2024-04-25T08:22:25.503 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:22:25.504 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990 --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose --owner scheduled_neesingh@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/orig.config.yaml 2024-04-25T08:22:25.548 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:22:25.640 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7672990' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnV197zNM2keAFWgxLmeuHORJvZxFF/n/DwFsHB1+LnjfNnCv5zdZFeUzBQTY55RlhiFnnxVfbZD77KD4Qaj3o= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/E0Sv9UZ2S2pyRFzug2wA1AHOsjHxQQsUJUJBnVdiz9OdDQ+vMnFvwn8h50D04IPISNy9aXKRG5u6B1d8sO9w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:22:25.640 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:22:25.641 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:22:25.641 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:22:25.643 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:22:25.667 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:22:25.668 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:22:25.668 INFO:teuthology.packaging:ref: None 2024-04-25T08:22:25.668 INFO:teuthology.packaging:tag: None 2024-04-25T08:22:25.668 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:22:25.668 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:22:25.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:22:25.831 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T08:22:25.833 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:22:25.839 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:22:25.840 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:22:25.850 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:22:25.867 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:22:25.873 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:22:25.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:17:35.151676', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnV197zNM2keAFWgxLmeuHORJvZxFF/n/DwFsHB1+LnjfNnCv5zdZFeUzBQTY55RlhiFnnxVfbZD77KD4Qaj3o='} 2024-04-25T08:22:25.913 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:17:35.149884', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/E0Sv9UZ2S2pyRFzug2wA1AHOsjHxQQsUJUJBnVdiz9OdDQ+vMnFvwn8h50D04IPISNy9aXKRG5u6B1d8sO9w='} 2024-04-25T08:22:25.914 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:22:25.920 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:22:25.920 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:22:25.920 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:22:25.991 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdeb00e3160>, signals=[15]) 2024-04-25T08:22:25.991 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:22:25.998 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:22:25.998 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:22:26.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:22:26.080 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:22:26.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:22:26.153 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:22:26.160 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-04-25T08:22:26.178 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-04-25T08:22:26.179 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-04-25T08:22:26.236 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:22:26.237 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:22:26.238 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-04-25T08:22:26.261 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-04-25T08:22:26.279 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-04-25T08:22:26.279 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-25T08:22:26.335 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-04-25T08:22:26.335 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-04-25T08:22:26.335 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:22:26.336 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:22:26.337 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-25T08:22:26.361 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:22:26.401 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:22:26.407 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:22:26.407 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:22:26.409 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:22:26.426 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:22:26.431 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:22:26.431 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:22:26.466 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:22:26.569 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:22:26.622 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:22:26.636 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T08:22:26.636 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-25T08:22:26.636 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-25T08:22:26.637 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:22:26.637 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:22:26.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:22:26.654 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-04-25T08:22:26.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:22:26.655 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-04-25T08:22:26.712 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:22:26.712 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-437.el9.x86_64 2024-04-25T08:22:26.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-04-25T08:22:26.713 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:22:26.713 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-437.el9.x86_64 2024-04-25T08:22:26.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2024-04-25T08:22:29.043 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T08:22:31.010 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T08:22:32.045 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-04-25T08:22:36.029 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-25T08:22:36.310 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-25T08:22:38.011 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-25T08:22:39.829 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T08:22:41.030 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-04-25T08:22:42.438 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-04-25T08:22:43.556 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-25T08:22:44.796 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-04-25T08:22:46.635 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T08:22:46.635 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T08:22:46.729 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-04-25T08:22:46.730 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-04-25T08:22:46.730 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-04-25T08:22:46.835 DEBUG:teuthology.orchestra.run.smithi100:> echo no | sudo yum reinstall kernel || true 2024-04-25T08:22:47.330 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:22:44 AM UTC. 2024-04-25T08:22:47.448 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-04-25T08:22:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-25T08:22:47.464 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-04-25T08:22:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-25T08:22:47.464 INFO:teuthology.orchestra.run.smithi100.stdout:Reinstalling: 2024-04-25T08:22:47.464 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:22:47.464 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:22:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-04-25T08:22:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-25T08:22:47.465 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:22:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 255 k 2024-04-25T08:22:47.465 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 0 2024-04-25T08:22:47.465 INFO:teuthology.orchestra.run.smithi100.stderr:Operation aborted. 2024-04-25T08:22:47.520 INFO:teuthology.orchestra.run.smithi100.stdout:Is this ok [y/N]: 2024-04-25T08:22:47.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum reinstall -y kernel || true 2024-04-25T08:22:48.009 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:22:44 AM UTC. 2024-04-25T08:22:48.124 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-04-25T08:22:48.125 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:Reinstalling: 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 255 k 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 0 2024-04-25T08:22:48.126 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-04-25T08:22:48.415 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-25T08:22:48.415 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:22:48.415 INFO:teuthology.orchestra.run.smithi100.stdout:Total 883 kB/s | 255 kB 00:00 2024-04-25T08:22:48.432 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-04-25T08:22:48.599 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-04-25T08:22:48.599 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-04-25T08:22:48.604 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-04-25T08:22:48.604 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-04-25T08:22:48.740 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T08:22:48.744 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-04-25T08:22:48.832 INFO:teuthology.orchestra.run.smithi100.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:22:48.832 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:22:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:22:49.478 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:22:50.148 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:22:50.149 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:22:50.149 INFO:teuthology.orchestra.run.smithi100.stdout:Reinstalled: 2024-04-25T08:22:50.149 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T08:22:50.149 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:22:50.149 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-04-25T08:22:50.318 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T08:22:50.365 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T08:22:50.366 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T08:22:50.366 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T08:22:50.366 DEBUG:teuthology.parallel:result is None 2024-04-25T08:22:50.688 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T08:22:50.688 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T08:22:50.784 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-04-25T08:22:50.785 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-04-25T08:22:50.785 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-04-25T08:22:50.902 DEBUG:teuthology.orchestra.run.smithi060:> echo no | sudo yum reinstall kernel || true 2024-04-25T08:22:51.424 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:22:48 AM UTC. 2024-04-25T08:22:51.555 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-04-25T08:22:51.570 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-04-25T08:22:51.570 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-04-25T08:22:51.570 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-04-25T08:22:51.570 INFO:teuthology.orchestra.run.smithi060.stdout:Reinstalling: 2024-04-25T08:22:51.570 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 255 k 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 0 2024-04-25T08:22:51.571 INFO:teuthology.orchestra.run.smithi060.stderr:Operation aborted. 2024-04-25T08:22:51.632 INFO:teuthology.orchestra.run.smithi060.stdout:Is this ok [y/N]: 2024-04-25T08:22:51.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum reinstall -y kernel || true 2024-04-25T08:22:52.151 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:22:48 AM UTC. 2024-04-25T08:22:52.272 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout:Reinstalling: 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:22:52.274 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-04-25T08:22:52.275 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-04-25T08:22:52.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:22:52.275 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 255 k 2024-04-25T08:22:52.275 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 0 2024-04-25T08:22:52.275 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-04-25T08:22:52.718 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-437.el9.x86_64.rpm 945 kB/s | 255 kB 00:00 2024-04-25T08:22:52.718 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:22:52.718 INFO:teuthology.orchestra.run.smithi060.stdout:Total 574 kB/s | 255 kB 00:00 2024-04-25T08:22:52.741 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-04-25T08:22:52.935 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-04-25T08:22:52.935 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-04-25T08:22:52.941 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-04-25T08:22:52.941 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-04-25T08:22:53.078 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-04-25T08:22:53.151 INFO:teuthology.orchestra.run.smithi060.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:22:53.152 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:22:53.825 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:22:53.825 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:22:54.553 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:22:54.554 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:22:54.554 INFO:teuthology.orchestra.run.smithi060.stdout:Reinstalled: 2024-04-25T08:22:54.554 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T08:22:54.554 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:22:54.554 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-04-25T08:22:54.715 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T08:22:54.763 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T08:22:54.763 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T08:22:54.763 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T08:22:54.763 DEBUG:teuthology.parallel:result is None 2024-04-25T08:22:54.764 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:22:54.771 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:22:54.772 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:22:54.774 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:22:54.792 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:22:54.815 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:22:54.830 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:22:54.830 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:22:54.833 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:22:54.860 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:22:54.866 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:22:54.867 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:22:54.893 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:22:54.926 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:22:54.933 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:22:54.941 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:22:54.952 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:22:54.954 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:22:54.961 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:22:54.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:22:54.985 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:22:55.021 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:22:55.030 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:22:55.030 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:22:55.050 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:22:55.077 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:22:55.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:22:55.192 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:22:55.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:22:55.313 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:22:55.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:22:55.377 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:22:55.419 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:22:55.496 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:22:55.536 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:22:55.577 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:22:55.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:22:55.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-04-25T08:22:55.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-04-25T08:22:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:22:55.719 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:22:56.161 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:22:56.168 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:22:56.168 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:22:56.178 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:22:56.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-04-25T08:22:56.265 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-04-25T08:22:56.267 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-04-25T08:22:56.328 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-04-25T08:22:56.330 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:22:56.330 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-04-25T08:22:56.355 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-04-25T08:22:56.355 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-04-25T08:22:56.375 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-04-25T08:22:56.375 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive'} 2024-04-25T08:22:56.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:22:56.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:22:56.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:22:56.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:22:56.448 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:22:56.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:22:56.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:22:56.520 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:22:56.529 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:22:56.545 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:22:56.546 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi100.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T08:26:27.171 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com')] 2024-04-25T08:26:27.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T08:26:27.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:26:27.251 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-04-25T08:26:27.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-04-25T08:26:27.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T08:26:27.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:26:27.409 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-25T08:26:27.480 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-25T08:26:27.481 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T08:26:27.492 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T08:26:27.492 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:26:27.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:26:27.495 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:26:27.495 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:26:27.525 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T08:26:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T08:26:27.549 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T08:26:27.563 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T08:26:27.566 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-04-25T08:26:27.577 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-04-25T08:26:27.587 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-04-25T08:26:27.591 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T08:26:27.605 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T08:26:27.608 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-04-25T08:26:27.622 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T08:26:27.635 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T08:26:27.659 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:26:27.663 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:26:27.663 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-04-25T08:26:27.663 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.663 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.663 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.663 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:26:27.698 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:26:27.698 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-04-25T08:26:27.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.698 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:26:27.699 INFO:teuthology.run_tasks:Running task install... 2024-04-25T08:26:27.708 DEBUG:teuthology.task.install:project ceph 2024-04-25T08:26:27.708 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T08:26:27.708 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T08:26:27.708 INFO:teuthology.task.install:Using flavor: default 2024-04-25T08:26:27.714 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T08:26:27.714 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T08:26:27.714 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T08:26:27.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:26:27.717 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T08:26:27.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:26:27.859 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T08:26:27.859 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T08:26:27.863 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T08:26:27.863 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T08:26:28.045 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T08:26:28.045 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:26:28.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T08:26:28.052 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T08:26:28.053 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:26:28.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T08:26:28.073 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T08:26:28.073 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T08:26:28.077 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T08:26:28.077 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T08:26:28.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T08:26:28.141 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T08:26:28.213 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T08:26:28.216 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T08:26:28.276 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-04-25T08:26:28.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-04-25T08:26:28.280 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-04-25T08:26:28.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-04-25T08:26:28.503 INFO:teuthology.orchestra.run.smithi100.stdout:48 files removed 2024-04-25T08:26:28.517 INFO:teuthology.orchestra.run.smithi060.stdout:48 files removed 2024-04-25T08:26:28.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T08:26:28.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T08:26:29.149 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 227 kB/s | 82 kB 00:00 2024-04-25T08:26:29.246 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 190 kB/s | 82 kB 00:00 2024-04-25T08:26:29.500 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 33 kB/s | 10 kB 00:00 2024-04-25T08:26:29.672 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-04-25T08:26:29.892 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-25T08:26:30.022 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T08:26:30.892 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T08:26:31.764 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-25T08:26:34.343 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-04-25T08:26:35.057 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-04-25T08:26:39.725 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-04-25T08:26:39.794 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-25T08:26:41.270 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-25T08:26:42.092 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-25T08:26:42.771 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T08:26:43.430 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T08:26:47.838 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T08:26:48.719 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T08:26:49.963 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:26:49.964 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:26:49.974 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T08:26:49.975 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T08:26:49.976 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T08:26:49.976 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T08:26:49.978 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T08:26:49.980 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T08:26:49.981 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T08:26:50.042 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T08:26:50.051 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T08:26:50.052 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T08:26:50.053 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T08:26:50.054 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T08:26:50.055 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T08:26:50.056 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T08:26:50.057 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T08:26:50.058 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T08:26:50.059 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T08:26:50.060 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T08:26:50.061 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T08:26:50.062 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:Install 148 Packages 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 4 Packages 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 215 M 2024-04-25T08:26:50.063 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-04-25T08:26:50.946 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:26:50.947 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:26:50.959 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T08:26:50.960 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T08:26:50.961 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T08:26:50.961 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T08:26:50.963 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T08:26:50.966 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T08:26:50.966 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T08:26:51.014 INFO:teuthology.orchestra.run.smithi100.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-25T08:26:51.032 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T08:26:51.042 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T08:26:51.043 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T08:26:51.044 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T08:26:51.045 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T08:26:51.046 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T08:26:51.047 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T08:26:51.048 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T08:26:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T08:26:51.050 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T08:26:51.051 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T08:26:51.052 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout:======================================================================================= 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout:Install 148 Packages 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 4 Packages 2024-04-25T08:26:51.053 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:26:51.054 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 215 M 2024-04-25T08:26:51.054 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-04-25T08:26:51.372 INFO:teuthology.orchestra.run.smithi100.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 2.4 MB/s | 887 kB 00:00 2024-04-25T08:26:51.448 INFO:teuthology.orchestra.run.smithi100.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.9 MB/s | 145 kB 00:00 2024-04-25T08:26:51.981 INFO:teuthology.orchestra.run.smithi100.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 4.0 MB/s | 2.1 MB 00:00 2024-04-25T08:26:52.037 INFO:teuthology.orchestra.run.smithi060.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-25T08:26:52.348 INFO:teuthology.orchestra.run.smithi100.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-04-25T08:26:53.382 INFO:teuthology.orchestra.run.smithi100.stdout:(6/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-04-25T08:26:53.620 INFO:teuthology.orchestra.run.smithi060.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 560 kB/s | 887 kB 00:01 2024-04-25T08:26:53.912 INFO:teuthology.orchestra.run.smithi060.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 499 kB/s | 145 kB 00:00 2024-04-25T08:26:54.979 INFO:teuthology.orchestra.run.smithi060.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 1.7 MB/s | 5.3 MB 00:03 2024-04-25T08:26:55.115 INFO:teuthology.orchestra.run.smithi100.stdout:(7/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 1.2 MB/s | 5.3 MB 00:04 2024-04-25T08:26:55.362 INFO:teuthology.orchestra.run.smithi060.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-04-25T08:26:56.246 INFO:teuthology.orchestra.run.smithi060.stdout:(6/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-04-25T08:26:57.079 INFO:teuthology.orchestra.run.smithi060.stdout:(7/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 688 kB/s | 2.1 MB 00:03 2024-04-25T08:26:57.298 INFO:teuthology.orchestra.run.smithi100.stdout:(8/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x86 4.3 MB/s | 17 MB 00:03 2024-04-25T08:26:57.382 INFO:teuthology.orchestra.run.smithi100.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 303 kB/s | 25 kB 00:00 2024-04-25T08:26:57.949 INFO:teuthology.orchestra.run.smithi100.stdout:(10/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 2.8 MB/s | 20 MB 00:07 2024-04-25T08:26:58.124 INFO:teuthology.orchestra.run.smithi100.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.2 MB/s | 220 kB 00:00 2024-04-25T08:26:58.257 INFO:teuthology.orchestra.run.smithi100.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 203 kB/s | 27 kB 00:00 2024-04-25T08:26:58.358 INFO:teuthology.orchestra.run.smithi100.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 312 kB/s | 31 kB 00:00 2024-04-25T08:26:59.074 INFO:teuthology.orchestra.run.smithi100.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 1.0 MB/s | 744 kB 00:00 2024-04-25T08:26:59.274 INFO:teuthology.orchestra.run.smithi100.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 816 kB/s | 163 kB 00:00 2024-04-25T08:26:59.450 INFO:teuthology.orchestra.run.smithi100.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 724 kB/s | 127 kB 00:00 2024-04-25T08:26:59.884 INFO:teuthology.orchestra.run.smithi100.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 1.1 MB/s | 501 kB 00:00 2024-04-25T08:27:00.488 INFO:teuthology.orchestra.run.smithi060.stdout:(8/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x86 4.0 MB/s | 17 MB 00:04 2024-04-25T08:27:00.579 INFO:teuthology.orchestra.run.smithi060.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 276 kB/s | 25 kB 00:00 2024-04-25T08:27:03.592 INFO:teuthology.orchestra.run.smithi100.stdout:(18/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 1.4 MB/s | 5.1 MB 00:03 2024-04-25T08:27:03.692 INFO:teuthology.orchestra.run.smithi100.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 452 kB/s | 45 kB 00:00 2024-04-25T08:27:03.850 INFO:teuthology.orchestra.run.smithi100.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 831 kB/s | 131 kB 00:00 2024-04-25T08:27:04.017 INFO:teuthology.orchestra.run.smithi100.stdout:(21/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 1.1 MB/s | 9.8 MB 00:08 2024-04-25T08:27:04.051 INFO:teuthology.orchestra.run.smithi100.stdout:(22/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 808 kB/s | 162 kB 00:00 2024-04-25T08:27:04.301 INFO:teuthology.orchestra.run.smithi100.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 1.2 MB/s | 298 kB 00:00 2024-04-25T08:27:04.419 INFO:teuthology.orchestra.run.smithi100.stdout:(24/152): python3-rados-19.0.0-1628.g97ae6cc9.e 801 kB/s | 321 kB 00:00 2024-04-25T08:27:04.444 INFO:teuthology.orchestra.run.smithi100.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 695 kB/s | 99 kB 00:00 2024-04-25T08:27:04.586 INFO:teuthology.orchestra.run.smithi100.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 511 kB/s | 85 kB 00:00 2024-04-25T08:27:04.596 INFO:teuthology.orchestra.run.smithi060.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 1.3 MB/s | 9.8 MB 00:07 2024-04-25T08:27:04.772 INFO:teuthology.orchestra.run.smithi060.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.2 MB/s | 220 kB 00:00 2024-04-25T08:27:04.828 INFO:teuthology.orchestra.run.smithi100.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 706 kB/s | 170 kB 00:00 2024-04-25T08:27:04.864 INFO:teuthology.orchestra.run.smithi060.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 295 kB/s | 27 kB 00:00 2024-04-25T08:27:04.928 INFO:teuthology.orchestra.run.smithi100.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 314 kB/s | 31 kB 00:00 2024-04-25T08:27:04.955 INFO:teuthology.orchestra.run.smithi060.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 340 kB/s | 31 kB 00:00 2024-04-25T08:27:05.087 INFO:teuthology.orchestra.run.smithi100.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 873 kB/s | 138 kB 00:00 2024-04-25T08:27:05.400 INFO:teuthology.orchestra.run.smithi060.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 1.6 MB/s | 744 kB 00:00 2024-04-25T08:27:05.567 INFO:teuthology.orchestra.run.smithi060.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 980 kB/s | 163 kB 00:00 2024-04-25T08:27:05.667 INFO:teuthology.orchestra.run.smithi060.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.2 MB/s | 127 kB 00:00 2024-04-25T08:27:05.951 INFO:teuthology.orchestra.run.smithi060.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 1.7 MB/s | 501 kB 00:00 2024-04-25T08:27:06.479 INFO:teuthology.orchestra.run.smithi100.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 1.5 MB/s | 3.1 MB 00:02 2024-04-25T08:27:06.804 INFO:teuthology.orchestra.run.smithi100.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 1.0 MB/s | 1.8 MB 00:01 2024-04-25T08:27:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.1 MB/s | 248 kB 00:00 2024-04-25T08:27:07.113 INFO:teuthology.orchestra.run.smithi100.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 592 kB/s | 49 kB 00:00 2024-04-25T08:27:07.213 INFO:teuthology.orchestra.run.smithi100.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 153 kB/s | 15 kB 00:00 2024-04-25T08:27:07.430 INFO:teuthology.orchestra.run.smithi100.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 1.2 MB/s | 264 kB 00:00 2024-04-25T08:27:07.467 INFO:teuthology.orchestra.run.smithi060.stdout:(18/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 20 MB 00:15 2024-04-25T08:27:07.559 INFO:teuthology.orchestra.run.smithi060.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 495 kB/s | 45 kB 00:00 2024-04-25T08:27:07.726 INFO:teuthology.orchestra.run.smithi060.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 788 kB/s | 131 kB 00:00 2024-04-25T08:27:07.843 INFO:teuthology.orchestra.run.smithi060.stdout:(21/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 2.7 MB/s | 5.1 MB 00:01 2024-04-25T08:27:07.910 INFO:teuthology.orchestra.run.smithi060.stdout:(22/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 882 kB/s | 162 kB 00:00 2024-04-25T08:27:07.993 INFO:teuthology.orchestra.run.smithi060.stdout:(23/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.1 MB/s | 321 kB 00:00 2024-04-25T08:27:07.997 INFO:teuthology.orchestra.run.smithi100.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 1.3 MB/s | 748 kB 00:00 2024-04-25T08:27:08.069 INFO:teuthology.orchestra.run.smithi060.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T08:27:08.081 INFO:teuthology.orchestra.run.smithi100.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 219 kB/s | 18 kB 00:00 2024-04-25T08:27:08.127 INFO:teuthology.orchestra.run.smithi060.stdout:(25/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 298 kB 00:00 2024-04-25T08:27:08.144 INFO:teuthology.orchestra.run.smithi060.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T08:27:08.228 INFO:teuthology.orchestra.run.smithi060.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T08:27:08.289 INFO:teuthology.orchestra.run.smithi100.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 386 kB/s | 80 kB 00:00 2024-04-25T08:27:08.311 INFO:teuthology.orchestra.run.smithi060.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 377 kB/s | 31 kB 00:00 2024-04-25T08:27:08.403 INFO:teuthology.orchestra.run.smithi060.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.5 MB/s | 138 kB 00:00 2024-04-25T08:27:08.673 INFO:teuthology.orchestra.run.smithi100.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-25T08:27:08.907 INFO:teuthology.orchestra.run.smithi100.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 837 kB/s | 195 kB 00:00 2024-04-25T08:27:08.921 INFO:teuthology.orchestra.run.smithi060.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 3.4 MB/s | 1.8 MB 00:00 2024-04-25T08:27:09.198 INFO:teuthology.orchestra.run.smithi100.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 869 kB/s | 253 kB 00:00 2024-04-25T08:27:09.765 INFO:teuthology.orchestra.run.smithi100.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-04-25T08:27:10.024 INFO:teuthology.orchestra.run.smithi100.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 412 kB/s | 106 kB 00:00 2024-04-25T08:27:10.232 INFO:teuthology.orchestra.run.smithi100.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 649 kB/s | 135 kB 00:00 2024-04-25T08:27:10.329 INFO:teuthology.orchestra.run.smithi060.stdout:(31/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 1.4 MB/s | 3.1 MB 00:02 2024-04-25T08:27:10.446 INFO:teuthology.orchestra.run.smithi060.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-04-25T08:27:10.449 INFO:teuthology.orchestra.run.smithi100.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 576 kB/s | 125 kB 00:00 2024-04-25T08:27:10.513 INFO:teuthology.orchestra.run.smithi060.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 743 kB/s | 49 kB 00:00 2024-04-25T08:27:10.538 INFO:teuthology.orchestra.run.smithi060.stdout:(34/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.2 MB/s | 248 kB 00:00 2024-04-25T08:27:10.588 INFO:teuthology.orchestra.run.smithi060.stdout:(35/152): ceph-prometheus-alerts-19.0.0-1628.g9 204 kB/s | 15 kB 00:00 2024-04-25T08:27:10.714 INFO:teuthology.orchestra.run.smithi060.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 5.8 MB/s | 748 kB 00:00 2024-04-25T08:27:10.739 INFO:teuthology.orchestra.run.smithi060.stdout:(37/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 264 kB 00:00 2024-04-25T08:27:10.798 INFO:teuthology.orchestra.run.smithi060.stdout:(38/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 218 kB/s | 18 kB 00:00 2024-04-25T08:27:10.873 INFO:teuthology.orchestra.run.smithi060.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-04-25T08:27:10.965 INFO:teuthology.orchestra.run.smithi060.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-25T08:27:10.990 INFO:teuthology.orchestra.run.smithi060.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-25T08:27:11.016 INFO:teuthology.orchestra.run.smithi060.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T08:27:11.025 INFO:teuthology.orchestra.run.smithi100.stdout:(46/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 3.5 MB/s | 47 MB 00:13 2024-04-25T08:27:11.074 INFO:teuthology.orchestra.run.smithi060.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T08:27:11.077 INFO:teuthology.orchestra.run.smithi100.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 342 kB/s | 215 kB 00:00 2024-04-25T08:27:11.099 INFO:teuthology.orchestra.run.smithi060.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T08:27:11.125 INFO:teuthology.orchestra.run.smithi060.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T08:27:11.150 INFO:teuthology.orchestra.run.smithi060.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T08:27:11.175 INFO:teuthology.orchestra.run.smithi060.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T08:27:11.226 INFO:teuthology.orchestra.run.smithi060.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-04-25T08:27:11.259 INFO:teuthology.orchestra.run.smithi100.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 1.5 MB/s | 7.4 MB 00:04 2024-04-25T08:27:11.268 INFO:teuthology.orchestra.run.smithi060.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-25T08:27:11.293 INFO:teuthology.orchestra.run.smithi060.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T08:27:11.318 INFO:teuthology.orchestra.run.smithi060.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T08:27:11.344 INFO:teuthology.orchestra.run.smithi060.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T08:27:11.369 INFO:teuthology.orchestra.run.smithi060.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-04-25T08:27:11.419 INFO:teuthology.orchestra.run.smithi060.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T08:27:11.426 INFO:teuthology.orchestra.run.smithi100.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2024-04-25T08:27:11.470 INFO:teuthology.orchestra.run.smithi060.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T08:27:11.495 INFO:teuthology.orchestra.run.smithi060.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-25T08:27:11.500 INFO:teuthology.orchestra.run.smithi100.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 442 kB/s | 107 kB 00:00 2024-04-25T08:27:11.521 INFO:teuthology.orchestra.run.smithi060.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T08:27:11.545 INFO:teuthology.orchestra.run.smithi060.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T08:27:11.559 INFO:teuthology.orchestra.run.smithi100.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 161 kB/s | 21 kB 00:00 2024-04-25T08:27:11.587 INFO:teuthology.orchestra.run.smithi060.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T08:27:11.595 INFO:teuthology.orchestra.run.smithi100.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-04-25T08:27:11.621 INFO:teuthology.orchestra.run.smithi060.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-25T08:27:11.647 INFO:teuthology.orchestra.run.smithi060.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-25T08:27:11.671 INFO:teuthology.orchestra.run.smithi060.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T08:27:11.697 INFO:teuthology.orchestra.run.smithi060.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T08:27:11.722 INFO:teuthology.orchestra.run.smithi060.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T08:27:11.729 INFO:teuthology.orchestra.run.smithi100.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 193 kB/s | 33 kB 00:00 2024-04-25T08:27:11.747 INFO:teuthology.orchestra.run.smithi060.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-25T08:27:11.854 INFO:teuthology.orchestra.run.smithi100.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 888 kB/s | 313 kB 00:00 2024-04-25T08:27:11.888 INFO:teuthology.orchestra.run.smithi100.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 115 kB/s | 18 kB 00:00 2024-04-25T08:27:11.923 INFO:teuthology.orchestra.run.smithi060.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-04-25T08:27:11.954 INFO:teuthology.orchestra.run.smithi100.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-04-25T08:27:11.998 INFO:teuthology.orchestra.run.smithi060.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-25T08:27:12.065 INFO:teuthology.orchestra.run.smithi060.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-25T08:27:12.116 INFO:teuthology.orchestra.run.smithi060.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-25T08:27:12.146 INFO:teuthology.orchestra.run.smithi100.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 620 kB/s | 160 kB 00:00 2024-04-25T08:27:12.171 INFO:teuthology.orchestra.run.smithi100.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 209 kB/s | 45 kB 00:00 2024-04-25T08:27:12.232 INFO:teuthology.orchestra.run.smithi060.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 9.3 MB/s | 4.7 MB 00:00 2024-04-25T08:27:12.316 INFO:teuthology.orchestra.run.smithi060.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T08:27:12.358 INFO:teuthology.orchestra.run.smithi060.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T08:27:12.383 INFO:teuthology.orchestra.run.smithi060.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T08:27:12.459 INFO:teuthology.orchestra.run.smithi060.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T08:27:12.542 INFO:teuthology.orchestra.run.smithi060.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-04-25T08:27:12.547 INFO:teuthology.orchestra.run.smithi100.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-04-25T08:27:12.567 INFO:teuthology.orchestra.run.smithi060.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-25T08:27:12.580 INFO:teuthology.orchestra.run.smithi100.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-04-25T08:27:12.593 INFO:teuthology.orchestra.run.smithi060.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-25T08:27:12.618 INFO:teuthology.orchestra.run.smithi060.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T08:27:12.643 INFO:teuthology.orchestra.run.smithi060.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T08:27:12.669 INFO:teuthology.orchestra.run.smithi060.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-04-25T08:27:12.681 INFO:teuthology.orchestra.run.smithi100.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 191 kB/s | 25 kB 00:00 2024-04-25T08:27:12.736 INFO:teuthology.orchestra.run.smithi060.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T08:27:12.772 INFO:teuthology.orchestra.run.smithi100.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-04-25T08:27:12.802 INFO:teuthology.orchestra.run.smithi060.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T08:27:12.820 INFO:teuthology.orchestra.run.smithi060.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-25T08:27:12.853 INFO:teuthology.orchestra.run.smithi060.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T08:27:12.878 INFO:teuthology.orchestra.run.smithi060.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-04-25T08:27:12.904 INFO:teuthology.orchestra.run.smithi060.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T08:27:12.929 INFO:teuthology.orchestra.run.smithi060.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T08:27:12.948 INFO:teuthology.orchestra.run.smithi100.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-04-25T08:27:12.954 INFO:teuthology.orchestra.run.smithi060.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T08:27:12.979 INFO:teuthology.orchestra.run.smithi060.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T08:27:13.005 INFO:teuthology.orchestra.run.smithi060.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T08:27:13.023 INFO:teuthology.orchestra.run.smithi100.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 753 kB/s | 188 kB 00:00 2024-04-25T08:27:13.064 INFO:teuthology.orchestra.run.smithi060.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-04-25T08:27:13.105 INFO:teuthology.orchestra.run.smithi060.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-25T08:27:13.106 INFO:teuthology.orchestra.run.smithi100.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 222 kB/s | 35 kB 00:00 2024-04-25T08:27:13.156 INFO:teuthology.orchestra.run.smithi060.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T08:27:13.206 INFO:teuthology.orchestra.run.smithi060.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-04-25T08:27:13.248 INFO:teuthology.orchestra.run.smithi060.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T08:27:13.274 INFO:teuthology.orchestra.run.smithi100.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.8 MB/s | 3.0 MB 00:01 2024-04-25T08:27:13.415 INFO:teuthology.orchestra.run.smithi060.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-25T08:27:13.441 INFO:teuthology.orchestra.run.smithi060.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-25T08:27:13.466 INFO:teuthology.orchestra.run.smithi060.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T08:27:13.508 INFO:teuthology.orchestra.run.smithi060.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-25T08:27:13.533 INFO:teuthology.orchestra.run.smithi060.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-04-25T08:27:13.558 INFO:teuthology.orchestra.run.smithi060.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-04-25T08:27:13.583 INFO:teuthology.orchestra.run.smithi060.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T08:27:13.617 INFO:teuthology.orchestra.run.smithi060.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T08:27:13.741 INFO:teuthology.orchestra.run.smithi100.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-04-25T08:27:14.074 INFO:teuthology.orchestra.run.smithi100.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-04-25T08:27:14.141 INFO:teuthology.orchestra.run.smithi100.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 2.2 MB/s | 911 kB 00:00 2024-04-25T08:27:14.167 INFO:teuthology.orchestra.run.smithi060.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 562 kB/s | 309 kB 00:00 2024-04-25T08:27:14.218 INFO:teuthology.orchestra.run.smithi060.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 865 kB/s | 548 kB 00:00 2024-04-25T08:27:14.293 INFO:teuthology.orchestra.run.smithi060.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-25T08:27:14.358 INFO:teuthology.orchestra.run.smithi100.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-04-25T08:27:14.400 INFO:teuthology.orchestra.run.smithi100.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 409 kB/s | 133 kB 00:00 2024-04-25T08:27:14.402 INFO:teuthology.orchestra.run.smithi060.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T08:27:14.510 INFO:teuthology.orchestra.run.smithi060.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-25T08:27:14.600 INFO:teuthology.orchestra.run.smithi100.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-04-25T08:27:14.669 INFO:teuthology.orchestra.run.smithi060.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-04-25T08:27:14.728 INFO:teuthology.orchestra.run.smithi060.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-04-25T08:27:14.811 INFO:teuthology.orchestra.run.smithi060.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-25T08:27:14.862 INFO:teuthology.orchestra.run.smithi060.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-04-25T08:27:14.917 INFO:teuthology.orchestra.run.smithi100.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-04-25T08:27:14.953 INFO:teuthology.orchestra.run.smithi060.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 475 kB/s | 43 kB 00:00 2024-04-25T08:27:15.037 INFO:teuthology.orchestra.run.smithi060.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-25T08:27:15.062 INFO:teuthology.orchestra.run.smithi060.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-25T08:27:15.137 INFO:teuthology.orchestra.run.smithi060.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-25T08:27:15.163 INFO:teuthology.orchestra.run.smithi060.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T08:27:15.200 INFO:teuthology.orchestra.run.smithi100.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 862 kB/s | 244 kB 00:00 2024-04-25T08:27:15.255 INFO:teuthology.orchestra.run.smithi060.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T08:27:15.288 INFO:teuthology.orchestra.run.smithi060.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-25T08:27:15.364 INFO:teuthology.orchestra.run.smithi060.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-25T08:27:15.389 INFO:teuthology.orchestra.run.smithi060.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T08:27:15.489 INFO:teuthology.orchestra.run.smithi060.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T08:27:15.509 INFO:teuthology.orchestra.run.smithi100.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 804 kB/s | 248 kB 00:00 2024-04-25T08:27:15.631 INFO:teuthology.orchestra.run.smithi060.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-04-25T08:27:15.656 INFO:teuthology.orchestra.run.smithi060.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-04-25T08:27:15.659 INFO:teuthology.orchestra.run.smithi100.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 171 kB/s | 26 kB 00:00 2024-04-25T08:27:15.748 INFO:teuthology.orchestra.run.smithi060.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T08:27:15.774 INFO:teuthology.orchestra.run.smithi060.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-04-25T08:27:15.818 INFO:teuthology.orchestra.run.smithi100.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 19 kB 00:00 2024-04-25T08:27:15.841 INFO:teuthology.orchestra.run.smithi060.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T08:27:15.866 INFO:teuthology.orchestra.run.smithi060.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T08:27:15.975 INFO:teuthology.orchestra.run.smithi060.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-04-25T08:27:16.000 INFO:teuthology.orchestra.run.smithi060.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-04-25T08:27:16.010 INFO:teuthology.orchestra.run.smithi100.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 3.1 MB/s | 5.8 MB 00:01 2024-04-25T08:27:16.084 INFO:teuthology.orchestra.run.smithi060.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-04-25T08:27:16.109 INFO:teuthology.orchestra.run.smithi060.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-25T08:27:16.118 INFO:teuthology.orchestra.run.smithi100.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 573 kB/s | 172 kB 00:00 2024-04-25T08:27:16.194 INFO:teuthology.orchestra.run.smithi100.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 189 kB/s | 35 kB 00:00 2024-04-25T08:27:16.201 INFO:teuthology.orchestra.run.smithi060.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T08:27:16.226 INFO:teuthology.orchestra.run.smithi060.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-25T08:27:16.293 INFO:teuthology.orchestra.run.smithi060.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 980 kB/s | 90 kB 00:00 2024-04-25T08:27:16.326 INFO:teuthology.orchestra.run.smithi060.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-25T08:27:16.393 INFO:teuthology.orchestra.run.smithi060.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-25T08:27:16.427 INFO:teuthology.orchestra.run.smithi060.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T08:27:16.486 INFO:teuthology.orchestra.run.smithi100.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 858 kB/s | 250 kB 00:00 2024-04-25T08:27:16.503 INFO:teuthology.orchestra.run.smithi060.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-04-25T08:27:16.527 INFO:teuthology.orchestra.run.smithi060.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-04-25T08:27:16.603 INFO:teuthology.orchestra.run.smithi060.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-04-25T08:27:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T08:27:16.761 INFO:teuthology.orchestra.run.smithi100.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 809 kB/s | 222 kB 00:00 2024-04-25T08:27:16.945 INFO:teuthology.orchestra.run.smithi100.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-04-25T08:27:17.104 INFO:teuthology.orchestra.run.smithi060.stdout:(143/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 2.9 MB/s | 47 MB 00:16 2024-04-25T08:27:17.129 INFO:teuthology.orchestra.run.smithi060.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 171 kB/s | 90 kB 00:00 2024-04-25T08:27:17.153 INFO:teuthology.orchestra.run.smithi100.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 203 kB/s | 42 kB 00:00 2024-04-25T08:27:17.154 INFO:teuthology.orchestra.run.smithi060.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 853 kB/s | 427 kB 00:00 2024-04-25T08:27:17.229 INFO:teuthology.orchestra.run.smithi060.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T08:27:17.245 INFO:teuthology.orchestra.run.smithi100.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-04-25T08:27:17.321 INFO:teuthology.orchestra.run.smithi060.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-25T08:27:17.380 INFO:teuthology.orchestra.run.smithi060.stdout:(148/152): python3-zc-lockfile-2.0-10.el9.noarc 72 kB/s | 20 kB 00:00 2024-04-25T08:27:17.479 INFO:teuthology.orchestra.run.smithi100.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 828 kB/s | 269 kB 00:00 2024-04-25T08:27:17.513 INFO:teuthology.orchestra.run.smithi100.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-04-25T08:27:17.580 INFO:teuthology.orchestra.run.smithi060.stdout:(149/152): librados2-19.0.0-1628.g97ae6cc9.el9. 9.6 MB/s | 3.4 MB 00:00 2024-04-25T08:27:17.649 INFO:teuthology.orchestra.run.smithi060.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.0 MB/s | 1.3 MB 00:00 2024-04-25T08:27:17.715 INFO:teuthology.orchestra.run.smithi100.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 253 kB/s | 51 kB 00:00 2024-04-25T08:27:17.788 INFO:teuthology.orchestra.run.smithi100.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 903 kB/s | 279 kB 00:00 2024-04-25T08:27:17.797 INFO:teuthology.orchestra.run.smithi060.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T08:27:17.856 INFO:teuthology.orchestra.run.smithi060.stdout:(152/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-04-25T08:27:17.859 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:27:17.859 INFO:teuthology.orchestra.run.smithi060.stdout:Total 8.0 MB/s | 215 MB 00:26 2024-04-25T08:27:17.913 INFO:teuthology.orchestra.run.smithi100.stdout:(90/152): python3-babel-2.9.1-2.el9.noarch.rpm 1.8 MB/s | 6.0 MB 00:03 2024-04-25T08:27:17.939 INFO:teuthology.orchestra.run.smithi100.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 240 kB/s | 54 kB 00:00 2024-04-25T08:27:18.156 INFO:teuthology.orchestra.run.smithi100.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 172 kB/s | 42 kB 00:00 2024-04-25T08:27:18.181 INFO:teuthology.orchestra.run.smithi100.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 183 kB/s | 44 kB 00:00 2024-04-25T08:27:18.316 INFO:teuthology.orchestra.run.smithi100.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 269 kB/s | 36 kB 00:00 2024-04-25T08:27:18.481 INFO:teuthology.orchestra.run.smithi100.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 678 kB/s | 221 kB 00:00 2024-04-25T08:27:18.507 INFO:teuthology.orchestra.run.smithi100.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-04-25T08:27:18.682 INFO:teuthology.orchestra.run.smithi100.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 330 kB/s | 58 kB 00:00 2024-04-25T08:27:18.824 INFO:teuthology.orchestra.run.smithi100.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 891 kB/s | 305 kB 00:00 2024-04-25T08:27:18.899 INFO:teuthology.orchestra.run.smithi100.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 432 kB/s | 94 kB 00:00 2024-04-25T08:27:18.995 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-04-25T08:27:19.153 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-04-25T08:27:19.153 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-04-25T08:27:19.200 INFO:teuthology.orchestra.run.smithi100.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-04-25T08:27:19.216 INFO:teuthology.orchestra.run.smithi100.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 141 kB/s | 45 kB 00:00 2024-04-25T08:27:19.334 INFO:teuthology.orchestra.run.smithi100.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-04-25T08:27:19.776 INFO:teuthology.orchestra.run.smithi100.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.2 MB/s | 16 MB 00:01 2024-04-25T08:27:19.809 INFO:teuthology.orchestra.run.smithi100.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-25T08:27:20.268 INFO:teuthology.orchestra.run.smithi100.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-25T08:27:20.585 INFO:teuthology.orchestra.run.smithi100.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 438 kB/s | 548 kB 00:01 2024-04-25T08:27:20.626 INFO:teuthology.orchestra.run.smithi100.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 363 kB/s | 309 kB 00:00 2024-04-25T08:27:20.685 INFO:teuthology.orchestra.run.smithi100.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T08:27:20.736 INFO:teuthology.orchestra.run.smithi100.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-04-25T08:27:20.869 INFO:teuthology.orchestra.run.smithi100.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 366 kB/s | 67 kB 00:00 2024-04-25T08:27:20.993 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-04-25T08:27:20.994 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-04-25T08:27:21.003 INFO:teuthology.orchestra.run.smithi100.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-04-25T08:27:21.770 INFO:teuthology.orchestra.run.smithi100.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 715 kB/s | 548 kB 00:00 2024-04-25T08:27:21.887 INFO:teuthology.orchestra.run.smithi100.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 373 kB/s | 43 kB 00:00 2024-04-25T08:27:21.962 INFO:teuthology.orchestra.run.smithi100.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 767 kB/s | 838 kB 00:01 2024-04-25T08:27:21.987 INFO:teuthology.orchestra.run.smithi100.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-25T08:27:22.062 INFO:teuthology.orchestra.run.smithi100.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-25T08:27:22.263 INFO:teuthology.orchestra.run.smithi100.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 624 kB/s | 172 kB 00:00 2024-04-25T08:27:22.632 INFO:teuthology.orchestra.run.smithi100.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 630 kB/s | 358 kB 00:00 2024-04-25T08:27:22.738 INFO:teuthology.orchestra.run.smithi100.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 638 kB/s | 303 kB 00:00 2024-04-25T08:27:22.763 INFO:teuthology.orchestra.run.smithi100.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 268 kB/s | 35 kB 00:00 2024-04-25T08:27:23.081 INFO:teuthology.orchestra.run.smithi100.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 601 kB/s | 206 kB 00:00 2024-04-25T08:27:23.342 INFO:teuthology.orchestra.run.smithi100.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 559 kB/s | 144 kB 00:00 2024-04-25T08:27:23.456 INFO:teuthology.orchestra.run.smithi100.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-25T08:27:23.573 INFO:teuthology.orchestra.run.smithi100.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-25T08:27:23.681 INFO:teuthology.orchestra.run.smithi100.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-04-25T08:27:23.782 INFO:teuthology.orchestra.run.smithi100.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T08:27:23.899 INFO:teuthology.orchestra.run.smithi100.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-04-25T08:27:25.207 INFO:teuthology.orchestra.run.smithi100.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 854 kB/s | 2.0 MB 00:02 2024-04-25T08:27:25.241 INFO:teuthology.orchestra.run.smithi100.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 779 kB/s | 1.0 MB 00:01 2024-04-25T08:27:25.316 INFO:teuthology.orchestra.run.smithi100.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-25T08:27:25.433 INFO:teuthology.orchestra.run.smithi100.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2024-04-25T08:27:25.509 INFO:teuthology.orchestra.run.smithi100.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 300 kB/s | 58 kB 00:00 2024-04-25T08:27:25.609 INFO:teuthology.orchestra.run.smithi100.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-25T08:27:25.842 INFO:teuthology.orchestra.run.smithi100.stdout:(134/152): libarrow-9.0.0-10.el9.x86_64.rpm 811 kB/s | 4.4 MB 00:05 2024-04-25T08:27:25.867 INFO:teuthology.orchestra.run.smithi100.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 348 kB/s | 90 kB 00:00 2024-04-25T08:27:25.893 INFO:teuthology.orchestra.run.smithi100.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 593 kB/s | 272 kB 00:00 2024-04-25T08:27:25.943 INFO:teuthology.orchestra.run.smithi100.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-25T08:27:25.969 INFO:teuthology.orchestra.run.smithi100.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-04-25T08:27:26.077 INFO:teuthology.orchestra.run.smithi100.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-25T08:27:26.127 INFO:teuthology.orchestra.run.smithi100.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 321 kB/s | 59 kB 00:00 2024-04-25T08:27:26.186 INFO:teuthology.orchestra.run.smithi100.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 643 kB/s | 188 kB 00:00 2024-04-25T08:27:26.211 INFO:teuthology.orchestra.run.smithi100.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-04-25T08:27:26.370 INFO:teuthology.orchestra.run.smithi100.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 489 kB/s | 90 kB 00:00 2024-04-25T08:27:26.487 INFO:teuthology.orchestra.run.smithi100.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 639 kB/s | 230 kB 00:00 2024-04-25T08:27:26.512 INFO:teuthology.orchestra.run.smithi100.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-04-25T08:27:26.762 INFO:teuthology.orchestra.run.smithi100.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 697 kB/s | 192 kB 00:00 2024-04-25T08:27:26.796 INFO:teuthology.orchestra.run.smithi100.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-25T08:27:27.130 INFO:teuthology.orchestra.run.smithi100.stdout:(148/152): librados2-19.0.0-1628.g97ae6cc9.el9. 9.2 MB/s | 3.4 MB 00:00 2024-04-25T08:27:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-04-25T08:27:27.335 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T08:27:27.374 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T08:27:27.428 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T08:27:27.470 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T08:27:27.503 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T08:27:27.547 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T08:27:27.647 INFO:teuthology.orchestra.run.smithi100.stdout:(149/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-25T08:27:27.839 INFO:teuthology.orchestra.run.smithi100.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.9 MB/s | 1.3 MB 00:00 2024-04-25T08:27:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T08:27:28.012 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:27:28.120 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:27:28.146 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:27:28.239 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:27:28.276 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:27:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T08:27:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T08:27:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:27:28.564 INFO:teuthology.orchestra.run.smithi100.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 789 kB/s | 1.6 MB 00:02 2024-04-25T08:27:28.605 INFO:teuthology.orchestra.run.smithi100.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 2.4 MB 00:00 2024-04-25T08:27:28.610 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:27:28.610 INFO:teuthology.orchestra.run.smithi100.stdout:Total 5.6 MB/s | 215 MB 00:38 2024-04-25T08:27:28.614 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:27:28.644 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:27:28.707 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:27:28.720 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:27:28.743 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:27:28.896 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:27:28.928 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:27:29.039 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T08:27:29.135 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T08:27:29.194 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T08:27:29.230 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T08:27:29.306 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T08:27:29.345 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T08:27:29.389 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T08:27:29.421 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T08:27:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T08:27:29.520 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T08:27:29.573 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T08:27:29.601 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T08:27:29.646 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T08:27:29.696 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-04-25T08:27:29.724 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T08:27:29.756 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T08:27:29.800 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T08:27:29.828 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T08:27:29.837 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-04-25T08:27:29.837 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-04-25T08:27:29.902 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T08:27:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T08:27:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T08:27:30.175 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T08:27:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T08:27:30.795 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T08:27:30.878 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T08:27:30.912 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T08:27:30.955 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T08:27:30.993 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T08:27:31.044 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T08:27:31.204 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T08:27:31.254 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T08:27:31.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T08:27:31.386 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T08:27:31.422 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T08:27:31.493 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T08:27:31.562 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-04-25T08:27:31.562 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-04-25T08:27:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T08:27:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T08:27:31.794 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T08:27:31.835 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T08:27:31.983 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T08:27:32.024 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T08:27:32.071 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T08:27:32.113 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T08:27:32.210 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T08:27:32.246 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T08:27:32.286 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T08:27:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T08:27:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T08:27:32.422 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T08:27:32.560 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T08:27:32.593 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T08:27:32.622 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T08:27:32.657 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T08:27:32.696 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T08:27:32.736 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T08:27:32.765 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T08:27:32.808 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T08:27:32.841 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T08:27:32.883 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T08:27:32.918 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T08:27:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T08:27:33.057 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T08:27:33.088 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T08:27:33.136 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T08:27:33.256 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T08:27:33.315 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T08:27:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T08:27:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T08:27:34.154 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T08:27:34.256 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T08:27:34.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T08:27:34.412 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T08:27:34.448 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T08:27:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T08:27:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T08:27:34.741 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T08:27:35.423 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T08:27:35.549 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T08:27:36.838 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T08:27:36.925 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T08:27:36.953 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T08:27:36.986 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T08:27:37.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T08:27:37.348 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:27:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:27:37.446 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-04-25T08:27:37.470 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T08:27:37.507 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T08:27:37.542 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T08:27:37.581 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T08:27:37.634 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T08:27:37.668 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T08:27:37.702 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T08:27:37.745 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T08:27:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T08:27:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:27:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:27:38.112 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T08:27:38.147 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T08:27:38.195 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:27:38.317 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:27:38.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:27:38.447 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:27:38.484 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:27:38.631 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T08:27:38.689 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T08:27:38.717 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:27:38.804 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:27:38.826 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:27:38.889 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:27:38.910 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:27:38.931 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:27:39.127 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:27:39.160 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:27:39.267 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T08:27:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T08:27:39.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T08:27:39.436 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T08:27:39.500 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T08:27:39.543 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T08:27:39.586 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T08:27:39.619 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T08:27:39.647 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T08:27:39.716 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T08:27:39.769 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T08:27:39.800 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T08:27:39.835 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T08:27:39.911 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T08:27:39.946 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T08:27:39.990 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T08:27:40.019 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T08:27:40.088 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T08:27:40.133 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T08:27:40.207 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T08:27:40.347 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T08:27:40.372 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:27:40.402 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T08:27:40.408 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:27:40.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T08:27:40.974 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:27:40.998 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T08:27:41.020 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:27:41.035 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T08:27:41.078 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T08:27:41.110 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:27:41.111 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T08:27:41.111 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T08:27:41.111 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:41.116 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T08:27:41.138 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:27:41.166 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T08:27:41.308 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T08:27:41.353 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T08:27:41.407 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T08:27:41.480 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T08:27:41.512 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T08:27:41.540 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T08:27:41.587 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T08:27:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T08:27:41.666 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T08:27:41.708 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T08:27:41.843 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T08:27:41.888 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T08:27:41.919 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T08:27:41.968 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T08:27:42.063 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T08:27:42.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T08:27:42.151 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T08:27:42.176 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T08:27:42.201 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T08:27:42.274 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T08:27:42.402 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T08:27:42.434 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T08:27:42.453 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T08:27:42.487 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T08:27:42.526 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T08:27:42.558 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T08:27:42.588 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T08:27:42.623 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T08:27:42.656 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T08:27:42.698 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T08:27:42.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T08:27:42.819 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T08:27:42.862 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T08:27:42.902 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T08:27:42.949 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T08:27:43.047 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T08:27:43.120 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T08:27:43.837 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T08:27:43.885 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T08:27:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T08:27:44.026 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T08:27:44.085 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T08:27:44.173 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T08:27:44.212 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T08:27:44.388 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T08:27:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T08:27:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T08:27:45.142 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T08:27:45.257 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T08:27:46.467 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T08:27:46.548 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T08:27:46.575 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T08:27:46.609 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T08:27:46.920 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T08:27:46.955 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:27:47.046 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:27:47.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T08:27:47.113 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T08:27:47.566 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T08:27:47.592 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:27:47.688 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:27:47.718 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T08:27:49.911 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:27:49.948 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:27:50.482 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:27:50.527 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:27:50.618 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:27:50.618 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T08:27:50.618 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T08:27:50.618 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:27:50.645 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-04-25T08:27:53.082 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-04-25T08:27:53.083 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:53.251 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:27:53.267 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:27:53.268 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:53.268 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T08:27:53.268 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:54.797 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:27:54.825 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:27:54.825 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:54.825 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T08:27:54.825 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:27:54.825 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:27:54.825 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:55.048 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:27:55.071 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:27:55.071 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:55.071 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T08:27:55.072 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:27:55.072 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:27:55.072 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:55.532 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:27:55.555 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:27:55.555 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:55.556 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T08:27:55.556 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:27:55.556 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:27:55.556 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:55.671 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T08:27:55.730 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T08:27:55.792 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T08:27:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T08:27:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T08:27:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T08:27:56.212 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:27:56.355 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:27:56.400 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T08:27:56.432 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T08:27:56.459 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T08:27:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:27:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:27:56.973 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T08:27:57.036 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T08:27:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:27:57.303 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:27:57.340 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:27:58.350 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:27:58.382 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:27:58.492 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:27:58.669 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T08:27:58.693 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:27:58.717 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:27:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T08:27:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:27:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:27:58.718 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:58.769 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:27:58.780 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:27:58.881 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T08:27:58.935 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T08:27:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T08:27:59.871 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:27:59.895 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:27:59.895 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:59.895 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T08:27:59.896 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:27:59.896 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:27:59.896 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:27:59.947 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:27:59.970 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:27:59.971 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:27:59.971 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T08:27:59.971 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:00.023 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T08:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:28:00.046 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:00.365 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:28:00.389 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:28:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T08:28:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:28:00.389 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:28:00.389 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-04-25T08:28:01.888 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:28:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:28:02.074 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:02.074 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T08:28:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:03.545 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:03.779 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:28:03.801 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:28:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T08:28:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:28:03.801 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:28:03.802 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:04.238 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:28:04.260 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:28:04.260 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:04.260 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T08:28:04.261 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:28:04.261 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:28:04.261 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:04.359 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T08:28:04.408 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T08:28:04.463 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T08:28:04.676 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T08:28:04.722 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T08:28:04.791 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T08:28:04.878 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:28:05.019 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:28:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T08:28:05.055 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T08:28:05.087 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T08:28:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T08:28:05.115 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T08:28:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T08:28:05.166 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T08:28:05.194 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T08:28:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T08:28:05.440 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T08:28:05.470 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T08:28:05.575 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:28:05.614 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:28:05.654 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T08:28:05.745 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:28:06.005 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:28:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:28:06.151 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:28:06.171 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:28:06.967 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:28:06.997 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:28:07.066 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T08:28:07.099 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:28:07.147 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T08:28:07.190 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T08:28:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T08:28:07.275 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T08:28:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:28:07.311 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T08:28:07.311 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:28:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:28:07.313 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:07.313 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T08:28:07.313 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:28:07.313 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:28:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:07.358 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:28:07.370 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:28:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:28:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:28:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:28:07.466 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T08:28:07.523 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T08:28:07.530 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:28:07.605 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:28:08.378 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T08:28:08.411 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:28:08.433 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:28:08.434 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:08.434 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T08:28:08.434 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:28:08.434 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:28:08.434 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:08.478 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:28:08.500 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:28:08.500 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:08.500 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T08:28:08.500 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:08.545 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:28:08.567 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:28:08.567 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:08.568 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T08:28:08.568 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:28:08.568 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:28:08.568 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:08.879 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:28:08.902 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:28:08.902 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:28:08.902 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T08:28:08.902 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:28:08.902 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:28:08.903 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:12.839 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:28:12.840 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T08:28:12.841 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T08:28:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T08:28:12.854 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T08:28:12.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T08:28:12.856 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T08:28:12.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T08:28:12.858 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T08:28:12.859 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T08:28:12.860 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:28:12.861 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T08:28:13.316 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T08:28:13.357 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T08:28:13.398 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T08:28:13.430 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T08:28:13.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T08:28:13.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T08:28:13.687 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T08:28:13.726 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T08:28:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:28:14.373 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:28:14.393 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:28:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:28:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:14.943 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-04-25T08:28:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.943 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.944 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.945 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T08:28:14.946 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T08:28:14.947 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T08:28:14.948 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T08:28:14.949 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T08:28:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T08:28:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T08:28:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T08:28:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T08:28:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.950 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T08:28:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T08:28:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T08:28:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T08:28:14.951 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T08:28:14.952 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T08:28:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T08:28:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T08:28:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T08:28:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T08:28:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T08:28:14.953 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T08:28:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T08:28:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T08:28:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T08:28:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T08:28:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T08:28:14.954 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T08:28:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T08:28:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T08:28:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T08:28:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T08:28:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T08:28:14.955 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T08:28:14.956 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T08:28:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T08:28:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T08:28:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T08:28:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.957 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T08:28:14.958 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:14.959 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:14.960 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-04-25T08:28:15.214 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T08:28:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T08:28:15.338 DEBUG:teuthology.parallel:result is None 2024-04-25T08:28:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T08:28:15.425 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T08:28:15.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T08:28:15.458 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:28:15.542 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:28:15.583 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:28:15.584 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:28:15.682 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:28:15.748 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T08:28:21.046 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T08:28:21.047 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T08:28:21.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T08:28:21.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T08:28:21.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T08:28:21.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T08:28:21.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T08:28:21.048 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T08:28:21.059 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T08:28:21.059 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T08:28:21.059 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T08:28:21.060 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T08:28:21.061 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T08:28:21.062 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T08:28:21.063 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T08:28:21.064 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T08:28:21.065 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T08:28:21.066 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T08:28:21.067 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:28:21.068 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T08:28:23.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.242 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.243 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T08:28:23.244 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T08:28:23.245 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T08:28:23.246 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.247 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T08:28:23.248 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T08:28:23.249 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T08:28:23.250 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T08:28:23.251 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.252 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T08:28:23.253 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:23.254 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-04-25T08:28:23.666 DEBUG:teuthology.parallel:result is None 2024-04-25T08:28:23.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:28:23.833 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T08:28:23.859 INFO:teuthology.orchestra.run.smithi060.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:28:23.860 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:28:23.860 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T08:28:23.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:28:24.019 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T08:28:24.041 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:28:24.041 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:28:24.041 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T08:28:24.043 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T08:28:24.043 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:24.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:28:24.075 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:24.075 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:28:24.108 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T08:28:24.108 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:24.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:28:24.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:28:24.212 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:24.212 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:28:24.236 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:28:24.299 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T08:28:24.300 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:24.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:28:24.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:28:24.396 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:24.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:28:24.420 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:28:24.484 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T08:28:24.484 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:24.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:28:24.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:28:24.577 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:24.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:28:24.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:28:24.667 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T08:28:24.752 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T08:28:24.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2024-04-25T08:28:24.772 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-04-25T08:28:24.796 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T08:28:24.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:28:24.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:28:24.864 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T08:28:24.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:28:24.907 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:28:24.938 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T08:28:24.938 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T08:28:24.938 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T08:28:24.938 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:24.938 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:28:24.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:28:25.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T08:28:25.131 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:25.131 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:28:25.145 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:28:25.249 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T08:28:25.298 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T08:28:25.298 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T08:28:25.299 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7672990', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnV197zNM2keAFWgxLmeuHORJvZxFF/n/DwFsHB1+LnjfNnCv5zdZFeUzBQTY55RlhiFnnxVfbZD77KD4Qaj3o=', 'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/E0Sv9UZ2S2pyRFzug2wA1AHOsjHxQQsUJUJBnVdiz9OdDQ+vMnFvwn8h50D04IPISNy9aXKRG5u6B1d8sO9w='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T08:28:25.299 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:28:25.302 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:28:25.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:28:25.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:28:25.412 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:25.412 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:28:25.439 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:28:25.439 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 08:28:08.038909910 +0000 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 08:26:08.661550207 +0000 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 08:26:08.661550207 +0000 2024-04-25T08:28:25.496 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-04-25 08:26:08.661550207 +0000 2024-04-25T08:28:25.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:28:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:28:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:28:25.560 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000109014 s, 4.7 MB/s 2024-04-25T08:28:25.561 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:28:25.619 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 08:28:08.038909910 +0000 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 08:26:08.988547288 +0000 2024-04-25T08:28:25.676 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 08:26:08.988547288 +0000 2024-04-25T08:28:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-04-25 08:26:08.988547288 +0000 2024-04-25T08:28:25.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:28:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:28:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:28:25.741 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 8.9915e-05 s, 5.7 MB/s 2024-04-25T08:28:25.743 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:28:25.799 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 08:28:08.038909910 +0000 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 08:26:09.334544200 +0000 2024-04-25T08:28:25.856 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 08:26:09.334544200 +0000 2024-04-25T08:28:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-04-25 08:26:09.334544200 +0000 2024-04-25T08:28:25.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:28:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:28:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:28:25.921 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000108498 s, 4.7 MB/s 2024-04-25T08:28:25.922 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:28:25.980 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-04-25 08:28:08.039909900 +0000 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-04-25 08:26:09.681541103 +0000 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-04-25 08:26:09.681541103 +0000 2024-04-25T08:28:26.037 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-04-25 08:26:09.681541103 +0000 2024-04-25T08:28:26.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:28:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:28:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:28:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 9.7842e-05 s, 5.2 MB/s 2024-04-25T08:28:26.103 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:28:26.160 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:26.160 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:26.160 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:28:26.176 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:28:26.176 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 08:28:16.108364697 +0000 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 08:26:04.887171534 +0000 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 08:26:04.887171534 +0000 2024-04-25T08:28:26.231 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-25 08:26:04.887171534 +0000 2024-04-25T08:28:26.231 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:28:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T08:28:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T08:28:26.293 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000214326 s, 2.4 MB/s 2024-04-25T08:28:26.294 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:28:26.350 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-04-25T08:28:26.405 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:28:26.405 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:26.405 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T08:28:26.405 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:26.406 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:26.406 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 08:28:16.108364697 +0000 2024-04-25T08:28:26.406 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 08:26:05.222162552 +0000 2024-04-25T08:28:26.406 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 08:26:05.222162552 +0000 2024-04-25T08:28:26.406 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-25 08:26:05.222162552 +0000 2024-04-25T08:28:26.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:28:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T08:28:26.468 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T08:28:26.469 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000184726 s, 2.8 MB/s 2024-04-25T08:28:26.470 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:28:26.525 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-04-25T08:28:26.580 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 08:28:16.108364697 +0000 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 08:26:05.556153596 +0000 2024-04-25T08:28:26.581 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 08:26:05.556153596 +0000 2024-04-25T08:28:26.582 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-25 08:26:05.556153596 +0000 2024-04-25T08:28:26.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:28:26.642 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T08:28:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T08:28:26.643 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000193838 s, 2.6 MB/s 2024-04-25T08:28:26.644 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:28:26.702 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-04-25T08:28:26.756 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:28:26.756 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:28:26.756 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T08:28:26.757 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:28:26.757 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:28:26.757 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-04-25 08:28:16.109364671 +0000 2024-04-25T08:28:26.757 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-04-25 08:26:05.886144748 +0000 2024-04-25T08:28:26.757 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-04-25 08:26:05.886144748 +0000 2024-04-25T08:28:26.757 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-04-25 08:26:05.886144748 +0000 2024-04-25T08:28:26.757 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:28:26.820 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-04-25T08:28:26.820 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-04-25T08:28:26.820 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000187494 s, 2.7 MB/s 2024-04-25T08:28:26.821 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:28:26.877 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:26.877 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:28:26.877 INFO:tasks.ceph:Generating config... 2024-04-25T08:28:26.878 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T08:28:26.878 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T08:28:26.878 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T08:28:26.878 INFO:tasks.ceph:[client] pid file = 2024-04-25T08:28:26.878 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T08:28:26.878 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T08:28:26.878 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T08:28:26.878 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T08:28:26.879 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T08:28:26.880 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T08:28:26.881 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T08:28:26.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T08:28:26.925 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T08:28:26.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T08:28:26.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:28:27.032 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.60'), ('mon.b', '172.21.15.100'), ('mon.c', '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]')] 2024-04-25T08:28:27.033 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T08:28:27.034 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:27.034 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:28:27.088 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --add b 172.21.15.100 --addv c '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid 46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2024-04-25T08:28:27.176556+0000 2024-04-25T08:28:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-04-25T08:28:27.176556+0000 2024-04-25T08:28:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 17 (quincy) 2024-04-25T08:28:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2024-04-25T08:28:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-04-25T08:28:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:1: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.b 2024-04-25T08:28:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:2: [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon.c 2024-04-25T08:28:27.175 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T08:28:27.176 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:28:27.191 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de... 2024-04-25T08:28:27.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:28:27.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:28:27.272 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-04-25T08:28:27.272 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-04-25T08:28:27.272 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-04-25T08:28:27.273 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-04-25T08:28:27.274 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-04-25T08:28:27.275 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on skipped message = False 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-04-25T08:28:27.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.277 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 20 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 20 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 10 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-04-25T08:28:27.278 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-04-25T08:28:27.279 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.280 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: mon op complaint time = 120 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: client mount timeout = 600 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: debug client = 20 2024-04-25T08:28:27.281 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = "" 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout:[mds] 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds = 20 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout: debug mds balancer = 20 2024-04-25T08:28:27.282 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: mds cache max size = 1000 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug frag = True 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: mds debug scatterstat = True 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: mds log max segments = 1 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: mds op complaint time = 180 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: mds verify scatter = True 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: osd op complaint time = 180 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: rados mon op timeout = 900 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi060.stdout: rados osd op timeout = 900 2024-04-25T08:28:27.283 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:28:27.284 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-04-25T08:28:27.285 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.60,172.21.15.100,[v2:172.21.15.100:3301,v1:172.21.15.100:6790]" 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on skipped message = False 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.286 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-04-25T08:28:27.287 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-04-25T08:28:27.288 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 20 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 20 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 10 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T08:28:27.289 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-04-25T08:28:27.290 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-04-25T08:28:27.291 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = "" 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-04-25T08:28:27.292 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: mds cache max size = 1000 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: mds log max segments = 1 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-04-25T08:28:27.293 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-04-25T08:28:27.296 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T08:28:27.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T08:28:27.390 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T08:28:27.390 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:27.390 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T08:28:27.405 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:27.405 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T08:28:27.460 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:28:27.461 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:27.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:28:27.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:28:27.533 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:27.534 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:27.589 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:28:27.589 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:27.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:28:27.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:28:27.621 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:27.622 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:27.677 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T08:28:27.677 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T08:28:27.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:28:27.727 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:28:27.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:28:27.784 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:28:27.787 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T08:28:27.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:28:27.837 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:28:27.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T08:28:27.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:28:27.952 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:28:27.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T08:28:27.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:28:28.027 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:28:28.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T08:28:28.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:28:28.141 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:28:28.143 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T08:28:28.166 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T08:28:28.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T08:28:28.206 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T08:28:28.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T08:28:28.299 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T08:28:28.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T08:28:28.351 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T08:28:28.362 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T08:28:28.441 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T08:28:28.453 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T08:28:28.453 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:28:28.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T08:28:28.478 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:28.478 INFO:tasks.ceph:role: osd.0 2024-04-25T08:28:28.478 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:28:28.478 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:28:28.542 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:28.543 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:28.575 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T08:28:28.577 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T08:28:28.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T08:28:28.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T08:28:28.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T08:28:28.780 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:28.780 INFO:tasks.ceph:role: osd.1 2024-04-25T08:28:28.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:28:28.780 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:28:28.843 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:28.843 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:28.844 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:28.877 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T08:28:28.878 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T08:28:28.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T08:28:28.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T08:28:29.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T08:28:29.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:29.079 INFO:tasks.ceph:role: osd.2 2024-04-25T08:28:29.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:28:29.079 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:29.142 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:29.174 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T08:28:29.176 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T08:28:29.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T08:28:29.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T08:28:29.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T08:28:29.380 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:29.380 INFO:tasks.ceph:role: osd.3 2024-04-25T08:28:29.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:28:29.380 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:29.444 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:29.445 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:29.445 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:29.477 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-04-25T08:28:29.479 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-04-25T08:28:29.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T08:28:29.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T08:28:29.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:29.717+0000 7f5da9c1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T08:28:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:29.717+0000 7f5da9c1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T08:28:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:29.717+0000 7f5da9c1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:28:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:29.717+0000 7f5da9c1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:28:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:29.717+0000 7f5da9c1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:28:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:29.718+0000 7f5da9c1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T08:28:32.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T08:28:32.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:32.687+0000 7fd3d0f1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T08:28:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:32.687+0000 7fd3d0f1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T08:28:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:32.687+0000 7fd3d0f1f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:28:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:32.687+0000 7fd3d0f1f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:28:32.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:32.687+0000 7fd3d0f1f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:28:32.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:32.687+0000 7fd3d0f1f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T08:28:35.532 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T08:28:35.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:35.652+0000 7f005cf4c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T08:28:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:35.652+0000 7f005cf4c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T08:28:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:35.652+0000 7f005cf4c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:28:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:35.652+0000 7f005cf4c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:28:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:35.652+0000 7f005cf4c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:28:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:35.653+0000 7f005cf4c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T08:28:38.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T08:28:38.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:38.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:38.616+0000 7fc28fe32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T08:28:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:38.616+0000 7fc28fe32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T08:28:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:38.616+0000 7fc28fe32640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:28:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:38.616+0000 7fc28fe32640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:28:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:38.616+0000 7fc28fe32640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:28:38.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:38.617+0000 7fc28fe32640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T08:28:41.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T08:28:41.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T08:28:41.518 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:41.518 INFO:tasks.ceph:role: osd.4 2024-04-25T08:28:41.519 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:28:41.519 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:41.594 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:41.595 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:41.595 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:42.081 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T08:28:42.083 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T08:28:42.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T08:28:42.198 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T08:28:42.230 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T08:28:42.294 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:42.294 INFO:tasks.ceph:role: osd.5 2024-04-25T08:28:42.294 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:28:42.294 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:42.365 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:42.366 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:42.366 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:42.366 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:42.816 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T08:28:42.818 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T08:28:42.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T08:28:42.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T08:28:42.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T08:28:43.019 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:43.020 INFO:tasks.ceph:role: osd.6 2024-04-25T08:28:43.020 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:28:43.020 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:28:43.089 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:43.089 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:43.089 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:43.090 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:43.579 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T08:28:43.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T08:28:43.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T08:28:43.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T08:28:43.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T08:28:43.767 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:28:43.767 INFO:tasks.ceph:role: osd.7 2024-04-25T08:28:43.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:28:43.767 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:28:43.837 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:28:43.837 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:28:43.837 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:28:43.837 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:28:43.837 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:28:43.838 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:28:43.838 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:28:43.838 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:28:43.838 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:28:43.838 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:28:44.330 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-04-25T08:28:44.332 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-04-25T08:28:44.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T08:28:44.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T08:28:44.490 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:44.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:44.585+0000 7f12ab81e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T08:28:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:44.587+0000 7f12ab81e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T08:28:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:44.587+0000 7f12ab81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:28:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:44.587+0000 7f12ab81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:28:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:44.587+0000 7f12ab81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:28:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:44.592+0000 7f12ab81e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T08:28:47.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T08:28:47.472 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:47.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:47.561+0000 7f32f31fc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T08:28:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:47.562+0000 7f32f31fc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T08:28:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:47.562+0000 7f32f31fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:28:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:47.562+0000 7f32f31fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:28:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:47.562+0000 7f32f31fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:28:47.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:47.562+0000 7f32f31fc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T08:28:50.409 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T08:28:50.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:50.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:50.605+0000 7ff56aa0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T08:28:50.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:50.605+0000 7ff56aa0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T08:28:50.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:50.605+0000 7ff56aa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:28:50.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:50.605+0000 7ff56aa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:28:50.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:50.605+0000 7ff56aa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:28:50.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:50.606+0000 7ff56aa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T08:28:53.449 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T08:28:53.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:53.557+0000 7f62635dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T08:28:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:53.557+0000 7f62635dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T08:28:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:53.557+0000 7f62635dc640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:28:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:53.557+0000 7f62635dc640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:28:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:53.557+0000 7f62635dc640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:28:53.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:53.558+0000 7f62635dc640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T08:28:55.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:28:55.304 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:28:56.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T08:28:56.426 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T08:28:56.426 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T08:28:56.455 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T08:28:56.523 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T08:28:56.591 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T08:28:56.658 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T08:28:56.725 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T08:28:56.792 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:56.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T08:28:56.859 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:56.859 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T08:28:56.887 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:56.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T08:28:56.954 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:56.954 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T08:28:57.020 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:57.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T08:28:57.085 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:57.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T08:28:57.150 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:57.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T08:28:57.215 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:57.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T08:28:57.279 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:57.279 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T08:28:57.298 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:57.298 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T08:28:57.356 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:57.356 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T08:28:57.371 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:57.371 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T08:28:57.426 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T08:28:57.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:28:57.428 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:28:57.469 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.y] 2024-04-25T08:28:57.469 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A== 2024-04-25T08:28:57.469 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.a] 2024-04-25T08:28:57.469 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ== 2024-04-25T08:28:57.469 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.c] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.b] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA== 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout:[mds.d] 2024-04-25T08:28:57.470 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.4] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.5] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.6] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.7] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[client.2] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA== 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2024-04-25T08:28:57.471 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw== 2024-04-25T08:28:57.472 INFO:teuthology.orchestra.run.smithi060.stdout:[client.3] 2024-04-25T08:28:57.472 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag== 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A== 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ== 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ== 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A== 2024-04-25T08:28:57.488 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw== 2024-04-25T08:28:57.489 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg== 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg== 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ== 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout:[client.2] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA== 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout:[client.1] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw== 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout:[client.3] 2024-04-25T08:28:57.490 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag== 2024-04-25T08:28:57.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:28:57.516 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:28:57.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.690 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.790 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:57.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:28:57.890 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:28:57.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:57.993 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:28:58.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.196 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:28:58.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.301 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.407 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:28:58.458 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T08:28:58.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T08:28:58.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:28:58.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T08:28:58.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T08:28:58.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:28:58.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T08:28:58.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T08:28:58.803 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:28:58.896 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T08:28:58.919 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:58.921 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:28:58.974 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T08:28:58.974 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T08:28:58.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T08:28:58.977 INFO:tasks.ceph.mon.a:Started 2024-04-25T08:28:58.977 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T08:28:58.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T08:28:59.016 INFO:tasks.ceph.mon.b:Started 2024-04-25T08:28:59.016 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T08:28:59.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T08:28:59.018 INFO:tasks.ceph.mon.c:Started 2024-04-25T08:28:59.019 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T08:28:59.019 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T08:28:59.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T08:28:59.021 INFO:tasks.ceph.mgr.y:Started 2024-04-25T08:28:59.021 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T08:28:59.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T08:28:59.023 INFO:tasks.ceph.mgr.x:Started 2024-04-25T08:28:59.023 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T08:28:59.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T08:28:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.126+0000 7f723e85d640 1 Processor -- start 2024-04-25T08:28:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.126+0000 7f723e85d640 1 -- start start 2024-04-25T08:28:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f723e85d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f723e85d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 0x7f72380ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f723e85d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7238161620 con 0x7f72380b54a0 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f723e85d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380b7120 con 0x7f7238161890 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f723e85d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7238161e00 con 0x7f7238161240 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f72377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 0x7f72380ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f72377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 0x7f72380ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35824/0 (socket says 172.21.15.60:35824) 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f72377fe640 1 -- 172.21.15.60:0/400511311 learned_addr learned my addr 172.21.15.60:0/400511311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.127+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f72377fe640 1 -- 172.21.15.60:0/400511311 >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 msgr2=0x7f72380ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f72377fe640 1 -- 172.21.15.60:0/400511311 >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 msgr2=0x7f72380ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f72377fe640 1 --2- 172.21.15.60:0/400511311 >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 0x7f72380ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:28:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f7237fff640 1 -- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 msgr2=0x7f72380b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:28:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f7237fff640 1 -- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 msgr2=0x7f72380b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:28:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:28:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f72377fe640 1 --2- 172.21.15.60:0/400511311 >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 0x7f72380ceac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:28:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.128+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b6be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:28:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.289+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530536654 0 0) 0x7f7238161620 con 0x7f72380b54a0 2024-04-25T08:28:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.289+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7220003520 con 0x7f72380b54a0 2024-04-25T08:28:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 260625450 0 0) 0x7f7220003520 con 0x7f72380b54a0 2024-04-25T08:28:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 msgr2=0x7f72380b6be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:28:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b6be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 msgr2=0x7f72380ceac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:28:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 --2- 172.21.15.60:0/400511311 >> v2:172.21.15.60:3300/0 conn(0x7f7238161890 0x7f72380ceac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72381620e0 con 0x7f72380b54a0 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f72300019f0 con 0x7f72380b54a0 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 >> v1:172.21.15.100:6789/0 conn(0x7f72380b54a0 legacy=0x7f72380d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7220004030 0x7f7220004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238161e00 con 0x7f7220004030 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722000baa0 con 0x7f7220004d40 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722000bca0 con 0x7f7238161240 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f72377fe640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7220004030 0x7f7220004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f723cdfa640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7237fff640 1 -- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7220004d40 msgr2=0x7f72200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7237fff640 1 -- 172.21.15.60:0/400511311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7220004030 msgr2=0x7f7220004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:28:59.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.295+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7220004030 0x7f7220004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.296+0000 7f7237fff640 1 -- 172.21.15.60:0/400511311 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7220003520 con 0x7f7238161240 2024-04-25T08:28:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.296+0000 7f72377fe640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7220004030 0x7f7220004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:28:59.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.296+0000 7f723cdfa640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7220004d40 0x7f72200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.296+0000 7f7237fff640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72200039b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f722800b9e0 tx=0x7f7220005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9bed2ef371d07fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228030070 con 0x7f7238161240 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722802adb0 con 0x7f7238161240 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f7236ffd640 1 -- 172.21.15.60:0/400511311 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722800b5e0 con 0x7f7238161240 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f723e85d640 1 -- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 msgr2=0x7f72200039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f723e85d640 1 --2- 172.21.15.60:0/400511311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72200039b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f722800b9e0 tx=0x7f7220005830 comp rx=0 tx=0).stop 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f723cdfa640 1 -- 172.21.15.60:0/400511311 reap_dead start 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f723e85d640 1 -- 172.21.15.60:0/400511311 shutdown_connections 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f723e85d640 1 -- 172.21.15.60:0/400511311 >> 172.21.15.60:0/400511311 conn(0x7f72380d04a0 msgr2=0x7f72380d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:28:59.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.297+0000 7f723e85d640 1 -- 172.21.15.60:0/400511311 shutdown_connections 2024-04-25T08:28:59.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.298+0000 7f723e85d640 1 -- 172.21.15.60:0/400511311 wait complete. 2024-04-25T08:28:59.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.299+0000 7f723e85d640 1 Processor -- start 2024-04-25T08:28:59.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.299+0000 7f723e85d640 1 -- start start 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 0x7f72380ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72380ca2e0 0x7f72380ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380d15f0 con 0x7f72380c9e60 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72381626a0 con 0x7f72380ca2e0 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380cf000 con 0x7f7238161240 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 0x7f72380ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 0x7f72380ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35854/0 (socket says 172.21.15.60:35854) 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 -- 172.21.15.60:0/790361960 learned_addr learned my addr 172.21.15.60:0/790361960 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723cdfa640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72380ca2e0 0x7f72380ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f7237fff640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 -- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 msgr2=0x7f72380b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 -- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72380ca2e0 msgr2=0x7f72380ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72380ca2e0 0x7f72380ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 -- 172.21.15.60:0/790361960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380c8340 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f72377fe640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 0x7f72380ac3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f722c0116c0 tx=0x7f722c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c019070 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c004220 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380c8620 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c00f720 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.300+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380c8be0 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.301+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f722c00f8c0 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.301+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f722c015460 con 0x7f72380c9e60 2024-04-25T08:28:59.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.301+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72381f3f50 con 0x7f72380c9e60 2024-04-25T08:28:59.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.304+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f722c00fca0 con 0x7f72380c9e60 2024-04-25T08:28:59.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.371+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7238073a00 con 0x7f72380c9e60 2024-04-25T08:28:59.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.372+0000 7f7234ff9640 1 -- 172.21.15.60:0/790361960 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f722c003610 con 0x7f72380c9e60 2024-04-25T08:28:59.371 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-04-25T08:28:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 msgr2=0x7f72380ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 0x7f72380ac3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f722c0116c0 tx=0x7f722c011ac0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 shutdown_connections 2024-04-25T08:28:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72380ca2e0 0x7f72380ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72380c9e60 0x7f72380ac3b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 --2- 172.21.15.60:0/790361960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7238161240 0x7f72380b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 >> 172.21.15.60:0/790361960 conn(0x7f72380d04a0 msgr2=0x7f723806ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:28:59.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 shutdown_connections 2024-04-25T08:28:59.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:28:59.374+0000 7f723e85d640 1 -- 172.21.15.60:0/790361960 wait complete. 2024-04-25T08:28:59.383 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T08:28:59.383 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T08:28:59.383 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:59.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T08:28:59.401 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:28:59.404+0000 7fd95ce12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:28:59.402 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:28:59.404+0000 7f2190d1d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:28:59.407 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:59.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:59.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T08:28:59.535 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:28:59.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T08:28:59.544 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:28:59.547+0000 7fd95ce12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:28:59.556 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:28:59.558+0000 7f2190d1d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:28:59.579 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:59.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T08:28:59.598 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:28:59.600+0000 7fd95ce12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:28:59.603 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:59.603 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T08:28:59.612 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:28:59.614+0000 7f2190d1d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:28:59.667 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:59.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T08:28:59.730 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:28:59.731 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T08:28:59.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 0309f754-7c34-473b-99a9-f12a336ecab5 0 2024-04-25T08:28:59.837 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:28:59.840+0000 7fd95ce12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:28:59.867 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:28:59.869+0000 7f2190d1d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:28:59.885 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:28:59.888+0000 7fd95ce12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:28:59.914 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:28:59.916+0000 7f2190d1d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:28:59.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.933+0000 7f8e76f68640 1 Processor -- start 2024-04-25T08:28:59.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.933+0000 7f8e76f68640 1 -- start start 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e76f68640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e700c6010 0x7f8e700ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e76f68640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700cd520 0x7f8e700cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e76f68640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e700c6940 con 0x7f8e7015a9f0 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e76f68640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e700b0020 con 0x7f8e700c6010 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e76f68640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700cb5a0 con 0x7f8e700cd520 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e75505640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700cd520 0x7f8e700cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e74d04640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8e7015a9f0 0x7f8e700cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:37362/0 (socket says 172.21.15.100:37362) 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e74d04640 1 -- 172.21.15.100:0/2886395251 learned_addr learned my addr 172.21.15.100:0/2886395251 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e6ffff640 1 --2- 172.21.15.100:0/2886395251 >> v2:172.21.15.60:3300/0 conn(0x7f8e700c6010 0x7f8e700ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e6f7fe640 1 -- 172.21.15.100:0/2886395251 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889135636 0 0) 0x7f8e64001110 con 0x7f8e7015a9f0 2024-04-25T08:28:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e6f7fe640 1 -- 172.21.15.100:0/2886395251 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e60003520 con 0x7f8e7015a9f0 2024-04-25T08:28:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e75505640 1 -- 172.21.15.100:0/2886395251 >> v2:172.21.15.60:3300/0 conn(0x7f8e700c6010 msgr2=0x7f8e700ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:28:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e75505640 1 --2- 172.21.15.100:0/2886395251 >> v2:172.21.15.60:3300/0 conn(0x7f8e700c6010 0x7f8e700ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e75505640 1 -- 172.21.15.100:0/2886395251 >> v1:172.21.15.100:6789/0 conn(0x7f8e7015a9f0 legacy=0x7f8e700cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.934+0000 7f8e75505640 1 -- 172.21.15.100:0/2886395251 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7015add0 con 0x7f8e700cd520 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.935+0000 7f8e75505640 1 --2- 172.21.15.100:0/2886395251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700cd520 0x7f8e700cba30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009d90 tx=0x7f8e6802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=535a91bef0c43a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.935+0000 7f8e6f7fe640 1 -- 172.21.15.100:0/2886395251 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007740 con 0x7f8e700cd520 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.935+0000 7f8e6f7fe640 1 -- 172.21.15.100:0/2886395251 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e680078e0 con 0x7f8e700cd520 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.935+0000 7f8e6f7fe640 1 -- 172.21.15.100:0/2886395251 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68007c90 con 0x7f8e700cd520 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 -- 172.21.15.100:0/2886395251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700cd520 msgr2=0x7f8e700cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2886395251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700cd520 0x7f8e700cba30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009d90 tx=0x7f8e6802a660 comp rx=0 tx=0).stop 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 -- 172.21.15.100:0/2886395251 shutdown_connections 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2886395251 >> v2:172.21.15.60:3300/0 conn(0x7f8e700c6010 0x7f8e700ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2886395251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700cd520 0x7f8e700cba30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 -- 172.21.15.100:0/2886395251 >> 172.21.15.100:0/2886395251 conn(0x7f8e700b2390 msgr2=0x7f8e700b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:28:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 -- 172.21.15.100:0/2886395251 shutdown_connections 2024-04-25T08:28:59.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.936+0000 7f8e76f68640 1 -- 172.21.15.100:0/2886395251 wait complete. 2024-04-25T08:28:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.937+0000 7f8e76f68640 1 Processor -- start 2024-04-25T08:28:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.937+0000 7f8e76f68640 1 -- start start 2024-04-25T08:28:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e76f68640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700c6010 0x7f8e70152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e76f68640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 0x7f8e700cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e76f68640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015a9f0 0x7f8e700cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e76f68640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700cb230 con 0x7f8e7015a9f0 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e76f68640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700caf50 con 0x7f8e700cd520 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e76f68640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700cbf70 con 0x7f8e700c6010 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e6ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 0x7f8e700cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e74d04640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700c6010 0x7f8e70152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e6ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 0x7f8e700cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50144/0 (socket says 172.21.15.100:50144) 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e6ffff640 1 -- 172.21.15.100:0/2748755395 learned_addr learned my addr 172.21.15.100:0/2748755395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e75505640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015a9f0 0x7f8e700cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e74d04640 1 -- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 msgr2=0x7f8e700cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e74d04640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 0x7f8e700cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e74d04640 1 -- 172.21.15.100:0/2748755395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015a9f0 msgr2=0x7f8e700cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e74d04640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015a9f0 0x7f8e700cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e74d04640 1 -- 172.21.15.100:0/2748755395 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700c7c10 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.938+0000 7f8e6ffff640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 0x7f8e700cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e74d04640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700c6010 0x7f8e70152d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400c3c0 tx=0x7f8e6400c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64017070 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e64005ce0 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e700c7ef0 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640072f0 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7009ab00 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8e6401a070 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.939+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e64012740 con 0x7f8e700c6010 2024-04-25T08:28:59.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.940+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7006cc80 con 0x7f8e700c6010 2024-04-25T08:28:59.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:28:59.942+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8e64021020 con 0x7f8e700c6010 2024-04-25T08:29:00.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.014+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0309f754-7c34-473b-99a9-f12a336ecab5", "id": 0} v 0) v1 -- 0x7f8e70073a00 con 0x7f8e700c6010 2024-04-25T08:29:00.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.020+0000 7f8e6d7fa640 1 -- 172.21.15.100:0/2748755395 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0309f754-7c34-473b-99a9-f12a336ecab5", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8e700c6940 con 0x7f8e700c6010 2024-04-25T08:29:00.017 INFO:teuthology.orchestra.run.smithi100.stdout:0 2024-04-25T08:29:00.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.020+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700c6010 msgr2=0x7f8e70152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.020+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700c6010 0x7f8e70152d20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400c3c0 tx=0x7f8e6400c7c0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 shutdown_connections 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7015a9f0 0x7f8e700cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e700cd520 0x7f8e700cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 --2- 172.21.15.100:0/2748755395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e700c6010 0x7f8e70152d20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 >> 172.21.15.100:0/2748755395 conn(0x7f8e700b2390 msgr2=0x7f8e7006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 shutdown_connections 2024-04-25T08:29:00.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.021+0000 7f8e76f68640 1 -- 172.21.15.100:0/2748755395 wait complete. 2024-04-25T08:29:00.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 7813a924-5ea3-47ac-a55e-cb91b1c687d9 1 2024-04-25T08:29:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.168+0000 7f2f9dd29640 1 Processor -- start 2024-04-25T08:29:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.168+0000 7f2f9dd29640 1 -- start start 2024-04-25T08:29:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f9dd29640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f9815a9f0 0x7f2f980c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f9dd29640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f980cdbf0 0x7f2f980ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f9dd29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980cd520 0x7f2f980ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f9dd29640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980b0020 con 0x7f2f980cdbf0 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f9dd29640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980cb5a0 con 0x7f2f9815a9f0 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f9dd29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980c6940 con 0x7f2f980cd520 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f9815a9f0 0x7f2f980c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f97fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980cd520 0x7f2f980ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f9815a9f0 0x7f2f980c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50158/0 (socket says 172.21.15.100:50158) 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 -- 172.21.15.100:0/4103197315 learned_addr learned my addr 172.21.15.100:0/4103197315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f96ffd640 1 --2- 172.21.15.100:0/4103197315 >> v2:172.21.15.60:3300/0 conn(0x7f2f980cdbf0 0x7f2f980ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 -- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980cd520 msgr2=0x7f2f980ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 --2- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980cd520 0x7f2f980ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 -- 172.21.15.100:0/4103197315 >> v2:172.21.15.60:3300/0 conn(0x7f2f980cdbf0 msgr2=0x7f2f980ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 --2- 172.21.15.100:0/4103197315 >> v2:172.21.15.60:3300/0 conn(0x7f2f980cdbf0 0x7f2f980ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f977fe640 1 -- 172.21.15.100:0/4103197315 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9815add0 con 0x7f2f9815a9f0 2024-04-25T08:29:00.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.169+0000 7f2f97fff640 1 --2- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980cd520 0x7f2f980ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.170+0000 7f2f977fe640 1 --2- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815a9f0 0x7f2f980c6010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f980c77e0 tx=0x7f2f8802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2eb81d2dbbd3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.170+0000 7f2f967fc640 1 -- 172.21.15.100:0/4103197315 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88037070 con 0x7f2f9815a9f0 2024-04-25T08:29:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.170+0000 7f2f967fc640 1 -- 172.21.15.100:0/4103197315 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88032440 con 0x7f2f9815a9f0 2024-04-25T08:29:00.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.170+0000 7f2f967fc640 1 -- 172.21.15.100:0/4103197315 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88032810 con 0x7f2f9815a9f0 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.170+0000 7f2f9dd29640 1 -- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815a9f0 msgr2=0x7f2f980c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.170+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815a9f0 0x7f2f980c6010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f980c77e0 tx=0x7f2f8802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 -- 172.21.15.100:0/4103197315 shutdown_connections 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980cd520 0x7f2f980ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/4103197315 >> v2:172.21.15.60:3300/0 conn(0x7f2f980cdbf0 0x7f2f980ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/4103197315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815a9f0 0x7f2f980c6010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 -- 172.21.15.100:0/4103197315 >> 172.21.15.100:0/4103197315 conn(0x7f2f980b2390 msgr2=0x7f2f980b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 -- 172.21.15.100:0/4103197315 shutdown_connections 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.171+0000 7f2f9dd29640 1 -- 172.21.15.100:0/4103197315 wait complete. 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.172+0000 7f2f9dd29640 1 Processor -- start 2024-04-25T08:29:00.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.172+0000 7f2f9dd29640 1 -- start start 2024-04-25T08:29:00.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980cd520 0x7f2f980c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 0x7f2f981f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f9815a9f0 0x7f2f981f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980cb230 con 0x7f2f980cd520 2024-04-25T08:29:00.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980caf50 con 0x7f2f980cdbf0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980ce070 con 0x7f2f9815a9f0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 0x7f2f981f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 0x7f2f981f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50162/0 (socket says 172.21.15.100:50162) 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f97fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f9815a9f0 0x7f2f981f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 -- 172.21.15.100:0/1924769008 learned_addr learned my addr 172.21.15.100:0/1924769008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f977fe640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980cd520 0x7f2f980c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 -- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f9815a9f0 msgr2=0x7f2f981f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f9815a9f0 0x7f2f981f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 -- 172.21.15.100:0/1924769008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980cd520 msgr2=0x7f2f980c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980cd520 0x7f2f980c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 -- 172.21.15.100:0/1924769008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f981f4e50 con 0x7f2f980cdbf0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f96ffd640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 0x7f2f981f1530 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00e9b0 tx=0x7f2f8c00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c017070 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c00c940 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f981f21b0 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c00cd30 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.173+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980ad6a0 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.174+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2f8c01a070 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.174+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2f8c013470 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.175+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9806cc80 con 0x7f2f980cdbf0 2024-04-25T08:29:00.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.177+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2f981f4e50 con 0x7f2f980cdbf0 2024-04-25T08:29:00.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.249+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7813a924-5ea3-47ac-a55e-cb91b1c687d9", "id": 1} v 0) v1 -- 0x7f2f98073a00 con 0x7f2f980cdbf0 2024-04-25T08:29:00.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.257+0000 7f2f7ffff640 1 -- 172.21.15.100:0/1924769008 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7813a924-5ea3-47ac-a55e-cb91b1c687d9", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2f8c006020 con 0x7f2f980cdbf0 2024-04-25T08:29:00.254 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-25T08:29:00.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 msgr2=0x7f2f981f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 0x7f2f981f1530 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c00e9b0 tx=0x7f2f8c00edb0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 shutdown_connections 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f9815a9f0 0x7f2f981f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f980cdbf0 0x7f2f981f1530 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 --2- 172.21.15.100:0/1924769008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980cd520 0x7f2f980c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 >> 172.21.15.100:0/1924769008 conn(0x7f2f980b2390 msgr2=0x7f2f9806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 shutdown_connections 2024-04-25T08:29:00.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.258+0000 7f2f9dd29640 1 -- 172.21.15.100:0/1924769008 wait complete. 2024-04-25T08:29:00.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841 2 2024-04-25T08:29:00.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.365+0000 7fcd9835c640 1 Processor -- start 2024-04-25T08:29:00.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.365+0000 7fcd9835c640 1 -- start start 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.366+0000 7fcd9835c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd9015bb20 0x7fcd900cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.366+0000 7fcd9835c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd900cb190 con 0x7fcd900d00e0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.366+0000 7fcd9835c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd900cb470 con 0x7fcd900ccdd0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.366+0000 7fcd9835c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd900cb750 con 0x7fcd9015bb20 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.366+0000 7fcd958f7640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcd900ccdd0 0x7fcd900cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:37378/0 (socket says 172.21.15.100:37378) 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.366+0000 7fcd958f7640 1 -- 172.21.15.100:0/1227356144 learned_addr learned my addr 172.21.15.100:0/1227356144 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd960f8640 1 --2- 172.21.15.100:0/1227356144 >> v2:172.21.15.60:3300/0 conn(0x7fcd9015bb20 0x7fcd900cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421061742 0 0) 0x7fcd900cb470 con 0x7fcd900ccdd0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd7c003520 con 0x7fcd900ccdd0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586294264 0 0) 0x7fcd900cb190 con 0x7fcd900d00e0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd900cb470 con 0x7fcd900d00e0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2507506833 0 0) 0x7fcd7c003520 con 0x7fcd900ccdd0 2024-04-25T08:29:00.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd960f8640 1 -- 172.21.15.100:0/1227356144 >> v1:172.21.15.100:6789/0 conn(0x7fcd900ccdd0 legacy=0x7fcd900cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd960f8640 1 -- 172.21.15.100:0/1227356144 >> v1:172.21.15.60:6789/0 conn(0x7fcd900d00e0 legacy=0x7fcd900ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.367+0000 7fcd960f8640 1 -- 172.21.15.100:0/1227356144 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900cba30 con 0x7fcd9015bb20 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd960f8640 1 --2- 172.21.15.100:0/1227356144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900cc230 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009720 tx=0x7fcd8002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b814ff23f9b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80037070 con 0x7fcd9015bb20 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8002abe0 con 0x7fcd9015bb20 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd950f6640 1 -- 172.21.15.100:0/1227356144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c001670 con 0x7fcd9015bb20 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 -- 172.21.15.100:0/1227356144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 msgr2=0x7fcd900cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 --2- 172.21.15.100:0/1227356144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900cc230 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009720 tx=0x7fcd8002a0d0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 -- 172.21.15.100:0/1227356144 shutdown_connections 2024-04-25T08:29:00.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 --2- 172.21.15.100:0/1227356144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900cc230 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 -- 172.21.15.100:0/1227356144 >> 172.21.15.100:0/1227356144 conn(0x7fcd900b2390 msgr2=0x7fcd900b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 -- 172.21.15.100:0/1227356144 shutdown_connections 2024-04-25T08:29:00.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.368+0000 7fcd9835c640 1 -- 172.21.15.100:0/1227356144 wait complete. 2024-04-25T08:29:00.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 Processor -- start 2024-04-25T08:29:00.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 -- start start 2024-04-25T08:29:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 0x7fcd901e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd900d00e0 0x7fcd900c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9015bf00 con 0x7fcd9015bb20 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd900cc770 con 0x7fcd900ccdd0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd9835c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9015c080 con 0x7fcd900d00e0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd960f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 0x7fcd901e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd960f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 0x7fcd901e7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50166/0 (socket says 172.21.15.100:50166) 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd960f8640 1 -- 172.21.15.100:0/3891894097 learned_addr learned my addr 172.21.15.100:0/3891894097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd958f7640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd900d00e0 0x7fcd900c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.370+0000 7fcd968f9640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd960f8640 1 -- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd900d00e0 msgr2=0x7fcd900c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd960f8640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd900d00e0 0x7fcd900c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd960f8640 1 -- 172.21.15.100:0/3891894097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 msgr2=0x7fcd900b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd960f8640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd960f8640 1 -- 172.21.15.100:0/3891894097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd900b0740 con 0x7fcd900ccdd0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd960f8640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 0x7fcd901e7490 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd80035040 tx=0x7fcd800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80041020 con 0x7fcd900ccdd0 2024-04-25T08:29:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8003d620 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd900b0a20 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80037040 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd901e79d0 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcd80044020 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.371+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fcd8003b3f0 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.372+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd90073a00 con 0x7fcd900ccdd0 2024-04-25T08:29:00.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.374+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcd8004b030 con 0x7fcd900ccdd0 2024-04-25T08:29:00.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.446+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841", "id": 2} v 0) v1 -- 0x7fcd9006f4c0 con 0x7fcd900ccdd0 2024-04-25T08:29:00.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.449+0000 7fcd86ffd640 1 -- 172.21.15.100:0/3891894097 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd80046400 con 0x7fcd900ccdd0 2024-04-25T08:29:00.447 INFO:teuthology.orchestra.run.smithi100.stdout:2 2024-04-25T08:29:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 msgr2=0x7fcd901e7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 0x7fcd901e7490 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd80035040 tx=0x7fcd800046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 shutdown_connections 2024-04-25T08:29:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd9015bb20 0x7fcd900b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd900d00e0 0x7fcd900c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 --2- 172.21.15.100:0/3891894097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd900ccdd0 0x7fcd901e7490 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 >> 172.21.15.100:0/3891894097 conn(0x7fcd900b2390 msgr2=0x7fcd9006f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 shutdown_connections 2024-04-25T08:29:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.450+0000 7fcd9835c640 1 -- 172.21.15.100:0/3891894097 wait complete. 2024-04-25T08:29:00.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 7228adc9-f853-45ff-aa9b-3dc9e965833b 3 2024-04-25T08:29:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.558+0000 7fd962bb7640 1 Processor -- start 2024-04-25T08:29:00.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.558+0000 7fd962bb7640 1 -- start start 2024-04-25T08:29:00.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd962bb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c072610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd962bb7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c072ca0 con 0x7fd95c15bd70 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd962bb7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c15c260 con 0x7fd95c06c300 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd962bb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c15c540 con 0x7fd95c15b990 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd960953640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd960953640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c072610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:35190/0 (socket says 172.21.15.100:35190) 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd960953640 1 -- 172.21.15.100:0/2570779152 learned_addr learned my addr 172.21.15.100:0/2570779152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd961154640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd95c06c300 0x7fd95c0acd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56288/0 (socket says 172.21.15.100:56288) 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.559+0000 7fd95b7fe640 1 -- 172.21.15.100:0/2570779152 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840046977 0 0) 0x7fd95c15c260 con 0x7fd95c06c300 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.560+0000 7fd95b7fe640 1 -- 172.21.15.100:0/2570779152 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd94c003520 con 0x7fd95c06c300 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.560+0000 7fd960953640 1 -- 172.21.15.100:0/2570779152 >> v1:172.21.15.100:6789/0 conn(0x7fd95c06c300 legacy=0x7fd95c0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.560+0000 7fd960953640 1 -- 172.21.15.100:0/2570779152 >> v1:172.21.15.60:6789/0 conn(0x7fd95c15bd70 legacy=0x7fd95c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.560+0000 7fd960953640 1 -- 172.21.15.100:0/2570779152 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0cb9d0 con 0x7fd95c15b990 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd960953640 1 --2- 172.21.15.100:0/2570779152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c072610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9500098a0 tx=0x7fd95002a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894c60fac5b69049 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd95b7fe640 1 -- 172.21.15.100:0/2570779152 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950037070 con 0x7fd95c15b990 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd95b7fe640 1 -- 172.21.15.100:0/2570779152 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950032640 con 0x7fd95c15b990 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd95b7fe640 1 -- 172.21.15.100:0/2570779152 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950032a10 con 0x7fd95c15b990 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 -- 172.21.15.100:0/2570779152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 msgr2=0x7fd95c072610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 --2- 172.21.15.100:0/2570779152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c072610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9500098a0 tx=0x7fd95002a2e0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 -- 172.21.15.100:0/2570779152 shutdown_connections 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 --2- 172.21.15.100:0/2570779152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c072610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 -- 172.21.15.100:0/2570779152 >> 172.21.15.100:0/2570779152 conn(0x7fd95c071af0 msgr2=0x7fd95c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 -- 172.21.15.100:0/2570779152 shutdown_connections 2024-04-25T08:29:00.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.561+0000 7fd962bb7640 1 -- 172.21.15.100:0/2570779152 wait complete. 2024-04-25T08:29:00.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.563+0000 7fd962bb7640 1 Processor -- start 2024-04-25T08:29:00.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.563+0000 7fd962bb7640 1 -- start start 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 0x7fd95c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd95c15bd70 0x7fd95c14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0af250 con 0x7fd95c15bd70 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0af0d0 con 0x7fd95c06c300 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0af3d0 con 0x7fd95c15b990 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd95bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 0x7fd95c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd95bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c0c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:35198/0 (socket says 172.21.15.100:35198) 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 0x7fd95c157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55104/0 (socket says 172.21.15.100:55104) 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd95bfff640 1 -- 172.21.15.100:0/370265550 learned_addr learned my addr 172.21.15.100:0/370265550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd961154640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd95c15bd70 0x7fd95c14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 -- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 msgr2=0x7fd95c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c0c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 -- 172.21.15.100:0/370265550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd95c15bd70 msgr2=0x7fd95c14f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd95c15bd70 0x7fd95c14f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 -- 172.21.15.100:0/370265550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c14ff10 con 0x7fd95c06c300 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd95bfff640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd960953640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 0x7fd95c157050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd950031000 tx=0x7fd9500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950041020 con 0x7fd95c06c300 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950030770 con 0x7fd95c06c300 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c1576e0 con 0x7fd95c06c300 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950037040 con 0x7fd95c06c300 2024-04-25T08:29:00.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.565+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c157ca0 con 0x7fd95c06c300 2024-04-25T08:29:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.566+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd950044020 con 0x7fd95c06c300 2024-04-25T08:29:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.566+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd95003dbe0 con 0x7fd95c06c300 2024-04-25T08:29:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.566+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c073a00 con 0x7fd95c06c300 2024-04-25T08:29:00.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.568+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd95004b030 con 0x7fd95c06c300 2024-04-25T08:29:00.609 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:00.611+0000 7fd95ce12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:29:00.635 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:00.637+0000 7f2190d1d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:29:00.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.640+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7228adc9-f853-45ff-aa9b-3dc9e965833b", "id": 3} v 0) v1 -- 0x7fd95c0b12c0 con 0x7fd95c06c300 2024-04-25T08:29:00.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.643+0000 7fd9597fa640 1 -- 172.21.15.100:0/370265550 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7228adc9-f853-45ff-aa9b-3dc9e965833b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd95c14ff10 con 0x7fd95c06c300 2024-04-25T08:29:00.641 INFO:teuthology.orchestra.run.smithi100.stdout:3 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 msgr2=0x7fd95c157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 0x7fd95c157050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd950031000 tx=0x7fd9500046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 shutdown_connections 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd95c15bd70 0x7fd95c14f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd95c15b990 0x7fd95c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 --2- 172.21.15.100:0/370265550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd95c06c300 0x7fd95c157050 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 >> 172.21.15.100:0/370265550 conn(0x7fd95c071af0 msgr2=0x7fd95c06ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 shutdown_connections 2024-04-25T08:29:00.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.644+0000 7fd962bb7640 1 -- 172.21.15.100:0/370265550 wait complete. 2024-04-25T08:29:00.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 55bf0b3b-3016-4d56-8b21-2c25c3ee741d 4 2024-04-25T08:29:00.702 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:00.705+0000 7fd95ce12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:29:00.726 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:00.728+0000 7f2190d1d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:29:00.747 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:00.749+0000 7fd95ce12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:29:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.752+0000 7f38ae476640 1 Processor -- start 2024-04-25T08:29:00.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.752+0000 7f38ae476640 1 -- start start 2024-04-25T08:29:00.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.752+0000 7f38ae476640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38a80cbf00 0x7f38a806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.752+0000 7f38ae476640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a80cae60 con 0x7f38a80cc390 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.752+0000 7f38ae476640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a80cb140 con 0x7f38a815dbb0 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.752+0000 7f38ae476640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a80cb420 con 0x7f38a80cbf00 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a7fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f38a815dbb0 0x7f38a806fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56304/0 (socket says 172.21.15.100:56304) 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a7fff640 1 -- 172.21.15.100:0/771833406 learned_addr learned my addr 172.21.15.100:0/771833406 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 --2- 172.21.15.100:0/771833406 >> v2:172.21.15.60:3300/0 conn(0x7f38a80cbf00 0x7f38a806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677570824 0 0) 0x7f38a80cb140 con 0x7f38a815dbb0 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3890003520 con 0x7f38a815dbb0 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122608418 0 0) 0x7f38a80cae60 con 0x7f38a80cc390 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a80cb140 con 0x7f38a80cc390 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574709965 0 0) 0x7f3890003520 con 0x7f38a815dbb0 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 >> v2:172.21.15.60:3300/0 conn(0x7f38a80cbf00 msgr2=0x7f38a806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 --2- 172.21.15.100:0/771833406 >> v2:172.21.15.60:3300/0 conn(0x7f38a80cbf00 0x7f38a806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 >> v1:172.21.15.60:6789/0 conn(0x7f38a80cc390 legacy=0x7f38a806db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80cb700 con 0x7f38a815dbb0 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3898001670 con 0x7f38a815dbb0 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 >> v1:172.21.15.100:6789/0 conn(0x7f38a815dbb0 legacy=0x7f38a806fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3890003b40 0x7f3890003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3890004800 0x7f3890004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38900051c0 0x7f389000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80cb420 con 0x7f38900051c0 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389000c040 con 0x7f3890004800 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389000c240 con 0x7f3890003b40 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3890004800 0x7f3890004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a7fff640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3890003b40 0x7f3890003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38aca13640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38900051c0 0x7f389000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 -- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3890003b40 msgr2=0x7f3890003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3890003b40 0x7f3890003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 -- 172.21.15.100:0/771833406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38900051c0 msgr2=0x7f389000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38900051c0 0x7f389000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a77fe640 1 -- 172.21.15.100:0/771833406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890003520 con 0x7f3890004800 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38aca13640 1 -- 172.21.15.100:0/771833406 reap_dead start 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.753+0000 7f38a7fff640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3890003b40 0x7f3890003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.754+0000 7f38a77fe640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3890004800 0x7f3890004c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f389c009ac0 tx=0x7f3890005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2e8f6a24d9b344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.754+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c036070 con 0x7f3890004800 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.754+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c031400 con 0x7f3890004800 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.754+0000 7f38a6ffd640 1 -- 172.21.15.100:0/771833406 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c009690 con 0x7f3890004800 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 -- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3890004800 msgr2=0x7f3890004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3890004800 0x7f3890004c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f389c009ac0 tx=0x7f3890005830 comp rx=0 tx=0).stop 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 -- 172.21.15.100:0/771833406 shutdown_connections 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 --2- 172.21.15.100:0/771833406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3890004800 0x7f3890004c80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 -- 172.21.15.100:0/771833406 >> 172.21.15.100:0/771833406 conn(0x7f38a8071af0 msgr2=0x7f38a8071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 -- 172.21.15.100:0/771833406 shutdown_connections 2024-04-25T08:29:00.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.755+0000 7f38ae476640 1 -- 172.21.15.100:0/771833406 wait complete. 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.756+0000 7f38ae476640 1 Processor -- start 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.756+0000 7f38ae476640 1 -- start start 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38ae476640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3890004800 0x7f38a80c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38ae476640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 0x7f38a81f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38ae476640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a81f2e50 0x7f38a815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38ae476640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a80cab10 con 0x7f3890004800 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38ae476640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a815df90 con 0x7f38a81f2e50 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38ae476640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a8072ac0 con 0x7f38a81f4700 2024-04-25T08:29:00.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 0x7f38a81f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38aca13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a81f2e50 0x7f38a815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38a77fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 0x7f38a81f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:35208/0 (socket says 172.21.15.100:35208) 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38a77fe640 1 -- 172.21.15.100:0/1381891308 learned_addr learned my addr 172.21.15.100:0/1381891308 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38a7fff640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3890004800 0x7f38a80c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38aca13640 1 -- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 msgr2=0x7f38a81f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38aca13640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 0x7f38a81f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38aca13640 1 -- 172.21.15.100:0/1381891308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3890004800 msgr2=0x7f38a80c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38aca13640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3890004800 0x7f38a80c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38aca13640 1 -- 172.21.15.100:0/1381891308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80c6340 con 0x7f38a81f2e50 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.757+0000 7f38a77fe640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 0x7f38a81f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38aca13640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a81f2e50 0x7f38a815e110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38a806d300 tx=0x7f38a000ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a001a070 con 0x7f38a81f2e50 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0003c40 con 0x7f38a81f2e50 2024-04-25T08:29:00.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80ace90 con 0x7f38a81f2e50 2024-04-25T08:29:00.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0005070 con 0x7f38a81f2e50 2024-04-25T08:29:00.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a80ad450 con 0x7f38a81f2e50 2024-04-25T08:29:00.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f38a001d070 con 0x7f38a81f2e50 2024-04-25T08:29:00.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.758+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f38a000db80 con 0x7f38a81f2e50 2024-04-25T08:29:00.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.759+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a8073a00 con 0x7f38a81f2e50 2024-04-25T08:29:00.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.761+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f38a000d680 con 0x7f38a81f2e50 2024-04-25T08:29:00.771 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:00.773+0000 7f2190d1d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:29:00.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.833+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "55bf0b3b-3016-4d56-8b21-2c25c3ee741d", "id": 4} v 0) v1 -- 0x7f38a80cec50 con 0x7f38a81f2e50 2024-04-25T08:29:00.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.836+0000 7f38a4ff9640 1 -- 172.21.15.100:0/1381891308 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55bf0b3b-3016-4d56-8b21-2c25c3ee741d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f38a000de20 con 0x7f38a81f2e50 2024-04-25T08:29:00.833 INFO:teuthology.orchestra.run.smithi100.stdout:4 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a81f2e50 msgr2=0x7f38a815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a81f2e50 0x7f38a815e110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38a806d300 tx=0x7f38a000ff70 comp rx=0 tx=0).stop 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 shutdown_connections 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38a81f2e50 0x7f38a815e110 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38a81f4700 0x7f38a81f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 --2- 172.21.15.100:0/1381891308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3890004800 0x7f38a80c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 >> 172.21.15.100:0/1381891308 conn(0x7f38a8071af0 msgr2=0x7f38a815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 shutdown_connections 2024-04-25T08:29:00.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.837+0000 7f38ae476640 1 -- 172.21.15.100:0/1381891308 wait complete. 2024-04-25T08:29:00.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new dd77d113-576d-4798-8ee1-6f46b49e7c6e 5 2024-04-25T08:29:00.921 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:00.923+0000 7fd95ce12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:29:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.944+0000 7f97bf122640 1 Processor -- start 2024-04-25T08:29:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.944+0000 7f97bf122640 1 -- start start 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97bf122640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97b80c6010 0x7f97b80ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97bf122640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b80c6940 con 0x7f97b80cd520 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97bf122640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b815add0 con 0x7f97b815a9f0 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97bf122640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b80b0020 con 0x7f97b80c6010 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97bcebe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f97b815a9f0 0x7f97b80cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:56318/0 (socket says 172.21.15.100:56318) 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97bcebe640 1 -- 172.21.15.100:0/1358515283 learned_addr learned my addr 172.21.15.100:0/1358515283 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b7fff640 1 --2- 172.21.15.100:0/1358515283 >> v2:172.21.15.60:3300/0 conn(0x7f97b80c6010 0x7f97b80ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b77fe640 1 -- 172.21.15.100:0/1358515283 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580059232 0 0) 0x7f97b815add0 con 0x7f97b815a9f0 2024-04-25T08:29:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b77fe640 1 -- 172.21.15.100:0/1358515283 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a8005730 con 0x7f97b815a9f0 2024-04-25T08:29:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b7fff640 1 -- 172.21.15.100:0/1358515283 >> v1:172.21.15.100:6789/0 conn(0x7f97b815a9f0 legacy=0x7f97b80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b7fff640 1 -- 172.21.15.100:0/1358515283 >> v1:172.21.15.60:6789/0 conn(0x7f97b80cd520 legacy=0x7f97b80cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b77fe640 1 -- 172.21.15.100:0/1358515283 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80981266 0 0) 0x7f97b0001490 con 0x7f97b80cd520 2024-04-25T08:29:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.945+0000 7f97b7fff640 1 -- 172.21.15.100:0/1358515283 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80c6660 con 0x7f97b80c6010 2024-04-25T08:29:00.944 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:00.946+0000 7f2190d1d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:29:00.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.946+0000 7f97b7fff640 1 --2- 172.21.15.100:0/1358515283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80c6010 0x7f97b80ccfe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97a40097a0 tx=0x7f97a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253ececfe7fc9ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.946+0000 7f97b77fe640 1 -- 172.21.15.100:0/1358515283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4037070 con 0x7f97b80c6010 2024-04-25T08:29:00.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.946+0000 7f97b77fe640 1 -- 172.21.15.100:0/1358515283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a4032440 con 0x7f97b80c6010 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.946+0000 7f97b77fe640 1 -- 172.21.15.100:0/1358515283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4032810 con 0x7f97b80c6010 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 -- 172.21.15.100:0/1358515283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80c6010 msgr2=0x7f97b80ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 --2- 172.21.15.100:0/1358515283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80c6010 0x7f97b80ccfe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f97a40097a0 tx=0x7f97a402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 -- 172.21.15.100:0/1358515283 shutdown_connections 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 --2- 172.21.15.100:0/1358515283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80c6010 0x7f97b80ccfe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 -- 172.21.15.100:0/1358515283 >> 172.21.15.100:0/1358515283 conn(0x7f97b80b2390 msgr2=0x7f97b80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 -- 172.21.15.100:0/1358515283 shutdown_connections 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.947+0000 7f97bf122640 1 -- 172.21.15.100:0/1358515283 wait complete. 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.948+0000 7f97bf122640 1 Processor -- start 2024-04-25T08:29:00.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.948+0000 7f97bf122640 1 -- start start 2024-04-25T08:29:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bf122640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 0x7f97b814d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bf122640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80cd520 0x7f97b80ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bf122640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b815a9f0 0x7f97b80c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bf122640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80cb230 con 0x7f97b80cd520 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bf122640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80cb5a0 con 0x7f97b80c6010 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bf122640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80cc140 con 0x7f97b815a9f0 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bcebe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 0x7f97b814d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bd6bf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b815a9f0 0x7f97b80c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bcebe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 0x7f97b814d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55140/0 (socket says 172.21.15.100:55140) 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bcebe640 1 -- 172.21.15.100:0/1303267250 learned_addr learned my addr 172.21.15.100:0/1303267250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97b7fff640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80cd520 0x7f97b80ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bd6bf640 1 -- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 msgr2=0x7f97b814d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bd6bf640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 0x7f97b814d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bd6bf640 1 -- 172.21.15.100:0/1303267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80cd520 msgr2=0x7f97b80ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bd6bf640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80cd520 0x7f97b80ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bd6bf640 1 -- 172.21.15.100:0/1303267250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80af030 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97bcebe640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 0x7f97b814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.949+0000 7f97b7fff640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80cd520 0x7f97b80ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97bd6bf640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b815a9f0 0x7f97b80c81a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97b0001730 tx=0x7f97b000cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0003b70 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b0003d10 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00052b0 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b80af310 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b80b49b0 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f97b0005660 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.950+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f97b001bc50 con 0x7f97b815a9f0 2024-04-25T08:29:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.951+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b806cc80 con 0x7f97b815a9f0 2024-04-25T08:29:00.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:00.953+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f97b80c6940 con 0x7f97b815a9f0 2024-04-25T08:29:00.983 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:00.986+0000 7fd95ce12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:29:01.004 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:01.006+0000 7f2190d1d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:29:01.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.026+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd77d113-576d-4798-8ee1-6f46b49e7c6e", "id": 5} v 0) v1 -- 0x7f97b80cac90 con 0x7f97b815a9f0 2024-04-25T08:29:01.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.029+0000 7f97b57fa640 1 -- 172.21.15.100:0/1303267250 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd77d113-576d-4798-8ee1-6f46b49e7c6e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97b001b6d0 con 0x7f97b815a9f0 2024-04-25T08:29:01.027 INFO:teuthology.orchestra.run.smithi100.stdout:5 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b815a9f0 msgr2=0x7f97b80c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b815a9f0 0x7f97b80c81a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f97b0001730 tx=0x7f97b000cb90 comp rx=0 tx=0).stop 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 shutdown_connections 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97b815a9f0 0x7f97b80c81a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97b80cd520 0x7f97b80ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 --2- 172.21.15.100:0/1303267250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97b80c6010 0x7f97b814d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 >> 172.21.15.100:0/1303267250 conn(0x7f97b80b2390 msgr2=0x7f97b806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 shutdown_connections 2024-04-25T08:29:01.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.030+0000 7f97bf122640 1 -- 172.21.15.100:0/1303267250 wait complete. 2024-04-25T08:29:01.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 8d9944d9-fae5-4a43-a375-b3cccd237dcb 6 2024-04-25T08:29:01.038 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:01.041+0000 7fd95ce12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:29:01.060 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:01.061+0000 7f2190d1d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:29:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.136+0000 7f7b8c509640 1 Processor -- start 2024-04-25T08:29:01.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.136+0000 7f7b8c509640 1 -- start start 2024-04-25T08:29:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.137+0000 7f7b8c509640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8415de80 0x7f7b840cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.137+0000 7f7b8c509640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b840cefb0 0x7f7b840cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.137+0000 7f7b8c509640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b840afc80 con 0x7f7b840cf950 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.137+0000 7f7b8c509640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b840ce4d0 con 0x7f7b840cefb0 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.137+0000 7f7b8c509640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840ce7b0 con 0x7f7b8415de80 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b840cefb0 0x7f7b840cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b8a2a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8415de80 0x7f7b840cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b840cefb0 0x7f7b840cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55148/0 (socket says 172.21.15.100:55148) 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b8a2a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8415de80 0x7f7b840cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:35214/0 (socket says 172.21.15.100:35214) 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 -- 172.21.15.100:0/1495905865 learned_addr learned my addr 172.21.15.100:0/1495905865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 -- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8415de80 msgr2=0x7f7b840cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 --2- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8415de80 0x7f7b840cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 -- 172.21.15.100:0/1495905865 >> v1:172.21.15.60:6789/0 conn(0x7f7b840cf950 legacy=0x7f7b8406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.138+0000 7f7b89aa4640 1 -- 172.21.15.100:0/1495905865 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840cea90 con 0x7f7b840cefb0 2024-04-25T08:29:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b89aa4640 1 --2- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cefb0 0x7f7b840cf410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009870 tx=0x7f7b7402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ec400ec3154f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b892a3640 1 -- 172.21.15.100:0/1495905865 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74037070 con 0x7f7b840cefb0 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b892a3640 1 -- 172.21.15.100:0/1495905865 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74032440 con 0x7f7b840cefb0 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b892a3640 1 -- 172.21.15.100:0/1495905865 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74032810 con 0x7f7b840cefb0 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 -- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cefb0 msgr2=0x7f7b840cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 --2- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cefb0 0x7f7b840cf410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009870 tx=0x7f7b7402a270 comp rx=0 tx=0).stop 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 -- 172.21.15.100:0/1495905865 shutdown_connections 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 --2- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cefb0 0x7f7b840cf410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 --2- 172.21.15.100:0/1495905865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b8415de80 0x7f7b840cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 -- 172.21.15.100:0/1495905865 >> 172.21.15.100:0/1495905865 conn(0x7f7b840b2390 msgr2=0x7f7b840b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 -- 172.21.15.100:0/1495905865 shutdown_connections 2024-04-25T08:29:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.139+0000 7f7b8c509640 1 -- 172.21.15.100:0/1495905865 wait complete. 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.140+0000 7f7b8c509640 1 Processor -- start 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 -- start start 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 0x7f7b840af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cf950 0x7f7b840b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8415de80 0x7f7b8414e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840b3030 con 0x7f7b8415de80 2024-04-25T08:29:01.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840b2eb0 con 0x7f7b840cf950 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8c509640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840c5600 con 0x7f7b840cefb0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8a2a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 0x7f7b840af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8a2a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 0x7f7b840af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:35230/0 (socket says 172.21.15.100:35230) 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8a2a5640 1 -- 172.21.15.100:0/3833486677 learned_addr learned my addr 172.21.15.100:0/3833486677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b89aa4640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cf950 0x7f7b840b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.141+0000 7f7b8aaa6640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8415de80 0x7f7b8414e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8a2a5640 1 -- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cf950 msgr2=0x7f7b840b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8a2a5640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cf950 0x7f7b840b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8a2a5640 1 -- 172.21.15.100:0/3833486677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8415de80 msgr2=0x7f7b8414e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8a2a5640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8415de80 0x7f7b8414e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8a2a5640 1 -- 172.21.15.100:0/3833486677 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840af8a0 con 0x7f7b840cefb0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8a2a5640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 0x7f7b840af360 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e870 tx=0x7f7b8000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017070 con 0x7f7b840cefb0 2024-04-25T08:29:01.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8000c940 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000cd30 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b841f4720 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b840aeb60 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7b8001a070 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.142+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7b80006020 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.143+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b84073a00 con 0x7f7b840cefb0 2024-04-25T08:29:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.145+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7b80006520 con 0x7f7b840cefb0 2024-04-25T08:29:01.169 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:01.172+0000 7fd95ce12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:29:01.176 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:01.178+0000 7f2190d1d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:29:01.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.231+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8d9944d9-fae5-4a43-a375-b3cccd237dcb", "id": 6} v 0) v1 -- 0x7f7b84158bd0 con 0x7f7b840cefb0 2024-04-25T08:29:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.235+0000 7f7b7affd640 1 -- 172.21.15.100:0/3833486677 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d9944d9-fae5-4a43-a375-b3cccd237dcb", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7b80006700 con 0x7f7b840cefb0 2024-04-25T08:29:01.232 INFO:teuthology.orchestra.run.smithi100.stdout:6 2024-04-25T08:29:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 msgr2=0x7f7b840af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 0x7f7b840af360 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e870 tx=0x7f7b8000ed30 comp rx=0 tx=0).stop 2024-04-25T08:29:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 shutdown_connections 2024-04-25T08:29:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b8415de80 0x7f7b8414e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b840cf950 0x7f7b840b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 --2- 172.21.15.100:0/3833486677 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b840cefb0 0x7f7b840af360 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 >> 172.21.15.100:0/3833486677 conn(0x7f7b840b2390 msgr2=0x7f7b8406f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 shutdown_connections 2024-04-25T08:29:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.236+0000 7f7b8c509640 1 -- 172.21.15.100:0/3833486677 wait complete. 2024-04-25T08:29:01.243 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd new 989211bc-e8f0-418b-8acf-378707ab4de7 7 2024-04-25T08:29:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 Processor -- start 2024-04-25T08:29:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 -- start start 2024-04-25T08:29:01.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ab41610f0 0x7f3ab406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 0x7f3ab4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab40c6960 con 0x7f3ab41616f0 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab41614d0 con 0x7f3ab41610f0 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3aba761640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4162260 con 0x7f3ab40ce7b0 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3ab8cfe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 0x7f3ab4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.344+0000 7f3ab3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ab41610f0 0x7f3ab406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ab41610f0 0x7f3ab406c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55166/0 (socket says 172.21.15.100:55166) 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab8cfe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 0x7f3ab4161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:35242/0 (socket says 172.21.15.100:35242) 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 -- 172.21.15.100:0/859273410 learned_addr learned my addr 172.21.15.100:0/859273410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 -- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 msgr2=0x7f3ab4161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 --2- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 0x7f3ab4161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 -- 172.21.15.100:0/859273410 >> v1:172.21.15.60:6789/0 conn(0x7f3ab41616f0 legacy=0x7f3ab40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 -- 172.21.15.100:0/859273410 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4162540 con 0x7f3ab41610f0 2024-04-25T08:29:01.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab8cfe640 1 --2- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 0x7f3ab4161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:01.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.345+0000 7f3ab3fff640 1 --2- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab406c1c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009870 tx=0x7f3aa402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82daa2343435d0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:01.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3ab2ffd640 1 -- 172.21.15.100:0/859273410 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4037070 con 0x7f3ab41610f0 2024-04-25T08:29:01.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3ab2ffd640 1 -- 172.21.15.100:0/859273410 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4032440 con 0x7f3ab41610f0 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3ab2ffd640 1 -- 172.21.15.100:0/859273410 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4032810 con 0x7f3ab41610f0 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 -- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 msgr2=0x7f3ab406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 --2- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab406c1c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009870 tx=0x7f3aa402a250 comp rx=0 tx=0).stop 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 -- 172.21.15.100:0/859273410 shutdown_connections 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 --2- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab40ce7b0 0x7f3ab4161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 --2- 172.21.15.100:0/859273410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab406c1c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 -- 172.21.15.100:0/859273410 >> 172.21.15.100:0/859273410 conn(0x7f3ab406f500 msgr2=0x7f3ab406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 -- 172.21.15.100:0/859273410 shutdown_connections 2024-04-25T08:29:01.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.346+0000 7f3aba761640 1 -- 172.21.15.100:0/859273410 wait complete. 2024-04-25T08:29:01.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.347+0000 7f3aba761640 1 Processor -- start 2024-04-25T08:29:01.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 -- start start 2024-04-25T08:29:01.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab40ce7b0 0x7f3ab40b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab40b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab41616f0 0x7f3ab4148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40cf690 con 0x7f3ab40ce7b0 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40cfce0 con 0x7f3ab41610f0 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3aba761640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab406c700 con 0x7f3ab41616f0 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab40b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3ab8cfe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab41616f0 0x7f3ab4148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab40b71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55174/0 (socket says 172.21.15.100:55174) 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3ab3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab40ce7b0 0x7f3ab40b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.348+0000 7f3ab37fe640 1 -- 172.21.15.100:0/2848243269 learned_addr learned my addr 172.21.15.100:0/2848243269 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab37fe640 1 -- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab41616f0 msgr2=0x7f3ab4148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab37fe640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab41616f0 0x7f3ab4148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab37fe640 1 -- 172.21.15.100:0/2848243269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab40ce7b0 msgr2=0x7f3ab40b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab37fe640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab40ce7b0 0x7f3ab40b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab37fe640 1 -- 172.21.15.100:0/2848243269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40b4a00 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab37fe640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab40b71d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c3f0 tx=0x7f3aa800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80052d0 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa8005490 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab40b4ce0 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8007260 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab40b52a0 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3aa8007400 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.349+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f3aa8011bf0 con 0x7f3ab41610f0 2024-04-25T08:29:01.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.350+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab4073a00 con 0x7f3ab41610f0 2024-04-25T08:29:01.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.354+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3aa80035f0 con 0x7f3ab41610f0 2024-04-25T08:29:01.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.426+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd new", "uuid": "989211bc-e8f0-418b-8acf-378707ab4de7", "id": 7} v 0) v1 -- 0x7f3ab40cf150 con 0x7f3ab41610f0 2024-04-25T08:29:01.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.432+0000 7f3ab0ff9640 1 -- 172.21.15.100:0/2848243269 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "989211bc-e8f0-418b-8acf-378707ab4de7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3aa8003a00 con 0x7f3ab41610f0 2024-04-25T08:29:01.430 INFO:teuthology.orchestra.run.smithi100.stdout:7 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 msgr2=0x7f3ab40b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab40b71d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c3f0 tx=0x7f3aa800c7f0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 shutdown_connections 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ab41616f0 0x7f3ab4148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ab41610f0 0x7f3ab40b71d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 --2- 172.21.15.100:0/2848243269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ab40ce7b0 0x7f3ab40b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 >> 172.21.15.100:0/2848243269 conn(0x7f3ab406f500 msgr2=0x7f3ab40d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 shutdown_connections 2024-04-25T08:29:01.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-04-25T08:29:01.433+0000 7f3aba761640 1 -- 172.21.15.100:0/2848243269 wait complete. 2024-04-25T08:29:01.441 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T08:29:01.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T08:29:01.444 INFO:tasks.ceph.osd.0:Started 2024-04-25T08:29:01.444 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T08:29:01.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T08:29:01.446 INFO:tasks.ceph.osd.1:Started 2024-04-25T08:29:01.446 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T08:29:01.446 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T08:29:01.448 INFO:tasks.ceph.osd.2:Started 2024-04-25T08:29:01.449 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T08:29:01.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T08:29:01.451 INFO:tasks.ceph.osd.3:Started 2024-04-25T08:29:01.451 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T08:29:01.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T08:29:01.453 INFO:tasks.ceph.osd.4:Started 2024-04-25T08:29:01.453 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T08:29:01.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T08:29:01.455 INFO:tasks.ceph.osd.5:Started 2024-04-25T08:29:01.456 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T08:29:01.456 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T08:29:01.458 INFO:tasks.ceph.osd.6:Started 2024-04-25T08:29:01.458 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T08:29:01.458 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T08:29:01.460 INFO:tasks.ceph.osd.7:Started 2024-04-25T08:29:01.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:01.587 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:01.589+0000 7fd95ce12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T08:29:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.607+0000 7f094441f640 1 Processor -- start 2024-04-25T08:29:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.607+0000 7f094441f640 1 -- start start 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f094441f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f094441f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0715e0 con 0x7f093c070cc0 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f094441f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0ce770 con 0x7f093c15dba0 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f094441f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c15df80 con 0x7f093c071150 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09419ba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f093c070cc0 0x7f093c0cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39916/0 (socket says 172.21.15.60:39916) 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09419ba640 1 -- 172.21.15.60:0/1207461669 learned_addr learned my addr 172.21.15.60:0/1207461669 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09429bc640 1 --2- 172.21.15.60:0/1207461669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09411b9640 1 -- 172.21.15.60:0/1207461669 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799469349 0 0) 0x7f093c0715e0 con 0x7f093c070cc0 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09411b9640 1 -- 172.21.15.60:0/1207461669 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0928003520 con 0x7f093c070cc0 2024-04-25T08:29:01.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09411b9640 1 -- 172.21.15.60:0/1207461669 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699834221 0 0) 0x7f093c0ce770 con 0x7f093c15dba0 2024-04-25T08:29:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09429bc640 1 -- 172.21.15.60:0/1207461669 >> v1:172.21.15.100:6789/0 conn(0x7f093c15dba0 legacy=0x7f093c06c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09429bc640 1 -- 172.21.15.60:0/1207461669 >> v1:172.21.15.60:6789/0 conn(0x7f093c070cc0 legacy=0x7f093c0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.608+0000 7f09429bc640 1 -- 172.21.15.60:0/1207461669 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0cea50 con 0x7f093c071150 2024-04-25T08:29:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.609+0000 7f09429bc640 1 --2- 172.21.15.60:0/1207461669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c0cff20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0934009dc0 tx=0x7f093402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42eec3fd87874a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.609+0000 7f09411b9640 1 -- 172.21.15.60:0/1207461669 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934007740 con 0x7f093c071150 2024-04-25T08:29:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.609+0000 7f09411b9640 1 -- 172.21.15.60:0/1207461669 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09340078e0 con 0x7f093c071150 2024-04-25T08:29:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.609+0000 7f09411b9640 1 -- 172.21.15.60:0/1207461669 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934007cb0 con 0x7f093c071150 2024-04-25T08:29:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 -- 172.21.15.60:0/1207461669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 msgr2=0x7f093c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 --2- 172.21.15.60:0/1207461669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c0cff20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0934009dc0 tx=0x7f093402a6d0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 -- 172.21.15.60:0/1207461669 shutdown_connections 2024-04-25T08:29:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 --2- 172.21.15.60:0/1207461669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c0cff20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 -- 172.21.15.60:0/1207461669 >> 172.21.15.60:0/1207461669 conn(0x7f093c06f520 msgr2=0x7f093c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 -- 172.21.15.60:0/1207461669 shutdown_connections 2024-04-25T08:29:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.610+0000 7f094441f640 1 -- 172.21.15.60:0/1207461669 wait complete. 2024-04-25T08:29:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.611+0000 7f094441f640 1 Processor -- start 2024-04-25T08:29:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.611+0000 7f094441f640 1 -- start start 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f094441f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 0x7f093c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f094441f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f094441f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 0x7f093c161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f094441f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0cf010 con 0x7f093c15dba0 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f094441f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0cf380 con 0x7f093c070cc0 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f094441f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b0ce0 con 0x7f093c071150 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09419ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 0x7f093c161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09421bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 0x7f093c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09421bb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 0x7f093c15c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37110/0 (socket says 172.21.15.60:37110) 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 0x7f093c161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35898/0 (socket says 172.21.15.60:35898) 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09419ba640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:41052/0 (socket says 172.21.15.60:41052) 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09421bb640 1 -- 172.21.15.60:0/4035791699 learned_addr learned my addr 172.21.15.60:0/4035791699 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 -- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 msgr2=0x7f093c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c15adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 -- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 msgr2=0x7f093c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 0x7f093c15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 -- 172.21.15.60:0/4035791699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c159d20 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09421bb640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 0x7f093c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.612+0000 7f09429bc640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 0x7f093c161410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0934031000 tx=0x7f0934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934040070 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0934031bc0 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c15a000 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934009450 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c15a5c0 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0934039040 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.613+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0934007840 con 0x7f093c15dba0 2024-04-25T08:29:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.614+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c06de80 con 0x7f093c15dba0 2024-04-25T08:29:01.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.616+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0934045cd0 con 0x7f093c15dba0 2024-04-25T08:29:01.628 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:01.630+0000 7f2190d1d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T08:29:01.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.679+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f093c0b0370 con 0x7f093c15dba0 2024-04-25T08:29:01.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.679+0000 7f0932ffd640 1 -- 172.21.15.60:0/4035791699 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f0934045460 con 0x7f093c15dba0 2024-04-25T08:29:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:01.678 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":10,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:01.427560+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 msgr2=0x7f093c161410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 0x7f093c161410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0934031000 tx=0x7f0934004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 shutdown_connections 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f093c15dba0 0x7f093c161410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f093c071150 0x7f093c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 --2- 172.21.15.60:0/4035791699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f093c070cc0 0x7f093c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 >> 172.21.15.60:0/4035791699 conn(0x7f093c06f520 msgr2=0x7f093c0d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 shutdown_connections 2024-04-25T08:29:01.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:01.680+0000 7f094441f640 1 -- 172.21.15.60:0/4035791699 wait complete. 2024-04-25T08:29:01.689 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T08:29:01.689 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T08:29:01.846 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:01.848+0000 7fd95ce12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:29:01.898 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:01.900+0000 7fd95ce12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:29:01.898 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:01.900+0000 7f2190d1d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:29:01.952 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:01.954+0000 7f2190d1d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:29:02.011 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.013+0000 7fd95ce12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T08:29:02.045 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T08:29:02.048+0000 7fe1487bc640 -1 Falling back to public interface 2024-04-25T08:29:02.053 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T08:29:02.055+0000 7fb5c15fc640 -1 Falling back to public interface 2024-04-25T08:29:02.053 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T08:29:02.055+0000 7feaee632640 -1 Falling back to public interface 2024-04-25T08:29:02.056 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T08:29:02.058+0000 7eff7c886640 -1 Falling back to public interface 2024-04-25T08:29:02.064 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T08:29:02.066+0000 7f1948d1d640 -1 Falling back to public interface 2024-04-25T08:29:02.065 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T08:29:02.066+0000 7f083b415640 -1 Falling back to public interface 2024-04-25T08:29:02.070 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T08:29:02.072+0000 7fa7a3c1c640 -1 Falling back to public interface 2024-04-25T08:29:02.073 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.075+0000 7f2190d1d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T08:29:02.073 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T08:29:02.076+0000 7f49f07ad640 -1 Falling back to public interface 2024-04-25T08:29:02.292 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.295+0000 7fd95ce12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:29:02.341 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.344+0000 7fd95ce12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:29:02.366 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.367+0000 7f2190d1d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:29:02.415 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.417+0000 7f2190d1d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:29:02.493 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:02.496 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.498+0000 7fd95ce12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:29:02.580 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.582+0000 7f2190d1d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:29:02.585 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.587+0000 7f7d4f71d640 1 Processor -- start 2024-04-25T08:29:02.585 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.587+0000 7f7d4f71d640 1 -- start start 2024-04-25T08:29:02.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4f71d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d481623e0 0x7f7d480c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4f71d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d480ce0b0 0x7f7d480ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4f71d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7d4815e460 0x7f7d4815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4f71d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480cea50 con 0x7f7d481623e0 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4f71d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480c6e50 con 0x7f7d4815e460 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4f71d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4815f3c0 con 0x7f7d480ce0b0 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d481623e0 0x7f7d480c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d481623e0 0x7f7d480c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42978/0 (socket says 172.21.15.60:42978) 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 -- 172.21.15.60:0/1543661546 learned_addr learned my addr 172.21.15.60:0/1543661546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4ccb8640 1 --2- 172.21.15.60:0/1543661546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d480ce0b0 0x7f7d480ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4dcba640 1 --2- 172.21.15.60:0/1543661546 >> v2:172.21.15.100:3300/0 conn(0x7f7d4815e460 0x7f7d4815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 -- 172.21.15.60:0/1543661546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d480ce0b0 msgr2=0x7f7d480ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 --2- 172.21.15.60:0/1543661546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d480ce0b0 0x7f7d480ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 -- 172.21.15.60:0/1543661546 >> v2:172.21.15.100:3300/0 conn(0x7f7d4815e460 msgr2=0x7f7d4815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:02.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 --2- 172.21.15.60:0/1543661546 >> v2:172.21.15.100:3300/0 conn(0x7f7d4815e460 0x7f7d4815e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.588+0000 7f7d4d4b9640 1 -- 172.21.15.60:0/1543661546 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4815f0e0 con 0x7f7d481623e0 2024-04-25T08:29:02.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.589+0000 7f7d4d4b9640 1 --2- 172.21.15.60:0/1543661546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d481623e0 0x7f7d480c72e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d4806bd30 tx=0x7f7d3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b1cc0dc8d1b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:02.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.589+0000 7f7d3ffff640 1 -- 172.21.15.60:0/1543661546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38037070 con 0x7f7d481623e0 2024-04-25T08:29:02.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.589+0000 7f7d3ffff640 1 -- 172.21.15.60:0/1543661546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d38032440 con 0x7f7d481623e0 2024-04-25T08:29:02.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.589+0000 7f7d3ffff640 1 -- 172.21.15.60:0/1543661546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38032810 con 0x7f7d481623e0 2024-04-25T08:29:02.591 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 -- 172.21.15.60:0/1543661546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d481623e0 msgr2=0x7f7d480c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/1543661546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d481623e0 0x7f7d480c72e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d4806bd30 tx=0x7f7d3802a180 comp rx=0 tx=0).stop 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 -- 172.21.15.60:0/1543661546 shutdown_connections 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/1543661546 >> v2:172.21.15.100:3300/0 conn(0x7f7d4815e460 0x7f7d4815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/1543661546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d480ce0b0 0x7f7d480ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/1543661546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d481623e0 0x7f7d480c72e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 -- 172.21.15.60:0/1543661546 >> 172.21.15.60:0/1543661546 conn(0x7f7d480b3460 msgr2=0x7f7d480b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 -- 172.21.15.60:0/1543661546 shutdown_connections 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.590+0000 7f7d4f71d640 1 -- 172.21.15.60:0/1543661546 wait complete. 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.591+0000 7f7d4f71d640 1 Processor -- start 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.591+0000 7f7d4f71d640 1 -- start start 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4f71d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d480ce0b0 0x7f7d481470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4f71d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 0x7f7d480b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4f71d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d481623e0 0x7f7d481606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4f71d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480c7820 con 0x7f7d4815e460 2024-04-25T08:29:02.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4f71d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480c6ae0 con 0x7f7d480ce0b0 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4f71d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4815edf0 con 0x7f7d481623e0 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 0x7f7d480b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 0x7f7d480b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42990/0 (socket says 172.21.15.60:42990) 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 -- 172.21.15.60:0/4085675887 learned_addr learned my addr 172.21.15.60:0/4085675887 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4d4b9640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d480ce0b0 0x7f7d481470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4dcba640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d481623e0 0x7f7d481606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 -- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d481623e0 msgr2=0x7f7d481606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d481623e0 0x7f7d481606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 -- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d480ce0b0 msgr2=0x7f7d481470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d480ce0b0 0x7f7d481470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 -- 172.21.15.60:0/4085675887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480b65b0 con 0x7f7d4815e460 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.592+0000 7f7d4ccb8640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 0x7f7d480b55a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400d800 tx=0x7f7d4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44018020 con 0x7f7d4815e460 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d440096a0 con 0x7f7d4815e460 2024-04-25T08:29:02.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d480b5ff0 con 0x7f7d4815e460 2024-04-25T08:29:02.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d44009a70 con 0x7f7d4815e460 2024-04-25T08:29:02.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d48147870 con 0x7f7d4815e460 2024-04-25T08:29:02.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7d4401b020 con 0x7f7d4815e460 2024-04-25T08:29:02.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.593+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7d4401dbc0 con 0x7f7d4815e460 2024-04-25T08:29:02.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.594+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4804ef20 con 0x7f7d4815e460 2024-04-25T08:29:02.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.596+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7d44022030 con 0x7f7d4815e460 2024-04-25T08:29:02.631 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.634+0000 7fd95ce12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:29:02.656 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.658+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d480d1770 con 0x7f7d4815e460 2024-04-25T08:29:02.657 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.659+0000 7f7d3dffb640 1 -- 172.21.15.60:0/4085675887 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f7d4401d510 con 0x7f7d4815e460 2024-04-25T08:29:02.657 INFO:teuthology.misc.health.smithi060.stdout: 2024-04-25T08:29:02.657 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":10,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:01.427560+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:02.658 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 msgr2=0x7f7d480b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:02.658 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 0x7f7d480b55a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400d800 tx=0x7f7d4400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 shutdown_connections 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d481623e0 0x7f7d481606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d4815e460 0x7f7d480b55a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 --2- 172.21.15.60:0/4085675887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d480ce0b0 0x7f7d481470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 >> 172.21.15.60:0/4085675887 conn(0x7f7d480b3460 msgr2=0x7f7d480d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 shutdown_connections 2024-04-25T08:29:02.659 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:02.660+0000 7f7d4f71d640 1 -- 172.21.15.60:0/4085675887 wait complete. 2024-04-25T08:29:02.667 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:29:02.682 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.683+0000 7fd95ce12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:29:02.726 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.728+0000 7f2190d1d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:29:02.749 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.752+0000 7fd95ce12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:29:02.778 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.780+0000 7f2190d1d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:29:02.793 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:02.796+0000 7fd95ce12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:29:02.851 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.853+0000 7f2190d1d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:29:02.898 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:02.900+0000 7f2190d1d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:29:03.266 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:03.268+0000 7fd95ce12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:29:03.381 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:03.383+0000 7fd95ce12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:29:03.399 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:03.401+0000 7f2190d1d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:29:03.521 INFO:tasks.ceph.mgr.y.smithi060.stderr:2024-04-25T08:29:03.523+0000 7f2190d1d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:29:04.399 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:04.401+0000 7fd8fd5ef640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T08:29:04.400 INFO:tasks.ceph.mgr.x.smithi100.stderr:2024-04-25T08:29:04.403+0000 7fd8fd5ef640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T08:29:05.520 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T08:29:05.523+0000 7fb5c15fc640 -1 osd.4 0 log_to_monitors true 2024-04-25T08:29:05.529 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T08:29:05.531+0000 7feaee632640 -1 osd.1 0 log_to_monitors true 2024-04-25T08:29:05.561 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T08:29:05.563+0000 7fa7a3c1c640 -1 osd.6 0 log_to_monitors true 2024-04-25T08:29:05.562 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T08:29:05.565+0000 7fe1487bc640 -1 osd.5 0 log_to_monitors true 2024-04-25T08:29:05.576 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T08:29:05.578+0000 7eff7c886640 -1 osd.2 0 log_to_monitors true 2024-04-25T08:29:05.586 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T08:29:05.588+0000 7f49f07ad640 -1 osd.7 0 log_to_monitors true 2024-04-25T08:29:05.596 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T08:29:05.598+0000 7f083b415640 -1 osd.0 0 log_to_monitors true 2024-04-25T08:29:05.598 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T08:29:05.600+0000 7f1948d1d640 -1 osd.3 0 log_to_monitors true 2024-04-25T08:29:08.555 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T08:29:08.556+0000 7f1944cb3640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T08:29:08.571 INFO:tasks.ceph.osd.3.smithi060.stderr:2024-04-25T08:29:08.571+0000 7f193cad3640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:08.591 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T08:29:08.592+0000 7feaea5c8640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T08:29:08.601 INFO:tasks.ceph.osd.1.smithi060.stderr:2024-04-25T08:29:08.603+0000 7feae23e8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:08.888 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T08:29:08.890+0000 7f0836d0c640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T08:29:08.898 INFO:tasks.ceph.osd.0.smithi060.stderr:2024-04-25T08:29:08.900+0000 7f082eb2c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:08.917 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T08:29:08.919+0000 7eff7881c640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T08:29:08.927 INFO:tasks.ceph.osd.2.smithi060.stderr:2024-04-25T08:29:08.929+0000 7eff7063c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:09.442 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T08:29:09.442+0000 7fb5bd592640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T08:29:09.464 INFO:tasks.ceph.osd.4.smithi100.stderr:2024-04-25T08:29:09.466+0000 7fb5b53b2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:09.475 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:09.493 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T08:29:09.495+0000 7fe144752640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T08:29:09.507 INFO:tasks.ceph.osd.5.smithi100.stderr:2024-04-25T08:29:09.509+0000 7fe13c572640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:09.572 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T08:29:09.573+0000 7fa79f50c640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T08:29:09.574 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T08:29:09.576+0000 7f49ec743640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T08:29:09.583 INFO:tasks.ceph.osd.6.smithi100.stderr:2024-04-25T08:29:09.585+0000 7fa79732c640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:09.585 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.587+0000 7fccd9334640 1 Processor -- start 2024-04-25T08:29:09.585 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.587+0000 7fccd9334640 1 -- start start 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd9334640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fccd40d2ea0 0x7fccd40d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd9334640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40b9950 con 0x7fccd40d0910 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd9334640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40d1e00 con 0x7fccd415dd40 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd9334640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd40d0d50 con 0x7fccd40d2ea0 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fccd40d0910 0x7fccd40b6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34826/0 (socket says 172.21.15.60:34826) 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd37fe640 1 -- 172.21.15.60:0/3511904873 learned_addr learned my addr 172.21.15.60:0/3511904873 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd27fc640 1 --2- 172.21.15.60:0/3511904873 >> v2:172.21.15.100:3300/0 conn(0x7fccd40d2ea0 0x7fccd40d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908770195 0 0) 0x7fccd40b9950 con 0x7fccd40d0910 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccbc003520 con 0x7fccd40d0910 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015422918 0 0) 0x7fccd40d1e00 con 0x7fccd415dd40 2024-04-25T08:29:09.587 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccd40b9950 con 0x7fccd415dd40 2024-04-25T08:29:09.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911475330 0 0) 0x7fccbc003520 con 0x7fccd40d0910 2024-04-25T08:29:09.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd27fc640 1 -- 172.21.15.60:0/3511904873 >> v1:172.21.15.100:6789/0 conn(0x7fccd415dd40 legacy=0x7fccd40b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd27fc640 1 -- 172.21.15.60:0/3511904873 >> v1:172.21.15.60:6789/0 conn(0x7fccd40d0910 legacy=0x7fccd40b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.588 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.588+0000 7fccd27fc640 1 -- 172.21.15.60:0/3511904873 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40d20e0 con 0x7fccd40d2ea0 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.589+0000 7fccd27fc640 1 --2- 172.21.15.60:0/3511904873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40d03d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccc80097a0 tx=0x7fccc802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4f09e8c0ec8e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.589+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8037070 con 0x7fccd40d2ea0 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.589+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc802ab30 con 0x7fccd40d2ea0 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.589+0000 7fccd1ffb640 1 -- 172.21.15.60:0/3511904873 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0019f0 con 0x7fccd40d2ea0 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 -- 172.21.15.60:0/3511904873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 msgr2=0x7fccd40d03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 --2- 172.21.15.60:0/3511904873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40d03d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fccc80097a0 tx=0x7fccc802a0a0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 -- 172.21.15.60:0/3511904873 shutdown_connections 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 --2- 172.21.15.60:0/3511904873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40d03d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 -- 172.21.15.60:0/3511904873 >> 172.21.15.60:0/3511904873 conn(0x7fccd40728f0 msgr2=0x7fccd406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:09.589 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 -- 172.21.15.60:0/3511904873 shutdown_connections 2024-04-25T08:29:09.590 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.590+0000 7fccd9334640 1 -- 172.21.15.60:0/3511904873 wait complete. 2024-04-25T08:29:09.590 INFO:tasks.ceph.osd.7.smithi100.stderr:2024-04-25T08:29:09.591+0000 7f49e4563640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.591+0000 7fccd9334640 1 Processor -- start 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.591+0000 7fccd9334640 1 -- start start 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd9334640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 0x7fccd40cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd9334640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd9334640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415dd40 0x7fccd4149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd9334640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40d2980 con 0x7fccd40d0910 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd9334640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40d26a0 con 0x7fccd40d2ea0 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd9334640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40d1a20 con 0x7fccd415dd40 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 0x7fccd40cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 0x7fccd40cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43038/0 (socket says 172.21.15.60:43038) 2024-04-25T08:29:09.592 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd2ffd640 1 -- 172.21.15.60:0/126810980 learned_addr learned my addr 172.21.15.60:0/126810980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd27fc640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd37fe640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415dd40 0x7fccd4149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.592+0000 7fccd2ffd640 1 -- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415dd40 msgr2=0x7fccd4149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd2ffd640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415dd40 0x7fccd4149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd2ffd640 1 -- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 msgr2=0x7fccd40cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd2ffd640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd2ffd640 1 -- 172.21.15.60:0/126810980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40cc100 con 0x7fccd40d0910 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd27fc640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd2ffd640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 0x7fccd40cd440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c490 tx=0x7fccc400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40052d0 con 0x7fccd40d0910 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc4005470 con 0x7fccd40d0910 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd414a2e0 con 0x7fccd40d0910 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4012850 con 0x7fccd40d0910 2024-04-25T08:29:09.593 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.593+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd40cc680 con 0x7fccd40d0910 2024-04-25T08:29:09.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.594+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd404ef20 con 0x7fccd40d0910 2024-04-25T08:29:09.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.594+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fccc40129f0 con 0x7fccd40d0910 2024-04-25T08:29:09.594 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.595+0000 7fccbb7fe640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccc006b440 0x7fccc006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:09.595 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.595+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3631+0+0 (secure 0 0 0) 0x7fccc40895a0 con 0x7fccd40d0910 2024-04-25T08:29:09.595 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.595+0000 7fccd27fc640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccc006b440 0x7fccc006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:09.595 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.596+0000 7fccd27fc640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccc006b440 0x7fccc006d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:09.598 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.600+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccc4058d20 con 0x7fccd40d0910 2024-04-25T08:29:09.778 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.780+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccd40b2910 con 0x7fccd40d0910 2024-04-25T08:29:09.779 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.781+0000 7fccbb7fe640 1 -- 172.21.15.60:0/126810980 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+9451 (secure 0 0 0) 0x7fccc4058620 con 0x7fccd40d0910 2024-04-25T08:29:09.779 INFO:teuthology.misc.health.smithi060.stdout: 2024-04-25T08:29:09.779 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":13,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:09.504614+0000","last_up_change":"2024-04-25T08:29:09.504614+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:09.781 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccc006b440 msgr2=0x7fccc006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.781 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccc006b440 0x7fccc006d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fccc8034000 tx=0x7fccc802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:09.781 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 msgr2=0x7fccd40cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:09.781 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 0x7fccd40cd440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c490 tx=0x7fccc400c950 comp rx=0 tx=0).stop 2024-04-25T08:29:09.781 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 shutdown_connections 2024-04-25T08:29:09.781 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccd415dd40 0x7fccd4149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.782 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccc006b440 0x7fccc006d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.782 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccd40d2ea0 0x7fccd40cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.782 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.782+0000 7fccd9334640 1 --2- 172.21.15.60:0/126810980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccd40d0910 0x7fccd40cd440 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:09.782 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.783+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 >> 172.21.15.60:0/126810980 conn(0x7fccd40728f0 msgr2=0x7fccd40c7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:09.782 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.783+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 shutdown_connections 2024-04-25T08:29:09.782 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:09.783+0000 7fccd9334640 1 -- 172.21.15.60:0/126810980 wait complete. 2024-04-25T08:29:09.790 DEBUG:teuthology.misc:5 of 8 OSDs are up 2024-04-25T08:29:16.596 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:16.700 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.702+0000 7fb72f015640 1 Processor -- start 2024-04-25T08:29:16.701 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.702+0000 7fb72f015640 1 -- start start 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb72f015640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb72815dd40 0x7fb7280b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb72f015640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7280d1e00 con 0x7fb7280b9780 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb72f015640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7280d20e0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb72f015640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7280d23c0 con 0x7fb72815dd40 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb72cdb1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb72815dd40 0x7fb7280b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb727fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb7280b9780 0x7fb7280d03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34346/0 (socket says 172.21.15.60:34346) 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.703+0000 7fb727fff640 1 -- 172.21.15.60:0/2348442585 learned_addr learned my addr 172.21.15.60:0/2348442585 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb72cdb1640 1 -- 172.21.15.60:0/2348442585 >> v1:172.21.15.100:6789/0 conn(0x7fb7280d0ae0 legacy=0x7fb7280b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb72cdb1640 1 -- 172.21.15.60:0/2348442585 >> v1:172.21.15.60:6789/0 conn(0x7fb7280b9780 legacy=0x7fb7280d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:16.702 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb72cdb1640 1 -- 172.21.15.60:0/2348442585 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7280d26a0 con 0x7fb72815dd40 2024-04-25T08:29:16.703 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb72cdb1640 1 --2- 172.21.15.60:0/2348442585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb72815dd40 0x7fb7280b9240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097a0 tx=0x7fb71c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67007ceb86125079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb7277fe640 1 -- 172.21.15.60:0/2348442585 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c037070 con 0x7fb72815dd40 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb7277fe640 1 -- 172.21.15.60:0/2348442585 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c02adc0 con 0x7fb72815dd40 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.704+0000 7fb7277fe640 1 -- 172.21.15.60:0/2348442585 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c032660 con 0x7fb72815dd40 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 -- 172.21.15.60:0/2348442585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb72815dd40 msgr2=0x7fb7280b9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 --2- 172.21.15.60:0/2348442585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb72815dd40 0x7fb7280b9240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097a0 tx=0x7fb71c02a0b0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 -- 172.21.15.60:0/2348442585 shutdown_connections 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 --2- 172.21.15.60:0/2348442585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb72815dd40 0x7fb7280b9240 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 -- 172.21.15.60:0/2348442585 >> 172.21.15.60:0/2348442585 conn(0x7fb7280728f0 msgr2=0x7fb72806c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 -- 172.21.15.60:0/2348442585 shutdown_connections 2024-04-25T08:29:16.704 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.705+0000 7fb72f015640 1 -- 172.21.15.60:0/2348442585 wait complete. 2024-04-25T08:29:16.705 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.706+0000 7fb72f015640 1 Processor -- start 2024-04-25T08:29:16.705 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.706+0000 7fb72f015640 1 -- start start 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb72f015640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7280b9780 0x7fb7281579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb72f015640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 0x7fb728149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb72f015640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb72815dd40 0x7fb728149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb72f015640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7280d1ba0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb72f015640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7280d1a20 con 0x7fb7280b9780 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb72f015640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7280d2ea0 con 0x7fb72815dd40 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb727fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 0x7fb728149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb727fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 0x7fb728149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39642/0 (socket says 172.21.15.60:39642) 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.707+0000 7fb727fff640 1 -- 172.21.15.60:0/1731353826 learned_addr learned my addr 172.21.15.60:0/1731353826 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb72cdb1640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7280b9780 0x7fb7281579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb72d5b2640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb72815dd40 0x7fb728149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb727fff640 1 -- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb72815dd40 msgr2=0x7fb728149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:16.707 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb727fff640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb72815dd40 0x7fb728149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb727fff640 1 -- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7280b9780 msgr2=0x7fb7281579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb727fff640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7280b9780 0x7fb7281579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb727fff640 1 -- 172.21.15.60:0/1731353826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728157f10 con 0x7fb7280d0ae0 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb72cdb1640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7280b9780 0x7fb7281579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb727fff640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 0x7fb728149890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb71400c400 tx=0x7fb71400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714017070 con 0x7fb7280d0ae0 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb714005ce0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7140072f0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb728154aa0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.708 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.708+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb728155060 con 0x7fb7280d0ae0 2024-04-25T08:29:16.709 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.709+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb728067570 con 0x7fb7280d0ae0 2024-04-25T08:29:16.709 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.710+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb71401a070 con 0x7fb7280d0ae0 2024-04-25T08:29:16.709 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.710+0000 7fb7257fa640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb70c06b4d0 0x7fb70c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:16.709 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.710+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb7140892f0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.709 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.710+0000 7fb72cdb1640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb70c06b4d0 0x7fb70c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:16.709 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.711+0000 7fb72cdb1640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb70c06b4d0 0x7fb70c06d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb71c034000 tx=0x7fb71c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:16.713 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.715+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb728067570 con 0x7fb7280d0ae0 2024-04-25T08:29:16.896 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.898+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb72804ef20 con 0x7fb7280d0ae0 2024-04-25T08:29:16.897 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.899+0000 7fb7257fa640 1 -- 172.21.15.60:0/1731353826 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fb7140584c0 con 0x7fb7280d0ae0 2024-04-25T08:29:16.897 INFO:teuthology.misc.health.smithi060.stdout: 2024-04-25T08:29:16.898 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":17,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:13.531094+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.901+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb70c06b4d0 msgr2=0x7fb70c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.901+0000 7fb72f015640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb70c06b4d0 0x7fb70c06d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb71c034000 tx=0x7fb71c02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.901+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 msgr2=0x7fb728149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.901+0000 7fb72f015640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 0x7fb728149890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb71400c400 tx=0x7fb71400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 shutdown_connections 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb72815dd40 0x7fb728149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7280d0ae0 0x7fb728149890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb70c06b4d0 0x7fb70c06d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 --2- 172.21.15.60:0/1731353826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7280b9780 0x7fb7281579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:16.900 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 >> 172.21.15.60:0/1731353826 conn(0x7fb7280728f0 msgr2=0x7fb7280c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:16.901 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 shutdown_connections 2024-04-25T08:29:16.901 INFO:teuthology.misc.health.smithi060.stderr:2024-04-25T08:29:16.902+0000 7fb72f015640 1 -- 172.21.15.60:0/1731353826 wait complete. 2024-04-25T08:29:16.909 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T08:29:16.910 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T08:29:16.910 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:29:16.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:29:16.913 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:29:16.913 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:29:16.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:29:16.915 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:29:16.915 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:29:16.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:29:16.918 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:29:16.918 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:29:16.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:29:16.920 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:29:16.920 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T08:29:16.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:16.987 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:29:16.990 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:29:17.001 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:29:17.002 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:29:17.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.028+0000 7f0b77e3a640 1 Processor -- start 2024-04-25T08:29:17.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.028+0000 7f0b77e3a640 1 -- start start 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b77e3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b700d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b77e3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b700d1020 0x7f0b700d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b77e3a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700b97a0 con 0x7f0b7015dbe0 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b77e3a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7015dfc0 con 0x7f0b700d1020 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b77e3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700d1a00 con 0x7f0b700b2cb0 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b763d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b700d1020 0x7f0b700d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b763d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0b700d1020 0x7f0b700d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39704/0 (socket says 172.21.15.60:39704) 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.029+0000 7f0b763d7640 1 -- 172.21.15.60:0/1137374276 learned_addr learned my addr 172.21.15.60:0/1137374276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.030+0000 7f0b753d5640 1 --2- 172.21.15.60:0/1137374276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b700d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.030+0000 7f0b763d7640 1 -- 172.21.15.60:0/1137374276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 msgr2=0x7f0b700d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.030+0000 7f0b763d7640 1 --2- 172.21.15.60:0/1137374276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b700d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.030+0000 7f0b763d7640 1 -- 172.21.15.60:0/1137374276 >> v1:172.21.15.100:6789/0 conn(0x7f0b7015dbe0 legacy=0x7f0b700d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.030+0000 7f0b763d7640 1 -- 172.21.15.60:0/1137374276 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700d1c50 con 0x7f0b700d1020 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.030+0000 7f0b763d7640 1 --2- 172.21.15.60:0/1137374276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b700d1020 0x7f0b700d2c70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009e50 tx=0x7f0b6802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ff368ba2afcbf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b74bd4640 1 -- 172.21.15.60:0/1137374276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68036070 con 0x7f0b700d1020 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b74bd4640 1 -- 172.21.15.60:0/1137374276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68031600 con 0x7f0b700d1020 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b74bd4640 1 -- 172.21.15.60:0/1137374276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68031a10 con 0x7f0b700d1020 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 -- 172.21.15.60:0/1137374276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b700d1020 msgr2=0x7f0b700d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/1137374276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b700d1020 0x7f0b700d2c70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009e50 tx=0x7f0b6802a730 comp rx=0 tx=0).stop 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 -- 172.21.15.60:0/1137374276 shutdown_connections 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/1137374276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b700d1020 0x7f0b700d2c70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/1137374276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b700d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 -- 172.21.15.60:0/1137374276 >> 172.21.15.60:0/1137374276 conn(0x7f0b70061a20 msgr2=0x7f0b70061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 -- 172.21.15.60:0/1137374276 shutdown_connections 2024-04-25T08:29:17.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.031+0000 7f0b77e3a640 1 -- 172.21.15.60:0/1137374276 wait complete. 2024-04-25T08:29:17.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 Processor -- start 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 -- start start 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b701556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b700d1020 0x7f0b70150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 0x7f0b70160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b92a0 con 0x7f0b7015dbe0 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700d2210 con 0x7f0b700d1020 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b77e3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b9420 con 0x7f0b700b2cb0 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b763d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 0x7f0b70160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b763d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 0x7f0b70160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39716/0 (socket says 172.21.15.60:39716) 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.033+0000 7f0b763d7640 1 -- 172.21.15.60:0/3735957362 learned_addr learned my addr 172.21.15.60:0/3735957362 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b753d5640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b700d1020 0x7f0b70150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b75bd6640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b701556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b763d7640 1 -- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 msgr2=0x7f0b701556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b763d7640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b701556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b763d7640 1 -- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b700d1020 msgr2=0x7f0b70150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b763d7640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b700d1020 0x7f0b70150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b763d7640 1 -- 172.21.15.60:0/3735957362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70155c20 con 0x7f0b7015dbe0 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b753d5640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b700d1020 0x7f0b70150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b763d7640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 0x7f0b70160590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009f70 tx=0x7f0b68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68036050 con 0x7f0b7015dbe0 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68042070 con 0x7f0b7015dbe0 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7015e280 con 0x7f0b7015dbe0 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007c70 con 0x7f0b7015dbe0 2024-04-25T08:29:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.034+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7015e800 con 0x7f0b7015dbe0 2024-04-25T08:29:17.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.035+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b70073990 con 0x7f0b7015dbe0 2024-04-25T08:29:17.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.035+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b68047460 con 0x7f0b7015dbe0 2024-04-25T08:29:17.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.036+0000 7f0b667fc640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b5406b500 0x7f0b5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.036+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0b680aab40 con 0x7f0b7015dbe0 2024-04-25T08:29:17.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.036+0000 7f0b75bd6640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b5406b500 0x7f0b5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.037+0000 7f0b75bd6640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b5406b500 0x7f0b5406d9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b600015a0 tx=0x7f0b60009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.040+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b68079ec0 con 0x7f0b7015dbe0 2024-04-25T08:29:17.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.222+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b7015bc90 con 0x7f0b7015dbe0 2024-04-25T08:29:17.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.222+0000 7f0b667fc640 1 -- 172.21.15.60:0/3735957362 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f0b680797c0 con 0x7f0b7015dbe0 2024-04-25T08:29:17.221 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:17.221 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:13.531094+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b5406b500 msgr2=0x7f0b5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b5406b500 0x7f0b5406d9c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b600015a0 tx=0x7f0b60009040 comp rx=0 tx=0).stop 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 msgr2=0x7f0b70160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 0x7f0b70160590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009f70 tx=0x7f0b68004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 shutdown_connections 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b7015dbe0 0x7f0b70160590 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b5406b500 0x7f0b5406d9c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b700d1020 0x7f0b70150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.224+0000 7f0b77e3a640 1 --2- 172.21.15.60:0/3735957362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b700b2cb0 0x7f0b701556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.225+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 >> 172.21.15.60:0/3735957362 conn(0x7f0b70061a20 msgr2=0x7f0b70061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:17.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.225+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 shutdown_connections 2024-04-25T08:29:17.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.225+0000 7f0b77e3a640 1 -- 172.21.15.60:0/3735957362 wait complete. 2024-04-25T08:29:17.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:29:17.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.339+0000 7fa2b933b640 1 Processor -- start 2024-04-25T08:29:17.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.339+0000 7fa2b933b640 1 -- start start 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b933b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 0x7fa2b40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b933b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b415e000 con 0x7fa2b415dc20 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b933b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b40d1de0 con 0x7fa2b4161710 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b933b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4161c60 con 0x7fa2b40ced80 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b2ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa2b415dc20 0x7fa2b40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34368/0 (socket says 172.21.15.60:34368) 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b2ffd640 1 -- 172.21.15.60:0/1438670473 learned_addr learned my addr 172.21.15.60:0/1438670473 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b27fc640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 0x7fa2b40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091892382 0 0) 0x7fa2b415e000 con 0x7fa2b415dc20 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa29c003520 con 0x7fa2b415dc20 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3524529455 0 0) 0x7fa29c003520 con 0x7fa2b415dc20 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 msgr2=0x7fa2b40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 0x7fa2b40d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 >> v1:172.21.15.100:6789/0 conn(0x7fa2b4161710 legacy=0x7fa2b40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.340+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4161f40 con 0x7fa2b415dc20 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fa2a4001670 con 0x7fa2b415dc20 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 >> v1:172.21.15.60:6789/0 conn(0x7fa2b415dc20 legacy=0x7fa2b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa29c0039e0 0x7fa29c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 0x7fa29c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4161c60 con 0x7fa29c004d40 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c00b9c0 con 0x7fa29c0039e0 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c00bbc0 con 0x7fa2b40ced80 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b37fe640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b2ffd640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa29c0039e0 0x7fa29c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b27fc640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 0x7fa29c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b37fe640 1 -- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 msgr2=0x7fa29c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b37fe640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b40ced80 0x7fa29c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b37fe640 1 -- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa29c0039e0 msgr2=0x7fa29c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b37fe640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa29c0039e0 0x7fa29c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.341+0000 7fa2b37fe640 1 -- 172.21.15.60:0/1438670473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c003520 con 0x7fa29c004d40 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b37fe640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac002010 tx=0x7fa29c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2553516becc076ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac036070 con 0x7fa29c004d40 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ac031600 con 0x7fa29c004d40 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b1ffb640 1 -- 172.21.15.60:0/1438670473 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac001be0 con 0x7fa29c004d40 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b933b640 1 -- 172.21.15.60:0/1438670473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 msgr2=0x7fa29c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b933b640 1 --2- 172.21.15.60:0/1438670473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa29c0051c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac002010 tx=0x7fa29c005830 comp rx=0 tx=0).stop 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b37fe640 1 -- 172.21.15.60:0/1438670473 reap_dead start 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b933b640 1 -- 172.21.15.60:0/1438670473 shutdown_connections 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b933b640 1 -- 172.21.15.60:0/1438670473 >> 172.21.15.60:0/1438670473 conn(0x7fa2b40d0420 msgr2=0x7fa2b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b933b640 1 -- 172.21.15.60:0/1438670473 shutdown_connections 2024-04-25T08:29:17.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.342+0000 7fa2b933b640 1 -- 172.21.15.60:0/1438670473 wait complete. 2024-04-25T08:29:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 Processor -- start 2024-04-25T08:29:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 -- start start 2024-04-25T08:29:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa2b4160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b4154810 0x7fa2b41567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b415c470 0x7fa2b40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40b5e50 con 0x7fa29c004d40 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4162500 con 0x7fa2b415c470 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.344+0000 7fa2b933b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40b5fd0 con 0x7fa2b4154810 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa2b4160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa2b4160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39734/0 (socket says 172.21.15.60:39734) 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 -- 172.21.15.60:0/2183625552 learned_addr learned my addr 172.21.15.60:0/2183625552 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b27fc640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b4154810 0x7fa2b41567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b37fe640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b415c470 0x7fa2b40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 -- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b4154810 msgr2=0x7fa2b41567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b4154810 0x7fa2b41567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 -- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b415c470 msgr2=0x7fa2b40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b415c470 0x7fa2b40cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 -- 172.21.15.60:0/2183625552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4156cf0 con 0x7fa29c004d40 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b27fc640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b4154810 0x7fa2b41567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b37fe640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b415c470 0x7fa2b40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b2ffd640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa2b4160b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4007290 tx=0x7fa2a4007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400f940 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4018070 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b4155730 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400fd20 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.345+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b4155cf0 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.346+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b40d16b0 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.347+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2a401d460 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.347+0000 7fa29b7fe640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a006b4d0 0x7fa2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.347+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2a4089fd0 con 0x7fa29c004d40 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.347+0000 7fa2b27fc640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a006b4d0 0x7fa2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.348+0000 7fa2b27fc640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a006b4d0 0x7fa2a006d990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009ab0 tx=0x7fa2a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.351+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2a40591a0 con 0x7fa29c004d40 2024-04-25T08:29:17.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.534+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2b40b5060 con 0x7fa29c004d40 2024-04-25T08:29:17.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.534+0000 7fa29b7fe640 1 -- 172.21.15.60:0/2183625552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2a4058aa0 con 0x7fa29c004d40 2024-04-25T08:29:17.533 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a006b4d0 msgr2=0x7fa2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a006b4d0 0x7fa2a006d990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009ab0 tx=0x7fa2a8008040 comp rx=0 tx=0).stop 2024-04-25T08:29:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 msgr2=0x7fa2b4160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa2b4160b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4007290 tx=0x7fa2a4007750 comp rx=0 tx=0).stop 2024-04-25T08:29:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 shutdown_connections 2024-04-25T08:29:17.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a006b4d0 0x7fa2a006d990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b415c470 0x7fa2b40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2b4154810 0x7fa2b41567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 --2- 172.21.15.60:0/2183625552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa29c004d40 0x7fa2b4160b50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 >> 172.21.15.60:0/2183625552 conn(0x7fa2b40d0420 msgr2=0x7fa2b40b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:17.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 shutdown_connections 2024-04-25T08:29:17.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.537+0000 7fa2b933b640 1 -- 172.21.15.60:0/2183625552 wait complete. 2024-04-25T08:29:17.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.647+0000 7f703f29e640 1 Processor -- start 2024-04-25T08:29:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.648+0000 7f703f29e640 1 -- start start 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.648+0000 7f703f29e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f703815dbe0 0x7f70380d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.648+0000 7f703f29e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b92a0 0x7f70380d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.648+0000 7f703f29e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70380b97a0 con 0x7f70380b2cb0 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.648+0000 7f703f29e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703815dfc0 con 0x7f703815dbe0 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.648+0000 7f703f29e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70380d1a00 con 0x7f70380b92a0 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d03a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f703815dbe0 0x7f70380d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703c839640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f70380b2cb0 0x7f70380b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34372/0 (socket says 172.21.15.60:34372) 2024-04-25T08:29:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703c839640 1 -- 172.21.15.60:0/771139283 learned_addr learned my addr 172.21.15.60:0/771139283 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d83b640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b92a0 0x7f70380d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d03a640 1 -- 172.21.15.60:0/771139283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b92a0 msgr2=0x7f70380d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d03a640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b92a0 0x7f70380d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d03a640 1 -- 172.21.15.60:0/771139283 >> v1:172.21.15.60:6789/0 conn(0x7f70380b2cb0 legacy=0x7f70380b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d03a640 1 -- 172.21.15.60:0/771139283 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380d1c50 con 0x7f703815dbe0 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f702ffff640 1 -- 172.21.15.60:0/771139283 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040157619 0 0) 0x7f70380b97a0 con 0x7f70380b2cb0 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.649+0000 7f703d83b640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b92a0 0x7f70380d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703d03a640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380d0910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70340097a0 tx=0x7f703402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a128470032a786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f702ffff640 1 -- 172.21.15.60:0/771139283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034037070 con 0x7f703815dbe0 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f702ffff640 1 -- 172.21.15.60:0/771139283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034032440 con 0x7f703815dbe0 2024-04-25T08:29:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f702ffff640 1 -- 172.21.15.60:0/771139283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034032810 con 0x7f703815dbe0 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 -- 172.21.15.60:0/771139283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 msgr2=0x7f70380d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380d0910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f70340097a0 tx=0x7f703402a180 comp rx=0 tx=0).stop 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 -- 172.21.15.60:0/771139283 shutdown_connections 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b92a0 0x7f70380d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 --2- 172.21.15.60:0/771139283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380d0910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 -- 172.21.15.60:0/771139283 >> 172.21.15.60:0/771139283 conn(0x7f7038061a20 msgr2=0x7f7038061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.650+0000 7f703f29e640 1 -- 172.21.15.60:0/771139283 shutdown_connections 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.651+0000 7f703f29e640 1 -- 172.21.15.60:0/771139283 wait complete. 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 Processor -- start 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 -- start start 2024-04-25T08:29:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b2cb0 0x7f70380b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70380b92a0 0x7f703815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70380b6ab0 con 0x7f703815dbe0 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70380d2210 con 0x7f70380b92a0 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703f29e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70380b6c30 con 0x7f70380b2cb0 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703d83b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703d83b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39754/0 (socket says 172.21.15.60:39754) 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.652+0000 7f703d83b640 1 -- 172.21.15.60:0/3506532821 learned_addr learned my addr 172.21.15.60:0/3506532821 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d03a640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b2cb0 0x7f70380b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703c839640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70380b92a0 0x7f703815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d83b640 1 -- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b2cb0 msgr2=0x7f70380b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d83b640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b2cb0 0x7f70380b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d83b640 1 -- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70380b92a0 msgr2=0x7f703815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d83b640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70380b92a0 0x7f703815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d83b640 1 -- 172.21.15.60:0/3506532821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380b3c10 con 0x7f703815dbe0 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d03a640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b2cb0 0x7f70380b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703c839640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70380b92a0 0x7f703815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703d83b640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380b32a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7030009f70 tx=0x7f703000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703001a070 con 0x7f703815dbe0 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030003bb0 con 0x7f703815dbe0 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70380b3ef0 con 0x7f703815dbe0 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030004030 con 0x7f703815dbe0 2024-04-25T08:29:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.653+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70380b44b0 con 0x7f703815dbe0 2024-04-25T08:29:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.654+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7038073990 con 0x7f703815dbe0 2024-04-25T08:29:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.655+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f703001d070 con 0x7f703815dbe0 2024-04-25T08:29:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.655+0000 7f702dffb640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f701c06b4d0 0x7f701c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.655+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f703008a1c0 con 0x7f703815dbe0 2024-04-25T08:29:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.655+0000 7f703d03a640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f701c06b4d0 0x7f701c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.656+0000 7f703d03a640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f701c06b4d0 0x7f701c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7034034000 tx=0x7f703402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.660+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7030010070 con 0x7f703815dbe0 2024-04-25T08:29:17.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.880+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f703815bf10 con 0x7f703815dbe0 2024-04-25T08:29:17.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.881+0000 7f702dffb640 1 -- 172.21.15.60:0/3506532821 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f7030059390 con 0x7f703815dbe0 2024-04-25T08:29:17.879 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:17.879 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:29:17.879 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 2 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f701c06b4d0 msgr2=0x7f701c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f701c06b4d0 0x7f701c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7034034000 tx=0x7f703402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 msgr2=0x7f70380b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380b32a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7030009f70 tx=0x7f703000d560 comp rx=0 tx=0).stop 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 shutdown_connections 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f703815dbe0 0x7f70380b32a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f701c06b4d0 0x7f701c06d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70380b92a0 0x7f703815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 --2- 172.21.15.60:0/3506532821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70380b2cb0 0x7f70380b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 >> 172.21.15.60:0/3506532821 conn(0x7f7038061a20 msgr2=0x7f703815b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:17.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 shutdown_connections 2024-04-25T08:29:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:17.883+0000 7f703f29e640 1 -- 172.21.15.60:0/3506532821 wait complete. 2024-04-25T08:29:19.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:20.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.008+0000 7f260bb22640 1 Processor -- start 2024-04-25T08:29:20.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.008+0000 7f260bb22640 1 -- start start 2024-04-25T08:29:20.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260bb22640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f260415db50 0x7f26040c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260bb22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26040c7050 0x7f26040cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260bb22640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26041625d0 con 0x7f26040cd5b0 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260bb22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260415df30 con 0x7f26040c7050 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260bb22640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040cc2b0 con 0x7f260415db50 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26090bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f26040c7050 0x7f26040cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260a0bf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f26040cd5b0 0x7f26040cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34384/0 (socket says 172.21.15.60:34384) 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26090bd640 1 --2- 172.21.15.60:0/634465761 >> v2:172.21.15.60:3300/0 conn(0x7f26040c7050 0x7f26040cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39768/0 (socket says 172.21.15.60:39768) 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f260a0bf640 1 -- 172.21.15.60:0/634465761 learned_addr learned my addr 172.21.15.60:0/634465761 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26098be640 1 --2- 172.21.15.60:0/634465761 >> v2:172.21.15.100:3300/0 conn(0x7f260415db50 0x7f26040c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26090bd640 1 -- 172.21.15.60:0/634465761 >> v2:172.21.15.100:3300/0 conn(0x7f260415db50 msgr2=0x7f26040c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26090bd640 1 --2- 172.21.15.60:0/634465761 >> v2:172.21.15.100:3300/0 conn(0x7f260415db50 0x7f26040c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26090bd640 1 -- 172.21.15.60:0/634465761 >> v1:172.21.15.60:6789/0 conn(0x7f26040cd5b0 legacy=0x7f26040cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26088bc640 1 -- 172.21.15.60:0/634465761 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451327066 0 0) 0x7f26041625d0 con 0x7f26040cd5b0 2024-04-25T08:29:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.009+0000 7f26090bd640 1 -- 172.21.15.60:0/634465761 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040cc590 con 0x7f26040c7050 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.010+0000 7f26090bd640 1 --2- 172.21.15.60:0/634465761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040cd070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26040713b0 tx=0x7f25f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b3e6e7f52d34d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.010+0000 7f26088bc640 1 -- 172.21.15.60:0/634465761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4037070 con 0x7f26040c7050 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.010+0000 7f26088bc640 1 -- 172.21.15.60:0/634465761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4032440 con 0x7f26040c7050 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.010+0000 7f26088bc640 1 -- 172.21.15.60:0/634465761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f402ae60 con 0x7f26040c7050 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 -- 172.21.15.60:0/634465761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 msgr2=0x7f26040cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 --2- 172.21.15.60:0/634465761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040cd070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26040713b0 tx=0x7f25f402a110 comp rx=0 tx=0).stop 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 -- 172.21.15.60:0/634465761 shutdown_connections 2024-04-25T08:29:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 --2- 172.21.15.60:0/634465761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040cd070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 --2- 172.21.15.60:0/634465761 >> v2:172.21.15.100:3300/0 conn(0x7f260415db50 0x7f26040c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 -- 172.21.15.60:0/634465761 >> 172.21.15.60:0/634465761 conn(0x7f26040d1800 msgr2=0x7f26040d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 -- 172.21.15.60:0/634465761 shutdown_connections 2024-04-25T08:29:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.011+0000 7f260bb22640 1 -- 172.21.15.60:0/634465761 wait complete. 2024-04-25T08:29:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.012+0000 7f260bb22640 1 Processor -- start 2024-04-25T08:29:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.012+0000 7f260bb22640 1 -- start start 2024-04-25T08:29:20.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26040cd5b0 0x7f2604157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260415db50 0x7f2604158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040d0240 con 0x7f26040c7050 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040ccb50 con 0x7f260415db50 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26040d03c0 con 0x7f26040cd5b0 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39784/0 (socket says 172.21.15.60:39784) 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 -- 172.21.15.60:0/919213441 learned_addr learned my addr 172.21.15.60:0/919213441 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260a0bf640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260415db50 0x7f2604158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26090bd640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26040cd5b0 0x7f2604157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 -- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26040cd5b0 msgr2=0x7f2604157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26040cd5b0 0x7f2604157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 -- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260415db50 msgr2=0x7f2604158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260415db50 0x7f2604158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 -- 172.21.15.60:0/919213441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26041497f0 con 0x7f26040c7050 2024-04-25T08:29:20.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f26098be640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040c91d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f260000e870 tx=0x7f260000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600017070 con 0x7f26040c7050 2024-04-25T08:29:20.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260000c940 con 0x7f26040c7050 2024-04-25T08:29:20.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260000cd30 con 0x7f26040c7050 2024-04-25T08:29:20.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.013+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2604159970 con 0x7f26040c7050 2024-04-25T08:29:20.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.014+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26040c8a90 con 0x7f26040c7050 2024-04-25T08:29:20.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.014+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2604073990 con 0x7f26040c7050 2024-04-25T08:29:20.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.015+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f260001a070 con 0x7f26040c7050 2024-04-25T08:29:20.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.015+0000 7f25fa7fc640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25e806b4d0 0x7f25e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.016+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f260008a040 con 0x7f26040c7050 2024-04-25T08:29:20.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.016+0000 7f26090bd640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25e806b4d0 0x7f25e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.016+0000 7f26090bd640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25e806b4d0 0x7f25e806d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.020+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26000592a0 con 0x7f26040c7050 2024-04-25T08:29:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.238+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26040c66c0 con 0x7f26040c7050 2024-04-25T08:29:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.239+0000 7f25fa7fc640 1 -- 172.21.15.60:0/919213441 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f2600058ba0 con 0x7f26040c7050 2024-04-25T08:29:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:20.237 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T08:29:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 3 2024-04-25T08:29:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25e806b4d0 msgr2=0x7f25e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25e806b4d0 0x7f25e806d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 msgr2=0x7f26040c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040c91d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f260000e870 tx=0x7f260000ed30 comp rx=0 tx=0).stop 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 shutdown_connections 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25e806b4d0 0x7f25e806d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f260415db50 0x7f2604158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26040cd5b0 0x7f2604157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 --2- 172.21.15.60:0/919213441 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26040c7050 0x7f26040c91d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 >> 172.21.15.60:0/919213441 conn(0x7f26040d1800 msgr2=0x7f26040c7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 shutdown_connections 2024-04-25T08:29:20.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.241+0000 7f260bb22640 1 -- 172.21.15.60:0/919213441 wait complete. 2024-04-25T08:29:20.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.352+0000 7f7af4149640 1 Processor -- start 2024-04-25T08:29:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.352+0000 7f7af4149640 1 -- start start 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af4149640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7aec0b2cb0 0x7f7aec0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af4149640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 0x7f7aec0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af4149640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aec0b97a0 con 0x7f7aec15dbe0 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af4149640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec15dfc0 con 0x7f7aec0b2cb0 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af4149640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0d1a00 con 0x7f7aec0d1020 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af1ee5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7aec15dbe0 0x7f7aec0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34392/0 (socket says 172.21.15.60:34392) 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af1ee5640 1 -- 172.21.15.60:0/1716603807 learned_addr learned my addr 172.21.15.60:0/1716603807 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af16e4640 1 --2- 172.21.15.60:0/1716603807 >> v2:172.21.15.100:3300/0 conn(0x7f7aec0b2cb0 0x7f7aec0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.353+0000 7f7af26e6640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 0x7f7aec0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993189751 0 0) 0x7f7aec0b97a0 con 0x7f7aec15dbe0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad4003520 con 0x7f7aec15dbe0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 706114980 0 0) 0x7f7ad4003520 con 0x7f7aec15dbe0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 msgr2=0x7f7aec0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 0x7f7aec0d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 >> v2:172.21.15.100:3300/0 conn(0x7f7aec0b2cb0 msgr2=0x7f7aec0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 --2- 172.21.15.60:0/1716603807 >> v2:172.21.15.100:3300/0 conn(0x7f7aec0b2cb0 0x7f7aec0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0d1c50 con 0x7f7aec15dbe0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f7ae4001670 con 0x7f7aec15dbe0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 >> v1:172.21.15.60:6789/0 conn(0x7f7aec15dbe0 legacy=0x7f7aec0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7ad4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad4004800 0x7f7ad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 0x7f7ad40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0d1a00 con 0x7f7ad40039b0 2024-04-25T08:29:20.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad400b9c0 con 0x7f7ad4004800 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad400bb40 con 0x7f7aec0d1020 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7ad4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af16e4640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad4004800 0x7f7ad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af26e6640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 0x7f7ad40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af1ee5640 1 -- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 msgr2=0x7f7ad40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0d1020 0x7f7ad40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af1ee5640 1 -- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad4004800 msgr2=0x7f7ad4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ad4004800 0x7f7ad4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.354+0000 7f7af1ee5640 1 -- 172.21.15.60:0/1716603807 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4003520 con 0x7f7ad40039b0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.355+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7ad4003dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c290 tx=0x7f7ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4538daa89a54bbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.355+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4007d50 con 0x7f7ad40039b0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.355+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4005ad0 con 0x7f7ad40039b0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.355+0000 7f7af0ee3640 1 -- 172.21.15.60:0/1716603807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4007070 con 0x7f7ad40039b0 2024-04-25T08:29:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af4149640 1 -- 172.21.15.60:0/1716603807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 msgr2=0x7f7ad4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af4149640 1 --2- 172.21.15.60:0/1716603807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7ad4003dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400c290 tx=0x7f7ad4005830 comp rx=0 tx=0).stop 2024-04-25T08:29:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af26e6640 1 -- 172.21.15.60:0/1716603807 reap_dead start 2024-04-25T08:29:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af4149640 1 -- 172.21.15.60:0/1716603807 shutdown_connections 2024-04-25T08:29:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af4149640 1 -- 172.21.15.60:0/1716603807 >> 172.21.15.60:0/1716603807 conn(0x7f7aec061a20 msgr2=0x7f7aec061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af4149640 1 -- 172.21.15.60:0/1716603807 shutdown_connections 2024-04-25T08:29:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.356+0000 7f7af4149640 1 -- 172.21.15.60:0/1716603807 wait complete. 2024-04-25T08:29:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.357+0000 7f7af4149640 1 Processor -- start 2024-04-25T08:29:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.357+0000 7f7af4149640 1 -- start start 2024-04-25T08:29:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7aec154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec15dfc0 0x7f7aec156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0ccc20 0x7f7aec0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b92a0 con 0x7f7ad40039b0 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0d2210 con 0x7f7aec15dfc0 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b9420 con 0x7f7aec0ccc20 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7aec154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7aec154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39800/0 (socket says 172.21.15.60:39800) 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 -- 172.21.15.60:0/2915847058 learned_addr learned my addr 172.21.15.60:0/2915847058 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af16e4640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec15dfc0 0x7f7aec156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af26e6640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0ccc20 0x7f7aec0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 -- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0ccc20 msgr2=0x7f7aec0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0ccc20 0x7f7aec0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 -- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec15dfc0 msgr2=0x7f7aec156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec15dfc0 0x7f7aec156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 -- 172.21.15.60:0/2915847058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec15e530 con 0x7f7ad40039b0 2024-04-25T08:29:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af1ee5640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7aec154f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ae402ae50 tx=0x7f7aec0d1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4042070 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4007480 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec15e810 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.358+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4001c80 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.359+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec15edd0 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.360+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec073990 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.360+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ae4045070 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.360+0000 7f7aea7fc640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ad806b4d0 0x7f7ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.361+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ae40aab20 con 0x7f7ad40039b0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.361+0000 7f7af16e4640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ad806b4d0 0x7f7ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.361+0000 7f7af16e4640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ad806b4d0 0x7f7ad806d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7aec1f6aa0 tx=0x7f7ae0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.365+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ae404c030 con 0x7f7ad40039b0 2024-04-25T08:29:20.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.545+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7aec0b2a70 con 0x7f7ad40039b0 2024-04-25T08:29:20.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.546+0000 7f7aea7fc640 1 -- 172.21.15.60:0/2915847058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f7ae4007df0 con 0x7f7ad40039b0 2024-04-25T08:29:20.544 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:20.544 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":17,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:13.531094+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ad806b4d0 msgr2=0x7f7ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ad806b4d0 0x7f7ad806d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7aec1f6aa0 tx=0x7f7ae0008040 comp rx=0 tx=0).stop 2024-04-25T08:29:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 msgr2=0x7f7aec154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7aec154f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7ae402ae50 tx=0x7f7aec0d1e20 comp rx=0 tx=0).stop 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 shutdown_connections 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7aec0ccc20 0x7f7aec0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ad806b4d0 0x7f7ad806d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec15dfc0 0x7f7aec156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 --2- 172.21.15.60:0/2915847058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ad40039b0 0x7f7aec154f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 >> 172.21.15.60:0/2915847058 conn(0x7f7aec061a20 msgr2=0x7f7aec15bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 shutdown_connections 2024-04-25T08:29:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.548+0000 7f7af4149640 1 -- 172.21.15.60:0/2915847058 wait complete. 2024-04-25T08:29:20.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:29:20.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.662+0000 7f699e8f7640 1 Processor -- start 2024-04-25T08:29:20.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.662+0000 7f699e8f7640 1 -- start start 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f699e8f7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69980cadb0 0x7f69980cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f699e8f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980ccfe0 0x7f69980cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f699e8f7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980afc80 con 0x7f699815de80 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f699e8f7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69980cb240 con 0x7f69980cadb0 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f699e8f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980ce4d0 con 0x7f69980ccfe0 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f69977fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69980cadb0 0x7f69980cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f69977fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69980cadb0 0x7f69980cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39812/0 (socket says 172.21.15.60:39812) 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.663+0000 7f69977fe640 1 -- 172.21.15.60:0/3411321228 learned_addr learned my addr 172.21.15.60:0/3411321228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f699ce94640 1 --2- 172.21.15.60:0/3411321228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980ccfe0 0x7f69980cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f69977fe640 1 -- 172.21.15.60:0/3411321228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980ccfe0 msgr2=0x7f69980cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f69977fe640 1 --2- 172.21.15.60:0/3411321228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980ccfe0 0x7f69980cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f69977fe640 1 -- 172.21.15.60:0/3411321228 >> v1:172.21.15.100:6789/0 conn(0x7f699815de80 legacy=0x7f69980cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:20.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f69977fe640 1 -- 172.21.15.60:0/3411321228 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980ce7b0 con 0x7f69980cadb0 2024-04-25T08:29:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f69977fe640 1 --2- 172.21.15.60:0/3411321228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980cadb0 0x7f69980cf6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35010ecedd142b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f6996ffd640 1 -- 172.21.15.60:0/3411321228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c037070 con 0x7f69980cadb0 2024-04-25T08:29:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f6996ffd640 1 -- 172.21.15.60:0/3411321228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c032440 con 0x7f69980cadb0 2024-04-25T08:29:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.664+0000 7f6996ffd640 1 -- 172.21.15.60:0/3411321228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c032810 con 0x7f69980cadb0 2024-04-25T08:29:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 -- 172.21.15.60:0/3411321228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980cadb0 msgr2=0x7f69980cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 --2- 172.21.15.60:0/3411321228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980cadb0 0x7f69980cf6c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c02a210 comp rx=0 tx=0).stop 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 -- 172.21.15.60:0/3411321228 shutdown_connections 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 --2- 172.21.15.60:0/3411321228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980cadb0 0x7f69980cf6c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 --2- 172.21.15.60:0/3411321228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980ccfe0 0x7f69980cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 -- 172.21.15.60:0/3411321228 >> 172.21.15.60:0/3411321228 conn(0x7f69980b2390 msgr2=0x7f69980b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 -- 172.21.15.60:0/3411321228 shutdown_connections 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.665+0000 7f699e8f7640 1 -- 172.21.15.60:0/3411321228 wait complete. 2024-04-25T08:29:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.666+0000 7f699e8f7640 1 Processor -- start 2024-04-25T08:29:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.666+0000 7f699e8f7640 1 -- start start 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f699e8f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980cadb0 0x7f69980b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f699e8f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 0x7f69980c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f699e8f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f699815de80 0x7f6998158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f699e8f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699815e260 con 0x7f69980ccfe0 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f699e8f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980cfc00 con 0x7f699815de80 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f699e8f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69980b2eb0 con 0x7f69980cadb0 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f69977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 0x7f69980c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f69977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 0x7f69980c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39820/0 (socket says 172.21.15.60:39820) 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f69977fe640 1 -- 172.21.15.60:0/177136822 learned_addr learned my addr 172.21.15.60:0/177136822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.667+0000 7f6997fff640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980cadb0 0x7f69980b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f699ce94640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f699815de80 0x7f6998158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f69977fe640 1 -- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980cadb0 msgr2=0x7f69980b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f69977fe640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980cadb0 0x7f69980b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f69977fe640 1 -- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f699815de80 msgr2=0x7f6998158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f69977fe640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f699815de80 0x7f6998158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f69977fe640 1 -- 172.21.15.60:0/177136822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980c8c90 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f6997fff640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980cadb0 0x7f69980b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f699ce94640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f699815de80 0x7f6998158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f69977fe640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 0x7f69980c8240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c041020 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c030970 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c037040 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69980c8f70 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.668+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69980c9530 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.669+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69980d03d0 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.669+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f698c044020 con 0x7f69980ccfe0 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.670+0000 7f6994ff9640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f698406b4d0 0x7f698406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.670+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f698c0aa8b0 con 0x7f69980ccfe0 2024-04-25T08:29:20.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.670+0000 7f6997fff640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f698406b4d0 0x7f698406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.671+0000 7f6997fff640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f698406b4d0 0x7f698406d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6988003eb0 tx=0x7f6988009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.674+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69980d03d0 con 0x7f69980ccfe0 2024-04-25T08:29:20.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.853+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6998073a00 con 0x7f69980ccfe0 2024-04-25T08:29:20.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.853+0000 7f6994ff9640 1 -- 172.21.15.60:0/177136822 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f698c04b030 con 0x7f69980ccfe0 2024-04-25T08:29:20.852 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:20.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f698406b4d0 msgr2=0x7f698406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f698406b4d0 0x7f698406d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6988003eb0 tx=0x7f6988009040 comp rx=0 tx=0).stop 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 msgr2=0x7f69980c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 0x7f69980c8240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 shutdown_connections 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f698406b4d0 0x7f698406d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f699815de80 0x7f6998158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69980ccfe0 0x7f69980c8240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 --2- 172.21.15.60:0/177136822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69980cadb0 0x7f69980b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 >> 172.21.15.60:0/177136822 conn(0x7f69980b2390 msgr2=0x7f699815c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 shutdown_connections 2024-04-25T08:29:20.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.855+0000 7f699e8f7640 1 -- 172.21.15.60:0/177136822 wait complete. 2024-04-25T08:29:20.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:29:20.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.970+0000 7f7003bab640 1 Processor -- start 2024-04-25T08:29:20.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.970+0000 7f7003bab640 1 -- start start 2024-04-25T08:29:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.970+0000 7f7003bab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ffc15dd30 0x7f6ffc0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7003bab640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc0d1df0 con 0x7f6ffc0d0ad0 2024-04-25T08:29:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7003bab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc0d20d0 con 0x7f6ffc0b9770 2024-04-25T08:29:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7003bab640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc0d23b0 con 0x7f6ffc15dd30 2024-04-25T08:29:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7001947640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ffc15dd30 0x7f6ffc0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7002148640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6ffc0d0ad0 0x7f6ffc0b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34408/0 (socket says 172.21.15.60:34408) 2024-04-25T08:29:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7002148640 1 -- 172.21.15.60:0/3748522899 learned_addr learned my addr 172.21.15.60:0/3748522899 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7001947640 1 -- 172.21.15.60:0/3748522899 >> v1:172.21.15.100:6789/0 conn(0x7f6ffc0b9770 legacy=0x7f6ffc0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7001947640 1 -- 172.21.15.60:0/3748522899 >> v1:172.21.15.60:6789/0 conn(0x7f6ffc0d0ad0 legacy=0x7f6ffc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7001947640 1 -- 172.21.15.60:0/3748522899 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc0d2690 con 0x7f6ffc15dd30 2024-04-25T08:29:20.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.971+0000 7f7000945640 1 -- 172.21.15.60:0/3748522899 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501123158 0 0) 0x7f6ffc0d1df0 con 0x7f6ffc0d0ad0 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7001947640 1 --2- 172.21.15.60:0/3748522899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc0b9230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80097a0 tx=0x7f6ff802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b794233bdedcacd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7000945640 1 -- 172.21.15.60:0/3748522899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8037070 con 0x7f6ffc15dd30 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7000945640 1 -- 172.21.15.60:0/3748522899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8032440 con 0x7f6ffc15dd30 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7000945640 1 -- 172.21.15.60:0/3748522899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff802ae60 con 0x7f6ffc15dd30 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7003bab640 1 -- 172.21.15.60:0/3748522899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc15dd30 msgr2=0x7f6ffc0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7003bab640 1 --2- 172.21.15.60:0/3748522899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc0b9230 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80097a0 tx=0x7f6ff802a110 comp rx=0 tx=0).stop 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7003bab640 1 -- 172.21.15.60:0/3748522899 shutdown_connections 2024-04-25T08:29:20.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7003bab640 1 --2- 172.21.15.60:0/3748522899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc0b9230 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7003bab640 1 -- 172.21.15.60:0/3748522899 >> 172.21.15.60:0/3748522899 conn(0x7f6ffc061c30 msgr2=0x7f6ffc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:20.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.972+0000 7f7003bab640 1 -- 172.21.15.60:0/3748522899 shutdown_connections 2024-04-25T08:29:20.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.973+0000 7f7003bab640 1 -- 172.21.15.60:0/3748522899 wait complete. 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 Processor -- start 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 -- start start 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 0x7f6ffc155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ffc0d0ad0 0x7f6ffc15f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc15fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0d1b90 con 0x7f6ffc0b9770 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0d1a10 con 0x7f6ffc15dd30 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7003bab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0d2e90 con 0x7f6ffc0d0ad0 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7001947640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 0x7f6ffc155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7001947640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 0x7f6ffc155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39836/0 (socket says 172.21.15.60:39836) 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.974+0000 7f7001947640 1 -- 172.21.15.60:0/1261730463 learned_addr learned my addr 172.21.15.60:0/1261730463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:20.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7002148640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc15fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001146640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ffc0d0ad0 0x7f6ffc15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001947640 1 -- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ffc0d0ad0 msgr2=0x7f6ffc15f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001947640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ffc0d0ad0 0x7f6ffc15f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001947640 1 -- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc15dd30 msgr2=0x7f6ffc15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001947640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc15fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001947640 1 -- 172.21.15.60:0/1261730463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc1567f0 con 0x7f6ffc0b9770 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7002148640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001146640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ffc0d0ad0 0x7f6ffc15f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7001947640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 0x7f6ffc155ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8000c00 tx=0x7f6ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8037040 con 0x7f6ffc0b9770 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8042070 con 0x7f6ffc0b9770 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc156ad0 con 0x7f6ffc0b9770 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8030a70 con 0x7f6ffc0b9770 2024-04-25T08:29:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.975+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc1563e0 con 0x7f6ffc0b9770 2024-04-25T08:29:20.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.976+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc073990 con 0x7f6ffc0b9770 2024-04-25T08:29:20.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.977+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ff8030c10 con 0x7f6ffc0b9770 2024-04-25T08:29:20.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.977+0000 7f6ff27fc640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fe006b500 0x7f6fe006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:20.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.977+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ff80aaa00 con 0x7f6ffc0b9770 2024-04-25T08:29:20.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.977+0000 7f7001146640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fe006b500 0x7f6fe006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:20.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.978+0000 7f7001146640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fe006b500 0x7f6fe006d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc1f6fa0 tx=0x7f6fec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:20.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:20.981+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ff8079b40 con 0x7f6ffc0b9770 2024-04-25T08:29:21.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.165+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ffc15c750 con 0x7f6ffc0b9770 2024-04-25T08:29:21.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.165+0000 7f6ff27fc640 1 -- 172.21.15.60:0/1261730463 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6ff8079440 con 0x7f6ffc0b9770 2024-04-25T08:29:21.221 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:21.222 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:29:21.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:29:21.222 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:29:21.222 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:29:21.222 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:29:21.222 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:29:21.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fe006b500 msgr2=0x7f6fe006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fe006b500 0x7f6fe006d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc1f6fa0 tx=0x7f6fec009040 comp rx=0 tx=0).stop 2024-04-25T08:29:21.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 msgr2=0x7f6ffc155ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 0x7f6ffc155ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8000c00 tx=0x7f6ff80046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 shutdown_connections 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fe006b500 0x7f6fe006d9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ffc15dd30 0x7f6ffc15fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ffc0d0ad0 0x7f6ffc15f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 --2- 172.21.15.60:0/1261730463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ffc0b9770 0x7f6ffc155ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 >> 172.21.15.60:0/1261730463 conn(0x7f6ffc061c30 msgr2=0x7f6ffc0c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 shutdown_connections 2024-04-25T08:29:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.168+0000 7f7003bab640 1 -- 172.21.15.60:0/1261730463 wait complete. 2024-04-25T08:29:21.224 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:29:21.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:29:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.331+0000 7f5364ad8640 1 Processor -- start 2024-04-25T08:29:21.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.332+0000 7f5364ad8640 1 -- start start 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.332+0000 7f5364ad8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f536015bb50 0x7f53600c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.332+0000 7f5364ad8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53600c75e0 con 0x7f53600c70d0 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.332+0000 7f5364ad8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53600c78c0 con 0x7f53600cfd10 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.332+0000 7f5364ad8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53600c8680 con 0x7f536015bb50 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.333+0000 7f535e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f536015bb50 0x7f53600c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.333+0000 7f5357fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f53600c70d0 0x7f53600cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34410/0 (socket says 172.21.15.60:34410) 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.333+0000 7f5357fff640 1 -- 172.21.15.60:0/3158189527 learned_addr learned my addr 172.21.15.60:0/3158189527 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.333+0000 7f535e59c640 1 -- 172.21.15.60:0/3158189527 >> v1:172.21.15.100:6789/0 conn(0x7f53600cfd10 legacy=0x7f53600cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.333+0000 7f535e59c640 1 -- 172.21.15.60:0/3158189527 >> v1:172.21.15.60:6789/0 conn(0x7f53600c70d0 legacy=0x7f53600cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.333+0000 7f535e59c640 1 -- 172.21.15.60:0/3158189527 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53600c7ba0 con 0x7f536015bb50 2024-04-25T08:29:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f535e59c640 1 --2- 172.21.15.60:0/3158189527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536015bb50 0x7f53600c6b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53500097a0 tx=0x7f535002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14567f132407cb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:21.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f535dd9b640 1 -- 172.21.15.60:0/3158189527 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350037070 con 0x7f536015bb50 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f535dd9b640 1 -- 172.21.15.60:0/3158189527 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350032640 con 0x7f536015bb50 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f535dd9b640 1 -- 172.21.15.60:0/3158189527 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350032a10 con 0x7f536015bb50 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 -- 172.21.15.60:0/3158189527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536015bb50 msgr2=0x7f53600c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 --2- 172.21.15.60:0/3158189527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536015bb50 0x7f53600c6b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53500097a0 tx=0x7f535002a2c0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 -- 172.21.15.60:0/3158189527 shutdown_connections 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 --2- 172.21.15.60:0/3158189527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536015bb50 0x7f53600c6b90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 -- 172.21.15.60:0/3158189527 >> 172.21.15.60:0/3158189527 conn(0x7f536006f500 msgr2=0x7f536006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 -- 172.21.15.60:0/3158189527 shutdown_connections 2024-04-25T08:29:21.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.334+0000 7f5364ad8640 1 -- 172.21.15.60:0/3158189527 wait complete. 2024-04-25T08:29:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 Processor -- start 2024-04-25T08:29:21.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 -- start start 2024-04-25T08:29:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 0x7f5360140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53600cfd10 0x7f53601533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536015bb50 0x7f53601e6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536015f6d0 con 0x7f53600c70d0 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600c8160 con 0x7f536015bb50 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f5364ad8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536015f850 con 0x7f53600cfd10 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f535e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 0x7f5360140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f535e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 0x7f5360140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39854/0 (socket says 172.21.15.60:39854) 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.336+0000 7f535e59c640 1 -- 172.21.15.60:0/2287272154 learned_addr learned my addr 172.21.15.60:0/2287272154 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535ed9d640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536015bb50 0x7f53601e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f5357fff640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53600cfd10 0x7f53601533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535e59c640 1 -- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53600cfd10 msgr2=0x7f53601533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535e59c640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53600cfd10 0x7f53601533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535e59c640 1 -- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536015bb50 msgr2=0x7f53601e6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535e59c640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536015bb50 0x7f53601e6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:21.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535e59c640 1 -- 172.21.15.60:0/2287272154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360140770 con 0x7f53600c70d0 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535ed9d640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536015bb50 0x7f53601e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f535e59c640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 0x7f5360140230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5350000c00 tx=0x7f53500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350041020 con 0x7f53600c70d0 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350030970 con 0x7f53600c70d0 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5360153d80 con 0x7f53600c70d0 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350037040 con 0x7f53600c70d0 2024-04-25T08:29:21.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.337+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536015d7a0 con 0x7f53600c70d0 2024-04-25T08:29:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.338+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5360073a00 con 0x7f53600c70d0 2024-04-25T08:29:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.338+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5350044020 con 0x7f53600c70d0 2024-04-25T08:29:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.339+0000 7f5356ffd640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.339+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53500aa7d0 con 0x7f53600c70d0 2024-04-25T08:29:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.339+0000 7f5357fff640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:21.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.339+0000 7f5357fff640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f534006b4d0 0x7f534006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f536015b540 tx=0x7f5348009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.343+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f535004b030 con 0x7f53600c70d0 2024-04-25T08:29:21.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:21.526+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f53600cf150 con 0x7f53600c70d0 2024-04-25T08:29:22.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.012+0000 7f5356ffd640 1 -- 172.21.15.60:0/2287272154 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5360140770 con 0x7f53600c70d0 2024-04-25T08:29:22.010 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f534006b4d0 msgr2=0x7f534006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f534006b4d0 0x7f534006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f536015b540 tx=0x7f5348009040 comp rx=0 tx=0).stop 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 msgr2=0x7f5360140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 0x7f5360140230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5350000c00 tx=0x7f53500046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 shutdown_connections 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536015bb50 0x7f53601e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53600cfd10 0x7f53601533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 --2- 172.21.15.60:0/2287272154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53600c70d0 0x7f5360140230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 >> 172.21.15.60:0/2287272154 conn(0x7f536006f500 msgr2=0x7f5360070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.014+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 shutdown_connections 2024-04-25T08:29:22.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.015+0000 7f5364ad8640 1 -- 172.21.15.60:0/2287272154 wait complete. 2024-04-25T08:29:22.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T08:29:22.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.144+0000 7fa9bd14c640 1 Processor -- start 2024-04-25T08:29:22.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.144+0000 7fa9bd14c640 1 -- start start 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9bd14c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9b815aa10 0x7fa9b80c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9bd14c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9b80ca5a0 0x7fa9b806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9bd14c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9bd14c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b806c5a0 con 0x7fa9b80ca5a0 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9bd14c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80cfa70 con 0x7fa9b815aa10 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9bd14c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80cc2e0 con 0x7fa9b806c130 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9b659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9b80ca5a0 0x7fa9b806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9b659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9b80ca5a0 0x7fa9b806bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49266/0 (socket says 172.21.15.60:49266) 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.145+0000 7fa9b659c640 1 -- 172.21.15.60:0/641059075 learned_addr learned my addr 172.21.15.60:0/641059075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:22.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b6d9d640 1 --2- 172.21.15.60:0/641059075 >> v2:172.21.15.100:3300/0 conn(0x7fa9b815aa10 0x7fa9b80c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b759e640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b659c640 1 -- 172.21.15.60:0/641059075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 msgr2=0x7fa9b80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b659c640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b659c640 1 -- 172.21.15.60:0/641059075 >> v2:172.21.15.100:3300/0 conn(0x7fa9b815aa10 msgr2=0x7fa9b80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b659c640 1 --2- 172.21.15.60:0/641059075 >> v2:172.21.15.100:3300/0 conn(0x7fa9b815aa10 0x7fa9b80c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b659c640 1 -- 172.21.15.60:0/641059075 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80cc5c0 con 0x7fa9b80ca5a0 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b6d9d640 1 --2- 172.21.15.60:0/641059075 >> v2:172.21.15.100:3300/0 conn(0x7fa9b815aa10 0x7fa9b80c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:22.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b759e640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:22.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b659c640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b806bbf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9b806c740 tx=0x7fa9a002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1b324fbe9effda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:22.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.146+0000 7fa9b5d9b640 1 -- 172.21.15.60:0/641059075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0037070 con 0x7fa9b80ca5a0 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9b5d9b640 1 -- 172.21.15.60:0/641059075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a0032440 con 0x7fa9b80ca5a0 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9b5d9b640 1 -- 172.21.15.60:0/641059075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0032810 con 0x7fa9b80ca5a0 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 -- 172.21.15.60:0/641059075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b80ca5a0 msgr2=0x7fa9b806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b806bbf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9b806c740 tx=0x7fa9a002a140 comp rx=0 tx=0).stop 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 -- 172.21.15.60:0/641059075 shutdown_connections 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/641059075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b806bbf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/641059075 >> v2:172.21.15.100:3300/0 conn(0x7fa9b815aa10 0x7fa9b80c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 -- 172.21.15.60:0/641059075 >> 172.21.15.60:0/641059075 conn(0x7fa9b8071af0 msgr2=0x7fa9b8071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 -- 172.21.15.60:0/641059075 shutdown_connections 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.147+0000 7fa9bd14c640 1 -- 172.21.15.60:0/641059075 wait complete. 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 Processor -- start 2024-04-25T08:29:22.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 -- start start 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b8153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 0x7fa9b80af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80cbe70 con 0x7fa9b815aa10 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80cf700 con 0x7fa9b80ca5a0 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9bd14c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80cbff0 con 0x7fa9b806c130 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 0x7fa9b80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9b759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 0x7fa9b80af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49280/0 (socket says 172.21.15.60:49280) 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.149+0000 7fa9b759e640 1 -- 172.21.15.60:0/811304370 learned_addr learned my addr 172.21.15.60:0/811304370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b659c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b8153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b6d9d640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b759e640 1 -- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 msgr2=0x7fa9b80ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b759e640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80ae270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b759e640 1 -- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b80ca5a0 msgr2=0x7fa9b8153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b759e640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b8153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b759e640 1 -- 172.21.15.60:0/811304370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b8157da0 con 0x7fa9b815aa10 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9b759e640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 0x7fa9b80af230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800cef0 tx=0x7fa9a800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a801a070 con 0x7fa9b815aa10 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a800d840 con 0x7fa9b815aa10 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b8158080 con 0x7fa9b815aa10 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800dc30 con 0x7fa9b815aa10 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.150+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b8158640 con 0x7fa9b815aa10 2024-04-25T08:29:22.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.151+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b8073a00 con 0x7fa9b815aa10 2024-04-25T08:29:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.152+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9a801d070 con 0x7fa9b815aa10 2024-04-25T08:29:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.152+0000 7fa9af7fe640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99806b4d0 0x7fa99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.152+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa9a808a8b0 con 0x7fa9b815aa10 2024-04-25T08:29:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.152+0000 7fa9b6d9d640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99806b4d0 0x7fa99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:22.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.153+0000 7fa9b6d9d640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99806b4d0 0x7fa99806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000b840 tx=0x7fa9b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.157+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9a8059910 con 0x7fa9b815aa10 2024-04-25T08:29:22.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:22.337+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fa9b815a110 con 0x7fa9b815aa10 2024-04-25T08:29:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.014+0000 7fa9af7fe640 1 -- 172.21.15.60:0/811304370 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fa9a8059210 con 0x7fa9b815aa10 2024-04-25T08:29:23.013 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.017+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99806b4d0 msgr2=0x7fa99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.017+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99806b4d0 0x7fa99806d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000b840 tx=0x7fa9b0008040 comp rx=0 tx=0).stop 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 msgr2=0x7fa9b80af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 0x7fa9b80af230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800cef0 tx=0x7fa9a800a710 comp rx=0 tx=0).stop 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 shutdown_connections 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b815aa10 0x7fa9b80af230 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99806b4d0 0x7fa99806d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b80ca5a0 0x7fa9b8153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 --2- 172.21.15.60:0/811304370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b806c130 0x7fa9b80ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 >> 172.21.15.60:0/811304370 conn(0x7fa9b8071af0 msgr2=0x7fa9b815c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 shutdown_connections 2024-04-25T08:29:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.018+0000 7fa9bd14c640 1 -- 172.21.15.60:0/811304370 wait complete. 2024-04-25T08:29:23.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:29:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.148+0000 7fe78b10a640 1 Processor -- start 2024-04-25T08:29:23.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.148+0000 7fe78b10a640 1 -- start start 2024-04-25T08:29:23.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.148+0000 7fe78b10a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe7840cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:23.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.148+0000 7fe78b10a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7840b5620 0x7fe7840b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:23.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe78b10a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78406cf50 con 0x7fe78406c2d0 2024-04-25T08:29:23.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe78b10a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7840b5fc0 con 0x7fe7840b5620 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe78b10a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840ce530 con 0x7fe78415b7d0 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe7896a7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe78406c2d0 0x7fe78406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38868/0 (socket says 172.21.15.60:38868) 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe7896a7640 1 -- 172.21.15.60:0/1936534680 learned_addr learned my addr 172.21.15.60:0/1936534680 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe788ea6640 1 --2- 172.21.15.60:0/1936534680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe7840cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe783fff640 1 --2- 172.21.15.60:0/1936534680 >> v2:172.21.15.100:3300/0 conn(0x7fe7840b5620 0x7fe7840b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe7837fe640 1 -- 172.21.15.60:0/1936534680 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898060572 0 0) 0x7fe78406cf50 con 0x7fe78406c2d0 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe7837fe640 1 -- 172.21.15.60:0/1936534680 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe774003520 con 0x7fe78406c2d0 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe788ea6640 1 -- 172.21.15.60:0/1936534680 >> v2:172.21.15.100:3300/0 conn(0x7fe7840b5620 msgr2=0x7fe7840b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe788ea6640 1 --2- 172.21.15.60:0/1936534680 >> v2:172.21.15.100:3300/0 conn(0x7fe7840b5620 0x7fe7840b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe788ea6640 1 -- 172.21.15.60:0/1936534680 >> v1:172.21.15.60:6789/0 conn(0x7fe78406c2d0 legacy=0x7fe78406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe788ea6640 1 -- 172.21.15.60:0/1936534680 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840ce810 con 0x7fe78415b7d0 2024-04-25T08:29:23.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.149+0000 7fe783fff640 1 --2- 172.21.15.60:0/1936534680 >> v2:172.21.15.100:3300/0 conn(0x7fe7840b5620 0x7fe7840b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:23.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe788ea6640 1 --2- 172.21.15.60:0/1936534680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe7840cf010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097a0 tx=0x7fe77802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4e737b51c74e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe7837fe640 1 -- 172.21.15.60:0/1936534680 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778037070 con 0x7fe78415b7d0 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe7837fe640 1 -- 172.21.15.60:0/1936534680 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe778032440 con 0x7fe78415b7d0 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe7837fe640 1 -- 172.21.15.60:0/1936534680 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7780327f0 con 0x7fe78415b7d0 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe78b10a640 1 -- 172.21.15.60:0/1936534680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 msgr2=0x7fe7840cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe78b10a640 1 --2- 172.21.15.60:0/1936534680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe7840cf010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7780097a0 tx=0x7fe77802a180 comp rx=0 tx=0).stop 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe78b10a640 1 -- 172.21.15.60:0/1936534680 shutdown_connections 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe78b10a640 1 --2- 172.21.15.60:0/1936534680 >> v2:172.21.15.100:3300/0 conn(0x7fe7840b5620 0x7fe7840b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe78b10a640 1 --2- 172.21.15.60:0/1936534680 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe7840cf010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.150+0000 7fe78b10a640 1 -- 172.21.15.60:0/1936534680 >> 172.21.15.60:0/1936534680 conn(0x7fe7840d0800 msgr2=0x7fe7840d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.151+0000 7fe78b10a640 1 -- 172.21.15.60:0/1936534680 shutdown_connections 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.151+0000 7fe78b10a640 1 -- 172.21.15.60:0/1936534680 wait complete. 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.152+0000 7fe78b10a640 1 Processor -- start 2024-04-25T08:29:23.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.152+0000 7fe78b10a640 1 -- start start 2024-04-25T08:29:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.152+0000 7fe78b10a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 0x7fe7841f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.152+0000 7fe78b10a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840b5620 0x7fe784157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe78b10a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe78415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe78b10a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840cf550 con 0x7fe78406c2d0 2024-04-25T08:29:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe78b10a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78406d230 con 0x7fe7840b5620 2024-04-25T08:29:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe78b10a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840cf6d0 con 0x7fe78415b7d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 0x7fe7841f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 0x7fe7841f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49286/0 (socket says 172.21.15.60:49286) 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 -- 172.21.15.60:0/2721296319 learned_addr learned my addr 172.21.15.60:0/2721296319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe7896a7640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe78415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe783fff640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840b5620 0x7fe784157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 -- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 msgr2=0x7fe78415fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe78415fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 -- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840b5620 msgr2=0x7fe784157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840b5620 0x7fe784157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 -- 172.21.15.60:0/2721296319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784158300 con 0x7fe78406c2d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe788ea6640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 0x7fe7841f76e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe77802a1c0 tx=0x7fe7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778041020 con 0x7fe78406c2d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77803d620 con 0x7fe78406c2d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe778037040 con 0x7fe78406c2d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7841585d0 con 0x7fe78406c2d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.153+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe784158b90 con 0x7fe78406c2d0 2024-04-25T08:29:23.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.154+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe784073a00 con 0x7fe78406c2d0 2024-04-25T08:29:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.155+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe778044020 con 0x7fe78406c2d0 2024-04-25T08:29:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.155+0000 7fe7817fa640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe76806b4d0 0x7fe76806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.155+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe7780aa750 con 0x7fe78406c2d0 2024-04-25T08:29:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.155+0000 7fe783fff640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe76806b4d0 0x7fe76806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.156+0000 7fe783fff640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe76806b4d0 0x7fe76806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe770009560 tx=0x7fe770009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:23.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.160+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe77804b030 con 0x7fe78406c2d0 2024-04-25T08:29:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:23.377+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe7840ce1d0 con 0x7fe78406c2d0 2024-04-25T08:29:24.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.029+0000 7fe7817fa640 1 -- 172.21.15.60:0/2721296319 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe778078ff0 con 0x7fe78406c2d0 2024-04-25T08:29:24.028 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T08:29:24.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.044+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe76806b4d0 msgr2=0x7fe76806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.044+0000 7fe78b10a640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe76806b4d0 0x7fe76806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe770009560 tx=0x7fe770009190 comp rx=0 tx=0).stop 2024-04-25T08:29:24.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.044+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 msgr2=0x7fe7841f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.044+0000 7fe78b10a640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 0x7fe7841f76e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe77802a1c0 tx=0x7fe7780046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.047+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 shutdown_connections 2024-04-25T08:29:24.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.048+0000 7fe78b10a640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe78415b7d0 0x7fe78415fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.048+0000 7fe78b10a640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe76806b4d0 0x7fe76806d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.048+0000 7fe78b10a640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7840b5620 0x7fe784157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.048+0000 7fe78b10a640 1 --2- 172.21.15.60:0/2721296319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe78406c2d0 0x7fe7841f76e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.048+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 >> 172.21.15.60:0/2721296319 conn(0x7fe7840d0800 msgr2=0x7fe78406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:24.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.050+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 shutdown_connections 2024-04-25T08:29:24.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.050+0000 7fe78b10a640 1 -- 172.21.15.60:0/2721296319 wait complete. 2024-04-25T08:29:24.064 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T08:29:24.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T08:29:24.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.177+0000 7f6c968e1640 1 Processor -- start 2024-04-25T08:29:24.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.177+0000 7f6c968e1640 1 -- start start 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c968e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c900ced80 0x7f6c900d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c968e1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9015e000 con 0x7f6c90161710 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c968e1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c900d1de0 con 0x7f6c9015dc20 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c968e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c90161c60 con 0x7f6c900ced80 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c8f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c900ced80 0x7f6c900d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c94e7e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6c90161710 0x7f6c900d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38872/0 (socket says 172.21.15.60:38872) 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c94e7e640 1 -- 172.21.15.60:0/2722516099 learned_addr learned my addr 172.21.15.60:0/2722516099 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c8f7fe640 1 -- 172.21.15.60:0/2722516099 >> v1:172.21.15.100:6789/0 conn(0x7f6c9015dc20 legacy=0x7f6c900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c8f7fe640 1 -- 172.21.15.60:0/2722516099 >> v1:172.21.15.60:6789/0 conn(0x7f6c90161710 legacy=0x7f6c900d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c8f7fe640 1 -- 172.21.15.60:0/2722516099 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90161f40 con 0x7f6c900ced80 2024-04-25T08:29:24.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.178+0000 7f6c8effd640 1 -- 172.21.15.60:0/2722516099 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154521864 0 0) 0x7f6c9015e000 con 0x7f6c90161710 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c8f7fe640 1 --2- 172.21.15.60:0/2722516099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c900ced80 0x7f6c900d22e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097a0 tx=0x7f6c8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b078852faf00906f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c8effd640 1 -- 172.21.15.60:0/2722516099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84037070 con 0x7f6c900ced80 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c8effd640 1 -- 172.21.15.60:0/2722516099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c84032440 con 0x7f6c900ced80 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c8effd640 1 -- 172.21.15.60:0/2722516099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84032810 con 0x7f6c900ced80 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c968e1640 1 -- 172.21.15.60:0/2722516099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c900ced80 msgr2=0x7f6c900d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c968e1640 1 --2- 172.21.15.60:0/2722516099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c900ced80 0x7f6c900d22e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097a0 tx=0x7f6c8402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c968e1640 1 -- 172.21.15.60:0/2722516099 shutdown_connections 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c968e1640 1 --2- 172.21.15.60:0/2722516099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c900ced80 0x7f6c900d22e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.179+0000 7f6c968e1640 1 -- 172.21.15.60:0/2722516099 >> 172.21.15.60:0/2722516099 conn(0x7f6c900d0420 msgr2=0x7f6c900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.180+0000 7f6c968e1640 1 -- 172.21.15.60:0/2722516099 shutdown_connections 2024-04-25T08:29:24.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.180+0000 7f6c968e1640 1 -- 172.21.15.60:0/2722516099 wait complete. 2024-04-25T08:29:24.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.181+0000 7f6c968e1640 1 Processor -- start 2024-04-25T08:29:24.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.181+0000 7f6c968e1640 1 -- start start 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c900ced80 0x7f6c90154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9015dc20 0x7f6c900c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 0x7f6c9015f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900b5e50 con 0x7f6c90161710 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c90162500 con 0x7f6c9015dc20 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900b5fd0 con 0x7f6c900ced80 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 0x7f6c9015f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 0x7f6c9015f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49314/0 (socket says 172.21.15.60:49314) 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 -- 172.21.15.60:0/1309979541 learned_addr learned my addr 172.21.15.60:0/1309979541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c8f7fe640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9015dc20 0x7f6c900c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c8ffff640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c900ced80 0x7f6c90154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 -- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c900ced80 msgr2=0x7f6c90154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c900ced80 0x7f6c90154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 -- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9015dc20 msgr2=0x7f6c900c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9015dc20 0x7f6c900c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 -- 172.21.15.60:0/1309979541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c900c7c00 con 0x7f6c90161710 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c8ffff640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c900ced80 0x7f6c90154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c94e7e640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 0x7f6c9015f870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c800 tx=0x7f6c8800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88018070 con 0x7f6c90161710 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88004d80 con 0x7f6c90161710 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800a880 con 0x7f6c90161710 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c900c7ee0 con 0x7f6c90161710 2024-04-25T08:29:24.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.182+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c900c8710 con 0x7f6c90161710 2024-04-25T08:29:24.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.183+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c90073a00 con 0x7f6c90161710 2024-04-25T08:29:24.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.184+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c8801b070 con 0x7f6c90161710 2024-04-25T08:29:24.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.184+0000 7f6c8cff9640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c7c06b500 0x7f6c7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:24.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.184+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c88089630 con 0x7f6c90161710 2024-04-25T08:29:24.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.184+0000 7f6c8ffff640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c7c06b500 0x7f6c7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:24.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.185+0000 7f6c8ffff640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c7c06b500 0x7f6c7c06d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c80007fd0 tx=0x7f6c80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:24.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.189+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c88057fb0 con 0x7f6c90161710 2024-04-25T08:29:24.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:24.376+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f6c9015b590 con 0x7f6c90161710 2024-04-25T08:29:25.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.030+0000 7f6c8cff9640 1 -- 172.21.15.60:0/1309979541 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f6c8805c050 con 0x7f6c90161710 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c7c06b500 msgr2=0x7f6c7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c7c06b500 0x7f6c7c06d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c80007fd0 tx=0x7f6c80009040 comp rx=0 tx=0).stop 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 msgr2=0x7f6c9015f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 0x7f6c9015f870 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c800 tx=0x7f6c8800ccc0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 shutdown_connections 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c90161710 0x7f6c9015f870 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c7c06b500 0x7f6c7c06d9c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9015dc20 0x7f6c900c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 --2- 172.21.15.60:0/1309979541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c900ced80 0x7f6c90154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.033+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 >> 172.21.15.60:0/1309979541 conn(0x7f6c900d0420 msgr2=0x7f6c900d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:25.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.034+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 shutdown_connections 2024-04-25T08:29:25.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.034+0000 7f6c968e1640 1 -- 172.21.15.60:0/1309979541 wait complete. 2024-04-25T08:29:25.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:29:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.163+0000 7f653fb7e640 1 Processor -- start 2024-04-25T08:29:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.163+0000 7f653fb7e640 1 -- start start 2024-04-25T08:29:25.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653fb7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653fb7e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653806cda0 con 0x7f6538161710 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653fb7e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653815e000 con 0x7f65380d32f0 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653fb7e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65380d1de0 con 0x7f653815dc20 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653d91a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653d119640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6538161710 0x7f653806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38880/0 (socket says 172.21.15.60:38880) 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653d119640 1 -- 172.21.15.60:0/2062351446 learned_addr learned my addr 172.21.15.60:0/2062351446 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.164+0000 7f653c918640 1 -- 172.21.15.60:0/2062351446 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341800607 0 0) 0x7f653806cda0 con 0x7f6538161710 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.165+0000 7f653c918640 1 -- 172.21.15.60:0/2062351446 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6520003520 con 0x7f6538161710 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.165+0000 7f653d91a640 1 -- 172.21.15.60:0/2062351446 >> v1:172.21.15.100:6789/0 conn(0x7f65380d32f0 legacy=0x7f653806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.165+0000 7f653d91a640 1 -- 172.21.15.60:0/2062351446 >> v1:172.21.15.60:6789/0 conn(0x7f6538161710 legacy=0x7f653806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.165+0000 7f653d91a640 1 -- 172.21.15.60:0/2062351446 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538161c60 con 0x7f653815dc20 2024-04-25T08:29:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.165+0000 7f653d91a640 1 --2- 172.21.15.60:0/2062351446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653815dc20 0x7f65380ce670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f65300097a0 tx=0x7f653002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e0bb582ad8df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.165+0000 7f653c918640 1 -- 172.21.15.60:0/2062351446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530037070 con 0x7f653815dc20 2024-04-25T08:29:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653c918640 1 -- 172.21.15.60:0/2062351446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530032440 con 0x7f653815dc20 2024-04-25T08:29:25.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653c918640 1 -- 172.21.15.60:0/2062351446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530032810 con 0x7f653815dc20 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 -- 172.21.15.60:0/2062351446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653815dc20 msgr2=0x7f65380ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 --2- 172.21.15.60:0/2062351446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653815dc20 0x7f65380ce670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f65300097a0 tx=0x7f653002a230 comp rx=0 tx=0).stop 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 -- 172.21.15.60:0/2062351446 shutdown_connections 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 --2- 172.21.15.60:0/2062351446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 -- 172.21.15.60:0/2062351446 >> 172.21.15.60:0/2062351446 conn(0x7f65380d0420 msgr2=0x7f65380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 -- 172.21.15.60:0/2062351446 shutdown_connections 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.166+0000 7f653fb7e640 1 -- 172.21.15.60:0/2062351446 wait complete. 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.167+0000 7f653fb7e640 1 Processor -- start 2024-04-25T08:29:25.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.167+0000 7f653fb7e640 1 -- start start 2024-04-25T08:29:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653fb7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 0x7f65380cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653fb7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f653815dc20 0x7f653814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653fb7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6538161710 0x7f653814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653fb7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6538162500 con 0x7f65380d32f0 2024-04-25T08:29:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653fb7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6538162220 con 0x7f6538161710 2024-04-25T08:29:25.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653fb7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380b5e50 con 0x7f653815dc20 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653d91a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 0x7f65380cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653d91a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 0x7f65380cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49332/0 (socket says 172.21.15.60:49332) 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653d91a640 1 -- 172.21.15.60:0/1122197922 learned_addr learned my addr 172.21.15.60:0/1122197922 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.168+0000 7f653e11b640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6538161710 0x7f653814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d119640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f653815dc20 0x7f653814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d91a640 1 -- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f653815dc20 msgr2=0x7f653814c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d91a640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f653815dc20 0x7f653814c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d91a640 1 -- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6538161710 msgr2=0x7f653814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d91a640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6538161710 0x7f653814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d91a640 1 -- 172.21.15.60:0/1122197922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380cb760 con 0x7f65380d32f0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653d91a640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 0x7f65380cc800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6530035040 tx=0x7f65300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530037040 con 0x7f65380d32f0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530030500 con 0x7f65380d32f0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65380cba40 con 0x7f65380d32f0 2024-04-25T08:29:25.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.169+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530030910 con 0x7f65380d32f0 2024-04-25T08:29:25.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.170+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65380cc000 con 0x7f65380d32f0 2024-04-25T08:29:25.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.171+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6530044020 con 0x7f65380d32f0 2024-04-25T08:29:25.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.171+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6538073a00 con 0x7f65380d32f0 2024-04-25T08:29:25.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.172+0000 7f65367fc640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652406b4d0 0x7f652406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:25.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.172+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f65300aa620 con 0x7f65380d32f0 2024-04-25T08:29:25.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.172+0000 7f653d119640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652406b4d0 0x7f652406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:25.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.173+0000 7f653d119640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652406b4d0 0x7f652406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f65380cdea0 tx=0x7f652c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:25.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.177+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65300792d0 con 0x7f65380d32f0 2024-04-25T08:29:25.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:29:25.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:29:25.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:25.362+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f65380d16b0 con 0x7f65380d32f0 2024-04-25T08:29:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.047+0000 7f65367fc640 1 -- 172.21.15.60:0/1122197922 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f6530078bd0 con 0x7f65380d32f0 2024-04-25T08:29:27.045 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' created 2024-04-25T08:29:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652406b4d0 msgr2=0x7f652406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652406b4d0 0x7f652406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f65380cdea0 tx=0x7f652c009040 comp rx=0 tx=0).stop 2024-04-25T08:29:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 msgr2=0x7f65380cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:27.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 0x7f65380cc800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6530035040 tx=0x7f65300046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 shutdown_connections 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652406b4d0 0x7f652406d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6538161710 0x7f653814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f653815dc20 0x7f653814c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 --2- 172.21.15.60:0/1122197922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65380d32f0 0x7f65380cc800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 >> 172.21.15.60:0/1122197922 conn(0x7f65380d0420 msgr2=0x7f65381601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 shutdown_connections 2024-04-25T08:29:27.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.050+0000 7f653fb7e640 1 -- 172.21.15.60:0/1122197922 wait complete. 2024-04-25T08:29:27.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T08:29:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.180+0000 7f5dd8934640 1 Processor -- start 2024-04-25T08:29:27.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.180+0000 7f5dd8934640 1 -- start start 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd8934640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5dd415dc20 0x7f5dd40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd8934640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5dd40d32f0 0x7f5dd40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd8934640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd415e000 con 0x7f5dd4161710 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd8934640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40d1de0 con 0x7f5dd415dc20 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd8934640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd4161c60 con 0x7f5dd40d32f0 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5dd415dc20 0x7f5dd40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5dd415dc20 0x7f5dd40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49334/0 (socket says 172.21.15.60:49334) 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 -- 172.21.15.60:0/1319726683 learned_addr learned my addr 172.21.15.60:0/1319726683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd2d9d640 1 --2- 172.21.15.60:0/1319726683 >> v2:172.21.15.100:3300/0 conn(0x7f5dd40d32f0 0x7f5dd40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 -- 172.21.15.60:0/1319726683 >> v2:172.21.15.100:3300/0 conn(0x7f5dd40d32f0 msgr2=0x7f5dd40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 --2- 172.21.15.60:0/1319726683 >> v2:172.21.15.100:3300/0 conn(0x7f5dd40d32f0 0x7f5dd40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 -- 172.21.15.60:0/1319726683 >> v1:172.21.15.100:6789/0 conn(0x7f5dd4161710 legacy=0x7f5dd406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:27.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd259c640 1 -- 172.21.15.60:0/1319726683 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4161f40 con 0x7f5dd415dc20 2024-04-25T08:29:27.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.181+0000 7f5dd2d9d640 1 --2- 172.21.15.60:0/1319726683 >> v2:172.21.15.100:3300/0 conn(0x7f5dd40d32f0 0x7f5dd40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.182+0000 7f5dd259c640 1 --2- 172.21.15.60:0/1319726683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd40ce670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40b6910 tx=0x7f5dc402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f98af3603e8832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.182+0000 7f5dd159a640 1 -- 172.21.15.60:0/1319726683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4037070 con 0x7f5dd415dc20 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.182+0000 7f5dd159a640 1 -- 172.21.15.60:0/1319726683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4032440 con 0x7f5dd415dc20 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.182+0000 7f5dd159a640 1 -- 172.21.15.60:0/1319726683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4032810 con 0x7f5dd415dc20 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 -- 172.21.15.60:0/1319726683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 msgr2=0x7f5dd40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 --2- 172.21.15.60:0/1319726683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd40ce670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40b6910 tx=0x7f5dc402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 -- 172.21.15.60:0/1319726683 shutdown_connections 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 --2- 172.21.15.60:0/1319726683 >> v2:172.21.15.100:3300/0 conn(0x7f5dd40d32f0 0x7f5dd40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 --2- 172.21.15.60:0/1319726683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd40ce670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 -- 172.21.15.60:0/1319726683 >> 172.21.15.60:0/1319726683 conn(0x7f5dd40d0420 msgr2=0x7f5dd40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 -- 172.21.15.60:0/1319726683 shutdown_connections 2024-04-25T08:29:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.183+0000 7f5dd8934640 1 -- 172.21.15.60:0/1319726683 wait complete. 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.184+0000 7f5dd8934640 1 Processor -- start 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.184+0000 7f5dd8934640 1 -- start start 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd8934640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dd40d32f0 0x7f5dd415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd8934640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd8934640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dd4161710 0x7f5dd415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd8934640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b5e50 con 0x7f5dd415dc20 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd8934640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4162500 con 0x7f5dd4161710 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd8934640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b5fd0 con 0x7f5dd40d32f0 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd415bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49338/0 (socket says 172.21.15.60:49338) 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd1d9b640 1 -- 172.21.15.60:0/3098247423 learned_addr learned my addr 172.21.15.60:0/3098247423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd2d9d640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dd4161710 0x7f5dd415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd259c640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dd40d32f0 0x7f5dd415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.185+0000 7f5dd1d9b640 1 -- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dd40d32f0 msgr2=0x7f5dd415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd1d9b640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dd40d32f0 0x7f5dd415e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd1d9b640 1 -- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dd4161710 msgr2=0x7f5dd415c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd1d9b640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dd4161710 0x7f5dd415c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd1d9b640 1 -- 172.21.15.60:0/3098247423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd415f080 con 0x7f5dd415dc20 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd1d9b640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd415bcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800b3f0 tx=0x7f5dc800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8015020 con 0x7f5dd415dc20 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc8011830 con 0x7f5dd415dc20 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd415f360 con 0x7f5dd415dc20 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011c20 con 0x7f5dd415dc20 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.186+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd41f3760 con 0x7f5dd415dc20 2024-04-25T08:29:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.187+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd4073a00 con 0x7f5dd415dc20 2024-04-25T08:29:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.187+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dc8018020 con 0x7f5dd415dc20 2024-04-25T08:29:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.188+0000 7f5dc2ffd640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.188+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5dc808a0f0 con 0x7f5dd415dc20 2024-04-25T08:29:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.188+0000 7f5dd259c640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.189+0000 7f5dd259c640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5db406b4d0 0x7f5db406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4034000 tx=0x7f5dc402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:27.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.192+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5dc801f030 con 0x7f5dd415dc20 2024-04-25T08:29:27.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:27.373+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f5dd40d16b0 con 0x7f5dd415dc20 2024-04-25T08:29:28.009 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-25T08:29:28.011+0000 7fdd51f8f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:29:28.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.048+0000 7f5dc2ffd640 1 -- 172.21.15.60:0/3098247423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5dc80172c0 con 0x7f5dd415dc20 2024-04-25T08:29:28.047 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T08:29:28.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5db406b4d0 msgr2=0x7f5db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5db406b4d0 0x7f5db406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4034000 tx=0x7f5dc402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:28.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 msgr2=0x7f5dd415bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd415bcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800b3f0 tx=0x7f5dc800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 shutdown_connections 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dd4161710 0x7f5dd415c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5dd415dc20 0x7f5dd415bcb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 --2- 172.21.15.60:0/3098247423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5dd40d32f0 0x7f5dd415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 >> 172.21.15.60:0/3098247423 conn(0x7f5dd40d0420 msgr2=0x7f5dd40b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 shutdown_connections 2024-04-25T08:29:28.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.053+0000 7f5dd8934640 1 -- 172.21.15.60:0/3098247423 wait complete. 2024-04-25T08:29:28.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T08:29:28.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.193+0000 7f1769589640 1 Processor -- start 2024-04-25T08:29:28.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.193+0000 7f1769589640 1 -- start start 2024-04-25T08:29:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f1769589640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f17640d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f1769589640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176415e000 con 0x7f17640ced80 2024-04-25T08:29:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f1769589640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17640d1de0 con 0x7f176415dc20 2024-04-25T08:29:28.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f1769589640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1764161c60 con 0x7f1764161710 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f175bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f17640ced80 0x7f17640d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38894/0 (socket says 172.21.15.60:38894) 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f175bfff640 1 -- 172.21.15.60:0/3036691466 learned_addr learned my addr 172.21.15.60:0/3036691466 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17637fe640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f17640d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310821319 0 0) 0x7f176415e000 con 0x7f17640ced80 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1750003520 con 0x7f17640ced80 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812746427 0 0) 0x7f17640d1de0 con 0x7f176415dc20 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f176415e000 con 0x7f176415dc20 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401340253 0 0) 0x7f1750003520 con 0x7f17640ced80 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 msgr2=0x7f17640d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f17640d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 >> v1:172.21.15.100:6789/0 conn(0x7f176415dc20 legacy=0x7f17640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764161f40 con 0x7f17640ced80 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.194+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f174c001670 con 0x7f17640ced80 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 >> v1:172.21.15.60:6789/0 conn(0x7f17640ced80 legacy=0x7f17640d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1750003b70 0x7f1750003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f1750004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1750004d40 0x7f17500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764161c60 con 0x7f1750004d40 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175000ba70 con 0x7f1750003b70 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175000bc20 con 0x7f1764161710 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f1762ffd640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1750003b70 0x7f1750003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f175bfff640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f1750004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17637fe640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1750004d40 0x7f17500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f175bfff640 1 -- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1750003b70 msgr2=0x7f1750003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f175bfff640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1750003b70 0x7f1750003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f175bfff640 1 -- 172.21.15.60:0/3036691466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1750004d40 msgr2=0x7f17500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f175bfff640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1750004d40 0x7f17500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f175bfff640 1 -- 172.21.15.60:0/3036691466 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750003520 con 0x7f1764161710 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f1762ffd640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1750003b70 0x7f1750003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.195+0000 7f17637fe640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1750004d40 0x7f17500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.196+0000 7f175bfff640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f1750004800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f174c00c160 tx=0x7f1750005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b18c8c683e900eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.196+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c007ca0 con 0x7f1764161710 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.196+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c005af0 con 0x7f1764161710 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.196+0000 7f17627fc640 1 -- 172.21.15.60:0/3036691466 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c007070 con 0x7f1764161710 2024-04-25T08:29:28.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.196+0000 7f1769589640 1 -- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 msgr2=0x7f1750004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.196+0000 7f1769589640 1 --2- 172.21.15.60:0/3036691466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1764161710 0x7f1750004800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f174c00c160 tx=0x7f1750005830 comp rx=0 tx=0).stop 2024-04-25T08:29:28.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.197+0000 7f17637fe640 1 -- 172.21.15.60:0/3036691466 reap_dead start 2024-04-25T08:29:28.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.197+0000 7f1769589640 1 -- 172.21.15.60:0/3036691466 shutdown_connections 2024-04-25T08:29:28.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.197+0000 7f1769589640 1 -- 172.21.15.60:0/3036691466 >> 172.21.15.60:0/3036691466 conn(0x7f17640d0420 msgr2=0x7f17640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:28.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.197+0000 7f1769589640 1 -- 172.21.15.60:0/3036691466 shutdown_connections 2024-04-25T08:29:28.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.197+0000 7f1769589640 1 -- 172.21.15.60:0/3036691466 wait complete. 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.198+0000 7f1769589640 1 Processor -- start 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.198+0000 7f1769589640 1 -- start start 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.198+0000 7f1769589640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1764161710 0x7f17640d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.198+0000 7f1769589640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 0x7f17640c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.198+0000 7f1769589640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f176414cd10 0x7f176414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f1769589640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b5e50 con 0x7f17640c9080 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f1769589640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764162500 con 0x7f1764161710 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f1769589640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b5fd0 con 0x7f176414cd10 2024-04-25T08:29:28.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 0x7f17640c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 0x7f17640c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49360/0 (socket says 172.21.15.60:49360) 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 -- 172.21.15.60:0/514876292 learned_addr learned my addr 172.21.15.60:0/514876292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f1762ffd640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1764161710 0x7f17640d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f17637fe640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f176414cd10 0x7f176414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 -- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f176414cd10 msgr2=0x7f176414d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f176414cd10 0x7f176414d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 -- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1764161710 msgr2=0x7f17640d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1764161710 0x7f17640d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 -- 172.21.15.60:0/514876292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176414da80 con 0x7f17640c9080 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f17637fe640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f176414cd10 0x7f176414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175bfff640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 0x7f17640c94e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f174c02b010 tx=0x7f1764162110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c041020 con 0x7f17640c9080 2024-04-25T08:29:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c001470 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176414aba0 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c007070 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.199+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176414b160 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.200+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764073a00 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.201+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f174c044020 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.201+0000 7f175b7fe640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f174406b4d0 0x7f174406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.201+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f174c0aa970 con 0x7f17640c9080 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.201+0000 7f1762ffd640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f174406b4d0 0x7f174406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.202+0000 7f1762ffd640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f174406b4d0 0x7f174406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1754005410 tx=0x7f1754009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.206+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f174c079560 con 0x7f17640c9080 2024-04-25T08:29:28.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:28.428+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f17640d16b0 con 0x7f17640c9080 2024-04-25T08:29:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.070+0000 7f175b7fe640 1 -- 172.21.15.60:0/514876292 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f174c078e60 con 0x7f17640c9080 2024-04-25T08:29:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:added data pool 4 to fsmap 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.074+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f174406b4d0 msgr2=0x7f174406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.074+0000 7f1769589640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f174406b4d0 0x7f174406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1754005410 tx=0x7f1754009040 comp rx=0 tx=0).stop 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.074+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 msgr2=0x7f17640c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.074+0000 7f1769589640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 0x7f17640c94e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f174c02b010 tx=0x7f1764162110 comp rx=0 tx=0).stop 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 shutdown_connections 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f176414cd10 0x7f176414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17640c9080 0x7f17640c94e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f174406b4d0 0x7f174406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 --2- 172.21.15.60:0/514876292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1764161710 0x7f17640d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 >> 172.21.15.60:0/514876292 conn(0x7f17640d0420 msgr2=0x7f17640c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:29.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 shutdown_connections 2024-04-25T08:29:29.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.075+0000 7f1769589640 1 -- 172.21.15.60:0/514876292 wait complete. 2024-04-25T08:29:29.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:29.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.195+0000 7fc3ceb42640 1 Processor -- start 2024-04-25T08:29:29.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.195+0000 7fc3ceb42640 1 -- start start 2024-04-25T08:29:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3ceb42640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3c80b2cb0 0x7fc3c80d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3ceb42640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3c80d1020 0x7fc3c80d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3ceb42640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c80b97a0 con 0x7fc3c815dbe0 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3ceb42640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c815dfc0 con 0x7fc3c80d1020 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3ceb42640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c80d1a00 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cd0df640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc3c80d1020 0x7fc3c80d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cc8de640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc3c815dbe0 0x7fc3c80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38902/0 (socket says 172.21.15.60:38902) 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cc8de640 1 -- 172.21.15.60:0/4145346732 learned_addr learned my addr 172.21.15.60:0/4145346732 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3c7fff640 1 --2- 172.21.15.60:0/4145346732 >> v2:172.21.15.100:3300/0 conn(0x7fc3c80b2cb0 0x7fc3c80d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cd0df640 1 -- 172.21.15.60:0/4145346732 >> v2:172.21.15.100:3300/0 conn(0x7fc3c80b2cb0 msgr2=0x7fc3c80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cd0df640 1 --2- 172.21.15.60:0/4145346732 >> v2:172.21.15.100:3300/0 conn(0x7fc3c80b2cb0 0x7fc3c80d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3c77fe640 1 -- 172.21.15.60:0/4145346732 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558640909 0 0) 0x7fc3c80b97a0 con 0x7fc3c815dbe0 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cd0df640 1 -- 172.21.15.60:0/4145346732 >> v1:172.21.15.60:6789/0 conn(0x7fc3c815dbe0 legacy=0x7fc3c80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3cd0df640 1 -- 172.21.15.60:0/4145346732 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80d1c50 con 0x7fc3c80d1020 2024-04-25T08:29:29.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.196+0000 7fc3c7fff640 1 --2- 172.21.15.60:0/4145346732 >> v2:172.21.15.100:3300/0 conn(0x7fc3c80b2cb0 0x7fc3c80d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.197+0000 7fc3cd0df640 1 --2- 172.21.15.60:0/4145346732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d1020 0x7fc3c80d2c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3c806cff0 tx=0x7fc3c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ee8e5489cc12ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.197+0000 7fc3c77fe640 1 -- 172.21.15.60:0/4145346732 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0007740 con 0x7fc3c80d1020 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.197+0000 7fc3c77fe640 1 -- 172.21.15.60:0/4145346732 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c00078e0 con 0x7fc3c80d1020 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.197+0000 7fc3c77fe640 1 -- 172.21.15.60:0/4145346732 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0007cb0 con 0x7fc3c80d1020 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 -- 172.21.15.60:0/4145346732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d1020 msgr2=0x7fc3c80d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/4145346732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d1020 0x7fc3c80d2c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3c806cff0 tx=0x7fc3c002a680 comp rx=0 tx=0).stop 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 -- 172.21.15.60:0/4145346732 shutdown_connections 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/4145346732 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80d1020 0x7fc3c80d2c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/4145346732 >> v2:172.21.15.100:3300/0 conn(0x7fc3c80b2cb0 0x7fc3c80d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 -- 172.21.15.60:0/4145346732 >> 172.21.15.60:0/4145346732 conn(0x7fc3c8061a20 msgr2=0x7fc3c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 -- 172.21.15.60:0/4145346732 shutdown_connections 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.198+0000 7fc3ceb42640 1 -- 172.21.15.60:0/4145346732 wait complete. 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.199+0000 7fc3ceb42640 1 Processor -- start 2024-04-25T08:29:29.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.199+0000 7fc3ceb42640 1 -- start start 2024-04-25T08:29:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3ceb42640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 0x7fc3c80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3ceb42640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3c80d1020 0x7fc3c80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3ceb42640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815dbe0 0x7fc3c80b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3ceb42640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80b92a0 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3ceb42640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80d2210 con 0x7fc3c815dbe0 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3ceb42640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c80b9420 con 0x7fc3c80d1020 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 0x7fc3c80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 0x7fc3c80c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49376/0 (socket says 172.21.15.60:49376) 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 -- 172.21.15.60:0/842103225 learned_addr learned my addr 172.21.15.60:0/842103225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cd0df640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815dbe0 0x7fc3c80b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3c7fff640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3c80d1020 0x7fc3c80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 -- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3c80d1020 msgr2=0x7fc3c80b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3c80d1020 0x7fc3c80b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 -- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815dbe0 msgr2=0x7fc3c80b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815dbe0 0x7fc3c80b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 -- 172.21.15.60:0/842103225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80cbb60 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3c7fff640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3c80d1020 0x7fc3c80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.200+0000 7fc3cc8de640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 0x7fc3c80c9bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c430 tx=0x7fc3bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.201+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0074e0 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.201+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc0047e0 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.201+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc004bd0 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.201+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c80cbe40 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.201+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c80ca0f0 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.202+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c8073990 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.202+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3bc011070 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.203+0000 7fc3c57fa640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.203+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc3bc08a090 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.203+0000 7fc3c7fff640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.203+0000 7fc3c7fff640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3c814bb60 tx=0x7fc3b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.207+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3bc058c60 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.424+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3c815aaf0 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.425+0000 7fc3c57fa640 1 -- 172.21.15.60:0/842103225 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4937 (secure 0 0 0) 0x7fc3bc058560 con 0x7fc3c80b2cb0 2024-04-25T08:29:29.423 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:29.423 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:28.429882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:29:29.423 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2024-04-25T08:29:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc3ac06b4d0 msgr2=0x7fc3ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3c814bb60 tx=0x7fc3b4008040 comp rx=0 tx=0).stop 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 msgr2=0x7fc3c80c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 0x7fc3c80c9bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00c430 tx=0x7fc3bc00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 shutdown_connections 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc3ac06b4d0 0x7fc3ac06d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3c815dbe0 0x7fc3c80b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc3c80d1020 0x7fc3c80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 --2- 172.21.15.60:0/842103225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc3c80b2cb0 0x7fc3c80c9bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 >> 172.21.15.60:0/842103225 conn(0x7fc3c8061a20 msgr2=0x7fc3c8154ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 shutdown_connections 2024-04-25T08:29:29.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.427+0000 7fc3ceb42640 1 -- 172.21.15.60:0/842103225 wait complete. 2024-04-25T08:29:29.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.538+0000 7fb01ed3b640 1 Processor -- start 2024-04-25T08:29:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.538+0000 7fb01ed3b640 1 -- start start 2024-04-25T08:29:29.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01ed3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01ed3b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0180b92a0 0x7fb0180d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01ed3b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0180b97a0 con 0x7fb0180b2cb0 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01ed3b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01815dfc0 con 0x7fb0180b92a0 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01ed3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180d1a00 con 0x7fb01815dbe0 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01d2d8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0180b92a0 0x7fb0180d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb017fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb0180b2cb0 0x7fb0180b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38918/0 (socket says 172.21.15.60:38918) 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01d2d8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb0180b92a0 0x7fb0180d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49378/0 (socket says 172.21.15.60:49378) 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb017fff640 1 -- 172.21.15.60:0/61020013 learned_addr learned my addr 172.21.15.60:0/61020013 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01cad7640 1 --2- 172.21.15.60:0/61020013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01d2d8640 1 -- 172.21.15.60:0/61020013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 msgr2=0x7fb0180d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.539+0000 7fb01d2d8640 1 --2- 172.21.15.60:0/61020013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.540+0000 7fb01d2d8640 1 -- 172.21.15.60:0/61020013 >> v1:172.21.15.60:6789/0 conn(0x7fb0180b2cb0 legacy=0x7fb0180b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.540+0000 7fb01d2d8640 1 -- 172.21.15.60:0/61020013 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180d1c50 con 0x7fb0180b92a0 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.540+0000 7fb01d2d8640 1 --2- 172.21.15.60:0/61020013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb0180d0e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb010009d90 tx=0x7fb01002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5df96f3dce2f438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.540+0000 7fb0177fe640 1 -- 172.21.15.60:0/61020013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010007740 con 0x7fb0180b92a0 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.540+0000 7fb0177fe640 1 -- 172.21.15.60:0/61020013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0100078e0 con 0x7fb0180b92a0 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.540+0000 7fb0177fe640 1 -- 172.21.15.60:0/61020013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010007cb0 con 0x7fb0180b92a0 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 -- 172.21.15.60:0/61020013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 msgr2=0x7fb0180d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/61020013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb0180d0e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb010009d90 tx=0x7fb01002a660 comp rx=0 tx=0).stop 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 -- 172.21.15.60:0/61020013 shutdown_connections 2024-04-25T08:29:29.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/61020013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb0180d0e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/61020013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 -- 172.21.15.60:0/61020013 >> 172.21.15.60:0/61020013 conn(0x7fb018061a20 msgr2=0x7fb018061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 -- 172.21.15.60:0/61020013 shutdown_connections 2024-04-25T08:29:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.541+0000 7fb01ed3b640 1 -- 172.21.15.60:0/61020013 wait complete. 2024-04-25T08:29:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.542+0000 7fb01ed3b640 1 Processor -- start 2024-04-25T08:29:29.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.542+0000 7fb01ed3b640 1 -- start start 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01ed3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0180b2cb0 0x7fb0180caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01ed3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb018151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01ed3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01ed3b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180b6ab0 con 0x7fb0180b92a0 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01ed3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180d2210 con 0x7fb0180b2cb0 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01ed3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180b6c30 con 0x7fb01815dbe0 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb018151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb018151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49384/0 (socket says 172.21.15.60:49384) 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 -- 172.21.15.60:0/2205802636 learned_addr learned my addr 172.21.15.60:0/2205802636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01cad7640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0180b2cb0 0x7fb0180caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01d2d8640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 -- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 msgr2=0x7fb0180b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 -- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0180b2cb0 msgr2=0x7fb0180caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0180b2cb0 0x7fb0180caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb017fff640 1 -- 172.21.15.60:0/2205802636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180b3e40 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.543+0000 7fb01d2d8640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.544+0000 7fb017fff640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb018151510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c430 tx=0x7fb00c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.544+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c017070 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.544+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c005ce0 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.544+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0180cc3b0 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.544+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0072f0 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.544+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0180b43c0 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.545+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018073990 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.545+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb00c01a070 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.546+0000 7fb0157fa640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb00406b500 0x7fb00406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.546+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb00c0898b0 con 0x7fb0180b92a0 2024-04-25T08:29:29.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.546+0000 7fb01cad7640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb00406b500 0x7fb00406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.546+0000 7fb01cad7640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb00406b500 0x7fb00406d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb0080079a0 tx=0x7fb008008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.550+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb00c057da0 con 0x7fb0180b92a0 2024-04-25T08:29:29.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.734+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb01815c9d0 con 0x7fb0180b92a0 2024-04-25T08:29:29.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.735+0000 7fb0157fa640 1 -- 172.21.15.60:0/2205802636 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fb00c05be40 con 0x7fb0180b92a0 2024-04-25T08:29:29.733 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:29.734 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.737+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb00406b500 msgr2=0x7fb00406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.737+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb00406b500 0x7fb00406d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb0080079a0 tx=0x7fb008008040 comp rx=0 tx=0).stop 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.737+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 msgr2=0x7fb018151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.737+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb018151510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00c430 tx=0x7fb00c00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 shutdown_connections 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb01815dbe0 0x7fb0180b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0180b92a0 0x7fb018151510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb00406b500 0x7fb00406d9c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 --2- 172.21.15.60:0/2205802636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0180b2cb0 0x7fb0180caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 >> 172.21.15.60:0/2205802636 conn(0x7fb018061a20 msgr2=0x7fb0180b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:29.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 shutdown_connections 2024-04-25T08:29:29.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.738+0000 7fb01ed3b640 1 -- 172.21.15.60:0/2205802636 wait complete. 2024-04-25T08:29:29.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.849+0000 7f783e2bd640 1 Processor -- start 2024-04-25T08:29:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.849+0000 7f783e2bd640 1 -- start start 2024-04-25T08:29:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f783e2bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78380c6df0 0x7f78380d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f783e2bd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783815dfe0 con 0x7f783815dc00 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f783e2bd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78380d00c0 con 0x7f78380d1c00 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f783e2bd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380d2070 con 0x7f78380c6df0 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f78377fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78380c6df0 0x7f78380d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f7837fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f783815dc00 0x7f78380ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38920/0 (socket says 172.21.15.60:38920) 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f7837fff640 1 -- 172.21.15.60:0/2393018088 learned_addr learned my addr 172.21.15.60:0/2393018088 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f78377fe640 1 -- 172.21.15.60:0/2393018088 >> v1:172.21.15.100:6789/0 conn(0x7f78380d1c00 legacy=0x7f78380ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f78377fe640 1 -- 172.21.15.60:0/2393018088 >> v1:172.21.15.60:6789/0 conn(0x7f783815dc00 legacy=0x7f78380ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.850+0000 7f78377fe640 1 -- 172.21.15.60:0/2393018088 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380d03a0 con 0x7f78380c6df0 2024-04-25T08:29:29.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.851+0000 7f78377fe640 1 --2- 172.21.15.60:0/2393018088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78380c6df0 0x7f78380d16c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78280097a0 tx=0x7f782802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b05bede40562a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.851+0000 7f7836ffd640 1 -- 172.21.15.60:0/2393018088 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828037070 con 0x7f78380c6df0 2024-04-25T08:29:29.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.851+0000 7f7836ffd640 1 -- 172.21.15.60:0/2393018088 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7828032440 con 0x7f78380c6df0 2024-04-25T08:29:29.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.851+0000 7f7836ffd640 1 -- 172.21.15.60:0/2393018088 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828032810 con 0x7f78380c6df0 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 -- 172.21.15.60:0/2393018088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78380c6df0 msgr2=0x7f78380d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 --2- 172.21.15.60:0/2393018088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78380c6df0 0x7f78380d16c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78280097a0 tx=0x7f782802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 -- 172.21.15.60:0/2393018088 shutdown_connections 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 --2- 172.21.15.60:0/2393018088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78380c6df0 0x7f78380d16c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 -- 172.21.15.60:0/2393018088 >> 172.21.15.60:0/2393018088 conn(0x7f783806f5d0 msgr2=0x7f783806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 -- 172.21.15.60:0/2393018088 shutdown_connections 2024-04-25T08:29:29.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.852+0000 7f783e2bd640 1 -- 172.21.15.60:0/2393018088 wait complete. 2024-04-25T08:29:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.853+0000 7f783e2bd640 1 Processor -- start 2024-04-25T08:29:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.853+0000 7f783e2bd640 1 -- start start 2024-04-25T08:29:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783e2bd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78380c6df0 0x7f78381e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783e2bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78380d1c00 0x7f78381e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783e2bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 0x7f78381483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783e2bd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cc130 con 0x7f783815dc00 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783e2bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cc720 con 0x7f78380d1c00 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783e2bd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cfd90 con 0x7f78380c6df0 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 0x7f78381483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 0x7f78381483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49406/0 (socket says 172.21.15.60:49406) 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 -- 172.21.15.60:0/145390119 learned_addr learned my addr 172.21.15.60:0/145390119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f78377fe640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78380d1c00 0x7f78381e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f7837fff640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78380c6df0 0x7f78381e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 -- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78380c6df0 msgr2=0x7f78381e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78380c6df0 0x7f78381e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 -- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78380d1c00 msgr2=0x7f78381e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78380d1c00 0x7f78381e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 -- 172.21.15.60:0/145390119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78381e7b60 con 0x7f783815dc00 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f783c85a640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 0x7f78381483b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f783000c780 tx=0x7f783000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.854+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830004b60 con 0x7f783815dc00 2024-04-25T08:29:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.855+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830004d00 con 0x7f783815dc00 2024-04-25T08:29:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.855+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78381e7e40 con 0x7f783815dc00 2024-04-25T08:29:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.855+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000aa90 con 0x7f783815dc00 2024-04-25T08:29:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.855+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78381e8400 con 0x7f783815dc00 2024-04-25T08:29:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.856+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7838073990 con 0x7f783815dc00 2024-04-25T08:29:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.856+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f783000ac30 con 0x7f783815dc00 2024-04-25T08:29:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.857+0000 7f7834ff9640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f781806b500 0x7f781806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.857+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f783008a7f0 con 0x7f783815dc00 2024-04-25T08:29:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.857+0000 7f7837fff640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f781806b500 0x7f781806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.857+0000 7f7837fff640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f781806b500 0x7f781806d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78380b1c00 tx=0x7f7820009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:29.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:29.861+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7830020070 con 0x7f783815dc00 2024-04-25T08:29:30.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.044+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78380b2fd0 con 0x7f783815dc00 2024-04-25T08:29:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.045+0000 7f7834ff9640 1 -- 172.21.15.60:0/145390119 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f7830059330 con 0x7f783815dc00 2024-04-25T08:29:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:30.044 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f781806b500 msgr2=0x7f781806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f781806b500 0x7f781806d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f78380b1c00 tx=0x7f7820009040 comp rx=0 tx=0).stop 2024-04-25T08:29:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 msgr2=0x7f78381483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 0x7f78381483b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f783000c780 tx=0x7f783000cc40 comp rx=0 tx=0).stop 2024-04-25T08:29:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 shutdown_connections 2024-04-25T08:29:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f783815dc00 0x7f78381483b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f781806b500 0x7f781806d9c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78380d1c00 0x7f78381e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 --2- 172.21.15.60:0/145390119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78380c6df0 0x7f78381e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 >> 172.21.15.60:0/145390119 conn(0x7f783806f5d0 msgr2=0x7f7838157110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:30.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 shutdown_connections 2024-04-25T08:29:30.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.047+0000 7f783e2bd640 1 -- 172.21.15.60:0/145390119 wait complete. 2024-04-25T08:29:30.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:30.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.176+0000 7f3133190640 1 Processor -- start 2024-04-25T08:29:30.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.176+0000 7f3133190640 1 -- start start 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.177+0000 7f3133190640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f312c0cf780 0x7f312c15c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.177+0000 7f3133190640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c0cfe10 con 0x7f312c06abe0 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.177+0000 7f3133190640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c0d00f0 con 0x7f312c0cc1d0 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.177+0000 7f3133190640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0ce6e0 con 0x7f312c0cf780 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.177+0000 7f3130f2c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f312c06abe0 0x7f312c06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38922/0 (socket says 172.21.15.60:38922) 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.177+0000 7f3130f2c640 1 -- 172.21.15.60:0/3187246735 learned_addr learned my addr 172.21.15.60:0/3187246735 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782881070 0 0) 0x7f312c0cfe10 con 0x7f312c06abe0 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118003520 con 0x7f312c06abe0 2024-04-25T08:29:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312bfff640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f312c0cf780 0x7f312c15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840991481 0 0) 0x7f3118003520 con 0x7f312c06abe0 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f312c0cf780 msgr2=0x7f312c15c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f312c0cf780 0x7f312c15c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 >> v1:172.21.15.100:6789/0 conn(0x7f312c0cc1d0 legacy=0x7f312c0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0d03d0 con 0x7f312c06abe0 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3124001670 con 0x7f312c06abe0 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 >> v1:172.21.15.60:6789/0 conn(0x7f312c06abe0 legacy=0x7f312c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31180039e0 0x7f3118003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3118004800 0x7f3118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31180051c0 0x7f311800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0ce6e0 con 0x7f31180039e0 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311800c090 con 0x7f31180051c0 2024-04-25T08:29:30.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311800c290 con 0x7f3118004800 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312bfff640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f312c0cf780 0x7f312c15c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f3130f2c640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31180039e0 0x7f3118003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f313172d640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31180051c0 0x7f311800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f312bfff640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3118004800 0x7f3118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f3130f2c640 1 -- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3118004800 msgr2=0x7f3118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f3130f2c640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3118004800 0x7f3118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f3130f2c640 1 -- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31180051c0 msgr2=0x7f311800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f3130f2c640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31180051c0 0x7f311800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.178+0000 7f3130f2c640 1 -- 172.21.15.60:0/3187246735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118003520 con 0x7f31180039e0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f3130f2c640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31180039e0 0x7f3118003e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f312400c240 tx=0x7f3118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964e83021fbe4a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124007d10 con 0x7f31180039e0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124005ad0 con 0x7f31180039e0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f312b7fe640 1 -- 172.21.15.60:0/3187246735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124007070 con 0x7f31180039e0 2024-04-25T08:29:30.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f3133190640 1 -- 172.21.15.60:0/3187246735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31180039e0 msgr2=0x7f3118003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f3133190640 1 --2- 172.21.15.60:0/3187246735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31180039e0 0x7f3118003e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f312400c240 tx=0x7f3118005830 comp rx=0 tx=0).stop 2024-04-25T08:29:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.179+0000 7f313172d640 1 -- 172.21.15.60:0/3187246735 reap_dead start 2024-04-25T08:29:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.180+0000 7f3133190640 1 -- 172.21.15.60:0/3187246735 shutdown_connections 2024-04-25T08:29:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.180+0000 7f3133190640 1 -- 172.21.15.60:0/3187246735 >> 172.21.15.60:0/3187246735 conn(0x7f312c0b2290 msgr2=0x7f312c0b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.180+0000 7f3133190640 1 -- 172.21.15.60:0/3187246735 shutdown_connections 2024-04-25T08:29:30.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.180+0000 7f3133190640 1 -- 172.21.15.60:0/3187246735 wait complete. 2024-04-25T08:29:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.181+0000 7f3133190640 1 Processor -- start 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.181+0000 7f3133190640 1 -- start start 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31180039e0 0x7f312c14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 0x7f312c0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312c15de80 0x7f312c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0cf280 con 0x7f312c070000 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0cfbc0 con 0x7f312c15de80 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0cf400 con 0x7f31180039e0 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 0x7f312c0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 0x7f312c0b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49428/0 (socket says 172.21.15.60:49428) 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 -- 172.21.15.60:0/2249443644 learned_addr learned my addr 172.21.15.60:0/2249443644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f313172d640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312c15de80 0x7f312c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3130f2c640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31180039e0 0x7f312c14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 -- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31180039e0 msgr2=0x7f312c14fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31180039e0 0x7f312c14fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 -- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312c15de80 msgr2=0x7f312c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312c15de80 0x7f312c15e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 -- 172.21.15.60:0/2249443644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c1f3de0 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3130f2c640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31180039e0 0x7f312c14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f312bfff640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 0x7f312c0b5cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f311c0046c0 tx=0x7f311c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c014410 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c019070 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0147b0 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c0b34b0 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.182+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c1f4380 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.183+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c15e840 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.184+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f311c014950 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.184+0000 7f31297fa640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.184+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f311c08ac60 con 0x7f312c070000 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.184+0000 7f3130f2c640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.185+0000 7f3130f2c640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f310c06b4d0 0x7f310c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f312400c240 tx=0x7f312402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:30.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.188+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f311c059830 con 0x7f312c070000 2024-04-25T08:29:30.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.411+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f312c073990 con 0x7f312c070000 2024-04-25T08:29:30.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.411+0000 7f31297fa640 1 -- 172.21.15.60:0/2249443644 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f311c059130 con 0x7f312c070000 2024-04-25T08:29:30.409 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:30.409 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:28.429882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:29:30.410 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 6 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.413+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f310c06b4d0 msgr2=0x7f310c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.413+0000 7f3133190640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f310c06b4d0 0x7f310c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f312400c240 tx=0x7f312402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 msgr2=0x7f312c0b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 0x7f312c0b5cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f311c0046c0 tx=0x7f311c007700 comp rx=0 tx=0).stop 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 shutdown_connections 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f312c15de80 0x7f312c15e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f312c070000 0x7f312c0b5cc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 --2- 172.21.15.60:0/2249443644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31180039e0 0x7f312c14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 >> 172.21.15.60:0/2249443644 conn(0x7f312c0b2290 msgr2=0x7f312c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:30.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 shutdown_connections 2024-04-25T08:29:30.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.414+0000 7f3133190640 1 -- 172.21.15.60:0/2249443644 wait complete. 2024-04-25T08:29:30.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.525+0000 7fdc2e61f640 1 Processor -- start 2024-04-25T08:29:30.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.525+0000 7fdc2e61f640 1 -- start start 2024-04-25T08:29:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.525+0000 7fdc2e61f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc280b2cb0 0x7fdc280d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc2e61f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc280d1020 0x7fdc280d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc2e61f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280b97a0 con 0x7fdc2815dbe0 2024-04-25T08:29:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc2e61f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2815dfc0 con 0x7fdc280b2cb0 2024-04-25T08:29:30.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc2e61f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280d1a00 con 0x7fdc280d1020 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc280b2cb0 0x7fdc280d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdc280b2cb0 0x7fdc280d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49438/0 (socket says 172.21.15.60:49438) 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 -- 172.21.15.60:0/3038418905 learned_addr learned my addr 172.21.15.60:0/3038418905 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc2cbbc640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc280d1020 0x7fdc280d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 -- 172.21.15.60:0/3038418905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc280d1020 msgr2=0x7fdc280d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc280d1020 0x7fdc280d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 -- 172.21.15.60:0/3038418905 >> v1:172.21.15.100:6789/0 conn(0x7fdc2815dbe0 legacy=0x7fdc280d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc277fe640 1 -- 172.21.15.60:0/3038418905 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280d1c50 con 0x7fdc280b2cb0 2024-04-25T08:29:30.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.526+0000 7fdc2cbbc640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc280d1020 0x7fdc280d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc277fe640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc280d2730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009870 tx=0x7fdc1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d00e42f97321c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc26ffd640 1 -- 172.21.15.60:0/3038418905 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c037070 con 0x7fdc280b2cb0 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc26ffd640 1 -- 172.21.15.60:0/3038418905 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c032440 con 0x7fdc280b2cb0 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc26ffd640 1 -- 172.21.15.60:0/3038418905 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c032810 con 0x7fdc280b2cb0 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3038418905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 msgr2=0x7fdc280d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc280d2730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009870 tx=0x7fdc1c02a270 comp rx=0 tx=0).stop 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3038418905 shutdown_connections 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc280d2730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3038418905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc280d1020 0x7fdc280d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3038418905 >> 172.21.15.60:0/3038418905 conn(0x7fdc28061a20 msgr2=0x7fdc28061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3038418905 shutdown_connections 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.527+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3038418905 wait complete. 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.529+0000 7fdc2e61f640 1 Processor -- start 2024-04-25T08:29:30.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.529+0000 7fdc2e61f640 1 -- start start 2024-04-25T08:29:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc281607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc280d1020 0x7fdc2814c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2815dbe0 0x7fdc281514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280b92a0 con 0x7fdc280b2cb0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280d2210 con 0x7fdc280d1020 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280b9420 con 0x7fdc2815dbe0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc281607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc281607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49440/0 (socket says 172.21.15.60:49440) 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 -- 172.21.15.60:0/3263008054 learned_addr learned my addr 172.21.15.60:0/3263008054 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc277fe640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc280d1020 0x7fdc2814c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2cbbc640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2815dbe0 0x7fdc281514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 -- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2815dbe0 msgr2=0x7fdc281514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2815dbe0 0x7fdc281514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 -- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc280d1020 msgr2=0x7fdc2814c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc280d1020 0x7fdc2814c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 -- 172.21.15.60:0/3263008054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280cadc0 con 0x7fdc280b2cb0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc27fff640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc281607e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c3c0 tx=0x7fdc1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180052d0 con 0x7fdc280b2cb0 2024-04-25T08:29:30.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18005490 con 0x7fdc280b2cb0 2024-04-25T08:29:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc280cb0a0 con 0x7fdc280b2cb0 2024-04-25T08:29:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18007260 con 0x7fdc280b2cb0 2024-04-25T08:29:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.530+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc280caa50 con 0x7fdc280b2cb0 2024-04-25T08:29:30.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.531+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc28073990 con 0x7fdc280b2cb0 2024-04-25T08:29:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.532+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc18007400 con 0x7fdc280b2cb0 2024-04-25T08:29:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.532+0000 7fdc24ff9640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdc1406b4d0 0x7fdc1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.532+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdc18089990 con 0x7fdc280b2cb0 2024-04-25T08:29:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.533+0000 7fdc277fe640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdc1406b4d0 0x7fdc1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.533+0000 7fdc277fe640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdc1406b4d0 0x7fdc1406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c034000 tx=0x7fdc1c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:30.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.536+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc18058560 con 0x7fdc280b2cb0 2024-04-25T08:29:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.721+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc280cb5c0 con 0x7fdc280b2cb0 2024-04-25T08:29:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.722+0000 7fdc24ff9640 1 -- 172.21.15.60:0/3263008054 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fdc18057e60 con 0x7fdc280b2cb0 2024-04-25T08:29:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:30.721 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdc1406b4d0 msgr2=0x7fdc1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdc1406b4d0 0x7fdc1406d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c034000 tx=0x7fdc1c02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 msgr2=0x7fdc281607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc281607e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800c3c0 tx=0x7fdc1800c880 comp rx=0 tx=0).stop 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 shutdown_connections 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdc2815dbe0 0x7fdc281514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdc1406b4d0 0x7fdc1406d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc280d1020 0x7fdc2814c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 --2- 172.21.15.60:0/3263008054 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdc280b2cb0 0x7fdc281607e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 >> 172.21.15.60:0/3263008054 conn(0x7fdc28061a20 msgr2=0x7fdc28158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 shutdown_connections 2024-04-25T08:29:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:30.724+0000 7fdc2e61f640 1 -- 172.21.15.60:0/3263008054 wait complete. 2024-04-25T08:29:31.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:29:23.379232+0000', 'modified': '2024-04-25T08:29:28.429882+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6836/3799475286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 3799475286}, {'type': 'v1', 'addr': '172.21.15.100:6836', 'nonce': 3799475286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T08:29:31.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:29:31.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:29:31.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.242+0000 7f07a42ac640 1 Processor -- start 2024-04-25T08:29:31.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.242+0000 7f07a42ac640 1 -- start start 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.243+0000 7f07a42ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f079c15bb50 0x7f079c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.243+0000 7f07a42ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0c70d0 0x7f079c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.243+0000 7f07a42ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079c0c75e0 con 0x7f079c0cfb40 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.243+0000 7f07a42ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079c0c8680 con 0x7f079c15bb50 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.243+0000 7f07a42ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0c78c0 con 0x7f079c0c70d0 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f079c15bb50 0x7f079c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f079c15bb50 0x7f079c0c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58532/0 (socket says 172.21.15.60:58532) 2024-04-25T08:29:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 -- 172.21.15.60:0/3184909315 learned_addr learned my addr 172.21.15.60:0/3184909315 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2849640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f079c0cfb40 0x7f079c070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46906/0 (socket says 172.21.15.60:46906) 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a1847640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0c70d0 0x7f079c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 -- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0c70d0 msgr2=0x7f079c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0c70d0 0x7f079c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 -- 172.21.15.60:0/3184909315 >> v1:172.21.15.100:6789/0 conn(0x7f079c0cfb40 legacy=0x7f079c070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a2048640 1 -- 172.21.15.60:0/3184909315 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0c7ba0 con 0x7f079c15bb50 2024-04-25T08:29:31.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.244+0000 7f07a1847640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0c70d0 0x7f079c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a2048640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c15bb50 0x7f079c0c6b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0794009870 tx=0x7f079402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e191d6f0d528ec88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a1046640 1 -- 172.21.15.60:0/3184909315 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794037070 con 0x7f079c15bb50 2024-04-25T08:29:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a1046640 1 -- 172.21.15.60:0/3184909315 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079402ae50 con 0x7f079c15bb50 2024-04-25T08:29:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a1046640 1 -- 172.21.15.60:0/3184909315 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794032860 con 0x7f079c15bb50 2024-04-25T08:29:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 -- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c15bb50 msgr2=0x7f079c0c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c15bb50 0x7f079c0c6b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0794009870 tx=0x7f079402a300 comp rx=0 tx=0).stop 2024-04-25T08:29:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 -- 172.21.15.60:0/3184909315 shutdown_connections 2024-04-25T08:29:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0c70d0 0x7f079c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 --2- 172.21.15.60:0/3184909315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c15bb50 0x7f079c0c6b90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 -- 172.21.15.60:0/3184909315 >> 172.21.15.60:0/3184909315 conn(0x7f079c06f500 msgr2=0x7f079c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.245+0000 7f07a42ac640 1 -- 172.21.15.60:0/3184909315 shutdown_connections 2024-04-25T08:29:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.246+0000 7f07a42ac640 1 -- 172.21.15.60:0/3184909315 wait complete. 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 Processor -- start 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 -- start start 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c0c70d0 0x7f079c0b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0cfb40 0x7f079c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 0x7f079c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15f6d0 con 0x7f079c15bb50 2024-04-25T08:29:31.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c0c8160 con 0x7f079c0c70d0 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.247+0000 7f07a42ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c15f850 con 0x7f079c0cfb40 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 0x7f079c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 0x7f079c15c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49452/0 (socket says 172.21.15.60:49452) 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 -- 172.21.15.60:0/1312814590 learned_addr learned my addr 172.21.15.60:0/1312814590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2048640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c0c70d0 0x7f079c0b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a1847640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0cfb40 0x7f079c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 -- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0cfb40 msgr2=0x7f079c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0cfb40 0x7f079c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 -- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c0c70d0 msgr2=0x7f079c0b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c0c70d0 0x7f079c0b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 -- 172.21.15.60:0/1312814590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0b5a70 con 0x7f079c15bb50 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a2849640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 0x7f079c15c500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c7b0 tx=0x7f078c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f079affd640 1 -- 172.21.15.60:0/1312814590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c017070 con 0x7f079c15bb50 2024-04-25T08:29:31.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f079affd640 1 -- 172.21.15.60:0/1312814590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c00a640 con 0x7f079c15bb50 2024-04-25T08:29:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c0b5d50 con 0x7f079c15bb50 2024-04-25T08:29:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f079affd640 1 -- 172.21.15.60:0/1312814590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00a9f0 con 0x7f079c15bb50 2024-04-25T08:29:31.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.248+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c0b6310 con 0x7f079c15bb50 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.250+0000 7f079affd640 1 -- 172.21.15.60:0/1312814590 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f078c01a070 con 0x7f079c15bb50 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.250+0000 7f079affd640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f078806b4d0 0x7f078806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.250+0000 7f079affd640 1 -- 172.21.15.60:0/1312814590 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f078c089a10 con 0x7f079c15bb50 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.250+0000 7f07a2048640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f078806b4d0 0x7f078806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.251+0000 7f07a2048640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f078806b4d0 0x7f078806d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0794034000 tx=0x7f079402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.251+0000 7f07a6fad740 1 Processor -- start 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.251+0000 7f07a6fad740 1 -- start start 2024-04-25T08:29:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4d7e0 0x56242cb25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 0x56242cb50040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb23de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a6fad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ca88a10 con 0x56242cb4dbc0 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ca31b30 con 0x56242cb4d7e0 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242ca84c50 con 0x56242cb50580 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 0x56242cb50040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 0x56242cb50040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49466/0 (socket says 172.21.15.60:49466) 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 -- 172.21.15.60:0/3860400168 learned_addr learned my addr 172.21.15.60:0/3860400168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a2048640 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4d7e0 0x56242cb25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 -- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 msgr2=0x56242cb23de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a2849640 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb23de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb23de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 -- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4d7e0 msgr2=0x56242cb25e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4d7e0 0x56242cb25e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a1847640 1 -- 172.21.15.60:0/3860400168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242ca46e10 con 0x56242cb4dbc0 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.252+0000 7f07a2048640 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4d7e0 0x56242cb25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a1847640 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 0x56242cb50040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0790009b60 tx=0x7f0790011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1a254f4aa6d6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f0783fff640 1 -- 172.21.15.60:0/3860400168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790009e60 con 0x56242cb4dbc0 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f0783fff640 1 -- 172.21.15.60:0/3860400168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790014440 con 0x56242cb4dbc0 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f0783fff640 1 -- 172.21.15.60:0/3860400168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790014810 con 0x56242cb4dbc0 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 -- 172.21.15.60:0/3860400168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 msgr2=0x56242cb50040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 0x56242cb50040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0790009b60 tx=0x7f0790011690 comp rx=0 tx=0).stop 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 -- 172.21.15.60:0/3860400168 shutdown_connections 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb23de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb4dbc0 0x56242cb50040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3860400168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4d7e0 0x56242cb25e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.253+0000 7f07a6fad740 1 -- 172.21.15.60:0/3860400168 >> 172.21.15.60:0/3860400168 conn(0x56242cb22840 msgr2=0x56242cb6b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.254+0000 7f07a6fad740 1 -- 172.21.15.60:0/3860400168 shutdown_connections 2024-04-25T08:29:31.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.254+0000 7f07a6fad740 1 -- 172.21.15.60:0/3860400168 wait complete. 2024-04-25T08:29:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.254+0000 7f07a6fad740 1 -- start start 2024-04-25T08:29:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.254+0000 7f07a6fad740 1 Processor -- start 2024-04-25T08:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242ca450c0 con 0x56242cb6ac20 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242cb68d40 con 0x56242cb4dbc0 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242cb51f10 con 0x56242cb50580 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a6fad740 10 client.0 _ll_register_callbacks cb 0x56242ca38ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49468/0 (socket says 172.21.15.60:49468) 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a2849640 1 -- 172.21.15.60:0/3647625156 learned_addr learned my addr 172.21.15.60:0/3647625156 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a2048640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.255+0000 7f07a1847640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a2849640 1 -- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 msgr2=0x56242cb6a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a2849640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb6a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a2849640 1 -- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 msgr2=0x56242cb67d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a2849640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb67d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a2849640 1 -- 172.21.15.60:0/3647625156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56242c79d2d0 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a2849640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb620b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f078c0543f0 tx=0x7f078c0579b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6e94bb2c155fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07817fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07817fa640 1 -- 172.21.15.60:0/3647625156 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0583f0 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07817fa640 1 -- 172.21.15.60:0/3647625156 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f078c060160 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0790014440 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07817fa640 1 -- 172.21.15.60:0/3647625156 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078c060380 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 10 client.4443 fetch_fsmap learned FSMap version 6 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56242c8676b0 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07817fa640 1 -- 172.21.15.60:0/3647625156 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790014440 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07817fa640 1 -- 172.21.15.60:0/3647625156 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f078c00ab90 con 0x56242cb6ac20 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 10 client.4443 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 10 client.4443 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:29:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 20 client.4443 populate_metadata read hostname 'smithi060' 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cb3aa50 0x56242cb39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 4 client.4443 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.256+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 --> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56242cae7920 con 0x56242cb3aa50 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.257+0000 7f07a2048640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cb3aa50 0x56242cb39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.257+0000 7f07a2048640 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cb3aa50 0x56242cb39720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdf585378c2f2f0 server_cookie=a86d926bc19c0f87 in_seq=0 out_seq=0 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.257+0000 7f07817fa640 10 client.4443 ms_handle_connect on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.257+0000 7f07817fa640 1 -- 172.21.15.60:0/3647625156 <== mds.0 v2:172.21.15.100:6834/3799475286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56242cae7920 con 0x56242cb3aa50 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.257+0000 7f07817fa640 10 client.4443 handle_command_reply: tid=0 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 client.4443 shutdown 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 msgr2=0x56242cb620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb620b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f078c0543f0 tx=0x7f078c0579b0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 shutdown_connections 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cb3aa50 0x56242cb39720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb620b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07817fa640 0 client.4443 ms_handle_reset on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:31.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 --2- 172.21.15.60:0/3647625156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb67d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 >> 172.21.15.60:0/3647625156 conn(0x56242cb4d7e0 msgr2=0x56242cb1f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 shutdown_connections 2024-04-25T08:29:31.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 1 -- 172.21.15.60:0/3647625156 wait complete. 2024-04-25T08:29:31.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.258+0000 7f07a6fad740 20 client.4443 trim_cache size 0 max 16384 2024-04-25T08:29:31.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.274+0000 7f07a6fad740 1 Processor -- start 2024-04-25T08:29:31.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.274+0000 7f07a6fad740 1 -- start start 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb6a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb1ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb20480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a6fad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1fc1e0 con 0x56242cb6ac20 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c06d8e0 con 0x56242cb4dbc0 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079c1fbde0 con 0x56242cb50580 2024-04-25T08:29:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb20480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb20480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49470/0 (socket says 172.21.15.60:49470) 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 -- 172.21.15.60:0/4154156352 learned_addr learned my addr 172.21.15.60:0/4154156352 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2048640 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb6a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a1847640 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb1ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 -- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 msgr2=0x56242cb1ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb1ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 -- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 msgr2=0x56242cb6a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb6a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 -- 172.21.15.60:0/4154156352 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56242cb38af0 con 0x56242cb6ac20 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07a2849640 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb20480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f078c089d30 tx=0x7f078c05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c719570984ec6f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07837fe640 1 -- 172.21.15.60:0/4154156352 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c00ab90 con 0x56242cb6ac20 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07837fe640 1 -- 172.21.15.60:0/4154156352 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0574d0 con 0x56242cb6ac20 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.275+0000 7f07837fe640 1 -- 172.21.15.60:0/4154156352 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c0581f0 con 0x56242cb6ac20 2024-04-25T08:29:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 -- 172.21.15.60:0/4154156352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 msgr2=0x56242cb20480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb20480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f078c089d30 tx=0x7f078c05a000 comp rx=0 tx=0).stop 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 -- 172.21.15.60:0/4154156352 shutdown_connections 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb20480 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb50580 0x56242cb1ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 --2- 172.21.15.60:0/4154156352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb4dbc0 0x56242cb6a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 -- 172.21.15.60:0/4154156352 >> 172.21.15.60:0/4154156352 conn(0x56242cb4d7e0 msgr2=0x56242cb37980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 -- 172.21.15.60:0/4154156352 shutdown_connections 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 -- 172.21.15.60:0/4154156352 wait complete. 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 -- start start 2024-04-25T08:29:31.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.276+0000 7f07a6fad740 1 Processor -- start 2024-04-25T08:29:31.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb4dbc0 0x56242cb618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb50580 0x56242cb68d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242cb3a2e0 con 0x56242cb6ac20 2024-04-25T08:29:31.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56242cb38970 con 0x56242cb50580 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56242cb3ae40 con 0x56242cb4dbc0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a6fad740 10 client.0 _ll_register_callbacks cb 0x7f07900094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a2849640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb69240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49486/0 (socket says 172.21.15.60:49486) 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.277+0000 7f07a2849640 1 -- 172.21.15.60:0/1673067099 learned_addr learned my addr 172.21.15.60:0/1673067099 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a1847640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb50580 0x56242cb68d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2048640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb4dbc0 0x56242cb618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2849640 1 -- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb4dbc0 msgr2=0x56242cb618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2849640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb4dbc0 0x56242cb618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2849640 1 -- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb50580 msgr2=0x56242cb68d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2849640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb50580 0x56242cb68d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2849640 1 -- 172.21.15.60:0/1673067099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56242cb26d50 con 0x56242cb6ac20 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a2849640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb69240 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f078c0543f0 tx=0x7f078c087b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af5c16d8d075ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07817fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07817fa640 1 -- 172.21.15.60:0/1673067099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c07a050 con 0x56242cb6ac20 2024-04-25T08:29:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07817fa640 1 -- 172.21.15.60:0/1673067099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f078c0583f0 con 0x56242cb6ac20 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f078c0574d0 con 0x56242cb6ac20 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07817fa640 1 -- 172.21.15.60:0/1673067099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078c065050 con 0x56242cb6ac20 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 10 client.4449 fetch_fsmap learned FSMap version 6 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56242cb67740 con 0x56242cb6ac20 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07817fa640 1 -- 172.21.15.60:0/1673067099 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0574d0 con 0x56242cb6ac20 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07817fa640 1 -- 172.21.15.60:0/1673067099 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f078c081080 con 0x56242cb6ac20 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 10 client.4449 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 10 client.4449 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 20 client.4449 populate_metadata read hostname 'smithi060' 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cbdb180 0x56242cbdb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 4 client.4449 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.278+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 --> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56242ca95f50 con 0x56242cbdb180 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07a2048640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cbdb180 0x56242cbdb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07a2048640 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cbdb180 0x56242cbdb560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ae8b2695ab51b5 server_cookie=a03c686b3ae7a17b in_seq=0 out_seq=0 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07817fa640 10 client.4449 ms_handle_connect on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07817fa640 1 -- 172.21.15.60:0/1673067099 <== mds.0 v2:172.21.15.100:6834/3799475286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56242ca95f50 con 0x56242cbdb180 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07817fa640 10 client.4449 handle_command_reply: tid=0 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07a6fad740 1 client.4449 shutdown 2024-04-25T08:29:31.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 msgr2=0x56242cb69240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.279+0000 7f07a6fad740 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb69240 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f078c0543f0 tx=0x7f078c087b60 comp rx=0 tx=0).stop 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 shutdown_connections 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x56242cbdb180 0x56242cbdb560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56242cb6ac20 0x56242cb69240 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56242cb50580 0x56242cb68d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07817fa640 0 client.4449 ms_handle_reset on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 --2- 172.21.15.60:0/1673067099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56242cb4dbc0 0x56242cb618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 >> 172.21.15.60:0/1673067099 conn(0x56242cb4d7e0 msgr2=0x56242cb25df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 shutdown_connections 2024-04-25T08:29:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 1 -- 172.21.15.60:0/1673067099 wait complete. 2024-04-25T08:29:31.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a6fad740 20 client.4449 trim_cache size 0 max 16384 2024-04-25T08:29:31.288 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4288, 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 7.2494432980000001, 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 6, 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 25, 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.274749602 2024-04-25T08:29:31.289 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:29:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f078806b4d0 msgr2=0x7f078806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f078806b4d0 0x7f078806d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0794034000 tx=0x7f079402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 msgr2=0x7f079c15c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 0x7f079c15c500 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c7b0 tx=0x7f078c00cc70 comp rx=0 tx=0).stop 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 shutdown_connections 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f079c15bb50 0x7f079c15c500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f079c0cfb40 0x7f079c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f078806b4d0 0x7f078806d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 --2- 172.21.15.60:0/1312814590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f079c0c70d0 0x7f079c0b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 >> 172.21.15.60:0/1312814590 conn(0x7f079c06f500 msgr2=0x7f079c0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 shutdown_connections 2024-04-25T08:29:31.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.280+0000 7f07a42ac640 1 -- 172.21.15.60:0/1312814590 wait complete. 2024-04-25T08:29:31.291 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:29:31.292 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:31.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:29:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.394+0000 7fd23aaf5640 1 Processor -- start 2024-04-25T08:29:31.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.394+0000 7fd23aaf5640 1 -- start start 2024-04-25T08:29:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd23aaf5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd23415dbe0 0x7fd2340d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd23aaf5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2340b6220 con 0x7fd2340b2cb0 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd23aaf5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2340b97a0 con 0x7fd2340b92a0 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd23aaf5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23415dfc0 con 0x7fd23415dbe0 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd233fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd2340b2cb0 0x7fd2340b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38932/0 (socket says 172.21.15.60:38932) 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd233fff640 1 -- 172.21.15.60:0/3392501636 learned_addr learned my addr 172.21.15.60:0/3392501636 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd238891640 1 --2- 172.21.15.60:0/3392501636 >> v2:172.21.15.100:3300/0 conn(0x7fd23415dbe0 0x7fd2340d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd2337fe640 1 -- 172.21.15.60:0/3392501636 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586152593 0 0) 0x7fd2340b6220 con 0x7fd2340b2cb0 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.395+0000 7fd2337fe640 1 -- 172.21.15.60:0/3392501636 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd21c003520 con 0x7fd2340b2cb0 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.396+0000 7fd238891640 1 -- 172.21.15.60:0/3392501636 >> v1:172.21.15.100:6789/0 conn(0x7fd2340b92a0 legacy=0x7fd2340b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.396+0000 7fd238891640 1 -- 172.21.15.60:0/3392501636 >> v1:172.21.15.60:6789/0 conn(0x7fd2340b2cb0 legacy=0x7fd2340b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.396+0000 7fd238891640 1 -- 172.21.15.60:0/3392501636 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340d1c50 con 0x7fd23415dbe0 2024-04-25T08:29:31.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.396+0000 7fd238891640 1 --2- 172.21.15.60:0/3392501636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340d0910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2240097a0 tx=0x7fd22402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c2303cf0d31f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd2337fe640 1 -- 172.21.15.60:0/3392501636 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224037070 con 0x7fd23415dbe0 2024-04-25T08:29:31.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd2337fe640 1 -- 172.21.15.60:0/3392501636 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd224032440 con 0x7fd23415dbe0 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd2337fe640 1 -- 172.21.15.60:0/3392501636 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224032810 con 0x7fd23415dbe0 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3392501636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 msgr2=0x7fd2340d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3392501636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340d0910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2240097a0 tx=0x7fd22402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3392501636 shutdown_connections 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3392501636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340d0910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3392501636 >> 172.21.15.60:0/3392501636 conn(0x7fd234061a20 msgr2=0x7fd234061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3392501636 shutdown_connections 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.397+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3392501636 wait complete. 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.398+0000 7fd23aaf5640 1 Processor -- start 2024-04-25T08:29:31.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.398+0000 7fd23aaf5640 1 -- start start 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd23aaf5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2340b2cb0 0x7fd2340b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd23aaf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 0x7fd2340ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd23aaf5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd23aaf5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2340d2210 con 0x7fd2340b92a0 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd23aaf5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2340d1a00 con 0x7fd23415dbe0 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd23aaf5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2340b6ab0 con 0x7fd2340b2cb0 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd233fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 0x7fd2340ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd233fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 0x7fd2340ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49492/0 (socket says 172.21.15.60:49492) 2024-04-25T08:29:31.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd233fff640 1 -- 172.21.15.60:0/3017856178 learned_addr learned my addr 172.21.15.60:0/3017856178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.399+0000 7fd239092640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd238891640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2340b2cb0 0x7fd2340b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd233fff640 1 -- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2340b2cb0 msgr2=0x7fd2340b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd233fff640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2340b2cb0 0x7fd2340b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd233fff640 1 -- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 msgr2=0x7fd2340cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd233fff640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd233fff640 1 -- 172.21.15.60:0/3017856178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2340b5290 con 0x7fd2340b92a0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd238891640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2340b2cb0 0x7fd2340b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd239092640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd233fff640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 0x7fd2340ccc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c4d0 tx=0x7fd22800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228017070 con 0x7fd2340b92a0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228005ce0 con 0x7fd2340b92a0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228007270 con 0x7fd2340b92a0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2340b5570 con 0x7fd2340b92a0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.400+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2340b5b30 con 0x7fd2340b92a0 2024-04-25T08:29:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.401+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd234073990 con 0x7fd2340b92a0 2024-04-25T08:29:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.401+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd22801a070 con 0x7fd2340b92a0 2024-04-25T08:29:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.402+0000 7fd2317fa640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd22006b4d0 0x7fd22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.402+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd228089900 con 0x7fd2340b92a0 2024-04-25T08:29:31.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.402+0000 7fd238891640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd22006b4d0 0x7fd22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:31.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.402+0000 7fd238891640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd22006b4d0 0x7fd22006d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd224034000 tx=0x7fd22402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:31.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.406+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd228057d50 con 0x7fd2340b92a0 2024-04-25T08:29:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.624+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd2340b4620 con 0x7fd2340b92a0 2024-04-25T08:29:31.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.624+0000 7fd2317fa640 1 -- 172.21.15.60:0/3017856178 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd22805bdf0 con 0x7fd2340b92a0 2024-04-25T08:29:31.623 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.626+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd22006b4d0 msgr2=0x7fd22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.626+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd22006b4d0 0x7fd22006d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd224034000 tx=0x7fd22402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.626+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 msgr2=0x7fd2340ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.626+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 0x7fd2340ccc20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd22800c4d0 tx=0x7fd22800c990 comp rx=0 tx=0).stop 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 shutdown_connections 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd22006b4d0 0x7fd22006d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd23415dbe0 0x7fd2340cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2340b92a0 0x7fd2340ccc20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 --2- 172.21.15.60:0/3017856178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2340b2cb0 0x7fd2340b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 >> 172.21.15.60:0/3017856178 conn(0x7fd234061a20 msgr2=0x7fd2340d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 shutdown_connections 2024-04-25T08:29:31.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:31.627+0000 7fd23aaf5640 1 -- 172.21.15.60:0/3017856178 wait complete. 2024-04-25T08:29:31.635 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:29:31.635 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:29:31.635 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:29:31.635 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T08:29:31.635 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:31.636 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:29:31.636 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:29:31.636 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:29:31.636 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:29:31.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:31.636 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:29:31.651 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:31.651 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:29:31.652 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42739sec preferred_lft 42739sec 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:31.653 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:29:31.654 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:29:31.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:29:31.654 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:31.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:31.788 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:29:31.813 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:29:31.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:31.815 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:29:31.878 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:29:31.878 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:29:31.878 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:29:31.878 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:29:31.878 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:29:31.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:31.879 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:31.879 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:31.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:29:31.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:29:31.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:29:31.879 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:32.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.101 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:29:32.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.116 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-04-25T08:29:32.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.171 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:32.171 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:32.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:29:32.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T08:29:32.171 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:32.249 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T08:29:32.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T08:29:32.249 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:32.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.379 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:32.379 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:32.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:29:32.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:29:32.379 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:32.424 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T08:29:32.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:32.424 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:29:32.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:32.440 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:32.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T08:29:32.563 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T08:29:32.619 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:29:32.620 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:29:32.621 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:29:32.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.622 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T08:29:32.675 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:29:32.675 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T08:29:32.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo modprobe fuse 2024-04-25T08:29:32.742 DEBUG:teuthology.orchestra.run.smithi060:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/proc 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/sys 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/dev 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/security 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/shm 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/pts 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/run 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/cgroup 2024-04-25T08:29:32.798 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/pstore 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/bpf 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/ 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/selinux 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/mqueue 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/dev/hugepages 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/debug 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/tracing 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/kernel/config 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/sys/fs/fuse/connections 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:29:32.799 INFO:teuthology.orchestra.run.smithi060.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/run/user/1000 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:29:32.800 INFO:teuthology.orchestra.run.smithi060.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:29:32.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.801 DEBUG:teuthology.orchestra.run.smithi060:> ls /sys/fs/fuse/connections 2024-04-25T08:29:32.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:2024-04-25T08:29:32.816+0000 7f1904320480 -1 init, newargv = 0x562bda6f29e0 newargc=15 2024-04-25T08:29:32.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[40627]: starting ceph client 2024-04-25T08:29:32.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[40627]: starting fuse 2024-04-25T08:29:32.830 INFO:teuthology.orchestra.run.smithi060.stdout:42 2024-04-25T08:29:32.830 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-25T08:29:32.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:29:32.830 DEBUG:teuthology.orchestra.run.smithi060:> import glob 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> import re 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> import subprocess 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> def _find_admin_socket(client_name): 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> files = glob.glob(asok_path) 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> # Given a non-glob path, it better be there 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> if "*" not in asok_path: 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> assert(len(files) == 1) 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> return files[0] 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:29:32.831 DEBUG:teuthology.orchestra.run.smithi060:> for f in files: 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> contents = proc_f.read() 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> if mountpoint in contents: 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> return f 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> print(_find_admin_socket("client.admin")) 2024-04-25T08:29:32.832 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:29:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:/var/run/ceph/ceph-client.admin.40627.asok 2024-04-25T08:29:32.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:29:32.959 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.40627.asok 2024-04-25T08:29:32.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:32.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.40627.asok status 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "metadata": { 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "entity_id": "admin", 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "hostname": "smithi060", 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "pid": "40627", 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "root": "/" 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_count": 0, 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "dentry_pinned_count": 0, 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4461, 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "inst": { 2024-04-25T08:29:33.097 INFO:teuthology.orchestra.run.smithi060.stdout: "name": { 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "client", 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num": 4461 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 532849164 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": { 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:29:33.098 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 532849164 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "inst_str": "client.4461 192.168.0.1:0/532849164", 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "addr_str": "192.168.0.1:0/532849164", 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "inode_count": 1, 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_epoch": 6, 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch": 25, 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklisted": false, 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs" 2024-04-25T08:29:33.099 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:29:33.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:33.105 DEBUG:teuthology.orchestra.run.smithi060:> stat --file-system '--printf=%T 2024-04-25T08:29:33.106 DEBUG:teuthology.orchestra.run.smithi060:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:33.121 INFO:teuthology.orchestra.run.smithi060.stdout:fuseblk 2024-04-25T08:29:33.121 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:33.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:33.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:33.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:33.187 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T08:29:33.262 INFO:teuthology.orchestra.run.smithi060.stdout:# file: . 2024-04-25T08:29:33.262 INFO:teuthology.orchestra.run.smithi060.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T08:29:33.262 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:33.262 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:29:33.262 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:29:33.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:33.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.461+0000 7fc58fd9e640 1 Processor -- start 2024-04-25T08:29:33.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.461+0000 7fc58fd9e640 1 -- start start 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58fd9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880d1020 0x7fc5880d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58fd9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5880b97a0 con 0x7fc5880b2cb0 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58fd9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5880d1c50 con 0x7fc58815dbe0 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58fd9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58815dfc0 con 0x7fc5880d1020 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58d339640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc5880b2cb0 0x7fc5880d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54332/0 (socket says 172.21.15.60:54332) 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58d339640 1 -- 172.21.15.60:0/3424665382 learned_addr learned my addr 172.21.15.60:0/3424665382 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58e33b640 1 --2- 172.21.15.60:0/3424665382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880d1020 0x7fc5880d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987601284 0 0) 0x7fc5880b97a0 con 0x7fc5880b2cb0 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.462+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc574003520 con 0x7fc5880b2cb0 2024-04-25T08:29:33.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524573034 0 0) 0x7fc5880d1c50 con 0x7fc58815dbe0 2024-04-25T08:29:33.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5880b97a0 con 0x7fc58815dbe0 2024-04-25T08:29:33.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58e33b640 1 -- 172.21.15.60:0/3424665382 >> v1:172.21.15.100:6789/0 conn(0x7fc58815dbe0 legacy=0x7fc5880d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58e33b640 1 -- 172.21.15.60:0/3424665382 >> v1:172.21.15.60:6789/0 conn(0x7fc5880b2cb0 legacy=0x7fc5880d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 547378876 0 0) 0x7fc574003520 con 0x7fc5880b2cb0 2024-04-25T08:29:33.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58e33b640 1 -- 172.21.15.60:0/3424665382 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880d1f30 con 0x7fc5880d1020 2024-04-25T08:29:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.463+0000 7fc58e33b640 1 --2- 172.21.15.60:0/3424665382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880d1020 0x7fc5880d2e40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc580009dc0 tx=0x7fc58002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879a6509b45767e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578001670 con 0x7fc5880d1020 2024-04-25T08:29:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc580007540 con 0x7fc5880d1020 2024-04-25T08:29:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58cb38640 1 -- 172.21.15.60:0/3424665382 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800078f0 con 0x7fc5880d1020 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 -- 172.21.15.60:0/3424665382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880d1020 msgr2=0x7fc5880d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/3424665382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880d1020 0x7fc5880d2e40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc580009dc0 tx=0x7fc58002a660 comp rx=0 tx=0).stop 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 -- 172.21.15.60:0/3424665382 shutdown_connections 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/3424665382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880d1020 0x7fc5880d2e40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 -- 172.21.15.60:0/3424665382 >> 172.21.15.60:0/3424665382 conn(0x7fc588061a20 msgr2=0x7fc588061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 -- 172.21.15.60:0/3424665382 shutdown_connections 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.464+0000 7fc58fd9e640 1 -- 172.21.15.60:0/3424665382 wait complete. 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.465+0000 7fc58fd9e640 1 Processor -- start 2024-04-25T08:29:33.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.465+0000 7fc58fd9e640 1 -- start start 2024-04-25T08:29:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58fd9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880b2cb0 0x7fc58814fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58fd9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880d1020 0x7fc5881f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58fd9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 0x7fc5880b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58fd9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880b92a0 con 0x7fc58815dbe0 2024-04-25T08:29:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58fd9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880d1a00 con 0x7fc5880d1020 2024-04-25T08:29:33.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58fd9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880b9420 con 0x7fc5880b2cb0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58e33b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 0x7fc5880b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58e33b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 0x7fc5880b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43538/0 (socket says 172.21.15.60:43538) 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58e33b640 1 -- 172.21.15.60:0/2334420580 learned_addr learned my addr 172.21.15.60:0/2334420580 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58d339640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880d1020 0x7fc5881f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.466+0000 7fc58db3a640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880b2cb0 0x7fc58814fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58e33b640 1 -- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880b2cb0 msgr2=0x7fc58814fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58e33b640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880b2cb0 0x7fc58814fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58e33b640 1 -- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880d1020 msgr2=0x7fc5881f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58e33b640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880d1020 0x7fc5881f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58e33b640 1 -- 172.21.15.60:0/2334420580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5881f49d0 con 0x7fc58815dbe0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58e33b640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 0x7fc5880b5830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc580009ee0 tx=0x7fc580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580040070 con 0x7fc58815dbe0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc580031c40 con 0x7fc58815dbe0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5881f4410 con 0x7fc58815dbe0 2024-04-25T08:29:33.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580009450 con 0x7fc58815dbe0 2024-04-25T08:29:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.467+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58815f530 con 0x7fc58815dbe0 2024-04-25T08:29:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.468+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc588073990 con 0x7fc58815dbe0 2024-04-25T08:29:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.468+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc580039040 con 0x7fc58815dbe0 2024-04-25T08:29:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.469+0000 7fc57e7fc640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc56c06b4d0 0x7fc56c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.469+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc5800ab6e0 con 0x7fc58815dbe0 2024-04-25T08:29:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.469+0000 7fc58db3a640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc56c06b4d0 0x7fc56c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.469+0000 7fc58db3a640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc56c06b4d0 0x7fc56c06d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5840011e0 tx=0x7fc584009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.473+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5881f49d0 con 0x7fc58815dbe0 2024-04-25T08:29:33.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.653+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5880ca2e0 con 0x7fc58815dbe0 2024-04-25T08:29:33.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.655+0000 7fc57e7fc640 1 -- 172.21.15.60:0/2334420580 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fc580079c60 con 0x7fc58815dbe0 2024-04-25T08:29:33.653 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:33.653 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc56c06b4d0 msgr2=0x7fc56c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc56c06b4d0 0x7fc56c06d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5840011e0 tx=0x7fc584009040 comp rx=0 tx=0).stop 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 msgr2=0x7fc5880b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 0x7fc5880b5830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc580009ee0 tx=0x7fc580004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 shutdown_connections 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc58815dbe0 0x7fc5880b5830 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc56c06b4d0 0x7fc56c06d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5880d1020 0x7fc5881f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 --2- 172.21.15.60:0/2334420580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5880b2cb0 0x7fc58814fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 >> 172.21.15.60:0/2334420580 conn(0x7fc588061a20 msgr2=0x7fc58806c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:33.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 shutdown_connections 2024-04-25T08:29:33.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.657+0000 7fc58fd9e640 1 -- 172.21.15.60:0/2334420580 wait complete. 2024-04-25T08:29:33.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:29:33.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.771+0000 7f66bb1b7640 1 Processor -- start 2024-04-25T08:29:33.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.771+0000 7f66bb1b7640 1 -- start start 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66bb1b7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66b40ced80 0x7f66b40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66bb1b7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b415e000 con 0x7f66b415dc20 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66bb1b7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b40d1de0 con 0x7f66b4161710 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66bb1b7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b4161c60 con 0x7f66b40ced80 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66b40ced80 0x7f66b40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66b40ced80 0x7f66b40d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43548/0 (socket says 172.21.15.60:43548) 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b8f53640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f66b415dc20 0x7f66b40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54344/0 (socket says 172.21.15.60:54344) 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b3fff640 1 -- 172.21.15.60:0/2464795069 learned_addr learned my addr 172.21.15.60:0/2464795069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b3fff640 1 -- 172.21.15.60:0/2464795069 >> v1:172.21.15.100:6789/0 conn(0x7f66b4161710 legacy=0x7f66b40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b3fff640 1 -- 172.21.15.60:0/2464795069 >> v1:172.21.15.60:6789/0 conn(0x7f66b415dc20 legacy=0x7f66b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.772+0000 7f66b37fe640 1 -- 172.21.15.60:0/2464795069 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272236713 0 0) 0x7f66b415e000 con 0x7f66b415dc20 2024-04-25T08:29:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.773+0000 7f66b3fff640 1 -- 172.21.15.60:0/2464795069 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b4161f40 con 0x7f66b40ced80 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.773+0000 7f66b3fff640 1 --2- 172.21.15.60:0/2464795069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b40ced80 0x7f66b40d22e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009870 tx=0x7f66a802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd7fdb48746174f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.773+0000 7f66b37fe640 1 -- 172.21.15.60:0/2464795069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8037070 con 0x7f66b40ced80 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.773+0000 7f66b37fe640 1 -- 172.21.15.60:0/2464795069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a802ae50 con 0x7f66b40ced80 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.773+0000 7f66b37fe640 1 -- 172.21.15.60:0/2464795069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8032860 con 0x7f66b40ced80 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 -- 172.21.15.60:0/2464795069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b40ced80 msgr2=0x7f66b40d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/2464795069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b40ced80 0x7f66b40d22e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009870 tx=0x7f66a802a300 comp rx=0 tx=0).stop 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 -- 172.21.15.60:0/2464795069 shutdown_connections 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/2464795069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b40ced80 0x7f66b40d22e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 -- 172.21.15.60:0/2464795069 >> 172.21.15.60:0/2464795069 conn(0x7f66b40d0420 msgr2=0x7f66b40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 -- 172.21.15.60:0/2464795069 shutdown_connections 2024-04-25T08:29:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.774+0000 7f66bb1b7640 1 -- 172.21.15.60:0/2464795069 wait complete. 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.775+0000 7f66bb1b7640 1 Processor -- start 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.775+0000 7f66bb1b7640 1 -- start start 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66bb1b7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40ced80 0x7f66b40c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66bb1b7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 0x7f66b40b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66bb1b7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b4161710 0x7f66b41f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66bb1b7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40b5e50 con 0x7f66b415dc20 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66bb1b7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4162500 con 0x7f66b40ced80 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66bb1b7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40b5fd0 con 0x7f66b4161710 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 0x7f66b40b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 0x7f66b40b8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43552/0 (socket says 172.21.15.60:43552) 2024-04-25T08:29:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66b3fff640 1 -- 172.21.15.60:0/4128310385 learned_addr learned my addr 172.21.15.60:0/4128310385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66b8f53640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40ced80 0x7f66b40c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.776+0000 7f66b9754640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b4161710 0x7f66b41f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b3fff640 1 -- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b4161710 msgr2=0x7f66b41f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b3fff640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b4161710 0x7f66b41f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b3fff640 1 -- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40ced80 msgr2=0x7f66b40c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b3fff640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40ced80 0x7f66b40c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b3fff640 1 -- 172.21.15.60:0/4128310385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b41f7590 con 0x7f66b415dc20 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b9754640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b4161710 0x7f66b41f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b3fff640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 0x7f66b40b8690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009870 tx=0x7f66a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8041070 con 0x7f66b415dc20 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a803c870 con 0x7f66b415dc20 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8037040 con 0x7f66b415dc20 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40cd850 con 0x7f66b415dc20 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.777+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b40cd520 con 0x7f66b415dc20 2024-04-25T08:29:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.778+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b4073a00 con 0x7f66b415dc20 2024-04-25T08:29:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.778+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66a8044070 con 0x7f66b415dc20 2024-04-25T08:29:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.779+0000 7f66b17fa640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66a006b4d0 0x7f66a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.779+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f66a80ab1a0 con 0x7f66b415dc20 2024-04-25T08:29:33.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.779+0000 7f66b8f53640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66a006b4d0 0x7f66a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.780+0000 7f66b8f53640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66a006b4d0 0x7f66a006d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66a4007f70 tx=0x7f66a4007f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.783+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66a80797a0 con 0x7f66b415dc20 2024-04-25T08:29:33.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.964+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66b40d16b0 con 0x7f66b415dc20 2024-04-25T08:29:33.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.965+0000 7f66b17fa640 1 -- 172.21.15.60:0/4128310385 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66a807d840 con 0x7f66b415dc20 2024-04-25T08:29:33.963 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66a006b4d0 msgr2=0x7f66a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66a006b4d0 0x7f66a006d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f66a4007f70 tx=0x7f66a4007f00 comp rx=0 tx=0).stop 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 msgr2=0x7f66b40b8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 0x7f66b40b8690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009870 tx=0x7f66a80046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 shutdown_connections 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66b4161710 0x7f66b41f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b415dc20 0x7f66b40b8690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66a006b4d0 0x7f66a006d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 --2- 172.21.15.60:0/4128310385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66b40ced80 0x7f66b40c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 >> 172.21.15.60:0/4128310385 conn(0x7f66b40d0420 msgr2=0x7f66b406c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.966+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 shutdown_connections 2024-04-25T08:29:33.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:33.967+0000 7f66bb1b7640 1 -- 172.21.15.60:0/4128310385 wait complete. 2024-04-25T08:29:33.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:29:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.081+0000 7faea1789640 1 Processor -- start 2024-04-25T08:29:34.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.081+0000 7faea1789640 1 -- start start 2024-04-25T08:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7faea1789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c15dc20 0x7fae9c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7faea1789640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fae9c161710 0x7fae9c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7faea1789640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c15e000 con 0x7fae9c0b9330 2024-04-25T08:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7faea1789640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c0d1de0 con 0x7fae9c161710 2024-04-25T08:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7faea1789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c161c60 con 0x7fae9c15dc20 2024-04-25T08:29:34.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fae9c0b9330 0x7fae9c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54350/0 (socket says 172.21.15.60:54350) 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9b7fe640 1 -- 172.21.15.60:0/1317062275 learned_addr learned my addr 172.21.15.60:0/1317062275 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae93fff640 1 --2- 172.21.15.60:0/1317062275 >> v2:172.21.15.100:3300/0 conn(0x7fae9c161710 0x7fae9c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9affd640 1 --2- 172.21.15.60:0/1317062275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c15dc20 0x7fae9c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9a7fc640 1 -- 172.21.15.60:0/1317062275 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381261481 0 0) 0x7fae9c15e000 con 0x7fae9c0b9330 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9a7fc640 1 -- 172.21.15.60:0/1317062275 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae88003520 con 0x7fae9c0b9330 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9affd640 1 -- 172.21.15.60:0/1317062275 >> v2:172.21.15.100:3300/0 conn(0x7fae9c161710 msgr2=0x7fae9c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9a7fc640 1 -- 172.21.15.60:0/1317062275 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598339215 0 0) 0x7fae88003520 con 0x7fae9c0b9330 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9affd640 1 --2- 172.21.15.60:0/1317062275 >> v2:172.21.15.100:3300/0 conn(0x7fae9c161710 0x7fae9c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9affd640 1 -- 172.21.15.60:0/1317062275 >> v1:172.21.15.60:6789/0 conn(0x7fae9c0b9330 legacy=0x7fae9c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae9affd640 1 -- 172.21.15.60:0/1317062275 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c161f40 con 0x7fae9c15dc20 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.082+0000 7fae93fff640 1 --2- 172.21.15.60:0/1317062275 >> v2:172.21.15.100:3300/0 conn(0x7fae9c161710 0x7fae9c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.083+0000 7fae9affd640 1 --2- 172.21.15.60:0/1317062275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c15dc20 0x7fae9c0ce670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0097a0 tx=0x7fae8c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4b3c7999b245e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.083+0000 7fae9a7fc640 1 -- 172.21.15.60:0/1317062275 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c037070 con 0x7fae9c15dc20 2024-04-25T08:29:34.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.083+0000 7fae9a7fc640 1 -- 172.21.15.60:0/1317062275 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c02aba0 con 0x7fae9c15dc20 2024-04-25T08:29:34.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.083+0000 7fae9a7fc640 1 -- 172.21.15.60:0/1317062275 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940019f0 con 0x7fae9c15dc20 2024-04-25T08:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 -- 172.21.15.60:0/1317062275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c15dc20 msgr2=0x7fae9c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 --2- 172.21.15.60:0/1317062275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c15dc20 0x7fae9c0ce670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0097a0 tx=0x7fae8c02a110 comp rx=0 tx=0).stop 2024-04-25T08:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 -- 172.21.15.60:0/1317062275 shutdown_connections 2024-04-25T08:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 --2- 172.21.15.60:0/1317062275 >> v2:172.21.15.100:3300/0 conn(0x7fae9c161710 0x7fae9c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 --2- 172.21.15.60:0/1317062275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c15dc20 0x7fae9c0ce670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 -- 172.21.15.60:0/1317062275 >> 172.21.15.60:0/1317062275 conn(0x7fae9c0d0420 msgr2=0x7fae9c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 -- 172.21.15.60:0/1317062275 shutdown_connections 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.084+0000 7faea1789640 1 -- 172.21.15.60:0/1317062275 wait complete. 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.085+0000 7faea1789640 1 Processor -- start 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.085+0000 7faea1789640 1 -- start start 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7faea1789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c0b9330 0x7fae9c1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7faea1789640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 0x7fae9c0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7faea1789640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae9c161710 0x7fae9c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7faea1789640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b5e50 con 0x7fae9c15dc20 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7faea1789640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c162500 con 0x7fae9c161710 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7faea1789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b5fd0 con 0x7fae9c0b9330 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 0x7fae9c0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 0x7fae9c0aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43556/0 (socket says 172.21.15.60:43556) 2024-04-25T08:29:34.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 -- 172.21.15.60:0/158535725 learned_addr learned my addr 172.21.15.60:0/158535725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae9affd640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c0b9330 0x7fae9c1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae9b7fe640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae9c161710 0x7fae9c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 -- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c0b9330 msgr2=0x7fae9c1f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c0b9330 0x7fae9c1f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 -- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae9c161710 msgr2=0x7fae9c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae9c161710 0x7fae9c0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.086+0000 7fae93fff640 1 -- 172.21.15.60:0/158535725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0b1c00 con 0x7fae9c15dc20 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7fae9b7fe640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae9c161710 0x7fae9c0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7fae93fff640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 0x7fae9c0aa380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae8400e960 tx=0x7fae8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84017070 con 0x7fae9c15dc20 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8400ca00 con 0x7fae9c15dc20 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c0b1ee0 con 0x7fae9c15dc20 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400cdd0 con 0x7fae9c15dc20 2024-04-25T08:29:34.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.087+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c1f3a70 con 0x7fae9c15dc20 2024-04-25T08:29:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.088+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c073a00 con 0x7fae9c15dc20 2024-04-25T08:29:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.089+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae8401a070 con 0x7fae9c15dc20 2024-04-25T08:29:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.089+0000 7fae937fe640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae7c06b4d0 0x7fae7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.089+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae8408a750 con 0x7fae9c15dc20 2024-04-25T08:29:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.089+0000 7fae9affd640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae7c06b4d0 0x7fae7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.090+0000 7fae9affd640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae7c06b4d0 0x7fae7c06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae8c034000 tx=0x7fae8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.094+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae84059320 con 0x7fae9c15dc20 2024-04-25T08:29:34.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.273+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae9c0b6e00 con 0x7fae9c15dc20 2024-04-25T08:29:34.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.273+0000 7fae937fe640 1 -- 172.21.15.60:0/158535725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae84058c20 con 0x7fae9c15dc20 2024-04-25T08:29:34.271 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:34.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae7c06b4d0 msgr2=0x7fae7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae7c06b4d0 0x7fae7c06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fae8c034000 tx=0x7fae8c02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:34.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 msgr2=0x7fae9c0aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 0x7fae9c0aa380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae8400e960 tx=0x7fae8400ee20 comp rx=0 tx=0).stop 2024-04-25T08:29:34.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 shutdown_connections 2024-04-25T08:29:34.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae7c06b4d0 0x7fae7c06d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae9c161710 0x7fae9c0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae9c15dc20 0x7fae9c0aa380 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 --2- 172.21.15.60:0/158535725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae9c0b9330 0x7fae9c1f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 >> 172.21.15.60:0/158535725 conn(0x7fae9c0d0420 msgr2=0x7fae9c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:34.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 shutdown_connections 2024-04-25T08:29:34.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.275+0000 7faea1789640 1 -- 172.21.15.60:0/158535725 wait complete. 2024-04-25T08:29:34.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:29:34.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.391+0000 7f4254638640 1 Processor -- start 2024-04-25T08:29:34.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.391+0000 7f4254638640 1 -- start start 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4254638640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f424c15dc20 0x7f424c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4254638640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f424c0d32f0 0x7f424c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4254638640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424c15e000 con 0x7f424c161710 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4254638640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c0d1de0 con 0x7f424c15dc20 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4254638640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c161c60 con 0x7f424c0d32f0 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f424c15dc20 0x7f424c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f424c15dc20 0x7f424c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43558/0 (socket says 172.21.15.60:43558) 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 -- 172.21.15.60:0/333537904 learned_addr learned my addr 172.21.15.60:0/333537904 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4251bd3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f424c161710 0x7f424c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54366/0 (socket says 172.21.15.60:54366) 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f4252bd5640 1 --2- 172.21.15.60:0/333537904 >> v2:172.21.15.100:3300/0 conn(0x7f424c0d32f0 0x7f424c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 -- 172.21.15.60:0/333537904 >> v2:172.21.15.100:3300/0 conn(0x7f424c0d32f0 msgr2=0x7f424c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 --2- 172.21.15.60:0/333537904 >> v2:172.21.15.100:3300/0 conn(0x7f424c0d32f0 0x7f424c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 -- 172.21.15.60:0/333537904 >> v1:172.21.15.60:6789/0 conn(0x7f424c161710 legacy=0x7f424c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42523d4640 1 -- 172.21.15.60:0/333537904 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c161f40 con 0x7f424c15dc20 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.392+0000 7f42513d2640 1 -- 172.21.15.60:0/333537904 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189161192 0 0) 0x7f424c15e000 con 0x7f424c161710 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.393+0000 7f42523d4640 1 --2- 172.21.15.60:0/333537904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c15dc20 0x7f424c0ce670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f424c0b6910 tx=0x7f423c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267f6523d195118c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.393+0000 7f42513d2640 1 -- 172.21.15.60:0/333537904 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c036070 con 0x7f424c15dc20 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.393+0000 7f42513d2640 1 -- 172.21.15.60:0/333537904 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c031600 con 0x7f424c15dc20 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.393+0000 7f42513d2640 1 -- 172.21.15.60:0/333537904 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0319d0 con 0x7f424c15dc20 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 -- 172.21.15.60:0/333537904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c15dc20 msgr2=0x7f424c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 --2- 172.21.15.60:0/333537904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c15dc20 0x7f424c0ce670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f424c0b6910 tx=0x7f423c02a730 comp rx=0 tx=0).stop 2024-04-25T08:29:34.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 -- 172.21.15.60:0/333537904 shutdown_connections 2024-04-25T08:29:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 --2- 172.21.15.60:0/333537904 >> v2:172.21.15.100:3300/0 conn(0x7f424c0d32f0 0x7f424c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 --2- 172.21.15.60:0/333537904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c15dc20 0x7f424c0ce670 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 -- 172.21.15.60:0/333537904 >> 172.21.15.60:0/333537904 conn(0x7f424c0d0420 msgr2=0x7f424c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 -- 172.21.15.60:0/333537904 shutdown_connections 2024-04-25T08:29:34.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.394+0000 7f4254638640 1 -- 172.21.15.60:0/333537904 wait complete. 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.395+0000 7f4254638640 1 Processor -- start 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.395+0000 7f4254638640 1 -- start start 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4254638640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 0x7f424c15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4254638640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424c15dc20 0x7f424c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4254638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f424c161710 0x7f424c1f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4254638640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0b5e50 con 0x7f424c0d32f0 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4254638640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c162500 con 0x7f424c15dc20 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4254638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0b5fd0 con 0x7f424c161710 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 0x7f424c15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 0x7f424c15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43562/0 (socket says 172.21.15.60:43562) 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 -- 172.21.15.60:0/3469749808 learned_addr learned my addr 172.21.15.60:0/3469749808 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:34.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4251bd3640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424c15dc20 0x7f424c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4252bd5640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f424c161710 0x7f424c1f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 -- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f424c161710 msgr2=0x7f424c1f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f424c161710 0x7f424c1f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 -- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424c15dc20 msgr2=0x7f424c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424c15dc20 0x7f424c15f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 -- 172.21.15.60:0/3469749808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c160130 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f4251bd3640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424c15dc20 0x7f424c15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.396+0000 7f42523d4640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 0x7f424c15d040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f423c009f40 tx=0x7f423c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.397+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c03f020 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.397+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c037040 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.397+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c15fb70 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.397+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c036070 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.397+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c0aa8f0 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.398+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c073a00 con 0x7f424c0d32f0 2024-04-25T08:29:34.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.398+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f423c042020 con 0x7f424c0d32f0 2024-04-25T08:29:34.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.399+0000 7f424affd640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f423806b500 0x7f423806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.399+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f423c0aa8e0 con 0x7f424c0d32f0 2024-04-25T08:29:34.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.399+0000 7f4251bd3640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f423806b500 0x7f423806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.400+0000 7f4251bd3640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f423806b500 0x7f423806d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4240000f60 tx=0x7f4240009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.403+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f423c049030 con 0x7f424c0d32f0 2024-04-25T08:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.584+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f424c0b35d0 con 0x7f424c0d32f0 2024-04-25T08:29:34.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.585+0000 7f424affd640 1 -- 172.21.15.60:0/3469749808 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f423c07de20 con 0x7f424c0d32f0 2024-04-25T08:29:34.583 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:29:34.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f423806b500 msgr2=0x7f423806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f423806b500 0x7f423806d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4240000f60 tx=0x7f4240009040 comp rx=0 tx=0).stop 2024-04-25T08:29:34.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 msgr2=0x7f424c15d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 0x7f424c15d040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f423c009f40 tx=0x7f423c004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 shutdown_connections 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f424c161710 0x7f424c1f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f423806b500 0x7f423806d9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424c15dc20 0x7f424c15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 --2- 172.21.15.60:0/3469749808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424c0d32f0 0x7f424c15d040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 >> 172.21.15.60:0/3469749808 conn(0x7f424c0d0420 msgr2=0x7f424c0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 shutdown_connections 2024-04-25T08:29:34.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.587+0000 7f4254638640 1 -- 172.21.15.60:0/3469749808 wait complete. 2024-04-25T08:29:34.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:29:34.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.701+0000 7f0b5168f640 1 Processor -- start 2024-04-25T08:29:34.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.701+0000 7f0b5168f640 1 -- start start 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b5168f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b4c161710 0x7f0b4c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b5168f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c15e000 con 0x7f0b4c0ced80 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b5168f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c0d1de0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b5168f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c161c60 con 0x7f0b4c161710 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b4a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0b4c0ced80 0x7f0b4c0d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54368/0 (socket says 172.21.15.60:54368) 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b4a7fc640 1 -- 172.21.15.60:0/2564918867 learned_addr learned my addr 172.21.15.60:0/2564918867 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b4b7fe640 1 --2- 172.21.15.60:0/2564918867 >> v2:172.21.15.100:3300/0 conn(0x7f0b4c161710 0x7f0b4c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632101355 0 0) 0x7f0b4c15e000 con 0x7f0b4c0ced80 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b34003520 con 0x7f0b4c0ced80 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.702+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965007611 0 0) 0x7f0b4c0d1de0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.703+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b4c15e000 con 0x7f0b4c15dc20 2024-04-25T08:29:34.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.703+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/2564918867 >> v1:172.21.15.100:6789/0 conn(0x7f0b4c15dc20 legacy=0x7f0b4c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.703+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/2564918867 >> v1:172.21.15.60:6789/0 conn(0x7f0b4c0ced80 legacy=0x7f0b4c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.703+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1734738108 0 0) 0x7f0b34003520 con 0x7f0b4c0ced80 2024-04-25T08:29:34.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.703+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/2564918867 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c161f40 con 0x7f0b4c161710 2024-04-25T08:29:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.703+0000 7f0b4b7fe640 1 --2- 172.21.15.60:0/2564918867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c161710 0x7f0b4c0d29f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b440097a0 tx=0x7f0b4402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c5c7469c24ca5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44037070 con 0x7f0b4c161710 2024-04-25T08:29:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4402ac50 con 0x7f0b4c161710 2024-04-25T08:29:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b49ffb640 1 -- 172.21.15.60:0/2564918867 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40001670 con 0x7f0b4c161710 2024-04-25T08:29:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 -- 172.21.15.60:0/2564918867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c161710 msgr2=0x7f0b4c0d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 --2- 172.21.15.60:0/2564918867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c161710 0x7f0b4c0d29f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b440097a0 tx=0x7f0b4402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 -- 172.21.15.60:0/2564918867 shutdown_connections 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 --2- 172.21.15.60:0/2564918867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c161710 0x7f0b4c0d29f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 -- 172.21.15.60:0/2564918867 >> 172.21.15.60:0/2564918867 conn(0x7f0b4c0d0420 msgr2=0x7f0b4c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 -- 172.21.15.60:0/2564918867 shutdown_connections 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.704+0000 7f0b5168f640 1 -- 172.21.15.60:0/2564918867 wait complete. 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.705+0000 7f0b5168f640 1 Processor -- start 2024-04-25T08:29:34.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.705+0000 7f0b5168f640 1 -- start start 2024-04-25T08:29:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b5168f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c0ced80 0x7f0b4c14cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b5168f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 0x7f0b4c0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b5168f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b4c161710 0x7f0b4c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b5168f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b5e50 con 0x7f0b4c15dc20 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b5168f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c162500 con 0x7f0b4c0ced80 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b5168f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0b5fd0 con 0x7f0b4c161710 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b4a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 0x7f0b4c0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b4a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 0x7f0b4c0caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43570/0 (socket says 172.21.15.60:43570) 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b4a7fc640 1 -- 172.21.15.60:0/3370636112 learned_addr learned my addr 172.21.15.60:0/3370636112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b4b7fe640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b4c161710 0x7f0b4c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.706+0000 7f0b4affd640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c0ced80 0x7f0b4c14cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4a7fc640 1 -- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b4c161710 msgr2=0x7f0b4c15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4a7fc640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b4c161710 0x7f0b4c15ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4a7fc640 1 -- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c0ced80 msgr2=0x7f0b4c14cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4a7fc640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c0ced80 0x7f0b4c14cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4a7fc640 1 -- 172.21.15.60:0/3370636112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c150700 con 0x7f0b4c15dc20 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4affd640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c0ced80 0x7f0b4c14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b4a7fc640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 0x7f0b4c0caa70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c750 tx=0x7f0b4000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40005ad0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b40005c70 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0cd990 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b40005e60 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.707+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c0cdf50 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.708+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c073a00 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.708+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b400072b0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.709+0000 7f0b337fe640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:34.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.709+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b4008a2a0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.709+0000 7f0b4affd640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:34.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.710+0000 7f0b4affd640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3806b4d0 0x7f0b3806d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c001e10 tx=0x7f0b3c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:34.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.713+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b40058ea0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.892+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0b4c15c660 con 0x7f0b4c15dc20 2024-04-25T08:29:34.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.892+0000 7f0b337fe640 1 -- 172.21.15.60:0/3370636112 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0b400587a0 con 0x7f0b4c15dc20 2024-04-25T08:29:34.891 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3806b4d0 msgr2=0x7f0b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3806b4d0 0x7f0b3806d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c001e10 tx=0x7f0b3c009040 comp rx=0 tx=0).stop 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 msgr2=0x7f0b4c0caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 0x7f0b4c0caa70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b4000c750 tx=0x7f0b4000cc10 comp rx=0 tx=0).stop 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 shutdown_connections 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b4c161710 0x7f0b4c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b4c15dc20 0x7f0b4c0caa70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 --2- 172.21.15.60:0/3370636112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b4c0ced80 0x7f0b4c14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 >> 172.21.15.60:0/3370636112 conn(0x7f0b4c0d0420 msgr2=0x7f0b4c0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 shutdown_connections 2024-04-25T08:29:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:34.894+0000 7f0b5168f640 1 -- 172.21.15.60:0/3370636112 wait complete. 2024-04-25T08:29:34.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:29:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.006+0000 7fe14899e640 1 Processor -- start 2024-04-25T08:29:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.006+0000 7fe14899e640 1 -- start start 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14899e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe14415bb50 0x7fe1440c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14899e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440c75e0 con 0x7fe1440c70d0 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14899e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440c78c0 con 0x7fe1440cfd10 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14899e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440c8680 con 0x7fe14415bb50 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe14415bb50 0x7fe1440c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe14415bb50 0x7fe1440c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43586/0 (socket says 172.21.15.60:43586) 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe14259c640 1 -- 172.21.15.60:0/4085943636 learned_addr learned my addr 172.21.15.60:0/4085943636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.007+0000 7fe141d9b640 1 --1- 172.21.15.60:0/4085943636 >> v1:172.21.15.60:6789/0 conn(0x7fe1440c70d0 0x7fe1440cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54370/0 (socket says 172.21.15.60:54370) 2024-04-25T08:29:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.008+0000 7fe14259c640 1 -- 172.21.15.60:0/4085943636 >> v1:172.21.15.100:6789/0 conn(0x7fe1440cfd10 legacy=0x7fe1440cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.008+0000 7fe14259c640 1 -- 172.21.15.60:0/4085943636 >> v1:172.21.15.60:6789/0 conn(0x7fe1440c70d0 legacy=0x7fe1440cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.008+0000 7fe14259c640 1 -- 172.21.15.60:0/4085943636 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440c7ba0 con 0x7fe14415bb50 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.008+0000 7fe14259c640 1 --2- 172.21.15.60:0/4085943636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe14415bb50 0x7fe1440c6b90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009870 tx=0x7fe13c02a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2a1c5de6b64f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14159a640 1 -- 172.21.15.60:0/4085943636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c037070 con 0x7fe14415bb50 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14159a640 1 -- 172.21.15.60:0/4085943636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c032640 con 0x7fe14415bb50 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14159a640 1 -- 172.21.15.60:0/4085943636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c032a10 con 0x7fe14415bb50 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 -- 172.21.15.60:0/4085943636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe14415bb50 msgr2=0x7fe1440c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 --2- 172.21.15.60:0/4085943636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe14415bb50 0x7fe1440c6b90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009870 tx=0x7fe13c02a3c0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 -- 172.21.15.60:0/4085943636 shutdown_connections 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 --2- 172.21.15.60:0/4085943636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe14415bb50 0x7fe1440c6b90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 -- 172.21.15.60:0/4085943636 >> 172.21.15.60:0/4085943636 conn(0x7fe14406f500 msgr2=0x7fe14406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 -- 172.21.15.60:0/4085943636 shutdown_connections 2024-04-25T08:29:35.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.009+0000 7fe14899e640 1 -- 172.21.15.60:0/4085943636 wait complete. 2024-04-25T08:29:35.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 Processor -- start 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 -- start start 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 0x7fe144157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1440cfd10 0x7fe14415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe14415bb50 0x7fe1441f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415f6d0 con 0x7fe1440c70d0 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440c8160 con 0x7fe14415bb50 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14899e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14415f850 con 0x7fe1440cfd10 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 0x7fe144157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 0x7fe144157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43602/0 (socket says 172.21.15.60:43602) 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.011+0000 7fe14259c640 1 -- 172.21.15.60:0/609261725 learned_addr learned my addr 172.21.15.60:0/609261725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe142d9d640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe14415bb50 0x7fe1441f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe141d9b640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1440cfd10 0x7fe14415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14259c640 1 -- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1440cfd10 msgr2=0x7fe14415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14259c640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1440cfd10 0x7fe14415adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14259c640 1 -- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe14415bb50 msgr2=0x7fe1441f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14259c640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe14415bb50 0x7fe1441f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14259c640 1 -- 172.21.15.60:0/609261725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14415da20 con 0x7fe1440c70d0 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14259c640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 0x7fe144157bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe13c000c00 tx=0x7fe13c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c037040 con 0x7fe1440c70d0 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c042070 con 0x7fe1440c70d0 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe144153dd0 con 0x7fe1440c70d0 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c030bd0 con 0x7fe1440c70d0 2024-04-25T08:29:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.012+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14415dfc0 con 0x7fe1440c70d0 2024-04-25T08:29:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.013+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe144073a00 con 0x7fe1440c70d0 2024-04-25T08:29:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.014+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe13c032500 con 0x7fe1440c70d0 2024-04-25T08:29:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.014+0000 7fe13affd640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe12406b500 0x7fe12406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.014+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe13c0ab010 con 0x7fe1440c70d0 2024-04-25T08:29:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.014+0000 7fe141d9b640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe12406b500 0x7fe12406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.015+0000 7fe141d9b640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe12406b500 0x7fe12406d9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1440b7850 tx=0x7fe12c007c80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.019+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe13c045360 con 0x7fe1440c70d0 2024-04-25T08:29:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.214+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe14415b300 con 0x7fe1440c70d0 2024-04-25T08:29:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.215+0000 7fe13affd640 1 -- 172.21.15.60:0/609261725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe13c079be0 con 0x7fe1440c70d0 2024-04-25T08:29:35.213 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe12406b500 msgr2=0x7fe12406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe12406b500 0x7fe12406d9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1440b7850 tx=0x7fe12c007c80 comp rx=0 tx=0).stop 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 msgr2=0x7fe144157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 0x7fe144157bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe13c000c00 tx=0x7fe13c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 shutdown_connections 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe12406b500 0x7fe12406d9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe14415bb50 0x7fe1441f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1440cfd10 0x7fe14415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 --2- 172.21.15.60:0/609261725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1440c70d0 0x7fe144157bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 >> 172.21.15.60:0/609261725 conn(0x7fe14406f500 msgr2=0x7fe1440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:35.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 shutdown_connections 2024-04-25T08:29:35.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.217+0000 7fe14899e640 1 -- 172.21.15.60:0/609261725 wait complete. 2024-04-25T08:29:35.225 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:29:35.225 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:29:35.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:35.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.383+0000 7f301c6ee640 1 Processor -- start 2024-04-25T08:29:35.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.383+0000 7f301c6ee640 1 -- start start 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301c6ee640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f301415dbe0 0x7f30140d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301c6ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b92a0 0x7f30140d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301c6ee640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140b97a0 con 0x7f30140b2cb0 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301c6ee640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301415dfc0 con 0x7f301415dbe0 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301c6ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30140d1a00 con 0x7f30140b92a0 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301a48a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f301415dbe0 0x7f30140d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f3019c89640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f30140b2cb0 0x7f30140b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54384/0 (socket says 172.21.15.60:54384) 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f3019c89640 1 -- 172.21.15.60:0/2881158387 learned_addr learned my addr 172.21.15.60:0/2881158387 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301ac8b640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b92a0 0x7f30140d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301a48a640 1 -- 172.21.15.60:0/2881158387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b92a0 msgr2=0x7f30140d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301a48a640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b92a0 0x7f30140d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301a48a640 1 -- 172.21.15.60:0/2881158387 >> v1:172.21.15.60:6789/0 conn(0x7f30140b2cb0 legacy=0x7f30140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f301a48a640 1 -- 172.21.15.60:0/2881158387 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140d1c50 con 0x7f301415dbe0 2024-04-25T08:29:35.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.384+0000 7f3019488640 1 -- 172.21.15.60:0/2881158387 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892416564 0 0) 0x7f30140b97a0 con 0x7f30140b2cb0 2024-04-25T08:29:35.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.385+0000 7f301ac8b640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b92a0 0x7f30140d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.385+0000 7f301a48a640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f301415dbe0 0x7f30140d0910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f30100097a0 tx=0x7f301002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fbeab8ebeee0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.385+0000 7f3019488640 1 -- 172.21.15.60:0/2881158387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010037070 con 0x7f301415dbe0 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.385+0000 7f3019488640 1 -- 172.21.15.60:0/2881158387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3010032440 con 0x7f301415dbe0 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.385+0000 7f3019488640 1 -- 172.21.15.60:0/2881158387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010032810 con 0x7f301415dbe0 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 -- 172.21.15.60:0/2881158387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f301415dbe0 msgr2=0x7f30140d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f301415dbe0 0x7f30140d0910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f30100097a0 tx=0x7f301002a180 comp rx=0 tx=0).stop 2024-04-25T08:29:35.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 -- 172.21.15.60:0/2881158387 shutdown_connections 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b92a0 0x7f30140d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 --2- 172.21.15.60:0/2881158387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f301415dbe0 0x7f30140d0910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 -- 172.21.15.60:0/2881158387 >> 172.21.15.60:0/2881158387 conn(0x7f3014061a20 msgr2=0x7f3014061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 -- 172.21.15.60:0/2881158387 shutdown_connections 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.386+0000 7f301c6ee640 1 -- 172.21.15.60:0/2881158387 wait complete. 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.387+0000 7f301c6ee640 1 Processor -- start 2024-04-25T08:29:35.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.387+0000 7f301c6ee640 1 -- start start 2024-04-25T08:29:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b2cb0 0x7f30140b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 0x7f301415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f301415dbe0 0x7f30140b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140b6ab0 con 0x7f30140b92a0 2024-04-25T08:29:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140d2210 con 0x7f301415dbe0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30140b6c30 con 0x7f30140b2cb0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 0x7f301415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 0x7f301415f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43620/0 (socket says 172.21.15.60:43620) 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 -- 172.21.15.60:0/3282325507 learned_addr learned my addr 172.21.15.60:0/3282325507 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301a48a640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b2cb0 0x7f30140b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301ac8b640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f301415dbe0 0x7f30140b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 -- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b2cb0 msgr2=0x7f30140b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b2cb0 0x7f30140b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 -- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f301415dbe0 msgr2=0x7f30140b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f301415dbe0 0x7f30140b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 -- 172.21.15.60:0/3282325507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140b3c10 con 0x7f30140b92a0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f3019c89640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 0x7f301415f7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f300400c4b0 tx=0x7f300400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004018070 con 0x7f30140b92a0 2024-04-25T08:29:35.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3004007500 con 0x7f30140b92a0 2024-04-25T08:29:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30140b3ef0 con 0x7f30140b92a0 2024-04-25T08:29:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004004a20 con 0x7f30140b92a0 2024-04-25T08:29:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.389+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30140b44b0 con 0x7f30140b92a0 2024-04-25T08:29:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.390+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3014073990 con 0x7f30140b92a0 2024-04-25T08:29:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.390+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f300401b070 con 0x7f30140b92a0 2024-04-25T08:29:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.391+0000 7f300affd640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.391+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f30040897f0 con 0x7f30140b92a0 2024-04-25T08:29:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.391+0000 7f301a48a640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.392+0000 7f301a48a640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ff806b4d0 0x7f2ff806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3010034000 tx=0x7f301002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.396+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3004057da0 con 0x7f30140b92a0 2024-04-25T08:29:35.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.577+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30140b2820 con 0x7f30140b92a0 2024-04-25T08:29:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.578+0000 7f300affd640 1 -- 172.21.15.60:0/3282325507 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f300405be40 con 0x7f30140b92a0 2024-04-25T08:29:35.577 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:35.577 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ff806b4d0 msgr2=0x7f2ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ff806b4d0 0x7f2ff806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3010034000 tx=0x7f301002e000 comp rx=0 tx=0).stop 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 msgr2=0x7f301415f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 0x7f301415f7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f300400c4b0 tx=0x7f300400c970 comp rx=0 tx=0).stop 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 shutdown_connections 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ff806b4d0 0x7f2ff806d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f301415dbe0 0x7f30140b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30140b92a0 0x7f301415f7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 --2- 172.21.15.60:0/3282325507 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30140b2cb0 0x7f30140b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 >> 172.21.15.60:0/3282325507 conn(0x7f3014061a20 msgr2=0x7f301415b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.580+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 shutdown_connections 2024-04-25T08:29:35.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.581+0000 7f301c6ee640 1 -- 172.21.15.60:0/3282325507 wait complete. 2024-04-25T08:29:35.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:29:35.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 Processor -- start 2024-04-25T08:29:35.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 -- start start 2024-04-25T08:29:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d815b7d0 0x7f83d80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83d806c9e0 0x7f83d80b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d806cf50 con 0x7f83d80b5620 2024-04-25T08:29:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d80b5fc0 con 0x7f83d806c9e0 2024-04-25T08:29:35.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83dd676640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80ce530 con 0x7f83d815b7d0 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83d806c9e0 0x7f83d80b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83cffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f83d80b5620 0x7f83d806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54392/0 (socket says 172.21.15.60:54392) 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83d806c9e0 0x7f83d80b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43628/0 (socket says 172.21.15.60:43628) 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83cffff640 1 -- 172.21.15.60:0/12353277 learned_addr learned my addr 172.21.15.60:0/12353277 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d77fe640 1 -- 172.21.15.60:0/12353277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d815b7d0 msgr2=0x7f83d80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d77fe640 1 --2- 172.21.15.60:0/12353277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d815b7d0 0x7f83d80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d77fe640 1 -- 172.21.15.60:0/12353277 >> v1:172.21.15.60:6789/0 conn(0x7f83d80b5620 legacy=0x7f83d806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d77fe640 1 -- 172.21.15.60:0/12353277 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80ce810 con 0x7f83d806c9e0 2024-04-25T08:29:35.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.697+0000 7f83d67fc640 1 -- 172.21.15.60:0/12353277 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120019347 0 0) 0x7f83d806cf50 con 0x7f83d80b5620 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.698+0000 7f83d77fe640 1 --2- 172.21.15.60:0/12353277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d806c9e0 0x7f83d80b5a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009d90 tx=0x7f83d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d16d1b6b5eaba65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.698+0000 7f83d67fc640 1 -- 172.21.15.60:0/12353277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0007740 con 0x7f83d806c9e0 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.698+0000 7f83d67fc640 1 -- 172.21.15.60:0/12353277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d00078e0 con 0x7f83d806c9e0 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.698+0000 7f83d67fc640 1 -- 172.21.15.60:0/12353277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0007cb0 con 0x7f83d806c9e0 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.698+0000 7f83dd676640 1 -- 172.21.15.60:0/12353277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d806c9e0 msgr2=0x7f83d80b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.698+0000 7f83dd676640 1 --2- 172.21.15.60:0/12353277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d806c9e0 0x7f83d80b5a80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009d90 tx=0x7f83d002a660 comp rx=0 tx=0).stop 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- 172.21.15.60:0/12353277 shutdown_connections 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 --2- 172.21.15.60:0/12353277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d806c9e0 0x7f83d80b5a80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 --2- 172.21.15.60:0/12353277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d815b7d0 0x7f83d80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- 172.21.15.60:0/12353277 >> 172.21.15.60:0/12353277 conn(0x7f83d80d0800 msgr2=0x7f83d80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- 172.21.15.60:0/12353277 shutdown_connections 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- 172.21.15.60:0/12353277 wait complete. 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 Processor -- start 2024-04-25T08:29:35.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- start start 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d806c9e0 0x7f83d81569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83d80b5620 0x7f83d80cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 0x7f83d81f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80cf550 con 0x7f83d815b7d0 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d806d230 con 0x7f83d80b5620 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.700+0000 7f83dd676640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80cf6d0 con 0x7f83d806c9e0 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 0x7f83d81f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 0x7f83d81f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43642/0 (socket says 172.21.15.60:43642) 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 -- 172.21.15.60:0/197878103 learned_addr learned my addr 172.21.15.60:0/197878103 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83cffff640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83d80b5620 0x7f83d80cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d6ffd640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d806c9e0 0x7f83d81569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 -- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d806c9e0 msgr2=0x7f83d81569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d806c9e0 0x7f83d81569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 -- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83d80b5620 msgr2=0x7f83d80cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83d80b5620 0x7f83d80cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 -- 172.21.15.60:0/197878103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80cb9d0 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83cffff640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83d80b5620 0x7f83d80cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83d77fe640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 0x7f83d81f6920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83d002ac50 tx=0x7f83d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0040070 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d0031bc0 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d80cce20 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0045460 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.701+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d81f75e0 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.702+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d8073a00 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.703+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83d0039040 con 0x7f83d815b7d0 2024-04-25T08:29:35.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.703+0000 7f83cf7fe640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83b806b4d0 0x7f83b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:35.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.703+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f83d00abf10 con 0x7f83d815b7d0 2024-04-25T08:29:35.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.704+0000 7f83d6ffd640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83b806b4d0 0x7f83b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:35.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.704+0000 7f83d6ffd640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83b806b4d0 0x7f83b806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b810 tx=0x7f83c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:35.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.709+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83d007a4c0 con 0x7f83d815b7d0 2024-04-25T08:29:35.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.888+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83d80c7270 con 0x7f83d815b7d0 2024-04-25T08:29:35.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.888+0000 7f83cf7fe640 1 -- 172.21.15.60:0/197878103 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83d007e560 con 0x7f83d815b7d0 2024-04-25T08:29:35.887 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83b806b4d0 msgr2=0x7f83b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83b806b4d0 0x7f83b806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83c800b810 tx=0x7f83c8008040 comp rx=0 tx=0).stop 2024-04-25T08:29:35.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 msgr2=0x7f83d81f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 0x7f83d81f6920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f83d002ac50 tx=0x7f83d0004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 shutdown_connections 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83d815b7d0 0x7f83d81f6920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83b806b4d0 0x7f83b806d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83d80b5620 0x7f83d80cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 --2- 172.21.15.60:0/197878103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83d806c9e0 0x7f83d81569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 >> 172.21.15.60:0/197878103 conn(0x7f83d80d0800 msgr2=0x7f83d80d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 shutdown_connections 2024-04-25T08:29:35.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:35.890+0000 7f83dd676640 1 -- 172.21.15.60:0/197878103 wait complete. 2024-04-25T08:29:35.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:29:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.004+0000 7f35275b0640 1 Processor -- start 2024-04-25T08:29:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.004+0000 7f35275b0640 1 -- start start 2024-04-25T08:29:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f35275b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f352015dc20 0x7f35200d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f35275b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35200d1ce0 0x7f35200b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f35275b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f35200ce670 0x7f35200ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f35275b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200d2120 con 0x7f352015dc20 2024-04-25T08:29:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f35275b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200d23a0 con 0x7f35200ce670 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f35275b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200d2680 con 0x7f35200d1ce0 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f352015dc20 0x7f35200d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f352015dc20 0x7f35200d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43646/0 (socket says 172.21.15.60:43646) 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 -- 172.21.15.60:0/1396261096 learned_addr learned my addr 172.21.15.60:0/1396261096 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f3524b4b640 1 --2- 172.21.15.60:0/1396261096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35200d1ce0 0x7f35200b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f3525b4d640 1 --2- 172.21.15.60:0/1396261096 >> v2:172.21.15.100:3300/0 conn(0x7f35200ce670 0x7f35200ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 -- 172.21.15.60:0/1396261096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35200d1ce0 msgr2=0x7f35200b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 --2- 172.21.15.60:0/1396261096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35200d1ce0 0x7f35200b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 -- 172.21.15.60:0/1396261096 >> v2:172.21.15.100:3300/0 conn(0x7f35200ce670 msgr2=0x7f35200ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 --2- 172.21.15.60:0/1396261096 >> v2:172.21.15.100:3300/0 conn(0x7f35200ce670 0x7f35200ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.005+0000 7f352534c640 1 -- 172.21.15.60:0/1396261096 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200d2960 con 0x7f352015dc20 2024-04-25T08:29:36.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.006+0000 7f352534c640 1 --2- 172.21.15.60:0/1396261096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352015dc20 0x7f35200d3440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35200b6080 tx=0x7f351802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0b2d81b18615ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.006+0000 7f351ffff640 1 -- 172.21.15.60:0/1396261096 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518037070 con 0x7f352015dc20 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.006+0000 7f351ffff640 1 -- 172.21.15.60:0/1396261096 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518032440 con 0x7f352015dc20 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.006+0000 7f351ffff640 1 -- 172.21.15.60:0/1396261096 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518032810 con 0x7f352015dc20 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 -- 172.21.15.60:0/1396261096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352015dc20 msgr2=0x7f35200d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 --2- 172.21.15.60:0/1396261096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352015dc20 0x7f35200d3440 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35200b6080 tx=0x7f351802a140 comp rx=0 tx=0).stop 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 -- 172.21.15.60:0/1396261096 shutdown_connections 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 --2- 172.21.15.60:0/1396261096 >> v2:172.21.15.100:3300/0 conn(0x7f35200ce670 0x7f35200ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 --2- 172.21.15.60:0/1396261096 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35200d1ce0 0x7f35200b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 --2- 172.21.15.60:0/1396261096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352015dc20 0x7f35200d3440 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 -- 172.21.15.60:0/1396261096 >> 172.21.15.60:0/1396261096 conn(0x7f3520061a20 msgr2=0x7f3520061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 -- 172.21.15.60:0/1396261096 shutdown_connections 2024-04-25T08:29:36.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.007+0000 7f35275b0640 1 -- 172.21.15.60:0/1396261096 wait complete. 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.008+0000 7f35275b0640 1 Processor -- start 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.008+0000 7f35275b0640 1 -- start start 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f35275b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35200ce670 0x7f35200b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f35275b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 0x7f35200c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f35275b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352015dc20 0x7f352015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f35275b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352006c120 con 0x7f35200d1ce0 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f35275b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200d2f20 con 0x7f35200ce670 2024-04-25T08:29:36.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f35275b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352006c2a0 con 0x7f352015dc20 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 0x7f35200c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 0x7f35200c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43652/0 (socket says 172.21.15.60:43652) 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 -- 172.21.15.60:0/47744936 learned_addr learned my addr 172.21.15.60:0/47744936 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f352534c640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35200ce670 0x7f35200b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3525b4d640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352015dc20 0x7f352015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 -- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352015dc20 msgr2=0x7f352015e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352015dc20 0x7f352015e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 -- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35200ce670 msgr2=0x7f35200b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35200ce670 0x7f35200b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 -- 172.21.15.60:0/47744936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3520156b00 con 0x7f35200d1ce0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.009+0000 7f3524b4b640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 0x7f35200c8460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f351400e9b0 tx=0x7f351400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.010+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514017070 con 0x7f35200d1ce0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.010+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351400c9a0 con 0x7f35200d1ce0 2024-04-25T08:29:36.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.010+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400cd90 con 0x7f35200d1ce0 2024-04-25T08:29:36.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.010+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3520157060 con 0x7f35200d1ce0 2024-04-25T08:29:36.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.010+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3520157620 con 0x7f35200d1ce0 2024-04-25T08:29:36.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.011+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3520073a00 con 0x7f35200d1ce0 2024-04-25T08:29:36.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.011+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f351401a070 con 0x7f35200d1ce0 2024-04-25T08:29:36.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.012+0000 7f351dffb640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f350c06b4d0 0x7f350c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.012+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f351408a4e0 con 0x7f35200d1ce0 2024-04-25T08:29:36.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.012+0000 7f352534c640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f350c06b4d0 0x7f350c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.012+0000 7f352534c640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f350c06b4d0 0x7f350c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3518034000 tx=0x7f351802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.017+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3520156b00 con 0x7f35200d1ce0 2024-04-25T08:29:36.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.201+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35200d0d80 con 0x7f35200d1ce0 2024-04-25T08:29:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.201+0000 7f351dffb640 1 -- 172.21.15.60:0/47744936 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3514058930 con 0x7f35200d1ce0 2024-04-25T08:29:36.199 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:36.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f350c06b4d0 msgr2=0x7f350c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f350c06b4d0 0x7f350c06d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3518034000 tx=0x7f351802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:36.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 msgr2=0x7f35200c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 0x7f35200c8460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f351400e9b0 tx=0x7f351400ee70 comp rx=0 tx=0).stop 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 shutdown_connections 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352015dc20 0x7f352015e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35200d1ce0 0x7f35200c8460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f350c06b4d0 0x7f350c06d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 --2- 172.21.15.60:0/47744936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35200ce670 0x7f35200b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 >> 172.21.15.60:0/47744936 conn(0x7f3520061a20 msgr2=0x7f35200c74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 shutdown_connections 2024-04-25T08:29:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.203+0000 7f35275b0640 1 -- 172.21.15.60:0/47744936 wait complete. 2024-04-25T08:29:36.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:29:36.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 Processor -- start 2024-04-25T08:29:36.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 -- start start 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7bb4161240 0x7f7bb40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 0x7f7bb40ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb4161620 con 0x7f7bb40b54a0 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40b7120 con 0x7f7bb4161240 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb8f0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4161e00 con 0x7f7bb4161890 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7bb4161240 0x7f7bb40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7babfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 0x7f7bb40ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7bb4161240 0x7f7bb40b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:51494/0 (socket says 172.21.15.60:51494) 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb2d9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7bb40b54a0 0x7f7bb40d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:42470/0 (socket says 172.21.15.60:42470) 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7babfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 0x7f7bb40ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43984/0 (socket says 172.21.15.60:43984) 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 -- 172.21.15.60:0/1371755645 learned_addr learned my addr 172.21.15.60:0/1371755645 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 -- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 msgr2=0x7f7bb40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 --2- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 0x7f7bb40ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 -- 172.21.15.60:0/1371755645 >> v1:172.21.15.100:6789/0 conn(0x7f7bb40b54a0 legacy=0x7f7bb40d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7bb259c640 1 -- 172.21.15.60:0/1371755645 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb41620e0 con 0x7f7bb4161240 2024-04-25T08:29:36.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.319+0000 7f7babfff640 1 --2- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 0x7f7bb40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.320+0000 7f7bb259c640 1 --2- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161240 0x7f7bb40b6be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009870 tx=0x7f7ba402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59d4bdadd4b73ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.320+0000 7f7bb1d9b640 1 -- 172.21.15.60:0/1371755645 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4037070 con 0x7f7bb4161240 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.320+0000 7f7bb1d9b640 1 -- 172.21.15.60:0/1371755645 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4032440 con 0x7f7bb4161240 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.320+0000 7f7bb1d9b640 1 -- 172.21.15.60:0/1371755645 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4032810 con 0x7f7bb4161240 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161240 msgr2=0x7f7bb40b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161240 0x7f7bb40b6be0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009870 tx=0x7f7ba402a270 comp rx=0 tx=0).stop 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1371755645 shutdown_connections 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161890 0x7f7bb40ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1371755645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161240 0x7f7bb40b6be0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1371755645 >> 172.21.15.60:0/1371755645 conn(0x7f7bb40d04a0 msgr2=0x7f7bb40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1371755645 shutdown_connections 2024-04-25T08:29:36.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.321+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1371755645 wait complete. 2024-04-25T08:29:36.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.322+0000 7f7bb8f0a640 1 Processor -- start 2024-04-25T08:29:36.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.322+0000 7f7bb8f0a640 1 -- start start 2024-04-25T08:29:36.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb8f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 0x7f7bb4154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb8f0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161240 0x7f7bb40d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb8f0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161890 0x7f7bb40d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb8f0a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40d15f0 con 0x7f7bb40b54a0 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb8f0a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb41626a0 con 0x7f7bb4161890 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb8f0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40cf000 con 0x7f7bb4161240 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 0x7f7bb4154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 0x7f7bb4154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43668/0 (socket says 172.21.15.60:43668) 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 -- 172.21.15.60:0/1196207243 learned_addr learned my addr 172.21.15.60:0/1196207243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb2d9d640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161890 0x7f7bb40d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7babfff640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161240 0x7f7bb40d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 -- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161240 msgr2=0x7f7bb40d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161240 0x7f7bb40d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 -- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161890 msgr2=0x7f7bb40d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161890 0x7f7bb40d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 -- 172.21.15.60:0/1196207243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40c75f0 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.323+0000 7f7bb259c640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 0x7f7bb4154050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009870 tx=0x7f7ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.324+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4041070 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.324+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba403c890 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.324+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4037040 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.324+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb40cc660 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.324+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb40cc940 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.325+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb4073a00 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.325+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ba4044070 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.325+0000 7f7baaffd640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b9406b4d0 0x7f7b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.326+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7ba40aae70 con 0x7f7bb40b54a0 2024-04-25T08:29:36.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.326+0000 7f7babfff640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b9406b4d0 0x7f7b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.326+0000 7f7babfff640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b9406b4d0 0x7f7b9406d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009560 tx=0x7f7b9c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.331+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ba4079480 con 0x7f7bb40b54a0 2024-04-25T08:29:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.508+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7bb415c670 con 0x7f7bb40b54a0 2024-04-25T08:29:36.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.508+0000 7f7baaffd640 1 -- 172.21.15.60:0/1196207243 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ba407d520 con 0x7f7bb40b54a0 2024-04-25T08:29:36.507 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:29:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b9406b4d0 msgr2=0x7f7b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b9406b4d0 0x7f7b9406d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009560 tx=0x7f7b9c009190 comp rx=0 tx=0).stop 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 msgr2=0x7f7bb4154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 0x7f7bb4154050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009870 tx=0x7f7ba40046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 shutdown_connections 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b9406b4d0 0x7f7b9406d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bb4161890 0x7f7bb40d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bb4161240 0x7f7bb40d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 --2- 172.21.15.60:0/1196207243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bb40b54a0 0x7f7bb4154050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 >> 172.21.15.60:0/1196207243 conn(0x7f7bb40d04a0 msgr2=0x7f7bb415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 shutdown_connections 2024-04-25T08:29:36.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.511+0000 7f7bb8f0a640 1 -- 172.21.15.60:0/1196207243 wait complete. 2024-04-25T08:29:36.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:29:36.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.625+0000 7f6ba8ab6640 1 Processor -- start 2024-04-25T08:29:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.625+0000 7f6ba8ab6640 1 -- start start 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.626+0000 7f6ba8ab6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ba415bb20 0x7f6ba40cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.626+0000 7f6ba8ab6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ba40ccdd0 0x7f6ba40cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.626+0000 7f6ba8ab6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba40cb190 con 0x7f6ba40cdcd0 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.626+0000 7f6ba8ab6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba40cb470 con 0x7f6ba415bb20 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.626+0000 7f6ba8ab6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba40cb750 con 0x7f6ba40ccdd0 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ba415bb20 0x7f6ba40cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ba415bb20 0x7f6ba40cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43674/0 (socket says 172.21.15.60:43674) 2024-04-25T08:29:36.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 -- 172.21.15.60:0/2204305272 learned_addr learned my addr 172.21.15.60:0/2204305272 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba1d9b640 1 --2- 172.21.15.60:0/2204305272 >> v2:172.21.15.100:3300/0 conn(0x7f6ba40ccdd0 0x7f6ba40cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 -- 172.21.15.60:0/2204305272 >> v2:172.21.15.100:3300/0 conn(0x7f6ba40ccdd0 msgr2=0x7f6ba40cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:36.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 --2- 172.21.15.60:0/2204305272 >> v2:172.21.15.100:3300/0 conn(0x7f6ba40ccdd0 0x7f6ba40cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 -- 172.21.15.60:0/2204305272 >> v1:172.21.15.100:6789/0 conn(0x7f6ba40cdcd0 legacy=0x7f6ba40ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.627+0000 7f6ba259c640 1 -- 172.21.15.60:0/2204305272 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40cba30 con 0x7f6ba415bb20 2024-04-25T08:29:36.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba259c640 1 --2- 172.21.15.60:0/2204305272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba40cc230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4071680 tx=0x7f6b9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc47476b662a7c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba159a640 1 -- 172.21.15.60:0/2204305272 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c037070 con 0x7f6ba415bb20 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba159a640 1 -- 172.21.15.60:0/2204305272 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c032440 con 0x7f6ba415bb20 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba159a640 1 -- 172.21.15.60:0/2204305272 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c032810 con 0x7f6ba415bb20 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/2204305272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 msgr2=0x7f6ba40cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/2204305272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba40cc230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4071680 tx=0x7f6b9c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/2204305272 shutdown_connections 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/2204305272 >> v2:172.21.15.100:3300/0 conn(0x7f6ba40ccdd0 0x7f6ba40cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/2204305272 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba40cc230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/2204305272 >> 172.21.15.60:0/2204305272 conn(0x7f6ba40b2390 msgr2=0x7f6ba40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/2204305272 shutdown_connections 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.628+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/2204305272 wait complete. 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.630+0000 7f6ba8ab6640 1 Processor -- start 2024-04-25T08:29:36.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.630+0000 7f6ba8ab6640 1 -- start start 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.630+0000 7f6ba8ab6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ba40ccdd0 0x7f6ba414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba40cdcd0 0x7f6ba40c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba4151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba415bf00 con 0x7f6ba415bb20 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40cc770 con 0x7f6ba40cdcd0 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba415c080 con 0x7f6ba40ccdd0 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba4151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba4151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43684/0 (socket says 172.21.15.60:43684) 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 -- 172.21.15.60:0/1475956592 learned_addr learned my addr 172.21.15.60:0/1475956592 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba1d9b640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba40cdcd0 0x7f6ba40c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba259c640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ba40ccdd0 0x7f6ba414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 -- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ba40ccdd0 msgr2=0x7f6ba414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ba40ccdd0 0x7f6ba414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 -- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba40cdcd0 msgr2=0x7f6ba40c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba40cdcd0 0x7f6ba40c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 -- 172.21.15.60:0/1475956592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40c6050 con 0x7f6ba415bb20 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba1d9b640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba40cdcd0 0x7f6ba40c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba259c640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ba40ccdd0 0x7f6ba414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba2d9d640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba4151d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c7a0 tx=0x7f6b9400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94003b70 con 0x7f6ba415bb20 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b94003d30 con 0x7f6ba415bb20 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba40c6330 con 0x7f6ba415bb20 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400aa90 con 0x7f6ba415bb20 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.631+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba40c68f0 con 0x7f6ba415bb20 2024-04-25T08:29:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.632+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba4073a00 con 0x7f6ba415bb20 2024-04-25T08:29:36.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.633+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b9400ac30 con 0x7f6ba415bb20 2024-04-25T08:29:36.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.633+0000 7f6b9affd640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b8406b4d0 0x7f6b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.633+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6b9408a490 con 0x7f6ba415bb20 2024-04-25T08:29:36.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.634+0000 7f6ba259c640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b8406b4d0 0x7f6b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.634+0000 7f6ba259c640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b8406b4d0 0x7f6b8406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c034000 tx=0x7f6b9c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.638+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b94058fd0 con 0x7f6ba415bb20 2024-04-25T08:29:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.817+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6ba40d0200 con 0x7f6ba415bb20 2024-04-25T08:29:36.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.817+0000 7f6b9affd640 1 -- 172.21.15.60:0/1475956592 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6b940588d0 con 0x7f6ba415bb20 2024-04-25T08:29:36.814 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b8406b4d0 msgr2=0x7f6b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b8406b4d0 0x7f6b8406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c034000 tx=0x7f6b9c02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 msgr2=0x7f6ba4151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba4151d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c7a0 tx=0x7f6b9400cc60 comp rx=0 tx=0).stop 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 shutdown_connections 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba415bb20 0x7f6ba4151d60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b8406b4d0 0x7f6b8406d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba40cdcd0 0x7f6ba40c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 --2- 172.21.15.60:0/1475956592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ba40ccdd0 0x7f6ba414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 >> 172.21.15.60:0/1475956592 conn(0x7f6ba40b2390 msgr2=0x7f6ba406f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 shutdown_connections 2024-04-25T08:29:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.819+0000 7f6ba8ab6640 1 -- 172.21.15.60:0/1475956592 wait complete. 2024-04-25T08:29:36.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.932+0000 7f602081f640 1 Processor -- start 2024-04-25T08:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.932+0000 7f602081f640 1 -- start start 2024-04-25T08:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f602081f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f601c15dc20 0x7f601c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f602081f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f601c0d32f0 0x7f601c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f602081f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601c15e000 con 0x7f601c161710 2024-04-25T08:29:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f602081f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c0d1de0 con 0x7f601c15dc20 2024-04-25T08:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f602081f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c161c60 con 0x7f601c0d32f0 2024-04-25T08:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f601c15dc20 0x7f601c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f6013fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f601c161710 0x7f601c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54394/0 (socket says 172.21.15.60:54394) 2024-04-25T08:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f6013fff640 1 -- 172.21.15.60:0/3426173476 learned_addr learned my addr 172.21.15.60:0/3426173476 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:36.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f601c15dc20 0x7f601c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43688/0 (socket says 172.21.15.60:43688) 2024-04-25T08:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601ad9d640 1 --2- 172.21.15.60:0/3426173476 >> v2:172.21.15.100:3300/0 conn(0x7f601c0d32f0 0x7f601c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601a59c640 1 -- 172.21.15.60:0/3426173476 >> v2:172.21.15.100:3300/0 conn(0x7f601c0d32f0 msgr2=0x7f601c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601a59c640 1 --2- 172.21.15.60:0/3426173476 >> v2:172.21.15.100:3300/0 conn(0x7f601c0d32f0 0x7f601c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601a59c640 1 -- 172.21.15.60:0/3426173476 >> v1:172.21.15.60:6789/0 conn(0x7f601c161710 legacy=0x7f601c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f601a59c640 1 -- 172.21.15.60:0/3426173476 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c161f40 con 0x7f601c15dc20 2024-04-25T08:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.933+0000 7f6019d9b640 1 -- 172.21.15.60:0/3426173476 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017946117 0 0) 0x7f601c15e000 con 0x7f601c161710 2024-04-25T08:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.934+0000 7f601a59c640 1 --2- 172.21.15.60:0/3426173476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c15dc20 0x7f601c0ce670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f601c0b6910 tx=0x7f600c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb26d762f9a678a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.934+0000 7f6019d9b640 1 -- 172.21.15.60:0/3426173476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c037070 con 0x7f601c15dc20 2024-04-25T08:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.934+0000 7f6019d9b640 1 -- 172.21.15.60:0/3426173476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c032440 con 0x7f601c15dc20 2024-04-25T08:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.934+0000 7f6019d9b640 1 -- 172.21.15.60:0/3426173476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c032810 con 0x7f601c15dc20 2024-04-25T08:29:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 -- 172.21.15.60:0/3426173476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c15dc20 msgr2=0x7f601c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 --2- 172.21.15.60:0/3426173476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c15dc20 0x7f601c0ce670 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f601c0b6910 tx=0x7f600c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 -- 172.21.15.60:0/3426173476 shutdown_connections 2024-04-25T08:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 --2- 172.21.15.60:0/3426173476 >> v2:172.21.15.100:3300/0 conn(0x7f601c0d32f0 0x7f601c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 --2- 172.21.15.60:0/3426173476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c15dc20 0x7f601c0ce670 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 -- 172.21.15.60:0/3426173476 >> 172.21.15.60:0/3426173476 conn(0x7f601c0d0420 msgr2=0x7f601c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 -- 172.21.15.60:0/3426173476 shutdown_connections 2024-04-25T08:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.935+0000 7f602081f640 1 -- 172.21.15.60:0/3426173476 wait complete. 2024-04-25T08:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.936+0000 7f602081f640 1 Processor -- start 2024-04-25T08:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.936+0000 7f602081f640 1 -- start start 2024-04-25T08:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 0x7f601c1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f601c15dc20 0x7f601c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f601c161710 0x7f601c0ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0b5e50 con 0x7f601c0d32f0 2024-04-25T08:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c162500 con 0x7f601c15dc20 2024-04-25T08:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0b5fd0 con 0x7f601c161710 2024-04-25T08:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 0x7f601c1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 0x7f601c1f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43690/0 (socket says 172.21.15.60:43690) 2024-04-25T08:29:36.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 -- 172.21.15.60:0/347447998 learned_addr learned my addr 172.21.15.60:0/347447998 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f6013fff640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f601c15dc20 0x7f601c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601ad9d640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f601c161710 0x7f601c0ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 -- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f601c161710 msgr2=0x7f601c0ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f601c161710 0x7f601c0ac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 -- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f601c15dc20 msgr2=0x7f601c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f601c15dc20 0x7f601c15ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 -- 172.21.15.60:0/347447998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0ab8e0 con 0x7f601c0d32f0 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f6013fff640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f601c15dc20 0x7f601c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f601a59c640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 0x7f601c1f4100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f600c000c00 tx=0x7f600c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c037050 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c038040 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c0d2dc0 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.937+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c004450 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.938+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c0aab20 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.938+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c073a00 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.939+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f600c03c6d0 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.939+0000 7f6012ffd640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.939+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f600c0aaf80 con 0x7f601c0d32f0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.940+0000 7f6013fff640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.940+0000 7f6013fff640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f601c161210 tx=0x7f6004007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:36.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:36.944+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f600c079ac0 con 0x7f601c0d32f0 2024-04-25T08:29:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.135+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f601c0d16b0 con 0x7f601c0d32f0 2024-04-25T08:29:37.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.135+0000 7f6012ffd640 1 -- 172.21.15.60:0/347447998 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f600c0793c0 con 0x7f601c0d32f0 2024-04-25T08:29:37.133 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ffc06b4d0 msgr2=0x7f5ffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f601c161210 tx=0x7f6004007ca0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 msgr2=0x7f601c1f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 0x7f601c1f4100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f600c000c00 tx=0x7f600c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 shutdown_connections 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f601c161710 0x7f601c0ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f601c15dc20 0x7f601c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 --2- 172.21.15.60:0/347447998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f601c0d32f0 0x7f601c1f4100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 >> 172.21.15.60:0/347447998 conn(0x7f601c0d0420 msgr2=0x7f601c0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 shutdown_connections 2024-04-25T08:29:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.137+0000 7f602081f640 1 -- 172.21.15.60:0/347447998 wait complete. 2024-04-25T08:29:37.145 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi060.front.sepia.ceph.com... 2024-04-25T08:29:37.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:37.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:29:37.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi060.stderr:ceph-fuse[40627]: fuse finished with error 0 and tester_r 0 2024-04-25T08:29:37.171 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:29:37.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:29:37.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:37.206 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T08:29:37.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.418+0000 7f8b870e2640 1 Processor -- start 2024-04-25T08:29:37.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.418+0000 7f8b870e2640 1 -- start start 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.418+0000 7f8b870e2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b8015db60 0x7f8b800ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.418+0000 7f8b870e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800cec00 con 0x7f8b800d0b10 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.418+0000 7f8b870e2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800ceee0 con 0x7f8b800b5f00 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.418+0000 7f8b870e2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b800ce930 con 0x7f8b8015db60 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b84e7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8b8015db60 0x7f8b800ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b8567f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8b800d0b10 0x7f8b800d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54398/0 (socket says 172.21.15.60:54398) 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b8567f640 1 -- 172.21.15.60:0/2985175657 learned_addr learned my addr 172.21.15.60:0/2985175657 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b84e7e640 1 -- 172.21.15.60:0/2985175657 >> v1:172.21.15.100:6789/0 conn(0x7f8b800b5f00 legacy=0x7f8b800d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b84e7e640 1 -- 172.21.15.60:0/2985175657 >> v1:172.21.15.60:6789/0 conn(0x7f8b800d0b10 legacy=0x7f8b800d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b84e7e640 1 -- 172.21.15.60:0/2985175657 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800cf1c0 con 0x7f8b8015db60 2024-04-25T08:29:37.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.419+0000 7f8b7f7fe640 1 -- 172.21.15.60:0/2985175657 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777137417 0 0) 0x7f8b800cec00 con 0x7f8b800d0b10 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b84e7e640 1 --2- 172.21.15.60:0/2985175657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b8015db60 0x7f8b800ce3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b740097d0 tx=0x7f8b7402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aa71529944c3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b7f7fe640 1 -- 172.21.15.60:0/2985175657 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74037070 con 0x7f8b8015db60 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b7f7fe640 1 -- 172.21.15.60:0/2985175657 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74032440 con 0x7f8b8015db60 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b7f7fe640 1 -- 172.21.15.60:0/2985175657 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74032810 con 0x7f8b8015db60 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 -- 172.21.15.60:0/2985175657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b8015db60 msgr2=0x7f8b800ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 --2- 172.21.15.60:0/2985175657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b8015db60 0x7f8b800ce3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b740097d0 tx=0x7f8b7402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 -- 172.21.15.60:0/2985175657 shutdown_connections 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 --2- 172.21.15.60:0/2985175657 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b8015db60 0x7f8b800ce3f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 -- 172.21.15.60:0/2985175657 >> 172.21.15.60:0/2985175657 conn(0x7f8b80061600 msgr2=0x7f8b80061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:37.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 -- 172.21.15.60:0/2985175657 shutdown_connections 2024-04-25T08:29:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.420+0000 7f8b870e2640 1 -- 172.21.15.60:0/2985175657 wait complete. 2024-04-25T08:29:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 Processor -- start 2024-04-25T08:29:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 -- start start 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b800b5f00 0x7f8b8015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 0x7f8b801f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b8015db60 0x7f8b800b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8015df40 con 0x7f8b800d0b10 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b6450 con 0x7f8b8015db60 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.422+0000 7f8b870e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8015e0c0 con 0x7f8b800b5f00 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 0x7f8b801f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 0x7f8b801f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43708/0 (socket says 172.21.15.60:43708) 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 -- 172.21.15.60:0/516634255 learned_addr learned my addr 172.21.15.60:0/516634255 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b84e7e640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b800b5f00 0x7f8b8015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b8567f640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b8015db60 0x7f8b800b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 -- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b800b5f00 msgr2=0x7f8b8015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b800b5f00 0x7f8b8015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 -- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b8015db60 msgr2=0x7f8b800b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b8015db60 0x7f8b800b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 -- 172.21.15.60:0/516634255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800c9840 con 0x7f8b800d0b10 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7ffff640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 0x7f8b801f48d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c480 tx=0x7f8b6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c0052d0 con 0x7f8b800d0b10 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c005470 con 0x7f8b800d0b10 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c012870 con 0x7f8b800d0b10 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b800c9b20 con 0x7f8b800d0b10 2024-04-25T08:29:37.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.423+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b801f43b0 con 0x7f8b800d0b10 2024-04-25T08:29:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.424+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b80073270 con 0x7f8b800d0b10 2024-04-25T08:29:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.425+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b6c012a10 con 0x7f8b800d0b10 2024-04-25T08:29:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.425+0000 7f8b7d7fa640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b6406b4d0 0x7f8b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.425+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b6c08a000 con 0x7f8b800d0b10 2024-04-25T08:29:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.425+0000 7f8b84e7e640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b6406b4d0 0x7f8b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.426+0000 7f8b84e7e640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b6406b4d0 0x7f8b6406d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b74034000 tx=0x7f8b7402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:37.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.430+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b6c058580 con 0x7f8b800d0b10 2024-04-25T08:29:37.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.611+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b801f45b0 con 0x7f8b800d0b10 2024-04-25T08:29:37.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.612+0000 7f8b7d7fa640 1 -- 172.21.15.60:0/516634255 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f8b801f45b0 con 0x7f8b800d0b10 2024-04-25T08:29:37.609 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:37.610 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b6406b4d0 msgr2=0x7f8b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b6406b4d0 0x7f8b6406d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b74034000 tx=0x7f8b7402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 msgr2=0x7f8b801f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 0x7f8b801f48d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00c480 tx=0x7f8b6c00c940 comp rx=0 tx=0).stop 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 shutdown_connections 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b6406b4d0 0x7f8b6406d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b8015db60 0x7f8b800b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b800d0b10 0x7f8b801f48d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 --2- 172.21.15.60:0/516634255 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b800b5f00 0x7f8b8015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 >> 172.21.15.60:0/516634255 conn(0x7f8b80061600 msgr2=0x7f8b800d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 shutdown_connections 2024-04-25T08:29:37.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.614+0000 7f8b870e2640 1 -- 172.21.15.60:0/516634255 wait complete. 2024-04-25T08:29:37.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.726+0000 7f549462a640 1 Processor -- start 2024-04-25T08:29:37.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.726+0000 7f549462a640 1 -- start start 2024-04-25T08:29:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f549462a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f549462a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0b6220 con 0x7f548c0b2cb0 2024-04-25T08:29:37.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f549462a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0b97a0 con 0x7f548c0b92a0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f549462a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c15dfc0 con 0x7f548c15dbe0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f5491bc5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f548c0b2cb0 0x7f548c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54404/0 (socket says 172.21.15.60:54404) 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f5491bc5640 1 -- 172.21.15.60:0/672344241 learned_addr learned my addr 172.21.15.60:0/672344241 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54923c6640 1 --2- 172.21.15.60:0/672344241 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326101185 0 0) 0x7f548c0b6220 con 0x7f548c0b2cb0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5478003520 con 0x7f548c0b2cb0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558693553 0 0) 0x7f548c0b97a0 con 0x7f548c0b92a0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f548c0b6220 con 0x7f548c0b92a0 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54923c6640 1 -- 172.21.15.60:0/672344241 >> v1:172.21.15.100:6789/0 conn(0x7f548c0b92a0 legacy=0x7f548c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54923c6640 1 -- 172.21.15.60:0/672344241 >> v1:172.21.15.60:6789/0 conn(0x7f548c0b2cb0 legacy=0x7f548c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.727+0000 7f54923c6640 1 -- 172.21.15.60:0/672344241 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0d1c50 con 0x7f548c15dbe0 2024-04-25T08:29:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.728+0000 7f54923c6640 1 --2- 172.21.15.60:0/672344241 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0d0910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54880097d0 tx=0x7f548802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f23c01d6bd276d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.728+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488037070 con 0x7f548c15dbe0 2024-04-25T08:29:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.728+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488032440 con 0x7f548c15dbe0 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.728+0000 7f54913c4640 1 -- 172.21.15.60:0/672344241 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880327f0 con 0x7f548c15dbe0 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 -- 172.21.15.60:0/672344241 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 msgr2=0x7f548c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 --2- 172.21.15.60:0/672344241 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0d0910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54880097d0 tx=0x7f548802a210 comp rx=0 tx=0).stop 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 -- 172.21.15.60:0/672344241 shutdown_connections 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 --2- 172.21.15.60:0/672344241 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0d0910 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 -- 172.21.15.60:0/672344241 >> 172.21.15.60:0/672344241 conn(0x7f548c061a20 msgr2=0x7f548c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 -- 172.21.15.60:0/672344241 shutdown_connections 2024-04-25T08:29:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.729+0000 7f549462a640 1 -- 172.21.15.60:0/672344241 wait complete. 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.730+0000 7f549462a640 1 Processor -- start 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.730+0000 7f549462a640 1 -- start start 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f549462a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f548c0b2cb0 0x7f548c1f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f549462a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 0x7f548c15fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f549462a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f549462a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0d2210 con 0x7f548c0b92a0 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f549462a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0d1a00 con 0x7f548c0b2cb0 2024-04-25T08:29:37.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f549462a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0b6ab0 con 0x7f548c15dbe0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 0x7f548c15fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 0x7f548c15fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43724/0 (socket says 172.21.15.60:43724) 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 -- 172.21.15.60:0/147646937 learned_addr learned my addr 172.21.15.60:0/147646937 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f54923c6640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f548c0b2cb0 0x7f548c1f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5492bc7640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 -- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 msgr2=0x7f548c0aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 -- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f548c0b2cb0 msgr2=0x7f548c1f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f548c0b2cb0 0x7f548c1f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.731+0000 7f5491bc5640 1 -- 172.21.15.60:0/147646937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c160730 con 0x7f548c0b92a0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.732+0000 7f5491bc5640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 0x7f548c15fce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f547c00c500 tx=0x7f547c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.732+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c017070 con 0x7f548c0b92a0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.732+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c005ce0 con 0x7f548c0b92a0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.732+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0072f0 con 0x7f548c0b92a0 2024-04-25T08:29:37.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.732+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c160a10 con 0x7f548c0b92a0 2024-04-25T08:29:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.732+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c0cb830 con 0x7f548c0b92a0 2024-04-25T08:29:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.733+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c073990 con 0x7f548c0b92a0 2024-04-25T08:29:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.733+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f547c01a070 con 0x7f548c0b92a0 2024-04-25T08:29:37.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.734+0000 7f5482ffd640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f547006b4d0 0x7f547006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.734+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f547c089800 con 0x7f548c0b92a0 2024-04-25T08:29:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.734+0000 7f54923c6640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f547006b4d0 0x7f547006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:37.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.734+0000 7f54923c6640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f547006b4d0 0x7f547006d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5488034000 tx=0x7f548802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.738+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f547c057ce0 con 0x7f548c0b92a0 2024-04-25T08:29:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.917+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f548c0b2990 con 0x7f548c0b92a0 2024-04-25T08:29:37.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.919+0000 7f5482ffd640 1 -- 172.21.15.60:0/147646937 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f547c05bd80 con 0x7f548c0b92a0 2024-04-25T08:29:37.916 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:37.916 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f547006b4d0 msgr2=0x7f547006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f547006b4d0 0x7f547006d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5488034000 tx=0x7f548802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 msgr2=0x7f548c15fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 0x7f548c15fce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f547c00c500 tx=0x7f547c00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 shutdown_connections 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f548c15dbe0 0x7f548c0aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f548c0b92a0 0x7f548c15fce0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f547006b4d0 0x7f547006d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 --2- 172.21.15.60:0/147646937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f548c0b2cb0 0x7f548c1f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 >> 172.21.15.60:0/147646937 conn(0x7f548c061a20 msgr2=0x7f548c0d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 shutdown_connections 2024-04-25T08:29:37.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:37.921+0000 7f549462a640 1 -- 172.21.15.60:0/147646937 wait complete. 2024-04-25T08:29:37.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:29:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.035+0000 7faf988cb640 1 Processor -- start 2024-04-25T08:29:38.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.035+0000 7faf988cb640 1 -- start start 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf988cb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf9415dc20 0x7faf940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf988cb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9406cda0 con 0x7faf940d32f0 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf988cb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9415e000 con 0x7faf94161710 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf988cb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf940d1de0 con 0x7faf9415dc20 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf9259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faf9415dc20 0x7faf940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf92d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faf940d32f0 0x7faf9406c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54420/0 (socket says 172.21.15.60:54420) 2024-04-25T08:29:38.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf92d9d640 1 -- 172.21.15.60:0/2088365811 learned_addr learned my addr 172.21.15.60:0/2088365811 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf9259c640 1 -- 172.21.15.60:0/2088365811 >> v1:172.21.15.100:6789/0 conn(0x7faf94161710 legacy=0x7faf9406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf9259c640 1 -- 172.21.15.60:0/2088365811 >> v1:172.21.15.60:6789/0 conn(0x7faf940d32f0 legacy=0x7faf9406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf9259c640 1 -- 172.21.15.60:0/2088365811 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf94161c60 con 0x7faf9415dc20 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.036+0000 7faf91d9b640 1 -- 172.21.15.60:0/2088365811 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612906665 0 0) 0x7faf9406cda0 con 0x7faf940d32f0 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.037+0000 7faf9259c640 1 --2- 172.21.15.60:0/2088365811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf9415dc20 0x7faf940ce670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faf840097a0 tx=0x7faf8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3676da381006775a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.037+0000 7faf91d9b640 1 -- 172.21.15.60:0/2088365811 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84037070 con 0x7faf9415dc20 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.037+0000 7faf91d9b640 1 -- 172.21.15.60:0/2088365811 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf84032440 con 0x7faf9415dc20 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.037+0000 7faf91d9b640 1 -- 172.21.15.60:0/2088365811 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84032810 con 0x7faf9415dc20 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 -- 172.21.15.60:0/2088365811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf9415dc20 msgr2=0x7faf940ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 --2- 172.21.15.60:0/2088365811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf9415dc20 0x7faf940ce670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faf840097a0 tx=0x7faf8402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 -- 172.21.15.60:0/2088365811 shutdown_connections 2024-04-25T08:29:38.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 --2- 172.21.15.60:0/2088365811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf9415dc20 0x7faf940ce670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 -- 172.21.15.60:0/2088365811 >> 172.21.15.60:0/2088365811 conn(0x7faf940d0420 msgr2=0x7faf940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 -- 172.21.15.60:0/2088365811 shutdown_connections 2024-04-25T08:29:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.038+0000 7faf988cb640 1 -- 172.21.15.60:0/2088365811 wait complete. 2024-04-25T08:29:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.039+0000 7faf988cb640 1 Processor -- start 2024-04-25T08:29:38.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.039+0000 7faf988cb640 1 -- start start 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf988cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf940d32f0 0x7faf9415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf988cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf9415dc20 0x7faf941e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf988cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 0x7faf941e9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf988cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf94162500 con 0x7faf94161710 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf988cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf94162220 con 0x7faf940d32f0 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf988cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf940b5e50 con 0x7faf9415dc20 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 0x7faf941e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 0x7faf941e9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43746/0 (socket says 172.21.15.60:43746) 2024-04-25T08:29:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 -- 172.21.15.60:0/1939762612 learned_addr learned my addr 172.21.15.60:0/1939762612 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf9259c640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf940d32f0 0x7faf9415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf8bfff640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf9415dc20 0x7faf941e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 -- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf9415dc20 msgr2=0x7faf941e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf9415dc20 0x7faf941e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 -- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf940d32f0 msgr2=0x7faf9415c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf940d32f0 0x7faf9415c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 -- 172.21.15.60:0/1939762612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9415cec0 con 0x7faf94161710 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.040+0000 7faf92d9d640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 0x7faf941e9c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c780 tx=0x7faf8c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.041+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c018070 con 0x7faf94161710 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.041+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c004d60 con 0x7faf94161710 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.041+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf940cb650 con 0x7faf94161710 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.041+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00a820 con 0x7faf94161710 2024-04-25T08:29:38.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.041+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf940cbc10 con 0x7faf94161710 2024-04-25T08:29:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.042+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf94073a00 con 0x7faf94161710 2024-04-25T08:29:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.042+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf8c01b070 con 0x7faf94161710 2024-04-25T08:29:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.042+0000 7faf8affd640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf7406b4d0 0x7faf7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.043+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faf8c089820 con 0x7faf94161710 2024-04-25T08:29:38.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.043+0000 7faf9259c640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf7406b4d0 0x7faf7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:38.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.043+0000 7faf9259c640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf7406b4d0 0x7faf7406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faf84034000 tx=0x7faf8402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:38.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.047+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf8c057da0 con 0x7faf94161710 2024-04-25T08:29:38.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:38.267+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faf940b48e0 con 0x7faf94161710 2024-04-25T08:29:39.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.110+0000 7faf8affd640 1 -- 172.21.15.60:0/1939762612 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7faf8c05be40 con 0x7faf94161710 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf7406b4d0 msgr2=0x7faf7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf7406b4d0 0x7faf7406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faf84034000 tx=0x7faf8402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 msgr2=0x7faf941e9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 0x7faf941e9c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c780 tx=0x7faf8c00cc40 comp rx=0 tx=0).stop 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 shutdown_connections 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf94161710 0x7faf941e9c20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf9415dc20 0x7faf941e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf7406b4d0 0x7faf7406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 --2- 172.21.15.60:0/1939762612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf940d32f0 0x7faf9415c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 >> 172.21.15.60:0/1939762612 conn(0x7faf940d0420 msgr2=0x7faf941601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 shutdown_connections 2024-04-25T08:29:39.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.113+0000 7faf988cb640 1 -- 172.21.15.60:0/1939762612 wait complete. 2024-04-25T08:29:39.124 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T08:29:39.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:29:39.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.237+0000 7fd7d4b9e640 1 Processor -- start 2024-04-25T08:29:39.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.237+0000 7fd7d4b9e640 1 -- start start 2024-04-25T08:29:39.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7d4b9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7d006bff0 0x7fd7d00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7d4b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d015e110 con 0x7fd7d00b5970 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7d4b9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00b2fd0 con 0x7fd7d015dd30 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7d4b9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d006d000 con 0x7fd7d006bff0 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7ceffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7d006bff0 0x7fd7d00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7cffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd7d00b5970 0x7fd7d00d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54436/0 (socket says 172.21.15.60:54436) 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.238+0000 7fd7cffff640 1 -- 172.21.15.60:0/1152402748 learned_addr learned my addr 172.21.15.60:0/1152402748 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.239+0000 7fd7ceffd640 1 -- 172.21.15.60:0/1152402748 >> v1:172.21.15.100:6789/0 conn(0x7fd7d015dd30 legacy=0x7fd7d00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.239+0000 7fd7ceffd640 1 -- 172.21.15.60:0/1152402748 >> v1:172.21.15.60:6789/0 conn(0x7fd7d00b5970 legacy=0x7fd7d00d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.239+0000 7fd7ceffd640 1 -- 172.21.15.60:0/1152402748 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d006c480 con 0x7fd7d006bff0 2024-04-25T08:29:39.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.239+0000 7fd7ce7fc640 1 -- 172.21.15.60:0/1152402748 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099183953 0 0) 0x7fd7d015e110 con 0x7fd7d00b5970 2024-04-25T08:29:39.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.239+0000 7fd7ceffd640 1 --2- 172.21.15.60:0/1152402748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d006bff0 0x7fd7d00b5430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40097a0 tx=0x7fd7c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2098772af10dd684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:39.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.239+0000 7fd7ce7fc640 1 -- 172.21.15.60:0/1152402748 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4037070 con 0x7fd7d006bff0 2024-04-25T08:29:39.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7ce7fc640 1 -- 172.21.15.60:0/1152402748 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4032440 con 0x7fd7d006bff0 2024-04-25T08:29:39.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7ce7fc640 1 -- 172.21.15.60:0/1152402748 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4032810 con 0x7fd7d006bff0 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/1152402748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d006bff0 msgr2=0x7fd7d00b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/1152402748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d006bff0 0x7fd7d00b5430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40097a0 tx=0x7fd7c402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/1152402748 shutdown_connections 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/1152402748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d006bff0 0x7fd7d00b5430 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/1152402748 >> 172.21.15.60:0/1152402748 conn(0x7fd7d00d0610 msgr2=0x7fd7d00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/1152402748 shutdown_connections 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.240+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/1152402748 wait complete. 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.241+0000 7fd7d4b9e640 1 Processor -- start 2024-04-25T08:29:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 -- start start 2024-04-25T08:29:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d00b5970 0x7fd7d01f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:39.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 0x7fd7d0158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d00cd690 0x7fd7d0160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d006cd20 con 0x7fd7d015dd30 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d006ca40 con 0x7fd7d00cd690 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.242+0000 7fd7d4b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00d2870 con 0x7fd7d00b5970 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 0x7fd7d0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 0x7fd7d0158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43750/0 (socket says 172.21.15.60:43750) 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 -- 172.21.15.60:0/359380864 learned_addr learned my addr 172.21.15.60:0/359380864 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7cffff640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d00cd690 0x7fd7d0160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7cf7fe640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d00b5970 0x7fd7d01f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 -- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d00b5970 msgr2=0x7fd7d01f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d00b5970 0x7fd7d01f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 -- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d00cd690 msgr2=0x7fd7d0160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d00cd690 0x7fd7d0160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 -- 172.21.15.60:0/359380864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0158780 con 0x7fd7d015dd30 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7ceffd640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 0x7fd7d0158240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40098c0 tx=0x7fd7c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:39.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4037040 con 0x7fd7d015dd30 2024-04-25T08:29:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4030770 con 0x7fd7d015dd30 2024-04-25T08:29:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0160930 con 0x7fd7d015dd30 2024-04-25T08:29:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4030b40 con 0x7fd7d015dd30 2024-04-25T08:29:39.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.243+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01f6c70 con 0x7fd7d015dd30 2024-04-25T08:29:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.244+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d0073990 con 0x7fd7d015dd30 2024-04-25T08:29:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.245+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7c4044020 con 0x7fd7d015dd30 2024-04-25T08:29:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.245+0000 7fd7b7fff640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.245+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd7c40aaa80 con 0x7fd7d015dd30 2024-04-25T08:29:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.245+0000 7fd7cf7fe640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:39.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.246+0000 7fd7cf7fe640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd7d015fc60 tx=0x7fd7c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.249+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7c4032e20 con 0x7fd7d015dd30 2024-04-25T08:29:39.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:39.468+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd7d015d400 con 0x7fd7d015dd30 2024-04-25T08:29:40.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.115+0000 7fd7b7fff640 1 -- 172.21.15.60:0/359380864 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7c4079650 con 0x7fd7d015dd30 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.117+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7bc06b4d0 msgr2=0x7fd7bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.117+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd7d015fc60 tx=0x7fd7c0009040 comp rx=0 tx=0).stop 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.117+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 msgr2=0x7fd7d0158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.117+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 0x7fd7d0158240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40098c0 tx=0x7fd7c40046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 shutdown_connections 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7bc06b4d0 0x7fd7bc06d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7d00cd690 0x7fd7d0160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7d015dd30 0x7fd7d0158240 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 --2- 172.21.15.60:0/359380864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7d00b5970 0x7fd7d01f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 >> 172.21.15.60:0/359380864 conn(0x7fd7d00d0610 msgr2=0x7fd7d00b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 shutdown_connections 2024-04-25T08:29:40.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.118+0000 7fd7d4b9e640 1 -- 172.21.15.60:0/359380864 wait complete. 2024-04-25T08:29:40.128 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:29:40.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.239+0000 7f2f9fe7c640 1 Processor -- start 2024-04-25T08:29:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.239+0000 7f2f9fe7c640 1 -- start start 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9fe7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f980d1020 0x7f2f980d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9fe7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980b97a0 con 0x7f2f9815dbe0 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9fe7c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980d1c50 con 0x7f2f980b2cb0 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9fe7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9815dfc0 con 0x7f2f980d1020 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9e419640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f980d1020 0x7f2f980d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9dc18640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2f9815dbe0 0x7f2f980d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54448/0 (socket says 172.21.15.60:54448) 2024-04-25T08:29:40.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9dc18640 1 -- 172.21.15.60:0/23990221 learned_addr learned my addr 172.21.15.60:0/23990221 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:40.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9e419640 1 -- 172.21.15.60:0/23990221 >> v1:172.21.15.100:6789/0 conn(0x7f2f980b2cb0 legacy=0x7f2f980d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9e419640 1 -- 172.21.15.60:0/23990221 >> v1:172.21.15.60:6789/0 conn(0x7f2f9815dbe0 legacy=0x7f2f980d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9cc16640 1 -- 172.21.15.60:0/23990221 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541913767 0 0) 0x7f2f980b97a0 con 0x7f2f9815dbe0 2024-04-25T08:29:40.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.240+0000 7f2f9e419640 1 -- 172.21.15.60:0/23990221 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980d1f30 con 0x7f2f980d1020 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.241+0000 7f2f9e419640 1 --2- 172.21.15.60:0/23990221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980d2e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009d90 tx=0x7f2f9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67feabcae1883c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.241+0000 7f2f9cc16640 1 -- 172.21.15.60:0/23990221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90007740 con 0x7f2f980d1020 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.241+0000 7f2f9cc16640 1 -- 172.21.15.60:0/23990221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f900078e0 con 0x7f2f980d1020 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.241+0000 7f2f9cc16640 1 -- 172.21.15.60:0/23990221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90007cb0 con 0x7f2f980d1020 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/23990221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 msgr2=0x7f2f980d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/23990221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980d2e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009d90 tx=0x7f2f9002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/23990221 shutdown_connections 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/23990221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980d2e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/23990221 >> 172.21.15.60:0/23990221 conn(0x7f2f98061a20 msgr2=0x7f2f98061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/23990221 shutdown_connections 2024-04-25T08:29:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.242+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/23990221 wait complete. 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.243+0000 7f2f9fe7c640 1 Processor -- start 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.243+0000 7f2f9fe7c640 1 -- start start 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9fe7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980b2cb0 0x7f2f980cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9fe7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9fe7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815dbe0 0x7f2f980c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9fe7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980b92a0 con 0x7f2f980d1020 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9fe7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d1a00 con 0x7f2f9815dbe0 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9fe7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980b9420 con 0x7f2f980b2cb0 2024-04-25T08:29:40.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43772/0 (socket says 172.21.15.60:43772) 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 -- 172.21.15.60:0/2436541655 learned_addr learned my addr 172.21.15.60:0/2436541655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9e419640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815dbe0 0x7f2f980c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9dc18640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980b2cb0 0x7f2f980cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 -- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980b2cb0 msgr2=0x7f2f980cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980b2cb0 0x7f2f980cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 -- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815dbe0 msgr2=0x7f2f980c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815dbe0 0x7f2f980c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 -- 172.21.15.60:0/2436541655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980cb1c0 con 0x7f2f980d1020 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9e419640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815dbe0 0x7f2f980c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f9d417640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980caa50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800c400 tx=0x7f2f8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.244+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88017070 con 0x7f2f980d1020 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.245+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f880052d0 con 0x7f2f980d1020 2024-04-25T08:29:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.245+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980cb4a0 con 0x7f2f980d1020 2024-04-25T08:29:40.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.245+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88003830 con 0x7f2f980d1020 2024-04-25T08:29:40.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.245+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980cba60 con 0x7f2f980d1020 2024-04-25T08:29:40.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.246+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f98073990 con 0x7f2f980d1020 2024-04-25T08:29:40.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.246+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f8801a070 con 0x7f2f980d1020 2024-04-25T08:29:40.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.247+0000 7f2f8e7fc640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f7c06b500 0x7f2f7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.247+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2f880898e0 con 0x7f2f980d1020 2024-04-25T08:29:40.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.247+0000 7f2f9dc18640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f7c06b500 0x7f2f7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.247+0000 7f2f9dc18640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f7c06b500 0x7f2f7c06d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f94000f60 tx=0x7f2f94007d00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.251+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f88058420 con 0x7f2f980d1020 2024-04-25T08:29:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.472+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f9815baf0 con 0x7f2f980d1020 2024-04-25T08:29:40.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.472+0000 7f2f8e7fc640 1 -- 172.21.15.60:0/2436541655 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f2f88057d20 con 0x7f2f980d1020 2024-04-25T08:29:40.469 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:40.470 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:29:40.470 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:29:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f7c06b500 msgr2=0x7f2f7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f7c06b500 0x7f2f7c06d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f94000f60 tx=0x7f2f94007d00 comp rx=0 tx=0).stop 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 msgr2=0x7f2f980caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980caa50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800c400 tx=0x7f2f8800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 shutdown_connections 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f7c06b500 0x7f2f7c06d9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f9815dbe0 0x7f2f980c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f980d1020 0x7f2f980caa50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 --2- 172.21.15.60:0/2436541655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f980b2cb0 0x7f2f980cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 >> 172.21.15.60:0/2436541655 conn(0x7f2f98061a20 msgr2=0x7f2f980b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 shutdown_connections 2024-04-25T08:29:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.474+0000 7f2f9fe7c640 1 -- 172.21.15.60:0/2436541655 wait complete. 2024-04-25T08:29:40.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:40.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.585+0000 7ff0a7607640 1 Processor -- start 2024-04-25T08:29:40.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.586+0000 7ff0a7607640 1 -- start start 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.586+0000 7ff0a7607640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0a015dbe0 0x7ff0a00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.586+0000 7ff0a7607640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff0a00b92a0 0x7ff0a00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.586+0000 7ff0a7607640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a00b97a0 con 0x7ff0a00b2cb0 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.586+0000 7ff0a7607640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a015dfc0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.586+0000 7ff0a7607640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a00d1a00 con 0x7ff0a00b92a0 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0a015dbe0 0x7ff0a00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0a015dbe0 0x7ff0a00d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43782/0 (socket says 172.21.15.60:43782) 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 -- 172.21.15.60:0/2826302752 learned_addr learned my addr 172.21.15.60:0/2826302752 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a5ba4640 1 --2- 172.21.15.60:0/2826302752 >> v2:172.21.15.100:3300/0 conn(0x7ff0a00b92a0 0x7ff0a00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 -- 172.21.15.60:0/2826302752 >> v2:172.21.15.100:3300/0 conn(0x7ff0a00b92a0 msgr2=0x7ff0a00d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:40.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 --2- 172.21.15.60:0/2826302752 >> v2:172.21.15.100:3300/0 conn(0x7ff0a00b92a0 0x7ff0a00d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 -- 172.21.15.60:0/2826302752 >> v1:172.21.15.100:6789/0 conn(0x7ff0a00b2cb0 legacy=0x7ff0a00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:40.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.587+0000 7ff0a53a3640 1 -- 172.21.15.60:0/2826302752 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00d1c50 con 0x7ff0a015dbe0 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a53a3640 1 --2- 172.21.15.60:0/2826302752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00d0910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00cfb90 tx=0x7ff09802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ed74820743df3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff09ffff640 1 -- 172.21.15.60:0/2826302752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098037070 con 0x7ff0a015dbe0 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff09ffff640 1 -- 172.21.15.60:0/2826302752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff098032440 con 0x7ff0a015dbe0 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff09ffff640 1 -- 172.21.15.60:0/2826302752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098032810 con 0x7ff0a015dbe0 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 -- 172.21.15.60:0/2826302752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 msgr2=0x7ff0a00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 --2- 172.21.15.60:0/2826302752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00d0910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00cfb90 tx=0x7ff09802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 -- 172.21.15.60:0/2826302752 shutdown_connections 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 --2- 172.21.15.60:0/2826302752 >> v2:172.21.15.100:3300/0 conn(0x7ff0a00b92a0 0x7ff0a00d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 --2- 172.21.15.60:0/2826302752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00d0910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 -- 172.21.15.60:0/2826302752 >> 172.21.15.60:0/2826302752 conn(0x7ff0a0061a20 msgr2=0x7ff0a0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 -- 172.21.15.60:0/2826302752 shutdown_connections 2024-04-25T08:29:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.588+0000 7ff0a7607640 1 -- 172.21.15.60:0/2826302752 wait complete. 2024-04-25T08:29:40.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.589+0000 7ff0a7607640 1 Processor -- start 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 -- start start 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a00b2cb0 0x7ff0a00c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a00b92a0 0x7ff0a00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00b6ab0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00d2210 con 0x7ff0a00b92a0 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a7607640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00b6c30 con 0x7ff0a00b2cb0 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a5ba4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a5ba4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43792/0 (socket says 172.21.15.60:43792) 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.590+0000 7ff0a5ba4640 1 -- 172.21.15.60:0/3670587649 learned_addr learned my addr 172.21.15.60:0/3670587649 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a4ba2640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a00b92a0 0x7ff0a00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a53a3640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a00b2cb0 0x7ff0a00c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a5ba4640 1 -- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a00b2cb0 msgr2=0x7ff0a00c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a5ba4640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a00b2cb0 0x7ff0a00c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a5ba4640 1 -- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a00b92a0 msgr2=0x7ff0a00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a5ba4640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a00b92a0 0x7ff0a00b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a5ba4640 1 -- 172.21.15.60:0/3670587649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00b33c0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a53a3640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a00b2cb0 0x7ff0a00c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a5ba4640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00b5710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff09000cd40 tx=0x7ff09000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090003b90 con 0x7ff0a015dbe0 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090003d30 con 0x7ff0a015dbe0 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a00cac60 con 0x7ff0a015dbe0 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090004240 con 0x7ff0a015dbe0 2024-04-25T08:29:40.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.591+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a00cb220 con 0x7ff0a015dbe0 2024-04-25T08:29:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.592+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a0073990 con 0x7ff0a015dbe0 2024-04-25T08:29:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.593+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0900043e0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.593+0000 7ff09dffb640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08c06b4d0 0x7ff08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.593+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff09008afc0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.593+0000 7ff0a53a3640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08c06b4d0 0x7ff08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.594+0000 7ff0a53a3640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08c06b4d0 0x7ff08c06d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff098034000 tx=0x7ff09802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.597+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0a00b33c0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.776+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0a00c9fb0 con 0x7ff0a015dbe0 2024-04-25T08:29:40.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.777+0000 7ff09dffb640 1 -- 172.21.15.60:0/3670587649 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7ff09001d340 con 0x7ff0a015dbe0 2024-04-25T08:29:40.775 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:40.775 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:40.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08c06b4d0 msgr2=0x7ff08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08c06b4d0 0x7ff08c06d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff098034000 tx=0x7ff09802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 msgr2=0x7ff0a00b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00b5710 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff09000cd40 tx=0x7ff09000a660 comp rx=0 tx=0).stop 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 shutdown_connections 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0a015dbe0 0x7ff0a00b5710 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08c06b4d0 0x7ff08c06d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0a00b92a0 0x7ff0a00b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 --2- 172.21.15.60:0/3670587649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0a00b2cb0 0x7ff0a00c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 >> 172.21.15.60:0/3670587649 conn(0x7ff0a0061a20 msgr2=0x7ff0a00d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.779+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 shutdown_connections 2024-04-25T08:29:40.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.780+0000 7ff0a7607640 1 -- 172.21.15.60:0/3670587649 wait complete. 2024-04-25T08:29:40.788 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:29:40.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:29:40.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.891+0000 7f8bf75ab640 1 Processor -- start 2024-04-25T08:29:40.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf75ab640 1 -- start start 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf75ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf00cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf75ab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8bf00ce590 0x7f8bf006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf75ab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf0071480 con 0x7f8bf00cfcf0 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf75ab640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf0072090 con 0x7f8bf00ce590 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf75ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00d0150 con 0x7f8bf0154ee0 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.892+0000 7f8bf5b48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8bf00ce590 0x7f8bf006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5b48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8bf00ce590 0x7f8bf006d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43806/0 (socket says 172.21.15.60:43806) 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5b48640 1 -- 172.21.15.60:0/3042072046 learned_addr learned my addr 172.21.15.60:0/3042072046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5347640 1 --2- 172.21.15.60:0/3042072046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf00cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5b48640 1 -- 172.21.15.60:0/3042072046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 msgr2=0x7f8bf00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5b48640 1 --2- 172.21.15.60:0/3042072046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf00cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5b48640 1 -- 172.21.15.60:0/3042072046 >> v1:172.21.15.100:6789/0 conn(0x7f8bf00cfcf0 legacy=0x7f8bf00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.893+0000 7f8bf5b48640 1 -- 172.21.15.60:0/3042072046 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0071760 con 0x7f8bf00ce590 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf5b48640 1 --2- 172.21.15.60:0/3042072046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf006d6f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009e50 tx=0x7f8be802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a49ad9e6d260096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8be7fff640 1 -- 172.21.15.60:0/3042072046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8036070 con 0x7f8bf00ce590 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8be7fff640 1 -- 172.21.15.60:0/3042072046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be8031600 con 0x7f8bf00ce590 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8be7fff640 1 -- 172.21.15.60:0/3042072046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80319d0 con 0x7f8bf00ce590 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 -- 172.21.15.60:0/3042072046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 msgr2=0x7f8bf006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/3042072046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf006d6f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009e50 tx=0x7f8be802a710 comp rx=0 tx=0).stop 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 -- 172.21.15.60:0/3042072046 shutdown_connections 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/3042072046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf006d6f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/3042072046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf00cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 -- 172.21.15.60:0/3042072046 >> 172.21.15.60:0/3042072046 conn(0x7f8bf006abe0 msgr2=0x7f8bf006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 -- 172.21.15.60:0/3042072046 shutdown_connections 2024-04-25T08:29:40.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.894+0000 7f8bf75ab640 1 -- 172.21.15.60:0/3042072046 wait complete. 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.895+0000 7f8bf75ab640 1 Processor -- start 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 -- start start 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf00cfcf0 0x7f8bf014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf014d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0071d20 con 0x7f8bf00ce590 2024-04-25T08:29:40.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0071230 con 0x7f8bf00cfcf0 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf75ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf01552c0 con 0x7f8bf0154ee0 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf5347640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf5347640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf015a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43820/0 (socket says 172.21.15.60:43820) 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.896+0000 7f8bf5347640 1 -- 172.21.15.60:0/487123828 learned_addr learned my addr 172.21.15.60:0/487123828 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf4b46640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf00cfcf0 0x7f8bf014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5b48640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf014d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5347640 1 -- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 msgr2=0x7f8bf014d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5347640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf014d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5347640 1 -- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf00cfcf0 msgr2=0x7f8bf014b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5347640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf00cfcf0 0x7f8bf014b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5347640 1 -- 172.21.15.60:0/487123828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf014c340 con 0x7f8bf00ce590 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5b48640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf5347640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf015a3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00d600 tx=0x7f8bec00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0125f0 con 0x7f8bf00ce590 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec019070 con 0x7f8bf00ce590 2024-04-25T08:29:40.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf014c620 con 0x7f8bf00ce590 2024-04-25T08:29:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0129f0 con 0x7f8bf00ce590 2024-04-25T08:29:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.897+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf014ef20 con 0x7f8bf00ce590 2024-04-25T08:29:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.898+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf0073990 con 0x7f8bf00ce590 2024-04-25T08:29:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.899+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bec012b90 con 0x7f8bf00ce590 2024-04-25T08:29:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.899+0000 7f8be5ffb640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8bd406b4d0 0x7f8bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:40.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.899+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8bec08a9c0 con 0x7f8bf00ce590 2024-04-25T08:29:40.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.899+0000 7f8bf4b46640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8bd406b4d0 0x7f8bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:40.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.900+0000 7f8bf4b46640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8bd406b4d0 0x7f8bd406d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bf006eb60 tx=0x7f8be0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:40.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:40.903+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bec08ce20 con 0x7f8bf00ce590 2024-04-25T08:29:41.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.080+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bf006d360 con 0x7f8bd406b4d0 2024-04-25T08:29:41.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.089+0000 7f8be5ffb640 1 -- 172.21.15.60:0/487123828 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f8bf006d360 con 0x7f8bd406b4d0 2024-04-25T08:29:41.090 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:41.092 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.092+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8bd406b4d0 msgr2=0x7f8bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.092+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8bd406b4d0 0x7f8bd406d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8bf006eb60 tx=0x7f8be0009040 comp rx=0 tx=0).stop 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.092+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 msgr2=0x7f8bf015a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.092+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf015a3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00d600 tx=0x7f8bec00dac0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 shutdown_connections 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bf0154ee0 0x7f8bf014d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8bd406b4d0 0x7f8bd406d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bf00cfcf0 0x7f8bf014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 --2- 172.21.15.60:0/487123828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bf00ce590 0x7f8bf015a3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 >> 172.21.15.60:0/487123828 conn(0x7f8bf006abe0 msgr2=0x7f8bf006d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:41.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 shutdown_connections 2024-04-25T08:29:41.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.093+0000 7f8bf75ab640 1 -- 172.21.15.60:0/487123828 wait complete. 2024-04-25T08:29:41.100 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T08:29:39.400079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231920,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742800,"statfs":{"total":773094113280,"available":772856627200,"internally_reserved":0,"allocated":3350528,"data_stored":2022430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.327225"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.028601+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.028315+0000","last_became_peered":"2024-04-25T08:29:23.028315+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075712+0000","last_change":"2024-04-25T08:29:24.048532+0000","last_active":"2024-04-25T08:29:29.075712+0000","last_peered":"2024-04-25T08:29:29.075712+0000","last_clean":"2024-04-25T08:29:29.075712+0000","last_became_active":"2024-04-25T08:29:24.046997+0000","last_became_peered":"2024-04-25T08:29:24.046997+0000","last_unstale":"2024-04-25T08:29:29.075712+0000","last_undegraded":"2024-04-25T08:29:29.075712+0000","last_fullsized":"2024-04-25T08:29:29.075712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074646+0000","last_change":"2024-04-25T08:29:28.085554+0000","last_active":"2024-04-25T08:29:29.074646+0000","last_peered":"2024-04-25T08:29:29.074646+0000","last_clean":"2024-04-25T08:29:29.074646+0000","last_became_active":"2024-04-25T08:29:28.085164+0000","last_became_peered":"2024-04-25T08:29:28.085164+0000","last_unstale":"2024-04-25T08:29:29.074646+0000","last_undegraded":"2024-04-25T08:29:29.074646+0000","last_fullsized":"2024-04-25T08:29:29.074646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:03.849914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075840+0000","last_change":"2024-04-25T08:29:23.035840+0000","last_active":"2024-04-25T08:29:29.075840+0000","last_peered":"2024-04-25T08:29:29.075840+0000","last_clean":"2024-04-25T08:29:29.075840+0000","last_became_active":"2024-04-25T08:29:23.035532+0000","last_became_peered":"2024-04-25T08:29:23.035532+0000","last_unstale":"2024-04-25T08:29:29.075840+0000","last_undegraded":"2024-04-25T08:29:29.075840+0000","last_fullsized":"2024-04-25T08:29:29.075840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906355+0000","last_change":"2024-04-25T08:29:24.037371+0000","last_active":"2024-04-25T08:29:29.906355+0000","last_peered":"2024-04-25T08:29:29.906355+0000","last_clean":"2024-04-25T08:29:29.906355+0000","last_became_active":"2024-04-25T08:29:24.037043+0000","last_became_peered":"2024-04-25T08:29:24.037043+0000","last_unstale":"2024-04-25T08:29:29.906355+0000","last_undegraded":"2024-04-25T08:29:29.906355+0000","last_fullsized":"2024-04-25T08:29:29.906355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.900465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074732+0000","last_change":"2024-04-25T08:29:28.080997+0000","last_active":"2024-04-25T08:29:29.074732+0000","last_peered":"2024-04-25T08:29:29.074732+0000","last_clean":"2024-04-25T08:29:29.074732+0000","last_became_active":"2024-04-25T08:29:28.080573+0000","last_became_peered":"2024-04-25T08:29:28.080573+0000","last_unstale":"2024-04-25T08:29:29.074732+0000","last_undegraded":"2024-04-25T08:29:29.074732+0000","last_fullsized":"2024-04-25T08:29:29.074732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:48.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895961+0000","last_change":"2024-04-25T08:29:23.041517+0000","last_active":"2024-04-25T08:29:29.895961+0000","last_peered":"2024-04-25T08:29:29.895961+0000","last_clean":"2024-04-25T08:29:29.895961+0000","last_became_active":"2024-04-25T08:29:23.041191+0000","last_became_peered":"2024-04-25T08:29:23.041191+0000","last_unstale":"2024-04-25T08:29:29.895961+0000","last_undegraded":"2024-04-25T08:29:29.895961+0000","last_fullsized":"2024-04-25T08:29:29.895961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574369+0000","last_change":"2024-04-25T08:29:24.042793+0000","last_active":"2024-04-25T08:29:29.574369+0000","last_peered":"2024-04-25T08:29:29.574369+0000","last_clean":"2024-04-25T08:29:29.574369+0000","last_became_active":"2024-04-25T08:29:24.042026+0000","last_became_peered":"2024-04-25T08:29:24.042026+0000","last_unstale":"2024-04-25T08:29:29.574369+0000","last_undegraded":"2024-04-25T08:29:29.574369+0000","last_fullsized":"2024-04-25T08:29:29.574369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075189+0000","last_change":"2024-04-25T08:29:28.081365+0000","last_active":"2024-04-25T08:29:29.075189+0000","last_peered":"2024-04-25T08:29:29.075189+0000","last_clean":"2024-04-25T08:29:29.075189+0000","last_became_active":"2024-04-25T08:29:28.080901+0000","last_became_peered":"2024-04-25T08:29:28.080901+0000","last_unstale":"2024-04-25T08:29:29.075189+0000","last_undegraded":"2024-04-25T08:29:29.075189+0000","last_fullsized":"2024-04-25T08:29:29.075189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.649430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895778+0000","last_change":"2024-04-25T08:29:23.036549+0000","last_active":"2024-04-25T08:29:29.895778+0000","last_peered":"2024-04-25T08:29:29.895778+0000","last_clean":"2024-04-25T08:29:29.895778+0000","last_became_active":"2024-04-25T08:29:23.036323+0000","last_became_peered":"2024-04-25T08:29:23.036323+0000","last_unstale":"2024-04-25T08:29:29.895778+0000","last_undegraded":"2024-04-25T08:29:29.895778+0000","last_fullsized":"2024-04-25T08:29:29.895778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574394+0000","last_change":"2024-04-25T08:29:24.041522+0000","last_active":"2024-04-25T08:29:29.574394+0000","last_peered":"2024-04-25T08:29:29.574394+0000","last_clean":"2024-04-25T08:29:29.574394+0000","last_became_active":"2024-04-25T08:29:24.041242+0000","last_became_peered":"2024-04-25T08:29:24.041242+0000","last_unstale":"2024-04-25T08:29:29.574394+0000","last_undegraded":"2024-04-25T08:29:29.574394+0000","last_fullsized":"2024-04-25T08:29:29.574394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075282+0000","last_change":"2024-04-25T08:29:28.081585+0000","last_active":"2024-04-25T08:29:29.075282+0000","last_peered":"2024-04-25T08:29:29.075282+0000","last_clean":"2024-04-25T08:29:29.075282+0000","last_became_active":"2024-04-25T08:29:28.080227+0000","last_became_peered":"2024-04-25T08:29:28.080227+0000","last_unstale":"2024-04-25T08:29:29.075282+0000","last_undegraded":"2024-04-25T08:29:29.075282+0000","last_fullsized":"2024-04-25T08:29:29.075282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075044+0000","last_change":"2024-04-25T08:29:24.040005+0000","last_active":"2024-04-25T08:29:29.075044+0000","last_peered":"2024-04-25T08:29:29.075044+0000","last_clean":"2024-04-25T08:29:29.075044+0000","last_became_active":"2024-04-25T08:29:24.039678+0000","last_became_peered":"2024-04-25T08:29:24.039678+0000","last_unstale":"2024-04-25T08:29:29.075044+0000","last_undegraded":"2024-04-25T08:29:29.075044+0000","last_fullsized":"2024-04-25T08:29:29.075044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895838+0000","last_change":"2024-04-25T08:29:23.034031+0000","last_active":"2024-04-25T08:29:29.895838+0000","last_peered":"2024-04-25T08:29:29.895838+0000","last_clean":"2024-04-25T08:29:29.895838+0000","last_became_active":"2024-04-25T08:29:23.033672+0000","last_became_peered":"2024-04-25T08:29:23.033672+0000","last_unstale":"2024-04-25T08:29:29.895838+0000","last_undegraded":"2024-04-25T08:29:29.895838+0000","last_fullsized":"2024-04-25T08:29:29.895838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076035+0000","last_change":"2024-04-25T08:29:28.067431+0000","last_active":"2024-04-25T08:29:29.076035+0000","last_peered":"2024-04-25T08:29:29.076035+0000","last_clean":"2024-04-25T08:29:29.076035+0000","last_became_active":"2024-04-25T08:29:28.066968+0000","last_became_peered":"2024-04-25T08:29:28.066968+0000","last_unstale":"2024-04-25T08:29:29.076035+0000","last_undegraded":"2024-04-25T08:29:29.076035+0000","last_fullsized":"2024-04-25T08:29:29.076035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075071+0000","last_change":"2024-04-25T08:29:24.052628+0000","last_active":"2024-04-25T08:29:29.075071+0000","last_peered":"2024-04-25T08:29:29.075071+0000","last_clean":"2024-04-25T08:29:29.075071+0000","last_became_active":"2024-04-25T08:29:24.052111+0000","last_became_peered":"2024-04-25T08:29:24.052111+0000","last_unstale":"2024-04-25T08:29:29.075071+0000","last_undegraded":"2024-04-25T08:29:29.075071+0000","last_fullsized":"2024-04-25T08:29:29.075071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074777+0000","last_change":"2024-04-25T08:29:23.040432+0000","last_active":"2024-04-25T08:29:30.074777+0000","last_peered":"2024-04-25T08:29:30.074777+0000","last_clean":"2024-04-25T08:29:30.074777+0000","last_became_active":"2024-04-25T08:29:23.040172+0000","last_became_peered":"2024-04-25T08:29:23.040172+0000","last_unstale":"2024-04-25T08:29:30.074777+0000","last_undegraded":"2024-04-25T08:29:30.074777+0000","last_fullsized":"2024-04-25T08:29:30.074777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075200+0000","last_change":"2024-04-25T08:29:28.081025+0000","last_active":"2024-04-25T08:29:29.075200+0000","last_peered":"2024-04-25T08:29:29.075200+0000","last_clean":"2024-04-25T08:29:29.075200+0000","last_became_active":"2024-04-25T08:29:28.080481+0000","last_became_peered":"2024-04-25T08:29:28.080481+0000","last_unstale":"2024-04-25T08:29:29.075200+0000","last_undegraded":"2024-04-25T08:29:29.075200+0000","last_fullsized":"2024-04-25T08:29:29.075200+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074861+0000","last_change":"2024-04-25T08:29:24.054299+0000","last_active":"2024-04-25T08:29:29.074861+0000","last_peered":"2024-04-25T08:29:29.074861+0000","last_clean":"2024-04-25T08:29:29.074861+0000","last_became_active":"2024-04-25T08:29:24.053996+0000","last_became_peered":"2024-04-25T08:29:24.053996+0000","last_unstale":"2024-04-25T08:29:29.074861+0000","last_undegraded":"2024-04-25T08:29:29.074861+0000","last_fullsized":"2024-04-25T08:29:29.074861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074526+0000","last_change":"2024-04-25T08:29:23.038382+0000","last_active":"2024-04-25T08:29:30.074526+0000","last_peered":"2024-04-25T08:29:30.074526+0000","last_clean":"2024-04-25T08:29:30.074526+0000","last_became_active":"2024-04-25T08:29:23.038060+0000","last_became_peered":"2024-04-25T08:29:23.038060+0000","last_unstale":"2024-04-25T08:29:30.074526+0000","last_undegraded":"2024-04-25T08:29:30.074526+0000","last_fullsized":"2024-04-25T08:29:30.074526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075123+0000","last_change":"2024-04-25T08:29:28.085260+0000","last_active":"2024-04-25T08:29:30.075123+0000","last_peered":"2024-04-25T08:29:30.075123+0000","last_clean":"2024-04-25T08:29:30.075123+0000","last_became_active":"2024-04-25T08:29:28.084884+0000","last_became_peered":"2024-04-25T08:29:28.084884+0000","last_unstale":"2024-04-25T08:29:30.075123+0000","last_undegraded":"2024-04-25T08:29:30.075123+0000","last_fullsized":"2024-04-25T08:29:30.075123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075420+0000","last_change":"2024-04-25T08:29:24.035523+0000","last_active":"2024-04-25T08:29:29.075420+0000","last_peered":"2024-04-25T08:29:29.075420+0000","last_clean":"2024-04-25T08:29:29.075420+0000","last_became_active":"2024-04-25T08:29:24.035185+0000","last_became_peered":"2024-04-25T08:29:24.035185+0000","last_unstale":"2024-04-25T08:29:29.075420+0000","last_undegraded":"2024-04-25T08:29:29.075420+0000","last_fullsized":"2024-04-25T08:29:29.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.274896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895386+0000","last_change":"2024-04-25T08:29:23.037104+0000","last_active":"2024-04-25T08:29:29.895386+0000","last_peered":"2024-04-25T08:29:29.895386+0000","last_clean":"2024-04-25T08:29:29.895386+0000","last_became_active":"2024-04-25T08:29:23.035864+0000","last_became_peered":"2024-04-25T08:29:23.035864+0000","last_unstale":"2024-04-25T08:29:29.895386+0000","last_undegraded":"2024-04-25T08:29:29.895386+0000","last_fullsized":"2024-04-25T08:29:29.895386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075821+0000","last_change":"2024-04-25T08:29:28.080714+0000","last_active":"2024-04-25T08:29:29.075821+0000","last_peered":"2024-04-25T08:29:29.075821+0000","last_clean":"2024-04-25T08:29:29.075821+0000","last_became_active":"2024-04-25T08:29:28.080291+0000","last_became_peered":"2024-04-25T08:29:28.080291+0000","last_unstale":"2024-04-25T08:29:29.075821+0000","last_undegraded":"2024-04-25T08:29:29.075821+0000","last_fullsized":"2024-04-25T08:29:29.075821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906353+0000","last_change":"2024-04-25T08:29:24.040261+0000","last_active":"2024-04-25T08:29:29.906353+0000","last_peered":"2024-04-25T08:29:29.906353+0000","last_clean":"2024-04-25T08:29:29.906353+0000","last_became_active":"2024-04-25T08:29:24.039953+0000","last_became_peered":"2024-04-25T08:29:24.039953+0000","last_unstale":"2024-04-25T08:29:29.906353+0000","last_undegraded":"2024-04-25T08:29:29.906353+0000","last_fullsized":"2024-04-25T08:29:29.906353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076570+0000","last_change":"2024-04-25T08:29:23.038961+0000","last_active":"2024-04-25T08:29:29.076570+0000","last_peered":"2024-04-25T08:29:29.076570+0000","last_clean":"2024-04-25T08:29:29.076570+0000","last_became_active":"2024-04-25T08:29:23.038722+0000","last_became_peered":"2024-04-25T08:29:23.038722+0000","last_unstale":"2024-04-25T08:29:29.076570+0000","last_undegraded":"2024-04-25T08:29:29.076570+0000","last_fullsized":"2024-04-25T08:29:29.076570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074879+0000","last_change":"2024-04-25T08:29:28.073848+0000","last_active":"2024-04-25T08:29:29.074879+0000","last_peered":"2024-04-25T08:29:29.074879+0000","last_clean":"2024-04-25T08:29:29.074879+0000","last_became_active":"2024-04-25T08:29:28.073489+0000","last_became_peered":"2024-04-25T08:29:28.073489+0000","last_unstale":"2024-04-25T08:29:29.074879+0000","last_undegraded":"2024-04-25T08:29:29.074879+0000","last_fullsized":"2024-04-25T08:29:29.074879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:54.262561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076145+0000","last_change":"2024-04-25T08:29:24.057631+0000","last_active":"2024-04-25T08:29:29.076145+0000","last_peered":"2024-04-25T08:29:29.076145+0000","last_clean":"2024-04-25T08:29:29.076145+0000","last_became_active":"2024-04-25T08:29:24.057387+0000","last_became_peered":"2024-04-25T08:29:24.057387+0000","last_unstale":"2024-04-25T08:29:29.076145+0000","last_undegraded":"2024-04-25T08:29:29.076145+0000","last_fullsized":"2024-04-25T08:29:29.076145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075211+0000","last_change":"2024-04-25T08:29:23.041608+0000","last_active":"2024-04-25T08:29:30.075211+0000","last_peered":"2024-04-25T08:29:30.075211+0000","last_clean":"2024-04-25T08:29:30.075211+0000","last_became_active":"2024-04-25T08:29:23.041305+0000","last_became_peered":"2024-04-25T08:29:23.041305+0000","last_unstale":"2024-04-25T08:29:30.075211+0000","last_undegraded":"2024-04-25T08:29:30.075211+0000","last_fullsized":"2024-04-25T08:29:30.075211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:28.085603+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:28.084444+0000","last_became_peered":"2024-04-25T08:29:28.084444+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:39.462705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075317+0000","last_change":"2024-04-25T08:29:24.036286+0000","last_active":"2024-04-25T08:29:30.075317+0000","last_peered":"2024-04-25T08:29:30.075317+0000","last_clean":"2024-04-25T08:29:30.075317+0000","last_became_active":"2024-04-25T08:29:24.035947+0000","last_became_peered":"2024-04-25T08:29:24.035947+0000","last_unstale":"2024-04-25T08:29:30.075317+0000","last_undegraded":"2024-04-25T08:29:30.075317+0000","last_fullsized":"2024-04-25T08:29:30.075317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896087+0000","last_change":"2024-04-25T08:29:23.035930+0000","last_active":"2024-04-25T08:29:29.896087+0000","last_peered":"2024-04-25T08:29:29.896087+0000","last_clean":"2024-04-25T08:29:29.896087+0000","last_became_active":"2024-04-25T08:29:23.035685+0000","last_became_peered":"2024-04-25T08:29:23.035685+0000","last_unstale":"2024-04-25T08:29:29.896087+0000","last_undegraded":"2024-04-25T08:29:29.896087+0000","last_fullsized":"2024-04-25T08:29:29.896087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906315+0000","last_change":"2024-04-25T08:29:28.076556+0000","last_active":"2024-04-25T08:29:29.906315+0000","last_peered":"2024-04-25T08:29:29.906315+0000","last_clean":"2024-04-25T08:29:29.906315+0000","last_became_active":"2024-04-25T08:29:28.076157+0000","last_became_peered":"2024-04-25T08:29:28.076157+0000","last_unstale":"2024-04-25T08:29:29.906315+0000","last_undegraded":"2024-04-25T08:29:29.906315+0000","last_fullsized":"2024-04-25T08:29:29.906315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574536+0000","last_change":"2024-04-25T08:29:24.055822+0000","last_active":"2024-04-25T08:29:29.574536+0000","last_peered":"2024-04-25T08:29:29.574536+0000","last_clean":"2024-04-25T08:29:29.574536+0000","last_became_active":"2024-04-25T08:29:24.055087+0000","last_became_peered":"2024-04-25T08:29:24.055087+0000","last_unstale":"2024-04-25T08:29:29.574536+0000","last_undegraded":"2024-04-25T08:29:29.574536+0000","last_fullsized":"2024-04-25T08:29:29.574536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075321+0000","last_change":"2024-04-25T08:29:23.037612+0000","last_active":"2024-04-25T08:29:30.075321+0000","last_peered":"2024-04-25T08:29:30.075321+0000","last_clean":"2024-04-25T08:29:30.075321+0000","last_became_active":"2024-04-25T08:29:23.037304+0000","last_became_peered":"2024-04-25T08:29:23.037304+0000","last_unstale":"2024-04-25T08:29:30.075321+0000","last_undegraded":"2024-04-25T08:29:30.075321+0000","last_fullsized":"2024-04-25T08:29:30.075321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074438+0000","last_change":"2024-04-25T08:29:28.068007+0000","last_active":"2024-04-25T08:29:30.074438+0000","last_peered":"2024-04-25T08:29:30.074438+0000","last_clean":"2024-04-25T08:29:30.074438+0000","last_became_active":"2024-04-25T08:29:28.067586+0000","last_became_peered":"2024-04-25T08:29:28.067586+0000","last_unstale":"2024-04-25T08:29:30.074438+0000","last_undegraded":"2024-04-25T08:29:30.074438+0000","last_fullsized":"2024-04-25T08:29:30.074438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075717+0000","last_change":"2024-04-25T08:29:24.038433+0000","last_active":"2024-04-25T08:29:29.075717+0000","last_peered":"2024-04-25T08:29:29.075717+0000","last_clean":"2024-04-25T08:29:29.075717+0000","last_became_active":"2024-04-25T08:29:24.038164+0000","last_became_peered":"2024-04-25T08:29:24.038164+0000","last_unstale":"2024-04-25T08:29:29.075717+0000","last_undegraded":"2024-04-25T08:29:29.075717+0000","last_fullsized":"2024-04-25T08:29:29.075717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574356+0000","last_change":"2024-04-25T08:29:23.036328+0000","last_active":"2024-04-25T08:29:29.574356+0000","last_peered":"2024-04-25T08:29:29.574356+0000","last_clean":"2024-04-25T08:29:29.574356+0000","last_became_active":"2024-04-25T08:29:23.036072+0000","last_became_peered":"2024-04-25T08:29:23.036072+0000","last_unstale":"2024-04-25T08:29:29.574356+0000","last_undegraded":"2024-04-25T08:29:29.574356+0000","last_fullsized":"2024-04-25T08:29:29.574356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075255+0000","last_change":"2024-04-25T08:29:28.081150+0000","last_active":"2024-04-25T08:29:30.075255+0000","last_peered":"2024-04-25T08:29:30.075255+0000","last_clean":"2024-04-25T08:29:30.075255+0000","last_became_active":"2024-04-25T08:29:28.080621+0000","last_became_peered":"2024-04-25T08:29:28.080621+0000","last_unstale":"2024-04-25T08:29:30.075255+0000","last_undegraded":"2024-04-25T08:29:30.075255+0000","last_fullsized":"2024-04-25T08:29:30.075255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074632+0000","last_change":"2024-04-25T08:29:24.036313+0000","last_active":"2024-04-25T08:29:30.074632+0000","last_peered":"2024-04-25T08:29:30.074632+0000","last_clean":"2024-04-25T08:29:30.074632+0000","last_became_active":"2024-04-25T08:29:24.035892+0000","last_became_peered":"2024-04-25T08:29:24.035892+0000","last_unstale":"2024-04-25T08:29:30.074632+0000","last_undegraded":"2024-04-25T08:29:30.074632+0000","last_fullsized":"2024-04-25T08:29:30.074632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074251+0000","last_change":"2024-04-25T08:29:23.033190+0000","last_active":"2024-04-25T08:29:30.074251+0000","last_peered":"2024-04-25T08:29:30.074251+0000","last_clean":"2024-04-25T08:29:30.074251+0000","last_became_active":"2024-04-25T08:29:23.032822+0000","last_became_peered":"2024-04-25T08:29:23.032822+0000","last_unstale":"2024-04-25T08:29:30.074251+0000","last_undegraded":"2024-04-25T08:29:30.074251+0000","last_fullsized":"2024-04-25T08:29:30.074251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075145+0000","last_change":"2024-04-25T08:29:28.085006+0000","last_active":"2024-04-25T08:29:30.075145+0000","last_peered":"2024-04-25T08:29:30.075145+0000","last_clean":"2024-04-25T08:29:30.075145+0000","last_became_active":"2024-04-25T08:29:28.084572+0000","last_became_peered":"2024-04-25T08:29:28.084572+0000","last_unstale":"2024-04-25T08:29:30.075145+0000","last_undegraded":"2024-04-25T08:29:30.075145+0000","last_fullsized":"2024-04-25T08:29:30.075145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906058+0000","last_change":"2024-04-25T08:29:24.043403+0000","last_active":"2024-04-25T08:29:29.906058+0000","last_peered":"2024-04-25T08:29:29.906058+0000","last_clean":"2024-04-25T08:29:29.906058+0000","last_became_active":"2024-04-25T08:29:24.042964+0000","last_became_peered":"2024-04-25T08:29:24.042964+0000","last_unstale":"2024-04-25T08:29:29.906058+0000","last_undegraded":"2024-04-25T08:29:29.906058+0000","last_fullsized":"2024-04-25T08:29:29.906058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895394+0000","last_change":"2024-04-25T08:29:23.035024+0000","last_active":"2024-04-25T08:29:29.895394+0000","last_peered":"2024-04-25T08:29:29.895394+0000","last_clean":"2024-04-25T08:29:29.895394+0000","last_became_active":"2024-04-25T08:29:23.034761+0000","last_became_peered":"2024-04-25T08:29:23.034761+0000","last_unstale":"2024-04-25T08:29:29.895394+0000","last_undegraded":"2024-04-25T08:29:29.895394+0000","last_fullsized":"2024-04-25T08:29:29.895394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906299+0000","last_change":"2024-04-25T08:29:28.078207+0000","last_active":"2024-04-25T08:29:29.906299+0000","last_peered":"2024-04-25T08:29:29.906299+0000","last_clean":"2024-04-25T08:29:29.906299+0000","last_became_active":"2024-04-25T08:29:28.077777+0000","last_became_peered":"2024-04-25T08:29:28.077777+0000","last_unstale":"2024-04-25T08:29:29.906299+0000","last_undegraded":"2024-04-25T08:29:29.906299+0000","last_fullsized":"2024-04-25T08:29:29.906299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.048117+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.047733+0000","last_became_peered":"2024-04-25T08:29:24.047733+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574166+0000","last_change":"2024-04-25T08:29:23.036154+0000","last_active":"2024-04-25T08:29:29.574166+0000","last_peered":"2024-04-25T08:29:29.574166+0000","last_clean":"2024-04-25T08:29:29.574166+0000","last_became_active":"2024-04-25T08:29:23.035873+0000","last_became_peered":"2024-04-25T08:29:23.035873+0000","last_unstale":"2024-04-25T08:29:29.574166+0000","last_undegraded":"2024-04-25T08:29:29.574166+0000","last_fullsized":"2024-04-25T08:29:29.574166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895594+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:29:29.895594+0000","last_peered":"2024-04-25T08:29:29.895594+0000","last_clean":"2024-04-25T08:29:29.895594+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:29:29.895594+0000","last_undegraded":"2024-04-25T08:29:29.895594+0000","last_fullsized":"2024-04-25T08:29:29.895594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076370+0000","last_change":"2024-04-25T08:29:28.076816+0000","last_active":"2024-04-25T08:29:29.076370+0000","last_peered":"2024-04-25T08:29:29.076370+0000","last_clean":"2024-04-25T08:29:29.076370+0000","last_became_active":"2024-04-25T08:29:28.076348+0000","last_became_peered":"2024-04-25T08:29:28.076348+0000","last_unstale":"2024-04-25T08:29:29.076370+0000","last_undegraded":"2024-04-25T08:29:29.076370+0000","last_fullsized":"2024-04-25T08:29:29.076370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895804+0000","last_change":"2024-04-25T08:29:24.047938+0000","last_active":"2024-04-25T08:29:29.895804+0000","last_peered":"2024-04-25T08:29:29.895804+0000","last_clean":"2024-04-25T08:29:29.895804+0000","last_became_active":"2024-04-25T08:29:24.047670+0000","last_became_peered":"2024-04-25T08:29:24.047670+0000","last_unstale":"2024-04-25T08:29:29.895804+0000","last_undegraded":"2024-04-25T08:29:29.895804+0000","last_fullsized":"2024-04-25T08:29:29.895804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074744+0000","last_change":"2024-04-25T08:29:23.037162+0000","last_active":"2024-04-25T08:29:30.074744+0000","last_peered":"2024-04-25T08:29:30.074744+0000","last_clean":"2024-04-25T08:29:30.074744+0000","last_became_active":"2024-04-25T08:29:23.036850+0000","last_became_peered":"2024-04-25T08:29:23.036850+0000","last_unstale":"2024-04-25T08:29:30.074744+0000","last_undegraded":"2024-04-25T08:29:30.074744+0000","last_fullsized":"2024-04-25T08:29:30.074744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074972+0000","last_change":"2024-04-25T08:29:28.073263+0000","last_active":"2024-04-25T08:29:29.074972+0000","last_peered":"2024-04-25T08:29:29.074972+0000","last_clean":"2024-04-25T08:29:29.074972+0000","last_became_active":"2024-04-25T08:29:28.072784+0000","last_became_peered":"2024-04-25T08:29:28.072784+0000","last_unstale":"2024-04-25T08:29:29.074972+0000","last_undegraded":"2024-04-25T08:29:29.074972+0000","last_fullsized":"2024-04-25T08:29:29.074972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574458+0000","last_change":"2024-04-25T08:29:24.056206+0000","last_active":"2024-04-25T08:29:29.574458+0000","last_peered":"2024-04-25T08:29:29.574458+0000","last_clean":"2024-04-25T08:29:29.574458+0000","last_became_active":"2024-04-25T08:29:24.055557+0000","last_became_peered":"2024-04-25T08:29:24.055557+0000","last_unstale":"2024-04-25T08:29:29.574458+0000","last_undegraded":"2024-04-25T08:29:29.574458+0000","last_fullsized":"2024-04-25T08:29:29.574458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075008+0000","last_change":"2024-04-25T08:29:23.031709+0000","last_active":"2024-04-25T08:29:30.075008+0000","last_peered":"2024-04-25T08:29:30.075008+0000","last_clean":"2024-04-25T08:29:30.075008+0000","last_became_active":"2024-04-25T08:29:23.031419+0000","last_became_peered":"2024-04-25T08:29:23.031419+0000","last_unstale":"2024-04-25T08:29:30.075008+0000","last_undegraded":"2024-04-25T08:29:30.075008+0000","last_fullsized":"2024-04-25T08:29:30.075008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906475+0000","last_change":"2024-04-25T08:29:28.070276+0000","last_active":"2024-04-25T08:29:29.906475+0000","last_peered":"2024-04-25T08:29:29.906475+0000","last_clean":"2024-04-25T08:29:29.906475+0000","last_became_active":"2024-04-25T08:29:28.069869+0000","last_became_peered":"2024-04-25T08:29:28.069869+0000","last_unstale":"2024-04-25T08:29:29.906475+0000","last_undegraded":"2024-04-25T08:29:29.906475+0000","last_fullsized":"2024-04-25T08:29:29.906475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896237+0000","last_change":"2024-04-25T08:29:24.057274+0000","last_active":"2024-04-25T08:29:29.896237+0000","last_peered":"2024-04-25T08:29:29.896237+0000","last_clean":"2024-04-25T08:29:29.896237+0000","last_became_active":"2024-04-25T08:29:24.056998+0000","last_became_peered":"2024-04-25T08:29:24.056998+0000","last_unstale":"2024-04-25T08:29:29.896237+0000","last_undegraded":"2024-04-25T08:29:29.896237+0000","last_fullsized":"2024-04-25T08:29:29.896237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:18.584258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074824+0000","last_change":"2024-04-25T08:29:23.039945+0000","last_active":"2024-04-25T08:29:30.074824+0000","last_peered":"2024-04-25T08:29:30.074824+0000","last_clean":"2024-04-25T08:29:30.074824+0000","last_became_active":"2024-04-25T08:29:23.039563+0000","last_became_peered":"2024-04-25T08:29:23.039563+0000","last_unstale":"2024-04-25T08:29:30.074824+0000","last_undegraded":"2024-04-25T08:29:30.074824+0000","last_fullsized":"2024-04-25T08:29:30.074824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573881+0000","last_change":"2024-04-25T08:29:28.076465+0000","last_active":"2024-04-25T08:29:29.573881+0000","last_peered":"2024-04-25T08:29:29.573881+0000","last_clean":"2024-04-25T08:29:29.573881+0000","last_became_active":"2024-04-25T08:29:28.076064+0000","last_became_peered":"2024-04-25T08:29:28.076064+0000","last_unstale":"2024-04-25T08:29:29.573881+0000","last_undegraded":"2024-04-25T08:29:29.573881+0000","last_fullsized":"2024-04-25T08:29:29.573881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896089+0000","last_change":"2024-04-25T08:29:24.050215+0000","last_active":"2024-04-25T08:29:29.896089+0000","last_peered":"2024-04-25T08:29:29.896089+0000","last_clean":"2024-04-25T08:29:29.896089+0000","last_became_active":"2024-04-25T08:29:24.048694+0000","last_became_peered":"2024-04-25T08:29:24.048694+0000","last_unstale":"2024-04-25T08:29:29.896089+0000","last_undegraded":"2024-04-25T08:29:29.896089+0000","last_fullsized":"2024-04-25T08:29:29.896089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075997+0000","last_change":"2024-04-25T08:29:23.033858+0000","last_active":"2024-04-25T08:29:29.075997+0000","last_peered":"2024-04-25T08:29:29.075997+0000","last_clean":"2024-04-25T08:29:29.075997+0000","last_became_active":"2024-04-25T08:29:23.033599+0000","last_became_peered":"2024-04-25T08:29:23.033599+0000","last_unstale":"2024-04-25T08:29:29.075997+0000","last_undegraded":"2024-04-25T08:29:29.075997+0000","last_fullsized":"2024-04-25T08:29:29.075997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074730+0000","last_change":"2024-04-25T08:29:28.081641+0000","last_active":"2024-04-25T08:29:29.074730+0000","last_peered":"2024-04-25T08:29:29.074730+0000","last_clean":"2024-04-25T08:29:29.074730+0000","last_became_active":"2024-04-25T08:29:28.081220+0000","last_became_peered":"2024-04-25T08:29:28.081220+0000","last_unstale":"2024-04-25T08:29:29.074730+0000","last_undegraded":"2024-04-25T08:29:29.074730+0000","last_fullsized":"2024-04-25T08:29:29.074730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:51.749033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895682+0000","last_change":"2024-04-25T08:29:24.050072+0000","last_active":"2024-04-25T08:29:29.895682+0000","last_peered":"2024-04-25T08:29:29.895682+0000","last_clean":"2024-04-25T08:29:29.895682+0000","last_became_active":"2024-04-25T08:29:24.049773+0000","last_became_peered":"2024-04-25T08:29:24.049773+0000","last_unstale":"2024-04-25T08:29:29.895682+0000","last_undegraded":"2024-04-25T08:29:29.895682+0000","last_fullsized":"2024-04-25T08:29:29.895682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074062+0000","last_change":"2024-04-25T08:29:23.035476+0000","last_active":"2024-04-25T08:29:30.074062+0000","last_peered":"2024-04-25T08:29:30.074062+0000","last_clean":"2024-04-25T08:29:30.074062+0000","last_became_active":"2024-04-25T08:29:23.035129+0000","last_became_peered":"2024-04-25T08:29:23.035129+0000","last_unstale":"2024-04-25T08:29:30.074062+0000","last_undegraded":"2024-04-25T08:29:30.074062+0000","last_fullsized":"2024-04-25T08:29:30.074062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.454939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906185+0000","last_change":"2024-04-25T08:29:28.076678+0000","last_active":"2024-04-25T08:29:29.906185+0000","last_peered":"2024-04-25T08:29:29.906185+0000","last_clean":"2024-04-25T08:29:29.906185+0000","last_became_active":"2024-04-25T08:29:28.076283+0000","last_became_peered":"2024-04-25T08:29:28.076283+0000","last_unstale":"2024-04-25T08:29:29.906185+0000","last_undegraded":"2024-04-25T08:29:29.906185+0000","last_fullsized":"2024-04-25T08:29:29.906185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075358+0000","last_change":"2024-04-25T08:29:24.041672+0000","last_active":"2024-04-25T08:29:29.075358+0000","last_peered":"2024-04-25T08:29:29.075358+0000","last_clean":"2024-04-25T08:29:29.075358+0000","last_became_active":"2024-04-25T08:29:24.040865+0000","last_became_peered":"2024-04-25T08:29:24.040865+0000","last_unstale":"2024-04-25T08:29:29.075358+0000","last_undegraded":"2024-04-25T08:29:29.075358+0000","last_fullsized":"2024-04-25T08:29:29.075358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075805+0000","last_change":"2024-04-25T08:29:23.028173+0000","last_active":"2024-04-25T08:29:29.075805+0000","last_peered":"2024-04-25T08:29:29.075805+0000","last_clean":"2024-04-25T08:29:29.075805+0000","last_became_active":"2024-04-25T08:29:23.027886+0000","last_became_peered":"2024-04-25T08:29:23.027886+0000","last_unstale":"2024-04-25T08:29:29.075805+0000","last_undegraded":"2024-04-25T08:29:29.075805+0000","last_fullsized":"2024-04-25T08:29:29.075805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076047+0000","last_change":"2024-04-25T08:29:28.084524+0000","last_active":"2024-04-25T08:29:29.076047+0000","last_peered":"2024-04-25T08:29:29.076047+0000","last_clean":"2024-04-25T08:29:29.076047+0000","last_became_active":"2024-04-25T08:29:28.083769+0000","last_became_peered":"2024-04-25T08:29:28.083769+0000","last_unstale":"2024-04-25T08:29:29.076047+0000","last_undegraded":"2024-04-25T08:29:29.076047+0000","last_fullsized":"2024-04-25T08:29:29.076047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:43.115368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895795+0000","last_change":"2024-04-25T08:29:24.057400+0000","last_active":"2024-04-25T08:29:29.895795+0000","last_peered":"2024-04-25T08:29:29.895795+0000","last_clean":"2024-04-25T08:29:29.895795+0000","last_became_active":"2024-04-25T08:29:24.057074+0000","last_became_peered":"2024-04-25T08:29:24.057074+0000","last_unstale":"2024-04-25T08:29:29.895795+0000","last_undegraded":"2024-04-25T08:29:29.895795+0000","last_fullsized":"2024-04-25T08:29:29.895795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574339+0000","last_change":"2024-04-25T08:29:23.036448+0000","last_active":"2024-04-25T08:29:29.574339+0000","last_peered":"2024-04-25T08:29:29.574339+0000","last_clean":"2024-04-25T08:29:29.574339+0000","last_became_active":"2024-04-25T08:29:23.036172+0000","last_became_peered":"2024-04-25T08:29:23.036172+0000","last_unstale":"2024-04-25T08:29:29.574339+0000","last_undegraded":"2024-04-25T08:29:29.574339+0000","last_fullsized":"2024-04-25T08:29:29.574339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075577+0000","last_change":"2024-04-25T08:29:28.083554+0000","last_active":"2024-04-25T08:29:29.075577+0000","last_peered":"2024-04-25T08:29:29.075577+0000","last_clean":"2024-04-25T08:29:29.075577+0000","last_became_active":"2024-04-25T08:29:28.083178+0000","last_became_peered":"2024-04-25T08:29:28.083178+0000","last_unstale":"2024-04-25T08:29:29.075577+0000","last_undegraded":"2024-04-25T08:29:29.075577+0000","last_fullsized":"2024-04-25T08:29:29.075577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906194+0000","last_change":"2024-04-25T08:29:24.049668+0000","last_active":"2024-04-25T08:29:29.906194+0000","last_peered":"2024-04-25T08:29:29.906194+0000","last_clean":"2024-04-25T08:29:29.906194+0000","last_became_active":"2024-04-25T08:29:24.049359+0000","last_became_peered":"2024-04-25T08:29:24.049359+0000","last_unstale":"2024-04-25T08:29:29.906194+0000","last_undegraded":"2024-04-25T08:29:29.906194+0000","last_fullsized":"2024-04-25T08:29:29.906194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:48.609719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.030224+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.029936+0000","last_became_peered":"2024-04-25T08:29:23.029936+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:17.202033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574628+0000","last_change":"2024-04-25T08:29:28.076915+0000","last_active":"2024-04-25T08:29:29.574628+0000","last_peered":"2024-04-25T08:29:29.574628+0000","last_clean":"2024-04-25T08:29:29.574628+0000","last_became_active":"2024-04-25T08:29:28.076527+0000","last_became_peered":"2024-04-25T08:29:28.076527+0000","last_unstale":"2024-04-25T08:29:29.574628+0000","last_undegraded":"2024-04-25T08:29:29.574628+0000","last_fullsized":"2024-04-25T08:29:29.574628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895697+0000","last_change":"2024-04-25T08:29:24.053639+0000","last_active":"2024-04-25T08:29:29.895697+0000","last_peered":"2024-04-25T08:29:29.895697+0000","last_clean":"2024-04-25T08:29:29.895697+0000","last_became_active":"2024-04-25T08:29:24.053028+0000","last_became_peered":"2024-04-25T08:29:24.053028+0000","last_unstale":"2024-04-25T08:29:29.895697+0000","last_undegraded":"2024-04-25T08:29:29.895697+0000","last_fullsized":"2024-04-25T08:29:29.895697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:32.823362+0000","last_change":"2024-04-25T08:29:23.031850+0000","last_active":"2024-04-25T08:29:32.823362+0000","last_peered":"2024-04-25T08:29:32.823362+0000","last_clean":"2024-04-25T08:29:32.823362+0000","last_became_active":"2024-04-25T08:29:23.031588+0000","last_became_peered":"2024-04-25T08:29:23.031588+0000","last_unstale":"2024-04-25T08:29:32.823362+0000","last_undegraded":"2024-04-25T08:29:32.823362+0000","last_fullsized":"2024-04-25T08:29:32.823362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074668+0000","last_change":"2024-04-25T08:29:28.066135+0000","last_active":"2024-04-25T08:29:30.074668+0000","last_peered":"2024-04-25T08:29:30.074668+0000","last_clean":"2024-04-25T08:29:30.074668+0000","last_became_active":"2024-04-25T08:29:28.065611+0000","last_became_peered":"2024-04-25T08:29:28.065611+0000","last_unstale":"2024-04-25T08:29:30.074668+0000","last_undegraded":"2024-04-25T08:29:30.074668+0000","last_fullsized":"2024-04-25T08:29:30.074668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075847+0000","last_change":"2024-04-25T08:29:24.050678+0000","last_active":"2024-04-25T08:29:29.075847+0000","last_peered":"2024-04-25T08:29:29.075847+0000","last_clean":"2024-04-25T08:29:29.075847+0000","last_became_active":"2024-04-25T08:29:24.049995+0000","last_became_peered":"2024-04-25T08:29:24.049995+0000","last_unstale":"2024-04-25T08:29:29.075847+0000","last_undegraded":"2024-04-25T08:29:29.075847+0000","last_fullsized":"2024-04-25T08:29:29.075847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.729278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075000+0000","last_change":"2024-04-25T08:29:23.036900+0000","last_active":"2024-04-25T08:29:30.075000+0000","last_peered":"2024-04-25T08:29:30.075000+0000","last_clean":"2024-04-25T08:29:30.075000+0000","last_became_active":"2024-04-25T08:29:23.036630+0000","last_became_peered":"2024-04-25T08:29:23.036630+0000","last_unstale":"2024-04-25T08:29:30.075000+0000","last_undegraded":"2024-04-25T08:29:30.075000+0000","last_fullsized":"2024-04-25T08:29:30.075000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075542+0000","last_change":"2024-04-25T08:29:28.066730+0000","last_active":"2024-04-25T08:29:29.075542+0000","last_peered":"2024-04-25T08:29:29.075542+0000","last_clean":"2024-04-25T08:29:29.075542+0000","last_became_active":"2024-04-25T08:29:28.066323+0000","last_became_peered":"2024-04-25T08:29:28.066323+0000","last_unstale":"2024-04-25T08:29:29.075542+0000","last_undegraded":"2024-04-25T08:29:29.075542+0000","last_fullsized":"2024-04-25T08:29:29.075542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906473+0000","last_change":"2024-04-25T08:29:24.035898+0000","last_active":"2024-04-25T08:29:29.906473+0000","last_peered":"2024-04-25T08:29:29.906473+0000","last_clean":"2024-04-25T08:29:29.906473+0000","last_became_active":"2024-04-25T08:29:24.035137+0000","last_became_peered":"2024-04-25T08:29:24.035137+0000","last_unstale":"2024-04-25T08:29:29.906473+0000","last_undegraded":"2024-04-25T08:29:29.906473+0000","last_fullsized":"2024-04-25T08:29:29.906473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574599+0000","last_change":"2024-04-25T08:29:23.036994+0000","last_active":"2024-04-25T08:29:29.574599+0000","last_peered":"2024-04-25T08:29:29.574599+0000","last_clean":"2024-04-25T08:29:29.574599+0000","last_became_active":"2024-04-25T08:29:23.036740+0000","last_became_peered":"2024-04-25T08:29:23.036740+0000","last_unstale":"2024-04-25T08:29:29.574599+0000","last_undegraded":"2024-04-25T08:29:29.574599+0000","last_fullsized":"2024-04-25T08:29:29.574599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074766+0000","last_change":"2024-04-25T08:29:28.080727+0000","last_active":"2024-04-25T08:29:29.074766+0000","last_peered":"2024-04-25T08:29:29.074766+0000","last_clean":"2024-04-25T08:29:29.074766+0000","last_became_active":"2024-04-25T08:29:28.080288+0000","last_became_peered":"2024-04-25T08:29:28.080288+0000","last_unstale":"2024-04-25T08:29:29.074766+0000","last_undegraded":"2024-04-25T08:29:29.074766+0000","last_fullsized":"2024-04-25T08:29:29.074766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.184309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574484+0000","last_change":"2024-04-25T08:29:24.054205+0000","last_active":"2024-04-25T08:29:29.574484+0000","last_peered":"2024-04-25T08:29:29.574484+0000","last_clean":"2024-04-25T08:29:29.574484+0000","last_became_active":"2024-04-25T08:29:24.050796+0000","last_became_peered":"2024-04-25T08:29:24.050796+0000","last_unstale":"2024-04-25T08:29:29.574484+0000","last_undegraded":"2024-04-25T08:29:29.574484+0000","last_fullsized":"2024-04-25T08:29:29.574484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076454+0000","last_change":"2024-04-25T08:29:23.037467+0000","last_active":"2024-04-25T08:29:29.076454+0000","last_peered":"2024-04-25T08:29:29.076454+0000","last_clean":"2024-04-25T08:29:29.076454+0000","last_became_active":"2024-04-25T08:29:23.037207+0000","last_became_peered":"2024-04-25T08:29:23.037207+0000","last_unstale":"2024-04-25T08:29:29.076454+0000","last_undegraded":"2024-04-25T08:29:29.076454+0000","last_fullsized":"2024-04-25T08:29:29.076454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075540+0000","last_change":"2024-04-25T08:29:28.085289+0000","last_active":"2024-04-25T08:29:29.075540+0000","last_peered":"2024-04-25T08:29:29.075540+0000","last_clean":"2024-04-25T08:29:29.075540+0000","last_became_active":"2024-04-25T08:29:28.084815+0000","last_became_peered":"2024-04-25T08:29:28.084815+0000","last_unstale":"2024-04-25T08:29:29.075540+0000","last_undegraded":"2024-04-25T08:29:29.075540+0000","last_fullsized":"2024-04-25T08:29:29.075540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906069+0000","last_change":"2024-04-25T08:29:28.077759+0000","last_active":"2024-04-25T08:29:29.906069+0000","last_peered":"2024-04-25T08:29:29.906069+0000","last_clean":"2024-04-25T08:29:29.906069+0000","last_became_active":"2024-04-25T08:29:28.077376+0000","last_became_peered":"2024-04-25T08:29:28.077376+0000","last_unstale":"2024-04-25T08:29:29.906069+0000","last_undegraded":"2024-04-25T08:29:29.906069+0000","last_fullsized":"2024-04-25T08:29:29.906069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074258+0000","last_change":"2024-04-25T08:29:24.053405+0000","last_active":"2024-04-25T08:29:30.074258+0000","last_peered":"2024-04-25T08:29:30.074258+0000","last_clean":"2024-04-25T08:29:30.074258+0000","last_became_active":"2024-04-25T08:29:24.053104+0000","last_became_peered":"2024-04-25T08:29:24.053104+0000","last_unstale":"2024-04-25T08:29:30.074258+0000","last_undegraded":"2024-04-25T08:29:30.074258+0000","last_fullsized":"2024-04-25T08:29:30.074258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574014+0000","last_change":"2024-04-25T08:29:23.034808+0000","last_active":"2024-04-25T08:29:29.574014+0000","last_peered":"2024-04-25T08:29:29.574014+0000","last_clean":"2024-04-25T08:29:29.574014+0000","last_became_active":"2024-04-25T08:29:23.034545+0000","last_became_peered":"2024-04-25T08:29:23.034545+0000","last_unstale":"2024-04-25T08:29:29.574014+0000","last_undegraded":"2024-04-25T08:29:29.574014+0000","last_fullsized":"2024-04-25T08:29:29.574014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076514+0000","last_change":"2024-04-25T08:29:28.068503+0000","last_active":"2024-04-25T08:29:29.076514+0000","last_peered":"2024-04-25T08:29:29.076514+0000","last_clean":"2024-04-25T08:29:29.076514+0000","last_became_active":"2024-04-25T08:29:28.068080+0000","last_became_peered":"2024-04-25T08:29:28.068080+0000","last_unstale":"2024-04-25T08:29:29.076514+0000","last_undegraded":"2024-04-25T08:29:29.076514+0000","last_fullsized":"2024-04-25T08:29:29.076514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074757+0000","last_change":"2024-04-25T08:29:24.053631+0000","last_active":"2024-04-25T08:29:29.074757+0000","last_peered":"2024-04-25T08:29:29.074757+0000","last_clean":"2024-04-25T08:29:29.074757+0000","last_became_active":"2024-04-25T08:29:24.052293+0000","last_became_peered":"2024-04-25T08:29:24.052293+0000","last_unstale":"2024-04-25T08:29:29.074757+0000","last_undegraded":"2024-04-25T08:29:29.074757+0000","last_fullsized":"2024-04-25T08:29:29.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074995+0000","last_change":"2024-04-25T08:29:23.029639+0000","last_active":"2024-04-25T08:29:29.074995+0000","last_peered":"2024-04-25T08:29:29.074995+0000","last_clean":"2024-04-25T08:29:29.074995+0000","last_became_active":"2024-04-25T08:29:23.027908+0000","last_became_peered":"2024-04-25T08:29:23.027908+0000","last_unstale":"2024-04-25T08:29:29.074995+0000","last_undegraded":"2024-04-25T08:29:29.074995+0000","last_fullsized":"2024-04-25T08:29:29.074995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:02.366427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075903+0000","last_change":"2024-04-25T08:29:23.024619+0000","last_active":"2024-04-25T08:29:29.075903+0000","last_peered":"2024-04-25T08:29:29.075903+0000","last_clean":"2024-04-25T08:29:29.075903+0000","last_became_active":"2024-04-25T08:29:23.024364+0000","last_became_peered":"2024-04-25T08:29:23.024364+0000","last_unstale":"2024-04-25T08:29:29.075903+0000","last_undegraded":"2024-04-25T08:29:29.075903+0000","last_fullsized":"2024-04-25T08:29:29.075903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906507+0000","last_change":"2024-04-25T08:29:24.040912+0000","last_active":"2024-04-25T08:29:29.906507+0000","last_peered":"2024-04-25T08:29:29.906507+0000","last_clean":"2024-04-25T08:29:29.906507+0000","last_became_active":"2024-04-25T08:29:24.040629+0000","last_became_peered":"2024-04-25T08:29:24.040629+0000","last_unstale":"2024-04-25T08:29:29.906507+0000","last_undegraded":"2024-04-25T08:29:29.906507+0000","last_fullsized":"2024-04-25T08:29:29.906507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074576+0000","last_change":"2024-04-25T08:29:28.081077+0000","last_active":"2024-04-25T08:29:30.074576+0000","last_peered":"2024-04-25T08:29:30.074576+0000","last_clean":"2024-04-25T08:29:30.074576+0000","last_became_active":"2024-04-25T08:29:28.080470+0000","last_became_peered":"2024-04-25T08:29:28.080470+0000","last_unstale":"2024-04-25T08:29:30.074576+0000","last_undegraded":"2024-04-25T08:29:30.074576+0000","last_fullsized":"2024-04-25T08:29:30.074576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:18.082525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075155+0000","last_change":"2024-04-25T08:29:23.039762+0000","last_active":"2024-04-25T08:29:29.075155+0000","last_peered":"2024-04-25T08:29:29.075155+0000","last_clean":"2024-04-25T08:29:29.075155+0000","last_became_active":"2024-04-25T08:29:23.039484+0000","last_became_peered":"2024-04-25T08:29:23.039484+0000","last_unstale":"2024-04-25T08:29:29.075155+0000","last_undegraded":"2024-04-25T08:29:29.075155+0000","last_fullsized":"2024-04-25T08:29:29.075155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074927+0000","last_change":"2024-04-25T08:29:24.037624+0000","last_active":"2024-04-25T08:29:29.074927+0000","last_peered":"2024-04-25T08:29:29.074927+0000","last_clean":"2024-04-25T08:29:29.074927+0000","last_became_active":"2024-04-25T08:29:24.037352+0000","last_became_peered":"2024-04-25T08:29:24.037352+0000","last_unstale":"2024-04-25T08:29:29.074927+0000","last_undegraded":"2024-04-25T08:29:29.074927+0000","last_fullsized":"2024-04-25T08:29:29.074927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:51.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075230+0000","last_change":"2024-04-25T08:29:28.083366+0000","last_active":"2024-04-25T08:29:29.075230+0000","last_peered":"2024-04-25T08:29:29.075230+0000","last_clean":"2024-04-25T08:29:29.075230+0000","last_became_active":"2024-04-25T08:29:28.082870+0000","last_became_peered":"2024-04-25T08:29:28.082870+0000","last_unstale":"2024-04-25T08:29:29.075230+0000","last_undegraded":"2024-04-25T08:29:29.075230+0000","last_fullsized":"2024-04-25T08:29:29.075230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:08.399424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075038+0000","last_change":"2024-04-25T08:29:24.042078+0000","last_active":"2024-04-25T08:29:29.075038+0000","last_peered":"2024-04-25T08:29:29.075038+0000","last_clean":"2024-04-25T08:29:29.075038+0000","last_became_active":"2024-04-25T08:29:24.041318+0000","last_became_peered":"2024-04-25T08:29:24.041318+0000","last_unstale":"2024-04-25T08:29:29.075038+0000","last_undegraded":"2024-04-25T08:29:29.075038+0000","last_fullsized":"2024-04-25T08:29:29.075038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074154+0000","last_change":"2024-04-25T08:29:24.043006+0000","last_active":"2024-04-25T08:29:30.074154+0000","last_peered":"2024-04-25T08:29:30.074154+0000","last_clean":"2024-04-25T08:29:30.074154+0000","last_became_active":"2024-04-25T08:29:24.042553+0000","last_became_peered":"2024-04-25T08:29:24.042553+0000","last_unstale":"2024-04-25T08:29:30.074154+0000","last_undegraded":"2024-04-25T08:29:30.074154+0000","last_fullsized":"2024-04-25T08:29:30.074154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574181+0000","last_change":"2024-04-25T08:29:24.054678+0000","last_active":"2024-04-25T08:29:29.574181+0000","last_peered":"2024-04-25T08:29:29.574181+0000","last_clean":"2024-04-25T08:29:29.574181+0000","last_became_active":"2024-04-25T08:29:24.054432+0000","last_became_peered":"2024-04-25T08:29:24.054432+0000","last_unstale":"2024-04-25T08:29:29.574181+0000","last_undegraded":"2024-04-25T08:29:29.574181+0000","last_fullsized":"2024-04-25T08:29:29.574181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906457+0000","last_change":"2024-04-25T08:29:24.047684+0000","last_active":"2024-04-25T08:29:29.906457+0000","last_peered":"2024-04-25T08:29:29.906457+0000","last_clean":"2024-04-25T08:29:29.906457+0000","last_became_active":"2024-04-25T08:29:24.047384+0000","last_became_peered":"2024-04-25T08:29:24.047384+0000","last_unstale":"2024-04-25T08:29:29.906457+0000","last_undegraded":"2024-04-25T08:29:29.906457+0000","last_fullsized":"2024-04-25T08:29:29.906457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075571+0000","last_change":"2024-04-25T08:29:24.044573+0000","last_active":"2024-04-25T08:29:29.075571+0000","last_peered":"2024-04-25T08:29:29.075571+0000","last_clean":"2024-04-25T08:29:29.075571+0000","last_became_active":"2024-04-25T08:29:24.043980+0000","last_became_peered":"2024-04-25T08:29:24.043980+0000","last_unstale":"2024-04-25T08:29:29.075571+0000","last_undegraded":"2024-04-25T08:29:29.075571+0000","last_fullsized":"2024-04-25T08:29:29.075571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074876+0000","last_change":"2024-04-25T08:29:24.053456+0000","last_active":"2024-04-25T08:29:30.074876+0000","last_peered":"2024-04-25T08:29:30.074876+0000","last_clean":"2024-04-25T08:29:30.074876+0000","last_became_active":"2024-04-25T08:29:24.053144+0000","last_became_peered":"2024-04-25T08:29:24.053144+0000","last_unstale":"2024-04-25T08:29:30.074876+0000","last_undegraded":"2024-04-25T08:29:30.074876+0000","last_fullsized":"2024-04-25T08:29:30.074876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076592+0000","last_change":"2024-04-25T08:29:24.052047+0000","last_active":"2024-04-25T08:29:29.076592+0000","last_peered":"2024-04-25T08:29:29.076592+0000","last_clean":"2024-04-25T08:29:29.076592+0000","last_became_active":"2024-04-25T08:29:24.051777+0000","last_became_peered":"2024-04-25T08:29:24.051777+0000","last_unstale":"2024-04-25T08:29:29.076592+0000","last_undegraded":"2024-04-25T08:29:29.076592+0000","last_fullsized":"2024-04-25T08:29:29.076592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075986+0000","last_change":"2024-04-25T08:29:24.044743+0000","last_active":"2024-04-25T08:29:29.075986+0000","last_peered":"2024-04-25T08:29:29.075986+0000","last_clean":"2024-04-25T08:29:29.075986+0000","last_became_active":"2024-04-25T08:29:24.044425+0000","last_became_peered":"2024-04-25T08:29:24.044425+0000","last_unstale":"2024-04-25T08:29:29.075986+0000","last_undegraded":"2024-04-25T08:29:29.075986+0000","last_fullsized":"2024-04-25T08:29:29.075986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573967+0000","last_change":"2024-04-25T08:29:24.047484+0000","last_active":"2024-04-25T08:29:29.573967+0000","last_peered":"2024-04-25T08:29:29.573967+0000","last_clean":"2024-04-25T08:29:29.573967+0000","last_became_active":"2024-04-25T08:29:24.046848+0000","last_became_peered":"2024-04-25T08:29:24.046848+0000","last_unstale":"2024-04-25T08:29:29.573967+0000","last_undegraded":"2024-04-25T08:29:29.573967+0000","last_fullsized":"2024-04-25T08:29:29.573967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074875+0000","last_change":"2024-04-25T08:29:24.048838+0000","last_active":"2024-04-25T08:29:29.074875+0000","last_peered":"2024-04-25T08:29:29.074875+0000","last_clean":"2024-04-25T08:29:29.074875+0000","last_became_active":"2024-04-25T08:29:24.047272+0000","last_became_peered":"2024-04-25T08:29:24.047272+0000","last_unstale":"2024-04-25T08:29:29.074875+0000","last_undegraded":"2024-04-25T08:29:29.074875+0000","last_fullsized":"2024-04-25T08:29:29.074875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574182+0000","last_change":"2024-04-25T08:29:24.052059+0000","last_active":"2024-04-25T08:29:29.574182+0000","last_peered":"2024-04-25T08:29:29.574182+0000","last_clean":"2024-04-25T08:29:29.574182+0000","last_became_active":"2024-04-25T08:29:24.051604+0000","last_became_peered":"2024-04-25T08:29:24.051604+0000","last_unstale":"2024-04-25T08:29:29.574182+0000","last_undegraded":"2024-04-25T08:29:29.574182+0000","last_fullsized":"2024-04-25T08:29:29.574182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906362+0000","last_change":"2024-04-25T08:29:24.049843+0000","last_active":"2024-04-25T08:29:29.906362+0000","last_peered":"2024-04-25T08:29:29.906362+0000","last_clean":"2024-04-25T08:29:29.906362+0000","last_became_active":"2024-04-25T08:29:24.049562+0000","last_became_peered":"2024-04-25T08:29:24.049562+0000","last_unstale":"2024-04-25T08:29:29.906362+0000","last_undegraded":"2024-04-25T08:29:29.906362+0000","last_fullsized":"2024-04-25T08:29:29.906362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.370927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.041378+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.040860+0000","last_became_peered":"2024-04-25T08:29:24.040860+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895964+0000","last_change":"2024-04-25T08:29:24.050860+0000","last_active":"2024-04-25T08:29:29.895964+0000","last_peered":"2024-04-25T08:29:29.895964+0000","last_clean":"2024-04-25T08:29:29.895964+0000","last_became_active":"2024-04-25T08:29:24.050610+0000","last_became_peered":"2024-04-25T08:29:24.050610+0000","last_unstale":"2024-04-25T08:29:29.895964+0000","last_undegraded":"2024-04-25T08:29:29.895964+0000","last_fullsized":"2024-04-25T08:29:29.895964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074757+0000","last_change":"2024-04-25T08:29:24.041470+0000","last_active":"2024-04-25T08:29:30.074757+0000","last_peered":"2024-04-25T08:29:30.074757+0000","last_clean":"2024-04-25T08:29:30.074757+0000","last_became_active":"2024-04-25T08:29:24.041172+0000","last_became_peered":"2024-04-25T08:29:24.041172+0000","last_unstale":"2024-04-25T08:29:30.074757+0000","last_undegraded":"2024-04-25T08:29:30.074757+0000","last_fullsized":"2024-04-25T08:29:30.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574663+0000","last_change":"2024-04-25T08:29:24.054784+0000","last_active":"2024-04-25T08:29:29.574663+0000","last_peered":"2024-04-25T08:29:29.574663+0000","last_clean":"2024-04-25T08:29:29.574663+0000","last_became_active":"2024-04-25T08:29:24.054538+0000","last_became_peered":"2024-04-25T08:29:24.054538+0000","last_unstale":"2024-04-25T08:29:29.574663+0000","last_undegraded":"2024-04-25T08:29:29.574663+0000","last_fullsized":"2024-04-25T08:29:29.574663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573870+0000","last_change":"2024-04-25T08:29:24.046213+0000","last_active":"2024-04-25T08:29:29.573870+0000","last_peered":"2024-04-25T08:29:29.573870+0000","last_clean":"2024-04-25T08:29:29.573870+0000","last_became_active":"2024-04-25T08:29:24.045850+0000","last_became_peered":"2024-04-25T08:29:24.045850+0000","last_unstale":"2024-04-25T08:29:29.573870+0000","last_undegraded":"2024-04-25T08:29:29.573870+0000","last_fullsized":"2024-04-25T08:29:29.573870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074307+0000","last_change":"2024-04-25T08:29:24.039662+0000","last_active":"2024-04-25T08:29:30.074307+0000","last_peered":"2024-04-25T08:29:30.074307+0000","last_clean":"2024-04-25T08:29:30.074307+0000","last_became_active":"2024-04-25T08:29:24.039338+0000","last_became_peered":"2024-04-25T08:29:24.039338+0000","last_unstale":"2024-04-25T08:29:30.074307+0000","last_undegraded":"2024-04-25T08:29:30.074307+0000","last_fullsized":"2024-04-25T08:29:30.074307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906168+0000","last_change":"2024-04-25T08:29:24.038228+0000","last_active":"2024-04-25T08:29:29.906168+0000","last_peered":"2024-04-25T08:29:29.906168+0000","last_clean":"2024-04-25T08:29:29.906168+0000","last_became_active":"2024-04-25T08:29:24.037900+0000","last_became_peered":"2024-04-25T08:29:24.037900+0000","last_unstale":"2024-04-25T08:29:29.906168+0000","last_undegraded":"2024-04-25T08:29:29.906168+0000","last_fullsized":"2024-04-25T08:29:29.906168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:35.121139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074948+0000","last_change":"2024-04-25T08:29:24.045041+0000","last_active":"2024-04-25T08:29:29.074948+0000","last_peered":"2024-04-25T08:29:29.074948+0000","last_clean":"2024-04-25T08:29:29.074948+0000","last_became_active":"2024-04-25T08:29:24.044581+0000","last_became_peered":"2024-04-25T08:29:24.044581+0000","last_unstale":"2024-04-25T08:29:29.074948+0000","last_undegraded":"2024-04-25T08:29:29.074948+0000","last_fullsized":"2024-04-25T08:29:29.074948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075863+0000","last_change":"2024-04-25T08:29:24.042402+0000","last_active":"2024-04-25T08:29:29.075863+0000","last_peered":"2024-04-25T08:29:29.075863+0000","last_clean":"2024-04-25T08:29:29.075863+0000","last_became_active":"2024-04-25T08:29:24.042074+0000","last_became_peered":"2024-04-25T08:29:24.042074+0000","last_unstale":"2024-04-25T08:29:29.075863+0000","last_undegraded":"2024-04-25T08:29:29.075863+0000","last_fullsized":"2024-04-25T08:29:29.075863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076327+0000","last_change":"2024-04-25T08:29:24.035522+0000","last_active":"2024-04-25T08:29:29.076327+0000","last_peered":"2024-04-25T08:29:29.076327+0000","last_clean":"2024-04-25T08:29:29.076327+0000","last_became_active":"2024-04-25T08:29:24.035220+0000","last_became_peered":"2024-04-25T08:29:24.035220+0000","last_unstale":"2024-04-25T08:29:29.076327+0000","last_undegraded":"2024-04-25T08:29:29.076327+0000","last_fullsized":"2024-04-25T08:29:29.076327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076595+0000","last_change":"2024-04-25T08:29:24.035459+0000","last_active":"2024-04-25T08:29:29.076595+0000","last_peered":"2024-04-25T08:29:29.076595+0000","last_clean":"2024-04-25T08:29:29.076595+0000","last_became_active":"2024-04-25T08:29:24.035170+0000","last_became_peered":"2024-04-25T08:29:24.035170+0000","last_unstale":"2024-04-25T08:29:29.076595+0000","last_undegraded":"2024-04-25T08:29:29.076595+0000","last_fullsized":"2024-04-25T08:29:29.076595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896232+0000","last_change":"2024-04-25T08:29:24.058439+0000","last_active":"2024-04-25T08:29:29.896232+0000","last_peered":"2024-04-25T08:29:29.896232+0000","last_clean":"2024-04-25T08:29:29.896232+0000","last_became_active":"2024-04-25T08:29:24.058172+0000","last_became_peered":"2024-04-25T08:29:24.058172+0000","last_unstale":"2024-04-25T08:29:29.896232+0000","last_undegraded":"2024-04-25T08:29:29.896232+0000","last_fullsized":"2024-04-25T08:29:29.896232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:24.045871+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:24.045477+0000","last_became_peered":"2024-04-25T08:29:24.045477+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895647+0000","last_change":"2024-04-25T08:29:24.054149+0000","last_active":"2024-04-25T08:29:29.895647+0000","last_peered":"2024-04-25T08:29:29.895647+0000","last_clean":"2024-04-25T08:29:29.895647+0000","last_became_active":"2024-04-25T08:29:24.053893+0000","last_became_peered":"2024-04-25T08:29:24.053893+0000","last_unstale":"2024-04-25T08:29:29.895647+0000","last_undegraded":"2024-04-25T08:29:29.895647+0000","last_fullsized":"2024-04-25T08:29:29.895647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574047+0000","last_change":"2024-04-25T08:29:24.046245+0000","last_active":"2024-04-25T08:29:29.574047+0000","last_peered":"2024-04-25T08:29:29.574047+0000","last_clean":"2024-04-25T08:29:29.574047+0000","last_became_active":"2024-04-25T08:29:24.045892+0000","last_became_peered":"2024-04-25T08:29:24.045892+0000","last_unstale":"2024-04-25T08:29:29.574047+0000","last_undegraded":"2024-04-25T08:29:29.574047+0000","last_fullsized":"2024-04-25T08:29:29.574047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895672+0000","last_change":"2024-04-25T08:29:24.048906+0000","last_active":"2024-04-25T08:29:29.895672+0000","last_peered":"2024-04-25T08:29:29.895672+0000","last_clean":"2024-04-25T08:29:29.895672+0000","last_became_active":"2024-04-25T08:29:24.048634+0000","last_became_peered":"2024-04-25T08:29:24.048634+0000","last_unstale":"2024-04-25T08:29:29.895672+0000","last_undegraded":"2024-04-25T08:29:29.895672+0000","last_fullsized":"2024-04-25T08:29:29.895672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574336+0000","last_change":"2024-04-25T08:29:24.049689+0000","last_active":"2024-04-25T08:29:29.574336+0000","last_peered":"2024-04-25T08:29:29.574336+0000","last_clean":"2024-04-25T08:29:29.574336+0000","last_became_active":"2024-04-25T08:29:24.049210+0000","last_became_peered":"2024-04-25T08:29:24.049210+0000","last_unstale":"2024-04-25T08:29:29.574336+0000","last_undegraded":"2024-04-25T08:29:29.574336+0000","last_fullsized":"2024-04-25T08:29:29.574336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906036+0000","last_change":"2024-04-25T08:29:24.037342+0000","last_active":"2024-04-25T08:29:29.906036+0000","last_peered":"2024-04-25T08:29:29.906036+0000","last_clean":"2024-04-25T08:29:29.906036+0000","last_became_active":"2024-04-25T08:29:24.036991+0000","last_became_peered":"2024-04-25T08:29:24.036991+0000","last_unstale":"2024-04-25T08:29:29.906036+0000","last_undegraded":"2024-04-25T08:29:29.906036+0000","last_fullsized":"2024-04-25T08:29:29.906036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:14.049743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075188+0000","last_change":"2024-04-25T08:29:24.041775+0000","last_active":"2024-04-25T08:29:30.075188+0000","last_peered":"2024-04-25T08:29:30.075188+0000","last_clean":"2024-04-25T08:29:30.075188+0000","last_became_active":"2024-04-25T08:29:24.041474+0000","last_became_peered":"2024-04-25T08:29:24.041474+0000","last_unstale":"2024-04-25T08:29:30.075188+0000","last_undegraded":"2024-04-25T08:29:30.075188+0000","last_fullsized":"2024-04-25T08:29:30.075188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896139+0000","last_change":"2024-04-25T08:29:24.054127+0000","last_active":"2024-04-25T08:29:29.896139+0000","last_peered":"2024-04-25T08:29:29.896139+0000","last_clean":"2024-04-25T08:29:29.896139+0000","last_became_active":"2024-04-25T08:29:24.053854+0000","last_became_peered":"2024-04-25T08:29:24.053854+0000","last_unstale":"2024-04-25T08:29:29.896139+0000","last_undegraded":"2024-04-25T08:29:29.896139+0000","last_fullsized":"2024-04-25T08:29:29.896139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":695037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":104087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":104247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:29:41.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:29:41.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.245+0000 7f953129a640 1 Processor -- start 2024-04-25T08:29:41.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.245+0000 7f953129a640 1 -- start start 2024-04-25T08:29:41.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f953129a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f953129a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f952c0d1020 0x7f952c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f953129a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0b97a0 con 0x7f952c15dbe0 2024-04-25T08:29:41.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f953129a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c15dfc0 con 0x7f952c0d1020 2024-04-25T08:29:41.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f953129a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0d1a00 con 0x7f952c0b2cb0 2024-04-25T08:29:41.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f9523fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f952b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f952c0d1020 0x7f952c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f952affd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f952c15dbe0 0x7f952c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:42564/0 (socket says 172.21.15.60:42564) 2024-04-25T08:29:41.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f952affd640 1 -- 172.21.15.60:0/2400775877 learned_addr learned my addr 172.21.15.60:0/2400775877 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f9523fff640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:44112/0 (socket says 172.21.15.60:44112) 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f952a7fc640 1 -- 172.21.15.60:0/2400775877 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727699619 0 0) 0x7f952c0b97a0 con 0x7f952c15dbe0 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.246+0000 7f952a7fc640 1 -- 172.21.15.60:0/2400775877 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9518003520 con 0x7f952c15dbe0 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952b7fe640 1 -- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 msgr2=0x7f952c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952b7fe640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952b7fe640 1 -- 172.21.15.60:0/2400775877 >> v1:172.21.15.100:6789/0 conn(0x7f952c15dbe0 legacy=0x7f952c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952b7fe640 1 -- 172.21.15.60:0/2400775877 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0d1c50 con 0x7f952c0d1020 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f9523fff640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952b7fe640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c0d1020 0x7f952c0d2c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9524009d90 tx=0x7f952402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feaad3a707b3bc48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952a7fc640 1 -- 172.21.15.60:0/2400775877 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524007740 con 0x7f952c0d1020 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952a7fc640 1 -- 172.21.15.60:0/2400775877 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95240078e0 con 0x7f952c0d1020 2024-04-25T08:29:41.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.247+0000 7f952a7fc640 1 -- 172.21.15.60:0/2400775877 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524007cb0 con 0x7f952c0d1020 2024-04-25T08:29:41.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 -- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c0d1020 msgr2=0x7f952c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c0d1020 0x7f952c0d2c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9524009d90 tx=0x7f952402a660 comp rx=0 tx=0).stop 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 -- 172.21.15.60:0/2400775877 shutdown_connections 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c0d1020 0x7f952c0d2c70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 --2- 172.21.15.60:0/2400775877 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 -- 172.21.15.60:0/2400775877 >> 172.21.15.60:0/2400775877 conn(0x7f952c061a20 msgr2=0x7f952c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 -- 172.21.15.60:0/2400775877 shutdown_connections 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.248+0000 7f953129a640 1 -- 172.21.15.60:0/2400775877 wait complete. 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.249+0000 7f953129a640 1 Processor -- start 2024-04-25T08:29:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.249+0000 7f953129a640 1 -- start start 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f953129a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f953129a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 0x7f952c0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f953129a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c15dbe0 0x7f952c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f953129a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b92a0 con 0x7f952c0d1020 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f953129a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0d2210 con 0x7f952c15dbe0 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f953129a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0b9420 con 0x7f952c0b2cb0 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f9523fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 0x7f952c0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f9523fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 0x7f952c0b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43824/0 (socket says 172.21.15.60:43824) 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f9523fff640 1 -- 172.21.15.60:0/1031165709 learned_addr learned my addr 172.21.15.60:0/1031165709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f952b7fe640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c15dbe0 0x7f952c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.250+0000 7f952affd640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f9523fff640 1 -- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 msgr2=0x7f952c0c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f9523fff640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f9523fff640 1 -- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c15dbe0 msgr2=0x7f952c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f9523fff640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c15dbe0 0x7f952c0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f9523fff640 1 -- 172.21.15.60:0/1031165709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0cd6a0 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f952b7fe640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c15dbe0 0x7f952c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f9523fff640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 0x7f952c0b3dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f951400ebe0 tx=0x7f951400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514017070 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951400cc30 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514013470 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c0b32e0 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.251+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c0cd020 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.252+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c073990 con 0x7f952c0d1020 2024-04-25T08:29:41.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.252+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f951401a070 con 0x7f952c0d1020 2024-04-25T08:29:41.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.253+0000 7f95237fe640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f950c06b4d0 0x7f950c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.253+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f951408a820 con 0x7f952c0d1020 2024-04-25T08:29:41.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.253+0000 7f952affd640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f950c06b4d0 0x7f950c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.254+0000 7f952affd640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f950c06b4d0 0x7f950c06d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f951c000f60 tx=0x7f951c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.257+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9514059350 con 0x7f952c0d1020 2024-04-25T08:29:41.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.434+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f952c154ec0 con 0x7f950c06b4d0 2024-04-25T08:29:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.442+0000 7f95237fe640 1 -- 172.21.15.60:0/1031165709 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f952c154ec0 con 0x7f950c06b4d0 2024-04-25T08:29:41.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:29:41.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f950c06b4d0 msgr2=0x7f950c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f950c06b4d0 0x7f950c06d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f951c000f60 tx=0x7f951c009040 comp rx=0 tx=0).stop 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 msgr2=0x7f952c0b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 0x7f952c0b3dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f951400ebe0 tx=0x7f951400c520 comp rx=0 tx=0).stop 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 shutdown_connections 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f950c06b4d0 0x7f950c06d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f952c15dbe0 0x7f952c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f952c0d1020 0x7f952c0b3dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 --2- 172.21.15.60:0/1031165709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f952c0b2cb0 0x7f952c0c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 >> 172.21.15.60:0/1031165709 conn(0x7f952c061a20 msgr2=0x7f952c0b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 shutdown_connections 2024-04-25T08:29:41.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.446+0000 7f953129a640 1 -- 172.21.15.60:0/1031165709 wait complete. 2024-04-25T08:29:41.454 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T08:29:41.400561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231920,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742800,"statfs":{"total":773094113280,"available":772856627200,"internally_reserved":0,"allocated":3350528,"data_stored":2022571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003097"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.028601+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.028315+0000","last_became_peered":"2024-04-25T08:29:23.028315+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075712+0000","last_change":"2024-04-25T08:29:24.048532+0000","last_active":"2024-04-25T08:29:29.075712+0000","last_peered":"2024-04-25T08:29:29.075712+0000","last_clean":"2024-04-25T08:29:29.075712+0000","last_became_active":"2024-04-25T08:29:24.046997+0000","last_became_peered":"2024-04-25T08:29:24.046997+0000","last_unstale":"2024-04-25T08:29:29.075712+0000","last_undegraded":"2024-04-25T08:29:29.075712+0000","last_fullsized":"2024-04-25T08:29:29.075712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074646+0000","last_change":"2024-04-25T08:29:28.085554+0000","last_active":"2024-04-25T08:29:29.074646+0000","last_peered":"2024-04-25T08:29:29.074646+0000","last_clean":"2024-04-25T08:29:29.074646+0000","last_became_active":"2024-04-25T08:29:28.085164+0000","last_became_peered":"2024-04-25T08:29:28.085164+0000","last_unstale":"2024-04-25T08:29:29.074646+0000","last_undegraded":"2024-04-25T08:29:29.074646+0000","last_fullsized":"2024-04-25T08:29:29.074646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:03.849914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075840+0000","last_change":"2024-04-25T08:29:23.035840+0000","last_active":"2024-04-25T08:29:29.075840+0000","last_peered":"2024-04-25T08:29:29.075840+0000","last_clean":"2024-04-25T08:29:29.075840+0000","last_became_active":"2024-04-25T08:29:23.035532+0000","last_became_peered":"2024-04-25T08:29:23.035532+0000","last_unstale":"2024-04-25T08:29:29.075840+0000","last_undegraded":"2024-04-25T08:29:29.075840+0000","last_fullsized":"2024-04-25T08:29:29.075840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906355+0000","last_change":"2024-04-25T08:29:24.037371+0000","last_active":"2024-04-25T08:29:29.906355+0000","last_peered":"2024-04-25T08:29:29.906355+0000","last_clean":"2024-04-25T08:29:29.906355+0000","last_became_active":"2024-04-25T08:29:24.037043+0000","last_became_peered":"2024-04-25T08:29:24.037043+0000","last_unstale":"2024-04-25T08:29:29.906355+0000","last_undegraded":"2024-04-25T08:29:29.906355+0000","last_fullsized":"2024-04-25T08:29:29.906355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.900465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074732+0000","last_change":"2024-04-25T08:29:28.080997+0000","last_active":"2024-04-25T08:29:29.074732+0000","last_peered":"2024-04-25T08:29:29.074732+0000","last_clean":"2024-04-25T08:29:29.074732+0000","last_became_active":"2024-04-25T08:29:28.080573+0000","last_became_peered":"2024-04-25T08:29:28.080573+0000","last_unstale":"2024-04-25T08:29:29.074732+0000","last_undegraded":"2024-04-25T08:29:29.074732+0000","last_fullsized":"2024-04-25T08:29:29.074732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:48.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895961+0000","last_change":"2024-04-25T08:29:23.041517+0000","last_active":"2024-04-25T08:29:29.895961+0000","last_peered":"2024-04-25T08:29:29.895961+0000","last_clean":"2024-04-25T08:29:29.895961+0000","last_became_active":"2024-04-25T08:29:23.041191+0000","last_became_peered":"2024-04-25T08:29:23.041191+0000","last_unstale":"2024-04-25T08:29:29.895961+0000","last_undegraded":"2024-04-25T08:29:29.895961+0000","last_fullsized":"2024-04-25T08:29:29.895961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574369+0000","last_change":"2024-04-25T08:29:24.042793+0000","last_active":"2024-04-25T08:29:29.574369+0000","last_peered":"2024-04-25T08:29:29.574369+0000","last_clean":"2024-04-25T08:29:29.574369+0000","last_became_active":"2024-04-25T08:29:24.042026+0000","last_became_peered":"2024-04-25T08:29:24.042026+0000","last_unstale":"2024-04-25T08:29:29.574369+0000","last_undegraded":"2024-04-25T08:29:29.574369+0000","last_fullsized":"2024-04-25T08:29:29.574369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075189+0000","last_change":"2024-04-25T08:29:28.081365+0000","last_active":"2024-04-25T08:29:29.075189+0000","last_peered":"2024-04-25T08:29:29.075189+0000","last_clean":"2024-04-25T08:29:29.075189+0000","last_became_active":"2024-04-25T08:29:28.080901+0000","last_became_peered":"2024-04-25T08:29:28.080901+0000","last_unstale":"2024-04-25T08:29:29.075189+0000","last_undegraded":"2024-04-25T08:29:29.075189+0000","last_fullsized":"2024-04-25T08:29:29.075189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.649430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895778+0000","last_change":"2024-04-25T08:29:23.036549+0000","last_active":"2024-04-25T08:29:29.895778+0000","last_peered":"2024-04-25T08:29:29.895778+0000","last_clean":"2024-04-25T08:29:29.895778+0000","last_became_active":"2024-04-25T08:29:23.036323+0000","last_became_peered":"2024-04-25T08:29:23.036323+0000","last_unstale":"2024-04-25T08:29:29.895778+0000","last_undegraded":"2024-04-25T08:29:29.895778+0000","last_fullsized":"2024-04-25T08:29:29.895778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574394+0000","last_change":"2024-04-25T08:29:24.041522+0000","last_active":"2024-04-25T08:29:29.574394+0000","last_peered":"2024-04-25T08:29:29.574394+0000","last_clean":"2024-04-25T08:29:29.574394+0000","last_became_active":"2024-04-25T08:29:24.041242+0000","last_became_peered":"2024-04-25T08:29:24.041242+0000","last_unstale":"2024-04-25T08:29:29.574394+0000","last_undegraded":"2024-04-25T08:29:29.574394+0000","last_fullsized":"2024-04-25T08:29:29.574394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075282+0000","last_change":"2024-04-25T08:29:28.081585+0000","last_active":"2024-04-25T08:29:29.075282+0000","last_peered":"2024-04-25T08:29:29.075282+0000","last_clean":"2024-04-25T08:29:29.075282+0000","last_became_active":"2024-04-25T08:29:28.080227+0000","last_became_peered":"2024-04-25T08:29:28.080227+0000","last_unstale":"2024-04-25T08:29:29.075282+0000","last_undegraded":"2024-04-25T08:29:29.075282+0000","last_fullsized":"2024-04-25T08:29:29.075282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075044+0000","last_change":"2024-04-25T08:29:24.040005+0000","last_active":"2024-04-25T08:29:29.075044+0000","last_peered":"2024-04-25T08:29:29.075044+0000","last_clean":"2024-04-25T08:29:29.075044+0000","last_became_active":"2024-04-25T08:29:24.039678+0000","last_became_peered":"2024-04-25T08:29:24.039678+0000","last_unstale":"2024-04-25T08:29:29.075044+0000","last_undegraded":"2024-04-25T08:29:29.075044+0000","last_fullsized":"2024-04-25T08:29:29.075044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895838+0000","last_change":"2024-04-25T08:29:23.034031+0000","last_active":"2024-04-25T08:29:29.895838+0000","last_peered":"2024-04-25T08:29:29.895838+0000","last_clean":"2024-04-25T08:29:29.895838+0000","last_became_active":"2024-04-25T08:29:23.033672+0000","last_became_peered":"2024-04-25T08:29:23.033672+0000","last_unstale":"2024-04-25T08:29:29.895838+0000","last_undegraded":"2024-04-25T08:29:29.895838+0000","last_fullsized":"2024-04-25T08:29:29.895838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076035+0000","last_change":"2024-04-25T08:29:28.067431+0000","last_active":"2024-04-25T08:29:29.076035+0000","last_peered":"2024-04-25T08:29:29.076035+0000","last_clean":"2024-04-25T08:29:29.076035+0000","last_became_active":"2024-04-25T08:29:28.066968+0000","last_became_peered":"2024-04-25T08:29:28.066968+0000","last_unstale":"2024-04-25T08:29:29.076035+0000","last_undegraded":"2024-04-25T08:29:29.076035+0000","last_fullsized":"2024-04-25T08:29:29.076035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075071+0000","last_change":"2024-04-25T08:29:24.052628+0000","last_active":"2024-04-25T08:29:29.075071+0000","last_peered":"2024-04-25T08:29:29.075071+0000","last_clean":"2024-04-25T08:29:29.075071+0000","last_became_active":"2024-04-25T08:29:24.052111+0000","last_became_peered":"2024-04-25T08:29:24.052111+0000","last_unstale":"2024-04-25T08:29:29.075071+0000","last_undegraded":"2024-04-25T08:29:29.075071+0000","last_fullsized":"2024-04-25T08:29:29.075071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074777+0000","last_change":"2024-04-25T08:29:23.040432+0000","last_active":"2024-04-25T08:29:30.074777+0000","last_peered":"2024-04-25T08:29:30.074777+0000","last_clean":"2024-04-25T08:29:30.074777+0000","last_became_active":"2024-04-25T08:29:23.040172+0000","last_became_peered":"2024-04-25T08:29:23.040172+0000","last_unstale":"2024-04-25T08:29:30.074777+0000","last_undegraded":"2024-04-25T08:29:30.074777+0000","last_fullsized":"2024-04-25T08:29:30.074777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075200+0000","last_change":"2024-04-25T08:29:28.081025+0000","last_active":"2024-04-25T08:29:29.075200+0000","last_peered":"2024-04-25T08:29:29.075200+0000","last_clean":"2024-04-25T08:29:29.075200+0000","last_became_active":"2024-04-25T08:29:28.080481+0000","last_became_peered":"2024-04-25T08:29:28.080481+0000","last_unstale":"2024-04-25T08:29:29.075200+0000","last_undegraded":"2024-04-25T08:29:29.075200+0000","last_fullsized":"2024-04-25T08:29:29.075200+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074861+0000","last_change":"2024-04-25T08:29:24.054299+0000","last_active":"2024-04-25T08:29:29.074861+0000","last_peered":"2024-04-25T08:29:29.074861+0000","last_clean":"2024-04-25T08:29:29.074861+0000","last_became_active":"2024-04-25T08:29:24.053996+0000","last_became_peered":"2024-04-25T08:29:24.053996+0000","last_unstale":"2024-04-25T08:29:29.074861+0000","last_undegraded":"2024-04-25T08:29:29.074861+0000","last_fullsized":"2024-04-25T08:29:29.074861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074526+0000","last_change":"2024-04-25T08:29:23.038382+0000","last_active":"2024-04-25T08:29:30.074526+0000","last_peered":"2024-04-25T08:29:30.074526+0000","last_clean":"2024-04-25T08:29:30.074526+0000","last_became_active":"2024-04-25T08:29:23.038060+0000","last_became_peered":"2024-04-25T08:29:23.038060+0000","last_unstale":"2024-04-25T08:29:30.074526+0000","last_undegraded":"2024-04-25T08:29:30.074526+0000","last_fullsized":"2024-04-25T08:29:30.074526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075123+0000","last_change":"2024-04-25T08:29:28.085260+0000","last_active":"2024-04-25T08:29:30.075123+0000","last_peered":"2024-04-25T08:29:30.075123+0000","last_clean":"2024-04-25T08:29:30.075123+0000","last_became_active":"2024-04-25T08:29:28.084884+0000","last_became_peered":"2024-04-25T08:29:28.084884+0000","last_unstale":"2024-04-25T08:29:30.075123+0000","last_undegraded":"2024-04-25T08:29:30.075123+0000","last_fullsized":"2024-04-25T08:29:30.075123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075420+0000","last_change":"2024-04-25T08:29:24.035523+0000","last_active":"2024-04-25T08:29:29.075420+0000","last_peered":"2024-04-25T08:29:29.075420+0000","last_clean":"2024-04-25T08:29:29.075420+0000","last_became_active":"2024-04-25T08:29:24.035185+0000","last_became_peered":"2024-04-25T08:29:24.035185+0000","last_unstale":"2024-04-25T08:29:29.075420+0000","last_undegraded":"2024-04-25T08:29:29.075420+0000","last_fullsized":"2024-04-25T08:29:29.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.274896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895386+0000","last_change":"2024-04-25T08:29:23.037104+0000","last_active":"2024-04-25T08:29:29.895386+0000","last_peered":"2024-04-25T08:29:29.895386+0000","last_clean":"2024-04-25T08:29:29.895386+0000","last_became_active":"2024-04-25T08:29:23.035864+0000","last_became_peered":"2024-04-25T08:29:23.035864+0000","last_unstale":"2024-04-25T08:29:29.895386+0000","last_undegraded":"2024-04-25T08:29:29.895386+0000","last_fullsized":"2024-04-25T08:29:29.895386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075821+0000","last_change":"2024-04-25T08:29:28.080714+0000","last_active":"2024-04-25T08:29:29.075821+0000","last_peered":"2024-04-25T08:29:29.075821+0000","last_clean":"2024-04-25T08:29:29.075821+0000","last_became_active":"2024-04-25T08:29:28.080291+0000","last_became_peered":"2024-04-25T08:29:28.080291+0000","last_unstale":"2024-04-25T08:29:29.075821+0000","last_undegraded":"2024-04-25T08:29:29.075821+0000","last_fullsized":"2024-04-25T08:29:29.075821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906353+0000","last_change":"2024-04-25T08:29:24.040261+0000","last_active":"2024-04-25T08:29:29.906353+0000","last_peered":"2024-04-25T08:29:29.906353+0000","last_clean":"2024-04-25T08:29:29.906353+0000","last_became_active":"2024-04-25T08:29:24.039953+0000","last_became_peered":"2024-04-25T08:29:24.039953+0000","last_unstale":"2024-04-25T08:29:29.906353+0000","last_undegraded":"2024-04-25T08:29:29.906353+0000","last_fullsized":"2024-04-25T08:29:29.906353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076570+0000","last_change":"2024-04-25T08:29:23.038961+0000","last_active":"2024-04-25T08:29:29.076570+0000","last_peered":"2024-04-25T08:29:29.076570+0000","last_clean":"2024-04-25T08:29:29.076570+0000","last_became_active":"2024-04-25T08:29:23.038722+0000","last_became_peered":"2024-04-25T08:29:23.038722+0000","last_unstale":"2024-04-25T08:29:29.076570+0000","last_undegraded":"2024-04-25T08:29:29.076570+0000","last_fullsized":"2024-04-25T08:29:29.076570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074879+0000","last_change":"2024-04-25T08:29:28.073848+0000","last_active":"2024-04-25T08:29:29.074879+0000","last_peered":"2024-04-25T08:29:29.074879+0000","last_clean":"2024-04-25T08:29:29.074879+0000","last_became_active":"2024-04-25T08:29:28.073489+0000","last_became_peered":"2024-04-25T08:29:28.073489+0000","last_unstale":"2024-04-25T08:29:29.074879+0000","last_undegraded":"2024-04-25T08:29:29.074879+0000","last_fullsized":"2024-04-25T08:29:29.074879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:54.262561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076145+0000","last_change":"2024-04-25T08:29:24.057631+0000","last_active":"2024-04-25T08:29:29.076145+0000","last_peered":"2024-04-25T08:29:29.076145+0000","last_clean":"2024-04-25T08:29:29.076145+0000","last_became_active":"2024-04-25T08:29:24.057387+0000","last_became_peered":"2024-04-25T08:29:24.057387+0000","last_unstale":"2024-04-25T08:29:29.076145+0000","last_undegraded":"2024-04-25T08:29:29.076145+0000","last_fullsized":"2024-04-25T08:29:29.076145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075211+0000","last_change":"2024-04-25T08:29:23.041608+0000","last_active":"2024-04-25T08:29:30.075211+0000","last_peered":"2024-04-25T08:29:30.075211+0000","last_clean":"2024-04-25T08:29:30.075211+0000","last_became_active":"2024-04-25T08:29:23.041305+0000","last_became_peered":"2024-04-25T08:29:23.041305+0000","last_unstale":"2024-04-25T08:29:30.075211+0000","last_undegraded":"2024-04-25T08:29:30.075211+0000","last_fullsized":"2024-04-25T08:29:30.075211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:28.085603+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:28.084444+0000","last_became_peered":"2024-04-25T08:29:28.084444+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:39.462705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075317+0000","last_change":"2024-04-25T08:29:24.036286+0000","last_active":"2024-04-25T08:29:30.075317+0000","last_peered":"2024-04-25T08:29:30.075317+0000","last_clean":"2024-04-25T08:29:30.075317+0000","last_became_active":"2024-04-25T08:29:24.035947+0000","last_became_peered":"2024-04-25T08:29:24.035947+0000","last_unstale":"2024-04-25T08:29:30.075317+0000","last_undegraded":"2024-04-25T08:29:30.075317+0000","last_fullsized":"2024-04-25T08:29:30.075317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896087+0000","last_change":"2024-04-25T08:29:23.035930+0000","last_active":"2024-04-25T08:29:29.896087+0000","last_peered":"2024-04-25T08:29:29.896087+0000","last_clean":"2024-04-25T08:29:29.896087+0000","last_became_active":"2024-04-25T08:29:23.035685+0000","last_became_peered":"2024-04-25T08:29:23.035685+0000","last_unstale":"2024-04-25T08:29:29.896087+0000","last_undegraded":"2024-04-25T08:29:29.896087+0000","last_fullsized":"2024-04-25T08:29:29.896087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906315+0000","last_change":"2024-04-25T08:29:28.076556+0000","last_active":"2024-04-25T08:29:29.906315+0000","last_peered":"2024-04-25T08:29:29.906315+0000","last_clean":"2024-04-25T08:29:29.906315+0000","last_became_active":"2024-04-25T08:29:28.076157+0000","last_became_peered":"2024-04-25T08:29:28.076157+0000","last_unstale":"2024-04-25T08:29:29.906315+0000","last_undegraded":"2024-04-25T08:29:29.906315+0000","last_fullsized":"2024-04-25T08:29:29.906315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574536+0000","last_change":"2024-04-25T08:29:24.055822+0000","last_active":"2024-04-25T08:29:29.574536+0000","last_peered":"2024-04-25T08:29:29.574536+0000","last_clean":"2024-04-25T08:29:29.574536+0000","last_became_active":"2024-04-25T08:29:24.055087+0000","last_became_peered":"2024-04-25T08:29:24.055087+0000","last_unstale":"2024-04-25T08:29:29.574536+0000","last_undegraded":"2024-04-25T08:29:29.574536+0000","last_fullsized":"2024-04-25T08:29:29.574536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075321+0000","last_change":"2024-04-25T08:29:23.037612+0000","last_active":"2024-04-25T08:29:30.075321+0000","last_peered":"2024-04-25T08:29:30.075321+0000","last_clean":"2024-04-25T08:29:30.075321+0000","last_became_active":"2024-04-25T08:29:23.037304+0000","last_became_peered":"2024-04-25T08:29:23.037304+0000","last_unstale":"2024-04-25T08:29:30.075321+0000","last_undegraded":"2024-04-25T08:29:30.075321+0000","last_fullsized":"2024-04-25T08:29:30.075321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074438+0000","last_change":"2024-04-25T08:29:28.068007+0000","last_active":"2024-04-25T08:29:30.074438+0000","last_peered":"2024-04-25T08:29:30.074438+0000","last_clean":"2024-04-25T08:29:30.074438+0000","last_became_active":"2024-04-25T08:29:28.067586+0000","last_became_peered":"2024-04-25T08:29:28.067586+0000","last_unstale":"2024-04-25T08:29:30.074438+0000","last_undegraded":"2024-04-25T08:29:30.074438+0000","last_fullsized":"2024-04-25T08:29:30.074438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075717+0000","last_change":"2024-04-25T08:29:24.038433+0000","last_active":"2024-04-25T08:29:29.075717+0000","last_peered":"2024-04-25T08:29:29.075717+0000","last_clean":"2024-04-25T08:29:29.075717+0000","last_became_active":"2024-04-25T08:29:24.038164+0000","last_became_peered":"2024-04-25T08:29:24.038164+0000","last_unstale":"2024-04-25T08:29:29.075717+0000","last_undegraded":"2024-04-25T08:29:29.075717+0000","last_fullsized":"2024-04-25T08:29:29.075717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574356+0000","last_change":"2024-04-25T08:29:23.036328+0000","last_active":"2024-04-25T08:29:29.574356+0000","last_peered":"2024-04-25T08:29:29.574356+0000","last_clean":"2024-04-25T08:29:29.574356+0000","last_became_active":"2024-04-25T08:29:23.036072+0000","last_became_peered":"2024-04-25T08:29:23.036072+0000","last_unstale":"2024-04-25T08:29:29.574356+0000","last_undegraded":"2024-04-25T08:29:29.574356+0000","last_fullsized":"2024-04-25T08:29:29.574356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075255+0000","last_change":"2024-04-25T08:29:28.081150+0000","last_active":"2024-04-25T08:29:30.075255+0000","last_peered":"2024-04-25T08:29:30.075255+0000","last_clean":"2024-04-25T08:29:30.075255+0000","last_became_active":"2024-04-25T08:29:28.080621+0000","last_became_peered":"2024-04-25T08:29:28.080621+0000","last_unstale":"2024-04-25T08:29:30.075255+0000","last_undegraded":"2024-04-25T08:29:30.075255+0000","last_fullsized":"2024-04-25T08:29:30.075255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074632+0000","last_change":"2024-04-25T08:29:24.036313+0000","last_active":"2024-04-25T08:29:30.074632+0000","last_peered":"2024-04-25T08:29:30.074632+0000","last_clean":"2024-04-25T08:29:30.074632+0000","last_became_active":"2024-04-25T08:29:24.035892+0000","last_became_peered":"2024-04-25T08:29:24.035892+0000","last_unstale":"2024-04-25T08:29:30.074632+0000","last_undegraded":"2024-04-25T08:29:30.074632+0000","last_fullsized":"2024-04-25T08:29:30.074632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074251+0000","last_change":"2024-04-25T08:29:23.033190+0000","last_active":"2024-04-25T08:29:30.074251+0000","last_peered":"2024-04-25T08:29:30.074251+0000","last_clean":"2024-04-25T08:29:30.074251+0000","last_became_active":"2024-04-25T08:29:23.032822+0000","last_became_peered":"2024-04-25T08:29:23.032822+0000","last_unstale":"2024-04-25T08:29:30.074251+0000","last_undegraded":"2024-04-25T08:29:30.074251+0000","last_fullsized":"2024-04-25T08:29:30.074251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075145+0000","last_change":"2024-04-25T08:29:28.085006+0000","last_active":"2024-04-25T08:29:30.075145+0000","last_peered":"2024-04-25T08:29:30.075145+0000","last_clean":"2024-04-25T08:29:30.075145+0000","last_became_active":"2024-04-25T08:29:28.084572+0000","last_became_peered":"2024-04-25T08:29:28.084572+0000","last_unstale":"2024-04-25T08:29:30.075145+0000","last_undegraded":"2024-04-25T08:29:30.075145+0000","last_fullsized":"2024-04-25T08:29:30.075145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906058+0000","last_change":"2024-04-25T08:29:24.043403+0000","last_active":"2024-04-25T08:29:29.906058+0000","last_peered":"2024-04-25T08:29:29.906058+0000","last_clean":"2024-04-25T08:29:29.906058+0000","last_became_active":"2024-04-25T08:29:24.042964+0000","last_became_peered":"2024-04-25T08:29:24.042964+0000","last_unstale":"2024-04-25T08:29:29.906058+0000","last_undegraded":"2024-04-25T08:29:29.906058+0000","last_fullsized":"2024-04-25T08:29:29.906058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895394+0000","last_change":"2024-04-25T08:29:23.035024+0000","last_active":"2024-04-25T08:29:29.895394+0000","last_peered":"2024-04-25T08:29:29.895394+0000","last_clean":"2024-04-25T08:29:29.895394+0000","last_became_active":"2024-04-25T08:29:23.034761+0000","last_became_peered":"2024-04-25T08:29:23.034761+0000","last_unstale":"2024-04-25T08:29:29.895394+0000","last_undegraded":"2024-04-25T08:29:29.895394+0000","last_fullsized":"2024-04-25T08:29:29.895394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906299+0000","last_change":"2024-04-25T08:29:28.078207+0000","last_active":"2024-04-25T08:29:29.906299+0000","last_peered":"2024-04-25T08:29:29.906299+0000","last_clean":"2024-04-25T08:29:29.906299+0000","last_became_active":"2024-04-25T08:29:28.077777+0000","last_became_peered":"2024-04-25T08:29:28.077777+0000","last_unstale":"2024-04-25T08:29:29.906299+0000","last_undegraded":"2024-04-25T08:29:29.906299+0000","last_fullsized":"2024-04-25T08:29:29.906299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.048117+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.047733+0000","last_became_peered":"2024-04-25T08:29:24.047733+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574166+0000","last_change":"2024-04-25T08:29:23.036154+0000","last_active":"2024-04-25T08:29:29.574166+0000","last_peered":"2024-04-25T08:29:29.574166+0000","last_clean":"2024-04-25T08:29:29.574166+0000","last_became_active":"2024-04-25T08:29:23.035873+0000","last_became_peered":"2024-04-25T08:29:23.035873+0000","last_unstale":"2024-04-25T08:29:29.574166+0000","last_undegraded":"2024-04-25T08:29:29.574166+0000","last_fullsized":"2024-04-25T08:29:29.574166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895594+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:29:29.895594+0000","last_peered":"2024-04-25T08:29:29.895594+0000","last_clean":"2024-04-25T08:29:29.895594+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:29:29.895594+0000","last_undegraded":"2024-04-25T08:29:29.895594+0000","last_fullsized":"2024-04-25T08:29:29.895594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076370+0000","last_change":"2024-04-25T08:29:28.076816+0000","last_active":"2024-04-25T08:29:29.076370+0000","last_peered":"2024-04-25T08:29:29.076370+0000","last_clean":"2024-04-25T08:29:29.076370+0000","last_became_active":"2024-04-25T08:29:28.076348+0000","last_became_peered":"2024-04-25T08:29:28.076348+0000","last_unstale":"2024-04-25T08:29:29.076370+0000","last_undegraded":"2024-04-25T08:29:29.076370+0000","last_fullsized":"2024-04-25T08:29:29.076370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895804+0000","last_change":"2024-04-25T08:29:24.047938+0000","last_active":"2024-04-25T08:29:29.895804+0000","last_peered":"2024-04-25T08:29:29.895804+0000","last_clean":"2024-04-25T08:29:29.895804+0000","last_became_active":"2024-04-25T08:29:24.047670+0000","last_became_peered":"2024-04-25T08:29:24.047670+0000","last_unstale":"2024-04-25T08:29:29.895804+0000","last_undegraded":"2024-04-25T08:29:29.895804+0000","last_fullsized":"2024-04-25T08:29:29.895804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074744+0000","last_change":"2024-04-25T08:29:23.037162+0000","last_active":"2024-04-25T08:29:30.074744+0000","last_peered":"2024-04-25T08:29:30.074744+0000","last_clean":"2024-04-25T08:29:30.074744+0000","last_became_active":"2024-04-25T08:29:23.036850+0000","last_became_peered":"2024-04-25T08:29:23.036850+0000","last_unstale":"2024-04-25T08:29:30.074744+0000","last_undegraded":"2024-04-25T08:29:30.074744+0000","last_fullsized":"2024-04-25T08:29:30.074744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074972+0000","last_change":"2024-04-25T08:29:28.073263+0000","last_active":"2024-04-25T08:29:29.074972+0000","last_peered":"2024-04-25T08:29:29.074972+0000","last_clean":"2024-04-25T08:29:29.074972+0000","last_became_active":"2024-04-25T08:29:28.072784+0000","last_became_peered":"2024-04-25T08:29:28.072784+0000","last_unstale":"2024-04-25T08:29:29.074972+0000","last_undegraded":"2024-04-25T08:29:29.074972+0000","last_fullsized":"2024-04-25T08:29:29.074972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574458+0000","last_change":"2024-04-25T08:29:24.056206+0000","last_active":"2024-04-25T08:29:29.574458+0000","last_peered":"2024-04-25T08:29:29.574458+0000","last_clean":"2024-04-25T08:29:29.574458+0000","last_became_active":"2024-04-25T08:29:24.055557+0000","last_became_peered":"2024-04-25T08:29:24.055557+0000","last_unstale":"2024-04-25T08:29:29.574458+0000","last_undegraded":"2024-04-25T08:29:29.574458+0000","last_fullsized":"2024-04-25T08:29:29.574458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075008+0000","last_change":"2024-04-25T08:29:23.031709+0000","last_active":"2024-04-25T08:29:30.075008+0000","last_peered":"2024-04-25T08:29:30.075008+0000","last_clean":"2024-04-25T08:29:30.075008+0000","last_became_active":"2024-04-25T08:29:23.031419+0000","last_became_peered":"2024-04-25T08:29:23.031419+0000","last_unstale":"2024-04-25T08:29:30.075008+0000","last_undegraded":"2024-04-25T08:29:30.075008+0000","last_fullsized":"2024-04-25T08:29:30.075008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906475+0000","last_change":"2024-04-25T08:29:28.070276+0000","last_active":"2024-04-25T08:29:29.906475+0000","last_peered":"2024-04-25T08:29:29.906475+0000","last_clean":"2024-04-25T08:29:29.906475+0000","last_became_active":"2024-04-25T08:29:28.069869+0000","last_became_peered":"2024-04-25T08:29:28.069869+0000","last_unstale":"2024-04-25T08:29:29.906475+0000","last_undegraded":"2024-04-25T08:29:29.906475+0000","last_fullsized":"2024-04-25T08:29:29.906475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896237+0000","last_change":"2024-04-25T08:29:24.057274+0000","last_active":"2024-04-25T08:29:29.896237+0000","last_peered":"2024-04-25T08:29:29.896237+0000","last_clean":"2024-04-25T08:29:29.896237+0000","last_became_active":"2024-04-25T08:29:24.056998+0000","last_became_peered":"2024-04-25T08:29:24.056998+0000","last_unstale":"2024-04-25T08:29:29.896237+0000","last_undegraded":"2024-04-25T08:29:29.896237+0000","last_fullsized":"2024-04-25T08:29:29.896237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:18.584258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074824+0000","last_change":"2024-04-25T08:29:23.039945+0000","last_active":"2024-04-25T08:29:30.074824+0000","last_peered":"2024-04-25T08:29:30.074824+0000","last_clean":"2024-04-25T08:29:30.074824+0000","last_became_active":"2024-04-25T08:29:23.039563+0000","last_became_peered":"2024-04-25T08:29:23.039563+0000","last_unstale":"2024-04-25T08:29:30.074824+0000","last_undegraded":"2024-04-25T08:29:30.074824+0000","last_fullsized":"2024-04-25T08:29:30.074824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573881+0000","last_change":"2024-04-25T08:29:28.076465+0000","last_active":"2024-04-25T08:29:29.573881+0000","last_peered":"2024-04-25T08:29:29.573881+0000","last_clean":"2024-04-25T08:29:29.573881+0000","last_became_active":"2024-04-25T08:29:28.076064+0000","last_became_peered":"2024-04-25T08:29:28.076064+0000","last_unstale":"2024-04-25T08:29:29.573881+0000","last_undegraded":"2024-04-25T08:29:29.573881+0000","last_fullsized":"2024-04-25T08:29:29.573881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896089+0000","last_change":"2024-04-25T08:29:24.050215+0000","last_active":"2024-04-25T08:29:29.896089+0000","last_peered":"2024-04-25T08:29:29.896089+0000","last_clean":"2024-04-25T08:29:29.896089+0000","last_became_active":"2024-04-25T08:29:24.048694+0000","last_became_peered":"2024-04-25T08:29:24.048694+0000","last_unstale":"2024-04-25T08:29:29.896089+0000","last_undegraded":"2024-04-25T08:29:29.896089+0000","last_fullsized":"2024-04-25T08:29:29.896089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075997+0000","last_change":"2024-04-25T08:29:23.033858+0000","last_active":"2024-04-25T08:29:29.075997+0000","last_peered":"2024-04-25T08:29:29.075997+0000","last_clean":"2024-04-25T08:29:29.075997+0000","last_became_active":"2024-04-25T08:29:23.033599+0000","last_became_peered":"2024-04-25T08:29:23.033599+0000","last_unstale":"2024-04-25T08:29:29.075997+0000","last_undegraded":"2024-04-25T08:29:29.075997+0000","last_fullsized":"2024-04-25T08:29:29.075997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074730+0000","last_change":"2024-04-25T08:29:28.081641+0000","last_active":"2024-04-25T08:29:29.074730+0000","last_peered":"2024-04-25T08:29:29.074730+0000","last_clean":"2024-04-25T08:29:29.074730+0000","last_became_active":"2024-04-25T08:29:28.081220+0000","last_became_peered":"2024-04-25T08:29:28.081220+0000","last_unstale":"2024-04-25T08:29:29.074730+0000","last_undegraded":"2024-04-25T08:29:29.074730+0000","last_fullsized":"2024-04-25T08:29:29.074730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:51.749033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895682+0000","last_change":"2024-04-25T08:29:24.050072+0000","last_active":"2024-04-25T08:29:29.895682+0000","last_peered":"2024-04-25T08:29:29.895682+0000","last_clean":"2024-04-25T08:29:29.895682+0000","last_became_active":"2024-04-25T08:29:24.049773+0000","last_became_peered":"2024-04-25T08:29:24.049773+0000","last_unstale":"2024-04-25T08:29:29.895682+0000","last_undegraded":"2024-04-25T08:29:29.895682+0000","last_fullsized":"2024-04-25T08:29:29.895682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074062+0000","last_change":"2024-04-25T08:29:23.035476+0000","last_active":"2024-04-25T08:29:30.074062+0000","last_peered":"2024-04-25T08:29:30.074062+0000","last_clean":"2024-04-25T08:29:30.074062+0000","last_became_active":"2024-04-25T08:29:23.035129+0000","last_became_peered":"2024-04-25T08:29:23.035129+0000","last_unstale":"2024-04-25T08:29:30.074062+0000","last_undegraded":"2024-04-25T08:29:30.074062+0000","last_fullsized":"2024-04-25T08:29:30.074062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.454939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906185+0000","last_change":"2024-04-25T08:29:28.076678+0000","last_active":"2024-04-25T08:29:29.906185+0000","last_peered":"2024-04-25T08:29:29.906185+0000","last_clean":"2024-04-25T08:29:29.906185+0000","last_became_active":"2024-04-25T08:29:28.076283+0000","last_became_peered":"2024-04-25T08:29:28.076283+0000","last_unstale":"2024-04-25T08:29:29.906185+0000","last_undegraded":"2024-04-25T08:29:29.906185+0000","last_fullsized":"2024-04-25T08:29:29.906185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075358+0000","last_change":"2024-04-25T08:29:24.041672+0000","last_active":"2024-04-25T08:29:29.075358+0000","last_peered":"2024-04-25T08:29:29.075358+0000","last_clean":"2024-04-25T08:29:29.075358+0000","last_became_active":"2024-04-25T08:29:24.040865+0000","last_became_peered":"2024-04-25T08:29:24.040865+0000","last_unstale":"2024-04-25T08:29:29.075358+0000","last_undegraded":"2024-04-25T08:29:29.075358+0000","last_fullsized":"2024-04-25T08:29:29.075358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075805+0000","last_change":"2024-04-25T08:29:23.028173+0000","last_active":"2024-04-25T08:29:29.075805+0000","last_peered":"2024-04-25T08:29:29.075805+0000","last_clean":"2024-04-25T08:29:29.075805+0000","last_became_active":"2024-04-25T08:29:23.027886+0000","last_became_peered":"2024-04-25T08:29:23.027886+0000","last_unstale":"2024-04-25T08:29:29.075805+0000","last_undegraded":"2024-04-25T08:29:29.075805+0000","last_fullsized":"2024-04-25T08:29:29.075805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076047+0000","last_change":"2024-04-25T08:29:28.084524+0000","last_active":"2024-04-25T08:29:29.076047+0000","last_peered":"2024-04-25T08:29:29.076047+0000","last_clean":"2024-04-25T08:29:29.076047+0000","last_became_active":"2024-04-25T08:29:28.083769+0000","last_became_peered":"2024-04-25T08:29:28.083769+0000","last_unstale":"2024-04-25T08:29:29.076047+0000","last_undegraded":"2024-04-25T08:29:29.076047+0000","last_fullsized":"2024-04-25T08:29:29.076047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:43.115368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895795+0000","last_change":"2024-04-25T08:29:24.057400+0000","last_active":"2024-04-25T08:29:29.895795+0000","last_peered":"2024-04-25T08:29:29.895795+0000","last_clean":"2024-04-25T08:29:29.895795+0000","last_became_active":"2024-04-25T08:29:24.057074+0000","last_became_peered":"2024-04-25T08:29:24.057074+0000","last_unstale":"2024-04-25T08:29:29.895795+0000","last_undegraded":"2024-04-25T08:29:29.895795+0000","last_fullsized":"2024-04-25T08:29:29.895795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574339+0000","last_change":"2024-04-25T08:29:23.036448+0000","last_active":"2024-04-25T08:29:29.574339+0000","last_peered":"2024-04-25T08:29:29.574339+0000","last_clean":"2024-04-25T08:29:29.574339+0000","last_became_active":"2024-04-25T08:29:23.036172+0000","last_became_peered":"2024-04-25T08:29:23.036172+0000","last_unstale":"2024-04-25T08:29:29.574339+0000","last_undegraded":"2024-04-25T08:29:29.574339+0000","last_fullsized":"2024-04-25T08:29:29.574339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075577+0000","last_change":"2024-04-25T08:29:28.083554+0000","last_active":"2024-04-25T08:29:29.075577+0000","last_peered":"2024-04-25T08:29:29.075577+0000","last_clean":"2024-04-25T08:29:29.075577+0000","last_became_active":"2024-04-25T08:29:28.083178+0000","last_became_peered":"2024-04-25T08:29:28.083178+0000","last_unstale":"2024-04-25T08:29:29.075577+0000","last_undegraded":"2024-04-25T08:29:29.075577+0000","last_fullsized":"2024-04-25T08:29:29.075577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906194+0000","last_change":"2024-04-25T08:29:24.049668+0000","last_active":"2024-04-25T08:29:29.906194+0000","last_peered":"2024-04-25T08:29:29.906194+0000","last_clean":"2024-04-25T08:29:29.906194+0000","last_became_active":"2024-04-25T08:29:24.049359+0000","last_became_peered":"2024-04-25T08:29:24.049359+0000","last_unstale":"2024-04-25T08:29:29.906194+0000","last_undegraded":"2024-04-25T08:29:29.906194+0000","last_fullsized":"2024-04-25T08:29:29.906194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:48.609719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.030224+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.029936+0000","last_became_peered":"2024-04-25T08:29:23.029936+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:17.202033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574628+0000","last_change":"2024-04-25T08:29:28.076915+0000","last_active":"2024-04-25T08:29:29.574628+0000","last_peered":"2024-04-25T08:29:29.574628+0000","last_clean":"2024-04-25T08:29:29.574628+0000","last_became_active":"2024-04-25T08:29:28.076527+0000","last_became_peered":"2024-04-25T08:29:28.076527+0000","last_unstale":"2024-04-25T08:29:29.574628+0000","last_undegraded":"2024-04-25T08:29:29.574628+0000","last_fullsized":"2024-04-25T08:29:29.574628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895697+0000","last_change":"2024-04-25T08:29:24.053639+0000","last_active":"2024-04-25T08:29:29.895697+0000","last_peered":"2024-04-25T08:29:29.895697+0000","last_clean":"2024-04-25T08:29:29.895697+0000","last_became_active":"2024-04-25T08:29:24.053028+0000","last_became_peered":"2024-04-25T08:29:24.053028+0000","last_unstale":"2024-04-25T08:29:29.895697+0000","last_undegraded":"2024-04-25T08:29:29.895697+0000","last_fullsized":"2024-04-25T08:29:29.895697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:32.823362+0000","last_change":"2024-04-25T08:29:23.031850+0000","last_active":"2024-04-25T08:29:32.823362+0000","last_peered":"2024-04-25T08:29:32.823362+0000","last_clean":"2024-04-25T08:29:32.823362+0000","last_became_active":"2024-04-25T08:29:23.031588+0000","last_became_peered":"2024-04-25T08:29:23.031588+0000","last_unstale":"2024-04-25T08:29:32.823362+0000","last_undegraded":"2024-04-25T08:29:32.823362+0000","last_fullsized":"2024-04-25T08:29:32.823362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074668+0000","last_change":"2024-04-25T08:29:28.066135+0000","last_active":"2024-04-25T08:29:30.074668+0000","last_peered":"2024-04-25T08:29:30.074668+0000","last_clean":"2024-04-25T08:29:30.074668+0000","last_became_active":"2024-04-25T08:29:28.065611+0000","last_became_peered":"2024-04-25T08:29:28.065611+0000","last_unstale":"2024-04-25T08:29:30.074668+0000","last_undegraded":"2024-04-25T08:29:30.074668+0000","last_fullsized":"2024-04-25T08:29:30.074668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075847+0000","last_change":"2024-04-25T08:29:24.050678+0000","last_active":"2024-04-25T08:29:29.075847+0000","last_peered":"2024-04-25T08:29:29.075847+0000","last_clean":"2024-04-25T08:29:29.075847+0000","last_became_active":"2024-04-25T08:29:24.049995+0000","last_became_peered":"2024-04-25T08:29:24.049995+0000","last_unstale":"2024-04-25T08:29:29.075847+0000","last_undegraded":"2024-04-25T08:29:29.075847+0000","last_fullsized":"2024-04-25T08:29:29.075847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.729278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075000+0000","last_change":"2024-04-25T08:29:23.036900+0000","last_active":"2024-04-25T08:29:30.075000+0000","last_peered":"2024-04-25T08:29:30.075000+0000","last_clean":"2024-04-25T08:29:30.075000+0000","last_became_active":"2024-04-25T08:29:23.036630+0000","last_became_peered":"2024-04-25T08:29:23.036630+0000","last_unstale":"2024-04-25T08:29:30.075000+0000","last_undegraded":"2024-04-25T08:29:30.075000+0000","last_fullsized":"2024-04-25T08:29:30.075000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075542+0000","last_change":"2024-04-25T08:29:28.066730+0000","last_active":"2024-04-25T08:29:29.075542+0000","last_peered":"2024-04-25T08:29:29.075542+0000","last_clean":"2024-04-25T08:29:29.075542+0000","last_became_active":"2024-04-25T08:29:28.066323+0000","last_became_peered":"2024-04-25T08:29:28.066323+0000","last_unstale":"2024-04-25T08:29:29.075542+0000","last_undegraded":"2024-04-25T08:29:29.075542+0000","last_fullsized":"2024-04-25T08:29:29.075542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906473+0000","last_change":"2024-04-25T08:29:24.035898+0000","last_active":"2024-04-25T08:29:29.906473+0000","last_peered":"2024-04-25T08:29:29.906473+0000","last_clean":"2024-04-25T08:29:29.906473+0000","last_became_active":"2024-04-25T08:29:24.035137+0000","last_became_peered":"2024-04-25T08:29:24.035137+0000","last_unstale":"2024-04-25T08:29:29.906473+0000","last_undegraded":"2024-04-25T08:29:29.906473+0000","last_fullsized":"2024-04-25T08:29:29.906473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574599+0000","last_change":"2024-04-25T08:29:23.036994+0000","last_active":"2024-04-25T08:29:29.574599+0000","last_peered":"2024-04-25T08:29:29.574599+0000","last_clean":"2024-04-25T08:29:29.574599+0000","last_became_active":"2024-04-25T08:29:23.036740+0000","last_became_peered":"2024-04-25T08:29:23.036740+0000","last_unstale":"2024-04-25T08:29:29.574599+0000","last_undegraded":"2024-04-25T08:29:29.574599+0000","last_fullsized":"2024-04-25T08:29:29.574599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074766+0000","last_change":"2024-04-25T08:29:28.080727+0000","last_active":"2024-04-25T08:29:29.074766+0000","last_peered":"2024-04-25T08:29:29.074766+0000","last_clean":"2024-04-25T08:29:29.074766+0000","last_became_active":"2024-04-25T08:29:28.080288+0000","last_became_peered":"2024-04-25T08:29:28.080288+0000","last_unstale":"2024-04-25T08:29:29.074766+0000","last_undegraded":"2024-04-25T08:29:29.074766+0000","last_fullsized":"2024-04-25T08:29:29.074766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.184309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574484+0000","last_change":"2024-04-25T08:29:24.054205+0000","last_active":"2024-04-25T08:29:29.574484+0000","last_peered":"2024-04-25T08:29:29.574484+0000","last_clean":"2024-04-25T08:29:29.574484+0000","last_became_active":"2024-04-25T08:29:24.050796+0000","last_became_peered":"2024-04-25T08:29:24.050796+0000","last_unstale":"2024-04-25T08:29:29.574484+0000","last_undegraded":"2024-04-25T08:29:29.574484+0000","last_fullsized":"2024-04-25T08:29:29.574484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076454+0000","last_change":"2024-04-25T08:29:23.037467+0000","last_active":"2024-04-25T08:29:29.076454+0000","last_peered":"2024-04-25T08:29:29.076454+0000","last_clean":"2024-04-25T08:29:29.076454+0000","last_became_active":"2024-04-25T08:29:23.037207+0000","last_became_peered":"2024-04-25T08:29:23.037207+0000","last_unstale":"2024-04-25T08:29:29.076454+0000","last_undegraded":"2024-04-25T08:29:29.076454+0000","last_fullsized":"2024-04-25T08:29:29.076454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075540+0000","last_change":"2024-04-25T08:29:28.085289+0000","last_active":"2024-04-25T08:29:29.075540+0000","last_peered":"2024-04-25T08:29:29.075540+0000","last_clean":"2024-04-25T08:29:29.075540+0000","last_became_active":"2024-04-25T08:29:28.084815+0000","last_became_peered":"2024-04-25T08:29:28.084815+0000","last_unstale":"2024-04-25T08:29:29.075540+0000","last_undegraded":"2024-04-25T08:29:29.075540+0000","last_fullsized":"2024-04-25T08:29:29.075540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906069+0000","last_change":"2024-04-25T08:29:28.077759+0000","last_active":"2024-04-25T08:29:29.906069+0000","last_peered":"2024-04-25T08:29:29.906069+0000","last_clean":"2024-04-25T08:29:29.906069+0000","last_became_active":"2024-04-25T08:29:28.077376+0000","last_became_peered":"2024-04-25T08:29:28.077376+0000","last_unstale":"2024-04-25T08:29:29.906069+0000","last_undegraded":"2024-04-25T08:29:29.906069+0000","last_fullsized":"2024-04-25T08:29:29.906069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074258+0000","last_change":"2024-04-25T08:29:24.053405+0000","last_active":"2024-04-25T08:29:30.074258+0000","last_peered":"2024-04-25T08:29:30.074258+0000","last_clean":"2024-04-25T08:29:30.074258+0000","last_became_active":"2024-04-25T08:29:24.053104+0000","last_became_peered":"2024-04-25T08:29:24.053104+0000","last_unstale":"2024-04-25T08:29:30.074258+0000","last_undegraded":"2024-04-25T08:29:30.074258+0000","last_fullsized":"2024-04-25T08:29:30.074258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574014+0000","last_change":"2024-04-25T08:29:23.034808+0000","last_active":"2024-04-25T08:29:29.574014+0000","last_peered":"2024-04-25T08:29:29.574014+0000","last_clean":"2024-04-25T08:29:29.574014+0000","last_became_active":"2024-04-25T08:29:23.034545+0000","last_became_peered":"2024-04-25T08:29:23.034545+0000","last_unstale":"2024-04-25T08:29:29.574014+0000","last_undegraded":"2024-04-25T08:29:29.574014+0000","last_fullsized":"2024-04-25T08:29:29.574014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076514+0000","last_change":"2024-04-25T08:29:28.068503+0000","last_active":"2024-04-25T08:29:29.076514+0000","last_peered":"2024-04-25T08:29:29.076514+0000","last_clean":"2024-04-25T08:29:29.076514+0000","last_became_active":"2024-04-25T08:29:28.068080+0000","last_became_peered":"2024-04-25T08:29:28.068080+0000","last_unstale":"2024-04-25T08:29:29.076514+0000","last_undegraded":"2024-04-25T08:29:29.076514+0000","last_fullsized":"2024-04-25T08:29:29.076514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074757+0000","last_change":"2024-04-25T08:29:24.053631+0000","last_active":"2024-04-25T08:29:29.074757+0000","last_peered":"2024-04-25T08:29:29.074757+0000","last_clean":"2024-04-25T08:29:29.074757+0000","last_became_active":"2024-04-25T08:29:24.052293+0000","last_became_peered":"2024-04-25T08:29:24.052293+0000","last_unstale":"2024-04-25T08:29:29.074757+0000","last_undegraded":"2024-04-25T08:29:29.074757+0000","last_fullsized":"2024-04-25T08:29:29.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074995+0000","last_change":"2024-04-25T08:29:23.029639+0000","last_active":"2024-04-25T08:29:29.074995+0000","last_peered":"2024-04-25T08:29:29.074995+0000","last_clean":"2024-04-25T08:29:29.074995+0000","last_became_active":"2024-04-25T08:29:23.027908+0000","last_became_peered":"2024-04-25T08:29:23.027908+0000","last_unstale":"2024-04-25T08:29:29.074995+0000","last_undegraded":"2024-04-25T08:29:29.074995+0000","last_fullsized":"2024-04-25T08:29:29.074995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:02.366427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075903+0000","last_change":"2024-04-25T08:29:23.024619+0000","last_active":"2024-04-25T08:29:29.075903+0000","last_peered":"2024-04-25T08:29:29.075903+0000","last_clean":"2024-04-25T08:29:29.075903+0000","last_became_active":"2024-04-25T08:29:23.024364+0000","last_became_peered":"2024-04-25T08:29:23.024364+0000","last_unstale":"2024-04-25T08:29:29.075903+0000","last_undegraded":"2024-04-25T08:29:29.075903+0000","last_fullsized":"2024-04-25T08:29:29.075903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906507+0000","last_change":"2024-04-25T08:29:24.040912+0000","last_active":"2024-04-25T08:29:29.906507+0000","last_peered":"2024-04-25T08:29:29.906507+0000","last_clean":"2024-04-25T08:29:29.906507+0000","last_became_active":"2024-04-25T08:29:24.040629+0000","last_became_peered":"2024-04-25T08:29:24.040629+0000","last_unstale":"2024-04-25T08:29:29.906507+0000","last_undegraded":"2024-04-25T08:29:29.906507+0000","last_fullsized":"2024-04-25T08:29:29.906507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074576+0000","last_change":"2024-04-25T08:29:28.081077+0000","last_active":"2024-04-25T08:29:30.074576+0000","last_peered":"2024-04-25T08:29:30.074576+0000","last_clean":"2024-04-25T08:29:30.074576+0000","last_became_active":"2024-04-25T08:29:28.080470+0000","last_became_peered":"2024-04-25T08:29:28.080470+0000","last_unstale":"2024-04-25T08:29:30.074576+0000","last_undegraded":"2024-04-25T08:29:30.074576+0000","last_fullsized":"2024-04-25T08:29:30.074576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:18.082525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075155+0000","last_change":"2024-04-25T08:29:23.039762+0000","last_active":"2024-04-25T08:29:29.075155+0000","last_peered":"2024-04-25T08:29:29.075155+0000","last_clean":"2024-04-25T08:29:29.075155+0000","last_became_active":"2024-04-25T08:29:23.039484+0000","last_became_peered":"2024-04-25T08:29:23.039484+0000","last_unstale":"2024-04-25T08:29:29.075155+0000","last_undegraded":"2024-04-25T08:29:29.075155+0000","last_fullsized":"2024-04-25T08:29:29.075155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074927+0000","last_change":"2024-04-25T08:29:24.037624+0000","last_active":"2024-04-25T08:29:29.074927+0000","last_peered":"2024-04-25T08:29:29.074927+0000","last_clean":"2024-04-25T08:29:29.074927+0000","last_became_active":"2024-04-25T08:29:24.037352+0000","last_became_peered":"2024-04-25T08:29:24.037352+0000","last_unstale":"2024-04-25T08:29:29.074927+0000","last_undegraded":"2024-04-25T08:29:29.074927+0000","last_fullsized":"2024-04-25T08:29:29.074927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:51.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075230+0000","last_change":"2024-04-25T08:29:28.083366+0000","last_active":"2024-04-25T08:29:29.075230+0000","last_peered":"2024-04-25T08:29:29.075230+0000","last_clean":"2024-04-25T08:29:29.075230+0000","last_became_active":"2024-04-25T08:29:28.082870+0000","last_became_peered":"2024-04-25T08:29:28.082870+0000","last_unstale":"2024-04-25T08:29:29.075230+0000","last_undegraded":"2024-04-25T08:29:29.075230+0000","last_fullsized":"2024-04-25T08:29:29.075230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:08.399424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075038+0000","last_change":"2024-04-25T08:29:24.042078+0000","last_active":"2024-04-25T08:29:29.075038+0000","last_peered":"2024-04-25T08:29:29.075038+0000","last_clean":"2024-04-25T08:29:29.075038+0000","last_became_active":"2024-04-25T08:29:24.041318+0000","last_became_peered":"2024-04-25T08:29:24.041318+0000","last_unstale":"2024-04-25T08:29:29.075038+0000","last_undegraded":"2024-04-25T08:29:29.075038+0000","last_fullsized":"2024-04-25T08:29:29.075038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074154+0000","last_change":"2024-04-25T08:29:24.043006+0000","last_active":"2024-04-25T08:29:30.074154+0000","last_peered":"2024-04-25T08:29:30.074154+0000","last_clean":"2024-04-25T08:29:30.074154+0000","last_became_active":"2024-04-25T08:29:24.042553+0000","last_became_peered":"2024-04-25T08:29:24.042553+0000","last_unstale":"2024-04-25T08:29:30.074154+0000","last_undegraded":"2024-04-25T08:29:30.074154+0000","last_fullsized":"2024-04-25T08:29:30.074154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574181+0000","last_change":"2024-04-25T08:29:24.054678+0000","last_active":"2024-04-25T08:29:29.574181+0000","last_peered":"2024-04-25T08:29:29.574181+0000","last_clean":"2024-04-25T08:29:29.574181+0000","last_became_active":"2024-04-25T08:29:24.054432+0000","last_became_peered":"2024-04-25T08:29:24.054432+0000","last_unstale":"2024-04-25T08:29:29.574181+0000","last_undegraded":"2024-04-25T08:29:29.574181+0000","last_fullsized":"2024-04-25T08:29:29.574181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906457+0000","last_change":"2024-04-25T08:29:24.047684+0000","last_active":"2024-04-25T08:29:29.906457+0000","last_peered":"2024-04-25T08:29:29.906457+0000","last_clean":"2024-04-25T08:29:29.906457+0000","last_became_active":"2024-04-25T08:29:24.047384+0000","last_became_peered":"2024-04-25T08:29:24.047384+0000","last_unstale":"2024-04-25T08:29:29.906457+0000","last_undegraded":"2024-04-25T08:29:29.906457+0000","last_fullsized":"2024-04-25T08:29:29.906457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075571+0000","last_change":"2024-04-25T08:29:24.044573+0000","last_active":"2024-04-25T08:29:29.075571+0000","last_peered":"2024-04-25T08:29:29.075571+0000","last_clean":"2024-04-25T08:29:29.075571+0000","last_became_active":"2024-04-25T08:29:24.043980+0000","last_became_peered":"2024-04-25T08:29:24.043980+0000","last_unstale":"2024-04-25T08:29:29.075571+0000","last_undegraded":"2024-04-25T08:29:29.075571+0000","last_fullsized":"2024-04-25T08:29:29.075571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074876+0000","last_change":"2024-04-25T08:29:24.053456+0000","last_active":"2024-04-25T08:29:30.074876+0000","last_peered":"2024-04-25T08:29:30.074876+0000","last_clean":"2024-04-25T08:29:30.074876+0000","last_became_active":"2024-04-25T08:29:24.053144+0000","last_became_peered":"2024-04-25T08:29:24.053144+0000","last_unstale":"2024-04-25T08:29:30.074876+0000","last_undegraded":"2024-04-25T08:29:30.074876+0000","last_fullsized":"2024-04-25T08:29:30.074876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076592+0000","last_change":"2024-04-25T08:29:24.052047+0000","last_active":"2024-04-25T08:29:29.076592+0000","last_peered":"2024-04-25T08:29:29.076592+0000","last_clean":"2024-04-25T08:29:29.076592+0000","last_became_active":"2024-04-25T08:29:24.051777+0000","last_became_peered":"2024-04-25T08:29:24.051777+0000","last_unstale":"2024-04-25T08:29:29.076592+0000","last_undegraded":"2024-04-25T08:29:29.076592+0000","last_fullsized":"2024-04-25T08:29:29.076592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075986+0000","last_change":"2024-04-25T08:29:24.044743+0000","last_active":"2024-04-25T08:29:29.075986+0000","last_peered":"2024-04-25T08:29:29.075986+0000","last_clean":"2024-04-25T08:29:29.075986+0000","last_became_active":"2024-04-25T08:29:24.044425+0000","last_became_peered":"2024-04-25T08:29:24.044425+0000","last_unstale":"2024-04-25T08:29:29.075986+0000","last_undegraded":"2024-04-25T08:29:29.075986+0000","last_fullsized":"2024-04-25T08:29:29.075986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573967+0000","last_change":"2024-04-25T08:29:24.047484+0000","last_active":"2024-04-25T08:29:29.573967+0000","last_peered":"2024-04-25T08:29:29.573967+0000","last_clean":"2024-04-25T08:29:29.573967+0000","last_became_active":"2024-04-25T08:29:24.046848+0000","last_became_peered":"2024-04-25T08:29:24.046848+0000","last_unstale":"2024-04-25T08:29:29.573967+0000","last_undegraded":"2024-04-25T08:29:29.573967+0000","last_fullsized":"2024-04-25T08:29:29.573967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074875+0000","last_change":"2024-04-25T08:29:24.048838+0000","last_active":"2024-04-25T08:29:29.074875+0000","last_peered":"2024-04-25T08:29:29.074875+0000","last_clean":"2024-04-25T08:29:29.074875+0000","last_became_active":"2024-04-25T08:29:24.047272+0000","last_became_peered":"2024-04-25T08:29:24.047272+0000","last_unstale":"2024-04-25T08:29:29.074875+0000","last_undegraded":"2024-04-25T08:29:29.074875+0000","last_fullsized":"2024-04-25T08:29:29.074875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574182+0000","last_change":"2024-04-25T08:29:24.052059+0000","last_active":"2024-04-25T08:29:29.574182+0000","last_peered":"2024-04-25T08:29:29.574182+0000","last_clean":"2024-04-25T08:29:29.574182+0000","last_became_active":"2024-04-25T08:29:24.051604+0000","last_became_peered":"2024-04-25T08:29:24.051604+0000","last_unstale":"2024-04-25T08:29:29.574182+0000","last_undegraded":"2024-04-25T08:29:29.574182+0000","last_fullsized":"2024-04-25T08:29:29.574182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906362+0000","last_change":"2024-04-25T08:29:24.049843+0000","last_active":"2024-04-25T08:29:29.906362+0000","last_peered":"2024-04-25T08:29:29.906362+0000","last_clean":"2024-04-25T08:29:29.906362+0000","last_became_active":"2024-04-25T08:29:24.049562+0000","last_became_peered":"2024-04-25T08:29:24.049562+0000","last_unstale":"2024-04-25T08:29:29.906362+0000","last_undegraded":"2024-04-25T08:29:29.906362+0000","last_fullsized":"2024-04-25T08:29:29.906362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.370927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.041378+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.040860+0000","last_became_peered":"2024-04-25T08:29:24.040860+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895964+0000","last_change":"2024-04-25T08:29:24.050860+0000","last_active":"2024-04-25T08:29:29.895964+0000","last_peered":"2024-04-25T08:29:29.895964+0000","last_clean":"2024-04-25T08:29:29.895964+0000","last_became_active":"2024-04-25T08:29:24.050610+0000","last_became_peered":"2024-04-25T08:29:24.050610+0000","last_unstale":"2024-04-25T08:29:29.895964+0000","last_undegraded":"2024-04-25T08:29:29.895964+0000","last_fullsized":"2024-04-25T08:29:29.895964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074757+0000","last_change":"2024-04-25T08:29:24.041470+0000","last_active":"2024-04-25T08:29:30.074757+0000","last_peered":"2024-04-25T08:29:30.074757+0000","last_clean":"2024-04-25T08:29:30.074757+0000","last_became_active":"2024-04-25T08:29:24.041172+0000","last_became_peered":"2024-04-25T08:29:24.041172+0000","last_unstale":"2024-04-25T08:29:30.074757+0000","last_undegraded":"2024-04-25T08:29:30.074757+0000","last_fullsized":"2024-04-25T08:29:30.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574663+0000","last_change":"2024-04-25T08:29:24.054784+0000","last_active":"2024-04-25T08:29:29.574663+0000","last_peered":"2024-04-25T08:29:29.574663+0000","last_clean":"2024-04-25T08:29:29.574663+0000","last_became_active":"2024-04-25T08:29:24.054538+0000","last_became_peered":"2024-04-25T08:29:24.054538+0000","last_unstale":"2024-04-25T08:29:29.574663+0000","last_undegraded":"2024-04-25T08:29:29.574663+0000","last_fullsized":"2024-04-25T08:29:29.574663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573870+0000","last_change":"2024-04-25T08:29:24.046213+0000","last_active":"2024-04-25T08:29:29.573870+0000","last_peered":"2024-04-25T08:29:29.573870+0000","last_clean":"2024-04-25T08:29:29.573870+0000","last_became_active":"2024-04-25T08:29:24.045850+0000","last_became_peered":"2024-04-25T08:29:24.045850+0000","last_unstale":"2024-04-25T08:29:29.573870+0000","last_undegraded":"2024-04-25T08:29:29.573870+0000","last_fullsized":"2024-04-25T08:29:29.573870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074307+0000","last_change":"2024-04-25T08:29:24.039662+0000","last_active":"2024-04-25T08:29:30.074307+0000","last_peered":"2024-04-25T08:29:30.074307+0000","last_clean":"2024-04-25T08:29:30.074307+0000","last_became_active":"2024-04-25T08:29:24.039338+0000","last_became_peered":"2024-04-25T08:29:24.039338+0000","last_unstale":"2024-04-25T08:29:30.074307+0000","last_undegraded":"2024-04-25T08:29:30.074307+0000","last_fullsized":"2024-04-25T08:29:30.074307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906168+0000","last_change":"2024-04-25T08:29:24.038228+0000","last_active":"2024-04-25T08:29:29.906168+0000","last_peered":"2024-04-25T08:29:29.906168+0000","last_clean":"2024-04-25T08:29:29.906168+0000","last_became_active":"2024-04-25T08:29:24.037900+0000","last_became_peered":"2024-04-25T08:29:24.037900+0000","last_unstale":"2024-04-25T08:29:29.906168+0000","last_undegraded":"2024-04-25T08:29:29.906168+0000","last_fullsized":"2024-04-25T08:29:29.906168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:35.121139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074948+0000","last_change":"2024-04-25T08:29:24.045041+0000","last_active":"2024-04-25T08:29:29.074948+0000","last_peered":"2024-04-25T08:29:29.074948+0000","last_clean":"2024-04-25T08:29:29.074948+0000","last_became_active":"2024-04-25T08:29:24.044581+0000","last_became_peered":"2024-04-25T08:29:24.044581+0000","last_unstale":"2024-04-25T08:29:29.074948+0000","last_undegraded":"2024-04-25T08:29:29.074948+0000","last_fullsized":"2024-04-25T08:29:29.074948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075863+0000","last_change":"2024-04-25T08:29:24.042402+0000","last_active":"2024-04-25T08:29:29.075863+0000","last_peered":"2024-04-25T08:29:29.075863+0000","last_clean":"2024-04-25T08:29:29.075863+0000","last_became_active":"2024-04-25T08:29:24.042074+0000","last_became_peered":"2024-04-25T08:29:24.042074+0000","last_unstale":"2024-04-25T08:29:29.075863+0000","last_undegraded":"2024-04-25T08:29:29.075863+0000","last_fullsized":"2024-04-25T08:29:29.075863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076327+0000","last_change":"2024-04-25T08:29:24.035522+0000","last_active":"2024-04-25T08:29:29.076327+0000","last_peered":"2024-04-25T08:29:29.076327+0000","last_clean":"2024-04-25T08:29:29.076327+0000","last_became_active":"2024-04-25T08:29:24.035220+0000","last_became_peered":"2024-04-25T08:29:24.035220+0000","last_unstale":"2024-04-25T08:29:29.076327+0000","last_undegraded":"2024-04-25T08:29:29.076327+0000","last_fullsized":"2024-04-25T08:29:29.076327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076595+0000","last_change":"2024-04-25T08:29:24.035459+0000","last_active":"2024-04-25T08:29:29.076595+0000","last_peered":"2024-04-25T08:29:29.076595+0000","last_clean":"2024-04-25T08:29:29.076595+0000","last_became_active":"2024-04-25T08:29:24.035170+0000","last_became_peered":"2024-04-25T08:29:24.035170+0000","last_unstale":"2024-04-25T08:29:29.076595+0000","last_undegraded":"2024-04-25T08:29:29.076595+0000","last_fullsized":"2024-04-25T08:29:29.076595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896232+0000","last_change":"2024-04-25T08:29:24.058439+0000","last_active":"2024-04-25T08:29:29.896232+0000","last_peered":"2024-04-25T08:29:29.896232+0000","last_clean":"2024-04-25T08:29:29.896232+0000","last_became_active":"2024-04-25T08:29:24.058172+0000","last_became_peered":"2024-04-25T08:29:24.058172+0000","last_unstale":"2024-04-25T08:29:29.896232+0000","last_undegraded":"2024-04-25T08:29:29.896232+0000","last_fullsized":"2024-04-25T08:29:29.896232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:24.045871+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:24.045477+0000","last_became_peered":"2024-04-25T08:29:24.045477+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895647+0000","last_change":"2024-04-25T08:29:24.054149+0000","last_active":"2024-04-25T08:29:29.895647+0000","last_peered":"2024-04-25T08:29:29.895647+0000","last_clean":"2024-04-25T08:29:29.895647+0000","last_became_active":"2024-04-25T08:29:24.053893+0000","last_became_peered":"2024-04-25T08:29:24.053893+0000","last_unstale":"2024-04-25T08:29:29.895647+0000","last_undegraded":"2024-04-25T08:29:29.895647+0000","last_fullsized":"2024-04-25T08:29:29.895647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574047+0000","last_change":"2024-04-25T08:29:24.046245+0000","last_active":"2024-04-25T08:29:29.574047+0000","last_peered":"2024-04-25T08:29:29.574047+0000","last_clean":"2024-04-25T08:29:29.574047+0000","last_became_active":"2024-04-25T08:29:24.045892+0000","last_became_peered":"2024-04-25T08:29:24.045892+0000","last_unstale":"2024-04-25T08:29:29.574047+0000","last_undegraded":"2024-04-25T08:29:29.574047+0000","last_fullsized":"2024-04-25T08:29:29.574047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895672+0000","last_change":"2024-04-25T08:29:24.048906+0000","last_active":"2024-04-25T08:29:29.895672+0000","last_peered":"2024-04-25T08:29:29.895672+0000","last_clean":"2024-04-25T08:29:29.895672+0000","last_became_active":"2024-04-25T08:29:24.048634+0000","last_became_peered":"2024-04-25T08:29:24.048634+0000","last_unstale":"2024-04-25T08:29:29.895672+0000","last_undegraded":"2024-04-25T08:29:29.895672+0000","last_fullsized":"2024-04-25T08:29:29.895672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574336+0000","last_change":"2024-04-25T08:29:24.049689+0000","last_active":"2024-04-25T08:29:29.574336+0000","last_peered":"2024-04-25T08:29:29.574336+0000","last_clean":"2024-04-25T08:29:29.574336+0000","last_became_active":"2024-04-25T08:29:24.049210+0000","last_became_peered":"2024-04-25T08:29:24.049210+0000","last_unstale":"2024-04-25T08:29:29.574336+0000","last_undegraded":"2024-04-25T08:29:29.574336+0000","last_fullsized":"2024-04-25T08:29:29.574336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906036+0000","last_change":"2024-04-25T08:29:24.037342+0000","last_active":"2024-04-25T08:29:29.906036+0000","last_peered":"2024-04-25T08:29:29.906036+0000","last_clean":"2024-04-25T08:29:29.906036+0000","last_became_active":"2024-04-25T08:29:24.036991+0000","last_became_peered":"2024-04-25T08:29:24.036991+0000","last_unstale":"2024-04-25T08:29:29.906036+0000","last_undegraded":"2024-04-25T08:29:29.906036+0000","last_fullsized":"2024-04-25T08:29:29.906036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:14.049743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075188+0000","last_change":"2024-04-25T08:29:24.041775+0000","last_active":"2024-04-25T08:29:30.075188+0000","last_peered":"2024-04-25T08:29:30.075188+0000","last_clean":"2024-04-25T08:29:30.075188+0000","last_became_active":"2024-04-25T08:29:24.041474+0000","last_became_peered":"2024-04-25T08:29:24.041474+0000","last_unstale":"2024-04-25T08:29:30.075188+0000","last_undegraded":"2024-04-25T08:29:30.075188+0000","last_fullsized":"2024-04-25T08:29:30.075188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896139+0000","last_change":"2024-04-25T08:29:24.054127+0000","last_active":"2024-04-25T08:29:29.896139+0000","last_peered":"2024-04-25T08:29:29.896139+0000","last_clean":"2024-04-25T08:29:29.896139+0000","last_became_active":"2024-04-25T08:29:24.053854+0000","last_became_peered":"2024-04-25T08:29:24.053854+0000","last_unstale":"2024-04-25T08:29:29.896139+0000","last_undegraded":"2024-04-25T08:29:29.896139+0000","last_fullsized":"2024-04-25T08:29:29.896139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":695037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":104087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":104247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:29:41.458 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:29:41.459 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T08:29:41.459 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T08:29:41.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T08:29:41.459 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T08:29:41.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.599+0000 7f488a56a640 1 Processor -- start 2024-04-25T08:29:41.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.599+0000 7f488a56a640 1 -- start start 2024-04-25T08:29:41.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f488a56a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f48840c6df0 0x7f48840d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f488a56a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48840d1c00 0x7f48840ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f488a56a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488415dfe0 con 0x7f488415dc00 2024-04-25T08:29:41.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f488a56a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48840d2090 con 0x7f48840c6df0 2024-04-25T08:29:41.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f488a56a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840d00c0 con 0x7f48840d1c00 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f48840c6df0 0x7f48840d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f48840c6df0 0x7f48840d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43832/0 (socket says 172.21.15.60:43832) 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 -- 172.21.15.60:0/4073978651 learned_addr learned my addr 172.21.15.60:0/4073978651 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f4888b07640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48840d1c00 0x7f48840ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 -- 172.21.15.60:0/4073978651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48840d1c00 msgr2=0x7f48840ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48840d1c00 0x7f48840ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 -- 172.21.15.60:0/4073978651 >> v1:172.21.15.100:6789/0 conn(0x7f488415dc00 legacy=0x7f48840ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f48837fe640 1 -- 172.21.15.60:0/4073978651 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840d03a0 con 0x7f48840c6df0 2024-04-25T08:29:41.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.600+0000 7f4888b07640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48840d1c00 0x7f48840ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:41.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.601+0000 7f48837fe640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f48840d16c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4878009870 tx=0x7f487802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8686095cf98ba16a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.601+0000 7f4882ffd640 1 -- 172.21.15.60:0/4073978651 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878037070 con 0x7f48840c6df0 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.601+0000 7f4882ffd640 1 -- 172.21.15.60:0/4073978651 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878032440 con 0x7f48840c6df0 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.601+0000 7f4882ffd640 1 -- 172.21.15.60:0/4073978651 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878032810 con 0x7f48840c6df0 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.601+0000 7f488a56a640 1 -- 172.21.15.60:0/4073978651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 msgr2=0x7f48840d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f48840d16c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4878009870 tx=0x7f487802a270 comp rx=0 tx=0).stop 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 -- 172.21.15.60:0/4073978651 shutdown_connections 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f48840d16c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 --2- 172.21.15.60:0/4073978651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48840d1c00 0x7f48840ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 -- 172.21.15.60:0/4073978651 >> 172.21.15.60:0/4073978651 conn(0x7f488406f5d0 msgr2=0x7f488406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 -- 172.21.15.60:0/4073978651 shutdown_connections 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.602+0000 7f488a56a640 1 -- 172.21.15.60:0/4073978651 wait complete. 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.603+0000 7f488a56a640 1 Processor -- start 2024-04-25T08:29:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.603+0000 7f488a56a640 1 -- start start 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f4884153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840d1c00 0x7f4884144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f488415dc00 0x7f48840c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840cc130 con 0x7f48840c6df0 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840cc720 con 0x7f48840d1c00 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840cfd90 con 0x7f488415dc00 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f4884153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f4884153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43848/0 (socket says 172.21.15.60:43848) 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 -- 172.21.15.60:0/1948386143 learned_addr learned my addr 172.21.15.60:0/1948386143 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f48837fe640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840d1c00 0x7f4884144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4888b07640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f488415dc00 0x7f48840c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 -- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f488415dc00 msgr2=0x7f48840c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f488415dc00 0x7f48840c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 -- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840d1c00 msgr2=0x7f4884144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840d1c00 0x7f4884144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 -- 172.21.15.60:0/1948386143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840cbce0 con 0x7f48840c6df0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4883fff640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f4884153aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f487400c3f0 tx=0x7f487400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740052d0 con 0x7f48840c6df0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874005490 con 0x7f48840c6df0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48841541a0 con 0x7f48840c6df0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.604+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874007260 con 0x7f48840c6df0 2024-04-25T08:29:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.605+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48841e8770 con 0x7f48840c6df0 2024-04-25T08:29:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.606+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4884073990 con 0x7f48840c6df0 2024-04-25T08:29:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.606+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4874007400 con 0x7f48840c6df0 2024-04-25T08:29:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.607+0000 7f4880ff9640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f487006b4d0 0x7f487006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.607+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4874089970 con 0x7f48840c6df0 2024-04-25T08:29:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.607+0000 7f48837fe640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f487006b4d0 0x7f487006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.607+0000 7f48837fe640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f487006b4d0 0x7f487006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4878034000 tx=0x7f487802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.611+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f488415dfe0 con 0x7f48840c6df0 2024-04-25T08:29:41.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.840+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f48840c66c0 con 0x7f48840c6df0 2024-04-25T08:29:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.843+0000 7f4880ff9640 1 -- 172.21.15.60:0/1948386143 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171787 (secure 0 0 0) 0x7f487405ce20 con 0x7f48840c6df0 2024-04-25T08:29:41.842 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:41.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f487006b4d0 msgr2=0x7f487006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f487006b4d0 0x7f487006d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4878034000 tx=0x7f487802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 msgr2=0x7f4884153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f4884153aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f487400c3f0 tx=0x7f487400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 shutdown_connections 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f488415dc00 0x7f48840c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f487006b4d0 0x7f487006d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48840d1c00 0x7f4884144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 --2- 172.21.15.60:0/1948386143 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48840c6df0 0x7f4884153aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 >> 172.21.15.60:0/1948386143 conn(0x7f488406f5d0 msgr2=0x7f488406ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 shutdown_connections 2024-04-25T08:29:41.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.846+0000 7f488a56a640 1 -- 172.21.15.60:0/1948386143 wait complete. 2024-04-25T08:29:41.853 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":416547562},{"type":"v1","addr":"172.21.15.100:6833","nonce":416547562}]},"active_addr":"172.21.15.100:6833/416547562","active_change":"2024-04-25T08:29:03.385641+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1889002889}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":317127153}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":387723979}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3742570945}]}]} 2024-04-25T08:29:41.857 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T08:29:41.857 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T08:29:41.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.961+0000 7feaa1d9e640 1 Processor -- start 2024-04-25T08:29:41.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.961+0000 7feaa1d9e640 1 -- start start 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7feaa1d9e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea9c15dbe0 0x7fea9c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7feaa1d9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b92a0 0x7fea9c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7feaa1d9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c0b97a0 con 0x7fea9c0b2cb0 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7feaa1d9e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c15dfc0 con 0x7fea9c15dbe0 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7feaa1d9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0d1a00 con 0x7fea9c0b92a0 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea9c15dbe0 0x7fea9c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fea9c0b2cb0 0x7fea9c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41978/0 (socket says 172.21.15.60:41978) 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9affd640 1 -- 172.21.15.60:0/4011376328 learned_addr learned my addr 172.21.15.60:0/4011376328 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9bfff640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b92a0 0x7fea9c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9b7fe640 1 -- 172.21.15.60:0/4011376328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b92a0 msgr2=0x7fea9c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9b7fe640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b92a0 0x7fea9c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9b7fe640 1 -- 172.21.15.60:0/4011376328 >> v1:172.21.15.60:6789/0 conn(0x7fea9c0b2cb0 legacy=0x7fea9c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9b7fe640 1 -- 172.21.15.60:0/4011376328 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0d1c50 con 0x7fea9c15dbe0 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9a7fc640 1 -- 172.21.15.60:0/4011376328 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197947031 0 0) 0x7fea9c0b97a0 con 0x7fea9c0b2cb0 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.962+0000 7fea9bfff640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b92a0 0x7fea9c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.963+0000 7fea9b7fe640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0d0910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0097a0 tx=0x7fea8c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64948db6977709ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.963+0000 7fea9a7fc640 1 -- 172.21.15.60:0/4011376328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c037070 con 0x7fea9c15dbe0 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.963+0000 7fea9a7fc640 1 -- 172.21.15.60:0/4011376328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c032440 con 0x7fea9c15dbe0 2024-04-25T08:29:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.963+0000 7fea9a7fc640 1 -- 172.21.15.60:0/4011376328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c032810 con 0x7fea9c15dbe0 2024-04-25T08:29:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 -- 172.21.15.60:0/4011376328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c15dbe0 msgr2=0x7fea9c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0d0910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0097a0 tx=0x7fea8c02a180 comp rx=0 tx=0).stop 2024-04-25T08:29:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 -- 172.21.15.60:0/4011376328 shutdown_connections 2024-04-25T08:29:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b92a0 0x7fea9c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/4011376328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0d0910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 -- 172.21.15.60:0/4011376328 >> 172.21.15.60:0/4011376328 conn(0x7fea9c061a20 msgr2=0x7fea9c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 -- 172.21.15.60:0/4011376328 shutdown_connections 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.964+0000 7feaa1d9e640 1 -- 172.21.15.60:0/4011376328 wait complete. 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.965+0000 7feaa1d9e640 1 Processor -- start 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.965+0000 7feaa1d9e640 1 -- start start 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7feaa1d9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b2cb0 0x7fea9c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7feaa1d9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 0x7fea9c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7feaa1d9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7feaa1d9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0b6ab0 con 0x7fea9c0b92a0 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7feaa1d9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0d2210 con 0x7fea9c15dbe0 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7feaa1d9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0b6c30 con 0x7fea9c0b2cb0 2024-04-25T08:29:41.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 0x7fea9c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 0x7fea9c15f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41314/0 (socket says 172.21.15.60:41314) 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 -- 172.21.15.60:0/3643647146 learned_addr learned my addr 172.21.15.60:0/3643647146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9b7fe640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b2cb0 0x7fea9c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9bfff640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 -- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b2cb0 msgr2=0x7fea9c0b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b2cb0 0x7fea9c0b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 -- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea9c15dbe0 msgr2=0x7fea9c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.966+0000 7fea9affd640 1 -- 172.21.15.60:0/3643647146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0b3c10 con 0x7fea9c0b92a0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.967+0000 7fea9affd640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 0x7fea9c15f7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c4b0 tx=0x7fea9000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.967+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90018070 con 0x7fea9c0b92a0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.967+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea90007500 con 0x7fea9c0b92a0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.967+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90004a20 con 0x7fea9c0b92a0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.967+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c0b3ef0 con 0x7fea9c0b92a0 2024-04-25T08:29:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.967+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c0b44b0 con 0x7fea9c0b92a0 2024-04-25T08:29:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.968+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c073990 con 0x7fea9c0b92a0 2024-04-25T08:29:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.968+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea9001b070 con 0x7fea9c0b92a0 2024-04-25T08:29:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.969+0000 7fea83fff640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fea8806b4d0 0x7fea8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.969+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea90089940 con 0x7fea9c0b92a0 2024-04-25T08:29:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.969+0000 7fea9b7fe640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fea8806b4d0 0x7fea8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.970+0000 7fea9b7fe640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fea8806b4d0 0x7fea8806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fea8c034000 tx=0x7fea8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:41.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:41.973+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea900584a0 con 0x7fea9c0b92a0 2024-04-25T08:29:42.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.153+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea9c0b2820 con 0x7fea9c0b92a0 2024-04-25T08:29:42.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.154+0000 7fea83fff640 1 -- 172.21.15.60:0/3643647146 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fea90057da0 con 0x7fea9c0b92a0 2024-04-25T08:29:42.152 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:42.152 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fea8806b4d0 msgr2=0x7fea8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fea8806b4d0 0x7fea8806d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fea8c034000 tx=0x7fea8c02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 msgr2=0x7fea9c15f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 0x7fea9c15f7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c4b0 tx=0x7fea9000c970 comp rx=0 tx=0).stop 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 shutdown_connections 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fea8806b4d0 0x7fea8806d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea9c15dbe0 0x7fea9c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea9c0b92a0 0x7fea9c15f7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 --2- 172.21.15.60:0/3643647146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea9c0b2cb0 0x7fea9c0b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.156+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 >> 172.21.15.60:0/3643647146 conn(0x7fea9c061a20 msgr2=0x7fea9c15b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.157+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 shutdown_connections 2024-04-25T08:29:42.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.157+0000 7feaa1d9e640 1 -- 172.21.15.60:0/3643647146 wait complete. 2024-04-25T08:29:42.165 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T08:29:42.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:42.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.269+0000 7f9c1941b640 1 Processor -- start 2024-04-25T08:29:42.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.269+0000 7f9c1941b640 1 -- start start 2024-04-25T08:29:42.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c1941b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c140d1020 0x7f9c140d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c1941b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c140b97a0 con 0x7f9c140b2cb0 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c1941b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c140d1c50 con 0x7f9c1415dbe0 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c1941b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1415dfc0 con 0x7f9c140d1020 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c0a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9c140b2cb0 0x7f9c140d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:41980/0 (socket says 172.21.15.60:41980) 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/3516688899 learned_addr learned my addr 172.21.15.60:0/3516688899 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c137fe640 1 --2- 172.21.15.60:0/3516688899 >> v2:172.21.15.100:3300/0 conn(0x7f9c140d1020 0x7f9c140d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964279433 0 0) 0x7f9c140b97a0 con 0x7f9c140b2cb0 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c00003520 con 0x7f9c140b2cb0 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110704955 0 0) 0x7f9c00003520 con 0x7f9c140b2cb0 2024-04-25T08:29:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 >> v2:172.21.15.100:3300/0 conn(0x7f9c140d1020 msgr2=0x7f9c140d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 --2- 172.21.15.60:0/3516688899 >> v2:172.21.15.100:3300/0 conn(0x7f9c140d1020 0x7f9c140d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 >> v1:172.21.15.100:6789/0 conn(0x7f9c1415dbe0 legacy=0x7f9c140d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c140d1f30 con 0x7f9c140b2cb0 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f9bfc001670 con 0x7f9c140b2cb0 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 >> v1:172.21.15.60:6789/0 conn(0x7f9c140b2cb0 legacy=0x7f9c140d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c000039b0 0x7f9c00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c00004800 0x7f9c00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c000051c0 0x7f9c0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0000c030 con 0x7f9c00004800 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0000c1e0 con 0x7f9c000039b0 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0000c3e0 con 0x7f9c000051c0 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c00004800 0x7f9c00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.270+0000 7f9c12ffd640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c000039b0 0x7f9c00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c137fe640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c000051c0 0x7f9c0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c000051c0 msgr2=0x7f9c0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c000051c0 0x7f9c0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c000039b0 msgr2=0x7f9c00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c000039b0 0x7f9c00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/3516688899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c00003520 con 0x7f9c00004800 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c137fe640 1 -- 172.21.15.60:0/3516688899 reap_dead start 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c00004800 0x7f9c00004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c210 tx=0x7f9c00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7c30d8bee8fdd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.271+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc007d10 con 0x7f9c00004800 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc005ad0 con 0x7f9c00004800 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c127fc640 1 -- 172.21.15.60:0/3516688899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc007070 con 0x7f9c00004800 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 -- 172.21.15.60:0/3516688899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c00004800 msgr2=0x7f9c00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c00004800 0x7f9c00004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c210 tx=0x7f9c00005830 comp rx=0 tx=0).stop 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 -- 172.21.15.60:0/3516688899 shutdown_connections 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 --2- 172.21.15.60:0/3516688899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c00004800 0x7f9c00004c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 -- 172.21.15.60:0/3516688899 >> 172.21.15.60:0/3516688899 conn(0x7f9c14061a20 msgr2=0x7f9c14061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 -- 172.21.15.60:0/3516688899 shutdown_connections 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.272+0000 7f9c1941b640 1 -- 172.21.15.60:0/3516688899 wait complete. 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.273+0000 7f9c1941b640 1 Processor -- start 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 -- start start 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c00004800 0x7f9c14159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 0x7f9c14155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c1415ff20 0x7f9c141603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c140b92a0 con 0x7f9c141556e0 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c140d1a00 con 0x7f9c1415ff20 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c1941b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c140b9420 con 0x7f9c00004800 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c0a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 0x7f9c14155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c0a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 0x7f9c14155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41332/0 (socket says 172.21.15.60:41332) 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.274+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/1498585774 learned_addr learned my addr 172.21.15.60:0/1498585774 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c12ffd640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c00004800 0x7f9c14159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c137fe640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c1415ff20 0x7f9c141603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c00004800 msgr2=0x7f9c14159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c00004800 0x7f9c14159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c1415ff20 msgr2=0x7f9c141603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c1415ff20 0x7f9c141603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0a7fc640 1 -- 172.21.15.60:0/1498585774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c14160c50 con 0x7f9c141556e0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c12ffd640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c00004800 0x7f9c14159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0a7fc640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 0x7f9c14155b60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c330 tx=0x7f9c140d1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc040020 con 0x7f9c141556e0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc001670 con 0x7f9c141556e0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c140cb810 con 0x7f9c141556e0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc037040 con 0x7f9c141556e0 2024-04-25T08:29:42.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.275+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c140cbdd0 con 0x7f9c141556e0 2024-04-25T08:29:42.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.276+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c14073990 con 0x7f9c141556e0 2024-04-25T08:29:42.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.277+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bfc043020 con 0x7f9c141556e0 2024-04-25T08:29:42.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.277+0000 7f9c0bfff640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9bf406b500 0x7f9bf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.277+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9bfc0aa200 con 0x7f9c141556e0 2024-04-25T08:29:42.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.277+0000 7f9c12ffd640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9bf406b500 0x7f9bf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.278+0000 7f9c12ffd640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9bf406b500 0x7f9bf406d9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c140692f0 tx=0x7f9c04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.281+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bfc078d40 con 0x7f9c141556e0 2024-04-25T08:29:42.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.463+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c140b2890 con 0x7f9c141556e0 2024-04-25T08:29:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.464+0000 7f9c0bfff640 1 -- 172.21.15.60:0/1498585774 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f9bfc078640 con 0x7f9c141556e0 2024-04-25T08:29:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:42.461 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9bf406b500 msgr2=0x7f9bf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9bf406b500 0x7f9bf406d9c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c140692f0 tx=0x7f9c04009040 comp rx=0 tx=0).stop 2024-04-25T08:29:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 msgr2=0x7f9c14155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 0x7f9c14155b60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c330 tx=0x7f9c140d1e20 comp rx=0 tx=0).stop 2024-04-25T08:29:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 shutdown_connections 2024-04-25T08:29:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9bf406b500 0x7f9bf406d9c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c1415ff20 0x7f9c141603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c141556e0 0x7f9c14155b60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 --2- 172.21.15.60:0/1498585774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c00004800 0x7f9c14159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 >> 172.21.15.60:0/1498585774 conn(0x7f9c14061a20 msgr2=0x7f9c140b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 shutdown_connections 2024-04-25T08:29:42.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.466+0000 7f9c1941b640 1 -- 172.21.15.60:0/1498585774 wait complete. 2024-04-25T08:29:42.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T08:29:42.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T08:29:42.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T08:29:42.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T08:29:42.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T08:29:42.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T08:29:42.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T08:29:42.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T08:29:42.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.655+0000 7f969f753640 1 Processor -- start 2024-04-25T08:29:42.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.655+0000 7f969f753640 1 -- start start 2024-04-25T08:29:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7fa9db0e6640 1 Processor -- start 2024-04-25T08:29:42.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7fd5c957e640 1 Processor -- start 2024-04-25T08:29:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7fa9db0e6640 1 -- start start 2024-04-25T08:29:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7fa9db0e6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9cc0e7700 0x7fa9cc04ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9db0e6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 0x7fa9cc007010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9db0e6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc057d80 con 0x7fa9cc0589b0 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9db0e6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc006250 con 0x7fa9cc0e7700 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9db0e6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc006530 con 0x7fa9cc053c30 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 0x7fa9cc007010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 0x7fa9cc007010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58838/0 (socket says 172.21.15.60:58838) 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 -- 172.21.15.60:0/369598699 learned_addr learned my addr 172.21.15.60:0/369598699 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f28e9f5f640 1 Processor -- start 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7f969f753640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9698157700 0x7f969815e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7f969f753640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f96980cd2e0 0x7f96980cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969f753640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9698071c50 con 0x7f969806b6d0 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969f753640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969815e7d0 con 0x7f96980cd2e0 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969f753640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969815d840 con 0x7f9698157700 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969d4ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9698157700 0x7f969815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969d4ef640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9698157700 0x7f969815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58820/0 (socket says 172.21.15.60:58820) 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969d4ef640 1 -- 172.21.15.60:0/2251972926 learned_addr learned my addr 172.21.15.60:0/2251972926 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f969ccee640 1 --2- 172.21.15.60:0/2251972926 >> v2:172.21.15.60:3300/0 conn(0x7f96980cd2e0 0x7f96980cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f35d809c640 1 Processor -- start 2024-04-25T08:29:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f45e7fff640 1 Processor -- start 2024-04-25T08:29:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f45e7fff640 1 -- start start 2024-04-25T08:29:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7f87a397d640 1 Processor -- start 2024-04-25T08:29:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7fd5c957e640 1 -- start start 2024-04-25T08:29:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7fd5c957e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5bc03e870 0x7fd5bc053c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7fd5c957e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc006900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7fd5c957e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc0585a0 con 0x7fd5bc0e30f0 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7fd5c957e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5bc058880 con 0x7fd5bc03e870 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7fd5c957e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc058b60 con 0x7fd5bc054160 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5bc03e870 0x7fd5bc053c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd5bc03e870 0x7fd5bc053c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41348/0 (socket says 172.21.15.60:41348) 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 -- 172.21.15.60:0/2812913558 learned_addr learned my addr 172.21.15.60:0/2812913558 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c37fe640 1 --2- 172.21.15.60:0/2812913558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc006900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f198d67b640 1 Processor -- start 2024-04-25T08:29:42.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f198d67b640 1 -- start start 2024-04-25T08:29:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f969ccee640 1 -- 172.21.15.60:0/2251972926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9698157700 msgr2=0x7f969815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f969ccee640 1 --2- 172.21.15.60:0/2251972926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9698157700 0x7f969815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f969ccee640 1 -- 172.21.15.60:0/2251972926 >> v1:172.21.15.60:6789/0 conn(0x7f969806b6d0 legacy=0x7f96980ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f969ccee640 1 -- 172.21.15.60:0/2251972926 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698071f30 con 0x7f96980cd2e0 2024-04-25T08:29:42.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.657+0000 7f28e9f5f640 1 -- start start 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f28e9f5f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc006b40 0x7f28dc007f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f28e9f5f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28dc0342e0 0x7f28dc053980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f28e9f5f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc055d00 con 0x7f28dc0dfa50 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f28e9f5f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc055fe0 con 0x7f28dc0342e0 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f28e9f5f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc036a70 con 0x7f28dc006b40 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f198d67b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19880af660 0x7f19880b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f198d67b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988162d10 0x7f1988158810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f198d67b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1988163190 con 0x7f1988158d50 2024-04-25T08:29:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f198d67b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880b3580 con 0x7f19880af660 2024-04-25T08:29:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f198d67b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880b3860 con 0x7f1988162d10 2024-04-25T08:29:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f28e3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f28dc0dfa50 0x7f28dc006f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42008/0 (socket says 172.21.15.60:42008) 2024-04-25T08:29:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.656+0000 7f87a397d640 1 -- start start 2024-04-25T08:29:42.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a397d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f879c06c180 0x7f879c0cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a397d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f879c0b2bb0 0x7f879c165b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a397d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c0d2a00 con 0x7f879c0d15f0 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a397d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0d2ce0 con 0x7f879c06c180 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a397d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0d2fc0 con 0x7f879c0b2bb0 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a1f1a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f879c0b2bb0 0x7f879c165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19880af660 0x7f19880b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f19880af660 0x7f19880b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33196/0 (socket says 172.21.15.60:33196) 2024-04-25T08:29:42.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 -- 172.21.15.60:0/3671596359 learned_addr learned my addr 172.21.15.60:0/3671596359 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a0f18640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f879c0d15f0 0x7f879c06c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42002/0 (socket says 172.21.15.60:42002) 2024-04-25T08:29:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a0f18640 1 -- 172.21.15.60:0/1042924254 learned_addr learned my addr 172.21.15.60:0/1042924254 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7f87a1f1a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f879c0b2bb0 0x7f879c165b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33188/0 (socket says 172.21.15.60:33188) 2024-04-25T08:29:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f28e3fff640 1 -- 172.21.15.60:0/3485230888 learned_addr learned my addr 172.21.15.60:0/3485230888 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f28e975e640 1 --2- 172.21.15.60:0/3485230888 >> v2:172.21.15.60:3300/0 conn(0x7f28dc0342e0 0x7f28dc053980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f87a1719640 1 --2- 172.21.15.60:0/1042924254 >> v2:172.21.15.60:3300/0 conn(0x7f879c06c180 0x7f879c0cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 -- 172.21.15.60:0/369598699 >> v2:172.21.15.100:3300/0 conn(0x7fa9cc0e7700 msgr2=0x7fa9cc04ca20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:29:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 --2- 172.21.15.60:0/369598699 >> v2:172.21.15.100:3300/0 conn(0x7fa9cc0e7700 0x7fa9cc04ca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 -- 172.21.15.60:0/369598699 >> v1:172.21.15.100:6789/0 conn(0x7fa9cc0589b0 legacy=0x7fa9cc053520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fa9da8e5640 1 -- 172.21.15.60:0/369598699 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc006810 con 0x7fa9cc053c30 2024-04-25T08:29:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f87a1f1a640 1 -- 172.21.15.60:0/1042924254 >> v2:172.21.15.60:3300/0 conn(0x7f879c06c180 msgr2=0x7f879c0cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f87a1f1a640 1 --2- 172.21.15.60:0/1042924254 >> v2:172.21.15.60:3300/0 conn(0x7f879c06c180 0x7f879c0cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f87a1f1a640 1 -- 172.21.15.60:0/1042924254 >> v1:172.21.15.60:6789/0 conn(0x7f879c0d15f0 legacy=0x7f879c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 -- 172.21.15.60:0/2812913558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 msgr2=0x7fd5bc006900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f87a1f1a640 1 -- 172.21.15.60:0/1042924254 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0d32a0 con 0x7f879c0b2bb0 2024-04-25T08:29:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f87a1719640 1 --2- 172.21.15.60:0/1042924254 >> v2:172.21.15.60:3300/0 conn(0x7f879c06c180 0x7f879c0cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 --2- 172.21.15.60:0/2812913558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc006900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 -- 172.21.15.60:0/2812913558 >> v1:172.21.15.100:6789/0 conn(0x7fd5bc0e30f0 legacy=0x7fd5bc057dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f28e8f5d640 1 --2- 172.21.15.60:0/3485230888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc006b40 0x7f28dc007f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f45e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45e80b95c0 0x7f45e80b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f45e6ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f45e815dbd0 0x7f45e8161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43658/0 (socket says 172.21.15.60:43658) 2024-04-25T08:29:42.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.658+0000 7fd5c27fc640 1 -- 172.21.15.60:0/2812913558 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc0063e0 con 0x7fd5bc03e870 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f28e8f5d640 1 -- 172.21.15.60:0/3485230888 >> v2:172.21.15.60:3300/0 conn(0x7f28dc0342e0 msgr2=0x7f28dc053980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f28e8f5d640 1 --2- 172.21.15.60:0/3485230888 >> v2:172.21.15.60:3300/0 conn(0x7f28dc0342e0 0x7f28dc053980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f28e8f5d640 1 -- 172.21.15.60:0/3485230888 >> v1:172.21.15.60:6789/0 conn(0x7f28dc0dfa50 legacy=0x7f28dc006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f28e8f5d640 1 -- 172.21.15.60:0/3485230888 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc034d10 con 0x7f28dc006b40 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f45e6ffd640 1 -- 172.21.15.60:0/1062924824 learned_addr learned my addr 172.21.15.60:0/1062924824 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7fd5c27fc640 1 --2- 172.21.15.60:0/2812913558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc03e870 0x7fd5bc053c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009870 tx=0x7fd5b402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa25c543a2f569f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f45e67fc640 1 --2- 172.21.15.60:0/1062924824 >> v2:172.21.15.60:3300/0 conn(0x7f45e80b95c0 0x7f45e80b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f28e975e640 1 --2- 172.21.15.60:0/3485230888 >> v2:172.21.15.60:3300/0 conn(0x7f28dc0342e0 0x7f28dc053980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7fd5c1ffb640 1 -- 172.21.15.60:0/2812913558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4037070 con 0x7fd5bc03e870 2024-04-25T08:29:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 -- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988162d10 msgr2=0x7f1988158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 --2- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988162d10 0x7f1988158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 -- 172.21.15.60:0/3671596359 >> v1:172.21.15.60:6789/0 conn(0x7f1988158d50 legacy=0x7f198806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.659+0000 7f1987fff640 1 -- 172.21.15.60:0/3671596359 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880b3b40 con 0x7f19880af660 2024-04-25T08:29:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e8161e70 con 0x7f45e80b5910 2024-04-25T08:29:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7fd5c1ffb640 1 -- 172.21.15.60:0/2812913558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b402ae50 con 0x7fd5bc03e870 2024-04-25T08:29:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f1987fff640 1 --2- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 0x7f19880b2e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1980009e50 tx=0x7f198002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb0b7ac8f151aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e8162150 con 0x7f45e815dbd0 2024-04-25T08:29:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e80d1ce0 con 0x7f45e80b95c0 2024-04-25T08:29:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f35d809c640 1 -- start start 2024-04-25T08:29:42.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7fd5c1ffb640 1 -- 172.21.15.60:0/2812913558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4032860 con 0x7fd5bc03e870 2024-04-25T08:29:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f969ccee640 1 --2- 172.21.15.60:0/2251972926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96980cd2e0 0x7f96980cd760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9690009d90 tx=0x7f969002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb483fcc1bab755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f1986ffd640 1 -- 172.21.15.60:0/3671596359 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980036070 con 0x7f19880af660 2024-04-25T08:29:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f1986ffd640 1 -- 172.21.15.60:0/3671596359 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980031600 con 0x7f19880af660 2024-04-25T08:29:42.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7fa9da8e5640 1 --2- 172.21.15.60:0/369598699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 0x7fa9cc007010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40692f0 tx=0x7fa9d40a51c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6413f49323eee2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f968ffff640 1 -- 172.21.15.60:0/2251972926 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690007740 con 0x7f96980cd2e0 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f968ffff640 1 -- 172.21.15.60:0/2251972926 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96900078e0 con 0x7f96980cd2e0 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f968ffff640 1 -- 172.21.15.60:0/2251972926 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690007cb0 con 0x7f96980cd2e0 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d809c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d015e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d809c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35d00cd2e0 0x7f35d00cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d5637640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35d00cd2e0 0x7f35d00cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d809c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d0071c50 con 0x7f35d00ae830 2024-04-25T08:29:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d809c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d015e7d0 con 0x7f35d00cd2e0 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d809c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d015d840 con 0x7f35d0157700 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d5637640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f35d00cd2e0 0x7f35d00cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41384/0 (socket says 172.21.15.60:41384) 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f35d5637640 1 -- 172.21.15.60:0/91567077 learned_addr learned my addr 172.21.15.60:0/91567077 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608960045 0 0) 0x7f45e8161e70 con 0x7f45e80b5910 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d0003520 con 0x7f45e80b5910 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252248622 0 0) 0x7f45e8162150 con 0x7f45e815dbd0 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e8161e70 con 0x7f45e815dbd0 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e67fc640 1 -- 172.21.15.60:0/1062924824 >> v1:172.21.15.100:6789/0 conn(0x7f45e815dbd0 legacy=0x7f45e8161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e67fc640 1 -- 172.21.15.60:0/1062924824 >> v1:172.21.15.60:6789/0 conn(0x7f45e80b5910 legacy=0x7f45e80d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f45e67fc640 1 -- 172.21.15.60:0/1062924824 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e8162430 con 0x7f45e80b95c0 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.660+0000 7f87a1f1a640 1 --2- 172.21.15.60:0/1042924254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0b2bb0 0x7f879c165b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f879c08ce10 tx=0x7f878c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7584547caf7e65a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f8793fff640 1 -- 172.21.15.60:0/1042924254 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c037070 con 0x7f879c0b2bb0 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f8793fff640 1 -- 172.21.15.60:0/1042924254 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c032440 con 0x7f879c0b2bb0 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f8793fff640 1 -- 172.21.15.60:0/1042924254 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c02ae60 con 0x7f879c0b2bb0 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7f1986ffd640 1 -- 172.21.15.60:0/3671596359 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800319d0 con 0x7f19880af660 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f198d67b640 1 -- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 msgr2=0x7f19880b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f198d67b640 1 --2- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 0x7f19880b2e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1980009e50 tx=0x7f198002a710 comp rx=0 tx=0).stop 2024-04-25T08:29:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fa9d90e2640 1 -- 172.21.15.60:0/369598699 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40b2070 con 0x7fa9cc053c30 2024-04-25T08:29:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fa9d90e2640 1 -- 172.21.15.60:0/369598699 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d40ad440 con 0x7fa9cc053c30 2024-04-25T08:29:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fa9d90e2640 1 -- 172.21.15.60:0/369598699 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40ad8b0 con 0x7fa9cc053c30 2024-04-25T08:29:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 -- 172.21.15.60:0/2251972926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96980cd2e0 msgr2=0x7f96980cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 --2- 172.21.15.60:0/2251972926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96980cd2e0 0x7f96980cd760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9690009d90 tx=0x7f969002a660 comp rx=0 tx=0).stop 2024-04-25T08:29:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f35d5e38640 1 --2- 172.21.15.60:0/91567077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f198d67b640 1 -- 172.21.15.60:0/3671596359 shutdown_connections 2024-04-25T08:29:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f198d67b640 1 --2- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988162d10 0x7f1988158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f198d67b640 1 --2- 172.21.15.60:0/3671596359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 0x7f19880b2e80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f198d67b640 1 -- 172.21.15.60:0/3671596359 >> 172.21.15.60:0/3671596359 conn(0x7f1988071ad0 msgr2=0x7f1988071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f35d4e36640 1 -- 172.21.15.60:0/91567077 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187319187 0 0) 0x7f35d0071c50 con 0x7f35d00ae830 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f35d5637640 1 -- 172.21.15.60:0/91567077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 msgr2=0x7f35d015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f35d5637640 1 --2- 172.21.15.60:0/91567077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f35d5637640 1 -- 172.21.15.60:0/91567077 >> v1:172.21.15.100:6789/0 conn(0x7f35d00ae830 legacy=0x7f35d015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 -- 172.21.15.60:0/2251972926 shutdown_connections 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 --2- 172.21.15.60:0/2251972926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96980cd2e0 0x7f96980cd760 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 --2- 172.21.15.60:0/2251972926 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9698157700 0x7f969815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 -- 172.21.15.60:0/2251972926 >> 172.21.15.60:0/2251972926 conn(0x7f969806abe0 msgr2=0x7f969806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f35d5637640 1 -- 172.21.15.60:0/91567077 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0071f30 con 0x7f35d00cd2e0 2024-04-25T08:29:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f28e8f5d640 1 --2- 172.21.15.60:0/3485230888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc006b40 0x7f28dc007f60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e9c77ff6aa8321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f45e67fc640 1 --2- 172.21.15.60:0/1062924824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b95c0 0x7f45e80b53d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097a0 tx=0x7f45dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87de79b41ec4fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7fd5c957e640 1 -- 172.21.15.60:0/2812913558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc03e870 msgr2=0x7fd5bc053c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f969f753640 1 -- 172.21.15.60:0/2251972926 shutdown_connections 2024-04-25T08:29:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.661+0000 7fd5c957e640 1 --2- 172.21.15.60:0/2812913558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc03e870 0x7fd5bc053c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009870 tx=0x7fd5b402a300 comp rx=0 tx=0).stop 2024-04-25T08:29:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7f28e37fe640 1 -- 172.21.15.60:0/3485230888 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8037070 con 0x7f28dc006b40 2024-04-25T08:29:42.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fd5c957e640 1 -- 172.21.15.60:0/2812913558 shutdown_connections 2024-04-25T08:29:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f28e37fe640 1 -- 172.21.15.60:0/3485230888 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d8032440 con 0x7f28dc006b40 2024-04-25T08:29:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f28e37fe640 1 -- 172.21.15.60:0/3485230888 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8032810 con 0x7f28dc006b40 2024-04-25T08:29:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fd5c957e640 1 --2- 172.21.15.60:0/2812913558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc03e870 0x7fd5bc053c20 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009870 tx=0x7fd5b402a300 comp rx=0 tx=0).stop 2024-04-25T08:29:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 -- 172.21.15.60:0/369598699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 msgr2=0x7fa9cc007010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 --2- 172.21.15.60:0/369598699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 0x7fa9cc007010 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa9d40692f0 tx=0x7fa9d40a51c0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f969f753640 1 -- 172.21.15.60:0/2251972926 wait complete. 2024-04-25T08:29:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc037070 con 0x7f45e80b95c0 2024-04-25T08:29:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fd5c957e640 1 --2- 172.21.15.60:0/2812913558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc006900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.662+0000 7fd5c957e640 1 -- 172.21.15.60:0/2812913558 >> 172.21.15.60:0/2812913558 conn(0x7fd5bc055a10 msgr2=0x7fd5bc055e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 -- 172.21.15.60:0/1042924254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0b2bb0 msgr2=0x7f879c165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc032440 con 0x7f45e80b95c0 2024-04-25T08:29:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fd5c957e640 1 -- 172.21.15.60:0/2812913558 shutdown_connections 2024-04-25T08:29:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f45e5ffb640 1 -- 172.21.15.60:0/1062924824 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc032810 con 0x7f45e80b95c0 2024-04-25T08:29:42.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fd5c957e640 1 -- 172.21.15.60:0/2812913558 wait complete. 2024-04-25T08:29:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 -- 172.21.15.60:0/369598699 shutdown_connections 2024-04-25T08:29:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 --2- 172.21.15.60:0/369598699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc053c30 0x7fa9cc007010 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 --2- 172.21.15.60:0/369598699 >> v2:172.21.15.100:3300/0 conn(0x7fa9cc0e7700 0x7fa9cc04ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 -- 172.21.15.60:0/369598699 >> 172.21.15.60:0/369598699 conn(0x7fa9cc058ed0 msgr2=0x7fa9cc0592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 -- 172.21.15.60:0/369598699 shutdown_connections 2024-04-25T08:29:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f35d5637640 1 --2- 172.21.15.60:0/91567077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00cd2e0 0x7f35d00cd760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009870 tx=0x7f35cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea400e2cfdc8e26d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b95c0 msgr2=0x7f45e80b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f45e7fff640 1 --2- 172.21.15.60:0/1062924824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b95c0 0x7f45e80b53d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0097a0 tx=0x7f45dc02a230 comp rx=0 tx=0).stop 2024-04-25T08:29:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f35d4e36640 1 -- 172.21.15.60:0/91567077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc037070 con 0x7f35d00cd2e0 2024-04-25T08:29:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7fa9db0e6640 1 -- 172.21.15.60:0/369598699 wait complete. 2024-04-25T08:29:42.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f35d4e36640 1 -- 172.21.15.60:0/91567077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc032440 con 0x7f35d00cd2e0 2024-04-25T08:29:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 shutdown_connections 2024-04-25T08:29:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f45e7fff640 1 --2- 172.21.15.60:0/1062924824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b95c0 0x7f45e80b53d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 >> 172.21.15.60:0/1062924824 conn(0x7f45e80d03d0 msgr2=0x7f45e80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f35d4e36640 1 -- 172.21.15.60:0/91567077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc032810 con 0x7f35d00cd2e0 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f969f753640 1 Processor -- start 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.664+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 shutdown_connections 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 --2- 172.21.15.60:0/1042924254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0b2bb0 0x7f879c165b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f879c08ce10 tx=0x7f878c02a110 comp rx=0 tx=0).stop 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 -- 172.21.15.60:0/1042924254 shutdown_connections 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 --2- 172.21.15.60:0/1042924254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0b2bb0 0x7f879c165b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 --2- 172.21.15.60:0/1042924254 >> v2:172.21.15.60:3300/0 conn(0x7f879c06c180 0x7f879c0cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 -- 172.21.15.60:0/1042924254 >> 172.21.15.60:0/1042924254 conn(0x7f879c0d3a20 msgr2=0x7f879c0d3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 -- 172.21.15.60:0/1042924254 shutdown_connections 2024-04-25T08:29:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.663+0000 7f87a397d640 1 -- 172.21.15.60:0/1042924254 wait complete. 2024-04-25T08:29:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f969f753640 1 -- start start 2024-04-25T08:29:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f45e7fff640 1 -- 172.21.15.60:0/1062924824 wait complete. 2024-04-25T08:29:42.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f35d809c640 1 -- 172.21.15.60:0/91567077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00cd2e0 msgr2=0x7f35d00cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7fa9db0e6640 1 Processor -- start 2024-04-25T08:29:42.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f35d809c640 1 --2- 172.21.15.60:0/91567077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00cd2e0 0x7f35d00cd760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009870 tx=0x7f35cc02a270 comp rx=0 tx=0).stop 2024-04-25T08:29:42.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7fa9db0e6640 1 -- start start 2024-04-25T08:29:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f35d809c640 1 -- 172.21.15.60:0/91567077 shutdown_connections 2024-04-25T08:29:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f35d809c640 1 --2- 172.21.15.60:0/91567077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00cd2e0 0x7f35d00cd760 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f35d809c640 1 --2- 172.21.15.60:0/91567077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7f35d809c640 1 -- 172.21.15.60:0/91567077 >> 172.21.15.60:0/91567077 conn(0x7f35d006abe0 msgr2=0x7f35d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f198d67b640 1 -- 172.21.15.60:0/3671596359 shutdown_connections 2024-04-25T08:29:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7fd5c957e640 1 Processor -- start 2024-04-25T08:29:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f87a397d640 1 Processor -- start 2024-04-25T08:29:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969f753640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969806b6d0 0x7f969812f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.665+0000 7fd5c957e640 1 -- start start 2024-04-25T08:29:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969f753640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96980cd2e0 0x7f9698147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969f753640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9698157700 0x7f9698158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969f753640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980718e0 con 0x7f9698157700 2024-04-25T08:29:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969f753640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969815dd70 con 0x7f969806b6d0 2024-04-25T08:29:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969f753640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9698157ae0 con 0x7f96980cd2e0 2024-04-25T08:29:42.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f969ccee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96980cd2e0 0x7f9698147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f198d67b640 1 -- 172.21.15.60:0/3671596359 wait complete. 2024-04-25T08:29:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96980cd2e0 0x7f9698147e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58874/0 (socket says 172.21.15.60:58874) 2024-04-25T08:29:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 -- 172.21.15.60:0/1698852390 learned_addr learned my addr 172.21.15.60:0/1698852390 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7fd5c957e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc0db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7fd5c957e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc0e30f0 0x7fd5bc0db7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7fd5c957e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc16eb20 0x7fd5bc03c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7fd5c957e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc006e40 con 0x7fd5bc16eb20 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7fd5c957e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc0e34d0 con 0x7fd5bc0e30f0 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7fd5c957e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc006fc0 con 0x7fd5bc054160 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fa9db0e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9cc053c30 0x7fa9cc0ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969d4ef640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969806b6d0 0x7f969812f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969dcf0640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9698157700 0x7f9698158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f35d809c640 1 -- 172.21.15.60:0/91567077 shutdown_connections 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f35d809c640 1 -- 172.21.15.60:0/91567077 wait complete. 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f45e7fff640 1 Processor -- start 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f45e7fff640 1 -- start start 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fa9da0e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9cc053c30 0x7fa9cc0ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 -- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969806b6d0 msgr2=0x7f969812f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f969806b6d0 0x7f969812f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 -- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9698157700 msgr2=0x7f9698158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9698157700 0x7f9698158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7f969ccee640 1 -- 172.21.15.60:0/1698852390 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698156500 con 0x7f96980cd2e0 2024-04-25T08:29:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fd5c2ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc0db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fd5c2ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc0db2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58876/0 (socket says 172.21.15.60:58876) 2024-04-25T08:29:42.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fd5c2ffd640 1 -- 172.21.15.60:0/4268572019 learned_addr learned my addr 172.21.15.60:0/4268572019 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fa9da0e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9cc053c30 0x7fa9cc0ddd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33220/0 (socket says 172.21.15.60:33220) 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.667+0000 7fa9db0e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9cc0589b0 0x7fa9cc03c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fa9da0e4640 1 -- 172.21.15.60:0/2663550186 learned_addr learned my addr 172.21.15.60:0/2663550186 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fa9db0e6640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc0e7700 0x7fa9cc04e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3485230888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc006b40 msgr2=0x7f28dc007f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 --2- 172.21.15.60:0/3485230888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc006b40 0x7f28dc007f60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802a180 comp rx=0 tx=0).stop 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3485230888 shutdown_connections 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 --2- 172.21.15.60:0/3485230888 >> v2:172.21.15.60:3300/0 conn(0x7f28dc0342e0 0x7f28dc053980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 --2- 172.21.15.60:0/3485230888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc006b40 0x7f28dc007f60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3485230888 >> 172.21.15.60:0/3485230888 conn(0x7f28dc01e6b0 msgr2=0x7f28dc0553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fa9d98e3640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9cc0589b0 0x7fa9cc03c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.666+0000 7f87a397d640 1 -- start start 2024-04-25T08:29:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3485230888 shutdown_connections 2024-04-25T08:29:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f87a397d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f879c06c180 0x7f879c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f87a397d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f879c0b2bb0 0x7f879c072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f87a397d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0d15f0 0x7f879c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f87a397d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0b3040 con 0x7f879c0b2bb0 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f87a397d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0d2580 con 0x7f879c0d15f0 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f87a397d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0b31c0 con 0x7f879c06c180 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b5910 0x7f45e81609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f969ccee640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96980cd2e0 0x7f9698147e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9690009eb0 tx=0x7f9690004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f198d67b640 1 Processor -- start 2024-04-25T08:29:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f198d67b640 1 -- start start 2024-04-25T08:29:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fd5c2ffd640 1 -- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc0e30f0 msgr2=0x7fd5bc0db7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:29:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3485230888 wait complete. 2024-04-25T08:29:42.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7f45e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b5910 0x7f45e81609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f87a1719640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f879c06c180 0x7f879c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e80b95c0 0x7f45e815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fd5c2ffd640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5bc0e30f0 0x7fd5bc0db7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fd5c2ffd640 1 -- 172.21.15.60:0/4268572019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc16eb20 msgr2=0x7fd5bc03c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fd5c2ffd640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5bc16eb20 0x7fd5bc03c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f87a1f1a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0d15f0 0x7f879c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fd5c2ffd640 1 -- 172.21.15.60:0/4268572019 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc04cbb0 con 0x7fd5bc054160 2024-04-25T08:29:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e815dbd0 0x7f45e80c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b5910 0x7f45e81609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41404/0 (socket says 172.21.15.60:41404) 2024-04-25T08:29:42.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f87a1f1a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0d15f0 0x7f879c1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33224/0 (socket says 172.21.15.60:33224) 2024-04-25T08:29:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e6ffd640 1 -- 172.21.15.60:0/755943599 learned_addr learned my addr 172.21.15.60:0/755943599 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7fd5c2ffd640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc0db2a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c430 tx=0x7fd5b800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e80d2a00 con 0x7f45e80b5910 2024-04-25T08:29:42.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f87a1f1a640 1 -- 172.21.15.60:0/3033602951 learned_addr learned my addr 172.21.15.60:0/3033602951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f35d809c640 1 Processor -- start 2024-04-25T08:29:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e80d2880 con 0x7f45e80b95c0 2024-04-25T08:29:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e77fe640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e815dbd0 0x7f45e80c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.668+0000 7fa9db0e6640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc057990 con 0x7fa9cc0589b0 2024-04-25T08:29:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7fa9db0e6640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc04cf60 con 0x7fa9cc053c30 2024-04-25T08:29:42.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7fa9db0e6640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc057b10 con 0x7fa9cc0e7700 2024-04-25T08:29:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f35d809c640 1 -- start start 2024-04-25T08:29:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e80d2b80 con 0x7f45e815dbd0 2024-04-25T08:29:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7fa9da8e5640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc0e7700 0x7fa9cc04e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f198d67b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 0x7f19881564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f198d67b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988158d50 0x7f198815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f198d67b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1988162d10 0x7f19880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f198d67b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198806c480 con 0x7f1988162d10 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f198d67b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198806c300 con 0x7f19880af660 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f198d67b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1988163af0 con 0x7f1988158d50 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f87a1719640 1 -- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0d15f0 msgr2=0x7f879c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f87a1719640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f879c0d15f0 0x7f879c1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f87a1719640 1 -- 172.21.15.60:0/3033602951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f879c0b2bb0 msgr2=0x7f879c072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f87a1719640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f879c0b2bb0 0x7f879c072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8017070 con 0x7fd5bc054160 2024-04-25T08:29:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b80052d0 con 0x7fd5bc054160 2024-04-25T08:29:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8003830 con 0x7fd5bc054160 2024-04-25T08:29:42.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f87a1719640 1 -- 172.21.15.60:0/3033602951 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c153560 con 0x7f879c06c180 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fa9da8e5640 1 -- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9cc053c30 msgr2=0x7fa9cc0ddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fa9da8e5640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9cc053c30 0x7fa9cc0ddd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fa9da8e5640 1 -- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9cc0589b0 msgr2=0x7fa9cc03c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fa9da8e5640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9cc0589b0 0x7fa9cc03c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fa9da8e5640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc03d330 con 0x7fa9cc0e7700 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f28e9f5f640 1 Processor -- start 2024-04-25T08:29:42.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f28e9f5f640 1 -- start start 2024-04-25T08:29:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f87a1719640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f879c06c180 0x7f879c152e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f879800c5e0 tx=0x7f879800caa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7fa9da8e5640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc0e7700 0x7fa9cc04e130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa9d406a560 tx=0x7fa9d4068960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f35d809c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00ae830 0x7f35d00aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f198ce7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1988162d10 0x7f19880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f198ce7a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1988162d10 0x7f19880cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41416/0 (socket says 172.21.15.60:41416) 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f198ce7a640 1 -- 172.21.15.60:0/3270764982 learned_addr learned my addr 172.21.15.60:0/3270764982 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690040070 con 0x7f96980cd2e0 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.669+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690031c40 con 0x7f96980cd2e0 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690009450 con 0x7f96980cd2e0 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96981567e0 con 0x7f96980cd2e0 2024-04-25T08:29:42.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969814be50 con 0x7f96980cd2e0 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f45e77fe640 1 -- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e80b95c0 msgr2=0x7f45e815ed60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f45e77fe640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e80b95c0 0x7f45e815ed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f45e77fe640 1 -- 172.21.15.60:0/755943599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b5910 msgr2=0x7f45e81609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f45e77fe640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e80b5910 0x7f45e81609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.670+0000 7f45e77fe640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e815e3b0 con 0x7f45e815dbd0 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f45e77fe640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e815dbd0 0x7f45e80c8e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c730 tx=0x7f45e000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798015070 con 0x7f879c06c180 2024-04-25T08:29:42.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f87a397d640 1 -- 172.21.15.60:0/3033602951 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c153840 con 0x7f879c06c180 2024-04-25T08:29:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f198ce7a640 1 -- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988158d50 msgr2=0x7f198815b140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:29:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f198ce7a640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1988158d50 0x7f198815b140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f198ce7a640 1 -- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 msgr2=0x7f19881564f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:29:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f198ce7a640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19880af660 0x7f19881564f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f198ce7a640 1 -- 172.21.15.60:0/3270764982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19881f89f0 con 0x7f1988162d10 2024-04-25T08:29:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f35d809c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35d00cd2e0 0x7f35d00b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f35d5e38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00ae830 0x7f35d00aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40bc070 con 0x7fa9cc0e7700 2024-04-25T08:29:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f35d5e38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00ae830 0x7f35d00aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41434/0 (socket says 172.21.15.60:41434) 2024-04-25T08:29:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f35d5e38640 1 -- 172.21.15.60:0/4216449056 learned_addr learned my addr 172.21.15.60:0/4216449056 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f35d809c640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d01515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7fa9db0e6640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc03d050 con 0x7fa9cc0e7700 2024-04-25T08:29:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00718e0 con 0x7f35d00ae830 2024-04-25T08:29:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d015dd70 con 0x7f35d00cd2e0 2024-04-25T08:29:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0157ae0 con 0x7f35d0157700 2024-04-25T08:29:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f198ce7a640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1988162d10 0x7f19880cea70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c3f0 tx=0x7f197c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7fa9db0e6640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc0e9310 con 0x7fa9cc0e7700 2024-04-25T08:29:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d40b7930 con 0x7fa9cc0e7700 2024-04-25T08:29:42.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40b2040 con 0x7fa9cc0e7700 2024-04-25T08:29:42.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9698073a00 con 0x7f96980cd2e0 2024-04-25T08:29:42.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f28e9f5f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc006b40 0x7f28dc0dee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d6639640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d01515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e0003b70 con 0x7f45e815dbd0 2024-04-25T08:29:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f28e9f5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28dc0342e0 0x7f28dc048f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d5637640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35d00cd2e0 0x7f35d00b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e0003d10 con 0x7f45e815dbd0 2024-04-25T08:29:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d5e38640 1 -- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 msgr2=0x7f35d01515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d5e38640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f35d0157700 0x7f35d01515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d5e38640 1 -- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35d00cd2e0 msgr2=0x7f35d00b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e00052d0 con 0x7f45e815dbd0 2024-04-25T08:29:42.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.671+0000 7f87a397d640 1 -- 172.21.15.60:0/3033602951 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c153e00 con 0x7f879c06c180 2024-04-25T08:29:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798011800 con 0x7f879c06c180 2024-04-25T08:29:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d5e38640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35d00cd2e0 0x7f35d00b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f35d5e38640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00af350 con 0x7f35d00ae830 2024-04-25T08:29:42.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f28e9f5f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc0dfa50 0x7f28dc049450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f35d5637640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35d00cd2e0 0x7f35d00b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28dc0342e0 0x7f28dc048f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28dc0342e0 0x7f28dc048f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33262/0 (socket says 172.21.15.60:33262) 2024-04-25T08:29:42.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f28e3fff640 1 -- 172.21.15.60:0/3687748967 learned_addr learned my addr 172.21.15.60:0/3687748967 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0052d0 con 0x7f1988162d10 2024-04-25T08:29:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.672+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c005470 con 0x7f1988162d10 2024-04-25T08:29:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c007260 con 0x7f1988162d10 2024-04-25T08:29:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f35d5e38640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00ae830 0x7f35d00aecc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35c4010fc0 tx=0x7f35c400e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19881f8cd0 con 0x7f1988162d10 2024-04-25T08:29:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc036810 con 0x7f28dc006b40 2024-04-25T08:29:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881f9290 con 0x7f1988162d10 2024-04-25T08:29:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798011bd0 con 0x7f879c06c180 2024-04-25T08:29:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e815e690 con 0x7f45e815dbd0 2024-04-25T08:29:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc036690 con 0x7f28dc0342e0 2024-04-25T08:29:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e80cac50 con 0x7f45e815dbd0 2024-04-25T08:29:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.673+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9690039040 con 0x7f96980cd2e0 2024-04-25T08:29:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc007630 con 0x7f28dc0dfa50 2024-04-25T08:29:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f968dffb640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f967c06b5a0 0x7f967c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9d40bf070 con 0x7fa9cc0e7700 2024-04-25T08:29:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f96900abc30 con 0x7f96980cd2e0 2024-04-25T08:29:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f968dffb640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7f967c074d20 0x7f967c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 --> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f969000aea0 con 0x7f967c074d20 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96900ae290 con 0x7f96980cd2e0 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e8f5d640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc006b40 0x7f28dc0dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e3fff640 1 -- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc0dfa50 msgr2=0x7f28dc049450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e3fff640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28dc0dfa50 0x7f28dc049450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e3fff640 1 -- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc006b40 msgr2=0x7f28dc0dee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e3fff640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28dc006b40 0x7f28dc0dee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f28e3fff640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc169660 con 0x7f28dc0342e0 2024-04-25T08:29:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45e815e160 con 0x7f45e815dbd0 2024-04-25T08:29:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9cc005380 con 0x7fa9cc0e7700 2024-04-25T08:29:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7fa9caffd640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9b806b5a0 0x7fa9b806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa9d4125af0 con 0x7fa9cc0e7700 2024-04-25T08:29:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7fa9caffd640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7fa9b8074ca0 0x7fa9b80770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8798018070 con 0x7f879c06c180 2024-04-25T08:29:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc186060 con 0x7fd5bc054160 2024-04-25T08:29:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.675+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc0dc350 con 0x7fd5bc054160 2024-04-25T08:29:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f969dcf0640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7f967c074d20 0x7f967c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f969d4ef640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f967c06b5a0 0x7f967c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7fa9d98e3640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7fa9b8074ca0 0x7fa9b80770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5bc005810 con 0x7fd5bc054160 2024-04-25T08:29:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c401b020 con 0x7f35d00ae830 2024-04-25T08:29:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c40175f0 con 0x7f35d00ae830 2024-04-25T08:29:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.674+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40179e0 con 0x7f35d00ae830 2024-04-25T08:29:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d00af630 con 0x7f35d00ae830 2024-04-25T08:29:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d00afbf0 con 0x7f35d00ae830 2024-04-25T08:29:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f196e7fc640 1 -- 172.21.15.60:0/3270764982 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19881f87a0 con 0x7f1988162d10 2024-04-25T08:29:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f28e3fff640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28dc0342e0 0x7f28dc048f10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28e402d370 tx=0x7f28e40686c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7fa9da0e4640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9b806b5a0 0x7fa9b806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4073280 con 0x7f28dc0342e0 2024-04-25T08:29:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7fa9d98e3640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7fa9b8074ca0 0x7fa9b80770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e408f020 con 0x7f28dc0342e0 2024-04-25T08:29:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.677+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc049a80 con 0x7f28dc0342e0 2024-04-25T08:29:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.677+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9d40a6e50 con 0x7fa9b8074ca0 2024-04-25T08:29:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.677+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e406abe0 con 0x7f28dc0342e0 2024-04-25T08:29:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.677+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc04a040 con 0x7f28dc0342e0 2024-04-25T08:29:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f969dcf0640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7f967c074d20 0x7f967c077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.677+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== osd.2 v2:172.21.15.60:6808/1418327746 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f969000aea0 con 0x7f967c074d20 2024-04-25T08:29:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.677+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5b801a070 con 0x7fd5bc054160 2024-04-25T08:29:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28e406c110 con 0x7f28dc0342e0 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7fd5ab7fe640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b006b5a0 0x7fd5b006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd5b8089970 con 0x7fd5bc054160 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7fd5ab7fe640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7fd5b0074d20 0x7fd5b0077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 --> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5b8004ec0 con 0x7fd5b0074d20 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b808e2a0 con 0x7fd5bc054160 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35c401e020 con 0x7f35d00ae830 2024-04-25T08:29:42.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f87a397d640 1 -- 172.21.15.60:0/3033602951 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f877c001030 con 0x7f879c06c180 2024-04-25T08:29:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7fd5c37fe640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7fd5b0074d20 0x7fd5b0077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f8791ffb640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f878006b5a0 0x7f878006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28dc005380 con 0x7f28dc0342e0 2024-04-25T08:29:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7fa9da0e4640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9b806b5a0 0x7fa9b806da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40099a0 tx=0x7fa9c40090e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9d412c050 con 0x7fa9cc0e7700 2024-04-25T08:29:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== osd.0 v2:172.21.15.60:6817/3630457049 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa9d40a6e50 con 0x7fa9b8074ca0 2024-04-25T08:29:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35d0073a00 con 0x7f35d00ae830 2024-04-25T08:29:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.676+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45e0005470 con 0x7f45e815dbd0 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7f45cf7fe640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45d406b5a0 0x7f45d406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f45e008a2a0 con 0x7f45e815dbd0 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7f45cf7fe640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7f45d4074d20 0x7f45d4077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45e0010e50 con 0x7f45d4074d20 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e0091050 con 0x7f45e815dbd0 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.678+0000 7f45e67fc640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7f45d4074d20 0x7f45d4077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f45e6ffd640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45d406b5a0 0x7f45d406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7fd5c27fc640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b006b5a0 0x7fd5b006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f197c007400 con 0x7f1988162d10 2024-04-25T08:29:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f35c27fc640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f35b406b5d0 0x7f35b406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7fd5c37fe640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7fd5b0074d20 0x7fd5b0077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f879808a260 con 0x7f879c06c180 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f8791ffb640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8780074ca0 0x7f87800770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 --> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8798011800 con 0x7f8780074ca0 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8798091050 con 0x7f879c06c180 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.679+0000 7f87a1f1a640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8780074ca0 0x7f87800770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f35d5637640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f35b406b5d0 0x7f35b406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f87a0f18640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f878006b5a0 0x7f878006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f35c408cfa0 con 0x7f35d00ae830 2024-04-25T08:29:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f1984ff9640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f197406b5a0 0x7f197406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7fd5c27fc640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b006b5a0 0x7fd5b006da60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4034000 tx=0x7fd5b402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f969d4ef640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f967c06b5a0 0x7f967c06da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9694009bd0 tx=0x7f9694009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f197c089940 con 0x7f1988162d10 2024-04-25T08:29:42.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f35c27fc640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f35b4074d50 0x7f35b4077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f1984ff9640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f1974074d20 0x7f1974077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 --> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f197c005470 con 0x7f1974074d20 2024-04-25T08:29:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197c08e2a0 con 0x7f1988162d10 2024-04-25T08:29:42.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f87a1f1a640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8780074ca0 0x7f87800770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f28e17fa640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28c806b5a0 0x7f28c806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f28e8f5d640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28c806b5a0 0x7f28c806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f45e67fc640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7f45d4074d20 0x7f45d4077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35c4007ea0 con 0x7f35b4074d50 2024-04-25T08:29:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.680+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35c4092590 con 0x7f35d00ae830 2024-04-25T08:29:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== osd.6 v2:172.21.15.100:6816/4212964990 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd5b8004ec0 con 0x7fd5b0074d20 2024-04-25T08:29:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f28e40b2060 con 0x7f28dc0342e0 2024-04-25T08:29:42.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f35d6639640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f35b4074d50 0x7f35b4077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f28e17fa640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f28c8074c90 0x7f28c80770d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28e40aee80 con 0x7f28c8074c90 2024-04-25T08:29:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e410b050 con 0x7f28dc0342e0 2024-04-25T08:29:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f35d5637640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f35b406b5d0 0x7f35b406da90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35cc034000 tx=0x7f35cc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f45e6ffd640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45d406b5a0 0x7f45d406da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000f60 tx=0x7f45d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== osd.4 v2:172.21.15.100:6808/2796877898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f45e0010e50 con 0x7f45d4074d20 2024-04-25T08:29:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f1987fff640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f197406b5a0 0x7f197406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f19877fe640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f1974074d20 0x7f1974077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f28e975e640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f28c8074c90 0x7f28c80770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.681+0000 7f28e8f5d640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28c806b5a0 0x7f28c806da60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f28d8034000 tx=0x7f28d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.682+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== osd.5 v2:172.21.15.100:6800/301366941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8798011800 con 0x7f8780074ca0 2024-04-25T08:29:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.682+0000 7f19877fe640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f1974074d20 0x7f1974077160 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.682+0000 7f87a0f18640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f878006b5a0 0x7f878006da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f879c0b5220 tx=0x7f87940015c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.682+0000 7f28e975e640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f28c8074c90 0x7f28c80770d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.682+0000 7f1987fff640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f197406b5a0 0x7f197406da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1980033000 tx=0x7f1980009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.683+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== osd.7 v2:172.21.15.100:6824/1558157189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f197c005470 con 0x7f1974074d20 2024-04-25T08:29:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.682+0000 7f35d6639640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f35b4074d50 0x7f35b4077190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:42.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.683+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== osd.3 v2:172.21.15.60:6816/426387744 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f35c4007ea0 con 0x7f35b4074d50 2024-04-25T08:29:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.686+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== osd.1 v2:172.21.15.60:6800/3720682638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f28e40aee80 con 0x7f28c8074c90 2024-04-25T08:29:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.710+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 --> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f96980cf480 con 0x7f967c074d20 2024-04-25T08:29:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.711+0000 7f196e7fc640 1 -- 172.21.15.60:0/3270764982 --> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1988073990 con 0x7f1974074d20 2024-04-25T08:29:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.712+0000 7f968dffb640 1 -- 172.21.15.60:0/1698852390 <== osd.2 v2:172.21.15.60:6808/1418327746 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f96980cf480 con 0x7f967c074d20 2024-04-25T08:29:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.712+0000 7f1984ff9640 1 -- 172.21.15.60:0/3270764982 <== osd.7 v2:172.21.15.100:6824/1558157189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1988073990 con 0x7f1974074d20 2024-04-25T08:29:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f1974074d20 msgr2=0x7f1974077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198d67b640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f1974074d20 0x7f1974077160 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f197406b5a0 msgr2=0x7f197406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198d67b640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f197406b5a0 0x7f197406da60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1980033000 tx=0x7f1980009000 comp rx=0 tx=0).stop 2024-04-25T08:29:42.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1988162d10 msgr2=0x7f19880cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 --> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5bc0dbd20 con 0x7fd5b0074d20 2024-04-25T08:29:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198d67b640 1 --2- 172.21.15.60:0/3270764982 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1988162d10 0x7f19880cea70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c3f0 tx=0x7f197c00c8b0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.713+0000 7f198ce7a640 1 -- 172.21.15.60:0/3270764982 reap_dead start 2024-04-25T08:29:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.714+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 shutdown_connections 2024-04-25T08:29:42.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.714+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 >> 172.21.15.60:0/3270764982 conn(0x7f1988071ad0 msgr2=0x7f19881581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.714+0000 7fd5ab7fe640 1 -- 172.21.15.60:0/4268572019 <== osd.6 v2:172.21.15.100:6816/4212964990 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd5bc0dbd20 con 0x7fd5b0074d20 2024-04-25T08:29:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.714+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 shutdown_connections 2024-04-25T08:29:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.714+0000 7f198d67b640 1 -- 172.21.15.60:0/3270764982 wait complete. 2024-04-25T08:29:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7fd5b0074d20 msgr2=0x7fd5b0077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c957e640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7fd5b0074d20 0x7fd5b0077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b006b5a0 msgr2=0x7fd5b006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c957e640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b006b5a0 0x7fd5b006da60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4034000 tx=0x7fd5b402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 msgr2=0x7fd5bc0db2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.714+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 --> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9cc005380 con 0x7fa9b8074ca0 2024-04-25T08:29:42.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fa9caffd640 1 -- 172.21.15.60:0/2663550186 <== osd.0 v2:172.21.15.60:6817/3630457049 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa9cc005380 con 0x7fa9b8074ca0 2024-04-25T08:29:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c957e640 1 --2- 172.21.15.60:0/4268572019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5bc054160 0x7fd5bc0db2a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c430 tx=0x7fd5b800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fd5c37fe640 1 -- 172.21.15.60:0/4268572019 reap_dead start 2024-04-25T08:29:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7fa9b8074ca0 msgr2=0x7fa9b80770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7fa9c8ff9640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7fa9b8074ca0 0x7fa9b80770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.715+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 --> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f45e80d1520 con 0x7f45d4074d20 2024-04-25T08:29:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 shutdown_connections 2024-04-25T08:29:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 >> 172.21.15.60:0/4268572019 conn(0x7fd5bc055a10 msgr2=0x7fd5bc038110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9b806b5a0 msgr2=0x7fa9b806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9c8ff9640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9b806b5a0 0x7fa9b806da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40099a0 tx=0x7fa9c40090e0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7f967c074d20 msgr2=0x7f967c077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc0e7700 msgr2=0x7fa9cc04e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969f753640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7f967c074d20 0x7f967c077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9c8ff9640 1 --2- 172.21.15.60:0/2663550186 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9cc0e7700 0x7fa9cc04e130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa9d406a560 tx=0x7fa9d4068960 comp rx=0 tx=0).stop 2024-04-25T08:29:42.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 shutdown_connections 2024-04-25T08:29:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f45cf7fe640 1 -- 172.21.15.60:0/755943599 <== osd.4 v2:172.21.15.100:6808/2796877898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f45e80d1520 con 0x7f45d4074d20 2024-04-25T08:29:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f967c06b5a0 msgr2=0x7f967c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969f753640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f967c06b5a0 0x7f967c06da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9694009bd0 tx=0x7f9694009240 comp rx=0 tx=0).stop 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96980cd2e0 msgr2=0x7f9698147e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969f753640 1 --2- 172.21.15.60:0/1698852390 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96980cd2e0 0x7f9698147e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9690009eb0 tx=0x7f9690004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7f969dcf0640 1 -- 172.21.15.60:0/1698852390 reap_dead start 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9da8e5640 1 -- 172.21.15.60:0/2663550186 reap_dead start 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 shutdown_connections 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.716+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 >> 172.21.15.60:0/2663550186 conn(0x7fa9cc058ed0 msgr2=0x7fa9cc0592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7fd5c957e640 1 -- 172.21.15.60:0/4268572019 wait complete. 2024-04-25T08:29:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 shutdown_connections 2024-04-25T08:29:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 >> 172.21.15.60:0/1698852390 conn(0x7f969806abe0 msgr2=0x7f96980b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 shutdown_connections 2024-04-25T08:29:42.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7fa9c8ff9640 1 -- 172.21.15.60:0/2663550186 wait complete. 2024-04-25T08:29:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f87a397d640 1 -- 172.21.15.60:0/3033602951 --> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f877c002d50 con 0x7f8780074ca0 2024-04-25T08:29:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 shutdown_connections 2024-04-25T08:29:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7f45d4074d20 msgr2=0x7f45d4077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f45e7fff640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7f45d4074d20 0x7f45d4077160 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f969f753640 1 -- 172.21.15.60:0/1698852390 wait complete. 2024-04-25T08:29:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45d406b5a0 msgr2=0x7f45d406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f45e7fff640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45d406b5a0 0x7f45d406da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000f60 tx=0x7f45d8009040 comp rx=0 tx=0).stop 2024-04-25T08:29:42.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e815dbd0 msgr2=0x7f45e80c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.717+0000 7f45e7fff640 1 --2- 172.21.15.60:0/755943599 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e815dbd0 0x7f45e80c8e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45e000c730 tx=0x7f45e000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f45e77fe640 1 -- 172.21.15.60:0/755943599 reap_dead start 2024-04-25T08:29:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 shutdown_connections 2024-04-25T08:29:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 >> 172.21.15.60:0/755943599 conn(0x7f45e80d03d0 msgr2=0x7f45e80b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f8791ffb640 1 -- 172.21.15.60:0/3033602951 <== osd.5 v2:172.21.15.100:6800/301366941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f877c002d50 con 0x7f8780074ca0 2024-04-25T08:29:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 shutdown_connections 2024-04-25T08:29:42.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f45e7fff640 1 -- 172.21.15.60:0/755943599 wait complete. 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8780074ca0 msgr2=0x7f87800770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.718+0000 7f87877fe640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8780074ca0 0x7f87800770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f878006b5a0 msgr2=0x7f878006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f878006b5a0 0x7f878006da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f879c0b5220 tx=0x7f87940015c0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f879c06c180 msgr2=0x7f879c152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 --2- 172.21.15.60:0/3033602951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f879c06c180 0x7f879c152e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f879800c5e0 tx=0x7f879800caa0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87a1f1a640 1 -- 172.21.15.60:0/3033602951 reap_dead start 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 shutdown_connections 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 >> 172.21.15.60:0/3033602951 conn(0x7f879c0d3a20 msgr2=0x7f879c0ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.719+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 shutdown_connections 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.721+0000 7f87877fe640 1 -- 172.21.15.60:0/3033602951 wait complete. 2024-04-25T08:29:42.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.723+0000 7f28e9f5f640 1 -- 172.21.15.60:0/3687748967 --> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28dc04ada0 con 0x7f28c8074c90 2024-04-25T08:29:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.724+0000 7f28e17fa640 1 -- 172.21.15.60:0/3687748967 <== osd.1 v2:172.21.15.60:6800/3720682638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f28dc04ada0 con 0x7f28c8074c90 2024-04-25T08:29:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f28c8074c90 msgr2=0x7f28c80770d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28ceffd640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f28c8074c90 0x7f28c80770d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28c806b5a0 msgr2=0x7f28c806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28ceffd640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28c806b5a0 0x7f28c806da60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f28d8034000 tx=0x7f28d802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28dc0342e0 msgr2=0x7f28dc048f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28ceffd640 1 --2- 172.21.15.60:0/3687748967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28dc0342e0 0x7f28dc048f10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28e402d370 tx=0x7f28e40686c0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.725+0000 7f28e975e640 1 -- 172.21.15.60:0/3687748967 reap_dead start 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.726+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 shutdown_connections 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.726+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 >> 172.21.15.60:0/3687748967 conn(0x7f28dc01e6b0 msgr2=0x7f28dc036e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.726+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 shutdown_connections 2024-04-25T08:29:42.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.726+0000 7f28ceffd640 1 -- 172.21.15.60:0/3687748967 wait complete. 2024-04-25T08:29:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.733+0000 7f35d809c640 1 -- 172.21.15.60:0/4216449056 --> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35d00a88a0 con 0x7f35b4074d50 2024-04-25T08:29:42.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.734+0000 7f35c27fc640 1 -- 172.21.15.60:0/4216449056 <== osd.3 v2:172.21.15.60:6816/426387744 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35d00a88a0 con 0x7f35b4074d50 2024-04-25T08:29:42.772 INFO:teuthology.orchestra.run.smithi060.stdout:60129542154 2024-04-25T08:29:42.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:29:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.736+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f35b4074d50 msgr2=0x7f35b4077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.736+0000 7f35b3fff640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f35b4074d50 0x7f35b4077190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.736+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f35b406b5d0 msgr2=0x7f35b406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.736+0000 7f35b3fff640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f35b406b5d0 0x7f35b406da90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35cc034000 tx=0x7f35cc02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.737+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00ae830 msgr2=0x7f35d00aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.737+0000 7f35b3fff640 1 --2- 172.21.15.60:0/4216449056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35d00ae830 0x7f35d00aecc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35c4010fc0 tx=0x7f35c400e490 comp rx=0 tx=0).stop 2024-04-25T08:29:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.737+0000 7f35d6639640 1 -- 172.21.15.60:0/4216449056 reap_dead start 2024-04-25T08:29:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.737+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 shutdown_connections 2024-04-25T08:29:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.737+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 >> 172.21.15.60:0/4216449056 conn(0x7f35d006abe0 msgr2=0x7f35d0159270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.738+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 shutdown_connections 2024-04-25T08:29:42.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.738+0000 7f35b3fff640 1 -- 172.21.15.60:0/4216449056 wait complete. 2024-04-25T08:29:42.776 INFO:teuthology.orchestra.run.smithi060.stdout:55834574857 2024-04-25T08:29:42.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:29:42.776 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:42.777 INFO:teuthology.orchestra.run.smithi060.stdout:60129542154 2024-04-25T08:29:42.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:29:42.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:29:42.778 INFO:teuthology.orchestra.run.smithi060.stdout:55834574857 2024-04-25T08:29:42.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:29:42.779 INFO:teuthology.orchestra.run.smithi060.stdout:60129542154 2024-04-25T08:29:42.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:29:42.780 INFO:teuthology.orchestra.run.smithi060.stdout:55834574857 2024-04-25T08:29:42.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:29:42.781 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:42.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:29:42.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.992+0000 7ff4df66f640 1 Processor -- start 2024-04-25T08:29:42.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.992+0000 7ff4df66f640 1 -- start start 2024-04-25T08:29:42.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.993+0000 7fb91524d640 1 Processor -- start 2024-04-25T08:29:42.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.993+0000 7fb91524d640 1 -- start start 2024-04-25T08:29:42.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.993+0000 7fbaa1330640 1 Processor -- start 2024-04-25T08:29:42.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4df66f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4d0053e50 0x7ff4d00542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4df66f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d004f930 0x7ff4d004fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4df66f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d0005ec0 con 0x7ff4d0048100 2024-04-25T08:29:42.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4df66f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d00e7b90 con 0x7ff4d0053e50 2024-04-25T08:29:42.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4df66f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0006a90 con 0x7ff4d004f930 2024-04-25T08:29:42.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7fd19f3d5640 1 Processor -- start 2024-04-25T08:29:42.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.993+0000 7fbaa1330640 1 -- start start 2024-04-25T08:29:42.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4dcc0a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4d0053e50 0x7ff4d00542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7fb91524d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9080e1200 0x7fb908036120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7f5a723b0640 1 Processor -- start 2024-04-25T08:29:42.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7f5a723b0640 1 -- start start 2024-04-25T08:29:42.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7fb91524d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9080545e0 con 0x7fb908051d90 2024-04-25T08:29:42.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7fb90e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb908051d90 0x7fb908038740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42034/0 (socket says 172.21.15.60:42034) 2024-04-25T08:29:42.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7fd19f3d5640 1 -- start start 2024-04-25T08:29:42.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7fb90e59c640 1 -- 172.21.15.60:0/3211424967 learned_addr learned my addr 172.21.15.60:0/3211424967 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:42.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fcd22b63640 1 Processor -- start 2024-04-25T08:29:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4dd40b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff4d0048100 0x7ff4d0006f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43676/0 (socket says 172.21.15.60:43676) 2024-04-25T08:29:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.994+0000 7ff4dd40b640 1 -- 172.21.15.60:0/1922965218 learned_addr learned my addr 172.21.15.60:0/1922965218 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7ff4ddc0c640 1 --2- 172.21.15.60:0/1922965218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d004f930 0x7ff4d004fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7ff4dcc0a640 1 -- 172.21.15.60:0/1922965218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d004f930 msgr2=0x7ff4d004fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7ff4dcc0a640 1 --2- 172.21.15.60:0/1922965218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d004f930 0x7ff4d004fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7ff4dcc0a640 1 -- 172.21.15.60:0/1922965218 >> v1:172.21.15.100:6789/0 conn(0x7ff4d0048100 legacy=0x7ff4d0006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7ff4dcc0a640 1 -- 172.21.15.60:0/1922965218 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0053280 con 0x7ff4d0053e50 2024-04-25T08:29:42.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fcd22b63640 1 -- start start 2024-04-25T08:29:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fb90ed9d640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9080e1200 0x7fb908036120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008824531 0 0) 0x7fb9080545e0 con 0x7fb908051d90 2024-04-25T08:29:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb908039310 con 0x7fb908038e50 2024-04-25T08:29:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080395f0 con 0x7fb9080e1200 2024-04-25T08:29:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19f3d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd19806c180 0x7fd1980cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:42.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19f3d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1980d15f0 0x7fd19806c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.995+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8f8003520 con 0x7fb908051d90 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19f3d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19806caa0 con 0x7fd1980b2bb0 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19f3d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1980d2a00 con 0x7fd19806c180 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19f3d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1980d2ce0 con 0x7fd1980d15f0 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19d972640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd1980b2bb0 0x7fd1980ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43694/0 (socket says 172.21.15.60:43694) 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19d972640 1 -- 172.21.15.60:0/1432172275 learned_addr learned my addr 172.21.15.60:0/1432172275 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:43.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19c970640 1 --2- 172.21.15.60:0/1432172275 >> v2:172.21.15.100:3300/0 conn(0x7fd1980d15f0 0x7fd19806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3532853284 0 0) 0x7fb8f8003520 con 0x7fb908051d90 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9080e1200 msgr2=0x7fb908036120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9080e1200 0x7fb908036120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 >> v1:172.21.15.100:6789/0 conn(0x7fb908038e50 legacy=0x7fb908039ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080515b0 con 0x7fb908051d90 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb904001670 con 0x7fb908051d90 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 >> v1:172.21.15.60:6789/0 conn(0x7fb908051d90 legacy=0x7fb908038740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8f80039e0 0x7fb8f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f8004800 0x7fb8f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb8f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080395f0 con 0x7fb8f8004800 2024-04-25T08:29:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f800c090 con 0x7fb8f80051c0 2024-04-25T08:29:43.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f800c290 con 0x7fb8f80039e0 2024-04-25T08:29:43.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fbaa1330640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba94054860 0x7fba94054ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd18ffff640 1 -- 172.21.15.60:0/1432172275 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209464987 0 0) 0x7fd19806caa0 con 0x7fd1980b2bb0 2024-04-25T08:29:43.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd18ffff640 1 -- 172.21.15.60:0/1432172275 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd184003520 con 0x7fd1980b2bb0 2024-04-25T08:29:43.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fb90f59e640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb8f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fbaa1330640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba9403c320 0x7fba9403c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fbaa1330640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba940e7df0 con 0x7fba940e7a10 2024-04-25T08:29:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fbaa1330640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9403b560 con 0x7fba94054860 2024-04-25T08:29:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fbaa1330640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9403b840 con 0x7fba9403c320 2024-04-25T08:29:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fba9affd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fba940e7a10 0x7fba94049b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43710/0 (socket says 172.21.15.60:43710) 2024-04-25T08:29:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fba9affd640 1 -- 172.21.15.60:0/1046025269 learned_addr learned my addr 172.21.15.60:0/1046025269 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:43.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7f5a723b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6c15dbb0 0x7f5a6c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90e59c640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f8004800 0x7fb8f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a723b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cbf00 0x7f5a6c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a723b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c0cae60 con 0x7f5a6c0cc360 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a723b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c0cb140 con 0x7f5a6c15dbb0 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a723b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0cb420 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fba9a7fc640 1 --2- 172.21.15.60:0/1046025269 >> v2:172.21.15.100:3300/0 conn(0x7fba94054860 0x7fba94054ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fba9b7fe640 1 --2- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba9403c320 0x7fba9403c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.996+0000 7fd19d171640 1 --2- 172.21.15.60:0/1432172275 >> v2:172.21.15.60:3300/0 conn(0x7fd19806c180 0x7fd1980cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fd18ffff640 1 -- 172.21.15.60:0/1432172275 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112726695 0 0) 0x7fd184003520 con 0x7fd1980b2bb0 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fd19c970640 1 -- 172.21.15.60:0/1432172275 >> v2:172.21.15.60:3300/0 conn(0x7fd19806c180 msgr2=0x7fd1980cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fd19c970640 1 --2- 172.21.15.60:0/1432172275 >> v2:172.21.15.60:3300/0 conn(0x7fd19806c180 0x7fd1980cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fd19c970640 1 -- 172.21.15.60:0/1432172275 >> v1:172.21.15.100:6789/0 conn(0x7fd1980b2bb0 legacy=0x7fd1980ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fd19c970640 1 -- 172.21.15.60:0/1432172275 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980d2fc0 con 0x7fd1980d15f0 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd22b63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd22b63640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcd1c144d50 0x7fcd1c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd22b63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c0cccc0 con 0x7fcd1c1452a0 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd22b63640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c0b2b20 con 0x7fcd1c144d50 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd22b63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c06d400 con 0x7fcd1c1446d0 2024-04-25T08:29:43.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd208ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd1bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcd1c144d50 0x7fcd1c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd208ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58992/0 (socket says 172.21.15.60:58992) 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd208ff640 1 -- 172.21.15.60:0/8994559 learned_addr learned my addr 172.21.15.60:0/8994559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fcd21100640 1 --1- 172.21.15.60:0/8994559 >> v1:172.21.15.100:6789/0 conn(0x7fcd1c1452a0 0x7fcd1c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43722/0 (socket says 172.21.15.60:43722) 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90ed9d640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8f80039e0 0x7fb8f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90f59e640 1 -- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8f80039e0 msgr2=0x7fb8f8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90f59e640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8f80039e0 0x7fb8f8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90f59e640 1 -- 172.21.15.60:0/3211424967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f8004800 msgr2=0x7fb8f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90f59e640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8f8004800 0x7fb8f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fb90f59e640 1 -- 172.21.15.60:0/3211424967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080515b0 con 0x7fb8f80051c0 2024-04-25T08:29:43.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fba99ffb640 1 -- 172.21.15.60:0/1046025269 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711968374 0 0) 0x7fba940e7df0 con 0x7fba940e7a10 2024-04-25T08:29:43.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7fba99ffb640 1 -- 172.21.15.60:0/1046025269 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba84003520 con 0x7fba940e7a10 2024-04-25T08:29:43.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7ff4dcc0a640 1 --2- 172.21.15.60:0/1922965218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0053e50 0x7ff4d00542e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006c10 tx=0x7ff4d402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344ff2ca885a86fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5e7ae6f640 1 Processor -- start 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cbf00 0x7f5a6c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cbf00 0x7f5a6c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58978/0 (socket says 172.21.15.60:58978) 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.997+0000 7f5a6b7fe640 1 -- 172.21.15.60:0/2150031442 learned_addr learned my addr 172.21.15.60:0/2150031442 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7ff4cffff640 1 -- 172.21.15.60:0/1922965218 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4037070 con 0x7ff4d0053e50 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7ff4cffff640 1 -- 172.21.15.60:0/1922965218 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d402ad40 con 0x7ff4d0053e50 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7ff4cffff640 1 -- 172.21.15.60:0/1922965218 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4032660 con 0x7ff4d0053e50 2024-04-25T08:29:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/2150031442 >> v2:172.21.15.60:3300/0 conn(0x7f5a6c15dbb0 0x7f5a6c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fba9a7fc640 1 -- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba9403c320 msgr2=0x7fba9403c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fba9a7fc640 1 --2- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba9403c320 0x7fba9403c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fba9a7fc640 1 -- 172.21.15.60:0/1046025269 >> v1:172.21.15.100:6789/0 conn(0x7fba940e7a10 legacy=0x7fba94049b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fba9a7fc640 1 -- 172.21.15.60:0/1046025269 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9403bb20 con 0x7fba94054860 2024-04-25T08:29:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fba99ffb640 1 -- 172.21.15.60:0/1046025269 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102003409 0 0) 0x7fba84003520 con 0x7fba940e7a10 2024-04-25T08:29:43.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7f5e7ae6f640 1 -- start start 2024-04-25T08:29:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fcd208ff640 1 -- 172.21.15.60:0/8994559 >> v2:172.21.15.100:3300/0 conn(0x7fcd1c144d50 msgr2=0x7fcd1c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fcd208ff640 1 --2- 172.21.15.60:0/8994559 >> v2:172.21.15.100:3300/0 conn(0x7fcd1c144d50 0x7fcd1c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fcd208ff640 1 -- 172.21.15.60:0/8994559 >> v1:172.21.15.100:6789/0 conn(0x7fcd1c1452a0 legacy=0x7fcd1c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7f5a6bfff640 1 -- 172.21.15.60:0/2150031442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cbf00 msgr2=0x7f5a6c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fb90f59e640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb8f800b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9100692f0 tx=0x7fb8f8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb91b49647aef20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fcd208ff640 1 -- 172.21.15.60:0/8994559 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0cc140 con 0x7fcd1c1446d0 2024-04-25T08:29:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/2150031442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cbf00 0x7f5a6c06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fcd1bfff640 1 --2- 172.21.15.60:0/8994559 >> v2:172.21.15.100:3300/0 conn(0x7fcd1c144d50 0x7fcd1c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:43.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7f5a6bfff640 1 -- 172.21.15.60:0/2150031442 >> v1:172.21.15.60:6789/0 conn(0x7f5a6c0cc360 legacy=0x7f5a6c06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100b2070 con 0x7fb8f80051c0 2024-04-25T08:29:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.998+0000 7f5a6bfff640 1 -- 172.21.15.60:0/2150031442 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0cb700 con 0x7f5a6c15dbb0 2024-04-25T08:29:43.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9100a5ab0 con 0x7fb8f80051c0 2024-04-25T08:29:43.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fb90dd9b640 1 -- 172.21.15.60:0/3211424967 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91004f490 con 0x7fb8f80051c0 2024-04-25T08:29:43.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fba9a7fc640 1 --2- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba94054860 0x7fba94054ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02d370 tx=0x7fba9c0a5080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae25f679c432fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7ff4df66f640 1 -- 172.21.15.60:0/1922965218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0053e50 msgr2=0x7ff4d00542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1922965218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0053e50 0x7ff4d00542e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006c10 tx=0x7ff4d402a0b0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fba99ffb640 1 -- 172.21.15.60:0/1046025269 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0b2070 con 0x7fba94054860 2024-04-25T08:29:43.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fba99ffb640 1 -- 172.21.15.60:0/1046025269 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c0a5bd0 con 0x7fba94054860 2024-04-25T08:29:43.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fba99ffb640 1 -- 172.21.15.60:0/1046025269 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90001670 con 0x7fba94054860 2024-04-25T08:29:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7ff4df66f640 1 -- 172.21.15.60:0/1922965218 shutdown_connections 2024-04-25T08:29:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1922965218 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0053e50 0x7ff4d00542e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1922965218 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d004f930 0x7ff4d004fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7ff4df66f640 1 -- 172.21.15.60:0/1922965218 >> 172.21.15.60:0/1922965218 conn(0x7ff4d0009fe0 msgr2=0x7ff4d000a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7fcd208ff640 1 --2- 172.21.15.60:0/8994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0b3f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009e50 tx=0x7fcd1402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71773feed23184b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7f5e7ae6f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e6c0eacf0 0x7f5e6c04fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:42.999+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/2150031442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c06fea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5a600097a0 tx=0x7f5a6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdd7a162380954e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 msgr2=0x7fb8f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb91524d640 1 --2- 172.21.15.60:0/3211424967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb8f800b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9100692f0 tx=0x7fb8f8005830 comp rx=0 tx=0).stop 2024-04-25T08:29:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7ff4df66f640 1 -- 172.21.15.60:0/1922965218 shutdown_connections 2024-04-25T08:29:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fcd1b7fe640 1 -- 172.21.15.60:0/8994559 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14036070 con 0x7fcd1c1446d0 2024-04-25T08:29:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fcd1b7fe640 1 -- 172.21.15.60:0/8994559 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd14031600 con 0x7fcd1c1446d0 2024-04-25T08:29:43.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5e78c0b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e6c0eacf0 0x7f5e6c04fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb90f59e640 1 -- 172.21.15.60:0/3211424967 reap_dead start 2024-04-25T08:29:43.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5a6affd640 1 -- 172.21.15.60:0/2150031442 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60037070 con 0x7f5a6c15dbb0 2024-04-25T08:29:43.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5e78c0b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e6c0eacf0 0x7f5e6c04fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33316/0 (socket says 172.21.15.60:33316) 2024-04-25T08:29:43.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fc6d9009640 1 Processor -- start 2024-04-25T08:29:43.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fcd1b7fe640 1 -- 172.21.15.60:0/8994559 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14036070 con 0x7fcd1c1446d0 2024-04-25T08:29:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5a6affd640 1 -- 172.21.15.60:0/2150031442 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60032440 con 0x7f5a6c15dbb0 2024-04-25T08:29:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5e73fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e6c04ffc0 0x7f5e6c037bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42050/0 (socket says 172.21.15.60:42050) 2024-04-25T08:29:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 shutdown_connections 2024-04-25T08:29:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 >> 172.21.15.60:0/3211424967 conn(0x7fb908053740 msgr2=0x7fb908053b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5a6affd640 1 -- 172.21.15.60:0/2150031442 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60032810 con 0x7f5a6c15dbb0 2024-04-25T08:29:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 -- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba94054860 msgr2=0x7fba94054ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5e7ae6f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c053710 con 0x7f5e6c04ffc0 2024-04-25T08:29:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5e78c0b640 1 -- 172.21.15.60:0/2664456016 learned_addr learned my addr 172.21.15.60:0/2664456016 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 --2- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba94054860 0x7fba94054ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02d370 tx=0x7fba9c0a5080 comp rx=0 tx=0).stop 2024-04-25T08:29:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 shutdown_connections 2024-04-25T08:29:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fb91524d640 1 -- 172.21.15.60:0/3211424967 wait complete. 2024-04-25T08:29:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fcd22b63640 1 -- 172.21.15.60:0/8994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 msgr2=0x7fcd1c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fcd22b63640 1 --2- 172.21.15.60:0/8994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0b3f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009e50 tx=0x7fcd1402a710 comp rx=0 tx=0).stop 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 -- 172.21.15.60:0/1046025269 shutdown_connections 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 --2- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba94054860 0x7fba94054ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 --2- 172.21.15.60:0/1046025269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba9403c320 0x7fba9403c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 -- 172.21.15.60:0/1046025269 >> 172.21.15.60:0/1046025269 conn(0x7fba94056c90 msgr2=0x7fba940570a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 -- 172.21.15.60:0/1046025269 shutdown_connections 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fbaa1330640 1 -- 172.21.15.60:0/1046025269 wait complete. 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c0539f0 con 0x7f5e6c006e90 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6c054320 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7ff4df66f640 1 -- 172.21.15.60:0/1922965218 wait complete. 2024-04-25T08:29:43.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fcd22b63640 1 -- 172.21.15.60:0/8994559 shutdown_connections 2024-04-25T08:29:43.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fcd22b63640 1 --2- 172.21.15.60:0/8994559 >> v2:172.21.15.100:3300/0 conn(0x7fcd1c144d50 0x7fcd1c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fcd22b63640 1 --2- 172.21.15.60:0/8994559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0b3f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fcd22b63640 1 -- 172.21.15.60:0/8994559 >> 172.21.15.60:0/8994559 conn(0x7fcd1c06abe0 msgr2=0x7fcd1c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fd19c970640 1 --2- 172.21.15.60:0/1432172275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980d15f0 0x7fd19806c560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd19808c2e0 tx=0x7fd19402a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7af5dd9d5e79ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fcd22b63640 1 -- 172.21.15.60:0/8994559 shutdown_connections 2024-04-25T08:29:43.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.000+0000 7fc6d9009640 1 -- start start 2024-04-25T08:29:43.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fcd22b63640 1 -- 172.21.15.60:0/8994559 wait complete. 2024-04-25T08:29:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7f5a723b0640 1 -- 172.21.15.60:0/2150031442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 msgr2=0x7f5a6c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7f5a723b0640 1 --2- 172.21.15.60:0/2150031442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c06fea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5a600097a0 tx=0x7f5a6002a180 comp rx=0 tx=0).stop 2024-04-25T08:29:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fd18ffff640 1 -- 172.21.15.60:0/1432172275 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194037070 con 0x7fd1980d15f0 2024-04-25T08:29:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fd18ffff640 1 -- 172.21.15.60:0/1432172275 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19402aaa0 con 0x7fd1980d15f0 2024-04-25T08:29:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7f5e737fe640 1 -- 172.21.15.60:0/2664456016 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847687321 0 0) 0x7f5e6c053710 con 0x7f5e6c04ffc0 2024-04-25T08:29:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7f5e737fe640 1 -- 172.21.15.60:0/2664456016 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e64003520 con 0x7f5e6c04ffc0 2024-04-25T08:29:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.001+0000 7fc6d9009640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc6d415c7c0 0x7fc6d40cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5e78c0b640 1 -- 172.21.15.60:0/2664456016 >> v1:172.21.15.100:6789/0 conn(0x7f5e6c006e90 legacy=0x7f5e6c0382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7ff4df66f640 1 Processor -- start 2024-04-25T08:29:43.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5a723b0640 1 -- 172.21.15.60:0/2150031442 shutdown_connections 2024-04-25T08:29:43.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5a723b0640 1 --2- 172.21.15.60:0/2150031442 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cbf00 0x7f5a6c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5a723b0640 1 --2- 172.21.15.60:0/2150031442 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c06fea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5a723b0640 1 -- 172.21.15.60:0/2150031442 >> 172.21.15.60:0/2150031442 conn(0x7f5a6c071af0 msgr2=0x7f5a6c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7ff4df66f640 1 -- start start 2024-04-25T08:29:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5a723b0640 1 -- 172.21.15.60:0/2150031442 shutdown_connections 2024-04-25T08:29:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5a723b0640 1 -- 172.21.15.60:0/2150031442 wait complete. 2024-04-25T08:29:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7fd19f3d5640 1 -- 172.21.15.60:0/1432172275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980d15f0 msgr2=0x7fd19806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/1432172275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980d15f0 0x7fd19806c560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd19808c2e0 tx=0x7fd19402a010 comp rx=0 tx=0).stop 2024-04-25T08:29:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5e78c0b640 1 -- 172.21.15.60:0/2664456016 >> v1:172.21.15.60:6789/0 conn(0x7f5e6c04ffc0 legacy=0x7f5e6c037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7f5e78c0b640 1 -- 172.21.15.60:0/2664456016 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c053cd0 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7fcd22b63640 1 Processor -- start 2024-04-25T08:29:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fcd22b63640 1 -- start start 2024-04-25T08:29:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7fc6d3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc6d40cf6d0 0x7fc6d40cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42052/0 (socket says 172.21.15.60:42052) 2024-04-25T08:29:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.002+0000 7fc6d9009640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d4099db0 con 0x7fc6d40cf6d0 2024-04-25T08:29:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fc6d3fff640 1 -- 172.21.15.60:0/1103707134 learned_addr learned my addr 172.21.15.60:0/1103707134 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fbaa1330640 1 Processor -- start 2024-04-25T08:29:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fbaa1330640 1 -- start start 2024-04-25T08:29:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fb91524d640 1 Processor -- start 2024-04-25T08:29:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fd19f3d5640 1 -- 172.21.15.60:0/1432172275 shutdown_connections 2024-04-25T08:29:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/1432172275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1980d15f0 0x7fd19806c560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/1432172275 >> v2:172.21.15.60:3300/0 conn(0x7fd19806c180 0x7fd1980cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fd19f3d5640 1 -- 172.21.15.60:0/1432172275 >> 172.21.15.60:0/1432172275 conn(0x7fd1980d3a20 msgr2=0x7fd1980d3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fb91524d640 1 -- start start 2024-04-25T08:29:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d409a090 con 0x7fc6d40cc710 2024-04-25T08:29:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7f5e78c0b640 1 --2- 172.21.15.60:0/2664456016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c04fa80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e680097f0 tx=0x7f5e6802a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35ca0f84725a0446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7ff4df66f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 0x7ff4d004d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/1103707134 >> v2:172.21.15.60:3300/0 conn(0x7fc6d415c7c0 0x7fc6d40cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821936376 0 0) 0x7fc6d4099db0 con 0x7fc6d40cf6d0 2024-04-25T08:29:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d409a370 con 0x7fc6d415c7c0 2024-04-25T08:29:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fd19f3d5640 1 -- 172.21.15.60:0/1432172275 shutdown_connections 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fd19f3d5640 1 -- 172.21.15.60:0/1432172275 wait complete. 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd22b63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd22b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c144d50 0x7fcd1c16a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd22b63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c1452a0 0x7fcd1c16aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd22b63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0cc700 con 0x7fcd1c1452a0 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd22b63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0b49a0 con 0x7fcd1c144d50 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd22b63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0cc9e0 con 0x7fcd1c1446d0 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd208ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd208ff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0aaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59010/0 (socket says 172.21.15.60:59010) 2024-04-25T08:29:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fcd208ff640 1 -- 172.21.15.60:0/4070752644 learned_addr learned my addr 172.21.15.60:0/4070752644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fb91524d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb908177b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb91524d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080da8f0 0x7fb90817a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fbaa1330640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 0x7fba940d5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7f5e737fe640 1 -- 172.21.15.60:0/2664456016 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68037070 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7f5e737fe640 1 -- 172.21.15.60:0/2664456016 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6802ae50 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7f5e737fe640 1 -- 172.21.15.60:0/2664456016 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68032660 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7f5a723b0640 1 Processor -- start 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4dd40b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 0x7ff4d004d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4dd40b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 0x7ff4d004d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41476/0 (socket says 172.21.15.60:41476) 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4dd40b640 1 -- 172.21.15.60:0/1680733394 learned_addr learned my addr 172.21.15.60:0/1680733394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d004f930 0x7ff4d00da0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d0053e50 0x7ff4d00e9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0006440 con 0x7ff4d0048100 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.003+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc003520 con 0x7fc6d40cf6d0 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2914116683 0 0) 0x7fc6bc003520 con 0x7fc6d40cf6d0 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 >> v2:172.21.15.60:3300/0 conn(0x7fc6d415c7c0 msgr2=0x7fc6d40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 --2- 172.21.15.60:0/1103707134 >> v2:172.21.15.60:3300/0 conn(0x7fc6d415c7c0 0x7fc6d40cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 >> v1:172.21.15.100:6789/0 conn(0x7fc6d40cc710 legacy=0x7fc6d40ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d409a650 con 0x7fc6d40cf6d0 2024-04-25T08:29:43.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fc6c4001670 con 0x7fc6d40cf6d0 2024-04-25T08:29:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 >> v1:172.21.15.60:6789/0 conn(0x7fc6d40cf6d0 legacy=0x7fc6d40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0039b0 0x7fc6bc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0052e0 0x7fc6bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d409a370 con 0x7fc6bc0052e0 2024-04-25T08:29:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc00c1e0 con 0x7fc6bc0039b0 2024-04-25T08:29:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc00c3e0 con 0x7fc6bc004800 2024-04-25T08:29:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7fc6d3fff640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0039b0 0x7fc6bc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd1bfff640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c144d50 0x7fcd1c16a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb90ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb908177b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fbaa1330640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba94054860 0x7fba940e3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c0eacf0 msgr2=0x7f5e6c04fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2664456016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c04fa80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e680097f0 tx=0x7f5e6802a140 comp rx=0 tx=0).stop 2024-04-25T08:29:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb90ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb908177b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33348/0 (socket says 172.21.15.60:33348) 2024-04-25T08:29:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fba9affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 0x7fba940d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fba9affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 0x7fba940d5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33362/0 (socket says 172.21.15.60:33362) 2024-04-25T08:29:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb90ed9d640 1 -- 172.21.15.60:0/1301314312 learned_addr learned my addr 172.21.15.60:0/1301314312 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fba9a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba94054860 0x7fba940e3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7f5a723b0640 1 -- start start 2024-04-25T08:29:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb91524d640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90817aaf0 0x7fb908036760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fba9a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba94054860 0x7fba940e3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41512/0 (socket says 172.21.15.60:41512) 2024-04-25T08:29:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fba9affd640 1 -- 172.21.15.60:0/4022793865 learned_addr learned my addr 172.21.15.60:0/4022793865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba940e7a10 0x7fba940e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd21100640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c1452a0 0x7fcd1c16aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080564e0 con 0x7fb9080da8f0 2024-04-25T08:29:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb908054260 con 0x7fb8f80051c0 2024-04-25T08:29:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb908056660 con 0x7fb90817aaf0 2024-04-25T08:29:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb90f59e640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90817aaf0 0x7fb908036760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fb90e59c640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080da8f0 0x7fb90817a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9404a230 con 0x7fba94054860 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fd19f3d5640 1 Processor -- start 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd208ff640 1 -- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c144d50 msgr2=0x7fcd1c16a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd208ff640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c144d50 0x7fcd1c16a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd208ff640 1 -- 172.21.15.60:0/4070752644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c1452a0 msgr2=0x7fcd1c16aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd208ff640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c1452a0 0x7fcd1c16aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd208ff640 1 -- 172.21.15.60:0/4070752644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0adf10 con 0x7fcd1c1446d0 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fcd21100640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c1452a0 0x7fcd1c16aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 shutdown_connections 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2664456016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c04fa80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 >> 172.21.15.60:0/2664456016 conn(0x7f5e6c009fa0 msgr2=0x7f5e6c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fba9b7fe640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba940e7a10 0x7fba940e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9404a570 con 0x7fba9403c320 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba94039c70 con 0x7fba940e7a10 2024-04-25T08:29:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fd19f3d5640 1 -- start start 2024-04-25T08:29:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d8808640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0052e0 0x7fc6bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d37fe640 1 -- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0039b0 msgr2=0x7fc6bc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6bc0039b0 0x7fc6bc003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d37fe640 1 -- 172.21.15.60:0/1103707134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0052e0 msgr2=0x7fc6bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0052e0 0x7fc6bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.005+0000 7fc6d37fe640 1 -- 172.21.15.60:0/1103707134 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d409a650 con 0x7fc6bc004800 2024-04-25T08:29:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d00502a0 con 0x7ff4d004f930 2024-04-25T08:29:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0006720 con 0x7ff4d0053e50 2024-04-25T08:29:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.004+0000 7ff4dcc0a640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d004f930 0x7ff4d00da0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fc6d8808640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6bc0052e0 0x7fc6bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fc6d8808640 1 -- 172.21.15.60:0/1103707134 reap_dead start 2024-04-25T08:29:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4ddc0c640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d0053e50 0x7ff4d00e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fcd208ff640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0aaf40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd14034040 tx=0x7fcd14004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90f59e640 1 -- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 msgr2=0x7fb908177b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90f59e640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb908177b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90f59e640 1 -- 172.21.15.60:0/1301314312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080da8f0 msgr2=0x7fb90817a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90f59e640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080da8f0 0x7fb90817a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90f59e640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080d66d0 con 0x7fb90817aaf0 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90e59c640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080da8f0 0x7fb90817a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 shutdown_connections 2024-04-25T08:29:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb90f59e640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90817aaf0 0x7fb908036760 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb91002d370 tx=0x7fb910068960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fba9affd640 1 -- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba940e7a10 msgr2=0x7fba940e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fba9affd640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba940e7a10 0x7fba940e35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fba9affd640 1 -- 172.21.15.60:0/4022793865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba94054860 msgr2=0x7fba940e3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fba9affd640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba94054860 0x7fba940e3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fba9affd640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940d5650 con 0x7fba9403c320 2024-04-25T08:29:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1403f070 con 0x7fcd1c1446d0 2024-04-25T08:29:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd140079b0 con 0x7fcd1c1446d0 2024-04-25T08:29:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14036070 con 0x7fcd1c1446d0 2024-04-25T08:29:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100bf070 con 0x7fb90817aaf0 2024-04-25T08:29:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7f5a723b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 0x7f5a6c147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fcd22b63640 1 -- 172.21.15.60:0/4070752644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c0ae1f0 con 0x7fcd1c1446d0 2024-04-25T08:29:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9100b3040 con 0x7fb90817aaf0 2024-04-25T08:29:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a723b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cc360 0x7f5a6c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9100b2050 con 0x7fb90817aaf0 2024-04-25T08:29:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a723b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c1f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 0x7f5a6c147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2664456016 wait complete. 2024-04-25T08:29:43.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fba9affd640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 0x7fba940d5110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba90001eb0 tx=0x7fba9000c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90817b120 con 0x7fb90817aaf0 2024-04-25T08:29:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a6bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 0x7f5a6c147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33372/0 (socket says 172.21.15.60:33372) 2024-04-25T08:29:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6bc004c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800a970 tx=0x7fc6bc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10d881e1a937579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19f3d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 0x7fd198153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19f3d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1980b2bb0 0x7fd1980b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19f3d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980d15f0 0x7fd1980b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb908177310 con 0x7fb90817aaf0 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19f3d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980d24f0 con 0x7fd1980b2bb0 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19f3d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980d2370 con 0x7fd19806c180 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19f3d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b3040 con 0x7fd1980d15f0 2024-04-25T08:29:43.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19d171640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 0x7fd198153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a7094d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c1f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90017070 con 0x7fba9403c320 2024-04-25T08:29:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8037070 con 0x7fc6bc004800 2024-04-25T08:29:43.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cc360 0x7f5a6c15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba90005ce0 con 0x7fba9403c320 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90007290 con 0x7fba9403c320 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7f5a7094d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c1f6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41522/0 (socket says 172.21.15.60:41522) 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19d171640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 0x7fd198153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33384/0 (socket says 172.21.15.60:33384) 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19d171640 1 -- 172.21.15.60:0/3675835350 learned_addr learned my addr 172.21.15.60:0/3675835350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19d972640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980d15f0 0x7fd1980b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.007+0000 7fd19c970640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1980b2bb0 0x7fd1980b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fd19d171640 1 -- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980d15f0 msgr2=0x7fd1980b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fd19d171640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980d15f0 0x7fd1980b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fd19d171640 1 -- 172.21.15.60:0/3675835350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1980b2bb0 msgr2=0x7fd1980b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fd19d171640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1980b2bb0 0x7fd1980b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fd19d171640 1 -- 172.21.15.60:0/3675835350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980c75d0 con 0x7fd19806c180 2024-04-25T08:29:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7fcd22b63640 1 -- 172.21.15.60:0/4070752644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c145d70 con 0x7fcd1c1446d0 2024-04-25T08:29:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba94007d50 con 0x7fba9403c320 2024-04-25T08:29:43.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba94008310 con 0x7fba9403c320 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7f5a6b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cc360 0x7f5a6c15e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59032/0 (socket says 172.21.15.60:59032) 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7f5a6bfff640 1 -- 172.21.15.60:0/4188991672 learned_addr learned my addr 172.21.15.60:0/4188991672 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0cab10 con 0x7f5a6c15dbb0 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c15df90 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0703e0 con 0x7f5a6c0cc360 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4dd40b640 1 -- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d0053e50 msgr2=0x7ff4d00e9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4dd40b640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d0053e50 0x7ff4d00e9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4dd40b640 1 -- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d004f930 msgr2=0x7ff4d00da0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4dd40b640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d004f930 0x7ff4d00da0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4dd40b640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d004c5d0 con 0x7ff4d0048100 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.006+0000 7ff4ddc0c640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d0053e50 0x7ff4d00e9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7ff4dd40b640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 0x7ff4d004d670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802d370 tx=0x7ff4d8068780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c802ab20 con 0x7fc6bc004800 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7fc6d2ffd640 1 -- 172.21.15.60:0/1103707134 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c800a570 con 0x7fc6bc004800 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.008+0000 7f5e7ae6f640 1 Processor -- start 2024-04-25T08:29:43.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80745b0 con 0x7ff4d0048100 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fd19d171640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 0x7fd198153420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d800 tx=0x7fd18800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800b6c0 con 0x7fd19806c180 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd188019070 con 0x7fd19806c180 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1981622a0 con 0x7fd19806c180 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800ba80 con 0x7fd19806c180 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fcd22b63640 1 -- 172.21.15.60:0/4070752644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c1458b0 con 0x7fcd1c1446d0 2024-04-25T08:29:43.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9080057d0 con 0x7fb90817aaf0 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5a6bfff640 1 -- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cc360 msgr2=0x7f5a6c15e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cc360 0x7f5a6c15e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5a6bfff640 1 -- 172.21.15.60:0/4188991672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 msgr2=0x7f5a6c1f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c1f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5a6bfff640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c1610a0 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5a7094d640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c1f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 msgr2=0x7fc6bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fc6d9009640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6bc004c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800a970 tx=0x7fc6bc005830 comp rx=0 tx=0).stop 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 shutdown_connections 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fc6d9009640 1 --2- 172.21.15.60:0/1103707134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6bc004c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 >> 172.21.15.60:0/1103707134 conn(0x7fc6d40b3140 msgr2=0x7fc6d40b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 shutdown_connections 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7fc6d9009640 1 -- 172.21.15.60:0/1103707134 wait complete. 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9100c2070 con 0x7fb90817aaf0 2024-04-25T08:29:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd14042070 con 0x7fcd1c1446d0 2024-04-25T08:29:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7fcd197fa640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd0806b5a0 0x7fcd0806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7fb8f77fe640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8fc06b520 0x7fb8fc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcd140aad00 con 0x7fcd1c1446d0 2024-04-25T08:29:43.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1980c78b0 con 0x7fd19806c180 2024-04-25T08:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7f5e7ae6f640 1 -- start start 2024-04-25T08:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7ae6f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 0x7f5e6c0e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7ae6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c04ffc0 0x7f5e6c044430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7ae6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c036580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 0x7f5a6c147260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098c0 tx=0x7f5a600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb910124a00 con 0x7fb90817aaf0 2024-04-25T08:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7fb90ed9d640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8fc06b520 0x7fb8fc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e78c0b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 0x7f5e6c0e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60037040 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e78c0b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 0x7f5e6c0e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59052/0 (socket says 172.21.15.60:59052) 2024-04-25T08:29:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60030770 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7940c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c036580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c160dc0 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c0b3f60 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60030b40 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7fcd1bfff640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd0806b5a0 0x7fcd0806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fc6d9009640 1 Processor -- start 2024-04-25T08:29:43.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd18800bc20 con 0x7fd19806c180 2024-04-25T08:29:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fd18dffb640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd1880565f0 con 0x7fd19806c180 2024-04-25T08:29:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba9001a070 con 0x7fba9403c320 2024-04-25T08:29:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fba837fe640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba8806b5d0 0x7fba8806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba900898e0 con 0x7fba9403c320 2024-04-25T08:29:43.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1980b9580 con 0x7fd19806c180 2024-04-25T08:29:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9400a910 con 0x7fba9403c320 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7940c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c036580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41542/0 (socket says 172.21.15.60:41542) 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7ae6f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c054930 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7ae6f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c0547b0 con 0x7f5e6c04ffc0 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e7ae6f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c04f540 con 0x7f5e6c006e90 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e78c0b640 1 -- 172.21.15.60:0/2528358434 learned_addr learned my addr 172.21.15.60:0/2528358434 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.011+0000 7f5e73fff640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c04ffc0 0x7f5e6c044430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7f5e7940c640 1 -- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 msgr2=0x7f5e6c0e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7f5e7940c640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 0x7f5e6c0e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7f5e7940c640 1 -- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c04ffc0 msgr2=0x7f5e6c044430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7f5e7940c640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c04ffc0 0x7f5e6c044430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7f5e7940c640 1 -- 172.21.15.60:0/2528358434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0e96b0 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7f5e7940c640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c036580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e74050700 tx=0x7f5e740686c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5e78c0b640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 0x7f5e6c0e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T08:29:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74074610 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.009+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d808f020 con 0x7ff4d0048100 2024-04-25T08:29:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d806ade0 con 0x7ff4d0048100 2024-04-25T08:29:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d004c8b0 con 0x7ff4d0048100 2024-04-25T08:29:43.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.010+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d004ce70 con 0x7ff4d0048100 2024-04-25T08:29:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.012+0000 7fc6d9009640 1 -- start start 2024-04-25T08:29:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e74092400 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74092870 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c073a00 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fcd1bfff640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd0806b5a0 0x7fcd0806da60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009560 tx=0x7fcd0c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fb90ed9d640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8fc06b520 0x7fb8fc06d9e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb900004650 tx=0x7fb9000046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fba9a7fc640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba8806b5d0 0x7fba8806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c04da50 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fd19c970640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fc6d9009640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6d40b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c043950 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d00053c0 con 0x7ff4d0048100 2024-04-25T08:29:43.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.013+0000 7fc6d9009640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 0x7fc6d40c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fba9a7fc640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba8806b5d0 0x7fba8806da90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba94054810 tx=0x7fba9c0a9000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4d806c110 con 0x7ff4d0048100 2024-04-25T08:29:43.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 0x7fc6d40c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d9009640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d416f2f0 0x7fc6d416f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d9009640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40d0390 con 0x7fc6d40c7440 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6d40b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6d40b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59068/0 (socket says 172.21.15.60:59068) 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 -- 172.21.15.60:0/179993636 learned_addr learned my addr 172.21.15.60:0/179993636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4099a40 con 0x7fc6d416f2f0 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40d0510 con 0x7fc6bc004800 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d8808640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d416f2f0 0x7fc6d416f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7ff4cdffb640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4bc06b520 0x7ff4bc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fd19c970640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd17c06b4d0 0x7fd17c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd194034000 tx=0x7fd19402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 0x7fc6d40c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41554/0 (socket says 172.21.15.60:41554) 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 -- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d416f2f0 msgr2=0x7fc6d416f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d416f2f0 0x7fc6d416f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 -- 172.21.15.60:0/179993636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 msgr2=0x7fc6d40c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 0x7fc6d40c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7fc6d3fff640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d40c87c0 con 0x7fc6bc004800 2024-04-25T08:29:43.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.014+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a60044020 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff4d8104b60 con 0x7ff4d0048100 2024-04-25T08:29:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 0x7fc6d40c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7fc6d3fff640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6d40b64f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4007320 tx=0x7fc6c4007720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7ff4dcc0a640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4bc06b520 0x7ff4bc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7f5a68ff9640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a5806b4d0 0x7f5a5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6c005380 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a600aa970 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e74092a10 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7ff4dcc0a640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4bc06b520 0x7ff4bc06d9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4034000 tx=0x7ff4d402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.015+0000 7f5a6b7fe640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a5806b4d0 0x7f5a5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.016+0000 7f5e717fa640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e5806b550 0x7f5e5806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.016+0000 7f5a6b7fe640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a5806b4d0 0x7f5a5806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c14aae0 tx=0x7f5a5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.016+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5e74104b40 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.016+0000 7f5e78c0b640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e5806b550 0x7f5e5806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.016+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd1c1458b0 con 0x7fcd1c1446d0 2024-04-25T08:29:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.016+0000 7f5e78c0b640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e5806b550 0x7f5e5806da10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009630 tx=0x7f5e6802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.018+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1980b9580 con 0x7fd19806c180 2024-04-25T08:29:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.020+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c401a070 con 0x7fc6bc004800 2024-04-25T08:29:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.020+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c400f960 con 0x7fc6bc004800 2024-04-25T08:29:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.020+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d416fcb0 con 0x7fc6bc004800 2024-04-25T08:29:43.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.020+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d40c8d10 con 0x7fc6bc004800 2024-04-25T08:29:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.020+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c400fd30 con 0x7fc6bc004800 2024-04-25T08:29:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.020+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9100f3180 con 0x7fb90817aaf0 2024-04-25T08:29:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.022+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4005290 con 0x7fc6bc004800 2024-04-25T08:29:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.022+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6c401d360 con 0x7fc6bc004800 2024-04-25T08:29:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.022+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4d8099030 con 0x7ff4d0048100 2024-04-25T08:29:43.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.022+0000 7fc6d0ff9640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6c006b500 0x7fc6c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:43.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.023+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6c4089e50 con 0x7fc6bc004800 2024-04-25T08:29:43.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.023+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba940d5650 con 0x7fba9403c320 2024-04-25T08:29:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.023+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6c006b500 0x7fc6c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.023+0000 7fc6d37fe640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6c006b500 0x7fc6c006d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8034000 tx=0x7fc6c802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.027+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a60032e20 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.029+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6d416fcb0 con 0x7fc6bc004800 2024-04-25T08:29:43.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.032+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e7408e030 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.442+0000 7fcd22b63640 1 -- 172.21.15.60:0/4070752644 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcd1c073a00 con 0x7fcd1c1446d0 2024-04-25T08:29:43.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.443+0000 7fcd197fa640 1 -- 172.21.15.60:0/4070752644 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd14079260 con 0x7fcd1c1446d0 2024-04-25T08:29:43.440 INFO:teuthology.orchestra.run.smithi060.stdout:60129542152 2024-04-25T08:29:43.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.446+0000 7fcd02ffd640 1 -- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd0806b5a0 msgr2=0x7fcd0806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.446+0000 7fcd02ffd640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd0806b5a0 0x7fcd0806da60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009560 tx=0x7fcd0c009190 comp rx=0 tx=0).stop 2024-04-25T08:29:43.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.446+0000 7fcd02ffd640 1 -- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 msgr2=0x7fcd1c0aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.446+0000 7fcd02ffd640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0aaf40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd14034040 tx=0x7fcd14004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:43.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 -- 172.21.15.60:0/4070752644 shutdown_connections 2024-04-25T08:29:43.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd1c1452a0 0x7fcd1c16aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd0806b5a0 0x7fcd0806da60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd1c144d50 0x7fcd1c16a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 --2- 172.21.15.60:0/4070752644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd1c1446d0 0x7fcd1c0aaf40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fb91524d640 1 -- 172.21.15.60:0/1301314312 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb9080e45b0 con 0x7fb90817aaf0 2024-04-25T08:29:43.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 -- 172.21.15.60:0/4070752644 >> 172.21.15.60:0/4070752644 conn(0x7fcd1c06abe0 msgr2=0x7fcd1c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 -- 172.21.15.60:0/4070752644 shutdown_connections 2024-04-25T08:29:43.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.447+0000 7fcd02ffd640 1 -- 172.21.15.60:0/4070752644 wait complete. 2024-04-25T08:29:43.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.448+0000 7fb8f77fe640 1 -- 172.21.15.60:0/1301314312 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb9100f7220 con 0x7fb90817aaf0 2024-04-25T08:29:43.446 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.451+0000 7fb8f57fa640 1 -- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8fc06b520 msgr2=0x7fb8fc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.451+0000 7fb8f57fa640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8fc06b520 0x7fb8fc06d9e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb900004650 tx=0x7fb9000046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 -- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90817aaf0 msgr2=0x7fb908036760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90817aaf0 0x7fb908036760 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb91002d370 tx=0x7fb910068960 comp rx=0 tx=0).stop 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 -- 172.21.15.60:0/1301314312 shutdown_connections 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb90817aaf0 0x7fb908036760 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9080da8f0 0x7fb90817a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8fc06b520 0x7fb8fc06d9e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 --2- 172.21.15.60:0/1301314312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8f80051c0 0x7fb908177b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 -- 172.21.15.60:0/1301314312 >> 172.21.15.60:0/1301314312 conn(0x7fb908053740 msgr2=0x7fb9080556b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 -- 172.21.15.60:0/1301314312 shutdown_connections 2024-04-25T08:29:43.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fb8f57fa640 1 -- 172.21.15.60:0/1301314312 wait complete. 2024-04-25T08:29:43.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.452+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd198073a00 con 0x7fd19806c180 2024-04-25T08:29:43.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.453+0000 7fd18dffb640 1 -- 172.21.15.60:0/3675835350 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd188058c50 con 0x7fd19806c180 2024-04-25T08:29:43.450 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:43.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.454+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4d0052850 con 0x7ff4d0048100 2024-04-25T08:29:43.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.455+0000 7ff4cdffb640 1 -- 172.21.15.60:0/1680733394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4d80d3670 con 0x7ff4d0048100 2024-04-25T08:29:43.451 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:43.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.456+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fba940de930 con 0x7fba9403c320 2024-04-25T08:29:43.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd17c06b4d0 msgr2=0x7fd17c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd17c06b4d0 0x7fd17c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd194034000 tx=0x7fd19402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:43.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 msgr2=0x7fd198153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fba837fe640 1 -- 172.21.15.60:0/4022793865 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba90057e40 con 0x7fba9403c320 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 0x7fd198153420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d800 tx=0x7fd18800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 shutdown_connections 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1980d15f0 0x7fd1980b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1980b2bb0 0x7fd1980b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 --2- 172.21.15.60:0/3675835350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd19806c180 0x7fd198153420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 >> 172.21.15.60:0/3675835350 conn(0x7fd1980d3a20 msgr2=0x7fd19806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.455 INFO:teuthology.orchestra.run.smithi060.stdout:60129542152 2024-04-25T08:29:43.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 shutdown_connections 2024-04-25T08:29:43.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.457+0000 7fd19f3d5640 1 -- 172.21.15.60:0/3675835350 wait complete. 2024-04-25T08:29:43.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.458+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5e6c04a490 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.458+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4bc06b520 msgr2=0x7ff4bc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.458+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4bc06b520 0x7ff4bc06d9e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4034000 tx=0x7ff4d402e000 comp rx=0 tx=0).stop 2024-04-25T08:29:43.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.458+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5a6c15c640 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.458+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 msgr2=0x7ff4d004d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.458+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 0x7ff4d004d670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802d370 tx=0x7ff4d8068780 comp rx=0 tx=0).stop 2024-04-25T08:29:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7f5e717fa640 1 -- 172.21.15.60:0/2528358434 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5e740d37f0 con 0x7f5e6c0eacf0 2024-04-25T08:29:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 shutdown_connections 2024-04-25T08:29:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d0053e50 0x7ff4d00e9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4bc06b520 0x7ff4bc06d9e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7f5a68ff9640 1 -- 172.21.15.60:0/4188991672 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a60078dc0 con 0x7f5a6c0cbf00 2024-04-25T08:29:43.459 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:43.459 INFO:teuthology.orchestra.run.smithi060.stdout:60129542152 2024-04-25T08:29:43.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d004f930 0x7ff4d00da0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 --2- 172.21.15.60:0/1680733394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d0048100 0x7ff4d004d670 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 >> 172.21.15.60:0/1680733394 conn(0x7ff4d0009fe0 msgr2=0x7ff4d003a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 shutdown_connections 2024-04-25T08:29:43.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.459+0000 7ff4df66f640 1 -- 172.21.15.60:0/1680733394 wait complete. 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.461+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba8806b5d0 msgr2=0x7fba8806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.461+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba8806b5d0 0x7fba8806da90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba94054810 tx=0x7fba9c0a9000 comp rx=0 tx=0).stop 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.461+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 msgr2=0x7fba940d5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.461+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 0x7fba940d5110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba90001eb0 tx=0x7fba9000c9b0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 shutdown_connections 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba940e7a10 0x7fba940e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba94054860 0x7fba940e3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba8806b5d0 0x7fba8806da90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 --2- 172.21.15.60:0/4022793865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba9403c320 0x7fba940d5110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 >> 172.21.15.60:0/4022793865 conn(0x7fba94056c90 msgr2=0x7fba94004f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 shutdown_connections 2024-04-25T08:29:43.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7fbaa1330640 1 -- 172.21.15.60:0/4022793865 wait complete. 2024-04-25T08:29:43.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e5806b550 msgr2=0x7f5e5806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e5806b550 0x7f5e5806da10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009630 tx=0x7f5e6802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:43.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 msgr2=0x7f5e6c036580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.462+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c036580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e74050700 tx=0x7f5e740686c0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a5806b4d0 msgr2=0x7f5a5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a5806b4d0 0x7f5a5806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c14aae0 tx=0x7f5a5c008040 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 msgr2=0x7f5a6c147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 0x7f5a6c147260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098c0 tx=0x7f5a600046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 shutdown_connections 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c15dbb0 0x7f5a6c1f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0cc360 0x7f5a6c15e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a5806b4d0 0x7f5a5806d990 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c14aae0 tx=0x7f5a5c008040 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 --2- 172.21.15.60:0/4188991672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0cbf00 0x7f5a6c147260 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a600098c0 tx=0x7f5a600046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 >> 172.21.15.60:0/4188991672 conn(0x7f5a6c071af0 msgr2=0x7f5a6c0c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 shutdown_connections 2024-04-25T08:29:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e6c0eacf0 0x7f5e6c036580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e5806b550 0x7f5e5806da10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e6c04ffc0 0x7f5e6c044430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 --2- 172.21.15.60:0/2528358434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e6c006e90 0x7f5e6c0e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 >> 172.21.15.60:0/2528358434 conn(0x7f5e6c009fa0 msgr2=0x7f5e6c0e9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 shutdown_connections 2024-04-25T08:29:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.464+0000 7f5a723b0640 1 -- 172.21.15.60:0/4188991672 wait complete. 2024-04-25T08:29:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.463+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 shutdown_connections 2024-04-25T08:29:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.464+0000 7f5e7ae6f640 1 -- 172.21.15.60:0/2528358434 wait complete. 2024-04-25T08:29:43.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542152 for osd.7 2024-04-25T08:29:43.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-04-25T08:29:43.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-04-25T08:29:43.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2024-04-25T08:29:43.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2024-04-25T08:29:43.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542152 for osd.5 2024-04-25T08:29:43.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542152 for osd.6 2024-04-25T08:29:43.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.488+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6b4005580 con 0x7fc6bc004800 2024-04-25T08:29:43.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.489+0000 7fc6d0ff9640 1 -- 172.21.15.60:0/179993636 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc6c4058980 con 0x7fc6bc004800 2024-04-25T08:29:43.486 INFO:teuthology.orchestra.run.smithi060.stdout:55834574855 2024-04-25T08:29:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6c006b500 msgr2=0x7fc6c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6c006b500 0x7fc6c006d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8034000 tx=0x7fc6c802e000 comp rx=0 tx=0).stop 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 msgr2=0x7fc6d40b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6d40b64f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4007320 tx=0x7fc6c4007720 comp rx=0 tx=0).stop 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 shutdown_connections 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6c006b500 0x7fc6c006d9c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc6d416f2f0 0x7fc6d416f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6d40c7440 0x7fc6d40c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 --2- 172.21.15.60:0/179993636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc6bc004800 0x7fc6d40b64f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 >> 172.21.15.60:0/179993636 conn(0x7fc6d40b3140 msgr2=0x7fc6d40ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 shutdown_connections 2024-04-25T08:29:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:43.491+0000 7fc6d9009640 1 -- 172.21.15.60:0/179993636 wait complete. 2024-04-25T08:29:43.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-04-25T08:29:44.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:29:44.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:29:44.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:29:44.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:29:44.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:29:44.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:29:44.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:29:44.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.643+0000 7fd25443f640 1 Processor -- start 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.643+0000 7fd25443f640 1 -- start start 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.645+0000 7fd25443f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd24c06bbf0 0x7fd24c06bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.645+0000 7fd25443f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.645+0000 7fd25443f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd24c0cda20 0x7fd24c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.645+0000 7fd25443f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c0c5ac0 con 0x7fd24c06bbf0 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.645+0000 7fd25443f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24c06c510 con 0x7fd24c0cda20 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.645+0000 7fd25443f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c4eb0 con 0x7fd24c0acd80 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.646+0000 7fd2529dc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd24c0cda20 0x7fd24c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.646+0000 7fd2529dc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd24c0cda20 0x7fd24c0cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33412/0 (socket says 172.21.15.60:33412) 2024-04-25T08:29:44.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.646+0000 7fd2529dc640 1 -- 172.21.15.60:0/2031964997 learned_addr learned my addr 172.21.15.60:0/2031964997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.646+0000 7fd2519da640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.646+0000 7fd2529dc640 1 -- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 msgr2=0x7fd24c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.647+0000 7fd2529dc640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.647+0000 7fd2529dc640 1 -- 172.21.15.60:0/2031964997 >> v2:172.21.15.60:3300/0 conn(0x7fd24c06bbf0 msgr2=0x7fd24c06bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.647+0000 7fd2521db640 1 --2- 172.21.15.60:0/2031964997 >> v2:172.21.15.60:3300/0 conn(0x7fd24c06bbf0 0x7fd24c06bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.647+0000 7fd2529dc640 1 --2- 172.21.15.60:0/2031964997 >> v2:172.21.15.60:3300/0 conn(0x7fd24c06bbf0 0x7fd24c06bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.647+0000 7fd2529dc640 1 -- 172.21.15.60:0/2031964997 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c0c5190 con 0x7fd24c0cda20 2024-04-25T08:29:44.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.648+0000 7fd2519da640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:44.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.648+0000 7fd2529dc640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c0cda20 0x7fd24c0cdea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd24c062010 tx=0x7fd24402c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467efa28dab8581d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.648+0000 7fd2521db640 1 --2- 172.21.15.60:0/2031964997 >> v2:172.21.15.60:3300/0 conn(0x7fd24c06bbf0 0x7fd24c06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:44.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.649+0000 7fd2511d9640 1 -- 172.21.15.60:0/2031964997 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24402f070 con 0x7fd24c0cda20 2024-04-25T08:29:44.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.649+0000 7fd2511d9640 1 -- 172.21.15.60:0/2031964997 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd244033750 con 0x7fd24c0cda20 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.650+0000 7fd25443f640 1 -- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c0cda20 msgr2=0x7fd24c0cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.650+0000 7fd25443f640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c0cda20 0x7fd24c0cdea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd24c062010 tx=0x7fd24402c8e0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 -- 172.21.15.60:0/2031964997 shutdown_connections 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c0cda20 0x7fd24c0cdea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 --2- 172.21.15.60:0/2031964997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 --2- 172.21.15.60:0/2031964997 >> v2:172.21.15.60:3300/0 conn(0x7fd24c06bbf0 0x7fd24c06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 -- 172.21.15.60:0/2031964997 >> 172.21.15.60:0/2031964997 conn(0x7fd24c071af0 msgr2=0x7fd24c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 -- 172.21.15.60:0/2031964997 shutdown_connections 2024-04-25T08:29:44.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.651+0000 7fd25443f640 1 -- 172.21.15.60:0/2031964997 wait complete. 2024-04-25T08:29:44.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.653+0000 7fd25443f640 1 Processor -- start 2024-04-25T08:29:44.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.654+0000 7fd25443f640 1 -- start start 2024-04-25T08:29:44.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd25443f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c06bbf0 0x7fd24c1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd25443f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c1524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd25443f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 0x7fd24c0affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd25443f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c6500 con 0x7fd24c06bbf0 2024-04-25T08:29:44.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd25443f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c5750 con 0x7fd24c153de0 2024-04-25T08:29:44.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd25443f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24c0c6960 con 0x7fd24c0acd80 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd2529dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 0x7fd24c0affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd2529dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 0x7fd24c0affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33424/0 (socket says 172.21.15.60:33424) 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.655+0000 7fd2529dc640 1 -- 172.21.15.60:0/1436108676 learned_addr learned my addr 172.21.15.60:0/1436108676 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.656+0000 7fd2529dc640 1 -- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 msgr2=0x7fd24c1524f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.656+0000 7fd2529dc640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c1524f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.656+0000 7fd2529dc640 1 -- 172.21.15.60:0/1436108676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c06bbf0 msgr2=0x7fd24c1495d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.656+0000 7fd2529dc640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c06bbf0 0x7fd24c1495d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.656+0000 7fd2529dc640 1 -- 172.21.15.60:0/1436108676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24c1543a0 con 0x7fd24c153de0 2024-04-25T08:29:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.656+0000 7fd2529dc640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 0x7fd24c0affd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d120 tx=0x7fd244004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.657+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244042030 con 0x7fd24c153de0 2024-04-25T08:29:44.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.657+0000 7fd25443f640 1 -- 172.21.15.60:0/1436108676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24c154680 con 0x7fd24c153de0 2024-04-25T08:29:44.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.657+0000 7fd25443f640 1 -- 172.21.15.60:0/1436108676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c154c40 con 0x7fd24c153de0 2024-04-25T08:29:44.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.657+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24400bcd0 con 0x7fd24c153de0 2024-04-25T08:29:44.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.657+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244033900 con 0x7fd24c153de0 2024-04-25T08:29:44.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.660+0000 7fd25443f640 1 -- 172.21.15.60:0/1436108676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd234005290 con 0x7fd24c153de0 2024-04-25T08:29:44.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.661+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd244045070 con 0x7fd24c153de0 2024-04-25T08:29:44.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.662+0000 7fd23effd640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd23806b640 0x7fd23806db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.662+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd2440ab050 con 0x7fd24c153de0 2024-04-25T08:29:44.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.663+0000 7fd2521db640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd23806b640 0x7fd23806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.663+0000 7fd2521db640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd23806b640 0x7fd23806db00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd240006b10 tx=0x7fd240008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.667+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd244079c10 con 0x7fd24c153de0 2024-04-25T08:29:44.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.696+0000 7f4e3817e640 1 Processor -- start 2024-04-25T08:29:44.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.697+0000 7f4e3817e640 1 -- start start 2024-04-25T08:29:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.696+0000 7f19e322a640 1 Processor -- start 2024-04-25T08:29:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.696+0000 7f19e322a640 1 -- start start 2024-04-25T08:29:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7fae1f85d640 1 Processor -- start 2024-04-25T08:29:44.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f4e3817e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e30161100 0x7f4e3006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f4e3817e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3006e0f0 con 0x7f4e300cc8b0 2024-04-25T08:29:44.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f4e3817e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e30161a50 con 0x7f4e300cfd80 2024-04-25T08:29:44.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f4e3817e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e30161800 con 0x7f4e30161100 2024-04-25T08:29:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e322a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19d40062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7fae1f85d640 1 -- start start 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e322a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19d4006820 0x7f19d4006cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e322a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d4048410 con 0x7f19d4048030 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e322a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d4050190 con 0x7f19d4006820 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e322a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4054790 con 0x7f19d4005e80 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e17c7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19d4006820 0x7f19d4006cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4e3671b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4e300cfd80 0x7f4e3006d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43754/0 (socket says 172.21.15.60:43754) 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4e3671b640 1 -- 172.21.15.60:0/3688084077 learned_addr learned my addr 172.21.15.60:0/3688084077 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:44.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4e35f1a640 1 --2- 172.21.15.60:0/3688084077 >> v2:172.21.15.100:3300/0 conn(0x7f4e30161100 0x7f4e3006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e17c7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f19d4006820 0x7f19d4006cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41564/0 (socket says 172.21.15.60:41564) 2024-04-25T08:29:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19e17c7640 1 -- 172.21.15.60:0/3381404146 learned_addr learned my addr 172.21.15.60:0/3381404146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4e34f18640 1 -- 172.21.15.60:0/3688084077 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650259925 0 0) 0x7f4e30161a50 con 0x7f4e300cfd80 2024-04-25T08:29:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.698+0000 7f19dbfff640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19d40062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746939582 0 0) 0x7f19d4048410 con 0x7f19d4048030 2024-04-25T08:29:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4e35f1a640 1 -- 172.21.15.60:0/3688084077 >> v1:172.21.15.100:6789/0 conn(0x7f4e300cfd80 legacy=0x7f4e3006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19cc003520 con 0x7f19d4048030 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790933615 0 0) 0x7f19cc003520 con 0x7f19d4048030 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 msgr2=0x7f19d40062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19d40062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 >> v2:172.21.15.60:3300/0 conn(0x7f19d4006820 msgr2=0x7f19d4006cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 --2- 172.21.15.60:0/3381404146 >> v2:172.21.15.60:3300/0 conn(0x7f19d4006820 0x7f19d4006cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40536f0 con 0x7f19d4048030 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19dbfff640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19d40062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19e17c7640 1 --2- 172.21.15.60:0/3381404146 >> v2:172.21.15.60:3300/0 conn(0x7f19d4006820 0x7f19d4006cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f19d0001670 con 0x7f19d4048030 2024-04-25T08:29:44.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4e35f1a640 1 -- 172.21.15.60:0/3688084077 >> v1:172.21.15.60:6789/0 conn(0x7f4e300cc8b0 legacy=0x7f4e3006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 >> v1:172.21.15.100:6789/0 conn(0x7f19d4048030 legacy=0x7f19d404fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1f85d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fae100e30b0 0x7fae10057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19cc0043b0 0x7f19cc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1f85d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae100542f0 0x7fae100068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f4e35f1a640 1 -- 172.21.15.60:0/3688084077 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e30161d30 con 0x7f4e30161100 2024-04-25T08:29:44.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1f85d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae100582c0 con 0x7fae1003e830 2024-04-25T08:29:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1cdf8640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fae1003e830 0x7fae10053be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43782/0 (socket says 172.21.15.60:43782) 2024-04-25T08:29:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1cdf8640 1 -- 172.21.15.60:0/3537395736 learned_addr learned my addr 172.21.15.60:0/3537395736 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae10058560 con 0x7fae100e30b0 2024-04-25T08:29:44.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae10058840 con 0x7fae100542f0 2024-04-25T08:29:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19cc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4050190 con 0x7f19cc0043b0 2024-04-25T08:29:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4054790 con 0x7f19cc0039b0 2024-04-25T08:29:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc00b9c0 con 0x7f19d4005e80 2024-04-25T08:29:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1ddfa640 1 --2- 172.21.15.60:0/3537395736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae100542f0 0x7fae100068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae1d5f9640 1 --2- 172.21.15.60:0/3537395736 >> v2:172.21.15.60:3300/0 conn(0x7fae100e30b0 0x7fae10057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19e17c7640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19cc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7fae0ffff640 1 -- 172.21.15.60:0/3537395736 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206060335 0 0) 0x7fae100582c0 con 0x7fae1003e830 2024-04-25T08:29:44.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.700+0000 7f19dbfff640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19cc0043b0 0x7f19cc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7fae0ffff640 1 -- 172.21.15.60:0/3537395736 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae04003520 con 0x7fae1003e830 2024-04-25T08:29:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.697+0000 7f4f694b6640 1 Processor -- start 2024-04-25T08:29:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.697+0000 7f4f694b6640 1 -- start start 2024-04-25T08:29:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f694b6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f640cef20 0x7f4f640b5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f694b6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4f6416e0d0 0x7f4f640b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f694b6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6416e690 con 0x7f4f640b3140 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f694b6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6416ad00 con 0x7f4f6416e0d0 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f694b6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6416e970 con 0x7f4f640cef20 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f62ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4f640cef20 0x7f4f640b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f637fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4f640b3140 0x7f4f6416b190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43774/0 (socket says 172.21.15.60:43774) 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f637fe640 1 -- 172.21.15.60:0/1137626559 learned_addr learned my addr 172.21.15.60:0/1137626559 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.699+0000 7f4f5bfff640 1 --2- 172.21.15.60:0/1137626559 >> v2:172.21.15.60:3300/0 conn(0x7f4f6416e0d0 0x7f4f640b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7fae1ddfa640 1 -- 172.21.15.60:0/3537395736 >> v2:172.21.15.60:3300/0 conn(0x7fae100e30b0 msgr2=0x7fae10057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7fae1ddfa640 1 --2- 172.21.15.60:0/3537395736 >> v2:172.21.15.60:3300/0 conn(0x7fae100e30b0 0x7fae10057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7fae1ddfa640 1 -- 172.21.15.60:0/3537395736 >> v1:172.21.15.100:6789/0 conn(0x7fae1003e830 legacy=0x7fae10053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7fae1ddfa640 1 -- 172.21.15.60:0/3537395736 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae10058b20 con 0x7fae100542f0 2024-04-25T08:29:44.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 msgr2=0x7f19cc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4005e80 0x7f19cc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3381404146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19cc0043b0 msgr2=0x7f19cc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19cc0043b0 0x7f19cc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3381404146 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc003520 con 0x7f19cc0039b0 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f4f62ffd640 1 -- 172.21.15.60:0/1137626559 >> v2:172.21.15.60:3300/0 conn(0x7f4f6416e0d0 msgr2=0x7f4f640b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f4f62ffd640 1 --2- 172.21.15.60:0/1137626559 >> v2:172.21.15.60:3300/0 conn(0x7f4f6416e0d0 0x7f4f640b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f4f62ffd640 1 -- 172.21.15.60:0/1137626559 >> v1:172.21.15.100:6789/0 conn(0x7f4f640b3140 legacy=0x7f4f6416b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.701+0000 7f4f62ffd640 1 -- 172.21.15.60:0/1137626559 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6416ec50 con 0x7f4f640cef20 2024-04-25T08:29:44.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4f5bfff640 1 --2- 172.21.15.60:0/1137626559 >> v2:172.21.15.60:3300/0 conn(0x7f4f6416e0d0 0x7f4f640b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:44.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4e35f1a640 1 --2- 172.21.15.60:0/3688084077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e30161100 0x7f4e3006e4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097a0 tx=0x7f4e2002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b92c39712c0769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f113a415640 1 Processor -- start 2024-04-25T08:29:44.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f113a415640 1 -- start start 2024-04-25T08:29:44.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19cc003d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19d0003d20 tx=0x7f19cc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d984c8c7d32661f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4e34f18640 1 -- 172.21.15.60:0/3688084077 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20037070 con 0x7f4e30161100 2024-04-25T08:29:44.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7fae1ddfa640 1 --2- 172.21.15.60:0/3537395736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae100542f0 0x7fae100068c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae140097a0 tx=0x7fae1402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a08f308baf4485f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4e34f18640 1 -- 172.21.15.60:0/3688084077 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20032440 con 0x7f4e30161100 2024-04-25T08:29:44.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4f62ffd640 1 --2- 172.21.15.60:0/1137626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b5220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f640b58f0 tx=0x7f4f5402e520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821ea0b14917572d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7fae0ffff640 1 -- 172.21.15.60:0/3537395736 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14037070 con 0x7fae100542f0 2024-04-25T08:29:44.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00038f0 con 0x7f19cc0039b0 2024-04-25T08:29:44.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4e34f18640 1 -- 172.21.15.60:0/3688084077 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20032810 con 0x7f4e30161100 2024-04-25T08:29:44.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7fae0ffff640 1 -- 172.21.15.60:0/3537395736 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14032440 con 0x7fae100542f0 2024-04-25T08:29:44.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7fae0ffff640 1 -- 172.21.15.60:0/3537395736 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14032810 con 0x7fae100542f0 2024-04-25T08:29:44.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d002f070 con 0x7f19cc0039b0 2024-04-25T08:29:44.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f19db7fe640 1 -- 172.21.15.60:0/3381404146 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d002b730 con 0x7f19cc0039b0 2024-04-25T08:29:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.702+0000 7f4f627fc640 1 -- 172.21.15.60:0/1137626559 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54033710 con 0x7f4f640cef20 2024-04-25T08:29:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f4f627fc640 1 -- 172.21.15.60:0/1137626559 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f540338b0 con 0x7f4f640cef20 2024-04-25T08:29:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f4f627fc640 1 -- 172.21.15.60:0/1137626559 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54033c80 con 0x7f4f640cef20 2024-04-25T08:29:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f6fcddd6640 1 Processor -- start 2024-04-25T08:29:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f19e322a640 1 -- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 msgr2=0x7f19cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f6fcddd6640 1 -- start start 2024-04-25T08:29:44.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f4e3817e640 1 -- 172.21.15.60:0/3688084077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e30161100 msgr2=0x7f4e3006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f4e3817e640 1 --2- 172.21.15.60:0/3688084077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e30161100 0x7f4e3006e4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097a0 tx=0x7f4e2002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae100542f0 msgr2=0x7fae100068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3537395736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae100542f0 0x7fae100068c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae140097a0 tx=0x7fae1402a210 comp rx=0 tx=0).stop 2024-04-25T08:29:44.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f19e322a640 1 --2- 172.21.15.60:0/3381404146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19cc003d90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19d0003d20 tx=0x7f19cc005830 comp rx=0 tx=0).stop 2024-04-25T08:29:44.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f19e17c7640 1 -- 172.21.15.60:0/3381404146 reap_dead start 2024-04-25T08:29:44.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.703+0000 7f113a415640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f4e3817e640 1 -- 172.21.15.60:0/3688084077 shutdown_connections 2024-04-25T08:29:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f4e3817e640 1 --2- 172.21.15.60:0/3688084077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e30161100 0x7f4e3006e4f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f4e3817e640 1 -- 172.21.15.60:0/3688084077 >> 172.21.15.60:0/3688084077 conn(0x7f4e3006abe0 msgr2=0x7f4e3006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 shutdown_connections 2024-04-25T08:29:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3537395736 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae100542f0 0x7fae100068c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f113a415640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f113406d570 0x7f11340b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3537395736 >> v2:172.21.15.60:3300/0 conn(0x7fae100e30b0 0x7fae10057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 >> 172.21.15.60:0/3537395736 conn(0x7fae100559d0 msgr2=0x7fae10055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f19e322a640 1 -- 172.21.15.60:0/3381404146 shutdown_connections 2024-04-25T08:29:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f19e322a640 1 -- 172.21.15.60:0/3381404146 >> 172.21.15.60:0/3381404146 conn(0x7f19d4009fa0 msgr2=0x7f19d400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f4e3817e640 1 -- 172.21.15.60:0/3688084077 shutdown_connections 2024-04-25T08:29:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f4e3817e640 1 -- 172.21.15.60:0/3688084077 wait complete. 2024-04-25T08:29:44.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 shutdown_connections 2024-04-25T08:29:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f6fcddd6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6fc80b3690 0x7f6fc806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f6fccdd4640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6fc815cdc0 0x7f6fc80b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43790/0 (socket says 172.21.15.60:43790) 2024-04-25T08:29:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f19e322a640 1 -- 172.21.15.60:0/3381404146 shutdown_connections 2024-04-25T08:29:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f19e322a640 1 -- 172.21.15.60:0/3381404146 wait complete. 2024-04-25T08:29:44.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fccdd4640 1 -- 172.21.15.60:0/2169513042 learned_addr learned my addr 172.21.15.60:0/2169513042 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7fae1f85d640 1 -- 172.21.15.60:0/3537395736 wait complete. 2024-04-25T08:29:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc815d4f0 con 0x7f6fc80b2710 2024-04-25T08:29:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fc7fff640 1 --2- 172.21.15.60:0/2169513042 >> v2:172.21.15.60:3300/0 conn(0x7f6fc80b3690 0x7f6fc806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc815d790 con 0x7f6fc815cdc0 2024-04-25T08:29:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc815da70 con 0x7f6fc80b3690 2024-04-25T08:29:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7ff662e3c640 1 Processor -- start 2024-04-25T08:29:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f113a415640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f113406bbf0 0x7f113406c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889653269 0 0) 0x7f6fc815d790 con 0x7f6fc815cdc0 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f113a415640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113406c580 con 0x7f113406d570 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f113a415640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113406da30 con 0x7f113406bbf0 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f113a415640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113406dd10 con 0x7f11341436b0 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f11337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f113406d570 0x7f11340b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.704+0000 7f1133fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb8003520 con 0x7f6fc815cdc0 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f1133fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59100/0 (socket says 172.21.15.60:59100) 2024-04-25T08:29:44.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7f1133fff640 1 -- 172.21.15.60:0/2994317295 learned_addr learned my addr 172.21.15.60:0/2994317295 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.705+0000 7ff662e3c640 1 -- start start 2024-04-25T08:29:44.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f11389b2640 1 --2- 172.21.15.60:0/2994317295 >> v2:172.21.15.100:3300/0 conn(0x7f113406bbf0 0x7f113406c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f1133fff640 1 -- 172.21.15.60:0/2994317295 >> v2:172.21.15.100:3300/0 conn(0x7f113406bbf0 msgr2=0x7f113406c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f4e3817e640 1 Processor -- start 2024-04-25T08:29:44.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f1133fff640 1 --2- 172.21.15.60:0/2994317295 >> v2:172.21.15.100:3300/0 conn(0x7f113406bbf0 0x7f113406c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f1133fff640 1 -- 172.21.15.60:0/2994317295 >> v2:172.21.15.60:3300/0 conn(0x7f113406d570 msgr2=0x7f11340b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f4f694b6640 1 -- 172.21.15.60:0/1137626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 msgr2=0x7f4f640b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f4f694b6640 1 --2- 172.21.15.60:0/1137626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b5220 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f640b58f0 tx=0x7f4f5402e520 comp rx=0 tx=0).stop 2024-04-25T08:29:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f1133fff640 1 --2- 172.21.15.60:0/2994317295 >> v2:172.21.15.60:3300/0 conn(0x7f113406d570 0x7f11340b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f4e3817e640 1 -- start start 2024-04-25T08:29:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f1133fff640 1 -- 172.21.15.60:0/2994317295 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113406dff0 con 0x7f11341436b0 2024-04-25T08:29:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 <== mon.1 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 305658549 0 0) 0x7f6fb8003520 con 0x7f6fc815cdc0 2024-04-25T08:29:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f11389b2640 1 --2- 172.21.15.60:0/2994317295 >> v2:172.21.15.100:3300/0 conn(0x7f113406bbf0 0x7f113406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 >> v2:172.21.15.60:3300/0 conn(0x7f6fc80b3690 msgr2=0x7f6fc806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 --2- 172.21.15.60:0/2169513042 >> v2:172.21.15.60:3300/0 conn(0x7f6fc80b3690 0x7f6fc806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f19e322a640 1 Processor -- start 2024-04-25T08:29:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 >> v1:172.21.15.60:6789/0 conn(0x7f6fc80b2710 legacy=0x7f6fc806df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc815dd50 con 0x7f6fc815cdc0 2024-04-25T08:29:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f19e322a640 1 -- start start 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 <== mon.1 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6fbc001670 con 0x7f6fc815cdc0 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.706+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 >> v1:172.21.15.100:6789/0 conn(0x7f6fc815cdc0 legacy=0x7f6fc80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc77fe640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fb80039b0 0x7f6fb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc77fe640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff662e3c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff65c161e50 0x7ff65c06daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff662e3c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 0x7ff65c06c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff662e3c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c06dfe0 con 0x7ff65c0b3a20 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff662e3c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c162230 con 0x7ff65c161e50 2024-04-25T08:29:44.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff662e3c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c06c590 con 0x7ff65c06bbf0 2024-04-25T08:29:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fb80039b0 0x7f6fb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 0x7ff65c06c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc77fe640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc815da70 con 0x7f6fb8004800 2024-04-25T08:29:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb800c010 con 0x7f6fb80039b0 2024-04-25T08:29:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb800c210 con 0x7f6fb80051c0 2024-04-25T08:29:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 0x7ff65c06c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59106/0 (socket says 172.21.15.60:59106) 2024-04-25T08:29:44.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f6fc7fff640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 -- 172.21.15.60:0/2991620541 learned_addr learned my addr 172.21.15.60:0/2991620541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 -- 172.21.15.60:0/2991620541 >> v2:172.21.15.100:3300/0 conn(0x7ff65c161e50 msgr2=0x7ff65c06daa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:29:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 --2- 172.21.15.60:0/2991620541 >> v2:172.21.15.100:3300/0 conn(0x7ff65c161e50 0x7ff65c06daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 -- 172.21.15.60:0/2991620541 >> v1:172.21.15.60:6789/0 conn(0x7ff65c0b3a20 legacy=0x7ff65c0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7ff661639640 1 -- 172.21.15.60:0/2991620541 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0b47f0 con 0x7ff65c06bbf0 2024-04-25T08:29:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f1133fff640 1 --2- 172.21.15.60:0/2994317295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340cd1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f113415a340 tx=0x7f112802b380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a1dd4fe90e3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.707+0000 7f1132ffd640 1 -- 172.21.15.60:0/2994317295 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128038070 con 0x7f11341436b0 2024-04-25T08:29:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f6fccdd4640 1 -- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80051c0 msgr2=0x7f6fb800b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:29:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80051c0 0x7f6fb800b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f6fccdd4640 1 -- 172.21.15.60:0/2169513042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fb8004800 msgr2=0x7f6fb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f1132ffd640 1 -- 172.21.15.60:0/2994317295 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128033640 con 0x7f11341436b0 2024-04-25T08:29:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f1132ffd640 1 -- 172.21.15.60:0/2994317295 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280339f0 con 0x7f11341436b0 2024-04-25T08:29:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e3817e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e300cc8b0 0x7f4e3014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f19e322a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19d40e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e3817e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e300cfd80 0x7f4e301531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e35f1a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e300cc8b0 0x7f4e3014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e35f1a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e300cc8b0 0x7f4e3014c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59118/0 (socket says 172.21.15.60:59118) 2024-04-25T08:29:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f19e0fc6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19d40e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f19e0fc6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19d40e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33472/0 (socket says 172.21.15.60:33472) 2024-04-25T08:29:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e35f1a640 1 -- 172.21.15.60:0/4235478113 learned_addr learned my addr 172.21.15.60:0/4235478113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3648017512 learned_addr learned my addr 172.21.15.60:0/3648017512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e35719640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e300cfd80 0x7f4e301531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19d4043bc0 0x7f19d40456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f4e3817e640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e30161100 0x7f4e30155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7ff661639640 1 --2- 172.21.15.60:0/2991620541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 0x7ff65c06c050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009870 tx=0x7ff64c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bace68309efdfdf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4044040 0x7f19d416d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4e3817e640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300cd5a0 con 0x7f4e30161100 2024-04-25T08:29:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4e3671b640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e30161100 0x7f4e30155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 -- 172.21.15.60:0/2994317295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 msgr2=0x7f11340cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 --2- 172.21.15.60:0/2994317295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340cd1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f113415a340 tx=0x7f112802b380 comp rx=0 tx=0).stop 2024-04-25T08:29:44.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4053f90 con 0x7f19d4043bc0 2024-04-25T08:29:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7fae1f85d640 1 Processor -- start 2024-04-25T08:29:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7fae1f85d640 1 -- start start 2024-04-25T08:29:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f6fccdd4640 1 -- 172.21.15.60:0/2169513042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb8003520 con 0x7f6fb80039b0 2024-04-25T08:29:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.708+0000 7f6fcd5d5640 1 -- 172.21.15.60:0/2169513042 reap_dead start 2024-04-25T08:29:44.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19dbfff640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19d4043bc0 0x7f19d40456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e17c7640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4044040 0x7f19d416d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4053cb0 con 0x7f19cc0039b0 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4054270 con 0x7f19d4044040 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f6fc7fff640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fb8004800 0x7f6fb8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7ff660e38640 1 -- 172.21.15.60:0/2991620541 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c037070 con 0x7ff65c06bbf0 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7ff660e38640 1 -- 172.21.15.60:0/2991620541 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c032440 con 0x7ff65c06bbf0 2024-04-25T08:29:44.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7ff660e38640 1 -- 172.21.15.60:0/2991620541 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c032810 con 0x7ff65c06bbf0 2024-04-25T08:29:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 -- 172.21.15.60:0/2994317295 shutdown_connections 2024-04-25T08:29:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 --2- 172.21.15.60:0/2994317295 >> v2:172.21.15.100:3300/0 conn(0x7f113406bbf0 0x7f113406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 --2- 172.21.15.60:0/2994317295 >> v2:172.21.15.60:3300/0 conn(0x7f113406d570 0x7f11340b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 --2- 172.21.15.60:0/2994317295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340cd1a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 -- 172.21.15.60:0/2994317295 >> 172.21.15.60:0/2994317295 conn(0x7f11340717f0 msgr2=0x7f1134071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4e3817e640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e301622f0 con 0x7f4e300cfd80 2024-04-25T08:29:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 -- 172.21.15.60:0/2994317295 shutdown_connections 2024-04-25T08:29:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4f694b6640 1 -- 172.21.15.60:0/1137626559 shutdown_connections 2024-04-25T08:29:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4f694b6640 1 --2- 172.21.15.60:0/1137626559 >> v2:172.21.15.60:3300/0 conn(0x7f4f6416e0d0 0x7f4f640b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4f694b6640 1 --2- 172.21.15.60:0/1137626559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b5220 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4f694b6640 1 -- 172.21.15.60:0/1137626559 >> 172.21.15.60:0/1137626559 conn(0x7f4f6406adc0 msgr2=0x7f4f6406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4f694b6640 1 -- 172.21.15.60:0/1137626559 shutdown_connections 2024-04-25T08:29:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4f694b6640 1 -- 172.21.15.60:0/1137626559 wait complete. 2024-04-25T08:29:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f4e3817e640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e301614e0 con 0x7f4e300cc8b0 2024-04-25T08:29:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f113a415640 1 -- 172.21.15.60:0/2994317295 wait complete. 2024-04-25T08:29:44.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4044040 msgr2=0x7f19d416d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4044040 0x7f19d416d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3648017512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19d4043bc0 msgr2=0x7f19d40456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.709+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19d4043bc0 0x7f19d40456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1f85d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae1003e830 0x7fae100d7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f19e0fc6640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4049390 con 0x7f19cc0039b0 2024-04-25T08:29:44.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 -- 172.21.15.60:0/2991620541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 msgr2=0x7ff65c06c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2991620541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 0x7ff65c06c050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009870 tx=0x7ff64c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1f85d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae100542f0 0x7fae100e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 -- 172.21.15.60:0/2991620541 shutdown_connections 2024-04-25T08:29:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2991620541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c06bbf0 0x7ff65c06c050 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2991620541 >> v2:172.21.15.100:3300/0 conn(0x7ff65c161e50 0x7ff65c06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 -- 172.21.15.60:0/2991620541 >> 172.21.15.60:0/2991620541 conn(0x7ff65c071880 msgr2=0x7ff65c071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1f85d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae100e30b0 0x7fae100d8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f19e0fc6640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19d40e63d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19d002b420 tx=0x7f19d0003610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 -- 172.21.15.60:0/2991620541 shutdown_connections 2024-04-25T08:29:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e35719640 1 -- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e300cc8b0 msgr2=0x7f4e3014c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e35719640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e300cc8b0 0x7f4e3014c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e35719640 1 -- 172.21.15.60:0/4235478113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e30161100 msgr2=0x7f4e30155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e35719640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e30161100 0x7f4e30155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e35719640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e301f3e30 con 0x7f4e300cfd80 2024-04-25T08:29:44.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e3671b640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e30161100 0x7f4e30155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1d5f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae1003e830 0x7fae100d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1d5f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae1003e830 0x7fae100d7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59124/0 (socket says 172.21.15.60:59124) 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1f85d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae100e3490 con 0x7fae100e30b0 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1d5f9640 1 -- 172.21.15.60:0/3312234638 learned_addr learned my addr 172.21.15.60:0/3312234638 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1003bfb0 con 0x7fae100542f0 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae10006e00 con 0x7fae1003e830 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f4e35719640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e300cfd80 0x7f4e301531f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b490 tx=0x7f4e2c00b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1cdf8640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae100542f0 0x7fae100e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fb80039b0 0x7f6fb8003d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e310 tx=0x7f6fb8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b317ccecf2d7563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00038f0 con 0x7f19cc0039b0 2024-04-25T08:29:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00e350 con 0x7f6fb80039b0 2024-04-25T08:29:44.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d002b930 con 0x7f19cc0039b0 2024-04-25T08:29:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d002bce0 con 0x7f19cc0039b0 2024-04-25T08:29:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d4049670 con 0x7f19cc0039b0 2024-04-25T08:29:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7fae1ddfa640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae100e30b0 0x7fae100d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d4049c30 con 0x7f19cc0039b0 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.710+0000 7ff662e3c640 1 -- 172.21.15.60:0/2991620541 wait complete. 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae1cdf8640 1 -- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae1003e830 msgr2=0x7fae100d7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae1cdf8640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae1003e830 0x7fae100d7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae1cdf8640 1 -- 172.21.15.60:0/3312234638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae100e30b0 msgr2=0x7fae100d8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae1cdf8640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae100e30b0 0x7fae100d8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae1cdf8640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100e50a0 con 0x7fae100542f0 2024-04-25T08:29:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae1cdf8640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae100542f0 0x7fae100e4650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c400 tx=0x7fae0800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f4f694b6640 1 Processor -- start 2024-04-25T08:29:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f4f694b6640 1 -- start start 2024-04-25T08:29:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c015020 con 0x7f4e300cfd80 2024-04-25T08:29:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c011830 con 0x7f4e300cfd80 2024-04-25T08:29:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c011c20 con 0x7f4e300cfd80 2024-04-25T08:29:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08017070 con 0x7fae100542f0 2024-04-25T08:29:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae08005ce0 con 0x7fae100542f0 2024-04-25T08:29:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc031030 con 0x7f6fb80039b0 2024-04-25T08:29:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f6fc77fe640 1 -- 172.21.15.60:0/2169513042 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0058d0 con 0x7f6fb80039b0 2024-04-25T08:29:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7ff662e3c640 1 Processor -- start 2024-04-25T08:29:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7ff662e3c640 1 -- start start 2024-04-25T08:29:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f4f694b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 0x7f4f640aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d40089c0 con 0x7f19cc0039b0 2024-04-25T08:29:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f4f62ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 0x7f4f640aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f4f62ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 0x7f4f640aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41660/0 (socket says 172.21.15.60:41660) 2024-04-25T08:29:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f4f62ffd640 1 -- 172.21.15.60:0/4251175089 learned_addr learned my addr 172.21.15.60:0/4251175089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fb80039b0 msgr2=0x7f6fb8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae080072f0 con 0x7fae100542f0 2024-04-25T08:29:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fb80039b0 0x7f6fb8003d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00e310 tx=0x7f6fb8005830 comp rx=0 tx=0).stop 2024-04-25T08:29:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f6416e0d0 0x7f4f640b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6416ba20 con 0x7f4f640b3140 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6416b8a0 con 0x7f4f640cef20 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640712f0 con 0x7f4f6416e0d0 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f5bfff640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f637fe640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f6416e0d0 0x7f4f640b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 shutdown_connections 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/2169513042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fb80039b0 0x7f6fb8003d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 >> 172.21.15.60:0/2169513042 conn(0x7f6fc8061c60 msgr2=0x7f6fc806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4f5bfff640 1 -- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f6416e0d0 msgr2=0x7f4f640b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.711+0000 7f113a415640 1 Processor -- start 2024-04-25T08:29:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.712+0000 7f113a415640 1 -- start start 2024-04-25T08:29:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff662e3c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 0x7ff65c149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4e3817e640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e30153730 con 0x7f4e300cfd80 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f113a415640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f113406bbf0 0x7f11341490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f113a415640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f113406d570 0x7f11340c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f113a415640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f113a415640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134143a90 con 0x7f113406bbf0 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f113a415640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113406e5b0 con 0x7f113406d570 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f113a415640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340b3ab0 con 0x7f11341436b0 2024-04-25T08:29:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f11389b2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f1133fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f113406bbf0 0x7f11341490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f4e3817e640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e301f43b0 con 0x7f4e300cfd80 2024-04-25T08:29:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f11389b2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59146/0 (socket says 172.21.15.60:59146) 2024-04-25T08:29:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f11389b2640 1 -- 172.21.15.60:0/2428754221 learned_addr learned my addr 172.21.15.60:0/2428754221 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae100e5380 con 0x7fae100542f0 2024-04-25T08:29:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff662e3c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c0b3a20 0x7ff65c0cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f11389b2640 1 -- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f113406d570 msgr2=0x7f11340c65f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:29:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f11389b2640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f113406d570 0x7f11340c65f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f11389b2640 1 -- 172.21.15.60:0/2428754221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f113406bbf0 msgr2=0x7f11341490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f11389b2640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f113406bbf0 0x7f11341490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f11389b2640 1 -- 172.21.15.60:0/2428754221 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340af480 con 0x7f11341436b0 2024-04-25T08:29:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff661e3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 0x7ff65c149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff661e3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 0x7ff65c149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41682/0 (socket says 172.21.15.60:41682) 2024-04-25T08:29:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f4f5bfff640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f6416e0d0 0x7f4f640b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f4f5bfff640 1 -- 172.21.15.60:0/4251175089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 msgr2=0x7f4f640aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f4f5bfff640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 0x7f4f640aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f4f5bfff640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640a8a90 con 0x7f4f640cef20 2024-04-25T08:29:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae100e5940 con 0x7fae100542f0 2024-04-25T08:29:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff661e3a640 1 -- 172.21.15.60:0/2649893203 learned_addr learned my addr 172.21.15.60:0/2649893203 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff65c161e50 0x7ff65c14a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0b4590 con 0x7ff65c06bbf0 2024-04-25T08:29:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.713+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19d0004620 con 0x7f19cc0039b0 2024-04-25T08:29:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f19d97fa640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f19c006b5a0 0x7f19c006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f19d00aa360 con 0x7f19cc0039b0 2024-04-25T08:29:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f4f62ffd640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 0x7f4f640aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f11389b2640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340aef40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f112c00cd80 tx=0x7f112c00a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff66263b640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff65c161e50 0x7ff65c14a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7f4f5bfff640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b3570 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00d710 tx=0x7f4f4c00dbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0b4db0 con 0x7ff65c161e50 2024-04-25T08:29:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.714+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0ad250 con 0x7ff65c0b3a20 2024-04-25T08:29:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 shutdown_connections 2024-04-25T08:29:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00f680 con 0x7f11341436b0 2024-04-25T08:29:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661639640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c0b3a20 0x7ff65c0cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c00f820 con 0x7f11341436b0 2024-04-25T08:29:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f19dbfff640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f19c006b5a0 0x7f19c006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00f680 con 0x7f11341436b0 2024-04-25T08:29:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1134149790 con 0x7f11341436b0 2024-04-25T08:29:44.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f6fcddd6640 1 -- 172.21.15.60:0/2169513042 wait complete. 2024-04-25T08:29:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e300ad850 con 0x7f4e300cfd80 2024-04-25T08:29:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661639640 1 -- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff65c161e50 msgr2=0x7ff65c14a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661639640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff65c161e50 0x7ff65c14a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661639640 1 -- 172.21.15.60:0/2649893203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 msgr2=0x7ff65c149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1134149d50 con 0x7f11341436b0 2024-04-25T08:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661639640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 0x7ff65c149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661639640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c149ff0 con 0x7ff65c0b3a20 2024-04-25T08:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.715+0000 7ff661e3a640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 0x7ff65c149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7ff661639640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c0b3a20 0x7ff65c0cf270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff64c031000 tx=0x7ff64c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e2c018020 con 0x7f4e300cfd80 2024-04-25T08:29:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7f19dbfff640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f19c006b5a0 0x7f19c006da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19d40059c0 tx=0x7f19c8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae100d9b80 con 0x7fae100542f0 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7f4e267fc640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4e1406b5a0 0x7f4e1406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4e2c08a2d0 con 0x7f4e300cfd80 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c041070 con 0x7ff65c0b3a20 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c03c930 con 0x7ff65c0b3a20 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c037040 con 0x7ff65c0b3a20 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c0ce4b0 con 0x7ff65c0b3a20 2024-04-25T08:29:44.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.716+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c0cea70 con 0x7ff65c0b3a20 2024-04-25T08:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134073990 con 0x7f11341436b0 2024-04-25T08:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7f4e35f1a640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4e1406b5a0 0x7f4e1406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7f6fcddd6640 1 Processor -- start 2024-04-25T08:29:44.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7f6fcddd6640 1 -- start start 2024-04-25T08:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c0ccd50 con 0x7ff65c0b3a20 2024-04-25T08:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.717+0000 7f4e35f1a640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4e1406b5a0 0x7f4e1406da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097a0 tx=0x7f4e2002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f112c00f9d0 con 0x7f11341436b0 2024-04-25T08:29:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff64c044070 con 0x7ff65c0b3a20 2024-04-25T08:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f1130ff9640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f112006b900 0x7f112006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fcddd6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80039b0 0x7f6fc814f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fcddd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 0x7f6fc8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fcddd6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fc81f3250 0x7f6fc815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fcddd6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc815d1a0 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fcddd6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc815e310 con 0x7f6fc81f3250 2024-04-25T08:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fcddd6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc806f100 con 0x7f6fb80039b0 2024-04-25T08:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f112c00f480 con 0x7f11341436b0 2024-04-25T08:29:44.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f1133fff640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f112006b900 0x7f112006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 0x7f6fc8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 0x7f6fc8158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41692/0 (socket says 172.21.15.60:41692) 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.718+0000 7f6fc7fff640 1 -- 172.21.15.60:0/3378216665 learned_addr learned my addr 172.21.15.60:0/3378216665 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fcd5d5640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fc81f3250 0x7f6fc815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80039b0 0x7f6fc814f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7ff6527fc640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff64006b4d0 0x7ff64006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19d0078830 con 0x7f19cc0039b0 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff64c0ab220 con 0x7ff65c0b3a20 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7ff661e3a640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff64006b4d0 0x7ff64006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc7fff640 1 -- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80039b0 msgr2=0x7f6fc814f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc7fff640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80039b0 0x7f6fc814f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc7fff640 1 -- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fc81f3250 msgr2=0x7f6fc815ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc7fff640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fc81f3250 0x7f6fc815ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:44.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc7fff640 1 -- 172.21.15.60:0/3378216665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc81efcd0 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c004230 con 0x7f4f640cef20 2024-04-25T08:29:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640a8d70 con 0x7f4f640cef20 2024-04-25T08:29:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f1133fff640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f112006b900 0x7f112006ddc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1128035000 tx=0x7f112802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640a9330 con 0x7f4f640cef20 2024-04-25T08:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc7fff640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 0x7f6fc8158d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40114e0 tx=0x7f6fb40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.719+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4015870 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.720+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc81effb0 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.720+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb4015a10 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.720+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4015e00 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.720+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc81f2d30 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.720+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae0801a070 con 0x7fae100542f0 2024-04-25T08:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.720+0000 7ff661e3a640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff64006b4d0 0x7ff64006d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d70 tx=0x7ff6580090e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.721+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6406bb70 con 0x7f4f640cef20 2024-04-25T08:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.721+0000 7fae0dffb640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadfc06b550 0x7fadfc06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.721+0000 7fae1d5f9640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadfc06b550 0x7fadfc06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.721+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc8073990 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.722+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fb401d400 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.722+0000 7fae1d5f9640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadfc06b550 0x7fadfc06da10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae180690f0 tx=0x7fae1808c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.722+0000 7f6fc57fa640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fac06b4d0 0x7f6fac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.722+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6fb408ac50 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.722+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fac06b4d0 0x7f6fac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.723+0000 7f6fccdd4640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fac06b4d0 0x7f6fac06d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc035000 tx=0x7f6fbc00a070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.723+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f112c014030 con 0x7f11341436b0 2024-04-25T08:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.724+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae08056140 con 0x7fae100542f0 2024-04-25T08:29:44.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.724+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e300ad850 con 0x7f4e300cfd80 2024-04-25T08:29:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.724+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c00ae50 con 0x7f4f640cef20 2024-04-25T08:29:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.724+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00b820 con 0x7f4f640cef20 2024-04-25T08:29:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.728+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff64c079700 con 0x7ff65c0b3a20 2024-04-25T08:29:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.728+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae100d9b80 con 0x7fae100542f0 2024-04-25T08:29:44.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.729+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fb4090020 con 0x7f6fc81ef4a0 2024-04-25T08:29:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.730+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f4c012070 con 0x7f4f640cef20 2024-04-25T08:29:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.731+0000 7f4f5b7fe640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4f4406b4d0 0x7f4f4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.731+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4f4c08aa60 con 0x7f4f640cef20 2024-04-25T08:29:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.731+0000 7f4f62ffd640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4f4406b4d0 0x7f4f4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.732+0000 7f4f62ffd640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4f4406b4d0 0x7f4f4406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f54010950 tx=0x7f4f54033000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:44.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:44.733+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f4c021070 con 0x7f4f640cef20 2024-04-25T08:29:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.068+0000 7fd25443f640 1 -- 172.21.15.60:0/1436108676 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd234005580 con 0x7fd24c153de0 2024-04-25T08:29:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.069+0000 7fd23effd640 1 -- 172.21.15.60:0/1436108676 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd244079510 con 0x7fd24c153de0 2024-04-25T08:29:45.066 INFO:teuthology.orchestra.run.smithi060.stdout:60129542154 2024-04-25T08:29:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 -- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd23806b640 msgr2=0x7fd23806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd23806b640 0x7fd23806db00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd240006b10 tx=0x7fd240008040 comp rx=0 tx=0).stop 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 -- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 msgr2=0x7fd24c0affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 0x7fd24c0affd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d120 tx=0x7fd244004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 -- 172.21.15.60:0/1436108676 shutdown_connections 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd23806b640 0x7fd23806db00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.072+0000 7fd23cff9640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd24c153de0 0x7fd24c0affd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.073+0000 7fd23cff9640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd24c0acd80 0x7fd24c1524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.073+0000 7fd23cff9640 1 --2- 172.21.15.60:0/1436108676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd24c06bbf0 0x7fd24c1495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.073+0000 7fd23cff9640 1 -- 172.21.15.60:0/1436108676 >> 172.21.15.60:0/1436108676 conn(0x7fd24c071af0 msgr2=0x7fd24c0ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.073+0000 7fd23cff9640 1 -- 172.21.15.60:0/1436108676 shutdown_connections 2024-04-25T08:29:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.073+0000 7fd23cff9640 1 -- 172.21.15.60:0/1436108676 wait complete. 2024-04-25T08:29:45.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.7 2024-04-25T08:29:45.087 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.113+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6fc806d500 con 0x7f6fc81ef4a0 2024-04-25T08:29:45.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.113+0000 7f6fc57fa640 1 -- 172.21.15.60:0/3378216665 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6fb4019030 con 0x7f6fc81ef4a0 2024-04-25T08:29:45.110 INFO:teuthology.orchestra.run.smithi060.stdout:60129542154 2024-04-25T08:29:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fac06b4d0 msgr2=0x7f6fac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fac06b4d0 0x7f6fac06d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc035000 tx=0x7f6fbc00a070 comp rx=0 tx=0).stop 2024-04-25T08:29:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 msgr2=0x7f6fc8158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 0x7f6fc8158d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40114e0 tx=0x7f6fb40119a0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 shutdown_connections 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fac06b4d0 0x7f6fac06d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fc81f3250 0x7f6fc815ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fc81ef4a0 0x7f6fc8158d90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 --2- 172.21.15.60:0/3378216665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6fb80039b0 0x7f6fc814f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 >> 172.21.15.60:0/3378216665 conn(0x7f6fc8061c60 msgr2=0x7f6fc806d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 shutdown_connections 2024-04-25T08:29:45.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.116+0000 7f6fcddd6640 1 -- 172.21.15.60:0/3378216665 wait complete. 2024-04-25T08:29:45.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.6 2024-04-25T08:29:45.124 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.148+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f19d4005380 con 0x7f19cc0039b0 2024-04-25T08:29:45.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.149+0000 7f19d97fa640 1 -- 172.21.15.60:0/3648017512 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f19d007c8d0 con 0x7f19cc0039b0 2024-04-25T08:29:45.147 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f19c006b5a0 msgr2=0x7f19c006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f19c006b5a0 0x7f19c006da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f19d40059c0 tx=0x7f19c8009190 comp rx=0 tx=0).stop 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 msgr2=0x7f19d40e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19d40e63d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19d002b420 tx=0x7f19d0003610 comp rx=0 tx=0).stop 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 shutdown_connections 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19d4044040 0x7f19d416d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f19d4043bc0 0x7f19d40456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff65c0b1710 con 0x7ff65c0b3a20 2024-04-25T08:29:45.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f19c006b5a0 0x7f19c006da60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 --2- 172.21.15.60:0/3648017512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19cc0039b0 0x7f19d40e63d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 >> 172.21.15.60:0/3648017512 conn(0x7f19d4009fa0 msgr2=0x7f19d40e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 shutdown_connections 2024-04-25T08:29:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7f19e322a640 1 -- 172.21.15.60:0/3648017512 wait complete. 2024-04-25T08:29:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.152+0000 7ff6527fc640 1 -- 172.21.15.60:0/2649893203 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff64c07d7a0 con 0x7ff65c0b3a20 2024-04-25T08:29:45.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.153+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4e301f4d40 con 0x7f4e300cfd80 2024-04-25T08:29:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:55834574856 2024-04-25T08:29:45.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.153+0000 7f4e267fc640 1 -- 172.21.15.60:0/4235478113 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e2c01f020 con 0x7f4e300cfd80 2024-04-25T08:29:45.153 INFO:teuthology.orchestra.run.smithi060.stdout:55834574857 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff64006b4d0 msgr2=0x7ff64006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff64006b4d0 0x7ff64006d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d70 tx=0x7ff6580090e0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c0b3a20 msgr2=0x7ff65c0cf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c0b3a20 0x7ff65c0cf270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff64c031000 tx=0x7ff64c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 shutdown_connections 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff64006b4d0 0x7ff64006d990 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d70 tx=0x7ff6580090e0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff65c161e50 0x7ff65c14a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff65c0b3a20 0x7ff65c0cf270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 --2- 172.21.15.60:0/2649893203 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff65c06bbf0 0x7ff65c149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 >> 172.21.15.60:0/2649893203 conn(0x7ff65c071880 msgr2=0x7ff65c1605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 shutdown_connections 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7ff662e3c640 1 -- 172.21.15.60:0/2649893203 wait complete. 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.155+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11340b40b0 con 0x7f11341436b0 2024-04-25T08:29:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.156+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4f6406ee20 con 0x7f4f640cef20 2024-04-25T08:29:45.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.156+0000 7f1130ff9640 1 -- 172.21.15.60:0/2428754221 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f112c058dc0 con 0x7f11341436b0 2024-04-25T08:29:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:55834574857 2024-04-25T08:29:45.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.156+0000 7f4f5b7fe640 1 -- 172.21.15.60:0/4251175089 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f4c058f30 con 0x7f4f640cef20 2024-04-25T08:29:45.156 INFO:teuthology.orchestra.run.smithi060.stdout:55834574857 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.156+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4e1406b5a0 msgr2=0x7f4e1406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4e1406b5a0 0x7f4e1406da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097a0 tx=0x7f4e2002e000 comp rx=0 tx=0).stop 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e300cfd80 msgr2=0x7f4e301531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e300cfd80 0x7f4e301531f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00b490 tx=0x7f4e2c00b950 comp rx=0 tx=0).stop 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 shutdown_connections 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e30161100 0x7f4e30155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4e1406b5a0 0x7f4e1406da60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e300cfd80 0x7f4e301531f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 --2- 172.21.15.60:0/4235478113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4e300cc8b0 0x7f4e3014c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 >> 172.21.15.60:0/4235478113 conn(0x7f4e3006abe0 msgr2=0x7f4e300ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 shutdown_connections 2024-04-25T08:29:45.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.157+0000 7f4e13fff640 1 -- 172.21.15.60:0/4235478113 wait complete. 2024-04-25T08:29:45.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f112006b900 msgr2=0x7f112006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fae100057d0 con 0x7fae100542f0 2024-04-25T08:29:45.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4f4406b4d0 msgr2=0x7f4f4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4f4406b4d0 0x7f4f4406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f54010950 tx=0x7f4f54033000 comp rx=0 tx=0).stop 2024-04-25T08:29:45.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 msgr2=0x7f4f640b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b3570 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00d710 tx=0x7f4f4c00dbd0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f113a415640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f112006b900 0x7f112006ddc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1128035000 tx=0x7f112802f000 comp rx=0 tx=0).stop 2024-04-25T08:29:45.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 msgr2=0x7f11340aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.159+0000 7f113a415640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340aef40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f112c00cd80 tx=0x7f112c00a650 comp rx=0 tx=0).stop 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 shutdown_connections 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11341436b0 0x7f11340aef40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f112006b900 0x7f112006ddc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f113406d570 0x7f11340c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 --2- 172.21.15.60:0/2428754221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f113406bbf0 0x7f11341490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 >> 172.21.15.60:0/2428754221 conn(0x7f11340717f0 msgr2=0x7f1134142890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 shutdown_connections 2024-04-25T08:29:45.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 shutdown_connections 2024-04-25T08:29:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4f6416e0d0 0x7f4f640b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4f4406b4d0 0x7f4f4406d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f640cef20 0x7f4f640b3570 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f113a415640 1 -- 172.21.15.60:0/2428754221 wait complete. 2024-04-25T08:29:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 --2- 172.21.15.60:0/4251175089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f640b3140 0x7f4f640aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 >> 172.21.15.60:0/4251175089 conn(0x7f4f6406adc0 msgr2=0x7f4f6406b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 shutdown_connections 2024-04-25T08:29:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7f4f694b6640 1 -- 172.21.15.60:0/4251175089 wait complete. 2024-04-25T08:29:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.160+0000 7fae0dffb640 1 -- 172.21.15.60:0/3312234638 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae08021020 con 0x7fae100542f0 2024-04-25T08:29:45.163 INFO:teuthology.orchestra.run.smithi060.stdout:60129542154 2024-04-25T08:29:45.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadfc06b550 msgr2=0x7fadfc06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadfc06b550 0x7fadfc06da10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae180690f0 tx=0x7fae1808c040 comp rx=0 tx=0).stop 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae100542f0 msgr2=0x7fae100e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae100542f0 0x7fae100e4650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c400 tx=0x7fae0800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 shutdown_connections 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae100e30b0 0x7fae100d8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadfc06b550 0x7fadfc06da10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae100542f0 0x7fae100e4650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 --2- 172.21.15.60:0/3312234638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae1003e830 0x7fae100d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 >> 172.21.15.60:0/3312234638 conn(0x7fae100559d0 msgr2=0x7fae10055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 shutdown_connections 2024-04-25T08:29:45.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.163+0000 7fae1f85d640 1 -- 172.21.15.60:0/3312234638 wait complete. 2024-04-25T08:29:45.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-04-25T08:29:45.170 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-25T08:29:45.171 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-25T08:29:45.176 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.5 2024-04-25T08:29:45.176 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-25T08:29:45.179 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-25T08:29:45.179 DEBUG:teuthology.parallel:result is None 2024-04-25T08:29:45.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:29:45.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:29:45.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.322+0000 7f973d37e640 1 Processor -- start 2024-04-25T08:29:45.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.322+0000 7f973d37e640 1 -- start start 2024-04-25T08:29:45.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f973d37e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f97380d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f973d37e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380b97a0 con 0x7f97380b2cb0 2024-04-25T08:29:45.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f973d37e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380d1c50 con 0x7f973815dbe0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f973d37e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973815dfc0 con 0x7f97380d1020 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f972ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f97380b2cb0 0x7f97380d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42082/0 (socket says 172.21.15.60:42082) 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f972ffff640 1 -- 172.21.15.60:0/3265766393 learned_addr learned my addr 172.21.15.60:0/3265766393 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97377fe640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f97380d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258007611 0 0) 0x7f97380b97a0 con 0x7f97380b2cb0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9724003520 con 0x7f97380b2cb0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477501339 0 0) 0x7f97380d1c50 con 0x7f973815dbe0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97380b97a0 con 0x7f973815dbe0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886918042 0 0) 0x7f9724003520 con 0x7f97380b2cb0 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 msgr2=0x7f97380d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f97380d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 >> v1:172.21.15.100:6789/0 conn(0x7f973815dbe0 legacy=0x7f97380d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380d1f30 con 0x7f97380b2cb0 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f9720001670 con 0x7f97380b2cb0 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 >> v1:172.21.15.60:6789/0 conn(0x7f97380b2cb0 legacy=0x7f97380d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724003b70 0x7f9724003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f9724004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9724004d40 0x7f97240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972400ba70 con 0x7f9724004d40 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972400bc20 con 0x7f9724003b70 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.323+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972400be20 con 0x7f97380d1020 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9724004d40 0x7f97240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f9736ffd640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724003b70 0x7f9724003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f972ffff640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f9724004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 -- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 msgr2=0x7f9724004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f9724004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 -- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724003b70 msgr2=0x7f9724003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724003b70 0x7f9724003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 -- 172.21.15.60:0/3265766393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9724003520 con 0x7f9724004d40 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f972ffff640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97380d1020 0x7f9724004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.324+0000 7f97377fe640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9724004d40 0x7f97240051c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f90 tx=0x7f9724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c172a109974045 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730009c10 con 0x7f9724004d40 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730036070 con 0x7f9724004d40 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f97367fc640 1 -- 172.21.15.60:0/3265766393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730031850 con 0x7f9724004d40 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f973d37e640 1 -- 172.21.15.60:0/3265766393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9724004d40 msgr2=0x7f97240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f973d37e640 1 --2- 172.21.15.60:0/3265766393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9724004d40 0x7f97240051c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f90 tx=0x7f9724005830 comp rx=0 tx=0).stop 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f97377fe640 1 -- 172.21.15.60:0/3265766393 reap_dead start 2024-04-25T08:29:45.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f973d37e640 1 -- 172.21.15.60:0/3265766393 shutdown_connections 2024-04-25T08:29:45.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f973d37e640 1 -- 172.21.15.60:0/3265766393 >> 172.21.15.60:0/3265766393 conn(0x7f9738061a20 msgr2=0x7f9738061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f973d37e640 1 -- 172.21.15.60:0/3265766393 shutdown_connections 2024-04-25T08:29:45.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.325+0000 7f973d37e640 1 -- 172.21.15.60:0/3265766393 wait complete. 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.326+0000 7f973d37e640 1 Processor -- start 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 -- start start 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724004d40 0x7f97380c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97381538f0 0x7f9738153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 0x7f97380cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380b92a0 con 0x7f97380cc450 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97380d1a00 con 0x7f9724004d40 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f973d37e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380b9420 con 0x7f97381538f0 2024-04-25T08:29:45.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f97377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 0x7f97380cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f97377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 0x7f97380cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41696/0 (socket says 172.21.15.60:41696) 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f97377fe640 1 -- 172.21.15.60:0/2855424268 learned_addr learned my addr 172.21.15.60:0/2855424268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.327+0000 7f972ffff640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97381538f0 0x7f9738153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f9736ffd640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724004d40 0x7f97380c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f97377fe640 1 -- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97381538f0 msgr2=0x7f9738153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f97377fe640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97381538f0 0x7f9738153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f97377fe640 1 -- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724004d40 msgr2=0x7f97380c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f97377fe640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724004d40 0x7f97380c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f97377fe640 1 -- 172.21.15.60:0/2855424268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380c9a10 con 0x7f97380cc450 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f97377fe640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 0x7f97380cc8d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f90 tx=0x7f9730009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730009c10 con 0x7f97380cc450 2024-04-25T08:29:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730003990 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97380cd030 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730003d80 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.328+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97380b3510 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.329+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9738073990 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.329+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9730005d10 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.330+0000 7f972f7fe640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.330+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f97300abd50 con 0x7f97380cc450 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.330+0000 7f9736ffd640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.330+0000 7f9736ffd640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f971806b4d0 0x7f971806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9728000c00 tx=0x7f9728009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.334+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f973004a060 con 0x7f97380cc450 2024-04-25T08:29:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.511+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97380cf700 con 0x7f971806b4d0 2024-04-25T08:29:45.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.519+0000 7f972f7fe640 1 -- 172.21.15.60:0/2855424268 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f97380cf700 con 0x7f971806b4d0 2024-04-25T08:29:45.521 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:45.523 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:29:45.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f971806b4d0 msgr2=0x7f971806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f971806b4d0 0x7f971806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9728000c00 tx=0x7f9728009040 comp rx=0 tx=0).stop 2024-04-25T08:29:45.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 msgr2=0x7f97380cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 0x7f97380cc8d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f90 tx=0x7f9730009750 comp rx=0 tx=0).stop 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 shutdown_connections 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97380cc450 0x7f97380cc8d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97381538f0 0x7f9738153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 --2- 172.21.15.60:0/2855424268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724004d40 0x7f97380c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 >> 172.21.15.60:0/2855424268 conn(0x7f9738061a20 msgr2=0x7f97380d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 shutdown_connections 2024-04-25T08:29:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.523+0000 7f973d37e640 1 -- 172.21.15.60:0/2855424268 wait complete. 2024-04-25T08:29:45.532 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T08:29:45.401871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231920,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742800,"statfs":{"total":773094113280,"available":772856627200,"internally_reserved":0,"allocated":3350528,"data_stored":2022712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003492"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.028601+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.028315+0000","last_became_peered":"2024-04-25T08:29:23.028315+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075712+0000","last_change":"2024-04-25T08:29:24.048532+0000","last_active":"2024-04-25T08:29:29.075712+0000","last_peered":"2024-04-25T08:29:29.075712+0000","last_clean":"2024-04-25T08:29:29.075712+0000","last_became_active":"2024-04-25T08:29:24.046997+0000","last_became_peered":"2024-04-25T08:29:24.046997+0000","last_unstale":"2024-04-25T08:29:29.075712+0000","last_undegraded":"2024-04-25T08:29:29.075712+0000","last_fullsized":"2024-04-25T08:29:29.075712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074646+0000","last_change":"2024-04-25T08:29:28.085554+0000","last_active":"2024-04-25T08:29:29.074646+0000","last_peered":"2024-04-25T08:29:29.074646+0000","last_clean":"2024-04-25T08:29:29.074646+0000","last_became_active":"2024-04-25T08:29:28.085164+0000","last_became_peered":"2024-04-25T08:29:28.085164+0000","last_unstale":"2024-04-25T08:29:29.074646+0000","last_undegraded":"2024-04-25T08:29:29.074646+0000","last_fullsized":"2024-04-25T08:29:29.074646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:03.849914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075840+0000","last_change":"2024-04-25T08:29:23.035840+0000","last_active":"2024-04-25T08:29:29.075840+0000","last_peered":"2024-04-25T08:29:29.075840+0000","last_clean":"2024-04-25T08:29:29.075840+0000","last_became_active":"2024-04-25T08:29:23.035532+0000","last_became_peered":"2024-04-25T08:29:23.035532+0000","last_unstale":"2024-04-25T08:29:29.075840+0000","last_undegraded":"2024-04-25T08:29:29.075840+0000","last_fullsized":"2024-04-25T08:29:29.075840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906355+0000","last_change":"2024-04-25T08:29:24.037371+0000","last_active":"2024-04-25T08:29:29.906355+0000","last_peered":"2024-04-25T08:29:29.906355+0000","last_clean":"2024-04-25T08:29:29.906355+0000","last_became_active":"2024-04-25T08:29:24.037043+0000","last_became_peered":"2024-04-25T08:29:24.037043+0000","last_unstale":"2024-04-25T08:29:29.906355+0000","last_undegraded":"2024-04-25T08:29:29.906355+0000","last_fullsized":"2024-04-25T08:29:29.906355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.900465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074732+0000","last_change":"2024-04-25T08:29:28.080997+0000","last_active":"2024-04-25T08:29:29.074732+0000","last_peered":"2024-04-25T08:29:29.074732+0000","last_clean":"2024-04-25T08:29:29.074732+0000","last_became_active":"2024-04-25T08:29:28.080573+0000","last_became_peered":"2024-04-25T08:29:28.080573+0000","last_unstale":"2024-04-25T08:29:29.074732+0000","last_undegraded":"2024-04-25T08:29:29.074732+0000","last_fullsized":"2024-04-25T08:29:29.074732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:48.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895961+0000","last_change":"2024-04-25T08:29:23.041517+0000","last_active":"2024-04-25T08:29:29.895961+0000","last_peered":"2024-04-25T08:29:29.895961+0000","last_clean":"2024-04-25T08:29:29.895961+0000","last_became_active":"2024-04-25T08:29:23.041191+0000","last_became_peered":"2024-04-25T08:29:23.041191+0000","last_unstale":"2024-04-25T08:29:29.895961+0000","last_undegraded":"2024-04-25T08:29:29.895961+0000","last_fullsized":"2024-04-25T08:29:29.895961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574369+0000","last_change":"2024-04-25T08:29:24.042793+0000","last_active":"2024-04-25T08:29:29.574369+0000","last_peered":"2024-04-25T08:29:29.574369+0000","last_clean":"2024-04-25T08:29:29.574369+0000","last_became_active":"2024-04-25T08:29:24.042026+0000","last_became_peered":"2024-04-25T08:29:24.042026+0000","last_unstale":"2024-04-25T08:29:29.574369+0000","last_undegraded":"2024-04-25T08:29:29.574369+0000","last_fullsized":"2024-04-25T08:29:29.574369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075189+0000","last_change":"2024-04-25T08:29:28.081365+0000","last_active":"2024-04-25T08:29:29.075189+0000","last_peered":"2024-04-25T08:29:29.075189+0000","last_clean":"2024-04-25T08:29:29.075189+0000","last_became_active":"2024-04-25T08:29:28.080901+0000","last_became_peered":"2024-04-25T08:29:28.080901+0000","last_unstale":"2024-04-25T08:29:29.075189+0000","last_undegraded":"2024-04-25T08:29:29.075189+0000","last_fullsized":"2024-04-25T08:29:29.075189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.649430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895778+0000","last_change":"2024-04-25T08:29:23.036549+0000","last_active":"2024-04-25T08:29:29.895778+0000","last_peered":"2024-04-25T08:29:29.895778+0000","last_clean":"2024-04-25T08:29:29.895778+0000","last_became_active":"2024-04-25T08:29:23.036323+0000","last_became_peered":"2024-04-25T08:29:23.036323+0000","last_unstale":"2024-04-25T08:29:29.895778+0000","last_undegraded":"2024-04-25T08:29:29.895778+0000","last_fullsized":"2024-04-25T08:29:29.895778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574394+0000","last_change":"2024-04-25T08:29:24.041522+0000","last_active":"2024-04-25T08:29:29.574394+0000","last_peered":"2024-04-25T08:29:29.574394+0000","last_clean":"2024-04-25T08:29:29.574394+0000","last_became_active":"2024-04-25T08:29:24.041242+0000","last_became_peered":"2024-04-25T08:29:24.041242+0000","last_unstale":"2024-04-25T08:29:29.574394+0000","last_undegraded":"2024-04-25T08:29:29.574394+0000","last_fullsized":"2024-04-25T08:29:29.574394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075282+0000","last_change":"2024-04-25T08:29:28.081585+0000","last_active":"2024-04-25T08:29:29.075282+0000","last_peered":"2024-04-25T08:29:29.075282+0000","last_clean":"2024-04-25T08:29:29.075282+0000","last_became_active":"2024-04-25T08:29:28.080227+0000","last_became_peered":"2024-04-25T08:29:28.080227+0000","last_unstale":"2024-04-25T08:29:29.075282+0000","last_undegraded":"2024-04-25T08:29:29.075282+0000","last_fullsized":"2024-04-25T08:29:29.075282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075044+0000","last_change":"2024-04-25T08:29:24.040005+0000","last_active":"2024-04-25T08:29:29.075044+0000","last_peered":"2024-04-25T08:29:29.075044+0000","last_clean":"2024-04-25T08:29:29.075044+0000","last_became_active":"2024-04-25T08:29:24.039678+0000","last_became_peered":"2024-04-25T08:29:24.039678+0000","last_unstale":"2024-04-25T08:29:29.075044+0000","last_undegraded":"2024-04-25T08:29:29.075044+0000","last_fullsized":"2024-04-25T08:29:29.075044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895838+0000","last_change":"2024-04-25T08:29:23.034031+0000","last_active":"2024-04-25T08:29:29.895838+0000","last_peered":"2024-04-25T08:29:29.895838+0000","last_clean":"2024-04-25T08:29:29.895838+0000","last_became_active":"2024-04-25T08:29:23.033672+0000","last_became_peered":"2024-04-25T08:29:23.033672+0000","last_unstale":"2024-04-25T08:29:29.895838+0000","last_undegraded":"2024-04-25T08:29:29.895838+0000","last_fullsized":"2024-04-25T08:29:29.895838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076035+0000","last_change":"2024-04-25T08:29:28.067431+0000","last_active":"2024-04-25T08:29:29.076035+0000","last_peered":"2024-04-25T08:29:29.076035+0000","last_clean":"2024-04-25T08:29:29.076035+0000","last_became_active":"2024-04-25T08:29:28.066968+0000","last_became_peered":"2024-04-25T08:29:28.066968+0000","last_unstale":"2024-04-25T08:29:29.076035+0000","last_undegraded":"2024-04-25T08:29:29.076035+0000","last_fullsized":"2024-04-25T08:29:29.076035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075071+0000","last_change":"2024-04-25T08:29:24.052628+0000","last_active":"2024-04-25T08:29:29.075071+0000","last_peered":"2024-04-25T08:29:29.075071+0000","last_clean":"2024-04-25T08:29:29.075071+0000","last_became_active":"2024-04-25T08:29:24.052111+0000","last_became_peered":"2024-04-25T08:29:24.052111+0000","last_unstale":"2024-04-25T08:29:29.075071+0000","last_undegraded":"2024-04-25T08:29:29.075071+0000","last_fullsized":"2024-04-25T08:29:29.075071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074777+0000","last_change":"2024-04-25T08:29:23.040432+0000","last_active":"2024-04-25T08:29:30.074777+0000","last_peered":"2024-04-25T08:29:30.074777+0000","last_clean":"2024-04-25T08:29:30.074777+0000","last_became_active":"2024-04-25T08:29:23.040172+0000","last_became_peered":"2024-04-25T08:29:23.040172+0000","last_unstale":"2024-04-25T08:29:30.074777+0000","last_undegraded":"2024-04-25T08:29:30.074777+0000","last_fullsized":"2024-04-25T08:29:30.074777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075200+0000","last_change":"2024-04-25T08:29:28.081025+0000","last_active":"2024-04-25T08:29:29.075200+0000","last_peered":"2024-04-25T08:29:29.075200+0000","last_clean":"2024-04-25T08:29:29.075200+0000","last_became_active":"2024-04-25T08:29:28.080481+0000","last_became_peered":"2024-04-25T08:29:28.080481+0000","last_unstale":"2024-04-25T08:29:29.075200+0000","last_undegraded":"2024-04-25T08:29:29.075200+0000","last_fullsized":"2024-04-25T08:29:29.075200+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074861+0000","last_change":"2024-04-25T08:29:24.054299+0000","last_active":"2024-04-25T08:29:29.074861+0000","last_peered":"2024-04-25T08:29:29.074861+0000","last_clean":"2024-04-25T08:29:29.074861+0000","last_became_active":"2024-04-25T08:29:24.053996+0000","last_became_peered":"2024-04-25T08:29:24.053996+0000","last_unstale":"2024-04-25T08:29:29.074861+0000","last_undegraded":"2024-04-25T08:29:29.074861+0000","last_fullsized":"2024-04-25T08:29:29.074861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074526+0000","last_change":"2024-04-25T08:29:23.038382+0000","last_active":"2024-04-25T08:29:30.074526+0000","last_peered":"2024-04-25T08:29:30.074526+0000","last_clean":"2024-04-25T08:29:30.074526+0000","last_became_active":"2024-04-25T08:29:23.038060+0000","last_became_peered":"2024-04-25T08:29:23.038060+0000","last_unstale":"2024-04-25T08:29:30.074526+0000","last_undegraded":"2024-04-25T08:29:30.074526+0000","last_fullsized":"2024-04-25T08:29:30.074526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075123+0000","last_change":"2024-04-25T08:29:28.085260+0000","last_active":"2024-04-25T08:29:30.075123+0000","last_peered":"2024-04-25T08:29:30.075123+0000","last_clean":"2024-04-25T08:29:30.075123+0000","last_became_active":"2024-04-25T08:29:28.084884+0000","last_became_peered":"2024-04-25T08:29:28.084884+0000","last_unstale":"2024-04-25T08:29:30.075123+0000","last_undegraded":"2024-04-25T08:29:30.075123+0000","last_fullsized":"2024-04-25T08:29:30.075123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075420+0000","last_change":"2024-04-25T08:29:24.035523+0000","last_active":"2024-04-25T08:29:29.075420+0000","last_peered":"2024-04-25T08:29:29.075420+0000","last_clean":"2024-04-25T08:29:29.075420+0000","last_became_active":"2024-04-25T08:29:24.035185+0000","last_became_peered":"2024-04-25T08:29:24.035185+0000","last_unstale":"2024-04-25T08:29:29.075420+0000","last_undegraded":"2024-04-25T08:29:29.075420+0000","last_fullsized":"2024-04-25T08:29:29.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.274896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895386+0000","last_change":"2024-04-25T08:29:23.037104+0000","last_active":"2024-04-25T08:29:29.895386+0000","last_peered":"2024-04-25T08:29:29.895386+0000","last_clean":"2024-04-25T08:29:29.895386+0000","last_became_active":"2024-04-25T08:29:23.035864+0000","last_became_peered":"2024-04-25T08:29:23.035864+0000","last_unstale":"2024-04-25T08:29:29.895386+0000","last_undegraded":"2024-04-25T08:29:29.895386+0000","last_fullsized":"2024-04-25T08:29:29.895386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075821+0000","last_change":"2024-04-25T08:29:28.080714+0000","last_active":"2024-04-25T08:29:29.075821+0000","last_peered":"2024-04-25T08:29:29.075821+0000","last_clean":"2024-04-25T08:29:29.075821+0000","last_became_active":"2024-04-25T08:29:28.080291+0000","last_became_peered":"2024-04-25T08:29:28.080291+0000","last_unstale":"2024-04-25T08:29:29.075821+0000","last_undegraded":"2024-04-25T08:29:29.075821+0000","last_fullsized":"2024-04-25T08:29:29.075821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906353+0000","last_change":"2024-04-25T08:29:24.040261+0000","last_active":"2024-04-25T08:29:29.906353+0000","last_peered":"2024-04-25T08:29:29.906353+0000","last_clean":"2024-04-25T08:29:29.906353+0000","last_became_active":"2024-04-25T08:29:24.039953+0000","last_became_peered":"2024-04-25T08:29:24.039953+0000","last_unstale":"2024-04-25T08:29:29.906353+0000","last_undegraded":"2024-04-25T08:29:29.906353+0000","last_fullsized":"2024-04-25T08:29:29.906353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076570+0000","last_change":"2024-04-25T08:29:23.038961+0000","last_active":"2024-04-25T08:29:29.076570+0000","last_peered":"2024-04-25T08:29:29.076570+0000","last_clean":"2024-04-25T08:29:29.076570+0000","last_became_active":"2024-04-25T08:29:23.038722+0000","last_became_peered":"2024-04-25T08:29:23.038722+0000","last_unstale":"2024-04-25T08:29:29.076570+0000","last_undegraded":"2024-04-25T08:29:29.076570+0000","last_fullsized":"2024-04-25T08:29:29.076570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074879+0000","last_change":"2024-04-25T08:29:28.073848+0000","last_active":"2024-04-25T08:29:29.074879+0000","last_peered":"2024-04-25T08:29:29.074879+0000","last_clean":"2024-04-25T08:29:29.074879+0000","last_became_active":"2024-04-25T08:29:28.073489+0000","last_became_peered":"2024-04-25T08:29:28.073489+0000","last_unstale":"2024-04-25T08:29:29.074879+0000","last_undegraded":"2024-04-25T08:29:29.074879+0000","last_fullsized":"2024-04-25T08:29:29.074879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:54.262561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076145+0000","last_change":"2024-04-25T08:29:24.057631+0000","last_active":"2024-04-25T08:29:29.076145+0000","last_peered":"2024-04-25T08:29:29.076145+0000","last_clean":"2024-04-25T08:29:29.076145+0000","last_became_active":"2024-04-25T08:29:24.057387+0000","last_became_peered":"2024-04-25T08:29:24.057387+0000","last_unstale":"2024-04-25T08:29:29.076145+0000","last_undegraded":"2024-04-25T08:29:29.076145+0000","last_fullsized":"2024-04-25T08:29:29.076145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075211+0000","last_change":"2024-04-25T08:29:23.041608+0000","last_active":"2024-04-25T08:29:30.075211+0000","last_peered":"2024-04-25T08:29:30.075211+0000","last_clean":"2024-04-25T08:29:30.075211+0000","last_became_active":"2024-04-25T08:29:23.041305+0000","last_became_peered":"2024-04-25T08:29:23.041305+0000","last_unstale":"2024-04-25T08:29:30.075211+0000","last_undegraded":"2024-04-25T08:29:30.075211+0000","last_fullsized":"2024-04-25T08:29:30.075211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:28.085603+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:28.084444+0000","last_became_peered":"2024-04-25T08:29:28.084444+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:39.462705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075317+0000","last_change":"2024-04-25T08:29:24.036286+0000","last_active":"2024-04-25T08:29:30.075317+0000","last_peered":"2024-04-25T08:29:30.075317+0000","last_clean":"2024-04-25T08:29:30.075317+0000","last_became_active":"2024-04-25T08:29:24.035947+0000","last_became_peered":"2024-04-25T08:29:24.035947+0000","last_unstale":"2024-04-25T08:29:30.075317+0000","last_undegraded":"2024-04-25T08:29:30.075317+0000","last_fullsized":"2024-04-25T08:29:30.075317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896087+0000","last_change":"2024-04-25T08:29:23.035930+0000","last_active":"2024-04-25T08:29:29.896087+0000","last_peered":"2024-04-25T08:29:29.896087+0000","last_clean":"2024-04-25T08:29:29.896087+0000","last_became_active":"2024-04-25T08:29:23.035685+0000","last_became_peered":"2024-04-25T08:29:23.035685+0000","last_unstale":"2024-04-25T08:29:29.896087+0000","last_undegraded":"2024-04-25T08:29:29.896087+0000","last_fullsized":"2024-04-25T08:29:29.896087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906315+0000","last_change":"2024-04-25T08:29:28.076556+0000","last_active":"2024-04-25T08:29:29.906315+0000","last_peered":"2024-04-25T08:29:29.906315+0000","last_clean":"2024-04-25T08:29:29.906315+0000","last_became_active":"2024-04-25T08:29:28.076157+0000","last_became_peered":"2024-04-25T08:29:28.076157+0000","last_unstale":"2024-04-25T08:29:29.906315+0000","last_undegraded":"2024-04-25T08:29:29.906315+0000","last_fullsized":"2024-04-25T08:29:29.906315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574536+0000","last_change":"2024-04-25T08:29:24.055822+0000","last_active":"2024-04-25T08:29:29.574536+0000","last_peered":"2024-04-25T08:29:29.574536+0000","last_clean":"2024-04-25T08:29:29.574536+0000","last_became_active":"2024-04-25T08:29:24.055087+0000","last_became_peered":"2024-04-25T08:29:24.055087+0000","last_unstale":"2024-04-25T08:29:29.574536+0000","last_undegraded":"2024-04-25T08:29:29.574536+0000","last_fullsized":"2024-04-25T08:29:29.574536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075321+0000","last_change":"2024-04-25T08:29:23.037612+0000","last_active":"2024-04-25T08:29:30.075321+0000","last_peered":"2024-04-25T08:29:30.075321+0000","last_clean":"2024-04-25T08:29:30.075321+0000","last_became_active":"2024-04-25T08:29:23.037304+0000","last_became_peered":"2024-04-25T08:29:23.037304+0000","last_unstale":"2024-04-25T08:29:30.075321+0000","last_undegraded":"2024-04-25T08:29:30.075321+0000","last_fullsized":"2024-04-25T08:29:30.075321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074438+0000","last_change":"2024-04-25T08:29:28.068007+0000","last_active":"2024-04-25T08:29:30.074438+0000","last_peered":"2024-04-25T08:29:30.074438+0000","last_clean":"2024-04-25T08:29:30.074438+0000","last_became_active":"2024-04-25T08:29:28.067586+0000","last_became_peered":"2024-04-25T08:29:28.067586+0000","last_unstale":"2024-04-25T08:29:30.074438+0000","last_undegraded":"2024-04-25T08:29:30.074438+0000","last_fullsized":"2024-04-25T08:29:30.074438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075717+0000","last_change":"2024-04-25T08:29:24.038433+0000","last_active":"2024-04-25T08:29:29.075717+0000","last_peered":"2024-04-25T08:29:29.075717+0000","last_clean":"2024-04-25T08:29:29.075717+0000","last_became_active":"2024-04-25T08:29:24.038164+0000","last_became_peered":"2024-04-25T08:29:24.038164+0000","last_unstale":"2024-04-25T08:29:29.075717+0000","last_undegraded":"2024-04-25T08:29:29.075717+0000","last_fullsized":"2024-04-25T08:29:29.075717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574356+0000","last_change":"2024-04-25T08:29:23.036328+0000","last_active":"2024-04-25T08:29:29.574356+0000","last_peered":"2024-04-25T08:29:29.574356+0000","last_clean":"2024-04-25T08:29:29.574356+0000","last_became_active":"2024-04-25T08:29:23.036072+0000","last_became_peered":"2024-04-25T08:29:23.036072+0000","last_unstale":"2024-04-25T08:29:29.574356+0000","last_undegraded":"2024-04-25T08:29:29.574356+0000","last_fullsized":"2024-04-25T08:29:29.574356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075255+0000","last_change":"2024-04-25T08:29:28.081150+0000","last_active":"2024-04-25T08:29:30.075255+0000","last_peered":"2024-04-25T08:29:30.075255+0000","last_clean":"2024-04-25T08:29:30.075255+0000","last_became_active":"2024-04-25T08:29:28.080621+0000","last_became_peered":"2024-04-25T08:29:28.080621+0000","last_unstale":"2024-04-25T08:29:30.075255+0000","last_undegraded":"2024-04-25T08:29:30.075255+0000","last_fullsized":"2024-04-25T08:29:30.075255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074632+0000","last_change":"2024-04-25T08:29:24.036313+0000","last_active":"2024-04-25T08:29:30.074632+0000","last_peered":"2024-04-25T08:29:30.074632+0000","last_clean":"2024-04-25T08:29:30.074632+0000","last_became_active":"2024-04-25T08:29:24.035892+0000","last_became_peered":"2024-04-25T08:29:24.035892+0000","last_unstale":"2024-04-25T08:29:30.074632+0000","last_undegraded":"2024-04-25T08:29:30.074632+0000","last_fullsized":"2024-04-25T08:29:30.074632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074251+0000","last_change":"2024-04-25T08:29:23.033190+0000","last_active":"2024-04-25T08:29:30.074251+0000","last_peered":"2024-04-25T08:29:30.074251+0000","last_clean":"2024-04-25T08:29:30.074251+0000","last_became_active":"2024-04-25T08:29:23.032822+0000","last_became_peered":"2024-04-25T08:29:23.032822+0000","last_unstale":"2024-04-25T08:29:30.074251+0000","last_undegraded":"2024-04-25T08:29:30.074251+0000","last_fullsized":"2024-04-25T08:29:30.074251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075145+0000","last_change":"2024-04-25T08:29:28.085006+0000","last_active":"2024-04-25T08:29:30.075145+0000","last_peered":"2024-04-25T08:29:30.075145+0000","last_clean":"2024-04-25T08:29:30.075145+0000","last_became_active":"2024-04-25T08:29:28.084572+0000","last_became_peered":"2024-04-25T08:29:28.084572+0000","last_unstale":"2024-04-25T08:29:30.075145+0000","last_undegraded":"2024-04-25T08:29:30.075145+0000","last_fullsized":"2024-04-25T08:29:30.075145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906058+0000","last_change":"2024-04-25T08:29:24.043403+0000","last_active":"2024-04-25T08:29:29.906058+0000","last_peered":"2024-04-25T08:29:29.906058+0000","last_clean":"2024-04-25T08:29:29.906058+0000","last_became_active":"2024-04-25T08:29:24.042964+0000","last_became_peered":"2024-04-25T08:29:24.042964+0000","last_unstale":"2024-04-25T08:29:29.906058+0000","last_undegraded":"2024-04-25T08:29:29.906058+0000","last_fullsized":"2024-04-25T08:29:29.906058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895394+0000","last_change":"2024-04-25T08:29:23.035024+0000","last_active":"2024-04-25T08:29:29.895394+0000","last_peered":"2024-04-25T08:29:29.895394+0000","last_clean":"2024-04-25T08:29:29.895394+0000","last_became_active":"2024-04-25T08:29:23.034761+0000","last_became_peered":"2024-04-25T08:29:23.034761+0000","last_unstale":"2024-04-25T08:29:29.895394+0000","last_undegraded":"2024-04-25T08:29:29.895394+0000","last_fullsized":"2024-04-25T08:29:29.895394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906299+0000","last_change":"2024-04-25T08:29:28.078207+0000","last_active":"2024-04-25T08:29:29.906299+0000","last_peered":"2024-04-25T08:29:29.906299+0000","last_clean":"2024-04-25T08:29:29.906299+0000","last_became_active":"2024-04-25T08:29:28.077777+0000","last_became_peered":"2024-04-25T08:29:28.077777+0000","last_unstale":"2024-04-25T08:29:29.906299+0000","last_undegraded":"2024-04-25T08:29:29.906299+0000","last_fullsized":"2024-04-25T08:29:29.906299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.048117+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.047733+0000","last_became_peered":"2024-04-25T08:29:24.047733+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574166+0000","last_change":"2024-04-25T08:29:23.036154+0000","last_active":"2024-04-25T08:29:29.574166+0000","last_peered":"2024-04-25T08:29:29.574166+0000","last_clean":"2024-04-25T08:29:29.574166+0000","last_became_active":"2024-04-25T08:29:23.035873+0000","last_became_peered":"2024-04-25T08:29:23.035873+0000","last_unstale":"2024-04-25T08:29:29.574166+0000","last_undegraded":"2024-04-25T08:29:29.574166+0000","last_fullsized":"2024-04-25T08:29:29.574166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895594+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:29:29.895594+0000","last_peered":"2024-04-25T08:29:29.895594+0000","last_clean":"2024-04-25T08:29:29.895594+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:29:29.895594+0000","last_undegraded":"2024-04-25T08:29:29.895594+0000","last_fullsized":"2024-04-25T08:29:29.895594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076370+0000","last_change":"2024-04-25T08:29:28.076816+0000","last_active":"2024-04-25T08:29:29.076370+0000","last_peered":"2024-04-25T08:29:29.076370+0000","last_clean":"2024-04-25T08:29:29.076370+0000","last_became_active":"2024-04-25T08:29:28.076348+0000","last_became_peered":"2024-04-25T08:29:28.076348+0000","last_unstale":"2024-04-25T08:29:29.076370+0000","last_undegraded":"2024-04-25T08:29:29.076370+0000","last_fullsized":"2024-04-25T08:29:29.076370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895804+0000","last_change":"2024-04-25T08:29:24.047938+0000","last_active":"2024-04-25T08:29:29.895804+0000","last_peered":"2024-04-25T08:29:29.895804+0000","last_clean":"2024-04-25T08:29:29.895804+0000","last_became_active":"2024-04-25T08:29:24.047670+0000","last_became_peered":"2024-04-25T08:29:24.047670+0000","last_unstale":"2024-04-25T08:29:29.895804+0000","last_undegraded":"2024-04-25T08:29:29.895804+0000","last_fullsized":"2024-04-25T08:29:29.895804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074744+0000","last_change":"2024-04-25T08:29:23.037162+0000","last_active":"2024-04-25T08:29:30.074744+0000","last_peered":"2024-04-25T08:29:30.074744+0000","last_clean":"2024-04-25T08:29:30.074744+0000","last_became_active":"2024-04-25T08:29:23.036850+0000","last_became_peered":"2024-04-25T08:29:23.036850+0000","last_unstale":"2024-04-25T08:29:30.074744+0000","last_undegraded":"2024-04-25T08:29:30.074744+0000","last_fullsized":"2024-04-25T08:29:30.074744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074972+0000","last_change":"2024-04-25T08:29:28.073263+0000","last_active":"2024-04-25T08:29:29.074972+0000","last_peered":"2024-04-25T08:29:29.074972+0000","last_clean":"2024-04-25T08:29:29.074972+0000","last_became_active":"2024-04-25T08:29:28.072784+0000","last_became_peered":"2024-04-25T08:29:28.072784+0000","last_unstale":"2024-04-25T08:29:29.074972+0000","last_undegraded":"2024-04-25T08:29:29.074972+0000","last_fullsized":"2024-04-25T08:29:29.074972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574458+0000","last_change":"2024-04-25T08:29:24.056206+0000","last_active":"2024-04-25T08:29:29.574458+0000","last_peered":"2024-04-25T08:29:29.574458+0000","last_clean":"2024-04-25T08:29:29.574458+0000","last_became_active":"2024-04-25T08:29:24.055557+0000","last_became_peered":"2024-04-25T08:29:24.055557+0000","last_unstale":"2024-04-25T08:29:29.574458+0000","last_undegraded":"2024-04-25T08:29:29.574458+0000","last_fullsized":"2024-04-25T08:29:29.574458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075008+0000","last_change":"2024-04-25T08:29:23.031709+0000","last_active":"2024-04-25T08:29:30.075008+0000","last_peered":"2024-04-25T08:29:30.075008+0000","last_clean":"2024-04-25T08:29:30.075008+0000","last_became_active":"2024-04-25T08:29:23.031419+0000","last_became_peered":"2024-04-25T08:29:23.031419+0000","last_unstale":"2024-04-25T08:29:30.075008+0000","last_undegraded":"2024-04-25T08:29:30.075008+0000","last_fullsized":"2024-04-25T08:29:30.075008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906475+0000","last_change":"2024-04-25T08:29:28.070276+0000","last_active":"2024-04-25T08:29:29.906475+0000","last_peered":"2024-04-25T08:29:29.906475+0000","last_clean":"2024-04-25T08:29:29.906475+0000","last_became_active":"2024-04-25T08:29:28.069869+0000","last_became_peered":"2024-04-25T08:29:28.069869+0000","last_unstale":"2024-04-25T08:29:29.906475+0000","last_undegraded":"2024-04-25T08:29:29.906475+0000","last_fullsized":"2024-04-25T08:29:29.906475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896237+0000","last_change":"2024-04-25T08:29:24.057274+0000","last_active":"2024-04-25T08:29:29.896237+0000","last_peered":"2024-04-25T08:29:29.896237+0000","last_clean":"2024-04-25T08:29:29.896237+0000","last_became_active":"2024-04-25T08:29:24.056998+0000","last_became_peered":"2024-04-25T08:29:24.056998+0000","last_unstale":"2024-04-25T08:29:29.896237+0000","last_undegraded":"2024-04-25T08:29:29.896237+0000","last_fullsized":"2024-04-25T08:29:29.896237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:18.584258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074824+0000","last_change":"2024-04-25T08:29:23.039945+0000","last_active":"2024-04-25T08:29:30.074824+0000","last_peered":"2024-04-25T08:29:30.074824+0000","last_clean":"2024-04-25T08:29:30.074824+0000","last_became_active":"2024-04-25T08:29:23.039563+0000","last_became_peered":"2024-04-25T08:29:23.039563+0000","last_unstale":"2024-04-25T08:29:30.074824+0000","last_undegraded":"2024-04-25T08:29:30.074824+0000","last_fullsized":"2024-04-25T08:29:30.074824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573881+0000","last_change":"2024-04-25T08:29:28.076465+0000","last_active":"2024-04-25T08:29:29.573881+0000","last_peered":"2024-04-25T08:29:29.573881+0000","last_clean":"2024-04-25T08:29:29.573881+0000","last_became_active":"2024-04-25T08:29:28.076064+0000","last_became_peered":"2024-04-25T08:29:28.076064+0000","last_unstale":"2024-04-25T08:29:29.573881+0000","last_undegraded":"2024-04-25T08:29:29.573881+0000","last_fullsized":"2024-04-25T08:29:29.573881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896089+0000","last_change":"2024-04-25T08:29:24.050215+0000","last_active":"2024-04-25T08:29:29.896089+0000","last_peered":"2024-04-25T08:29:29.896089+0000","last_clean":"2024-04-25T08:29:29.896089+0000","last_became_active":"2024-04-25T08:29:24.048694+0000","last_became_peered":"2024-04-25T08:29:24.048694+0000","last_unstale":"2024-04-25T08:29:29.896089+0000","last_undegraded":"2024-04-25T08:29:29.896089+0000","last_fullsized":"2024-04-25T08:29:29.896089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075997+0000","last_change":"2024-04-25T08:29:23.033858+0000","last_active":"2024-04-25T08:29:29.075997+0000","last_peered":"2024-04-25T08:29:29.075997+0000","last_clean":"2024-04-25T08:29:29.075997+0000","last_became_active":"2024-04-25T08:29:23.033599+0000","last_became_peered":"2024-04-25T08:29:23.033599+0000","last_unstale":"2024-04-25T08:29:29.075997+0000","last_undegraded":"2024-04-25T08:29:29.075997+0000","last_fullsized":"2024-04-25T08:29:29.075997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074730+0000","last_change":"2024-04-25T08:29:28.081641+0000","last_active":"2024-04-25T08:29:29.074730+0000","last_peered":"2024-04-25T08:29:29.074730+0000","last_clean":"2024-04-25T08:29:29.074730+0000","last_became_active":"2024-04-25T08:29:28.081220+0000","last_became_peered":"2024-04-25T08:29:28.081220+0000","last_unstale":"2024-04-25T08:29:29.074730+0000","last_undegraded":"2024-04-25T08:29:29.074730+0000","last_fullsized":"2024-04-25T08:29:29.074730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:51.749033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895682+0000","last_change":"2024-04-25T08:29:24.050072+0000","last_active":"2024-04-25T08:29:29.895682+0000","last_peered":"2024-04-25T08:29:29.895682+0000","last_clean":"2024-04-25T08:29:29.895682+0000","last_became_active":"2024-04-25T08:29:24.049773+0000","last_became_peered":"2024-04-25T08:29:24.049773+0000","last_unstale":"2024-04-25T08:29:29.895682+0000","last_undegraded":"2024-04-25T08:29:29.895682+0000","last_fullsized":"2024-04-25T08:29:29.895682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074062+0000","last_change":"2024-04-25T08:29:23.035476+0000","last_active":"2024-04-25T08:29:30.074062+0000","last_peered":"2024-04-25T08:29:30.074062+0000","last_clean":"2024-04-25T08:29:30.074062+0000","last_became_active":"2024-04-25T08:29:23.035129+0000","last_became_peered":"2024-04-25T08:29:23.035129+0000","last_unstale":"2024-04-25T08:29:30.074062+0000","last_undegraded":"2024-04-25T08:29:30.074062+0000","last_fullsized":"2024-04-25T08:29:30.074062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.454939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906185+0000","last_change":"2024-04-25T08:29:28.076678+0000","last_active":"2024-04-25T08:29:29.906185+0000","last_peered":"2024-04-25T08:29:29.906185+0000","last_clean":"2024-04-25T08:29:29.906185+0000","last_became_active":"2024-04-25T08:29:28.076283+0000","last_became_peered":"2024-04-25T08:29:28.076283+0000","last_unstale":"2024-04-25T08:29:29.906185+0000","last_undegraded":"2024-04-25T08:29:29.906185+0000","last_fullsized":"2024-04-25T08:29:29.906185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075358+0000","last_change":"2024-04-25T08:29:24.041672+0000","last_active":"2024-04-25T08:29:29.075358+0000","last_peered":"2024-04-25T08:29:29.075358+0000","last_clean":"2024-04-25T08:29:29.075358+0000","last_became_active":"2024-04-25T08:29:24.040865+0000","last_became_peered":"2024-04-25T08:29:24.040865+0000","last_unstale":"2024-04-25T08:29:29.075358+0000","last_undegraded":"2024-04-25T08:29:29.075358+0000","last_fullsized":"2024-04-25T08:29:29.075358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075805+0000","last_change":"2024-04-25T08:29:23.028173+0000","last_active":"2024-04-25T08:29:29.075805+0000","last_peered":"2024-04-25T08:29:29.075805+0000","last_clean":"2024-04-25T08:29:29.075805+0000","last_became_active":"2024-04-25T08:29:23.027886+0000","last_became_peered":"2024-04-25T08:29:23.027886+0000","last_unstale":"2024-04-25T08:29:29.075805+0000","last_undegraded":"2024-04-25T08:29:29.075805+0000","last_fullsized":"2024-04-25T08:29:29.075805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076047+0000","last_change":"2024-04-25T08:29:28.084524+0000","last_active":"2024-04-25T08:29:29.076047+0000","last_peered":"2024-04-25T08:29:29.076047+0000","last_clean":"2024-04-25T08:29:29.076047+0000","last_became_active":"2024-04-25T08:29:28.083769+0000","last_became_peered":"2024-04-25T08:29:28.083769+0000","last_unstale":"2024-04-25T08:29:29.076047+0000","last_undegraded":"2024-04-25T08:29:29.076047+0000","last_fullsized":"2024-04-25T08:29:29.076047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:43.115368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895795+0000","last_change":"2024-04-25T08:29:24.057400+0000","last_active":"2024-04-25T08:29:29.895795+0000","last_peered":"2024-04-25T08:29:29.895795+0000","last_clean":"2024-04-25T08:29:29.895795+0000","last_became_active":"2024-04-25T08:29:24.057074+0000","last_became_peered":"2024-04-25T08:29:24.057074+0000","last_unstale":"2024-04-25T08:29:29.895795+0000","last_undegraded":"2024-04-25T08:29:29.895795+0000","last_fullsized":"2024-04-25T08:29:29.895795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574339+0000","last_change":"2024-04-25T08:29:23.036448+0000","last_active":"2024-04-25T08:29:29.574339+0000","last_peered":"2024-04-25T08:29:29.574339+0000","last_clean":"2024-04-25T08:29:29.574339+0000","last_became_active":"2024-04-25T08:29:23.036172+0000","last_became_peered":"2024-04-25T08:29:23.036172+0000","last_unstale":"2024-04-25T08:29:29.574339+0000","last_undegraded":"2024-04-25T08:29:29.574339+0000","last_fullsized":"2024-04-25T08:29:29.574339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075577+0000","last_change":"2024-04-25T08:29:28.083554+0000","last_active":"2024-04-25T08:29:29.075577+0000","last_peered":"2024-04-25T08:29:29.075577+0000","last_clean":"2024-04-25T08:29:29.075577+0000","last_became_active":"2024-04-25T08:29:28.083178+0000","last_became_peered":"2024-04-25T08:29:28.083178+0000","last_unstale":"2024-04-25T08:29:29.075577+0000","last_undegraded":"2024-04-25T08:29:29.075577+0000","last_fullsized":"2024-04-25T08:29:29.075577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906194+0000","last_change":"2024-04-25T08:29:24.049668+0000","last_active":"2024-04-25T08:29:29.906194+0000","last_peered":"2024-04-25T08:29:29.906194+0000","last_clean":"2024-04-25T08:29:29.906194+0000","last_became_active":"2024-04-25T08:29:24.049359+0000","last_became_peered":"2024-04-25T08:29:24.049359+0000","last_unstale":"2024-04-25T08:29:29.906194+0000","last_undegraded":"2024-04-25T08:29:29.906194+0000","last_fullsized":"2024-04-25T08:29:29.906194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:48.609719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.030224+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.029936+0000","last_became_peered":"2024-04-25T08:29:23.029936+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:17.202033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574628+0000","last_change":"2024-04-25T08:29:28.076915+0000","last_active":"2024-04-25T08:29:29.574628+0000","last_peered":"2024-04-25T08:29:29.574628+0000","last_clean":"2024-04-25T08:29:29.574628+0000","last_became_active":"2024-04-25T08:29:28.076527+0000","last_became_peered":"2024-04-25T08:29:28.076527+0000","last_unstale":"2024-04-25T08:29:29.574628+0000","last_undegraded":"2024-04-25T08:29:29.574628+0000","last_fullsized":"2024-04-25T08:29:29.574628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895697+0000","last_change":"2024-04-25T08:29:24.053639+0000","last_active":"2024-04-25T08:29:29.895697+0000","last_peered":"2024-04-25T08:29:29.895697+0000","last_clean":"2024-04-25T08:29:29.895697+0000","last_became_active":"2024-04-25T08:29:24.053028+0000","last_became_peered":"2024-04-25T08:29:24.053028+0000","last_unstale":"2024-04-25T08:29:29.895697+0000","last_undegraded":"2024-04-25T08:29:29.895697+0000","last_fullsized":"2024-04-25T08:29:29.895697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:33.263062+0000","last_change":"2024-04-25T08:29:23.031850+0000","last_active":"2024-04-25T08:29:33.263062+0000","last_peered":"2024-04-25T08:29:33.263062+0000","last_clean":"2024-04-25T08:29:33.263062+0000","last_became_active":"2024-04-25T08:29:23.031588+0000","last_became_peered":"2024-04-25T08:29:23.031588+0000","last_unstale":"2024-04-25T08:29:33.263062+0000","last_undegraded":"2024-04-25T08:29:33.263062+0000","last_fullsized":"2024-04-25T08:29:33.263062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074668+0000","last_change":"2024-04-25T08:29:28.066135+0000","last_active":"2024-04-25T08:29:30.074668+0000","last_peered":"2024-04-25T08:29:30.074668+0000","last_clean":"2024-04-25T08:29:30.074668+0000","last_became_active":"2024-04-25T08:29:28.065611+0000","last_became_peered":"2024-04-25T08:29:28.065611+0000","last_unstale":"2024-04-25T08:29:30.074668+0000","last_undegraded":"2024-04-25T08:29:30.074668+0000","last_fullsized":"2024-04-25T08:29:30.074668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075847+0000","last_change":"2024-04-25T08:29:24.050678+0000","last_active":"2024-04-25T08:29:29.075847+0000","last_peered":"2024-04-25T08:29:29.075847+0000","last_clean":"2024-04-25T08:29:29.075847+0000","last_became_active":"2024-04-25T08:29:24.049995+0000","last_became_peered":"2024-04-25T08:29:24.049995+0000","last_unstale":"2024-04-25T08:29:29.075847+0000","last_undegraded":"2024-04-25T08:29:29.075847+0000","last_fullsized":"2024-04-25T08:29:29.075847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.729278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075000+0000","last_change":"2024-04-25T08:29:23.036900+0000","last_active":"2024-04-25T08:29:30.075000+0000","last_peered":"2024-04-25T08:29:30.075000+0000","last_clean":"2024-04-25T08:29:30.075000+0000","last_became_active":"2024-04-25T08:29:23.036630+0000","last_became_peered":"2024-04-25T08:29:23.036630+0000","last_unstale":"2024-04-25T08:29:30.075000+0000","last_undegraded":"2024-04-25T08:29:30.075000+0000","last_fullsized":"2024-04-25T08:29:30.075000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075542+0000","last_change":"2024-04-25T08:29:28.066730+0000","last_active":"2024-04-25T08:29:29.075542+0000","last_peered":"2024-04-25T08:29:29.075542+0000","last_clean":"2024-04-25T08:29:29.075542+0000","last_became_active":"2024-04-25T08:29:28.066323+0000","last_became_peered":"2024-04-25T08:29:28.066323+0000","last_unstale":"2024-04-25T08:29:29.075542+0000","last_undegraded":"2024-04-25T08:29:29.075542+0000","last_fullsized":"2024-04-25T08:29:29.075542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906473+0000","last_change":"2024-04-25T08:29:24.035898+0000","last_active":"2024-04-25T08:29:29.906473+0000","last_peered":"2024-04-25T08:29:29.906473+0000","last_clean":"2024-04-25T08:29:29.906473+0000","last_became_active":"2024-04-25T08:29:24.035137+0000","last_became_peered":"2024-04-25T08:29:24.035137+0000","last_unstale":"2024-04-25T08:29:29.906473+0000","last_undegraded":"2024-04-25T08:29:29.906473+0000","last_fullsized":"2024-04-25T08:29:29.906473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574599+0000","last_change":"2024-04-25T08:29:23.036994+0000","last_active":"2024-04-25T08:29:29.574599+0000","last_peered":"2024-04-25T08:29:29.574599+0000","last_clean":"2024-04-25T08:29:29.574599+0000","last_became_active":"2024-04-25T08:29:23.036740+0000","last_became_peered":"2024-04-25T08:29:23.036740+0000","last_unstale":"2024-04-25T08:29:29.574599+0000","last_undegraded":"2024-04-25T08:29:29.574599+0000","last_fullsized":"2024-04-25T08:29:29.574599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074766+0000","last_change":"2024-04-25T08:29:28.080727+0000","last_active":"2024-04-25T08:29:29.074766+0000","last_peered":"2024-04-25T08:29:29.074766+0000","last_clean":"2024-04-25T08:29:29.074766+0000","last_became_active":"2024-04-25T08:29:28.080288+0000","last_became_peered":"2024-04-25T08:29:28.080288+0000","last_unstale":"2024-04-25T08:29:29.074766+0000","last_undegraded":"2024-04-25T08:29:29.074766+0000","last_fullsized":"2024-04-25T08:29:29.074766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.184309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574484+0000","last_change":"2024-04-25T08:29:24.054205+0000","last_active":"2024-04-25T08:29:29.574484+0000","last_peered":"2024-04-25T08:29:29.574484+0000","last_clean":"2024-04-25T08:29:29.574484+0000","last_became_active":"2024-04-25T08:29:24.050796+0000","last_became_peered":"2024-04-25T08:29:24.050796+0000","last_unstale":"2024-04-25T08:29:29.574484+0000","last_undegraded":"2024-04-25T08:29:29.574484+0000","last_fullsized":"2024-04-25T08:29:29.574484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076454+0000","last_change":"2024-04-25T08:29:23.037467+0000","last_active":"2024-04-25T08:29:29.076454+0000","last_peered":"2024-04-25T08:29:29.076454+0000","last_clean":"2024-04-25T08:29:29.076454+0000","last_became_active":"2024-04-25T08:29:23.037207+0000","last_became_peered":"2024-04-25T08:29:23.037207+0000","last_unstale":"2024-04-25T08:29:29.076454+0000","last_undegraded":"2024-04-25T08:29:29.076454+0000","last_fullsized":"2024-04-25T08:29:29.076454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075540+0000","last_change":"2024-04-25T08:29:28.085289+0000","last_active":"2024-04-25T08:29:29.075540+0000","last_peered":"2024-04-25T08:29:29.075540+0000","last_clean":"2024-04-25T08:29:29.075540+0000","last_became_active":"2024-04-25T08:29:28.084815+0000","last_became_peered":"2024-04-25T08:29:28.084815+0000","last_unstale":"2024-04-25T08:29:29.075540+0000","last_undegraded":"2024-04-25T08:29:29.075540+0000","last_fullsized":"2024-04-25T08:29:29.075540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906069+0000","last_change":"2024-04-25T08:29:28.077759+0000","last_active":"2024-04-25T08:29:29.906069+0000","last_peered":"2024-04-25T08:29:29.906069+0000","last_clean":"2024-04-25T08:29:29.906069+0000","last_became_active":"2024-04-25T08:29:28.077376+0000","last_became_peered":"2024-04-25T08:29:28.077376+0000","last_unstale":"2024-04-25T08:29:29.906069+0000","last_undegraded":"2024-04-25T08:29:29.906069+0000","last_fullsized":"2024-04-25T08:29:29.906069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074258+0000","last_change":"2024-04-25T08:29:24.053405+0000","last_active":"2024-04-25T08:29:30.074258+0000","last_peered":"2024-04-25T08:29:30.074258+0000","last_clean":"2024-04-25T08:29:30.074258+0000","last_became_active":"2024-04-25T08:29:24.053104+0000","last_became_peered":"2024-04-25T08:29:24.053104+0000","last_unstale":"2024-04-25T08:29:30.074258+0000","last_undegraded":"2024-04-25T08:29:30.074258+0000","last_fullsized":"2024-04-25T08:29:30.074258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574014+0000","last_change":"2024-04-25T08:29:23.034808+0000","last_active":"2024-04-25T08:29:29.574014+0000","last_peered":"2024-04-25T08:29:29.574014+0000","last_clean":"2024-04-25T08:29:29.574014+0000","last_became_active":"2024-04-25T08:29:23.034545+0000","last_became_peered":"2024-04-25T08:29:23.034545+0000","last_unstale":"2024-04-25T08:29:29.574014+0000","last_undegraded":"2024-04-25T08:29:29.574014+0000","last_fullsized":"2024-04-25T08:29:29.574014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076514+0000","last_change":"2024-04-25T08:29:28.068503+0000","last_active":"2024-04-25T08:29:29.076514+0000","last_peered":"2024-04-25T08:29:29.076514+0000","last_clean":"2024-04-25T08:29:29.076514+0000","last_became_active":"2024-04-25T08:29:28.068080+0000","last_became_peered":"2024-04-25T08:29:28.068080+0000","last_unstale":"2024-04-25T08:29:29.076514+0000","last_undegraded":"2024-04-25T08:29:29.076514+0000","last_fullsized":"2024-04-25T08:29:29.076514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074757+0000","last_change":"2024-04-25T08:29:24.053631+0000","last_active":"2024-04-25T08:29:29.074757+0000","last_peered":"2024-04-25T08:29:29.074757+0000","last_clean":"2024-04-25T08:29:29.074757+0000","last_became_active":"2024-04-25T08:29:24.052293+0000","last_became_peered":"2024-04-25T08:29:24.052293+0000","last_unstale":"2024-04-25T08:29:29.074757+0000","last_undegraded":"2024-04-25T08:29:29.074757+0000","last_fullsized":"2024-04-25T08:29:29.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074995+0000","last_change":"2024-04-25T08:29:23.029639+0000","last_active":"2024-04-25T08:29:29.074995+0000","last_peered":"2024-04-25T08:29:29.074995+0000","last_clean":"2024-04-25T08:29:29.074995+0000","last_became_active":"2024-04-25T08:29:23.027908+0000","last_became_peered":"2024-04-25T08:29:23.027908+0000","last_unstale":"2024-04-25T08:29:29.074995+0000","last_undegraded":"2024-04-25T08:29:29.074995+0000","last_fullsized":"2024-04-25T08:29:29.074995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:02.366427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075903+0000","last_change":"2024-04-25T08:29:23.024619+0000","last_active":"2024-04-25T08:29:29.075903+0000","last_peered":"2024-04-25T08:29:29.075903+0000","last_clean":"2024-04-25T08:29:29.075903+0000","last_became_active":"2024-04-25T08:29:23.024364+0000","last_became_peered":"2024-04-25T08:29:23.024364+0000","last_unstale":"2024-04-25T08:29:29.075903+0000","last_undegraded":"2024-04-25T08:29:29.075903+0000","last_fullsized":"2024-04-25T08:29:29.075903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906507+0000","last_change":"2024-04-25T08:29:24.040912+0000","last_active":"2024-04-25T08:29:29.906507+0000","last_peered":"2024-04-25T08:29:29.906507+0000","last_clean":"2024-04-25T08:29:29.906507+0000","last_became_active":"2024-04-25T08:29:24.040629+0000","last_became_peered":"2024-04-25T08:29:24.040629+0000","last_unstale":"2024-04-25T08:29:29.906507+0000","last_undegraded":"2024-04-25T08:29:29.906507+0000","last_fullsized":"2024-04-25T08:29:29.906507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074576+0000","last_change":"2024-04-25T08:29:28.081077+0000","last_active":"2024-04-25T08:29:30.074576+0000","last_peered":"2024-04-25T08:29:30.074576+0000","last_clean":"2024-04-25T08:29:30.074576+0000","last_became_active":"2024-04-25T08:29:28.080470+0000","last_became_peered":"2024-04-25T08:29:28.080470+0000","last_unstale":"2024-04-25T08:29:30.074576+0000","last_undegraded":"2024-04-25T08:29:30.074576+0000","last_fullsized":"2024-04-25T08:29:30.074576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:18.082525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075155+0000","last_change":"2024-04-25T08:29:23.039762+0000","last_active":"2024-04-25T08:29:29.075155+0000","last_peered":"2024-04-25T08:29:29.075155+0000","last_clean":"2024-04-25T08:29:29.075155+0000","last_became_active":"2024-04-25T08:29:23.039484+0000","last_became_peered":"2024-04-25T08:29:23.039484+0000","last_unstale":"2024-04-25T08:29:29.075155+0000","last_undegraded":"2024-04-25T08:29:29.075155+0000","last_fullsized":"2024-04-25T08:29:29.075155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074927+0000","last_change":"2024-04-25T08:29:24.037624+0000","last_active":"2024-04-25T08:29:29.074927+0000","last_peered":"2024-04-25T08:29:29.074927+0000","last_clean":"2024-04-25T08:29:29.074927+0000","last_became_active":"2024-04-25T08:29:24.037352+0000","last_became_peered":"2024-04-25T08:29:24.037352+0000","last_unstale":"2024-04-25T08:29:29.074927+0000","last_undegraded":"2024-04-25T08:29:29.074927+0000","last_fullsized":"2024-04-25T08:29:29.074927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:51.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075230+0000","last_change":"2024-04-25T08:29:28.083366+0000","last_active":"2024-04-25T08:29:29.075230+0000","last_peered":"2024-04-25T08:29:29.075230+0000","last_clean":"2024-04-25T08:29:29.075230+0000","last_became_active":"2024-04-25T08:29:28.082870+0000","last_became_peered":"2024-04-25T08:29:28.082870+0000","last_unstale":"2024-04-25T08:29:29.075230+0000","last_undegraded":"2024-04-25T08:29:29.075230+0000","last_fullsized":"2024-04-25T08:29:29.075230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:08.399424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075038+0000","last_change":"2024-04-25T08:29:24.042078+0000","last_active":"2024-04-25T08:29:29.075038+0000","last_peered":"2024-04-25T08:29:29.075038+0000","last_clean":"2024-04-25T08:29:29.075038+0000","last_became_active":"2024-04-25T08:29:24.041318+0000","last_became_peered":"2024-04-25T08:29:24.041318+0000","last_unstale":"2024-04-25T08:29:29.075038+0000","last_undegraded":"2024-04-25T08:29:29.075038+0000","last_fullsized":"2024-04-25T08:29:29.075038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074154+0000","last_change":"2024-04-25T08:29:24.043006+0000","last_active":"2024-04-25T08:29:30.074154+0000","last_peered":"2024-04-25T08:29:30.074154+0000","last_clean":"2024-04-25T08:29:30.074154+0000","last_became_active":"2024-04-25T08:29:24.042553+0000","last_became_peered":"2024-04-25T08:29:24.042553+0000","last_unstale":"2024-04-25T08:29:30.074154+0000","last_undegraded":"2024-04-25T08:29:30.074154+0000","last_fullsized":"2024-04-25T08:29:30.074154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574181+0000","last_change":"2024-04-25T08:29:24.054678+0000","last_active":"2024-04-25T08:29:29.574181+0000","last_peered":"2024-04-25T08:29:29.574181+0000","last_clean":"2024-04-25T08:29:29.574181+0000","last_became_active":"2024-04-25T08:29:24.054432+0000","last_became_peered":"2024-04-25T08:29:24.054432+0000","last_unstale":"2024-04-25T08:29:29.574181+0000","last_undegraded":"2024-04-25T08:29:29.574181+0000","last_fullsized":"2024-04-25T08:29:29.574181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906457+0000","last_change":"2024-04-25T08:29:24.047684+0000","last_active":"2024-04-25T08:29:29.906457+0000","last_peered":"2024-04-25T08:29:29.906457+0000","last_clean":"2024-04-25T08:29:29.906457+0000","last_became_active":"2024-04-25T08:29:24.047384+0000","last_became_peered":"2024-04-25T08:29:24.047384+0000","last_unstale":"2024-04-25T08:29:29.906457+0000","last_undegraded":"2024-04-25T08:29:29.906457+0000","last_fullsized":"2024-04-25T08:29:29.906457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075571+0000","last_change":"2024-04-25T08:29:24.044573+0000","last_active":"2024-04-25T08:29:29.075571+0000","last_peered":"2024-04-25T08:29:29.075571+0000","last_clean":"2024-04-25T08:29:29.075571+0000","last_became_active":"2024-04-25T08:29:24.043980+0000","last_became_peered":"2024-04-25T08:29:24.043980+0000","last_unstale":"2024-04-25T08:29:29.075571+0000","last_undegraded":"2024-04-25T08:29:29.075571+0000","last_fullsized":"2024-04-25T08:29:29.075571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074876+0000","last_change":"2024-04-25T08:29:24.053456+0000","last_active":"2024-04-25T08:29:30.074876+0000","last_peered":"2024-04-25T08:29:30.074876+0000","last_clean":"2024-04-25T08:29:30.074876+0000","last_became_active":"2024-04-25T08:29:24.053144+0000","last_became_peered":"2024-04-25T08:29:24.053144+0000","last_unstale":"2024-04-25T08:29:30.074876+0000","last_undegraded":"2024-04-25T08:29:30.074876+0000","last_fullsized":"2024-04-25T08:29:30.074876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076592+0000","last_change":"2024-04-25T08:29:24.052047+0000","last_active":"2024-04-25T08:29:29.076592+0000","last_peered":"2024-04-25T08:29:29.076592+0000","last_clean":"2024-04-25T08:29:29.076592+0000","last_became_active":"2024-04-25T08:29:24.051777+0000","last_became_peered":"2024-04-25T08:29:24.051777+0000","last_unstale":"2024-04-25T08:29:29.076592+0000","last_undegraded":"2024-04-25T08:29:29.076592+0000","last_fullsized":"2024-04-25T08:29:29.076592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075986+0000","last_change":"2024-04-25T08:29:24.044743+0000","last_active":"2024-04-25T08:29:29.075986+0000","last_peered":"2024-04-25T08:29:29.075986+0000","last_clean":"2024-04-25T08:29:29.075986+0000","last_became_active":"2024-04-25T08:29:24.044425+0000","last_became_peered":"2024-04-25T08:29:24.044425+0000","last_unstale":"2024-04-25T08:29:29.075986+0000","last_undegraded":"2024-04-25T08:29:29.075986+0000","last_fullsized":"2024-04-25T08:29:29.075986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573967+0000","last_change":"2024-04-25T08:29:24.047484+0000","last_active":"2024-04-25T08:29:29.573967+0000","last_peered":"2024-04-25T08:29:29.573967+0000","last_clean":"2024-04-25T08:29:29.573967+0000","last_became_active":"2024-04-25T08:29:24.046848+0000","last_became_peered":"2024-04-25T08:29:24.046848+0000","last_unstale":"2024-04-25T08:29:29.573967+0000","last_undegraded":"2024-04-25T08:29:29.573967+0000","last_fullsized":"2024-04-25T08:29:29.573967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074875+0000","last_change":"2024-04-25T08:29:24.048838+0000","last_active":"2024-04-25T08:29:29.074875+0000","last_peered":"2024-04-25T08:29:29.074875+0000","last_clean":"2024-04-25T08:29:29.074875+0000","last_became_active":"2024-04-25T08:29:24.047272+0000","last_became_peered":"2024-04-25T08:29:24.047272+0000","last_unstale":"2024-04-25T08:29:29.074875+0000","last_undegraded":"2024-04-25T08:29:29.074875+0000","last_fullsized":"2024-04-25T08:29:29.074875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574182+0000","last_change":"2024-04-25T08:29:24.052059+0000","last_active":"2024-04-25T08:29:29.574182+0000","last_peered":"2024-04-25T08:29:29.574182+0000","last_clean":"2024-04-25T08:29:29.574182+0000","last_became_active":"2024-04-25T08:29:24.051604+0000","last_became_peered":"2024-04-25T08:29:24.051604+0000","last_unstale":"2024-04-25T08:29:29.574182+0000","last_undegraded":"2024-04-25T08:29:29.574182+0000","last_fullsized":"2024-04-25T08:29:29.574182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906362+0000","last_change":"2024-04-25T08:29:24.049843+0000","last_active":"2024-04-25T08:29:29.906362+0000","last_peered":"2024-04-25T08:29:29.906362+0000","last_clean":"2024-04-25T08:29:29.906362+0000","last_became_active":"2024-04-25T08:29:24.049562+0000","last_became_peered":"2024-04-25T08:29:24.049562+0000","last_unstale":"2024-04-25T08:29:29.906362+0000","last_undegraded":"2024-04-25T08:29:29.906362+0000","last_fullsized":"2024-04-25T08:29:29.906362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.370927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.041378+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.040860+0000","last_became_peered":"2024-04-25T08:29:24.040860+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895964+0000","last_change":"2024-04-25T08:29:24.050860+0000","last_active":"2024-04-25T08:29:29.895964+0000","last_peered":"2024-04-25T08:29:29.895964+0000","last_clean":"2024-04-25T08:29:29.895964+0000","last_became_active":"2024-04-25T08:29:24.050610+0000","last_became_peered":"2024-04-25T08:29:24.050610+0000","last_unstale":"2024-04-25T08:29:29.895964+0000","last_undegraded":"2024-04-25T08:29:29.895964+0000","last_fullsized":"2024-04-25T08:29:29.895964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074757+0000","last_change":"2024-04-25T08:29:24.041470+0000","last_active":"2024-04-25T08:29:30.074757+0000","last_peered":"2024-04-25T08:29:30.074757+0000","last_clean":"2024-04-25T08:29:30.074757+0000","last_became_active":"2024-04-25T08:29:24.041172+0000","last_became_peered":"2024-04-25T08:29:24.041172+0000","last_unstale":"2024-04-25T08:29:30.074757+0000","last_undegraded":"2024-04-25T08:29:30.074757+0000","last_fullsized":"2024-04-25T08:29:30.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574663+0000","last_change":"2024-04-25T08:29:24.054784+0000","last_active":"2024-04-25T08:29:29.574663+0000","last_peered":"2024-04-25T08:29:29.574663+0000","last_clean":"2024-04-25T08:29:29.574663+0000","last_became_active":"2024-04-25T08:29:24.054538+0000","last_became_peered":"2024-04-25T08:29:24.054538+0000","last_unstale":"2024-04-25T08:29:29.574663+0000","last_undegraded":"2024-04-25T08:29:29.574663+0000","last_fullsized":"2024-04-25T08:29:29.574663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573870+0000","last_change":"2024-04-25T08:29:24.046213+0000","last_active":"2024-04-25T08:29:29.573870+0000","last_peered":"2024-04-25T08:29:29.573870+0000","last_clean":"2024-04-25T08:29:29.573870+0000","last_became_active":"2024-04-25T08:29:24.045850+0000","last_became_peered":"2024-04-25T08:29:24.045850+0000","last_unstale":"2024-04-25T08:29:29.573870+0000","last_undegraded":"2024-04-25T08:29:29.573870+0000","last_fullsized":"2024-04-25T08:29:29.573870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074307+0000","last_change":"2024-04-25T08:29:24.039662+0000","last_active":"2024-04-25T08:29:30.074307+0000","last_peered":"2024-04-25T08:29:30.074307+0000","last_clean":"2024-04-25T08:29:30.074307+0000","last_became_active":"2024-04-25T08:29:24.039338+0000","last_became_peered":"2024-04-25T08:29:24.039338+0000","last_unstale":"2024-04-25T08:29:30.074307+0000","last_undegraded":"2024-04-25T08:29:30.074307+0000","last_fullsized":"2024-04-25T08:29:30.074307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906168+0000","last_change":"2024-04-25T08:29:24.038228+0000","last_active":"2024-04-25T08:29:29.906168+0000","last_peered":"2024-04-25T08:29:29.906168+0000","last_clean":"2024-04-25T08:29:29.906168+0000","last_became_active":"2024-04-25T08:29:24.037900+0000","last_became_peered":"2024-04-25T08:29:24.037900+0000","last_unstale":"2024-04-25T08:29:29.906168+0000","last_undegraded":"2024-04-25T08:29:29.906168+0000","last_fullsized":"2024-04-25T08:29:29.906168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:35.121139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074948+0000","last_change":"2024-04-25T08:29:24.045041+0000","last_active":"2024-04-25T08:29:29.074948+0000","last_peered":"2024-04-25T08:29:29.074948+0000","last_clean":"2024-04-25T08:29:29.074948+0000","last_became_active":"2024-04-25T08:29:24.044581+0000","last_became_peered":"2024-04-25T08:29:24.044581+0000","last_unstale":"2024-04-25T08:29:29.074948+0000","last_undegraded":"2024-04-25T08:29:29.074948+0000","last_fullsized":"2024-04-25T08:29:29.074948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075863+0000","last_change":"2024-04-25T08:29:24.042402+0000","last_active":"2024-04-25T08:29:29.075863+0000","last_peered":"2024-04-25T08:29:29.075863+0000","last_clean":"2024-04-25T08:29:29.075863+0000","last_became_active":"2024-04-25T08:29:24.042074+0000","last_became_peered":"2024-04-25T08:29:24.042074+0000","last_unstale":"2024-04-25T08:29:29.075863+0000","last_undegraded":"2024-04-25T08:29:29.075863+0000","last_fullsized":"2024-04-25T08:29:29.075863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076327+0000","last_change":"2024-04-25T08:29:24.035522+0000","last_active":"2024-04-25T08:29:29.076327+0000","last_peered":"2024-04-25T08:29:29.076327+0000","last_clean":"2024-04-25T08:29:29.076327+0000","last_became_active":"2024-04-25T08:29:24.035220+0000","last_became_peered":"2024-04-25T08:29:24.035220+0000","last_unstale":"2024-04-25T08:29:29.076327+0000","last_undegraded":"2024-04-25T08:29:29.076327+0000","last_fullsized":"2024-04-25T08:29:29.076327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076595+0000","last_change":"2024-04-25T08:29:24.035459+0000","last_active":"2024-04-25T08:29:29.076595+0000","last_peered":"2024-04-25T08:29:29.076595+0000","last_clean":"2024-04-25T08:29:29.076595+0000","last_became_active":"2024-04-25T08:29:24.035170+0000","last_became_peered":"2024-04-25T08:29:24.035170+0000","last_unstale":"2024-04-25T08:29:29.076595+0000","last_undegraded":"2024-04-25T08:29:29.076595+0000","last_fullsized":"2024-04-25T08:29:29.076595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896232+0000","last_change":"2024-04-25T08:29:24.058439+0000","last_active":"2024-04-25T08:29:29.896232+0000","last_peered":"2024-04-25T08:29:29.896232+0000","last_clean":"2024-04-25T08:29:29.896232+0000","last_became_active":"2024-04-25T08:29:24.058172+0000","last_became_peered":"2024-04-25T08:29:24.058172+0000","last_unstale":"2024-04-25T08:29:29.896232+0000","last_undegraded":"2024-04-25T08:29:29.896232+0000","last_fullsized":"2024-04-25T08:29:29.896232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:24.045871+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:24.045477+0000","last_became_peered":"2024-04-25T08:29:24.045477+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895647+0000","last_change":"2024-04-25T08:29:24.054149+0000","last_active":"2024-04-25T08:29:29.895647+0000","last_peered":"2024-04-25T08:29:29.895647+0000","last_clean":"2024-04-25T08:29:29.895647+0000","last_became_active":"2024-04-25T08:29:24.053893+0000","last_became_peered":"2024-04-25T08:29:24.053893+0000","last_unstale":"2024-04-25T08:29:29.895647+0000","last_undegraded":"2024-04-25T08:29:29.895647+0000","last_fullsized":"2024-04-25T08:29:29.895647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574047+0000","last_change":"2024-04-25T08:29:24.046245+0000","last_active":"2024-04-25T08:29:29.574047+0000","last_peered":"2024-04-25T08:29:29.574047+0000","last_clean":"2024-04-25T08:29:29.574047+0000","last_became_active":"2024-04-25T08:29:24.045892+0000","last_became_peered":"2024-04-25T08:29:24.045892+0000","last_unstale":"2024-04-25T08:29:29.574047+0000","last_undegraded":"2024-04-25T08:29:29.574047+0000","last_fullsized":"2024-04-25T08:29:29.574047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895672+0000","last_change":"2024-04-25T08:29:24.048906+0000","last_active":"2024-04-25T08:29:29.895672+0000","last_peered":"2024-04-25T08:29:29.895672+0000","last_clean":"2024-04-25T08:29:29.895672+0000","last_became_active":"2024-04-25T08:29:24.048634+0000","last_became_peered":"2024-04-25T08:29:24.048634+0000","last_unstale":"2024-04-25T08:29:29.895672+0000","last_undegraded":"2024-04-25T08:29:29.895672+0000","last_fullsized":"2024-04-25T08:29:29.895672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574336+0000","last_change":"2024-04-25T08:29:24.049689+0000","last_active":"2024-04-25T08:29:29.574336+0000","last_peered":"2024-04-25T08:29:29.574336+0000","last_clean":"2024-04-25T08:29:29.574336+0000","last_became_active":"2024-04-25T08:29:24.049210+0000","last_became_peered":"2024-04-25T08:29:24.049210+0000","last_unstale":"2024-04-25T08:29:29.574336+0000","last_undegraded":"2024-04-25T08:29:29.574336+0000","last_fullsized":"2024-04-25T08:29:29.574336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906036+0000","last_change":"2024-04-25T08:29:24.037342+0000","last_active":"2024-04-25T08:29:29.906036+0000","last_peered":"2024-04-25T08:29:29.906036+0000","last_clean":"2024-04-25T08:29:29.906036+0000","last_became_active":"2024-04-25T08:29:24.036991+0000","last_became_peered":"2024-04-25T08:29:24.036991+0000","last_unstale":"2024-04-25T08:29:29.906036+0000","last_undegraded":"2024-04-25T08:29:29.906036+0000","last_fullsized":"2024-04-25T08:29:29.906036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:14.049743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075188+0000","last_change":"2024-04-25T08:29:24.041775+0000","last_active":"2024-04-25T08:29:30.075188+0000","last_peered":"2024-04-25T08:29:30.075188+0000","last_clean":"2024-04-25T08:29:30.075188+0000","last_became_active":"2024-04-25T08:29:24.041474+0000","last_became_peered":"2024-04-25T08:29:24.041474+0000","last_unstale":"2024-04-25T08:29:30.075188+0000","last_undegraded":"2024-04-25T08:29:30.075188+0000","last_fullsized":"2024-04-25T08:29:30.075188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896139+0000","last_change":"2024-04-25T08:29:24.054127+0000","last_active":"2024-04-25T08:29:29.896139+0000","last_peered":"2024-04-25T08:29:29.896139+0000","last_clean":"2024-04-25T08:29:29.896139+0000","last_became_active":"2024-04-25T08:29:24.053854+0000","last_became_peered":"2024-04-25T08:29:24.053854+0000","last_unstale":"2024-04-25T08:29:29.896139+0000","last_undegraded":"2024-04-25T08:29:29.896139+0000","last_fullsized":"2024-04-25T08:29:29.896139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":695037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":104087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":104247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:29:45.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:29:45.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.674+0000 7fbba47e0640 1 Processor -- start 2024-04-25T08:29:45.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.674+0000 7fbba47e0640 1 -- start start 2024-04-25T08:29:45.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba47e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba47e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb9c0d0b10 0x7fbb9c0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba47e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9c0cec00 con 0x7fbb9c0b5f00 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba47e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9c0ce930 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba47e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c0b6450 con 0x7fbb9c15db60 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba2d7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb9c0d0b10 0x7fbb9c0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba2d7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbb9c0d0b10 0x7fbb9c0d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41706/0 (socket says 172.21.15.60:41706) 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.675+0000 7fbba2d7d640 1 -- 172.21.15.60:0/3321936395 learned_addr learned my addr 172.21.15.60:0/3321936395 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba2d7d640 1 -- 172.21.15.60:0/3321936395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 msgr2=0x7fbb9c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba257c640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba2d7d640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba2d7d640 1 -- 172.21.15.60:0/3321936395 >> v1:172.21.15.100:6789/0 conn(0x7fbb9c0b5f00 legacy=0x7fbb9c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba2d7d640 1 -- 172.21.15.60:0/3321936395 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c0ceee0 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba257c640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.676+0000 7fbba2d7d640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0d2270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009e50 tx=0x7fbb9402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aad49cc62db57a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:45.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba157a640 1 -- 172.21.15.60:0/3321936395 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb94036070 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba157a640 1 -- 172.21.15.60:0/3321936395 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb94031600 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba157a640 1 -- 172.21.15.60:0/3321936395 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb940319d0 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 -- 172.21.15.60:0/3321936395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 msgr2=0x7fbb9c0d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0d2270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009e50 tx=0x7fbb9402a680 comp rx=0 tx=0).stop 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 -- 172.21.15.60:0/3321936395 shutdown_connections 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0d2270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 --2- 172.21.15.60:0/3321936395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 -- 172.21.15.60:0/3321936395 >> 172.21.15.60:0/3321936395 conn(0x7fbb9c061600 msgr2=0x7fbb9c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 -- 172.21.15.60:0/3321936395 shutdown_connections 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.677+0000 7fbba47e0640 1 -- 172.21.15.60:0/3321936395 wait complete. 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.678+0000 7fbba47e0640 1 Processor -- start 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.678+0000 7fbba47e0640 1 -- start start 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba47e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0b5f00 0x7fbb9c158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba47e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba47e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba47e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c15df40 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba47e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c0d1d50 con 0x7fbb9c0b5f00 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba47e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c15e0c0 con 0x7fbb9c15db60 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba1d7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba1d7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41714/0 (socket says 172.21.15.60:41714) 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba1d7b640 1 -- 172.21.15.60:0/2215305465 learned_addr learned my addr 172.21.15.60:0/2215305465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba257c640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0b5f00 0x7fbb9c158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.679+0000 7fbba2d7d640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba1d7b640 1 -- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 msgr2=0x7fbb9c0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba1d7b640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba1d7b640 1 -- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0b5f00 msgr2=0x7fbb9c158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba1d7b640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0b5f00 0x7fbb9c158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba1d7b640 1 -- 172.21.15.60:0/2215305465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c15e4a0 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba1d7b640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0cdcf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00c440 tx=0x7fbb8c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0052d0 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c005470 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c007240 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c15e740 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.680+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c15ed00 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.681+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c073270 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.681+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb8c0073e0 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.682+0000 7fbb92ffd640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb8006b4d0 0x7fbb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:45.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.682+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbb8c089950 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.682+0000 7fbba257c640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb8006b4d0 0x7fbb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:45.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.682+0000 7fbba257c640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb8006b4d0 0x7fbb8006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800a950 tx=0x7fbb98008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:45.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.686+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb8c058520 con 0x7fbb9c0d0b10 2024-04-25T08:29:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.863+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb9c0d1860 con 0x7fbb8006b4d0 2024-04-25T08:29:45.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.871+0000 7fbb92ffd640 1 -- 172.21.15.60:0/2215305465 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fbb9c0d1860 con 0x7fbb8006b4d0 2024-04-25T08:29:45.873 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:45.875 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:29:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb8006b4d0 msgr2=0x7fbb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb8006b4d0 0x7fbb8006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800a950 tx=0x7fbb98008040 comp rx=0 tx=0).stop 2024-04-25T08:29:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 msgr2=0x7fbb9c0cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0cdcf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00c440 tx=0x7fbb8c001e70 comp rx=0 tx=0).stop 2024-04-25T08:29:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 shutdown_connections 2024-04-25T08:29:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb9c15db60 0x7fbb9c0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb9c0d0b10 0x7fbb9c0cdcf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb8006b4d0 0x7fbb8006d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 --2- 172.21.15.60:0/2215305465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb9c0b5f00 0x7fbb9c158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.874+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 >> 172.21.15.60:0/2215305465 conn(0x7fbb9c061600 msgr2=0x7fbb9c061bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.875+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 shutdown_connections 2024-04-25T08:29:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:45.875+0000 7fbba47e0640 1 -- 172.21.15.60:0/2215305465 wait complete. 2024-04-25T08:29:45.883 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T08:29:45.401871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231920,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742800,"statfs":{"total":773094113280,"available":772856627200,"internally_reserved":0,"allocated":3350528,"data_stored":2022712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003492"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.028601+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.028315+0000","last_became_peered":"2024-04-25T08:29:23.028315+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075712+0000","last_change":"2024-04-25T08:29:24.048532+0000","last_active":"2024-04-25T08:29:29.075712+0000","last_peered":"2024-04-25T08:29:29.075712+0000","last_clean":"2024-04-25T08:29:29.075712+0000","last_became_active":"2024-04-25T08:29:24.046997+0000","last_became_peered":"2024-04-25T08:29:24.046997+0000","last_unstale":"2024-04-25T08:29:29.075712+0000","last_undegraded":"2024-04-25T08:29:29.075712+0000","last_fullsized":"2024-04-25T08:29:29.075712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074646+0000","last_change":"2024-04-25T08:29:28.085554+0000","last_active":"2024-04-25T08:29:29.074646+0000","last_peered":"2024-04-25T08:29:29.074646+0000","last_clean":"2024-04-25T08:29:29.074646+0000","last_became_active":"2024-04-25T08:29:28.085164+0000","last_became_peered":"2024-04-25T08:29:28.085164+0000","last_unstale":"2024-04-25T08:29:29.074646+0000","last_undegraded":"2024-04-25T08:29:29.074646+0000","last_fullsized":"2024-04-25T08:29:29.074646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:03.849914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075840+0000","last_change":"2024-04-25T08:29:23.035840+0000","last_active":"2024-04-25T08:29:29.075840+0000","last_peered":"2024-04-25T08:29:29.075840+0000","last_clean":"2024-04-25T08:29:29.075840+0000","last_became_active":"2024-04-25T08:29:23.035532+0000","last_became_peered":"2024-04-25T08:29:23.035532+0000","last_unstale":"2024-04-25T08:29:29.075840+0000","last_undegraded":"2024-04-25T08:29:29.075840+0000","last_fullsized":"2024-04-25T08:29:29.075840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906355+0000","last_change":"2024-04-25T08:29:24.037371+0000","last_active":"2024-04-25T08:29:29.906355+0000","last_peered":"2024-04-25T08:29:29.906355+0000","last_clean":"2024-04-25T08:29:29.906355+0000","last_became_active":"2024-04-25T08:29:24.037043+0000","last_became_peered":"2024-04-25T08:29:24.037043+0000","last_unstale":"2024-04-25T08:29:29.906355+0000","last_undegraded":"2024-04-25T08:29:29.906355+0000","last_fullsized":"2024-04-25T08:29:29.906355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.900465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074732+0000","last_change":"2024-04-25T08:29:28.080997+0000","last_active":"2024-04-25T08:29:29.074732+0000","last_peered":"2024-04-25T08:29:29.074732+0000","last_clean":"2024-04-25T08:29:29.074732+0000","last_became_active":"2024-04-25T08:29:28.080573+0000","last_became_peered":"2024-04-25T08:29:28.080573+0000","last_unstale":"2024-04-25T08:29:29.074732+0000","last_undegraded":"2024-04-25T08:29:29.074732+0000","last_fullsized":"2024-04-25T08:29:29.074732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:48.362958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895961+0000","last_change":"2024-04-25T08:29:23.041517+0000","last_active":"2024-04-25T08:29:29.895961+0000","last_peered":"2024-04-25T08:29:29.895961+0000","last_clean":"2024-04-25T08:29:29.895961+0000","last_became_active":"2024-04-25T08:29:23.041191+0000","last_became_peered":"2024-04-25T08:29:23.041191+0000","last_unstale":"2024-04-25T08:29:29.895961+0000","last_undegraded":"2024-04-25T08:29:29.895961+0000","last_fullsized":"2024-04-25T08:29:29.895961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574369+0000","last_change":"2024-04-25T08:29:24.042793+0000","last_active":"2024-04-25T08:29:29.574369+0000","last_peered":"2024-04-25T08:29:29.574369+0000","last_clean":"2024-04-25T08:29:29.574369+0000","last_became_active":"2024-04-25T08:29:24.042026+0000","last_became_peered":"2024-04-25T08:29:24.042026+0000","last_unstale":"2024-04-25T08:29:29.574369+0000","last_undegraded":"2024-04-25T08:29:29.574369+0000","last_fullsized":"2024-04-25T08:29:29.574369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075189+0000","last_change":"2024-04-25T08:29:28.081365+0000","last_active":"2024-04-25T08:29:29.075189+0000","last_peered":"2024-04-25T08:29:29.075189+0000","last_clean":"2024-04-25T08:29:29.075189+0000","last_became_active":"2024-04-25T08:29:28.080901+0000","last_became_peered":"2024-04-25T08:29:28.080901+0000","last_unstale":"2024-04-25T08:29:29.075189+0000","last_undegraded":"2024-04-25T08:29:29.075189+0000","last_fullsized":"2024-04-25T08:29:29.075189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.649430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895778+0000","last_change":"2024-04-25T08:29:23.036549+0000","last_active":"2024-04-25T08:29:29.895778+0000","last_peered":"2024-04-25T08:29:29.895778+0000","last_clean":"2024-04-25T08:29:29.895778+0000","last_became_active":"2024-04-25T08:29:23.036323+0000","last_became_peered":"2024-04-25T08:29:23.036323+0000","last_unstale":"2024-04-25T08:29:29.895778+0000","last_undegraded":"2024-04-25T08:29:29.895778+0000","last_fullsized":"2024-04-25T08:29:29.895778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574394+0000","last_change":"2024-04-25T08:29:24.041522+0000","last_active":"2024-04-25T08:29:29.574394+0000","last_peered":"2024-04-25T08:29:29.574394+0000","last_clean":"2024-04-25T08:29:29.574394+0000","last_became_active":"2024-04-25T08:29:24.041242+0000","last_became_peered":"2024-04-25T08:29:24.041242+0000","last_unstale":"2024-04-25T08:29:29.574394+0000","last_undegraded":"2024-04-25T08:29:29.574394+0000","last_fullsized":"2024-04-25T08:29:29.574394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075282+0000","last_change":"2024-04-25T08:29:28.081585+0000","last_active":"2024-04-25T08:29:29.075282+0000","last_peered":"2024-04-25T08:29:29.075282+0000","last_clean":"2024-04-25T08:29:29.075282+0000","last_became_active":"2024-04-25T08:29:28.080227+0000","last_became_peered":"2024-04-25T08:29:28.080227+0000","last_unstale":"2024-04-25T08:29:29.075282+0000","last_undegraded":"2024-04-25T08:29:29.075282+0000","last_fullsized":"2024-04-25T08:29:29.075282+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075044+0000","last_change":"2024-04-25T08:29:24.040005+0000","last_active":"2024-04-25T08:29:29.075044+0000","last_peered":"2024-04-25T08:29:29.075044+0000","last_clean":"2024-04-25T08:29:29.075044+0000","last_became_active":"2024-04-25T08:29:24.039678+0000","last_became_peered":"2024-04-25T08:29:24.039678+0000","last_unstale":"2024-04-25T08:29:29.075044+0000","last_undegraded":"2024-04-25T08:29:29.075044+0000","last_fullsized":"2024-04-25T08:29:29.075044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895838+0000","last_change":"2024-04-25T08:29:23.034031+0000","last_active":"2024-04-25T08:29:29.895838+0000","last_peered":"2024-04-25T08:29:29.895838+0000","last_clean":"2024-04-25T08:29:29.895838+0000","last_became_active":"2024-04-25T08:29:23.033672+0000","last_became_peered":"2024-04-25T08:29:23.033672+0000","last_unstale":"2024-04-25T08:29:29.895838+0000","last_undegraded":"2024-04-25T08:29:29.895838+0000","last_fullsized":"2024-04-25T08:29:29.895838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.725230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076035+0000","last_change":"2024-04-25T08:29:28.067431+0000","last_active":"2024-04-25T08:29:29.076035+0000","last_peered":"2024-04-25T08:29:29.076035+0000","last_clean":"2024-04-25T08:29:29.076035+0000","last_became_active":"2024-04-25T08:29:28.066968+0000","last_became_peered":"2024-04-25T08:29:28.066968+0000","last_unstale":"2024-04-25T08:29:29.076035+0000","last_undegraded":"2024-04-25T08:29:29.076035+0000","last_fullsized":"2024-04-25T08:29:29.076035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075071+0000","last_change":"2024-04-25T08:29:24.052628+0000","last_active":"2024-04-25T08:29:29.075071+0000","last_peered":"2024-04-25T08:29:29.075071+0000","last_clean":"2024-04-25T08:29:29.075071+0000","last_became_active":"2024-04-25T08:29:24.052111+0000","last_became_peered":"2024-04-25T08:29:24.052111+0000","last_unstale":"2024-04-25T08:29:29.075071+0000","last_undegraded":"2024-04-25T08:29:29.075071+0000","last_fullsized":"2024-04-25T08:29:29.075071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074777+0000","last_change":"2024-04-25T08:29:23.040432+0000","last_active":"2024-04-25T08:29:30.074777+0000","last_peered":"2024-04-25T08:29:30.074777+0000","last_clean":"2024-04-25T08:29:30.074777+0000","last_became_active":"2024-04-25T08:29:23.040172+0000","last_became_peered":"2024-04-25T08:29:23.040172+0000","last_unstale":"2024-04-25T08:29:30.074777+0000","last_undegraded":"2024-04-25T08:29:30.074777+0000","last_fullsized":"2024-04-25T08:29:30.074777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075200+0000","last_change":"2024-04-25T08:29:28.081025+0000","last_active":"2024-04-25T08:29:29.075200+0000","last_peered":"2024-04-25T08:29:29.075200+0000","last_clean":"2024-04-25T08:29:29.075200+0000","last_became_active":"2024-04-25T08:29:28.080481+0000","last_became_peered":"2024-04-25T08:29:28.080481+0000","last_unstale":"2024-04-25T08:29:29.075200+0000","last_undegraded":"2024-04-25T08:29:29.075200+0000","last_fullsized":"2024-04-25T08:29:29.075200+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074861+0000","last_change":"2024-04-25T08:29:24.054299+0000","last_active":"2024-04-25T08:29:29.074861+0000","last_peered":"2024-04-25T08:29:29.074861+0000","last_clean":"2024-04-25T08:29:29.074861+0000","last_became_active":"2024-04-25T08:29:24.053996+0000","last_became_peered":"2024-04-25T08:29:24.053996+0000","last_unstale":"2024-04-25T08:29:29.074861+0000","last_undegraded":"2024-04-25T08:29:29.074861+0000","last_fullsized":"2024-04-25T08:29:29.074861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074526+0000","last_change":"2024-04-25T08:29:23.038382+0000","last_active":"2024-04-25T08:29:30.074526+0000","last_peered":"2024-04-25T08:29:30.074526+0000","last_clean":"2024-04-25T08:29:30.074526+0000","last_became_active":"2024-04-25T08:29:23.038060+0000","last_became_peered":"2024-04-25T08:29:23.038060+0000","last_unstale":"2024-04-25T08:29:30.074526+0000","last_undegraded":"2024-04-25T08:29:30.074526+0000","last_fullsized":"2024-04-25T08:29:30.074526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075123+0000","last_change":"2024-04-25T08:29:28.085260+0000","last_active":"2024-04-25T08:29:30.075123+0000","last_peered":"2024-04-25T08:29:30.075123+0000","last_clean":"2024-04-25T08:29:30.075123+0000","last_became_active":"2024-04-25T08:29:28.084884+0000","last_became_peered":"2024-04-25T08:29:28.084884+0000","last_unstale":"2024-04-25T08:29:30.075123+0000","last_undegraded":"2024-04-25T08:29:30.075123+0000","last_fullsized":"2024-04-25T08:29:30.075123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075420+0000","last_change":"2024-04-25T08:29:24.035523+0000","last_active":"2024-04-25T08:29:29.075420+0000","last_peered":"2024-04-25T08:29:29.075420+0000","last_clean":"2024-04-25T08:29:29.075420+0000","last_became_active":"2024-04-25T08:29:24.035185+0000","last_became_peered":"2024-04-25T08:29:24.035185+0000","last_unstale":"2024-04-25T08:29:29.075420+0000","last_undegraded":"2024-04-25T08:29:29.075420+0000","last_fullsized":"2024-04-25T08:29:29.075420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.274896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895386+0000","last_change":"2024-04-25T08:29:23.037104+0000","last_active":"2024-04-25T08:29:29.895386+0000","last_peered":"2024-04-25T08:29:29.895386+0000","last_clean":"2024-04-25T08:29:29.895386+0000","last_became_active":"2024-04-25T08:29:23.035864+0000","last_became_peered":"2024-04-25T08:29:23.035864+0000","last_unstale":"2024-04-25T08:29:29.895386+0000","last_undegraded":"2024-04-25T08:29:29.895386+0000","last_fullsized":"2024-04-25T08:29:29.895386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075821+0000","last_change":"2024-04-25T08:29:28.080714+0000","last_active":"2024-04-25T08:29:29.075821+0000","last_peered":"2024-04-25T08:29:29.075821+0000","last_clean":"2024-04-25T08:29:29.075821+0000","last_became_active":"2024-04-25T08:29:28.080291+0000","last_became_peered":"2024-04-25T08:29:28.080291+0000","last_unstale":"2024-04-25T08:29:29.075821+0000","last_undegraded":"2024-04-25T08:29:29.075821+0000","last_fullsized":"2024-04-25T08:29:29.075821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906353+0000","last_change":"2024-04-25T08:29:24.040261+0000","last_active":"2024-04-25T08:29:29.906353+0000","last_peered":"2024-04-25T08:29:29.906353+0000","last_clean":"2024-04-25T08:29:29.906353+0000","last_became_active":"2024-04-25T08:29:24.039953+0000","last_became_peered":"2024-04-25T08:29:24.039953+0000","last_unstale":"2024-04-25T08:29:29.906353+0000","last_undegraded":"2024-04-25T08:29:29.906353+0000","last_fullsized":"2024-04-25T08:29:29.906353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076570+0000","last_change":"2024-04-25T08:29:23.038961+0000","last_active":"2024-04-25T08:29:29.076570+0000","last_peered":"2024-04-25T08:29:29.076570+0000","last_clean":"2024-04-25T08:29:29.076570+0000","last_became_active":"2024-04-25T08:29:23.038722+0000","last_became_peered":"2024-04-25T08:29:23.038722+0000","last_unstale":"2024-04-25T08:29:29.076570+0000","last_undegraded":"2024-04-25T08:29:29.076570+0000","last_fullsized":"2024-04-25T08:29:29.076570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074879+0000","last_change":"2024-04-25T08:29:28.073848+0000","last_active":"2024-04-25T08:29:29.074879+0000","last_peered":"2024-04-25T08:29:29.074879+0000","last_clean":"2024-04-25T08:29:29.074879+0000","last_became_active":"2024-04-25T08:29:28.073489+0000","last_became_peered":"2024-04-25T08:29:28.073489+0000","last_unstale":"2024-04-25T08:29:29.074879+0000","last_undegraded":"2024-04-25T08:29:29.074879+0000","last_fullsized":"2024-04-25T08:29:29.074879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:54.262561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076145+0000","last_change":"2024-04-25T08:29:24.057631+0000","last_active":"2024-04-25T08:29:29.076145+0000","last_peered":"2024-04-25T08:29:29.076145+0000","last_clean":"2024-04-25T08:29:29.076145+0000","last_became_active":"2024-04-25T08:29:24.057387+0000","last_became_peered":"2024-04-25T08:29:24.057387+0000","last_unstale":"2024-04-25T08:29:29.076145+0000","last_undegraded":"2024-04-25T08:29:29.076145+0000","last_fullsized":"2024-04-25T08:29:29.076145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075211+0000","last_change":"2024-04-25T08:29:23.041608+0000","last_active":"2024-04-25T08:29:30.075211+0000","last_peered":"2024-04-25T08:29:30.075211+0000","last_clean":"2024-04-25T08:29:30.075211+0000","last_became_active":"2024-04-25T08:29:23.041305+0000","last_became_peered":"2024-04-25T08:29:23.041305+0000","last_unstale":"2024-04-25T08:29:30.075211+0000","last_undegraded":"2024-04-25T08:29:30.075211+0000","last_fullsized":"2024-04-25T08:29:30.075211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:28.085603+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:28.084444+0000","last_became_peered":"2024-04-25T08:29:28.084444+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:39.462705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075317+0000","last_change":"2024-04-25T08:29:24.036286+0000","last_active":"2024-04-25T08:29:30.075317+0000","last_peered":"2024-04-25T08:29:30.075317+0000","last_clean":"2024-04-25T08:29:30.075317+0000","last_became_active":"2024-04-25T08:29:24.035947+0000","last_became_peered":"2024-04-25T08:29:24.035947+0000","last_unstale":"2024-04-25T08:29:30.075317+0000","last_undegraded":"2024-04-25T08:29:30.075317+0000","last_fullsized":"2024-04-25T08:29:30.075317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896087+0000","last_change":"2024-04-25T08:29:23.035930+0000","last_active":"2024-04-25T08:29:29.896087+0000","last_peered":"2024-04-25T08:29:29.896087+0000","last_clean":"2024-04-25T08:29:29.896087+0000","last_became_active":"2024-04-25T08:29:23.035685+0000","last_became_peered":"2024-04-25T08:29:23.035685+0000","last_unstale":"2024-04-25T08:29:29.896087+0000","last_undegraded":"2024-04-25T08:29:29.896087+0000","last_fullsized":"2024-04-25T08:29:29.896087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906315+0000","last_change":"2024-04-25T08:29:28.076556+0000","last_active":"2024-04-25T08:29:29.906315+0000","last_peered":"2024-04-25T08:29:29.906315+0000","last_clean":"2024-04-25T08:29:29.906315+0000","last_became_active":"2024-04-25T08:29:28.076157+0000","last_became_peered":"2024-04-25T08:29:28.076157+0000","last_unstale":"2024-04-25T08:29:29.906315+0000","last_undegraded":"2024-04-25T08:29:29.906315+0000","last_fullsized":"2024-04-25T08:29:29.906315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574536+0000","last_change":"2024-04-25T08:29:24.055822+0000","last_active":"2024-04-25T08:29:29.574536+0000","last_peered":"2024-04-25T08:29:29.574536+0000","last_clean":"2024-04-25T08:29:29.574536+0000","last_became_active":"2024-04-25T08:29:24.055087+0000","last_became_peered":"2024-04-25T08:29:24.055087+0000","last_unstale":"2024-04-25T08:29:29.574536+0000","last_undegraded":"2024-04-25T08:29:29.574536+0000","last_fullsized":"2024-04-25T08:29:29.574536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075321+0000","last_change":"2024-04-25T08:29:23.037612+0000","last_active":"2024-04-25T08:29:30.075321+0000","last_peered":"2024-04-25T08:29:30.075321+0000","last_clean":"2024-04-25T08:29:30.075321+0000","last_became_active":"2024-04-25T08:29:23.037304+0000","last_became_peered":"2024-04-25T08:29:23.037304+0000","last_unstale":"2024-04-25T08:29:30.075321+0000","last_undegraded":"2024-04-25T08:29:30.075321+0000","last_fullsized":"2024-04-25T08:29:30.075321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074438+0000","last_change":"2024-04-25T08:29:28.068007+0000","last_active":"2024-04-25T08:29:30.074438+0000","last_peered":"2024-04-25T08:29:30.074438+0000","last_clean":"2024-04-25T08:29:30.074438+0000","last_became_active":"2024-04-25T08:29:28.067586+0000","last_became_peered":"2024-04-25T08:29:28.067586+0000","last_unstale":"2024-04-25T08:29:30.074438+0000","last_undegraded":"2024-04-25T08:29:30.074438+0000","last_fullsized":"2024-04-25T08:29:30.074438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075717+0000","last_change":"2024-04-25T08:29:24.038433+0000","last_active":"2024-04-25T08:29:29.075717+0000","last_peered":"2024-04-25T08:29:29.075717+0000","last_clean":"2024-04-25T08:29:29.075717+0000","last_became_active":"2024-04-25T08:29:24.038164+0000","last_became_peered":"2024-04-25T08:29:24.038164+0000","last_unstale":"2024-04-25T08:29:29.075717+0000","last_undegraded":"2024-04-25T08:29:29.075717+0000","last_fullsized":"2024-04-25T08:29:29.075717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574356+0000","last_change":"2024-04-25T08:29:23.036328+0000","last_active":"2024-04-25T08:29:29.574356+0000","last_peered":"2024-04-25T08:29:29.574356+0000","last_clean":"2024-04-25T08:29:29.574356+0000","last_became_active":"2024-04-25T08:29:23.036072+0000","last_became_peered":"2024-04-25T08:29:23.036072+0000","last_unstale":"2024-04-25T08:29:29.574356+0000","last_undegraded":"2024-04-25T08:29:29.574356+0000","last_fullsized":"2024-04-25T08:29:29.574356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075255+0000","last_change":"2024-04-25T08:29:28.081150+0000","last_active":"2024-04-25T08:29:30.075255+0000","last_peered":"2024-04-25T08:29:30.075255+0000","last_clean":"2024-04-25T08:29:30.075255+0000","last_became_active":"2024-04-25T08:29:28.080621+0000","last_became_peered":"2024-04-25T08:29:28.080621+0000","last_unstale":"2024-04-25T08:29:30.075255+0000","last_undegraded":"2024-04-25T08:29:30.075255+0000","last_fullsized":"2024-04-25T08:29:30.075255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074632+0000","last_change":"2024-04-25T08:29:24.036313+0000","last_active":"2024-04-25T08:29:30.074632+0000","last_peered":"2024-04-25T08:29:30.074632+0000","last_clean":"2024-04-25T08:29:30.074632+0000","last_became_active":"2024-04-25T08:29:24.035892+0000","last_became_peered":"2024-04-25T08:29:24.035892+0000","last_unstale":"2024-04-25T08:29:30.074632+0000","last_undegraded":"2024-04-25T08:29:30.074632+0000","last_fullsized":"2024-04-25T08:29:30.074632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074251+0000","last_change":"2024-04-25T08:29:23.033190+0000","last_active":"2024-04-25T08:29:30.074251+0000","last_peered":"2024-04-25T08:29:30.074251+0000","last_clean":"2024-04-25T08:29:30.074251+0000","last_became_active":"2024-04-25T08:29:23.032822+0000","last_became_peered":"2024-04-25T08:29:23.032822+0000","last_unstale":"2024-04-25T08:29:30.074251+0000","last_undegraded":"2024-04-25T08:29:30.074251+0000","last_fullsized":"2024-04-25T08:29:30.074251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075145+0000","last_change":"2024-04-25T08:29:28.085006+0000","last_active":"2024-04-25T08:29:30.075145+0000","last_peered":"2024-04-25T08:29:30.075145+0000","last_clean":"2024-04-25T08:29:30.075145+0000","last_became_active":"2024-04-25T08:29:28.084572+0000","last_became_peered":"2024-04-25T08:29:28.084572+0000","last_unstale":"2024-04-25T08:29:30.075145+0000","last_undegraded":"2024-04-25T08:29:30.075145+0000","last_fullsized":"2024-04-25T08:29:30.075145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:22.084545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906058+0000","last_change":"2024-04-25T08:29:24.043403+0000","last_active":"2024-04-25T08:29:29.906058+0000","last_peered":"2024-04-25T08:29:29.906058+0000","last_clean":"2024-04-25T08:29:29.906058+0000","last_became_active":"2024-04-25T08:29:24.042964+0000","last_became_peered":"2024-04-25T08:29:24.042964+0000","last_unstale":"2024-04-25T08:29:29.906058+0000","last_undegraded":"2024-04-25T08:29:29.906058+0000","last_fullsized":"2024-04-25T08:29:29.906058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895394+0000","last_change":"2024-04-25T08:29:23.035024+0000","last_active":"2024-04-25T08:29:29.895394+0000","last_peered":"2024-04-25T08:29:29.895394+0000","last_clean":"2024-04-25T08:29:29.895394+0000","last_became_active":"2024-04-25T08:29:23.034761+0000","last_became_peered":"2024-04-25T08:29:23.034761+0000","last_unstale":"2024-04-25T08:29:29.895394+0000","last_undegraded":"2024-04-25T08:29:29.895394+0000","last_fullsized":"2024-04-25T08:29:29.895394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906299+0000","last_change":"2024-04-25T08:29:28.078207+0000","last_active":"2024-04-25T08:29:29.906299+0000","last_peered":"2024-04-25T08:29:29.906299+0000","last_clean":"2024-04-25T08:29:29.906299+0000","last_became_active":"2024-04-25T08:29:28.077777+0000","last_became_peered":"2024-04-25T08:29:28.077777+0000","last_unstale":"2024-04-25T08:29:29.906299+0000","last_undegraded":"2024-04-25T08:29:29.906299+0000","last_fullsized":"2024-04-25T08:29:29.906299+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.389434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.048117+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.047733+0000","last_became_peered":"2024-04-25T08:29:24.047733+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574166+0000","last_change":"2024-04-25T08:29:23.036154+0000","last_active":"2024-04-25T08:29:29.574166+0000","last_peered":"2024-04-25T08:29:29.574166+0000","last_clean":"2024-04-25T08:29:29.574166+0000","last_became_active":"2024-04-25T08:29:23.035873+0000","last_became_peered":"2024-04-25T08:29:23.035873+0000","last_unstale":"2024-04-25T08:29:29.574166+0000","last_undegraded":"2024-04-25T08:29:29.574166+0000","last_fullsized":"2024-04-25T08:29:29.574166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895594+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:29:29.895594+0000","last_peered":"2024-04-25T08:29:29.895594+0000","last_clean":"2024-04-25T08:29:29.895594+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:29:29.895594+0000","last_undegraded":"2024-04-25T08:29:29.895594+0000","last_fullsized":"2024-04-25T08:29:29.895594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076370+0000","last_change":"2024-04-25T08:29:28.076816+0000","last_active":"2024-04-25T08:29:29.076370+0000","last_peered":"2024-04-25T08:29:29.076370+0000","last_clean":"2024-04-25T08:29:29.076370+0000","last_became_active":"2024-04-25T08:29:28.076348+0000","last_became_peered":"2024-04-25T08:29:28.076348+0000","last_unstale":"2024-04-25T08:29:29.076370+0000","last_undegraded":"2024-04-25T08:29:29.076370+0000","last_fullsized":"2024-04-25T08:29:29.076370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895804+0000","last_change":"2024-04-25T08:29:24.047938+0000","last_active":"2024-04-25T08:29:29.895804+0000","last_peered":"2024-04-25T08:29:29.895804+0000","last_clean":"2024-04-25T08:29:29.895804+0000","last_became_active":"2024-04-25T08:29:24.047670+0000","last_became_peered":"2024-04-25T08:29:24.047670+0000","last_unstale":"2024-04-25T08:29:29.895804+0000","last_undegraded":"2024-04-25T08:29:29.895804+0000","last_fullsized":"2024-04-25T08:29:29.895804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074744+0000","last_change":"2024-04-25T08:29:23.037162+0000","last_active":"2024-04-25T08:29:30.074744+0000","last_peered":"2024-04-25T08:29:30.074744+0000","last_clean":"2024-04-25T08:29:30.074744+0000","last_became_active":"2024-04-25T08:29:23.036850+0000","last_became_peered":"2024-04-25T08:29:23.036850+0000","last_unstale":"2024-04-25T08:29:30.074744+0000","last_undegraded":"2024-04-25T08:29:30.074744+0000","last_fullsized":"2024-04-25T08:29:30.074744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:44.847488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074972+0000","last_change":"2024-04-25T08:29:28.073263+0000","last_active":"2024-04-25T08:29:29.074972+0000","last_peered":"2024-04-25T08:29:29.074972+0000","last_clean":"2024-04-25T08:29:29.074972+0000","last_became_active":"2024-04-25T08:29:28.072784+0000","last_became_peered":"2024-04-25T08:29:28.072784+0000","last_unstale":"2024-04-25T08:29:29.074972+0000","last_undegraded":"2024-04-25T08:29:29.074972+0000","last_fullsized":"2024-04-25T08:29:29.074972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:56.666318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574458+0000","last_change":"2024-04-25T08:29:24.056206+0000","last_active":"2024-04-25T08:29:29.574458+0000","last_peered":"2024-04-25T08:29:29.574458+0000","last_clean":"2024-04-25T08:29:29.574458+0000","last_became_active":"2024-04-25T08:29:24.055557+0000","last_became_peered":"2024-04-25T08:29:24.055557+0000","last_unstale":"2024-04-25T08:29:29.574458+0000","last_undegraded":"2024-04-25T08:29:29.574458+0000","last_fullsized":"2024-04-25T08:29:29.574458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:18.051764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075008+0000","last_change":"2024-04-25T08:29:23.031709+0000","last_active":"2024-04-25T08:29:30.075008+0000","last_peered":"2024-04-25T08:29:30.075008+0000","last_clean":"2024-04-25T08:29:30.075008+0000","last_became_active":"2024-04-25T08:29:23.031419+0000","last_became_peered":"2024-04-25T08:29:23.031419+0000","last_unstale":"2024-04-25T08:29:30.075008+0000","last_undegraded":"2024-04-25T08:29:30.075008+0000","last_fullsized":"2024-04-25T08:29:30.075008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906475+0000","last_change":"2024-04-25T08:29:28.070276+0000","last_active":"2024-04-25T08:29:29.906475+0000","last_peered":"2024-04-25T08:29:29.906475+0000","last_clean":"2024-04-25T08:29:29.906475+0000","last_became_active":"2024-04-25T08:29:28.069869+0000","last_became_peered":"2024-04-25T08:29:28.069869+0000","last_unstale":"2024-04-25T08:29:29.906475+0000","last_undegraded":"2024-04-25T08:29:29.906475+0000","last_fullsized":"2024-04-25T08:29:29.906475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:22.617039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896237+0000","last_change":"2024-04-25T08:29:24.057274+0000","last_active":"2024-04-25T08:29:29.896237+0000","last_peered":"2024-04-25T08:29:29.896237+0000","last_clean":"2024-04-25T08:29:29.896237+0000","last_became_active":"2024-04-25T08:29:24.056998+0000","last_became_peered":"2024-04-25T08:29:24.056998+0000","last_unstale":"2024-04-25T08:29:29.896237+0000","last_undegraded":"2024-04-25T08:29:29.896237+0000","last_fullsized":"2024-04-25T08:29:29.896237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:18.584258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074824+0000","last_change":"2024-04-25T08:29:23.039945+0000","last_active":"2024-04-25T08:29:30.074824+0000","last_peered":"2024-04-25T08:29:30.074824+0000","last_clean":"2024-04-25T08:29:30.074824+0000","last_became_active":"2024-04-25T08:29:23.039563+0000","last_became_peered":"2024-04-25T08:29:23.039563+0000","last_unstale":"2024-04-25T08:29:30.074824+0000","last_undegraded":"2024-04-25T08:29:30.074824+0000","last_fullsized":"2024-04-25T08:29:30.074824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573881+0000","last_change":"2024-04-25T08:29:28.076465+0000","last_active":"2024-04-25T08:29:29.573881+0000","last_peered":"2024-04-25T08:29:29.573881+0000","last_clean":"2024-04-25T08:29:29.573881+0000","last_became_active":"2024-04-25T08:29:28.076064+0000","last_became_peered":"2024-04-25T08:29:28.076064+0000","last_unstale":"2024-04-25T08:29:29.573881+0000","last_undegraded":"2024-04-25T08:29:29.573881+0000","last_fullsized":"2024-04-25T08:29:29.573881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896089+0000","last_change":"2024-04-25T08:29:24.050215+0000","last_active":"2024-04-25T08:29:29.896089+0000","last_peered":"2024-04-25T08:29:29.896089+0000","last_clean":"2024-04-25T08:29:29.896089+0000","last_became_active":"2024-04-25T08:29:24.048694+0000","last_became_peered":"2024-04-25T08:29:24.048694+0000","last_unstale":"2024-04-25T08:29:29.896089+0000","last_undegraded":"2024-04-25T08:29:29.896089+0000","last_fullsized":"2024-04-25T08:29:29.896089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075997+0000","last_change":"2024-04-25T08:29:23.033858+0000","last_active":"2024-04-25T08:29:29.075997+0000","last_peered":"2024-04-25T08:29:29.075997+0000","last_clean":"2024-04-25T08:29:29.075997+0000","last_became_active":"2024-04-25T08:29:23.033599+0000","last_became_peered":"2024-04-25T08:29:23.033599+0000","last_unstale":"2024-04-25T08:29:29.075997+0000","last_undegraded":"2024-04-25T08:29:29.075997+0000","last_fullsized":"2024-04-25T08:29:29.075997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:05.732421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074730+0000","last_change":"2024-04-25T08:29:28.081641+0000","last_active":"2024-04-25T08:29:29.074730+0000","last_peered":"2024-04-25T08:29:29.074730+0000","last_clean":"2024-04-25T08:29:29.074730+0000","last_became_active":"2024-04-25T08:29:28.081220+0000","last_became_peered":"2024-04-25T08:29:28.081220+0000","last_unstale":"2024-04-25T08:29:29.074730+0000","last_undegraded":"2024-04-25T08:29:29.074730+0000","last_fullsized":"2024-04-25T08:29:29.074730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:51.749033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895682+0000","last_change":"2024-04-25T08:29:24.050072+0000","last_active":"2024-04-25T08:29:29.895682+0000","last_peered":"2024-04-25T08:29:29.895682+0000","last_clean":"2024-04-25T08:29:29.895682+0000","last_became_active":"2024-04-25T08:29:24.049773+0000","last_became_peered":"2024-04-25T08:29:24.049773+0000","last_unstale":"2024-04-25T08:29:29.895682+0000","last_undegraded":"2024-04-25T08:29:29.895682+0000","last_fullsized":"2024-04-25T08:29:29.895682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074062+0000","last_change":"2024-04-25T08:29:23.035476+0000","last_active":"2024-04-25T08:29:30.074062+0000","last_peered":"2024-04-25T08:29:30.074062+0000","last_clean":"2024-04-25T08:29:30.074062+0000","last_became_active":"2024-04-25T08:29:23.035129+0000","last_became_peered":"2024-04-25T08:29:23.035129+0000","last_unstale":"2024-04-25T08:29:30.074062+0000","last_undegraded":"2024-04-25T08:29:30.074062+0000","last_fullsized":"2024-04-25T08:29:30.074062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:52.454939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906185+0000","last_change":"2024-04-25T08:29:28.076678+0000","last_active":"2024-04-25T08:29:29.906185+0000","last_peered":"2024-04-25T08:29:29.906185+0000","last_clean":"2024-04-25T08:29:29.906185+0000","last_became_active":"2024-04-25T08:29:28.076283+0000","last_became_peered":"2024-04-25T08:29:28.076283+0000","last_unstale":"2024-04-25T08:29:29.906185+0000","last_undegraded":"2024-04-25T08:29:29.906185+0000","last_fullsized":"2024-04-25T08:29:29.906185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075358+0000","last_change":"2024-04-25T08:29:24.041672+0000","last_active":"2024-04-25T08:29:29.075358+0000","last_peered":"2024-04-25T08:29:29.075358+0000","last_clean":"2024-04-25T08:29:29.075358+0000","last_became_active":"2024-04-25T08:29:24.040865+0000","last_became_peered":"2024-04-25T08:29:24.040865+0000","last_unstale":"2024-04-25T08:29:29.075358+0000","last_undegraded":"2024-04-25T08:29:29.075358+0000","last_fullsized":"2024-04-25T08:29:29.075358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075805+0000","last_change":"2024-04-25T08:29:23.028173+0000","last_active":"2024-04-25T08:29:29.075805+0000","last_peered":"2024-04-25T08:29:29.075805+0000","last_clean":"2024-04-25T08:29:29.075805+0000","last_became_active":"2024-04-25T08:29:23.027886+0000","last_became_peered":"2024-04-25T08:29:23.027886+0000","last_unstale":"2024-04-25T08:29:29.075805+0000","last_undegraded":"2024-04-25T08:29:29.075805+0000","last_fullsized":"2024-04-25T08:29:29.075805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076047+0000","last_change":"2024-04-25T08:29:28.084524+0000","last_active":"2024-04-25T08:29:29.076047+0000","last_peered":"2024-04-25T08:29:29.076047+0000","last_clean":"2024-04-25T08:29:29.076047+0000","last_became_active":"2024-04-25T08:29:28.083769+0000","last_became_peered":"2024-04-25T08:29:28.083769+0000","last_unstale":"2024-04-25T08:29:29.076047+0000","last_undegraded":"2024-04-25T08:29:29.076047+0000","last_fullsized":"2024-04-25T08:29:29.076047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:43.115368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895795+0000","last_change":"2024-04-25T08:29:24.057400+0000","last_active":"2024-04-25T08:29:29.895795+0000","last_peered":"2024-04-25T08:29:29.895795+0000","last_clean":"2024-04-25T08:29:29.895795+0000","last_became_active":"2024-04-25T08:29:24.057074+0000","last_became_peered":"2024-04-25T08:29:24.057074+0000","last_unstale":"2024-04-25T08:29:29.895795+0000","last_undegraded":"2024-04-25T08:29:29.895795+0000","last_fullsized":"2024-04-25T08:29:29.895795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574339+0000","last_change":"2024-04-25T08:29:23.036448+0000","last_active":"2024-04-25T08:29:29.574339+0000","last_peered":"2024-04-25T08:29:29.574339+0000","last_clean":"2024-04-25T08:29:29.574339+0000","last_became_active":"2024-04-25T08:29:23.036172+0000","last_became_peered":"2024-04-25T08:29:23.036172+0000","last_unstale":"2024-04-25T08:29:29.574339+0000","last_undegraded":"2024-04-25T08:29:29.574339+0000","last_fullsized":"2024-04-25T08:29:29.574339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:15.024133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075577+0000","last_change":"2024-04-25T08:29:28.083554+0000","last_active":"2024-04-25T08:29:29.075577+0000","last_peered":"2024-04-25T08:29:29.075577+0000","last_clean":"2024-04-25T08:29:29.075577+0000","last_became_active":"2024-04-25T08:29:28.083178+0000","last_became_peered":"2024-04-25T08:29:28.083178+0000","last_unstale":"2024-04-25T08:29:29.075577+0000","last_undegraded":"2024-04-25T08:29:29.075577+0000","last_fullsized":"2024-04-25T08:29:29.075577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906194+0000","last_change":"2024-04-25T08:29:24.049668+0000","last_active":"2024-04-25T08:29:29.906194+0000","last_peered":"2024-04-25T08:29:29.906194+0000","last_clean":"2024-04-25T08:29:29.906194+0000","last_became_active":"2024-04-25T08:29:24.049359+0000","last_became_peered":"2024-04-25T08:29:24.049359+0000","last_unstale":"2024-04-25T08:29:29.906194+0000","last_undegraded":"2024-04-25T08:29:29.906194+0000","last_fullsized":"2024-04-25T08:29:29.906194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:48.609719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906548+0000","last_change":"2024-04-25T08:29:23.030224+0000","last_active":"2024-04-25T08:29:29.906548+0000","last_peered":"2024-04-25T08:29:29.906548+0000","last_clean":"2024-04-25T08:29:29.906548+0000","last_became_active":"2024-04-25T08:29:23.029936+0000","last_became_peered":"2024-04-25T08:29:23.029936+0000","last_unstale":"2024-04-25T08:29:29.906548+0000","last_undegraded":"2024-04-25T08:29:29.906548+0000","last_fullsized":"2024-04-25T08:29:29.906548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:17.202033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574628+0000","last_change":"2024-04-25T08:29:28.076915+0000","last_active":"2024-04-25T08:29:29.574628+0000","last_peered":"2024-04-25T08:29:29.574628+0000","last_clean":"2024-04-25T08:29:29.574628+0000","last_became_active":"2024-04-25T08:29:28.076527+0000","last_became_peered":"2024-04-25T08:29:28.076527+0000","last_unstale":"2024-04-25T08:29:29.574628+0000","last_undegraded":"2024-04-25T08:29:29.574628+0000","last_fullsized":"2024-04-25T08:29:29.574628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:10.090750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895697+0000","last_change":"2024-04-25T08:29:24.053639+0000","last_active":"2024-04-25T08:29:29.895697+0000","last_peered":"2024-04-25T08:29:29.895697+0000","last_clean":"2024-04-25T08:29:29.895697+0000","last_became_active":"2024-04-25T08:29:24.053028+0000","last_became_peered":"2024-04-25T08:29:24.053028+0000","last_unstale":"2024-04-25T08:29:29.895697+0000","last_undegraded":"2024-04-25T08:29:29.895697+0000","last_fullsized":"2024-04-25T08:29:29.895697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:33.263062+0000","last_change":"2024-04-25T08:29:23.031850+0000","last_active":"2024-04-25T08:29:33.263062+0000","last_peered":"2024-04-25T08:29:33.263062+0000","last_clean":"2024-04-25T08:29:33.263062+0000","last_became_active":"2024-04-25T08:29:23.031588+0000","last_became_peered":"2024-04-25T08:29:23.031588+0000","last_unstale":"2024-04-25T08:29:33.263062+0000","last_undegraded":"2024-04-25T08:29:33.263062+0000","last_fullsized":"2024-04-25T08:29:33.263062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:18.936438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074668+0000","last_change":"2024-04-25T08:29:28.066135+0000","last_active":"2024-04-25T08:29:30.074668+0000","last_peered":"2024-04-25T08:29:30.074668+0000","last_clean":"2024-04-25T08:29:30.074668+0000","last_became_active":"2024-04-25T08:29:28.065611+0000","last_became_peered":"2024-04-25T08:29:28.065611+0000","last_unstale":"2024-04-25T08:29:30.074668+0000","last_undegraded":"2024-04-25T08:29:30.074668+0000","last_fullsized":"2024-04-25T08:29:30.074668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075847+0000","last_change":"2024-04-25T08:29:24.050678+0000","last_active":"2024-04-25T08:29:29.075847+0000","last_peered":"2024-04-25T08:29:29.075847+0000","last_clean":"2024-04-25T08:29:29.075847+0000","last_became_active":"2024-04-25T08:29:24.049995+0000","last_became_peered":"2024-04-25T08:29:24.049995+0000","last_unstale":"2024-04-25T08:29:29.075847+0000","last_undegraded":"2024-04-25T08:29:29.075847+0000","last_fullsized":"2024-04-25T08:29:29.075847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:57.729278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075000+0000","last_change":"2024-04-25T08:29:23.036900+0000","last_active":"2024-04-25T08:29:30.075000+0000","last_peered":"2024-04-25T08:29:30.075000+0000","last_clean":"2024-04-25T08:29:30.075000+0000","last_became_active":"2024-04-25T08:29:23.036630+0000","last_became_peered":"2024-04-25T08:29:23.036630+0000","last_unstale":"2024-04-25T08:29:30.075000+0000","last_undegraded":"2024-04-25T08:29:30.075000+0000","last_fullsized":"2024-04-25T08:29:30.075000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075542+0000","last_change":"2024-04-25T08:29:28.066730+0000","last_active":"2024-04-25T08:29:29.075542+0000","last_peered":"2024-04-25T08:29:29.075542+0000","last_clean":"2024-04-25T08:29:29.075542+0000","last_became_active":"2024-04-25T08:29:28.066323+0000","last_became_peered":"2024-04-25T08:29:28.066323+0000","last_unstale":"2024-04-25T08:29:29.075542+0000","last_undegraded":"2024-04-25T08:29:29.075542+0000","last_fullsized":"2024-04-25T08:29:29.075542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906473+0000","last_change":"2024-04-25T08:29:24.035898+0000","last_active":"2024-04-25T08:29:29.906473+0000","last_peered":"2024-04-25T08:29:29.906473+0000","last_clean":"2024-04-25T08:29:29.906473+0000","last_became_active":"2024-04-25T08:29:24.035137+0000","last_became_peered":"2024-04-25T08:29:24.035137+0000","last_unstale":"2024-04-25T08:29:29.906473+0000","last_undegraded":"2024-04-25T08:29:29.906473+0000","last_fullsized":"2024-04-25T08:29:29.906473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574599+0000","last_change":"2024-04-25T08:29:23.036994+0000","last_active":"2024-04-25T08:29:29.574599+0000","last_peered":"2024-04-25T08:29:29.574599+0000","last_clean":"2024-04-25T08:29:29.574599+0000","last_became_active":"2024-04-25T08:29:23.036740+0000","last_became_peered":"2024-04-25T08:29:23.036740+0000","last_unstale":"2024-04-25T08:29:29.574599+0000","last_undegraded":"2024-04-25T08:29:29.574599+0000","last_fullsized":"2024-04-25T08:29:29.574599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074766+0000","last_change":"2024-04-25T08:29:28.080727+0000","last_active":"2024-04-25T08:29:29.074766+0000","last_peered":"2024-04-25T08:29:29.074766+0000","last_clean":"2024-04-25T08:29:29.074766+0000","last_became_active":"2024-04-25T08:29:28.080288+0000","last_became_peered":"2024-04-25T08:29:28.080288+0000","last_unstale":"2024-04-25T08:29:29.074766+0000","last_undegraded":"2024-04-25T08:29:29.074766+0000","last_fullsized":"2024-04-25T08:29:29.074766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:00.184309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574484+0000","last_change":"2024-04-25T08:29:24.054205+0000","last_active":"2024-04-25T08:29:29.574484+0000","last_peered":"2024-04-25T08:29:29.574484+0000","last_clean":"2024-04-25T08:29:29.574484+0000","last_became_active":"2024-04-25T08:29:24.050796+0000","last_became_peered":"2024-04-25T08:29:24.050796+0000","last_unstale":"2024-04-25T08:29:29.574484+0000","last_undegraded":"2024-04-25T08:29:29.574484+0000","last_fullsized":"2024-04-25T08:29:29.574484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076454+0000","last_change":"2024-04-25T08:29:23.037467+0000","last_active":"2024-04-25T08:29:29.076454+0000","last_peered":"2024-04-25T08:29:29.076454+0000","last_clean":"2024-04-25T08:29:29.076454+0000","last_became_active":"2024-04-25T08:29:23.037207+0000","last_became_peered":"2024-04-25T08:29:23.037207+0000","last_unstale":"2024-04-25T08:29:29.076454+0000","last_undegraded":"2024-04-25T08:29:29.076454+0000","last_fullsized":"2024-04-25T08:29:29.076454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075540+0000","last_change":"2024-04-25T08:29:28.085289+0000","last_active":"2024-04-25T08:29:29.075540+0000","last_peered":"2024-04-25T08:29:29.075540+0000","last_clean":"2024-04-25T08:29:29.075540+0000","last_became_active":"2024-04-25T08:29:28.084815+0000","last_became_peered":"2024-04-25T08:29:28.084815+0000","last_unstale":"2024-04-25T08:29:29.075540+0000","last_undegraded":"2024-04-25T08:29:29.075540+0000","last_fullsized":"2024-04-25T08:29:29.075540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906069+0000","last_change":"2024-04-25T08:29:28.077759+0000","last_active":"2024-04-25T08:29:29.906069+0000","last_peered":"2024-04-25T08:29:29.906069+0000","last_clean":"2024-04-25T08:29:29.906069+0000","last_became_active":"2024-04-25T08:29:28.077376+0000","last_became_peered":"2024-04-25T08:29:28.077376+0000","last_unstale":"2024-04-25T08:29:29.906069+0000","last_undegraded":"2024-04-25T08:29:29.906069+0000","last_fullsized":"2024-04-25T08:29:29.906069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:26.891798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074258+0000","last_change":"2024-04-25T08:29:24.053405+0000","last_active":"2024-04-25T08:29:30.074258+0000","last_peered":"2024-04-25T08:29:30.074258+0000","last_clean":"2024-04-25T08:29:30.074258+0000","last_became_active":"2024-04-25T08:29:24.053104+0000","last_became_peered":"2024-04-25T08:29:24.053104+0000","last_unstale":"2024-04-25T08:29:30.074258+0000","last_undegraded":"2024-04-25T08:29:30.074258+0000","last_fullsized":"2024-04-25T08:29:30.074258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574014+0000","last_change":"2024-04-25T08:29:23.034808+0000","last_active":"2024-04-25T08:29:29.574014+0000","last_peered":"2024-04-25T08:29:29.574014+0000","last_clean":"2024-04-25T08:29:29.574014+0000","last_became_active":"2024-04-25T08:29:23.034545+0000","last_became_peered":"2024-04-25T08:29:23.034545+0000","last_unstale":"2024-04-25T08:29:29.574014+0000","last_undegraded":"2024-04-25T08:29:29.574014+0000","last_fullsized":"2024-04-25T08:29:29.574014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.854971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076514+0000","last_change":"2024-04-25T08:29:28.068503+0000","last_active":"2024-04-25T08:29:29.076514+0000","last_peered":"2024-04-25T08:29:29.076514+0000","last_clean":"2024-04-25T08:29:29.076514+0000","last_became_active":"2024-04-25T08:29:28.068080+0000","last_became_peered":"2024-04-25T08:29:28.068080+0000","last_unstale":"2024-04-25T08:29:29.076514+0000","last_undegraded":"2024-04-25T08:29:29.076514+0000","last_fullsized":"2024-04-25T08:29:29.076514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:42.259878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074757+0000","last_change":"2024-04-25T08:29:24.053631+0000","last_active":"2024-04-25T08:29:29.074757+0000","last_peered":"2024-04-25T08:29:29.074757+0000","last_clean":"2024-04-25T08:29:29.074757+0000","last_became_active":"2024-04-25T08:29:24.052293+0000","last_became_peered":"2024-04-25T08:29:24.052293+0000","last_unstale":"2024-04-25T08:29:29.074757+0000","last_undegraded":"2024-04-25T08:29:29.074757+0000","last_fullsized":"2024-04-25T08:29:29.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074995+0000","last_change":"2024-04-25T08:29:23.029639+0000","last_active":"2024-04-25T08:29:29.074995+0000","last_peered":"2024-04-25T08:29:29.074995+0000","last_clean":"2024-04-25T08:29:29.074995+0000","last_became_active":"2024-04-25T08:29:23.027908+0000","last_became_peered":"2024-04-25T08:29:23.027908+0000","last_unstale":"2024-04-25T08:29:29.074995+0000","last_undegraded":"2024-04-25T08:29:29.074995+0000","last_fullsized":"2024-04-25T08:29:29.074995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:02.366427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075903+0000","last_change":"2024-04-25T08:29:23.024619+0000","last_active":"2024-04-25T08:29:29.075903+0000","last_peered":"2024-04-25T08:29:29.075903+0000","last_clean":"2024-04-25T08:29:29.075903+0000","last_became_active":"2024-04-25T08:29:23.024364+0000","last_became_peered":"2024-04-25T08:29:23.024364+0000","last_unstale":"2024-04-25T08:29:29.075903+0000","last_undegraded":"2024-04-25T08:29:29.075903+0000","last_fullsized":"2024-04-25T08:29:29.075903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:50.946005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906507+0000","last_change":"2024-04-25T08:29:24.040912+0000","last_active":"2024-04-25T08:29:29.906507+0000","last_peered":"2024-04-25T08:29:29.906507+0000","last_clean":"2024-04-25T08:29:29.906507+0000","last_became_active":"2024-04-25T08:29:24.040629+0000","last_became_peered":"2024-04-25T08:29:24.040629+0000","last_unstale":"2024-04-25T08:29:29.906507+0000","last_undegraded":"2024-04-25T08:29:29.906507+0000","last_fullsized":"2024-04-25T08:29:29.906507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074576+0000","last_change":"2024-04-25T08:29:28.081077+0000","last_active":"2024-04-25T08:29:30.074576+0000","last_peered":"2024-04-25T08:29:30.074576+0000","last_clean":"2024-04-25T08:29:30.074576+0000","last_became_active":"2024-04-25T08:29:28.080470+0000","last_became_peered":"2024-04-25T08:29:28.080470+0000","last_unstale":"2024-04-25T08:29:30.074576+0000","last_undegraded":"2024-04-25T08:29:30.074576+0000","last_fullsized":"2024-04-25T08:29:30.074576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:18.082525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075155+0000","last_change":"2024-04-25T08:29:23.039762+0000","last_active":"2024-04-25T08:29:29.075155+0000","last_peered":"2024-04-25T08:29:29.075155+0000","last_clean":"2024-04-25T08:29:29.075155+0000","last_became_active":"2024-04-25T08:29:23.039484+0000","last_became_peered":"2024-04-25T08:29:23.039484+0000","last_unstale":"2024-04-25T08:29:29.075155+0000","last_undegraded":"2024-04-25T08:29:29.075155+0000","last_fullsized":"2024-04-25T08:29:29.075155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:22.009286+0000","last_clean_scrub_stamp":"2024-04-25T08:29:22.009286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:31.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074927+0000","last_change":"2024-04-25T08:29:24.037624+0000","last_active":"2024-04-25T08:29:29.074927+0000","last_peered":"2024-04-25T08:29:29.074927+0000","last_clean":"2024-04-25T08:29:29.074927+0000","last_became_active":"2024-04-25T08:29:24.037352+0000","last_became_peered":"2024-04-25T08:29:24.037352+0000","last_unstale":"2024-04-25T08:29:29.074927+0000","last_undegraded":"2024-04-25T08:29:29.074927+0000","last_fullsized":"2024-04-25T08:29:29.074927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:51.950054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075230+0000","last_change":"2024-04-25T08:29:28.083366+0000","last_active":"2024-04-25T08:29:29.075230+0000","last_peered":"2024-04-25T08:29:29.075230+0000","last_clean":"2024-04-25T08:29:29.075230+0000","last_became_active":"2024-04-25T08:29:28.082870+0000","last_became_peered":"2024-04-25T08:29:28.082870+0000","last_unstale":"2024-04-25T08:29:29.075230+0000","last_undegraded":"2024-04-25T08:29:29.075230+0000","last_fullsized":"2024-04-25T08:29:29.075230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:27.046115+0000","last_clean_scrub_stamp":"2024-04-25T08:29:27.046115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:08.399424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075038+0000","last_change":"2024-04-25T08:29:24.042078+0000","last_active":"2024-04-25T08:29:29.075038+0000","last_peered":"2024-04-25T08:29:29.075038+0000","last_clean":"2024-04-25T08:29:29.075038+0000","last_became_active":"2024-04-25T08:29:24.041318+0000","last_became_peered":"2024-04-25T08:29:24.041318+0000","last_unstale":"2024-04-25T08:29:29.075038+0000","last_undegraded":"2024-04-25T08:29:29.075038+0000","last_fullsized":"2024-04-25T08:29:29.075038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074154+0000","last_change":"2024-04-25T08:29:24.043006+0000","last_active":"2024-04-25T08:29:30.074154+0000","last_peered":"2024-04-25T08:29:30.074154+0000","last_clean":"2024-04-25T08:29:30.074154+0000","last_became_active":"2024-04-25T08:29:24.042553+0000","last_became_peered":"2024-04-25T08:29:24.042553+0000","last_unstale":"2024-04-25T08:29:30.074154+0000","last_undegraded":"2024-04-25T08:29:30.074154+0000","last_fullsized":"2024-04-25T08:29:30.074154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574181+0000","last_change":"2024-04-25T08:29:24.054678+0000","last_active":"2024-04-25T08:29:29.574181+0000","last_peered":"2024-04-25T08:29:29.574181+0000","last_clean":"2024-04-25T08:29:29.574181+0000","last_became_active":"2024-04-25T08:29:24.054432+0000","last_became_peered":"2024-04-25T08:29:24.054432+0000","last_unstale":"2024-04-25T08:29:29.574181+0000","last_undegraded":"2024-04-25T08:29:29.574181+0000","last_fullsized":"2024-04-25T08:29:29.574181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:52.633536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906457+0000","last_change":"2024-04-25T08:29:24.047684+0000","last_active":"2024-04-25T08:29:29.906457+0000","last_peered":"2024-04-25T08:29:29.906457+0000","last_clean":"2024-04-25T08:29:29.906457+0000","last_became_active":"2024-04-25T08:29:24.047384+0000","last_became_peered":"2024-04-25T08:29:24.047384+0000","last_unstale":"2024-04-25T08:29:29.906457+0000","last_undegraded":"2024-04-25T08:29:29.906457+0000","last_fullsized":"2024-04-25T08:29:29.906457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075571+0000","last_change":"2024-04-25T08:29:24.044573+0000","last_active":"2024-04-25T08:29:29.075571+0000","last_peered":"2024-04-25T08:29:29.075571+0000","last_clean":"2024-04-25T08:29:29.075571+0000","last_became_active":"2024-04-25T08:29:24.043980+0000","last_became_peered":"2024-04-25T08:29:24.043980+0000","last_unstale":"2024-04-25T08:29:29.075571+0000","last_undegraded":"2024-04-25T08:29:29.075571+0000","last_fullsized":"2024-04-25T08:29:29.075571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074876+0000","last_change":"2024-04-25T08:29:24.053456+0000","last_active":"2024-04-25T08:29:30.074876+0000","last_peered":"2024-04-25T08:29:30.074876+0000","last_clean":"2024-04-25T08:29:30.074876+0000","last_became_active":"2024-04-25T08:29:24.053144+0000","last_became_peered":"2024-04-25T08:29:24.053144+0000","last_unstale":"2024-04-25T08:29:30.074876+0000","last_undegraded":"2024-04-25T08:29:30.074876+0000","last_fullsized":"2024-04-25T08:29:30.074876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:15.637486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076592+0000","last_change":"2024-04-25T08:29:24.052047+0000","last_active":"2024-04-25T08:29:29.076592+0000","last_peered":"2024-04-25T08:29:29.076592+0000","last_clean":"2024-04-25T08:29:29.076592+0000","last_became_active":"2024-04-25T08:29:24.051777+0000","last_became_peered":"2024-04-25T08:29:24.051777+0000","last_unstale":"2024-04-25T08:29:29.076592+0000","last_undegraded":"2024-04-25T08:29:29.076592+0000","last_fullsized":"2024-04-25T08:29:29.076592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075986+0000","last_change":"2024-04-25T08:29:24.044743+0000","last_active":"2024-04-25T08:29:29.075986+0000","last_peered":"2024-04-25T08:29:29.075986+0000","last_clean":"2024-04-25T08:29:29.075986+0000","last_became_active":"2024-04-25T08:29:24.044425+0000","last_became_peered":"2024-04-25T08:29:24.044425+0000","last_unstale":"2024-04-25T08:29:29.075986+0000","last_undegraded":"2024-04-25T08:29:29.075986+0000","last_fullsized":"2024-04-25T08:29:29.075986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573967+0000","last_change":"2024-04-25T08:29:24.047484+0000","last_active":"2024-04-25T08:29:29.573967+0000","last_peered":"2024-04-25T08:29:29.573967+0000","last_clean":"2024-04-25T08:29:29.573967+0000","last_became_active":"2024-04-25T08:29:24.046848+0000","last_became_peered":"2024-04-25T08:29:24.046848+0000","last_unstale":"2024-04-25T08:29:29.573967+0000","last_undegraded":"2024-04-25T08:29:29.573967+0000","last_fullsized":"2024-04-25T08:29:29.573967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.616648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074875+0000","last_change":"2024-04-25T08:29:24.048838+0000","last_active":"2024-04-25T08:29:29.074875+0000","last_peered":"2024-04-25T08:29:29.074875+0000","last_clean":"2024-04-25T08:29:29.074875+0000","last_became_active":"2024-04-25T08:29:24.047272+0000","last_became_peered":"2024-04-25T08:29:24.047272+0000","last_unstale":"2024-04-25T08:29:29.074875+0000","last_undegraded":"2024-04-25T08:29:29.074875+0000","last_fullsized":"2024-04-25T08:29:29.074875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574182+0000","last_change":"2024-04-25T08:29:24.052059+0000","last_active":"2024-04-25T08:29:29.574182+0000","last_peered":"2024-04-25T08:29:29.574182+0000","last_clean":"2024-04-25T08:29:29.574182+0000","last_became_active":"2024-04-25T08:29:24.051604+0000","last_became_peered":"2024-04-25T08:29:24.051604+0000","last_unstale":"2024-04-25T08:29:29.574182+0000","last_undegraded":"2024-04-25T08:29:29.574182+0000","last_fullsized":"2024-04-25T08:29:29.574182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906362+0000","last_change":"2024-04-25T08:29:24.049843+0000","last_active":"2024-04-25T08:29:29.906362+0000","last_peered":"2024-04-25T08:29:29.906362+0000","last_clean":"2024-04-25T08:29:29.906362+0000","last_became_active":"2024-04-25T08:29:24.049562+0000","last_became_peered":"2024-04-25T08:29:24.049562+0000","last_unstale":"2024-04-25T08:29:29.906362+0000","last_undegraded":"2024-04-25T08:29:29.906362+0000","last_fullsized":"2024-04-25T08:29:29.906362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:47.370927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075161+0000","last_change":"2024-04-25T08:29:24.041378+0000","last_active":"2024-04-25T08:29:29.075161+0000","last_peered":"2024-04-25T08:29:29.075161+0000","last_clean":"2024-04-25T08:29:29.075161+0000","last_became_active":"2024-04-25T08:29:24.040860+0000","last_became_peered":"2024-04-25T08:29:24.040860+0000","last_unstale":"2024-04-25T08:29:29.075161+0000","last_undegraded":"2024-04-25T08:29:29.075161+0000","last_fullsized":"2024-04-25T08:29:29.075161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:09.206586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895964+0000","last_change":"2024-04-25T08:29:24.050860+0000","last_active":"2024-04-25T08:29:29.895964+0000","last_peered":"2024-04-25T08:29:29.895964+0000","last_clean":"2024-04-25T08:29:29.895964+0000","last_became_active":"2024-04-25T08:29:24.050610+0000","last_became_peered":"2024-04-25T08:29:24.050610+0000","last_unstale":"2024-04-25T08:29:29.895964+0000","last_undegraded":"2024-04-25T08:29:29.895964+0000","last_fullsized":"2024-04-25T08:29:29.895964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074757+0000","last_change":"2024-04-25T08:29:24.041470+0000","last_active":"2024-04-25T08:29:30.074757+0000","last_peered":"2024-04-25T08:29:30.074757+0000","last_clean":"2024-04-25T08:29:30.074757+0000","last_became_active":"2024-04-25T08:29:24.041172+0000","last_became_peered":"2024-04-25T08:29:24.041172+0000","last_unstale":"2024-04-25T08:29:30.074757+0000","last_undegraded":"2024-04-25T08:29:30.074757+0000","last_fullsized":"2024-04-25T08:29:30.074757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:19.940487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574663+0000","last_change":"2024-04-25T08:29:24.054784+0000","last_active":"2024-04-25T08:29:29.574663+0000","last_peered":"2024-04-25T08:29:29.574663+0000","last_clean":"2024-04-25T08:29:29.574663+0000","last_became_active":"2024-04-25T08:29:24.054538+0000","last_became_peered":"2024-04-25T08:29:24.054538+0000","last_unstale":"2024-04-25T08:29:29.574663+0000","last_undegraded":"2024-04-25T08:29:29.574663+0000","last_fullsized":"2024-04-25T08:29:29.574663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.573870+0000","last_change":"2024-04-25T08:29:24.046213+0000","last_active":"2024-04-25T08:29:29.573870+0000","last_peered":"2024-04-25T08:29:29.573870+0000","last_clean":"2024-04-25T08:29:29.573870+0000","last_became_active":"2024-04-25T08:29:24.045850+0000","last_became_peered":"2024-04-25T08:29:24.045850+0000","last_unstale":"2024-04-25T08:29:29.573870+0000","last_undegraded":"2024-04-25T08:29:29.573870+0000","last_fullsized":"2024-04-25T08:29:29.573870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.074307+0000","last_change":"2024-04-25T08:29:24.039662+0000","last_active":"2024-04-25T08:29:30.074307+0000","last_peered":"2024-04-25T08:29:30.074307+0000","last_clean":"2024-04-25T08:29:30.074307+0000","last_became_active":"2024-04-25T08:29:24.039338+0000","last_became_peered":"2024-04-25T08:29:24.039338+0000","last_unstale":"2024-04-25T08:29:30.074307+0000","last_undegraded":"2024-04-25T08:29:30.074307+0000","last_fullsized":"2024-04-25T08:29:30.074307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:05.537134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906168+0000","last_change":"2024-04-25T08:29:24.038228+0000","last_active":"2024-04-25T08:29:29.906168+0000","last_peered":"2024-04-25T08:29:29.906168+0000","last_clean":"2024-04-25T08:29:29.906168+0000","last_became_active":"2024-04-25T08:29:24.037900+0000","last_became_peered":"2024-04-25T08:29:24.037900+0000","last_unstale":"2024-04-25T08:29:29.906168+0000","last_undegraded":"2024-04-25T08:29:29.906168+0000","last_fullsized":"2024-04-25T08:29:29.906168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:47:35.121139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074948+0000","last_change":"2024-04-25T08:29:24.045041+0000","last_active":"2024-04-25T08:29:29.074948+0000","last_peered":"2024-04-25T08:29:29.074948+0000","last_clean":"2024-04-25T08:29:29.074948+0000","last_became_active":"2024-04-25T08:29:24.044581+0000","last_became_peered":"2024-04-25T08:29:24.044581+0000","last_unstale":"2024-04-25T08:29:29.074948+0000","last_undegraded":"2024-04-25T08:29:29.074948+0000","last_fullsized":"2024-04-25T08:29:29.074948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.075863+0000","last_change":"2024-04-25T08:29:24.042402+0000","last_active":"2024-04-25T08:29:29.075863+0000","last_peered":"2024-04-25T08:29:29.075863+0000","last_clean":"2024-04-25T08:29:29.075863+0000","last_became_active":"2024-04-25T08:29:24.042074+0000","last_became_peered":"2024-04-25T08:29:24.042074+0000","last_unstale":"2024-04-25T08:29:29.075863+0000","last_undegraded":"2024-04-25T08:29:29.075863+0000","last_fullsized":"2024-04-25T08:29:29.075863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:32.919774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076327+0000","last_change":"2024-04-25T08:29:24.035522+0000","last_active":"2024-04-25T08:29:29.076327+0000","last_peered":"2024-04-25T08:29:29.076327+0000","last_clean":"2024-04-25T08:29:29.076327+0000","last_became_active":"2024-04-25T08:29:24.035220+0000","last_became_peered":"2024-04-25T08:29:24.035220+0000","last_unstale":"2024-04-25T08:29:29.076327+0000","last_undegraded":"2024-04-25T08:29:29.076327+0000","last_fullsized":"2024-04-25T08:29:29.076327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.619852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.076595+0000","last_change":"2024-04-25T08:29:24.035459+0000","last_active":"2024-04-25T08:29:29.076595+0000","last_peered":"2024-04-25T08:29:29.076595+0000","last_clean":"2024-04-25T08:29:29.076595+0000","last_became_active":"2024-04-25T08:29:24.035170+0000","last_became_peered":"2024-04-25T08:29:24.035170+0000","last_unstale":"2024-04-25T08:29:29.076595+0000","last_undegraded":"2024-04-25T08:29:29.076595+0000","last_fullsized":"2024-04-25T08:29:29.076595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896232+0000","last_change":"2024-04-25T08:29:24.058439+0000","last_active":"2024-04-25T08:29:29.896232+0000","last_peered":"2024-04-25T08:29:29.896232+0000","last_clean":"2024-04-25T08:29:29.896232+0000","last_became_active":"2024-04-25T08:29:24.058172+0000","last_became_peered":"2024-04-25T08:29:24.058172+0000","last_unstale":"2024-04-25T08:29:29.896232+0000","last_undegraded":"2024-04-25T08:29:29.896232+0000","last_fullsized":"2024-04-25T08:29:29.896232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:36.356652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.074560+0000","last_change":"2024-04-25T08:29:24.045871+0000","last_active":"2024-04-25T08:29:29.074560+0000","last_peered":"2024-04-25T08:29:29.074560+0000","last_clean":"2024-04-25T08:29:29.074560+0000","last_became_active":"2024-04-25T08:29:24.045477+0000","last_became_peered":"2024-04-25T08:29:24.045477+0000","last_unstale":"2024-04-25T08:29:29.074560+0000","last_undegraded":"2024-04-25T08:29:29.074560+0000","last_fullsized":"2024-04-25T08:29:29.074560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.601648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895647+0000","last_change":"2024-04-25T08:29:24.054149+0000","last_active":"2024-04-25T08:29:29.895647+0000","last_peered":"2024-04-25T08:29:29.895647+0000","last_clean":"2024-04-25T08:29:29.895647+0000","last_became_active":"2024-04-25T08:29:24.053893+0000","last_became_peered":"2024-04-25T08:29:24.053893+0000","last_unstale":"2024-04-25T08:29:29.895647+0000","last_undegraded":"2024-04-25T08:29:29.895647+0000","last_fullsized":"2024-04-25T08:29:29.895647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:44.330176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574047+0000","last_change":"2024-04-25T08:29:24.046245+0000","last_active":"2024-04-25T08:29:29.574047+0000","last_peered":"2024-04-25T08:29:29.574047+0000","last_clean":"2024-04-25T08:29:29.574047+0000","last_became_active":"2024-04-25T08:29:24.045892+0000","last_became_peered":"2024-04-25T08:29:24.045892+0000","last_unstale":"2024-04-25T08:29:29.574047+0000","last_undegraded":"2024-04-25T08:29:29.574047+0000","last_fullsized":"2024-04-25T08:29:29.574047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:47.716252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.895672+0000","last_change":"2024-04-25T08:29:24.048906+0000","last_active":"2024-04-25T08:29:29.895672+0000","last_peered":"2024-04-25T08:29:29.895672+0000","last_clean":"2024-04-25T08:29:29.895672+0000","last_became_active":"2024-04-25T08:29:24.048634+0000","last_became_peered":"2024-04-25T08:29:24.048634+0000","last_unstale":"2024-04-25T08:29:29.895672+0000","last_undegraded":"2024-04-25T08:29:29.895672+0000","last_fullsized":"2024-04-25T08:29:29.895672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:50.229779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.574336+0000","last_change":"2024-04-25T08:29:24.049689+0000","last_active":"2024-04-25T08:29:29.574336+0000","last_peered":"2024-04-25T08:29:29.574336+0000","last_clean":"2024-04-25T08:29:29.574336+0000","last_became_active":"2024-04-25T08:29:24.049210+0000","last_became_peered":"2024-04-25T08:29:24.049210+0000","last_unstale":"2024-04-25T08:29:29.574336+0000","last_undegraded":"2024-04-25T08:29:29.574336+0000","last_fullsized":"2024-04-25T08:29:29.574336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.859016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.906036+0000","last_change":"2024-04-25T08:29:24.037342+0000","last_active":"2024-04-25T08:29:29.906036+0000","last_peered":"2024-04-25T08:29:29.906036+0000","last_clean":"2024-04-25T08:29:29.906036+0000","last_became_active":"2024-04-25T08:29:24.036991+0000","last_became_peered":"2024-04-25T08:29:24.036991+0000","last_unstale":"2024-04-25T08:29:29.906036+0000","last_undegraded":"2024-04-25T08:29:29.906036+0000","last_fullsized":"2024-04-25T08:29:29.906036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:14.049743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:30.075188+0000","last_change":"2024-04-25T08:29:24.041775+0000","last_active":"2024-04-25T08:29:30.075188+0000","last_peered":"2024-04-25T08:29:30.075188+0000","last_clean":"2024-04-25T08:29:30.075188+0000","last_became_active":"2024-04-25T08:29:24.041474+0000","last_became_peered":"2024-04-25T08:29:24.041474+0000","last_unstale":"2024-04-25T08:29:30.075188+0000","last_undegraded":"2024-04-25T08:29:30.075188+0000","last_fullsized":"2024-04-25T08:29:30.075188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.016632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T08:29:29.896139+0000","last_change":"2024-04-25T08:29:24.054127+0000","last_active":"2024-04-25T08:29:29.896139+0000","last_peered":"2024-04-25T08:29:29.896139+0000","last_clean":"2024-04-25T08:29:29.896139+0000","last_became_active":"2024-04-25T08:29:24.053854+0000","last_became_peered":"2024-04-25T08:29:24.053854+0000","last_unstale":"2024-04-25T08:29:29.896139+0000","last_undegraded":"2024-04-25T08:29:29.896139+0000","last_fullsized":"2024-04-25T08:29:29.896139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:23.013334+0000","last_clean_scrub_stamp":"2024-04-25T08:29:23.013334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:38.227097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":695037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":104177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":104213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":262144,"data_stored":104087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":104247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":695093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:29:45.888 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:29:45.888 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T08:29:45.888 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T08:29:45.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:29:46.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.025+0000 7f8d9437e640 1 Processor -- start 2024-04-25T08:29:46.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.025+0000 7f8d9437e640 1 -- start start 2024-04-25T08:29:46.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9437e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d8c06c1e0 0x7f8d8c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9437e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c0b3d70 con 0x7f8d8c15a810 2024-04-25T08:29:46.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9437e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c0ae2e0 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9437e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c0ae5c0 con 0x7f8d8c06c1e0 2024-04-25T08:29:46.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9291b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d8c06c1e0 0x7f8d8c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9211a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8d8c15a810 0x7f8d8c0af0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42086/0 (socket says 172.21.15.60:42086) 2024-04-25T08:29:46.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.026+0000 7f8d9211a640 1 -- 172.21.15.60:0/2194558875 learned_addr learned my addr 172.21.15.60:0/2194558875 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:46.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.027+0000 7f8d9291b640 1 -- 172.21.15.60:0/2194558875 >> v1:172.21.15.100:6789/0 conn(0x7f8d8c0b2e80 legacy=0x7f8d8c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.027+0000 7f8d9291b640 1 -- 172.21.15.60:0/2194558875 >> v1:172.21.15.60:6789/0 conn(0x7f8d8c15a810 legacy=0x7f8d8c0af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.027+0000 7f8d91118640 1 -- 172.21.15.60:0/2194558875 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172954876 0 0) 0x7f8d7c001490 con 0x7f8d8c15a810 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.027+0000 7f8d9291b640 1 -- 172.21.15.60:0/2194558875 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0ae8a0 con 0x7f8d8c06c1e0 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9291b640 1 --2- 172.21.15.60:0/2194558875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c0b4730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d840097a0 tx=0x7f8d8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a212f9689030e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d91118640 1 -- 172.21.15.60:0/2194558875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84037070 con 0x7f8d8c06c1e0 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d91118640 1 -- 172.21.15.60:0/2194558875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d84032440 con 0x7f8d8c06c1e0 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d91118640 1 -- 172.21.15.60:0/2194558875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84032810 con 0x7f8d8c06c1e0 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 -- 172.21.15.60:0/2194558875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c06c1e0 msgr2=0x7f8d8c0b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 --2- 172.21.15.60:0/2194558875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c0b4730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d840097a0 tx=0x7f8d8402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 -- 172.21.15.60:0/2194558875 shutdown_connections 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 --2- 172.21.15.60:0/2194558875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c0b4730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 -- 172.21.15.60:0/2194558875 >> 172.21.15.60:0/2194558875 conn(0x7f8d8c0cfe10 msgr2=0x7f8d8c0d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 -- 172.21.15.60:0/2194558875 shutdown_connections 2024-04-25T08:29:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.028+0000 7f8d9437e640 1 -- 172.21.15.60:0/2194558875 wait complete. 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 Processor -- start 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 -- start start 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 0x7f8d8c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8c15a810 0x7f8d8c0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0af7b0 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c15abf0 con 0x7f8d8c06c1e0 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d9437e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0b3ff0 con 0x7f8d8c15a810 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d91919640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 0x7f8d8c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d91919640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 0x7f8d8c0afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41736/0 (socket says 172.21.15.60:41736) 2024-04-25T08:29:46.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.030+0000 7f8d91919640 1 -- 172.21.15.60:0/3239878747 learned_addr learned my addr 172.21.15.60:0/3239878747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d9211a640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d9291b640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8c15a810 0x7f8d8c0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d91919640 1 -- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8c15a810 msgr2=0x7f8d8c0b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d91919640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8c15a810 0x7f8d8c0b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d91919640 1 -- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8c06c1e0 msgr2=0x7f8d8c14a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d91919640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c14a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d91919640 1 -- 172.21.15.60:0/3239878747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c152e80 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d91919640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 0x7f8d8c0afbb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000c450 tx=0x7f8d8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800052d0 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80005470 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c153160 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80012870 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.031+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c153720 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.032+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c159520 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.033+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d80012a10 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.033+0000 7f8d8affd640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d7806b4d0 0x7f8d7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.033+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8d8008a0f0 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.033+0000 7f8d9211a640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d7806b4d0 0x7f8d7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.034+0000 7f8d9211a640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d7806b4d0 0x7f8d7806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0012e0 tx=0x7f8d7c005bf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.037+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d8c159520 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.259+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8d8c073990 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.260+0000 7f8d8affd640 1 -- 172.21.15.60:0/3239878747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8d80058cc0 con 0x7f8d8c0b2e80 2024-04-25T08:29:46.258 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:46.258 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:29:46.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d7806b4d0 msgr2=0x7f8d7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d7806b4d0 0x7f8d7806d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0012e0 tx=0x7f8d7c005bf0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 msgr2=0x7f8d8c0afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 0x7f8d8c0afbb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000c450 tx=0x7f8d8000c910 comp rx=0 tx=0).stop 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 shutdown_connections 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d8c15a810 0x7f8d8c0b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d8c0b2e80 0x7f8d8c0afbb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d7806b4d0 0x7f8d7806d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 --2- 172.21.15.60:0/3239878747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d8c06c1e0 0x7f8d8c14a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 >> 172.21.15.60:0/3239878747 conn(0x7f8d8c0cfe10 msgr2=0x7f8d8c074060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 shutdown_connections 2024-04-25T08:29:46.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.262+0000 7f8d9437e640 1 -- 172.21.15.60:0/3239878747 wait complete. 2024-04-25T08:29:46.270 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T08:29:46.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.374+0000 7f6165807640 1 Processor -- start 2024-04-25T08:29:46.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.374+0000 7f6165807640 1 -- start start 2024-04-25T08:29:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6165807640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61600b9730 0x7f61600ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6165807640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61600ceb90 0x7f61600d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6165807640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6160161d50 con 0x7f616015da70 2024-04-25T08:29:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6165807640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61601625e0 con 0x7f61600ceb90 2024-04-25T08:29:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6165807640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61600cf000 con 0x7f61600b9730 2024-04-25T08:29:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61600ceb90 0x7f61600d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61600ceb90 0x7f61600d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41748/0 (socket says 172.21.15.60:41748) 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 -- 172.21.15.60:0/611973012 learned_addr learned my addr 172.21.15.60:0/611973012 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6157fff640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61600b9730 0x7f61600ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 -- 172.21.15.60:0/611973012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61600b9730 msgr2=0x7f61600ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61600b9730 0x7f61600ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 -- 172.21.15.60:0/611973012 >> v1:172.21.15.100:6789/0 conn(0x7f616015da70 legacy=0x7f6160161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f615f7fe640 1 -- 172.21.15.60:0/611973012 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160162020 con 0x7f61600ceb90 2024-04-25T08:29:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.375+0000 7f6157fff640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61600b9730 0x7f61600ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.376+0000 7f615f7fe640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600ceb90 0x7f61600d22f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6158009e50 tx=0x7f615802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65bffefea932755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.376+0000 7f615e7fc640 1 -- 172.21.15.60:0/611973012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158036070 con 0x7f61600ceb90 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.376+0000 7f615e7fc640 1 -- 172.21.15.60:0/611973012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6158031600 con 0x7f61600ceb90 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.376+0000 7f615e7fc640 1 -- 172.21.15.60:0/611973012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61580319d0 con 0x7f61600ceb90 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 -- 172.21.15.60:0/611973012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600ceb90 msgr2=0x7f61600d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600ceb90 0x7f61600d22f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6158009e50 tx=0x7f615802a680 comp rx=0 tx=0).stop 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 -- 172.21.15.60:0/611973012 shutdown_connections 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600ceb90 0x7f61600d22f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 --2- 172.21.15.60:0/611973012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61600b9730 0x7f61600ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 -- 172.21.15.60:0/611973012 >> 172.21.15.60:0/611973012 conn(0x7f6160061a20 msgr2=0x7f6160061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 -- 172.21.15.60:0/611973012 shutdown_connections 2024-04-25T08:29:46.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.377+0000 7f6165807640 1 -- 172.21.15.60:0/611973012 wait complete. 2024-04-25T08:29:46.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.378+0000 7f6165807640 1 Processor -- start 2024-04-25T08:29:46.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.378+0000 7f6165807640 1 -- start start 2024-04-25T08:29:46.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61600ceb90 0x7f6160158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f616015da70 0x7f61600cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 0x7f61600d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61600d1dd0 con 0x7f61600d34d0 2024-04-25T08:29:46.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61600d1af0 con 0x7f61600ceb90 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616015de50 con 0x7f616015da70 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 0x7f61600d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 0x7f61600d3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41756/0 (socket says 172.21.15.60:41756) 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 -- 172.21.15.60:0/849789978 learned_addr learned my addr 172.21.15.60:0/849789978 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615effd640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61600ceb90 0x7f6160158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6157fff640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f616015da70 0x7f61600cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 -- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f616015da70 msgr2=0x7f61600cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f616015da70 0x7f61600cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 -- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61600ceb90 msgr2=0x7f6160158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61600ceb90 0x7f6160158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 -- 172.21.15.60:0/849789978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61600cabf0 con 0x7f61600d34d0 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6157fff640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f616015da70 0x7f61600cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615f7fe640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 0x7f61600d3980 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6158034040 tx=0x7f6158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158036050 con 0x7f61600d34d0 2024-04-25T08:29:46.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6158042070 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61600cae90 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.379+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158009510 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.380+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61600cb450 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.381+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160073990 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.381+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6158007bd0 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.381+0000 7f615cff9640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f614006b500 0x7f614006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.381+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f61580ab250 con 0x7f61600d34d0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.382+0000 7f615effd640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f614006b500 0x7f614006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.382+0000 7f615effd640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f614006b500 0x7f614006d9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61500016e0 tx=0x7f6150009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.386+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6158079e20 con 0x7f61600d34d0 2024-04-25T08:29:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.566+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61600d1570 con 0x7f61600d34d0 2024-04-25T08:29:46.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.568+0000 7f615cff9640 1 -- 172.21.15.60:0/849789978 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f6158079720 con 0x7f61600d34d0 2024-04-25T08:29:46.566 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:46.566 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.569+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f614006b500 msgr2=0x7f614006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.569+0000 7f6165807640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f614006b500 0x7f614006d9c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61500016e0 tx=0x7f6150009040 comp rx=0 tx=0).stop 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.569+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 msgr2=0x7f61600d3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.569+0000 7f6165807640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 0x7f61600d3980 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6158034040 tx=0x7f6158004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 shutdown_connections 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61600d34d0 0x7f61600d3980 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f616015da70 0x7f61600cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f614006b500 0x7f614006d9c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 --2- 172.21.15.60:0/849789978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61600ceb90 0x7f6160158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 >> 172.21.15.60:0/849789978 conn(0x7f6160061a20 msgr2=0x7f6160061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 shutdown_connections 2024-04-25T08:29:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.570+0000 7f6165807640 1 -- 172.21.15.60:0/849789978 wait complete. 2024-04-25T08:29:46.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:29:46.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.684+0000 7f5067932640 1 Processor -- start 2024-04-25T08:29:46.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.684+0000 7f5067932640 1 -- start start 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5067932640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f506015bb50 0x7f50600c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5067932640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50600c70d0 0x7f50600cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5067932640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600c75e0 con 0x7f50600cfb40 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5067932640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50600c8680 con 0x7f50600c70d0 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5067932640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50600c78c0 con 0x7f506015bb50 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5064ecd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50600c70d0 0x7f50600cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5065ecf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50600cfb40 0x7f5060070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42098/0 (socket says 172.21.15.60:42098) 2024-04-25T08:29:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5064ecd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50600c70d0 0x7f50600cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41770/0 (socket says 172.21.15.60:41770) 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.685+0000 7f5065ecf640 1 -- 172.21.15.60:0/3578585756 learned_addr learned my addr 172.21.15.60:0/3578585756 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f50656ce640 1 --2- 172.21.15.60:0/3578585756 >> v2:172.21.15.100:3300/0 conn(0x7f506015bb50 0x7f50600c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f5064ecd640 1 -- 172.21.15.60:0/3578585756 >> v2:172.21.15.100:3300/0 conn(0x7f506015bb50 msgr2=0x7f50600c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f5064ecd640 1 --2- 172.21.15.60:0/3578585756 >> v2:172.21.15.100:3300/0 conn(0x7f506015bb50 0x7f50600c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f5064ecd640 1 -- 172.21.15.60:0/3578585756 >> v1:172.21.15.60:6789/0 conn(0x7f50600cfb40 legacy=0x7f5060070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f5057fff640 1 -- 172.21.15.60:0/3578585756 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385012934 0 0) 0x7f50600c75e0 con 0x7f50600cfb40 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f5064ecd640 1 -- 172.21.15.60:0/3578585756 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600c7ba0 con 0x7f50600c70d0 2024-04-25T08:29:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.686+0000 7f5064ecd640 1 --2- 172.21.15.60:0/3578585756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f50600cf600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5060070ab0 tx=0x7f505002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68228957c6452cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5057fff640 1 -- 172.21.15.60:0/3578585756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050037070 con 0x7f50600c70d0 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5057fff640 1 -- 172.21.15.60:0/3578585756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050032440 con 0x7f50600c70d0 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5057fff640 1 -- 172.21.15.60:0/3578585756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050032810 con 0x7f50600c70d0 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 -- 172.21.15.60:0/3578585756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 msgr2=0x7f50600cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 --2- 172.21.15.60:0/3578585756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f50600cf600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5060070ab0 tx=0x7f505002a230 comp rx=0 tx=0).stop 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 -- 172.21.15.60:0/3578585756 shutdown_connections 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 --2- 172.21.15.60:0/3578585756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f50600cf600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 --2- 172.21.15.60:0/3578585756 >> v2:172.21.15.100:3300/0 conn(0x7f506015bb50 0x7f50600c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 -- 172.21.15.60:0/3578585756 >> 172.21.15.60:0/3578585756 conn(0x7f506006f500 msgr2=0x7f506006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 -- 172.21.15.60:0/3578585756 shutdown_connections 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.687+0000 7f5067932640 1 -- 172.21.15.60:0/3578585756 wait complete. 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.688+0000 7f5067932640 1 Processor -- start 2024-04-25T08:29:46.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.688+0000 7f5067932640 1 -- start start 2024-04-25T08:29:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5067932640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f5060156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5067932640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600cfb40 0x7f5060154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5067932640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015bb50 0x7f506015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5067932640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506015f6d0 con 0x7f50600c70d0 2024-04-25T08:29:46.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5067932640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600c8160 con 0x7f506015bb50 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5067932640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506015f850 con 0x7f50600cfb40 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f5060156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f5060156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41780/0 (socket says 172.21.15.60:41780) 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 -- 172.21.15.60:0/3979884238 learned_addr learned my addr 172.21.15.60:0/3979884238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5065ecf640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015bb50 0x7f506015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f5064ecd640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600cfb40 0x7f5060154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 -- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600cfb40 msgr2=0x7f5060154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600cfb40 0x7f5060154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 -- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015bb50 msgr2=0x7f506015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.689+0000 7f50656ce640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015bb50 0x7f506015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f50656ce640 1 -- 172.21.15.60:0/3979884238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600ad5e0 con 0x7f50600c70d0 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5065ecf640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015bb50 0x7f506015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5064ecd640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600cfb40 0x7f5060154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:46.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f50656ce640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f5060156e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f505800e8a0 tx=0x7f505800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058017070 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505800c940 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505800cd30 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50600ad8c0 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.690+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5060154340 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.691+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5060073a00 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.692+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f505801a070 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.692+0000 7f5055ffb640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f504406b4d0 0x7f504406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.692+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f505808a6e0 con 0x7f50600c70d0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.692+0000 7f5064ecd640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f504406b4d0 0x7f504406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.693+0000 7f5064ecd640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f504406b4d0 0x7f504406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5050034000 tx=0x7f505002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.696+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50580592b0 con 0x7f50600c70d0 2024-04-25T08:29:46.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.875+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50600cf150 con 0x7f50600c70d0 2024-04-25T08:29:46.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.875+0000 7f5055ffb640 1 -- 172.21.15.60:0/3979884238 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5058058bb0 con 0x7f50600c70d0 2024-04-25T08:29:46.873 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:46.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f504406b4d0 msgr2=0x7f504406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f504406b4d0 0x7f504406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5050034000 tx=0x7f505002e000 comp rx=0 tx=0).stop 2024-04-25T08:29:46.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 msgr2=0x7f5060156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f5060156e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f505800e8a0 tx=0x7f505800ed60 comp rx=0 tx=0).stop 2024-04-25T08:29:46.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 shutdown_connections 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f504406b4d0 0x7f504406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f506015bb50 0x7f506015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50600cfb40 0x7f5060154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 --2- 172.21.15.60:0/3979884238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50600c70d0 0x7f5060156e80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 >> 172.21.15.60:0/3979884238 conn(0x7f506006f500 msgr2=0x7f50600b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 shutdown_connections 2024-04-25T08:29:46.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.877+0000 7f5067932640 1 -- 172.21.15.60:0/3979884238 wait complete. 2024-04-25T08:29:46.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:29:46.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.991+0000 7efde8688640 1 Processor -- start 2024-04-25T08:29:46.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.991+0000 7efde8688640 1 -- start start 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde8688640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efde01610f0 0x7efde006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde8688640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efde01616f0 0x7efde0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde8688640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde00ce0a0 0x7efde00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde8688640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde01614d0 con 0x7efde01610f0 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde8688640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde00c6960 con 0x7efde01616f0 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde8688640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde0162260 con 0x7efde00ce0a0 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.992+0000 7efde6424640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efde01610f0 0x7efde006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efde01610f0 0x7efde006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41786/0 (socket says 172.21.15.60:41786) 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 -- 172.21.15.60:0/986527901 learned_addr learned my addr 172.21.15.60:0/986527901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:46.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde5c23640 1 --2- 172.21.15.60:0/986527901 >> v2:172.21.15.100:3300/0 conn(0x7efde01616f0 0x7efde0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6c25640 1 --2- 172.21.15.60:0/986527901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde00ce0a0 0x7efde00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 -- 172.21.15.60:0/986527901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde00ce0a0 msgr2=0x7efde00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:46.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 --2- 172.21.15.60:0/986527901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde00ce0a0 0x7efde00ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 -- 172.21.15.60:0/986527901 >> v2:172.21.15.100:3300/0 conn(0x7efde01616f0 msgr2=0x7efde0161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:46.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 --2- 172.21.15.60:0/986527901 >> v2:172.21.15.100:3300/0 conn(0x7efde01616f0 0x7efde0161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.993+0000 7efde6424640 1 -- 172.21.15.60:0/986527901 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0162540 con 0x7efde01610f0 2024-04-25T08:29:46.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde6424640 1 --2- 172.21.15.60:0/986527901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01610f0 0x7efde006c1c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efde006f220 tx=0x7efddc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467563226faa9e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde5422640 1 -- 172.21.15.60:0/986527901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc037070 con 0x7efde01610f0 2024-04-25T08:29:46.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde5422640 1 -- 172.21.15.60:0/986527901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc032440 con 0x7efde01610f0 2024-04-25T08:29:46.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde5422640 1 -- 172.21.15.60:0/986527901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc032810 con 0x7efde01610f0 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 -- 172.21.15.60:0/986527901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01610f0 msgr2=0x7efde006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 --2- 172.21.15.60:0/986527901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01610f0 0x7efde006c1c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efde006f220 tx=0x7efddc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 -- 172.21.15.60:0/986527901 shutdown_connections 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 --2- 172.21.15.60:0/986527901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde00ce0a0 0x7efde00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 --2- 172.21.15.60:0/986527901 >> v2:172.21.15.100:3300/0 conn(0x7efde01616f0 0x7efde0161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 --2- 172.21.15.60:0/986527901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01610f0 0x7efde006c1c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 -- 172.21.15.60:0/986527901 >> 172.21.15.60:0/986527901 conn(0x7efde006f500 msgr2=0x7efde006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 -- 172.21.15.60:0/986527901 shutdown_connections 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.994+0000 7efde8688640 1 -- 172.21.15.60:0/986527901 wait complete. 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 Processor -- start 2024-04-25T08:29:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 -- start start 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efde00ce0a0 0x7efde01452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde01610f0 0x7efde00ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 0x7efde00ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00cf690 con 0x7efde01616f0 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00cfce0 con 0x7efde00ce0a0 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde8688640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde006c700 con 0x7efde01610f0 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde6c25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 0x7efde00ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde6c25640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 0x7efde00ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41790/0 (socket says 172.21.15.60:41790) 2024-04-25T08:29:46.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.996+0000 7efde6c25640 1 -- 172.21.15.60:0/333751197 learned_addr learned my addr 172.21.15.60:0/333751197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6424640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efde00ce0a0 0x7efde01452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde5c23640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde01610f0 0x7efde00ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6c25640 1 -- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde01610f0 msgr2=0x7efde00ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6c25640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde01610f0 0x7efde00ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6c25640 1 -- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efde00ce0a0 msgr2=0x7efde01452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6c25640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efde00ce0a0 0x7efde01452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6c25640 1 -- 172.21.15.60:0/333751197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde01433e0 con 0x7efde01616f0 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde5c23640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde01610f0 0x7efde00ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde6c25640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 0x7efde00ae500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdd8007e90 tx=0x7efdd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8014420 con 0x7efde01616f0 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd8019070 con 0x7efde01616f0 2024-04-25T08:29:46.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde01436c0 con 0x7efde01616f0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8014820 con 0x7efde01616f0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.997+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde0143f60 con 0x7efde01616f0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.998+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde0143890 con 0x7efde01616f0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.999+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdd80149c0 con 0x7efde01616f0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.999+0000 7efdd6ffd640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efdc406b4d0 0x7efdc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.999+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efdd808ad30 con 0x7efde01616f0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:46.999+0000 7efde6424640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efdc406b4d0 0x7efdc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.000+0000 7efde6424640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efdc406b4d0 0x7efdc406d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efddc034000 tx=0x7efddc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.003+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdd8059870 con 0x7efde01616f0 2024-04-25T08:29:47.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.185+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efde0073a00 con 0x7efde01616f0 2024-04-25T08:29:47.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.186+0000 7efdd6ffd640 1 -- 172.21.15.60:0/333751197 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7efdd8059170 con 0x7efde01616f0 2024-04-25T08:29:47.184 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efdc406b4d0 msgr2=0x7efdc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efdc406b4d0 0x7efdc406d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efddc034000 tx=0x7efddc02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 msgr2=0x7efde00ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 0x7efde00ae500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdd8007e90 tx=0x7efdd800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 shutdown_connections 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efde01616f0 0x7efde00ae500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efde01610f0 0x7efde00ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efdc406b4d0 0x7efdc406d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 --2- 172.21.15.60:0/333751197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efde00ce0a0 0x7efde01452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 >> 172.21.15.60:0/333751197 conn(0x7efde006f500 msgr2=0x7efde0160410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 shutdown_connections 2024-04-25T08:29:47.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.188+0000 7efde8688640 1 -- 172.21.15.60:0/333751197 wait complete. 2024-04-25T08:29:47.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:29:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.302+0000 7f8a68f5c640 1 Processor -- start 2024-04-25T08:29:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.302+0000 7f8a68f5c640 1 -- start start 2024-04-25T08:29:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a68f5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 0x7f8a640ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a68f5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6406cda0 con 0x7f8a64161710 2024-04-25T08:29:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a68f5c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6415e000 con 0x7f8a640d32f0 2024-04-25T08:29:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a68f5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640d1de0 con 0x7f8a6415dc20 2024-04-25T08:29:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a59d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8a64161710 0x7f8a6406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42106/0 (socket says 172.21.15.60:42106) 2024-04-25T08:29:47.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a59d9b640 1 -- 172.21.15.60:0/2688039937 learned_addr learned my addr 172.21.15.60:0/2688039937 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a6259c640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 0x7f8a640ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199813752 0 0) 0x7f8a6406cda0 con 0x7f8a64161710 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.303+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a50003520 con 0x7f8a64161710 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621887843 0 0) 0x7f8a6415e000 con 0x7f8a640d32f0 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a6406cda0 con 0x7f8a640d32f0 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3043370224 0 0) 0x7f8a50003520 con 0x7f8a64161710 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 msgr2=0x7f8a640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 0x7f8a640ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 >> v1:172.21.15.100:6789/0 conn(0x7f8a640d32f0 legacy=0x7f8a6406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a64161c60 con 0x7f8a64161710 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f8a4c001670 con 0x7f8a64161710 2024-04-25T08:29:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 >> v1:172.21.15.60:6789/0 conn(0x7f8a64161710 legacy=0x7f8a6406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a50003b70 0x7f8a50003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 0x7f8a50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a50004d40 0x7f8a500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640d1de0 con 0x7f8a50004d40 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5000ba70 con 0x7f8a50003b70 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5000bc70 con 0x7f8a6415dc20 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a50004d40 0x7f8a500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a6259c640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a50003b70 0x7f8a50003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a59d9b640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 0x7f8a50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a62d9d640 1 -- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 msgr2=0x7f8a50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a6415dc20 0x7f8a50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a62d9d640 1 -- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a50003b70 msgr2=0x7f8a50003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a50003b70 0x7f8a50003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.304+0000 7f8a62d9d640 1 -- 172.21.15.60:0/2688039937 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50003520 con 0x7f8a50004d40 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.305+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a50004d40 0x7f8a500051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001fa0 tx=0x7f8a50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e7ab300d91c46a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.305+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c001bf0 con 0x7f8a50004d40 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.305+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c003b70 con 0x7f8a50004d40 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.305+0000 7f8a61d9b640 1 -- 172.21.15.60:0/2688039937 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0050d0 con 0x7f8a50004d40 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a68f5c640 1 -- 172.21.15.60:0/2688039937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a50004d40 msgr2=0x7f8a500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/2688039937 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a50004d40 0x7f8a500051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c001fa0 tx=0x7f8a50005830 comp rx=0 tx=0).stop 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a62d9d640 1 -- 172.21.15.60:0/2688039937 reap_dead start 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a68f5c640 1 -- 172.21.15.60:0/2688039937 shutdown_connections 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a68f5c640 1 -- 172.21.15.60:0/2688039937 >> 172.21.15.60:0/2688039937 conn(0x7f8a640d0420 msgr2=0x7f8a640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a68f5c640 1 -- 172.21.15.60:0/2688039937 shutdown_connections 2024-04-25T08:29:47.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.306+0000 7f8a68f5c640 1 -- 172.21.15.60:0/2688039937 wait complete. 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.307+0000 7f8a68f5c640 1 Processor -- start 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.307+0000 7f8a68f5c640 1 -- start start 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a68f5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a641599a0 0x7f8a640cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a68f5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a641f3880 0x7f8a641f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a68f5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 0x7f8a640cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a68f5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a64162500 con 0x7f8a640cd420 2024-04-25T08:29:47.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a68f5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a64162220 con 0x7f8a641599a0 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a68f5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640b5e50 con 0x7f8a641f3880 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 0x7f8a640cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 0x7f8a640cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41812/0 (socket says 172.21.15.60:41812) 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 -- 172.21.15.60:0/721386979 learned_addr learned my addr 172.21.15.60:0/721386979 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a59d9b640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a641f3880 0x7f8a641f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a6259c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a641599a0 0x7f8a640cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 -- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a641f3880 msgr2=0x7f8a641f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a641f3880 0x7f8a641f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 -- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a641599a0 msgr2=0x7f8a640cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a641599a0 0x7f8a640cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.308+0000 7f8a62d9d640 1 -- 172.21.15.60:0/721386979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640cc0e0 con 0x7f8a640cd420 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.309+0000 7f8a62d9d640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 0x7f8a640cd8a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c005040 tx=0x7f8a5c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.309+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c001bf0 con 0x7f8a640cd420 2024-04-25T08:29:47.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.309+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c003970 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.309+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640cc380 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.309+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c005590 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.309+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a640cc940 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.310+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a64073a00 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.310+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a5c045020 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.311+0000 7f8a5b7fe640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a4406b4d0 0x7f8a4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.311+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8a5c0aa810 con 0x7f8a640cd420 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.311+0000 7f8a6259c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a4406b4d0 0x7f8a4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.311+0000 7f8a6259c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a4406b4d0 0x7f8a4406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009b00 tx=0x7f8a54008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.315+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a5c079470 con 0x7f8a640cd420 2024-04-25T08:29:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.499+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a640d16b0 con 0x7f8a640cd420 2024-04-25T08:29:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.500+0000 7f8a5b7fe640 1 -- 172.21.15.60:0/721386979 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a5c0a8e20 con 0x7f8a640cd420 2024-04-25T08:29:47.498 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a4406b4d0 msgr2=0x7f8a4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a4406b4d0 0x7f8a4406d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009b00 tx=0x7f8a54008040 comp rx=0 tx=0).stop 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 msgr2=0x7f8a640cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 0x7f8a640cd8a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c005040 tx=0x7f8a5c001650 comp rx=0 tx=0).stop 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 shutdown_connections 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a640cd420 0x7f8a640cd8a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a641f3880 0x7f8a641f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a4406b4d0 0x7f8a4406d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 --2- 172.21.15.60:0/721386979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a641599a0 0x7f8a640cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 >> 172.21.15.60:0/721386979 conn(0x7f8a640d0420 msgr2=0x7f8a640b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:47.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 shutdown_connections 2024-04-25T08:29:47.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.502+0000 7f8a68f5c640 1 -- 172.21.15.60:0/721386979 wait complete. 2024-04-25T08:29:47.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:29:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.616+0000 7f21d983f640 1 Processor -- start 2024-04-25T08:29:47.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.617+0000 7f21d983f640 1 -- start start 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.617+0000 7f21d983f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 0x7f21d40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.617+0000 7f21d983f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 0x7f21d40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.617+0000 7f21d983f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d415e000 con 0x7f21d40b9330 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.617+0000 7f21d983f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d40d1de0 con 0x7f21d415dc20 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.617+0000 7f21d983f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4161c60 con 0x7f21d4161710 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 0x7f21d40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d27fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 0x7f21d40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d37fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f21d40b9330 0x7f21d406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43856/0 (socket says 172.21.15.60:43856) 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 0x7f21d40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33616/0 (socket says 172.21.15.60:33616) 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d37fe640 1 -- 172.21.15.60:0/1427887910 learned_addr learned my addr 172.21.15.60:0/1427887910 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d27fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 0x7f21d40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59308/0 (socket says 172.21.15.60:59308) 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d27fc640 1 -- 172.21.15.60:0/1427887910 >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 msgr2=0x7f21d40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d27fc640 1 --2- 172.21.15.60:0/1427887910 >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 0x7f21d40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d27fc640 1 -- 172.21.15.60:0/1427887910 >> v1:172.21.15.100:6789/0 conn(0x7f21d40b9330 legacy=0x7f21d406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d1ffb640 1 -- 172.21.15.60:0/1427887910 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154882055 0 0) 0x7f21d415e000 con 0x7f21d40b9330 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d27fc640 1 -- 172.21.15.60:0/1427887910 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4161f40 con 0x7f21d4161710 2024-04-25T08:29:47.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.618+0000 7f21d2ffd640 1 --2- 172.21.15.60:0/1427887910 >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 0x7f21d40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.619+0000 7f21d27fc640 1 --2- 172.21.15.60:0/1427887910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 0x7f21d40cebb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0097a0 tx=0x7f21bc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=486384d5d9bef710 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.619+0000 7f21d1ffb640 1 -- 172.21.15.60:0/1427887910 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc037070 con 0x7f21d4161710 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.619+0000 7f21d1ffb640 1 -- 172.21.15.60:0/1427887910 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc032440 con 0x7f21d4161710 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.619+0000 7f21d1ffb640 1 -- 172.21.15.60:0/1427887910 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc032810 con 0x7f21d4161710 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 -- 172.21.15.60:0/1427887910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 msgr2=0x7f21d40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 --2- 172.21.15.60:0/1427887910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 0x7f21d40cebb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0097a0 tx=0x7f21bc02a180 comp rx=0 tx=0).stop 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 -- 172.21.15.60:0/1427887910 shutdown_connections 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 --2- 172.21.15.60:0/1427887910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d4161710 0x7f21d40cebb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 --2- 172.21.15.60:0/1427887910 >> v2:172.21.15.100:3300/0 conn(0x7f21d415dc20 0x7f21d40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 -- 172.21.15.60:0/1427887910 >> 172.21.15.60:0/1427887910 conn(0x7f21d40d0420 msgr2=0x7f21d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 -- 172.21.15.60:0/1427887910 shutdown_connections 2024-04-25T08:29:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.620+0000 7f21d983f640 1 -- 172.21.15.60:0/1427887910 wait complete. 2024-04-25T08:29:47.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.621+0000 7f21d983f640 1 Processor -- start 2024-04-25T08:29:47.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.621+0000 7f21d983f640 1 -- start start 2024-04-25T08:29:47.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d983f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40b9330 0x7f21d41551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d983f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d415dc20 0x7f21d40d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d983f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 0x7f21d40d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d983f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40b5e50 con 0x7f21d4161710 2024-04-25T08:29:47.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d983f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4162500 con 0x7f21d415dc20 2024-04-25T08:29:47.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d983f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d40b5fd0 con 0x7f21d40b9330 2024-04-25T08:29:47.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 0x7f21d40d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d2ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40b9330 0x7f21d41551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d415dc20 0x7f21d40d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 0x7f21d40d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41816/0 (socket says 172.21.15.60:41816) 2024-04-25T08:29:47.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d2ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40b9330 0x7f21d41551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:59312/0 (socket says 172.21.15.60:59312) 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 -- 172.21.15.60:0/4256922616 learned_addr learned my addr 172.21.15.60:0/4256922616 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 -- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40b9330 msgr2=0x7f21d41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40b9330 0x7f21d41551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 -- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d415dc20 msgr2=0x7f21d40d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d415dc20 0x7f21d40d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 -- 172.21.15.60:0/4256922616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d41f37c0 con 0x7f21d4161710 2024-04-25T08:29:47.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.622+0000 7f21d37fe640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 0x7f21d40d3ec0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c410 tx=0x7f21cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.623+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0074e0 con 0x7f21d4161710 2024-04-25T08:29:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.623+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc007680 con 0x7f21d4161710 2024-04-25T08:29:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.623+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d41f3aa0 con 0x7f21d4161710 2024-04-25T08:29:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.623+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc004a20 con 0x7f21d4161710 2024-04-25T08:29:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.623+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d41f4060 con 0x7f21d4161710 2024-04-25T08:29:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.624+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4073a00 con 0x7f21d4161710 2024-04-25T08:29:47.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.624+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21cc011070 con 0x7f21d4161710 2024-04-25T08:29:47.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.625+0000 7f21cbfff640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21b406b500 0x7f21b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.625+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f21cc089f20 con 0x7f21d4161710 2024-04-25T08:29:47.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.625+0000 7f21d2ffd640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21b406b500 0x7f21b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.625+0000 7f21d2ffd640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21b406b500 0x7f21b406d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009bd0 tx=0x7f21c400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.629+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21cc058410 con 0x7f21d4161710 2024-04-25T08:29:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.809+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f21d40d16b0 con 0x7f21d4161710 2024-04-25T08:29:47.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.809+0000 7f21cbfff640 1 -- 172.21.15.60:0/4256922616 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f21cc05c4b0 con 0x7f21d4161710 2024-04-25T08:29:47.807 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21b406b500 msgr2=0x7f21b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21b406b500 0x7f21b406d9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009bd0 tx=0x7f21c400b040 comp rx=0 tx=0).stop 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 msgr2=0x7f21d40d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 0x7f21d40d3ec0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c410 tx=0x7f21cc00c8d0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 shutdown_connections 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d4161710 0x7f21d40d3ec0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21b406b500 0x7f21b406d9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d415dc20 0x7f21d40d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 --2- 172.21.15.60:0/4256922616 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d40b9330 0x7f21d41551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.811+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 >> 172.21.15.60:0/4256922616 conn(0x7f21d40d0420 msgr2=0x7f21d40d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.812+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 shutdown_connections 2024-04-25T08:29:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.812+0000 7f21d983f640 1 -- 172.21.15.60:0/4256922616 wait complete. 2024-04-25T08:29:47.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.923+0000 7f262b3db640 1 Processor -- start 2024-04-25T08:29:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.924+0000 7f262b3db640 1 -- start start 2024-04-25T08:29:47.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.924+0000 7f262b3db640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f262415db70 0x7f2624162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.924+0000 7f262b3db640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f26240b6cb0 0x7f26240b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.924+0000 7f262b3db640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2624162690 con 0x7f26240b5a60 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.924+0000 7f262b3db640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240d3800 con 0x7f262415db70 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.924+0000 7f262b3db640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240ceee0 con 0x7f26240b6cb0 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629177640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f262415db70 0x7f2624162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629978640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f26240b5a60 0x7f262406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42110/0 (socket says 172.21.15.60:42110) 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629978640 1 -- 172.21.15.60:0/3740722920 learned_addr learned my addr 172.21.15.60:0/3740722920 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2628976640 1 --2- 172.21.15.60:0/3740722920 >> v2:172.21.15.100:3300/0 conn(0x7f26240b6cb0 0x7f26240b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629177640 1 -- 172.21.15.60:0/3740722920 >> v2:172.21.15.100:3300/0 conn(0x7f26240b6cb0 msgr2=0x7f26240b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629177640 1 --2- 172.21.15.60:0/3740722920 >> v2:172.21.15.100:3300/0 conn(0x7f26240b6cb0 0x7f26240b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629177640 1 -- 172.21.15.60:0/3740722920 >> v1:172.21.15.60:6789/0 conn(0x7f26240b5a60 legacy=0x7f262406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f261bfff640 1 -- 172.21.15.60:0/3740722920 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005697157 0 0) 0x7f2624162690 con 0x7f26240b5a60 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.925+0000 7f2629177640 1 -- 172.21.15.60:0/3740722920 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240d1d10 con 0x7f262415db70 2024-04-25T08:29:47.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f2629177640 1 --2- 172.21.15.60:0/3740722920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f2624162150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26240d3980 tx=0x7f262002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0816c8ca00e476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f261bfff640 1 -- 172.21.15.60:0/3740722920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620037070 con 0x7f262415db70 2024-04-25T08:29:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f261bfff640 1 -- 172.21.15.60:0/3740722920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620032440 con 0x7f262415db70 2024-04-25T08:29:47.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f261bfff640 1 -- 172.21.15.60:0/3740722920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262002ae60 con 0x7f262415db70 2024-04-25T08:29:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 -- 172.21.15.60:0/3740722920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 msgr2=0x7f2624162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 --2- 172.21.15.60:0/3740722920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f2624162150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26240d3980 tx=0x7f262002a110 comp rx=0 tx=0).stop 2024-04-25T08:29:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 -- 172.21.15.60:0/3740722920 shutdown_connections 2024-04-25T08:29:47.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 --2- 172.21.15.60:0/3740722920 >> v2:172.21.15.100:3300/0 conn(0x7f26240b6cb0 0x7f26240b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 --2- 172.21.15.60:0/3740722920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f2624162150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 -- 172.21.15.60:0/3740722920 >> 172.21.15.60:0/3740722920 conn(0x7f26240d0520 msgr2=0x7f26240d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 -- 172.21.15.60:0/3740722920 shutdown_connections 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.926+0000 7f262b3db640 1 -- 172.21.15.60:0/3740722920 wait complete. 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 Processor -- start 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 -- start start 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26240b5a60 0x7f26240c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26240b6cb0 0x7f26240ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f26240cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262415df50 con 0x7f262415db70 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240d22d0 con 0x7f26240b6cb0 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f262b3db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240ce530 con 0x7f26240b5a60 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f2629978640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f26240cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f2629978640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f26240cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41836/0 (socket says 172.21.15.60:41836) 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.928+0000 7f2629978640 1 -- 172.21.15.60:0/2451742655 learned_addr learned my addr 172.21.15.60:0/2451742655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629177640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26240b5a60 0x7f26240c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2628976640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26240b6cb0 0x7f26240ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629978640 1 -- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26240b5a60 msgr2=0x7f26240c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629978640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26240b5a60 0x7f26240c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629978640 1 -- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26240b6cb0 msgr2=0x7f26240ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629978640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26240b6cb0 0x7f26240ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629978640 1 -- 172.21.15.60:0/2451742655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240cbc00 con 0x7f262415db70 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2629978640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f26240cd740 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c7a0 tx=0x7f261c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c018070 con 0x7f262415db70 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00a640 con 0x7f262415db70 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26240cbee0 con 0x7f262415db70 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00aab0 con 0x7f262415db70 2024-04-25T08:29:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.929+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240cc4a0 con 0x7f262415db70 2024-04-25T08:29:47.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.930+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624073990 con 0x7f262415db70 2024-04-25T08:29:47.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.930+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f261c01b070 con 0x7f262415db70 2024-04-25T08:29:47.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.931+0000 7f2619ffb640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:47.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.931+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f261c089c20 con 0x7f262415db70 2024-04-25T08:29:47.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.931+0000 7f2629177640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:47.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.931+0000 7f2629177640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f260806b4d0 0x7f260806d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2620034000 tx=0x7f262002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:47.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:47.935+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f261c0586c0 con 0x7f262415db70 2024-04-25T08:29:48.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.153+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f262415d320 con 0x7f262415db70 2024-04-25T08:29:48.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.154+0000 7f2619ffb640 1 -- 172.21.15.60:0/2451742655 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f261c057fc0 con 0x7f262415db70 2024-04-25T08:29:48.152 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:48.152 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:29:48.152 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:29:48.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f260806b4d0 msgr2=0x7f260806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f260806b4d0 0x7f260806d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2620034000 tx=0x7f262002e000 comp rx=0 tx=0).stop 2024-04-25T08:29:48.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 msgr2=0x7f26240cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f26240cd740 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c7a0 tx=0x7f261c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 shutdown_connections 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f262415db70 0x7f26240cd740 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26240b6cb0 0x7f26240ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 --2- 172.21.15.60:0/2451742655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26240b5a60 0x7f26240c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 >> 172.21.15.60:0/2451742655 conn(0x7f26240d0520 msgr2=0x7f2624162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 shutdown_connections 2024-04-25T08:29:48.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.156+0000 7f262b3db640 1 -- 172.21.15.60:0/2451742655 wait complete. 2024-04-25T08:29:48.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.268+0000 7f3ad081f640 1 Processor -- start 2024-04-25T08:29:48.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.268+0000 7f3ad081f640 1 -- start start 2024-04-25T08:29:48.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ad081f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3acc15dbe0 0x7f3acc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ad081f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3acc0b2cb0 0x7f3acc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ad081f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc0b97a0 con 0x7f3acc0b92a0 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ad081f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc15dfc0 con 0x7f3acc15dbe0 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ad081f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc0d1a00 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3aca59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3acc15dbe0 0x7f3acc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3acad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3acc0b92a0 0x7f3acc0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42122/0 (socket says 172.21.15.60:42122) 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3acad9d640 1 -- 172.21.15.60:0/4158424473 learned_addr learned my addr 172.21.15.60:0/4158424473 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ac9d9b640 1 --2- 172.21.15.60:0/4158424473 >> v2:172.21.15.100:3300/0 conn(0x7f3acc0b2cb0 0x7f3acc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3aca59c640 1 -- 172.21.15.60:0/4158424473 >> v2:172.21.15.100:3300/0 conn(0x7f3acc0b2cb0 msgr2=0x7f3acc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3aca59c640 1 --2- 172.21.15.60:0/4158424473 >> v2:172.21.15.100:3300/0 conn(0x7f3acc0b2cb0 0x7f3acc0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3aca59c640 1 -- 172.21.15.60:0/4158424473 >> v1:172.21.15.60:6789/0 conn(0x7f3acc0b92a0 legacy=0x7f3acc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.270+0000 7f3aca59c640 1 -- 172.21.15.60:0/4158424473 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc0d1c50 con 0x7f3acc15dbe0 2024-04-25T08:29:48.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.269+0000 7f3ac959a640 1 -- 172.21.15.60:0/4158424473 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122665155 0 0) 0x7f3acc0b97a0 con 0x7f3acc0b92a0 2024-04-25T08:29:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.270+0000 7f3ac9d9b640 1 --2- 172.21.15.60:0/4158424473 >> v2:172.21.15.100:3300/0 conn(0x7f3acc0b2cb0 0x7f3acc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.270+0000 7f3aca59c640 1 --2- 172.21.15.60:0/4158424473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc15dbe0 0x7f3acc0d0910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3acc06cff0 tx=0x7f3ac402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91ba29b863654b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.270+0000 7f3ac959a640 1 -- 172.21.15.60:0/4158424473 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4037070 con 0x7f3acc15dbe0 2024-04-25T08:29:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.270+0000 7f3ac959a640 1 -- 172.21.15.60:0/4158424473 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4032440 con 0x7f3acc15dbe0 2024-04-25T08:29:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.270+0000 7f3ac959a640 1 -- 172.21.15.60:0/4158424473 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac402ae60 con 0x7f3acc15dbe0 2024-04-25T08:29:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 -- 172.21.15.60:0/4158424473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc15dbe0 msgr2=0x7f3acc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 --2- 172.21.15.60:0/4158424473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc15dbe0 0x7f3acc0d0910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3acc06cff0 tx=0x7f3ac402a110 comp rx=0 tx=0).stop 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 -- 172.21.15.60:0/4158424473 shutdown_connections 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 --2- 172.21.15.60:0/4158424473 >> v2:172.21.15.100:3300/0 conn(0x7f3acc0b2cb0 0x7f3acc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 --2- 172.21.15.60:0/4158424473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc15dbe0 0x7f3acc0d0910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 -- 172.21.15.60:0/4158424473 >> 172.21.15.60:0/4158424473 conn(0x7f3acc061a20 msgr2=0x7f3acc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 -- 172.21.15.60:0/4158424473 shutdown_connections 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.271+0000 7f3ad081f640 1 -- 172.21.15.60:0/4158424473 wait complete. 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.272+0000 7f3ad081f640 1 Processor -- start 2024-04-25T08:29:48.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.272+0000 7f3ad081f640 1 -- start start 2024-04-25T08:29:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3ad081f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 0x7f3acc15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3ad081f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc0b92a0 0x7f3acc1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3ad081f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc15dbe0 0x7f3acc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3ad081f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0b6ab0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3ad081f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0d2210 con 0x7f3acc0b92a0 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3ad081f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0b6c30 con 0x7f3acc15dbe0 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3aca59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 0x7f3acc15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3aca59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 0x7f3acc15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41860/0 (socket says 172.21.15.60:41860) 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.273+0000 7f3aca59c640 1 -- 172.21.15.60:0/1866300654 learned_addr learned my addr 172.21.15.60:0/1866300654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3aca59c640 1 -- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc15dbe0 msgr2=0x7f3acc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3aca59c640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc15dbe0 0x7f3acc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3aca59c640 1 -- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc0b92a0 msgr2=0x7f3acc1f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3ac9d9b640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc0b92a0 0x7f3acc1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3aca59c640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc0b92a0 0x7f3acc1f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3aca59c640 1 -- 172.21.15.60:0/1866300654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc1f37d0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3aca59c640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 0x7f3acc15a010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000c00 tx=0x7f3ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4037040 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4042070 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4030bd0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc1f3ab0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.274+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc0b3480 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.275+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc1f3fd0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.275+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ac4047460 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.276+0000 7f3abaffd640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.276+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ac40ab1e0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.276+0000 7f3ac9d9b640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.277+0000 7f3ac9d9b640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3acc160450 tx=0x7f3ac0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.280+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3acc1f3fd0 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.461+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3acc15be30 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.462+0000 7f3abaffd640 1 -- 172.21.15.60:0/1866300654 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f3acc15be30 con 0x7f3acc0b2cb0 2024-04-25T08:29:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:48.461 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 msgr2=0x7f3aac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3acc160450 tx=0x7f3ac0008040 comp rx=0 tx=0).stop 2024-04-25T08:29:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 msgr2=0x7f3acc15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 0x7f3acc15a010 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000c00 tx=0x7f3ac40046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 shutdown_connections 2024-04-25T08:29:48.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3acc15dbe0 0x7f3acc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3acc0b92a0 0x7f3acc1f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 --2- 172.21.15.60:0/1866300654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3acc0b2cb0 0x7f3acc15a010 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 >> 172.21.15.60:0/1866300654 conn(0x7f3acc061a20 msgr2=0x7f3acc1609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 shutdown_connections 2024-04-25T08:29:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.465+0000 7f3ad081f640 1 -- 172.21.15.60:0/1866300654 wait complete. 2024-04-25T08:29:48.474 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:29:48.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.580+0000 7f4d58a5d640 1 Processor -- start 2024-04-25T08:29:48.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.580+0000 7f4d58a5d640 1 -- start start 2024-04-25T08:29:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d58a5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4d540cadb0 0x7f4d540cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d58a5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4d540ccfe0 0x7f4d540cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d58a5d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d540afc80 con 0x7f4d5415de80 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d58a5d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d540cb240 con 0x7f4d540cadb0 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d58a5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d540ce4d0 con 0x7f4d540ccfe0 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d5259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4d5415de80 0x7f4d540cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42128/0 (socket says 172.21.15.60:42128) 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d5259c640 1 -- 172.21.15.60:0/3597809214 learned_addr learned my addr 172.21.15.60:0/3597809214 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d51d9b640 1 --2- 172.21.15.60:0/3597809214 >> v2:172.21.15.60:3300/0 conn(0x7f4d540cadb0 0x7f4d540cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d52d9d640 1 --2- 172.21.15.60:0/3597809214 >> v2:172.21.15.100:3300/0 conn(0x7f4d540ccfe0 0x7f4d540cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d5159a640 1 -- 172.21.15.60:0/3597809214 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128769018 0 0) 0x7f4d540afc80 con 0x7f4d5415de80 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.581+0000 7f4d5159a640 1 -- 172.21.15.60:0/3597809214 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d3c003520 con 0x7f4d5415de80 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d51d9b640 1 -- 172.21.15.60:0/3597809214 >> v2:172.21.15.100:3300/0 conn(0x7f4d540ccfe0 msgr2=0x7f4d540cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d51d9b640 1 --2- 172.21.15.60:0/3597809214 >> v2:172.21.15.100:3300/0 conn(0x7f4d540ccfe0 0x7f4d540cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d51d9b640 1 -- 172.21.15.60:0/3597809214 >> v1:172.21.15.60:6789/0 conn(0x7f4d5415de80 legacy=0x7f4d540cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d51d9b640 1 -- 172.21.15.60:0/3597809214 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540ce7b0 con 0x7f4d540cadb0 2024-04-25T08:29:48.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d51d9b640 1 --2- 172.21.15.60:0/3597809214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540cadb0 0x7f4d540cf6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d5406e2a0 tx=0x7f4d4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336ccc5c8a072049 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d5159a640 1 -- 172.21.15.60:0/3597809214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037070 con 0x7f4d540cadb0 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d5159a640 1 -- 172.21.15.60:0/3597809214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48032440 con 0x7f4d540cadb0 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.582+0000 7f4d5159a640 1 -- 172.21.15.60:0/3597809214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48032810 con 0x7f4d540cadb0 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 -- 172.21.15.60:0/3597809214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540cadb0 msgr2=0x7f4d540cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/3597809214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540cadb0 0x7f4d540cf6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d5406e2a0 tx=0x7f4d4802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 -- 172.21.15.60:0/3597809214 shutdown_connections 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/3597809214 >> v2:172.21.15.100:3300/0 conn(0x7f4d540ccfe0 0x7f4d540cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/3597809214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540cadb0 0x7f4d540cf6c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 -- 172.21.15.60:0/3597809214 >> 172.21.15.60:0/3597809214 conn(0x7f4d540b2390 msgr2=0x7f4d540b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 -- 172.21.15.60:0/3597809214 shutdown_connections 2024-04-25T08:29:48.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.583+0000 7f4d58a5d640 1 -- 172.21.15.60:0/3597809214 wait complete. 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.584+0000 7f4d58a5d640 1 Processor -- start 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.584+0000 7f4d58a5d640 1 -- start start 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d58a5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d540cadb0 0x7f4d5415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d58a5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 0x7f4d540b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d58a5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5415de80 0x7f4d5415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d58a5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5415e260 con 0x7f4d540ccfe0 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d58a5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540cfc00 con 0x7f4d5415de80 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d58a5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540b2eb0 con 0x7f4d540cadb0 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 0x7f4d540b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 0x7f4d540b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41872/0 (socket says 172.21.15.60:41872) 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d51d9b640 1 -- 172.21.15.60:0/14279515 learned_addr learned my addr 172.21.15.60:0/14279515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d52d9d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5415de80 0x7f4d5415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.585+0000 7f4d5259c640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d540cadb0 0x7f4d5415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d51d9b640 1 -- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d540cadb0 msgr2=0x7f4d5415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d51d9b640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d540cadb0 0x7f4d5415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d51d9b640 1 -- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5415de80 msgr2=0x7f4d5415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d51d9b640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5415de80 0x7f4d5415c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d51d9b640 1 -- 172.21.15.60:0/14279515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540c7830 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d5259c640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d540cadb0 0x7f4d5415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d52d9d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5415de80 0x7f4d5415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d51d9b640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 0x7f4d540b3480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4d48000c00 tx=0x7f4d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48041020 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4803d700 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48037040 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d540c7b10 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.586+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d540c80d0 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.587+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d54073a00 con 0x7f4d540ccfe0 2024-04-25T08:29:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.587+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d48044020 con 0x7f4d540ccfe0 2024-04-25T08:29:48.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.588+0000 7f4d42ffd640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d3406b4d0 0x7f4d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.588+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4d480aac90 con 0x7f4d540ccfe0 2024-04-25T08:29:48.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.588+0000 7f4d5259c640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d3406b4d0 0x7f4d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.589+0000 7f4d5259c640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d3406b4d0 0x7f4d3406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d44005880 tx=0x7f4d44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.592+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d4804b030 con 0x7f4d540ccfe0 2024-04-25T08:29:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.772+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d540cb3d0 con 0x7f4d540ccfe0 2024-04-25T08:29:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.772+0000 7f4d42ffd640 1 -- 172.21.15.60:0/14279515 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d480797e0 con 0x7f4d540ccfe0 2024-04-25T08:29:48.770 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d3406b4d0 msgr2=0x7f4d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d3406b4d0 0x7f4d3406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d44005880 tx=0x7f4d44009040 comp rx=0 tx=0).stop 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 msgr2=0x7f4d540b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 0x7f4d540b3480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4d48000c00 tx=0x7f4d480046c0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 shutdown_connections 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d3406b4d0 0x7f4d3406d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d5415de80 0x7f4d5415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4d540ccfe0 0x7f4d540b3480 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 --2- 172.21.15.60:0/14279515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4d540cadb0 0x7f4d5415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 >> 172.21.15.60:0/14279515 conn(0x7f4d540b2390 msgr2=0x7f4d540cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 shutdown_connections 2024-04-25T08:29:48.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.774+0000 7f4d58a5d640 1 -- 172.21.15.60:0/14279515 wait complete. 2024-04-25T08:29:48.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:29:48.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.889+0000 7ff93b638640 1 Processor -- start 2024-04-25T08:29:48.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.889+0000 7ff93b638640 1 -- start start 2024-04-25T08:29:48.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff93b638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 0x7ff9340b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff93b638640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93406cf50 con 0x7ff93415b7d0 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff93b638640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340ce530 con 0x7ff9340b5620 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff93b638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340ce810 con 0x7ff93406c9e0 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff9393d4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff93415b7d0 0x7ff9340cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42140/0 (socket says 172.21.15.60:42140) 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff9393d4640 1 -- 172.21.15.60:0/9359179 learned_addr learned my addr 172.21.15.60:0/9359179 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff939bd5640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 0x7ff9340b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101211325 0 0) 0x7ff93406cf50 con 0x7ff93415b7d0 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff920003520 con 0x7ff93415b7d0 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254835871 0 0) 0x7ff9340ce530 con 0x7ff9340b5620 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff93406cf50 con 0x7ff9340b5620 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795803854 0 0) 0x7ff920003520 con 0x7ff93415b7d0 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 msgr2=0x7ff9340b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 0x7ff9340b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 >> v1:172.21.15.100:6789/0 conn(0x7ff9340b5620 legacy=0x7ff93406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9340ceaf0 con 0x7ff93415b7d0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ff930001690 con 0x7ff93415b7d0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 >> v1:172.21.15.60:6789/0 conn(0x7ff93415b7d0 legacy=0x7ff9340cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 0x7ff920003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9200040d0 0x7ff920004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920004d40 0x7ff9200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340ce810 con 0x7ff920004d40 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92000baa0 con 0x7ff9200040d0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.890+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92000bca0 con 0x7ff93406c9e0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920004d40 0x7ff9200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff938bd3640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9200040d0 0x7ff920004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff9393d4640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 0x7ff920003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 -- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 msgr2=0x7ff920003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff93406c9e0 0x7ff920003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 -- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9200040d0 msgr2=0x7ff920004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9200040d0 0x7ff920004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 -- 172.21.15.60:0/9359179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920003520 con 0x7ff920004d40 2024-04-25T08:29:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.891+0000 7ff939bd5640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920004d40 0x7ff9200051c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009fc0 tx=0x7ff920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651fa7eaee157ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c009c10 con 0x7ff920004d40 2024-04-25T08:29:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c036070 con 0x7ff920004d40 2024-04-25T08:29:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff92bfff640 1 -- 172.21.15.60:0/9359179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c031880 con 0x7ff920004d40 2024-04-25T08:29:48.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff93b638640 1 -- 172.21.15.60:0/9359179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920004d40 msgr2=0x7ff9200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff93b638640 1 --2- 172.21.15.60:0/9359179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff920004d40 0x7ff9200051c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009fc0 tx=0x7ff920005830 comp rx=0 tx=0).stop 2024-04-25T08:29:48.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff939bd5640 1 -- 172.21.15.60:0/9359179 reap_dead start 2024-04-25T08:29:48.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff93b638640 1 -- 172.21.15.60:0/9359179 shutdown_connections 2024-04-25T08:29:48.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff93b638640 1 -- 172.21.15.60:0/9359179 >> 172.21.15.60:0/9359179 conn(0x7ff9340d0800 msgr2=0x7ff9340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:48.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff93b638640 1 -- 172.21.15.60:0/9359179 shutdown_connections 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.892+0000 7ff93b638640 1 -- 172.21.15.60:0/9359179 wait complete. 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.893+0000 7ff93b638640 1 Processor -- start 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 -- start start 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934151810 0x7ff9340cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 0x7ff93415fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934151bf0 0x7ff9341602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93415bd30 con 0x7ff9340cac70 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93415bbb0 con 0x7ff934151bf0 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff93b638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340b5ab0 con 0x7ff934151810 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff938bd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 0x7ff93415fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff938bd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 0x7ff93415fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41886/0 (socket says 172.21.15.60:41886) 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff938bd3640 1 -- 172.21.15.60:0/4096803461 learned_addr learned my addr 172.21.15.60:0/4096803461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff939bd5640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934151bf0 0x7ff9341602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.894+0000 7ff9393d4640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934151810 0x7ff9340cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff938bd3640 1 -- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934151810 msgr2=0x7ff9340cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff938bd3640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934151810 0x7ff9340cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff938bd3640 1 -- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934151bf0 msgr2=0x7ff9341602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff938bd3640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934151bf0 0x7ff9341602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff938bd3640 1 -- 172.21.15.60:0/4096803461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9340cb1a0 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff938bd3640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 0x7ff93415fdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff92400f770 tx=0x7ff92400fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400fe80 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff924018070 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9340cb480 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400d6d0 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.895+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9340cba40 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.896+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff934073a00 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.897+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff92400d870 con 0x7ff9340cac70 2024-04-25T08:29:48.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.897+0000 7ff929ffb640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff91806b500 0x7ff91806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:48.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.897+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff92408a3c0 con 0x7ff9340cac70 2024-04-25T08:29:48.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.897+0000 7ff9393d4640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff91806b500 0x7ff91806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:48.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.898+0000 7ff9393d4640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff91806b500 0x7ff91806d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff930001220 tx=0x7ff930001e40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:48.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:48.901+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff924058820 con 0x7ff9340cac70 2024-04-25T08:29:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.081+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9340c70f0 con 0x7ff9340cac70 2024-04-25T08:29:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.081+0000 7ff929ffb640 1 -- 172.21.15.60:0/4096803461 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff92405c8c0 con 0x7ff9340cac70 2024-04-25T08:29:49.079 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff91806b500 msgr2=0x7ff91806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff91806b500 0x7ff91806d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff930001220 tx=0x7ff930001e40 comp rx=0 tx=0).stop 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 msgr2=0x7ff93415fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 0x7ff93415fdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff92400f770 tx=0x7ff92400fc30 comp rx=0 tx=0).stop 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 shutdown_connections 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff91806b500 0x7ff91806d9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff934151bf0 0x7ff9341602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9340cac70 0x7ff93415fdb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 --2- 172.21.15.60:0/4096803461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff934151810 0x7ff9340cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 >> 172.21.15.60:0/4096803461 conn(0x7ff9340d0800 msgr2=0x7ff93406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 shutdown_connections 2024-04-25T08:29:49.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.083+0000 7ff93b638640 1 -- 172.21.15.60:0/4096803461 wait complete. 2024-04-25T08:29:49.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:29:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.199+0000 7f1ee498b640 1 Processor -- start 2024-04-25T08:29:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.199+0000 7f1ee498b640 1 -- start start 2024-04-25T08:29:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1ee498b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1ee015bb50 0x7f1ee00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1ee498b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1ee00c70d0 0x7f1ee00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1ee498b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee00c75e0 con 0x7f1ee00cfb40 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1ee498b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00c8680 con 0x7f1ee00c70d0 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1ee498b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee00c78c0 con 0x7f1ee015bb50 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1eddd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1ee00c70d0 0x7f1ee00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1eded9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1ee00cfb40 0x7f1ee0070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42156/0 (socket says 172.21.15.60:42156) 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.200+0000 7f1eded9d640 1 -- 172.21.15.60:0/1670472593 learned_addr learned my addr 172.21.15.60:0/1670472593 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1ede59c640 1 --2- 172.21.15.60:0/1670472593 >> v2:172.21.15.100:3300/0 conn(0x7f1ee015bb50 0x7f1ee00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1eddd9b640 1 -- 172.21.15.60:0/1670472593 >> v2:172.21.15.100:3300/0 conn(0x7f1ee015bb50 msgr2=0x7f1ee00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1eddd9b640 1 --2- 172.21.15.60:0/1670472593 >> v2:172.21.15.100:3300/0 conn(0x7f1ee015bb50 0x7f1ee00c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1eddd9b640 1 -- 172.21.15.60:0/1670472593 >> v1:172.21.15.60:6789/0 conn(0x7f1ee00cfb40 legacy=0x7f1ee0070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1eddd9b640 1 -- 172.21.15.60:0/1670472593 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00c7ba0 con 0x7f1ee00c70d0 2024-04-25T08:29:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1edd59a640 1 -- 172.21.15.60:0/1670472593 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840998631 0 0) 0x7f1ee00c75e0 con 0x7f1ee00cfb40 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.201+0000 7f1eddd9b640 1 --2- 172.21.15.60:0/1670472593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee00cf600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0070ab0 tx=0x7f1ed402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10867a1a975b2dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1edd59a640 1 -- 172.21.15.60:0/1670472593 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4037070 con 0x7f1ee00c70d0 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1edd59a640 1 -- 172.21.15.60:0/1670472593 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4032440 con 0x7f1ee00c70d0 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1edd59a640 1 -- 172.21.15.60:0/1670472593 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4032810 con 0x7f1ee00c70d0 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 -- 172.21.15.60:0/1670472593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 msgr2=0x7f1ee00cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 --2- 172.21.15.60:0/1670472593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee00cf600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0070ab0 tx=0x7f1ed402a230 comp rx=0 tx=0).stop 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 -- 172.21.15.60:0/1670472593 shutdown_connections 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 --2- 172.21.15.60:0/1670472593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee00cf600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 --2- 172.21.15.60:0/1670472593 >> v2:172.21.15.100:3300/0 conn(0x7f1ee015bb50 0x7f1ee00c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 -- 172.21.15.60:0/1670472593 >> 172.21.15.60:0/1670472593 conn(0x7f1ee006f500 msgr2=0x7f1ee006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 -- 172.21.15.60:0/1670472593 shutdown_connections 2024-04-25T08:29:49.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.202+0000 7f1ee498b640 1 -- 172.21.15.60:0/1670472593 wait complete. 2024-04-25T08:29:49.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 Processor -- start 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 -- start start 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee014a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee00cfb40 0x7f1ee015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee015bb50 0x7f1ee015c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee015f6d0 con 0x7f1ee00c70d0 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00c8160 con 0x7f1ee015bb50 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ee498b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee015f850 con 0x7f1ee00cfb40 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ede59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee014a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.204+0000 7f1ede59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee014a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41912/0 (socket says 172.21.15.60:41912) 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 -- 172.21.15.60:0/206442522 learned_addr learned my addr 172.21.15.60:0/206442522 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1eded9d640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee015bb50 0x7f1ee015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 -- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee00cfb40 msgr2=0x7f1ee015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee00cfb40 0x7f1ee015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 -- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee015bb50 msgr2=0x7f1ee015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee015bb50 0x7f1ee015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 -- 172.21.15.60:0/206442522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee014a810 con 0x7f1ee00c70d0 2024-04-25T08:29:49.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ede59c640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee014a2d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800e870 tx=0x7f1ed800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8017070 con 0x7f1ee00c70d0 2024-04-25T08:29:49.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800c940 con 0x7f1ee00c70d0 2024-04-25T08:29:49.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee006cc20 con 0x7f1ee00c70d0 2024-04-25T08:29:49.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800cd30 con 0x7f1ee00c70d0 2024-04-25T08:29:49.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.205+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee006d1e0 con 0x7f1ee00c70d0 2024-04-25T08:29:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.206+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee0073a00 con 0x7f1ee00c70d0 2024-04-25T08:29:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.207+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ed801a070 con 0x7f1ee00c70d0 2024-04-25T08:29:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.207+0000 7f1eceffd640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ec006b4d0 0x7f1ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.207+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1ed808a400 con 0x7f1ee00c70d0 2024-04-25T08:29:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.207+0000 7f1eddd9b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ec006b4d0 0x7f1ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.208+0000 7f1eddd9b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ec006b4d0 0x7f1ec006d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4034000 tx=0x7f1ed402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.212+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ee014a810 con 0x7f1ee00c70d0 2024-04-25T08:29:49.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.391+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ee00cf150 con 0x7f1ee00c70d0 2024-04-25T08:29:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.392+0000 7f1eceffd640 1 -- 172.21.15.60:0/206442522 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ed8058f40 con 0x7f1ee00c70d0 2024-04-25T08:29:49.390 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ec006b4d0 msgr2=0x7f1ec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ec006b4d0 0x7f1ec006d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4034000 tx=0x7f1ed402c000 comp rx=0 tx=0).stop 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 msgr2=0x7f1ee014a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee014a2d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800e870 tx=0x7f1ed800ed30 comp rx=0 tx=0).stop 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 shutdown_connections 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ec006b4d0 0x7f1ec006d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ee015bb50 0x7f1ee015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1ee00cfb40 0x7f1ee015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 --2- 172.21.15.60:0/206442522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1ee00c70d0 0x7f1ee014a2d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 >> 172.21.15.60:0/206442522 conn(0x7f1ee006f500 msgr2=0x7f1ee0159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 shutdown_connections 2024-04-25T08:29:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.394+0000 7f1ee498b640 1 -- 172.21.15.60:0/206442522 wait complete. 2024-04-25T08:29:49.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:29:49.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.508+0000 7feee0076640 1 Processor -- start 2024-04-25T08:29:49.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.508+0000 7feee0076640 1 -- start start 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feee0076640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 0x7feed80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feee0076640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed806c2d0 0x7feed806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feee0076640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed806cc80 con 0x7feed815b7d0 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feee0076640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed806cf50 con 0x7feed80b5620 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feee0076640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80ce530 con 0x7feed806c2d0 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feedd611640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 0x7feed80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feede613640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed806c2d0 0x7feed806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feedd611640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 0x7feed80b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:33738/0 (socket says 172.21.15.60:33738) 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feedd611640 1 -- 172.21.15.60:0/1446554756 learned_addr learned my addr 172.21.15.60:0/1446554756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feedde12640 1 --1- 172.21.15.60:0/1446554756 >> v1:172.21.15.100:6789/0 conn(0x7feed815b7d0 0x7feed80cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:43886/0 (socket says 172.21.15.60:43886) 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feede613640 1 -- 172.21.15.60:0/1446554756 >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 msgr2=0x7feed80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feede613640 1 --2- 172.21.15.60:0/1446554756 >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 0x7feed80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feede613640 1 -- 172.21.15.60:0/1446554756 >> v1:172.21.15.100:6789/0 conn(0x7feed815b7d0 legacy=0x7feed80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feede613640 1 -- 172.21.15.60:0/1446554756 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80ce810 con 0x7feed806c2d0 2024-04-25T08:29:49.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.509+0000 7feedd611640 1 --2- 172.21.15.60:0/1446554756 >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 0x7feed80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:49.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.510+0000 7feede613640 1 --2- 172.21.15.60:0/1446554756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed806c2d0 0x7feed806c740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feed0009d90 tx=0x7feed002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f106858404e978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.510+0000 7feedce10640 1 -- 172.21.15.60:0/1446554756 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0007740 con 0x7feed806c2d0 2024-04-25T08:29:49.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.510+0000 7feedce10640 1 -- 172.21.15.60:0/1446554756 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed00078e0 con 0x7feed806c2d0 2024-04-25T08:29:49.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.510+0000 7feedce10640 1 -- 172.21.15.60:0/1446554756 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0007cb0 con 0x7feed806c2d0 2024-04-25T08:29:49.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 -- 172.21.15.60:0/1446554756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed806c2d0 msgr2=0x7feed806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 --2- 172.21.15.60:0/1446554756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed806c2d0 0x7feed806c740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feed0009d90 tx=0x7feed002a660 comp rx=0 tx=0).stop 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 -- 172.21.15.60:0/1446554756 shutdown_connections 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 --2- 172.21.15.60:0/1446554756 >> v2:172.21.15.100:3300/0 conn(0x7feed80b5620 0x7feed80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 --2- 172.21.15.60:0/1446554756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed806c2d0 0x7feed806c740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 -- 172.21.15.60:0/1446554756 >> 172.21.15.60:0/1446554756 conn(0x7feed80d0800 msgr2=0x7feed80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 -- 172.21.15.60:0/1446554756 shutdown_connections 2024-04-25T08:29:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.511+0000 7feee0076640 1 -- 172.21.15.60:0/1446554756 wait complete. 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.512+0000 7feee0076640 1 Processor -- start 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.512+0000 7feee0076640 1 -- start start 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feee0076640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 0x7feed8157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feee0076640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed80b5620 0x7feed80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feee0076640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed815b7d0 0x7feed80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feee0076640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed815bbb0 con 0x7feed806c2d0 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feee0076640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed806d230 con 0x7feed80b5620 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feee0076640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed815bd30 con 0x7feed815b7d0 2024-04-25T08:29:49.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 0x7feed8157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 0x7feed8157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41926/0 (socket says 172.21.15.60:41926) 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 -- 172.21.15.60:0/1440177698 learned_addr learned my addr 172.21.15.60:0/1440177698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedd611640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed80b5620 0x7feed80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feede613640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed815b7d0 0x7feed80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 -- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed815b7d0 msgr2=0x7feed80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed815b7d0 0x7feed80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 -- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed80b5620 msgr2=0x7feed80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed80b5620 0x7feed80cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 -- 172.21.15.60:0/1440177698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80b4e50 con 0x7feed806c2d0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.513+0000 7feedde12640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 0x7feed8157cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feed400c3c0 tx=0x7feed400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.514+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40074e0 con 0x7feed806c2d0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.514+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4007680 con 0x7feed806c2d0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.514+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed80ce2b0 con 0x7feed806c2d0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.514+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4004a00 con 0x7feed806c2d0 2024-04-25T08:29:49.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.514+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed80b4890 con 0x7feed806c2d0 2024-04-25T08:29:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.515+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed8073a00 con 0x7feed806c2d0 2024-04-25T08:29:49.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.515+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feed401a020 con 0x7feed806c2d0 2024-04-25T08:29:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.516+0000 7feece7fc640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feebc06b4d0 0x7feebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.516+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feed4089ad0 con 0x7feed806c2d0 2024-04-25T08:29:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.516+0000 7feedd611640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feebc06b4d0 0x7feebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.516+0000 7feedd611640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feebc06b4d0 0x7feebc06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feed80ce500 tx=0x7feec8009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.520+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feed4058050 con 0x7feed806c2d0 2024-04-25T08:29:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.702+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feed80c8410 con 0x7feed806c2d0 2024-04-25T08:29:49.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.702+0000 7feece7fc640 1 -- 172.21.15.60:0/1440177698 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7feed405c0f0 con 0x7feed806c2d0 2024-04-25T08:29:49.700 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:49.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feebc06b4d0 msgr2=0x7feebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feebc06b4d0 0x7feebc06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feed80ce500 tx=0x7feec8009240 comp rx=0 tx=0).stop 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 msgr2=0x7feed8157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 0x7feed8157cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feed400c3c0 tx=0x7feed400c880 comp rx=0 tx=0).stop 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 shutdown_connections 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed815b7d0 0x7feed80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feebc06b4d0 0x7feebc06d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed80b5620 0x7feed80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 --2- 172.21.15.60:0/1440177698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed806c2d0 0x7feed8157cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 >> 172.21.15.60:0/1440177698 conn(0x7feed80d0800 msgr2=0x7feed815d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.704+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 shutdown_connections 2024-04-25T08:29:49.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.705+0000 7feee0076640 1 -- 172.21.15.60:0/1440177698 wait complete. 2024-04-25T08:29:49.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:49.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.816+0000 7f4bc4b9e640 1 Processor -- start 2024-04-25T08:29:49.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.816+0000 7f4bc4b9e640 1 -- start start 2024-04-25T08:29:49.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bc4b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 0x7f4bc00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bc4b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00b97a0 con 0x7f4bc015dbe0 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bc4b9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00d1c50 con 0x7f4bc00b2cb0 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bc4b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc015dfc0 con 0x7f4bc00d1020 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbe59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4bc015dbe0 0x7f4bc00d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42170/0 (socket says 172.21.15.60:42170) 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbe59c640 1 -- 172.21.15.60:0/535132385 learned_addr learned my addr 172.21.15.60:0/535132385 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbed9d640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 0x7f4bc00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616559443 0 0) 0x7f4bc00b97a0 con 0x7f4bc015dbe0 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bac003520 con 0x7f4bc015dbe0 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3264274826 0 0) 0x7f4bac003520 con 0x7f4bc015dbe0 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 msgr2=0x7f4bc00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbd59a640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 0x7f4bc00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.817+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 >> v1:172.21.15.100:6789/0 conn(0x7f4bc00b2cb0 legacy=0x7f4bc00d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00d1f30 con 0x7f4bc015dbe0 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f4bb8001670 con 0x7f4bc015dbe0 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 >> v1:172.21.15.60:6789/0 conn(0x7f4bc015dbe0 legacy=0x7f4bc00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac0039e0 0x7f4bac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bac004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 0x7f4bac0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00b9c0 con 0x7f4bac004390 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00bb70 con 0x7f4bac0039e0 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac00bd70 con 0x7f4bc00d1020 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bac004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbe59c640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac0039e0 0x7f4bac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbed9d640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 0x7f4bac0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbdd9b640 1 -- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 msgr2=0x7f4bac0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc00d1020 0x7f4bac0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbdd9b640 1 -- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac0039e0 msgr2=0x7f4bac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bac0039e0 0x7f4bac003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.818+0000 7f4bbdd9b640 1 -- 172.21.15.60:0/535132385 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac003520 con 0x7f4bac004390 2024-04-25T08:29:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bac004860 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8001830 tx=0x7f4bac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f51f824ffe0139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8037070 con 0x7f4bac004390 2024-04-25T08:29:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba802abe0 con 0x7f4bac004390 2024-04-25T08:29:49.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bbd59a640 1 -- 172.21.15.60:0/535132385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8001430 con 0x7f4bac004390 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/535132385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 msgr2=0x7f4bac004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/535132385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bac004860 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8001830 tx=0x7f4bac005830 comp rx=0 tx=0).stop 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.819+0000 7f4bbed9d640 1 -- 172.21.15.60:0/535132385 reap_dead start 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.820+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/535132385 shutdown_connections 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.820+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/535132385 >> 172.21.15.60:0/535132385 conn(0x7f4bc0061a20 msgr2=0x7f4bc0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.820+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/535132385 shutdown_connections 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.820+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/535132385 wait complete. 2024-04-25T08:29:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.821+0000 7f4bc4b9e640 1 Processor -- start 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.821+0000 7f4bc4b9e640 1 -- start start 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bc00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01556e0 0x7f4bc0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd70 0x7f4bc01601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b92a0 con 0x7f4bac004390 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00d1a00 con 0x7f4bc015fd70 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00b9420 con 0x7f4bc01556e0 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bc00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bc00c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41954/0 (socket says 172.21.15.60:41954) 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 -- 172.21.15.60:0/1031353209 learned_addr learned my addr 172.21.15.60:0/1031353209 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01556e0 0x7f4bc0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbed9d640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd70 0x7f4bc01601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 -- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01556e0 msgr2=0x7f4bc0155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01556e0 0x7f4bc0155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 -- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd70 msgr2=0x7f4bc01601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd70 0x7f4bc01601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 -- 172.21.15.60:0/1031353209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00cb630 con 0x7f4bac004390 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bbe59c640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bc00c9e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80072f0 tx=0x7f4bb80077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb801a070 con 0x7f4bac004390 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800f860 con 0x7f4bac004390 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00cb910 con 0x7f4bac004390 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800fc30 con 0x7f4bac004390 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.822+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00cbed0 con 0x7f4bac004390 2024-04-25T08:29:49.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.823+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0073990 con 0x7f4bac004390 2024-04-25T08:29:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.824+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bb801d070 con 0x7f4bac004390 2024-04-25T08:29:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.824+0000 7f4bb6ffd640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ba006b4d0 0x7f4ba006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.824+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4bb8089e40 con 0x7f4bac004390 2024-04-25T08:29:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.824+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ba006b4d0 0x7f4ba006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.825+0000 7f4bbdd9b640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ba006b4d0 0x7f4ba006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8034000 tx=0x7f4ba8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:49.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:49.829+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bb8058330 con 0x7f4bac004390 2024-04-25T08:29:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.044+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bc015d000 con 0x7f4bac004390 2024-04-25T08:29:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.044+0000 7f4bb6ffd640 1 -- 172.21.15.60:0/1031353209 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f4bb805c3d0 con 0x7f4bac004390 2024-04-25T08:29:50.042 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:50.043 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:29:50.043 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:29:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ba006b4d0 msgr2=0x7f4ba006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ba006b4d0 0x7f4ba006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8034000 tx=0x7f4ba8009030 comp rx=0 tx=0).stop 2024-04-25T08:29:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 msgr2=0x7f4bc00c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bc00c9e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80072f0 tx=0x7f4bb80077b0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 shutdown_connections 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ba006b4d0 0x7f4ba006d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4bc015fd70 0x7f4bc01601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4bc01556e0 0x7f4bc0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 --2- 172.21.15.60:0/1031353209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4bac004390 0x7f4bc00c9e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 >> 172.21.15.60:0/1031353209 conn(0x7f4bc0061a20 msgr2=0x7f4bc00b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 shutdown_connections 2024-04-25T08:29:50.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.046+0000 7f4bc4b9e640 1 -- 172.21.15.60:0/1031353209 wait complete. 2024-04-25T08:29:50.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:50.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.158+0000 7fa342ce4640 1 Processor -- start 2024-04-25T08:29:50.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.158+0000 7fa342ce4640 1 -- start start 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.159+0000 7fa342ce4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa33c06ce10 0x7fa33c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.159+0000 7fa342ce4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c06c220 con 0x7fa33c15dc70 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.159+0000 7fa342ce4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c0d0080 con 0x7fa33c09a450 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.159+0000 7fa342ce4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c0d0c90 con 0x7fa33c06ce10 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.159+0000 7fa340a80640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa33c15dc70 0x7fa33c06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42182/0 (socket says 172.21.15.60:42182) 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.159+0000 7fa340a80640 1 -- 172.21.15.60:0/57988724 learned_addr learned my addr 172.21.15.60:0/57988724 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33bfff640 1 --2- 172.21.15.60:0/57988724 >> v2:172.21.15.100:3300/0 conn(0x7fa33c06ce10 0x7fa33c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327166771 0 0) 0x7fa33c06c220 con 0x7fa33c15dc70 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa324003520 con 0x7fa33c15dc70 2024-04-25T08:29:50.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1159702472 0 0) 0x7fa324003520 con 0x7fa33c15dc70 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 >> v2:172.21.15.100:3300/0 conn(0x7fa33c06ce10 msgr2=0x7fa33c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 --2- 172.21.15.60:0/57988724 >> v2:172.21.15.100:3300/0 conn(0x7fa33c06ce10 0x7fa33c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 >> v1:172.21.15.100:6789/0 conn(0x7fa33c09a450 legacy=0x7fa33c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0d0360 con 0x7fa33c15dc70 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fa3340019f0 con 0x7fa33c15dc70 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 >> v1:172.21.15.60:6789/0 conn(0x7fa33c15dc70 legacy=0x7fa33c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240039b0 0x7fa324003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa324004800 0x7fa324004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3240051c0 0x7fa32400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d0c90 con 0x7fa3240051c0 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32400c060 con 0x7fa324004800 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32400c260 con 0x7fa3240039b0 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa341281640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3240051c0 0x7fa32400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa33bfff640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa324004800 0x7fa324004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa340a80640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240039b0 0x7fa324003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa341281640 1 -- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240039b0 msgr2=0x7fa324003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa341281640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240039b0 0x7fa324003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa341281640 1 -- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa324004800 msgr2=0x7fa324004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa341281640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa324004800 0x7fa324004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.160+0000 7fa341281640 1 -- 172.21.15.60:0/57988724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324003520 con 0x7fa3240051c0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.161+0000 7fa341281640 1 -- 172.21.15.60:0/57988724 reap_dead start 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.161+0000 7fa341281640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3240051c0 0x7fa32400b9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa32c001750 tx=0x7fa324005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd06209fc67b23ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.161+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c001370 con 0x7fa3240051c0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.161+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c005310 con 0x7fa3240051c0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.161+0000 7fa33b7fe640 1 -- 172.21.15.60:0/57988724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c02abe0 con 0x7fa3240051c0 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 -- 172.21.15.60:0/57988724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3240051c0 msgr2=0x7fa32400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3240051c0 0x7fa32400b9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa32c001750 tx=0x7fa324005830 comp rx=0 tx=0).stop 2024-04-25T08:29:50.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 -- 172.21.15.60:0/57988724 shutdown_connections 2024-04-25T08:29:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 --2- 172.21.15.60:0/57988724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3240051c0 0x7fa32400b9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 -- 172.21.15.60:0/57988724 >> 172.21.15.60:0/57988724 conn(0x7fa33c0b3270 msgr2=0x7fa33c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 -- 172.21.15.60:0/57988724 shutdown_connections 2024-04-25T08:29:50.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.162+0000 7fa342ce4640 1 -- 172.21.15.60:0/57988724 wait complete. 2024-04-25T08:29:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.163+0000 7fa342ce4640 1 Processor -- start 2024-04-25T08:29:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.163+0000 7fa342ce4640 1 -- start start 2024-04-25T08:29:50.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa342ce4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240051c0 0x7fa33c160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa342ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 0x7fa33c14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa342ce4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa33c157d10 0x7fa33c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa342ce4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c15e050 con 0x7fa33c1578d0 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa342ce4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d0920 con 0x7fa33c157d10 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa342ce4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0d1120 con 0x7fa3240051c0 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 0x7fa33c14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 0x7fa33c14abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41978/0 (socket says 172.21.15.60:41978) 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa33bfff640 1 -- 172.21.15.60:0/1462975303 learned_addr learned my addr 172.21.15.60:0/1462975303 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa340a80640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240051c0 0x7fa33c160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.164+0000 7fa341281640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa33c157d10 0x7fa33c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa33bfff640 1 -- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240051c0 msgr2=0x7fa33c160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa33bfff640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240051c0 0x7fa33c160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa33bfff640 1 -- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa33c157d10 msgr2=0x7fa33c14b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa33bfff640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa33c157d10 0x7fa33c14b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa33bfff640 1 -- 172.21.15.60:0/1462975303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c158430 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa33bfff640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 0x7fa33c14abd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa3300118e0 tx=0x7fa330011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33000fa30 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33000fbd0 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c158700 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330014440 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.165+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c15fe10 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.166+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c15fc30 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.166+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa330012040 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.167+0000 7fa3397fa640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa32806b4d0 0x7fa32806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.167+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa33008aa50 con 0x7fa33c1578d0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.167+0000 7fa340a80640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa32806b4d0 0x7fa32806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.168+0000 7fa340a80640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa32806b4d0 0x7fa32806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3340012e0 tx=0x7fa334006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:50.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.171+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa33c15fc30 con 0x7fa33c1578d0 2024-04-25T08:29:50.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.357+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa33c073990 con 0x7fa33c1578d0 2024-04-25T08:29:50.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.358+0000 7fa3397fa640 1 -- 172.21.15.60:0/1462975303 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fa330019030 con 0x7fa33c1578d0 2024-04-25T08:29:50.356 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:50.356 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa32806b4d0 msgr2=0x7fa32806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa32806b4d0 0x7fa32806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3340012e0 tx=0x7fa334006040 comp rx=0 tx=0).stop 2024-04-25T08:29:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 msgr2=0x7fa33c14abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 0x7fa33c14abd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa3300118e0 tx=0x7fa330011ce0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 shutdown_connections 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa32806b4d0 0x7fa32806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa33c157d10 0x7fa33c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa33c1578d0 0x7fa33c14abd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 --2- 172.21.15.60:0/1462975303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3240051c0 0x7fa33c160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 >> 172.21.15.60:0/1462975303 conn(0x7fa33c0b3270 msgr2=0x7fa33c0cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 shutdown_connections 2024-04-25T08:29:50.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.360+0000 7fa342ce4640 1 -- 172.21.15.60:0/1462975303 wait complete. 2024-04-25T08:29:50.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:29:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.472+0000 7fd824f89640 1 Processor -- start 2024-04-25T08:29:50.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.472+0000 7fd824f89640 1 -- start start 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd824f89640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd82015ef00 0x7fd8200d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd824f89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8200c7310 0x7fd8200cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd824f89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8200c7970 con 0x7fd8200d1ca0 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd824f89640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8200d2110 con 0x7fd8200c7310 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd824f89640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82015f2e0 con 0x7fd82015ef00 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd81ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8200c7310 0x7fd8200cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd81ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8200c7310 0x7fd8200cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41992/0 (socket says 172.21.15.60:41992) 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.473+0000 7fd81ed9d640 1 -- 172.21.15.60:0/2196661423 learned_addr learned my addr 172.21.15.60:0/2196661423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.474+0000 7fd81e59c640 1 --2- 172.21.15.60:0/2196661423 >> v2:172.21.15.100:3300/0 conn(0x7fd82015ef00 0x7fd8200d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.474+0000 7fd81ed9d640 1 -- 172.21.15.60:0/2196661423 >> v2:172.21.15.100:3300/0 conn(0x7fd82015ef00 msgr2=0x7fd8200d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:50.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.474+0000 7fd81ed9d640 1 --2- 172.21.15.60:0/2196661423 >> v2:172.21.15.100:3300/0 conn(0x7fd82015ef00 0x7fd8200d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.474+0000 7fd81ed9d640 1 -- 172.21.15.60:0/2196661423 >> v1:172.21.15.100:6789/0 conn(0x7fd8200d1ca0 legacy=0x7fd8200c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.474+0000 7fd81ed9d640 1 -- 172.21.15.60:0/2196661423 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200c7c50 con 0x7fd8200c7310 2024-04-25T08:29:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.474+0000 7fd81ed9d640 1 --2- 172.21.15.60:0/2196661423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd8200cc270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8200d2470 tx=0x7fd81802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afe7ff5090dc659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd81d59a640 1 -- 172.21.15.60:0/2196661423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818036070 con 0x7fd8200c7310 2024-04-25T08:29:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd81d59a640 1 -- 172.21.15.60:0/2196661423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818031600 con 0x7fd8200c7310 2024-04-25T08:29:50.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd81d59a640 1 -- 172.21.15.60:0/2196661423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8180319f0 con 0x7fd8200c7310 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 -- 172.21.15.60:0/2196661423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 msgr2=0x7fd8200cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 --2- 172.21.15.60:0/2196661423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd8200cc270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8200d2470 tx=0x7fd81802a7c0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 -- 172.21.15.60:0/2196661423 shutdown_connections 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 --2- 172.21.15.60:0/2196661423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd8200cc270 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 --2- 172.21.15.60:0/2196661423 >> v2:172.21.15.100:3300/0 conn(0x7fd82015ef00 0x7fd8200d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 -- 172.21.15.60:0/2196661423 >> 172.21.15.60:0/2196661423 conn(0x7fd82006f5d0 msgr2=0x7fd82006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 -- 172.21.15.60:0/2196661423 shutdown_connections 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.475+0000 7fd824f89640 1 -- 172.21.15.60:0/2196661423 wait complete. 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 Processor -- start 2024-04-25T08:29:50.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 -- start start 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd82013d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8200d1ca0 0x7fd8200cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82015ef00 0x7fd8200c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200c84f0 con 0x7fd8200c7310 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200c8210 con 0x7fd82015ef00 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd824f89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8200cc7b0 con 0x7fd8200d1ca0 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd81e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd82013d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd81e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd82013d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41998/0 (socket says 172.21.15.60:41998) 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.477+0000 7fd81e59c640 1 -- 172.21.15.60:0/4273659798 learned_addr learned my addr 172.21.15.60:0/4273659798 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:50.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81dd9b640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8200d1ca0 0x7fd8200cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81ed9d640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82015ef00 0x7fd8200c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81e59c640 1 -- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8200d1ca0 msgr2=0x7fd8200cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81e59c640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8200d1ca0 0x7fd8200cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81e59c640 1 -- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82015ef00 msgr2=0x7fd8200c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81e59c640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82015ef00 0x7fd8200c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81e59c640 1 -- 172.21.15.60:0/4273659798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8200b55d0 con 0x7fd8200c7310 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81dd9b640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8200d1ca0 0x7fd8200cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81ed9d640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82015ef00 0x7fd8200c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd81e59c640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd82013d890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd81000d5f0 tx=0x7fd81000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100125f0 con 0x7fd8200c7310 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810019070 con 0x7fd8200c7310 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8200b5840 con 0x7fd8200c7310 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100129d0 con 0x7fd8200c7310 2024-04-25T08:29:50.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.478+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82013ddd0 con 0x7fd8200c7310 2024-04-25T08:29:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.479+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd820073990 con 0x7fd8200c7310 2024-04-25T08:29:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.480+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd810012b70 con 0x7fd8200c7310 2024-04-25T08:29:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.480+0000 7fd80effd640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd80006b4d0 0x7fd80006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.480+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd81008a930 con 0x7fd8200c7310 2024-04-25T08:29:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.480+0000 7fd81dd9b640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd80006b4d0 0x7fd80006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:50.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.481+0000 7fd81dd9b640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd80006b4d0 0x7fd80006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd820143fa0 tx=0x7fd814009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:50.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.485+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd810059470 con 0x7fd8200c7310 2024-04-25T08:29:50.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.707+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd82006b9e0 con 0x7fd8200c7310 2024-04-25T08:29:50.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.708+0000 7fd80effd640 1 -- 172.21.15.60:0/4273659798 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7fd810058d70 con 0x7fd8200c7310 2024-04-25T08:29:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:50.706 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:29:50.706 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:29:50.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd80006b4d0 msgr2=0x7fd80006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd80006b4d0 0x7fd80006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd820143fa0 tx=0x7fd814009040 comp rx=0 tx=0).stop 2024-04-25T08:29:50.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 msgr2=0x7fd82013d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd82013d890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd81000d5f0 tx=0x7fd81000dab0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 shutdown_connections 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd80006b4d0 0x7fd80006d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd82015ef00 0x7fd8200c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8200d1ca0 0x7fd8200cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 --2- 172.21.15.60:0/4273659798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8200c7310 0x7fd82013d890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 >> 172.21.15.60:0/4273659798 conn(0x7fd82006f5d0 msgr2=0x7fd82015a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 shutdown_connections 2024-04-25T08:29:50.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:50.710+0000 7fd824f89640 1 -- 172.21.15.60:0/4273659798 wait complete. 2024-04-25T08:29:51.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:29:23.379232+0000', 'modified': '2024-04-25T08:29:39.468631+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6836/3799475286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 3799475286}, {'type': 'v1', 'addr': '172.21.15.100:6836', 'nonce': 3799475286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:29:51.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:29:51.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:29:51.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.230+0000 7fd8f7a1f640 1 Processor -- start 2024-04-25T08:29:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.230+0000 7fd8f7a1f640 1 -- start start 2024-04-25T08:29:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f7a1f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8f0161710 0x7fd8f00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f7a1f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f015e000 con 0x7fd8f015dc20 2024-04-25T08:29:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f7a1f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f00d1de0 con 0x7fd8f00ced80 2024-04-25T08:29:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f7a1f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f0161c60 con 0x7fd8f0161710 2024-04-25T08:29:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f5fbc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8f0161710 0x7fd8f00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f57bb640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd8f015dc20 0x7fd8f00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42190/0 (socket says 172.21.15.60:42190) 2024-04-25T08:29:51.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f57bb640 1 -- 172.21.15.60:0/3200952420 learned_addr learned my addr 172.21.15.60:0/3200952420 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/3200952420 >> v1:172.21.15.100:6789/0 conn(0x7fd8f00ced80 legacy=0x7fd8f00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/3200952420 >> v1:172.21.15.60:6789/0 conn(0x7fd8f015dc20 legacy=0x7fd8f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.231+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/3200952420 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0161f40 con 0x7fd8f0161710 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/3200952420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00d29f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009d90 tx=0x7fd8e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a83486f582fb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8e7fff640 1 -- 172.21.15.60:0/3200952420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8007740 con 0x7fd8f0161710 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8e7fff640 1 -- 172.21.15.60:0/3200952420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e80078e0 con 0x7fd8f0161710 2024-04-25T08:29:51.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8e7fff640 1 -- 172.21.15.60:0/3200952420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8007cb0 con 0x7fd8f0161710 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/3200952420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 msgr2=0x7fd8f00d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/3200952420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00d29f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009d90 tx=0x7fd8e802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/3200952420 shutdown_connections 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.232+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/3200952420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00d29f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.233+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/3200952420 >> 172.21.15.60:0/3200952420 conn(0x7fd8f00d0420 msgr2=0x7fd8f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.233+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/3200952420 shutdown_connections 2024-04-25T08:29:51.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.233+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/3200952420 wait complete. 2024-04-25T08:29:51.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 Processor -- start 2024-04-25T08:29:51.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 -- start start 2024-04-25T08:29:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f00ced80 0x7fd8f01e9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f015dc20 0x7fd8f00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b5e50 con 0x7fd8f0161710 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0162500 con 0x7fd8f00ced80 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.234+0000 7fd8f7a1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00b5fd0 con 0x7fd8f015dc20 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42026/0 (socket says 172.21.15.60:42026) 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/1192272644 learned_addr learned my addr 172.21.15.60:0/1192272644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f00ced80 0x7fd8f01e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f015dc20 0x7fd8f00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f015dc20 msgr2=0x7fd8f00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f015dc20 0x7fd8f00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f00ced80 msgr2=0x7fd8f01e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f00ced80 0x7fd8f01e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 -- 172.21.15.60:0/1192272644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00cb4f0 con 0x7fd8f0161710 2024-04-25T08:29:51.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00cafb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8e802ac90 tx=0x7fd8e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8e5ffb640 1 -- 172.21.15.60:0/1192272644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8040070 con 0x7fd8f0161710 2024-04-25T08:29:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8e5ffb640 1 -- 172.21.15.60:0/1192272644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8031c40 con 0x7fd8f0161710 2024-04-25T08:29:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f00cd990 con 0x7fd8f0161710 2024-04-25T08:29:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.235+0000 7fd8e5ffb640 1 -- 172.21.15.60:0/1192272644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8009450 con 0x7fd8f0161710 2024-04-25T08:29:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.236+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f00cdf50 con 0x7fd8f0161710 2024-04-25T08:29:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.237+0000 7fd8e5ffb640 1 -- 172.21.15.60:0/1192272644 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8e8039040 con 0x7fd8f0161710 2024-04-25T08:29:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.238+0000 7fd8e5ffb640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8d406b4d0 0x7fd8d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.238+0000 7fd8e5ffb640 1 -- 172.21.15.60:0/1192272644 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8e80aba60 con 0x7fd8f0161710 2024-04-25T08:29:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.238+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8d406b4d0 0x7fd8d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.238+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8d406b4d0 0x7fd8d406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec000f60 tx=0x7fd8ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 Processor -- start 2024-04-25T08:29:51.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 -- start start 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c321033610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321051020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321008f10 0x55c32104f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c320f6ba10 con 0x55c321033230 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c320f14b30 con 0x55c321008f10 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.239+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c320f67c50 con 0x55c321050ad0 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c321033610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c321033610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42038/0 (socket says 172.21.15.60:42038) 2024-04-25T08:29:51.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 -- 172.21.15.60:0/1160565528 learned_addr learned my addr 172.21.15.60:0/1160565528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321008f10 0x55c32104f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321051020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 -- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 msgr2=0x55c321051020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321051020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 -- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321008f10 msgr2=0x55c32104f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321008f10 0x55c32104f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 -- 172.21.15.60:0/1160565528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c320f29e10 con 0x55c321033230 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321051020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c321033610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec005a90 tx=0x7fd8ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884160b55764eaa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8daffd640 1 -- 172.21.15.60:0/1160565528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec007710 con 0x55c321033230 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8daffd640 1 -- 172.21.15.60:0/1160565528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00e740 con 0x55c321033230 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.240+0000 7fd8daffd640 1 -- 172.21.15.60:0/1160565528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00eaf0 con 0x55c321033230 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 -- 172.21.15.60:0/1160565528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 msgr2=0x55c321033610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c321033610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec005a90 tx=0x7fd8ec011000 comp rx=0 tx=0).stop 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 -- 172.21.15.60:0/1160565528 shutdown_connections 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321008f10 0x55c32104f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321051020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 --2- 172.21.15.60:0/1160565528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c321033610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 -- 172.21.15.60:0/1160565528 >> 172.21.15.60:0/1160565528 conn(0x55c321005840 msgr2=0x55c32104e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 -- 172.21.15.60:0/1160565528 shutdown_connections 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 -- 172.21.15.60:0/1160565528 wait complete. 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.241+0000 7fd8fa803740 1 -- start start 2024-04-25T08:29:51.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 Processor -- start 2024-04-25T08:29:51.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321033230 0x55c321035970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 0x55c321033ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c32102f140 0x55c3210368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321009cb0 con 0x55c321050ad0 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c320f280c0 con 0x55c321033230 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c321003770 con 0x55c32102f140 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.242+0000 7fd8fa803740 10 client.0 _ll_register_callbacks cb 0x55c320f1bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 0x55c321033ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 0x55c321033ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42050/0 (socket says 172.21.15.60:42050) 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 -- 172.21.15.60:0/3105417890 learned_addr learned my addr 172.21.15.60:0/3105417890 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321033230 0x55c321035970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c32102f140 0x55c3210368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 -- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c32102f140 msgr2=0x55c3210368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c32102f140 0x55c3210368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 -- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321033230 msgr2=0x55c321035970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321033230 0x55c321035970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 -- 172.21.15.60:0/3105417890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c320c802d0 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c32102f140 0x55c3210368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 0x55c321033ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000bf10 tx=0x7fd8e000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a8da7a1e9ef28e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8cffff640 1 -- 172.21.15.60:0/3105417890 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0018020 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8ec00e740 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8cffff640 1 -- 172.21.15.60:0/3105417890 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8e0014660 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8cffff640 1 -- 172.21.15.60:0/3105417890 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e0022050 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8fa803740 10 client.4881 fetch_fsmap learned FSMap version 8 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c320d4a6b0 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.243+0000 7fd8cffff640 1 -- 172.21.15.60:0/3105417890 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00e740 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8cffff640 1 -- 172.21.15.60:0/3105417890 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fd8e0025020 con 0x55c321050ad0 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8fa803740 10 client.4881 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8fa803740 10 client.4881 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8fa803740 20 client.4881 populate_metadata read hostname 'smithi060' 2024-04-25T08:29:51.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8fa803740 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c32102f890 0x55c32102fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8fa803740 4 client.4881 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 --> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c320fca920 con 0x55c32102f890 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c32102f890 0x55c32102fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8cffff640 10 client.4881 ms_handle_connect on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.244+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c32102f890 0x55c32102fc70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273f0065956ab965 server_cookie=99cfe841d9f55265 in_seq=0 out_seq=0 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8cffff640 1 -- 172.21.15.60:0/3105417890 <== mds.0 v2:172.21.15.100:6834/3799475286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c320fca920 con 0x55c32102f890 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8cffff640 10 client.4881 handle_command_reply: tid=0 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 client.4881 shutdown 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 msgr2=0x55c321033ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 0x55c321033ef0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000bf10 tx=0x7fd8e000f5b0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 shutdown_connections 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c32102f890 0x55c32102fc70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c32102f140 0x55c3210368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8cffff640 0 client.4881 ms_handle_reset on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321050ad0 0x55c321033ef0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 --2- 172.21.15.60:0/3105417890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321033230 0x55c321035970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 >> 172.21.15.60:0/3105417890 conn(0x55c321008f10 msgr2=0x55c3210515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 shutdown_connections 2024-04-25T08:29:51.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 1 -- 172.21.15.60:0/3105417890 wait complete. 2024-04-25T08:29:51.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.245+0000 7fd8fa803740 20 client.4881 trim_cache size 0 max 16384 2024-04-25T08:29:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.261+0000 7fd8fa803740 1 Processor -- start 2024-04-25T08:29:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.261+0000 7fd8fa803740 1 -- start start 2024-04-25T08:29:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 0x55c32104cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321033230 0x55c3210094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321050ad0 0x55c32102d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8fa803740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0204a60 con 0x55c32102f140 2024-04-25T08:29:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0203e60 con 0x55c321050ad0 2024-04-25T08:29:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0204460 con 0x55c321033230 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 0x55c32104cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 0x55c32104cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42054/0 (socket says 172.21.15.60:42054) 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 -- 172.21.15.60:0/2221427957 learned_addr learned my addr 172.21.15.60:0/2221427957 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321033230 0x55c3210094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321050ad0 0x55c32102d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 -- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321033230 msgr2=0x55c3210094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321033230 0x55c3210094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 -- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321050ad0 msgr2=0x55c32102d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321050ad0 0x55c32102d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f57bb640 1 -- 172.21.15.60:0/2221427957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c321034bb0 con 0x55c32102f140 2024-04-25T08:29:51.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.262+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321033230 0x55c3210094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321050ad0 0x55c32102d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 0x55c32104cc20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0074a0 tx=0x7fd8ec007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cd953d80cbf44b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8da7fc640 1 -- 172.21.15.60:0/2221427957 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec033070 con 0x55c32102f140 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8da7fc640 1 -- 172.21.15.60:0/2221427957 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec027040 con 0x55c32102f140 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8da7fc640 1 -- 172.21.15.60:0/2221427957 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00edf0 con 0x55c32102f140 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 -- 172.21.15.60:0/2221427957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 msgr2=0x55c32104cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 0x55c32104cc20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec0074a0 tx=0x7fd8ec007000 comp rx=0 tx=0).stop 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 -- 172.21.15.60:0/2221427957 shutdown_connections 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c321050ad0 0x55c32102d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321033230 0x55c3210094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 --2- 172.21.15.60:0/2221427957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c32102f140 0x55c32104cc20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 -- 172.21.15.60:0/2221427957 >> 172.21.15.60:0/2221427957 conn(0x55c321008f10 msgr2=0x55c321035a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 -- 172.21.15.60:0/2221427957 shutdown_connections 2024-04-25T08:29:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.263+0000 7fd8fa803740 1 -- 172.21.15.60:0/2221427957 wait complete. 2024-04-25T08:29:51.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.264+0000 7fd8fa803740 1 -- start start 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.264+0000 7fd8fa803740 1 Processor -- start 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c32102f140 0x55c32104dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c32102d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321002900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321034f30 con 0x55c321033230 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c321036e80 con 0x55c32102f140 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c32105f620 con 0x55c321050ad0 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8fa803740 10 client.0 _ll_register_callbacks cb 0x55c321003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c32102d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c32102d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42068/0 (socket says 172.21.15.60:42068) 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 -- 172.21.15.60:0/4016612350 learned_addr learned my addr 172.21.15.60:0/4016612350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c32102f140 0x55c32104dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f5fbc640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321002900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 -- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 msgr2=0x55c321002900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321002900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 -- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c32102f140 msgr2=0x55c32104dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c32102f140 0x55c32104dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 -- 172.21.15.60:0/4016612350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3210b9b40 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8f4fba640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c32102d170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0001eb0 tx=0x7fd8e00046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ac7a2e7c62dcdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.265+0000 7fd8cffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8cffff640 1 -- 172.21.15.60:0/4016612350 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0018040 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8cffff640 1 -- 172.21.15.60:0/4016612350 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8e0004240 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8ec027040 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8cffff640 1 -- 172.21.15.60:0/4016612350 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e0039050 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 10 client.4887 fetch_fsmap learned FSMap version 8 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c32104e900 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8cffff640 1 -- 172.21.15.60:0/4016612350 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec027040 con 0x55c321033230 2024-04-25T08:29:51.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8cffff640 1 -- 172.21.15.60:0/4016612350 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fd8e003c020 con 0x55c321033230 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 10 client.4887 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 10 client.4887 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 20 client.4887 populate_metadata read hostname 'smithi060' 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c321067df0 0x55c3210681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 4 client.4887 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 --> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c320f78f50 con 0x55c321067df0 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.266+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c321067df0 0x55c3210681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8f57bb640 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c321067df0 0x55c3210681d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f7c654701ca6689 server_cookie=8bda744836a8cfcf in_seq=0 out_seq=0 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8cffff640 10 client.4887 ms_handle_connect on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8cffff640 1 -- 172.21.15.60:0/4016612350 <== mds.0 v2:172.21.15.100:6834/3799475286 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55c320f78f50 con 0x55c321067df0 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8cffff640 10 client.4887 handle_command_reply: tid=0 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 client.4887 shutdown 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 msgr2=0x55c32102d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c32102d170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0001eb0 tx=0x7fd8e00046f0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 shutdown_connections 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:6834/3799475286,v1:172.21.15.100:6836/3799475286] conn(0x55c321067df0 0x55c3210681d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c321050ad0 0x55c321002900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c321033230 0x55c32102d170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 --2- 172.21.15.60:0/4016612350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c32102f140 0x55c32104dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8cffff640 0 client.4887 ms_handle_reset on v2:172.21.15.100:6834/3799475286 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 >> 172.21.15.60:0/4016612350 conn(0x55c321008f10 msgr2=0x55c3210094e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 shutdown_connections 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 1 -- 172.21.15.60:0/4016612350 wait complete. 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.267+0000 7fd8fa803740 20 client.4887 trim_cache size 0 max 16384 2024-04-25T08:29:51.271 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 4288, 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 27.237090677000001, 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 8, 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 25, 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 34.262399805000001 2024-04-25T08:29:51.272 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8d406b4d0 msgr2=0x7fd8d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8d406b4d0 0x7fd8d406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec000f60 tx=0x7fd8ec009040 comp rx=0 tx=0).stop 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 msgr2=0x7fd8f00cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00cafb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8e802ac90 tx=0x7fd8e8004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 shutdown_connections 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8f0161710 0x7fd8f00cafb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8f015dc20 0x7fd8f00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8d406b4d0 0x7fd8d406d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 --2- 172.21.15.60:0/1192272644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8f00ced80 0x7fd8f01e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 >> 172.21.15.60:0/1192272644 conn(0x7fd8f00d0420 msgr2=0x7fd8f00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 shutdown_connections 2024-04-25T08:29:51.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.268+0000 7fd8f7a1f640 1 -- 172.21.15.60:0/1192272644 wait complete. 2024-04-25T08:29:51.276 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T08:29:51.289 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T08:29:51.290 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T08:29:51.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.290 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:29:51.305 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T08:29:51.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:29:51.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-04-25T08:29:51.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.462 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:29:51.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-04-25T08:29:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-04-25T08:29:51.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:29:51.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.548 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T08:29:51.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-25T08:29:51.637 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-04-25T08:29:51.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:29:51.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.639 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T08:29:51.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:51.697 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-25T08:29:51.764 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-04-25T08:29:51.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:29:51.765 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T08:29:51.766 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:29:51.766 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:29:51.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:29:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.870+0000 7f39c1eb3640 1 Processor -- start 2024-04-25T08:29:51.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.870+0000 7f39c1eb3640 1 -- start start 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39c1eb3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39bc15dbe0 0x7f39bc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39c1eb3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc0b6220 con 0x7f39bc0b92a0 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39c1eb3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc0b97a0 con 0x7f39bc0b2cb0 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39c1eb3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc15dfc0 con 0x7f39bc15dbe0 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39bb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39bc15dbe0 0x7f39bc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39bbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f39bc0b92a0 0x7f39bc0b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60146/0 (socket says 172.21.15.60:60146) 2024-04-25T08:29:51.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.871+0000 7f39bbfff640 1 -- 172.21.15.60:0/2672041091 learned_addr learned my addr 172.21.15.60:0/2672041091 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39bb7fe640 1 -- 172.21.15.60:0/2672041091 >> v1:172.21.15.100:6789/0 conn(0x7f39bc0b2cb0 legacy=0x7f39bc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39bb7fe640 1 -- 172.21.15.60:0/2672041091 >> v1:172.21.15.60:6789/0 conn(0x7f39bc0b92a0 legacy=0x7f39bc0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39ba7fc640 1 -- 172.21.15.60:0/2672041091 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925732138 0 0) 0x7f39bc0b6220 con 0x7f39bc0b92a0 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39bb7fe640 1 -- 172.21.15.60:0/2672041091 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0d1c50 con 0x7f39bc15dbe0 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39bb7fe640 1 --2- 172.21.15.60:0/2672041091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0d0910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0097a0 tx=0x7f39ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5555882f7369fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39ba7fc640 1 -- 172.21.15.60:0/2672041091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac037070 con 0x7f39bc15dbe0 2024-04-25T08:29:51.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39ba7fc640 1 -- 172.21.15.60:0/2672041091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac032440 con 0x7f39bc15dbe0 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.872+0000 7f39ba7fc640 1 -- 172.21.15.60:0/2672041091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac032810 con 0x7f39bc15dbe0 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 -- 172.21.15.60:0/2672041091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc15dbe0 msgr2=0x7f39bc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/2672041091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0d0910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0097a0 tx=0x7f39ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 -- 172.21.15.60:0/2672041091 shutdown_connections 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/2672041091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0d0910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 -- 172.21.15.60:0/2672041091 >> 172.21.15.60:0/2672041091 conn(0x7f39bc061a20 msgr2=0x7f39bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 -- 172.21.15.60:0/2672041091 shutdown_connections 2024-04-25T08:29:51.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.873+0000 7f39c1eb3640 1 -- 172.21.15.60:0/2672041091 wait complete. 2024-04-25T08:29:51.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.874+0000 7f39c1eb3640 1 Processor -- start 2024-04-25T08:29:51.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.874+0000 7f39c1eb3640 1 -- start start 2024-04-25T08:29:51.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39c1eb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc0b2cb0 0x7f39bc151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39c1eb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 0x7f39bc0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39c1eb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39c1eb3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0d2210 con 0x7f39bc0b92a0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39c1eb3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0d1a00 con 0x7f39bc15dbe0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39c1eb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0b6ab0 con 0x7f39bc0b2cb0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39baffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 0x7f39bc0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39baffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 0x7f39bc0cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55382/0 (socket says 172.21.15.60:55382) 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39baffd640 1 -- 172.21.15.60:0/465160290 learned_addr learned my addr 172.21.15.60:0/465160290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39bbfff640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.875+0000 7f39bb7fe640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc0b2cb0 0x7f39bc151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39baffd640 1 -- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc0b2cb0 msgr2=0x7f39bc151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39baffd640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc0b2cb0 0x7f39bc151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39baffd640 1 -- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc15dbe0 msgr2=0x7f39bc0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39baffd640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39baffd640 1 -- 172.21.15.60:0/465160290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc15e2d0 con 0x7f39bc0b92a0 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39bb7fe640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc0b2cb0 0x7f39bc151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:51.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39baffd640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 0x7f39bc0cd1f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f39b000c400 tx=0x7f39b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0017070 con 0x7f39bc0b92a0 2024-04-25T08:29:51.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b00052d0 con 0x7f39bc0b92a0 2024-04-25T08:29:51.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc15e590 con 0x7f39bc0b92a0 2024-04-25T08:29:51.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0003830 con 0x7f39bc0b92a0 2024-04-25T08:29:51.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.876+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc15eb50 con 0x7f39bc0b92a0 2024-04-25T08:29:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.877+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc073990 con 0x7f39bc0b92a0 2024-04-25T08:29:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.877+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39b001a070 con 0x7f39bc0b92a0 2024-04-25T08:29:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.878+0000 7f39a3fff640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f39a806b500 0x7f39a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.878+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f39b0089830 con 0x7f39bc0b92a0 2024-04-25T08:29:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.878+0000 7f39bb7fe640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f39a806b500 0x7f39a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:51.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.879+0000 7f39bb7fe640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f39a806b500 0x7f39a806d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39ac034000 tx=0x7f39ac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:51.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:51.882+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39b0057d20 con 0x7f39bc0b92a0 2024-04-25T08:29:52.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.107+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39bc06b080 con 0x7f39bc0b92a0 2024-04-25T08:29:52.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.107+0000 7f39a3fff640 1 -- 172.21.15.60:0/465160290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f39b005bdc0 con 0x7f39bc0b92a0 2024-04-25T08:29:52.105 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f39a806b500 msgr2=0x7f39a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f39a806b500 0x7f39a806d9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39ac034000 tx=0x7f39ac02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 msgr2=0x7f39bc0cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 0x7f39bc0cd1f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f39b000c400 tx=0x7f39b000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 shutdown_connections 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f39a806b500 0x7f39a806d9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39bc15dbe0 0x7f39bc0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39bc0b92a0 0x7f39bc0cd1f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 --2- 172.21.15.60:0/465160290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39bc0b2cb0 0x7f39bc151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 >> 172.21.15.60:0/465160290 conn(0x7f39bc061a20 msgr2=0x7f39bc15a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:52.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 shutdown_connections 2024-04-25T08:29:52.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:52.110+0000 7f39c1eb3640 1 -- 172.21.15.60:0/465160290 wait complete. 2024-04-25T08:29:52.118 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:29:52.118 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:29:52.119 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:29:52.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.119 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:29:52.135 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42718sec preferred_lft 42718sec 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:29:52.136 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:29:52.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.136 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:52.136 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:52.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:29:52.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr flush dev ceph-brx 2024-04-25T08:29:52.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set ceph-brx up 2024-04-25T08:29:52.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:29:52.137 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:52.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.231 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:29:52.254 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:29:52.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.256 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:29:52.313 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:29:52.313 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:29:52.313 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:29:52.313 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:29:52.313 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:29:52.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.313 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:52.313 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:52.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:29:52.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:29:52.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:29:52.314 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:52.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.399 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:29:52.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.414 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-04-25T08:29:52.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.469 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:52.469 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:52.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:29:52.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T08:29:52.469 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:52.546 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T08:29:52.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.546 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T08:29:52.547 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:52.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.658 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:52.658 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:52.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:29:52.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:29:52.658 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:52.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:52.694 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:29:52.749 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:29:52.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:52.750 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:29:52.804 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:29:52.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:52.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:29:52.887 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:29:52.887 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:29:52.887 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:29:52.986 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:29:52.986 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:29:52.986 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:29:52.987 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:52.987 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:53.004 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:29:53.004 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:29:53.005 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:29:53.006 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:29:53.006 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:29:53.006 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:29:53.007 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:29:53.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:29:53.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.072 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:53.129 INFO:teuthology.orchestra.run.smithi060.stdout:client4899 2024-04-25T08:29:53.130 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:29:53.130 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:29:53.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.186 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:53.242 INFO:teuthology.orchestra.run.smithi060.stdout:client4899 2024-04-25T08:29:53.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.243 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:53.299 INFO:teuthology.orchestra.run.smithi060.stdout:client4899 2024-04-25T08:29:53.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.300 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de.client4899/status') 2024-04-25T08:29:53.363 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.4899 (3)192.168.0.1:0/1921347756 2024-04-25T08:29:53.363 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-04-25T08:29:53.364 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-04-25T08:29:53.364 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-04-25T08:29:53.364 INFO:teuthology.orchestra.run.smithi060.stderr:69 bytes copied, 2.5884e-05 s, 2.7 MB/s 2024-04-25T08:29:53.365 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T08:29:53.365 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:29:53.365 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:29:53.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.508+0000 7fdb78638640 1 Processor -- start 2024-04-25T08:29:53.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.508+0000 7fdb78638640 1 -- start start 2024-04-25T08:29:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb78638640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb700cedf0 0x7fdb701616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb78638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb700d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb78638640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7015e000 con 0x7fdb7015dc20 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb78638640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb700b6bb0 con 0x7fdb700cedf0 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb78638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700b6030 con 0x7fdb700b93a0 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb76bd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb700d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb75bd3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb700cedf0 0x7fdb701616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb763d4640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdb7015dc20 0x7fdb700ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:45974/0 (socket says 172.21.15.60:45974) 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.509+0000 7fdb763d4640 1 -- 172.21.15.60:0/1311113960 learned_addr learned my addr 172.21.15.60:0/1311113960 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb753d2640 1 -- 172.21.15.60:0/1311113960 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243154932 0 0) 0x7fdb6c001110 con 0x7fdb7015dc20 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb753d2640 1 -- 172.21.15.60:0/1311113960 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c003520 con 0x7fdb7015dc20 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb75bd3640 1 -- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 msgr2=0x7fdb700d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb75bd3640 1 --2- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb700d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb75bd3640 1 -- 172.21.15.60:0/1311113960 >> v1:172.21.15.100:6789/0 conn(0x7fdb7015dc20 legacy=0x7fdb700ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb75bd3640 1 -- 172.21.15.60:0/1311113960 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70161de0 con 0x7fdb700cedf0 2024-04-25T08:29:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb76bd5640 1 --2- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb700d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.510+0000 7fdb75bd3640 1 --2- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701616b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb600097a0 tx=0x7fdb6002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf87cc7e7b1cc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb753d2640 1 -- 172.21.15.60:0/1311113960 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60037070 con 0x7fdb700cedf0 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb753d2640 1 -- 172.21.15.60:0/1311113960 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60032440 con 0x7fdb700cedf0 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb753d2640 1 -- 172.21.15.60:0/1311113960 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60032810 con 0x7fdb700cedf0 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 -- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 msgr2=0x7fdb701616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 --2- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701616b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb600097a0 tx=0x7fdb6002a180 comp rx=0 tx=0).stop 2024-04-25T08:29:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 -- 172.21.15.60:0/1311113960 shutdown_connections 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 --2- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701616b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 --2- 172.21.15.60:0/1311113960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb700d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 -- 172.21.15.60:0/1311113960 >> 172.21.15.60:0/1311113960 conn(0x7fdb700d0490 msgr2=0x7fdb700d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 -- 172.21.15.60:0/1311113960 shutdown_connections 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.511+0000 7fdb78638640 1 -- 172.21.15.60:0/1311113960 wait complete. 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.512+0000 7fdb78638640 1 Processor -- start 2024-04-25T08:29:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 -- start start 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb70160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 0x7fdb7015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700d2a50 con 0x7fdb7015dc20 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb701623a0 con 0x7fdb700cedf0 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb78638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb700d2bd0 con 0x7fdb700b93a0 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb76bd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 0x7fdb7015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb76bd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 0x7fdb7015bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55386/0 (socket says 172.21.15.60:55386) 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.513+0000 7fdb76bd5640 1 -- 172.21.15.60:0/1340357698 learned_addr learned my addr 172.21.15.60:0/1340357698 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb75bd3640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb763d4640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb70160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:53.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb76bd5640 1 -- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 msgr2=0x7fdb70160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb76bd5640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb70160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb76bd5640 1 -- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 msgr2=0x7fdb701e83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb76bd5640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701e83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb76bd5640 1 -- 172.21.15.60:0/1340357698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb701e8ea0 con 0x7fdb7015dc20 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb76bd5640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 0x7fdb7015bfe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009f00 tx=0x7fdb6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68018030 con 0x7fdb7015dc20 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6800ce50 con 0x7fdb7015dc20 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb701e9180 con 0x7fdb7015dc20 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68003bb0 con 0x7fdb7015dc20 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.514+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb701e9a20 con 0x7fdb7015dc20 2024-04-25T08:29:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.515+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb70073990 con 0x7fdb7015dc20 2024-04-25T08:29:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.516+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb6801b070 con 0x7fdb7015dc20 2024-04-25T08:29:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.516+0000 7fdb66ffd640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb5406b4d0 0x7fdb5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.516+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdb6808afc0 con 0x7fdb7015dc20 2024-04-25T08:29:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.516+0000 7fdb763d4640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb5406b4d0 0x7fdb5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.517+0000 7fdb763d4640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb5406b4d0 0x7fdb5406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c005880 tx=0x7fdb6c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:53.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.520+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb68010040 con 0x7fdb7015dc20 2024-04-25T08:29:53.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.738+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb701e9bf0 con 0x7fdb7015dc20 2024-04-25T08:29:53.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.738+0000 7fdb66ffd640 1 -- 172.21.15.60:0/1340357698 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fdb68012340 con 0x7fdb7015dc20 2024-04-25T08:29:53.736 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:29:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb5406b4d0 msgr2=0x7fdb5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb5406b4d0 0x7fdb5406d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c005880 tx=0x7fdb6c009040 comp rx=0 tx=0).stop 2024-04-25T08:29:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 msgr2=0x7fdb7015bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 0x7fdb7015bfe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009f00 tx=0x7fdb6800c490 comp rx=0 tx=0).stop 2024-04-25T08:29:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 shutdown_connections 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb7015dc20 0x7fdb7015bfe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb5406b4d0 0x7fdb5406d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb700cedf0 0x7fdb701e83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 --2- 172.21.15.60:0/1340357698 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb700b93a0 0x7fdb70160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 >> 172.21.15.60:0/1340357698 conn(0x7fdb700d0490 msgr2=0x7fdb700ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 shutdown_connections 2024-04-25T08:29:53.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:53.740+0000 7fdb78638640 1 -- 172.21.15.60:0/1340357698 wait complete. 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:29:53.749 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:29:53.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.749 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:29:53.765 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42716sec preferred_lft 42716sec 2024-04-25T08:29:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link tentative 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link tentative 2024-04-25T08:29:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:53.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.768 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:29:53.828 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:29:53.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.830 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:29:53.886 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:29:53.887 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:29:53.887 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:29:53.887 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:29:53.887 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:29:53.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.887 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:53.887 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:53.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:29:53.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:29:53.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:29:53.888 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:53.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.972 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:29:53.987 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:29:53.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:53.988 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list-id 2024-04-25T08:29:54.042 INFO:teuthology.orchestra.run.smithi060.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T08:29:54.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.043 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:54.043 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:54.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:29:54.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T08:29:54.043 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:54.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:29:54.182 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:54.182 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 42:26:50:8a:37:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4026:50ff:fe8a:3789/64 scope link 2024-04-25T08:29:54.183 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:29:54.248 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:54.248 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:54.248 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 42:26:50:8a:37:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::4026:50ff:fe8a:3789/64 scope link 2024-04-25T08:29:54.249 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:54.250 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T08:29:54.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.250 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:54.250 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:54.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T08:29:54.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:29:54.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T08:29:54.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T08:29:54.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T08:29:54.251 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:54.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.360 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:29:54.360 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:29:54.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.1 up 2024-04-25T08:29:54.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:29:54.361 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:29:54.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:54.396 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T08:29:54.451 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:29:54.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:54.452 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:29:54.506 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:29:54.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:29:54.588 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:29:54.588 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T08:29:54.588 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:29:54.597 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:29:54.597 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T08:29:54.597 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:29:54.597 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:54.597 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:54.604 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:29:54.604 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T08:29:54.604 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:29:54.604 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:29:54.605 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:29:54.607 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:29:54.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:29:54.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.672 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:54.729 INFO:teuthology.orchestra.run.smithi060.stdout:client4905 2024-04-25T08:29:54.730 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:29:54.730 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:29:54.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.785 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:54.842 INFO:teuthology.orchestra.run.smithi060.stdout:client4905 2024-04-25T08:29:54.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.842 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:54.899 INFO:teuthology.orchestra.run.smithi060.stdout:client4905 2024-04-25T08:29:54.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:54.899 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de.client4905/status') 2024-04-25T08:29:54.963 INFO:teuthology.orchestra.run.smithi060.stdout:instance: client.4905 (3)192.168.0.2:0/1663657312 2024-04-25T08:29:54.963 INFO:teuthology.orchestra.run.smithi060.stdout:blocklisted: false 2024-04-25T08:29:54.964 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records in 2024-04-25T08:29:54.964 INFO:teuthology.orchestra.run.smithi060.stderr:0+1 records out 2024-04-25T08:29:54.964 INFO:teuthology.orchestra.run.smithi060.stderr:69 bytes copied, 2.376e-05 s, 2.9 MB/s 2024-04-25T08:29:54.965 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T08:29:54.965 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:29:54.965 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:29:54.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:29:55.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.108+0000 7f9f4dc1f640 1 Processor -- start 2024-04-25T08:29:55.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.108+0000 7f9f4dc1f640 1 -- start start 2024-04-25T08:29:55.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f4dc1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f480d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:55.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f4dc1f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f480d0fe0 0x7f9f480d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f4dc1f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4815e810 con 0x7f9f4815dd20 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f4dc1f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f480b9b30 con 0x7f9f480d0fe0 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f4dc1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480d1c00 con 0x7f9f480b96d0 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f47fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f480d0fe0 0x7f9f480d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f47fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f480d0fe0 0x7f9f480d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55388/0 (socket says 172.21.15.60:55388) 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f47fff640 1 -- 172.21.15.60:0/4141797125 learned_addr learned my addr 172.21.15.60:0/4141797125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.109+0000 7f9f46ffd640 1 --2- 172.21.15.60:0/4141797125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f480d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f47fff640 1 -- 172.21.15.60:0/4141797125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 msgr2=0x7f9f480d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f47fff640 1 --2- 172.21.15.60:0/4141797125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f480d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f47fff640 1 -- 172.21.15.60:0/4141797125 >> v1:172.21.15.100:6789/0 conn(0x7f9f4815dd20 legacy=0x7f9f4815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:55.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f47fff640 1 -- 172.21.15.60:0/4141797125 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4815eac0 con 0x7f9f480d0fe0 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f47fff640 1 --2- 172.21.15.60:0/4141797125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480d0fe0 0x7f9f480d2370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009e50 tx=0x7f9f4002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46bc4d6ace35f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f467fc640 1 -- 172.21.15.60:0/4141797125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40036070 con 0x7f9f480d0fe0 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f467fc640 1 -- 172.21.15.60:0/4141797125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40031600 con 0x7f9f480d0fe0 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.110+0000 7f9f467fc640 1 -- 172.21.15.60:0/4141797125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f400319d0 con 0x7f9f480d0fe0 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/4141797125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480d0fe0 msgr2=0x7f9f480d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/4141797125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480d0fe0 0x7f9f480d2370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009e50 tx=0x7f9f4002a680 comp rx=0 tx=0).stop 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/4141797125 shutdown_connections 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/4141797125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480d0fe0 0x7f9f480d2370 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/4141797125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f480d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/4141797125 >> 172.21.15.60:0/4141797125 conn(0x7f9f48061a20 msgr2=0x7f9f48061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/4141797125 shutdown_connections 2024-04-25T08:29:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.111+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/4141797125 wait complete. 2024-04-25T08:29:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.112+0000 7f9f4dc1f640 1 Processor -- start 2024-04-25T08:29:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.112+0000 7f9f4dc1f640 1 -- start start 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f4dc1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f481f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f4dc1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f480d0fe0 0x7f9f48158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f4dc1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 0x7f9f4815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f4dc1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480d1e50 con 0x7f9f4815dd20 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f4dc1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815f080 con 0x7f9f480d0fe0 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f4dc1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480d28b0 con 0x7f9f480b96d0 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 0x7f9f4815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 0x7f9f4815ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55402/0 (socket says 172.21.15.60:55402) 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 -- 172.21.15.60:0/1649283508 learned_addr learned my addr 172.21.15.60:0/1649283508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 -- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 msgr2=0x7f9f481f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:55.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f481f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 -- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f480d0fe0 msgr2=0x7f9f48158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f46ffd640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f480d0fe0 0x7f9f48158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.113+0000 7f9f47fff640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f480d0fe0 0x7f9f48158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f47fff640 1 -- 172.21.15.60:0/1649283508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48159ec0 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f47fff640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 0x7f9f4815ac00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009f70 tx=0x7f9f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4003f020 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40037040 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40036070 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f480cae30 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.114+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f481f3db0 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.115+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f48073990 con 0x7f9f4815dd20 2024-04-25T08:29:55.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.115+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f40042020 con 0x7f9f4815dd20 2024-04-25T08:29:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.116+0000 7f9f2ffff640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f3406b4d0 0x7f9f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.116+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9f400aa910 con 0x7f9f4815dd20 2024-04-25T08:29:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.116+0000 7f9f477fe640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f3406b4d0 0x7f9f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:55.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.116+0000 7f9f477fe640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f3406b4d0 0x7f9f3406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000f60 tx=0x7f9f380079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:55.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.120+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f40049030 con 0x7f9f4815dd20 2024-04-25T08:29:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.343+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9f480b8e50 con 0x7f9f4815dd20 2024-04-25T08:29:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.344+0000 7f9f2ffff640 1 -- 172.21.15.60:0/1649283508 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9f400794e0 con 0x7f9f4815dd20 2024-04-25T08:29:55.342 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f3406b4d0 msgr2=0x7f9f3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f3406b4d0 0x7f9f3406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000f60 tx=0x7f9f380079d0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 msgr2=0x7f9f4815ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 0x7f9f4815ac00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009f70 tx=0x7f9f40004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 shutdown_connections 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f4815dd20 0x7f9f4815ac00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f3406b4d0 0x7f9f3406d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f480d0fe0 0x7f9f48158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 --2- 172.21.15.60:0/1649283508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480b96d0 0x7f9f481f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 >> 172.21.15.60:0/1649283508 conn(0x7f9f48061a20 msgr2=0x7f9f480b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 shutdown_connections 2024-04-25T08:29:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:55.346+0000 7f9f4dc1f640 1 -- 172.21.15.60:0/1649283508 wait complete. 2024-04-25T08:29:55.354 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:29:55.354 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:29:55.355 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:29:55.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.355 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:55.374 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42715sec preferred_lft 42715sec 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::8f61:db9f:14f5:aa00/64 scope link noprefixroute 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:55.375 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:29:55.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:29:55.376 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:55.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:29:55.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:29:55.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.521 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:29:55.585 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-25T08:29:55.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.586 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-25T08:29:55.654 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-25T08:29:55.654 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:29:55.654 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:29:55.655 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:29:55.655 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:29:55.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.655 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:55.655 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:55.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:29:55.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:29:55.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:29:55.655 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:55.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.879 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T08:29:55.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.893 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-25T08:29:55.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:55.949 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:55.949 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:55.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:29:55.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T08:29:55.949 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:56.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T08:29:56.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.024 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:56.024 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:56.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T08:29:56.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:29:56.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T08:29:56.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T08:29:56.025 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T08:29:56.025 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:56.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.144 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:56.144 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:56.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-04-25T08:29:56.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:29:56.144 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:56.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:56.179 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T08:29:56.234 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:29:56.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:56.235 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:29:56.289 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:29:56.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:29:56.378 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:29:56.378 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T08:29:56.378 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-04-25T08:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T08:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-04-25T08:29:56.481 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:56.482 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:56.489 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:29:56.490 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T08:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:29:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:29:56.492 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:29:56.493 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:29:56.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.493 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:29:56.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.516 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:56.571 INFO:teuthology.orchestra.run.smithi100.stdout:client4914 2024-04-25T08:29:56.571 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:29:56.571 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:29:56.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.627 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:56.683 INFO:teuthology.orchestra.run.smithi100.stdout:client4914 2024-04-25T08:29:56.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.683 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:56.739 INFO:teuthology.orchestra.run.smithi100.stdout:client4914 2024-04-25T08:29:56.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:56.739 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de.client4914/status') 2024-04-25T08:29:56.802 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4914 (3)172.21.15.100:0/181246700 2024-04-25T08:29:56.802 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-04-25T08:29:56.803 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-04-25T08:29:56.803 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-04-25T08:29:56.803 INFO:teuthology.orchestra.run.smithi100.stderr:70 bytes copied, 3.0461e-05 s, 2.3 MB/s 2024-04-25T08:29:56.804 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T08:29:56.804 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:29:56.804 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:29:56.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:29:56.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.919+0000 7f24c85ed640 1 Processor -- start 2024-04-25T08:29:56.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.919+0000 7f24c85ed640 1 -- start start 2024-04-25T08:29:56.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.920+0000 7f24c85ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24c015ba00 0x7f24c009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.920+0000 7f24c85ed640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24c01602e0 0x7f24c00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.920+0000 7f24c85ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00cd710 con 0x7f24c015f5f0 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.920+0000 7f24c85ed640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c0099b70 con 0x7f24c01602e0 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.920+0000 7f24c85ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c0099e50 con 0x7f24c015ba00 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24c01602e0 0x7f24c00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f24c01602e0 0x7f24c00ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55408/0 (socket says 172.21.15.60:55408) 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 -- 172.21.15.60:0/4103024726 learned_addr learned my addr 172.21.15.60:0/4103024726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 -- 172.21.15.60:0/4103024726 >> v2:172.21.15.100:3300/0 conn(0x7f24c015ba00 msgr2=0x7f24c009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 --2- 172.21.15.60:0/4103024726 >> v2:172.21.15.100:3300/0 conn(0x7f24c015ba00 0x7f24c009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 -- 172.21.15.60:0/4103024726 >> v1:172.21.15.100:6789/0 conn(0x7f24c015f5f0 legacy=0x7f24c00c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:56.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.921+0000 7f24c5b88640 1 -- 172.21.15.60:0/4103024726 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c009a130 con 0x7f24c01602e0 2024-04-25T08:29:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c5b88640 1 --2- 172.21.15.60:0/4103024726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00ccda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24c00b4370 tx=0x7f24b402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57345b4d001412b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:56.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c5387640 1 -- 172.21.15.60:0/4103024726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4037070 con 0x7f24c01602e0 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c5387640 1 -- 172.21.15.60:0/4103024726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b402ae20 con 0x7f24c01602e0 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c5387640 1 -- 172.21.15.60:0/4103024726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4032660 con 0x7f24c01602e0 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 -- 172.21.15.60:0/4103024726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 msgr2=0x7f24c00ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 --2- 172.21.15.60:0/4103024726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00ccda0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f24c00b4370 tx=0x7f24b402a110 comp rx=0 tx=0).stop 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 -- 172.21.15.60:0/4103024726 shutdown_connections 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 --2- 172.21.15.60:0/4103024726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00ccda0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 --2- 172.21.15.60:0/4103024726 >> v2:172.21.15.100:3300/0 conn(0x7f24c015ba00 0x7f24c009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 -- 172.21.15.60:0/4103024726 >> 172.21.15.60:0/4103024726 conn(0x7f24c00b3270 msgr2=0x7f24c00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 -- 172.21.15.60:0/4103024726 shutdown_connections 2024-04-25T08:29:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.922+0000 7f24c85ed640 1 -- 172.21.15.60:0/4103024726 wait complete. 2024-04-25T08:29:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 Processor -- start 2024-04-25T08:29:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 -- start start 2024-04-25T08:29:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24c015ba00 0x7f24c0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:56.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c015f5f0 0x7f24c00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015bde0 con 0x7f24c01602e0 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00cd2e0 con 0x7f24c015ba00 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c85ed640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00b6610 con 0x7f24c015f5f0 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.924+0000 7f24c6b8a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55418/0 (socket says 172.21.15.60:55418) 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6389640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24c015ba00 0x7f24c0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 -- 172.21.15.60:0/1566241629 learned_addr learned my addr 172.21.15.60:0/1566241629 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c5b88640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c015f5f0 0x7f24c00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 -- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c015f5f0 msgr2=0x7f24c00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c015f5f0 0x7f24c00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 -- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24c015ba00 msgr2=0x7f24c0154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24c015ba00 0x7f24c0154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:56.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 -- 172.21.15.60:0/1566241629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00c9820 con 0x7f24c01602e0 2024-04-25T08:29:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c6b8a640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00b4b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b9c0 tx=0x7f24b000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0019070 con 0x7f24c01602e0 2024-04-25T08:29:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b000cba0 con 0x7f24c01602e0 2024-04-25T08:29:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c00b6840 con 0x7f24c01602e0 2024-04-25T08:29:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0014440 con 0x7f24c01602e0 2024-04-25T08:29:56.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.925+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c00b6ae0 con 0x7f24c01602e0 2024-04-25T08:29:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.926+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c0073990 con 0x7f24c01602e0 2024-04-25T08:29:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.927+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24b001c070 con 0x7f24c01602e0 2024-04-25T08:29:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.927+0000 7f24beffd640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ac06b4d0 0x7f24ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.927+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f24b008a210 con 0x7f24c01602e0 2024-04-25T08:29:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.927+0000 7f24c6389640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ac06b4d0 0x7f24ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:56.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.928+0000 7f24c6389640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ac06b4d0 0x7f24ac06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f24c015eca0 tx=0x7f24b8009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:56.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:56.932+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24b0058700 con 0x7f24c01602e0 2024-04-25T08:29:57.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.157+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24c00b40a0 con 0x7f24c01602e0 2024-04-25T08:29:57.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.158+0000 7f24beffd640 1 -- 172.21.15.60:0/1566241629 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f24b005c7a0 con 0x7f24c01602e0 2024-04-25T08:29:57.156 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:29:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ac06b4d0 msgr2=0x7f24ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ac06b4d0 0x7f24ac06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f24c015eca0 tx=0x7f24b8009060 comp rx=0 tx=0).stop 2024-04-25T08:29:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 msgr2=0x7f24c00b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00b4b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b9c0 tx=0x7f24b000be80 comp rx=0 tx=0).stop 2024-04-25T08:29:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 shutdown_connections 2024-04-25T08:29:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24c01602e0 0x7f24c00b4b40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24c015f5f0 0x7f24c00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ac06b4d0 0x7f24ac06d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 --2- 172.21.15.60:0/1566241629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24c015ba00 0x7f24c0154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 >> 172.21.15.60:0/1566241629 conn(0x7f24c00b3270 msgr2=0x7f24c00b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 shutdown_connections 2024-04-25T08:29:57.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:57.160+0000 7f24c85ed640 1 -- 172.21.15.60:0/1566241629 wait complete. 2024-04-25T08:29:57.169 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:29:57.169 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:29:57.169 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-04-25T08:29:57.169 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T08:29:57.169 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:29:57.169 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:29:57.170 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:29:57.170 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:29:57.170 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:29:57.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.170 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:57.186 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42713sec preferred_lft 42713sec 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::8f61:db9f:14f5:aa00/64 scope link noprefixroute 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 4e:30:82:48:a2:06 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.187 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::10ab:f6ff:fe04:40ae/64 scope link tentative 2024-04-25T08:29:57.188 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.188 INFO:teuthology.orchestra.run.smithi100.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:29:57.188 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 4e:30:82:48:a2:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:29:57.188 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::4c30:82ff:fe48:a206/64 scope link tentative 2024-04-25T08:29:57.188 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.188 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:29:57.249 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-04-25T08:29:57.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.250 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-04-25T08:29:57.306 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-04-25T08:29:57.306 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:29:57.306 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:29:57.306 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:29:57.306 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:29:57.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.307 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:57.307 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:57.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:29:57.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:29:57.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:29:57.307 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:57.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.391 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T08:29:57.446 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:29:57.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.447 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-04-25T08:29:57.501 INFO:teuthology.orchestra.run.smithi100.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T08:29:57.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.502 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:57.502 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:57.502 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:29:57.502 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T08:29:57.502 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:57.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:36:cb:90:fb:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.640 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::436:cbff:fe90:fbb1/64 scope link 2024-04-25T08:29:57.641 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:29:57.704 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:29:57.704 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 06:36:cb:90:fb:b1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::436:cbff:fe90:fbb1/64 scope link 2024-04-25T08:29:57.705 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:29:57.706 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T08:29:57.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.706 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:57.706 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:57.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T08:29:57.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:29:57.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T08:29:57.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T08:29:57.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T08:29:57.707 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:57.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.815 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec bash -c ' 2024-04-25T08:29:57.815 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-04-25T08:29:57.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.1 up 2024-04-25T08:29:57.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:29:57.815 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-04-25T08:29:57.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:57.849 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T08:29:57.907 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:29:57.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:29:57.907 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:29:57.962 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:29:57.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:57.962 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:29:58.043 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:29:58.043 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T08:29:58.043 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-04-25T08:29:58.051 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:29:58.052 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T08:29:58.052 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-04-25T08:29:58.052 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:58.052 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:58.056 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:29:58.056 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T08:29:58.056 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-04-25T08:29:58.056 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:29:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:29:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:29:58.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:29:58.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:58.059 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:29:58.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:58.122 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:58.178 INFO:teuthology.orchestra.run.smithi100.stdout:client4571 2024-04-25T08:29:58.178 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:29:58.179 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:29:58.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:58.234 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:58.289 INFO:teuthology.orchestra.run.smithi100.stdout:client4571 2024-04-25T08:29:58.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:58.290 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:29:58.345 INFO:teuthology.orchestra.run.smithi100.stdout:client4571 2024-04-25T08:29:58.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:29:58.345 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de.client4571/status') 2024-04-25T08:29:58.408 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4571 (3)192.168.0.2:0/617480869 2024-04-25T08:29:58.408 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-04-25T08:29:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-04-25T08:29:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-04-25T08:29:58.408 INFO:teuthology.orchestra.run.smithi100.stderr:68 bytes copied, 3.9534e-05 s, 1.7 MB/s 2024-04-25T08:29:58.410 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T08:29:58.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:29:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.532+0000 7fa962b9e640 1 Processor -- start 2024-04-25T08:29:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.532+0000 7fa962b9e640 1 -- start start 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa962b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c154ee0 0x7fa95c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa962b9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa95c0ce590 0x7fa95c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa962b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c071480 con 0x7fa95c0cfcf0 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa962b9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c072090 con 0x7fa95c0ce590 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa962b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0d0150 con 0x7fa95c154ee0 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa95bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa95c0cfcf0 0x7fa95c0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60148/0 (socket says 172.21.15.60:60148) 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa95bfff640 1 -- 172.21.15.60:0/2579176572 learned_addr learned my addr 172.21.15.60:0/2579176572 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96093a640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c154ee0 0x7fa95c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96113b640 1 --2- 172.21.15.60:0/2579176572 >> v2:172.21.15.100:3300/0 conn(0x7fa95c0ce590 0x7fa95c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa95b7fe640 1 -- 172.21.15.60:0/2579176572 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683699317 0 0) 0x7fa95c071480 con 0x7fa95c0cfcf0 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa95b7fe640 1 -- 172.21.15.60:0/2579176572 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa94c005730 con 0x7fa95c0cfcf0 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96113b640 1 -- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c154ee0 msgr2=0x7fa95c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96113b640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c154ee0 0x7fa95c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96113b640 1 -- 172.21.15.60:0/2579176572 >> v1:172.21.15.60:6789/0 conn(0x7fa95c0cfcf0 legacy=0x7fa95c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa95b7fe640 1 -- 172.21.15.60:0/2579176572 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3308265398 0 0) 0x7fa94c005730 con 0x7fa95c0cfcf0 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96113b640 1 -- 172.21.15.60:0/2579176572 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c071760 con 0x7fa95c0ce590 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.533+0000 7fa96093a640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c154ee0 0x7fa95c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.534+0000 7fa96113b640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c0ce590 0x7fa95c06d6f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa954009d90 tx=0x7fa95402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a582c8e6449e7b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.534+0000 7fa95b7fe640 1 -- 172.21.15.60:0/2579176572 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948001670 con 0x7fa95c0ce590 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.534+0000 7fa95b7fe640 1 -- 172.21.15.60:0/2579176572 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954007540 con 0x7fa95c0ce590 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.534+0000 7fa95b7fe640 1 -- 172.21.15.60:0/2579176572 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954007910 con 0x7fa95c0ce590 2024-04-25T08:29:58.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 -- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c0ce590 msgr2=0x7fa95c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c0ce590 0x7fa95c06d6f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa954009d90 tx=0x7fa95402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 -- 172.21.15.60:0/2579176572 shutdown_connections 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c0ce590 0x7fa95c06d6f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 --2- 172.21.15.60:0/2579176572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c154ee0 0x7fa95c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 -- 172.21.15.60:0/2579176572 >> 172.21.15.60:0/2579176572 conn(0x7fa95c06abe0 msgr2=0x7fa95c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 -- 172.21.15.60:0/2579176572 shutdown_connections 2024-04-25T08:29:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.535+0000 7fa962b9e640 1 -- 172.21.15.60:0/2579176572 wait complete. 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.536+0000 7fa962b9e640 1 Processor -- start 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.536+0000 7fa962b9e640 1 -- start start 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa962b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 0x7fa95c1473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa962b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c0cfcf0 0x7fa95c0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa962b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c154ee0 0x7fa95c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa962b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c071d20 con 0x7fa95c0ce590 2024-04-25T08:29:58.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa962b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c071230 con 0x7fa95c154ee0 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa962b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1552c0 con 0x7fa95c0cfcf0 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 0x7fa95c1473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 0x7fa95c1473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55426/0 (socket says 172.21.15.60:55426) 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 -- 172.21.15.60:0/1719041039 learned_addr learned my addr 172.21.15.60:0/1719041039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96113b640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c154ee0 0x7fa95c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa95bfff640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c0cfcf0 0x7fa95c0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 -- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c0cfcf0 msgr2=0x7fa95c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c0cfcf0 0x7fa95c0b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 -- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c154ee0 msgr2=0x7fa95c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c154ee0 0x7fa95c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96093a640 1 -- 172.21.15.60:0/1719041039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c1f4930 con 0x7fa95c0ce590 2024-04-25T08:29:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.537+0000 7fa96113b640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c154ee0 0x7fa95c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.538+0000 7fa96093a640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 0x7fa95c1473c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e960 tx=0x7fa95000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.538+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950017070 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.538+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000ca00 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.538+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c1f4c10 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.538+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000cdd0 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.538+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c1f51d0 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.539+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c073990 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.539+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa95001a070 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.540+0000 7fa9597fa640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa94006b500 0x7fa94006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.540+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa95008a3f0 con 0x7fa95c0ce590 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.540+0000 7fa95bfff640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa94006b500 0x7fa94006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.540+0000 7fa95bfff640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa94006b500 0x7fa94006d9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0cfc40 tx=0x7fa9480057f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:58.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.544+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa950058840 con 0x7fa95c0ce590 2024-04-25T08:29:58.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.728+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa95c06d360 con 0x7fa95c0ce590 2024-04-25T08:29:58.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.729+0000 7fa9597fa640 1 -- 172.21.15.60:0/1719041039 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fa95005c8e0 con 0x7fa95c0ce590 2024-04-25T08:29:58.727 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:29:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa94006b500 msgr2=0x7fa94006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa94006b500 0x7fa94006d9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0cfc40 tx=0x7fa9480057f0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 msgr2=0x7fa95c1473c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 0x7fa95c1473c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e960 tx=0x7fa95000ee20 comp rx=0 tx=0).stop 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 shutdown_connections 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa94006b500 0x7fa94006d9c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95c154ee0 0x7fa95c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa95c0cfcf0 0x7fa95c0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 --2- 172.21.15.60:0/1719041039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa95c0ce590 0x7fa95c1473c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 >> 172.21.15.60:0/1719041039 conn(0x7fa95c06abe0 msgr2=0x7fa95c15d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 shutdown_connections 2024-04-25T08:29:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.731+0000 7fa962b9e640 1 -- 172.21.15.60:0/1719041039 wait complete. 2024-04-25T08:29:58.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:29:58.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:29:58.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.846+0000 7fd6041c8640 1 Processor -- start 2024-04-25T08:29:58.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.846+0000 7fd6041c8640 1 -- start start 2024-04-25T08:29:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd6041c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5fc0ced80 0x7fd5fc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd6041c8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc15e000 con 0x7fd5fc15dc20 2024-04-25T08:29:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd6041c8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0d1de0 con 0x7fd5fc161710 2024-04-25T08:29:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd6041c8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc161c60 con 0x7fd5fc0ced80 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd601f64640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd5fc15dc20 0x7fd5fc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60152/0 (socket says 172.21.15.60:60152) 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd601f64640 1 -- 172.21.15.60:0/1816018061 learned_addr learned my addr 172.21.15.60:0/1816018061 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd601763640 1 --2- 172.21.15.60:0/1816018061 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc0ced80 0x7fd5fc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046197984 0 0) 0x7fd5fc15e000 con 0x7fd5fc15dc20 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e8003520 con 0x7fd5fc15dc20 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4010132266 0 0) 0x7fd5e8003520 con 0x7fd5fc15dc20 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc0ced80 msgr2=0x7fd5fc0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 --2- 172.21.15.60:0/1816018061 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc0ced80 0x7fd5fc0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 >> v1:172.21.15.100:6789/0 conn(0x7fd5fc161710 legacy=0x7fd5fc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc161f40 con 0x7fd5fc15dc20 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fd5f8001670 con 0x7fd5fc15dc20 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd601763640 1 --2- 172.21.15.60:0/1816018061 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc0ced80 0x7fd5fc0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:29:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 >> v1:172.21.15.60:6789/0 conn(0x7fd5fc15dc20 legacy=0x7fd5fc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e80039b0 0x7fd5e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e8004800 0x7fd5e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5e80051c0 0x7fd5e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc161c60 con 0x7fd5e8004800 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e800c060 con 0x7fd5e80039b0 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.847+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e800c260 con 0x7fd5e80051c0 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601763640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e8004800 0x7fd5e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601f64640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e80039b0 0x7fd5e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd602765640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5e80051c0 0x7fd5e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601763640 1 -- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5e80051c0 msgr2=0x7fd5e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601763640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5e80051c0 0x7fd5e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601763640 1 -- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e80039b0 msgr2=0x7fd5e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:58.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601763640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e80039b0 0x7fd5e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd602765640 1 -- 172.21.15.60:0/1816018061 reap_dead start 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.848+0000 7fd601763640 1 -- 172.21.15.60:0/1816018061 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8003520 con 0x7fd5e8004800 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd601763640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e8004800 0x7fd5e8004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009c90 tx=0x7fd5e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca7b9477c4d80fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec036070 con 0x7fd5e8004800 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec031400 con 0x7fd5e8004800 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd600f62640 1 -- 172.21.15.60:0/1816018061 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec009860 con 0x7fd5e8004800 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 -- 172.21.15.60:0/1816018061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e8004800 msgr2=0x7fd5e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e8004800 0x7fd5e8004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009c90 tx=0x7fd5e8005830 comp rx=0 tx=0).stop 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 -- 172.21.15.60:0/1816018061 shutdown_connections 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1816018061 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e8004800 0x7fd5e8004c80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 -- 172.21.15.60:0/1816018061 >> 172.21.15.60:0/1816018061 conn(0x7fd5fc0d0420 msgr2=0x7fd5fc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 -- 172.21.15.60:0/1816018061 shutdown_connections 2024-04-25T08:29:58.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.849+0000 7fd6041c8640 1 -- 172.21.15.60:0/1816018061 wait complete. 2024-04-25T08:29:58.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 Processor -- start 2024-04-25T08:29:58.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 -- start start 2024-04-25T08:29:58.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e8004800 0x7fd5fc0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0d2cd0 0x7fd5fc1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 0x7fd5fc146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b5e50 con 0x7fd5fc160920 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc162500 con 0x7fd5e8004800 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd6041c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b5fd0 con 0x7fd5fc0d2cd0 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd602765640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 0x7fd5fc146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd602765640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 0x7fd5fc146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55440/0 (socket says 172.21.15.60:55440) 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.851+0000 7fd602765640 1 -- 172.21.15.60:0/1251325550 learned_addr learned my addr 172.21.15.60:0/1251325550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd601763640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0d2cd0 0x7fd5fc1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd601f64640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e8004800 0x7fd5fc0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd602765640 1 -- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0d2cd0 msgr2=0x7fd5fc1603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd602765640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0d2cd0 0x7fd5fc1603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd602765640 1 -- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e8004800 msgr2=0x7fd5fc0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd602765640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e8004800 0x7fd5fc0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd602765640 1 -- 172.21.15.60:0/1251325550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc156f30 con 0x7fd5fc160920 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd601f64640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e8004800 0x7fd5fc0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:29:58.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd602765640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 0x7fd5fc146b60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400f770 tx=0x7fd5f400fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4003970 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4018070 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc159800 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4003d70 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.852+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc159dc0 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.853+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc073a00 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.854+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5f40052e0 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.854+0000 7fd5f27fc640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e006b500 0x7fd5e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.854+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd5f408a3d0 con 0x7fd5fc160920 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.854+0000 7fd601f64640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e006b500 0x7fd5e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.855+0000 7fd601f64640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e006b500 0x7fd5e006d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc068c30 tx=0x7fd5f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:58.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:58.858+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5f4058f10 con 0x7fd5fc160920 2024-04-25T08:29:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.037+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5fc0d16b0 con 0x7fd5fc160920 2024-04-25T08:29:59.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.037+0000 7fd5f27fc640 1 -- 172.21.15.60:0/1251325550 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5f4058810 con 0x7fd5fc160920 2024-04-25T08:29:59.035 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:29:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e006b500 msgr2=0x7fd5e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e006b500 0x7fd5e006d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc068c30 tx=0x7fd5f8009040 comp rx=0 tx=0).stop 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 msgr2=0x7fd5fc146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 0x7fd5fc146b60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400f770 tx=0x7fd5f400fc30 comp rx=0 tx=0).stop 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 shutdown_connections 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc160920 0x7fd5fc146b60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0d2cd0 0x7fd5fc1603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e006b500 0x7fd5e006d9c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 --2- 172.21.15.60:0/1251325550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e8004800 0x7fd5fc0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 >> 172.21.15.60:0/1251325550 conn(0x7fd5fc0d0420 msgr2=0x7fd5fc0b8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 shutdown_connections 2024-04-25T08:29:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.039+0000 7fd6041c8640 1 -- 172.21.15.60:0/1251325550 wait complete. 2024-04-25T08:29:59.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:29:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.154+0000 7fae55f34640 1 Processor -- start 2024-04-25T08:29:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.154+0000 7fae55f34640 1 -- start start 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae55f34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fae500ced80 0x7fae500d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae55f34640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fae50161710 0x7fae500d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae55f34640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5015e000 con 0x7fae5015dc20 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae55f34640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae500d1de0 con 0x7fae50161710 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae55f34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae50161c60 con 0x7fae500ced80 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fae50161710 0x7fae500d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fae5015dc20 0x7fae500ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60164/0 (socket says 172.21.15.60:60164) 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4f7fe640 1 -- 172.21.15.60:0/3174906656 learned_addr learned my addr 172.21.15.60:0/3174906656 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4effd640 1 --2- 172.21.15.60:0/3174906656 >> v2:172.21.15.100:3300/0 conn(0x7fae500ced80 0x7fae500d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4ffff640 1 -- 172.21.15.60:0/3174906656 >> v2:172.21.15.100:3300/0 conn(0x7fae500ced80 msgr2=0x7fae500d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4ffff640 1 --2- 172.21.15.60:0/3174906656 >> v2:172.21.15.100:3300/0 conn(0x7fae500ced80 0x7fae500d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4ffff640 1 -- 172.21.15.60:0/3174906656 >> v1:172.21.15.60:6789/0 conn(0x7fae5015dc20 legacy=0x7fae500ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4e7fc640 1 -- 172.21.15.60:0/3174906656 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165072693 0 0) 0x7fae5015e000 con 0x7fae5015dc20 2024-04-25T08:29:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4ffff640 1 -- 172.21.15.60:0/3174906656 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50161f40 con 0x7fae50161710 2024-04-25T08:29:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.155+0000 7fae4effd640 1 --2- 172.21.15.60:0/3174906656 >> v2:172.21.15.100:3300/0 conn(0x7fae500ced80 0x7fae500d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:59.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae4ffff640 1 --2- 172.21.15.60:0/3174906656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae500d2820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fae5006da90 tx=0x7fae4802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad98a5914ae2791 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae4e7fc640 1 -- 172.21.15.60:0/3174906656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48007740 con 0x7fae50161710 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae4e7fc640 1 -- 172.21.15.60:0/3174906656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae480078e0 con 0x7fae50161710 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae4e7fc640 1 -- 172.21.15.60:0/3174906656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48007cb0 con 0x7fae50161710 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 -- 172.21.15.60:0/3174906656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 msgr2=0x7fae500d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 --2- 172.21.15.60:0/3174906656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae500d2820 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fae5006da90 tx=0x7fae4802a680 comp rx=0 tx=0).stop 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 -- 172.21.15.60:0/3174906656 shutdown_connections 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 --2- 172.21.15.60:0/3174906656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae500d2820 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 --2- 172.21.15.60:0/3174906656 >> v2:172.21.15.100:3300/0 conn(0x7fae500ced80 0x7fae500d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 -- 172.21.15.60:0/3174906656 >> 172.21.15.60:0/3174906656 conn(0x7fae500d0420 msgr2=0x7fae500d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 -- 172.21.15.60:0/3174906656 shutdown_connections 2024-04-25T08:29:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.156+0000 7fae55f34640 1 -- 172.21.15.60:0/3174906656 wait complete. 2024-04-25T08:29:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.158+0000 7fae55f34640 1 Processor -- start 2024-04-25T08:29:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.158+0000 7fae55f34640 1 -- start start 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.158+0000 7fae55f34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae500ced80 0x7fae500c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.158+0000 7fae55f34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae5015dc20 0x7fae500c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.158+0000 7fae55f34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae501494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae55f34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae500b5e50 con 0x7fae50161710 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae55f34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae50162500 con 0x7fae500ced80 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae55f34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae500b5fd0 con 0x7fae5015dc20 2024-04-25T08:29:59.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae501494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae501494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55462/0 (socket says 172.21.15.60:55462) 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 -- 172.21.15.60:0/2065932638 learned_addr learned my addr 172.21.15.60:0/2065932638 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4f7fe640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae500ced80 0x7fae500c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4effd640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae5015dc20 0x7fae500c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 -- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae5015dc20 msgr2=0x7fae500c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae5015dc20 0x7fae500c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 -- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae500ced80 msgr2=0x7fae500c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae500ced80 0x7fae500c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 -- 172.21.15.60:0/2065932638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5014ca10 con 0x7fae50161710 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae4ffff640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae501494b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae4802ac70 tx=0x7fae48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4803f070 con 0x7fae50161710 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae48031bc0 con 0x7fae50161710 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5014ccf0 con 0x7fae50161710 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48009450 con 0x7fae50161710 2024-04-25T08:29:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.159+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae501e87b0 con 0x7fae50161710 2024-04-25T08:29:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.160+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae500cf380 con 0x7fae50161710 2024-04-25T08:29:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.161+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae4803b410 con 0x7fae50161710 2024-04-25T08:29:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.161+0000 7fae37fff640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae3c06b4d0 0x7fae3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.161+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae480abbf0 con 0x7fae50161710 2024-04-25T08:29:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.161+0000 7fae4f7fe640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae3c06b4d0 0x7fae3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.162+0000 7fae4f7fe640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae3c06b4d0 0x7fae3c06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae40007fd0 tx=0x7fae40007de0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.165+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae500cf380 con 0x7fae50161710 2024-04-25T08:29:59.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.347+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae501e89b0 con 0x7fae50161710 2024-04-25T08:29:59.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.348+0000 7fae37fff640 1 -- 172.21.15.60:0/2065932638 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae501e89b0 con 0x7fae50161710 2024-04-25T08:29:59.346 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae3c06b4d0 msgr2=0x7fae3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae3c06b4d0 0x7fae3c06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae40007fd0 tx=0x7fae40007de0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 msgr2=0x7fae501494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae501494b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae4802ac70 tx=0x7fae48004a40 comp rx=0 tx=0).stop 2024-04-25T08:29:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 shutdown_connections 2024-04-25T08:29:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae50161710 0x7fae501494b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae5015dc20 0x7fae500c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae3c06b4d0 0x7fae3c06d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 --2- 172.21.15.60:0/2065932638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae500ced80 0x7fae500c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 >> 172.21.15.60:0/2065932638 conn(0x7fae500d0420 msgr2=0x7fae500b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 shutdown_connections 2024-04-25T08:29:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.350+0000 7fae55f34640 1 -- 172.21.15.60:0/2065932638 wait complete. 2024-04-25T08:29:59.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:29:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.465+0000 7f1fa1553640 1 Processor -- start 2024-04-25T08:29:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.465+0000 7f1fa1553640 1 -- start start 2024-04-25T08:29:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1fa1553640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1f9c15dc20 0x7f1f9c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1fa1553640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c06cda0 con 0x7f1f9c161710 2024-04-25T08:29:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1fa1553640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c15e000 con 0x7f1f9c0d32f0 2024-04-25T08:29:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1fa1553640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c0d1de0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f93fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1f9c161710 0x7f1f9c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60180/0 (socket says 172.21.15.60:60180) 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f93fff640 1 -- 172.21.15.60:0/927087951 learned_addr learned my addr 172.21.15.60:0/927087951 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9affd640 1 --2- 172.21.15.60:0/927087951 >> v2:172.21.15.100:3300/0 conn(0x7f1f9c15dc20 0x7f1f9c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063697614 0 0) 0x7f1f9c06cda0 con 0x7f1f9c161710 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f88003520 con 0x7f1f9c161710 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052087771 0 0) 0x7f1f9c15e000 con 0x7f1f9c0d32f0 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f9c06cda0 con 0x7f1f9c0d32f0 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9affd640 1 -- 172.21.15.60:0/927087951 >> v1:172.21.15.100:6789/0 conn(0x7f1f9c0d32f0 legacy=0x7f1f9c06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9affd640 1 -- 172.21.15.60:0/927087951 >> v1:172.21.15.60:6789/0 conn(0x7f1f9c161710 legacy=0x7f1f9c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401468490 0 0) 0x7f1f88003520 con 0x7f1f9c161710 2024-04-25T08:29:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.466+0000 7f1f9affd640 1 -- 172.21.15.60:0/927087951 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c161c60 con 0x7f1f9c15dc20 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.467+0000 7f1f9affd640 1 --2- 172.21.15.60:0/927087951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c0ce670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0097a0 tx=0x7f1f8c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381e78fb50b86810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.467+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c037070 con 0x7f1f9c15dc20 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.467+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c02ac50 con 0x7f1f9c15dc20 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.467+0000 7f1f9a7fc640 1 -- 172.21.15.60:0/927087951 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84001670 con 0x7f1f9c15dc20 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 -- 172.21.15.60:0/927087951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c15dc20 msgr2=0x7f1f9c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 --2- 172.21.15.60:0/927087951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c0ce670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0097a0 tx=0x7f1f8c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 -- 172.21.15.60:0/927087951 shutdown_connections 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 --2- 172.21.15.60:0/927087951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c0ce670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 -- 172.21.15.60:0/927087951 >> 172.21.15.60:0/927087951 conn(0x7f1f9c0d0420 msgr2=0x7f1f9c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 -- 172.21.15.60:0/927087951 shutdown_connections 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.468+0000 7f1fa1553640 1 -- 172.21.15.60:0/927087951 wait complete. 2024-04-25T08:29:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.469+0000 7f1fa1553640 1 Processor -- start 2024-04-25T08:29:59.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.469+0000 7f1fa1553640 1 -- start start 2024-04-25T08:29:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1fa1553640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9c0d32f0 0x7f1f9c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1fa1553640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1fa1553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c161710 0x7f1f9c0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1fa1553640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c162500 con 0x7f1f9c15dc20 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1fa1553640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c162220 con 0x7f1f9c161710 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1fa1553640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0b5e50 con 0x7f1f9c0d32f0 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c15c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55478/0 (socket says 172.21.15.60:55478) 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 -- 172.21.15.60:0/730173791 learned_addr learned my addr 172.21.15.60:0/730173791 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f9affd640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9c0d32f0 0x7f1f9c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f9b7fe640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c161710 0x7f1f9c0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 -- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9c0d32f0 msgr2=0x7f1f9c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9c0d32f0 0x7f1f9c160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 -- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c161710 msgr2=0x7f1f9c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c161710 0x7f1f9c0b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f93fff640 1 -- 172.21.15.60:0/730173791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0aa6f0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f9affd640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9c0d32f0 0x7f1f9c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.470+0000 7f1f9b7fe640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c161710 0x7f1f9c0b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.471+0000 7f1f93fff640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c15c480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400c5a0 tx=0x7f1f8400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.471+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84017070 con 0x7f1f9c15dc20 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.471+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f84005ce0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.471+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c0aa9d0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.471+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840072b0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.471+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c0aaf90 con 0x7f1f9c15dc20 2024-04-25T08:29:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.472+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c073a00 con 0x7f1f9c15dc20 2024-04-25T08:29:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.472+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f8401a070 con 0x7f1f9c15dc20 2024-04-25T08:29:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.473+0000 7f1f937fe640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f7c06b4d0 0x7f1f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.473+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1f840899f0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.473+0000 7f1f9affd640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f7c06b4d0 0x7f1f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.474+0000 7f1f9affd640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f7c06b4d0 0x7f1f7c06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c034000 tx=0x7f1f8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.477+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f840585c0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.657+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f9c0d16b0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.657+0000 7f1f937fe640 1 -- 172.21.15.60:0/730173791 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f84057ec0 con 0x7f1f9c15dc20 2024-04-25T08:29:59.655 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:29:59.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f7c06b4d0 msgr2=0x7f1f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f7c06b4d0 0x7f1f7c06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c034000 tx=0x7f1f8c02e000 comp rx=0 tx=0).stop 2024-04-25T08:29:59.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 msgr2=0x7f1f9c15c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c15c480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400c5a0 tx=0x7f1f8400ca60 comp rx=0 tx=0).stop 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 shutdown_connections 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f7c06b4d0 0x7f1f7c06d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f9c161710 0x7f1f9c0b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f9c15dc20 0x7f1f9c15c480 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 --2- 172.21.15.60:0/730173791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f9c0d32f0 0x7f1f9c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 >> 172.21.15.60:0/730173791 conn(0x7f1f9c0d0420 msgr2=0x7f1f9c0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 shutdown_connections 2024-04-25T08:29:59.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.659+0000 7f1fa1553640 1 -- 172.21.15.60:0/730173791 wait complete. 2024-04-25T08:29:59.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:29:59.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.773+0000 7ffaf769a640 1 Processor -- start 2024-04-25T08:29:59.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.773+0000 7ffaf769a640 1 -- start start 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaf769a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffaf00ce230 0x7ffaf00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaf769a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf0072280 con 0x7ffaf00cc4f0 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaf769a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf00714c0 con 0x7ffaf015b270 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaf769a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf0071270 con 0x7ffaf00ce230 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaf5436640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffaf00cc4f0 0x7ffaf00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60196/0 (socket says 172.21.15.60:60196) 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaf5436640 1 -- 172.21.15.60:0/3745281830 learned_addr learned my addr 172.21.15.60:0/3745281830 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804899753 0 0) 0x7ffaf0072280 con 0x7ffaf00cc4f0 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.774+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad8003520 con 0x7ffaf00cc4f0 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf4c35640 1 --2- 172.21.15.60:0/3745281830 >> v2:172.21.15.100:3300/0 conn(0x7ffaf00ce230 0x7ffaf00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1853068888 0 0) 0x7ffad8003520 con 0x7ffaf00cc4f0 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 >> v2:172.21.15.100:3300/0 conn(0x7ffaf00ce230 msgr2=0x7ffaf00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 --2- 172.21.15.60:0/3745281830 >> v2:172.21.15.100:3300/0 conn(0x7ffaf00ce230 0x7ffaf00cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 >> v1:172.21.15.100:6789/0 conn(0x7ffaf015b270 legacy=0x7ffaf015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00717a0 con 0x7ffaf00cc4f0 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ffae8001670 con 0x7ffaf00cc4f0 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 >> v1:172.21.15.60:6789/0 conn(0x7ffaf00cc4f0 legacy=0x7ffaf00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffad80039b0 0x7ffad8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8004800 0x7ffad8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffad80051c0 0x7ffad800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0071270 con 0x7ffad80051c0 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad800c110 con 0x7ffad8004800 2024-04-25T08:29:59.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad800c310 con 0x7ffad80039b0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffad80051c0 0x7ffad800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5436640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffad80039b0 0x7ffad8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf4c35640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8004800 0x7ffad8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5c37640 1 -- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffad80039b0 msgr2=0x7ffad8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffad80039b0 0x7ffad8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5c37640 1 -- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8004800 msgr2=0x7ffad8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8004800 0x7ffad8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.775+0000 7ffaf5c37640 1 -- 172.21.15.60:0/3745281830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad8003520 con 0x7ffad80051c0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.776+0000 7ffaf5c37640 1 -- 172.21.15.60:0/3745281830 reap_dead start 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.776+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffad80051c0 0x7ffad800b9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001e50 tx=0x7ffad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddacc9cb7f4f3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.776+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0001a70 con 0x7ffad80051c0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.776+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae0003970 con 0x7ffad80051c0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.776+0000 7ffaeffff640 1 -- 172.21.15.60:0/3745281830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0003d60 con 0x7ffad80051c0 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 -- 172.21.15.60:0/3745281830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffad80051c0 msgr2=0x7ffad800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffad80051c0 0x7ffad800b9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001e50 tx=0x7ffad8005830 comp rx=0 tx=0).stop 2024-04-25T08:29:59.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 -- 172.21.15.60:0/3745281830 shutdown_connections 2024-04-25T08:29:59.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 --2- 172.21.15.60:0/3745281830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffad80051c0 0x7ffad800b9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 -- 172.21.15.60:0/3745281830 >> 172.21.15.60:0/3745281830 conn(0x7ffaf006abe0 msgr2=0x7ffaf006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 -- 172.21.15.60:0/3745281830 shutdown_connections 2024-04-25T08:29:59.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.777+0000 7ffaf769a640 1 -- 172.21.15.60:0/3745281830 wait complete. 2024-04-25T08:29:59.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.778+0000 7ffaf769a640 1 Processor -- start 2024-04-25T08:29:59.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.778+0000 7ffaf769a640 1 -- start start 2024-04-25T08:29:59.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80051c0 0x7ffaf01471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaf01f5550 0x7ffaf0154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 0x7ffaf015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00cc8d0 con 0x7ffaf01f1f50 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0071d60 con 0x7ffad80051c0 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00cca50 con 0x7ffaf01f5550 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 0x7ffaf015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 0x7ffaf015bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55496/0 (socket says 172.21.15.60:55496) 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 -- 172.21.15.60:0/1663705927 learned_addr learned my addr 172.21.15.60:0/1663705927 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5436640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80051c0 0x7ffaf01471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf4c35640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaf01f5550 0x7ffaf0154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 -- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaf01f5550 msgr2=0x7ffaf0154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaf01f5550 0x7ffaf0154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 -- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80051c0 msgr2=0x7ffaf01471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80051c0 0x7ffaf01471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 -- 172.21.15.60:0/1663705927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf01476f0 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf5c37640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 0x7ffaf015bfe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001f70 tx=0x7ffae00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0001a70 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae0006b50 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf0154b60 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.779+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0007030 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.780+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf01f4d50 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.781+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf0073a00 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.781+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffae00071d0 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.781+0000 7ffaedffb640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffadc06b500 0x7ffadc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.781+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffae00aa0d0 con 0x7ffaf01f1f50 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.782+0000 7ffaf5436640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffadc06b500 0x7ffadc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:29:59.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.782+0000 7ffaf5436640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffadc06b500 0x7ffadc06d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffae80017b0 tx=0x7ffae8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:29:59.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.786+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffae0043060 con 0x7ffaf01f1f50 2024-04-25T08:29:59.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.965+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffaf00b4020 con 0x7ffaf01f1f50 2024-04-25T08:29:59.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.966+0000 7ffaedffb640 1 -- 172.21.15.60:0/1663705927 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffaf01476f0 con 0x7ffaf01f1f50 2024-04-25T08:29:59.963 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffadc06b500 msgr2=0x7ffadc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffadc06b500 0x7ffadc06d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffae80017b0 tx=0x7ffae8009040 comp rx=0 tx=0).stop 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 msgr2=0x7ffaf015bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 0x7ffaf015bfe0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001f70 tx=0x7ffae00014a0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 shutdown_connections 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffaf01f1f50 0x7ffaf015bfe0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffaf01f5550 0x7ffaf0154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffadc06b500 0x7ffadc06d9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 --2- 172.21.15.60:0/1663705927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad80051c0 0x7ffaf01471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:29:59.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 >> 172.21.15.60:0/1663705927 conn(0x7ffaf006abe0 msgr2=0x7ffaf00cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:29:59.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 shutdown_connections 2024-04-25T08:29:59.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:29:59.968+0000 7ffaf769a640 1 -- 172.21.15.60:0/1663705927 wait complete. 2024-04-25T08:29:59.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:00.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.082+0000 7f6d555f9640 1 Processor -- start 2024-04-25T08:30:00.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.082+0000 7f6d555f9640 1 -- start start 2024-04-25T08:30:00.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d555f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d500ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d555f9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6d500d0b10 0x7f6d500d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d555f9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d500cec00 con 0x7f6d500b5f00 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d555f9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d500ce930 con 0x7f6d500d0b10 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d555f9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500b6450 con 0x7f6d5015db60 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d4e7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6d500b5f00 0x7f6d500d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60202/0 (socket says 172.21.15.60:60202) 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.083+0000 7f6d4e7fc640 1 -- 172.21.15.60:0/4248569624 learned_addr learned my addr 172.21.15.60:0/4248569624 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4effd640 1 --2- 172.21.15.60:0/4248569624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d500ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4f7fe640 1 --2- 172.21.15.60:0/4248569624 >> v2:172.21.15.100:3300/0 conn(0x7f6d500d0b10 0x7f6d500d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4effd640 1 -- 172.21.15.60:0/4248569624 >> v2:172.21.15.100:3300/0 conn(0x7f6d500d0b10 msgr2=0x7f6d500d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4dffb640 1 -- 172.21.15.60:0/4248569624 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256076966 0 0) 0x7f6d500cec00 con 0x7f6d500b5f00 2024-04-25T08:30:00.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4effd640 1 --2- 172.21.15.60:0/4248569624 >> v2:172.21.15.100:3300/0 conn(0x7f6d500d0b10 0x7f6d500d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4effd640 1 -- 172.21.15.60:0/4248569624 >> v1:172.21.15.60:6789/0 conn(0x7f6d500b5f00 legacy=0x7f6d500d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.084+0000 7f6d4effd640 1 -- 172.21.15.60:0/4248569624 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500ceee0 con 0x7f6d5015db60 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.085+0000 7f6d4effd640 1 --2- 172.21.15.60:0/4248569624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d500ce3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d400097a0 tx=0x7f6d4002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e67b569ebcfde3de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.085+0000 7f6d4dffb640 1 -- 172.21.15.60:0/4248569624 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40037070 con 0x7f6d5015db60 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.085+0000 7f6d4dffb640 1 -- 172.21.15.60:0/4248569624 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4002ae50 con 0x7f6d5015db60 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.085+0000 7f6d4dffb640 1 -- 172.21.15.60:0/4248569624 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40032660 con 0x7f6d5015db60 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 -- 172.21.15.60:0/4248569624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 msgr2=0x7f6d500ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 --2- 172.21.15.60:0/4248569624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d500ce3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d400097a0 tx=0x7f6d4002a100 comp rx=0 tx=0).stop 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 -- 172.21.15.60:0/4248569624 shutdown_connections 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 --2- 172.21.15.60:0/4248569624 >> v2:172.21.15.100:3300/0 conn(0x7f6d500d0b10 0x7f6d500d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 --2- 172.21.15.60:0/4248569624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d500ce3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 -- 172.21.15.60:0/4248569624 >> 172.21.15.60:0/4248569624 conn(0x7f6d50061600 msgr2=0x7f6d50061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 -- 172.21.15.60:0/4248569624 shutdown_connections 2024-04-25T08:30:00.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.086+0000 7f6d555f9640 1 -- 172.21.15.60:0/4248569624 wait complete. 2024-04-25T08:30:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.087+0000 7f6d555f9640 1 Processor -- start 2024-04-25T08:30:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.087+0000 7f6d555f9640 1 -- start start 2024-04-25T08:30:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 0x7f6d50151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d500d0b10 0x7f6d501514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d5015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5015df40 con 0x7f6d500b5f00 2024-04-25T08:30:00.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500d1d50 con 0x7f6d500d0b10 2024-04-25T08:30:00.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5015e0c0 con 0x7f6d5015db60 2024-04-25T08:30:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 0x7f6d50151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 0x7f6d50151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55506/0 (socket says 172.21.15.60:55506) 2024-04-25T08:30:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 -- 172.21.15.60:0/2163509723 learned_addr learned my addr 172.21.15.60:0/2163509723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4e7fc640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d500d0b10 0x7f6d501514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4f7fe640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d5015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 -- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 msgr2=0x7f6d5015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d5015efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 -- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d500d0b10 msgr2=0x7f6d501514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d500d0b10 0x7f6d501514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 -- 172.21.15.60:0/2163509723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500cc1e0 con 0x7f6d500b5f00 2024-04-25T08:30:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4e7fc640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d500d0b10 0x7f6d501514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:00.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d4effd640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 0x7f6d50151fa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40041070 con 0x7f6d500b5f00 2024-04-25T08:30:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4003c460 con 0x7f6d500b5f00 2024-04-25T08:30:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5015f870 con 0x7f6d500b5f00 2024-04-25T08:30:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.088+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40037040 con 0x7f6d500b5f00 2024-04-25T08:30:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.089+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5015f510 con 0x7f6d500b5f00 2024-04-25T08:30:00.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.089+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d50073270 con 0x7f6d500b5f00 2024-04-25T08:30:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.090+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d40044070 con 0x7f6d500b5f00 2024-04-25T08:30:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.090+0000 7f6d377fe640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d3c06b500 0x7f6d3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.090+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6d400aab00 con 0x7f6d500b5f00 2024-04-25T08:30:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.091+0000 7f6d4e7fc640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d3c06b500 0x7f6d3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.091+0000 7f6d4e7fc640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d3c06b500 0x7f6d3c06d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d500c9fa0 tx=0x7f6d44007c10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.095+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d40078f60 con 0x7f6d500b5f00 2024-04-25T08:30:00.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.277+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d50073990 con 0x7f6d500b5f00 2024-04-25T08:30:00.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.278+0000 7f6d377fe640 1 -- 172.21.15.60:0/2163509723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f6d4007d000 con 0x7f6d500b5f00 2024-04-25T08:30:00.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:00.276 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d3c06b500 msgr2=0x7f6d3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d3c06b500 0x7f6d3c06d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d500c9fa0 tx=0x7f6d44007c10 comp rx=0 tx=0).stop 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 msgr2=0x7f6d50151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 0x7f6d50151fa0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6d40000c00 tx=0x7f6d400046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 shutdown_connections 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d5015db60 0x7f6d5015efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d3c06b500 0x7f6d3c06d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d500d0b10 0x7f6d501514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 --2- 172.21.15.60:0/2163509723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d500b5f00 0x7f6d50151fa0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 >> 172.21.15.60:0/2163509723 conn(0x7f6d50061600 msgr2=0x7f6d500b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 shutdown_connections 2024-04-25T08:30:00.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.280+0000 7f6d555f9640 1 -- 172.21.15.60:0/2163509723 wait complete. 2024-04-25T08:30:00.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:00.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.395+0000 7fdb946f5640 1 Processor -- start 2024-04-25T08:30:00.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.395+0000 7fdb946f5640 1 -- start start 2024-04-25T08:30:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb946f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb8c15dc20 0x7fdb8c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb946f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c161710 0x7fdb8c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb946f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c15e000 con 0x7fdb8c0b9330 2024-04-25T08:30:00.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb946f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0d1de0 con 0x7fdb8c15dc20 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb946f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c161c60 con 0x7fdb8c161710 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb92491640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb8c15dc20 0x7fdb8c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb92491640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb8c15dc20 0x7fdb8c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55508/0 (socket says 172.21.15.60:55508) 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb92491640 1 -- 172.21.15.60:0/1504500980 learned_addr learned my addr 172.21.15.60:0/1504500980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.396+0000 7fdb92491640 1 -- 172.21.15.60:0/1504500980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c161710 msgr2=0x7fdb8c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb92491640 1 --2- 172.21.15.60:0/1504500980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c161710 0x7fdb8c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb92491640 1 -- 172.21.15.60:0/1504500980 >> v1:172.21.15.100:6789/0 conn(0x7fdb8c0b9330 legacy=0x7fdb8c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb92491640 1 -- 172.21.15.60:0/1504500980 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c161f40 con 0x7fdb8c15dc20 2024-04-25T08:30:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb92491640 1 --2- 172.21.15.60:0/1504500980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c15dc20 0x7fdb8c0ce670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009870 tx=0x7fdb8402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944083f3de2e6425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb9148f640 1 -- 172.21.15.60:0/1504500980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84037070 con 0x7fdb8c15dc20 2024-04-25T08:30:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb9148f640 1 -- 172.21.15.60:0/1504500980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84032440 con 0x7fdb8c15dc20 2024-04-25T08:30:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.397+0000 7fdb9148f640 1 -- 172.21.15.60:0/1504500980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84032810 con 0x7fdb8c15dc20 2024-04-25T08:30:00.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 -- 172.21.15.60:0/1504500980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c15dc20 msgr2=0x7fdb8c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 --2- 172.21.15.60:0/1504500980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c15dc20 0x7fdb8c0ce670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009870 tx=0x7fdb8402a210 comp rx=0 tx=0).stop 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 -- 172.21.15.60:0/1504500980 shutdown_connections 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 --2- 172.21.15.60:0/1504500980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c161710 0x7fdb8c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 --2- 172.21.15.60:0/1504500980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c15dc20 0x7fdb8c0ce670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 -- 172.21.15.60:0/1504500980 >> 172.21.15.60:0/1504500980 conn(0x7fdb8c0d0420 msgr2=0x7fdb8c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 -- 172.21.15.60:0/1504500980 shutdown_connections 2024-04-25T08:30:00.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.398+0000 7fdb946f5640 1 -- 172.21.15.60:0/1504500980 wait complete. 2024-04-25T08:30:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.399+0000 7fdb946f5640 1 Processor -- start 2024-04-25T08:30:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.399+0000 7fdb946f5640 1 -- start start 2024-04-25T08:30:00.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb946f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c0b9330 0x7fdb8c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb946f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c15dc20 0x7fdb8c148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb946f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 0x7fdb8c0b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb946f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0b5e50 con 0x7fdb8c161710 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb946f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c162500 con 0x7fdb8c0b9330 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb946f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0b5fd0 con 0x7fdb8c15dc20 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 0x7fdb8c0b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 0x7fdb8c0b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55510/0 (socket says 172.21.15.60:55510) 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 -- 172.21.15.60:0/4203949716 learned_addr learned my addr 172.21.15.60:0/4203949716 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92491640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c0b9330 0x7fdb8c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 -- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c15dc20 msgr2=0x7fdb8c148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb91c90640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c15dc20 0x7fdb8c148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c15dc20 0x7fdb8c148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 -- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c0b9330 msgr2=0x7fdb8c14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c0b9330 0x7fdb8c14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92c92640 1 -- 172.21.15.60:0/4203949716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c1494a0 con 0x7fdb8c161710 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.400+0000 7fdb92491640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c0b9330 0x7fdb8c14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.401+0000 7fdb92c92640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 0x7fdb8c0b2ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00ba10 tx=0x7fdb7c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.401+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c019070 con 0x7fdb8c161710 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.401+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00cac0 con 0x7fdb8c161710 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.401+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c149740 con 0x7fdb8c161710 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.401+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c014440 con 0x7fdb8c161710 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.401+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c149d00 con 0x7fdb8c161710 2024-04-25T08:30:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.402+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c073a00 con 0x7fdb8c161710 2024-04-25T08:30:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.402+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb7c01c070 con 0x7fdb8c161710 2024-04-25T08:30:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.403+0000 7fdb8affd640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb7806b4d0 0x7fdb7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.403+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdb7c08a040 con 0x7fdb8c161710 2024-04-25T08:30:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.403+0000 7fdb92491640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb7806b4d0 0x7fdb7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.403+0000 7fdb92491640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb7806b4d0 0x7fdb7806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb84034000 tx=0x7fdb8402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.407+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb7c058b80 con 0x7fdb8c161710 2024-04-25T08:30:00.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.587+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb8c15c790 con 0x7fdb8c161710 2024-04-25T08:30:00.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.588+0000 7fdb8affd640 1 -- 172.21.15.60:0/4203949716 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb7c058480 con 0x7fdb8c161710 2024-04-25T08:30:00.585 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb7806b4d0 msgr2=0x7fdb7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb7806b4d0 0x7fdb7806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb84034000 tx=0x7fdb8402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 msgr2=0x7fdb8c0b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 0x7fdb8c0b2ad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00ba10 tx=0x7fdb7c00bed0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 shutdown_connections 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb8c161710 0x7fdb8c0b2ad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb8c15dc20 0x7fdb8c148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb7806b4d0 0x7fdb7806d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 --2- 172.21.15.60:0/4203949716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb8c0b9330 0x7fdb8c14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 >> 172.21.15.60:0/4203949716 conn(0x7fdb8c0d0420 msgr2=0x7fdb8c15bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 shutdown_connections 2024-04-25T08:30:00.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.590+0000 7fdb946f5640 1 -- 172.21.15.60:0/4203949716 wait complete. 2024-04-25T08:30:00.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:00.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.704+0000 7f34c7038640 1 Processor -- start 2024-04-25T08:30:00.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.704+0000 7f34c7038640 1 -- start start 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.705+0000 7f34c7038640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f34c00cadb0 0x7f34c00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.705+0000 7f34c7038640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c00afc80 con 0x7f34c015de80 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.705+0000 7f34c7038640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c00ce4d0 con 0x7f34c00ccfe0 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.705+0000 7f34c7038640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c00cb240 con 0x7f34c00cadb0 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.705+0000 7f34c4dd4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f34c015de80 0x7f34c00cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60218/0 (socket says 172.21.15.60:60218) 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.705+0000 7f34c4dd4640 1 -- 172.21.15.60:0/122614518 learned_addr learned my addr 172.21.15.60:0/122614518 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bffff640 1 --2- 172.21.15.60:0/122614518 >> v2:172.21.15.100:3300/0 conn(0x7f34c00cadb0 0x7f34c00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941752271 0 0) 0x7f34c00afc80 con 0x7f34c015de80 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34ac003520 con 0x7f34c015de80 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036110866 0 0) 0x7f34ac003520 con 0x7f34c015de80 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 >> v2:172.21.15.100:3300/0 conn(0x7f34c00cadb0 msgr2=0x7f34c00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 --2- 172.21.15.60:0/122614518 >> v2:172.21.15.100:3300/0 conn(0x7f34c00cadb0 0x7f34c00cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 >> v1:172.21.15.100:6789/0 conn(0x7f34c00ccfe0 legacy=0x7f34c00cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00ce7b0 con 0x7f34c015de80 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f34b80019f0 con 0x7f34c015de80 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 >> v1:172.21.15.60:6789/0 conn(0x7f34c015de80 legacy=0x7f34c00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0039b0 0x7f34ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34ac004800 0x7f34ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34ac0051c0 0x7f34ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac00c030 con 0x7f34ac0051c0 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac00c1e0 con 0x7f34ac004800 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac00c3e0 con 0x7f34ac0039b0 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34bffff640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34ac004800 0x7f34ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34c55d5640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34ac0051c0 0x7f34ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.706+0000 7f34c4dd4640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0039b0 0x7f34ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 -- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0039b0 msgr2=0x7f34ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0039b0 0x7f34ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 -- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34ac004800 msgr2=0x7f34ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34ac004800 0x7f34ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 -- 172.21.15.60:0/122614518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac003520 con 0x7f34ac0051c0 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34bffff640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34ac004800 0x7f34ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 -- 172.21.15.60:0/122614518 reap_dead start 2024-04-25T08:30:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34c55d5640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34ac0051c0 0x7f34ac00b9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34a8001750 tx=0x7f34ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b0148d736dc722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0009690 con 0x7f34ac0051c0 2024-04-25T08:30:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8005310 con 0x7f34ac0051c0 2024-04-25T08:30:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.707+0000 7f34bf7fe640 1 -- 172.21.15.60:0/122614518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8001370 con 0x7f34ac0051c0 2024-04-25T08:30:00.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 -- 172.21.15.60:0/122614518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34ac0051c0 msgr2=0x7f34ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34ac0051c0 0x7f34ac00b9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34a8001750 tx=0x7f34ac005830 comp rx=0 tx=0).stop 2024-04-25T08:30:00.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 -- 172.21.15.60:0/122614518 shutdown_connections 2024-04-25T08:30:00.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 --2- 172.21.15.60:0/122614518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34ac0051c0 0x7f34ac00b9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 -- 172.21.15.60:0/122614518 >> 172.21.15.60:0/122614518 conn(0x7f34c00b2390 msgr2=0x7f34c00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:00.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 -- 172.21.15.60:0/122614518 shutdown_connections 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.708+0000 7f34c7038640 1 -- 172.21.15.60:0/122614518 wait complete. 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.709+0000 7f34c7038640 1 Processor -- start 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.709+0000 7f34c7038640 1 -- start start 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c7038640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0051c0 0x7f34c00aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c7038640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34c0156cf0 0x7f34c0158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c7038640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 0x7f34c00d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c7038640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00b2eb0 con 0x7f34c00d06f0 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c7038640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c015e260 con 0x7f34c0156cf0 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c7038640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c00b3030 con 0x7f34ac0051c0 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c55d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 0x7f34c00d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c55d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 0x7f34c00d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55530/0 (socket says 172.21.15.60:55530) 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c55d5640 1 -- 172.21.15.60:0/1719499170 learned_addr learned my addr 172.21.15.60:0/1719499170 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34c4dd4640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0051c0 0x7f34c00aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.710+0000 7f34bffff640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34c0156cf0 0x7f34c0158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c55d5640 1 -- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0051c0 msgr2=0x7f34c00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c55d5640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0051c0 0x7f34c00aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c55d5640 1 -- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34c0156cf0 msgr2=0x7f34c0158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c55d5640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34c0156cf0 0x7f34c0158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c55d5640 1 -- 172.21.15.60:0/1719499170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c0159300 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c55d5640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 0x7f34c00d0ba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f34a8001870 tx=0x7f34a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8001370 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a802ae20 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c00b0de0 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8007a30 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.711+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c00b0820 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.712+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c0073a00 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.712+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34a8005310 con 0x7f34c00d06f0 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.713+0000 7f34bd7fa640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a006b500 0x7f34a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:00.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.713+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f34a80a9680 con 0x7f34c00d06f0 2024-04-25T08:30:00.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.713+0000 7f34c4dd4640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a006b500 0x7f34a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:00.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.714+0000 7f34c4dd4640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a006b500 0x7f34a006d9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f34b8004d00 tx=0x7f34b8006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:00.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.717+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34a8078280 con 0x7f34c00d06f0 2024-04-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.897+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34c006d330 con 0x7f34c00d06f0 2024-04-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.897+0000 7f34bd7fa640 1 -- 172.21.15.60:0/1719499170 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34a8077b80 con 0x7f34c00d06f0 2024-04-25T08:30:00.895 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a006b500 msgr2=0x7f34a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a006b500 0x7f34a006d9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f34b8004d00 tx=0x7f34b8006040 comp rx=0 tx=0).stop 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 msgr2=0x7f34c00d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 0x7f34c00d0ba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f34a8001870 tx=0x7f34a8001120 comp rx=0 tx=0).stop 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 shutdown_connections 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34c00d06f0 0x7f34c00d0ba0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a006b500 0x7f34a006d9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34c0156cf0 0x7f34c0158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 --2- 172.21.15.60:0/1719499170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34ac0051c0 0x7f34c00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:00.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 >> 172.21.15.60:0/1719499170 conn(0x7f34c00b2390 msgr2=0x7f34c00b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:00.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 shutdown_connections 2024-04-25T08:30:00.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:00.899+0000 7f34c7038640 1 -- 172.21.15.60:0/1719499170 wait complete. 2024-04-25T08:30:00.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.017+0000 7f8086adb640 1 Processor -- start 2024-04-25T08:30:01.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.017+0000 7f8086adb640 1 -- start start 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8086adb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80800ced80 0x7f80800d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8086adb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8080161710 0x7f80800d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8086adb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808015e000 con 0x7f808015dc20 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8086adb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80800d1de0 con 0x7f80800ced80 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8086adb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8080161c60 con 0x7f8080161710 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8077fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80800ced80 0x7f80800d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8084877640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f808015dc20 0x7f80800ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60228/0 (socket says 172.21.15.60:60228) 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8077fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80800ced80 0x7f80800d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55534/0 (socket says 172.21.15.60:55534) 2024-04-25T08:30:01.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8084877640 1 -- 172.21.15.60:0/1625648893 learned_addr learned my addr 172.21.15.60:0/1625648893 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8077fff640 1 -- 172.21.15.60:0/1625648893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8080161710 msgr2=0x7f80800d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8077fff640 1 --2- 172.21.15.60:0/1625648893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8080161710 0x7f80800d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8077fff640 1 -- 172.21.15.60:0/1625648893 >> v1:172.21.15.60:6789/0 conn(0x7f808015dc20 legacy=0x7f80800ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.018+0000 7f8077fff640 1 -- 172.21.15.60:0/1625648893 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080161f40 con 0x7f80800ced80 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.019+0000 7f8077fff640 1 --2- 172.21.15.60:0/1625648893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80800ced80 0x7f80800d22e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8c21bc75c8a8b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.019+0000 7f80777fe640 1 -- 172.21.15.60:0/1625648893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070037070 con 0x7f80800ced80 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.019+0000 7f80777fe640 1 -- 172.21.15.60:0/1625648893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807002ad10 con 0x7f80800ced80 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.019+0000 7f80777fe640 1 -- 172.21.15.60:0/1625648893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070032660 con 0x7f80800ced80 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 -- 172.21.15.60:0/1625648893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80800ced80 msgr2=0x7f80800d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 --2- 172.21.15.60:0/1625648893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80800ced80 0x7f80800d22e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a080 comp rx=0 tx=0).stop 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 -- 172.21.15.60:0/1625648893 shutdown_connections 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 --2- 172.21.15.60:0/1625648893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8080161710 0x7f80800d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 --2- 172.21.15.60:0/1625648893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80800ced80 0x7f80800d22e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 -- 172.21.15.60:0/1625648893 >> 172.21.15.60:0/1625648893 conn(0x7f80800d0420 msgr2=0x7f80800d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 -- 172.21.15.60:0/1625648893 shutdown_connections 2024-04-25T08:30:01.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.020+0000 7f8086adb640 1 -- 172.21.15.60:0/1625648893 wait complete. 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.021+0000 7f8086adb640 1 Processor -- start 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.021+0000 7f8086adb640 1 -- start start 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8086adb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80800ced80 0x7f8080157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8086adb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f808015dc20 0x7f808015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8086adb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 0x7f80800c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8086adb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800b5e50 con 0x7f8080161710 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8086adb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8080162500 con 0x7f80800ced80 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8086adb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800b5fd0 con 0x7f808015dc20 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 0x7f80800c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 0x7f80800c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55548/0 (socket says 172.21.15.60:55548) 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 -- 172.21.15.60:0/2753131686 learned_addr learned my addr 172.21.15.60:0/2753131686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:01.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8077fff640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f808015dc20 0x7f808015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8084877640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80800ced80 0x7f8080157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 -- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f808015dc20 msgr2=0x7f808015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f808015dc20 0x7f808015f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 -- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80800ced80 msgr2=0x7f8080157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80800ced80 0x7f8080157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 -- 172.21.15.60:0/2753131686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808015e1f0 con 0x7f8080161710 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8077fff640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f808015dc20 0x7f808015f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.022+0000 7f8085078640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 0x7f80800c92a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f807800ba10 tx=0x7f807800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.023+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078019070 con 0x7f8080161710 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.023+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807800ca80 con 0x7f8080161710 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.023+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078014440 con 0x7f8080161710 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.023+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808015e4d0 con 0x7f8080161710 2024-04-25T08:30:01.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.023+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808015ea90 con 0x7f8080161710 2024-04-25T08:30:01.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.024+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8080073a00 con 0x7f8080161710 2024-04-25T08:30:01.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.024+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f807801c070 con 0x7f8080161710 2024-04-25T08:30:01.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.024+0000 7f80757fa640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f806c06b4d0 0x7f806c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.024+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f807808a220 con 0x7f8080161710 2024-04-25T08:30:01.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.025+0000 7f8084877640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f806c06b4d0 0x7f806c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.025+0000 7f8084877640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f806c06b4d0 0x7f806c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f807c000f60 tx=0x7f807c007c80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.029+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8078058cc0 con 0x7f8080161710 2024-04-25T08:30:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.209+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f808015d230 con 0x7f8080161710 2024-04-25T08:30:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.210+0000 7f80757fa640 1 -- 172.21.15.60:0/2753131686 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80780585c0 con 0x7f8080161710 2024-04-25T08:30:01.208 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f806c06b4d0 msgr2=0x7f806c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f806c06b4d0 0x7f806c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f807c000f60 tx=0x7f807c007c80 comp rx=0 tx=0).stop 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 msgr2=0x7f80800c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 0x7f80800c92a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f807800ba10 tx=0x7f807800bed0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 shutdown_connections 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8080161710 0x7f80800c92a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f808015dc20 0x7f808015f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f806c06b4d0 0x7f806c06d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 --2- 172.21.15.60:0/2753131686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80800ced80 0x7f8080157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 >> 172.21.15.60:0/2753131686 conn(0x7f80800d0420 msgr2=0x7f80800b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 shutdown_connections 2024-04-25T08:30:01.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.212+0000 7f8086adb640 1 -- 172.21.15.60:0/2753131686 wait complete. 2024-04-25T08:30:01.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.327+0000 7fa688110640 1 Processor -- start 2024-04-25T08:30:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.327+0000 7fa688110640 1 -- start start 2024-04-25T08:30:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa688110640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 0x7fa6800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa688110640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68006cf50 con 0x7fa6800b5620 2024-04-25T08:30:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa688110640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6800ce530 con 0x7fa68015b7d0 2024-04-25T08:30:01.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa688110640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800ce810 con 0x7fa68006c9e0 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa6856ab640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa6800b5620 0x7fa68006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60242/0 (socket says 172.21.15.60:60242) 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa6856ab640 1 -- 172.21.15.60:0/859557848 learned_addr learned my addr 172.21.15.60:0/859557848 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa6866ad640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 0x7fa6800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111023243 0 0) 0x7fa68006cf50 con 0x7fa6800b5620 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa66c003520 con 0x7fa6800b5620 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193738068 0 0) 0x7fa6800ce530 con 0x7fa68015b7d0 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa68006cf50 con 0x7fa68015b7d0 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3201895048 0 0) 0x7fa66c003520 con 0x7fa6800b5620 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 msgr2=0x7fa6800b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 0x7fa6800b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 >> v1:172.21.15.100:6789/0 conn(0x7fa68015b7d0 legacy=0x7fa6800cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6800ceaf0 con 0x7fa6800b5620 2024-04-25T08:30:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fa670001690 con 0x7fa6800b5620 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 >> v1:172.21.15.60:6789/0 conn(0x7fa6800b5620 legacy=0x7fa68006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 0x7fa66c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa66c0040d0 0x7fa66c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa66c004d40 0x7fa66c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800ce810 con 0x7fa66c0040d0 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c00baa0 con 0x7fa66c004d40 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.328+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c00bca0 con 0x7fa68006c9e0 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6856ab640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa66c0040d0 0x7fa66c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa685eac640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 0x7fa66c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6866ad640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa66c004d40 0x7fa66c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6856ab640 1 -- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 msgr2=0x7fa66c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6856ab640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa68006c9e0 0x7fa66c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6856ab640 1 -- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa66c004d40 msgr2=0x7fa66c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6856ab640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa66c004d40 0x7fa66c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.329+0000 7fa6856ab640 1 -- 172.21.15.60:0/859557848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c003520 con 0x7fa66c0040d0 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa6856ab640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa66c0040d0 0x7fa66c004800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c190 tx=0x7fa66c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c2158b7b21eafa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670007cc0 con 0x7fa66c0040d0 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670005af0 con 0x7fa66c0040d0 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa684eaa640 1 -- 172.21.15.60:0/859557848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670007070 con 0x7fa66c0040d0 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa688110640 1 -- 172.21.15.60:0/859557848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa66c0040d0 msgr2=0x7fa66c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa688110640 1 --2- 172.21.15.60:0/859557848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa66c0040d0 0x7fa66c004800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c190 tx=0x7fa66c005830 comp rx=0 tx=0).stop 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa6866ad640 1 -- 172.21.15.60:0/859557848 reap_dead start 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa688110640 1 -- 172.21.15.60:0/859557848 shutdown_connections 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa688110640 1 -- 172.21.15.60:0/859557848 >> 172.21.15.60:0/859557848 conn(0x7fa6800d0800 msgr2=0x7fa6800d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa688110640 1 -- 172.21.15.60:0/859557848 shutdown_connections 2024-04-25T08:30:01.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.330+0000 7fa688110640 1 -- 172.21.15.60:0/859557848 wait complete. 2024-04-25T08:30:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 Processor -- start 2024-04-25T08:30:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 -- start start 2024-04-25T08:30:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa66c0040d0 0x7fa6800cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 0x7fa6800d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680147d20 0x7fa6801481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015bd30 con 0x7fa6800d27f0 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015bbb0 con 0x7fa680147d20 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa688110640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800b5ab0 con 0x7fa66c0040d0 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa6856ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 0x7fa6800d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa6856ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 0x7fa6800d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55570/0 (socket says 172.21.15.60:55570) 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.332+0000 7fa6856ab640 1 -- 172.21.15.60:0/3041896492 learned_addr learned my addr 172.21.15.60:0/3041896492 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa685eac640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa66c0040d0 0x7fa6800cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6866ad640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680147d20 0x7fa6801481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6856ab640 1 -- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa66c0040d0 msgr2=0x7fa6800cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6856ab640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa66c0040d0 0x7fa6800cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6856ab640 1 -- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680147d20 msgr2=0x7fa6801481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6856ab640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680147d20 0x7fa6801481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6856ab640 1 -- 172.21.15.60:0/3041896492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6801f3690 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6866ad640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680147d20 0x7fa6801481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6856ab640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 0x7fa6800d2c70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa67002ad50 tx=0x7fa6800ce700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670043070 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670038040 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670001690 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6801f3910 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.333+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6801f3ed0 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.334+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa680073a00 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.335+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6700072b0 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.335+0000 7fa6767fc640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa66406b4d0 0x7fa66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.335+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa6700aa030 con 0x7fa6800d27f0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.335+0000 7fa685eac640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa66406b4d0 0x7fa66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.336+0000 7fa685eac640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa66406b4d0 0x7fa66406d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0018d0 tx=0x7fa67c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.340+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6801f3690 con 0x7fa6800d27f0 2024-04-25T08:30:01.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.519+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa6800cda20 con 0x7fa6800d27f0 2024-04-25T08:30:01.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.519+0000 7fa6767fc640 1 -- 172.21.15.60:0/3041896492 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa670078c00 con 0x7fa6800d27f0 2024-04-25T08:30:01.517 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa66406b4d0 msgr2=0x7fa66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa66406b4d0 0x7fa66406d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0018d0 tx=0x7fa67c009040 comp rx=0 tx=0).stop 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 msgr2=0x7fa6800d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 0x7fa6800d2c70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa67002ad50 tx=0x7fa6800ce700 comp rx=0 tx=0).stop 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 shutdown_connections 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa66406b4d0 0x7fa66406d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa680147d20 0x7fa6801481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6800d27f0 0x7fa6800d2c70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 --2- 172.21.15.60:0/3041896492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa66c0040d0 0x7fa6800cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 >> 172.21.15.60:0/3041896492 conn(0x7fa6800d0800 msgr2=0x7fa6800b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 shutdown_connections 2024-04-25T08:30:01.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.521+0000 7fa688110640 1 -- 172.21.15.60:0/3041896492 wait complete. 2024-04-25T08:30:01.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.632+0000 7faa7f85d640 1 Processor -- start 2024-04-25T08:30:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.632+0000 7faa7f85d640 1 -- start start 2024-04-25T08:30:01.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa7f85d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa7815dbe0 0x7faa780d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa7f85d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780b6220 con 0x7faa780b92a0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa7f85d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780b97a0 con 0x7faa780b2cb0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa7f85d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7815dfc0 con 0x7faa7815dbe0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa7ddfa640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa780b92a0 0x7faa780b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60248/0 (socket says 172.21.15.60:60248) 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa7ddfa640 1 -- 172.21.15.60:0/239269422 learned_addr learned my addr 172.21.15.60:0/239269422 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.633+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590869511 0 0) 0x7faa780b6220 con 0x7faa780b92a0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa64003520 con 0x7faa780b92a0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7d5f9640 1 --2- 172.21.15.60:0/239269422 >> v2:172.21.15.100:3300/0 conn(0x7faa7815dbe0 0x7faa780d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756511741 0 0) 0x7faa64003520 con 0x7faa780b92a0 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 >> v2:172.21.15.100:3300/0 conn(0x7faa7815dbe0 msgr2=0x7faa780d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 --2- 172.21.15.60:0/239269422 >> v2:172.21.15.100:3300/0 conn(0x7faa7815dbe0 0x7faa780d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 >> v1:172.21.15.100:6789/0 conn(0x7faa780b2cb0 legacy=0x7faa780b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780d1c50 con 0x7faa780b92a0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7faa700019f0 con 0x7faa780b92a0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 >> v1:172.21.15.60:6789/0 conn(0x7faa780b92a0 legacy=0x7faa780b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa640039b0 0x7faa64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa64004800 0x7faa64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa640051c0 0x7faa6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6400c130 con 0x7faa640051c0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6400c2e0 con 0x7faa64004800 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6400c4e0 con 0x7faa640039b0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa640051c0 0x7faa6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7cdf8640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa64004800 0x7faa64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7d5f9640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa640039b0 0x7faa64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7ddfa640 1 -- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa640039b0 msgr2=0x7faa64003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa640039b0 0x7faa64003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7ddfa640 1 -- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa64004800 msgr2=0x7faa64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa64004800 0x7faa64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.634+0000 7faa7ddfa640 1 -- 172.21.15.60:0/239269422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa64003520 con 0x7faa640051c0 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.635+0000 7faa7ddfa640 1 -- 172.21.15.60:0/239269422 reap_dead start 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.635+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa640051c0 0x7faa6400b9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faa7000a830 tx=0x7faa64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1249d1be8fab70fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.635+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7000a480 con 0x7faa640051c0 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.635+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7002f070 con 0x7faa640051c0 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.635+0000 7faa6ffff640 1 -- 172.21.15.60:0/239269422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7002bc90 con 0x7faa640051c0 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 -- 172.21.15.60:0/239269422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa640051c0 msgr2=0x7faa6400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa640051c0 0x7faa6400b9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faa7000a830 tx=0x7faa64005830 comp rx=0 tx=0).stop 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 -- 172.21.15.60:0/239269422 shutdown_connections 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 --2- 172.21.15.60:0/239269422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa640051c0 0x7faa6400b9c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 -- 172.21.15.60:0/239269422 >> 172.21.15.60:0/239269422 conn(0x7faa78061a20 msgr2=0x7faa78061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 -- 172.21.15.60:0/239269422 shutdown_connections 2024-04-25T08:30:01.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.636+0000 7faa7f85d640 1 -- 172.21.15.60:0/239269422 wait complete. 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.637+0000 7faa7f85d640 1 Processor -- start 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.637+0000 7faa7f85d640 1 -- start start 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7f85d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa640051c0 0x7faa781f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7f85d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa781fba70 0x7faa7815ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7f85d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 0x7faa7815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7f85d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa780d2210 con 0x7faa781fa6d0 2024-04-25T08:30:01.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7f85d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa780d1a00 con 0x7faa640051c0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7f85d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa780b6ab0 con 0x7faa781fba70 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 0x7faa7815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 0x7faa7815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55582/0 (socket says 172.21.15.60:55582) 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 -- 172.21.15.60:0/4179269587 learned_addr learned my addr 172.21.15.60:0/4179269587 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7d5f9640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa640051c0 0x7faa781f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7cdf8640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa781fba70 0x7faa7815ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 -- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa781fba70 msgr2=0x7faa7815ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa781fba70 0x7faa7815ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 -- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa640051c0 msgr2=0x7faa781f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa640051c0 0x7faa781f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 -- 172.21.15.60:0/4179269587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7815f5c0 con 0x7faa781fa6d0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.638+0000 7faa7ddfa640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 0x7faa7815c690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa7000a950 tx=0x7faa780b9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.639+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa70044070 con 0x7faa781fa6d0 2024-04-25T08:30:01.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.639+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7002bd90 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.639+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7815f8a0 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.639+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7000a480 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.639+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa78156430 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.640+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa78155f40 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.640+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa70047070 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.641+0000 7faa6dffb640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa5c06b500 0x7faa5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.641+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faa700a9e80 con 0x7faa781fa6d0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.641+0000 7faa7d5f9640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa5c06b500 0x7faa5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.641+0000 7faa7d5f9640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa5c06b500 0x7faa5c06d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa74009b40 tx=0x7faa74009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.645+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa70078490 con 0x7faa781fa6d0 2024-04-25T08:30:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.824+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa78073990 con 0x7faa781fa6d0 2024-04-25T08:30:01.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.825+0000 7faa6dffb640 1 -- 172.21.15.60:0/4179269587 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7faa7007c530 con 0x7faa781fa6d0 2024-04-25T08:30:01.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:01.824 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa5c06b500 msgr2=0x7faa5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa5c06b500 0x7faa5c06d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa74009b40 tx=0x7faa74009190 comp rx=0 tx=0).stop 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 msgr2=0x7faa7815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 0x7faa7815c690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faa7000a950 tx=0x7faa780b9970 comp rx=0 tx=0).stop 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 shutdown_connections 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa781fa6d0 0x7faa7815c690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.827+0000 7faa7f85d640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa781fba70 0x7faa7815ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.828+0000 7faa7f85d640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa5c06b500 0x7faa5c06d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.828+0000 7faa7f85d640 1 --2- 172.21.15.60:0/4179269587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa640051c0 0x7faa781f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.828+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 >> 172.21.15.60:0/4179269587 conn(0x7faa78061a20 msgr2=0x7faa780b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.828+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 shutdown_connections 2024-04-25T08:30:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.828+0000 7faa7f85d640 1 -- 172.21.15.60:0/4179269587 wait complete. 2024-04-25T08:30:01.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:01.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.944+0000 7f51dc238640 1 Processor -- start 2024-04-25T08:30:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.944+0000 7f51dc238640 1 -- start start 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51dc238640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51d4161710 0x7f51d40d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51dc238640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d415e000 con 0x7f51d415dc20 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51dc238640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d40d1de0 con 0x7f51d40ced80 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51dc238640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d4161c60 con 0x7f51d4161710 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51da7d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51d4161710 0x7f51d40d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51d9fd4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f51d415dc20 0x7f51d40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:47994/0 (socket says 172.21.15.60:47994) 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51d9fd4640 1 -- 172.21.15.60:0/2300636255 learned_addr learned my addr 172.21.15.60:0/2300636255 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51da7d5640 1 -- 172.21.15.60:0/2300636255 >> v1:172.21.15.100:6789/0 conn(0x7f51d40ced80 legacy=0x7f51d40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51da7d5640 1 -- 172.21.15.60:0/2300636255 >> v1:172.21.15.60:6789/0 conn(0x7f51d415dc20 legacy=0x7f51d40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51da7d5640 1 -- 172.21.15.60:0/2300636255 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4161f40 con 0x7f51d4161710 2024-04-25T08:30:01.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.945+0000 7f51d8fd2640 1 -- 172.21.15.60:0/2300636255 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676830945 0 0) 0x7f51d415e000 con 0x7f51d415dc20 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51da7d5640 1 --2- 172.21.15.60:0/2300636255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d4161710 0x7f51d40d29f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009d90 tx=0x7f51c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43bc5c2af7dc007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51d8fd2640 1 -- 172.21.15.60:0/2300636255 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4007740 con 0x7f51d4161710 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51d8fd2640 1 -- 172.21.15.60:0/2300636255 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c40078e0 con 0x7f51d4161710 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51d8fd2640 1 -- 172.21.15.60:0/2300636255 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4007cb0 con 0x7f51d4161710 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51dc238640 1 -- 172.21.15.60:0/2300636255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d4161710 msgr2=0x7f51d40d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51dc238640 1 --2- 172.21.15.60:0/2300636255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d4161710 0x7f51d40d29f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009d90 tx=0x7f51c402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51dc238640 1 -- 172.21.15.60:0/2300636255 shutdown_connections 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51dc238640 1 --2- 172.21.15.60:0/2300636255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d4161710 0x7f51d40d29f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.946+0000 7f51dc238640 1 -- 172.21.15.60:0/2300636255 >> 172.21.15.60:0/2300636255 conn(0x7f51d40d0420 msgr2=0x7f51d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.947+0000 7f51dc238640 1 -- 172.21.15.60:0/2300636255 shutdown_connections 2024-04-25T08:30:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.947+0000 7f51dc238640 1 -- 172.21.15.60:0/2300636255 wait complete. 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 Processor -- start 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 -- start start 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51d40ced80 0x7f51d415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 0x7f51d40d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d4161710 0x7f51d4154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40b5e50 con 0x7f51d415dc20 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d4162500 con 0x7f51d4161710 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51dc238640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40b5fd0 con 0x7f51d40ced80 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.948+0000 7f51d97d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 0x7f51d40d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 0x7f51d40d2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55042/0 (socket says 172.21.15.60:55042) 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 -- 172.21.15.60:0/3057184296 learned_addr learned my addr 172.21.15.60:0/3057184296 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51da7d5640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d4161710 0x7f51d4154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d9fd4640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51d40ced80 0x7f51d415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 -- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51d40ced80 msgr2=0x7f51d415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51d40ced80 0x7f51d415f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 -- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d4161710 msgr2=0x7f51d4154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d4161710 0x7f51d4154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 -- 172.21.15.60:0/3057184296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d41f3560 con 0x7f51d415dc20 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d9fd4640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51d40ced80 0x7f51d415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d97d3640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 0x7f51d40d2cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c400 tx=0x7f51c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8017070 con 0x7f51d415dc20 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c80052d0 con 0x7f51d415dc20 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d41f3820 con 0x7f51d415dc20 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8003830 con 0x7f51d415dc20 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.949+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d41f3de0 con 0x7f51d415dc20 2024-04-25T08:30:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.950+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d4073a00 con 0x7f51d415dc20 2024-04-25T08:30:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.951+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51c801a070 con 0x7f51d415dc20 2024-04-25T08:30:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.951+0000 7f51d27fc640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f51c006b500 0x7f51c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.951+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51c8089850 con 0x7f51d415dc20 2024-04-25T08:30:01.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.951+0000 7f51d9fd4640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f51c006b500 0x7f51c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:01.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.952+0000 7f51d9fd4640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f51c006b500 0x7f51c006d9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000f60 tx=0x7f51cc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:01.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:01.956+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51c8058390 con 0x7f51d415dc20 2024-04-25T08:30:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.134+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51d40cdfd0 con 0x7f51d415dc20 2024-04-25T08:30:02.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.135+0000 7f51d27fc640 1 -- 172.21.15.60:0/3057184296 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51c8057c90 con 0x7f51d415dc20 2024-04-25T08:30:02.133 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f51c006b500 msgr2=0x7f51c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f51c006b500 0x7f51c006d9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51cc000f60 tx=0x7f51cc009040 comp rx=0 tx=0).stop 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 msgr2=0x7f51d40d2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 0x7f51d40d2cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c400 tx=0x7f51c800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 shutdown_connections 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f51c006b500 0x7f51c006d9c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51d4161710 0x7f51d4154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51d415dc20 0x7f51d40d2cc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 --2- 172.21.15.60:0/3057184296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51d40ced80 0x7f51d415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 >> 172.21.15.60:0/3057184296 conn(0x7f51d40d0420 msgr2=0x7f51d40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:02.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 shutdown_connections 2024-04-25T08:30:02.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.137+0000 7f51dc238640 1 -- 172.21.15.60:0/3057184296 wait complete. 2024-04-25T08:30:02.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:02.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.251+0000 7f016995c640 1 Processor -- start 2024-04-25T08:30:02.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.251+0000 7f016995c640 1 -- start start 2024-04-25T08:30:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f016995c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f016415dc20 0x7f01640ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f016995c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0164161710 0x7f01640cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f016995c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f01640b9330 0x7f016406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f016995c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016415e000 con 0x7f016415dc20 2024-04-25T08:30:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f016995c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01640d1de0 con 0x7f01640b9330 2024-04-25T08:30:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f016995c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0164161c60 con 0x7f0164161710 2024-04-25T08:30:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f016415dc20 0x7f01640ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f016415dc20 0x7f01640ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55054/0 (socket says 172.21.15.60:55054) 2024-04-25T08:30:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 -- 172.21.15.60:0/1478802831 learned_addr learned my addr 172.21.15.60:0/1478802831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f01637fe640 1 --2- 172.21.15.60:0/1478802831 >> v2:172.21.15.100:3300/0 conn(0x7f01640b9330 0x7f016406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f01627fc640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0164161710 0x7f01640cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 -- 172.21.15.60:0/1478802831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0164161710 msgr2=0x7f01640cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0164161710 0x7f01640cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 -- 172.21.15.60:0/1478802831 >> v2:172.21.15.100:3300/0 conn(0x7f01640b9330 msgr2=0x7f016406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 --2- 172.21.15.60:0/1478802831 >> v2:172.21.15.100:3300/0 conn(0x7f01640b9330 0x7f016406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f0162ffd640 1 -- 172.21.15.60:0/1478802831 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164161f40 con 0x7f016415dc20 2024-04-25T08:30:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f01637fe640 1 --2- 172.21.15.60:0/1478802831 >> v2:172.21.15.100:3300/0 conn(0x7f01640b9330 0x7f016406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.252+0000 7f01627fc640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0164161710 0x7f01640cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.253+0000 7f0162ffd640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016415dc20 0x7f01640ce670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f01640b6570 tx=0x7f015402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05dc8b338d302f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.253+0000 7f0161ffb640 1 -- 172.21.15.60:0/1478802831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154037070 con 0x7f016415dc20 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.253+0000 7f0161ffb640 1 -- 172.21.15.60:0/1478802831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154032440 con 0x7f016415dc20 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.253+0000 7f0161ffb640 1 -- 172.21.15.60:0/1478802831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154032810 con 0x7f016415dc20 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 -- 172.21.15.60:0/1478802831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016415dc20 msgr2=0x7f01640ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016415dc20 0x7f01640ce670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f01640b6570 tx=0x7f015402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 -- 172.21.15.60:0/1478802831 shutdown_connections 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 --2- 172.21.15.60:0/1478802831 >> v2:172.21.15.100:3300/0 conn(0x7f01640b9330 0x7f016406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0164161710 0x7f01640cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 --2- 172.21.15.60:0/1478802831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f016415dc20 0x7f01640ce670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 -- 172.21.15.60:0/1478802831 >> 172.21.15.60:0/1478802831 conn(0x7f01640d0420 msgr2=0x7f01640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 -- 172.21.15.60:0/1478802831 shutdown_connections 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.254+0000 7f016995c640 1 -- 172.21.15.60:0/1478802831 wait complete. 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.256+0000 7f016995c640 1 Processor -- start 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.256+0000 7f016995c640 1 -- start start 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f016995c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01640b9330 0x7f01640c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f016995c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016415dc20 0x7f01641e8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f016995c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 0x7f01640caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f016995c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01640b5e50 con 0x7f0164161710 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f016995c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0164162500 con 0x7f01640b9330 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f016995c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01640b5fd0 con 0x7f016415dc20 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 0x7f01640caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 0x7f01640caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55066/0 (socket says 172.21.15.60:55066) 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 -- 172.21.15.60:0/101039843 learned_addr learned my addr 172.21.15.60:0/101039843 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f0162ffd640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01640b9330 0x7f01640c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01627fc640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016415dc20 0x7f01641e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 -- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016415dc20 msgr2=0x7f01641e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016415dc20 0x7f01641e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 -- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01640b9330 msgr2=0x7f01640c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01640b9330 0x7f01640c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 -- 172.21.15.60:0/101039843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01640ca2f0 con 0x7f0164161710 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.257+0000 7f01637fe640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 0x7f01640caa70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f015c00cd40 tx=0x7f015c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.258+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c00f680 con 0x7f0164161710 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.258+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015c00f840 con 0x7f0164161710 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.258+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01641e8eb0 con 0x7f0164161710 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.258+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c00fc30 con 0x7f0164161710 2024-04-25T08:30:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.258+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01641e9470 con 0x7f0164161710 2024-04-25T08:30:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.259+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0164073a00 con 0x7f0164161710 2024-04-25T08:30:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.259+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f015c0213f0 con 0x7f0164161710 2024-04-25T08:30:02.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.259+0000 7f014bfff640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f015006b4d0 0x7f015006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.260+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f015c08b110 con 0x7f0164161710 2024-04-25T08:30:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.260+0000 7f0162ffd640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f015006b4d0 0x7f015006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.260+0000 7f0162ffd640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f015006b4d0 0x7f015006d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0154034000 tx=0x7f015402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.264+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f015c014030 con 0x7f0164161710 2024-04-25T08:30:02.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.444+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01640d16b0 con 0x7f0164161710 2024-04-25T08:30:02.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.445+0000 7f014bfff640 1 -- 172.21.15.60:0/101039843 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f015c059690 con 0x7f0164161710 2024-04-25T08:30:02.443 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f015006b4d0 msgr2=0x7f015006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f015006b4d0 0x7f015006d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0154034000 tx=0x7f015402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 msgr2=0x7f01640caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 0x7f01640caa70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f015c00cd40 tx=0x7f015c00a650 comp rx=0 tx=0).stop 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 shutdown_connections 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0164161710 0x7f01640caa70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f016415dc20 0x7f01641e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f015006b4d0 0x7f015006d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 --2- 172.21.15.60:0/101039843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01640b9330 0x7f01640c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 >> 172.21.15.60:0/101039843 conn(0x7f01640d0420 msgr2=0x7f01640b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:02.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 shutdown_connections 2024-04-25T08:30:02.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.447+0000 7f016995c640 1 -- 172.21.15.60:0/101039843 wait complete. 2024-04-25T08:30:02.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:02.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.562+0000 7f286ba09640 1 Processor -- start 2024-04-25T08:30:02.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.562+0000 7f286ba09640 1 -- start start 2024-04-25T08:30:02.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f286ba09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 0x7f28640ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f286ba09640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28640d32f0 0x7f28640cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f286ba09640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286415e000 con 0x7f2864161710 2024-04-25T08:30:02.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f286ba09640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28640d1de0 con 0x7f28640d32f0 2024-04-25T08:30:02.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f286ba09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2864161c60 con 0x7f286415dc20 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f2869fa6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28640d32f0 0x7f28640cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f28697a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 0x7f28640ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f28697a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 0x7f28640ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54448/0 (socket says 172.21.15.60:54448) 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f2869fa6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28640d32f0 0x7f28640cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58972/0 (socket says 172.21.15.60:58972) 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f2868fa4640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2864161710 0x7f286406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57122/0 (socket says 172.21.15.60:57122) 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.563+0000 7f28697a5640 1 -- 172.21.15.60:0/3303731229 learned_addr learned my addr 172.21.15.60:0/3303731229 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.564+0000 7f2869fa6640 1 -- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 msgr2=0x7f28640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.564+0000 7f2869fa6640 1 --2- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 0x7f28640ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.564+0000 7f2869fa6640 1 -- 172.21.15.60:0/3303731229 >> v1:172.21.15.100:6789/0 conn(0x7f2864161710 legacy=0x7f286406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.564+0000 7f2869fa6640 1 -- 172.21.15.60:0/3303731229 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864161f40 con 0x7f28640d32f0 2024-04-25T08:30:02.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.564+0000 7f28697a5640 1 --2- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 0x7f28640ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:02.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.564+0000 7f2869fa6640 1 --2- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28640d32f0 0x7f28640cebb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f285c009d90 tx=0x7f285c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45f68a78e220feb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f285bfff640 1 -- 172.21.15.60:0/3303731229 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c007740 con 0x7f28640d32f0 2024-04-25T08:30:02.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f285bfff640 1 -- 172.21.15.60:0/3303731229 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c0078e0 con 0x7f28640d32f0 2024-04-25T08:30:02.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f285bfff640 1 -- 172.21.15.60:0/3303731229 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c007cb0 con 0x7f28640d32f0 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 -- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28640d32f0 msgr2=0x7f28640cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 --2- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28640d32f0 0x7f28640cebb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f285c009d90 tx=0x7f285c02a660 comp rx=0 tx=0).stop 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 -- 172.21.15.60:0/3303731229 shutdown_connections 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 --2- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28640d32f0 0x7f28640cebb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 --2- 172.21.15.60:0/3303731229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f286415dc20 0x7f28640ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 -- 172.21.15.60:0/3303731229 >> 172.21.15.60:0/3303731229 conn(0x7f28640d0420 msgr2=0x7f28640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 -- 172.21.15.60:0/3303731229 shutdown_connections 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.565+0000 7f286ba09640 1 -- 172.21.15.60:0/3303731229 wait complete. 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.566+0000 7f286ba09640 1 Processor -- start 2024-04-25T08:30:02.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.566+0000 7f286ba09640 1 -- start start 2024-04-25T08:30:02.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f286ba09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28640d32f0 0x7f2864152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f286ba09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286415dc20 0x7f28640d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f286ba09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 0x7f28640d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f286ba09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640b5e50 con 0x7f2864161710 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f286ba09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2864162500 con 0x7f286415dc20 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f286ba09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28640b5fd0 con 0x7f28640d32f0 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f2869fa6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 0x7f28640d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f2869fa6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 0x7f28640d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55072/0 (socket says 172.21.15.60:55072) 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f2869fa6640 1 -- 172.21.15.60:0/479162579 learned_addr learned my addr 172.21.15.60:0/479162579 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.567+0000 7f2868fa4640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286415dc20 0x7f28640d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f28697a5640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28640d32f0 0x7f2864152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2869fa6640 1 -- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28640d32f0 msgr2=0x7f2864152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2869fa6640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28640d32f0 0x7f2864152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2869fa6640 1 -- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286415dc20 msgr2=0x7f28640d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2869fa6640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286415dc20 0x7f28640d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2869fa6640 1 -- 172.21.15.60:0/479162579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28641f39b0 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f28697a5640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28640d32f0 0x7f2864152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2868fa4640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286415dc20 0x7f28640d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2869fa6640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 0x7f28640d3ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f285c009eb0 tx=0x7f285c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c040070 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c031c60 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28640ab240 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c009450 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.568+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28640ab800 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.569+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2864073a00 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.570+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f285c039040 con 0x7f2864161710 2024-04-25T08:30:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.570+0000 7f2859ffb640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f284806b4d0 0x7f284806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.570+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f285c0ab900 con 0x7f2864161710 2024-04-25T08:30:02.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.571+0000 7f28697a5640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f284806b4d0 0x7f284806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.571+0000 7f28697a5640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f284806b4d0 0x7f284806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2860009b80 tx=0x7f2860008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.575+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f285c07a440 con 0x7f2864161710 2024-04-25T08:30:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.757+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28640b48e0 con 0x7f2864161710 2024-04-25T08:30:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.758+0000 7f2859ffb640 1 -- 172.21.15.60:0/479162579 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f285c079d40 con 0x7f2864161710 2024-04-25T08:30:02.756 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.759+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f284806b4d0 msgr2=0x7f284806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.759+0000 7f286ba09640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f284806b4d0 0x7f284806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2860009b80 tx=0x7f2860008040 comp rx=0 tx=0).stop 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.759+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 msgr2=0x7f28640d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.759+0000 7f286ba09640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 0x7f28640d3ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f285c009eb0 tx=0x7f285c004a40 comp rx=0 tx=0).stop 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 shutdown_connections 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2864161710 0x7f28640d3ec0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f284806b4d0 0x7f284806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f286415dc20 0x7f28640d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 --2- 172.21.15.60:0/479162579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28640d32f0 0x7f2864152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 >> 172.21.15.60:0/479162579 conn(0x7f28640d0420 msgr2=0x7f28640c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 shutdown_connections 2024-04-25T08:30:02.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.760+0000 7f286ba09640 1 -- 172.21.15.60:0/479162579 wait complete. 2024-04-25T08:30:02.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.874+0000 7f27c521f640 1 Processor -- start 2024-04-25T08:30:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.874+0000 7f27c521f640 1 -- start start 2024-04-25T08:30:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27c521f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c015dac0 0x7f27c00d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27c521f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27c00d0ab0 0x7f27c00b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27c521f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c00d2420 con 0x7f27c00d1f30 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27c521f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c00d3030 con 0x7f27c00d0ab0 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27c521f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00d2700 con 0x7f27c015dac0 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27be59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f27c00d1f30 0x7f27c00d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48004/0 (socket says 172.21.15.60:48004) 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27be59c640 1 -- 172.21.15.60:0/1379647674 learned_addr learned my addr 172.21.15.60:0/1379647674 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27bf59e640 1 --2- 172.21.15.60:0/1379647674 >> v2:172.21.15.100:3300/0 conn(0x7f27c00d0ab0 0x7f27c00b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27bed9d640 1 --2- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c015dac0 0x7f27c00d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27bdd9b640 1 -- 172.21.15.60:0/1379647674 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007398214 0 0) 0x7f27c00d2420 con 0x7f27c00d1f30 2024-04-25T08:30:02.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.875+0000 7f27bdd9b640 1 -- 172.21.15.60:0/1379647674 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac003520 con 0x7f27c00d1f30 2024-04-25T08:30:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.876+0000 7f27bf59e640 1 -- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c015dac0 msgr2=0x7f27c00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.876+0000 7f27bdd9b640 1 -- 172.21.15.60:0/1379647674 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 852241825 0 0) 0x7f27ac003520 con 0x7f27c00d1f30 2024-04-25T08:30:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.876+0000 7f27bf59e640 1 --2- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c015dac0 0x7f27c00d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.876+0000 7f27bf59e640 1 -- 172.21.15.60:0/1379647674 >> v1:172.21.15.60:6789/0 conn(0x7f27c00d1f30 legacy=0x7f27c00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.876+0000 7f27bf59e640 1 -- 172.21.15.60:0/1379647674 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00d29e0 con 0x7f27c00d0ab0 2024-04-25T08:30:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.876+0000 7f27bf59e640 1 --2- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c00d0ab0 0x7f27c00b6990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27b80097a0 tx=0x7f27b802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc57df2208a413c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27bdd9b640 1 -- 172.21.15.60:0/1379647674 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8037070 con 0x7f27c00d0ab0 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27bdd9b640 1 -- 172.21.15.60:0/1379647674 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b802ac30 con 0x7f27c00d0ab0 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27bdd9b640 1 -- 172.21.15.60:0/1379647674 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a80017f0 con 0x7f27c00d0ab0 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 -- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c00d0ab0 msgr2=0x7f27c00b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 --2- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c00d0ab0 0x7f27c00b6990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27b80097a0 tx=0x7f27b802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 -- 172.21.15.60:0/1379647674 shutdown_connections 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 --2- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c00d0ab0 0x7f27c00b6990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 --2- 172.21.15.60:0/1379647674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c015dac0 0x7f27c00d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 -- 172.21.15.60:0/1379647674 >> 172.21.15.60:0/1379647674 conn(0x7f27c0061c30 msgr2=0x7f27c006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 -- 172.21.15.60:0/1379647674 shutdown_connections 2024-04-25T08:30:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.877+0000 7f27c521f640 1 -- 172.21.15.60:0/1379647674 wait complete. 2024-04-25T08:30:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 Processor -- start 2024-04-25T08:30:02.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 -- start start 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 0x7f27c0150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c00d1f30 0x7f27c00cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c015dac0 0x7f27c00cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00b97d0 con 0x7f27c00d0ab0 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c015dea0 con 0x7f27c015dac0 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27c521f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c00b9950 con 0x7f27c00d1f30 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.879+0000 7f27bed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 0x7f27c0150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 0x7f27c0150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55078/0 (socket says 172.21.15.60:55078) 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 -- 172.21.15.60:0/1906098940 learned_addr learned my addr 172.21.15.60:0/1906098940 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:02.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bf59e640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c015dac0 0x7f27c00cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 -- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c00d1f30 msgr2=0x7f27c00cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c00d1f30 0x7f27c00cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 -- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c015dac0 msgr2=0x7f27c00cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c015dac0 0x7f27c00cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 -- 172.21.15.60:0/1906098940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c015fd40 con 0x7f27c00d0ab0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27bed9d640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 0x7f27c0150650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f27b000efc0 tx=0x7f27b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0018020 con 0x7f27c00d0ab0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b0014480 con 0x7f27c00d0ab0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c00caf30 con 0x7f27c00d0ab0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0014850 con 0x7f27c00d0ab0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.880+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c00cb4f0 con 0x7f27c00d0ab0 2024-04-25T08:30:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.881+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c00cbc10 con 0x7f27c00d0ab0 2024-04-25T08:30:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.882+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27b0005d10 con 0x7f27c00d0ab0 2024-04-25T08:30:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.882+0000 7f27b77fe640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a006b4d0 0x7f27a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.882+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f27b008b370 con 0x7f27c00d0ab0 2024-04-25T08:30:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.882+0000 7f27be59c640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a006b4d0 0x7f27a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.883+0000 7f27be59c640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a006b4d0 0x7f27a006d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27c01537e0 tx=0x7f27a8001e10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:02.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:02.886+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27b0022030 con 0x7f27c00d0ab0 2024-04-25T08:30:03.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.068+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f27c0073a00 con 0x7f27c00d0ab0 2024-04-25T08:30:03.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.069+0000 7f27b77fe640 1 -- 172.21.15.60:0/1906098940 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f27b0010040 con 0x7f27c00d0ab0 2024-04-25T08:30:03.066 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:03.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a006b4d0 msgr2=0x7f27a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a006b4d0 0x7f27a006d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27c01537e0 tx=0x7f27a8001e10 comp rx=0 tx=0).stop 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 msgr2=0x7f27c0150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 0x7f27c0150650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f27b000efc0 tx=0x7f27b000c490 comp rx=0 tx=0).stop 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 shutdown_connections 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a006b4d0 0x7f27a006d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c015dac0 0x7f27c00cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c00d1f30 0x7f27c00cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 --2- 172.21.15.60:0/1906098940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c00d0ab0 0x7f27c0150650 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 >> 172.21.15.60:0/1906098940 conn(0x7f27c0061c30 msgr2=0x7f27c006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.070+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 shutdown_connections 2024-04-25T08:30:03.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.071+0000 7f27c521f640 1 -- 172.21.15.60:0/1906098940 wait complete. 2024-04-25T08:30:03.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:03.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.182+0000 7f81f7a6b640 1 Processor -- start 2024-04-25T08:30:03.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.182+0000 7f81f7a6b640 1 -- start start 2024-04-25T08:30:03.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f7a6b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81f015db50 0x7f81f00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f7a6b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00cfb30 0x7f81f00cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f7a6b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f01625d0 con 0x7f81f00c7050 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f7a6b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f015df30 con 0x7f81f015db50 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f7a6b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00cc2b0 con 0x7f81f00cfb30 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f5807640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81f015db50 0x7f81f00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f5006640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f81f00c7050 0x7f81f00cd070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57138/0 (socket says 172.21.15.60:57138) 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f5006640 1 -- 172.21.15.60:0/1400888124 learned_addr learned my addr 172.21.15.60:0/1400888124 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.183+0000 7f81f6008640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00cfb30 0x7f81f00cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.184+0000 7f81f5807640 1 -- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00cfb30 msgr2=0x7f81f00cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.184+0000 7f81f5807640 1 --2- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00cfb30 0x7f81f00cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.184+0000 7f81f5807640 1 -- 172.21.15.60:0/1400888124 >> v1:172.21.15.100:6789/0 conn(0x7f81f00c7050 legacy=0x7f81f00cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.184+0000 7f81f5807640 1 -- 172.21.15.60:0/1400888124 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00cc590 con 0x7f81f015db50 2024-04-25T08:30:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.184+0000 7f81f5807640 1 --2- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f015db50 0x7f81f00c6b10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81e80097a0 tx=0x7f81e802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b99e8fa9cb6dc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f4805640 1 -- 172.21.15.60:0/1400888124 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8037070 con 0x7f81f015db50 2024-04-25T08:30:03.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f4805640 1 -- 172.21.15.60:0/1400888124 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e802ae50 con 0x7f81f015db50 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f4805640 1 -- 172.21.15.60:0/1400888124 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8032660 con 0x7f81f015db50 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 -- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f015db50 msgr2=0x7f81f00c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f015db50 0x7f81f00c6b10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81e80097a0 tx=0x7f81e802a100 comp rx=0 tx=0).stop 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 -- 172.21.15.60:0/1400888124 shutdown_connections 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00cfb30 0x7f81f00cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/1400888124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f015db50 0x7f81f00c6b10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 -- 172.21.15.60:0/1400888124 >> 172.21.15.60:0/1400888124 conn(0x7f81f00d1800 msgr2=0x7f81f00d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 -- 172.21.15.60:0/1400888124 shutdown_connections 2024-04-25T08:30:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.185+0000 7f81f7a6b640 1 -- 172.21.15.60:0/1400888124 wait complete. 2024-04-25T08:30:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.186+0000 7f81f7a6b640 1 Processor -- start 2024-04-25T08:30:03.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 -- start start 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00c7050 0x7f81f0154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f00cfb30 0x7f81f015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 0x7f81f015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00cd780 con 0x7f81f015db50 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00ccb50 con 0x7f81f00cfb30 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f7a6b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00cd900 con 0x7f81f00c7050 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f6008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 0x7f81f015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f6008640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 0x7f81f015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55092/0 (socket says 172.21.15.60:55092) 2024-04-25T08:30:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f6008640 1 -- 172.21.15.60:0/4257508988 learned_addr learned my addr 172.21.15.60:0/4257508988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f5006640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f00cfb30 0x7f81f015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.187+0000 7f81f5807640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00c7050 0x7f81f0154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f6008640 1 -- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00c7050 msgr2=0x7f81f0154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f6008640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00c7050 0x7f81f0154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f6008640 1 -- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f00cfb30 msgr2=0x7f81f015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f6008640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f00cfb30 0x7f81f015e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f6008640 1 -- 172.21.15.60:0/4257508988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0155450 con 0x7f81f015db50 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f6008640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 0x7f81f015fcb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f81e000efc0 tx=0x7f81e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0018020 con 0x7f81f015db50 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e0014480 con 0x7f81f015db50 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0014870 con 0x7f81f015db50 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f00caf60 con 0x7f81f015db50 2024-04-25T08:30:03.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.188+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f01e7c90 con 0x7f81f015db50 2024-04-25T08:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.189+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f00cb240 con 0x7f81f015db50 2024-04-25T08:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.189+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81e0005d10 con 0x7f81f015db50 2024-04-25T08:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.190+0000 7f81ee7fc640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f81dc06b4d0 0x7f81dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.190+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f81e008aec0 con 0x7f81f015db50 2024-04-25T08:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.190+0000 7f81f5807640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f81dc06b4d0 0x7f81dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.191+0000 7f81f5807640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f81dc06b4d0 0x7f81dc06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f81e8034000 tx=0x7f81e802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.194+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81f00cb240 con 0x7f81f015db50 2024-04-25T08:30:03.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.377+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81f0073990 con 0x7f81f015db50 2024-04-25T08:30:03.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.378+0000 7f81ee7fc640 1 -- 172.21.15.60:0/4257508988 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f81e0022070 con 0x7f81f015db50 2024-04-25T08:30:03.376 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:03.376 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:29:28.429733+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.380+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f81dc06b4d0 msgr2=0x7f81dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.380+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f81dc06b4d0 0x7f81dc06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f81e8034000 tx=0x7f81e802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.380+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 msgr2=0x7f81f015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.380+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 0x7f81f015fcb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f81e000efc0 tx=0x7f81e000c490 comp rx=0 tx=0).stop 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 shutdown_connections 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81f015db50 0x7f81f015fcb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f81dc06b4d0 0x7f81dc06d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81f00cfb30 0x7f81f015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 --2- 172.21.15.60:0/4257508988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81f00c7050 0x7f81f0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 >> 172.21.15.60:0/4257508988 conn(0x7f81f00d1800 msgr2=0x7f81f00cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 shutdown_connections 2024-04-25T08:30:03.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.381+0000 7f81f7a6b640 1 -- 172.21.15.60:0/4257508988 wait complete. 2024-04-25T08:30:03.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:03.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.495+0000 7f0f0056a640 1 Processor -- start 2024-04-25T08:30:03.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.496+0000 7f0f0056a640 1 -- start start 2024-04-25T08:30:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.496+0000 7f0f0056a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 0x7f0ef80d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.496+0000 7f0f0056a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef815e000 con 0x7f0ef80ced80 2024-04-25T08:30:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.496+0000 7f0f0056a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef80d1de0 con 0x7f0ef815dc20 2024-04-25T08:30:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.496+0000 7f0f0056a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef8161c60 con 0x7f0ef8161710 2024-04-25T08:30:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efdb05640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0ef80ced80 0x7f0ef80d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48008/0 (socket says 172.21.15.60:48008) 2024-04-25T08:30:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efdb05640 1 -- 172.21.15.60:0/3394205951 learned_addr learned my addr 172.21.15.60:0/3394205951 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efeb07640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 0x7f0ef80d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401720644 0 0) 0x7f0ef815e000 con 0x7f0ef80ced80 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ee4003520 con 0x7f0ef80ced80 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755817047 0 0) 0x7f0ef80d1de0 con 0x7f0ef815dc20 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef815e000 con 0x7f0ef815dc20 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740165503 0 0) 0x7f0ee4003520 con 0x7f0ef80ced80 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 msgr2=0x7f0ef80d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 0x7f0ef80d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 >> v1:172.21.15.100:6789/0 conn(0x7f0ef815dc20 legacy=0x7f0ef80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8161f40 con 0x7f0ef80ced80 2024-04-25T08:30:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f0ee8001670 con 0x7f0ef80ced80 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 >> v1:172.21.15.60:6789/0 conn(0x7f0ef80ced80 legacy=0x7f0ef80d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 0x7f0ee4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ee4004120 0x7f0ee4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef8161c60 con 0x7f0ee4004d40 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee400bb00 con 0x7f0ee4004120 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee400bd00 con 0x7f0ef8161710 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efeb07640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efe306640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 0x7f0ee4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.497+0000 7f0efdb05640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ee4004120 0x7f0ee4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efeb07640 1 -- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 msgr2=0x7f0ee4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efeb07640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef8161710 0x7f0ee4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efeb07640 1 -- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ee4004120 msgr2=0x7f0ee4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efeb07640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ee4004120 0x7f0ee4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efeb07640 1 -- 172.21.15.60:0/3394205951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4003520 con 0x7f0ee4004d40 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efeb07640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ee40051c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009fc0 tx=0x7f0ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d62635c57a999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.498+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0009c10 con 0x7f0ee4004d40 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef0036070 con 0x7f0ee4004d40 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0efd304640 1 -- 172.21.15.60:0/3394205951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0031880 con 0x7f0ee4004d40 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0f0056a640 1 -- 172.21.15.60:0/3394205951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 msgr2=0x7f0ee40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0f0056a640 1 --2- 172.21.15.60:0/3394205951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ee40051c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009fc0 tx=0x7f0ee4005830 comp rx=0 tx=0).stop 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0efeb07640 1 -- 172.21.15.60:0/3394205951 reap_dead start 2024-04-25T08:30:03.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0f0056a640 1 -- 172.21.15.60:0/3394205951 shutdown_connections 2024-04-25T08:30:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0f0056a640 1 -- 172.21.15.60:0/3394205951 >> 172.21.15.60:0/3394205951 conn(0x7f0ef80d0420 msgr2=0x7f0ef80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0f0056a640 1 -- 172.21.15.60:0/3394205951 shutdown_connections 2024-04-25T08:30:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.499+0000 7f0f0056a640 1 -- 172.21.15.60:0/3394205951 wait complete. 2024-04-25T08:30:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.500+0000 7f0f0056a640 1 Processor -- start 2024-04-25T08:30:03.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.500+0000 7f0f0056a640 1 -- start start 2024-04-25T08:30:03.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0f0056a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ef8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0f0056a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80c9ae0 0x7f0ef8158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0f0056a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef81585a0 0x7f0ef81f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0f0056a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80b5e50 con 0x7f0ee4004d40 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0f0056a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef8162500 con 0x7f0ef80c9ae0 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0f0056a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef80b5fd0 con 0x7f0ef81585a0 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ef8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ef8158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55100/0 (socket says 172.21.15.60:55100) 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 -- 172.21.15.60:0/384899678 learned_addr learned my addr 172.21.15.60:0/384899678 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:03.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efdb05640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80c9ae0 0x7f0ef8158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efeb07640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef81585a0 0x7f0ef81f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 -- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef81585a0 msgr2=0x7f0ef81f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef81585a0 0x7f0ef81f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 -- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80c9ae0 msgr2=0x7f0ef8158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80c9ae0 0x7f0ef8158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.501+0000 7f0efe306640 1 -- 172.21.15.60:0/384899678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef815ec30 con 0x7f0ee4004d40 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.502+0000 7f0efe306640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ef8158a30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400f7a0 tx=0x7f0ef400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.502+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef401a070 con 0x7f0ee4004d40 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.502+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef4005110 con 0x7f0ee4004d40 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.502+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef80ca030 con 0x7f0ee4004d40 2024-04-25T08:30:03.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.502+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4007050 con 0x7f0ee4004d40 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.502+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef815f4a0 con 0x7f0ee4004d40 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.503+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef8161450 con 0x7f0ee4004d40 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.503+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ef401d070 con 0x7f0ee4004d40 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.504+0000 7f0eeeffd640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0edc06b4d0 0x7f0edc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.504+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0ef4089bc0 con 0x7f0ee4004d40 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.504+0000 7f0efdb05640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0edc06b4d0 0x7f0edc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.504+0000 7f0efdb05640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0edc06b4d0 0x7f0edc06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80c9f60 tx=0x7f0ee8000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.508+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ef8161450 con 0x7f0ee4004d40 2024-04-25T08:30:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.687+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ef8073a00 con 0x7f0ee4004d40 2024-04-25T08:30:03.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.688+0000 7f0eeeffd640 1 -- 172.21.15.60:0/384899678 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ef4058140 con 0x7f0ee4004d40 2024-04-25T08:30:03.686 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0edc06b4d0 msgr2=0x7f0edc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0edc06b4d0 0x7f0edc06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ef80c9f60 tx=0x7f0ee8000ed0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 msgr2=0x7f0ef8158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ef8158a30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400f7a0 tx=0x7f0ef400fba0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 shutdown_connections 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ef81585a0 0x7f0ef81f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0edc06b4d0 0x7f0edc06d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80c9ae0 0x7f0ef8158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 --2- 172.21.15.60:0/384899678 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ee4004d40 0x7f0ef8158a30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 >> 172.21.15.60:0/384899678 conn(0x7f0ef80d0420 msgr2=0x7f0ef815bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 shutdown_connections 2024-04-25T08:30:03.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.690+0000 7f0f0056a640 1 -- 172.21.15.60:0/384899678 wait complete. 2024-04-25T08:30:03.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.805+0000 7f7faf44a640 1 Processor -- start 2024-04-25T08:30:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.805+0000 7f7faf44a640 1 -- start start 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7faf44a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 0x7f7fa8161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7faf44a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7fa80b9750 0x7f7fa80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7faf44a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa8162040 con 0x7f7fa8161ba0 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7faf44a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa8162600 con 0x7f7fa80b9750 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7faf44a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8162320 con 0x7f7fa815da90 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fad1e6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 0x7f7fa8161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fad1e6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 0x7f7fa8161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54510/0 (socket says 172.21.15.60:54510) 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fad1e6640 1 -- 172.21.15.60:0/997719471 learned_addr learned my addr 172.21.15.60:0/997719471 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fac9e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7fa80b9750 0x7f7fa80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fac9e5640 1 -- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 msgr2=0x7f7fa8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fac9e5640 1 --2- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 0x7f7fa8161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fac9e5640 1 -- 172.21.15.60:0/997719471 >> v1:172.21.15.100:6789/0 conn(0x7f7fa8161ba0 legacy=0x7f7fa806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fac9e5640 1 -- 172.21.15.60:0/997719471 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80d1b10 con 0x7f7fa80b9750 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7f9ffff640 1 -- 172.21.15.60:0/997719471 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746892178 0 0) 0x7f7fa8162040 con 0x7f7fa8161ba0 2024-04-25T08:30:03.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.806+0000 7f7fad1e6640 1 --2- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 0x7f7fa8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.807+0000 7f7fac9e5640 1 --2- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa80b9750 0x7f7fa80ce670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f980097a0 tx=0x7f7f9802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ca899c64fa3a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.807+0000 7f7f9ffff640 1 -- 172.21.15.60:0/997719471 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98037070 con 0x7f7fa80b9750 2024-04-25T08:30:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.807+0000 7f7f9ffff640 1 -- 172.21.15.60:0/997719471 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9802ae50 con 0x7f7fa80b9750 2024-04-25T08:30:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.807+0000 7f7f9ffff640 1 -- 172.21.15.60:0/997719471 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98032660 con 0x7f7fa80b9750 2024-04-25T08:30:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 -- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa80b9750 msgr2=0x7f7fa80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 --2- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa80b9750 0x7f7fa80ce670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f980097a0 tx=0x7f7f9802a100 comp rx=0 tx=0).stop 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 -- 172.21.15.60:0/997719471 shutdown_connections 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 --2- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa80b9750 0x7f7fa80ce670 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 --2- 172.21.15.60:0/997719471 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa815da90 0x7f7fa8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 -- 172.21.15.60:0/997719471 >> 172.21.15.60:0/997719471 conn(0x7f7fa8061a20 msgr2=0x7f7fa8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 -- 172.21.15.60:0/997719471 shutdown_connections 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.808+0000 7f7faf44a640 1 -- 172.21.15.60:0/997719471 wait complete. 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.809+0000 7f7faf44a640 1 Processor -- start 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.809+0000 7f7faf44a640 1 -- start start 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7faf44a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 0x7f7fa80cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7faf44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa8161ba0 0x7f7fa80b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7faf44a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa8160500 0x7f7fa8160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7faf44a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80d3320 con 0x7f7fa815da90 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7faf44a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815de70 con 0x7f7fa8161ba0 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7faf44a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80d34a0 con 0x7f7fa8160500 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 0x7f7fa80cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 0x7f7fa80cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55102/0 (socket says 172.21.15.60:55102) 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 -- 172.21.15.60:0/3794042267 learned_addr learned my addr 172.21.15.60:0/3794042267 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fac9e5640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa8161ba0 0x7f7fa80b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad9e7640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa8160500 0x7f7fa8160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 -- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa8160500 msgr2=0x7f7fa8160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa8160500 0x7f7fa8160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 -- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa8161ba0 msgr2=0x7f7fa80b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa8161ba0 0x7f7fa80b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 -- 172.21.15.60:0/3794042267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8160ea0 con 0x7f7fa815da90 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.810+0000 7f7fad1e6640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 0x7f7fa80cac50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9d0 tx=0x7f7fa400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.811+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4017070 con 0x7f7fa815da90 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.811+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa400ca60 con 0x7f7fa815da90 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.811+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa80c7570 con 0x7f7fa815da90 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.811+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400ce10 con 0x7f7fa815da90 2024-04-25T08:30:03.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.811+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa80c7b30 con 0x7f7fa815da90 2024-04-25T08:30:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.812+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa8073a00 con 0x7f7fa815da90 2024-04-25T08:30:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.812+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fa401a070 con 0x7f7fa815da90 2024-04-25T08:30:03.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.813+0000 7f7f9dffb640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8c06b4d0 0x7f7f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.813+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7fa408a460 con 0x7f7fa815da90 2024-04-25T08:30:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.813+0000 7f7fac9e5640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8c06b4d0 0x7f7f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.813+0000 7f7fac9e5640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8c06b4d0 0x7f7f8c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f98034000 tx=0x7f7f9802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.817+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fa40588c0 con 0x7f7fa815da90 2024-04-25T08:30:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.996+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7fa80d1590 con 0x7f7fa815da90 2024-04-25T08:30:03.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.996+0000 7f7f9dffb640 1 -- 172.21.15.60:0/3794042267 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7fa405c960 con 0x7f7fa815da90 2024-04-25T08:30:03.994 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8c06b4d0 msgr2=0x7f7f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8c06b4d0 0x7f7f8c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f98034000 tx=0x7f7f9802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:03.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 msgr2=0x7f7fa80cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 0x7f7fa80cac50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400e9d0 tx=0x7f7fa400ee90 comp rx=0 tx=0).stop 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 shutdown_connections 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa8160500 0x7f7fa8160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8c06b4d0 0x7f7f8c06d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa8161ba0 0x7f7fa80b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 --2- 172.21.15.60:0/3794042267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa815da90 0x7f7fa80cac50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 >> 172.21.15.60:0/3794042267 conn(0x7f7fa8061a20 msgr2=0x7f7fa80b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 shutdown_connections 2024-04-25T08:30:03.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:03.998+0000 7f7faf44a640 1 -- 172.21.15.60:0/3794042267 wait complete. 2024-04-25T08:30:04.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.115+0000 7f058c288640 1 Processor -- start 2024-04-25T08:30:04.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.115+0000 7f058c288640 1 -- start start 2024-04-25T08:30:04.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.116+0000 7f058c288640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f05840c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.116+0000 7f058c288640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05840c70d0 0x7f05840cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.116+0000 7f058c288640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05840c75e0 con 0x7f05840cfb40 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.116+0000 7f058c288640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05840c8680 con 0x7f05840c70d0 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.116+0000 7f058c288640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05840c78c0 con 0x7f058415bb50 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f058a024640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f05840c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f0589823640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05840c70d0 0x7f05840cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f058a825640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f05840cfb40 0x7f0584070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57178/0 (socket says 172.21.15.60:57178) 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f058a825640 1 -- 172.21.15.60:0/887657648 learned_addr learned my addr 172.21.15.60:0/887657648 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f0589823640 1 -- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 msgr2=0x7f05840c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f0589823640 1 --2- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f05840c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f0589823640 1 -- 172.21.15.60:0/887657648 >> v1:172.21.15.100:6789/0 conn(0x7f05840cfb40 legacy=0x7f0584070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f0589022640 1 -- 172.21.15.60:0/887657648 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361138693 0 0) 0x7f05840c75e0 con 0x7f05840cfb40 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f0589823640 1 -- 172.21.15.60:0/887657648 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840c7ba0 con 0x7f05840c70d0 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.117+0000 7f058a024640 1 --2- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f05840c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.118+0000 7f0589823640 1 --2- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840c70d0 0x7f05840cf600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05740097a0 tx=0x7f057402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f86203e7f47189e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.118+0000 7f0589022640 1 -- 172.21.15.60:0/887657648 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574037070 con 0x7f05840c70d0 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.118+0000 7f0589022640 1 -- 172.21.15.60:0/887657648 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574032440 con 0x7f05840c70d0 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.118+0000 7f0589022640 1 -- 172.21.15.60:0/887657648 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574032810 con 0x7f05840c70d0 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 -- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840c70d0 msgr2=0x7f05840cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 --2- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840c70d0 0x7f05840cf600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05740097a0 tx=0x7f057402a210 comp rx=0 tx=0).stop 2024-04-25T08:30:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 -- 172.21.15.60:0/887657648 shutdown_connections 2024-04-25T08:30:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 --2- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840c70d0 0x7f05840cf600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 --2- 172.21.15.60:0/887657648 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f05840c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 -- 172.21.15.60:0/887657648 >> 172.21.15.60:0/887657648 conn(0x7f058406f500 msgr2=0x7f058406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 -- 172.21.15.60:0/887657648 shutdown_connections 2024-04-25T08:30:04.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.119+0000 7f058c288640 1 -- 172.21.15.60:0/887657648 wait complete. 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.120+0000 7f058c288640 1 Processor -- start 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.120+0000 7f058c288640 1 -- start start 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 0x7f058406c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840cfb40 0x7f05841f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f058415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058415f6d0 con 0x7f05840c70d0 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05840c8160 con 0x7f05840cfb40 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058415f850 con 0x7f058415bb50 2024-04-25T08:30:04.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 0x7f058406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 0x7f058406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55110/0 (socket says 172.21.15.60:55110) 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 -- 172.21.15.60:0/2920527297 learned_addr learned my addr 172.21.15.60:0/2920527297 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f0589823640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840cfb40 0x7f05841f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a825640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f058415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 -- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 msgr2=0x7f058415d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f058415d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 -- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840cfb40 msgr2=0x7f05841f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840cfb40 0x7f05841f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 -- 172.21.15.60:0/2920527297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05841f4670 con 0x7f05840c70d0 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058a024640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 0x7f058406c770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f058000ebe0 tx=0x7f058000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580017070 con 0x7f05840c70d0 2024-04-25T08:30:04.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058000cc30 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.121+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05841f4950 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.122+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580013470 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.122+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05840b5ee0 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.122+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0584073a00 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.123+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f058001a070 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.123+0000 7f057affd640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.124+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f058008a840 con 0x7f05840c70d0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.124+0000 7f0589823640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.124+0000 7f0589823640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f056806b4d0 0x7f056806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0574034000 tx=0x7f057402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.128+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0580058cb0 con 0x7f05840c70d0 2024-04-25T08:30:04.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.312+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f05840c6eb0 con 0x7f05840c70d0 2024-04-25T08:30:04.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.312+0000 7f057affd640 1 -- 172.21.15.60:0/2920527297 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f058005cd50 con 0x7f05840c70d0 2024-04-25T08:30:04.310 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:04.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f056806b4d0 msgr2=0x7f056806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f056806b4d0 0x7f056806d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0574034000 tx=0x7f057402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:04.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 msgr2=0x7f058406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 0x7f058406c770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f058000ebe0 tx=0x7f058000c520 comp rx=0 tx=0).stop 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 shutdown_connections 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f058415bb50 0x7f058415d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05840cfb40 0x7f05841f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 --2- 172.21.15.60:0/2920527297 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05840c70d0 0x7f058406c770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 >> 172.21.15.60:0/2920527297 conn(0x7f058406f500 msgr2=0x7f05840cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 shutdown_connections 2024-04-25T08:30:04.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.314+0000 7f058c288640 1 -- 172.21.15.60:0/2920527297 wait complete. 2024-04-25T08:30:04.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:04.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.428+0000 7f89ac57e640 1 Processor -- start 2024-04-25T08:30:04.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.428+0000 7f89ac57e640 1 -- start start 2024-04-25T08:30:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89ac57e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89a415dc20 0x7f89a40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89ac57e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a406cda0 con 0x7f89a4161710 2024-04-25T08:30:04.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89ac57e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a415e000 con 0x7f89a40d32f0 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89ac57e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a40d1de0 con 0x7f89a415dc20 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89aa31a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f89a415dc20 0x7f89a40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89a9b19640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f89a4161710 0x7f89a406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48010/0 (socket says 172.21.15.60:48010) 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.429+0000 7f89a9b19640 1 -- 172.21.15.60:0/2101871893 learned_addr learned my addr 172.21.15.60:0/2101871893 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.430+0000 7f89aa31a640 1 -- 172.21.15.60:0/2101871893 >> v1:172.21.15.100:6789/0 conn(0x7f89a40d32f0 legacy=0x7f89a406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.430+0000 7f89aa31a640 1 -- 172.21.15.60:0/2101871893 >> v1:172.21.15.60:6789/0 conn(0x7f89a4161710 legacy=0x7f89a406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.430+0000 7f89aa31a640 1 -- 172.21.15.60:0/2101871893 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a4161c60 con 0x7f89a415dc20 2024-04-25T08:30:04.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.430+0000 7f89a9318640 1 -- 172.21.15.60:0/2101871893 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311786573 0 0) 0x7f89a406cda0 con 0x7f89a4161710 2024-04-25T08:30:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.430+0000 7f89aa31a640 1 --2- 172.21.15.60:0/2101871893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a415dc20 0x7f89a40ce670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f899c0097a0 tx=0x7f899c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6facab0b2371ccf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89a9318640 1 -- 172.21.15.60:0/2101871893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c037070 con 0x7f89a415dc20 2024-04-25T08:30:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89a9318640 1 -- 172.21.15.60:0/2101871893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c032440 con 0x7f89a415dc20 2024-04-25T08:30:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89a9318640 1 -- 172.21.15.60:0/2101871893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c032810 con 0x7f89a415dc20 2024-04-25T08:30:04.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 -- 172.21.15.60:0/2101871893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a415dc20 msgr2=0x7f89a40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 --2- 172.21.15.60:0/2101871893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a415dc20 0x7f89a40ce670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f899c0097a0 tx=0x7f899c02a230 comp rx=0 tx=0).stop 2024-04-25T08:30:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 -- 172.21.15.60:0/2101871893 shutdown_connections 2024-04-25T08:30:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 --2- 172.21.15.60:0/2101871893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a415dc20 0x7f89a40ce670 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 -- 172.21.15.60:0/2101871893 >> 172.21.15.60:0/2101871893 conn(0x7f89a40d0420 msgr2=0x7f89a40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 -- 172.21.15.60:0/2101871893 shutdown_connections 2024-04-25T08:30:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.431+0000 7f89ac57e640 1 -- 172.21.15.60:0/2101871893 wait complete. 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.432+0000 7f89ac57e640 1 Processor -- start 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.432+0000 7f89ac57e640 1 -- start start 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89ac57e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89a40d32f0 0x7f89a40cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89ac57e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89a415dc20 0x7f89a4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89ac57e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 0x7f89a40ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89ac57e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4162500 con 0x7f89a4161710 2024-04-25T08:30:04.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89ac57e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a4162220 con 0x7f89a40d32f0 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89ac57e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40b5e50 con 0x7f89a415dc20 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89aab1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 0x7f89a40ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89aab1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 0x7f89a40ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55124/0 (socket says 172.21.15.60:55124) 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89aab1b640 1 -- 172.21.15.60:0/1074223233 learned_addr learned my addr 172.21.15.60:0/1074223233 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89aa31a640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89a40d32f0 0x7f89a40cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.433+0000 7f89a9b19640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89a415dc20 0x7f89a4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89aab1b640 1 -- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89a415dc20 msgr2=0x7f89a4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89aab1b640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89a415dc20 0x7f89a4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89aab1b640 1 -- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89a40d32f0 msgr2=0x7f89a40cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89aab1b640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89a40d32f0 0x7f89a40cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89aab1b640 1 -- 172.21.15.60:0/1074223233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40cb520 con 0x7f89a4161710 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89aab1b640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 0x7f89a40ac120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f899400c760 tx=0x7f899400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994003b70 con 0x7f89a4161710 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8994003d10 con 0x7f89a4161710 2024-04-25T08:30:04.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a40cb800 con 0x7f89a4161710 2024-04-25T08:30:04.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994005310 con 0x7f89a4161710 2024-04-25T08:30:04.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.434+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a40cbdc0 con 0x7f89a4161710 2024-04-25T08:30:04.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.435+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a4073a00 con 0x7f89a4161710 2024-04-25T08:30:04.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.436+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89940054b0 con 0x7f89a4161710 2024-04-25T08:30:04.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.436+0000 7f89a2ffd640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.436+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f899408a2b0 con 0x7f89a4161710 2024-04-25T08:30:04.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.436+0000 7f89aa31a640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.437+0000 7f89aa31a640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f899006b4d0 0x7f899006d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f899c034000 tx=0x7f899c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.440+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89940586e0 con 0x7f89a4161710 2024-04-25T08:30:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.619+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f89a40d16b0 con 0x7f89a4161710 2024-04-25T08:30:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.619+0000 7f89a2ffd640 1 -- 172.21.15.60:0/1074223233 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f899405c780 con 0x7f89a4161710 2024-04-25T08:30:04.617 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f899006b4d0 msgr2=0x7f899006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f899006b4d0 0x7f899006d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f899c034000 tx=0x7f899c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 msgr2=0x7f89a40ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 0x7f89a40ac120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f899400c760 tx=0x7f899400cc20 comp rx=0 tx=0).stop 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 shutdown_connections 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f89a4161710 0x7f89a40ac120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f89a415dc20 0x7f89a4152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 --2- 172.21.15.60:0/1074223233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89a40d32f0 0x7f89a40cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 >> 172.21.15.60:0/1074223233 conn(0x7f89a40d0420 msgr2=0x7f89a40c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 shutdown_connections 2024-04-25T08:30:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.621+0000 7f89ac57e640 1 -- 172.21.15.60:0/1074223233 wait complete. 2024-04-25T08:30:04.640 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T08:30:04.640 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:30:04.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:04.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.744+0000 7f8a4084d640 1 Processor -- start 2024-04-25T08:30:04.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.744+0000 7f8a4084d640 1 -- start start 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a4084d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8a3c0b5c30 0x7f8a3c168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a4084d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c0ceee0 con 0x7f8a3c0b6cb0 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a4084d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c0d1d10 con 0x7f8a3c15db70 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a4084d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0d3800 con 0x7f8a3c0b5c30 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a31d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8a3c0b6cb0 0x7f8a3c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48022/0 (socket says 172.21.15.60:48022) 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a31d9b640 1 -- 172.21.15.60:0/1542980835 learned_addr learned my addr 172.21.15.60:0/1542980835 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875384302 0 0) 0x7f8a3c0ceee0 con 0x7f8a3c0b6cb0 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a28003520 con 0x7f8a3c0b6cb0 2024-04-25T08:30:04.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a3ad9d640 1 --2- 172.21.15.60:0/1542980835 >> v2:172.21.15.100:3300/0 conn(0x7f8a3c0b5c30 0x7f8a3c168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991305035 0 0) 0x7f8a28003520 con 0x7f8a3c0b6cb0 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 >> v2:172.21.15.100:3300/0 conn(0x7f8a3c0b5c30 msgr2=0x7f8a3c168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 --2- 172.21.15.60:0/1542980835 >> v2:172.21.15.100:3300/0 conn(0x7f8a3c0b5c30 0x7f8a3c168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 >> v1:172.21.15.100:6789/0 conn(0x7f8a3c15db70 legacy=0x7f8a3c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0d1ff0 con 0x7f8a3c0b6cb0 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f8a24001670 con 0x7f8a3c0b6cb0 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 >> v1:172.21.15.60:6789/0 conn(0x7f8a3c0b6cb0 legacy=0x7f8a3c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a28003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a280051c0 0x7f8a2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0d3800 con 0x7f8a28003a00 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2800c0f0 con 0x7f8a28004800 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2800c2f0 con 0x7f8a280051c0 2024-04-25T08:30:04.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.745+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a28003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a31d9b640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3ad9d640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a280051c0 0x7f8a2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3a59c640 1 -- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a280051c0 msgr2=0x7f8a2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a280051c0 0x7f8a2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3a59c640 1 -- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a28004800 msgr2=0x7f8a28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3ad9d640 1 -- 172.21.15.60:0/1542980835 reap_dead start 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3a59c640 1 -- 172.21.15.60:0/1542980835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a28003520 con 0x7f8a28003a00 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.746+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a28003e40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c001750 tx=0x7f8a28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2abc1a4c9e5beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c001370 con 0x7f8a28003a00 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c0052d0 con 0x7f8a28003a00 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a39d9b640 1 -- 172.21.15.60:0/1542980835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c02abe0 con 0x7f8a28003a00 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 -- 172.21.15.60:0/1542980835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 msgr2=0x7f8a28003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a28003e40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c001750 tx=0x7f8a28005830 comp rx=0 tx=0).stop 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 -- 172.21.15.60:0/1542980835 shutdown_connections 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 --2- 172.21.15.60:0/1542980835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a28003e40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 -- 172.21.15.60:0/1542980835 >> 172.21.15.60:0/1542980835 conn(0x7f8a3c0d0520 msgr2=0x7f8a3c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:04.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 -- 172.21.15.60:0/1542980835 shutdown_connections 2024-04-25T08:30:04.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.747+0000 7f8a4084d640 1 -- 172.21.15.60:0/1542980835 wait complete. 2024-04-25T08:30:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 Processor -- start 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 -- start start 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a3c0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c1e6450 0x7f8a3c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a3c1f3de0 0x7f8a3c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0ce530 con 0x7f8a28003a00 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c15df50 con 0x7f8a3c1e6450 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.749+0000 7f8a4084d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c0cf1c0 con 0x7f8a3c1f3de0 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a3c0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a3c0b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55134/0 (socket says 172.21.15.60:55134) 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 -- 172.21.15.60:0/987253547 learned_addr learned my addr 172.21.15.60:0/987253547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3ad9d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a3c1f3de0 0x7f8a3c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a31d9b640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c1e6450 0x7f8a3c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 -- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a3c1f3de0 msgr2=0x7f8a3c15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a3c1f3de0 0x7f8a3c15c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 -- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c1e6450 msgr2=0x7f8a3c158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c1e6450 0x7f8a3c158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 -- 172.21.15.60:0/987253547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0cc870 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a31d9b640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c1e6450 0x7f8a3c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3ad9d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a3c1f3de0 0x7f8a3c15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a3a59c640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a3c0b4b30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02a590 tx=0x7f8a2c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c037040 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c040070 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c001370 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c0ccb50 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.750+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c0cd110 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.751+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c073990 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.752+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a2c0052d0 con 0x7f8a28003a00 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.752+0000 7f8a337fe640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a1c06b4d0 0x7f8a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.752+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8a2c0aa340 con 0x7f8a28003a00 2024-04-25T08:30:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.752+0000 7f8a31d9b640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a1c06b4d0 0x7f8a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.753+0000 7f8a31d9b640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a1c06b4d0 0x7f8a1c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a24000c00 tx=0x7f8a24009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.757+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a2c078d30 con 0x7f8a28003a00 2024-04-25T08:30:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.939+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a3c0d1820 con 0x7f8a28003a00 2024-04-25T08:30:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.940+0000 7f8a337fe640 1 -- 172.21.15.60:0/987253547 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18889 (secure 0 0 0) 0x7f8a2c078630 con 0x7f8a28003a00 2024-04-25T08:30:04.938 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:04.938 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:04.179588+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a1c06b4d0 msgr2=0x7f8a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a1c06b4d0 0x7f8a1c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a24000c00 tx=0x7f8a24009040 comp rx=0 tx=0).stop 2024-04-25T08:30:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 msgr2=0x7f8a3c0b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a3c0b4b30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02a590 tx=0x7f8a2c001120 comp rx=0 tx=0).stop 2024-04-25T08:30:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 shutdown_connections 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8a3c1f3de0 0x7f8a3c15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8a1c06b4d0 0x7f8a1c06d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a3c1e6450 0x7f8a3c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 --2- 172.21.15.60:0/987253547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8a28003a00 0x7f8a3c0b4b30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 >> 172.21.15.60:0/987253547 conn(0x7f8a3c0d0520 msgr2=0x7f8a3c0c8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 shutdown_connections 2024-04-25T08:30:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:04.942+0000 7f8a4084d640 1 -- 172.21.15.60:0/987253547 wait complete. 2024-04-25T08:30:04.951 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:04.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:05.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.057+0000 7efe5039b640 1 Processor -- start 2024-04-25T08:30:05.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.058+0000 7efe5039b640 1 -- start start 2024-04-25T08:30:05.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.058+0000 7efe5039b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efe4815da90 0x7efe48161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe5039b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe48162040 con 0x7efe480b9750 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe5039b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe48162320 con 0x7efe48161ba0 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe5039b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe48162600 con 0x7efe4815da90 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe4e137640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efe4815da90 0x7efe48161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe4d936640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efe480b9750 0x7efe480ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48032/0 (socket says 172.21.15.60:48032) 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe4d936640 1 -- 172.21.15.60:0/2735416553 learned_addr learned my addr 172.21.15.60:0/2735416553 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe4e137640 1 -- 172.21.15.60:0/2735416553 >> v1:172.21.15.100:6789/0 conn(0x7efe48161ba0 legacy=0x7efe480d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe4e137640 1 -- 172.21.15.60:0/2735416553 >> v1:172.21.15.60:6789/0 conn(0x7efe480b9750 legacy=0x7efe480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.059+0000 7efe4e137640 1 -- 172.21.15.60:0/2735416553 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe480d1b10 con 0x7efe4815da90 2024-04-25T08:30:05.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe4e137640 1 --2- 172.21.15.60:0/2735416553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815da90 0x7efe48161660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efe440097a0 tx=0x7efe4402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef2eb6be0b104ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe4d135640 1 -- 172.21.15.60:0/2735416553 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44037070 con 0x7efe4815da90 2024-04-25T08:30:05.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe4d135640 1 -- 172.21.15.60:0/2735416553 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4402ae50 con 0x7efe4815da90 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe4d135640 1 -- 172.21.15.60:0/2735416553 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44032660 con 0x7efe4815da90 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe5039b640 1 -- 172.21.15.60:0/2735416553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815da90 msgr2=0x7efe48161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe5039b640 1 --2- 172.21.15.60:0/2735416553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815da90 0x7efe48161660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efe440097a0 tx=0x7efe4402a140 comp rx=0 tx=0).stop 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe5039b640 1 -- 172.21.15.60:0/2735416553 shutdown_connections 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe5039b640 1 --2- 172.21.15.60:0/2735416553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815da90 0x7efe48161660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.060+0000 7efe5039b640 1 -- 172.21.15.60:0/2735416553 >> 172.21.15.60:0/2735416553 conn(0x7efe48061a20 msgr2=0x7efe48061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.061+0000 7efe5039b640 1 -- 172.21.15.60:0/2735416553 shutdown_connections 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.061+0000 7efe5039b640 1 -- 172.21.15.60:0/2735416553 wait complete. 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 Processor -- start 2024-04-25T08:30:05.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 -- start start 2024-04-25T08:30:05.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4815da90 0x7efe481567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe48161ba0 0x7efe4815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 0x7efe480cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480ced80 con 0x7efe4815a5e0 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4815de70 con 0x7efe4815da90 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.062+0000 7efe5039b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe480cef00 con 0x7efe48161ba0 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 0x7efe480cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 0x7efe480cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55150/0 (socket says 172.21.15.60:55150) 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 -- 172.21.15.60:0/1192999410 learned_addr learned my addr 172.21.15.60:0/1192999410 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e137640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4815da90 0x7efe481567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4d936640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe48161ba0 0x7efe4815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 -- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe48161ba0 msgr2=0x7efe4815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe48161ba0 0x7efe4815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 -- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4815da90 msgr2=0x7efe481567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4815da90 0x7efe481567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 -- 172.21.15.60:0/1192999410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe481f7440 con 0x7efe4815a5e0 2024-04-25T08:30:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe4e938640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 0x7efe480cd3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b970 tx=0x7efe4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40019070 con 0x7efe4815a5e0 2024-04-25T08:30:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4000cb40 con 0x7efe4815a5e0 2024-04-25T08:30:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40014440 con 0x7efe4815a5e0 2024-04-25T08:30:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe481f7720 con 0x7efe4815a5e0 2024-04-25T08:30:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.063+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe481f7ce0 con 0x7efe4815a5e0 2024-04-25T08:30:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.064+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe48073a00 con 0x7efe4815a5e0 2024-04-25T08:30:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.065+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe4001c070 con 0x7efe4815a5e0 2024-04-25T08:30:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.065+0000 7efe3effd640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.065+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efe4008a230 con 0x7efe4815a5e0 2024-04-25T08:30:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.065+0000 7efe4e137640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.066+0000 7efe4e137640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe44034000 tx=0x7efe4402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.070+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe40058cb0 con 0x7efe4815a5e0 2024-04-25T08:30:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.255+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe480d1590 con 0x7efe4815a5e0 2024-04-25T08:30:05.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.256+0000 7efe3effd640 1 -- 172.21.15.60:0/1192999410 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe400585b0 con 0x7efe4815a5e0 2024-04-25T08:30:05.253 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 msgr2=0x7efe2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe44034000 tx=0x7efe4402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 msgr2=0x7efe480cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 0x7efe480cd3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b970 tx=0x7efe4000be30 comp rx=0 tx=0).stop 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 shutdown_connections 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4815a5e0 0x7efe480cd3c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe48161ba0 0x7efe4815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 --2- 172.21.15.60:0/1192999410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4815da90 0x7efe481567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.257+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 >> 172.21.15.60:0/1192999410 conn(0x7efe48061a20 msgr2=0x7efe480b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.258+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 shutdown_connections 2024-04-25T08:30:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.258+0000 7efe5039b640 1 -- 172.21.15.60:0/1192999410 wait complete. 2024-04-25T08:30:05.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.371+0000 7f1637730640 1 Processor -- start 2024-04-25T08:30:05.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 -- start start 2024-04-25T08:30:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1630161240 0x7f16300b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1630161890 0x7f16300ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16300b54a0 0x7f16300b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16300b7120 con 0x7f1630161890 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1630161620 con 0x7f1630161240 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1637730640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1630161e00 con 0x7f16300b54a0 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1634ccb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1630161890 0x7f16300ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.372+0000 7f1634ccb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1630161890 0x7f16300ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55158/0 (socket says 172.21.15.60:55158) 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1634ccb640 1 -- 172.21.15.60:0/3537113690 learned_addr learned my addr 172.21.15.60:0/3537113690 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f16354cc640 1 --2- 172.21.15.60:0/3537113690 >> v2:172.21.15.100:3300/0 conn(0x7f1630161240 0x7f16300b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1635ccd640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16300b54a0 0x7f16300b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1634ccb640 1 -- 172.21.15.60:0/3537113690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16300b54a0 msgr2=0x7f16300b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1634ccb640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16300b54a0 0x7f16300b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1634ccb640 1 -- 172.21.15.60:0/3537113690 >> v2:172.21.15.100:3300/0 conn(0x7f1630161240 msgr2=0x7f16300b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1634ccb640 1 --2- 172.21.15.60:0/3537113690 >> v2:172.21.15.100:3300/0 conn(0x7f1630161240 0x7f16300b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1634ccb640 1 -- 172.21.15.60:0/3537113690 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16301620e0 con 0x7f1630161890 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.373+0000 7f1635ccd640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16300b54a0 0x7f16300b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1634ccb640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300ceac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16300d0400 tx=0x7f162402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07717a75389fd0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f162ffff640 1 -- 172.21.15.60:0/3537113690 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624037070 con 0x7f1630161890 2024-04-25T08:30:05.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f162ffff640 1 -- 172.21.15.60:0/3537113690 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624032440 con 0x7f1630161890 2024-04-25T08:30:05.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f162ffff640 1 -- 172.21.15.60:0/3537113690 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624032810 con 0x7f1630161890 2024-04-25T08:30:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 -- 172.21.15.60:0/3537113690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 msgr2=0x7f16300ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300ceac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16300d0400 tx=0x7f162402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 -- 172.21.15.60:0/3537113690 shutdown_connections 2024-04-25T08:30:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16300b54a0 0x7f16300b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 --2- 172.21.15.60:0/3537113690 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300ceac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 --2- 172.21.15.60:0/3537113690 >> v2:172.21.15.100:3300/0 conn(0x7f1630161240 0x7f16300b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 -- 172.21.15.60:0/3537113690 >> 172.21.15.60:0/3537113690 conn(0x7f16300d04a0 msgr2=0x7f16300d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 -- 172.21.15.60:0/3537113690 shutdown_connections 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.374+0000 7f1637730640 1 -- 172.21.15.60:0/3537113690 wait complete. 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.375+0000 7f1637730640 1 Processor -- start 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 -- start start 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16300b54a0 0x7f16300ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1630161240 0x7f163015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300d15f0 con 0x7f1630161890 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16301626a0 con 0x7f16300b54a0 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1637730640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300cf000 con 0x7f1630161240 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1635ccd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1635ccd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55170/0 (socket says 172.21.15.60:55170) 2024-04-25T08:30:05.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.376+0000 7f1635ccd640 1 -- 172.21.15.60:0/2755710557 learned_addr learned my addr 172.21.15.60:0/2755710557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f16354cc640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16300b54a0 0x7f16300ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1634ccb640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1630161240 0x7f163015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1635ccd640 1 -- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1630161240 msgr2=0x7f163015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1635ccd640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1630161240 0x7f163015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1635ccd640 1 -- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16300b54a0 msgr2=0x7f16300ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1635ccd640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16300b54a0 0x7f16300ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1635ccd640 1 -- 172.21.15.60:0/2755710557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16301f4bd0 con 0x7f1630161890 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f16354cc640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16300b54a0 0x7f16300ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1635ccd640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300b17c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1620007e10 tx=0x7f162000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162001a070 con 0x7f1630161890 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620003b90 con 0x7f1630161890 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163015bdc0 con 0x7f1630161890 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620004030 con 0x7f1630161890 2024-04-25T08:30:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.377+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16301f4330 con 0x7f1630161890 2024-04-25T08:30:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.378+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630073a00 con 0x7f1630161890 2024-04-25T08:30:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.379+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f162001d070 con 0x7f1630161890 2024-04-25T08:30:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.379+0000 7f162dffb640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f161c06b4d0 0x7f161c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.379+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f162008a830 con 0x7f1630161890 2024-04-25T08:30:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.380+0000 7f16354cc640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f161c06b4d0 0x7f161c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.380+0000 7f16354cc640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f161c06b4d0 0x7f161c06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16300cb770 tx=0x7f1628008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.384+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1620010070 con 0x7f1630161890 2024-04-25T08:30:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.572+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f163015d220 con 0x7f1630161890 2024-04-25T08:30:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.573+0000 7f162dffb640 1 -- 172.21.15.60:0/2755710557 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1620059340 con 0x7f1630161890 2024-04-25T08:30:05.570 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f161c06b4d0 msgr2=0x7f161c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f161c06b4d0 0x7f161c06d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16300cb770 tx=0x7f1628008040 comp rx=0 tx=0).stop 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 msgr2=0x7f16300b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300b17c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1620007e10 tx=0x7f162000d4a0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 shutdown_connections 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1630161890 0x7f16300b17c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1630161240 0x7f163015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f161c06b4d0 0x7f161c06d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 --2- 172.21.15.60:0/2755710557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16300b54a0 0x7f16300ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 >> 172.21.15.60:0/2755710557 conn(0x7f16300d04a0 msgr2=0x7f16300b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 shutdown_connections 2024-04-25T08:30:05.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.575+0000 7f1637730640 1 -- 172.21.15.60:0/2755710557 wait complete. 2024-04-25T08:30:05.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.689+0000 7f0b595c1640 1 Processor -- start 2024-04-25T08:30:05.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.689+0000 7f0b595c1640 1 -- start start 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b595c1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b5415bb20 0x7f0b540cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b595c1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b540ccdd0 0x7f0b540cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b595c1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b540cb190 con 0x7f0b540cdcd0 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b595c1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540cb470 con 0x7f0b540ccdd0 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b595c1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cb750 con 0x7f0b5415bb20 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b527fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b540ccdd0 0x7f0b540cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b52ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b5415bb20 0x7f0b540cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b537fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0b540cdcd0 0x7f0b540ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57222/0 (socket says 172.21.15.60:57222) 2024-04-25T08:30:05.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.690+0000 7f0b537fe640 1 -- 172.21.15.60:0/3701364191 learned_addr learned my addr 172.21.15.60:0/3701364191 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.691+0000 7f0b52ffd640 1 -- 172.21.15.60:0/3701364191 >> v2:172.21.15.100:3300/0 conn(0x7f0b540ccdd0 msgr2=0x7f0b540cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.691+0000 7f0b52ffd640 1 --2- 172.21.15.60:0/3701364191 >> v2:172.21.15.100:3300/0 conn(0x7f0b540ccdd0 0x7f0b540cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.691+0000 7f0b52ffd640 1 -- 172.21.15.60:0/3701364191 >> v1:172.21.15.100:6789/0 conn(0x7f0b540cdcd0 legacy=0x7f0b540ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.691+0000 7f0b52ffd640 1 -- 172.21.15.60:0/3701364191 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540cba30 con 0x7f0b5415bb20 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.691+0000 7f0b527fc640 1 --2- 172.21.15.60:0/3701364191 >> v2:172.21.15.100:3300/0 conn(0x7f0b540ccdd0 0x7f0b540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b52ffd640 1 --2- 172.21.15.60:0/3701364191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b5415bb20 0x7f0b540cc230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0097a0 tx=0x7f0b4c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce678d9aa768a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b51ffb640 1 -- 172.21.15.60:0/3701364191 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c037070 con 0x7f0b5415bb20 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b51ffb640 1 -- 172.21.15.60:0/3701364191 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c02ae50 con 0x7f0b5415bb20 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b51ffb640 1 -- 172.21.15.60:0/3701364191 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c032660 con 0x7f0b5415bb20 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 -- 172.21.15.60:0/3701364191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b5415bb20 msgr2=0x7f0b540cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 --2- 172.21.15.60:0/3701364191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b5415bb20 0x7f0b540cc230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0097a0 tx=0x7f0b4c02a100 comp rx=0 tx=0).stop 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 -- 172.21.15.60:0/3701364191 shutdown_connections 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 --2- 172.21.15.60:0/3701364191 >> v2:172.21.15.100:3300/0 conn(0x7f0b540ccdd0 0x7f0b540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 --2- 172.21.15.60:0/3701364191 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b5415bb20 0x7f0b540cc230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 -- 172.21.15.60:0/3701364191 >> 172.21.15.60:0/3701364191 conn(0x7f0b540b2390 msgr2=0x7f0b540b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 -- 172.21.15.60:0/3701364191 shutdown_connections 2024-04-25T08:30:05.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.692+0000 7f0b595c1640 1 -- 172.21.15.60:0/3701364191 wait complete. 2024-04-25T08:30:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.694+0000 7f0b595c1640 1 Processor -- start 2024-04-25T08:30:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.694+0000 7f0b595c1640 1 -- start start 2024-04-25T08:30:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.694+0000 7f0b595c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b540ccdd0 0x7f0b54152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.694+0000 7f0b595c1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b540cdcd0 0x7f0b54158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.694+0000 7f0b595c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 0x7f0b540b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b595c1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5415bf00 con 0x7f0b5415bb20 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b595c1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cc770 con 0x7f0b540ccdd0 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b595c1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5415c080 con 0x7f0b540cdcd0 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 0x7f0b540b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 0x7f0b540b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55176/0 (socket says 172.21.15.60:55176) 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 -- 172.21.15.60:0/1696630119 learned_addr learned my addr 172.21.15.60:0/1696630119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b52ffd640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b540ccdd0 0x7f0b54152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b527fc640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b540cdcd0 0x7f0b54158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 -- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b540cdcd0 msgr2=0x7f0b54158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b540cdcd0 0x7f0b54158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 -- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b540ccdd0 msgr2=0x7f0b54152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b540ccdd0 0x7f0b54152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 -- 172.21.15.60:0/1696630119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5415a5b0 con 0x7f0b5415bb20 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b537fe640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 0x7f0b540b3a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400cbc0 tx=0x7f0b4400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44017070 con 0x7f0b5415bb20 2024-04-25T08:30:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4400ad40 con 0x7f0b5415bb20 2024-04-25T08:30:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5415a890 con 0x7f0b5415bb20 2024-04-25T08:30:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44012600 con 0x7f0b5415bb20 2024-04-25T08:30:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.695+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b540c58e0 con 0x7f0b5415bb20 2024-04-25T08:30:05.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.696+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54073a00 con 0x7f0b5415bb20 2024-04-25T08:30:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.697+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b4401a070 con 0x7f0b5415bb20 2024-04-25T08:30:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.697+0000 7f0b4b7fe640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3406b500 0x7f0b3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.698+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0b4408a250 con 0x7f0b5415bb20 2024-04-25T08:30:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.698+0000 7f0b52ffd640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3406b500 0x7f0b3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:05.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.698+0000 7f0b52ffd640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3406b500 0x7f0b3406d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c034000 tx=0x7f0b4c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:05.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.702+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b44058690 con 0x7f0b5415bb20 2024-04-25T08:30:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.884+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b540ada60 con 0x7f0b5415bb20 2024-04-25T08:30:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.885+0000 7f0b4b7fe640 1 -- 172.21.15.60:0/1696630119 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b4405c730 con 0x7f0b5415bb20 2024-04-25T08:30:05.883 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:05.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.886+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3406b500 msgr2=0x7f0b3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.886+0000 7f0b595c1640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3406b500 0x7f0b3406d9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c034000 tx=0x7f0b4c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.886+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 msgr2=0x7f0b540b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.886+0000 7f0b595c1640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 0x7f0b540b3a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400cbc0 tx=0x7f0b4400a430 comp rx=0 tx=0).stop 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 shutdown_connections 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b5415bb20 0x7f0b540b3a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0b540cdcd0 0x7f0b54158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0b3406b500 0x7f0b3406d9c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 --2- 172.21.15.60:0/1696630119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b540ccdd0 0x7f0b54152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 >> 172.21.15.60:0/1696630119 conn(0x7f0b540b2390 msgr2=0x7f0b540b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 shutdown_connections 2024-04-25T08:30:05.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:05.887+0000 7f0b595c1640 1 -- 172.21.15.60:0/1696630119 wait complete. 2024-04-25T08:30:05.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:05.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.001+0000 7f8fb197e640 1 Processor -- start 2024-04-25T08:30:05.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.001+0000 7f8fb197e640 1 -- start start 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fb197e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 0x7f8fac161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fb197e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8fac161ba0 0x7f8fac0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fb197e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac162040 con 0x7f8fac0b9750 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fb197e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fac162600 con 0x7f8fac161ba0 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fb197e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac162320 con 0x7f8fac15da90 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 0x7f8fac161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fab7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8fac161ba0 0x7f8fac0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 0x7f8fac161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54652/0 (socket says 172.21.15.60:54652) 2024-04-25T08:30:06.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 -- 172.21.15.60:0/1368548213 learned_addr learned my addr 172.21.15.60:0/1368548213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:06.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 -- 172.21.15.60:0/1368548213 >> v2:172.21.15.100:3300/0 conn(0x7f8fac161ba0 msgr2=0x7f8fac0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:06.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 --2- 172.21.15.60:0/1368548213 >> v2:172.21.15.100:3300/0 conn(0x7f8fac161ba0 0x7f8fac0d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 -- 172.21.15.60:0/1368548213 >> v1:172.21.15.100:6789/0 conn(0x7f8fac0b9750 legacy=0x7f8fac0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8faaffd640 1 -- 172.21.15.60:0/1368548213 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0d1b10 con 0x7f8fac15da90 2024-04-25T08:30:06.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.002+0000 7f8fab7fe640 1 --2- 172.21.15.60:0/1368548213 >> v2:172.21.15.100:3300/0 conn(0x7f8fac161ba0 0x7f8fac0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.003+0000 7f8faaffd640 1 --2- 172.21.15.60:0/1368548213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 0x7f8fac161660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009870 tx=0x7f8f9c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fff9f222e675ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.003+0000 7f8fa9ffb640 1 -- 172.21.15.60:0/1368548213 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c037070 con 0x7f8fac15da90 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.003+0000 7f8fa9ffb640 1 -- 172.21.15.60:0/1368548213 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c032440 con 0x7f8fac15da90 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.003+0000 7f8fa9ffb640 1 -- 172.21.15.60:0/1368548213 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c032810 con 0x7f8fac15da90 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 -- 172.21.15.60:0/1368548213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 msgr2=0x7f8fac161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1368548213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 0x7f8fac161660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009870 tx=0x7f8f9c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 -- 172.21.15.60:0/1368548213 shutdown_connections 2024-04-25T08:30:06.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1368548213 >> v2:172.21.15.100:3300/0 conn(0x7f8fac161ba0 0x7f8fac0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1368548213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15da90 0x7f8fac161660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 -- 172.21.15.60:0/1368548213 >> 172.21.15.60:0/1368548213 conn(0x7f8fac061a20 msgr2=0x7f8fac061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:06.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 -- 172.21.15.60:0/1368548213 shutdown_connections 2024-04-25T08:30:06.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.004+0000 7f8fb197e640 1 -- 172.21.15.60:0/1368548213 wait complete. 2024-04-25T08:30:06.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.005+0000 7f8fb197e640 1 Processor -- start 2024-04-25T08:30:06.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.005+0000 7f8fb197e640 1 -- start start 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fb197e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 0x7f8fac1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fb197e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fac161ba0 0x7f8fac15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fb197e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15e030 0x7f8fac0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fb197e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0ced80 con 0x7f8fac15da90 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fb197e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac15de70 con 0x7f8fac161ba0 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fb197e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0cef00 con 0x7f8fac15e030 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8faaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 0x7f8fac1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8faaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 0x7f8fac1608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55186/0 (socket says 172.21.15.60:55186) 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8faaffd640 1 -- 172.21.15.60:0/1337568916 learned_addr learned my addr 172.21.15.60:0/1337568916 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:06.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8faa7fc640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fac161ba0 0x7f8fac15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8fab7fe640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15e030 0x7f8fac0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8faaffd640 1 -- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15e030 msgr2=0x7f8fac0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.006+0000 7f8faaffd640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15e030 0x7f8fac0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8faaffd640 1 -- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fac161ba0 msgr2=0x7f8fac15c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8faaffd640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fac161ba0 0x7f8fac15c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8faaffd640 1 -- 172.21.15.60:0/1337568916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac15c950 con 0x7f8fac15da90 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8faaffd640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 0x7f8fac1608c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c000c00 tx=0x7f8f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c037040 con 0x7f8fac15da90 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c042070 con 0x7f8fac15da90 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac0cd570 con 0x7f8fac15da90 2024-04-25T08:30:06.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c030880 con 0x7f8fac15da90 2024-04-25T08:30:06.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.007+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac0cdb30 con 0x7f8fac15da90 2024-04-25T08:30:06.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.008+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac073a00 con 0x7f8fac15da90 2024-04-25T08:30:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.008+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f9c02ad10 con 0x7f8fac15da90 2024-04-25T08:30:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.009+0000 7f8f93fff640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f9806b4d0 0x7f8f9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.009+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8f9c0aac30 con 0x7f8fac15da90 2024-04-25T08:30:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.009+0000 7f8faa7fc640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f9806b4d0 0x7f8f9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.009+0000 7f8faa7fc640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f9806b4d0 0x7f8f9806d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007fd0 tx=0x7f8fa0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:06.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.013+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f9c079630 con 0x7f8fac15da90 2024-04-25T08:30:06.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.195+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8fac061e00 con 0x7f8fac15da90 2024-04-25T08:30:06.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.195+0000 7f8f93fff640 1 -- 172.21.15.60:0/1337568916 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8f9c078f30 con 0x7f8fac15da90 2024-04-25T08:30:06.193 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f9806b4d0 msgr2=0x7f8f9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f9806b4d0 0x7f8f9806d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007fd0 tx=0x7f8fa0009040 comp rx=0 tx=0).stop 2024-04-25T08:30:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 msgr2=0x7f8fac1608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 0x7f8fac1608c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c000c00 tx=0x7f8f9c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 shutdown_connections 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8fac15e030 0x7f8fac0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f9806b4d0 0x7f8f9806d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fac161ba0 0x7f8fac15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 --2- 172.21.15.60:0/1337568916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8fac15da90 0x7f8fac1608c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.197+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 >> 172.21.15.60:0/1337568916 conn(0x7f8fac061a20 msgr2=0x7f8fac0d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.198+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 shutdown_connections 2024-04-25T08:30:06.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.198+0000 7f8fb197e640 1 -- 172.21.15.60:0/1337568916 wait complete. 2024-04-25T08:30:06.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T08:30:06.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.310+0000 7f3fb299e640 1 Processor -- start 2024-04-25T08:30:06.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.310+0000 7f3fb299e640 1 -- start start 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fb299e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fac15db70 0x7f3fac162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fb299e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac162690 con 0x7f3fac0b5c30 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fb299e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac0ceee0 con 0x7f3fac0b6cb0 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fb299e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac0d1d10 con 0x7f3fac15db70 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fabfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fac15db70 0x7f3fac162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fb0f3b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3fac0b5c30 0x7f3fac0d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48044/0 (socket says 172.21.15.60:48044) 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fb0f3b640 1 -- 172.21.15.60:0/184866962 learned_addr learned my addr 172.21.15.60:0/184866962 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fabfff640 1 -- 172.21.15.60:0/184866962 >> v1:172.21.15.100:6789/0 conn(0x7f3fac0b6cb0 legacy=0x7f3fac0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fabfff640 1 -- 172.21.15.60:0/184866962 >> v1:172.21.15.60:6789/0 conn(0x7f3fac0b5c30 legacy=0x7f3fac0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.311+0000 7f3fabfff640 1 -- 172.21.15.60:0/184866962 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0d1ff0 con 0x7f3fac15db70 2024-04-25T08:30:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.312+0000 7f3fabfff640 1 --2- 172.21.15.60:0/184866962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac162150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd2769fc78dffb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:06.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.312+0000 7f3faaffd640 1 -- 172.21.15.60:0/184866962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0037070 con 0x7f3fac15db70 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.312+0000 7f3faaffd640 1 -- 172.21.15.60:0/184866962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0032440 con 0x7f3fac15db70 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.312+0000 7f3faaffd640 1 -- 172.21.15.60:0/184866962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0032810 con 0x7f3fac15db70 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.312+0000 7f3fb299e640 1 -- 172.21.15.60:0/184866962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 msgr2=0x7f3fac162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.312+0000 7f3fb299e640 1 --2- 172.21.15.60:0/184866962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac162150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00097a0 tx=0x7f3fa002a230 comp rx=0 tx=0).stop 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.313+0000 7f3fb299e640 1 -- 172.21.15.60:0/184866962 shutdown_connections 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.313+0000 7f3fb299e640 1 --2- 172.21.15.60:0/184866962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac162150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.313+0000 7f3fb299e640 1 -- 172.21.15.60:0/184866962 >> 172.21.15.60:0/184866962 conn(0x7f3fac0d0520 msgr2=0x7f3fac0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.313+0000 7f3fb299e640 1 -- 172.21.15.60:0/184866962 shutdown_connections 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.313+0000 7f3fb299e640 1 -- 172.21.15.60:0/184866962 wait complete. 2024-04-25T08:30:06.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.314+0000 7f3fb299e640 1 Processor -- start 2024-04-25T08:30:06.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.314+0000 7f3fb299e640 1 -- start start 2024-04-25T08:30:06.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb299e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0b5c30 0x7f3fac0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb299e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fac0b6cb0 0x7f3fac155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb299e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb299e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0ce530 con 0x7f3fac15db70 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb299e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac15df50 con 0x7f3fac0b5c30 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb299e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0cf1c0 con 0x7f3fac0b6cb0 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55212/0 (socket says 172.21.15.60:55212) 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 -- 172.21.15.60:0/1800912707 learned_addr learned my addr 172.21.15.60:0/1800912707 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fabfff640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0b5c30 0x7f3fac0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fab7fe640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fac0b6cb0 0x7f3fac155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 -- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fac0b6cb0 msgr2=0x7f3fac155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fac0b6cb0 0x7f3fac155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 -- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0b5c30 msgr2=0x7f3fac0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0b5c30 0x7f3fac0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 -- 172.21.15.60:0/1800912707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac153a30 con 0x7f3fac15db70 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.315+0000 7f3fb0f3b640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac155f90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400cb10 tx=0x7f3fa400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.316+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4017070 con 0x7f3fac15db70 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.316+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa400ac70 con 0x7f3fac15db70 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.316+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac15f070 con 0x7f3fac15db70 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.316+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4012600 con 0x7f3fac15db70 2024-04-25T08:30:06.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.316+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac15f630 con 0x7f3fac15db70 2024-04-25T08:30:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.317+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac153c00 con 0x7f3fac15db70 2024-04-25T08:30:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.317+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fa401a070 con 0x7f3fac15db70 2024-04-25T08:30:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.317+0000 7f3fa8ff9640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9806b4d0 0x7f3f9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.317+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3fa408a240 con 0x7f3fac15db70 2024-04-25T08:30:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.318+0000 7f3fabfff640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9806b4d0 0x7f3f9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:06.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.318+0000 7f3fabfff640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9806b4d0 0x7f3f9806d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0034000 tx=0x7f3fa002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:06.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.322+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fa400ae20 con 0x7f3fac15db70 2024-04-25T08:30:06.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:06.570+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f3fac0d1820 con 0x7f3fac15db70 2024-04-25T08:30:07.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.194+0000 7f3fa8ff9640 1 -- 172.21.15.60:0/1800912707 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v66) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3fa4058670 con 0x7f3fac15db70 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9806b4d0 msgr2=0x7f3f9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9806b4d0 0x7f3f9806d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0034000 tx=0x7f3fa002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 msgr2=0x7f3fac155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac155f90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400cb10 tx=0x7f3fa400a060 comp rx=0 tx=0).stop 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 shutdown_connections 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fac15db70 0x7f3fac155f90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fac0b6cb0 0x7f3fac155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9806b4d0 0x7f3f9806d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 --2- 172.21.15.60:0/1800912707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fac0b5c30 0x7f3fac0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 >> 172.21.15.60:0/1800912707 conn(0x7f3fac0d0520 msgr2=0x7f3fac0b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 shutdown_connections 2024-04-25T08:30:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.197+0000 7f3fb299e640 1 -- 172.21.15.60:0/1800912707 wait complete. 2024-04-25T08:30:07.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:30:07.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.321+0000 7efc09eb3640 1 Processor -- start 2024-04-25T08:30:07.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.321+0000 7efc09eb3640 1 -- start start 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc09eb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc0415dc20 0x7efc040ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc09eb3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc04161710 0x7efc040cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc09eb3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc040b9330 0x7efc0406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc09eb3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0415e000 con 0x7efc040b9330 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc09eb3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc040d1de0 con 0x7efc04161710 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc09eb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc04161c60 con 0x7efc0415dc20 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc040b9330 0x7efc0406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc040b9330 0x7efc0406bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55224/0 (socket says 172.21.15.60:55224) 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 -- 172.21.15.60:0/3837118962 learned_addr learned my addr 172.21.15.60:0/3837118962 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc037fe640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc0415dc20 0x7efc040ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 -- 172.21.15.60:0/3837118962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc0415dc20 msgr2=0x7efc040ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc02ffd640 1 --2- 172.21.15.60:0/3837118962 >> v2:172.21.15.100:3300/0 conn(0x7efc04161710 0x7efc040cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc0415dc20 0x7efc040ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 -- 172.21.15.60:0/3837118962 >> v2:172.21.15.100:3300/0 conn(0x7efc04161710 msgr2=0x7efc040cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 --2- 172.21.15.60:0/3837118962 >> v2:172.21.15.100:3300/0 conn(0x7efc04161710 0x7efc040cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc03fff640 1 -- 172.21.15.60:0/3837118962 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc04161f40 con 0x7efc040b9330 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc037fe640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc0415dc20 0x7efc040ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.322+0000 7efc02ffd640 1 --2- 172.21.15.60:0/3837118962 >> v2:172.21.15.100:3300/0 conn(0x7efc04161710 0x7efc040cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.323+0000 7efc03fff640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc0406bf80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc0406d9a0 tx=0x7efbfc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13be5cf21ecc1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.323+0000 7efc027fc640 1 -- 172.21.15.60:0/3837118962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc036070 con 0x7efc040b9330 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.323+0000 7efc027fc640 1 -- 172.21.15.60:0/3837118962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc031600 con 0x7efc040b9330 2024-04-25T08:30:07.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.323+0000 7efc027fc640 1 -- 172.21.15.60:0/3837118962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0319d0 con 0x7efc040b9330 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 -- 172.21.15.60:0/3837118962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 msgr2=0x7efc0406bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc0406bf80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc0406d9a0 tx=0x7efbfc02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 -- 172.21.15.60:0/3837118962 shutdown_connections 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc0406bf80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 --2- 172.21.15.60:0/3837118962 >> v2:172.21.15.100:3300/0 conn(0x7efc04161710 0x7efc040cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 --2- 172.21.15.60:0/3837118962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc0415dc20 0x7efc040ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 -- 172.21.15.60:0/3837118962 >> 172.21.15.60:0/3837118962 conn(0x7efc040d0420 msgr2=0x7efc040d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 -- 172.21.15.60:0/3837118962 shutdown_connections 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.324+0000 7efc09eb3640 1 -- 172.21.15.60:0/3837118962 wait complete. 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.325+0000 7efc09eb3640 1 Processor -- start 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.325+0000 7efc09eb3640 1 -- start start 2024-04-25T08:30:07.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc09eb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc041f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc09eb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0415dc20 0x7efc041f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc09eb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc04161710 0x7efc040d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc09eb3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc040b5e50 con 0x7efc040b9330 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc09eb3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04162500 con 0x7efc0415dc20 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc09eb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc040b5fd0 con 0x7efc04161710 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc041f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc041f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55228/0 (socket says 172.21.15.60:55228) 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 -- 172.21.15.60:0/876793126 learned_addr learned my addr 172.21.15.60:0/876793126 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc02ffd640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0415dc20 0x7efc041f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc03fff640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc04161710 0x7efc040d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 -- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc04161710 msgr2=0x7efc040d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc04161710 0x7efc040d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 -- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0415dc20 msgr2=0x7efc041f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0415dc20 0x7efc041f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 -- 172.21.15.60:0/876793126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc040cbc20 con 0x7efc040b9330 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc02ffd640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0415dc20 0x7efc041f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.326+0000 7efc037fe640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc041f3a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efbf400c820 tx=0x7efbf400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.327+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4015070 con 0x7efc040b9330 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.327+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf4007a30 con 0x7efc040b9330 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.327+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4007e20 con 0x7efc040b9330 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.327+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc040cbf00 con 0x7efc040b9330 2024-04-25T08:30:07.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.327+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc040cc4c0 con 0x7efc040b9330 2024-04-25T08:30:07.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.328+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc04073a00 con 0x7efc040b9330 2024-04-25T08:30:07.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.328+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efbf4018070 con 0x7efc040b9330 2024-04-25T08:30:07.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.329+0000 7efbebfff640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efbf006b4d0 0x7efbf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.329+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efbf408a550 con 0x7efc040b9330 2024-04-25T08:30:07.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.329+0000 7efc02ffd640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efbf006b4d0 0x7efbf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.330+0000 7efc02ffd640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efbf006b4d0 0x7efbf006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc040b7280 tx=0x7efbf8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:07.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.333+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efbf4058f50 con 0x7efc040b9330 2024-04-25T08:30:07.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.513+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efc040d16b0 con 0x7efc040b9330 2024-04-25T08:30:07.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.516+0000 7efbebfff640 1 -- 172.21.15.60:0/876793126 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7efbf4058850 con 0x7efc040b9330 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efbf006b4d0 msgr2=0x7efbf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efbf006b4d0 0x7efbf006d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc040b7280 tx=0x7efbf8008040 comp rx=0 tx=0).stop 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 msgr2=0x7efc041f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc041f3a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efbf400c820 tx=0x7efbf400cce0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 shutdown_connections 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc04161710 0x7efc040d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efbf006b4d0 0x7efbf006d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc0415dc20 0x7efc041f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 --2- 172.21.15.60:0/876793126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc040b9330 0x7efc041f3a50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 >> 172.21.15.60:0/876793126 conn(0x7efc040d0420 msgr2=0x7efc040b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 shutdown_connections 2024-04-25T08:30:07.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.518+0000 7efc09eb3640 1 -- 172.21.15.60:0/876793126 wait complete. 2024-04-25T08:30:07.527 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:30:07.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:30:07.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:30:07.613 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:30:07.613 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:07.613 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:07.629 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:07.629 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:07.685 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:07.685 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:07.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:07.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.885+0000 7fa3f95f5640 1 Processor -- start 2024-04-25T08:30:07.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.885+0000 7fa3f95f5640 1 -- start start 2024-04-25T08:30:07.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f95f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa3f415dc70 0x7fa3f40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f95f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3f409a450 0x7fa3f40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f95f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f406c220 con 0x7fa3f406c700 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f95f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f40d0c90 con 0x7fa3f409a450 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f95f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f40d0080 con 0x7fa3f415dc70 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa3f409a450 0x7fa3f40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa3f406c700 0x7fa3f40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48056/0 (socket says 172.21.15.60:48056) 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.886+0000 7fa3f37fe640 1 -- 172.21.15.60:0/2192145843 learned_addr learned my addr 172.21.15.60:0/2192145843 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f2ffd640 1 --2- 172.21.15.60:0/2192145843 >> v2:172.21.15.100:3300/0 conn(0x7fa3f415dc70 0x7fa3f40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f27fc640 1 -- 172.21.15.60:0/2192145843 >> v2:172.21.15.100:3300/0 conn(0x7fa3f415dc70 msgr2=0x7fa3f40d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f27fc640 1 --2- 172.21.15.60:0/2192145843 >> v2:172.21.15.100:3300/0 conn(0x7fa3f415dc70 0x7fa3f40d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f27fc640 1 -- 172.21.15.60:0/2192145843 >> v1:172.21.15.60:6789/0 conn(0x7fa3f406c700 legacy=0x7fa3f40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f1ffb640 1 -- 172.21.15.60:0/2192145843 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625386557 0 0) 0x7fa3f406c220 con 0x7fa3f406c700 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f27fc640 1 -- 172.21.15.60:0/2192145843 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40d0360 con 0x7fa3f409a450 2024-04-25T08:30:07.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f2ffd640 1 --2- 172.21.15.60:0/2192145843 >> v2:172.21.15.100:3300/0 conn(0x7fa3f415dc70 0x7fa3f40d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:07.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.887+0000 7fa3f27fc640 1 --2- 172.21.15.60:0/2192145843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f40cf9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40726a0 tx=0x7fa3e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d432cbfd5518f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:07.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f1ffb640 1 -- 172.21.15.60:0/2192145843 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8037070 con 0x7fa3f409a450 2024-04-25T08:30:07.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f1ffb640 1 -- 172.21.15.60:0/2192145843 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8032440 con 0x7fa3f409a450 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f1ffb640 1 -- 172.21.15.60:0/2192145843 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8032810 con 0x7fa3f409a450 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 -- 172.21.15.60:0/2192145843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 msgr2=0x7fa3f40cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/2192145843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f40cf9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa3f40726a0 tx=0x7fa3e802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 -- 172.21.15.60:0/2192145843 shutdown_connections 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/2192145843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f40cf9c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/2192145843 >> v2:172.21.15.100:3300/0 conn(0x7fa3f415dc70 0x7fa3f40d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 -- 172.21.15.60:0/2192145843 >> 172.21.15.60:0/2192145843 conn(0x7fa3f40b3270 msgr2=0x7fa3f40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 -- 172.21.15.60:0/2192145843 shutdown_connections 2024-04-25T08:30:07.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.888+0000 7fa3f95f5640 1 -- 172.21.15.60:0/2192145843 wait complete. 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 Processor -- start 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 -- start start 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3f406c700 0x7fa3f4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f409cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f415dc70 0x7fa3f409c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f415e050 con 0x7fa3f409a450 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40d0920 con 0x7fa3f415dc70 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f95f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f406cb90 con 0x7fa3f406c700 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f409cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f409cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55256/0 (socket says 172.21.15.60:55256) 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.890+0000 7fa3f27fc640 1 -- 172.21.15.60:0/950185294 learned_addr learned my addr 172.21.15.60:0/950185294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f37fe640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f415dc70 0x7fa3f409c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f27fc640 1 -- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3f406c700 msgr2=0x7fa3f4156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f27fc640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3f406c700 0x7fa3f4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f27fc640 1 -- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f415dc70 msgr2=0x7fa3f409c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f27fc640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f415dc70 0x7fa3f409c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f27fc640 1 -- 172.21.15.60:0/950185294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f409d8e0 con 0x7fa3f409a450 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f37fe640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f415dc70 0x7fa3f409c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f27fc640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f409cfe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000c00 tx=0x7fa3e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8041070 con 0x7fa3f409a450 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e803c890 con 0x7fa3f409a450 2024-04-25T08:30:07.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f409dbc0 con 0x7fa3f409a450 2024-04-25T08:30:07.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8037040 con 0x7fa3f409a450 2024-04-25T08:30:07.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.891+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f409b920 con 0x7fa3f409a450 2024-04-25T08:30:07.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.892+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f4073990 con 0x7fa3f409a450 2024-04-25T08:30:07.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.893+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3e8044070 con 0x7fa3f409a450 2024-04-25T08:30:07.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.893+0000 7fa3db7fe640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa3e006b4d0 0x7fa3e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:07.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.893+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa3e80ab350 con 0x7fa3f409a450 2024-04-25T08:30:07.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.893+0000 7fa3f2ffd640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa3e006b4d0 0x7fa3e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:07.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.894+0000 7fa3f2ffd640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa3e006b4d0 0x7fa3e006d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec006fd0 tx=0x7fa3ec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:07.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:07.897+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3e8079f80 con 0x7fa3f409a450 2024-04-25T08:30:08.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.080+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f40b40a0 con 0x7fa3f409a450 2024-04-25T08:30:08.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.082+0000 7fa3db7fe640 1 -- 172.21.15.60:0/950185294 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7fa3e8079880 con 0x7fa3f409a450 2024-04-25T08:30:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:08.080 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa3e006b4d0 msgr2=0x7fa3e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa3e006b4d0 0x7fa3e006d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec006fd0 tx=0x7fa3ec008040 comp rx=0 tx=0).stop 2024-04-25T08:30:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 msgr2=0x7fa3f409cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f409cfe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000c00 tx=0x7fa3e80046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 shutdown_connections 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa3e006b4d0 0x7fa3e006d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3f415dc70 0x7fa3f409c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa3f409a450 0x7fa3f409cfe0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 --2- 172.21.15.60:0/950185294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3f406c700 0x7fa3f4156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 >> 172.21.15.60:0/950185294 conn(0x7fa3f40b3270 msgr2=0x7fa3f40b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 shutdown_connections 2024-04-25T08:30:08.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.084+0000 7fa3f95f5640 1 -- 172.21.15.60:0/950185294 wait complete. 2024-04-25T08:30:08.093 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:08.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.200+0000 7facb078b640 1 Processor -- start 2024-04-25T08:30:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.200+0000 7facb078b640 1 -- start start 2024-04-25T08:30:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facb078b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faca81610f0 0x7faca806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facb078b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca80c6960 con 0x7faca80ce7b0 2024-04-25T08:30:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facb078b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca8162260 con 0x7faca81616f0 2024-04-25T08:30:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facb078b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca81614d0 con 0x7faca81610f0 2024-04-25T08:30:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facaed28640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faca80ce7b0 0x7faca80d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48058/0 (socket says 172.21.15.60:48058) 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facaed28640 1 -- 172.21.15.60:0/742690285 learned_addr learned my addr 172.21.15.60:0/742690285 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facae527640 1 --2- 172.21.15.60:0/742690285 >> v2:172.21.15.100:3300/0 conn(0x7faca81610f0 0x7faca806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facad525640 1 -- 172.21.15.60:0/742690285 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541643686 0 0) 0x7faca80c6960 con 0x7faca80ce7b0 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facad525640 1 -- 172.21.15.60:0/742690285 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac94003520 con 0x7faca80ce7b0 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facad525640 1 -- 172.21.15.60:0/742690285 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516524480 0 0) 0x7faca8162260 con 0x7faca81616f0 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facad525640 1 -- 172.21.15.60:0/742690285 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca80c6960 con 0x7faca81616f0 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facae527640 1 -- 172.21.15.60:0/742690285 >> v1:172.21.15.100:6789/0 conn(0x7faca81616f0 legacy=0x7faca80ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facae527640 1 -- 172.21.15.60:0/742690285 >> v1:172.21.15.60:6789/0 conn(0x7faca80ce7b0 legacy=0x7faca80d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.201+0000 7facae527640 1 -- 172.21.15.60:0/742690285 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8162540 con 0x7faca81610f0 2024-04-25T08:30:08.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.202+0000 7facae527640 1 --2- 172.21.15.60:0/742690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca81610f0 0x7faca806c1c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faca40097a0 tx=0x7faca402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e06a515148d971 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.202+0000 7facad525640 1 -- 172.21.15.60:0/742690285 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4037070 con 0x7faca81610f0 2024-04-25T08:30:08.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facad525640 1 -- 172.21.15.60:0/742690285 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca4032440 con 0x7faca81610f0 2024-04-25T08:30:08.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facad525640 1 -- 172.21.15.60:0/742690285 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca4032810 con 0x7faca81610f0 2024-04-25T08:30:08.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 -- 172.21.15.60:0/742690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca81610f0 msgr2=0x7faca806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 --2- 172.21.15.60:0/742690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca81610f0 0x7faca806c1c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faca40097a0 tx=0x7faca402a230 comp rx=0 tx=0).stop 2024-04-25T08:30:08.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 -- 172.21.15.60:0/742690285 shutdown_connections 2024-04-25T08:30:08.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 --2- 172.21.15.60:0/742690285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca81610f0 0x7faca806c1c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 -- 172.21.15.60:0/742690285 >> 172.21.15.60:0/742690285 conn(0x7faca806f500 msgr2=0x7faca806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:08.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 -- 172.21.15.60:0/742690285 shutdown_connections 2024-04-25T08:30:08.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.203+0000 7facb078b640 1 -- 172.21.15.60:0/742690285 wait complete. 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.204+0000 7facb078b640 1 Processor -- start 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.204+0000 7facb078b640 1 -- start start 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facb078b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca80ce7b0 0x7faca814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facb078b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 0x7faca80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facb078b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca81616f0 0x7faca80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facb078b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca80cf690 con 0x7faca81610f0 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facb078b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca80cfce0 con 0x7faca80ce7b0 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facb078b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca806c700 con 0x7faca81616f0 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facadd26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 0x7faca80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facadd26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 0x7faca80d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55266/0 (socket says 172.21.15.60:55266) 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facadd26640 1 -- 172.21.15.60:0/1191669008 learned_addr learned my addr 172.21.15.60:0/1191669008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facae527640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca80ce7b0 0x7faca814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.205+0000 7facaed28640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca81616f0 0x7faca80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facadd26640 1 -- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca81616f0 msgr2=0x7faca80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facadd26640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca81616f0 0x7faca80d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facadd26640 1 -- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca80ce7b0 msgr2=0x7faca814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facadd26640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca80ce7b0 0x7faca814c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facadd26640 1 -- 172.21.15.60:0/1191669008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca80cd1a0 con 0x7faca81610f0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facae527640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca80ce7b0 0x7faca814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facadd26640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 0x7faca80d0100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c430 tx=0x7fac9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98017070 con 0x7faca81610f0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac980052f0 con 0x7faca81610f0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca80cd480 con 0x7faca81610f0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98003830 con 0x7faca81610f0 2024-04-25T08:30:08.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.206+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca80c88f0 con 0x7faca81610f0 2024-04-25T08:30:08.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.207+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8073a00 con 0x7faca81610f0 2024-04-25T08:30:08.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.207+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac9801a070 con 0x7faca81610f0 2024-04-25T08:30:08.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.208+0000 7fac9effd640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac8c06b4d0 0x7fac8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.208+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fac980899f0 con 0x7faca81610f0 2024-04-25T08:30:08.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.208+0000 7facae527640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac8c06b4d0 0x7fac8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.209+0000 7facae527640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac8c06b4d0 0x7fac8c06d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faca4034000 tx=0x7faca402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.212+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac98058480 con 0x7faca81610f0 2024-04-25T08:30:08.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.394+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faca80cf150 con 0x7faca81610f0 2024-04-25T08:30:08.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.394+0000 7fac9effd640 1 -- 172.21.15.60:0/1191669008 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac98057d80 con 0x7faca81610f0 2024-04-25T08:30:08.392 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.396+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac8c06b4d0 msgr2=0x7fac8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.396+0000 7facb078b640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac8c06b4d0 0x7fac8c06d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faca4034000 tx=0x7faca402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.396+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 msgr2=0x7faca80d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.396+0000 7facb078b640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 0x7faca80d0100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fac9800c430 tx=0x7fac9800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 shutdown_connections 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faca81616f0 0x7faca80d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca81610f0 0x7faca80d0100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac8c06b4d0 0x7fac8c06d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 --2- 172.21.15.60:0/1191669008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faca80ce7b0 0x7faca814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 >> 172.21.15.60:0/1191669008 conn(0x7faca806f500 msgr2=0x7faca80c6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 shutdown_connections 2024-04-25T08:30:08.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.397+0000 7facb078b640 1 -- 172.21.15.60:0/1191669008 wait complete. 2024-04-25T08:30:08.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:08.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.513+0000 7f9705a1f640 1 Processor -- start 2024-04-25T08:30:08.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.513+0000 7f9705a1f640 1 -- start start 2024-04-25T08:30:08.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f9705a1f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f9705a1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f9705a1f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97000c6940 con 0x7f97000cdbf0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f9705a1f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000b0020 con 0x7f970015a9f0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f9705a1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000cb5a0 con 0x7f97000cd6f0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f96feffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f96fe7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f97000cdbf0 0x7f97000ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48072/0 (socket says 172.21.15.60:48072) 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.514+0000 7f96fe7fc640 1 -- 172.21.15.60:0/4166720321 learned_addr learned my addr 172.21.15.60:0/4166720321 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96feffd640 1 -- 172.21.15.60:0/4166720321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97000cd6f0 msgr2=0x7f97000cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96ff7fe640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96feffd640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96feffd640 1 -- 172.21.15.60:0/4166720321 >> v1:172.21.15.60:6789/0 conn(0x7f97000cdbf0 legacy=0x7f97000ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96fdffb640 1 -- 172.21.15.60:0/4166720321 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183867791 0 0) 0x7f96e8001110 con 0x7f97000cdbf0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96feffd640 1 -- 172.21.15.60:0/4166720321 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970015add0 con 0x7f970015a9f0 2024-04-25T08:30:08.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96ff7fe640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.515+0000 7f96feffd640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f970015a9f0 0x7f97000c6010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f96f000bfa0 tx=0x7f96f002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65537a8701d63034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f96fdffb640 1 -- 172.21.15.60:0/4166720321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0039070 con 0x7f970015a9f0 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f96fdffb640 1 -- 172.21.15.60:0/4166720321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0009540 con 0x7f970015a9f0 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f96fdffb640 1 -- 172.21.15.60:0/4166720321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0009910 con 0x7f970015a9f0 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 -- 172.21.15.60:0/4166720321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f970015a9f0 msgr2=0x7f97000c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f970015a9f0 0x7f97000c6010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f96f000bfa0 tx=0x7f96f002c860 comp rx=0 tx=0).stop 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 -- 172.21.15.60:0/4166720321 shutdown_connections 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97000cd6f0 0x7f97000cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 --2- 172.21.15.60:0/4166720321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f970015a9f0 0x7f97000c6010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 -- 172.21.15.60:0/4166720321 >> 172.21.15.60:0/4166720321 conn(0x7f97000b2390 msgr2=0x7f97000b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 -- 172.21.15.60:0/4166720321 shutdown_connections 2024-04-25T08:30:08.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.516+0000 7f9705a1f640 1 -- 172.21.15.60:0/4166720321 wait complete. 2024-04-25T08:30:08.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.517+0000 7f9705a1f640 1 Processor -- start 2024-04-25T08:30:08.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 -- start start 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97000cd6f0 0x7f9700153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 0x7f97001f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f970015a9f0 0x7f97001591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000cb230 con 0x7f97000cdbf0 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000caf50 con 0x7f97000cd6f0 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f9705a1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ce070 con 0x7f970015a9f0 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f96fe7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 0x7f97001f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f96fe7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 0x7f97001f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55282/0 (socket says 172.21.15.60:55282) 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.518+0000 7f96fe7fc640 1 -- 172.21.15.60:0/2633199450 learned_addr learned my addr 172.21.15.60:0/2633199450 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96feffd640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97000cd6f0 0x7f9700153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96ff7fe640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f970015a9f0 0x7f97001591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96fe7fc640 1 -- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f970015a9f0 msgr2=0x7f97001591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96fe7fc640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f970015a9f0 0x7f97001591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96fe7fc640 1 -- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97000cd6f0 msgr2=0x7f9700153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96fe7fc640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97000cd6f0 0x7f9700153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96fe7fc640 1 -- 172.21.15.60:0/2633199450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97001f7230 con 0x7f97000cdbf0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96feffd640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97000cd6f0 0x7f9700153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96fe7fc640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 0x7f97001f6cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c7b0 tx=0x7f96e800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8018070 con 0x7f97000cdbf0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e80072d0 con 0x7f97000cdbf0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97000ae460 con 0x7f97000cdbf0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80047e0 con 0x7f97000cdbf0 2024-04-25T08:30:08.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.519+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97000ad850 con 0x7f97000cdbf0 2024-04-25T08:30:08.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.520+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970006cc80 con 0x7f97000cdbf0 2024-04-25T08:30:08.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.521+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96e801b070 con 0x7f97000cdbf0 2024-04-25T08:30:08.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.521+0000 7f96f7fff640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96e006b4d0 0x7f96e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.521+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f96e80898e0 con 0x7f97000cdbf0 2024-04-25T08:30:08.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.521+0000 7f96feffd640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96e006b4d0 0x7f96e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.522+0000 7f96feffd640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96e006b4d0 0x7f96e006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96f0036000 tx=0x7f96f000b000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.526+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97000c6940 con 0x7f97000cdbf0 2024-04-25T08:30:08.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.704+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9700073a00 con 0x7f97000cdbf0 2024-04-25T08:30:08.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.704+0000 7f96f7fff640 1 -- 172.21.15.60:0/2633199450 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96e8058360 con 0x7f97000cdbf0 2024-04-25T08:30:08.702 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96e006b4d0 msgr2=0x7f96e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96e006b4d0 0x7f96e006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96f0036000 tx=0x7f96f000b000 comp rx=0 tx=0).stop 2024-04-25T08:30:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 msgr2=0x7f97001f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 0x7f97001f6cf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f96e800c7b0 tx=0x7f96e800cc70 comp rx=0 tx=0).stop 2024-04-25T08:30:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 shutdown_connections 2024-04-25T08:30:08.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f970015a9f0 0x7f97001591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97000cdbf0 0x7f97001f6cf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96e006b4d0 0x7f96e006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 --2- 172.21.15.60:0/2633199450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97000cd6f0 0x7f9700153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 >> 172.21.15.60:0/2633199450 conn(0x7f97000b2390 msgr2=0x7f970006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 shutdown_connections 2024-04-25T08:30:08.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.706+0000 7f9705a1f640 1 -- 172.21.15.60:0/2633199450 wait complete. 2024-04-25T08:30:08.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:08.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.821+0000 7fb84b742640 1 Processor -- start 2024-04-25T08:30:08.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.821+0000 7fb84b742640 1 -- start start 2024-04-25T08:30:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb84b742640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84415b7d0 0x7fb8440cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb84b742640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8440b5620 0x7fb8440b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb84b742640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84406cf50 con 0x7fb84406c2d0 2024-04-25T08:30:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb84b742640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8440b5fc0 con 0x7fb8440b5620 2024-04-25T08:30:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb84b742640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440ce530 con 0x7fb84415b7d0 2024-04-25T08:30:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8440b5620 0x7fb8440b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8440b5620 0x7fb8440b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55290/0 (socket says 172.21.15.60:55290) 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 -- 172.21.15.60:0/3775942011 learned_addr learned my addr 172.21.15.60:0/3775942011 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb8494de640 1 --2- 172.21.15.60:0/3775942011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84415b7d0 0x7fb8440cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 -- 172.21.15.60:0/3775942011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84415b7d0 msgr2=0x7fb8440cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 --2- 172.21.15.60:0/3775942011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84415b7d0 0x7fb8440cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 -- 172.21.15.60:0/3775942011 >> v1:172.21.15.100:6789/0 conn(0x7fb84406c2d0 legacy=0x7fb84406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.822+0000 7fb848cdd640 1 -- 172.21.15.60:0/3775942011 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8440ce810 con 0x7fb8440b5620 2024-04-25T08:30:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.823+0000 7fb848cdd640 1 --2- 172.21.15.60:0/3775942011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440b5a80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb834009870 tx=0x7fb83402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aece34160c68f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.823+0000 7fb83bfff640 1 -- 172.21.15.60:0/3775942011 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834037070 con 0x7fb8440b5620 2024-04-25T08:30:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.823+0000 7fb83bfff640 1 -- 172.21.15.60:0/3775942011 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834032440 con 0x7fb8440b5620 2024-04-25T08:30:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.823+0000 7fb83bfff640 1 -- 172.21.15.60:0/3775942011 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834032810 con 0x7fb8440b5620 2024-04-25T08:30:08.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 -- 172.21.15.60:0/3775942011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 msgr2=0x7fb8440b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 --2- 172.21.15.60:0/3775942011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440b5a80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb834009870 tx=0x7fb83402a270 comp rx=0 tx=0).stop 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 -- 172.21.15.60:0/3775942011 shutdown_connections 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 --2- 172.21.15.60:0/3775942011 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440b5a80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 --2- 172.21.15.60:0/3775942011 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84415b7d0 0x7fb8440cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 -- 172.21.15.60:0/3775942011 >> 172.21.15.60:0/3775942011 conn(0x7fb8440d0800 msgr2=0x7fb8440d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 -- 172.21.15.60:0/3775942011 shutdown_connections 2024-04-25T08:30:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.824+0000 7fb84b742640 1 -- 172.21.15.60:0/3775942011 wait complete. 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.825+0000 7fb84b742640 1 Processor -- start 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.825+0000 7fb84b742640 1 -- start start 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb84b742640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84406c2d0 0x7fb84414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb84b742640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb84b742640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84415b7d0 0x7fb844157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb84b742640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440cf550 con 0x7fb8440b5620 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb84b742640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84406d230 con 0x7fb84415b7d0 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb84b742640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440cf6d0 con 0x7fb84406c2d0 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55302/0 (socket says 172.21.15.60:55302) 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 -- 172.21.15.60:0/3946027243 learned_addr learned my addr 172.21.15.60:0/3946027243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb8494de640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84406c2d0 0x7fb84414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb849cdf640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84415b7d0 0x7fb844157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 -- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84406c2d0 msgr2=0x7fb84414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84406c2d0 0x7fb84414bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 -- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84415b7d0 msgr2=0x7fb844157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84415b7d0 0x7fb844157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.826+0000 7fb848cdd640 1 -- 172.21.15.60:0/3946027243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8440cda80 con 0x7fb8440b5620 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.827+0000 7fb848cdd640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440cd540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb834000c00 tx=0x7fb8340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.827+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834037040 con 0x7fb8440b5620 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.827+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834042070 con 0x7fb8440b5620 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.827+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834030bd0 con 0x7fb8440b5620 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.827+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb844158570 con 0x7fb8440b5620 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.827+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb844158b30 con 0x7fb8440b5620 2024-04-25T08:30:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.828+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb844073a00 con 0x7fb8440b5620 2024-04-25T08:30:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.829+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb834047460 con 0x7fb8440b5620 2024-04-25T08:30:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.829+0000 7fb839ffb640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb82806b4d0 0x7fb82806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.829+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb8340aaff0 con 0x7fb8440b5620 2024-04-25T08:30:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.829+0000 7fb8494de640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb82806b4d0 0x7fb82806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.830+0000 7fb8494de640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb82806b4d0 0x7fb82806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb84000aa20 tx=0x7fb840008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:08.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:08.833+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8340794b0 con 0x7fb8440b5620 2024-04-25T08:30:09.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.012+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8440ca620 con 0x7fb8440b5620 2024-04-25T08:30:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.012+0000 7fb839ffb640 1 -- 172.21.15.60:0/3946027243 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb83407d550 con 0x7fb8440b5620 2024-04-25T08:30:09.010 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb82806b4d0 msgr2=0x7fb82806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb82806b4d0 0x7fb82806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb84000aa20 tx=0x7fb840008040 comp rx=0 tx=0).stop 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 msgr2=0x7fb8440cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440cd540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb834000c00 tx=0x7fb8340046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 shutdown_connections 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb82806b4d0 0x7fb82806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84415b7d0 0x7fb844157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8440b5620 0x7fb8440cd540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 --2- 172.21.15.60:0/3946027243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84406c2d0 0x7fb84414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 >> 172.21.15.60:0/3946027243 conn(0x7fb8440d0800 msgr2=0x7fb8440b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.014+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 shutdown_connections 2024-04-25T08:30:09.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.015+0000 7fb84b742640 1 -- 172.21.15.60:0/3946027243 wait complete. 2024-04-25T08:30:09.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:09.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.129+0000 7f3fc0dcf640 1 Processor -- start 2024-04-25T08:30:09.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.129+0000 7f3fc0dcf640 1 -- start start 2024-04-25T08:30:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fc0dcf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3fbc0ced80 0x7f3fbc0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fc0dcf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc0b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fc0dcf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc0d23a0 con 0x7f3fbc15dc20 2024-04-25T08:30:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fc0dcf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc0d2170 con 0x7f3fbc0ced80 2024-04-25T08:30:09.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fc0dcf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0d2680 con 0x7f3fbc0d1ce0 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fba59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3fbc15dc20 0x7f3fbc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48076/0 (socket says 172.21.15.60:48076) 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fba59c640 1 -- 172.21.15.60:0/3961841779 learned_addr learned my addr 172.21.15.60:0/3961841779 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fbad9d640 1 --2- 172.21.15.60:0/3961841779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc0b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fb9d9b640 1 --2- 172.21.15.60:0/3961841779 >> v2:172.21.15.100:3300/0 conn(0x7f3fbc0ced80 0x7f3fbc0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fb959a640 1 -- 172.21.15.60:0/3961841779 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701811523 0 0) 0x7f3fbc0d23a0 con 0x7f3fbc15dc20 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.130+0000 7f3fb959a640 1 -- 172.21.15.60:0/3961841779 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4003520 con 0x7f3fbc15dc20 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.131+0000 7f3fbad9d640 1 -- 172.21.15.60:0/3961841779 >> v2:172.21.15.100:3300/0 conn(0x7f3fbc0ced80 msgr2=0x7f3fbc0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.131+0000 7f3fbad9d640 1 --2- 172.21.15.60:0/3961841779 >> v2:172.21.15.100:3300/0 conn(0x7f3fbc0ced80 0x7f3fbc0d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.131+0000 7f3fbad9d640 1 -- 172.21.15.60:0/3961841779 >> v1:172.21.15.60:6789/0 conn(0x7f3fbc15dc20 legacy=0x7f3fbc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.131+0000 7f3fbad9d640 1 -- 172.21.15.60:0/3961841779 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0d2960 con 0x7f3fbc0d1ce0 2024-04-25T08:30:09.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.131+0000 7f3fbad9d640 1 --2- 172.21.15.60:0/3961841779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc0b9490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009d90 tx=0x7f3fb402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64dfe680bbae125c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fb959a640 1 -- 172.21.15.60:0/3961841779 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4031600 con 0x7f3fbc0d1ce0 2024-04-25T08:30:09.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fb959a640 1 -- 172.21.15.60:0/3961841779 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb40317a0 con 0x7f3fbc0d1ce0 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fb959a640 1 -- 172.21.15.60:0/3961841779 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4031b50 con 0x7f3fbc0d1ce0 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/3961841779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 msgr2=0x7f3fbc0b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/3961841779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc0b9490 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009d90 tx=0x7f3fb402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/3961841779 shutdown_connections 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/3961841779 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc0b9490 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/3961841779 >> v2:172.21.15.100:3300/0 conn(0x7f3fbc0ced80 0x7f3fbc0d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/3961841779 >> 172.21.15.60:0/3961841779 conn(0x7f3fbc061a20 msgr2=0x7f3fbc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/3961841779 shutdown_connections 2024-04-25T08:30:09.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.132+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/3961841779 wait complete. 2024-04-25T08:30:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 Processor -- start 2024-04-25T08:30:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 -- start start 2024-04-25T08:30:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 0x7f3fbc15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc1e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc15dc20 0x7f3fbc1e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b6a30 con 0x7f3fbc0ced80 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0d2f20 con 0x7f3fbc15dc20 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fc0dcf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0b6bb0 con 0x7f3fbc0d1ce0 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 0x7f3fbc15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fba59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 0x7f3fbc15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55318/0 (socket says 172.21.15.60:55318) 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.134+0000 7f3fba59c640 1 -- 172.21.15.60:0/1374867706 learned_addr learned my addr 172.21.15.60:0/1374867706 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fba59c640 1 -- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 msgr2=0x7f3fbc1e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fba59c640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc1e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fba59c640 1 -- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc15dc20 msgr2=0x7f3fbc1e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fba59c640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc15dc20 0x7f3fbc1e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fba59c640 1 -- 172.21.15.60:0/1374867706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc15ebe0 con 0x7f3fbc0ced80 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fba59c640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 0x7f3fbc15c250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0010e0 tx=0x7f3fac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac017070 con 0x7f3fbc0ced80 2024-04-25T08:30:09.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac005d00 con 0x7f3fbc0ced80 2024-04-25T08:30:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc0c9c60 con 0x7f3fbc0ced80 2024-04-25T08:30:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac007310 con 0x7f3fbc0ced80 2024-04-25T08:30:09.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.135+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc15f180 con 0x7f3fbc0ced80 2024-04-25T08:30:09.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.136+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc073a00 con 0x7f3fbc0ced80 2024-04-25T08:30:09.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.137+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fac01a070 con 0x7f3fbc0ced80 2024-04-25T08:30:09.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.137+0000 7f3faaffd640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.137+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3fac0899f0 con 0x7f3fbc0ced80 2024-04-25T08:30:09.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.137+0000 7f3fb9d9b640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.138+0000 7f3fb9d9b640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006a90 tx=0x7f3fb0006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.141+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fac058470 con 0x7f3fbc0ced80 2024-04-25T08:30:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.320+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3fbc062050 con 0x7f3fbc0ced80 2024-04-25T08:30:09.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.320+0000 7f3faaffd640 1 -- 172.21.15.60:0/1374867706 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3fac057d70 con 0x7f3fbc0ced80 2024-04-25T08:30:09.318 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:09.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9c06b4d0 msgr2=0x7f3f9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006a90 tx=0x7f3fb0006a20 comp rx=0 tx=0).stop 2024-04-25T08:30:09.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 msgr2=0x7f3fbc15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 0x7f3fbc15c250 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0010e0 tx=0x7f3fac00c890 comp rx=0 tx=0).stop 2024-04-25T08:30:09.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 shutdown_connections 2024-04-25T08:30:09.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f9c06b4d0 0x7f3f9c06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fbc15dc20 0x7f3fbc1e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fbc0d1ce0 0x7f3fbc1e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 --2- 172.21.15.60:0/1374867706 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fbc0ced80 0x7f3fbc15c250 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 >> 172.21.15.60:0/1374867706 conn(0x7f3fbc061a20 msgr2=0x7f3fbc0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 shutdown_connections 2024-04-25T08:30:09.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.322+0000 7f3fc0dcf640 1 -- 172.21.15.60:0/1374867706 wait complete. 2024-04-25T08:30:09.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:30:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.435+0000 7f28567c4640 1 Processor -- start 2024-04-25T08:30:09.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.436+0000 7f28567c4640 1 -- start start 2024-04-25T08:30:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.436+0000 7f28567c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f285015bb50 0x7f28500c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.436+0000 7f28567c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.436+0000 7f28567c4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500c75e0 con 0x7f28500c70d0 2024-04-25T08:30:09.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.436+0000 7f28567c4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28500c8680 con 0x7f285015bb50 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.436+0000 7f28567c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500c78c0 con 0x7f28500cfd10 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f285015bb50 0x7f28500c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f28500c70d0 0x7f28500cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48090/0 (socket says 172.21.15.60:48090) 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284f7fe640 1 -- 172.21.15.60:0/32947569 learned_addr learned my addr 172.21.15.60:0/32947569 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f2854d61640 1 --2- 172.21.15.60:0/32947569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284ffff640 1 -- 172.21.15.60:0/32947569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 msgr2=0x7f28500cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284ffff640 1 --2- 172.21.15.60:0/32947569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284ffff640 1 -- 172.21.15.60:0/32947569 >> v1:172.21.15.60:6789/0 conn(0x7f28500c70d0 legacy=0x7f28500cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.437+0000 7f284ffff640 1 -- 172.21.15.60:0/32947569 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500c7ba0 con 0x7f285015bb50 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f284ffff640 1 --2- 172.21.15.60:0/32947569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f285015bb50 0x7f28500c6b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f28400097a0 tx=0x7f284002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e3b4a327456958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f284effd640 1 -- 172.21.15.60:0/32947569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840037070 con 0x7f285015bb50 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f284effd640 1 -- 172.21.15.60:0/32947569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840032440 con 0x7f285015bb50 2024-04-25T08:30:09.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f284effd640 1 -- 172.21.15.60:0/32947569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840032810 con 0x7f285015bb50 2024-04-25T08:30:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 -- 172.21.15.60:0/32947569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f285015bb50 msgr2=0x7f28500c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 --2- 172.21.15.60:0/32947569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f285015bb50 0x7f28500c6b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f28400097a0 tx=0x7f284002a210 comp rx=0 tx=0).stop 2024-04-25T08:30:09.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 -- 172.21.15.60:0/32947569 shutdown_connections 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 --2- 172.21.15.60:0/32947569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 --2- 172.21.15.60:0/32947569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f285015bb50 0x7f28500c6b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 -- 172.21.15.60:0/32947569 >> 172.21.15.60:0/32947569 conn(0x7f285006f500 msgr2=0x7f285006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 -- 172.21.15.60:0/32947569 shutdown_connections 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.438+0000 7f28567c4640 1 -- 172.21.15.60:0/32947569 wait complete. 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.439+0000 7f28567c4640 1 Processor -- start 2024-04-25T08:30:09.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 -- start start 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 0x7f28500b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f285015bb50 0x7f285006c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285015f6d0 con 0x7f28500c70d0 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500c8160 con 0x7f285015bb50 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.440+0000 7f28567c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285015f850 con 0x7f28500cfd10 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 0x7f28500b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 0x7f28500b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55338/0 (socket says 172.21.15.60:55338) 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 -- 172.21.15.60:0/2053329723 learned_addr learned my addr 172.21.15.60:0/2053329723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284f7fe640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f2854d61640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f285015bb50 0x7f285006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 -- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 msgr2=0x7f28500b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 -- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f285015bb50 msgr2=0x7f285006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f285015bb50 0x7f285006c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 -- 172.21.15.60:0/2053329723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500b75b0 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284f7fe640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284ffff640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 0x7f28500b4660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f28400098c0 tx=0x7f28400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840037040 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2840030770 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28500c88f0 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840030b40 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.441+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28500c8eb0 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.442+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2850073a00 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.443+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2840044020 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.443+0000 7f284cff9640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f283c06b4d0 0x7f283c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.443+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f28400aaa80 con 0x7f28500c70d0 2024-04-25T08:30:09.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.443+0000 7f284f7fe640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f283c06b4d0 0x7f283c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.444+0000 7f284f7fe640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f283c06b4d0 0x7f283c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f285015af80 tx=0x7f2844009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.447+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2840032e20 con 0x7f28500c70d0 2024-04-25T08:30:09.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.639+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28500b1d90 con 0x7f28500c70d0 2024-04-25T08:30:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.639+0000 7f284cff9640 1 -- 172.21.15.60:0/2053329723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2840079590 con 0x7f28500c70d0 2024-04-25T08:30:09.637 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:30:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f283c06b4d0 msgr2=0x7f283c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f283c06b4d0 0x7f283c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f285015af80 tx=0x7f2844009040 comp rx=0 tx=0).stop 2024-04-25T08:30:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 msgr2=0x7f28500b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 0x7f28500b4660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f28400098c0 tx=0x7f28400046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 shutdown_connections 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f283c06b4d0 0x7f283c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f285015bb50 0x7f285006c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28500cfd10 0x7f28500b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 --2- 172.21.15.60:0/2053329723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28500c70d0 0x7f28500b4660 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 >> 172.21.15.60:0/2053329723 conn(0x7f285006f500 msgr2=0x7f28500cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.641+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 shutdown_connections 2024-04-25T08:30:09.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.642+0000 7f28567c4640 1 -- 172.21.15.60:0/2053329723 wait complete. 2024-04-25T08:30:09.650 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:30:09.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:09.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T08:30:09.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:30:09.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:09.676 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:30:09.731 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:30:09.731 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:09.750 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:30:09.750 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:09.808 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:30:09.808 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:09.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:09.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.970+0000 7f7639c22640 1 Processor -- start 2024-04-25T08:30:09.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.970+0000 7f7639c22640 1 -- start start 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f7639c22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f763415dd20 0x7f76340b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f7639c22640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763415eac0 con 0x7f76340d11b0 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f7639c22640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763415eda0 con 0x7f763415e100 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f7639c22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76340d1c00 con 0x7f763415dd20 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f76337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f763415dd20 0x7f76340b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f76337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f763415dd20 0x7f76340b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55348/0 (socket says 172.21.15.60:55348) 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f76337fe640 1 -- 172.21.15.60:0/57311150 learned_addr learned my addr 172.21.15.60:0/57311150 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f7633fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f76340d11b0 0x7f76340d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48102/0 (socket says 172.21.15.60:48102) 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f76337fe640 1 -- 172.21.15.60:0/57311150 >> v1:172.21.15.100:6789/0 conn(0x7f763415e100 legacy=0x7f76340d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f76337fe640 1 -- 172.21.15.60:0/57311150 >> v1:172.21.15.60:6789/0 conn(0x7f76340d11b0 legacy=0x7f76340d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.971+0000 7f76337fe640 1 -- 172.21.15.60:0/57311150 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763415f080 con 0x7f763415dd20 2024-04-25T08:30:09.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.972+0000 7f76337fe640 1 --2- 172.21.15.60:0/57311150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340b96d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7624009870 tx=0x7f762402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ee4649f2d892d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.972+0000 7f76327fc640 1 -- 172.21.15.60:0/57311150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624037070 con 0x7f763415dd20 2024-04-25T08:30:09.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.972+0000 7f76327fc640 1 -- 172.21.15.60:0/57311150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624032440 con 0x7f763415dd20 2024-04-25T08:30:09.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.972+0000 7f76327fc640 1 -- 172.21.15.60:0/57311150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624032810 con 0x7f763415dd20 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 -- 172.21.15.60:0/57311150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 msgr2=0x7f76340b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 --2- 172.21.15.60:0/57311150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340b96d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7624009870 tx=0x7f762402a210 comp rx=0 tx=0).stop 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 -- 172.21.15.60:0/57311150 shutdown_connections 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 --2- 172.21.15.60:0/57311150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340b96d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 -- 172.21.15.60:0/57311150 >> 172.21.15.60:0/57311150 conn(0x7f7634061a20 msgr2=0x7f7634061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 -- 172.21.15.60:0/57311150 shutdown_connections 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.973+0000 7f7639c22640 1 -- 172.21.15.60:0/57311150 wait complete. 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.974+0000 7f7639c22640 1 Processor -- start 2024-04-25T08:30:09.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.974+0000 7f7639c22640 1 -- start start 2024-04-25T08:30:09.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7639c22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76340d11b0 0x7f7634158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7639c22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7639c22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763415e100 0x7f7634157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7639c22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763415e710 con 0x7f763415dd20 2024-04-25T08:30:09.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7639c22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763415e590 con 0x7f763415e100 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7639c22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7634158130 con 0x7f76340d11b0 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55364/0 (socket says 172.21.15.60:55364) 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 -- 172.21.15.60:0/3113652091 learned_addr learned my addr 172.21.15.60:0/3113652091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7633fff640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763415e100 0x7f7634157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f76337fe640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76340d11b0 0x7f7634158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 -- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76340d11b0 msgr2=0x7f7634158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76340d11b0 0x7f7634158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 -- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763415e100 msgr2=0x7f7634157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763415e100 0x7f7634157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7632ffd640 1 -- 172.21.15.60:0/3113652091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340cd5f0 con 0x7f763415dd20 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f7633fff640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763415e100 0x7f7634157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.975+0000 7f76337fe640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76340d11b0 0x7f7634158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.976+0000 7f7632ffd640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340cabe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f762800b420 tx=0x7f762800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.976+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628011830 con 0x7f763415dd20 2024-04-25T08:30:09.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.976+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7628016070 con 0x7f763415dd20 2024-04-25T08:30:09.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.976+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76340cd8d0 con 0x7f763415dd20 2024-04-25T08:30:09.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.976+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628011c10 con 0x7f763415dd20 2024-04-25T08:30:09.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.976+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76340c7f70 con 0x7f763415dd20 2024-04-25T08:30:09.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.977+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76340c8170 con 0x7f763415dd20 2024-04-25T08:30:09.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.977+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f762801b460 con 0x7f763415dd20 2024-04-25T08:30:09.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.978+0000 7f761bfff640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:09.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.978+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f762808a120 con 0x7f763415dd20 2024-04-25T08:30:09.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.978+0000 7f76337fe640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:09.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.979+0000 7f76337fe640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f762006b4d0 0x7f762006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7624034000 tx=0x7f762402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:09.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:09.982+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76340c8170 con 0x7f763415dd20 2024-04-25T08:30:10.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.163+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7634073990 con 0x7f763415dd20 2024-04-25T08:30:10.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.164+0000 7f761bfff640 1 -- 172.21.15.60:0/3113652091 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7f7628020020 con 0x7f763415dd20 2024-04-25T08:30:10.162 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:10.162 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:10.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f762006b4d0 msgr2=0x7f762006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f762006b4d0 0x7f762006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7624034000 tx=0x7f762402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:10.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 msgr2=0x7f76340cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340cabe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f762800b420 tx=0x7f762800b8e0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 shutdown_connections 2024-04-25T08:30:10.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f762006b4d0 0x7f762006d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f763415e100 0x7f7634157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f763415dd20 0x7f76340cabe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 --2- 172.21.15.60:0/3113652091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f76340d11b0 0x7f7634158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 >> 172.21.15.60:0/3113652091 conn(0x7f7634061a20 msgr2=0x7f76340d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 shutdown_connections 2024-04-25T08:30:10.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.166+0000 7f7639c22640 1 -- 172.21.15.60:0/3113652091 wait complete. 2024-04-25T08:30:10.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:10.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:10.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.282+0000 7f057415c640 1 Processor -- start 2024-04-25T08:30:10.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.282+0000 7f057415c640 1 -- start start 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f057415c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f056c15dc20 0x7f056c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f057415c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f056c161710 0x7f056c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f057415c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c15e000 con 0x7f056c0b9330 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f057415c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c0d1de0 con 0x7f056c15dc20 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f057415c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c161c60 con 0x7f056c161710 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f0571ef8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f056c15dc20 0x7f056c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f05726f9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f056c0b9330 0x7f056c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48110/0 (socket says 172.21.15.60:48110) 2024-04-25T08:30:10.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f0571ef8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f056c15dc20 0x7f056c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55370/0 (socket says 172.21.15.60:55370) 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f05726f9640 1 -- 172.21.15.60:0/1546883745 learned_addr learned my addr 172.21.15.60:0/1546883745 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f05716f7640 1 --2- 172.21.15.60:0/1546883745 >> v2:172.21.15.100:3300/0 conn(0x7f056c161710 0x7f056c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f0571ef8640 1 -- 172.21.15.60:0/1546883745 >> v2:172.21.15.100:3300/0 conn(0x7f056c161710 msgr2=0x7f056c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f0571ef8640 1 --2- 172.21.15.60:0/1546883745 >> v2:172.21.15.100:3300/0 conn(0x7f056c161710 0x7f056c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.283+0000 7f0571ef8640 1 -- 172.21.15.60:0/1546883745 >> v1:172.21.15.60:6789/0 conn(0x7f056c0b9330 legacy=0x7f056c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.284+0000 7f0571ef8640 1 -- 172.21.15.60:0/1546883745 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c161f40 con 0x7f056c15dc20 2024-04-25T08:30:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.284+0000 7f05716f7640 1 --2- 172.21.15.60:0/1546883745 >> v2:172.21.15.100:3300/0 conn(0x7f056c161710 0x7f056c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.284+0000 7f0571ef8640 1 --2- 172.21.15.60:0/1546883745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c15dc20 0x7f056c0ce670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f056c06da90 tx=0x7f056802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efe54f15afdecd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.284+0000 7f0570ef6640 1 -- 172.21.15.60:0/1546883745 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568037070 con 0x7f056c15dc20 2024-04-25T08:30:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.284+0000 7f0570ef6640 1 -- 172.21.15.60:0/1546883745 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568032440 con 0x7f056c15dc20 2024-04-25T08:30:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.284+0000 7f0570ef6640 1 -- 172.21.15.60:0/1546883745 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568032810 con 0x7f056c15dc20 2024-04-25T08:30:10.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 -- 172.21.15.60:0/1546883745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c15dc20 msgr2=0x7f056c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 --2- 172.21.15.60:0/1546883745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c15dc20 0x7f056c0ce670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f056c06da90 tx=0x7f056802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 -- 172.21.15.60:0/1546883745 shutdown_connections 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 --2- 172.21.15.60:0/1546883745 >> v2:172.21.15.100:3300/0 conn(0x7f056c161710 0x7f056c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 --2- 172.21.15.60:0/1546883745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c15dc20 0x7f056c0ce670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 -- 172.21.15.60:0/1546883745 >> 172.21.15.60:0/1546883745 conn(0x7f056c0d0420 msgr2=0x7f056c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 -- 172.21.15.60:0/1546883745 shutdown_connections 2024-04-25T08:30:10.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.285+0000 7f057415c640 1 -- 172.21.15.60:0/1546883745 wait complete. 2024-04-25T08:30:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.286+0000 7f057415c640 1 Processor -- start 2024-04-25T08:30:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.286+0000 7f057415c640 1 -- start start 2024-04-25T08:30:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f057415c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 0x7f056c1f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f057415c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f056c15dc20 0x7f056c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f057415c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056c161710 0x7f056c1f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f057415c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0b5e50 con 0x7f056c0b9330 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f057415c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c162500 con 0x7f056c161710 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f057415c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c0b5fd0 con 0x7f056c15dc20 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 0x7f056c1f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 0x7f056c1f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55378/0 (socket says 172.21.15.60:55378) 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 -- 172.21.15.60:0/2207127813 learned_addr learned my addr 172.21.15.60:0/2207127813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f05726f9640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056c161710 0x7f056c1f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f05716f7640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f056c15dc20 0x7f056c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 -- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f056c15dc20 msgr2=0x7f056c0b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f056c15dc20 0x7f056c0b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 -- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056c161710 msgr2=0x7f056c1f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056c161710 0x7f056c1f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.287+0000 7f0571ef8640 1 -- 172.21.15.60:0/2207127813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c1f4bb0 con 0x7f056c0b9330 2024-04-25T08:30:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.288+0000 7f0571ef8640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 0x7f056c1f4670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f05680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.288+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568037040 con 0x7f056c0b9330 2024-04-25T08:30:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.288+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568042070 con 0x7f056c0b9330 2024-04-25T08:30:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.288+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568030bd0 con 0x7f056c0b9330 2024-04-25T08:30:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.288+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c0aa720 con 0x7f056c0b9330 2024-04-25T08:30:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.288+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c1f5130 con 0x7f056c0b9330 2024-04-25T08:30:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.289+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c0ce340 con 0x7f056c0b9330 2024-04-25T08:30:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.289+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0568047460 con 0x7f056c0b9330 2024-04-25T08:30:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.290+0000 7f05627fc640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f055006b4d0 0x7f055006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.290+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f05680ab200 con 0x7f056c0b9330 2024-04-25T08:30:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.290+0000 7f05716f7640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f055006b4d0 0x7f055006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.290+0000 7f05716f7640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f055006b4d0 0x7f055006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f056c0cf480 tx=0x7f055c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.294+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f056c0ce340 con 0x7f056c0b9330 2024-04-25T08:30:10.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.473+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f056c0cfc20 con 0x7f056c0b9330 2024-04-25T08:30:10.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.473+0000 7f05627fc640 1 -- 172.21.15.60:0/2207127813 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f056c0cfc20 con 0x7f056c0b9330 2024-04-25T08:30:10.471 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f055006b4d0 msgr2=0x7f055006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f055006b4d0 0x7f055006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f056c0cf480 tx=0x7f055c008040 comp rx=0 tx=0).stop 2024-04-25T08:30:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 msgr2=0x7f056c1f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 0x7f056c1f4670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0568000c00 tx=0x7f05680046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 shutdown_connections 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f055006b4d0 0x7f055006d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f056c161710 0x7f056c1f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f056c15dc20 0x7f056c0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 --2- 172.21.15.60:0/2207127813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f056c0b9330 0x7f056c1f4670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 >> 172.21.15.60:0/2207127813 conn(0x7f056c0d0420 msgr2=0x7f056c0b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 shutdown_connections 2024-04-25T08:30:10.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.475+0000 7f057415c640 1 -- 172.21.15.60:0/2207127813 wait complete. 2024-04-25T08:30:10.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.589+0000 7f587bd70640 1 Processor -- start 2024-04-25T08:30:10.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.589+0000 7f587bd70640 1 -- start start 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587bd70640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f587415b7d0 0x7f58740cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587bd70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587bd70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f587406c2d0 0x7f587406c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587bd70640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58740b5fc0 con 0x7f587406c2d0 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587bd70640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587406cca0 con 0x7f587415b7d0 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587bd70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587406cf50 con 0x7f58740b5620 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f587406c2d0 0x7f587406c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f587406c2d0 0x7f587406c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55384/0 (socket says 172.21.15.60:55384) 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 -- 172.21.15.60:0/508838023 learned_addr learned my addr 172.21.15.60:0/508838023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:10.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f5879b0c640 1 --2- 172.21.15.60:0/508838023 >> v2:172.21.15.100:3300/0 conn(0x7f587415b7d0 0x7f58740cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587930b640 1 --2- 172.21.15.60:0/508838023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 -- 172.21.15.60:0/508838023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 msgr2=0x7f58740b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 --2- 172.21.15.60:0/508838023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 -- 172.21.15.60:0/508838023 >> v2:172.21.15.100:3300/0 conn(0x7f587415b7d0 msgr2=0x7f58740cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 --2- 172.21.15.60:0/508838023 >> v2:172.21.15.100:3300/0 conn(0x7f587415b7d0 0x7f58740cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.590+0000 7f587a30d640 1 -- 172.21.15.60:0/508838023 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740ce530 con 0x7f587406c2d0 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.591+0000 7f587a30d640 1 --2- 172.21.15.60:0/508838023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f587406c760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f587408c0d0 tx=0x7f586c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f77cbeec5fd9833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.591+0000 7f5878b0a640 1 -- 172.21.15.60:0/508838023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c036070 con 0x7f587406c2d0 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.591+0000 7f5878b0a640 1 -- 172.21.15.60:0/508838023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586c031600 con 0x7f587406c2d0 2024-04-25T08:30:10.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.591+0000 7f5878b0a640 1 -- 172.21.15.60:0/508838023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c0319d0 con 0x7f587406c2d0 2024-04-25T08:30:10.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 -- 172.21.15.60:0/508838023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 msgr2=0x7f587406c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 --2- 172.21.15.60:0/508838023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f587406c760 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f587408c0d0 tx=0x7f586c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 -- 172.21.15.60:0/508838023 shutdown_connections 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 --2- 172.21.15.60:0/508838023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f587406c760 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 --2- 172.21.15.60:0/508838023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 --2- 172.21.15.60:0/508838023 >> v2:172.21.15.100:3300/0 conn(0x7f587415b7d0 0x7f58740cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 -- 172.21.15.60:0/508838023 >> 172.21.15.60:0/508838023 conn(0x7f58740d0800 msgr2=0x7f58740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 -- 172.21.15.60:0/508838023 shutdown_connections 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.592+0000 7f587bd70640 1 -- 172.21.15.60:0/508838023 wait complete. 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.593+0000 7f587bd70640 1 Processor -- start 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.593+0000 7f587bd70640 1 -- start start 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587bd70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f58741550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587bd70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587bd70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587415b7d0 0x7f5874160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587bd70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587406d230 con 0x7f587406c2d0 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587bd70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740ceaf0 con 0x7f587415b7d0 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587bd70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58740cf550 con 0x7f58740b5620 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f58741550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f58741550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55390/0 (socket says 172.21.15.60:55390) 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 -- 172.21.15.60:0/2226783146 learned_addr learned my addr 172.21.15.60:0/2226783146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587930b640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587a30d640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587415b7d0 0x7f5874160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 -- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 msgr2=0x7f58740cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 -- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587415b7d0 msgr2=0x7f5874160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587415b7d0 0x7f5874160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 -- 172.21.15.60:0/2226783146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740cb850 con 0x7f587406c2d0 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f587a30d640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587415b7d0 0x7f5874160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:10.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.594+0000 7f5879b0c640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f58741550e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f587000e8a0 tx=0x7f587000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.595+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870017070 con 0x7f587406c2d0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.595+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587000c940 con 0x7f587406c2d0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.595+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58740cbb30 con 0x7f587406c2d0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.595+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000cd30 con 0x7f587406c2d0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.595+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58740cc0f0 con 0x7f587406c2d0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.596+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5874073a00 con 0x7f587406c2d0 2024-04-25T08:30:10.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.596+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f587001a070 con 0x7f587406c2d0 2024-04-25T08:30:10.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.597+0000 7f586a7fc640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f585806b4d0 0x7f585806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.597+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f587008a830 con 0x7f587406c2d0 2024-04-25T08:30:10.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.597+0000 7f587930b640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f585806b4d0 0x7f585806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.597+0000 7f587930b640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f585806b4d0 0x7f585806d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5874068150 tx=0x7f5864008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.601+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58700592b0 con 0x7f587406c2d0 2024-04-25T08:30:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.779+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58740d36d0 con 0x7f587406c2d0 2024-04-25T08:30:10.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.780+0000 7f586a7fc640 1 -- 172.21.15.60:0/2226783146 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5870058bb0 con 0x7f587406c2d0 2024-04-25T08:30:10.777 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.781+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f585806b4d0 msgr2=0x7f585806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.781+0000 7f587bd70640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f585806b4d0 0x7f585806d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5874068150 tx=0x7f5864008040 comp rx=0 tx=0).stop 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.781+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 msgr2=0x7f58741550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.781+0000 7f587bd70640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f58741550e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f587000e8a0 tx=0x7f587000ed60 comp rx=0 tx=0).stop 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.781+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 shutdown_connections 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f585806b4d0 0x7f585806d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f587415b7d0 0x7f5874160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58740b5620 0x7f58740cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 --2- 172.21.15.60:0/2226783146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f587406c2d0 0x7f58741550e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 >> 172.21.15.60:0/2226783146 conn(0x7f58740d0800 msgr2=0x7f58740d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 shutdown_connections 2024-04-25T08:30:10.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.782+0000 7f587bd70640 1 -- 172.21.15.60:0/2226783146 wait complete. 2024-04-25T08:30:10.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:10.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.896+0000 7f152765a640 1 Processor -- start 2024-04-25T08:30:10.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.896+0000 7f152765a640 1 -- start start 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f152765a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15200d3440 0x7f15200d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f152765a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200d23a0 con 0x7f15200ced80 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f152765a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200d2680 con 0x7f152015dc20 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f152765a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200d2960 con 0x7f15200d3440 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f1524bf5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f15200ced80 0x7f15200b6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48120/0 (socket says 172.21.15.60:48120) 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f1524bf5640 1 -- 172.21.15.60:0/3063195170 learned_addr learned my addr 172.21.15.60:0/3063195170 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f1525bf7640 1 --2- 172.21.15.60:0/3063195170 >> v2:172.21.15.100:3300/0 conn(0x7f15200d3440 0x7f15200d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815907303 0 0) 0x7f15200d23a0 con 0x7f15200ced80 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.897+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f150c003520 con 0x7f15200ced80 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2302744768 0 0) 0x7f150c003520 con 0x7f15200ced80 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 >> v2:172.21.15.100:3300/0 conn(0x7f15200d3440 msgr2=0x7f15200d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:10.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 --2- 172.21.15.60:0/3063195170 >> v2:172.21.15.100:3300/0 conn(0x7f15200d3440 0x7f15200d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 >> v1:172.21.15.100:6789/0 conn(0x7f152015dc20 legacy=0x7f15200ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200d2c40 con 0x7f15200ced80 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1510001670 con 0x7f15200ced80 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 >> v1:172.21.15.60:6789/0 conn(0x7f15200ced80 legacy=0x7f15200b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f150c0039b0 0x7f150c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f150c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f150c005120 0x7f150c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200d2960 con 0x7f150c0043b0 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c00c060 con 0x7f150c0039b0 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c00c260 con 0x7f150c005120 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1524bf5640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f150c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f15253f6640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f150c0039b0 0x7f150c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1525bf7640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f150c005120 0x7f150c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1524bf5640 1 -- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f150c005120 msgr2=0x7f150c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1524bf5640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f150c005120 0x7f150c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1524bf5640 1 -- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f150c0039b0 msgr2=0x7f150c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1524bf5640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f150c0039b0 0x7f150c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1524bf5640 1 -- 172.21.15.60:0/3063195170 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c003520 con 0x7f150c0043b0 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.898+0000 7f1525bf7640 1 -- 172.21.15.60:0/3063195170 reap_dead start 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.899+0000 7f1524bf5640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f150c004880 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f151000c240 tx=0x7f150c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46343a3a84e61305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.899+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510007d30 con 0x7f150c0043b0 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.899+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510005af0 con 0x7f150c0043b0 2024-04-25T08:30:10.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.899+0000 7f1517fff640 1 -- 172.21.15.60:0/3063195170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510007070 con 0x7f150c0043b0 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 -- 172.21.15.60:0/3063195170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 msgr2=0x7f150c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f150c004880 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f151000c240 tx=0x7f150c005830 comp rx=0 tx=0).stop 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 -- 172.21.15.60:0/3063195170 shutdown_connections 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 --2- 172.21.15.60:0/3063195170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f150c004880 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 -- 172.21.15.60:0/3063195170 >> 172.21.15.60:0/3063195170 conn(0x7f1520061a20 msgr2=0x7f1520061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 -- 172.21.15.60:0/3063195170 shutdown_connections 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.900+0000 7f152765a640 1 -- 172.21.15.60:0/3063195170 wait complete. 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.901+0000 7f152765a640 1 Processor -- start 2024-04-25T08:30:10.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.901+0000 7f152765a640 1 -- start start 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f1520159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1520159fb0 0x7f152015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f152015a410 0x7f15201604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200d1e60 con 0x7f150c0043b0 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200d1ce0 con 0x7f1520159fb0 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200d1fe0 con 0x7f152015a410 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f1520159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f1520159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55414/0 (socket says 172.21.15.60:55414) 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 -- 172.21.15.60:0/658967636 learned_addr learned my addr 172.21.15.60:0/658967636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f1524bf5640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1520159fb0 0x7f152015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f1525bf7640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f152015a410 0x7f15201604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 -- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f152015a410 msgr2=0x7f15201604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f152015a410 0x7f15201604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 -- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1520159fb0 msgr2=0x7f152015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1520159fb0 0x7f152015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 -- 172.21.15.60:0/658967636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520159d60 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f15253f6640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f1520159820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f151c00f7a0 tx=0x7f151c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00fe50 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c018070 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200c7610 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.902+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00d6b0 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.903+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200c7bd0 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.904+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520073a00 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.904+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f151c00d850 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.904+0000 7f1515ffb640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f150406b4d0 0x7f150406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.905+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f151c08a4d0 con 0x7f150c0043b0 2024-04-25T08:30:10.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.905+0000 7f1524bf5640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f150406b4d0 0x7f150406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:10.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.905+0000 7f1524bf5640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f150406b4d0 0x7f150406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f151000c210 tx=0x7f151002d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:10.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:10.909+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f151c058f50 con 0x7f150c0043b0 2024-04-25T08:30:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.090+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15200b87b0 con 0x7f150c0043b0 2024-04-25T08:30:11.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.090+0000 7f1515ffb640 1 -- 172.21.15.60:0/658967636 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f151c058850 con 0x7f150c0043b0 2024-04-25T08:30:11.088 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:11.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.092+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f150406b4d0 msgr2=0x7f150406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.092+0000 7f152765a640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f150406b4d0 0x7f150406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f151000c210 tx=0x7f151002d000 comp rx=0 tx=0).stop 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.092+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 msgr2=0x7f1520159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.092+0000 7f152765a640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f1520159820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f151c00f7a0 tx=0x7f151c00fc60 comp rx=0 tx=0).stop 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.092+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 shutdown_connections 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f152015a410 0x7f15201604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f150406b4d0 0x7f150406d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1520159fb0 0x7f152015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 --2- 172.21.15.60:0/658967636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f150c0043b0 0x7f1520159820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 >> 172.21.15.60:0/658967636 conn(0x7f1520061a20 msgr2=0x7f152015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 shutdown_connections 2024-04-25T08:30:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.093+0000 7f152765a640 1 -- 172.21.15.60:0/658967636 wait complete. 2024-04-25T08:30:11.101 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:11.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.207+0000 7f461fe9a640 1 Processor -- start 2024-04-25T08:30:11.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.207+0000 7f461fe9a640 1 -- start start 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461fe9a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46180cadb0 0x7f46180cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461fe9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f46180ccfe0 0x7f46180cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461fe9a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180afc80 con 0x7f461815de80 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461fe9a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180cb240 con 0x7f46180cadb0 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461fe9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180ce4d0 con 0x7f46180ccfe0 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46180cadb0 0x7f46180cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f46180cadb0 0x7f46180cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55416/0 (socket says 172.21.15.60:55416) 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 -- 172.21.15.60:0/96734066 learned_addr learned my addr 172.21.15.60:0/96734066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461e437640 1 --2- 172.21.15.60:0/96734066 >> v2:172.21.15.100:3300/0 conn(0x7f46180ccfe0 0x7f46180cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 -- 172.21.15.60:0/96734066 >> v2:172.21.15.100:3300/0 conn(0x7f46180ccfe0 msgr2=0x7f46180cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 --2- 172.21.15.60:0/96734066 >> v2:172.21.15.100:3300/0 conn(0x7f46180ccfe0 0x7f46180cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 -- 172.21.15.60:0/96734066 >> v1:172.21.15.100:6789/0 conn(0x7f461815de80 legacy=0x7f46180cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.208+0000 7f461d435640 1 -- 172.21.15.60:0/96734066 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180ce7b0 con 0x7f46180cadb0 2024-04-25T08:30:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.209+0000 7f461d435640 1 --2- 172.21.15.60:0/96734066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f46180cf6c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f461806e2a0 tx=0x7f460802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee8a6361c5174ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.209+0000 7f461cc34640 1 -- 172.21.15.60:0/96734066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608037070 con 0x7f46180cadb0 2024-04-25T08:30:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.209+0000 7f461cc34640 1 -- 172.21.15.60:0/96734066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608032440 con 0x7f46180cadb0 2024-04-25T08:30:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.209+0000 7f461cc34640 1 -- 172.21.15.60:0/96734066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608032810 con 0x7f46180cadb0 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 -- 172.21.15.60:0/96734066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 msgr2=0x7f46180cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 --2- 172.21.15.60:0/96734066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f46180cf6c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f461806e2a0 tx=0x7f460802a270 comp rx=0 tx=0).stop 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 -- 172.21.15.60:0/96734066 shutdown_connections 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 --2- 172.21.15.60:0/96734066 >> v2:172.21.15.100:3300/0 conn(0x7f46180ccfe0 0x7f46180cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 --2- 172.21.15.60:0/96734066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f46180cf6c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 -- 172.21.15.60:0/96734066 >> 172.21.15.60:0/96734066 conn(0x7f46180b2390 msgr2=0x7f46180b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 -- 172.21.15.60:0/96734066 shutdown_connections 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.210+0000 7f461fe9a640 1 -- 172.21.15.60:0/96734066 wait complete. 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.211+0000 7f461fe9a640 1 Processor -- start 2024-04-25T08:30:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.211+0000 7f461fe9a640 1 -- start start 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461fe9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f461814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461fe9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46180ccfe0 0x7f4618159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461fe9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f461815de80 0x7f461815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461fe9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461815e260 con 0x7f46180cadb0 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461fe9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180cfc00 con 0x7f46180ccfe0 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461fe9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46180b2eb0 con 0x7f461815de80 2024-04-25T08:30:11.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f461814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f461814cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55432/0 (socket says 172.21.15.60:55432) 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 -- 172.21.15.60:0/2816556480 learned_addr learned my addr 172.21.15.60:0/2816556480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461d435640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46180ccfe0 0x7f4618159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461e437640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f461815de80 0x7f461815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 -- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f461815de80 msgr2=0x7f461815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f461815de80 0x7f461815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 -- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46180ccfe0 msgr2=0x7f4618159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46180ccfe0 0x7f4618159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.212+0000 7f461dc36640 1 -- 172.21.15.60:0/2816556480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461814e000 con 0x7f46180cadb0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.213+0000 7f461dc36640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f461814cac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f461400c3c0 tx=0x7f461400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.213+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46140052d0 con 0x7f46180cadb0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.213+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614005490 con 0x7f46180cadb0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.213+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461814d240 con 0x7f46180cadb0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.213+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614007260 con 0x7f46180cadb0 2024-04-25T08:30:11.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.213+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461814d800 con 0x7f46180cadb0 2024-04-25T08:30:11.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.214+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4618067be0 con 0x7f46180cadb0 2024-04-25T08:30:11.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.214+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4614007400 con 0x7f46180cadb0 2024-04-25T08:30:11.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.215+0000 7f460e7fc640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45fc06b4d0 0x7f45fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.215+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4614089aa0 con 0x7f46180cadb0 2024-04-25T08:30:11.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.215+0000 7f461d435640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45fc06b4d0 0x7f45fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.215+0000 7f461d435640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45fc06b4d0 0x7f45fc06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4608034000 tx=0x7f460802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:11.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.219+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4618067be0 con 0x7f46180cadb0 2024-04-25T08:30:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.400+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4618073a00 con 0x7f46180cadb0 2024-04-25T08:30:11.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.400+0000 7f460e7fc640 1 -- 172.21.15.60:0/2816556480 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4614058520 con 0x7f46180cadb0 2024-04-25T08:30:11.398 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.402+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45fc06b4d0 msgr2=0x7f45fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.402+0000 7f461fe9a640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45fc06b4d0 0x7f45fc06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4608034000 tx=0x7f460802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.402+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 msgr2=0x7f461814cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.402+0000 7f461fe9a640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f461814cac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f461400c3c0 tx=0x7f461400c880 comp rx=0 tx=0).stop 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 shutdown_connections 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f461815de80 0x7f461815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45fc06b4d0 0x7f45fc06d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46180ccfe0 0x7f4618159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 --2- 172.21.15.60:0/2816556480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f46180cadb0 0x7f461814cac0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 >> 172.21.15.60:0/2816556480 conn(0x7f46180b2390 msgr2=0x7f461815b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 shutdown_connections 2024-04-25T08:30:11.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.403+0000 7f461fe9a640 1 -- 172.21.15.60:0/2816556480 wait complete. 2024-04-25T08:30:11.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:30:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.515+0000 7f48d03e2640 1 Processor -- start 2024-04-25T08:30:11.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.515+0000 7f48d03e2640 1 -- start start 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.516+0000 7f48d03e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 0x7f48c80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.516+0000 7f48d03e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c80cf320 con 0x7f48c80b5c40 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.516+0000 7f48d03e2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c80cf040 con 0x7f48c815b5a0 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.516+0000 7f48d03e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8161970 con 0x7f48c815b980 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd97d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f48c80b5c40 0x7f48c80ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48130/0 (socket says 172.21.15.60:48130) 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd97d640 1 -- 172.21.15.60:0/58806609 learned_addr learned my addr 172.21.15.60:0/58806609 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48ce97f640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 0x7f48c80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884369062 0 0) 0x7f48bc001290 con 0x7f48c80b5c40 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b0003520 con 0x7f48c80b5c40 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 521642463 0 0) 0x7f48b0003520 con 0x7f48c80b5c40 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 msgr2=0x7f48c80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 0x7f48c80cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 >> v1:172.21.15.100:6789/0 conn(0x7f48c815b5a0 legacy=0x7f48c80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c8161c50 con 0x7f48c80b5c40 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f48bc0019c0 con 0x7f48c80b5c40 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 >> v1:172.21.15.60:6789/0 conn(0x7f48c80b5c40 legacy=0x7f48c80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 0x7f48b00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0003f20 0x7f48b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b0004d40 0x7f48b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8161970 con 0x7f48b0004d40 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b000ba20 con 0x7f48b0003f20 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b000bc20 con 0x7f48c815b980 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48ce97f640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b0004d40 0x7f48b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48cd97d640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0003f20 0x7f48b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.517+0000 7f48ce17e640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 0x7f48b00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48ce97f640 1 -- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 msgr2=0x7f48b00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48ce97f640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c815b980 0x7f48b00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48ce97f640 1 -- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0003f20 msgr2=0x7f48b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48ce97f640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0003f20 0x7f48b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48ce97f640 1 -- 172.21.15.60:0/58806609 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b0003520 con 0x7f48b0004d40 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48ce97f640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b0004d40 0x7f48b00051c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009a90 tx=0x7f48b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee2901fbd4025b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0036070 con 0x7f48b0004d40 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0031400 con 0x7f48b0004d40 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.518+0000 7f48cd17c640 1 -- 172.21.15.60:0/58806609 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0009690 con 0x7f48b0004d40 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48d03e2640 1 -- 172.21.15.60:0/58806609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b0004d40 msgr2=0x7f48b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48d03e2640 1 --2- 172.21.15.60:0/58806609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b0004d40 0x7f48b00051c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009a90 tx=0x7f48b0005830 comp rx=0 tx=0).stop 2024-04-25T08:30:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48ce97f640 1 -- 172.21.15.60:0/58806609 reap_dead start 2024-04-25T08:30:11.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48d03e2640 1 -- 172.21.15.60:0/58806609 shutdown_connections 2024-04-25T08:30:11.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48d03e2640 1 -- 172.21.15.60:0/58806609 >> 172.21.15.60:0/58806609 conn(0x7f48c80d0530 msgr2=0x7f48c80d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:11.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48d03e2640 1 -- 172.21.15.60:0/58806609 shutdown_connections 2024-04-25T08:30:11.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.519+0000 7f48d03e2640 1 -- 172.21.15.60:0/58806609 wait complete. 2024-04-25T08:30:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.520+0000 7f48d03e2640 1 Processor -- start 2024-04-25T08:30:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.520+0000 7f48d03e2640 1 -- start start 2024-04-25T08:30:11.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48d03e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0004d40 0x7f48c814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48d03e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c8158560 0x7f48c80c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48d03e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 0x7f48c80c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48d03e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c81625d0 con 0x7f48c80b6d50 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48d03e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8162450 con 0x7f48b0004d40 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48d03e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8162750 con 0x7f48c8158560 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 0x7f48c80c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 0x7f48c80c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55442/0 (socket says 172.21.15.60:55442) 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 -- 172.21.15.60:0/396402115 learned_addr learned my addr 172.21.15.60:0/396402115 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 -- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c8158560 msgr2=0x7f48c80c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c8158560 0x7f48c80c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce17e640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0004d40 0x7f48c814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 -- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0004d40 msgr2=0x7f48c814d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0004d40 0x7f48c814d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.521+0000 7f48ce97f640 1 -- 172.21.15.60:0/396402115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c80b7da0 con 0x7f48c80b6d50 2024-04-25T08:30:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.522+0000 7f48ce97f640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 0x7f48c80c7cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48c002ab70 tx=0x7f48c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.522+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0036040 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.522+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0041070 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.522+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c80b8080 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.522+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0009690 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.522+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c80b8640 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.523+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c8073a00 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.523+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48c0039070 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.524+0000 7f48c6ffd640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f48b406b4d0 0x7f48b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.524+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f48c00a9dc0 con 0x7f48c80b6d50 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.524+0000 7f48ce17e640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f48b406b4d0 0x7f48b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.524+0000 7f48ce17e640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f48b406b4d0 0x7f48b406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48b8003f20 tx=0x7f48b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:11.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.528+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48c004a020 con 0x7f48c80b6d50 2024-04-25T08:30:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.718+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f48c80d17c0 con 0x7f48c80b6d50 2024-04-25T08:30:11.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.719+0000 7f48c6ffd640 1 -- 172.21.15.60:0/396402115 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f48c00787b0 con 0x7f48c80b6d50 2024-04-25T08:30:11.717 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f48b406b4d0 msgr2=0x7f48b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f48b406b4d0 0x7f48b406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48b8003f20 tx=0x7f48b8009040 comp rx=0 tx=0).stop 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 msgr2=0x7f48c80c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 0x7f48c80c7cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48c002ab70 tx=0x7f48c0004730 comp rx=0 tx=0).stop 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 shutdown_connections 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48c80b6d50 0x7f48c80c7cd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48c8158560 0x7f48c80c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f48b406b4d0 0x7f48b406d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 --2- 172.21.15.60:0/396402115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48b0004d40 0x7f48c814d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 >> 172.21.15.60:0/396402115 conn(0x7f48c80d0530 msgr2=0x7f48c80d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 shutdown_connections 2024-04-25T08:30:11.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:11.721+0000 7f48d03e2640 1 -- 172.21.15.60:0/396402115 wait complete. 2024-04-25T08:30:11.729 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T08:30:11.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:11.729 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T08:30:11.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:30:11.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:11.760 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:30:11.830 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:11.830 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:11.846 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:11.846 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:11.902 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:11.902 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:11.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.103+0000 7f625a610640 1 Processor -- start 2024-04-25T08:30:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.103+0000 7f625a610640 1 -- start start 2024-04-25T08:30:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.103+0000 7f625a610640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f62540d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.103+0000 7f625a610640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62540b97a0 con 0x7f62540b2cb0 2024-04-25T08:30:12.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.103+0000 7f625a610640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62540d1c50 con 0x7f625415dbe0 2024-04-25T08:30:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.103+0000 7f625a610640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625415dfc0 con 0x7f62540d1020 2024-04-25T08:30:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f62537fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f62540b2cb0 0x7f62540d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39180/0 (socket says 172.21.15.60:39180) 2024-04-25T08:30:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f62537fe640 1 -- 172.21.15.60:0/2290826966 learned_addr learned my addr 172.21.15.60:0/2290826966 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6258bad640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f62540d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628937850 0 0) 0x7f62540b97a0 con 0x7f62540b2cb0 2024-04-25T08:30:12.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f623c003520 con 0x7f62540b2cb0 2024-04-25T08:30:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332134439 0 0) 0x7f62540d1c50 con 0x7f625415dbe0 2024-04-25T08:30:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62540b97a0 con 0x7f625415dbe0 2024-04-25T08:30:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101887058 0 0) 0x7f623c003520 con 0x7f62540b2cb0 2024-04-25T08:30:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 msgr2=0x7f62540d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f62540d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 >> v1:172.21.15.100:6789/0 conn(0x7f625415dbe0 legacy=0x7f62540d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540d1f30 con 0x7f62540b2cb0 2024-04-25T08:30:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6248001670 con 0x7f62540b2cb0 2024-04-25T08:30:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6258bad640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f62540d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 >> v1:172.21.15.60:6789/0 conn(0x7f62540b2cb0 legacy=0x7f62540d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c003b70 0x7f623c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f623c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f623c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00b9f0 con 0x7f623c004800 2024-04-25T08:30:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00bba0 con 0x7f623c003b70 2024-04-25T08:30:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00bda0 con 0x7f62540d1020 2024-04-25T08:30:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.104+0000 7f62537fe640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f623c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f6253fff640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c003b70 0x7f623c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f6258bad640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f623c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f62537fe640 1 -- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 msgr2=0x7f623c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:12.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f62537fe640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540d1020 0x7f623c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f62537fe640 1 -- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c003b70 msgr2=0x7f623c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:12.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f62537fe640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c003b70 0x7f623c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f62537fe640 1 -- 172.21.15.60:0/2290826966 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c003520 con 0x7f623c004800 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.105+0000 7f62537fe640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f623c004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f62540d3930 tx=0x7f623c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d0df8264e9d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248007ca0 con 0x7f623c004800 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248005ad0 con 0x7f623c004800 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f6252ffd640 1 -- 172.21.15.60:0/2290826966 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248007070 con 0x7f623c004800 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f625a610640 1 -- 172.21.15.60:0/2290826966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 msgr2=0x7f623c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f625a610640 1 --2- 172.21.15.60:0/2290826966 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f623c004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f62540d3930 tx=0x7f623c005830 comp rx=0 tx=0).stop 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f6258bad640 1 -- 172.21.15.60:0/2290826966 reap_dead start 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f625a610640 1 -- 172.21.15.60:0/2290826966 shutdown_connections 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f625a610640 1 -- 172.21.15.60:0/2290826966 >> 172.21.15.60:0/2290826966 conn(0x7f6254061a20 msgr2=0x7f6254061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f625a610640 1 -- 172.21.15.60:0/2290826966 shutdown_connections 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.106+0000 7f625a610640 1 -- 172.21.15.60:0/2290826966 wait complete. 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.107+0000 7f625a610640 1 Processor -- start 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 -- start start 2024-04-25T08:30:12.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f62541453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62540c94d0 0x7f6254145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540b4d60 0x7f62540b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540b92a0 con 0x7f623c004800 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540d1a00 con 0x7f62540c94d0 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f625a610640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540b9420 con 0x7f62540b4d60 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f6253fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f62541453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f6253fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f62541453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45524/0 (socket says 172.21.15.60:45524) 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.108+0000 7f6253fff640 1 -- 172.21.15.60:0/2362131832 learned_addr learned my addr 172.21.15.60:0/2362131832 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f62537fe640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62540c94d0 0x7f6254145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6258bad640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540b4d60 0x7f62540b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6253fff640 1 -- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540b4d60 msgr2=0x7f62540b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6253fff640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540b4d60 0x7f62540b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6253fff640 1 -- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62540c94d0 msgr2=0x7f6254145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6253fff640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62540c94d0 0x7f6254145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6253fff640 1 -- 172.21.15.60:0/2362131832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62541458e0 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f62537fe640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62540c94d0 0x7f6254145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6258bad640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540b4d60 0x7f62540b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6253fff640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f62541453a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f624400f7d0 tx=0x7f624400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624401a070 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624400fe80 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244007050 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254142c30 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.109+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541431f0 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.110+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254073990 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.111+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f624401d070 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.111+0000 7f6250ff9640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f624006b500 0x7f624006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.111+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6244089d50 con 0x7f623c004800 2024-04-25T08:30:12.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.111+0000 7f62537fe640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f624006b500 0x7f624006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.112+0000 7f62537fe640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f624006b500 0x7f624006d9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6248032000 tx=0x7f6248009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.115+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f624408d030 con 0x7f623c004800 2024-04-25T08:30:12.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.299+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6254155280 con 0x7f623c004800 2024-04-25T08:30:12.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.301+0000 7f6250ff9640 1 -- 172.21.15.60:0/2362131832 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7f62541458e0 con 0x7f623c004800 2024-04-25T08:30:12.298 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:12.299 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f624006b500 msgr2=0x7f624006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f624006b500 0x7f624006d9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6248032000 tx=0x7f6248009030 comp rx=0 tx=0).stop 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 msgr2=0x7f62541453a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f62541453a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f624400f7d0 tx=0x7f624400fc90 comp rx=0 tx=0).stop 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 shutdown_connections 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62540b4d60 0x7f62540b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f624006b500 0x7f624006d9c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62540c94d0 0x7f6254145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 --2- 172.21.15.60:0/2362131832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f623c004800 0x7f62541453a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 >> 172.21.15.60:0/2362131832 conn(0x7f6254061a20 msgr2=0x7f62540b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 shutdown_connections 2024-04-25T08:30:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.303+0000 7f625a610640 1 -- 172.21.15.60:0/2362131832 wait complete. 2024-04-25T08:30:12.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:12.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:12.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.419+0000 7feba3207640 1 Processor -- start 2024-04-25T08:30:12.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.419+0000 7feba3207640 1 -- start start 2024-04-25T08:30:12.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.419+0000 7feba3207640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb9c161240 0x7feb9c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.419+0000 7feba3207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba3207640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9c161620 con 0x7feb9c0b54a0 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba3207640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9c0b7120 con 0x7feb9c161240 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba3207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c161e00 con 0x7feb9c161890 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb9c161240 0x7feb9c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb9c161240 0x7feb9c0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45538/0 (socket says 172.21.15.60:45538) 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 -- 172.21.15.60:0/2873607920 learned_addr learned my addr 172.21.15.60:0/2873607920 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feb9bfff640 1 --2- 172.21.15.60:0/2873607920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 -- 172.21.15.60:0/2873607920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 msgr2=0x7feb9c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 --2- 172.21.15.60:0/2873607920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 -- 172.21.15.60:0/2873607920 >> v1:172.21.15.100:6789/0 conn(0x7feb9c0b54a0 legacy=0x7feb9c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.420+0000 7feba0fa3640 1 -- 172.21.15.60:0/2873607920 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c1620e0 con 0x7feb9c161240 2024-04-25T08:30:12.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba0fa3640 1 --2- 172.21.15.60:0/2873607920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c161240 0x7feb9c0b6be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb90009870 tx=0x7feb9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d572cd54f2454501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feb9b7fe640 1 -- 172.21.15.60:0/2873607920 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90037070 con 0x7feb9c161240 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feb9b7fe640 1 -- 172.21.15.60:0/2873607920 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb90032440 con 0x7feb9c161240 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feb9b7fe640 1 -- 172.21.15.60:0/2873607920 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90032810 con 0x7feb9c161240 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 -- 172.21.15.60:0/2873607920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c161240 msgr2=0x7feb9c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 --2- 172.21.15.60:0/2873607920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c161240 0x7feb9c0b6be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb90009870 tx=0x7feb9002a270 comp rx=0 tx=0).stop 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 -- 172.21.15.60:0/2873607920 shutdown_connections 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 --2- 172.21.15.60:0/2873607920 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 --2- 172.21.15.60:0/2873607920 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c161240 0x7feb9c0b6be0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 -- 172.21.15.60:0/2873607920 >> 172.21.15.60:0/2873607920 conn(0x7feb9c0d04a0 msgr2=0x7feb9c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 -- 172.21.15.60:0/2873607920 shutdown_connections 2024-04-25T08:30:12.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.421+0000 7feba3207640 1 -- 172.21.15.60:0/2873607920 wait complete. 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 Processor -- start 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 -- start start 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 0x7feb9c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c161240 0x7feb9c0d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c0d15f0 con 0x7feb9c0b54a0 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c1626a0 con 0x7feb9c161240 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.423+0000 7feba3207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c0cf000 con 0x7feb9c161890 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 0x7feb9c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 0x7feb9c154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45550/0 (socket says 172.21.15.60:45550) 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 -- 172.21.15.60:0/2023231797 learned_addr learned my addr 172.21.15.60:0/2023231797 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feb9bfff640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c161240 0x7feb9c0d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba17a4640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 -- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 msgr2=0x7feb9c0d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 -- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c161240 msgr2=0x7feb9c0d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c161240 0x7feb9c0d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 -- 172.21.15.60:0/2023231797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c0c75f0 con 0x7feb9c0b54a0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feb9bfff640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c161240 0x7feb9c0d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba0fa3640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 0x7feb9c154050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb90009870 tx=0x7feb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90041070 con 0x7feb9c0b54a0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9003c890 con 0x7feb9c0b54a0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9c0cc660 con 0x7feb9c0b54a0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb90037040 con 0x7feb9c0b54a0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.424+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c0cc940 con 0x7feb9c0b54a0 2024-04-25T08:30:12.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.425+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9c073a00 con 0x7feb9c0b54a0 2024-04-25T08:30:12.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.426+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb90044070 con 0x7feb9c0b54a0 2024-04-25T08:30:12.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.426+0000 7feb997fa640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb8006b4d0 0x7feb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.426+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feb900ab080 con 0x7feb9c0b54a0 2024-04-25T08:30:12.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.426+0000 7feb9bfff640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb8006b4d0 0x7feb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.427+0000 7feb9bfff640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb8006b4d0 0x7feb8006d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb88006fd0 tx=0x7feb88008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.431+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb90079c20 con 0x7feb9c0b54a0 2024-04-25T08:30:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.610+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb9c15c670 con 0x7feb9c0b54a0 2024-04-25T08:30:12.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.610+0000 7feb997fa640 1 -- 172.21.15.60:0/2023231797 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb90079520 con 0x7feb9c0b54a0 2024-04-25T08:30:12.608 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb8006b4d0 msgr2=0x7feb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb8006b4d0 0x7feb8006d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb88006fd0 tx=0x7feb88008040 comp rx=0 tx=0).stop 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 msgr2=0x7feb9c154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 0x7feb9c154050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb90009870 tx=0x7feb900046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 shutdown_connections 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb9c161890 0x7feb9c0d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb8006b4d0 0x7feb8006d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb9c161240 0x7feb9c0d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 --2- 172.21.15.60:0/2023231797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb9c0b54a0 0x7feb9c154050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 >> 172.21.15.60:0/2023231797 conn(0x7feb9c0d04a0 msgr2=0x7feb9c15b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 shutdown_connections 2024-04-25T08:30:12.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.612+0000 7feba3207640 1 -- 172.21.15.60:0/2023231797 wait complete. 2024-04-25T08:30:12.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:12.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.727+0000 7f9312948640 1 Processor -- start 2024-04-25T08:30:12.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.727+0000 7f9312948640 1 -- start start 2024-04-25T08:30:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9312948640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f930c06abe0 0x7f930c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9312948640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f930c15dd80 0x7f930c0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9312948640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930c06b500 con 0x7f930c0ad570 2024-04-25T08:30:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9312948640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930c06e410 con 0x7f930c15dd80 2024-04-25T08:30:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9312948640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930c0b3930 con 0x7f930c06abe0 2024-04-25T08:30:12.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f930c15dd80 0x7f930c0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9310ee5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f930c0ad570 0x7f930c06ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39188/0 (socket says 172.21.15.60:39188) 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f9310ee5640 1 -- 172.21.15.60:0/4218306028 learned_addr learned my addr 172.21.15.60:0/4218306028 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930bfff640 1 --2- 172.21.15.60:0/4218306028 >> v2:172.21.15.100:3300/0 conn(0x7f930c06abe0 0x7f930c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930b7fe640 1 -- 172.21.15.60:0/4218306028 >> v2:172.21.15.100:3300/0 conn(0x7f930c06abe0 msgr2=0x7f930c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930b7fe640 1 --2- 172.21.15.60:0/4218306028 >> v2:172.21.15.100:3300/0 conn(0x7f930c06abe0 0x7f930c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930b7fe640 1 -- 172.21.15.60:0/4218306028 >> v1:172.21.15.60:6789/0 conn(0x7f930c0ad570 legacy=0x7f930c06ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930affd640 1 -- 172.21.15.60:0/4218306028 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488907917 0 0) 0x7f930c06b500 con 0x7f930c0ad570 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.728+0000 7f930b7fe640 1 -- 172.21.15.60:0/4218306028 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0b3c10 con 0x7f930c15dd80 2024-04-25T08:30:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.729+0000 7f930bfff640 1 --2- 172.21.15.60:0/4218306028 >> v2:172.21.15.100:3300/0 conn(0x7f930c06abe0 0x7f930c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:12.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.729+0000 7f930b7fe640 1 --2- 172.21.15.60:0/4218306028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c15dd80 0x7f930c0b5290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f930c06e590 tx=0x7f930002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7509d887e05a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.729+0000 7f930affd640 1 -- 172.21.15.60:0/4218306028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300037070 con 0x7f930c15dd80 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.729+0000 7f930affd640 1 -- 172.21.15.60:0/4218306028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300032440 con 0x7f930c15dd80 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.729+0000 7f930affd640 1 -- 172.21.15.60:0/4218306028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300032810 con 0x7f930c15dd80 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 -- 172.21.15.60:0/4218306028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c15dd80 msgr2=0x7f930c0b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 --2- 172.21.15.60:0/4218306028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c15dd80 0x7f930c0b5290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f930c06e590 tx=0x7f930002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 -- 172.21.15.60:0/4218306028 shutdown_connections 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 --2- 172.21.15.60:0/4218306028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c15dd80 0x7f930c0b5290 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 --2- 172.21.15.60:0/4218306028 >> v2:172.21.15.100:3300/0 conn(0x7f930c06abe0 0x7f930c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 -- 172.21.15.60:0/4218306028 >> 172.21.15.60:0/4218306028 conn(0x7f930c0b5af0 msgr2=0x7f930c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 -- 172.21.15.60:0/4218306028 shutdown_connections 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.730+0000 7f9312948640 1 -- 172.21.15.60:0/4218306028 wait complete. 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.731+0000 7f9312948640 1 Processor -- start 2024-04-25T08:30:12.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.731+0000 7f9312948640 1 -- start start 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9312948640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 0x7f930c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9312948640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c0ad570 0x7f930c1e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9312948640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f930c15dd80 0x7f930c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9312948640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c0b57d0 con 0x7f930c06abe0 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9312948640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c06e940 con 0x7f930c0ad570 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9312948640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930c070280 con 0x7f930c15dd80 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 0x7f930c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 0x7f930c1491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45578/0 (socket says 172.21.15.60:45578) 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 -- 172.21.15.60:0/675395860 learned_addr learned my addr 172.21.15.60:0/675395860 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930b7fe640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c0ad570 0x7f930c1e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f9310ee5640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f930c15dd80 0x7f930c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 -- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f930c15dd80 msgr2=0x7f930c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f930c15dd80 0x7f930c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 -- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c0ad570 msgr2=0x7f930c1e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c0ad570 0x7f930c1e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.732+0000 7f930bfff640 1 -- 172.21.15.60:0/675395860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c1496e0 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f9310ee5640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f930c15dd80 0x7f930c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f930bfff640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 0x7f930c1491a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00c7f0 tx=0x7f92fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc015070 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc007a30 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c1e6ae0 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc007e00 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.733+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c1e70a0 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.734+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c1e77b0 con 0x7f930c06abe0 2024-04-25T08:30:12.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.734+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92fc018070 con 0x7f930c06abe0 2024-04-25T08:30:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.735+0000 7f9308ff9640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92f806b4d0 0x7f92f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.735+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f92fc08a570 con 0x7f930c06abe0 2024-04-25T08:30:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.735+0000 7f930b7fe640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92f806b4d0 0x7f92f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:12.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.736+0000 7f930b7fe640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92f806b4d0 0x7f92f806d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9300034000 tx=0x7f930002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:12.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.739+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92fc058ff0 con 0x7f930c06abe0 2024-04-25T08:30:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.923+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f930c073a00 con 0x7f930c06abe0 2024-04-25T08:30:12.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.924+0000 7f9308ff9640 1 -- 172.21.15.60:0/675395860 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92fc0588f0 con 0x7f930c06abe0 2024-04-25T08:30:12.921 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92f806b4d0 msgr2=0x7f92f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92f806b4d0 0x7f92f806d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9300034000 tx=0x7f930002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 msgr2=0x7f930c1491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 0x7f930c1491a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00c7f0 tx=0x7f92fc00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 shutdown_connections 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f930c15dd80 0x7f930c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92f806b4d0 0x7f92f806d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.925+0000 7f9312948640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f930c0ad570 0x7f930c1e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.926+0000 7f9312948640 1 --2- 172.21.15.60:0/675395860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f930c06abe0 0x7f930c1491a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.926+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 >> 172.21.15.60:0/675395860 conn(0x7f930c0b5af0 msgr2=0x7f930c0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.926+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 shutdown_connections 2024-04-25T08:30:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:12.926+0000 7f9312948640 1 -- 172.21.15.60:0/675395860 wait complete. 2024-04-25T08:30:12.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:13.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.040+0000 7fcf0f683640 1 Processor -- start 2024-04-25T08:30:13.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.040+0000 7fcf0f683640 1 -- start start 2024-04-25T08:30:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0f683640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 0x7fcf080b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0f683640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0806cf50 con 0x7fcf080b5620 2024-04-25T08:30:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0f683640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf080ce530 con 0x7fcf0815b7d0 2024-04-25T08:30:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0f683640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf080ce810 con 0x7fcf0806c9e0 2024-04-25T08:30:13.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0cc1e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcf080b5620 0x7fcf0806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39190/0 (socket says 172.21.15.60:39190) 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0cc1e640 1 -- 172.21.15.60:0/2750316693 learned_addr learned my addr 172.21.15.60:0/2750316693 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 0x7fcf080b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205148603 0 0) 0x7fcf0806cf50 con 0x7fcf080b5620 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcef4003520 con 0x7fcf080b5620 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885431066 0 0) 0x7fcf080ce530 con 0x7fcf0815b7d0 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.041+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf0806cf50 con 0x7fcf0815b7d0 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 469597786 0 0) 0x7fcef4003520 con 0x7fcf080b5620 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 msgr2=0x7fcf080b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 0x7fcf080b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 >> v1:172.21.15.100:6789/0 conn(0x7fcf0815b7d0 legacy=0x7fcf080cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf080ceaf0 con 0x7fcf080b5620 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fcef8001690 con 0x7fcf080b5620 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 >> v1:172.21.15.60:6789/0 conn(0x7fcf080b5620 legacy=0x7fcf0806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4003b70 0x7fcef4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 0x7fcef4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef4004d40 0x7fcef40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf080ce810 con 0x7fcef4004d40 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef400ba70 con 0x7fcef4003b70 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef400bc70 con 0x7fcf0806c9e0 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef4004d40 0x7fcef40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0cc1e640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 0x7fcef4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0d41f640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4003b70 0x7fcef4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0dc20640 1 -- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 msgr2=0x7fcef4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf0806c9e0 0x7fcef4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0dc20640 1 -- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4003b70 msgr2=0x7fcef4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef4003b70 0x7fcef4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.042+0000 7fcf0dc20640 1 -- 172.21.15.60:0/2750316693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4003520 con 0x7fcef4004d40 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef4004d40 0x7fcef40051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009fc0 tx=0x7fcef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fade8c1fe60cb4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00009c10 con 0x7fcef4004d40 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00036070 con 0x7fcef4004d40 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcefffff640 1 -- 172.21.15.60:0/2750316693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00031880 con 0x7fcef4004d40 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcf0f683640 1 -- 172.21.15.60:0/2750316693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef4004d40 msgr2=0x7fcef40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcf0f683640 1 --2- 172.21.15.60:0/2750316693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef4004d40 0x7fcef40051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009fc0 tx=0x7fcef4005830 comp rx=0 tx=0).stop 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.043+0000 7fcf0dc20640 1 -- 172.21.15.60:0/2750316693 reap_dead start 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.044+0000 7fcf0f683640 1 -- 172.21.15.60:0/2750316693 shutdown_connections 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.044+0000 7fcf0f683640 1 -- 172.21.15.60:0/2750316693 >> 172.21.15.60:0/2750316693 conn(0x7fcf080d0800 msgr2=0x7fcf080d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:13.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.044+0000 7fcf0f683640 1 -- 172.21.15.60:0/2750316693 shutdown_connections 2024-04-25T08:30:13.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.044+0000 7fcf0f683640 1 -- 172.21.15.60:0/2750316693 wait complete. 2024-04-25T08:30:13.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.045+0000 7fcf0f683640 1 Processor -- start 2024-04-25T08:30:13.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.045+0000 7fcf0f683640 1 -- start start 2024-04-25T08:30:13.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0f683640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef4004d40 0x7fcf0814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0f683640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf080b8300 0x7fcf080cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0f683640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 0x7fcf080cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0f683640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0815bd30 con 0x7fcf080b8760 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0f683640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0815bbb0 con 0x7fcf080b8300 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0f683640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf080b5ab0 con 0x7fcef4004d40 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 0x7fcf080cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 0x7fcf080cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45606/0 (socket says 172.21.15.60:45606) 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 -- 172.21.15.60:0/1328354906 learned_addr learned my addr 172.21.15.60:0/1328354906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0cc1e640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf080b8300 0x7fcf080cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0d41f640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef4004d40 0x7fcf0814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 -- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef4004d40 msgr2=0x7fcf0814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef4004d40 0x7fcf0814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 -- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf080b8300 msgr2=0x7fcf080cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf080b8300 0x7fcf080cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 -- 172.21.15.60:0/1328354906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf080c7570 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcf0dc20640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 0x7fcf080cd8a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000e230 tx=0x7fcf00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.046+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00009c10 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.047+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00031dd0 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.047+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf080c7850 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.047+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00003bd0 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.047+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf080c7e10 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.048+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf08073a00 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.048+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf00005d10 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.048+0000 7fcefdffb640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fceec06b4d0 0x7fceec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.049+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcf000abb30 con 0x7fcf080b8760 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.049+0000 7fcf0d41f640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fceec06b4d0 0x7fceec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.049+0000 7fcf0d41f640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fceec06b4d0 0x7fceec06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf040018d0 tx=0x7fcf04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.053+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf0004a060 con 0x7fcf080b8760 2024-04-25T08:30:13.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.235+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf080b39f0 con 0x7fcf080b8760 2024-04-25T08:30:13.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.236+0000 7fcefdffb640 1 -- 172.21.15.60:0/1328354906 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf00079ed0 con 0x7fcf080b8760 2024-04-25T08:30:13.233 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fceec06b4d0 msgr2=0x7fceec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fceec06b4d0 0x7fceec06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf040018d0 tx=0x7fcf04009040 comp rx=0 tx=0).stop 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 msgr2=0x7fcf080cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 0x7fcf080cd8a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000e230 tx=0x7fcf00009750 comp rx=0 tx=0).stop 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 shutdown_connections 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcf080b8760 0x7fcf080cd8a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fceec06b4d0 0x7fceec06d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf080b8300 0x7fcf080cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 --2- 172.21.15.60:0/1328354906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef4004d40 0x7fcf0814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 >> 172.21.15.60:0/1328354906 conn(0x7fcf080d0800 msgr2=0x7fcf080cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 shutdown_connections 2024-04-25T08:30:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.238+0000 7fcf0f683640 1 -- 172.21.15.60:0/1328354906 wait complete. 2024-04-25T08:30:13.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:13.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.352+0000 7faced39e640 1 Processor -- start 2024-04-25T08:30:13.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.352+0000 7faced39e640 1 -- start start 2024-04-25T08:30:13.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7faced39e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7face815dbd0 0x7face80caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7faced39e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face806ff30 0x7face80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7faced39e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80cc6c0 con 0x7face806bbf0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7faced39e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face80cb0e0 con 0x7face815dbd0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7faced39e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80cb3c0 con 0x7face806ff30 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7face77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7face806bbf0 0x7face806db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39198/0 (socket says 172.21.15.60:39198) 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.353+0000 7face77fe640 1 -- 172.21.15.60:0/3617103187 learned_addr learned my addr 172.21.15.60:0/3617103187 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face6ffd640 1 --2- 172.21.15.60:0/3617103187 >> v2:172.21.15.100:3300/0 conn(0x7face815dbd0 0x7face80caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face67fc640 1 --2- 172.21.15.60:0/3617103187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face806ff30 0x7face80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face5ffb640 1 -- 172.21.15.60:0/3617103187 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139967654 0 0) 0x7face80cc6c0 con 0x7face806bbf0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face5ffb640 1 -- 172.21.15.60:0/3617103187 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd0003520 con 0x7face806bbf0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face5ffb640 1 -- 172.21.15.60:0/3617103187 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9621563 0 0) 0x7facd0003520 con 0x7face806bbf0 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face67fc640 1 -- 172.21.15.60:0/3617103187 >> v2:172.21.15.100:3300/0 conn(0x7face815dbd0 msgr2=0x7face80caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face67fc640 1 --2- 172.21.15.60:0/3617103187 >> v2:172.21.15.100:3300/0 conn(0x7face815dbd0 0x7face80caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face67fc640 1 -- 172.21.15.60:0/3617103187 >> v1:172.21.15.60:6789/0 conn(0x7face806bbf0 legacy=0x7face806db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face67fc640 1 -- 172.21.15.60:0/3617103187 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80cb6a0 con 0x7face806ff30 2024-04-25T08:30:13.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.354+0000 7face6ffd640 1 --2- 172.21.15.60:0/3617103187 >> v2:172.21.15.100:3300/0 conn(0x7face815dbd0 0x7face80caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:13.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.355+0000 7face67fc640 1 --2- 172.21.15.60:0/3617103187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face806ff30 0x7face80cc180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097d0 tx=0x7facdc02a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47661dd964cc3d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.355+0000 7face5ffb640 1 -- 172.21.15.60:0/3617103187 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc037070 con 0x7face806ff30 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.355+0000 7face5ffb640 1 -- 172.21.15.60:0/3617103187 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc02abd0 con 0x7face806ff30 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.355+0000 7face5ffb640 1 -- 172.21.15.60:0/3617103187 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd80019f0 con 0x7face806ff30 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.355+0000 7faced39e640 1 -- 172.21.15.60:0/3617103187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face806ff30 msgr2=0x7face80cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.355+0000 7faced39e640 1 --2- 172.21.15.60:0/3617103187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face806ff30 0x7face80cc180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097d0 tx=0x7facdc02a140 comp rx=0 tx=0).stop 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.356+0000 7faced39e640 1 -- 172.21.15.60:0/3617103187 shutdown_connections 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.356+0000 7faced39e640 1 --2- 172.21.15.60:0/3617103187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face806ff30 0x7face80cc180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.356+0000 7faced39e640 1 --2- 172.21.15.60:0/3617103187 >> v2:172.21.15.100:3300/0 conn(0x7face815dbd0 0x7face80caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.356+0000 7faced39e640 1 -- 172.21.15.60:0/3617103187 >> 172.21.15.60:0/3617103187 conn(0x7face8071af0 msgr2=0x7face8071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.356+0000 7faced39e640 1 -- 172.21.15.60:0/3617103187 shutdown_connections 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.356+0000 7faced39e640 1 -- 172.21.15.60:0/3617103187 wait complete. 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.357+0000 7faced39e640 1 Processor -- start 2024-04-25T08:30:13.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.357+0000 7faced39e640 1 -- start start 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face806bbf0 0x7face80ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 0x7face814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815dbd0 0x7face814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face815dfb0 con 0x7face806ff30 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80cbc60 con 0x7face806bbf0 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face815e130 con 0x7face815dbd0 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 0x7face814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 0x7face814f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45620/0 (socket says 172.21.15.60:45620) 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 -- 172.21.15.60:0/3969552923 learned_addr learned my addr 172.21.15.60:0/3969552923 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face77fe640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815dbd0 0x7face814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face6ffd640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face806bbf0 0x7face80ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 -- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815dbd0 msgr2=0x7face814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815dbd0 0x7face814fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 -- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face806bbf0 msgr2=0x7face80ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face806bbf0 0x7face80ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 -- 172.21.15.60:0/3969552923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80b2c30 con 0x7face806ff30 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face6ffd640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face806bbf0 0x7face80ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7face67fc640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 0x7face814f860 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7facdc02a180 tx=0x7facdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc037040 con 0x7face806ff30 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc042070 con 0x7face806ff30 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc03d440 con 0x7face806ff30 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face80b2f10 con 0x7face806ff30 2024-04-25T08:30:13.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.358+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face80b34d0 con 0x7face806ff30 2024-04-25T08:30:13.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.359+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face8070310 con 0x7face806ff30 2024-04-25T08:30:13.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.360+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facdc03d5e0 con 0x7face806ff30 2024-04-25T08:30:13.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.360+0000 7faccf7fe640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7facd406b4d0 0x7facd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.360+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7facdc0aacc0 con 0x7face806ff30 2024-04-25T08:30:13.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.361+0000 7face6ffd640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7facd406b4d0 0x7facd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.361+0000 7face6ffd640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7facd406b4d0 0x7facd406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7face0009ac0 tx=0x7face0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.365+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7face8070310 con 0x7face806ff30 2024-04-25T08:30:13.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.547+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7face8061b30 con 0x7face806ff30 2024-04-25T08:30:13.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.547+0000 7faccf7fe640 1 -- 172.21.15.60:0/3969552923 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7facdc0797d0 con 0x7face806ff30 2024-04-25T08:30:13.545 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7facd406b4d0 msgr2=0x7facd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7facd406b4d0 0x7facd406d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7face0009ac0 tx=0x7face0008040 comp rx=0 tx=0).stop 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 msgr2=0x7face814f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 0x7face814f860 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7facdc02a180 tx=0x7facdc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 shutdown_connections 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7face815dbd0 0x7face814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7face806ff30 0x7face814f860 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7facd406b4d0 0x7facd406d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 --2- 172.21.15.60:0/3969552923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7face806bbf0 0x7face80ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 >> 172.21.15.60:0/3969552923 conn(0x7face8071af0 msgr2=0x7face80c4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 shutdown_connections 2024-04-25T08:30:13.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.549+0000 7faced39e640 1 -- 172.21.15.60:0/3969552923 wait complete. 2024-04-25T08:30:13.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:30:13.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.663+0000 7f84348fc640 1 Processor -- start 2024-04-25T08:30:13.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.663+0000 7f84348fc640 1 -- start start 2024-04-25T08:30:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.663+0000 7f84348fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 0x7f84300cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.663+0000 7f84348fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84300ccdd0 0x7f84300cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.663+0000 7f84348fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84300cdcd0 0x7f84300cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.663+0000 7f84348fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300d0450 con 0x7f84300ccdd0 2024-04-25T08:30:13.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f84348fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84300cb190 con 0x7f84300cdcd0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f84348fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84300cb470 con 0x7f843015bb20 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84300ccdd0 0x7f84300cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f84300ccdd0 0x7f84300cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45626/0 (socket says 172.21.15.60:45626) 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 -- 172.21.15.60:0/674921886 learned_addr learned my addr 172.21.15.60:0/674921886 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 -- 172.21.15.60:0/674921886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 msgr2=0x7f84300cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 --2- 172.21.15.60:0/674921886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 0x7f84300cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f842ed9d640 1 --2- 172.21.15.60:0/674921886 >> v2:172.21.15.100:3300/0 conn(0x7f84300cdcd0 0x7f84300cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 -- 172.21.15.60:0/674921886 >> v2:172.21.15.100:3300/0 conn(0x7f84300cdcd0 msgr2=0x7f84300cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 --2- 172.21.15.60:0/674921886 >> v2:172.21.15.100:3300/0 conn(0x7f84300cdcd0 0x7f84300cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f8427fff640 1 -- 172.21.15.60:0/674921886 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300cb750 con 0x7f84300ccdd0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.664+0000 7f842ed9d640 1 --2- 172.21.15.60:0/674921886 >> v2:172.21.15.100:3300/0 conn(0x7f84300cdcd0 0x7f84300cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f8427fff640 1 --2- 172.21.15.60:0/674921886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300ccdd0 0x7f84300cf9d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8430071d80 tx=0x7f841802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b528dc4446898051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f842dd9b640 1 -- 172.21.15.60:0/674921886 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418037070 con 0x7f84300ccdd0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f842dd9b640 1 -- 172.21.15.60:0/674921886 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418032440 con 0x7f84300ccdd0 2024-04-25T08:30:13.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f842dd9b640 1 -- 172.21.15.60:0/674921886 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418032810 con 0x7f84300ccdd0 2024-04-25T08:30:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 -- 172.21.15.60:0/674921886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300ccdd0 msgr2=0x7f84300cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 --2- 172.21.15.60:0/674921886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300ccdd0 0x7f84300cf9d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8430071d80 tx=0x7f841802a140 comp rx=0 tx=0).stop 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 -- 172.21.15.60:0/674921886 shutdown_connections 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 --2- 172.21.15.60:0/674921886 >> v2:172.21.15.100:3300/0 conn(0x7f84300cdcd0 0x7f84300cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 --2- 172.21.15.60:0/674921886 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300ccdd0 0x7f84300cf9d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 --2- 172.21.15.60:0/674921886 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 0x7f84300cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 -- 172.21.15.60:0/674921886 >> 172.21.15.60:0/674921886 conn(0x7f84300b2390 msgr2=0x7f84300b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.665+0000 7f84348fc640 1 -- 172.21.15.60:0/674921886 shutdown_connections 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.666+0000 7f84348fc640 1 -- 172.21.15.60:0/674921886 wait complete. 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.667+0000 7f84348fc640 1 Processor -- start 2024-04-25T08:30:13.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.667+0000 7f84348fc640 1 -- start start 2024-04-25T08:30:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.667+0000 7f84348fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84300ccdd0 0x7f8430155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.667+0000 7f84348fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 0x7f84301f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.667+0000 7f84348fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 0x7f84300b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.667+0000 7f84348fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300cc770 con 0x7f84300cdcd0 2024-04-25T08:30:13.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f84348fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84300cbd10 con 0x7f84300ccdd0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f84348fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843015bf00 con 0x7f843015bb20 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 0x7f84301f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 0x7f84301f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45642/0 (socket says 172.21.15.60:45642) 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 -- 172.21.15.60:0/1752026561 learned_addr learned my addr 172.21.15.60:0/1752026561 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f842e59c640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84300ccdd0 0x7f8430155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 -- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 msgr2=0x7f84300b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 0x7f84300b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 -- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84300ccdd0 msgr2=0x7f8430155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84300ccdd0 0x7f8430155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 -- 172.21.15.60:0/1752026561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84301f1d60 con 0x7f84300cdcd0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f842e59c640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84300ccdd0 0x7f8430155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8427fff640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 0x7f84301f1470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f841802a180 tx=0x7f84180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418037040 con 0x7f84300cdcd0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8418042070 con 0x7f84300cdcd0 2024-04-25T08:30:13.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84301f2040 con 0x7f84300cdcd0 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418030bb0 con 0x7f84300cdcd0 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.668+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84300b3480 con 0x7f84300cdcd0 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.669+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8430073a00 con 0x7f84300cdcd0 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.670+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8418047460 con 0x7f84300cdcd0 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.670+0000 7f8426ffd640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f841006b4d0 0x7f841006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.670+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f84180ab290 con 0x7f84300cdcd0 2024-04-25T08:30:13.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.671+0000 7f842e59c640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f841006b4d0 0x7f841006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:13.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.671+0000 7f842e59c640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f841006b4d0 0x7f841006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8420006fd0 tx=0x7f8420008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:13.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.675+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8418045360 con 0x7f84300cdcd0 2024-04-25T08:30:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.870+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8430156540 con 0x7f84300cdcd0 2024-04-25T08:30:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.870+0000 7f8426ffd640 1 -- 172.21.15.60:0/1752026561 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8418079d10 con 0x7f84300cdcd0 2024-04-25T08:30:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:30:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f841006b4d0 msgr2=0x7f841006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f841006b4d0 0x7f841006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8420006fd0 tx=0x7f8420008040 comp rx=0 tx=0).stop 2024-04-25T08:30:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 msgr2=0x7f84301f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 0x7f84301f1470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f841802a180 tx=0x7f84180046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 shutdown_connections 2024-04-25T08:30:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f843015bb20 0x7f84300b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84300cdcd0 0x7f84301f1470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f841006b4d0 0x7f841006d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 --2- 172.21.15.60:0/1752026561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84300ccdd0 0x7f8430155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 >> 172.21.15.60:0/1752026561 conn(0x7f84300b2390 msgr2=0x7f84300adba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 shutdown_connections 2024-04-25T08:30:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:13.872+0000 7f84348fc640 1 -- 172.21.15.60:0/1752026561 wait complete. 2024-04-25T08:30:13.881 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T08:30:13.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:13.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T08:30:13.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:30:13.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:13.906 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:30:13.962 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:30:13.962 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:13.980 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:30:13.980 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:14.039 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:30:14.039 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:14.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:14.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.198+0000 7f6a584f5640 1 Processor -- start 2024-04-25T08:30:14.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.198+0000 7f6a584f5640 1 -- start start 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a584f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a5015dbe0 0x7f6a500d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a584f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a500d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a584f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a500b92a0 0x7f6a500b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a584f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5015dfc0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a584f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a500d1a00 con 0x7f6a500b92a0 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a584f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500b97a0 con 0x7f6a500b2cb0 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a56291640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a5015dbe0 0x7f6a500d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a56291640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a5015dbe0 0x7f6a500d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45654/0 (socket says 172.21.15.60:45654) 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.199+0000 7f6a56291640 1 -- 172.21.15.60:0/3256885435 learned_addr learned my addr 172.21.15.60:0/3256885435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56a92640 1 --2- 172.21.15.60:0/3256885435 >> v2:172.21.15.100:3300/0 conn(0x7f6a500b92a0 0x7f6a500b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56291640 1 -- 172.21.15.60:0/3256885435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 msgr2=0x7f6a500d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:14.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a55a90640 1 --2- 172.21.15.60:0/3256885435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a500d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56291640 1 --2- 172.21.15.60:0/3256885435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a500d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56291640 1 -- 172.21.15.60:0/3256885435 >> v2:172.21.15.100:3300/0 conn(0x7f6a500b92a0 msgr2=0x7f6a500b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56291640 1 --2- 172.21.15.60:0/3256885435 >> v2:172.21.15.100:3300/0 conn(0x7f6a500b92a0 0x7f6a500b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56291640 1 -- 172.21.15.60:0/3256885435 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500d1c50 con 0x7f6a5015dbe0 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.200+0000 7f6a56a92640 1 --2- 172.21.15.60:0/3256885435 >> v2:172.21.15.100:3300/0 conn(0x7f6a500b92a0 0x7f6a500b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a56291640 1 --2- 172.21.15.60:0/3256885435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a500d0910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a5006c640 tx=0x7f6a4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f1656e6ab257be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a5528f640 1 -- 172.21.15.60:0/3256885435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40036070 con 0x7f6a5015dbe0 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a5528f640 1 -- 172.21.15.60:0/3256885435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40031600 con 0x7f6a5015dbe0 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a5528f640 1 -- 172.21.15.60:0/3256885435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400319d0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 -- 172.21.15.60:0/3256885435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 msgr2=0x7f6a500d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 --2- 172.21.15.60:0/3256885435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a500d0910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6a5006c640 tx=0x7f6a4002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 -- 172.21.15.60:0/3256885435 shutdown_connections 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 --2- 172.21.15.60:0/3256885435 >> v2:172.21.15.100:3300/0 conn(0x7f6a500b92a0 0x7f6a500b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 --2- 172.21.15.60:0/3256885435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a500d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 --2- 172.21.15.60:0/3256885435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a500d0910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 -- 172.21.15.60:0/3256885435 >> 172.21.15.60:0/3256885435 conn(0x7f6a50061a20 msgr2=0x7f6a50061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:14.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 -- 172.21.15.60:0/3256885435 shutdown_connections 2024-04-25T08:30:14.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.201+0000 7f6a584f5640 1 -- 172.21.15.60:0/3256885435 wait complete. 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.202+0000 7f6a584f5640 1 Processor -- start 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 -- start start 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a50151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a500b92a0 0x7f6a5015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a5015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500b6ab0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500d2210 con 0x7f6a500b92a0 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a584f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500b6c30 con 0x7f6a500b2cb0 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a56a92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a5015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a56a92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a5015f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45664/0 (socket says 172.21.15.60:45664) 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.203+0000 7f6a56a92640 1 -- 172.21.15.60:0/2049290614 learned_addr learned my addr 172.21.15.60:0/2049290614 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a55a90640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a500b92a0 0x7f6a5015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56291640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a50151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56a92640 1 -- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 msgr2=0x7f6a50151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56a92640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a50151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56a92640 1 -- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a500b92a0 msgr2=0x7f6a5015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56a92640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a500b92a0 0x7f6a5015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56a92640 1 -- 172.21.15.60:0/2049290614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500cb710 con 0x7f6a5015dbe0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a56a92640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a5015f6e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c7f0 tx=0x7f6a4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48015070 con 0x7f6a5015dbe0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48004240 con 0x7f6a5015dbe0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a500b4620 con 0x7f6a5015dbe0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48007620 con 0x7f6a5015dbe0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.204+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a500b4be0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.205+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a501f3400 con 0x7f6a5015dbe0 2024-04-25T08:30:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.206+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a480079d0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.206+0000 7f6a4effd640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.206+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6a4808a5a0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.206+0000 7f6a56291640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.207+0000 7f6a56291640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a40033000 tx=0x7f6a40009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.211+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a480589d0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.393+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a500b4db0 con 0x7f6a5015dbe0 2024-04-25T08:30:14.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.394+0000 7f6a4effd640 1 -- 172.21.15.60:0/2049290614 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7f6a4805ca70 con 0x7f6a5015dbe0 2024-04-25T08:30:14.392 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:14.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3c06b4d0 msgr2=0x7f6a3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a40033000 tx=0x7f6a40009000 comp rx=0 tx=0).stop 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 msgr2=0x7f6a5015f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a5015f6e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c7f0 tx=0x7f6a4800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 shutdown_connections 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a5015dbe0 0x7f6a5015f6e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a500b92a0 0x7f6a5015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 --2- 172.21.15.60:0/2049290614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b2cb0 0x7f6a50151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 >> 172.21.15.60:0/2049290614 conn(0x7f6a50061a20 msgr2=0x7f6a500ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 shutdown_connections 2024-04-25T08:30:14.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.396+0000 7f6a584f5640 1 -- 172.21.15.60:0/2049290614 wait complete. 2024-04-25T08:30:14.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:14.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:14.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.510+0000 7f27cccbc640 1 Processor -- start 2024-04-25T08:30:14.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.511+0000 7f27cccbc640 1 -- start start 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.511+0000 7f27cccbc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 0x7f27c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.511+0000 7f27cccbc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27c806c7e0 0x7f27c80b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.511+0000 7f27cccbc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80d1d50 con 0x7f27c8158230 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.511+0000 7f27cccbc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c8158610 con 0x7f27c806c7e0 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.511+0000 7f27cccbc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c806bea0 con 0x7f27c806c330 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27c6d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27c806c7e0 0x7f27c80b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 0x7f27c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 0x7f27c80b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:60362/0 (socket says 172.21.15.60:60362) 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27c659c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f27c8158230 0x7f27c80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47998/0 (socket says 172.21.15.60:47998) 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 -- 172.21.15.60:0/2003644758 learned_addr learned my addr 172.21.15.60:0/2003644758 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27c5d9b640 1 -- 172.21.15.60:0/2003644758 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425143590 0 0) 0x7f27c80d1d50 con 0x7f27c8158230 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27c5d9b640 1 -- 172.21.15.60:0/2003644758 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b4003520 con 0x7f27c8158230 2024-04-25T08:30:14.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 -- 172.21.15.60:0/2003644758 >> v2:172.21.15.100:3300/0 conn(0x7f27c806c7e0 msgr2=0x7f27c80b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:14.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 --2- 172.21.15.60:0/2003644758 >> v2:172.21.15.100:3300/0 conn(0x7f27c806c7e0 0x7f27c80b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 -- 172.21.15.60:0/2003644758 >> v1:172.21.15.100:6789/0 conn(0x7f27c8158230 legacy=0x7f27c80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27bdd9b640 1 -- 172.21.15.60:0/2003644758 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80d2020 con 0x7f27c806c330 2024-04-25T08:30:14.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.512+0000 7f27c6d9d640 1 --2- 172.21.15.60:0/2003644758 >> v2:172.21.15.100:3300/0 conn(0x7f27c806c7e0 0x7f27c80b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.513+0000 7f27bdd9b640 1 --2- 172.21.15.60:0/2003644758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 0x7f27c80b4e60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009850 tx=0x7f27b002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc8f918a18fd1235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.513+0000 7f27c5d9b640 1 -- 172.21.15.60:0/2003644758 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004030 con 0x7f27c806c330 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.513+0000 7f27c5d9b640 1 -- 172.21.15.60:0/2003644758 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b00041d0 con 0x7f27c806c330 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.513+0000 7f27c5d9b640 1 -- 172.21.15.60:0/2003644758 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0003610 con 0x7f27c806c330 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 -- 172.21.15.60:0/2003644758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 msgr2=0x7f27c80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 --2- 172.21.15.60:0/2003644758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 0x7f27c80b4e60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009850 tx=0x7f27b002a230 comp rx=0 tx=0).stop 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 -- 172.21.15.60:0/2003644758 shutdown_connections 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 --2- 172.21.15.60:0/2003644758 >> v2:172.21.15.100:3300/0 conn(0x7f27c806c7e0 0x7f27c80b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 --2- 172.21.15.60:0/2003644758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c330 0x7f27c80b4e60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 -- 172.21.15.60:0/2003644758 >> 172.21.15.60:0/2003644758 conn(0x7f27c8061600 msgr2=0x7f27c80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:14.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 -- 172.21.15.60:0/2003644758 shutdown_connections 2024-04-25T08:30:14.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.514+0000 7f27cccbc640 1 -- 172.21.15.60:0/2003644758 wait complete. 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.515+0000 7f27cccbc640 1 Processor -- start 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.515+0000 7f27cccbc640 1 -- start start 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27cccbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 0x7f27c8153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27cccbc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c7e0 0x7f27c815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27cccbc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c8158230 0x7f27c815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27cccbc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80d25e0 con 0x7f27c806c330 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27cccbc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c806bb30 con 0x7f27c8158230 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27cccbc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c815e320 con 0x7f27c806c7e0 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 0x7f27c8153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 0x7f27c8153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45672/0 (socket says 172.21.15.60:45672) 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 -- 172.21.15.60:0/4030252640 learned_addr learned my addr 172.21.15.60:0/4030252640 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c6d9d640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c8158230 0x7f27c815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27bdd9b640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c7e0 0x7f27c815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 -- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c7e0 msgr2=0x7f27c815b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c7e0 0x7f27c815b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 -- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c8158230 msgr2=0x7f27c815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c8158230 0x7f27c815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.516+0000 7f27c659c640 1 -- 172.21.15.60:0/4030252640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80cca40 con 0x7f27c806c330 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.517+0000 7f27c659c640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 0x7f27c8153f10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c3c0 tx=0x7f27b800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.517+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80074e0 con 0x7f27c806c330 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.517+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b80047e0 con 0x7f27c806c330 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.517+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c80ccd20 con 0x7f27c806c330 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.517+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004bd0 con 0x7f27c806c330 2024-04-25T08:30:14.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.517+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c80cd2e0 con 0x7f27c806c330 2024-04-25T08:30:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.518+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c806de50 con 0x7f27c806c330 2024-04-25T08:30:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.518+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27b801a020 con 0x7f27c806c330 2024-04-25T08:30:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.519+0000 7f27bf7fe640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a806b4d0 0x7f27a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.519+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f27b8089d50 con 0x7f27c806c330 2024-04-25T08:30:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.519+0000 7f27bdd9b640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a806b4d0 0x7f27a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.519+0000 7f27bdd9b640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a806b4d0 0x7f27a806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27b0033000 tx=0x7f27b002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.523+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27b808d030 con 0x7f27c806c330 2024-04-25T08:30:14.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.705+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27c8073a00 con 0x7f27c806c330 2024-04-25T08:30:14.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.705+0000 7f27bf7fe640 1 -- 172.21.15.60:0/4030252640 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27b80587d0 con 0x7f27c806c330 2024-04-25T08:30:14.703 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a806b4d0 msgr2=0x7f27a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a806b4d0 0x7f27a806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27b0033000 tx=0x7f27b002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 msgr2=0x7f27c8153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 0x7f27c8153f10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c3c0 tx=0x7f27b800c7c0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 shutdown_connections 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27a806b4d0 0x7f27a806d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27c8158230 0x7f27c815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27c806c7e0 0x7f27c815b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 --2- 172.21.15.60:0/4030252640 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27c806c330 0x7f27c8153f10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 >> 172.21.15.60:0/4030252640 conn(0x7f27c8061600 msgr2=0x7f27c80d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 shutdown_connections 2024-04-25T08:30:14.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.707+0000 7f27cccbc640 1 -- 172.21.15.60:0/4030252640 wait complete. 2024-04-25T08:30:14.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:14.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.822+0000 7f2b46b34640 1 Processor -- start 2024-04-25T08:30:14.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.822+0000 7f2b46b34640 1 -- start start 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b46b34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b400ced80 0x7f2b400d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b46b34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b46b34640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4015e000 con 0x7f2b4015dc20 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b46b34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b400d1de0 con 0x7f2b400ced80 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b46b34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40161c60 con 0x7f2b40161710 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b448d0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2b4015dc20 0x7f2b400ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39208/0 (socket says 172.21.15.60:39208) 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b448d0640 1 -- 172.21.15.60:0/2366617627 learned_addr learned my addr 172.21.15.60:0/2366617627 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2366617627 >> v2:172.21.15.100:3300/0 conn(0x7f2b400ced80 0x7f2b400d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b450d1640 1 --2- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3f7fe640 1 -- 172.21.15.60:0/2366617627 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210516230 0 0) 0x7f2b4015e000 con 0x7f2b4015dc20 2024-04-25T08:30:14.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3f7fe640 1 -- 172.21.15.60:0/2366617627 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b30003520 con 0x7f2b4015dc20 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3ffff640 1 -- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 msgr2=0x7f2b400d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3f7fe640 1 -- 172.21.15.60:0/2366617627 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 283595341 0 0) 0x7f2b30003520 con 0x7f2b4015dc20 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3ffff640 1 -- 172.21.15.60:0/2366617627 >> v1:172.21.15.60:6789/0 conn(0x7f2b4015dc20 legacy=0x7f2b400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.823+0000 7f2b3ffff640 1 -- 172.21.15.60:0/2366617627 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b40161f40 con 0x7f2b400ced80 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.824+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b400ced80 0x7f2b400d22e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0097a0 tx=0x7f2b2c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73dbfa090ec55a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.824+0000 7f2b3f7fe640 1 -- 172.21.15.60:0/2366617627 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c037070 con 0x7f2b400ced80 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.824+0000 7f2b3f7fe640 1 -- 172.21.15.60:0/2366617627 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c02aba0 con 0x7f2b400ced80 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.824+0000 7f2b3f7fe640 1 -- 172.21.15.60:0/2366617627 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34001670 con 0x7f2b400ced80 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 -- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b400ced80 msgr2=0x7f2b400d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b400ced80 0x7f2b400d22e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0097a0 tx=0x7f2b2c02a110 comp rx=0 tx=0).stop 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 -- 172.21.15.60:0/2366617627 shutdown_connections 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2366617627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b400ced80 0x7f2b400d22e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 -- 172.21.15.60:0/2366617627 >> 172.21.15.60:0/2366617627 conn(0x7f2b400d0420 msgr2=0x7f2b400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 -- 172.21.15.60:0/2366617627 shutdown_connections 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.825+0000 7f2b46b34640 1 -- 172.21.15.60:0/2366617627 wait complete. 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.826+0000 7f2b46b34640 1 Processor -- start 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.826+0000 7f2b46b34640 1 -- start start 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b46b34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 0x7f2b400cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b46b34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b4015dc20 0x7f2b40150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b46b34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b46b34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400b5e50 con 0x7f2b400ced80 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b46b34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40162500 con 0x7f2b4015dc20 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b46b34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400b5fd0 con 0x7f2b40161710 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 0x7f2b400cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 0x7f2b400cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45676/0 (socket says 172.21.15.60:45676) 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 -- 172.21.15.60:0/2732764671 learned_addr learned my addr 172.21.15.60:0/2732764671 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:14.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b4015dc20 0x7f2b40150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b450d1640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 -- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 msgr2=0x7f2b400aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 -- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b4015dc20 msgr2=0x7f2b40150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b4015dc20 0x7f2b40150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b448d0640 1 -- 172.21.15.60:0/2732764671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4015b200 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.827+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b4015dc20 0x7f2b40150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.828+0000 7f2b448d0640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 0x7f2b400cdad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400c700 tx=0x7f2b3400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.828+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34017070 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.828+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b34005ad0 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.828+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b400caf80 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.828+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34007070 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.828+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b400c8c80 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.829+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b40073a00 con 0x7f2b400ced80 2024-04-25T08:30:14.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.829+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b3401a070 con 0x7f2b400ced80 2024-04-25T08:30:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.830+0000 7f2b3d7fa640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b2406b4d0 0x7f2b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.830+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2b340898d0 con 0x7f2b400ced80 2024-04-25T08:30:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.830+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b2406b4d0 0x7f2b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.831+0000 7f2b3ffff640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b2406b4d0 0x7f2b2406d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c034000 tx=0x7f2b2c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:14.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:14.834+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b34057d90 con 0x7f2b400ced80 2024-04-25T08:30:15.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.017+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b400d16b0 con 0x7f2b400ced80 2024-04-25T08:30:15.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.017+0000 7f2b3d7fa640 1 -- 172.21.15.60:0/2732764671 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b34087e20 con 0x7f2b400ced80 2024-04-25T08:30:15.015 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b2406b4d0 msgr2=0x7f2b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b2406b4d0 0x7f2b2406d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c034000 tx=0x7f2b2c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 msgr2=0x7f2b400cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 0x7f2b400cdad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400c700 tx=0x7f2b3400cbc0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 shutdown_connections 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b40161710 0x7f2b400aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b2406b4d0 0x7f2b2406d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b4015dc20 0x7f2b40150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 --2- 172.21.15.60:0/2732764671 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b400ced80 0x7f2b400cdad0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 >> 172.21.15.60:0/2732764671 conn(0x7f2b400d0420 msgr2=0x7f2b400d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 shutdown_connections 2024-04-25T08:30:15.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.019+0000 7f2b46b34640 1 -- 172.21.15.60:0/2732764671 wait complete. 2024-04-25T08:30:15.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.134+0000 7f9f4cf70640 1 Processor -- start 2024-04-25T08:30:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.135+0000 7f9f4cf70640 1 -- start start 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.135+0000 7f9f4cf70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f480ccdd0 0x7f9f480cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.135+0000 7f9f4cf70640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f480cb190 con 0x7f9f480cdcd0 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.135+0000 7f9f4cf70640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f480cb470 con 0x7f9f4815bb20 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.135+0000 7f9f4cf70640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f480cb750 con 0x7f9f480ccdd0 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.136+0000 7f9f3ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9f480ccdd0 0x7f9f480cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.136+0000 7f9f46d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9f480cdcd0 0x7f9f480ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39216/0 (socket says 172.21.15.60:39216) 2024-04-25T08:30:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.136+0000 7f9f46d9d640 1 -- 172.21.15.60:0/763635191 learned_addr learned my addr 172.21.15.60:0/763635191 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.136+0000 7f9f3ffff640 1 -- 172.21.15.60:0/763635191 >> v1:172.21.15.100:6789/0 conn(0x7f9f4815bb20 legacy=0x7f9f480cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.136+0000 7f9f3ffff640 1 -- 172.21.15.60:0/763635191 >> v1:172.21.15.60:6789/0 conn(0x7f9f480cdcd0 legacy=0x7f9f480ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.136+0000 7f9f3ffff640 1 -- 172.21.15.60:0/763635191 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480cba30 con 0x7f9f480ccdd0 2024-04-25T08:30:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/763635191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480ccdd0 0x7f9f480cf9d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f300097a0 tx=0x7f9f3002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb991f979571be27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f45d9b640 1 -- 172.21.15.60:0/763635191 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30037070 con 0x7f9f480ccdd0 2024-04-25T08:30:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f45d9b640 1 -- 172.21.15.60:0/763635191 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30032440 con 0x7f9f480ccdd0 2024-04-25T08:30:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f45d9b640 1 -- 172.21.15.60:0/763635191 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30032810 con 0x7f9f480ccdd0 2024-04-25T08:30:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 -- 172.21.15.60:0/763635191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480ccdd0 msgr2=0x7f9f480cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/763635191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480ccdd0 0x7f9f480cf9d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9f300097a0 tx=0x7f9f3002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 -- 172.21.15.60:0/763635191 shutdown_connections 2024-04-25T08:30:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/763635191 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480ccdd0 0x7f9f480cf9d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 -- 172.21.15.60:0/763635191 >> 172.21.15.60:0/763635191 conn(0x7f9f480b2390 msgr2=0x7f9f480b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 -- 172.21.15.60:0/763635191 shutdown_connections 2024-04-25T08:30:15.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.137+0000 7f9f4cf70640 1 -- 172.21.15.60:0/763635191 wait complete. 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 Processor -- start 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 -- start start 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480ccdd0 0x7f9f480c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 0x7f9f48154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f4815bb20 0x7f9f48155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815c080 con 0x7f9f480cdcd0 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815bf00 con 0x7f9f4815bb20 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f4cf70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480cadb0 con 0x7f9f480ccdd0 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 0x7f9f48154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f3ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 0x7f9f48154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45696/0 (socket says 172.21.15.60:45696) 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.139+0000 7f9f3ffff640 1 -- 172.21.15.60:0/3676817464 learned_addr learned my addr 172.21.15.60:0/3676817464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f46d9d640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f4815bb20 0x7f9f48155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f4659c640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480ccdd0 0x7f9f480c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3ffff640 1 -- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480ccdd0 msgr2=0x7f9f480c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480ccdd0 0x7f9f480c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3ffff640 1 -- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f4815bb20 msgr2=0x7f9f48155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f4815bb20 0x7f9f48155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3ffff640 1 -- 172.21.15.60:0/3676817464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480c7fd0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3ffff640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 0x7f9f48154cc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f30035040 tx=0x7f9f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30037040 con 0x7f9f480cdcd0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30030770 con 0x7f9f480cdcd0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f480c82b0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30030b80 con 0x7f9f480cdcd0 2024-04-25T08:30:15.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.140+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f480c92b0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.141+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f480c87d0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.142+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f30044020 con 0x7f9f480cdcd0 2024-04-25T08:30:15.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.142+0000 7f9f3effd640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f2806b500 0x7f9f2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.142+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9f300aabb0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.142+0000 7f9f4659c640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f2806b500 0x7f9f2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.143+0000 7f9f4659c640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f2806b500 0x7f9f2806d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f38001640 tx=0x7f9f38009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.146+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f480c87d0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.327+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f480c94b0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.328+0000 7f9f3effd640 1 -- 172.21.15.60:0/3676817464 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f480c94b0 con 0x7f9f480cdcd0 2024-04-25T08:30:15.325 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:15.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.329+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f2806b500 msgr2=0x7f9f2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.329+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f2806b500 0x7f9f2806d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9f38001640 tx=0x7f9f38009040 comp rx=0 tx=0).stop 2024-04-25T08:30:15.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 msgr2=0x7f9f48154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 0x7f9f48154cc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f30035040 tx=0x7f9f300046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 shutdown_connections 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9f2806b500 0x7f9f2806d9c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f4815bb20 0x7f9f48155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f480cdcd0 0x7f9f48154cc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 --2- 172.21.15.60:0/3676817464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f480ccdd0 0x7f9f480c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 >> 172.21.15.60:0/3676817464 conn(0x7f9f480b2390 msgr2=0x7f9f4815aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 shutdown_connections 2024-04-25T08:30:15.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.330+0000 7f9f4cf70640 1 -- 172.21.15.60:0/3676817464 wait complete. 2024-04-25T08:30:15.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.444+0000 7f101763f640 1 Processor -- start 2024-04-25T08:30:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f101763f640 1 -- start start 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f101763f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 0x7f10100ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f101763f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101006cda0 con 0x7f10100d32f0 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f101763f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101015e000 con 0x7f1010161710 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f101763f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10100d1de0 con 0x7f101015dc20 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f1015bdc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f10100d32f0 0x7f101006c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39232/0 (socket says 172.21.15.60:39232) 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.445+0000 7f1015bdc640 1 -- 172.21.15.60:0/2140370895 learned_addr learned my addr 172.21.15.60:0/2140370895 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f10153db640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 0x7f10100ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578343536 0 0) 0x7f101006cda0 con 0x7f10100d32f0 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ffc003520 con 0x7f10100d32f0 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943695083 0 0) 0x7f101015e000 con 0x7f1010161710 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101006cda0 con 0x7f1010161710 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168853994 0 0) 0x7f0ffc003520 con 0x7f10100d32f0 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 msgr2=0x7f10100ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 0x7f10100ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 >> v1:172.21.15.100:6789/0 conn(0x7f1010161710 legacy=0x7f101006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010161c60 con 0x7f10100d32f0 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f10080019f0 con 0x7f10100d32f0 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 >> v1:172.21.15.60:6789/0 conn(0x7f10100d32f0 legacy=0x7f101006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc003b70 0x7f0ffc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 0x7f0ffc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100d1de0 con 0x7f0ffc004d40 2024-04-25T08:30:15.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc00ba70 con 0x7f0ffc003b70 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc014480 con 0x7f101015dc20 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1015bdc640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f10153db640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc003b70 0x7f0ffc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.446+0000 7f1014bda640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 0x7f0ffc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1015bdc640 1 -- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 msgr2=0x7f0ffc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1015bdc640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f101015dc20 0x7f0ffc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1015bdc640 1 -- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc003b70 msgr2=0x7f0ffc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1015bdc640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ffc003b70 0x7f0ffc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1015bdc640 1 -- 172.21.15.60:0/2140370895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc003520 con 0x7f0ffc004d40 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1015bdc640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f100800a700 tx=0x7f0ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0591aae92a700d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.447+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800a350 con 0x7f0ffc004d40 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100802f070 con 0x7f0ffc004d40 2024-04-25T08:30:15.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f1007fff640 1 -- 172.21.15.60:0/2140370895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100802bb50 con 0x7f0ffc004d40 2024-04-25T08:30:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f101763f640 1 -- 172.21.15.60:0/2140370895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ffc004d40 msgr2=0x7f0ffc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f101763f640 1 --2- 172.21.15.60:0/2140370895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ffc004d40 0x7f0ffc0051c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f100800a700 tx=0x7f0ffc005830 comp rx=0 tx=0).stop 2024-04-25T08:30:15.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f1015bdc640 1 -- 172.21.15.60:0/2140370895 reap_dead start 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f101763f640 1 -- 172.21.15.60:0/2140370895 shutdown_connections 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f101763f640 1 -- 172.21.15.60:0/2140370895 >> 172.21.15.60:0/2140370895 conn(0x7f10100d0420 msgr2=0x7f10100d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f101763f640 1 -- 172.21.15.60:0/2140370895 shutdown_connections 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.448+0000 7f101763f640 1 -- 172.21.15.60:0/2140370895 wait complete. 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.449+0000 7f101763f640 1 Processor -- start 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.449+0000 7f101763f640 1 -- start start 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f101763f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ffc004d40 0x7f101014a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f101763f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 0x7f10100b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f101763f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f101015bca0 0x7f101015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f101763f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010162500 con 0x7f101015c900 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f101763f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010162220 con 0x7f101015bca0 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f101763f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10100b5e50 con 0x7f0ffc004d40 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 0x7f10100b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 0x7f10100b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45706/0 (socket says 172.21.15.60:45706) 2024-04-25T08:30:15.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 -- 172.21.15.60:0/3004902581 learned_addr learned my addr 172.21.15.60:0/3004902581 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f10153db640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ffc004d40 0x7f101014a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1015bdc640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f101015bca0 0x7f101015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 -- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ffc004d40 msgr2=0x7f101014a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ffc004d40 0x7f101014a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 -- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f101015bca0 msgr2=0x7f101015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f101015bca0 0x7f101015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1014bda640 1 -- 172.21.15.60:0/3004902581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101014afb0 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.450+0000 7f1015bdc640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f101015bca0 0x7f101015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.451+0000 7f1014bda640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 0x7f10100b4db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f100000f840 tx=0x7f100000fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.451+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000fe90 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.451+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000018070 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.451+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101014b290 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.451+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000d730 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.451+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1010148970 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.452+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1010148b70 con 0x7f101015c900 2024-04-25T08:30:15.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.452+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f100000d8d0 con 0x7f101015c900 2024-04-25T08:30:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.453+0000 7f1005ffb640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ff406b6a0 0x7f0ff406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.453+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f100008a510 con 0x7f101015c900 2024-04-25T08:30:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.453+0000 7f10153db640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ff406b6a0 0x7f0ff406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.453+0000 7f10153db640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ff406b6a0 0x7f0ff406db60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f100c009bc0 tx=0x7f100c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.457+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1010148b70 con 0x7f101015c900 2024-04-25T08:30:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.635+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1010073a00 con 0x7f101015c900 2024-04-25T08:30:15.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.636+0000 7f1005ffb640 1 -- 172.21.15.60:0/3004902581 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1000058f90 con 0x7f101015c900 2024-04-25T08:30:15.633 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ff406b6a0 msgr2=0x7f0ff406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ff406b6a0 0x7f0ff406db60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f100c009bc0 tx=0x7f100c009190 comp rx=0 tx=0).stop 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 msgr2=0x7f10100b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 0x7f10100b4db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f100000f840 tx=0x7f100000fc40 comp rx=0 tx=0).stop 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 shutdown_connections 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ff406b6a0 0x7f0ff406db60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f101015bca0 0x7f101015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f101015c900 0x7f10100b4db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 --2- 172.21.15.60:0/3004902581 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ffc004d40 0x7f101014a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 >> 172.21.15.60:0/3004902581 conn(0x7f10100d0420 msgr2=0x7f10100b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 shutdown_connections 2024-04-25T08:30:15.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.638+0000 7f101763f640 1 -- 172.21.15.60:0/3004902581 wait complete. 2024-04-25T08:30:15.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:30:15.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.750+0000 7f7d2de83640 1 Processor -- start 2024-04-25T08:30:15.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.750+0000 7f7d2de83640 1 -- start start 2024-04-25T08:30:15.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d2de83640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7d280cf600 0x7f7d280cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d2de83640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 0x7f7d280cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d2de83640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d280c75e0 con 0x7f7d2815bb50 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d2de83640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280c8680 con 0x7f7d280cbf20 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d2de83640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d280cff80 con 0x7f7d280cf600 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d27fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 0x7f7d280cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d26ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7d280cf600 0x7f7d280cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d27fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 0x7f7d280cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45722/0 (socket says 172.21.15.60:45722) 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.751+0000 7f7d27fff640 1 -- 172.21.15.60:0/1981401751 learned_addr learned my addr 172.21.15.60:0/1981401751 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.752+0000 7f7d26ffd640 1 -- 172.21.15.60:0/1981401751 >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 msgr2=0x7f7d280cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.752+0000 7f7d26ffd640 1 --2- 172.21.15.60:0/1981401751 >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 0x7f7d280cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.752+0000 7f7d26ffd640 1 -- 172.21.15.60:0/1981401751 >> v1:172.21.15.100:6789/0 conn(0x7f7d2815bb50 legacy=0x7f7d280c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.752+0000 7f7d26ffd640 1 -- 172.21.15.60:0/1981401751 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280c78c0 con 0x7f7d280cf600 2024-04-25T08:30:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.752+0000 7f7d27fff640 1 --2- 172.21.15.60:0/1981401751 >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 0x7f7d280cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d26ffd640 1 --2- 172.21.15.60:0/1981401751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cfa40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d2806f220 tx=0x7f7d1c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1830ea8c862a0c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d267fc640 1 -- 172.21.15.60:0/1981401751 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c037070 con 0x7f7d280cf600 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d267fc640 1 -- 172.21.15.60:0/1981401751 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c032440 con 0x7f7d280cf600 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d267fc640 1 -- 172.21.15.60:0/1981401751 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c032810 con 0x7f7d280cf600 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 -- 172.21.15.60:0/1981401751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 msgr2=0x7f7d280cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 --2- 172.21.15.60:0/1981401751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cfa40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7d2806f220 tx=0x7f7d1c02a230 comp rx=0 tx=0).stop 2024-04-25T08:30:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 -- 172.21.15.60:0/1981401751 shutdown_connections 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 --2- 172.21.15.60:0/1981401751 >> v2:172.21.15.60:3300/0 conn(0x7f7d280cbf20 0x7f7d280cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 --2- 172.21.15.60:0/1981401751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cfa40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 -- 172.21.15.60:0/1981401751 >> 172.21.15.60:0/1981401751 conn(0x7f7d2806f500 msgr2=0x7f7d2806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 -- 172.21.15.60:0/1981401751 shutdown_connections 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.753+0000 7f7d2de83640 1 -- 172.21.15.60:0/1981401751 wait complete. 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 Processor -- start 2024-04-25T08:30:15.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 -- start start 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 0x7f7d2806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d2815bb50 0x7f7d280c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c8160 con 0x7f7d280cbf20 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c7e80 con 0x7f7d280cf600 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.755+0000 7f7d2de83640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d280c72a0 con 0x7f7d2815bb50 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 0x7f7d2806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 0x7f7d2806d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45730/0 (socket says 172.21.15.60:45730) 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 -- 172.21.15.60:0/2566977476 learned_addr learned my addr 172.21.15.60:0/2566977476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d26ffd640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d27fff640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d2815bb50 0x7f7d280c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 -- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d2815bb50 msgr2=0x7f7d280c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d2815bb50 0x7f7d280c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 -- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 msgr2=0x7f7d280cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 -- 172.21.15.60:0/2566977476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2815a610 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d26ffd640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d277fe640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 0x7f7d2806d670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c3f0 tx=0x7f7d1800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d180052d0 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d18005490 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2815a050 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18007260 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.756+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d28158320 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.757+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2815a440 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.758+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d18007400 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.758+0000 7f7d0ffff640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7d1406b4d0 0x7f7d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.758+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7d18089b80 con 0x7f7d280cbf20 2024-04-25T08:30:15.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.758+0000 7f7d26ffd640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7d1406b4d0 0x7f7d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:15.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.759+0000 7f7d26ffd640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7d1406b4d0 0x7f7d1406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c034000 tx=0x7f7d1c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:15.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.762+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d18058560 con 0x7f7d280cbf20 2024-04-25T08:30:15.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.955+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7d28073a00 con 0x7f7d280cbf20 2024-04-25T08:30:15.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.955+0000 7f7d0ffff640 1 -- 172.21.15.60:0/2566977476 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7d18057e60 con 0x7f7d280cbf20 2024-04-25T08:30:15.953 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7d1406b4d0 msgr2=0x7f7d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7d1406b4d0 0x7f7d1406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c034000 tx=0x7f7d1c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 msgr2=0x7f7d2806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 0x7f7d2806d670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800c3f0 tx=0x7f7d1800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 shutdown_connections 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7d2815bb50 0x7f7d280c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7d1406b4d0 0x7f7d1406d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d280cf600 0x7f7d280cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 --2- 172.21.15.60:0/2566977476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7d280cbf20 0x7f7d2806d670 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 >> 172.21.15.60:0/2566977476 conn(0x7f7d2806f500 msgr2=0x7f7d281568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 shutdown_connections 2024-04-25T08:30:15.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:15.957+0000 7f7d2de83640 1 -- 172.21.15.60:0/2566977476 wait complete. 2024-04-25T08:30:15.965 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T08:30:15.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:15.965 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T08:30:15.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:30:15.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:15.995 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:30:16.053 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:30:16.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:30:16.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.158+0000 7f418e17e640 1 Processor -- start 2024-04-25T08:30:16.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.158+0000 7f418e17e640 1 -- start start 2024-04-25T08:30:16.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f418e17e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f418815db70 0x7f4188162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f418e17e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4188162690 con 0x7f41880b6cb0 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f418e17e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41880ceee0 con 0x7f41880b5c30 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f418e17e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41880d1d10 con 0x7f418815db70 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f4186ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f41880b6cb0 0x7f41880b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39238/0 (socket says 172.21.15.60:39238) 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f4186ffd640 1 -- 172.21.15.60:0/3502697102 learned_addr learned my addr 172.21.15.60:0/3502697102 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f41877fe640 1 --2- 172.21.15.60:0/3502697102 >> v2:172.21.15.100:3300/0 conn(0x7f418815db70 0x7f4188162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817620973 0 0) 0x7f4188162690 con 0x7f41880b6cb0 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.159+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4170003520 con 0x7f41880b6cb0 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.160+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728259988 0 0) 0x7f41880ceee0 con 0x7f41880b5c30 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.160+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4188162690 con 0x7f41880b5c30 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.160+0000 7f41877fe640 1 -- 172.21.15.60:0/3502697102 >> v1:172.21.15.100:6789/0 conn(0x7f41880b5c30 legacy=0x7f41880d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.160+0000 7f41877fe640 1 -- 172.21.15.60:0/3502697102 >> v1:172.21.15.60:6789/0 conn(0x7f41880b6cb0 legacy=0x7f41880b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.160+0000 7f41877fe640 1 -- 172.21.15.60:0/3502697102 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41880d1ff0 con 0x7f418815db70 2024-04-25T08:30:16.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.160+0000 7f41877fe640 1 --2- 172.21.15.60:0/3502697102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f418815db70 0x7f4188162150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f41780097a0 tx=0x7f417802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6f72f9000f7574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178037070 con 0x7f418815db70 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4178032440 con 0x7f418815db70 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f41867fc640 1 -- 172.21.15.60:0/3502697102 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41780327f0 con 0x7f418815db70 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 -- 172.21.15.60:0/3502697102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f418815db70 msgr2=0x7f4188162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 --2- 172.21.15.60:0/3502697102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f418815db70 0x7f4188162150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f41780097a0 tx=0x7f417802a230 comp rx=0 tx=0).stop 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 -- 172.21.15.60:0/3502697102 shutdown_connections 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 --2- 172.21.15.60:0/3502697102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f418815db70 0x7f4188162150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 -- 172.21.15.60:0/3502697102 >> 172.21.15.60:0/3502697102 conn(0x7f41880d0520 msgr2=0x7f41880d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 -- 172.21.15.60:0/3502697102 shutdown_connections 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.161+0000 7f418e17e640 1 -- 172.21.15.60:0/3502697102 wait complete. 2024-04-25T08:30:16.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.162+0000 7f418e17e640 1 Processor -- start 2024-04-25T08:30:16.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 -- start start 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b5c30 0x7f41880c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41880b6cb0 0x7f41881486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 0x7f4188160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41880ce530 con 0x7f418815db70 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418815df50 con 0x7f41880b6cb0 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f418e17e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41880cf1c0 con 0x7f41880b5c30 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f4187fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 0x7f4188160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f4187fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 0x7f4188160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45740/0 (socket says 172.21.15.60:45740) 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.163+0000 7f4187fff640 1 -- 172.21.15.60:0/2761444669 learned_addr learned my addr 172.21.15.60:0/2761444669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4186ffd640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41880b6cb0 0x7f41881486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f41877fe640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b5c30 0x7f41880c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4187fff640 1 -- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b5c30 msgr2=0x7f41880c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4187fff640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b5c30 0x7f41880c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4187fff640 1 -- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41880b6cb0 msgr2=0x7f41881486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4187fff640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41880b6cb0 0x7f41881486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4187fff640 1 -- 172.21.15.60:0/2761444669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41880cbff0 con 0x7f418815db70 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f41877fe640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b5c30 0x7f41880c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f4187fff640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 0x7f4188160d00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f418000c730 tx=0x7f418000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180003b70 con 0x7f418815db70 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4180003d10 con 0x7f418815db70 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41880cc2d0 con 0x7f418815db70 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418000aa90 con 0x7f418815db70 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.164+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41880ccb70 con 0x7f418815db70 2024-04-25T08:30:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.165+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41880cc6c0 con 0x7f418815db70 2024-04-25T08:30:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.166+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f418000ac30 con 0x7f418815db70 2024-04-25T08:30:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.166+0000 7f416ffff640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417406b4d0 0x7f417406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.166+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f418008a3f0 con 0x7f418815db70 2024-04-25T08:30:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.166+0000 7f41877fe640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417406b4d0 0x7f417406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.167+0000 7f41877fe640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417406b4d0 0x7f417406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4178034000 tx=0x7f417802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.170+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4180058f00 con 0x7f418815db70 2024-04-25T08:30:16.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.387+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4188073990 con 0x7f418815db70 2024-04-25T08:30:16.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.388+0000 7f416ffff640 1 -- 172.21.15.60:0/2761444669 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f4180058800 con 0x7f418815db70 2024-04-25T08:30:16.386 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:16.386 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:30:16.386 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417406b4d0 msgr2=0x7f417406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417406b4d0 0x7f417406d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4178034000 tx=0x7f417802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 msgr2=0x7f4188160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 0x7f4188160d00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f418000c730 tx=0x7f418000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 shutdown_connections 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f418815db70 0x7f4188160d00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417406b4d0 0x7f417406d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41880b6cb0 0x7f41881486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 --2- 172.21.15.60:0/2761444669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b5c30 0x7f41880c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 >> 172.21.15.60:0/2761444669 conn(0x7f41880d0520 msgr2=0x7f41881588b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:16.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 shutdown_connections 2024-04-25T08:30:16.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.390+0000 7f418e17e640 1 -- 172.21.15.60:0/2761444669 wait complete. 2024-04-25T08:30:16.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.501+0000 7fd35b90a640 1 Processor -- start 2024-04-25T08:30:16.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.501+0000 7fd35b90a640 1 -- start start 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd35b90a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd354154ed0 0x7fd3540aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd35b90a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd354071ac0 0x7fd3540ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd35b90a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35415ec60 con 0x7fd35406c1d0 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd35b90a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35415ef40 con 0x7fd354154ed0 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd35b90a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35406db90 con 0x7fd354071ac0 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd3596a6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd354154ed0 0x7fd3540aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd359ea7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd35406c1d0 0x7fd3541617d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39250/0 (socket says 172.21.15.60:39250) 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.502+0000 7fd359ea7640 1 -- 172.21.15.60:0/2895620962 learned_addr learned my addr 172.21.15.60:0/2895620962 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd358ea5640 1 --2- 172.21.15.60:0/2895620962 >> v2:172.21.15.100:3300/0 conn(0x7fd354071ac0 0x7fd3540ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd353fff640 1 -- 172.21.15.60:0/2895620962 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227977667 0 0) 0x7fd35415ec60 con 0x7fd35406c1d0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd353fff640 1 -- 172.21.15.60:0/2895620962 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd33c003520 con 0x7fd35406c1d0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd3596a6640 1 -- 172.21.15.60:0/2895620962 >> v2:172.21.15.100:3300/0 conn(0x7fd354071ac0 msgr2=0x7fd3540ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd3596a6640 1 --2- 172.21.15.60:0/2895620962 >> v2:172.21.15.100:3300/0 conn(0x7fd354071ac0 0x7fd3540ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd3596a6640 1 -- 172.21.15.60:0/2895620962 >> v1:172.21.15.60:6789/0 conn(0x7fd35406c1d0 legacy=0x7fd3541617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.503+0000 7fd3596a6640 1 -- 172.21.15.60:0/2895620962 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35406de70 con 0x7fd354154ed0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd3596a6640 1 --2- 172.21.15.60:0/2895620962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354154ed0 0x7fd3540aced0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd354061790 tx=0x7fd34c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a07185d0786fb02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd353fff640 1 -- 172.21.15.60:0/2895620962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c037070 con 0x7fd354154ed0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd353fff640 1 -- 172.21.15.60:0/2895620962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c032440 con 0x7fd354154ed0 2024-04-25T08:30:16.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd353fff640 1 -- 172.21.15.60:0/2895620962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c02ae60 con 0x7fd354154ed0 2024-04-25T08:30:16.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 -- 172.21.15.60:0/2895620962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354154ed0 msgr2=0x7fd3540aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 --2- 172.21.15.60:0/2895620962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354154ed0 0x7fd3540aced0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd354061790 tx=0x7fd34c02a110 comp rx=0 tx=0).stop 2024-04-25T08:30:16.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 -- 172.21.15.60:0/2895620962 shutdown_connections 2024-04-25T08:30:16.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 --2- 172.21.15.60:0/2895620962 >> v2:172.21.15.100:3300/0 conn(0x7fd354071ac0 0x7fd3540ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 --2- 172.21.15.60:0/2895620962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354154ed0 0x7fd3540aced0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 -- 172.21.15.60:0/2895620962 >> 172.21.15.60:0/2895620962 conn(0x7fd3540af670 msgr2=0x7fd3540b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 -- 172.21.15.60:0/2895620962 shutdown_connections 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.504+0000 7fd35b90a640 1 -- 172.21.15.60:0/2895620962 wait complete. 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.505+0000 7fd35b90a640 1 Processor -- start 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 -- start start 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354071ac0 0x7fd354149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd354154ed0 0x7fd35414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 0x7fd35414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540add20 con 0x7fd354157c90 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540adba0 con 0x7fd354071ac0 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd35b90a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd354161ee0 con 0x7fd354154ed0 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd359ea7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 0x7fd35414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd359ea7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 0x7fd35414ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45756/0 (socket says 172.21.15.60:45756) 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.506+0000 7fd359ea7640 1 -- 172.21.15.60:0/1840429810 learned_addr learned my addr 172.21.15.60:0/1840429810 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd3596a6640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354071ac0 0x7fd354149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd358ea5640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd354154ed0 0x7fd35414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd359ea7640 1 -- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd354154ed0 msgr2=0x7fd35414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd359ea7640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd354154ed0 0x7fd35414e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd359ea7640 1 -- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354071ac0 msgr2=0x7fd354149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd359ea7640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354071ac0 0x7fd354149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd359ea7640 1 -- 172.21.15.60:0/1840429810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35414a530 con 0x7fd354157c90 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd359ea7640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 0x7fd35414ee30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd34400caa0 tx=0x7fd34400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344017070 con 0x7fd354157c90 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34400aba0 con 0x7fd354157c90 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd354156bf0 con 0x7fd354157c90 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344012400 con 0x7fd354157c90 2024-04-25T08:30:16.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.507+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3541571b0 con 0x7fd354157c90 2024-04-25T08:30:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.508+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd354073990 con 0x7fd354157c90 2024-04-25T08:30:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.509+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd34401a070 con 0x7fd354157c90 2024-04-25T08:30:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.509+0000 7fd351ffb640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd34006b4d0 0x7fd34006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.509+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd34408a1e0 con 0x7fd354157c90 2024-04-25T08:30:16.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.510+0000 7fd3596a6640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd34006b4d0 0x7fd34006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.510+0000 7fd3596a6640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd34006b4d0 0x7fd34006d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd34c034000 tx=0x7fd34c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.514+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3440586a0 con 0x7fd354157c90 2024-04-25T08:30:16.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.693+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd35414fd10 con 0x7fd354157c90 2024-04-25T08:30:16.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.694+0000 7fd351ffb640 1 -- 172.21.15.60:0/1840429810 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7fd34405c740 con 0x7fd354157c90 2024-04-25T08:30:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:16.692 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:16.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd34006b4d0 msgr2=0x7fd34006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd34006b4d0 0x7fd34006d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd34c034000 tx=0x7fd34c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 msgr2=0x7fd35414ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 0x7fd35414ee30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd34400caa0 tx=0x7fd34400cf60 comp rx=0 tx=0).stop 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 shutdown_connections 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd354157c90 0x7fd35414ee30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd354154ed0 0x7fd35414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd34006b4d0 0x7fd34006d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 --2- 172.21.15.60:0/1840429810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd354071ac0 0x7fd354149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 >> 172.21.15.60:0/1840429810 conn(0x7fd35406c1d0 msgr2=0x7fd35415e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 shutdown_connections 2024-04-25T08:30:16.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.696+0000 7fd35b90a640 1 -- 172.21.15.60:0/1840429810 wait complete. 2024-04-25T08:30:16.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:29:21.558014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:29:22.368855+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:29:26.081283+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:30:16.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.811+0000 7f7b95d22640 1 Processor -- start 2024-04-25T08:30:16.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.811+0000 7f7b95d22640 1 -- start start 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b95d22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b90158230 0x7f7b900d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b95d22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b900d1b80 0x7f7b9006c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b95d22640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b9006c870 0x7f7b900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b95d22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b90158610 con 0x7f7b90158230 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b95d22640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9006bea0 con 0x7f7b9006c870 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b95d22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900d2020 con 0x7f7b900d1b80 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b90158230 0x7f7b900d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b90158230 0x7f7b900d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45758/0 (socket says 172.21.15.60:45758) 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 -- 172.21.15.60:0/1484731683 learned_addr learned my addr 172.21.15.60:0/1484731683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8effd640 1 --2- 172.21.15.60:0/1484731683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b900d1b80 0x7f7b9006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8ffff640 1 --2- 172.21.15.60:0/1484731683 >> v2:172.21.15.100:3300/0 conn(0x7f7b9006c870 0x7f7b900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 -- 172.21.15.60:0/1484731683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b900d1b80 msgr2=0x7f7b9006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 --2- 172.21.15.60:0/1484731683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b900d1b80 0x7f7b9006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 -- 172.21.15.60:0/1484731683 >> v2:172.21.15.100:3300/0 conn(0x7f7b9006c870 msgr2=0x7f7b900b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 --2- 172.21.15.60:0/1484731683 >> v2:172.21.15.100:3300/0 conn(0x7f7b9006c870 0x7f7b900b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.812+0000 7f7b8f7fe640 1 -- 172.21.15.60:0/1484731683 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900d2300 con 0x7f7b90158230 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.813+0000 7f7b8f7fe640 1 --2- 172.21.15.60:0/1484731683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b90158230 0x7f7b900d1640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7b900b4800 tx=0x7f7b8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82f564296d6548d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.813+0000 7f7b8e7fc640 1 -- 172.21.15.60:0/1484731683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80037070 con 0x7f7b90158230 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.813+0000 7f7b8e7fc640 1 -- 172.21.15.60:0/1484731683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80032440 con 0x7f7b90158230 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.813+0000 7f7b8e7fc640 1 -- 172.21.15.60:0/1484731683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80032810 con 0x7f7b90158230 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 -- 172.21.15.60:0/1484731683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b90158230 msgr2=0x7f7b900d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 --2- 172.21.15.60:0/1484731683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b90158230 0x7f7b900d1640 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7b900b4800 tx=0x7f7b8002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 -- 172.21.15.60:0/1484731683 shutdown_connections 2024-04-25T08:30:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 --2- 172.21.15.60:0/1484731683 >> v2:172.21.15.100:3300/0 conn(0x7f7b9006c870 0x7f7b900b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 --2- 172.21.15.60:0/1484731683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b900d1b80 0x7f7b9006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 --2- 172.21.15.60:0/1484731683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b90158230 0x7f7b900d1640 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 -- 172.21.15.60:0/1484731683 >> 172.21.15.60:0/1484731683 conn(0x7f7b90061600 msgr2=0x7f7b900619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 -- 172.21.15.60:0/1484731683 shutdown_connections 2024-04-25T08:30:16.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.814+0000 7f7b95d22640 1 -- 172.21.15.60:0/1484731683 wait complete. 2024-04-25T08:30:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.815+0000 7f7b95d22640 1 Processor -- start 2024-04-25T08:30:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.815+0000 7f7b95d22640 1 -- start start 2024-04-25T08:30:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b95d22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9006c870 0x7f7b90146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b95d22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 0x7f7b900cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b95d22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b90158230 0x7f7b901e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b95d22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9015e320 con 0x7f7b900d1b80 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b95d22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900d25e0 con 0x7f7b9006c870 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b95d22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9015e4a0 con 0x7f7b90158230 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 0x7f7b900cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b8effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 0x7f7b900cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45766/0 (socket says 172.21.15.60:45766) 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b8effd640 1 -- 172.21.15.60:0/274694145 learned_addr learned my addr 172.21.15.60:0/274694145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b8f7fe640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9006c870 0x7f7b90146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.816+0000 7f7b8ffff640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b90158230 0x7f7b901e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b8effd640 1 -- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b90158230 msgr2=0x7f7b901e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b8effd640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b90158230 0x7f7b901e5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b8effd640 1 -- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9006c870 msgr2=0x7f7b90146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b8effd640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9006c870 0x7f7b90146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b8effd640 1 -- 172.21.15.60:0/274694145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900cadd0 con 0x7f7b900d1b80 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b8effd640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 0x7f7b900cbe70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400d800 tx=0x7f7b8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400b6c0 con 0x7f7b900d1b80 2024-04-25T08:30:16.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84019070 con 0x7f7b900d1b80 2024-04-25T08:30:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b900cb0b0 con 0x7f7b900d1b80 2024-04-25T08:30:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400baa0 con 0x7f7b900d1b80 2024-04-25T08:30:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.817+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b900cb670 con 0x7f7b900d1b80 2024-04-25T08:30:16.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.818+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b90073a00 con 0x7f7b900d1b80 2024-04-25T08:30:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.818+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b8400bc40 con 0x7f7b900d1b80 2024-04-25T08:30:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.819+0000 7f7b77fff640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b7c06b4d0 0x7f7b7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.819+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7b8408a8e0 con 0x7f7b900d1b80 2024-04-25T08:30:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.819+0000 7f7b8f7fe640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b7c06b4d0 0x7f7b7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:16.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.820+0000 7f7b8f7fe640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b7c06b4d0 0x7f7b7c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b80034000 tx=0x7f7b8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:16.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:16.823+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b84058d20 con 0x7f7b900d1b80 2024-04-25T08:30:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.003+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b900b5f30 con 0x7f7b900d1b80 2024-04-25T08:30:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.003+0000 7f7b77fff640 1 -- 172.21.15.60:0/274694145 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b8405cdc0 con 0x7f7b900d1b80 2024-04-25T08:30:17.001 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b7c06b4d0 msgr2=0x7f7b7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b7c06b4d0 0x7f7b7c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b80034000 tx=0x7f7b8002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 msgr2=0x7f7b900cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 0x7f7b900cbe70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400d800 tx=0x7f7b8400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 shutdown_connections 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b90158230 0x7f7b901e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b900d1b80 0x7f7b900cbe70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b7c06b4d0 0x7f7b7c06d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 --2- 172.21.15.60:0/274694145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b9006c870 0x7f7b90146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 >> 172.21.15.60:0/274694145 conn(0x7f7b90061600 msgr2=0x7f7b9015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 shutdown_connections 2024-04-25T08:30:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.005+0000 7f7b95d22640 1 -- 172.21.15.60:0/274694145 wait complete. 2024-04-25T08:30:17.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:30:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.120+0000 7fc7feed1640 1 Processor -- start 2024-04-25T08:30:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.120+0000 7fc7feed1640 1 -- start start 2024-04-25T08:30:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.121+0000 7fc7feed1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7f80d00e0 0x7fc7f80ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.121+0000 7fc7feed1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80cb190 con 0x7fc7f815bb20 2024-04-25T08:30:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.121+0000 7fc7feed1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80cb470 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.121+0000 7fc7feed1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f80cb750 con 0x7fc7f80d00e0 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.121+0000 7fc7fcc6d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc7f815bb20 0x7fc7f80cc230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39266/0 (socket says 172.21.15.60:39266) 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.121+0000 7fc7fcc6d640 1 -- 172.21.15.60:0/1946092327 learned_addr learned my addr 172.21.15.60:0/1946092327 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7fd46e640 1 --2- 172.21.15.60:0/1946092327 >> v2:172.21.15.100:3300/0 conn(0x7fc7f80d00e0 0x7fc7f80ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7f77fe640 1 -- 172.21.15.60:0/1946092327 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425422168 0 0) 0x7fc7f80cb190 con 0x7fc7f815bb20 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7f77fe640 1 -- 172.21.15.60:0/1946092327 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e4003520 con 0x7fc7f815bb20 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7f77fe640 1 -- 172.21.15.60:0/1946092327 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 712189752 0 0) 0x7fc7e4003520 con 0x7fc7f815bb20 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7fd46e640 1 -- 172.21.15.60:0/1946092327 >> v1:172.21.15.100:6789/0 conn(0x7fc7f80ccdd0 legacy=0x7fc7f80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7fd46e640 1 -- 172.21.15.60:0/1946092327 >> v1:172.21.15.60:6789/0 conn(0x7fc7f815bb20 legacy=0x7fc7f80cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.122+0000 7fc7fd46e640 1 -- 172.21.15.60:0/1946092327 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80cba30 con 0x7fc7f80d00e0 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.123+0000 7fc7fd46e640 1 --2- 172.21.15.60:0/1946092327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f80ad600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097a0 tx=0x7fc7e802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=681b9a1ab7d7ce31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.123+0000 7fc7f77fe640 1 -- 172.21.15.60:0/1946092327 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8037070 con 0x7fc7f80d00e0 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.123+0000 7fc7f77fe640 1 -- 172.21.15.60:0/1946092327 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e802abc0 con 0x7fc7f80d00e0 2024-04-25T08:30:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.123+0000 7fc7f77fe640 1 -- 172.21.15.60:0/1946092327 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00019f0 con 0x7fc7f80d00e0 2024-04-25T08:30:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 -- 172.21.15.60:0/1946092327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 msgr2=0x7fc7f80ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 --2- 172.21.15.60:0/1946092327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f80ad600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80097a0 tx=0x7fc7e802a130 comp rx=0 tx=0).stop 2024-04-25T08:30:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 -- 172.21.15.60:0/1946092327 shutdown_connections 2024-04-25T08:30:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 --2- 172.21.15.60:0/1946092327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f80ad600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 -- 172.21.15.60:0/1946092327 >> 172.21.15.60:0/1946092327 conn(0x7fc7f80b2390 msgr2=0x7fc7f80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 -- 172.21.15.60:0/1946092327 shutdown_connections 2024-04-25T08:30:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.124+0000 7fc7feed1640 1 -- 172.21.15.60:0/1946092327 wait complete. 2024-04-25T08:30:17.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.125+0000 7fc7feed1640 1 Processor -- start 2024-04-25T08:30:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.125+0000 7fc7feed1640 1 -- start start 2024-04-25T08:30:17.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 0x7fc7f8149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f8156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815bb20 0x7fc7f814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f815c080 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f815bf00 con 0x7fc7f80d00e0 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f80cadb0 con 0x7fc7f815bb20 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 0x7fc7f8149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 0x7fc7f8149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45768/0 (socket says 172.21.15.60:45768) 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 -- 172.21.15.60:0/494962837 learned_addr learned my addr 172.21.15.60:0/494962837 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7f7fff640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f8156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fd46e640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815bb20 0x7fc7f814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 -- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815bb20 msgr2=0x7fc7f814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815bb20 0x7fc7f814f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 -- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 msgr2=0x7fc7f8156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f8156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 -- 172.21.15.60:0/494962837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8158a20 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7f7fff640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fd46e640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815bb20 0x7fc7f814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7fcc6d640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 0x7fc7f8149170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0001890 tx=0x7fc7f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0017070 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f0012400 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8158cc0 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.126+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00127f0 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.127+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f8159280 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.127+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f80cb230 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.128+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7f001a070 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.128+0000 7fc7f57fa640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.128+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc7f008a540 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.129+0000 7fc7f7fff640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.129+0000 7fc7f7fff640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7d806b4d0 0x7fc7d806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc7f814f320 tx=0x7fc7e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.133+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7f80cb230 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.314+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7f80b1a70 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.314+0000 7fc7f57fa640 1 -- 172.21.15.60:0/494962837 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7f80b1a70 con 0x7fc7f80ccdd0 2024-04-25T08:30:17.311 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:17.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7d806b4d0 msgr2=0x7fc7d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7d806b4d0 0x7fc7d806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc7f814f320 tx=0x7fc7e0009040 comp rx=0 tx=0).stop 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 msgr2=0x7fc7f8149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 0x7fc7f8149170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0001890 tx=0x7fc7f000a430 comp rx=0 tx=0).stop 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 shutdown_connections 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7f815bb20 0x7fc7f814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7f80d00e0 0x7fc7f8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 --2- 172.21.15.60:0/494962837 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7f80ccdd0 0x7fc7f8149170 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 >> 172.21.15.60:0/494962837 conn(0x7fc7f80b2390 msgr2=0x7fc7f80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 shutdown_connections 2024-04-25T08:30:17.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.316+0000 7fc7feed1640 1 -- 172.21.15.60:0/494962837 wait complete. 2024-04-25T08:30:17.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:30:17.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.430+0000 7f117077e640 1 Processor -- start 2024-04-25T08:30:17.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.430+0000 7f117077e640 1 -- start start 2024-04-25T08:30:17.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.431+0000 7f117077e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f116806c330 0x7f11680b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.431+0000 7f117077e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680d1d50 con 0x7f1168158230 2024-04-25T08:30:17.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.431+0000 7f117077e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680d2020 con 0x7f116806c7e0 2024-04-25T08:30:17.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.431+0000 7f117077e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1168158610 con 0x7f116806c330 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116e51a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1168158230 0x7f11680d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39276/0 (socket says 172.21.15.60:39276) 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116e51a640 1 -- 172.21.15.60:0/3336819656 learned_addr learned my addr 172.21.15.60:0/3336819656 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643820335 0 0) 0x7f11680d1d50 con 0x7f1168158230 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1154003520 con 0x7f1168158230 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116dd19640 1 --2- 172.21.15.60:0/3336819656 >> v2:172.21.15.100:3300/0 conn(0x7f116806c330 0x7f11680b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190327130 0 0) 0x7f1154003520 con 0x7f1168158230 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 >> v2:172.21.15.100:3300/0 conn(0x7f116806c330 msgr2=0x7f11680b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 --2- 172.21.15.60:0/3336819656 >> v2:172.21.15.100:3300/0 conn(0x7f116806c330 0x7f11680b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 >> v1:172.21.15.100:6789/0 conn(0x7f116806c7e0 legacy=0x7f116806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680d2300 con 0x7f1168158230 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1164001670 con 0x7f1168158230 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 >> v1:172.21.15.60:6789/0 conn(0x7f1168158230 legacy=0x7f11680d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11540039b0 0x7f1154003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1154004800 0x7f1154004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f115400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1168158610 con 0x7f11540051c0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115400c060 con 0x7f11540039b0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115400c260 con 0x7f1154004800 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116ed1b640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f115400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116e51a640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11540039b0 0x7f1154003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.432+0000 7f116dd19640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1154004800 0x7f1154004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 -- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1154004800 msgr2=0x7f1154004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1154004800 0x7f1154004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 -- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11540039b0 msgr2=0x7f1154003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11540039b0 0x7f1154003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 -- 172.21.15.60:0/3336819656 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1154003520 con 0x7f11540051c0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 -- 172.21.15.60:0/3336819656 reap_dead start 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.433+0000 7f116ed1b640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f115400b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1160001e50 tx=0x7f1154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bfcf06e863130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160001a70 con 0x7f11540051c0 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1160003970 con 0x7f11540051c0 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f116d518640 1 -- 172.21.15.60:0/3336819656 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160003d60 con 0x7f11540051c0 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 -- 172.21.15.60:0/3336819656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 msgr2=0x7f115400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f115400b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1160001e50 tx=0x7f1154005830 comp rx=0 tx=0).stop 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 -- 172.21.15.60:0/3336819656 shutdown_connections 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 --2- 172.21.15.60:0/3336819656 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f115400b9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 -- 172.21.15.60:0/3336819656 >> 172.21.15.60:0/3336819656 conn(0x7f1168061600 msgr2=0x7f11680619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 -- 172.21.15.60:0/3336819656 shutdown_connections 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.434+0000 7f117077e640 1 -- 172.21.15.60:0/3336819656 wait complete. 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.435+0000 7f117077e640 1 Processor -- start 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.435+0000 7f117077e640 1 -- start start 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f117077e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f11681591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f117077e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116815b010 0x7f11681460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f117077e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11681e3fa0 0x7f11681e4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f117077e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680d25e0 con 0x7f11540051c0 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f117077e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116806bea0 con 0x7f116815b010 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f117077e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116815e320 con 0x7f11681e3fa0 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f11681591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f11681591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45780/0 (socket says 172.21.15.60:45780) 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 -- 172.21.15.60:0/2076490845 learned_addr learned my addr 172.21.15.60:0/2076490845 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116ed1b640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11681e3fa0 0x7f11681e4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116dd19640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116815b010 0x7f11681460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 -- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11681e3fa0 msgr2=0x7f11681e4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11681e3fa0 0x7f11681e4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 -- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116815b010 msgr2=0x7f11681460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116815b010 0x7f11681460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116e51a640 1 -- 172.21.15.60:0/2076490845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116815b490 con 0x7f11540051c0 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.436+0000 7f116dd19640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116815b010 0x7f11681460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:17.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.437+0000 7f116e51a640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f11681591a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f11640073a0 tx=0x7f1164007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.437+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116401a070 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.437+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400f960 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.437+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11681598a0 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.437+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400fd30 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.437+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1168159e60 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.438+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116806de50 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.438+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f116401d070 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.439+0000 7f115effd640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f114c06b4d0 0x7f114c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.439+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1164089e80 con 0x7f11540051c0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.439+0000 7f116dd19640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f114c06b4d0 0x7f114c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.439+0000 7f116dd19640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f114c06b4d0 0x7f114c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f116815a980 tx=0x7f1158008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.443+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11640589b0 con 0x7f11540051c0 2024-04-25T08:30:17.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.623+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1168073a00 con 0x7f11540051c0 2024-04-25T08:30:17.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.624+0000 7f115effd640 1 -- 172.21.15.60:0/2076490845 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11640582b0 con 0x7f11540051c0 2024-04-25T08:30:17.622 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f114c06b4d0 msgr2=0x7f114c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f114c06b4d0 0x7f114c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f116815a980 tx=0x7f1158008040 comp rx=0 tx=0).stop 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 msgr2=0x7f11681591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f11681591a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f11640073a0 tx=0x7f1164007860 comp rx=0 tx=0).stop 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 shutdown_connections 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11681e3fa0 0x7f11681e4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f114c06b4d0 0x7f114c06d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f116815b010 0x7f11681460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 --2- 172.21.15.60:0/2076490845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11540051c0 0x7f11681591a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 >> 172.21.15.60:0/2076490845 conn(0x7f1168061600 msgr2=0x7f11680b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 shutdown_connections 2024-04-25T08:30:17.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.626+0000 7f117077e640 1 -- 172.21.15.60:0/2076490845 wait complete. 2024-04-25T08:30:17.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:30:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.740+0000 7fb6adb1b640 1 Processor -- start 2024-04-25T08:30:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.740+0000 7fb6adb1b640 1 -- start start 2024-04-25T08:30:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6adb1b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6adb1b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6a8161710 0x7fb6a80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6adb1b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6a80b9330 0x7fb6a806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6adb1b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a815e000 con 0x7fb6a80b9330 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6adb1b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a80d1de0 con 0x7fb6a8161710 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6adb1b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8161c60 con 0x7fb6a815dc20 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6a7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6a80b9330 0x7fb6a806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6a7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6a80b9330 0x7fb6a806bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45782/0 (socket says 172.21.15.60:45782) 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3961064515 learned_addr learned my addr 172.21.15.60:0/3961064515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6a77fe640 1 --2- 172.21.15.60:0/3961064515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.741+0000 7fb6a6ffd640 1 --2- 172.21.15.60:0/3961064515 >> v2:172.21.15.100:3300/0 conn(0x7fb6a8161710 0x7fb6a80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.742+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3961064515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 msgr2=0x7fb6a80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.742+0000 7fb6a7fff640 1 --2- 172.21.15.60:0/3961064515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.742+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3961064515 >> v2:172.21.15.100:3300/0 conn(0x7fb6a8161710 msgr2=0x7fb6a80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.742+0000 7fb6a7fff640 1 --2- 172.21.15.60:0/3961064515 >> v2:172.21.15.100:3300/0 conn(0x7fb6a8161710 0x7fb6a80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.742+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3961064515 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8161f40 con 0x7fb6a80b9330 2024-04-25T08:30:17.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.742+0000 7fb6a7fff640 1 --2- 172.21.15.60:0/3961064515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80b9330 0x7fb6a806bf80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6a806d9a0 tx=0x7fb6a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823857aff92ab9fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6a67fc640 1 -- 172.21.15.60:0/3961064515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0036070 con 0x7fb6a80b9330 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6a67fc640 1 -- 172.21.15.60:0/3961064515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0031600 con 0x7fb6a80b9330 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6a67fc640 1 -- 172.21.15.60:0/3961064515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00319d0 con 0x7fb6a80b9330 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3961064515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80b9330 msgr2=0x7fb6a806bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3961064515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80b9330 0x7fb6a806bf80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6a806d9a0 tx=0x7fb6a002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3961064515 shutdown_connections 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3961064515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a80b9330 0x7fb6a806bf80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3961064515 >> v2:172.21.15.100:3300/0 conn(0x7fb6a8161710 0x7fb6a80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3961064515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3961064515 >> 172.21.15.60:0/3961064515 conn(0x7fb6a80d0420 msgr2=0x7fb6a80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3961064515 shutdown_connections 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.743+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3961064515 wait complete. 2024-04-25T08:30:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.744+0000 7fb6adb1b640 1 Processor -- start 2024-04-25T08:30:17.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 -- start start 2024-04-25T08:30:17.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80b9330 0x7fb6a81f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a81f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 0x7fb6a815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80b5e50 con 0x7fb6a8161710 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8162500 con 0x7fb6a80b9330 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6adb1b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80b5fd0 con 0x7fb6a815dc20 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 0x7fb6a815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 0x7fb6a815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45798/0 (socket says 172.21.15.60:45798) 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.745+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3774864355 learned_addr learned my addr 172.21.15.60:0/3774864355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a6ffd640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a81f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a77fe640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80b9330 0x7fb6a81f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 msgr2=0x7fb6a81f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a7fff640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a81f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80b9330 msgr2=0x7fb6a81f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a7fff640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80b9330 0x7fb6a81f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a7fff640 1 -- 172.21.15.60:0/3774864355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80aa380 con 0x7fb6a8161710 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6a7fff640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 0x7fb6a815cc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009f40 tx=0x7fb6a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0041020 con 0x7fb6a8161710 2024-04-25T08:30:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00077a0 con 0x7fb6a8161710 2024-04-25T08:30:17.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a81f3ec0 con 0x7fb6a8161710 2024-04-25T08:30:17.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0036040 con 0x7fb6a8161710 2024-04-25T08:30:17.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.746+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a80aa620 con 0x7fb6a8161710 2024-04-25T08:30:17.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.747+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a81f42b0 con 0x7fb6a8161710 2024-04-25T08:30:17.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.748+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6a0044020 con 0x7fb6a8161710 2024-04-25T08:30:17.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.748+0000 7fb68ffff640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb69406b4d0 0x7fb69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:17.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.748+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb6a00aafc0 con 0x7fb6a8161710 2024-04-25T08:30:17.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.748+0000 7fb6a77fe640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb69406b4d0 0x7fb69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:17.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.749+0000 7fb6a77fe640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb69406b4d0 0x7fb69406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb698009a20 tx=0x7fb698009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:17.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.753+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6a004b030 con 0x7fb6a8161710 2024-04-25T08:30:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.931+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb6a8073a00 con 0x7fb6a8161710 2024-04-25T08:30:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.932+0000 7fb68ffff640 1 -- 172.21.15.60:0/3774864355 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb6a0079540 con 0x7fb6a8161710 2024-04-25T08:30:17.929 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb69406b4d0 msgr2=0x7fb69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb69406b4d0 0x7fb69406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb698009a20 tx=0x7fb698009190 comp rx=0 tx=0).stop 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 msgr2=0x7fb6a815cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 0x7fb6a815cc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009f40 tx=0x7fb6a0004a40 comp rx=0 tx=0).stop 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 shutdown_connections 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6a8161710 0x7fb6a815cc10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6a815dc20 0x7fb6a81f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb69406b4d0 0x7fb69406d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 --2- 172.21.15.60:0/3774864355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6a80b9330 0x7fb6a81f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 >> 172.21.15.60:0/3774864355 conn(0x7fb6a80d0420 msgr2=0x7fb6a80d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:17.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 shutdown_connections 2024-04-25T08:30:17.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:17.934+0000 7fb6adb1b640 1 -- 172.21.15.60:0/3774864355 wait complete. 2024-04-25T08:30:17.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:30:18.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.046+0000 7f28db39e640 1 Processor -- start 2024-04-25T08:30:18.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.046+0000 7f28db39e640 1 -- start start 2024-04-25T08:30:18.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28db39e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415dd20 0x7f28d40b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28db39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28d415e100 0x7f28d40d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28db39e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f28d415e560 0x7f28d40d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28db39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d40d1c00 con 0x7f28d415e100 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28db39e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d415eac0 con 0x7f28d415e560 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28db39e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d415eda0 con 0x7f28d415dd20 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28d415e100 0x7f28d40d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28d415e100 0x7f28d40d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45814/0 (socket says 172.21.15.60:45814) 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 -- 172.21.15.60:0/2023142074 learned_addr learned my addr 172.21.15.60:0/2023142074 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d993b640 1 --2- 172.21.15.60:0/2023142074 >> v2:172.21.15.100:3300/0 conn(0x7f28d415e560 0x7f28d40d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d913a640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415dd20 0x7f28d40b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 -- 172.21.15.60:0/2023142074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415dd20 msgr2=0x7f28d40b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415dd20 0x7f28d40b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 -- 172.21.15.60:0/2023142074 >> v2:172.21.15.100:3300/0 conn(0x7f28d415e560 msgr2=0x7f28d40d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:18.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.047+0000 7f28d8939640 1 --2- 172.21.15.60:0/2023142074 >> v2:172.21.15.100:3300/0 conn(0x7f28d415e560 0x7f28d40d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.048+0000 7f28d8939640 1 -- 172.21.15.60:0/2023142074 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d415f080 con 0x7f28d415e100 2024-04-25T08:30:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.048+0000 7f28d913a640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415dd20 0x7f28d40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.048+0000 7f28d8939640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415e100 0x7f28d40d0aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28d40cab60 tx=0x7f28c402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a94be526c6c60c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.048+0000 7f28cbfff640 1 -- 172.21.15.60:0/2023142074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4037070 con 0x7f28d415e100 2024-04-25T08:30:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.048+0000 7f28cbfff640 1 -- 172.21.15.60:0/2023142074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4032440 con 0x7f28d415e100 2024-04-25T08:30:18.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.048+0000 7f28cbfff640 1 -- 172.21.15.60:0/2023142074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4032810 con 0x7f28d415e100 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 -- 172.21.15.60:0/2023142074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415e100 msgr2=0x7f28d40d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415e100 0x7f28d40d0aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28d40cab60 tx=0x7f28c402a140 comp rx=0 tx=0).stop 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 -- 172.21.15.60:0/2023142074 shutdown_connections 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 --2- 172.21.15.60:0/2023142074 >> v2:172.21.15.100:3300/0 conn(0x7f28d415e560 0x7f28d40d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415e100 0x7f28d40d0aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 --2- 172.21.15.60:0/2023142074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415dd20 0x7f28d40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 -- 172.21.15.60:0/2023142074 >> 172.21.15.60:0/2023142074 conn(0x7f28d4061a20 msgr2=0x7f28d4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 -- 172.21.15.60:0/2023142074 shutdown_connections 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.049+0000 7f28db39e640 1 -- 172.21.15.60:0/2023142074 wait complete. 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.050+0000 7f28db39e640 1 Processor -- start 2024-04-25T08:30:18.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.050+0000 7f28db39e640 1 -- start start 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28db39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 0x7f28d40ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28db39e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415e100 0x7f28d40aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28db39e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415e560 0x7f28d4152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28db39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40d3c20 con 0x7f28d415dd20 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28db39e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40d3aa0 con 0x7f28d415e560 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28db39e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d40d3da0 con 0x7f28d415e100 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 0x7f28d40ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 0x7f28d40ab7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45822/0 (socket says 172.21.15.60:45822) 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 -- 172.21.15.60:0/247056261 learned_addr learned my addr 172.21.15.60:0/247056261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d8939640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415e100 0x7f28d40aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d993b640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415e560 0x7f28d4152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 -- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415e100 msgr2=0x7f28d40aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415e100 0x7f28d40aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 -- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415e560 msgr2=0x7f28d4152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415e560 0x7f28d4152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d913a640 1 -- 172.21.15.60:0/247056261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d40b1ab0 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.051+0000 7f28d993b640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415e560 0x7f28d4152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.052+0000 7f28d913a640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 0x7f28d40ab7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f28d000c7c0 tx=0x7f28d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.052+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0015070 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.052+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d0007a30 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.052+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0007e00 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.052+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d40b1d90 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.052+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d40b2350 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.053+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d4073990 con 0x7f28d415dd20 2024-04-25T08:30:18.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.053+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28d0018070 con 0x7f28d415dd20 2024-04-25T08:30:18.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.054+0000 7f28c9ffb640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28b806b4d0 0x7f28b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.054+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f28d008a4c0 con 0x7f28d415dd20 2024-04-25T08:30:18.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.054+0000 7f28d8939640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28b806b4d0 0x7f28b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.054+0000 7f28d8939640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28b806b4d0 0x7f28b806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28c4034000 tx=0x7f28c402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.058+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28d00588f0 con 0x7f28d415dd20 2024-04-25T08:30:18.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.280+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28d4062050 con 0x7f28d415dd20 2024-04-25T08:30:18.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.280+0000 7f28c9ffb640 1 -- 172.21.15.60:0/247056261 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f28d005c990 con 0x7f28d415dd20 2024-04-25T08:30:18.278 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:18.278 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:30:18.278 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:30:18.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.282+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28b806b4d0 msgr2=0x7f28b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.282+0000 7f28db39e640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28b806b4d0 0x7f28b806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28c4034000 tx=0x7f28c402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:18.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.282+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 msgr2=0x7f28d40ab7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.282+0000 7f28db39e640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 0x7f28d40ab7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f28d000c7c0 tx=0x7f28d000cc80 comp rx=0 tx=0).stop 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 shutdown_connections 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f28b806b4d0 0x7f28b806d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28d415e560 0x7f28d4152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28d415e100 0x7f28d40aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 --2- 172.21.15.60:0/247056261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28d415dd20 0x7f28d40ab7e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 >> 172.21.15.60:0/247056261 conn(0x7f28d4061a20 msgr2=0x7f28d415f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 shutdown_connections 2024-04-25T08:30:18.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.283+0000 7f28db39e640 1 -- 172.21.15.60:0/247056261 wait complete. 2024-04-25T08:30:18.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:18.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.393+0000 7f1895a41640 1 Processor -- start 2024-04-25T08:30:18.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 -- start start 2024-04-25T08:30:18.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f1890162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f18900b6cb0 0x7f18900b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f18900b5a60 0x7f189006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1890162690 con 0x7f18900b6cb0 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18900ceee0 con 0x7f18900b5a60 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f1895a41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18900d1d10 con 0x7f189015db70 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f188e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f18900b6cb0 0x7f18900b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.394+0000 7f188e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f18900b6cb0 0x7f18900b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45834/0 (socket says 172.21.15.60:45834) 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188e7fc640 1 -- 172.21.15.60:0/4201892846 learned_addr learned my addr 172.21.15.60:0/4201892846 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188effd640 1 --2- 172.21.15.60:0/4201892846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f1890162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188f7fe640 1 --2- 172.21.15.60:0/4201892846 >> v2:172.21.15.100:3300/0 conn(0x7f18900b5a60 0x7f189006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188e7fc640 1 -- 172.21.15.60:0/4201892846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 msgr2=0x7f1890162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188e7fc640 1 --2- 172.21.15.60:0/4201892846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f1890162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188e7fc640 1 -- 172.21.15.60:0/4201892846 >> v2:172.21.15.100:3300/0 conn(0x7f18900b5a60 msgr2=0x7f189006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188e7fc640 1 --2- 172.21.15.60:0/4201892846 >> v2:172.21.15.100:3300/0 conn(0x7f18900b5a60 0x7f189006c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188e7fc640 1 -- 172.21.15.60:0/4201892846 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900d1ff0 con 0x7f18900b6cb0 2024-04-25T08:30:18.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.395+0000 7f188f7fe640 1 --2- 172.21.15.60:0/4201892846 >> v2:172.21.15.100:3300/0 conn(0x7f18900b5a60 0x7f189006c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f188e7fc640 1 --2- 172.21.15.60:0/4201892846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900b5520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f189008bfb0 tx=0x7f187802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d446b5a38e0a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f188dffb640 1 -- 172.21.15.60:0/4201892846 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878037070 con 0x7f18900b6cb0 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f188dffb640 1 -- 172.21.15.60:0/4201892846 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1878032440 con 0x7f18900b6cb0 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f188dffb640 1 -- 172.21.15.60:0/4201892846 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878032810 con 0x7f18900b6cb0 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 -- 172.21.15.60:0/4201892846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 msgr2=0x7f18900b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 --2- 172.21.15.60:0/4201892846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900b5520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f189008bfb0 tx=0x7f187802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 -- 172.21.15.60:0/4201892846 shutdown_connections 2024-04-25T08:30:18.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 --2- 172.21.15.60:0/4201892846 >> v2:172.21.15.100:3300/0 conn(0x7f18900b5a60 0x7f189006c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 --2- 172.21.15.60:0/4201892846 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900b5520 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 --2- 172.21.15.60:0/4201892846 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f1890162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 -- 172.21.15.60:0/4201892846 >> 172.21.15.60:0/4201892846 conn(0x7f18900d0520 msgr2=0x7f18900d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 -- 172.21.15.60:0/4201892846 shutdown_connections 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.396+0000 7f1895a41640 1 -- 172.21.15.60:0/4201892846 wait complete. 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 Processor -- start 2024-04-25T08:30:18.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 -- start start 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18900b5a60 0x7f18901f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f18901f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18900ce530 con 0x7f18900b6cb0 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189015df50 con 0x7f18900b5a60 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f1895a41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18900cf1c0 con 0x7f189015db70 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.398+0000 7f188e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45846/0 (socket says 172.21.15.60:45846) 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 -- 172.21.15.60:0/2999509183 learned_addr learned my addr 172.21.15.60:0/2999509183 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188f7fe640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f18901f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188effd640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18900b5a60 0x7f18901f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 -- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 msgr2=0x7f18901f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f18901f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 -- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18900b5a60 msgr2=0x7f18901f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18900b5a60 0x7f18901f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 -- 172.21.15.60:0/2999509183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18901f12a0 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f188e7fc640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900cab70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f187802a220 tx=0x7f18780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878041070 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187803c870 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18900cb270 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878037040 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.399+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18900cb830 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.400+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1890073990 con 0x7f18900b6cb0 2024-04-25T08:30:18.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.401+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1878044070 con 0x7f18900b6cb0 2024-04-25T08:30:18.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.401+0000 7f1887fff640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f187006b4d0 0x7f187006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.401+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f18780ab410 con 0x7f18900b6cb0 2024-04-25T08:30:18.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.401+0000 7f188effd640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f187006b4d0 0x7f187006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.402+0000 7f188effd640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f187006b4d0 0x7f187006d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f18800099c0 tx=0x7f1880003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.405+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18901f12a0 con 0x7f18900b6cb0 2024-04-25T08:30:18.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.587+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18900d1820 con 0x7f18900b6cb0 2024-04-25T08:30:18.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.589+0000 7f1887fff640 1 -- 172.21.15.60:0/2999509183 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7f187807a040 con 0x7f18900b6cb0 2024-04-25T08:30:18.586 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:18.587 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f187006b4d0 msgr2=0x7f187006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f187006b4d0 0x7f187006d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f18800099c0 tx=0x7f1880003990 comp rx=0 tx=0).stop 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 msgr2=0x7f18900cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900cab70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f187802a220 tx=0x7f18780046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 shutdown_connections 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f189015db70 0x7f18901f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18900b6cb0 0x7f18900cab70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f187006b4d0 0x7f187006d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 --2- 172.21.15.60:0/2999509183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18900b5a60 0x7f18901f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 >> 172.21.15.60:0/2999509183 conn(0x7f18900d0520 msgr2=0x7f18900c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 shutdown_connections 2024-04-25T08:30:18.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.591+0000 7f1895a41640 1 -- 172.21.15.60:0/2999509183 wait complete. 2024-04-25T08:30:18.600 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:30:18.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:30:18.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.703+0000 7f51aae1c640 1 Processor -- start 2024-04-25T08:30:18.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.703+0000 7f51aae1c640 1 -- start start 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51aae1c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51a4158220 0x7f51a415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51aae1c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a40af720 0x7f51a40afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51aae1c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a415f180 con 0x7f51a41617b0 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51aae1c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40b3670 con 0x7f51a4158220 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51aae1c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4070a10 con 0x7f51a40af720 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51a93b9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f51a41617b0 0x7f51a4161c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39278/0 (socket says 172.21.15.60:39278) 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.704+0000 7f51a93b9640 1 -- 172.21.15.60:0/387180383 learned_addr learned my addr 172.21.15.60:0/387180383 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a8bb8640 1 --2- 172.21.15.60:0/387180383 >> v2:172.21.15.100:3300/0 conn(0x7f51a4158220 0x7f51a415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a3fff640 1 --2- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a40af720 0x7f51a40afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a37fe640 1 -- 172.21.15.60:0/387180383 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705845497 0 0) 0x7f519c001490 con 0x7f51a41617b0 2024-04-25T08:30:18.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a37fe640 1 -- 172.21.15.60:0/387180383 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c003520 con 0x7f51a41617b0 2024-04-25T08:30:18.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a8bb8640 1 -- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a40af720 msgr2=0x7f51a40afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:18.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a8bb8640 1 --2- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a40af720 0x7f51a40afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a8bb8640 1 -- 172.21.15.60:0/387180383 >> v1:172.21.15.60:6789/0 conn(0x7f51a41617b0 legacy=0x7f51a4161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.705+0000 7f51a8bb8640 1 -- 172.21.15.60:0/387180383 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a406fe00 con 0x7f51a4158220 2024-04-25T08:30:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51a8bb8640 1 --2- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415ec40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75a3fb30eb2c230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51a37fe640 1 -- 172.21.15.60:0/387180383 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194037070 con 0x7f51a4158220 2024-04-25T08:30:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51a37fe640 1 -- 172.21.15.60:0/387180383 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194032440 con 0x7f51a4158220 2024-04-25T08:30:18.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51a37fe640 1 -- 172.21.15.60:0/387180383 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194032810 con 0x7f51a4158220 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 -- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 msgr2=0x7f51a415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 --2- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415ec40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402a180 comp rx=0 tx=0).stop 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 -- 172.21.15.60:0/387180383 shutdown_connections 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 --2- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a40af720 0x7f51a40afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 --2- 172.21.15.60:0/387180383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415ec40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 -- 172.21.15.60:0/387180383 >> 172.21.15.60:0/387180383 conn(0x7f51a40b2370 msgr2=0x7f51a40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.706+0000 7f51aae1c640 1 -- 172.21.15.60:0/387180383 shutdown_connections 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.707+0000 7f51aae1c640 1 -- 172.21.15.60:0/387180383 wait complete. 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 Processor -- start 2024-04-25T08:30:18.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 -- start start 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 0x7f51a40cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41617b0 0x7f51a415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40706a0 con 0x7f51a40af720 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40703c0 con 0x7f51a4158220 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.708+0000 7f51aae1c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4158600 con 0x7f51a41617b0 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 0x7f51a40cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 0x7f51a40cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45862/0 (socket says 172.21.15.60:45862) 2024-04-25T08:30:18.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 -- 172.21.15.60:0/1070152033 learned_addr learned my addr 172.21.15.60:0/1070152033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a3fff640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a93b9640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41617b0 0x7f51a415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 -- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41617b0 msgr2=0x7f51a415c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41617b0 0x7f51a415c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 -- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 msgr2=0x7f51a415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 -- 172.21.15.60:0/1070152033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40b1b50 con 0x7f51a40af720 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a93b9640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41617b0 0x7f51a415c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a3fff640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a8bb8640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 0x7f51a40cc320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5194035040 tx=0x7f51940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194041030 con 0x7f51a40af720 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519403d4e0 con 0x7f51a40af720 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194037040 con 0x7f51a40af720 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a40b1e30 con 0x7f51a40af720 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.709+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a40cc860 con 0x7f51a40af720 2024-04-25T08:30:18.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.710+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4073990 con 0x7f51a40af720 2024-04-25T08:30:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.711+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5194044070 con 0x7f51a40af720 2024-04-25T08:30:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.711+0000 7f51a17fa640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f519006b4d0 0x7f519006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.711+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f51940aac80 con 0x7f51a40af720 2024-04-25T08:30:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.712+0000 7f51a3fff640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f519006b4d0 0x7f519006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:18.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.712+0000 7f51a3fff640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f519006b4d0 0x7f519006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f51a41f72d0 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:18.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.716+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5194044e40 con 0x7f51a40af720 2024-04-25T08:30:18.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.935+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51a40ccbb0 con 0x7f51a40af720 2024-04-25T08:30:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.935+0000 7f51a17fa640 1 -- 172.21.15.60:0/1070152033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f5194079710 con 0x7f51a40af720 2024-04-25T08:30:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:18.933 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:30:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:30:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f519006b4d0 msgr2=0x7f519006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f519006b4d0 0x7f519006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f51a41f72d0 tx=0x7f5198008040 comp rx=0 tx=0).stop 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 msgr2=0x7f51a40cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 0x7f51a40cc320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5194035040 tx=0x7f51940046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 shutdown_connections 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51a41617b0 0x7f51a415c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f519006b4d0 0x7f519006d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51a4158220 0x7f51a415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 --2- 172.21.15.60:0/1070152033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51a40af720 0x7f51a40cc320 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 >> 172.21.15.60:0/1070152033 conn(0x7f51a40b2370 msgr2=0x7f51a415fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 shutdown_connections 2024-04-25T08:30:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:18.938+0000 7f51aae1c640 1 -- 172.21.15.60:0/1070152033 wait complete. 2024-04-25T08:30:18.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:30:19.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.050+0000 7f02ce3d5640 1 Processor -- start 2024-04-25T08:30:19.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.050+0000 7f02ce3d5640 1 -- start start 2024-04-25T08:30:19.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02ce3d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 0x7f02c80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02ce3d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02c80cfcf0 0x7f02c80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02ce3d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c8071480 con 0x7f02c80ce3c0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02ce3d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c80ce830 con 0x7f02c80cfcf0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02ce3d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8072090 con 0x7f02c8154ee0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02cc972640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f02c80ce3c0 0x7f02c815e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39290/0 (socket says 172.21.15.60:39290) 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02cc972640 1 -- 172.21.15.60:0/1403599603 learned_addr learned my addr 172.21.15.60:0/1403599603 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602521141 0 0) 0x7f02c8071480 con 0x7f02c80ce3c0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02c7fff640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 0x7f02c80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02c77fe640 1 --2- 172.21.15.60:0/1403599603 >> v2:172.21.15.100:3300/0 conn(0x7f02c80cfcf0 0x7f02c80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.051+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02b4003520 con 0x7f02c80ce3c0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719999981 0 0) 0x7f02b4003520 con 0x7f02c80ce3c0 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 msgr2=0x7f02c80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 0x7f02c80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 >> v2:172.21.15.100:3300/0 conn(0x7f02c80cfcf0 msgr2=0x7f02c80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 --2- 172.21.15.60:0/1403599603 >> v2:172.21.15.100:3300/0 conn(0x7f02c80cfcf0 0x7f02c80cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c8071760 con 0x7f02c80ce3c0 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f02c00019f0 con 0x7f02c80ce3c0 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 >> v1:172.21.15.60:6789/0 conn(0x7f02c80ce3c0 legacy=0x7f02c815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 0x7f02b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02b4003f30 0x7f02b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02b4004d40 0x7f02b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c80ce830 con 0x7f02b4003f30 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b400b9c0 con 0x7f02b4004d40 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b400bb40 con 0x7f02c8154ee0 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c77fe640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02b4003f30 0x7f02b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c7fff640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 0x7f02b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02cc972640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02b4004d40 0x7f02b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c77fe640 1 -- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 msgr2=0x7f02b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c77fe640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02c8154ee0 0x7f02b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c77fe640 1 -- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02b4004d40 msgr2=0x7f02b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c77fe640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02b4004d40 0x7f02b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.052+0000 7f02c77fe640 1 -- 172.21.15.60:0/1403599603 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4003520 con 0x7f02b4003f30 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.053+0000 7f02c77fe640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02b4003f30 0x7f02b4004800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009ac0 tx=0x7f02b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bc9b5dda08632e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.053+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8036070 con 0x7f02b4003f30 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.053+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b8031400 con 0x7f02b4003f30 2024-04-25T08:30:19.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.053+0000 7f02c6ffd640 1 -- 172.21.15.60:0/1403599603 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8009690 con 0x7f02b4003f30 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1403599603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02b4003f30 msgr2=0x7f02b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1403599603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02b4003f30 0x7f02b4004800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009ac0 tx=0x7f02b4005830 comp rx=0 tx=0).stop 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02cc972640 1 -- 172.21.15.60:0/1403599603 reap_dead start 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1403599603 shutdown_connections 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1403599603 >> 172.21.15.60:0/1403599603 conn(0x7f02c806abe0 msgr2=0x7f02c806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1403599603 shutdown_connections 2024-04-25T08:30:19.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.054+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1403599603 wait complete. 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.055+0000 7f02ce3d5640 1 Processor -- start 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.055+0000 7f02ce3d5640 1 -- start start 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02ce3d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02b4003f30 0x7f02c8148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02ce3d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c8151410 0x7f02c81599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02ce3d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 0x7f02c8166980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02ce3d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8071230 con 0x7f02c8159f20 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02ce3d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c8071d20 con 0x7f02c8151410 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02ce3d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c81552c0 con 0x7f02b4003f30 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 0x7f02c8166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 0x7f02c8166980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45880/0 (socket says 172.21.15.60:45880) 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 -- 172.21.15.60:0/1506636956 learned_addr learned my addr 172.21.15.60:0/1506636956 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02c77fe640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c8151410 0x7f02c81599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02c7fff640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02b4003f30 0x7f02c8148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 -- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02b4003f30 msgr2=0x7f02c8148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02b4003f30 0x7f02c8148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 -- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c8151410 msgr2=0x7f02c81599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c8151410 0x7f02c81599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 -- 172.21.15.60:0/1506636956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c81676d0 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02cc972640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 0x7f02c8166980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f02c00013d0 tx=0x7f02c0007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.056+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c000abb0 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.057+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c000ad50 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.057+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c814bcd0 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.057+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0013630 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.057+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c814c290 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.058+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c8073990 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.058+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02c00137d0 con 0x7f02c8159f20 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.058+0000 7f02c4ff9640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02a806b500 0x7f02a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.059+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f02c008aa60 con 0x7f02c8159f20 2024-04-25T08:30:19.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.059+0000 7f02c7fff640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02a806b500 0x7f02a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.059+0000 7f02c7fff640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02a806b500 0x7f02a806d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009a20 tx=0x7f02b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.063+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02c0059570 con 0x7f02c8159f20 2024-04-25T08:30:19.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.281+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02c806db20 con 0x7f02c8159f20 2024-04-25T08:30:19.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.282+0000 7f02c4ff9640 1 -- 172.21.15.60:0/1506636956 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f02c0058e70 con 0x7f02c8159f20 2024-04-25T08:30:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:19.279 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3481027691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3481027691},{"type":"v1","addr":"172.21.15.100:6837","nonce":3481027691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3927225252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3927225252},{"type":"v1","addr":"172.21.15.60:6833","nonce":3927225252}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/159034642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":159034642},{"type":"v1","addr":"172.21.15.60:6835","nonce":159034642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:29:23.379232+0000","modified":"2024-04-25T08:29:39.468631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/3799475286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3799475286},{"type":"v1","addr":"172.21.15.100:6836","nonce":3799475286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:30:19.280 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 8 2024-04-25T08:30:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.283+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02a806b500 msgr2=0x7f02a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.283+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02a806b500 0x7f02a806d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009a20 tx=0x7f02b0008040 comp rx=0 tx=0).stop 2024-04-25T08:30:19.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.283+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 msgr2=0x7f02c8166980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.283+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 0x7f02c8166980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f02c00013d0 tx=0x7f02c0007b20 comp rx=0 tx=0).stop 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 shutdown_connections 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f02c8159f20 0x7f02c8166980 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02a806b500 0x7f02a806d9c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02c8151410 0x7f02c81599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 --2- 172.21.15.60:0/1506636956 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f02b4003f30 0x7f02c8148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 >> 172.21.15.60:0/1506636956 conn(0x7f02c806abe0 msgr2=0x7f02c80af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 shutdown_connections 2024-04-25T08:30:19.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.284+0000 7f02ce3d5640 1 -- 172.21.15.60:0/1506636956 wait complete. 2024-04-25T08:30:19.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.395+0000 7f3841b7e640 1 Processor -- start 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f3841b7e640 1 -- start start 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f3841b7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f383c15ba00 0x7f383c09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f3841b7e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0cd710 con 0x7f383c1602e0 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f3841b7e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c099b70 con 0x7f383c15f5f0 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f3841b7e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383c0cd4b0 con 0x7f383c15ba00 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f383b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f383c15ba00 0x7f383c09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f383affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f383c1602e0 0x7f383c0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39302/0 (socket says 172.21.15.60:39302) 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.396+0000 7f383affd640 1 -- 172.21.15.60:0/173091215 learned_addr learned my addr 172.21.15.60:0/173091215 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:19.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.397+0000 7f383a7fc640 1 -- 172.21.15.60:0/173091215 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935265721 0 0) 0x7f3830001110 con 0x7f383c1602e0 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.397+0000 7f383a7fc640 1 -- 172.21.15.60:0/173091215 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3824003520 con 0x7f383c1602e0 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.397+0000 7f383b7fe640 1 -- 172.21.15.60:0/173091215 >> v1:172.21.15.100:6789/0 conn(0x7f383c15f5f0 legacy=0x7f383c0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.397+0000 7f383b7fe640 1 -- 172.21.15.60:0/173091215 >> v1:172.21.15.60:6789/0 conn(0x7f383c1602e0 legacy=0x7f383c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.397+0000 7f383b7fe640 1 -- 172.21.15.60:0/173091215 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c099e50 con 0x7f383c15ba00 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f383b7fe640 1 --2- 172.21.15.60:0/173091215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15ba00 0x7f383c09a930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f382c0097a0 tx=0x7f382c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabece757ed08acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f383a7fc640 1 -- 172.21.15.60:0/173091215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c037070 con 0x7f383c15ba00 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f383a7fc640 1 -- 172.21.15.60:0/173091215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c032440 con 0x7f383c15ba00 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f383a7fc640 1 -- 172.21.15.60:0/173091215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c02ae60 con 0x7f383c15ba00 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 -- 172.21.15.60:0/173091215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15ba00 msgr2=0x7f383c09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 --2- 172.21.15.60:0/173091215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15ba00 0x7f383c09a930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f382c0097a0 tx=0x7f382c02a110 comp rx=0 tx=0).stop 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 -- 172.21.15.60:0/173091215 shutdown_connections 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 --2- 172.21.15.60:0/173091215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15ba00 0x7f383c09a930 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 -- 172.21.15.60:0/173091215 >> 172.21.15.60:0/173091215 conn(0x7f383c0b3270 msgr2=0x7f383c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 -- 172.21.15.60:0/173091215 shutdown_connections 2024-04-25T08:30:19.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.398+0000 7f3841b7e640 1 -- 172.21.15.60:0/173091215 wait complete. 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.400+0000 7f3841b7e640 1 Processor -- start 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.400+0000 7f3841b7e640 1 -- start start 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.400+0000 7f3841b7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c15ba00 0x7f383c15adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.400+0000 7f3841b7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 0x7f383c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3841b7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c1602e0 0x7f383c06cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3841b7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c15bde0 con 0x7f383c15f5f0 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3841b7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c09ae70 con 0x7f383c1602e0 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3841b7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1e5ae0 con 0x7f383c15ba00 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 0x7f383c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 0x7f383c06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45894/0 (socket says 172.21.15.60:45894) 2024-04-25T08:30:19.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 -- 172.21.15.60:0/1028122548 learned_addr learned my addr 172.21.15.60:0/1028122548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383bfff640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c1602e0 0x7f383c06cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383b7fe640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c15ba00 0x7f383c15adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 -- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c15ba00 msgr2=0x7f383c15adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c15ba00 0x7f383c15adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 -- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c1602e0 msgr2=0x7f383c06cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c1602e0 0x7f383c06cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 -- 172.21.15.60:0/1028122548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0caf00 con 0x7f383c15f5f0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383b7fe640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c15ba00 0x7f383c15adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f383affd640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 0x7f383c06c670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f383000c730 tx=0x7f383000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830017070 con 0x7f383c15f5f0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3830005ab0 con 0x7f383c15f5f0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383c0cb1e0 con 0x7f383c15f5f0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830007070 con 0x7f383c15f5f0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.401+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383c1e5770 con 0x7f383c15f5f0 2024-04-25T08:30:19.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.402+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383c073990 con 0x7f383c15f5f0 2024-04-25T08:30:19.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.403+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f383001a070 con 0x7f383c15f5f0 2024-04-25T08:30:19.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.403+0000 7f3823fff640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.403+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3830089ae0 con 0x7f383c15f5f0 2024-04-25T08:30:19.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.404+0000 7f383b7fe640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.404+0000 7f383b7fe640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382806b4d0 0x7f382806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f382c034000 tx=0x7f382c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.408+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f383c0cd710 con 0x7f383c15f5f0 2024-04-25T08:30:19.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.589+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f383c0b75c0 con 0x7f383c15f5f0 2024-04-25T08:30:19.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.590+0000 7f3823fff640 1 -- 172.21.15.60:0/1028122548 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18889 (secure 0 0 0) 0x7f38300584d0 con 0x7f383c15f5f0 2024-04-25T08:30:19.588 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:19.588 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":27,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:05.183585+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:29:21.558014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:29:22.368855+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:29:26.081283+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":1}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":3,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382806b4d0 msgr2=0x7f382806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382806b4d0 0x7f382806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f382c034000 tx=0x7f382c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 msgr2=0x7f383c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 0x7f383c06c670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f383000c730 tx=0x7f383000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 shutdown_connections 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383c1602e0 0x7f383c06cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f383c15f5f0 0x7f383c06c670 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 --2- 172.21.15.60:0/1028122548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383c15ba00 0x7f383c15adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 >> 172.21.15.60:0/1028122548 conn(0x7f383c0b3270 msgr2=0x7f383c0c5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 shutdown_connections 2024-04-25T08:30:19.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.592+0000 7f3841b7e640 1 -- 172.21.15.60:0/1028122548 wait complete. 2024-04-25T08:30:19.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:30:19.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.704+0000 7f5eab64a640 1 Processor -- start 2024-04-25T08:30:19.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.704+0000 7f5eab64a640 1 -- start start 2024-04-25T08:30:19.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.705+0000 7f5eab64a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ea41610f0 0x7f5ea406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.705+0000 7f5eab64a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea40ce7b0 0x7f5ea4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.705+0000 7f5eab64a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea40c6960 con 0x7f5ea41616f0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5eab64a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea41614d0 con 0x7f5ea41610f0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5eab64a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4162260 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea8be5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5ea41616f0 0x7f5ea40ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39318/0 (socket says 172.21.15.60:39318) 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea8be5640 1 -- 172.21.15.60:0/3479810854 learned_addr learned my addr 172.21.15.60:0/3479810854 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea93e6640 1 --2- 172.21.15.60:0/3479810854 >> v2:172.21.15.60:3300/0 conn(0x7f5ea41610f0 0x7f5ea406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea9be7640 1 --2- 172.21.15.60:0/3479810854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea40ce7b0 0x7f5ea4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea3fff640 1 -- 172.21.15.60:0/3479810854 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129643376 0 0) 0x7f5ea40c6960 con 0x7f5ea41616f0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea3fff640 1 -- 172.21.15.60:0/3479810854 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e8c003520 con 0x7f5ea41616f0 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3479810854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea40ce7b0 msgr2=0x7f5ea4161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea93e6640 1 --2- 172.21.15.60:0/3479810854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea40ce7b0 0x7f5ea4161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3479810854 >> v1:172.21.15.60:6789/0 conn(0x7f5ea41616f0 legacy=0x7f5ea40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.706+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3479810854 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea4162540 con 0x7f5ea41610f0 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5ea93e6640 1 --2- 172.21.15.60:0/3479810854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea41610f0 0x7f5ea406c1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0097a0 tx=0x7f5e9c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446ff275916f8343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5ea3fff640 1 -- 172.21.15.60:0/3479810854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c037070 con 0x7f5ea41610f0 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5ea3fff640 1 -- 172.21.15.60:0/3479810854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c032440 con 0x7f5ea41610f0 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5ea3fff640 1 -- 172.21.15.60:0/3479810854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0327f0 con 0x7f5ea41610f0 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5eab64a640 1 -- 172.21.15.60:0/3479810854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea41610f0 msgr2=0x7f5ea406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3479810854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea41610f0 0x7f5ea406c1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0097a0 tx=0x7f5e9c02a180 comp rx=0 tx=0).stop 2024-04-25T08:30:19.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5eab64a640 1 -- 172.21.15.60:0/3479810854 shutdown_connections 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3479810854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea40ce7b0 0x7f5ea4161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3479810854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea41610f0 0x7f5ea406c1c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.707+0000 7f5eab64a640 1 -- 172.21.15.60:0/3479810854 >> 172.21.15.60:0/3479810854 conn(0x7f5ea406f500 msgr2=0x7f5ea406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.708+0000 7f5eab64a640 1 -- 172.21.15.60:0/3479810854 shutdown_connections 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.708+0000 7f5eab64a640 1 -- 172.21.15.60:0/3479810854 wait complete. 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 Processor -- start 2024-04-25T08:30:19.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 -- start start 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 0x7f5ea41441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ea41610f0 0x7f5ea40adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea41616f0 0x7f5ea40ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40cf690 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea40cfce0 con 0x7f5ea41610f0 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5eab64a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea406c700 con 0x7f5ea41616f0 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5ea93e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 0x7f5ea41441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.709+0000 7f5ea93e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 0x7f5ea41441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45912/0 (socket says 172.21.15.60:45912) 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3059645752 learned_addr learned my addr 172.21.15.60:0/3059645752 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea8be5640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ea41610f0 0x7f5ea40adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea41616f0 msgr2=0x7f5ea40ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea9be7640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea41616f0 0x7f5ea40ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea41616f0 0x7f5ea40ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ea41610f0 msgr2=0x7f5ea40adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ea41610f0 0x7f5ea40adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 -- 172.21.15.60:0/3059645752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea40b0a80 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea8be5640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ea41610f0 0x7f5ea40adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea93e6640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 0x7f5ea41441c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c035040 tx=0x7f5e9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c037040 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0044c0 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0325f0 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea40b0d60 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.710+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4144b30 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.711+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea4073a00 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.712+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e9c044020 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.712+0000 7f5ea1ffb640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9006b4d0 0x7f5e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:19.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.713+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5e9c0aa8f0 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.713+0000 7f5ea8be5640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9006b4d0 0x7f5e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:19.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.713+0000 7f5ea8be5640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9006b4d0 0x7f5e9006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4161b30 tx=0x7f5e98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:19.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.717+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e9c079360 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.935+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5ea40008d0 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.933 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:30:19.936+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:30:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.937+0000 7f5ea1ffb640 1 -- 172.21.15.60:0/3059645752 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5e9c078c60 con 0x7f5ea40ce7b0 2024-04-25T08:30:19.934 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:30:19.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9006b4d0 msgr2=0x7f5e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9006b4d0 0x7f5e9006d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4161b30 tx=0x7f5e98009040 comp rx=0 tx=0).stop 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 msgr2=0x7f5ea41441c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 0x7f5ea41441c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c035040 tx=0x7f5e9c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 shutdown_connections 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ea41616f0 0x7f5ea40ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9006b4d0 0x7f5e9006d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ea41610f0 0x7f5ea40adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 --2- 172.21.15.60:0/3059645752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ea40ce7b0 0x7f5ea41441c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 >> 172.21.15.60:0/3059645752 conn(0x7f5ea406f500 msgr2=0x7f5ea41603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 shutdown_connections 2024-04-25T08:30:19.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:19.940+0000 7f5eab64a640 1 -- 172.21.15.60:0/3059645752 wait complete. 2024-04-25T08:30:19.940 INFO:tasks.ceph.mds.b.smithi100.stderr: -4869> 2024-04-25T08:29:28.011+0000 7fdd51f8f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:30:19.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:30:19.973 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:30:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.055+0000 7f5edebe7640 1 Processor -- start 2024-04-25T08:30:20.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.055+0000 7f5edebe7640 1 -- start start 2024-04-25T08:30:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edebe7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ed80cfa40 0x7f5ed80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edebe7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed80c75e0 con 0x7f5ed815bb50 2024-04-25T08:30:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edebe7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed80c78c0 con 0x7f5ed80cf600 2024-04-25T08:30:20.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edebe7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed80c8680 con 0x7f5ed80cfa40 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edc983640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5ed815bb50 0x7f5ed80c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39324/0 (socket says 172.21.15.60:39324) 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edc983640 1 -- 172.21.15.60:0/1563025688 learned_addr learned my addr 172.21.15.60:0/1563025688 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5edd184640 1 --2- 172.21.15.60:0/1563025688 >> v2:172.21.15.100:3300/0 conn(0x7f5ed80cfa40 0x7f5ed80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.056+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996717498 0 0) 0x7f5ed80c75e0 con 0x7f5ed815bb50 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ec0003520 con 0x7f5ed815bb50 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704979037 0 0) 0x7f5ed80c78c0 con 0x7f5ed80cf600 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed80c75e0 con 0x7f5ed80cf600 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5edd184640 1 -- 172.21.15.60:0/1563025688 >> v1:172.21.15.100:6789/0 conn(0x7f5ed80cf600 legacy=0x7f5ed80cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5edd184640 1 -- 172.21.15.60:0/1563025688 >> v1:172.21.15.60:6789/0 conn(0x7f5ed815bb50 legacy=0x7f5ed80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5edd184640 1 -- 172.21.15.60:0/1563025688 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed80c7ba0 con 0x7f5ed80cfa40 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.057+0000 7f5edd184640 1 --2- 172.21.15.60:0/1563025688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cfa40 0x7f5ed80cc630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009d90 tx=0x7f5ed002a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b04c053ddcc4a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0037070 con 0x7f5ed80cfa40 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed0007740 con 0x7f5ed80cfa40 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5ecf7fe640 1 -- 172.21.15.60:0/1563025688 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0007b10 con 0x7f5ed80cfa40 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 -- 172.21.15.60:0/1563025688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cfa40 msgr2=0x7f5ed80cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 --2- 172.21.15.60:0/1563025688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cfa40 0x7f5ed80cc630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009d90 tx=0x7f5ed002a780 comp rx=0 tx=0).stop 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 -- 172.21.15.60:0/1563025688 shutdown_connections 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 --2- 172.21.15.60:0/1563025688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cfa40 0x7f5ed80cc630 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 -- 172.21.15.60:0/1563025688 >> 172.21.15.60:0/1563025688 conn(0x7f5ed806f500 msgr2=0x7f5ed806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 -- 172.21.15.60:0/1563025688 shutdown_connections 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.058+0000 7f5edebe7640 1 -- 172.21.15.60:0/1563025688 wait complete. 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.059+0000 7f5edebe7640 1 Processor -- start 2024-04-25T08:30:20.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.059+0000 7f5edebe7640 1 -- start start 2024-04-25T08:30:20.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edebe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cf600 0x7f5ed8152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edebe7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ed80cfa40 0x7f5ed80c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edebe7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 0x7f5ed80ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edebe7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80c72a0 con 0x7f5ed815bb50 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edebe7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80c8160 con 0x7f5ed80cf600 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edebe7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80cfe90 con 0x7f5ed80cfa40 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edd184640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 0x7f5ed80ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edd184640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 0x7f5ed80ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45920/0 (socket says 172.21.15.60:45920) 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edd184640 1 -- 172.21.15.60:0/2158457292 learned_addr learned my addr 172.21.15.60:0/2158457292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5ecffff640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ed80cfa40 0x7f5ed80c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.060+0000 7f5edc983640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cf600 0x7f5ed8152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edd184640 1 -- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ed80cfa40 msgr2=0x7f5ed80c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edd184640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ed80cfa40 0x7f5ed80c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edd184640 1 -- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cf600 msgr2=0x7f5ed8152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edd184640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cf600 0x7f5ed8152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edd184640 1 -- 172.21.15.60:0/2158457292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed815a360 con 0x7f5ed815bb50 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edd184640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 0x7f5ed80ca170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009d90 tx=0x7f5ed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0042020 con 0x7f5ed815bb50 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed00327f0 con 0x7f5ed815bb50 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed815db90 con 0x7f5ed815bb50 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0037040 con 0x7f5ed815bb50 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.061+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed815e150 con 0x7f5ed815bb50 2024-04-25T08:30:20.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.062+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed8073a00 con 0x7f5ed815bb50 2024-04-25T08:30:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.063+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ed0045020 con 0x7f5ed815bb50 2024-04-25T08:30:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.063+0000 7f5ecd7fa640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ec406b500 0x7f5ec406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.063+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f5ed00ab2d0 con 0x7f5ed815bb50 2024-04-25T08:30:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.063+0000 7f5edc983640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ec406b500 0x7f5ec406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:20.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.064+0000 7f5edc983640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ec406b500 0x7f5ec406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4000f60 tx=0x7f5ed4007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:20.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.068+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ed004c070 con 0x7f5ed815bb50 2024-04-25T08:30:20.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.286+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5ed80cf150 con 0x7f5ed815bb50 2024-04-25T08:30:20.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.939+0000 7f5ecd7fa640 1 -- 172.21.15.60:0/2158457292 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5ed815a360 con 0x7f5ed815bb50 2024-04-25T08:30:20.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.941+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ec406b500 msgr2=0x7f5ec406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.941+0000 7f5edebe7640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ec406b500 0x7f5ec406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4000f60 tx=0x7f5ed4007d50 comp rx=0 tx=0).stop 2024-04-25T08:30:20.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.941+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 msgr2=0x7f5ed80ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:20.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 0x7f5ed80ca170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009d90 tx=0x7f5ed0004a40 comp rx=0 tx=0).stop 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 shutdown_connections 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ed815bb50 0x7f5ed80ca170 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ed80cfa40 0x7f5ed80c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ec406b500 0x7f5ec406d9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 --2- 172.21.15.60:0/2158457292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ed80cf600 0x7f5ed8152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 >> 172.21.15.60:0/2158457292 conn(0x7f5ed806f500 msgr2=0x7f5ed80b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 shutdown_connections 2024-04-25T08:30:20.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:20.942+0000 7f5edebe7640 1 -- 172.21.15.60:0/2158457292 wait complete. 2024-04-25T08:30:20.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:30:21.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.066+0000 7f1e5b5b6640 1 Processor -- start 2024-04-25T08:30:21.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.067+0000 7f1e5b5b6640 1 -- start start 2024-04-25T08:30:21.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.067+0000 7f1e5b5b6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 0x7f1e540d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:21.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.067+0000 7f1e5b5b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:21.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.067+0000 7f1e5b5b6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540d2020 con 0x7f1e540d1b80 2024-04-25T08:30:21.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.067+0000 7f1e5b5b6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e54158610 con 0x7f1e54158230 2024-04-25T08:30:21.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e5b5b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b57c0 con 0x7f1e540b4e60 2024-04-25T08:30:21.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59b53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:21.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59352640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 0x7f1e540d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:21.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59352640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 0x7f1e540d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40680/0 (socket says 172.21.15.60:40680) 2024-04-25T08:30:21.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59352640 1 -- 172.21.15.60:0/1240451740 learned_addr learned my addr 172.21.15.60:0/1240451740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:21.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59b53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:60644/0 (socket says 172.21.15.60:60644) 2024-04-25T08:30:21.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59b53640 1 -- 172.21.15.60:0/1240451740 >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 msgr2=0x7f1e540d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:21.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59b53640 1 --2- 172.21.15.60:0/1240451740 >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 0x7f1e540d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:21.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59b53640 1 -- 172.21.15.60:0/1240451740 >> v1:172.21.15.100:6789/0 conn(0x7f1e540d1b80 legacy=0x7f1e5406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:21.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59b53640 1 -- 172.21.15.60:0/1240451740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540d2300 con 0x7f1e540b4e60 2024-04-25T08:30:21.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.068+0000 7f1e59352640 1 --2- 172.21.15.60:0/1240451740 >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 0x7f1e540d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:21.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.069+0000 7f1e59b53640 1 --2- 172.21.15.60:0/1240451740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540b5280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009d90 tx=0x7f1e4402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95442418842d5d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.069+0000 7f1e53fff640 1 -- 172.21.15.60:0/1240451740 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44007740 con 0x7f1e540b4e60 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.069+0000 7f1e53fff640 1 -- 172.21.15.60:0/1240451740 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e440078e0 con 0x7f1e540b4e60 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.069+0000 7f1e53fff640 1 -- 172.21.15.60:0/1240451740 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44007cb0 con 0x7f1e540b4e60 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/1240451740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 msgr2=0x7f1e540b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/1240451740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540b5280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009d90 tx=0x7f1e4402a660 comp rx=0 tx=0).stop 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/1240451740 shutdown_connections 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/1240451740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540b5280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:21.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/1240451740 >> v2:172.21.15.100:3300/0 conn(0x7f1e54158230 0x7f1e540d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:21.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/1240451740 >> 172.21.15.60:0/1240451740 conn(0x7f1e54061600 msgr2=0x7f1e540619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:21.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/1240451740 shutdown_connections 2024-04-25T08:30:21.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.070+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/1240451740 wait complete. 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.071+0000 7f1e5b5b6640 1 Processor -- start 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.071+0000 7f1e5b5b6640 1 -- start start 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e5b5b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e5b5b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e540d1b80 0x7f1e5415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e5b5b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 0x7f1e54149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e5b5b6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540d25e0 con 0x7f1e54158230 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e5b5b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5406bea0 con 0x7f1e540d1b80 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e5b5b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5406ca40 con 0x7f1e540b4e60 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 0x7f1e54149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:21.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 0x7f1e54149390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45928/0 (socket says 172.21.15.60:45928) 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 -- 172.21.15.60:0/721575457 learned_addr learned my addr 172.21.15.60:0/721575457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59352640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e58b51640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e540d1b80 0x7f1e5415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 -- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 msgr2=0x7f1e540cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 -- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e540d1b80 msgr2=0x7f1e5415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e540d1b80 0x7f1e5415af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59b53640 1 -- 172.21.15.60:0/721575457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e541498d0 con 0x7f1e54158230 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.072+0000 7f1e59352640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.073+0000 7f1e59b53640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 0x7f1e54149390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402ac50 tx=0x7f1e44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.073+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44040070 con 0x7f1e54158230 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.073+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44031c60 con 0x7f1e54158230 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.073+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5414b980 con 0x7f1e54158230 2024-04-25T08:30:21.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.073+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44009450 con 0x7f1e54158230 2024-04-25T08:30:21.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.073+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5414bf20 con 0x7f1e54158230 2024-04-25T08:30:21.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.074+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5406de50 con 0x7f1e54158230 2024-04-25T08:30:21.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.074+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e44039040 con 0x7f1e54158230 2024-04-25T08:30:21.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.075+0000 7f1e51ffb640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e4006b4d0 0x7f1e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:21.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.075+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f1e440aba10 con 0x7f1e54158230 2024-04-25T08:30:21.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.075+0000 7f1e59352640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e4006b4d0 0x7f1e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:21.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.076+0000 7f1e59352640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e4006b4d0 0x7f1e4006d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009b80 tx=0x7f1e4c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.079+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e541498d0 con 0x7f1e54158230 2024-04-25T08:30:21.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:21.263+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e54073a00 con 0x7f1e54158230 2024-04-25T08:30:22.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.243+0000 7f1e51ffb640 1 -- 172.21.15.60:0/721575457 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1e4407a4c0 con 0x7f1e54158230 2024-04-25T08:30:22.240 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e4006b4d0 msgr2=0x7f1e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e4006b4d0 0x7f1e4006d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c009b80 tx=0x7f1e4c008040 comp rx=0 tx=0).stop 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 msgr2=0x7f1e54149390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 0x7f1e54149390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402ac50 tx=0x7f1e44004a40 comp rx=0 tx=0).stop 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 shutdown_connections 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e54158230 0x7f1e54149390 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e4006b4d0 0x7f1e4006d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e540d1b80 0x7f1e5415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 --2- 172.21.15.60:0/721575457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e540b4e60 0x7f1e540cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 >> 172.21.15.60:0/721575457 conn(0x7f1e54061600 msgr2=0x7f1e540b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 shutdown_connections 2024-04-25T08:30:22.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.247+0000 7f1e5b5b6640 1 -- 172.21.15.60:0/721575457 wait complete. 2024-04-25T08:30:22.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:30:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.371+0000 7fc1f6c9a640 1 Processor -- start 2024-04-25T08:30:22.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.371+0000 7fc1f6c9a640 1 -- start start 2024-04-25T08:30:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f6c9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc1f015dc20 0x7fc1f00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f6c9a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1f00d32f0 0x7fc1f00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:22.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f6c9a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f015e000 con 0x7fc1f0161710 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f6c9a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00d1de0 con 0x7fc1f00d32f0 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f6c9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f0161c60 con 0x7fc1f015dc20 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f5237640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1f00d32f0 0x7fc1f00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.372+0000 7fc1f5237640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc1f00d32f0 0x7fc1f00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47560/0 (socket says 172.21.15.60:47560) 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.373+0000 7fc1f5237640 1 -- 172.21.15.60:0/3653944948 learned_addr learned my addr 172.21.15.60:0/3653944948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.373+0000 7fc1f4a36640 1 --2- 172.21.15.60:0/3653944948 >> v2:172.21.15.100:3300/0 conn(0x7fc1f015dc20 0x7fc1f00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.373+0000 7fc1f5237640 1 -- 172.21.15.60:0/3653944948 >> v2:172.21.15.100:3300/0 conn(0x7fc1f015dc20 msgr2=0x7fc1f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.373+0000 7fc1f5237640 1 --2- 172.21.15.60:0/3653944948 >> v2:172.21.15.100:3300/0 conn(0x7fc1f015dc20 0x7fc1f00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.373+0000 7fc1f5237640 1 -- 172.21.15.60:0/3653944948 >> v1:172.21.15.100:6789/0 conn(0x7fc1f0161710 legacy=0x7fc1f006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.373+0000 7fc1f5237640 1 -- 172.21.15.60:0/3653944948 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0161f40 con 0x7fc1f00d32f0 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f5237640 1 --2- 172.21.15.60:0/3653944948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f00cebb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00d1520 tx=0x7fc1e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6457bf3b683a2739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:22.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1ef7fe640 1 -- 172.21.15.60:0/3653944948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0037070 con 0x7fc1f00d32f0 2024-04-25T08:30:22.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1ef7fe640 1 -- 172.21.15.60:0/3653944948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e0032440 con 0x7fc1f00d32f0 2024-04-25T08:30:22.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1ef7fe640 1 -- 172.21.15.60:0/3653944948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0032810 con 0x7fc1f00d32f0 2024-04-25T08:30:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3653944948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 msgr2=0x7fc1f00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3653944948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f00cebb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00d1520 tx=0x7fc1e002a270 comp rx=0 tx=0).stop 2024-04-25T08:30:22.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3653944948 shutdown_connections 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3653944948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f00cebb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3653944948 >> v2:172.21.15.100:3300/0 conn(0x7fc1f015dc20 0x7fc1f00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3653944948 >> 172.21.15.60:0/3653944948 conn(0x7fc1f00d0420 msgr2=0x7fc1f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3653944948 shutdown_connections 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.374+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3653944948 wait complete. 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 Processor -- start 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 -- start start 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f015b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1f015dc20 0x7fc1f00cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1f0161710 0x7fc1f00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00b5e50 con 0x7fc1f00d32f0 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0162500 con 0x7fc1f0161710 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.376+0000 7fc1f6c9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00b5fd0 con 0x7fc1f015dc20 2024-04-25T08:30:22.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f015b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f015b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47564/0 (socket says 172.21.15.60:47564) 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 -- 172.21.15.60:0/3016775136 learned_addr learned my addr 172.21.15.60:0/3016775136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f5237640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1f0161710 0x7fc1f00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 -- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1f015dc20 msgr2=0x7fc1f00cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1effff640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1f015dc20 0x7fc1f00cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1f015dc20 0x7fc1f00cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 -- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1f0161710 msgr2=0x7fc1f00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1f0161710 0x7fc1f00b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 -- 172.21.15.60:0/3016775136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00cd2e0 con 0x7fc1f00d32f0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f5237640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1f0161710 0x7fc1f00b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1effff640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1f015dc20 0x7fc1f00cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f4a36640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f015b1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8007e60 tx=0x7fc1e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8014620 con 0x7fc1f00d32f0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8019070 con 0x7fc1f00d32f0 2024-04-25T08:30:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f00cd5c0 con 0x7fc1f00d32f0 2024-04-25T08:30:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8014a20 con 0x7fc1f00d32f0 2024-04-25T08:30:22.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.377+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f00cdb80 con 0x7fc1f00d32f0 2024-04-25T08:30:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.378+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f0073a00 con 0x7fc1f00d32f0 2024-04-25T08:30:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.379+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1e8014bc0 con 0x7fc1f00d32f0 2024-04-25T08:30:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.379+0000 7fc1ed7fa640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc1dc06b4d0 0x7fc1dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.379+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6383+0+0 (secure 0 0 0) 0x7fc1e808ad60 con 0x7fc1f00d32f0 2024-04-25T08:30:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.379+0000 7fc1effff640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc1dc06b4d0 0x7fc1dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:22.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.380+0000 7fc1effff640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc1dc06b4d0 0x7fc1dc06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1f015b960 tx=0x7fc1e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:22.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.384+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1e8059940 con 0x7fc1f00d32f0 2024-04-25T08:30:22.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:22.573+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1f00d16b0 con 0x7fc1f00d32f0 2024-04-25T08:30:23.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.244+0000 7fc1ed7fa640 1 -- 172.21.15.60:0/3016775136 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc1e8059240 con 0x7fc1f00d32f0 2024-04-25T08:30:23.242 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:30:23.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc1dc06b4d0 msgr2=0x7fc1dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:23.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc1dc06b4d0 0x7fc1dc06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc1f015b960 tx=0x7fc1e4009040 comp rx=0 tx=0).stop 2024-04-25T08:30:23.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 msgr2=0x7fc1f015b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:23.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f015b1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8007e60 tx=0x7fc1e800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 shutdown_connections 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc1dc06b4d0 0x7fc1dc06d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1f0161710 0x7fc1f00b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc1f015dc20 0x7fc1f00cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 --2- 172.21.15.60:0/3016775136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1f00d32f0 0x7fc1f015b1d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 >> 172.21.15.60:0/3016775136 conn(0x7fc1f00d0420 msgr2=0x7fc1f015c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 shutdown_connections 2024-04-25T08:30:23.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.249+0000 7fc1f6c9a640 1 -- 172.21.15.60:0/3016775136 wait complete. 2024-04-25T08:30:23.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T08:30:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 Processor -- start 2024-04-25T08:30:23.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 -- start start 2024-04-25T08:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85d4161890 0x7f85d40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d40b54a0 0x7f85d40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d40cf1d0 con 0x7f85d4161240 2024-04-25T08:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d40b7120 con 0x7f85d4161890 2024-04-25T08:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85db169640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4161620 con 0x7f85d40b54a0 2024-04-25T08:30:23.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.378+0000 7f85d3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85d4161890 0x7f85d40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85d4161890 0x7f85d40b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47574/0 (socket says 172.21.15.60:47574) 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d3fff640 1 -- 172.21.15.60:0/987894358 learned_addr learned my addr 172.21.15.60:0/987894358 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d9706640 1 --2- 172.21.15.60:0/987894358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d40b54a0 0x7f85d40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d3fff640 1 -- 172.21.15.60:0/987894358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d40b54a0 msgr2=0x7f85d40b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d3fff640 1 --2- 172.21.15.60:0/987894358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d40b54a0 0x7f85d40b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d3fff640 1 -- 172.21.15.60:0/987894358 >> v1:172.21.15.100:6789/0 conn(0x7f85d4161240 legacy=0x7f85d40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:23.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.379+0000 7f85d3fff640 1 -- 172.21.15.60:0/987894358 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4161e00 con 0x7f85d4161890 2024-04-25T08:30:23.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85d3fff640 1 --2- 172.21.15.60:0/987894358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d4161890 0x7f85d40b6be0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009870 tx=0x7f85c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa440902084a631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:23.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85d37fe640 1 -- 172.21.15.60:0/987894358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8037070 con 0x7f85d4161890 2024-04-25T08:30:23.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85d37fe640 1 -- 172.21.15.60:0/987894358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c8032440 con 0x7f85d4161890 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85d37fe640 1 -- 172.21.15.60:0/987894358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8032810 con 0x7f85d4161890 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 -- 172.21.15.60:0/987894358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d4161890 msgr2=0x7f85d40b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 --2- 172.21.15.60:0/987894358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d4161890 0x7f85d40b6be0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009870 tx=0x7f85c802a270 comp rx=0 tx=0).stop 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 -- 172.21.15.60:0/987894358 shutdown_connections 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 --2- 172.21.15.60:0/987894358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d4161890 0x7f85d40b6be0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 --2- 172.21.15.60:0/987894358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d40b54a0 0x7f85d40b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 -- 172.21.15.60:0/987894358 >> 172.21.15.60:0/987894358 conn(0x7f85d40d04a0 msgr2=0x7f85d40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 -- 172.21.15.60:0/987894358 shutdown_connections 2024-04-25T08:30:23.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.380+0000 7f85db169640 1 -- 172.21.15.60:0/987894358 wait complete. 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 Processor -- start 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 -- start start 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 0x7f85d40d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d4161240 0x7f85d40c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d4161890 0x7f85d40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d41626a0 con 0x7f85d40b54a0 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d41623c0 con 0x7f85d4161240 2024-04-25T08:30:23.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.382+0000 7f85db169640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d40d15f0 con 0x7f85d4161890 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 0x7f85d40d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 0x7f85d40d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47590/0 (socket says 172.21.15.60:47590) 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 -- 172.21.15.60:0/868505825 learned_addr learned my addr 172.21.15.60:0/868505825 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d3fff640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d4161240 0x7f85d40c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d9706640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d4161890 0x7f85d40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 -- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d4161890 msgr2=0x7f85d40b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d4161890 0x7f85d40b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 -- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d4161240 msgr2=0x7f85d40c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d4161240 0x7f85d40c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 -- 172.21.15.60:0/868505825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d40d1ff0 con 0x7f85d40b54a0 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d8f05640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 0x7f85d40d1a10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c790 tx=0x7f85cc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc016020 con 0x7f85d40b54a0 2024-04-25T08:30:23.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc006d50 con 0x7f85d40b54a0 2024-04-25T08:30:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d40d22d0 con 0x7f85d40b54a0 2024-04-25T08:30:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc003bb0 con 0x7f85d40b54a0 2024-04-25T08:30:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.383+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d40d2890 con 0x7f85d40b54a0 2024-04-25T08:30:23.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.385+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d4073a00 con 0x7f85d40b54a0 2024-04-25T08:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.385+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85cc00a850 con 0x7f85d40b54a0 2024-04-25T08:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.385+0000 7f85d17fa640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85c006b500 0x7f85c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.385+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f85cc089e10 con 0x7f85d40b54a0 2024-04-25T08:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.386+0000 7f85d3fff640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85c006b500 0x7f85c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:23.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.386+0000 7f85d3fff640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85c006b500 0x7f85c006d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f85c8034000 tx=0x7f85c802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:23.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.390+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85cc020020 con 0x7f85d40b54a0 2024-04-25T08:30:23.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:23.569+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85d406d240 con 0x7f85d40b54a0 2024-04-25T08:30:24.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.258+0000 7f85d17fa640 1 -- 172.21.15.60:0/868505825 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f85cc0585b0 con 0x7f85d40b54a0 2024-04-25T08:30:24.256 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data_ec' removed 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85c006b500 msgr2=0x7f85c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85c006b500 0x7f85c006d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f85c8034000 tx=0x7f85c802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 msgr2=0x7f85d40d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 0x7f85d40d1a10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c790 tx=0x7f85cc00cc50 comp rx=0 tx=0).stop 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 shutdown_connections 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85d4161890 0x7f85d40b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85c006b500 0x7f85c006d9c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85d4161240 0x7f85d40c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 --2- 172.21.15.60:0/868505825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85d40b54a0 0x7f85d40d1a10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 >> 172.21.15.60:0/868505825 conn(0x7f85d40d04a0 msgr2=0x7f85d40b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 shutdown_connections 2024-04-25T08:30:24.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.262+0000 7f85db169640 1 -- 172.21.15.60:0/868505825 wait complete. 2024-04-25T08:30:24.274 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:30:24.274 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:30:24.274 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:30:24.274 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:30:24.275 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:30:24.275 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:30:24.275 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:30:24.275 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:30:24.275 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:30:24.275 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:30:24.276 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:30:24.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:30:24.276 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:30:24.276 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:30:24.276 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:30:24.276 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:30:24.285 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:30:24.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:30:24.286 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:30:24.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:30:24.288 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:30:24.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:30:24.288 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:30:24.289 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:30:24.329 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:30:24.329 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:24.330 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:30:24.330 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:24.330 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:30:24.330 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:24.331 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:30:24.331 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:24.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:30:24.407 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:30:24.407 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:30:24.410 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:30:24.411 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:30:24.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.445+0000 7f839b722640 1 Processor -- start 2024-04-25T08:30:24.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.445+0000 7f839b722640 1 -- start start 2024-04-25T08:30:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f839b722640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f839415a9f0 0x7f83940c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:24.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f839b722640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cdbf0 0x7f83940ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f839b722640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83940c6940 con 0x7f83940cd520 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f839b722640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83940b0020 con 0x7f839415a9f0 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f839b722640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83940cb5a0 con 0x7f83940cdbf0 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f83994be640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f839415a9f0 0x7f83940c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f83994be640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f839415a9f0 0x7f83940c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47626/0 (socket says 172.21.15.60:47626) 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f83994be640 1 -- 172.21.15.60:0/3681392092 learned_addr learned my addr 172.21.15.60:0/3681392092 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.446+0000 7f8398cbd640 1 --2- 172.21.15.60:0/3681392092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cdbf0 0x7f83940ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.447+0000 7f83994be640 1 -- 172.21.15.60:0/3681392092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cdbf0 msgr2=0x7f83940ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.447+0000 7f83994be640 1 --2- 172.21.15.60:0/3681392092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cdbf0 0x7f83940ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.447+0000 7f83994be640 1 -- 172.21.15.60:0/3681392092 >> v1:172.21.15.100:6789/0 conn(0x7f83940cd520 legacy=0x7f83940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.447+0000 7f838bfff640 1 -- 172.21.15.60:0/3681392092 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956458509 0 0) 0x7f838c001490 con 0x7f83940cd520 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.447+0000 7f83994be640 1 -- 172.21.15.60:0/3681392092 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839415add0 con 0x7f839415a9f0 2024-04-25T08:30:24.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.447+0000 7f83994be640 1 --2- 172.21.15.60:0/3681392092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940c6010 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8384009870 tx=0x7f838402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fb4ec7a9f53357 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f838bfff640 1 -- 172.21.15.60:0/3681392092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384037070 con 0x7f839415a9f0 2024-04-25T08:30:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f838bfff640 1 -- 172.21.15.60:0/3681392092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384032440 con 0x7f839415a9f0 2024-04-25T08:30:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f838bfff640 1 -- 172.21.15.60:0/3681392092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384032810 con 0x7f839415a9f0 2024-04-25T08:30:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 -- 172.21.15.60:0/3681392092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 msgr2=0x7f83940c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:24.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 --2- 172.21.15.60:0/3681392092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940c6010 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8384009870 tx=0x7f838402a270 comp rx=0 tx=0).stop 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 -- 172.21.15.60:0/3681392092 shutdown_connections 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 --2- 172.21.15.60:0/3681392092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cdbf0 0x7f83940ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 --2- 172.21.15.60:0/3681392092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940c6010 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 -- 172.21.15.60:0/3681392092 >> 172.21.15.60:0/3681392092 conn(0x7f83940b2390 msgr2=0x7f83940b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 -- 172.21.15.60:0/3681392092 shutdown_connections 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.448+0000 7f839b722640 1 -- 172.21.15.60:0/3681392092 wait complete. 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.449+0000 7f839b722640 1 Processor -- start 2024-04-25T08:30:24.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 -- start start 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cd520 0x7f83941527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83940cdbf0 0x7f83940ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940cb230 con 0x7f839415a9f0 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940caf50 con 0x7f83940cdbf0 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f839b722640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83940ce070 con 0x7f83940cd520 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f8399cbf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f8399cbf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47632/0 (socket says 172.21.15.60:47632) 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f8399cbf640 1 -- 172.21.15.60:0/3248434411 learned_addr learned my addr 172.21.15.60:0/3248434411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f8398cbd640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83940cdbf0 0x7f83940ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.450+0000 7f83994be640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cd520 0x7f83941527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8399cbf640 1 -- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cd520 msgr2=0x7f83941527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:24.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8399cbf640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cd520 0x7f83941527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8399cbf640 1 -- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83940cdbf0 msgr2=0x7f83940ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8399cbf640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83940cdbf0 0x7f83940ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8399cbf640 1 -- 172.21.15.60:0/3248434411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83940b5740 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8399cbf640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940b4cf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f838c001730 tx=0x7f838c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c017070 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c003dc0 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c0052b0 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83940ce950 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.451+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83940cef10 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.453+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839406cc80 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.453+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f838c01a070 con 0x7f839415a9f0 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.453+0000 7f8389ffb640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:24.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.453+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f838c089690 con 0x7f839415a9f0 2024-04-25T08:30:24.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.453+0000 7f83994be640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:24.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.454+0000 7f83994be640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8384034000 tx=0x7f838402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:24.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.458+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83940c6940 con 0x7f839415a9f0 2024-04-25T08:30:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:24.649+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8394073a00 con 0x7f839415a9f0 2024-04-25T08:30:25.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.267+0000 7f8389ffb640 1 -- 172.21.15.60:0/3248434411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f838c0580c0 con 0x7f839415a9f0 2024-04-25T08:30:25.264 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f837806b4d0 msgr2=0x7f837806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8384034000 tx=0x7f838402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 msgr2=0x7f83940b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940b4cf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f838c001730 tx=0x7f838c00cc70 comp rx=0 tx=0).stop 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 shutdown_connections 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f839415a9f0 0x7f83940b4cf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83940cdbf0 0x7f83940ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 --2- 172.21.15.60:0/3248434411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83940cd520 0x7f83941527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 >> 172.21.15.60:0/3248434411 conn(0x7f83940b2390 msgr2=0x7f839406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 shutdown_connections 2024-04-25T08:30:25.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.270+0000 7f839b722640 1 -- 172.21.15.60:0/3248434411 wait complete. 2024-04-25T08:30:25.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:30:25.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.392+0000 7effcd741640 1 Processor -- start 2024-04-25T08:30:25.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.392+0000 7effcd741640 1 -- start start 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effcd741640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effc815dc20 0x7effc81616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effcd741640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc80b93a0 0x7effc80ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effcd741640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7effc80cec20 0x7effc806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effcd741640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc815e000 con 0x7effc815dc20 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effcd741640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80b6bb0 con 0x7effc80cec20 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effcd741640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc80b6030 con 0x7effc80b93a0 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effc6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effc815dc20 0x7effc81616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effc6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7effc815dc20 0x7effc81616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47648/0 (socket says 172.21.15.60:47648) 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effc6ffd640 1 -- 172.21.15.60:0/2209049904 learned_addr learned my addr 172.21.15.60:0/2209049904 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:25.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effc67fc640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc80b93a0 0x7effc80ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.393+0000 7effc77fe640 1 --2- 172.21.15.60:0/2209049904 >> v2:172.21.15.100:3300/0 conn(0x7effc80cec20 0x7effc806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc6ffd640 1 -- 172.21.15.60:0/2209049904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc80b93a0 msgr2=0x7effc80ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc6ffd640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc80b93a0 0x7effc80ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc6ffd640 1 -- 172.21.15.60:0/2209049904 >> v2:172.21.15.100:3300/0 conn(0x7effc80cec20 msgr2=0x7effc806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc6ffd640 1 --2- 172.21.15.60:0/2209049904 >> v2:172.21.15.100:3300/0 conn(0x7effc80cec20 0x7effc806c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc6ffd640 1 -- 172.21.15.60:0/2209049904 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8161de0 con 0x7effc815dc20 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc77fe640 1 --2- 172.21.15.60:0/2209049904 >> v2:172.21.15.100:3300/0 conn(0x7effc80cec20 0x7effc806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc67fc640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc80b93a0 0x7effc80ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc6ffd640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc815dc20 0x7effc81616b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effc80d1ff0 tx=0x7effc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751fc8ac1ee1c61a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc5ffb640 1 -- 172.21.15.60:0/2209049904 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc0037070 con 0x7effc815dc20 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc5ffb640 1 -- 172.21.15.60:0/2209049904 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc0032440 con 0x7effc815dc20 2024-04-25T08:30:25.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.394+0000 7effc5ffb640 1 -- 172.21.15.60:0/2209049904 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc0032810 con 0x7effc815dc20 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 -- 172.21.15.60:0/2209049904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc815dc20 msgr2=0x7effc81616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc815dc20 0x7effc81616b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effc80d1ff0 tx=0x7effc002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 -- 172.21.15.60:0/2209049904 shutdown_connections 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 --2- 172.21.15.60:0/2209049904 >> v2:172.21.15.100:3300/0 conn(0x7effc80cec20 0x7effc806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc80b93a0 0x7effc80ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 --2- 172.21.15.60:0/2209049904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc815dc20 0x7effc81616b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 -- 172.21.15.60:0/2209049904 >> 172.21.15.60:0/2209049904 conn(0x7effc80d0490 msgr2=0x7effc80d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:25.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 -- 172.21.15.60:0/2209049904 shutdown_connections 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.395+0000 7effcd741640 1 -- 172.21.15.60:0/2209049904 wait complete. 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.396+0000 7effcd741640 1 Processor -- start 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.396+0000 7effcd741640 1 -- start start 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effcd741640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80b93a0 0x7effc81f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effcd741640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 0x7effc81f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effcd741640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc815dc20 0x7effc80c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effcd741640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc806c610 con 0x7effc80cec20 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effcd741640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc81623a0 con 0x7effc80b93a0 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effcd741640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc806c790 con 0x7effc815dc20 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effc67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 0x7effc81f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effc67fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 0x7effc81f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47660/0 (socket says 172.21.15.60:47660) 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effc67fc640 1 -- 172.21.15.60:0/359896710 learned_addr learned my addr 172.21.15.60:0/359896710 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effc77fe640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc815dc20 0x7effc80c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.397+0000 7effc6ffd640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80b93a0 0x7effc81f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effc67fc640 1 -- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc815dc20 msgr2=0x7effc80c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effc67fc640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc815dc20 0x7effc80c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effc67fc640 1 -- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80b93a0 msgr2=0x7effc81f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effc67fc640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80b93a0 0x7effc81f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effc67fc640 1 -- 172.21.15.60:0/359896710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc81f3c10 con 0x7effc80cec20 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effc67fc640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 0x7effc81f75c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effb000e810 tx=0x7effb000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000f040 con 0x7effc80cec20 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb0004030 con 0x7effc80cec20 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc81f3ef0 con 0x7effc80cec20 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0004400 con 0x7effc80cec20 2024-04-25T08:30:25.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.398+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc81f44b0 con 0x7effc80cec20 2024-04-25T08:30:25.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.399+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc8073990 con 0x7effc80cec20 2024-04-25T08:30:25.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.399+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effb000c6d0 con 0x7effc80cec20 2024-04-25T08:30:25.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.400+0000 7effbf7fe640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7effa806b4d0 0x7effa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.400+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7effb0089bc0 con 0x7effc80cec20 2024-04-25T08:30:25.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.400+0000 7effc6ffd640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7effa806b4d0 0x7effa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.400+0000 7effc6ffd640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7effa806b4d0 0x7effa806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7effc0034000 tx=0x7effc002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:25.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.404+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7effb0017020 con 0x7effc80cec20 2024-04-25T08:30:25.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:25.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:25.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.654+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7effc80d1720 con 0x7effc80cec20 2024-04-25T08:30:25.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.655+0000 7effbf7fe640 1 -- 172.21.15.60:0/359896710 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7effb0058640 con 0x7effc80cec20 2024-04-25T08:30:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:25.652 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:30:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:30:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:30:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:30:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:30:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:30:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:30:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:30:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mds", 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBLFCpmVpFUEBAAC/VX4/1Zkid9VItRZ7l9xQ==", 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBLFCpm7ppUEBAA7Vr7RpPNKfhz8BCffJZ3qA==", 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-osd", 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBLFCpmP6JUEBAA6CXdiYMPD36SK5wuk/xJPA==", 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBLFCpmRqlUEBAAs2bEwj1mw0WilQPpYtn2eQ==", 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBLFCpmebBUEBAAIyvrNZP3iWp4eK9YuGqfHQ==", 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBLFCpmcrlUEBAA4KmFuJS2RgWZzjgChGSq2A==", 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:30:25.667 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7effa806b4d0 msgr2=0x7effa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7effa806b4d0 0x7effa806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7effc0034000 tx=0x7effc002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 msgr2=0x7effc81f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 0x7effc81f75c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effb000e810 tx=0x7effb000ecd0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 shutdown_connections 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7effc815dc20 0x7effc80c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7effc80cec20 0x7effc81f75c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7effa806b4d0 0x7effa806d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 --2- 172.21.15.60:0/359896710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7effc80b93a0 0x7effc81f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 >> 172.21.15.60:0/359896710 conn(0x7effc80d0490 msgr2=0x7effc80b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 shutdown_connections 2024-04-25T08:30:25.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.657+0000 7effcd741640 1 -- 172.21.15.60:0/359896710 wait complete. 2024-04-25T08:30:25.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T08:30:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.775+0000 7fbddd6bd640 1 Processor -- start 2024-04-25T08:30:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.775+0000 7fbddd6bd640 1 -- start start 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.775+0000 7fbddd6bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbdd80714d0 0x7fbdd8166980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.775+0000 7fbddd6bd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd80ceba0 con 0x7fbdd8144d20 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.775+0000 7fbddd6bd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd80cee80 con 0x7fbdd81454b0 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.775+0000 7fbddd6bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80b4830 con 0x7fbdd80714d0 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbdd8144d20 0x7fbdd80ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52748/0 (socket says 172.21.15.60:52748) 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd6ffd640 1 -- 172.21.15.60:0/539773819 learned_addr learned my addr 172.21.15.60:0/539773819 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd77fe640 1 --2- 172.21.15.60:0/539773819 >> v2:172.21.15.100:3300/0 conn(0x7fbdd80714d0 0x7fbdd8166980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029529640 0 0) 0x7fbdd80ceba0 con 0x7fbdd8144d20 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc4003520 con 0x7fbdd8144d20 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434216773 0 0) 0x7fbdd80cee80 con 0x7fbdd81454b0 2024-04-25T08:30:25.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd80ceba0 con 0x7fbdd81454b0 2024-04-25T08:30:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd77fe640 1 -- 172.21.15.60:0/539773819 >> v1:172.21.15.100:6789/0 conn(0x7fbdd81454b0 legacy=0x7fbdd8070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd77fe640 1 -- 172.21.15.60:0/539773819 >> v1:172.21.15.60:6789/0 conn(0x7fbdd8144d20 legacy=0x7fbdd80ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.776+0000 7fbdd77fe640 1 -- 172.21.15.60:0/539773819 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80cf160 con 0x7fbdd80714d0 2024-04-25T08:30:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.777+0000 7fbdd77fe640 1 --2- 172.21.15.60:0/539773819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd80714d0 0x7fbdd8166980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b49b0 tx=0x7fbdc802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4e4089f7a7a46e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.777+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8037070 con 0x7fbdd80714d0 2024-04-25T08:30:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.777+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc8032440 con 0x7fbdd80714d0 2024-04-25T08:30:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.777+0000 7fbdd5ffb640 1 -- 172.21.15.60:0/539773819 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80327f0 con 0x7fbdd80714d0 2024-04-25T08:30:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 -- 172.21.15.60:0/539773819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd80714d0 msgr2=0x7fbdd8166980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/539773819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd80714d0 0x7fbdd8166980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b49b0 tx=0x7fbdc802a210 comp rx=0 tx=0).stop 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 -- 172.21.15.60:0/539773819 shutdown_connections 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/539773819 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd80714d0 0x7fbdd8166980 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 -- 172.21.15.60:0/539773819 >> 172.21.15.60:0/539773819 conn(0x7fbdd80b3530 msgr2=0x7fbdd80b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 -- 172.21.15.60:0/539773819 shutdown_connections 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.778+0000 7fbddd6bd640 1 -- 172.21.15.60:0/539773819 wait complete. 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.779+0000 7fbddd6bd640 1 Processor -- start 2024-04-25T08:30:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.779+0000 7fbddd6bd640 1 -- start start 2024-04-25T08:30:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbddd6bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 0x7fbdd81e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbddd6bd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdd8144d20 0x7fbdd81d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbddd6bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd81454b0 0x7fbdd81e5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbddd6bd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8145280 con 0x7fbdd80714d0 2024-04-25T08:30:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbddd6bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8145100 con 0x7fbdd81454b0 2024-04-25T08:30:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbddd6bd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b5cb0 con 0x7fbdd8144d20 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 0x7fbdd81e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 0x7fbdd81e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47662/0 (socket says 172.21.15.60:47662) 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 -- 172.21.15.60:0/3521684079 learned_addr learned my addr 172.21.15.60:0/3521684079 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd77fe640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd81454b0 0x7fbdd81e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd67fc640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdd8144d20 0x7fbdd81d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 -- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdd8144d20 msgr2=0x7fbdd81d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdd8144d20 0x7fbdd81d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 -- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd81454b0 msgr2=0x7fbdd81e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd81454b0 0x7fbdd81e5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 -- 172.21.15.60:0/3521684079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd81e0180 con 0x7fbdd80714d0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdd6ffd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 0x7fbdd81e7370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000cbc0 tx=0x7fbdd000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.780+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0017070 con 0x7fbdd80714d0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.781+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd000ad40 con 0x7fbdd80714d0 2024-04-25T08:30:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.781+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd81e0460 con 0x7fbdd80714d0 2024-04-25T08:30:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.781+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0012600 con 0x7fbdd80714d0 2024-04-25T08:30:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.781+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd81e0a20 con 0x7fbdd80714d0 2024-04-25T08:30:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.782+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8073a00 con 0x7fbdd80714d0 2024-04-25T08:30:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.782+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdd001a070 con 0x7fbdd80714d0 2024-04-25T08:30:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.783+0000 7fbdcf7fe640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbdb806b4d0 0x7fbdb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.783+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbdd0089dd0 con 0x7fbdd80714d0 2024-04-25T08:30:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.783+0000 7fbdd67fc640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbdb806b4d0 0x7fbdb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.783+0000 7fbdd67fc640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbdb806b4d0 0x7fbdb806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0007fd0 tx=0x7fbdc0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:25.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:25.787+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdd00587c0 con 0x7fbdd80714d0 2024-04-25T08:30:26.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.042+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fbdd80b0f50 con 0x7fbdd80714d0 2024-04-25T08:30:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.044+0000 7fbdcf7fe640 1 -- 172.21.15.60:0/3521684079 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbdd008b0a0 con 0x7fbdd80714d0 2024-04-25T08:30:26.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.046+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbdb806b4d0 msgr2=0x7fbdb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.046+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbdb806b4d0 0x7fbdb806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0007fd0 tx=0x7fbdc0009040 comp rx=0 tx=0).stop 2024-04-25T08:30:26.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 msgr2=0x7fbdd81e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 0x7fbdd81e7370 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000cbc0 tx=0x7fbdd000a430 comp rx=0 tx=0).stop 2024-04-25T08:30:26.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 shutdown_connections 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbdb806b4d0 0x7fbdb806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbdd81454b0 0x7fbdd81e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbdd8144d20 0x7fbdd81d8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 --2- 172.21.15.60:0/3521684079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbdd80714d0 0x7fbdd81e7370 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 >> 172.21.15.60:0/3521684079 conn(0x7fbdd80b3530 msgr2=0x7fbdd81d9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 shutdown_connections 2024-04-25T08:30:26.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.047+0000 7fbddd6bd640 1 -- 172.21.15.60:0/3521684079 wait complete. 2024-04-25T08:30:26.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T08:30:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 Processor -- start 2024-04-25T08:30:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 -- start start 2024-04-25T08:30:26.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41980b5620 0x7f41980b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f419806c2d0 0x7f419806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419806cc80 con 0x7f419815b7d0 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419806cf50 con 0x7f419806c2d0 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.159+0000 7f419edc8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41980ce530 con 0x7f41980b5620 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419d365640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f419806c2d0 0x7f419806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419cb64640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f419815b7d0 0x7f41980cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52756/0 (socket says 172.21.15.60:52756) 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419d365640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f419806c2d0 0x7f419806c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47666/0 (socket says 172.21.15.60:47666) 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419cb64640 1 -- 172.21.15.60:0/4293230066 learned_addr learned my addr 172.21.15.60:0/4293230066 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f4197fff640 1 --2- 172.21.15.60:0/4293230066 >> v2:172.21.15.100:3300/0 conn(0x7f41980b5620 0x7f41980b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419d365640 1 -- 172.21.15.60:0/4293230066 >> v2:172.21.15.100:3300/0 conn(0x7f41980b5620 msgr2=0x7f41980b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419d365640 1 --2- 172.21.15.60:0/4293230066 >> v2:172.21.15.100:3300/0 conn(0x7f41980b5620 0x7f41980b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419d365640 1 -- 172.21.15.60:0/4293230066 >> v1:172.21.15.60:6789/0 conn(0x7f419815b7d0 legacy=0x7f41980cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f41977fe640 1 -- 172.21.15.60:0/4293230066 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615595186 0 0) 0x7f419806cc80 con 0x7f419815b7d0 2024-04-25T08:30:26.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.160+0000 7f419d365640 1 -- 172.21.15.60:0/4293230066 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980ce810 con 0x7f419806c2d0 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419d365640 1 --2- 172.21.15.60:0/4293230066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f419806c2d0 0x7f419806c740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41980ce180 tx=0x7f418802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600d0b2ef07fb1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f41977fe640 1 -- 172.21.15.60:0/4293230066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188037070 con 0x7f419806c2d0 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f41977fe640 1 -- 172.21.15.60:0/4293230066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4188032440 con 0x7f419806c2d0 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f41977fe640 1 -- 172.21.15.60:0/4293230066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188032810 con 0x7f419806c2d0 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 -- 172.21.15.60:0/4293230066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f419806c2d0 msgr2=0x7f419806c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 --2- 172.21.15.60:0/4293230066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f419806c2d0 0x7f419806c740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41980ce180 tx=0x7f418802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 -- 172.21.15.60:0/4293230066 shutdown_connections 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 --2- 172.21.15.60:0/4293230066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f419806c2d0 0x7f419806c740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 --2- 172.21.15.60:0/4293230066 >> v2:172.21.15.100:3300/0 conn(0x7f41980b5620 0x7f41980b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 -- 172.21.15.60:0/4293230066 >> 172.21.15.60:0/4293230066 conn(0x7f41980d0800 msgr2=0x7f41980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 -- 172.21.15.60:0/4293230066 shutdown_connections 2024-04-25T08:30:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.161+0000 7f419edc8640 1 -- 172.21.15.60:0/4293230066 wait complete. 2024-04-25T08:30:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.162+0000 7f419edc8640 1 Processor -- start 2024-04-25T08:30:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 -- start start 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f419806c2d0 0x7f4198157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 0x7f41980c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f419815b7d0 0x7f41980c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419815bbb0 con 0x7f41980b5620 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419806d230 con 0x7f419815b7d0 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.163+0000 7f419edc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419815bd30 con 0x7f419806c2d0 2024-04-25T08:30:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 0x7f41980c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 0x7f41980c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47674/0 (socket says 172.21.15.60:47674) 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 -- 172.21.15.60:0/4221255404 learned_addr learned my addr 172.21.15.60:0/4221255404 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f419d365640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f419815b7d0 0x7f41980c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f419cb64640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f419806c2d0 0x7f4198157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 -- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f419806c2d0 msgr2=0x7f4198157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f419806c2d0 0x7f4198157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 -- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f419815b7d0 msgr2=0x7f41980c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f419815b7d0 0x7f41980c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 -- 172.21.15.60:0/4221255404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198159830 con 0x7f41980b5620 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f419d365640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f419815b7d0 0x7f41980c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f419cb64640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f419806c2d0 0x7f4198157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f4197fff640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 0x7f41980c7f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f418000e9e0 tx=0x7f418000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180017070 con 0x7f41980b5620 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418000c9d0 con 0x7f41980b5620 2024-04-25T08:30:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4198158f90 con 0x7f41980b5620 2024-04-25T08:30:26.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418000cdc0 con 0x7f41980b5620 2024-04-25T08:30:26.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.164+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4198159550 con 0x7f41980b5620 2024-04-25T08:30:26.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.165+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4198073a00 con 0x7f41980b5620 2024-04-25T08:30:26.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.166+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f418001a070 con 0x7f41980b5620 2024-04-25T08:30:26.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.166+0000 7f41957fa640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417806b4d0 0x7f417806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.166+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4180089ff0 con 0x7f41980b5620 2024-04-25T08:30:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.166+0000 7f419cb64640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417806b4d0 0x7f417806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.167+0000 7f419cb64640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417806b4d0 0x7f417806d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4190007fd0 tx=0x7f4190009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.171+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41800590c0 con 0x7f41980b5620 2024-04-25T08:30:26.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.419+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4198068070 con 0x7f41980b5620 2024-04-25T08:30:26.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.421+0000 7f41957fa640 1 -- 172.21.15.60:0/4221255404 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f41800589c0 con 0x7f41980b5620 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417806b4d0 msgr2=0x7f417806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417806b4d0 0x7f417806d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4190007fd0 tx=0x7f4190009040 comp rx=0 tx=0).stop 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 msgr2=0x7f41980c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 0x7f41980c7f80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f418000e9e0 tx=0x7f418000eea0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 shutdown_connections 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f417806b4d0 0x7f417806d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f419815b7d0 0x7f41980c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41980b5620 0x7f41980c7f80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 --2- 172.21.15.60:0/4221255404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f419806c2d0 0x7f4198157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 >> 172.21.15.60:0/4221255404 conn(0x7f41980d0800 msgr2=0x7f4198159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 shutdown_connections 2024-04-25T08:30:26.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.423+0000 7f419edc8640 1 -- 172.21.15.60:0/4221255404 wait complete. 2024-04-25T08:30:26.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T08:30:26.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.535+0000 7fae797f5640 1 Processor -- start 2024-04-25T08:30:26.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.535+0000 7fae797f5640 1 -- start start 2024-04-25T08:30:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae797f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fae7415b7d0 0x7fae740cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae797f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae7406c9e0 0x7fae740b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae797f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7406cf50 con 0x7fae740b5620 2024-04-25T08:30:26.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae797f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae740ce530 con 0x7fae7415b7d0 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae797f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740ce810 con 0x7fae7406c9e0 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae72ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fae7415b7d0 0x7fae740cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae727fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fae740b5620 0x7fae7406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52766/0 (socket says 172.21.15.60:52766) 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae727fc640 1 -- 172.21.15.60:0/1217472703 learned_addr learned my addr 172.21.15.60:0/1217472703 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae737fe640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae7406c9e0 0x7fae740b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.536+0000 7fae72ffd640 1 -- 172.21.15.60:0/1217472703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae7406c9e0 msgr2=0x7fae740b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae72ffd640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae7406c9e0 0x7fae740b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae72ffd640 1 -- 172.21.15.60:0/1217472703 >> v1:172.21.15.60:6789/0 conn(0x7fae740b5620 legacy=0x7fae7406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae72ffd640 1 -- 172.21.15.60:0/1217472703 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae740ceaf0 con 0x7fae7415b7d0 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae71ffb640 1 -- 172.21.15.60:0/1217472703 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364101048 0 0) 0x7fae7406cf50 con 0x7fae740b5620 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae737fe640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae7406c9e0 0x7fae740b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:26.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae72ffd640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740cf010 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5320e3f5dc61871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae71ffb640 1 -- 172.21.15.60:0/1217472703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c037070 con 0x7fae7415b7d0 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae71ffb640 1 -- 172.21.15.60:0/1217472703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c032440 con 0x7fae7415b7d0 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.537+0000 7fae71ffb640 1 -- 172.21.15.60:0/1217472703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c032810 con 0x7fae7415b7d0 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 -- 172.21.15.60:0/1217472703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 msgr2=0x7fae740cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740cf010 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097a0 tx=0x7fae6c02a180 comp rx=0 tx=0).stop 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 -- 172.21.15.60:0/1217472703 shutdown_connections 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae7406c9e0 0x7fae740b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 --2- 172.21.15.60:0/1217472703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740cf010 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 -- 172.21.15.60:0/1217472703 >> 172.21.15.60:0/1217472703 conn(0x7fae740d0800 msgr2=0x7fae740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 -- 172.21.15.60:0/1217472703 shutdown_connections 2024-04-25T08:30:26.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.538+0000 7fae797f5640 1 -- 172.21.15.60:0/1217472703 wait complete. 2024-04-25T08:30:26.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.539+0000 7fae797f5640 1 Processor -- start 2024-04-25T08:30:26.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.539+0000 7fae797f5640 1 -- start start 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae797f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7406c9e0 0x7fae74159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae797f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae740b5620 0x7fae740d2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae797f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae797f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740cf6d0 con 0x7fae7415b7d0 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae797f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740cf550 con 0x7fae7406c9e0 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae797f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7415bbb0 con 0x7fae740b5620 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae737fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae737fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740d2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47696/0 (socket says 172.21.15.60:47696) 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae737fe640 1 -- 172.21.15.60:0/1556358162 learned_addr learned my addr 172.21.15.60:0/1556358162 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae727fc640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae740b5620 0x7fae740d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.540+0000 7fae72ffd640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7406c9e0 0x7fae74159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae737fe640 1 -- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae740b5620 msgr2=0x7fae740d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae737fe640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae740b5620 0x7fae740d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae737fe640 1 -- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7406c9e0 msgr2=0x7fae74159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae737fe640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7406c9e0 0x7fae74159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae737fe640 1 -- 172.21.15.60:0/1556358162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae740cdf00 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae72ffd640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7406c9e0 0x7fae74159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae737fe640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740d2ca0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae64007de0 tx=0x7fae6400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6401a070 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64003990 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae741e6590 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64003d80 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.541+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae740c8e60 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.542+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae74073a00 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.542+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae6401d070 con 0x7fae7415b7d0 2024-04-25T08:30:26.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.543+0000 7fae6b7fe640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae5406b500 0x7fae5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.543+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fae6408a380 con 0x7fae7415b7d0 2024-04-25T08:30:26.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.543+0000 7fae72ffd640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae5406b500 0x7fae5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.543+0000 7fae72ffd640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae5406b500 0x7fae5406d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fae6c034000 tx=0x7fae6c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.547+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae740cdf00 con 0x7fae7415b7d0 2024-04-25T08:30:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.797+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fae740d0be0 con 0x7fae7415b7d0 2024-04-25T08:30:26.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.799+0000 7fae6b7fe640 1 -- 172.21.15.60:0/1556358162 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fae64010070 con 0x7fae7415b7d0 2024-04-25T08:30:26.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae5406b500 msgr2=0x7fae5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae5406b500 0x7fae5406d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fae6c034000 tx=0x7fae6c02e000 comp rx=0 tx=0).stop 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 msgr2=0x7fae740d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740d2ca0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae64007de0 tx=0x7fae6400d4a0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 shutdown_connections 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae7415b7d0 0x7fae740d2ca0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae740b5620 0x7fae740d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae5406b500 0x7fae5406d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 --2- 172.21.15.60:0/1556358162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae7406c9e0 0x7fae74159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 >> 172.21.15.60:0/1556358162 conn(0x7fae740d0800 msgr2=0x7fae74160e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 shutdown_connections 2024-04-25T08:30:26.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.801+0000 7fae797f5640 1 -- 172.21.15.60:0/1556358162 wait complete. 2024-04-25T08:30:26.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T08:30:26.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.913+0000 7fd88b5cf640 1 Processor -- start 2024-04-25T08:30:26.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.913+0000 7fd88b5cf640 1 -- start start 2024-04-25T08:30:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88b5cf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8841446d0 0x7fd8840b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88b5cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd884144d50 0x7fd8840b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88b5cf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd8841452a0 0x7fd8840cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88b5cf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8840b2b20 con 0x7fd8841446d0 2024-04-25T08:30:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88b5cf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88406d400 con 0x7fd8841452a0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88b5cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840b49a0 con 0x7fd884144d50 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8841446d0 0x7fd8840b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8841446d0 0x7fd8840b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47700/0 (socket says 172.21.15.60:47700) 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 -- 172.21.15.60:0/742155870 learned_addr learned my addr 172.21.15.60:0/742155870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd888b6a640 1 --2- 172.21.15.60:0/742155870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd884144d50 0x7fd8840b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 -- 172.21.15.60:0/742155870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd884144d50 msgr2=0x7fd8840b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 --2- 172.21.15.60:0/742155870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd884144d50 0x7fd8840b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd889b6c640 1 --2- 172.21.15.60:0/742155870 >> v2:172.21.15.100:3300/0 conn(0x7fd8841452a0 0x7fd8840cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 -- 172.21.15.60:0/742155870 >> v2:172.21.15.100:3300/0 conn(0x7fd8841452a0 msgr2=0x7fd8840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 --2- 172.21.15.60:0/742155870 >> v2:172.21.15.100:3300/0 conn(0x7fd8841452a0 0x7fd8840cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd88936b640 1 -- 172.21.15.60:0/742155870 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840cccc0 con 0x7fd8841446d0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.914+0000 7fd889b6c640 1 --2- 172.21.15.60:0/742155870 >> v2:172.21.15.100:3300/0 conn(0x7fd8841452a0 0x7fd8840cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.915+0000 7fd88936b640 1 --2- 172.21.15.60:0/742155870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd8840b3f20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd884145880 tx=0x7fd88002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e264327ab6cf0144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.915+0000 7fd87bfff640 1 -- 172.21.15.60:0/742155870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880037070 con 0x7fd8841446d0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.915+0000 7fd87bfff640 1 -- 172.21.15.60:0/742155870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd880032440 con 0x7fd8841446d0 2024-04-25T08:30:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.915+0000 7fd87bfff640 1 -- 172.21.15.60:0/742155870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880032810 con 0x7fd8841446d0 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.915+0000 7fd88b5cf640 1 -- 172.21.15.60:0/742155870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 msgr2=0x7fd8840b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.915+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/742155870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd8840b3f20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd884145880 tx=0x7fd88002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 -- 172.21.15.60:0/742155870 shutdown_connections 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/742155870 >> v2:172.21.15.100:3300/0 conn(0x7fd8841452a0 0x7fd8840cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/742155870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd884144d50 0x7fd8840b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/742155870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd8840b3f20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 -- 172.21.15.60:0/742155870 >> 172.21.15.60:0/742155870 conn(0x7fd88406abe0 msgr2=0x7fd88406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 -- 172.21.15.60:0/742155870 shutdown_connections 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.916+0000 7fd88b5cf640 1 -- 172.21.15.60:0/742155870 wait complete. 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.917+0000 7fd88b5cf640 1 Processor -- start 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.917+0000 7fd88b5cf640 1 -- start start 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd88413dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884144d50 0x7fd8841e4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8841452a0 0x7fd8840b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840cc700 con 0x7fd8841446d0 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840cc420 con 0x7fd884144d50 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840cc9e0 con 0x7fd8841452a0 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd88413dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd88413dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47714/0 (socket says 172.21.15.60:47714) 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 -- 172.21.15.60:0/3108302210 learned_addr learned my addr 172.21.15.60:0/3108302210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 -- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8841452a0 msgr2=0x7fd8840b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd888b6a640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884144d50 0x7fd8841e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd889b6c640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8841452a0 0x7fd8840b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8841452a0 0x7fd8840b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 -- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884144d50 msgr2=0x7fd8841e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884144d50 0x7fd8841e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 -- 172.21.15.60:0/3108302210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8841e3170 con 0x7fd8841446d0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd888b6a640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884144d50 0x7fd8841e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd889b6c640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8841452a0 0x7fd8840b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88936b640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd88413dcd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd880000c00 tx=0x7fd8800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880041070 con 0x7fd8841446d0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88003c870 con 0x7fd8841446d0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880037040 con 0x7fd8841446d0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.918+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8841e3450 con 0x7fd8841446d0 2024-04-25T08:30:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.919+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8841e3a10 con 0x7fd8841446d0 2024-04-25T08:30:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.920+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd884073a00 con 0x7fd8841446d0 2024-04-25T08:30:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.920+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd880044070 con 0x7fd8841446d0 2024-04-25T08:30:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.920+0000 7fd879ffb640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd86806b4d0 0x7fd86806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.920+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd8800aab30 con 0x7fd8841446d0 2024-04-25T08:30:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.921+0000 7fd888b6a640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd86806b4d0 0x7fd86806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.921+0000 7fd888b6a640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd86806b4d0 0x7fd86806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:26.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:26.925+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd880079c00 con 0x7fd8841446d0 2024-04-25T08:30:27.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.174+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd88406eb00 con 0x7fd8841446d0 2024-04-25T08:30:27.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.177+0000 7fd879ffb640 1 -- 172.21.15.60:0/3108302210 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd880079500 con 0x7fd8841446d0 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd86806b4d0 msgr2=0x7fd86806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd86806b4d0 0x7fd86806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd874006fd0 tx=0x7fd874008040 comp rx=0 tx=0).stop 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 msgr2=0x7fd88413dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd88413dcd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd880000c00 tx=0x7fd8800046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 shutdown_connections 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8841452a0 0x7fd8840b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd86806b4d0 0x7fd86806d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd884144d50 0x7fd8841e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 --2- 172.21.15.60:0/3108302210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8841446d0 0x7fd88413dcd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 >> 172.21.15.60:0/3108302210 conn(0x7fd88406abe0 msgr2=0x7fd8841437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 shutdown_connections 2024-04-25T08:30:27.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.179+0000 7fd88b5cf640 1 -- 172.21.15.60:0/3108302210 wait complete. 2024-04-25T08:30:27.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T08:30:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.294+0000 7fb8cf838640 1 Processor -- start 2024-04-25T08:30:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.294+0000 7fb8cf838640 1 -- start start 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cf838640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 0x7fb8c80ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cf838640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8c80cd520 0x7fb8c80cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cf838640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80c6940 con 0x7fb8c815a9f0 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cf838640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c80b0020 con 0x7fb8c80cd520 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cf838640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80cb5a0 con 0x7fb8c80c6010 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cd5d4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8c815a9f0 0x7fb8c80cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52782/0 (socket says 172.21.15.60:52782) 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cd5d4640 1 -- 172.21.15.60:0/2605541445 learned_addr learned my addr 172.21.15.60:0/2605541445 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127938849 0 0) 0x7fb8c4001110 con 0x7fb8c815a9f0 2024-04-25T08:30:27.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cddd5640 1 --2- 172.21.15.60:0/2605541445 >> v2:172.21.15.100:3300/0 conn(0x7fb8c80cd520 0x7fb8c80cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b4003520 con 0x7fb8c815a9f0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 0x7fb8c80ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071504916 0 0) 0x7fb8b4003520 con 0x7fb8c815a9f0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 msgr2=0x7fb8c80ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 0x7fb8c80ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 >> v2:172.21.15.100:3300/0 conn(0x7fb8c80cd520 msgr2=0x7fb8c80cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 --2- 172.21.15.60:0/2605541445 >> v2:172.21.15.100:3300/0 conn(0x7fb8c80cd520 0x7fb8c80cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c815add0 con 0x7fb8c815a9f0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb8c4001840 con 0x7fb8c815a9f0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8cddd5640 1 --2- 172.21.15.60:0/2605541445 >> v2:172.21.15.100:3300/0 conn(0x7fb8c80cd520 0x7fb8c80cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 >> v1:172.21.15.60:6789/0 conn(0x7fb8c815a9f0 legacy=0x7fb8c80cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 0x7fb8b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4003fe0 0x7fb8b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4004d40 0x7fb8b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80cb5a0 con 0x7fb8b4003fe0 2024-04-25T08:30:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b400ba70 con 0x7fb8b4004d40 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.295+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b400bc40 con 0x7fb8c80c6010 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4003fe0 0x7fb8b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8cd5d4640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 0x7fb8b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8cddd5640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4004d40 0x7fb8b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 msgr2=0x7fb8b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80c6010 0x7fb8b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4004d40 msgr2=0x7fb8b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4004d40 0x7fb8b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.296+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/2605541445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b4003520 con 0x7fb8b4003fe0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4003fe0 0x7fb8b4004800 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800ba30 tx=0x7fb8b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894c1b0ba1d90c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8030070 con 0x7fb8b4003fe0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b802ade0 con 0x7fb8b4003fe0 2024-04-25T08:30:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8bffff640 1 -- 172.21.15.60:0/2605541445 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800b600 con 0x7fb8b4003fe0 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cf838640 1 -- 172.21.15.60:0/2605541445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4003fe0 msgr2=0x7fb8b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cf838640 1 --2- 172.21.15.60:0/2605541445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4003fe0 0x7fb8b4004800 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800ba30 tx=0x7fb8b4005830 comp rx=0 tx=0).stop 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cddd5640 1 -- 172.21.15.60:0/2605541445 reap_dead start 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cf838640 1 -- 172.21.15.60:0/2605541445 shutdown_connections 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cf838640 1 -- 172.21.15.60:0/2605541445 >> 172.21.15.60:0/2605541445 conn(0x7fb8c80b2390 msgr2=0x7fb8c80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cf838640 1 -- 172.21.15.60:0/2605541445 shutdown_connections 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.297+0000 7fb8cf838640 1 -- 172.21.15.60:0/2605541445 wait complete. 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 Processor -- start 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 -- start start 2024-04-25T08:30:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4003fe0 0x7fb8c80b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 0x7fb8c815d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80b3d90 0x7fb8c80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80cb230 con 0x7fb8c814a0c0 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80caf50 con 0x7fb8b4003fe0 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8cf838640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80ccfe0 con 0x7fb8c80b3d90 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.299+0000 7fb8ccdd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 0x7fb8c815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 0x7fb8c815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47724/0 (socket says 172.21.15.60:47724) 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/3769095461 learned_addr learned my addr 172.21.15.60:0/3769095461 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8cd5d4640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4003fe0 0x7fb8c80b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8cddd5640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80b3d90 0x7fb8c80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80b3d90 msgr2=0x7fb8c80b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80b3d90 0x7fb8c80b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4003fe0 msgr2=0x7fb8c80b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4003fe0 0x7fb8c80b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 -- 172.21.15.60:0/3769095461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8152c10 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8ccdd3640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 0x7fb8c815d720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800bb20 tx=0x7fb8b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8043030 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b802abe0 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c815c680 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b800b600 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.300+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c815cc40 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.301+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c806cc80 con 0x7fb8c814a0c0 2024-04-25T08:30:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.302+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8b8046070 con 0x7fb8c814a0c0 2024-04-25T08:30:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.302+0000 7fb8bdffb640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8ac06b500 0x7fb8ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.302+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb8b80aaab0 con 0x7fb8c814a0c0 2024-04-25T08:30:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.302+0000 7fb8cd5d4640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8ac06b500 0x7fb8ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.303+0000 7fb8cd5d4640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8ac06b500 0x7fb8ac06d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40012f0 tx=0x7fb8c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.307+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8b803c3a0 con 0x7fb8c814a0c0 2024-04-25T08:30:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.559+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb8c80c57a0 con 0x7fb8c814a0c0 2024-04-25T08:30:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.562+0000 7fb8bdffb640 1 -- 172.21.15.60:0/3769095461 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb8b8079a60 con 0x7fb8c814a0c0 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8ac06b500 msgr2=0x7fb8ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8ac06b500 0x7fb8ac06d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40012f0 tx=0x7fb8c4009040 comp rx=0 tx=0).stop 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 msgr2=0x7fb8c815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 0x7fb8c815d720 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800bb20 tx=0x7fb8b80046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 shutdown_connections 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8c80b3d90 0x7fb8c80b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8c814a0c0 0x7fb8c815d720 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb8ac06b500 0x7fb8ac06d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 --2- 172.21.15.60:0/3769095461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b4003fe0 0x7fb8c80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 >> 172.21.15.60:0/3769095461 conn(0x7fb8c80b2390 msgr2=0x7fb8c806e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 shutdown_connections 2024-04-25T08:30:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.564+0000 7fb8cf838640 1 -- 172.21.15.60:0/3769095461 wait complete. 2024-04-25T08:30:27.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T08:30:27.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.677+0000 7f30cbf30640 1 Processor -- start 2024-04-25T08:30:27.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.678+0000 7f30cbf30640 1 -- start start 2024-04-25T08:30:27.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.678+0000 7f30cbf30640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f30c40ced80 0x7f30c40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.678+0000 7f30cbf30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c415e000 con 0x7f30c4161710 2024-04-25T08:30:27.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.678+0000 7f30cbf30640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c40d1de0 con 0x7f30c415dc20 2024-04-25T08:30:27.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.678+0000 7f30cbf30640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c4161c60 con 0x7f30c40ced80 2024-04-25T08:30:27.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.678+0000 7f30ca4cd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f30c4161710 0x7f30c40d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52790/0 (socket says 172.21.15.60:52790) 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30ca4cd640 1 -- 172.21.15.60:0/1445608141 learned_addr learned my addr 172.21.15.60:0/1445608141 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30c94cb640 1 --2- 172.21.15.60:0/1445608141 >> v2:172.21.15.100:3300/0 conn(0x7f30c40ced80 0x7f30c40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30c8cca640 1 -- 172.21.15.60:0/1445608141 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355539810 0 0) 0x7f30c415e000 con 0x7f30c4161710 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30c8cca640 1 -- 172.21.15.60:0/1445608141 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b0003520 con 0x7f30c4161710 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30c94cb640 1 -- 172.21.15.60:0/1445608141 >> v1:172.21.15.100:6789/0 conn(0x7f30c415dc20 legacy=0x7f30c40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30c94cb640 1 -- 172.21.15.60:0/1445608141 >> v1:172.21.15.60:6789/0 conn(0x7f30c4161710 legacy=0x7f30c40d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.679+0000 7f30c94cb640 1 -- 172.21.15.60:0/1445608141 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c4161f40 con 0x7f30c40ced80 2024-04-25T08:30:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.680+0000 7f30c94cb640 1 --2- 172.21.15.60:0/1445608141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c40d22e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f30c00097a0 tx=0x7f30c002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5a9d4efcb34fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.680+0000 7f30c8cca640 1 -- 172.21.15.60:0/1445608141 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0037070 con 0x7f30c40ced80 2024-04-25T08:30:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.680+0000 7f30c8cca640 1 -- 172.21.15.60:0/1445608141 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0032440 con 0x7f30c40ced80 2024-04-25T08:30:27.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.680+0000 7f30c8cca640 1 -- 172.21.15.60:0/1445608141 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c00327f0 con 0x7f30c40ced80 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 -- 172.21.15.60:0/1445608141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 msgr2=0x7f30c40d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1445608141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c40d22e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f30c00097a0 tx=0x7f30c002a230 comp rx=0 tx=0).stop 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 -- 172.21.15.60:0/1445608141 shutdown_connections 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1445608141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c40d22e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 -- 172.21.15.60:0/1445608141 >> 172.21.15.60:0/1445608141 conn(0x7f30c40d0420 msgr2=0x7f30c40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 -- 172.21.15.60:0/1445608141 shutdown_connections 2024-04-25T08:30:27.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.681+0000 7f30cbf30640 1 -- 172.21.15.60:0/1445608141 wait complete. 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.682+0000 7f30cbf30640 1 Processor -- start 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.682+0000 7f30cbf30640 1 -- start start 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c4160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 0x7f30c41f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c4161710 0x7f30c4157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40b5e50 con 0x7f30c415dc20 2024-04-25T08:30:27.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c4162500 con 0x7f30c40ced80 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40b5fd0 con 0x7f30c4161710 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 0x7f30c41f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 0x7f30c41f4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47736/0 (socket says 172.21.15.60:47736) 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 -- 172.21.15.60:0/1892474391 learned_addr learned my addr 172.21.15.60:0/1892474391 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c9ccc640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c4160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30ca4cd640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c4161710 0x7f30c4157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 -- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c4161710 msgr2=0x7f30c4157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c4161710 0x7f30c4157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 -- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 msgr2=0x7f30c4160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c4160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 -- 172.21.15.60:0/1892474391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c415f6f0 con 0x7f30c415dc20 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30ca4cd640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c4161710 0x7f30c4157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30c94cb640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 0x7f30c41f4860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30c00097a0 tx=0x7f30c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0037050 con 0x7f30c415dc20 2024-04-25T08:30:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0038040 con 0x7f30c415dc20 2024-04-25T08:30:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c415f9d0 con 0x7f30c415dc20 2024-04-25T08:30:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.683+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0004470 con 0x7f30c415dc20 2024-04-25T08:30:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.684+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c415ff90 con 0x7f30c415dc20 2024-04-25T08:30:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.684+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c4073a00 con 0x7f30c415dc20 2024-04-25T08:30:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.685+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30c003c6d0 con 0x7f30c415dc20 2024-04-25T08:30:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.685+0000 7f30ba7fc640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.685+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f30c00aaa90 con 0x7f30c415dc20 2024-04-25T08:30:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.686+0000 7f30c9ccc640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.686+0000 7f30c9ccc640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30a806b4d0 0x7f30a806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f30b4001640 tx=0x7f30b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:27.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.690+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30c0079ad0 con 0x7f30c415dc20 2024-04-25T08:30:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.939+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f30c40d16b0 con 0x7f30c415dc20 2024-04-25T08:30:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.941+0000 7f30ba7fc640 1 -- 172.21.15.60:0/1892474391 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f30c00793d0 con 0x7f30c415dc20 2024-04-25T08:30:27.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.943+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30a806b4d0 msgr2=0x7f30a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.943+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30a806b4d0 0x7f30a806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f30b4001640 tx=0x7f30b4009040 comp rx=0 tx=0).stop 2024-04-25T08:30:27.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.943+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 msgr2=0x7f30c41f4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:27.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.943+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 0x7f30c41f4860 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30c00097a0 tx=0x7f30c00046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 shutdown_connections 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30c4161710 0x7f30c4157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30c415dc20 0x7f30c41f4860 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30a806b4d0 0x7f30a806d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 --2- 172.21.15.60:0/1892474391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30c40ced80 0x7f30c4160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 >> 172.21.15.60:0/1892474391 conn(0x7f30c40d0420 msgr2=0x7f30c40b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 shutdown_connections 2024-04-25T08:30:27.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:27.944+0000 7f30cbf30640 1 -- 172.21.15.60:0/1892474391 wait complete. 2024-04-25T08:30:27.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.056+0000 7f87429b0640 1 Processor -- start 2024-04-25T08:30:28.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.056+0000 7f87429b0640 1 -- start start 2024-04-25T08:30:28.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f87429b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f873c15dc90 0x7f873c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f87429b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c0d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f87429b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c0cce50 con 0x7f873c0cc8e0 2024-04-25T08:30:28.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f87429b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c0d21c0 con 0x7f873c15dc90 2024-04-25T08:30:28.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f87429b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c161af0 con 0x7f873c0d1810 2024-04-25T08:30:28.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f873bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f873c15dc90 0x7f873c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f873bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f873c15dc90 0x7f873c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47742/0 (socket says 172.21.15.60:47742) 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f873b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f873c0cc8e0 0x7f873c15ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52802/0 (socket says 172.21.15.60:52802) 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f873bfff640 1 -- 172.21.15.60:0/1422918133 learned_addr learned my addr 172.21.15.60:0/1422918133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f8740f4d640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c0d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.057+0000 7f873bfff640 1 -- 172.21.15.60:0/1422918133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 msgr2=0x7f873c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873bfff640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c0d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873bfff640 1 -- 172.21.15.60:0/1422918133 >> v1:172.21.15.60:6789/0 conn(0x7f873c0cc8e0 legacy=0x7f873c15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873bfff640 1 -- 172.21.15.60:0/1422918133 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c161dd0 con 0x7f873c15dc90 2024-04-25T08:30:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873affd640 1 -- 172.21.15.60:0/1422918133 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293822197 0 0) 0x7f873c0cce50 con 0x7f873c0cc8e0 2024-04-25T08:30:28.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f8740f4d640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873bfff640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c15dc90 0x7f873c1625d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8730009870 tx=0x7f873002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6343ef927574cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873affd640 1 -- 172.21.15.60:0/1422918133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730037070 con 0x7f873c15dc90 2024-04-25T08:30:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873affd640 1 -- 172.21.15.60:0/1422918133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730032440 con 0x7f873c15dc90 2024-04-25T08:30:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.058+0000 7f873affd640 1 -- 172.21.15.60:0/1422918133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730032810 con 0x7f873c15dc90 2024-04-25T08:30:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 -- 172.21.15.60:0/1422918133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c15dc90 msgr2=0x7f873c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c15dc90 0x7f873c1625d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8730009870 tx=0x7f873002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 -- 172.21.15.60:0/1422918133 shutdown_connections 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c0d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 --2- 172.21.15.60:0/1422918133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c15dc90 0x7f873c1625d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 -- 172.21.15.60:0/1422918133 >> 172.21.15.60:0/1422918133 conn(0x7f873c06f590 msgr2=0x7f873c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 -- 172.21.15.60:0/1422918133 shutdown_connections 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.059+0000 7f87429b0640 1 -- 172.21.15.60:0/1422918133 wait complete. 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.060+0000 7f87429b0640 1 Processor -- start 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.060+0000 7f87429b0640 1 -- start start 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f87429b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 0x7f873c0ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f87429b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f87429b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c15dc90 0x7f873c1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f87429b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c15e070 con 0x7f873c0cc8e0 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f87429b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0cd130 con 0x7f873c15dc90 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f87429b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c15e1f0 con 0x7f873c0d1810 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f873bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 0x7f873c0ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f873bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 0x7f873c0ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47748/0 (socket says 172.21.15.60:47748) 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f873bfff640 1 -- 172.21.15.60:0/2262122529 learned_addr learned my addr 172.21.15.60:0/2262122529 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.061+0000 7f873b7fe640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f8740f4d640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c15dc90 0x7f873c1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f873bfff640 1 -- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 msgr2=0x7f873c15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f873bfff640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f873bfff640 1 -- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c15dc90 msgr2=0x7f873c1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f873bfff640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c15dc90 0x7f873c1f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f873bfff640 1 -- 172.21.15.60:0/2262122529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c15adc0 con 0x7f873c0cc8e0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f873bfff640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 0x7f873c0ca180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8730035040 tx=0x7f87300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730037040 con 0x7f873c0cc8e0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730004230 con 0x7f873c0cc8e0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c0ca910 con 0x7f873c0cc8e0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730003610 con 0x7f873c0cc8e0 2024-04-25T08:30:28.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.062+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c0caed0 con 0x7f873c0cc8e0 2024-04-25T08:30:28.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.063+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c0d2550 con 0x7f873c0cc8e0 2024-04-25T08:30:28.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.064+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f873002ad10 con 0x7f873c0cc8e0 2024-04-25T08:30:28.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.064+0000 7f8738ff9640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f872806b4d0 0x7f872806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.065+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f87300aaa90 con 0x7f873c0cc8e0 2024-04-25T08:30:28.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.065+0000 7f873b7fe640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f872806b4d0 0x7f872806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.065+0000 7f873b7fe640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f872806b4d0 0x7f872806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f873c15d440 tx=0x7f872c007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.069+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f873c0d2550 con 0x7f873c0cc8e0 2024-04-25T08:30:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.252+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873c073990 con 0x7f873c0cc8e0 2024-04-25T08:30:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.253+0000 7f8738ff9640 1 -- 172.21.15.60:0/2262122529 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f8730079c80 con 0x7f873c0cc8e0 2024-04-25T08:30:28.250 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:28.250 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:25.274778+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:28.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f872806b4d0 msgr2=0x7f872806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f872806b4d0 0x7f872806d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f873c15d440 tx=0x7f872c007c00 comp rx=0 tx=0).stop 2024-04-25T08:30:28.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 msgr2=0x7f873c0ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 0x7f873c0ca180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8730035040 tx=0x7f87300046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 shutdown_connections 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f872806b4d0 0x7f872806d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f873c15dc90 0x7f873c1f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f873c0d1810 0x7f873c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 --2- 172.21.15.60:0/2262122529 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f873c0cc8e0 0x7f873c0ca180 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 >> 172.21.15.60:0/2262122529 conn(0x7f873c06f590 msgr2=0x7f873c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 shutdown_connections 2024-04-25T08:30:28.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.255+0000 7f87429b0640 1 -- 172.21.15.60:0/2262122529 wait complete. 2024-04-25T08:30:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:30:28.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:30:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.371+0000 7fb8bbb63640 1 Processor -- start 2024-04-25T08:30:28.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.371+0000 7fb8bbb63640 1 -- start start 2024-04-25T08:30:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8bbb63640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8b415da90 0x7fb8b4161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8bbb63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8b4161ba0 0x7fb8b40d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8bbb63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b4162040 con 0x7fb8b40b9750 2024-04-25T08:30:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8bbb63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b4162600 con 0x7fb8b4161ba0 2024-04-25T08:30:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8bbb63640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b4162320 con 0x7fb8b415da90 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8ba100640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8b4161ba0 0x7fb8b40d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8ba100640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8b4161ba0 0x7fb8b40d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47750/0 (socket says 172.21.15.60:47750) 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8ba100640 1 -- 172.21.15.60:0/2348114483 learned_addr learned my addr 172.21.15.60:0/2348114483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8ba100640 1 -- 172.21.15.60:0/2348114483 >> v2:172.21.15.100:3300/0 conn(0x7fb8b415da90 msgr2=0x7fb8b4161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8b98ff640 1 --2- 172.21.15.60:0/2348114483 >> v2:172.21.15.100:3300/0 conn(0x7fb8b415da90 0x7fb8b4161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.372+0000 7fb8ba100640 1 --2- 172.21.15.60:0/2348114483 >> v2:172.21.15.100:3300/0 conn(0x7fb8b415da90 0x7fb8b4161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.373+0000 7fb8ba100640 1 -- 172.21.15.60:0/2348114483 >> v1:172.21.15.100:6789/0 conn(0x7fb8b40b9750 legacy=0x7fb8b40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.373+0000 7fb8ba100640 1 -- 172.21.15.60:0/2348114483 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40d1b10 con 0x7fb8b4161ba0 2024-04-25T08:30:28.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.373+0000 7fb8ba100640 1 --2- 172.21.15.60:0/2348114483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40d2310 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40b61c0 tx=0x7fb8ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580e929b693d545f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.373+0000 7fb8b88fd640 1 -- 172.21.15.60:0/2348114483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac036070 con 0x7fb8b4161ba0 2024-04-25T08:30:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.373+0000 7fb8b88fd640 1 -- 172.21.15.60:0/2348114483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac031600 con 0x7fb8b4161ba0 2024-04-25T08:30:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.373+0000 7fb8b88fd640 1 -- 172.21.15.60:0/2348114483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0319d0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2348114483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 msgr2=0x7fb8b40d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2348114483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40d2310 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40b61c0 tx=0x7fb8ac02a660 comp rx=0 tx=0).stop 2024-04-25T08:30:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2348114483 shutdown_connections 2024-04-25T08:30:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2348114483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40d2310 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2348114483 >> v2:172.21.15.100:3300/0 conn(0x7fb8b415da90 0x7fb8b4161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2348114483 >> 172.21.15.60:0/2348114483 conn(0x7fb8b4061a20 msgr2=0x7fb8b4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2348114483 shutdown_connections 2024-04-25T08:30:28.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.374+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2348114483 wait complete. 2024-04-25T08:30:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 Processor -- start 2024-04-25T08:30:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 -- start start 2024-04-25T08:30:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b415da90 0x7fb8b414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b40b4990 0x7fb8b4151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40ced80 con 0x7fb8b4161ba0 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b415de70 con 0x7fb8b40b4990 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8bbb63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b40cef00 con 0x7fb8b415da90 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.376+0000 7fb8b90fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47758/0 (socket says 172.21.15.60:47758) 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 -- 172.21.15.60:0/2952735002 learned_addr learned my addr 172.21.15.60:0/2952735002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8ba100640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b40b4990 0x7fb8b4151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b98ff640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b415da90 0x7fb8b414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 -- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b415da90 msgr2=0x7fb8b414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b415da90 0x7fb8b414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 -- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b40b4990 msgr2=0x7fb8b4151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b40b4990 0x7fb8b4151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 -- 172.21.15.60:0/2952735002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40cc0a0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8ba100640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b40b4990 0x7fb8b4151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8b90fe640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40b4450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400c430 tx=0x7fb8a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4017070 con 0x7fb8b4161ba0 2024-04-25T08:30:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a40035f0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b4151ed0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40039e0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.377+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b4152490 con 0x7fb8b4161ba0 2024-04-25T08:30:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.378+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b40b4da0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.379+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8a401a070 con 0x7fb8b4161ba0 2024-04-25T08:30:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.379+0000 7fb8aa7fc640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.379+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb8a4089260 con 0x7fb8b4161ba0 2024-04-25T08:30:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.379+0000 7fb8b98ff640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.380+0000 7fb8b98ff640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89806b4d0 0x7fb89806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000a970 tx=0x7fb8b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.384+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8b40b4da0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.563+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8b4073a00 con 0x7fb8b4161ba0 2024-04-25T08:30:28.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.563+0000 7fb8aa7fc640 1 -- 172.21.15.60:0/2952735002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8a4057ce0 con 0x7fb8b4161ba0 2024-04-25T08:30:28.561 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89806b4d0 msgr2=0x7fb89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89806b4d0 0x7fb89806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000a970 tx=0x7fb8b0008040 comp rx=0 tx=0).stop 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 msgr2=0x7fb8b40b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40b4450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb8a400c430 tx=0x7fb8a400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 shutdown_connections 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b40b4990 0x7fb8b4151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b4161ba0 0x7fb8b40b4450 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 --2- 172.21.15.60:0/2952735002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b415da90 0x7fb8b414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 >> 172.21.15.60:0/2952735002 conn(0x7fb8b4061a20 msgr2=0x7fb8b40d2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 shutdown_connections 2024-04-25T08:30:28.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.565+0000 7fb8bbb63640 1 -- 172.21.15.60:0/2952735002 wait complete. 2024-04-25T08:30:28.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:30:28.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.677+0000 7f734296a640 1 Processor -- start 2024-04-25T08:30:28.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.677+0000 7f734296a640 1 -- start start 2024-04-25T08:30:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f734296a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f733c15ba00 0x7f733c09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f734296a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c15f5f0 0x7f733c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f734296a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c0cd710 con 0x7f733c1602e0 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f734296a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c0cd4b0 con 0x7f733c15ba00 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f734296a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c099b70 con 0x7f733c15f5f0 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f733bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f733c15ba00 0x7f733c09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f733b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f733c1602e0 0x7f733c0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52818/0 (socket says 172.21.15.60:52818) 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.678+0000 7f733b7fe640 1 -- 172.21.15.60:0/854478952 learned_addr learned my addr 172.21.15.60:0/854478952 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.679+0000 7f7340f07640 1 --2- 172.21.15.60:0/854478952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c15f5f0 0x7f733c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.679+0000 7f733bfff640 1 -- 172.21.15.60:0/854478952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c15f5f0 msgr2=0x7f733c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.679+0000 7f733bfff640 1 --2- 172.21.15.60:0/854478952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c15f5f0 0x7f733c15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.679+0000 7f733bfff640 1 -- 172.21.15.60:0/854478952 >> v1:172.21.15.60:6789/0 conn(0x7f733c1602e0 legacy=0x7f733c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.679+0000 7f733bfff640 1 -- 172.21.15.60:0/854478952 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c099e50 con 0x7f733c15ba00 2024-04-25T08:30:28.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.679+0000 7f733affd640 1 -- 172.21.15.60:0/854478952 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805366320 0 0) 0x7f7330001110 con 0x7f733c1602e0 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f733bfff640 1 --2- 172.21.15.60:0/854478952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c09a930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f732c0097a0 tx=0x7f732c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6feca1c67165e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f733affd640 1 -- 172.21.15.60:0/854478952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c037070 con 0x7f733c15ba00 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f733affd640 1 -- 172.21.15.60:0/854478952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c02adb0 con 0x7f733c15ba00 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f733affd640 1 -- 172.21.15.60:0/854478952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c032660 con 0x7f733c15ba00 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 -- 172.21.15.60:0/854478952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 msgr2=0x7f733c09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 --2- 172.21.15.60:0/854478952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c09a930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f732c0097a0 tx=0x7f732c02a060 comp rx=0 tx=0).stop 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 -- 172.21.15.60:0/854478952 shutdown_connections 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 --2- 172.21.15.60:0/854478952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c15f5f0 0x7f733c15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 --2- 172.21.15.60:0/854478952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c09a930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 -- 172.21.15.60:0/854478952 >> 172.21.15.60:0/854478952 conn(0x7f733c0b3270 msgr2=0x7f733c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 -- 172.21.15.60:0/854478952 shutdown_connections 2024-04-25T08:30:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.680+0000 7f734296a640 1 -- 172.21.15.60:0/854478952 wait complete. 2024-04-25T08:30:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.681+0000 7f734296a640 1 Processor -- start 2024-04-25T08:30:28.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 -- start start 2024-04-25T08:30:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c14abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f733c15f5f0 0x7f733c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c1602e0 0x7f733c0c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c15bde0 con 0x7f733c15ba00 2024-04-25T08:30:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c09ae70 con 0x7f733c15f5f0 2024-04-25T08:30:28.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f734296a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0b5120 con 0x7f733c1602e0 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f733bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c14abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f733bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c14abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47776/0 (socket says 172.21.15.60:47776) 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.682+0000 7f733bfff640 1 -- 172.21.15.60:0/646827227 learned_addr learned my addr 172.21.15.60:0/646827227 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733bfff640 1 -- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c1602e0 msgr2=0x7f733c0c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733bfff640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c1602e0 0x7f733c0c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733bfff640 1 -- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f733c15f5f0 msgr2=0x7f733c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733b7fe640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f733c15f5f0 0x7f733c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733bfff640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f733c15f5f0 0x7f733c155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733bfff640 1 -- 172.21.15.60:0/646827227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0b5400 con 0x7f733c15ba00 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f733bfff640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c14abb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f732c0098c0 tx=0x7f732c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c041030 con 0x7f733c15ba00 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c02abb0 con 0x7f733c15ba00 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c0b56e0 con 0x7f733c15ba00 2024-04-25T08:30:28.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c037040 con 0x7f733c15ba00 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.683+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c0b5ca0 con 0x7f733c15ba00 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.684+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c0b3bb0 con 0x7f733c15ba00 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.685+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f732c044070 con 0x7f733c15ba00 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.685+0000 7f7338ff9640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f732806b500 0x7f732806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.685+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f732c0aa780 con 0x7f733c15ba00 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.685+0000 7f733b7fe640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f732806b500 0x7f732806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.686+0000 7f733b7fe640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f732806b500 0x7f732806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f733c1553b0 tx=0x7f7330009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.689+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f733c0b3bb0 con 0x7f733c15ba00 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.872+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f733c073990 con 0x7f733c15ba00 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.872+0000 7f7338ff9640 1 -- 172.21.15.60:0/646827227 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f732c079850 con 0x7f733c15ba00 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:30:28.870 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:30:28.871 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:30:28.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f732806b500 msgr2=0x7f732806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f732806b500 0x7f732806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f733c1553b0 tx=0x7f7330009040 comp rx=0 tx=0).stop 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 msgr2=0x7f733c14abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c14abb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f732c0098c0 tx=0x7f732c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 shutdown_connections 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f733c1602e0 0x7f733c0c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f732806b500 0x7f732806d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f733c15f5f0 0x7f733c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 --2- 172.21.15.60:0/646827227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f733c15ba00 0x7f733c14abb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 >> 172.21.15.60:0/646827227 conn(0x7f733c0b3270 msgr2=0x7f733c0cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 shutdown_connections 2024-04-25T08:30:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.875+0000 7f734296a640 1 -- 172.21.15.60:0/646827227 wait complete. 2024-04-25T08:30:28.883 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:30:28.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:30:28.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.990+0000 7f84cd0e2640 1 Processor -- start 2024-04-25T08:30:28.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.990+0000 7f84cd0e2640 1 -- start start 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84cd0e2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84c815dd80 0x7f84c80b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84cd0e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 0x7f84c806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84cd0e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80b3930 con 0x7f84c806abe0 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84cd0e2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c806e410 con 0x7f84c815dd80 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84cd0e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b3c10 con 0x7f84c80ad570 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f84c806abe0 0x7f84c806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52828/0 (socket says 172.21.15.60:52828) 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c6d9d640 1 -- 172.21.15.60:0/14801618 learned_addr learned my addr 172.21.15.60:0/14801618 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84bffff640 1 --2- 172.21.15.60:0/14801618 >> v2:172.21.15.100:3300/0 conn(0x7f84c815dd80 0x7f84c80b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c759e640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 0x7f84c806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271405183 0 0) 0x7f84c80b3930 con 0x7f84c806abe0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b4003520 con 0x7f84c806abe0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731940813 0 0) 0x7f84b4003520 con 0x7f84c806abe0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 msgr2=0x7f84c806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 0x7f84c806ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 >> v2:172.21.15.100:3300/0 conn(0x7f84c815dd80 msgr2=0x7f84c80b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.991+0000 7f84c659c640 1 --2- 172.21.15.60:0/14801618 >> v2:172.21.15.100:3300/0 conn(0x7f84c815dd80 0x7f84c80b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c806e660 con 0x7f84c806abe0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f84b8001670 con 0x7f84c806abe0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 >> v1:172.21.15.60:6789/0 conn(0x7f84c806abe0 legacy=0x7f84c806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b40039b0 0x7f84b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40043b0 0x7f84b4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 0x7f84b4004f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c806e410 con 0x7f84b40039b0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b3c10 con 0x7f84b40043b0 2024-04-25T08:30:28.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b400b9c0 con 0x7f84c80ad570 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b40039b0 0x7f84b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84bffff640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40043b0 0x7f84b4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c759e640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 0x7f84b4004f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c6d9d640 1 -- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 msgr2=0x7f84b4004f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c80ad570 0x7f84b4004f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c6d9d640 1 -- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40043b0 msgr2=0x7f84b4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40043b0 0x7f84b4004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84c6d9d640 1 -- 172.21.15.60:0/14801618 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4003520 con 0x7f84b40039b0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.992+0000 7f84bffff640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40043b0 0x7f84b4004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b40039b0 0x7f84b4003d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005000 tx=0x7f84b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a17b93a4ea4e73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004c00 con 0x7f84b40039b0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b802f030 con 0x7f84b40039b0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84c659c640 1 -- 172.21.15.60:0/14801618 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8005d30 con 0x7f84b40039b0 2024-04-25T08:30:28.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84cd0e2640 1 -- 172.21.15.60:0/14801618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b40039b0 msgr2=0x7f84b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/14801618 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b40039b0 0x7f84b4003d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005000 tx=0x7f84b4005830 comp rx=0 tx=0).stop 2024-04-25T08:30:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84c759e640 1 -- 172.21.15.60:0/14801618 reap_dead start 2024-04-25T08:30:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84cd0e2640 1 -- 172.21.15.60:0/14801618 shutdown_connections 2024-04-25T08:30:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.993+0000 7f84cd0e2640 1 -- 172.21.15.60:0/14801618 >> 172.21.15.60:0/14801618 conn(0x7f84c80b5af0 msgr2=0x7f84c80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.994+0000 7f84cd0e2640 1 -- 172.21.15.60:0/14801618 shutdown_connections 2024-04-25T08:30:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.994+0000 7f84cd0e2640 1 -- 172.21.15.60:0/14801618 wait complete. 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.995+0000 7f84cd0e2640 1 Processor -- start 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.995+0000 7f84cd0e2640 1 -- start start 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84cd0e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40039b0 0x7f84c8160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84cd0e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c81583c0 0x7f84c8150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84cd0e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 0x7f84c8142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84cd0e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8070280 con 0x7f84c8158840 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84cd0e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b57d0 con 0x7f84b40039b0 2024-04-25T08:30:28.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84cd0e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8070400 con 0x7f84c81583c0 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 0x7f84c8142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 0x7f84c8142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47790/0 (socket says 172.21.15.60:47790) 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 -- 172.21.15.60:0/1630849117 learned_addr learned my addr 172.21.15.60:0/1630849117 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84bffff640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c81583c0 0x7f84c8150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40039b0 0x7f84c8160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 -- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c81583c0 msgr2=0x7f84c8150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c81583c0 0x7f84c8150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 -- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40039b0 msgr2=0x7f84c8160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40039b0 0x7f84c8160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 -- 172.21.15.60:0/1630849117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c815f970 con 0x7f84c8158840 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84c759e640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 0x7f84c8142d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f84c80b01d0 tx=0x7f84c00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.996+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0011e10 con 0x7f84c8158840 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.997+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c001b070 con 0x7f84c8158840 2024-04-25T08:30:28.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.997+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c815fc50 con 0x7f84c8158840 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.997+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0003990 con 0x7f84c8158840 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.997+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c8160210 con 0x7f84c8158840 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.998+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c8073a00 con 0x7f84c8158840 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.998+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84c0005d10 con 0x7f84c8158840 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.998+0000 7f84bf7fe640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84a806b4d0 0x7f84a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.999+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f84c008a5a0 con 0x7f84c8158840 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.999+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84a806b4d0 0x7f84a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:28.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:28.999+0000 7f84c6d9d640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84a806b4d0 0x7f84a806d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84b802f000 tx=0x7f84b8007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:29.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.003+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84c0025020 con 0x7f84c8158840 2024-04-25T08:30:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.183+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f84c80adb20 con 0x7f84c8158840 2024-04-25T08:30:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.276+0000 7f84bf7fe640 1 -- 172.21.15.60:0/1630849117 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f84c0058ef0 con 0x7f84c8158840 2024-04-25T08:30:29.274 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:30:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84a806b4d0 msgr2=0x7f84a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84a806b4d0 0x7f84a806d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84b802f000 tx=0x7f84b8007000 comp rx=0 tx=0).stop 2024-04-25T08:30:29.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 msgr2=0x7f84c8142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 0x7f84c8142d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f84c80b01d0 tx=0x7f84c00115a0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 shutdown_connections 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84c8158840 0x7f84c8142d30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84c81583c0 0x7f84c8150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84a806b4d0 0x7f84a806d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 --2- 172.21.15.60:0/1630849117 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b40039b0 0x7f84c8160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 >> 172.21.15.60:0/1630849117 conn(0x7f84c80b5af0 msgr2=0x7f84c80ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 shutdown_connections 2024-04-25T08:30:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.279+0000 7f84cd0e2640 1 -- 172.21.15.60:0/1630849117 wait complete. 2024-04-25T08:30:29.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:30:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 Processor -- start 2024-04-25T08:30:29.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 -- start start 2024-04-25T08:30:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 0x7f8b4c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 0x7f8b4c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c0d2020 con 0x7f8b4c06c870 2024-04-25T08:30:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c158610 con 0x7f8b4c158230 2024-04-25T08:30:29.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.401+0000 7f8b51b34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c06bea0 con 0x7f8b4c0d1b80 2024-04-25T08:30:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 0x7f8b4c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 0x7f8b4c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 0x7f8b4c0d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:57436/0 (socket says 172.21.15.60:57436) 2024-04-25T08:30:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4b7fe640 1 -- 172.21.15.60:0/3682297074 learned_addr learned my addr 172.21.15.60:0/3682297074 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 0x7f8b4c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33308/0 (socket says 172.21.15.60:33308) 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4affd640 1 -- 172.21.15.60:0/3682297074 >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 msgr2=0x7f8b4c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4affd640 1 --2- 172.21.15.60:0/3682297074 >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 0x7f8b4c0d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4affd640 1 -- 172.21.15.60:0/3682297074 >> v1:172.21.15.100:6789/0 conn(0x7f8b4c06c870 legacy=0x7f8b4c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4affd640 1 -- 172.21.15.60:0/3682297074 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0d2300 con 0x7f8b4c0d1b80 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.402+0000 7f8b4b7fe640 1 --2- 172.21.15.60:0/3682297074 >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 0x7f8b4c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.403+0000 7f8b4affd640 1 --2- 172.21.15.60:0/3682297074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 0x7f8b4c06c330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009790 tx=0x7f8b4002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e817978a29612c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.403+0000 7f8b4a7fc640 1 -- 172.21.15.60:0/3682297074 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40037070 con 0x7f8b4c0d1b80 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.403+0000 7f8b4a7fc640 1 -- 172.21.15.60:0/3682297074 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40032440 con 0x7f8b4c0d1b80 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.403+0000 7f8b4a7fc640 1 -- 172.21.15.60:0/3682297074 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40032810 con 0x7f8b4c0d1b80 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 -- 172.21.15.60:0/3682297074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 msgr2=0x7f8b4c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3682297074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 0x7f8b4c06c330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009790 tx=0x7f8b4002a180 comp rx=0 tx=0).stop 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 -- 172.21.15.60:0/3682297074 shutdown_connections 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3682297074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c0d1b80 0x7f8b4c06c330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3682297074 >> v2:172.21.15.100:3300/0 conn(0x7f8b4c158230 0x7f8b4c0d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 -- 172.21.15.60:0/3682297074 >> 172.21.15.60:0/3682297074 conn(0x7f8b4c061600 msgr2=0x7f8b4c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:29.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 -- 172.21.15.60:0/3682297074 shutdown_connections 2024-04-25T08:30:29.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.404+0000 7f8b51b34640 1 -- 172.21.15.60:0/3682297074 wait complete. 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.405+0000 7f8b51b34640 1 Processor -- start 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.405+0000 7f8b51b34640 1 -- start start 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b51b34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 0x7f8b4c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b51b34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b4c0d1b80 0x7f8b4c0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b51b34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c158230 0x7f8b4c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b51b34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c15e320 con 0x7f8b4c06c870 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b51b34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d25e0 con 0x7f8b4c0d1b80 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b51b34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c15e4a0 con 0x7f8b4c158230 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 0x7f8b4c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 0x7f8b4c14bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47802/0 (socket says 172.21.15.60:47802) 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 -- 172.21.15.60:0/3663477332 learned_addr learned my addr 172.21.15.60:0/3663477332 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4affd640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b4c0d1b80 0x7f8b4c0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4bfff640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c158230 0x7f8b4c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:29.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 -- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c158230 msgr2=0x7f8b4c155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c158230 0x7f8b4c155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 -- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b4c0d1b80 msgr2=0x7f8b4c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b4c0d1b80 0x7f8b4c0cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.406+0000 7f8b4b7fe640 1 -- 172.21.15.60:0/3663477332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0cb9b0 con 0x7f8b4c06c870 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.407+0000 7f8b4b7fe640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 0x7f8b4c14bf50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9b0 tx=0x7f8b3c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.407+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c017070 con 0x7f8b4c06c870 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.407+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c00c9a0 con 0x7f8b4c06c870 2024-04-25T08:30:29.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.407+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c0cc740 con 0x7f8b4c06c870 2024-04-25T08:30:29.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.407+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00cd50 con 0x7f8b4c06c870 2024-04-25T08:30:29.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.407+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c0ccd00 con 0x7f8b4c06c870 2024-04-25T08:30:29.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.408+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c06de50 con 0x7f8b4c06c870 2024-04-25T08:30:29.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.408+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b3c01a070 con 0x7f8b4c06c870 2024-04-25T08:30:29.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.409+0000 7f8b33fff640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b3806b4d0 0x7f8b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:29.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.409+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f8b3c089ff0 con 0x7f8b4c06c870 2024-04-25T08:30:29.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.409+0000 7f8b4affd640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b3806b4d0 0x7f8b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:29.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.410+0000 7f8b4affd640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b3806b4d0 0x7f8b3806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b40034000 tx=0x7f8b4002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.414+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b4c0cb9b0 con 0x7f8b4c06c870 2024-04-25T08:30:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:29.593+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8b4c073a00 con 0x7f8b4c06c870 2024-04-25T08:30:30.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.279+0000 7f8b33fff640 1 -- 172.21.15.60:0/3663477332 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8b3c058870 con 0x7f8b4c06c870 2024-04-25T08:30:30.277 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:30:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b3806b4d0 msgr2=0x7f8b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b3806b4d0 0x7f8b3806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b40034000 tx=0x7f8b4002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 msgr2=0x7f8b4c14bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 0x7f8b4c14bf50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9b0 tx=0x7f8b3c00ee70 comp rx=0 tx=0).stop 2024-04-25T08:30:30.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 shutdown_connections 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b4c158230 0x7f8b4c155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8b3806b4d0 0x7f8b3806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b4c0d1b80 0x7f8b4c0cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 --2- 172.21.15.60:0/3663477332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8b4c06c870 0x7f8b4c14bf50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.282+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 >> 172.21.15.60:0/3663477332 conn(0x7f8b4c061600 msgr2=0x7f8b4c0b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.283+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 shutdown_connections 2024-04-25T08:30:30.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.283+0000 7f8b51b34640 1 -- 172.21.15.60:0/3663477332 wait complete. 2024-04-25T08:30:30.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:30:30.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.421+0000 7fd38d81f640 1 Processor -- start 2024-04-25T08:30:30.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.421+0000 7fd38d81f640 1 -- start start 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd38d81f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3880b5620 0x7fd3880b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd38d81f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd38806c2d0 0x7fd38806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd38d81f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38806cc80 con 0x7fd38815b7d0 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd38d81f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38806cf50 con 0x7fd3880b5620 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd38d81f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3880ce530 con 0x7fd38806c2d0 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd3867fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3880b5620 0x7fd3880b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd3867fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3880b5620 0x7fd3880b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47808/0 (socket says 172.21.15.60:47808) 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd3867fc640 1 -- 172.21.15.60:0/4145252908 learned_addr learned my addr 172.21.15.60:0/4145252908 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd3877fe640 1 --2- 172.21.15.60:0/4145252908 >> v2:172.21.15.100:3300/0 conn(0x7fd38806c2d0 0x7fd38806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.422+0000 7fd3867fc640 1 -- 172.21.15.60:0/4145252908 >> v2:172.21.15.100:3300/0 conn(0x7fd38806c2d0 msgr2=0x7fd38806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd3867fc640 1 --2- 172.21.15.60:0/4145252908 >> v2:172.21.15.100:3300/0 conn(0x7fd38806c2d0 0x7fd38806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd3867fc640 1 -- 172.21.15.60:0/4145252908 >> v1:172.21.15.100:6789/0 conn(0x7fd38815b7d0 legacy=0x7fd3880cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd3867fc640 1 -- 172.21.15.60:0/4145252908 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880ce810 con 0x7fd3880b5620 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd3867fc640 1 --2- 172.21.15.60:0/4145252908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3880b5ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd388161890 tx=0x7fd37c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d23a6ac2aab080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd385ffb640 1 -- 172.21.15.60:0/4145252908 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c037070 con 0x7fd3880b5620 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd385ffb640 1 -- 172.21.15.60:0/4145252908 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c032440 con 0x7fd3880b5620 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.423+0000 7fd385ffb640 1 -- 172.21.15.60:0/4145252908 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c032810 con 0x7fd3880b5620 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 -- 172.21.15.60:0/4145252908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 msgr2=0x7fd3880b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 --2- 172.21.15.60:0/4145252908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3880b5ab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd388161890 tx=0x7fd37c02a270 comp rx=0 tx=0).stop 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 -- 172.21.15.60:0/4145252908 shutdown_connections 2024-04-25T08:30:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 --2- 172.21.15.60:0/4145252908 >> v2:172.21.15.100:3300/0 conn(0x7fd38806c2d0 0x7fd38806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 --2- 172.21.15.60:0/4145252908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3880b5ab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 -- 172.21.15.60:0/4145252908 >> 172.21.15.60:0/4145252908 conn(0x7fd3880d0800 msgr2=0x7fd3880d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 -- 172.21.15.60:0/4145252908 shutdown_connections 2024-04-25T08:30:30.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.424+0000 7fd38d81f640 1 -- 172.21.15.60:0/4145252908 wait complete. 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.425+0000 7fd38d81f640 1 Processor -- start 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 -- start start 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd38806c2d0 0x7fd3880cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3881502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38815b7d0 0x7fd3880cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38815bbb0 con 0x7fd3880b5620 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38806d230 con 0x7fd38815b7d0 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd38d81f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38815bd30 con 0x7fd38806c2d0 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd3867fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3881502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd3867fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3881502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47818/0 (socket says 172.21.15.60:47818) 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.426+0000 7fd3867fc640 1 -- 172.21.15.60:0/1802877489 learned_addr learned my addr 172.21.15.60:0/1802877489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd386ffd640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd38806c2d0 0x7fd3880cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3877fe640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38815b7d0 0x7fd3880cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3867fc640 1 -- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd38806c2d0 msgr2=0x7fd3880cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3867fc640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd38806c2d0 0x7fd3880cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3867fc640 1 -- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38815b7d0 msgr2=0x7fd3880cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3867fc640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38815b7d0 0x7fd3880cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3867fc640 1 -- 172.21.15.60:0/1802877489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3881507f0 con 0x7fd3880b5620 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd3867fc640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3881502b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02a2b0 tx=0x7fd37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c037040 con 0x7fd3880b5620 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c042070 con 0x7fd3880b5620 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c030880 con 0x7fd3880b5620 2024-04-25T08:30:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3880ccf80 con 0x7fd3880b5620 2024-04-25T08:30:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.427+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3880cae10 con 0x7fd3880b5620 2024-04-25T08:30:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.428+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd388073a00 con 0x7fd3880b5620 2024-04-25T08:30:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.429+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd37c030a20 con 0x7fd3880b5620 2024-04-25T08:30:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.429+0000 7fd36ffff640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd37406b4d0 0x7fd37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.429+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd37c0aabd0 con 0x7fd3880b5620 2024-04-25T08:30:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.429+0000 7fd386ffd640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd37406b4d0 0x7fd37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:30.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.430+0000 7fd386ffd640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd37406b4d0 0x7fd37406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd380007fd0 tx=0x7fd380009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:30.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.434+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd37c079920 con 0x7fd3880b5620 2024-04-25T08:30:30.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:30.650+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd388068070 con 0x7fd3880b5620 2024-04-25T08:30:31.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.321+0000 7fd36ffff640 1 -- 172.21.15.60:0/1802877489 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd37c079220 con 0x7fd3880b5620 2024-04-25T08:30:31.320 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T08:30:31.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.327+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd37406b4d0 msgr2=0x7fd37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.327+0000 7fd38d81f640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd37406b4d0 0x7fd37406d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd380007fd0 tx=0x7fd380009040 comp rx=0 tx=0).stop 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.327+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 msgr2=0x7fd3881502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.327+0000 7fd38d81f640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3881502b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd37c02a2b0 tx=0x7fd37c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.329+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 shutdown_connections 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.329+0000 7fd38d81f640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd37406b4d0 0x7fd37406d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.329+0000 7fd38d81f640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38815b7d0 0x7fd3880cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.329+0000 7fd38d81f640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3880b5620 0x7fd3881502b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.329+0000 7fd38d81f640 1 --2- 172.21.15.60:0/1802877489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd38806c2d0 0x7fd3880cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.329+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 >> 172.21.15.60:0/1802877489 conn(0x7fd3880d0800 msgr2=0x7fd3880b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:31.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.330+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 shutdown_connections 2024-04-25T08:30:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.333+0000 7fd38d81f640 1 -- 172.21.15.60:0/1802877489 wait complete. 2024-04-25T08:30:31.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.456+0000 7f1cb9a7d640 1 Processor -- start 2024-04-25T08:30:31.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.456+0000 7f1cb9a7d640 1 -- start start 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb9a7d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1cb415dd30 0x7f1cb40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb9a7d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b9770 0x7f1cb40d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb9a7d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb40d1df0 con 0x7f1cb40d2e90 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb9a7d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb40b7a50 con 0x7f1cb415dd30 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb9a7d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40d20d0 con 0x7f1cb40b9770 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1cb40d2e90 0x7f1cb40b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52838/0 (socket says 172.21.15.60:52838) 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb3fff640 1 -- 172.21.15.60:0/205389132 learned_addr learned my addr 172.21.15.60:0/205389132 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb2ffd640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b9770 0x7f1cb40d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb37fe640 1 --2- 172.21.15.60:0/205389132 >> v2:172.21.15.100:3300/0 conn(0x7f1cb415dd30 0x7f1cb40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956844424 0 0) 0x7f1cb40d1df0 con 0x7f1cb40d2e90 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c9c003520 con 0x7f1cb40d2e90 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430105623 0 0) 0x7f1c9c003520 con 0x7f1cb40d2e90 2024-04-25T08:30:31.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b9770 msgr2=0x7f1cb40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b9770 0x7f1cb40d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 >> v2:172.21.15.100:3300/0 conn(0x7f1cb415dd30 msgr2=0x7f1cb40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 --2- 172.21.15.60:0/205389132 >> v2:172.21.15.100:3300/0 conn(0x7f1cb415dd30 0x7f1cb40b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb40d23b0 con 0x7f1cb40d2e90 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1cac0019f0 con 0x7f1cb40d2e90 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 >> v1:172.21.15.60:6789/0 conn(0x7f1cb40d2e90 legacy=0x7f1cb40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c9c0039b0 0x7f1c9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c0043a0 0x7f1c9c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c9c004f90 0x7f1c9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40b7a50 con 0x7f1c9c004f90 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40d20d0 con 0x7f1c9c0043a0 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c00bf30 con 0x7f1c9c0039b0 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb37fe640 1 --2- 172.21.15.60:0/205389132 >> v2:172.21.15.100:3300/0 conn(0x7f1cb415dd30 0x7f1cb40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.457+0000 7f1cb2ffd640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b9770 0x7f1cb40d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c9c004f90 0x7f1c9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb2ffd640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c0043a0 0x7f1c9c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb37fe640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c9c0039b0 0x7f1c9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 -- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c9c0039b0 msgr2=0x7f1c9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c9c0039b0 0x7f1c9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 -- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c0043a0 msgr2=0x7f1c9c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c9c0043a0 0x7f1c9c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 -- 172.21.15.60:0/205389132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c003520 con 0x7f1c9c004f90 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.458+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c9c004f90 0x7f1c9c00b9c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00a920 tx=0x7f1c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a4cd8ba2d274af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00a4c0 con 0x7f1c9c004f90 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac02f070 con 0x7f1c9c004f90 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb27fc640 1 -- 172.21.15.60:0/205389132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac02bd40 con 0x7f1c9c004f90 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/205389132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c9c004f90 msgr2=0x7f1c9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/205389132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c9c004f90 0x7f1c9c00b9c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00a920 tx=0x7f1c9c005830 comp rx=0 tx=0).stop 2024-04-25T08:30:31.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb3fff640 1 -- 172.21.15.60:0/205389132 reap_dead start 2024-04-25T08:30:31.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/205389132 shutdown_connections 2024-04-25T08:30:31.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/205389132 >> 172.21.15.60:0/205389132 conn(0x7f1cb4061c30 msgr2=0x7f1cb406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:31.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/205389132 shutdown_connections 2024-04-25T08:30:31.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.459+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/205389132 wait complete. 2024-04-25T08:30:31.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.460+0000 7f1cb9a7d640 1 Processor -- start 2024-04-25T08:30:31.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 -- start start 2024-04-25T08:30:31.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cb40b1c10 0x7f1cb40ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b1ff0 0x7f1cb40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 0x7f1cb4150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40d1a10 con 0x7f1cb414fc50 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40d2970 con 0x7f1cb40b1c10 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.461+0000 7f1cb9a7d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb40d1b90 con 0x7f1cb40b1ff0 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 0x7f1cb4150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 0x7f1cb4150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:47844/0 (socket says 172.21.15.60:47844) 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 -- 172.21.15.60:0/1464033453 learned_addr learned my addr 172.21.15.60:0/1464033453 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb2ffd640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b1ff0 0x7f1cb40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb37fe640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cb40b1c10 0x7f1cb40ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 -- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b1ff0 msgr2=0x7f1cb40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b1ff0 0x7f1cb40cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 -- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cb40b1c10 msgr2=0x7f1cb40ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cb40b1c10 0x7f1cb40ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 -- 172.21.15.60:0/1464033453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4150980 con 0x7f1cb414fc50 2024-04-25T08:30:31.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb3fff640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 0x7f1cb4150100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00a920 tx=0x7f1cac001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00a4c0 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac034470 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac002000 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb414daa0 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.462+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb414e060 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.463+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb4073990 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.464+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cac034820 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.464+0000 7f1c9bfff640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b500 0x7f1ca006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.464+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cac0a9f70 con 0x7f1cb414fc50 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.464+0000 7f1cb37fe640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b500 0x7f1ca006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.465+0000 7f1cb37fe640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b500 0x7f1ca006d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40093d0 tx=0x7f1ca4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:31.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.469+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cac046060 con 0x7f1cb414fc50 2024-04-25T08:30:31.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.650+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cb40ce670 con 0x7f1cb414fc50 2024-04-25T08:30:31.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.651+0000 7f1c9bfff640 1 -- 172.21.15.60:0/1464033453 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f1cac0786a0 con 0x7f1cb414fc50 2024-04-25T08:30:31.648 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:31.649 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":38,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:30.652380+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b500 msgr2=0x7f1ca006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b500 0x7f1ca006d9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40093d0 tx=0x7f1ca4009190 comp rx=0 tx=0).stop 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 msgr2=0x7f1cb4150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 0x7f1cb4150100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00a920 tx=0x7f1cac001dc0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 shutdown_connections 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cb414fc50 0x7f1cb4150100 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cb40b1ff0 0x7f1cb40cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b500 0x7f1ca006d9c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 --2- 172.21.15.60:0/1464033453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cb40b1c10 0x7f1cb40ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 >> 172.21.15.60:0/1464033453 conn(0x7f1cb4061c30 msgr2=0x7f1cb40d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 shutdown_connections 2024-04-25T08:30:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.653+0000 7f1cb9a7d640 1 -- 172.21.15.60:0/1464033453 wait complete. 2024-04-25T08:30:31.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.765+0000 7f275d99e640 1 Processor -- start 2024-04-25T08:30:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.765+0000 7f275d99e640 1 -- start start 2024-04-25T08:30:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f275d99e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f275815dd30 0x7f27580b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f275d99e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27580d0ad0 0x7f27580b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f275d99e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27580d1df0 con 0x7f27580b9770 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f275d99e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580d20d0 con 0x7f275815dd30 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f275d99e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580d23b0 con 0x7f27580d0ad0 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f275815dd30 0x7f27580b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f275815dd30 0x7f27580b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45816/0 (socket says 172.21.15.60:45816) 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 -- 172.21.15.60:0/2417708692 learned_addr learned my addr 172.21.15.60:0/2417708692 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f27577fe640 1 --2- 172.21.15.60:0/2417708692 >> v2:172.21.15.100:3300/0 conn(0x7f27580d0ad0 0x7f27580b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 -- 172.21.15.60:0/2417708692 >> v2:172.21.15.100:3300/0 conn(0x7f27580d0ad0 msgr2=0x7f27580b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 --2- 172.21.15.60:0/2417708692 >> v2:172.21.15.100:3300/0 conn(0x7f27580d0ad0 0x7f27580b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 -- 172.21.15.60:0/2417708692 >> v1:172.21.15.100:6789/0 conn(0x7f27580b9770 legacy=0x7f27580d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:31.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f2756ffd640 1 -- 172.21.15.60:0/2417708692 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27580d2690 con 0x7f275815dd30 2024-04-25T08:30:31.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.766+0000 7f27577fe640 1 --2- 172.21.15.60:0/2417708692 >> v2:172.21.15.100:3300/0 conn(0x7f27580d0ad0 0x7f27580b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f2756ffd640 1 --2- 172.21.15.60:0/2417708692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f275815dd30 0x7f27580b9230 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f27580ce200 tx=0x7f274802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e794b7328abf8bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f2755ffb640 1 -- 172.21.15.60:0/2417708692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748037070 con 0x7f275815dd30 2024-04-25T08:30:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f2755ffb640 1 -- 172.21.15.60:0/2417708692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274802ae10 con 0x7f275815dd30 2024-04-25T08:30:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f2755ffb640 1 -- 172.21.15.60:0/2417708692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748032660 con 0x7f275815dd30 2024-04-25T08:30:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f275d99e640 1 -- 172.21.15.60:0/2417708692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f275815dd30 msgr2=0x7f27580b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f275d99e640 1 --2- 172.21.15.60:0/2417708692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f275815dd30 0x7f27580b9230 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f27580ce200 tx=0x7f274802a100 comp rx=0 tx=0).stop 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f275d99e640 1 -- 172.21.15.60:0/2417708692 shutdown_connections 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f275d99e640 1 --2- 172.21.15.60:0/2417708692 >> v2:172.21.15.100:3300/0 conn(0x7f27580d0ad0 0x7f27580b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.767+0000 7f275d99e640 1 --2- 172.21.15.60:0/2417708692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f275815dd30 0x7f27580b9230 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.768+0000 7f275d99e640 1 -- 172.21.15.60:0/2417708692 >> 172.21.15.60:0/2417708692 conn(0x7f2758061c30 msgr2=0x7f275806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.768+0000 7f275d99e640 1 -- 172.21.15.60:0/2417708692 shutdown_connections 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.768+0000 7f275d99e640 1 -- 172.21.15.60:0/2417708692 wait complete. 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.769+0000 7f275d99e640 1 Processor -- start 2024-04-25T08:30:31.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.769+0000 7f275d99e640 1 -- start start 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27580b9770 0x7f27581f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 0x7f2758155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275815dd30 0x7f27580c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580d1b90 con 0x7f27580d0ad0 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580d1a10 con 0x7f275815dd30 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580d2e90 con 0x7f27580b9770 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 0x7f2758155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 0x7f2758155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45830/0 (socket says 172.21.15.60:45830) 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 -- 172.21.15.60:0/4086994464 learned_addr learned my addr 172.21.15.60:0/4086994464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27577fe640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275815dd30 0x7f27580c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f2756ffd640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27580b9770 0x7f27581f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 -- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27580b9770 msgr2=0x7f27581f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27580b9770 0x7f27581f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 -- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275815dd30 msgr2=0x7f27580c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275815dd30 0x7f27580c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 -- 172.21.15.60:0/4086994464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758156050 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f27567fc640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 0x7f2758155670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f274c004640 tx=0x7f274c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c011790 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c017070 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c011b70 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2758156330 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.770+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2758153380 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.771+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27580b6790 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.772+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f274c011d10 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.772+0000 7f273ffff640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f274406b4d0 0x7f274406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.772+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f274c089f40 con 0x7f27580d0ad0 2024-04-25T08:30:31.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.773+0000 7f2756ffd640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f274406b4d0 0x7f274406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:31.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.773+0000 7f2756ffd640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f274406b4d0 0x7f274406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2748034000 tx=0x7f274802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:31.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.777+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27580b6790 con 0x7f27580d0ad0 2024-04-25T08:30:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.956+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2758073990 con 0x7f27580d0ad0 2024-04-25T08:30:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.957+0000 7f273ffff640 1 -- 172.21.15.60:0/4086994464 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f274c058c30 con 0x7f27580d0ad0 2024-04-25T08:30:31.955 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:31.955 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":38,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:30.652380+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f274406b4d0 msgr2=0x7f274406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f274406b4d0 0x7f274406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2748034000 tx=0x7f274802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 msgr2=0x7f2758155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 0x7f2758155670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f274c004640 tx=0x7f274c00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 shutdown_connections 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f274406b4d0 0x7f274406d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f275815dd30 0x7f27580c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27580d0ad0 0x7f2758155670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 --2- 172.21.15.60:0/4086994464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27580b9770 0x7f27581f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 >> 172.21.15.60:0/4086994464 conn(0x7f2758061c30 msgr2=0x7f27580c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 shutdown_connections 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:31.959+0000 7f275d99e640 1 -- 172.21.15.60:0/4086994464 wait complete. 2024-04-25T08:30:31.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:30:32.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.075+0000 7feeee412640 1 Processor -- start 2024-04-25T08:30:32.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.075+0000 7feeee412640 1 -- start start 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feeee412640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feee8161710 0x7feee80d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feeee412640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee815e000 con 0x7feee80ced80 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feeee412640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80d1de0 con 0x7feee815dc20 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feeee412640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee8161c60 con 0x7feee8161710 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feee80ced80 0x7feee80d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53468/0 (socket says 172.21.15.60:53468) 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee77fe640 1 -- 172.21.15.60:0/3000308322 learned_addr learned my addr 172.21.15.60:0/3000308322 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feeec9af640 1 --2- 172.21.15.60:0/3000308322 >> v2:172.21.15.100:3300/0 conn(0x7feee8161710 0x7feee80d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850498263 0 0) 0x7feee815e000 con 0x7feee80ced80 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed0003520 con 0x7feee80ced80 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834691147 0 0) 0x7feed0003520 con 0x7feee80ced80 2024-04-25T08:30:32.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 >> v2:172.21.15.100:3300/0 conn(0x7feee8161710 msgr2=0x7feee80d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 --2- 172.21.15.60:0/3000308322 >> v2:172.21.15.100:3300/0 conn(0x7feee8161710 0x7feee80d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 >> v1:172.21.15.100:6789/0 conn(0x7feee815dc20 legacy=0x7feee80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8161f40 con 0x7feee80ced80 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7feedc001670 con 0x7feee80ced80 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 >> v1:172.21.15.60:6789/0 conn(0x7feee80ced80 legacy=0x7feee80d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00039b0 0x7feed0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed0004800 0x7feed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00051c0 0x7feed000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8161c60 con 0x7feed00051c0 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed000c060 con 0x7feed0004800 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.076+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed000c260 con 0x7feed00039b0 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00051c0 0x7feed000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feee77fe640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed0004800 0x7feed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feee7fff640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00039b0 0x7feed0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 -- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00039b0 msgr2=0x7feed0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00039b0 0x7feed0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 -- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed0004800 msgr2=0x7feed0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feed0004800 0x7feed0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 -- 172.21.15.60:0/3000308322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed0003520 con 0x7feed00051c0 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feee7fff640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00039b0 0x7feed0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.077+0000 7feeec9af640 1 -- 172.21.15.60:0/3000308322 reap_dead start 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeec9af640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00051c0 0x7feed000b9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feee0009fc0 tx=0x7feed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646020d3b49766d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0035020 con 0x7feed00051c0 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0003b90 con 0x7feed00051c0 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feee6ffd640 1 -- 172.21.15.60:0/3000308322 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0009c10 con 0x7feed00051c0 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 -- 172.21.15.60:0/3000308322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00051c0 msgr2=0x7feed000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00051c0 0x7feed000b9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7feee0009fc0 tx=0x7feed0005830 comp rx=0 tx=0).stop 2024-04-25T08:30:32.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 -- 172.21.15.60:0/3000308322 shutdown_connections 2024-04-25T08:30:32.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 --2- 172.21.15.60:0/3000308322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feed00051c0 0x7feed000b9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 -- 172.21.15.60:0/3000308322 >> 172.21.15.60:0/3000308322 conn(0x7feee80d0420 msgr2=0x7feee80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:32.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 -- 172.21.15.60:0/3000308322 shutdown_connections 2024-04-25T08:30:32.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.078+0000 7feeee412640 1 -- 172.21.15.60:0/3000308322 wait complete. 2024-04-25T08:30:32.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.079+0000 7feeee412640 1 Processor -- start 2024-04-25T08:30:32.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.080+0000 7feeee412640 1 -- start start 2024-04-25T08:30:32.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.080+0000 7feeee412640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00051c0 0x7feee81f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.080+0000 7feeee412640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee8160900 0x7feee80c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeee412640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 0x7feee8154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeee412640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee80b5e50 con 0x7feee8154790 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeee412640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee8162500 con 0x7feee8160900 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeee412640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee80b5fd0 con 0x7feed00051c0 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 0x7feee8154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 0x7feee8154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45850/0 (socket says 172.21.15.60:45850) 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 -- 172.21.15.60:0/37776296 learned_addr learned my addr 172.21.15.60:0/37776296 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feee77fe640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee8160900 0x7feee80c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feee7fff640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00051c0 0x7feee81f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 -- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00051c0 msgr2=0x7feee81f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00051c0 0x7feee81f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 -- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee8160900 msgr2=0x7feee80c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee8160900 0x7feee80c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 -- 172.21.15.60:0/37776296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee81f4460 con 0x7feee8154790 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeec9af640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 0x7feee8154c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feee000e230 tx=0x7feee0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee003f030 con 0x7feee8154790 2024-04-25T08:30:32.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0003990 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0035070 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee80c7430 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.081+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee80c7cd0 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.083+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee8073a00 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.083+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feee0042030 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.083+0000 7feee4ff9640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feed406b4d0 0x7feed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.083+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feee00aafb0 con 0x7feee8154790 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.083+0000 7feee7fff640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feed406b4d0 0x7feed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.084+0000 7feee7fff640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feed406b4d0 0x7feed406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed8000c00 tx=0x7feed8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:32.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.088+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feee81f4460 con 0x7feee8154790 2024-04-25T08:30:32.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.304+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feee80c77c0 con 0x7feee8154790 2024-04-25T08:30:32.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.320+0000 7feee4ff9640 1 -- 172.21.15.60:0/37776296 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7feee0079650 con 0x7feee8154790 2024-04-25T08:30:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feed406b4d0 msgr2=0x7feed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feed406b4d0 0x7feed406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feed8000c00 tx=0x7feed8009040 comp rx=0 tx=0).stop 2024-04-25T08:30:32.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 msgr2=0x7feee8154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 0x7feee8154c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feee000e230 tx=0x7feee0009750 comp rx=0 tx=0).stop 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 shutdown_connections 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8154790 0x7feee8154c40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feed406b4d0 0x7feed406d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feee8160900 0x7feee80c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 --2- 172.21.15.60:0/37776296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feed00051c0 0x7feee81f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 >> 172.21.15.60:0/37776296 conn(0x7feee80d0420 msgr2=0x7feee806c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 shutdown_connections 2024-04-25T08:30:32.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.323+0000 7feeee412640 1 -- 172.21.15.60:0/37776296 wait complete. 2024-04-25T08:30:32.331 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:30:32.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:30:32.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.438+0000 7f1210de7640 1 Processor -- start 2024-04-25T08:30:32.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.438+0000 7f1210de7640 1 -- start start 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1210de7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c0b9750 0x7f120c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1210de7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f120c0cebb0 0x7f120c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1210de7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c161d70 con 0x7f120c15da90 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1210de7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120c162600 con 0x7f120c0cebb0 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1210de7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0cf020 con 0x7f120c0b9750 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120ad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f120c0cebb0 0x7f120c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f120c15da90 0x7f120c161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53476/0 (socket says 172.21.15.60:53476) 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120a59c640 1 -- 172.21.15.60:0/3119405896 learned_addr learned my addr 172.21.15.60:0/3119405896 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1203fff640 1 --2- 172.21.15.60:0/3119405896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c0b9750 0x7f120c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120ad9d640 1 -- 172.21.15.60:0/3119405896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c0b9750 msgr2=0x7f120c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:32.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120ad9d640 1 --2- 172.21.15.60:0/3119405896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c0b9750 0x7f120c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120ad9d640 1 -- 172.21.15.60:0/3119405896 >> v1:172.21.15.60:6789/0 conn(0x7f120c15da90 legacy=0x7f120c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f120ad9d640 1 -- 172.21.15.60:0/3119405896 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c162040 con 0x7f120c0cebb0 2024-04-25T08:30:32.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.439+0000 7f1209d9b640 1 -- 172.21.15.60:0/3119405896 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017824872 0 0) 0x7f120c161d70 con 0x7f120c15da90 2024-04-25T08:30:32.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.440+0000 7f120ad9d640 1 --2- 172.21.15.60:0/3119405896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0d2310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1204009d90 tx=0x7f120402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52e605b9aba7a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:32.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.440+0000 7f1209d9b640 1 -- 172.21.15.60:0/3119405896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204031600 con 0x7f120c0cebb0 2024-04-25T08:30:32.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.440+0000 7f1209d9b640 1 -- 172.21.15.60:0/3119405896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12040317a0 con 0x7f120c0cebb0 2024-04-25T08:30:32.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.440+0000 7f1209d9b640 1 -- 172.21.15.60:0/3119405896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204031b70 con 0x7f120c0cebb0 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.440+0000 7f1210de7640 1 -- 172.21.15.60:0/3119405896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 msgr2=0x7f120c0d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.440+0000 7f1210de7640 1 --2- 172.21.15.60:0/3119405896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0d2310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1204009d90 tx=0x7f120402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.441+0000 7f1210de7640 1 -- 172.21.15.60:0/3119405896 shutdown_connections 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.441+0000 7f1210de7640 1 --2- 172.21.15.60:0/3119405896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0d2310 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.441+0000 7f1210de7640 1 --2- 172.21.15.60:0/3119405896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c0b9750 0x7f120c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.441+0000 7f1210de7640 1 -- 172.21.15.60:0/3119405896 >> 172.21.15.60:0/3119405896 conn(0x7f120c061a20 msgr2=0x7f120c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.441+0000 7f1210de7640 1 -- 172.21.15.60:0/3119405896 shutdown_connections 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.441+0000 7f1210de7640 1 -- 172.21.15.60:0/3119405896 wait complete. 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.442+0000 7f1210de7640 1 Processor -- start 2024-04-25T08:30:32.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.442+0000 7f1210de7640 1 -- start start 2024-04-25T08:30:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15da90 0x7f120c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c152e30 0x7f120c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0d1df0 con 0x7f120c0cebb0 2024-04-25T08:30:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0d1b10 con 0x7f120c15da90 2024-04-25T08:30:32.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15de70 con 0x7f120c152e30 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45866/0 (socket says 172.21.15.60:45866) 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 -- 172.21.15.60:0/1750046822 learned_addr learned my addr 172.21.15.60:0/1750046822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1203fff640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15da90 0x7f120c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120ad9d640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c152e30 0x7f120c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 -- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c152e30 msgr2=0x7f120c15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c152e30 0x7f120c15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 -- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15da90 msgr2=0x7f120c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15da90 0x7f120c0cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 -- 172.21.15.60:0/1750046822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c15e2b0 con 0x7f120c0cebb0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f120a59c640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0ccd80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c430 tx=0x7f11fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc0074e0 con 0x7f120c0cebb0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc0047e0 con 0x7f120c0cebb0 2024-04-25T08:30:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc004bd0 con 0x7f120c0cebb0 2024-04-25T08:30:32.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.443+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c15e540 con 0x7f120c0cebb0 2024-04-25T08:30:32.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.444+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c15eb00 con 0x7f120c0cebb0 2024-04-25T08:30:32.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.445+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c073a00 con 0x7f120c0cebb0 2024-04-25T08:30:32.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.445+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11fc011070 con 0x7f120c0cebb0 2024-04-25T08:30:32.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.445+0000 7f1202ffd640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b500 0x7f11ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:32.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.446+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11fc08a000 con 0x7f120c0cebb0 2024-04-25T08:30:32.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.446+0000 7f1203fff640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b500 0x7f11ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:32.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.446+0000 7f1203fff640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b500 0x7f11ec06d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f120c0b29f0 tx=0x7f11f4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:32.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.450+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11fc058cf0 con 0x7f120c0cebb0 2024-04-25T08:30:32.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:32.669+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f120c062010 con 0x7f120c0cebb0 2024-04-25T08:30:33.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.322+0000 7f1202ffd640 1 -- 172.21.15.60:0/1750046822 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11fc0585f0 con 0x7f120c0cebb0 2024-04-25T08:30:33.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b500 msgr2=0x7f11ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b500 0x7f11ec06d9c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f120c0b29f0 tx=0x7f11f4008040 comp rx=0 tx=0).stop 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 msgr2=0x7f120c0ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0ccd80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c430 tx=0x7f11fc00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 shutdown_connections 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f120c152e30 0x7f120c15b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b500 0x7f11ec06d9c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f120c15da90 0x7f120c0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 --2- 172.21.15.60:0/1750046822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f120c0cebb0 0x7f120c0ccd80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 >> 172.21.15.60:0/1750046822 conn(0x7f120c061a20 msgr2=0x7f120c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 shutdown_connections 2024-04-25T08:30:33.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.325+0000 7f1210de7640 1 -- 172.21.15.60:0/1750046822 wait complete. 2024-04-25T08:30:33.335 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:30:33.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:30:33.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.446+0000 7fe177fc1640 1 Processor -- start 2024-04-25T08:30:33.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.446+0000 7fe177fc1640 1 -- start start 2024-04-25T08:30:33.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe177fc1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700aecf0 0x7fe1700cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe177fc1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1700b2650 0x7fe1700cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe177fc1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1700cf330 con 0x7fe170165680 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe177fc1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe170072340 con 0x7fe1700b2650 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe177fc1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe170071670 con 0x7fe1700aecf0 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1700b2650 0x7fe1700cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1700b2650 0x7fe1700cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45878/0 (socket says 172.21.15.60:45878) 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 -- 172.21.15.60:0/3853059333 learned_addr learned my addr 172.21.15.60:0/3853059333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17555c640 1 --2- 172.21.15.60:0/3853059333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700aecf0 0x7fe1700cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 -- 172.21.15.60:0/3853059333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700aecf0 msgr2=0x7fe1700cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 --2- 172.21.15.60:0/3853059333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700aecf0 0x7fe1700cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 -- 172.21.15.60:0/3853059333 >> v1:172.21.15.100:6789/0 conn(0x7fe170165680 legacy=0x7fe1700cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.447+0000 7fe17655e640 1 -- 172.21.15.60:0/3853059333 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170071d80 con 0x7fe1700b2650 2024-04-25T08:30:33.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.448+0000 7fe17655e640 1 --2- 172.21.15.60:0/3853059333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1700b2650 0x7fe1700cf580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe168009e50 tx=0x7fe16802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d01d1be1112bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:33.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.448+0000 7fe174d5b640 1 -- 172.21.15.60:0/3853059333 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168036070 con 0x7fe1700b2650 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.448+0000 7fe174d5b640 1 -- 172.21.15.60:0/3853059333 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168031600 con 0x7fe1700b2650 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.448+0000 7fe174d5b640 1 -- 172.21.15.60:0/3853059333 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1680319d0 con 0x7fe1700b2650 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 -- 172.21.15.60:0/3853059333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1700b2650 msgr2=0x7fe1700cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 --2- 172.21.15.60:0/3853059333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1700b2650 0x7fe1700cf580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe168009e50 tx=0x7fe16802a680 comp rx=0 tx=0).stop 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 -- 172.21.15.60:0/3853059333 shutdown_connections 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 --2- 172.21.15.60:0/3853059333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1700b2650 0x7fe1700cf580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 --2- 172.21.15.60:0/3853059333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700aecf0 0x7fe1700cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 -- 172.21.15.60:0/3853059333 >> 172.21.15.60:0/3853059333 conn(0x7fe17006abe0 msgr2=0x7fe17006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 -- 172.21.15.60:0/3853059333 shutdown_connections 2024-04-25T08:30:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.449+0000 7fe177fc1640 1 -- 172.21.15.60:0/3853059333 wait complete. 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.450+0000 7fe177fc1640 1 Processor -- start 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.450+0000 7fe177fc1640 1 -- start start 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe177fc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1700aecf0 0x7fe17014d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe177fc1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700b2650 0x7fe17015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe177fc1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 0x7fe17015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe177fc1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe170165a60 con 0x7fe170165680 2024-04-25T08:30:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe177fc1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17006ea30 con 0x7fe1700aecf0 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe177fc1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b2a80 con 0x7fe1700b2650 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 0x7fe17015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 0x7fe17015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45884/0 (socket says 172.21.15.60:45884) 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 -- 172.21.15.60:0/985281761 learned_addr learned my addr 172.21.15.60:0/985281761 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe175d5d640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1700aecf0 0x7fe17014d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17555c640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700b2650 0x7fe17015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 -- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700b2650 msgr2=0x7fe17015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700b2650 0x7fe17015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 -- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1700aecf0 msgr2=0x7fe17014d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1700aecf0 0x7fe17014d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 -- 172.21.15.60:0/985281761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17014ddd0 con 0x7fe170165680 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe17655e640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 0x7fe17015ac60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe168034040 tx=0x7fe168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.451+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168036050 con 0x7fe170165680 2024-04-25T08:30:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.452+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168042070 con 0x7fe170165680 2024-04-25T08:30:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.452+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17015fee0 con 0x7fe170165680 2024-04-25T08:30:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.452+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168009510 con 0x7fe170165680 2024-04-25T08:30:33.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.452+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe170160180 con 0x7fe170165680 2024-04-25T08:30:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.453+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe170073990 con 0x7fe170165680 2024-04-25T08:30:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.453+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe168007bd0 con 0x7fe170165680 2024-04-25T08:30:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.453+0000 7fe1667fc640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe15406b4d0 0x7fe15406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.453+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1680ab250 con 0x7fe170165680 2024-04-25T08:30:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.454+0000 7fe175d5d640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe15406b4d0 0x7fe15406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.454+0000 7fe175d5d640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe15406b4d0 0x7fe15406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe16c007fd0 tx=0x7fe16c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.458+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe17014ddd0 con 0x7fe170165680 2024-04-25T08:30:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.675+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe17015cd80 con 0x7fe170165680 2024-04-25T08:30:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.676+0000 7fe1667fc640 1 -- 172.21.15.60:0/985281761 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe168045360 con 0x7fe170165680 2024-04-25T08:30:33.673 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:33.673 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:30:30.652688+0000","modified":"2024-04-25T08:30:32.670840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2407433231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2407433231},{"type":"v1","addr":"172.21.15.60:6835","nonce":2407433231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:30:33.673 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T08:30:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.677+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe15406b4d0 msgr2=0x7fe15406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.677+0000 7fe177fc1640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe15406b4d0 0x7fe15406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe16c007fd0 tx=0x7fe16c009040 comp rx=0 tx=0).stop 2024-04-25T08:30:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.677+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 msgr2=0x7fe17015ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.677+0000 7fe177fc1640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 0x7fe17015ac60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe168034040 tx=0x7fe168004a40 comp rx=0 tx=0).stop 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 shutdown_connections 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe170165680 0x7fe17015ac60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1700b2650 0x7fe17015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe15406b4d0 0x7fe15406d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 --2- 172.21.15.60:0/985281761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1700aecf0 0x7fe17014d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 >> 172.21.15.60:0/985281761 conn(0x7fe17006abe0 msgr2=0x7fe1700b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 shutdown_connections 2024-04-25T08:30:33.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.678+0000 7fe177fc1640 1 -- 172.21.15.60:0/985281761 wait complete. 2024-04-25T08:30:33.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:30:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.788+0000 7f732021f640 1 Processor -- start 2024-04-25T08:30:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.789+0000 7f732021f640 1 -- start start 2024-04-25T08:30:33.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.789+0000 7f732021f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f731815db70 0x7f7318162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.789+0000 7f732021f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7318162690 con 0x7f73180b5c30 2024-04-25T08:30:33.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f732021f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73180ceee0 con 0x7f73180b6cb0 2024-04-25T08:30:33.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f732021f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73180d1d10 con 0x7f731815db70 2024-04-25T08:30:33.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f731dfbb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f731815db70 0x7f7318162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f731e7bc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f73180b5c30 0x7f73180d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53488/0 (socket says 172.21.15.60:53488) 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f731e7bc640 1 -- 172.21.15.60:0/1520989393 learned_addr learned my addr 172.21.15.60:0/1520989393 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f731dfbb640 1 -- 172.21.15.60:0/1520989393 >> v1:172.21.15.100:6789/0 conn(0x7f73180b6cb0 legacy=0x7f73180b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f731dfbb640 1 -- 172.21.15.60:0/1520989393 >> v1:172.21.15.60:6789/0 conn(0x7f73180b5c30 legacy=0x7f73180d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.790+0000 7f731dfbb640 1 -- 172.21.15.60:0/1520989393 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180d1ff0 con 0x7f731815db70 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f731dfbb640 1 --2- 172.21.15.60:0/1520989393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f731815db70 0x7f7318162150 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f73100097a0 tx=0x7f731002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dea3206bd687fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f731cfb9640 1 -- 172.21.15.60:0/1520989393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310037070 con 0x7f731815db70 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f731cfb9640 1 -- 172.21.15.60:0/1520989393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7310032440 con 0x7f731815db70 2024-04-25T08:30:33.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f731cfb9640 1 -- 172.21.15.60:0/1520989393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310032810 con 0x7f731815db70 2024-04-25T08:30:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 -- 172.21.15.60:0/1520989393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f731815db70 msgr2=0x7f7318162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 --2- 172.21.15.60:0/1520989393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f731815db70 0x7f7318162150 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f73100097a0 tx=0x7f731002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 -- 172.21.15.60:0/1520989393 shutdown_connections 2024-04-25T08:30:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 --2- 172.21.15.60:0/1520989393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f731815db70 0x7f7318162150 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 -- 172.21.15.60:0/1520989393 >> 172.21.15.60:0/1520989393 conn(0x7f73180d0520 msgr2=0x7f73180d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 -- 172.21.15.60:0/1520989393 shutdown_connections 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.791+0000 7f732021f640 1 -- 172.21.15.60:0/1520989393 wait complete. 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 Processor -- start 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 -- start start 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 0x7f73180ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73180b6cb0 0x7f7318158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731815db70 0x7f73180b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73180ce530 con 0x7f73180b5c30 2024-04-25T08:30:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731815df50 con 0x7f731815db70 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f732021f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73180cf1c0 con 0x7f73180b6cb0 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f731dfbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 0x7f73180ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f731dfbb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 0x7f73180ac150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45900/0 (socket says 172.21.15.60:45900) 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.793+0000 7f731dfbb640 1 -- 172.21.15.60:0/3681108623 learned_addr learned my addr 172.21.15.60:0/3681108623 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f731dfbb640 1 -- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73180b6cb0 msgr2=0x7f7318158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f731dfbb640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73180b6cb0 0x7f7318158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f731dfbb640 1 -- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731815db70 msgr2=0x7f73180b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f731dfbb640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731815db70 0x7f73180b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f731dfbb640 1 -- 172.21.15.60:0/3681108623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180ac690 con 0x7f73180b5c30 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f731dfbb640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 0x7f73180ac150 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f73100098c0 tx=0x7f73100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:33.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310041070 con 0x7f73180b5c30 2024-04-25T08:30:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731003c870 con 0x7f73180b5c30 2024-04-25T08:30:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310037040 con 0x7f73180b5c30 2024-04-25T08:30:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73180aa350 con 0x7f73180b5c30 2024-04-25T08:30:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.794+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73180aabf0 con 0x7f73180b5c30 2024-04-25T08:30:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.795+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73180cf7d0 con 0x7f73180b5c30 2024-04-25T08:30:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.795+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7310044070 con 0x7f73180b5c30 2024-04-25T08:30:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.796+0000 7f730e7fc640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72fc06b4d0 0x7f72fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.796+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73100aae50 con 0x7f73180b5c30 2024-04-25T08:30:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.796+0000 7f731d7ba640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72fc06b4d0 0x7f72fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:33.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.797+0000 7f731d7ba640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72fc06b4d0 0x7f72fc06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73180cca00 tx=0x7f7308007990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:33.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.800+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73180cf7d0 con 0x7f73180b5c30 2024-04-25T08:30:33.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.982+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73180d1820 con 0x7f73180b5c30 2024-04-25T08:30:33.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.983+0000 7f730e7fc640 1 -- 172.21.15.60:0/3681108623 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f7310079cf0 con 0x7f73180b5c30 2024-04-25T08:30:33.981 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:33.981 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:30:32.306408+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72fc06b4d0 msgr2=0x7f72fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72fc06b4d0 0x7f72fc06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73180cca00 tx=0x7f7308007990 comp rx=0 tx=0).stop 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 msgr2=0x7f73180ac150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 0x7f73180ac150 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f73100098c0 tx=0x7f73100046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 shutdown_connections 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72fc06b4d0 0x7f72fc06d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f731815db70 0x7f73180b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f73180b6cb0 0x7f7318158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 --2- 172.21.15.60:0/3681108623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f73180b5c30 0x7f73180ac150 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 >> 172.21.15.60:0/3681108623 conn(0x7f73180d0520 msgr2=0x7f731815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:33.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 shutdown_connections 2024-04-25T08:30:33.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:33.985+0000 7f732021f640 1 -- 172.21.15.60:0/3681108623 wait complete. 2024-04-25T08:30:33.993 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:30:33.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:30:34.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.096+0000 7f6ef006b640 1 Processor -- start 2024-04-25T08:30:34.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.097+0000 7f6ef006b640 1 -- start start 2024-04-25T08:30:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.097+0000 7f6ef006b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ee80b6cb0 0x7f6ee80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.097+0000 7f6ef006b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80ceee0 con 0x7f6ee815db70 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.097+0000 7f6ef006b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80d1d10 con 0x7f6ee80b5a60 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.097+0000 7f6ef006b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80d1ff0 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eede07640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6ee815db70 0x7f6ee8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53498/0 (socket says 172.21.15.60:53498) 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eede07640 1 -- 172.21.15.60:0/4025677296 learned_addr learned my addr 172.21.15.60:0/4025677296 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eed606640 1 --2- 172.21.15.60:0/4025677296 >> v2:172.21.15.100:3300/0 conn(0x7f6ee80b6cb0 0x7f6ee80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873726180 0 0) 0x7f6ee80ceee0 con 0x7f6ee815db70 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed4003520 con 0x7f6ee815db70 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000369190 0 0) 0x7f6ee80d1d10 con 0x7f6ee80b5a60 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee80ceee0 con 0x7f6ee80b5a60 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690353025 0 0) 0x7f6ed4003520 con 0x7f6ee815db70 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eed606640 1 -- 172.21.15.60:0/4025677296 >> v1:172.21.15.100:6789/0 conn(0x7f6ee80b5a60 legacy=0x7f6ee80d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eed606640 1 -- 172.21.15.60:0/4025677296 >> v1:172.21.15.60:6789/0 conn(0x7f6ee815db70 legacy=0x7f6ee8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.098+0000 7f6eed606640 1 -- 172.21.15.60:0/4025677296 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80d22d0 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.099+0000 7f6eed606640 1 --2- 172.21.15.60:0/4025677296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80b5520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097a0 tx=0x7f6ed802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d73733bd75725a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.099+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8037070 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.099+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed802abc0 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.099+0000 7f6eece05640 1 -- 172.21.15.60:0/4025677296 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4001670 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 -- 172.21.15.60:0/4025677296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee80b6cb0 msgr2=0x7f6ee80b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 --2- 172.21.15.60:0/4025677296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80b5520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097a0 tx=0x7f6ed802a130 comp rx=0 tx=0).stop 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 -- 172.21.15.60:0/4025677296 shutdown_connections 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 --2- 172.21.15.60:0/4025677296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80b5520 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 -- 172.21.15.60:0/4025677296 >> 172.21.15.60:0/4025677296 conn(0x7f6ee80d0520 msgr2=0x7f6ee80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 -- 172.21.15.60:0/4025677296 shutdown_connections 2024-04-25T08:30:34.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.100+0000 7f6ef006b640 1 -- 172.21.15.60:0/4025677296 wait complete. 2024-04-25T08:30:34.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.101+0000 7f6ef006b640 1 Processor -- start 2024-04-25T08:30:34.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.101+0000 7f6ef006b640 1 -- start start 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6ef006b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ee80b5a60 0x7f6ee80c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6ef006b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6ef006b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee815db70 0x7f6ee815ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6ef006b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80cf1c0 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6ef006b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80ce530 con 0x7f6ee815db70 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6ef006b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80cf340 con 0x7f6ee80b5a60 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45908/0 (socket says 172.21.15.60:45908) 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 -- 172.21.15.60:0/2068521349 learned_addr learned my addr 172.21.15.60:0/2068521349 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eede07640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ee80b5a60 0x7f6ee80c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eee608640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee815db70 0x7f6ee815ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 -- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ee80b5a60 msgr2=0x7f6ee80c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ee80b5a60 0x7f6ee80c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 -- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee815db70 msgr2=0x7f6ee815ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee815db70 0x7f6ee815ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 -- 172.21.15.60:0/2068521349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee8146c90 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.102+0000 7f6eed606640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80cb420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8035040 tx=0x7f6ed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.103+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8041020 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.103+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed803d620 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.103+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8146f70 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.103+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8037040 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.103+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee8147530 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.104+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8073990 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.104+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ed8044020 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.104+0000 7f6ede7fc640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ecc06b4d0 0x7f6ecc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.105+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ed80aa810 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.105+0000 7f6eede07640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ecc06b4d0 0x7f6ecc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.105+0000 7f6eede07640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ecc06b4d0 0x7f6ecc06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40011e0 tx=0x7f6ee4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.109+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ed804b030 con 0x7f6ee80b6cb0 2024-04-25T08:30:34.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.285+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ee80d1820 con 0x7f6ecc06b4d0 2024-04-25T08:30:34.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.292+0000 7f6ede7fc640 1 -- 172.21.15.60:0/2068521349 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248860 (secure 0 0 0) 0x7f6ee80d1820 con 0x7f6ecc06b4d0 2024-04-25T08:30:34.292 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:34.293 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:30:34.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ecc06b4d0 msgr2=0x7f6ecc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ecc06b4d0 0x7f6ecc06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40011e0 tx=0x7f6ee4009040 comp rx=0 tx=0).stop 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 msgr2=0x7f6ee80cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80cb420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8035040 tx=0x7f6ed80046c0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 shutdown_connections 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ecc06b4d0 0x7f6ecc06d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ee815db70 0x7f6ee815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee80b6cb0 0x7f6ee80cb420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 --2- 172.21.15.60:0/2068521349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ee80b5a60 0x7f6ee80c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 >> 172.21.15.60:0/2068521349 conn(0x7f6ee80d0520 msgr2=0x7f6ee80b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 shutdown_connections 2024-04-25T08:30:34.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.295+0000 7f6ef006b640 1 -- 172.21.15.60:0/2068521349 wait complete. 2024-04-25T08:30:34.303 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-25T08:30:33.413583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237196,"kb_used_data":4396,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737524,"statfs":{"total":773094113280,"available":772851224576,"internally_reserved":0,"allocated":4501504,"data_stored":2587401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.132042"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292388+0000","last_change":"2024-04-25T08:30:30.292388+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292388+0000","last_undegraded":"2024-04-25T08:30:30.292388+0000","last_fullsized":"2024-04-25T08:30:30.292388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293940+0000","last_change":"2024-04-25T08:30:30.293940+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293940+0000","last_undegraded":"2024-04-25T08:30:30.293940+0000","last_fullsized":"2024-04-25T08:30:30.293940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287912+0000","last_change":"2024-04-25T08:30:30.287912+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287912+0000","last_undegraded":"2024-04-25T08:30:30.287912+0000","last_fullsized":"2024-04-25T08:30:30.287912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286932+0000","last_change":"2024-04-25T08:30:30.286932+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286932+0000","last_undegraded":"2024-04-25T08:30:30.286932+0000","last_fullsized":"2024-04-25T08:30:30.286932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286790+0000","last_change":"2024-04-25T08:30:30.286790+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286790+0000","last_undegraded":"2024-04-25T08:30:30.286790+0000","last_fullsized":"2024-04-25T08:30:30.286790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292036+0000","last_change":"2024-04-25T08:30:30.292036+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292036+0000","last_undegraded":"2024-04-25T08:30:30.292036+0000","last_fullsized":"2024-04-25T08:30:30.292036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341093+0000","last_change":"2024-04-25T08:30:31.339672+0000","last_active":"2024-04-25T08:30:31.341093+0000","last_peered":"2024-04-25T08:30:31.341093+0000","last_clean":"2024-04-25T08:30:31.341093+0000","last_became_active":"2024-04-25T08:30:31.339395+0000","last_became_peered":"2024-04-25T08:30:31.339395+0000","last_unstale":"2024-04-25T08:30:31.341093+0000","last_undegraded":"2024-04-25T08:30:31.341093+0000","last_fullsized":"2024-04-25T08:30:31.341093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.788878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289376+0000","last_change":"2024-04-25T08:30:30.289376+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289376+0000","last_undegraded":"2024-04-25T08:30:30.289376+0000","last_fullsized":"2024-04-25T08:30:30.289376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287333+0000","last_change":"2024-04-25T08:30:30.287333+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287333+0000","last_undegraded":"2024-04-25T08:30:30.287333+0000","last_fullsized":"2024-04-25T08:30:30.287333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.336655+0000","last_change":"2024-04-25T08:30:31.331300+0000","last_active":"2024-04-25T08:30:31.336655+0000","last_peered":"2024-04-25T08:30:31.336655+0000","last_clean":"2024-04-25T08:30:31.336655+0000","last_became_active":"2024-04-25T08:30:31.330974+0000","last_became_peered":"2024-04-25T08:30:31.330974+0000","last_unstale":"2024-04-25T08:30:31.336655+0000","last_undegraded":"2024-04-25T08:30:31.336655+0000","last_fullsized":"2024-04-25T08:30:31.336655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:43.081894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340959+0000","last_change":"2024-04-25T08:30:31.336378+0000","last_active":"2024-04-25T08:30:31.340959+0000","last_peered":"2024-04-25T08:30:31.340959+0000","last_clean":"2024-04-25T08:30:31.340959+0000","last_became_active":"2024-04-25T08:30:31.335915+0000","last_became_peered":"2024-04-25T08:30:31.335915+0000","last_unstale":"2024-04-25T08:30:31.340959+0000","last_undegraded":"2024-04-25T08:30:31.340959+0000","last_fullsized":"2024-04-25T08:30:31.340959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:47.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287710+0000","last_change":"2024-04-25T08:30:30.287710+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287710+0000","last_undegraded":"2024-04-25T08:30:30.287710+0000","last_fullsized":"2024-04-25T08:30:30.287710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337039+0000","last_change":"2024-04-25T08:30:31.331560+0000","last_active":"2024-04-25T08:30:31.337039+0000","last_peered":"2024-04-25T08:30:31.337039+0000","last_clean":"2024-04-25T08:30:31.337039+0000","last_became_active":"2024-04-25T08:30:31.331207+0000","last_became_peered":"2024-04-25T08:30:31.331207+0000","last_unstale":"2024-04-25T08:30:31.337039+0000","last_undegraded":"2024-04-25T08:30:31.337039+0000","last_fullsized":"2024-04-25T08:30:31.337039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:12.355397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288425+0000","last_change":"2024-04-25T08:30:30.288425+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288425+0000","last_undegraded":"2024-04-25T08:30:30.288425+0000","last_fullsized":"2024-04-25T08:30:30.288425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340097+0000","last_change":"2024-04-25T08:30:31.334369+0000","last_active":"2024-04-25T08:30:31.340097+0000","last_peered":"2024-04-25T08:30:31.340097+0000","last_clean":"2024-04-25T08:30:31.340097+0000","last_became_active":"2024-04-25T08:30:31.334071+0000","last_became_peered":"2024-04-25T08:30:31.334071+0000","last_unstale":"2024-04-25T08:30:31.340097+0000","last_undegraded":"2024-04-25T08:30:31.340097+0000","last_fullsized":"2024-04-25T08:30:31.340097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.453764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332186+0000","last_change":"2024-04-25T08:30:31.331292+0000","last_active":"2024-04-25T08:30:31.332186+0000","last_peered":"2024-04-25T08:30:31.332186+0000","last_clean":"2024-04-25T08:30:31.332186+0000","last_became_active":"2024-04-25T08:30:31.330966+0000","last_became_peered":"2024-04-25T08:30:31.330966+0000","last_unstale":"2024-04-25T08:30:31.332186+0000","last_undegraded":"2024-04-25T08:30:31.332186+0000","last_fullsized":"2024-04-25T08:30:31.332186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332626+0000","last_change":"2024-04-25T08:30:31.331334+0000","last_active":"2024-04-25T08:30:31.332626+0000","last_peered":"2024-04-25T08:30:31.332626+0000","last_clean":"2024-04-25T08:30:31.332626+0000","last_became_active":"2024-04-25T08:30:31.331048+0000","last_became_peered":"2024-04-25T08:30:31.331048+0000","last_unstale":"2024-04-25T08:30:31.332626+0000","last_undegraded":"2024-04-25T08:30:31.332626+0000","last_fullsized":"2024-04-25T08:30:31.332626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325080+0000","last_change":"2024-04-25T08:30:31.322980+0000","last_active":"2024-04-25T08:30:31.325080+0000","last_peered":"2024-04-25T08:30:31.325080+0000","last_clean":"2024-04-25T08:30:31.325080+0000","last_became_active":"2024-04-25T08:30:31.322718+0000","last_became_peered":"2024-04-25T08:30:31.322718+0000","last_unstale":"2024-04-25T08:30:31.325080+0000","last_undegraded":"2024-04-25T08:30:31.325080+0000","last_fullsized":"2024-04-25T08:30:31.325080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:34.413297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286631+0000","last_change":"2024-04-25T08:30:30.286631+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286631+0000","last_undegraded":"2024-04-25T08:30:30.286631+0000","last_fullsized":"2024-04-25T08:30:30.286631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323347+0000","last_change":"2024-04-25T08:30:31.321932+0000","last_active":"2024-04-25T08:30:31.323347+0000","last_peered":"2024-04-25T08:30:31.323347+0000","last_clean":"2024-04-25T08:30:31.323347+0000","last_became_active":"2024-04-25T08:30:31.321696+0000","last_became_peered":"2024-04-25T08:30:31.321696+0000","last_unstale":"2024-04-25T08:30:31.323347+0000","last_undegraded":"2024-04-25T08:30:31.323347+0000","last_fullsized":"2024-04-25T08:30:31.323347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:39.300333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337070+0000","last_change":"2024-04-25T08:30:31.330998+0000","last_active":"2024-04-25T08:30:31.337070+0000","last_peered":"2024-04-25T08:30:31.337070+0000","last_clean":"2024-04-25T08:30:31.337070+0000","last_became_active":"2024-04-25T08:30:31.330539+0000","last_became_peered":"2024-04-25T08:30:31.330539+0000","last_unstale":"2024-04-25T08:30:31.337070+0000","last_undegraded":"2024-04-25T08:30:31.337070+0000","last_fullsized":"2024-04-25T08:30:31.337070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287479+0000","last_change":"2024-04-25T08:30:30.287479+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287479+0000","last_undegraded":"2024-04-25T08:30:30.287479+0000","last_fullsized":"2024-04-25T08:30:30.287479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323285+0000","last_change":"2024-04-25T08:30:31.320712+0000","last_active":"2024-04-25T08:30:31.323285+0000","last_peered":"2024-04-25T08:30:31.323285+0000","last_clean":"2024-04-25T08:30:31.323285+0000","last_became_active":"2024-04-25T08:30:31.320446+0000","last_became_peered":"2024-04-25T08:30:31.320446+0000","last_unstale":"2024-04-25T08:30:31.323285+0000","last_undegraded":"2024-04-25T08:30:31.323285+0000","last_fullsized":"2024-04-25T08:30:31.323285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.070232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286827+0000","last_change":"2024-04-25T08:30:30.286827+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286827+0000","last_undegraded":"2024-04-25T08:30:30.286827+0000","last_fullsized":"2024-04-25T08:30:30.286827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.327272+0000","last_change":"2024-04-25T08:30:31.324771+0000","last_active":"2024-04-25T08:30:31.327272+0000","last_peered":"2024-04-25T08:30:31.327272+0000","last_clean":"2024-04-25T08:30:31.327272+0000","last_became_active":"2024-04-25T08:30:31.324509+0000","last_became_peered":"2024-04-25T08:30:31.324509+0000","last_unstale":"2024-04-25T08:30:31.327272+0000","last_undegraded":"2024-04-25T08:30:31.327272+0000","last_fullsized":"2024-04-25T08:30:31.327272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:32.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.298366+0000","last_change":"2024-04-25T08:30:30.298366+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.298366+0000","last_undegraded":"2024-04-25T08:30:30.298366+0000","last_fullsized":"2024-04-25T08:30:30.298366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.319706+0000","last_change":"2024-04-25T08:30:31.316804+0000","last_active":"2024-04-25T08:30:31.319706+0000","last_peered":"2024-04-25T08:30:31.319706+0000","last_clean":"2024-04-25T08:30:31.319706+0000","last_became_active":"2024-04-25T08:30:31.316503+0000","last_became_peered":"2024-04-25T08:30:31.316503+0000","last_unstale":"2024-04-25T08:30:31.319706+0000","last_undegraded":"2024-04-25T08:30:31.319706+0000","last_fullsized":"2024-04-25T08:30:31.319706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:43.729410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.339771+0000","last_change":"2024-04-25T08:30:31.331474+0000","last_active":"2024-04-25T08:30:31.339771+0000","last_peered":"2024-04-25T08:30:31.339771+0000","last_clean":"2024-04-25T08:30:31.339771+0000","last_became_active":"2024-04-25T08:30:31.331058+0000","last_became_peered":"2024-04-25T08:30:31.331058+0000","last_unstale":"2024-04-25T08:30:31.339771+0000","last_undegraded":"2024-04-25T08:30:31.339771+0000","last_fullsized":"2024-04-25T08:30:31.339771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320679+0000","last_change":"2024-04-25T08:30:31.319410+0000","last_active":"2024-04-25T08:30:31.320679+0000","last_peered":"2024-04-25T08:30:31.320679+0000","last_clean":"2024-04-25T08:30:31.320679+0000","last_became_active":"2024-04-25T08:30:31.318913+0000","last_became_peered":"2024-04-25T08:30:31.318913+0000","last_unstale":"2024-04-25T08:30:31.320679+0000","last_undegraded":"2024-04-25T08:30:31.320679+0000","last_fullsized":"2024-04-25T08:30:31.320679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:14.713545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320655+0000","last_change":"2024-04-25T08:30:31.318909+0000","last_active":"2024-04-25T08:30:31.320655+0000","last_peered":"2024-04-25T08:30:31.320655+0000","last_clean":"2024-04-25T08:30:31.320655+0000","last_became_active":"2024-04-25T08:30:31.318639+0000","last_became_peered":"2024-04-25T08:30:31.318639+0000","last_unstale":"2024-04-25T08:30:31.320655+0000","last_undegraded":"2024-04-25T08:30:31.320655+0000","last_fullsized":"2024-04-25T08:30:31.320655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:50.624325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323302+0000","last_change":"2024-04-25T08:30:31.321924+0000","last_active":"2024-04-25T08:30:31.323302+0000","last_peered":"2024-04-25T08:30:31.323302+0000","last_clean":"2024-04-25T08:30:31.323302+0000","last_became_active":"2024-04-25T08:30:31.321434+0000","last_became_peered":"2024-04-25T08:30:31.321434+0000","last_unstale":"2024-04-25T08:30:31.323302+0000","last_undegraded":"2024-04-25T08:30:31.323302+0000","last_fullsized":"2024-04-25T08:30:31.323302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289797+0000","last_change":"2024-04-25T08:30:30.289797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289797+0000","last_undegraded":"2024-04-25T08:30:30.289797+0000","last_fullsized":"2024-04-25T08:30:30.289797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338955+0000","last_change":"2024-04-25T08:30:31.332758+0000","last_active":"2024-04-25T08:30:31.338955+0000","last_peered":"2024-04-25T08:30:31.338955+0000","last_clean":"2024-04-25T08:30:31.338955+0000","last_became_active":"2024-04-25T08:30:31.332458+0000","last_became_peered":"2024-04-25T08:30:31.332458+0000","last_unstale":"2024-04-25T08:30:31.338955+0000","last_undegraded":"2024-04-25T08:30:31.338955+0000","last_fullsized":"2024-04-25T08:30:31.338955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340572+0000","last_change":"2024-04-25T08:30:30.294755+0000","last_active":"2024-04-25T08:30:31.340572+0000","last_peered":"2024-04-25T08:30:31.340572+0000","last_clean":"2024-04-25T08:30:31.340572+0000","last_became_active":"2024-04-25T08:30:30.293812+0000","last_became_peered":"2024-04-25T08:30:30.293812+0000","last_unstale":"2024-04-25T08:30:31.340572+0000","last_undegraded":"2024-04-25T08:30:31.340572+0000","last_fullsized":"2024-04-25T08:30:31.340572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.299433+0000","last_change":"2024-04-25T08:30:30.299433+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.299433+0000","last_undegraded":"2024-04-25T08:30:30.299433+0000","last_fullsized":"2024-04-25T08:30:30.299433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304327+0000","last_change":"2024-04-25T08:30:30.301932+0000","last_active":"2024-04-25T08:30:30.304327+0000","last_peered":"2024-04-25T08:30:30.304327+0000","last_clean":"2024-04-25T08:30:30.304327+0000","last_became_active":"2024-04-25T08:30:30.301633+0000","last_became_peered":"2024-04-25T08:30:30.301633+0000","last_unstale":"2024-04-25T08:30:30.304327+0000","last_undegraded":"2024-04-25T08:30:30.304327+0000","last_fullsized":"2024-04-25T08:30:30.304327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286830+0000","last_change":"2024-04-25T08:30:30.286830+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286830+0000","last_undegraded":"2024-04-25T08:30:30.286830+0000","last_fullsized":"2024-04-25T08:30:30.286830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306192+0000","last_change":"2024-04-25T08:30:30.304552+0000","last_active":"2024-04-25T08:30:30.306192+0000","last_peered":"2024-04-25T08:30:30.306192+0000","last_clean":"2024-04-25T08:30:30.306192+0000","last_became_active":"2024-04-25T08:30:30.304168+0000","last_became_peered":"2024-04-25T08:30:30.304168+0000","last_unstale":"2024-04-25T08:30:30.306192+0000","last_undegraded":"2024-04-25T08:30:30.306192+0000","last_fullsized":"2024-04-25T08:30:30.306192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292899+0000","last_change":"2024-04-25T08:30:30.292899+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292899+0000","last_undegraded":"2024-04-25T08:30:30.292899+0000","last_fullsized":"2024-04-25T08:30:30.292899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304321+0000","last_change":"2024-04-25T08:30:30.301217+0000","last_active":"2024-04-25T08:30:30.304321+0000","last_peered":"2024-04-25T08:30:30.304321+0000","last_clean":"2024-04-25T08:30:30.304321+0000","last_became_active":"2024-04-25T08:30:30.300899+0000","last_became_peered":"2024-04-25T08:30:30.300899+0000","last_unstale":"2024-04-25T08:30:30.304321+0000","last_undegraded":"2024-04-25T08:30:30.304321+0000","last_fullsized":"2024-04-25T08:30:30.304321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289501+0000","last_change":"2024-04-25T08:30:30.289501+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289501+0000","last_undegraded":"2024-04-25T08:30:30.289501+0000","last_fullsized":"2024-04-25T08:30:30.289501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338973+0000","last_change":"2024-04-25T08:30:30.302842+0000","last_active":"2024-04-25T08:30:31.338973+0000","last_peered":"2024-04-25T08:30:31.338973+0000","last_clean":"2024-04-25T08:30:31.338973+0000","last_became_active":"2024-04-25T08:30:30.302540+0000","last_became_peered":"2024-04-25T08:30:30.302540+0000","last_unstale":"2024-04-25T08:30:31.338973+0000","last_undegraded":"2024-04-25T08:30:31.338973+0000","last_fullsized":"2024-04-25T08:30:31.338973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.946206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335411+0000","last_change":"2024-04-25T08:30:31.330857+0000","last_active":"2024-04-25T08:30:31.335411+0000","last_peered":"2024-04-25T08:30:31.335411+0000","last_clean":"2024-04-25T08:30:31.335411+0000","last_became_active":"2024-04-25T08:30:31.329563+0000","last_became_peered":"2024-04-25T08:30:31.329563+0000","last_unstale":"2024-04-25T08:30:31.335411+0000","last_undegraded":"2024-04-25T08:30:31.335411+0000","last_fullsized":"2024-04-25T08:30:31.335411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341386+0000","last_change":"2024-04-25T08:30:30.298067+0000","last_active":"2024-04-25T08:30:31.341386+0000","last_peered":"2024-04-25T08:30:31.341386+0000","last_clean":"2024-04-25T08:30:31.341386+0000","last_became_active":"2024-04-25T08:30:30.297718+0000","last_became_peered":"2024-04-25T08:30:30.297718+0000","last_unstale":"2024-04-25T08:30:31.341386+0000","last_undegraded":"2024-04-25T08:30:31.341386+0000","last_fullsized":"2024-04-25T08:30:31.341386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287002+0000","last_change":"2024-04-25T08:30:30.287002+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287002+0000","last_undegraded":"2024-04-25T08:30:30.287002+0000","last_fullsized":"2024-04-25T08:30:30.287002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306219+0000","last_change":"2024-04-25T08:30:30.305406+0000","last_active":"2024-04-25T08:30:30.306219+0000","last_peered":"2024-04-25T08:30:30.306219+0000","last_clean":"2024-04-25T08:30:30.306219+0000","last_became_active":"2024-04-25T08:30:30.305041+0000","last_became_peered":"2024-04-25T08:30:30.305041+0000","last_unstale":"2024-04-25T08:30:30.306219+0000","last_undegraded":"2024-04-25T08:30:30.306219+0000","last_fullsized":"2024-04-25T08:30:30.306219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291475+0000","last_change":"2024-04-25T08:30:30.291475+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291475+0000","last_undegraded":"2024-04-25T08:30:30.291475+0000","last_fullsized":"2024-04-25T08:30:30.291475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295501+0000","last_change":"2024-04-25T08:30:30.290419+0000","last_active":"2024-04-25T08:30:30.295501+0000","last_peered":"2024-04-25T08:30:30.295501+0000","last_clean":"2024-04-25T08:30:30.295501+0000","last_became_active":"2024-04-25T08:30:30.290143+0000","last_became_peered":"2024-04-25T08:30:30.290143+0000","last_unstale":"2024-04-25T08:30:30.295501+0000","last_undegraded":"2024-04-25T08:30:30.295501+0000","last_fullsized":"2024-04-25T08:30:30.295501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:19.910492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286356+0000","last_change":"2024-04-25T08:30:30.286356+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286356+0000","last_undegraded":"2024-04-25T08:30:30.286356+0000","last_fullsized":"2024-04-25T08:30:30.286356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305495+0000","last_change":"2024-04-25T08:30:30.302163+0000","last_active":"2024-04-25T08:30:30.305495+0000","last_peered":"2024-04-25T08:30:30.305495+0000","last_clean":"2024-04-25T08:30:30.305495+0000","last_became_active":"2024-04-25T08:30:30.301848+0000","last_became_peered":"2024-04-25T08:30:30.301848+0000","last_unstale":"2024-04-25T08:30:30.305495+0000","last_undegraded":"2024-04-25T08:30:30.305495+0000","last_fullsized":"2024-04-25T08:30:30.305495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:50.715081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323281+0000","last_change":"2024-04-25T08:30:31.319799+0000","last_active":"2024-04-25T08:30:31.323281+0000","last_peered":"2024-04-25T08:30:31.323281+0000","last_clean":"2024-04-25T08:30:31.323281+0000","last_became_active":"2024-04-25T08:30:31.319152+0000","last_became_peered":"2024-04-25T08:30:31.319152+0000","last_unstale":"2024-04-25T08:30:31.323281+0000","last_undegraded":"2024-04-25T08:30:31.323281+0000","last_fullsized":"2024-04-25T08:30:31.323281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:11.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325859+0000","last_change":"2024-04-25T08:30:30.299038+0000","last_active":"2024-04-25T08:30:31.325859+0000","last_peered":"2024-04-25T08:30:31.325859+0000","last_clean":"2024-04-25T08:30:31.325859+0000","last_became_active":"2024-04-25T08:30:30.298782+0000","last_became_peered":"2024-04-25T08:30:30.298782+0000","last_unstale":"2024-04-25T08:30:31.325859+0000","last_undegraded":"2024-04-25T08:30:31.325859+0000","last_fullsized":"2024-04-25T08:30:31.325859+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:47.771125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287963+0000","last_change":"2024-04-25T08:30:30.287963+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287963+0000","last_undegraded":"2024-04-25T08:30:30.287963+0000","last_fullsized":"2024-04-25T08:30:30.287963+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341021+0000","last_change":"2024-04-25T08:30:30.301686+0000","last_active":"2024-04-25T08:30:31.341021+0000","last_peered":"2024-04-25T08:30:31.341021+0000","last_clean":"2024-04-25T08:30:31.341021+0000","last_became_active":"2024-04-25T08:30:30.301391+0000","last_became_peered":"2024-04-25T08:30:30.301391+0000","last_unstale":"2024-04-25T08:30:31.341021+0000","last_undegraded":"2024-04-25T08:30:31.341021+0000","last_fullsized":"2024-04-25T08:30:31.341021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.852169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288660+0000","last_change":"2024-04-25T08:30:30.288660+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288660+0000","last_undegraded":"2024-04-25T08:30:30.288660+0000","last_fullsized":"2024-04-25T08:30:30.288660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305486+0000","last_change":"2024-04-25T08:30:30.303220+0000","last_active":"2024-04-25T08:30:30.305486+0000","last_peered":"2024-04-25T08:30:30.305486+0000","last_clean":"2024-04-25T08:30:30.305486+0000","last_became_active":"2024-04-25T08:30:30.302914+0000","last_became_peered":"2024-04-25T08:30:30.302914+0000","last_unstale":"2024-04-25T08:30:30.305486+0000","last_undegraded":"2024-04-25T08:30:30.305486+0000","last_fullsized":"2024-04-25T08:30:30.305486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:10.215234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289039+0000","last_change":"2024-04-25T08:30:30.289039+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289039+0000","last_undegraded":"2024-04-25T08:30:30.289039+0000","last_fullsized":"2024-04-25T08:30:30.289039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304930+0000","last_change":"2024-04-25T08:30:30.301164+0000","last_active":"2024-04-25T08:30:30.304930+0000","last_peered":"2024-04-25T08:30:30.304930+0000","last_clean":"2024-04-25T08:30:30.304930+0000","last_became_active":"2024-04-25T08:30:30.300887+0000","last_became_peered":"2024-04-25T08:30:30.300887+0000","last_unstale":"2024-04-25T08:30:30.304930+0000","last_undegraded":"2024-04-25T08:30:30.304930+0000","last_fullsized":"2024-04-25T08:30:30.304930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.290447+0000","last_change":"2024-04-25T08:30:30.290447+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.290447+0000","last_undegraded":"2024-04-25T08:30:30.290447+0000","last_fullsized":"2024-04-25T08:30:30.290447+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306196+0000","last_change":"2024-04-25T08:30:30.304789+0000","last_active":"2024-04-25T08:30:30.306196+0000","last_peered":"2024-04-25T08:30:30.306196+0000","last_clean":"2024-04-25T08:30:30.306196+0000","last_became_active":"2024-04-25T08:30:30.304493+0000","last_became_peered":"2024-04-25T08:30:30.304493+0000","last_unstale":"2024-04-25T08:30:30.306196+0000","last_undegraded":"2024-04-25T08:30:30.306196+0000","last_fullsized":"2024-04-25T08:30:30.306196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330098+0000","last_change":"2024-04-25T08:30:31.326518+0000","last_active":"2024-04-25T08:30:31.330098+0000","last_peered":"2024-04-25T08:30:31.330098+0000","last_clean":"2024-04-25T08:30:31.330098+0000","last_became_active":"2024-04-25T08:30:31.326241+0000","last_became_peered":"2024-04-25T08:30:31.326241+0000","last_unstale":"2024-04-25T08:30:31.330098+0000","last_undegraded":"2024-04-25T08:30:31.330098+0000","last_fullsized":"2024-04-25T08:30:31.330098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310441+0000","last_change":"2024-04-25T08:30:30.294767+0000","last_active":"2024-04-25T08:30:31.310441+0000","last_peered":"2024-04-25T08:30:31.310441+0000","last_clean":"2024-04-25T08:30:31.310441+0000","last_became_active":"2024-04-25T08:30:30.294475+0000","last_became_peered":"2024-04-25T08:30:30.294475+0000","last_unstale":"2024-04-25T08:30:31.310441+0000","last_undegraded":"2024-04-25T08:30:31.310441+0000","last_fullsized":"2024-04-25T08:30:31.310441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291037+0000","last_change":"2024-04-25T08:30:30.291037+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291037+0000","last_undegraded":"2024-04-25T08:30:30.291037+0000","last_fullsized":"2024-04-25T08:30:30.291037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.342812+0000","last_change":"2024-04-25T08:30:30.297207+0000","last_active":"2024-04-25T08:30:31.342812+0000","last_peered":"2024-04-25T08:30:31.342812+0000","last_clean":"2024-04-25T08:30:31.342812+0000","last_became_active":"2024-04-25T08:30:30.296922+0000","last_became_peered":"2024-04-25T08:30:30.296922+0000","last_unstale":"2024-04-25T08:30:31.342812+0000","last_undegraded":"2024-04-25T08:30:31.342812+0000","last_fullsized":"2024-04-25T08:30:31.342812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:13.711704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.297011+0000","last_change":"2024-04-25T08:30:30.297011+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.297011+0000","last_undegraded":"2024-04-25T08:30:30.297011+0000","last_fullsized":"2024-04-25T08:30:30.297011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340937+0000","last_change":"2024-04-25T08:30:30.297702+0000","last_active":"2024-04-25T08:30:31.340937+0000","last_peered":"2024-04-25T08:30:31.340937+0000","last_clean":"2024-04-25T08:30:31.340937+0000","last_became_active":"2024-04-25T08:30:30.297420+0000","last_became_peered":"2024-04-25T08:30:30.297420+0000","last_unstale":"2024-04-25T08:30:31.340937+0000","last_undegraded":"2024-04-25T08:30:31.340937+0000","last_fullsized":"2024-04-25T08:30:31.340937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.137643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341074+0000","last_change":"2024-04-25T08:30:31.337230+0000","last_active":"2024-04-25T08:30:31.341074+0000","last_peered":"2024-04-25T08:30:31.341074+0000","last_clean":"2024-04-25T08:30:31.341074+0000","last_became_active":"2024-04-25T08:30:31.336785+0000","last_became_peered":"2024-04-25T08:30:31.336785+0000","last_unstale":"2024-04-25T08:30:31.341074+0000","last_undegraded":"2024-04-25T08:30:31.341074+0000","last_fullsized":"2024-04-25T08:30:31.341074+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:28.421465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301349+0000","last_change":"2024-04-25T08:30:30.296836+0000","last_active":"2024-04-25T08:30:30.301349+0000","last_peered":"2024-04-25T08:30:30.301349+0000","last_clean":"2024-04-25T08:30:30.301349+0000","last_became_active":"2024-04-25T08:30:30.296490+0000","last_became_peered":"2024-04-25T08:30:30.296490+0000","last_unstale":"2024-04-25T08:30:30.301349+0000","last_undegraded":"2024-04-25T08:30:30.301349+0000","last_fullsized":"2024-04-25T08:30:30.301349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289132+0000","last_change":"2024-04-25T08:30:30.289132+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289132+0000","last_undegraded":"2024-04-25T08:30:30.289132+0000","last_fullsized":"2024-04-25T08:30:30.289132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.300070+0000","last_change":"2024-04-25T08:30:30.297703+0000","last_active":"2024-04-25T08:30:30.300070+0000","last_peered":"2024-04-25T08:30:30.300070+0000","last_clean":"2024-04-25T08:30:30.300070+0000","last_became_active":"2024-04-25T08:30:30.297424+0000","last_became_peered":"2024-04-25T08:30:30.297424+0000","last_unstale":"2024-04-25T08:30:30.300070+0000","last_undegraded":"2024-04-25T08:30:30.300070+0000","last_fullsized":"2024-04-25T08:30:30.300070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.320688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286213+0000","last_change":"2024-04-25T08:30:30.286213+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286213+0000","last_undegraded":"2024-04-25T08:30:30.286213+0000","last_fullsized":"2024-04-25T08:30:30.286213+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305672+0000","last_change":"2024-04-25T08:30:30.303603+0000","last_active":"2024-04-25T08:30:30.305672+0000","last_peered":"2024-04-25T08:30:30.305672+0000","last_clean":"2024-04-25T08:30:30.305672+0000","last_became_active":"2024-04-25T08:30:30.303322+0000","last_became_peered":"2024-04-25T08:30:30.303322+0000","last_unstale":"2024-04-25T08:30:30.305672+0000","last_undegraded":"2024-04-25T08:30:30.305672+0000","last_fullsized":"2024-04-25T08:30:30.305672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330062+0000","last_change":"2024-04-25T08:30:31.325746+0000","last_active":"2024-04-25T08:30:31.330062+0000","last_peered":"2024-04-25T08:30:31.330062+0000","last_clean":"2024-04-25T08:30:31.330062+0000","last_became_active":"2024-04-25T08:30:31.325449+0000","last_became_peered":"2024-04-25T08:30:31.325449+0000","last_unstale":"2024-04-25T08:30:31.330062+0000","last_undegraded":"2024-04-25T08:30:31.330062+0000","last_fullsized":"2024-04-25T08:30:31.330062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295651+0000","last_change":"2024-04-25T08:30:30.293002+0000","last_active":"2024-04-25T08:30:30.295651+0000","last_peered":"2024-04-25T08:30:30.295651+0000","last_clean":"2024-04-25T08:30:30.295651+0000","last_became_active":"2024-04-25T08:30:30.292702+0000","last_became_peered":"2024-04-25T08:30:30.292702+0000","last_unstale":"2024-04-25T08:30:30.295651+0000","last_undegraded":"2024-04-25T08:30:30.295651+0000","last_fullsized":"2024-04-25T08:30:30.295651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.414247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287883+0000","last_change":"2024-04-25T08:30:30.287883+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287883+0000","last_undegraded":"2024-04-25T08:30:30.287883+0000","last_fullsized":"2024-04-25T08:30:30.287883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295421+0000","last_change":"2024-04-25T08:30:30.292431+0000","last_active":"2024-04-25T08:30:30.295421+0000","last_peered":"2024-04-25T08:30:30.295421+0000","last_clean":"2024-04-25T08:30:30.295421+0000","last_became_active":"2024-04-25T08:30:30.292078+0000","last_became_peered":"2024-04-25T08:30:30.292078+0000","last_unstale":"2024-04-25T08:30:30.295421+0000","last_undegraded":"2024-04-25T08:30:30.295421+0000","last_fullsized":"2024-04-25T08:30:30.295421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:06.079852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335419+0000","last_change":"2024-04-25T08:30:31.331176+0000","last_active":"2024-04-25T08:30:31.335419+0000","last_peered":"2024-04-25T08:30:31.335419+0000","last_clean":"2024-04-25T08:30:31.335419+0000","last_became_active":"2024-04-25T08:30:31.330823+0000","last_became_peered":"2024-04-25T08:30:31.330823+0000","last_unstale":"2024-04-25T08:30:31.335419+0000","last_undegraded":"2024-04-25T08:30:31.335419+0000","last_fullsized":"2024-04-25T08:30:31.335419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:21.292627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311697+0000","last_change":"2024-04-25T08:30:30.299966+0000","last_active":"2024-04-25T08:30:31.311697+0000","last_peered":"2024-04-25T08:30:31.311697+0000","last_clean":"2024-04-25T08:30:31.311697+0000","last_became_active":"2024-04-25T08:30:30.299699+0000","last_became_peered":"2024-04-25T08:30:30.299699+0000","last_unstale":"2024-04-25T08:30:31.311697+0000","last_undegraded":"2024-04-25T08:30:31.311697+0000","last_fullsized":"2024-04-25T08:30:31.311697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289401+0000","last_change":"2024-04-25T08:30:30.289401+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289401+0000","last_undegraded":"2024-04-25T08:30:30.289401+0000","last_fullsized":"2024-04-25T08:30:30.289401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.297186+0000","last_change":"2024-04-25T08:30:30.293467+0000","last_active":"2024-04-25T08:30:30.297186+0000","last_peered":"2024-04-25T08:30:30.297186+0000","last_clean":"2024-04-25T08:30:30.297186+0000","last_became_active":"2024-04-25T08:30:30.293179+0000","last_became_peered":"2024-04-25T08:30:30.293179+0000","last_unstale":"2024-04-25T08:30:30.297186+0000","last_undegraded":"2024-04-25T08:30:30.297186+0000","last_fullsized":"2024-04-25T08:30:30.297186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.629362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.296797+0000","last_change":"2024-04-25T08:30:30.296797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.296797+0000","last_undegraded":"2024-04-25T08:30:30.296797+0000","last_fullsized":"2024-04-25T08:30:30.296797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340294+0000","last_change":"2024-04-25T08:30:30.301760+0000","last_active":"2024-04-25T08:30:31.340294+0000","last_peered":"2024-04-25T08:30:31.340294+0000","last_clean":"2024-04-25T08:30:31.340294+0000","last_became_active":"2024-04-25T08:30:30.301331+0000","last_became_peered":"2024-04-25T08:30:30.301331+0000","last_unstale":"2024-04-25T08:30:31.340294+0000","last_undegraded":"2024-04-25T08:30:31.340294+0000","last_fullsized":"2024-04-25T08:30:31.340294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323304+0000","last_change":"2024-04-25T08:30:31.320048+0000","last_active":"2024-04-25T08:30:31.323304+0000","last_peered":"2024-04-25T08:30:31.323304+0000","last_clean":"2024-04-25T08:30:31.323304+0000","last_became_active":"2024-04-25T08:30:31.319495+0000","last_became_peered":"2024-04-25T08:30:31.319495+0000","last_unstale":"2024-04-25T08:30:31.323304+0000","last_undegraded":"2024-04-25T08:30:31.323304+0000","last_fullsized":"2024-04-25T08:30:31.323304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:05.728351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.299425+0000","last_change":"2024-04-25T08:30:30.296903+0000","last_active":"2024-04-25T08:30:30.299425+0000","last_peered":"2024-04-25T08:30:30.299425+0000","last_clean":"2024-04-25T08:30:30.299425+0000","last_became_active":"2024-04-25T08:30:30.296623+0000","last_became_peered":"2024-04-25T08:30:30.296623+0000","last_unstale":"2024-04-25T08:30:30.299425+0000","last_undegraded":"2024-04-25T08:30:30.299425+0000","last_fullsized":"2024-04-25T08:30:30.299425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:03.007004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293539+0000","last_change":"2024-04-25T08:30:30.293539+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293539+0000","last_undegraded":"2024-04-25T08:30:30.293539+0000","last_fullsized":"2024-04-25T08:30:30.293539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305561+0000","last_change":"2024-04-25T08:30:30.303229+0000","last_active":"2024-04-25T08:30:30.305561+0000","last_peered":"2024-04-25T08:30:30.305561+0000","last_clean":"2024-04-25T08:30:30.305561+0000","last_became_active":"2024-04-25T08:30:30.302894+0000","last_became_peered":"2024-04-25T08:30:30.302894+0000","last_unstale":"2024-04-25T08:30:30.305561+0000","last_undegraded":"2024-04-25T08:30:30.305561+0000","last_fullsized":"2024-04-25T08:30:30.305561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288107+0000","last_change":"2024-04-25T08:30:30.288107+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288107+0000","last_undegraded":"2024-04-25T08:30:30.288107+0000","last_fullsized":"2024-04-25T08:30:30.288107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301519+0000","last_change":"2024-04-25T08:30:30.300280+0000","last_active":"2024-04-25T08:30:30.301519+0000","last_peered":"2024-04-25T08:30:30.301519+0000","last_clean":"2024-04-25T08:30:30.301519+0000","last_became_active":"2024-04-25T08:30:30.299985+0000","last_became_peered":"2024-04-25T08:30:30.299985+0000","last_unstale":"2024-04-25T08:30:30.301519+0000","last_undegraded":"2024-04-25T08:30:30.301519+0000","last_fullsized":"2024-04-25T08:30:30.301519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292115+0000","last_change":"2024-04-25T08:30:30.292115+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292115+0000","last_undegraded":"2024-04-25T08:30:30.292115+0000","last_fullsized":"2024-04-25T08:30:30.292115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.312138+0000","last_change":"2024-04-25T08:30:30.293212+0000","last_active":"2024-04-25T08:30:31.312138+0000","last_peered":"2024-04-25T08:30:31.312138+0000","last_clean":"2024-04-25T08:30:31.312138+0000","last_became_active":"2024-04-25T08:30:30.292931+0000","last_became_peered":"2024-04-25T08:30:30.292931+0000","last_unstale":"2024-04-25T08:30:31.312138+0000","last_undegraded":"2024-04-25T08:30:31.312138+0000","last_fullsized":"2024-04-25T08:30:31.312138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287931+0000","last_change":"2024-04-25T08:30:30.287931+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287931+0000","last_undegraded":"2024-04-25T08:30:30.287931+0000","last_fullsized":"2024-04-25T08:30:30.287931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.302154+0000","last_change":"2024-04-25T08:30:30.298769+0000","last_active":"2024-04-25T08:30:30.302154+0000","last_peered":"2024-04-25T08:30:30.302154+0000","last_clean":"2024-04-25T08:30:30.302154+0000","last_became_active":"2024-04-25T08:30:30.298489+0000","last_became_peered":"2024-04-25T08:30:30.298489+0000","last_unstale":"2024-04-25T08:30:30.302154+0000","last_undegraded":"2024-04-25T08:30:30.302154+0000","last_fullsized":"2024-04-25T08:30:30.302154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323460+0000","last_change":"2024-04-25T08:30:31.318757+0000","last_active":"2024-04-25T08:30:31.323460+0000","last_peered":"2024-04-25T08:30:31.323460+0000","last_clean":"2024-04-25T08:30:31.323460+0000","last_became_active":"2024-04-25T08:30:31.318482+0000","last_became_peered":"2024-04-25T08:30:31.318482+0000","last_unstale":"2024-04-25T08:30:31.323460+0000","last_undegraded":"2024-04-25T08:30:31.323460+0000","last_fullsized":"2024-04-25T08:30:31.323460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.716923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.296818+0000","last_change":"2024-04-25T08:30:30.295325+0000","last_active":"2024-04-25T08:30:30.296818+0000","last_peered":"2024-04-25T08:30:30.296818+0000","last_clean":"2024-04-25T08:30:30.296818+0000","last_became_active":"2024-04-25T08:30:30.295030+0000","last_became_peered":"2024-04-25T08:30:30.295030+0000","last_unstale":"2024-04-25T08:30:30.296818+0000","last_undegraded":"2024-04-25T08:30:30.296818+0000","last_fullsized":"2024-04-25T08:30:30.296818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341677+0000","last_change":"2024-04-25T08:30:31.340453+0000","last_active":"2024-04-25T08:30:31.341677+0000","last_peered":"2024-04-25T08:30:31.341677+0000","last_clean":"2024-04-25T08:30:31.341677+0000","last_became_active":"2024-04-25T08:30:31.340188+0000","last_became_peered":"2024-04-25T08:30:31.340188+0000","last_unstale":"2024-04-25T08:30:31.341677+0000","last_undegraded":"2024-04-25T08:30:31.341677+0000","last_fullsized":"2024-04-25T08:30:31.341677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311406+0000","last_change":"2024-04-25T08:30:30.300697+0000","last_active":"2024-04-25T08:30:31.311406+0000","last_peered":"2024-04-25T08:30:31.311406+0000","last_clean":"2024-04-25T08:30:31.311406+0000","last_became_active":"2024-04-25T08:30:30.300410+0000","last_became_peered":"2024-04-25T08:30:30.300410+0000","last_unstale":"2024-04-25T08:30:31.311406+0000","last_undegraded":"2024-04-25T08:30:31.311406+0000","last_fullsized":"2024-04-25T08:30:31.311406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:38.298492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310971+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:30:31.310971+0000","last_peered":"2024-04-25T08:30:31.310971+0000","last_clean":"2024-04-25T08:30:31.310971+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:30:31.310971+0000","last_undegraded":"2024-04-25T08:30:31.310971+0000","last_fullsized":"2024-04-25T08:30:31.310971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":763701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:30:34.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:30:34.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.446+0000 7f65fb3f9640 1 Processor -- start 2024-04-25T08:30:34.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.446+0000 7f65fb3f9640 1 -- start start 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65fb3f9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65f40cedf0 0x7f65f41616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65fb3f9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65f40b93a0 0x7f65f40d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65fb3f9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f415e000 con 0x7f65f415dc20 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65fb3f9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f40b6bb0 con 0x7f65f40b93a0 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65fb3f9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f40b6030 con 0x7f65f40cedf0 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65f40b93a0 0x7f65f40d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f65f40b93a0 0x7f65f40d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45918/0 (socket says 172.21.15.60:45918) 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 -- 172.21.15.60:0/4092282571 learned_addr learned my addr 172.21.15.60:0/4092282571 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f8994640 1 --2- 172.21.15.60:0/4092282571 >> v2:172.21.15.100:3300/0 conn(0x7f65f40cedf0 0x7f65f41616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 -- 172.21.15.60:0/4092282571 >> v2:172.21.15.100:3300/0 conn(0x7f65f40cedf0 msgr2=0x7f65f41616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 --2- 172.21.15.60:0/4092282571 >> v2:172.21.15.100:3300/0 conn(0x7f65f40cedf0 0x7f65f41616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 -- 172.21.15.60:0/4092282571 >> v1:172.21.15.100:6789/0 conn(0x7f65f415dc20 legacy=0x7f65f40ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f9996640 1 -- 172.21.15.60:0/4092282571 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4161de0 con 0x7f65f40b93a0 2024-04-25T08:30:34.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.447+0000 7f65f8994640 1 --2- 172.21.15.60:0/4092282571 >> v2:172.21.15.100:3300/0 conn(0x7f65f40cedf0 0x7f65f41616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:34.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.448+0000 7f65f9996640 1 --2- 172.21.15.60:0/4092282571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40b93a0 0x7f65f40d2510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f65f40d4b10 tx=0x7f65e402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e7e7e114b28ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.448+0000 7f65f3fff640 1 -- 172.21.15.60:0/4092282571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4036070 con 0x7f65f40b93a0 2024-04-25T08:30:34.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.448+0000 7f65f3fff640 1 -- 172.21.15.60:0/4092282571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4031600 con 0x7f65f40b93a0 2024-04-25T08:30:34.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.448+0000 7f65f3fff640 1 -- 172.21.15.60:0/4092282571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40319d0 con 0x7f65f40b93a0 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 -- 172.21.15.60:0/4092282571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40b93a0 msgr2=0x7f65f40d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/4092282571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40b93a0 0x7f65f40d2510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f65f40d4b10 tx=0x7f65e402a730 comp rx=0 tx=0).stop 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 -- 172.21.15.60:0/4092282571 shutdown_connections 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/4092282571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40b93a0 0x7f65f40d2510 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/4092282571 >> v2:172.21.15.100:3300/0 conn(0x7f65f40cedf0 0x7f65f41616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 -- 172.21.15.60:0/4092282571 >> 172.21.15.60:0/4092282571 conn(0x7f65f40d0490 msgr2=0x7f65f40d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 -- 172.21.15.60:0/4092282571 shutdown_connections 2024-04-25T08:30:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.449+0000 7f65fb3f9640 1 -- 172.21.15.60:0/4092282571 wait complete. 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.450+0000 7f65fb3f9640 1 Processor -- start 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.450+0000 7f65fb3f9640 1 -- start start 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65fb3f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f40b93a0 0x7f65f4160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65fb3f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 0x7f65f40aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65fb3f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f415dc20 0x7f65f4152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65fb3f9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40d2a50 con 0x7f65f40cedf0 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65fb3f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f41623a0 con 0x7f65f415dc20 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65fb3f9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f40d2bd0 con 0x7f65f40b93a0 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 0x7f65f40aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 0x7f65f40aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45926/0 (socket says 172.21.15.60:45926) 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 -- 172.21.15.60:0/3234776263 learned_addr learned my addr 172.21.15.60:0/3234776263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f9195640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f40b93a0 0x7f65f4160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f9996640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f415dc20 0x7f65f4152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 -- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f40b93a0 msgr2=0x7f65f4160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f40b93a0 0x7f65f4160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 -- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f415dc20 msgr2=0x7f65f4152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f415dc20 0x7f65f4152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 -- 172.21.15.60:0/3234776263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f415f440 con 0x7f65f40cedf0 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.451+0000 7f65f8994640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 0x7f65f40aa370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e810 tx=0x7f65e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.452+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e800f040 con 0x7f65f40cedf0 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.452+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8004030 con 0x7f65f40cedf0 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.452+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8004400 con 0x7f65f40cedf0 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.452+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f415f720 con 0x7f65f40cedf0 2024-04-25T08:30:34.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.452+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f4160200 con 0x7f65f40cedf0 2024-04-25T08:30:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.453+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f40d19a0 con 0x7f65f40cedf0 2024-04-25T08:30:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.453+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65e800c6d0 con 0x7f65f40cedf0 2024-04-25T08:30:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.453+0000 7f65f1ffb640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f65e006b500 0x7f65e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.454+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65e8089ff0 con 0x7f65f40cedf0 2024-04-25T08:30:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.454+0000 7f65f9195640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f65e006b500 0x7f65e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.454+0000 7f65f9195640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f65e006b500 0x7f65e006d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0016e0 tx=0x7f65ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.458+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65f40d19a0 con 0x7f65f40cedf0 2024-04-25T08:30:34.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.636+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65f40d2510 con 0x7f65e006b500 2024-04-25T08:30:34.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.643+0000 7f65f1ffb640 1 -- 172.21.15.60:0/3234776263 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248860 (secure 0 0 0) 0x7f65f40d2510 con 0x7f65e006b500 2024-04-25T08:30:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:34.644 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:30:34.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f65e006b500 msgr2=0x7f65e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f65e006b500 0x7f65e006d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0016e0 tx=0x7f65ec009040 comp rx=0 tx=0).stop 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 msgr2=0x7f65f40aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 0x7f65f40aa370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e810 tx=0x7f65e800ecd0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 shutdown_connections 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f65e006b500 0x7f65e006d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f65f415dc20 0x7f65f4152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f65f40cedf0 0x7f65f40aa370 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 --2- 172.21.15.60:0/3234776263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65f40b93a0 0x7f65f4160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.646+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 >> 172.21.15.60:0/3234776263 conn(0x7f65f40d0490 msgr2=0x7f65f40b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.647+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 shutdown_connections 2024-04-25T08:30:34.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.647+0000 7f65fb3f9640 1 -- 172.21.15.60:0/3234776263 wait complete. 2024-04-25T08:30:34.654 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-25T08:30:33.413583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237196,"kb_used_data":4396,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737524,"statfs":{"total":773094113280,"available":772851224576,"internally_reserved":0,"allocated":4501504,"data_stored":2587401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.132042"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292388+0000","last_change":"2024-04-25T08:30:30.292388+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292388+0000","last_undegraded":"2024-04-25T08:30:30.292388+0000","last_fullsized":"2024-04-25T08:30:30.292388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293940+0000","last_change":"2024-04-25T08:30:30.293940+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293940+0000","last_undegraded":"2024-04-25T08:30:30.293940+0000","last_fullsized":"2024-04-25T08:30:30.293940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287912+0000","last_change":"2024-04-25T08:30:30.287912+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287912+0000","last_undegraded":"2024-04-25T08:30:30.287912+0000","last_fullsized":"2024-04-25T08:30:30.287912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286932+0000","last_change":"2024-04-25T08:30:30.286932+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286932+0000","last_undegraded":"2024-04-25T08:30:30.286932+0000","last_fullsized":"2024-04-25T08:30:30.286932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286790+0000","last_change":"2024-04-25T08:30:30.286790+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286790+0000","last_undegraded":"2024-04-25T08:30:30.286790+0000","last_fullsized":"2024-04-25T08:30:30.286790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292036+0000","last_change":"2024-04-25T08:30:30.292036+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292036+0000","last_undegraded":"2024-04-25T08:30:30.292036+0000","last_fullsized":"2024-04-25T08:30:30.292036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341093+0000","last_change":"2024-04-25T08:30:31.339672+0000","last_active":"2024-04-25T08:30:31.341093+0000","last_peered":"2024-04-25T08:30:31.341093+0000","last_clean":"2024-04-25T08:30:31.341093+0000","last_became_active":"2024-04-25T08:30:31.339395+0000","last_became_peered":"2024-04-25T08:30:31.339395+0000","last_unstale":"2024-04-25T08:30:31.341093+0000","last_undegraded":"2024-04-25T08:30:31.341093+0000","last_fullsized":"2024-04-25T08:30:31.341093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.788878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289376+0000","last_change":"2024-04-25T08:30:30.289376+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289376+0000","last_undegraded":"2024-04-25T08:30:30.289376+0000","last_fullsized":"2024-04-25T08:30:30.289376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287333+0000","last_change":"2024-04-25T08:30:30.287333+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287333+0000","last_undegraded":"2024-04-25T08:30:30.287333+0000","last_fullsized":"2024-04-25T08:30:30.287333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.336655+0000","last_change":"2024-04-25T08:30:31.331300+0000","last_active":"2024-04-25T08:30:31.336655+0000","last_peered":"2024-04-25T08:30:31.336655+0000","last_clean":"2024-04-25T08:30:31.336655+0000","last_became_active":"2024-04-25T08:30:31.330974+0000","last_became_peered":"2024-04-25T08:30:31.330974+0000","last_unstale":"2024-04-25T08:30:31.336655+0000","last_undegraded":"2024-04-25T08:30:31.336655+0000","last_fullsized":"2024-04-25T08:30:31.336655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:43.081894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340959+0000","last_change":"2024-04-25T08:30:31.336378+0000","last_active":"2024-04-25T08:30:31.340959+0000","last_peered":"2024-04-25T08:30:31.340959+0000","last_clean":"2024-04-25T08:30:31.340959+0000","last_became_active":"2024-04-25T08:30:31.335915+0000","last_became_peered":"2024-04-25T08:30:31.335915+0000","last_unstale":"2024-04-25T08:30:31.340959+0000","last_undegraded":"2024-04-25T08:30:31.340959+0000","last_fullsized":"2024-04-25T08:30:31.340959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:47.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287710+0000","last_change":"2024-04-25T08:30:30.287710+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287710+0000","last_undegraded":"2024-04-25T08:30:30.287710+0000","last_fullsized":"2024-04-25T08:30:30.287710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337039+0000","last_change":"2024-04-25T08:30:31.331560+0000","last_active":"2024-04-25T08:30:31.337039+0000","last_peered":"2024-04-25T08:30:31.337039+0000","last_clean":"2024-04-25T08:30:31.337039+0000","last_became_active":"2024-04-25T08:30:31.331207+0000","last_became_peered":"2024-04-25T08:30:31.331207+0000","last_unstale":"2024-04-25T08:30:31.337039+0000","last_undegraded":"2024-04-25T08:30:31.337039+0000","last_fullsized":"2024-04-25T08:30:31.337039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:12.355397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288425+0000","last_change":"2024-04-25T08:30:30.288425+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288425+0000","last_undegraded":"2024-04-25T08:30:30.288425+0000","last_fullsized":"2024-04-25T08:30:30.288425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340097+0000","last_change":"2024-04-25T08:30:31.334369+0000","last_active":"2024-04-25T08:30:31.340097+0000","last_peered":"2024-04-25T08:30:31.340097+0000","last_clean":"2024-04-25T08:30:31.340097+0000","last_became_active":"2024-04-25T08:30:31.334071+0000","last_became_peered":"2024-04-25T08:30:31.334071+0000","last_unstale":"2024-04-25T08:30:31.340097+0000","last_undegraded":"2024-04-25T08:30:31.340097+0000","last_fullsized":"2024-04-25T08:30:31.340097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.453764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332186+0000","last_change":"2024-04-25T08:30:31.331292+0000","last_active":"2024-04-25T08:30:31.332186+0000","last_peered":"2024-04-25T08:30:31.332186+0000","last_clean":"2024-04-25T08:30:31.332186+0000","last_became_active":"2024-04-25T08:30:31.330966+0000","last_became_peered":"2024-04-25T08:30:31.330966+0000","last_unstale":"2024-04-25T08:30:31.332186+0000","last_undegraded":"2024-04-25T08:30:31.332186+0000","last_fullsized":"2024-04-25T08:30:31.332186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332626+0000","last_change":"2024-04-25T08:30:31.331334+0000","last_active":"2024-04-25T08:30:31.332626+0000","last_peered":"2024-04-25T08:30:31.332626+0000","last_clean":"2024-04-25T08:30:31.332626+0000","last_became_active":"2024-04-25T08:30:31.331048+0000","last_became_peered":"2024-04-25T08:30:31.331048+0000","last_unstale":"2024-04-25T08:30:31.332626+0000","last_undegraded":"2024-04-25T08:30:31.332626+0000","last_fullsized":"2024-04-25T08:30:31.332626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325080+0000","last_change":"2024-04-25T08:30:31.322980+0000","last_active":"2024-04-25T08:30:31.325080+0000","last_peered":"2024-04-25T08:30:31.325080+0000","last_clean":"2024-04-25T08:30:31.325080+0000","last_became_active":"2024-04-25T08:30:31.322718+0000","last_became_peered":"2024-04-25T08:30:31.322718+0000","last_unstale":"2024-04-25T08:30:31.325080+0000","last_undegraded":"2024-04-25T08:30:31.325080+0000","last_fullsized":"2024-04-25T08:30:31.325080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:34.413297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286631+0000","last_change":"2024-04-25T08:30:30.286631+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286631+0000","last_undegraded":"2024-04-25T08:30:30.286631+0000","last_fullsized":"2024-04-25T08:30:30.286631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323347+0000","last_change":"2024-04-25T08:30:31.321932+0000","last_active":"2024-04-25T08:30:31.323347+0000","last_peered":"2024-04-25T08:30:31.323347+0000","last_clean":"2024-04-25T08:30:31.323347+0000","last_became_active":"2024-04-25T08:30:31.321696+0000","last_became_peered":"2024-04-25T08:30:31.321696+0000","last_unstale":"2024-04-25T08:30:31.323347+0000","last_undegraded":"2024-04-25T08:30:31.323347+0000","last_fullsized":"2024-04-25T08:30:31.323347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:39.300333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337070+0000","last_change":"2024-04-25T08:30:31.330998+0000","last_active":"2024-04-25T08:30:31.337070+0000","last_peered":"2024-04-25T08:30:31.337070+0000","last_clean":"2024-04-25T08:30:31.337070+0000","last_became_active":"2024-04-25T08:30:31.330539+0000","last_became_peered":"2024-04-25T08:30:31.330539+0000","last_unstale":"2024-04-25T08:30:31.337070+0000","last_undegraded":"2024-04-25T08:30:31.337070+0000","last_fullsized":"2024-04-25T08:30:31.337070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287479+0000","last_change":"2024-04-25T08:30:30.287479+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287479+0000","last_undegraded":"2024-04-25T08:30:30.287479+0000","last_fullsized":"2024-04-25T08:30:30.287479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323285+0000","last_change":"2024-04-25T08:30:31.320712+0000","last_active":"2024-04-25T08:30:31.323285+0000","last_peered":"2024-04-25T08:30:31.323285+0000","last_clean":"2024-04-25T08:30:31.323285+0000","last_became_active":"2024-04-25T08:30:31.320446+0000","last_became_peered":"2024-04-25T08:30:31.320446+0000","last_unstale":"2024-04-25T08:30:31.323285+0000","last_undegraded":"2024-04-25T08:30:31.323285+0000","last_fullsized":"2024-04-25T08:30:31.323285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.070232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286827+0000","last_change":"2024-04-25T08:30:30.286827+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286827+0000","last_undegraded":"2024-04-25T08:30:30.286827+0000","last_fullsized":"2024-04-25T08:30:30.286827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.327272+0000","last_change":"2024-04-25T08:30:31.324771+0000","last_active":"2024-04-25T08:30:31.327272+0000","last_peered":"2024-04-25T08:30:31.327272+0000","last_clean":"2024-04-25T08:30:31.327272+0000","last_became_active":"2024-04-25T08:30:31.324509+0000","last_became_peered":"2024-04-25T08:30:31.324509+0000","last_unstale":"2024-04-25T08:30:31.327272+0000","last_undegraded":"2024-04-25T08:30:31.327272+0000","last_fullsized":"2024-04-25T08:30:31.327272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:32.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.298366+0000","last_change":"2024-04-25T08:30:30.298366+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.298366+0000","last_undegraded":"2024-04-25T08:30:30.298366+0000","last_fullsized":"2024-04-25T08:30:30.298366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.319706+0000","last_change":"2024-04-25T08:30:31.316804+0000","last_active":"2024-04-25T08:30:31.319706+0000","last_peered":"2024-04-25T08:30:31.319706+0000","last_clean":"2024-04-25T08:30:31.319706+0000","last_became_active":"2024-04-25T08:30:31.316503+0000","last_became_peered":"2024-04-25T08:30:31.316503+0000","last_unstale":"2024-04-25T08:30:31.319706+0000","last_undegraded":"2024-04-25T08:30:31.319706+0000","last_fullsized":"2024-04-25T08:30:31.319706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:43.729410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.339771+0000","last_change":"2024-04-25T08:30:31.331474+0000","last_active":"2024-04-25T08:30:31.339771+0000","last_peered":"2024-04-25T08:30:31.339771+0000","last_clean":"2024-04-25T08:30:31.339771+0000","last_became_active":"2024-04-25T08:30:31.331058+0000","last_became_peered":"2024-04-25T08:30:31.331058+0000","last_unstale":"2024-04-25T08:30:31.339771+0000","last_undegraded":"2024-04-25T08:30:31.339771+0000","last_fullsized":"2024-04-25T08:30:31.339771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320679+0000","last_change":"2024-04-25T08:30:31.319410+0000","last_active":"2024-04-25T08:30:31.320679+0000","last_peered":"2024-04-25T08:30:31.320679+0000","last_clean":"2024-04-25T08:30:31.320679+0000","last_became_active":"2024-04-25T08:30:31.318913+0000","last_became_peered":"2024-04-25T08:30:31.318913+0000","last_unstale":"2024-04-25T08:30:31.320679+0000","last_undegraded":"2024-04-25T08:30:31.320679+0000","last_fullsized":"2024-04-25T08:30:31.320679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:14.713545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320655+0000","last_change":"2024-04-25T08:30:31.318909+0000","last_active":"2024-04-25T08:30:31.320655+0000","last_peered":"2024-04-25T08:30:31.320655+0000","last_clean":"2024-04-25T08:30:31.320655+0000","last_became_active":"2024-04-25T08:30:31.318639+0000","last_became_peered":"2024-04-25T08:30:31.318639+0000","last_unstale":"2024-04-25T08:30:31.320655+0000","last_undegraded":"2024-04-25T08:30:31.320655+0000","last_fullsized":"2024-04-25T08:30:31.320655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:50.624325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323302+0000","last_change":"2024-04-25T08:30:31.321924+0000","last_active":"2024-04-25T08:30:31.323302+0000","last_peered":"2024-04-25T08:30:31.323302+0000","last_clean":"2024-04-25T08:30:31.323302+0000","last_became_active":"2024-04-25T08:30:31.321434+0000","last_became_peered":"2024-04-25T08:30:31.321434+0000","last_unstale":"2024-04-25T08:30:31.323302+0000","last_undegraded":"2024-04-25T08:30:31.323302+0000","last_fullsized":"2024-04-25T08:30:31.323302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289797+0000","last_change":"2024-04-25T08:30:30.289797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289797+0000","last_undegraded":"2024-04-25T08:30:30.289797+0000","last_fullsized":"2024-04-25T08:30:30.289797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338955+0000","last_change":"2024-04-25T08:30:31.332758+0000","last_active":"2024-04-25T08:30:31.338955+0000","last_peered":"2024-04-25T08:30:31.338955+0000","last_clean":"2024-04-25T08:30:31.338955+0000","last_became_active":"2024-04-25T08:30:31.332458+0000","last_became_peered":"2024-04-25T08:30:31.332458+0000","last_unstale":"2024-04-25T08:30:31.338955+0000","last_undegraded":"2024-04-25T08:30:31.338955+0000","last_fullsized":"2024-04-25T08:30:31.338955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340572+0000","last_change":"2024-04-25T08:30:30.294755+0000","last_active":"2024-04-25T08:30:31.340572+0000","last_peered":"2024-04-25T08:30:31.340572+0000","last_clean":"2024-04-25T08:30:31.340572+0000","last_became_active":"2024-04-25T08:30:30.293812+0000","last_became_peered":"2024-04-25T08:30:30.293812+0000","last_unstale":"2024-04-25T08:30:31.340572+0000","last_undegraded":"2024-04-25T08:30:31.340572+0000","last_fullsized":"2024-04-25T08:30:31.340572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.299433+0000","last_change":"2024-04-25T08:30:30.299433+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.299433+0000","last_undegraded":"2024-04-25T08:30:30.299433+0000","last_fullsized":"2024-04-25T08:30:30.299433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304327+0000","last_change":"2024-04-25T08:30:30.301932+0000","last_active":"2024-04-25T08:30:30.304327+0000","last_peered":"2024-04-25T08:30:30.304327+0000","last_clean":"2024-04-25T08:30:30.304327+0000","last_became_active":"2024-04-25T08:30:30.301633+0000","last_became_peered":"2024-04-25T08:30:30.301633+0000","last_unstale":"2024-04-25T08:30:30.304327+0000","last_undegraded":"2024-04-25T08:30:30.304327+0000","last_fullsized":"2024-04-25T08:30:30.304327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286830+0000","last_change":"2024-04-25T08:30:30.286830+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286830+0000","last_undegraded":"2024-04-25T08:30:30.286830+0000","last_fullsized":"2024-04-25T08:30:30.286830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306192+0000","last_change":"2024-04-25T08:30:30.304552+0000","last_active":"2024-04-25T08:30:30.306192+0000","last_peered":"2024-04-25T08:30:30.306192+0000","last_clean":"2024-04-25T08:30:30.306192+0000","last_became_active":"2024-04-25T08:30:30.304168+0000","last_became_peered":"2024-04-25T08:30:30.304168+0000","last_unstale":"2024-04-25T08:30:30.306192+0000","last_undegraded":"2024-04-25T08:30:30.306192+0000","last_fullsized":"2024-04-25T08:30:30.306192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292899+0000","last_change":"2024-04-25T08:30:30.292899+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292899+0000","last_undegraded":"2024-04-25T08:30:30.292899+0000","last_fullsized":"2024-04-25T08:30:30.292899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304321+0000","last_change":"2024-04-25T08:30:30.301217+0000","last_active":"2024-04-25T08:30:30.304321+0000","last_peered":"2024-04-25T08:30:30.304321+0000","last_clean":"2024-04-25T08:30:30.304321+0000","last_became_active":"2024-04-25T08:30:30.300899+0000","last_became_peered":"2024-04-25T08:30:30.300899+0000","last_unstale":"2024-04-25T08:30:30.304321+0000","last_undegraded":"2024-04-25T08:30:30.304321+0000","last_fullsized":"2024-04-25T08:30:30.304321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289501+0000","last_change":"2024-04-25T08:30:30.289501+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289501+0000","last_undegraded":"2024-04-25T08:30:30.289501+0000","last_fullsized":"2024-04-25T08:30:30.289501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338973+0000","last_change":"2024-04-25T08:30:30.302842+0000","last_active":"2024-04-25T08:30:31.338973+0000","last_peered":"2024-04-25T08:30:31.338973+0000","last_clean":"2024-04-25T08:30:31.338973+0000","last_became_active":"2024-04-25T08:30:30.302540+0000","last_became_peered":"2024-04-25T08:30:30.302540+0000","last_unstale":"2024-04-25T08:30:31.338973+0000","last_undegraded":"2024-04-25T08:30:31.338973+0000","last_fullsized":"2024-04-25T08:30:31.338973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.946206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335411+0000","last_change":"2024-04-25T08:30:31.330857+0000","last_active":"2024-04-25T08:30:31.335411+0000","last_peered":"2024-04-25T08:30:31.335411+0000","last_clean":"2024-04-25T08:30:31.335411+0000","last_became_active":"2024-04-25T08:30:31.329563+0000","last_became_peered":"2024-04-25T08:30:31.329563+0000","last_unstale":"2024-04-25T08:30:31.335411+0000","last_undegraded":"2024-04-25T08:30:31.335411+0000","last_fullsized":"2024-04-25T08:30:31.335411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341386+0000","last_change":"2024-04-25T08:30:30.298067+0000","last_active":"2024-04-25T08:30:31.341386+0000","last_peered":"2024-04-25T08:30:31.341386+0000","last_clean":"2024-04-25T08:30:31.341386+0000","last_became_active":"2024-04-25T08:30:30.297718+0000","last_became_peered":"2024-04-25T08:30:30.297718+0000","last_unstale":"2024-04-25T08:30:31.341386+0000","last_undegraded":"2024-04-25T08:30:31.341386+0000","last_fullsized":"2024-04-25T08:30:31.341386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287002+0000","last_change":"2024-04-25T08:30:30.287002+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287002+0000","last_undegraded":"2024-04-25T08:30:30.287002+0000","last_fullsized":"2024-04-25T08:30:30.287002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306219+0000","last_change":"2024-04-25T08:30:30.305406+0000","last_active":"2024-04-25T08:30:30.306219+0000","last_peered":"2024-04-25T08:30:30.306219+0000","last_clean":"2024-04-25T08:30:30.306219+0000","last_became_active":"2024-04-25T08:30:30.305041+0000","last_became_peered":"2024-04-25T08:30:30.305041+0000","last_unstale":"2024-04-25T08:30:30.306219+0000","last_undegraded":"2024-04-25T08:30:30.306219+0000","last_fullsized":"2024-04-25T08:30:30.306219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291475+0000","last_change":"2024-04-25T08:30:30.291475+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291475+0000","last_undegraded":"2024-04-25T08:30:30.291475+0000","last_fullsized":"2024-04-25T08:30:30.291475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295501+0000","last_change":"2024-04-25T08:30:30.290419+0000","last_active":"2024-04-25T08:30:30.295501+0000","last_peered":"2024-04-25T08:30:30.295501+0000","last_clean":"2024-04-25T08:30:30.295501+0000","last_became_active":"2024-04-25T08:30:30.290143+0000","last_became_peered":"2024-04-25T08:30:30.290143+0000","last_unstale":"2024-04-25T08:30:30.295501+0000","last_undegraded":"2024-04-25T08:30:30.295501+0000","last_fullsized":"2024-04-25T08:30:30.295501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:19.910492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286356+0000","last_change":"2024-04-25T08:30:30.286356+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286356+0000","last_undegraded":"2024-04-25T08:30:30.286356+0000","last_fullsized":"2024-04-25T08:30:30.286356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305495+0000","last_change":"2024-04-25T08:30:30.302163+0000","last_active":"2024-04-25T08:30:30.305495+0000","last_peered":"2024-04-25T08:30:30.305495+0000","last_clean":"2024-04-25T08:30:30.305495+0000","last_became_active":"2024-04-25T08:30:30.301848+0000","last_became_peered":"2024-04-25T08:30:30.301848+0000","last_unstale":"2024-04-25T08:30:30.305495+0000","last_undegraded":"2024-04-25T08:30:30.305495+0000","last_fullsized":"2024-04-25T08:30:30.305495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:50.715081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323281+0000","last_change":"2024-04-25T08:30:31.319799+0000","last_active":"2024-04-25T08:30:31.323281+0000","last_peered":"2024-04-25T08:30:31.323281+0000","last_clean":"2024-04-25T08:30:31.323281+0000","last_became_active":"2024-04-25T08:30:31.319152+0000","last_became_peered":"2024-04-25T08:30:31.319152+0000","last_unstale":"2024-04-25T08:30:31.323281+0000","last_undegraded":"2024-04-25T08:30:31.323281+0000","last_fullsized":"2024-04-25T08:30:31.323281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:11.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325859+0000","last_change":"2024-04-25T08:30:30.299038+0000","last_active":"2024-04-25T08:30:31.325859+0000","last_peered":"2024-04-25T08:30:31.325859+0000","last_clean":"2024-04-25T08:30:31.325859+0000","last_became_active":"2024-04-25T08:30:30.298782+0000","last_became_peered":"2024-04-25T08:30:30.298782+0000","last_unstale":"2024-04-25T08:30:31.325859+0000","last_undegraded":"2024-04-25T08:30:31.325859+0000","last_fullsized":"2024-04-25T08:30:31.325859+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:47.771125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287963+0000","last_change":"2024-04-25T08:30:30.287963+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287963+0000","last_undegraded":"2024-04-25T08:30:30.287963+0000","last_fullsized":"2024-04-25T08:30:30.287963+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341021+0000","last_change":"2024-04-25T08:30:30.301686+0000","last_active":"2024-04-25T08:30:31.341021+0000","last_peered":"2024-04-25T08:30:31.341021+0000","last_clean":"2024-04-25T08:30:31.341021+0000","last_became_active":"2024-04-25T08:30:30.301391+0000","last_became_peered":"2024-04-25T08:30:30.301391+0000","last_unstale":"2024-04-25T08:30:31.341021+0000","last_undegraded":"2024-04-25T08:30:31.341021+0000","last_fullsized":"2024-04-25T08:30:31.341021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.852169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288660+0000","last_change":"2024-04-25T08:30:30.288660+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288660+0000","last_undegraded":"2024-04-25T08:30:30.288660+0000","last_fullsized":"2024-04-25T08:30:30.288660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305486+0000","last_change":"2024-04-25T08:30:30.303220+0000","last_active":"2024-04-25T08:30:30.305486+0000","last_peered":"2024-04-25T08:30:30.305486+0000","last_clean":"2024-04-25T08:30:30.305486+0000","last_became_active":"2024-04-25T08:30:30.302914+0000","last_became_peered":"2024-04-25T08:30:30.302914+0000","last_unstale":"2024-04-25T08:30:30.305486+0000","last_undegraded":"2024-04-25T08:30:30.305486+0000","last_fullsized":"2024-04-25T08:30:30.305486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:10.215234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289039+0000","last_change":"2024-04-25T08:30:30.289039+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289039+0000","last_undegraded":"2024-04-25T08:30:30.289039+0000","last_fullsized":"2024-04-25T08:30:30.289039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304930+0000","last_change":"2024-04-25T08:30:30.301164+0000","last_active":"2024-04-25T08:30:30.304930+0000","last_peered":"2024-04-25T08:30:30.304930+0000","last_clean":"2024-04-25T08:30:30.304930+0000","last_became_active":"2024-04-25T08:30:30.300887+0000","last_became_peered":"2024-04-25T08:30:30.300887+0000","last_unstale":"2024-04-25T08:30:30.304930+0000","last_undegraded":"2024-04-25T08:30:30.304930+0000","last_fullsized":"2024-04-25T08:30:30.304930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.290447+0000","last_change":"2024-04-25T08:30:30.290447+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.290447+0000","last_undegraded":"2024-04-25T08:30:30.290447+0000","last_fullsized":"2024-04-25T08:30:30.290447+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306196+0000","last_change":"2024-04-25T08:30:30.304789+0000","last_active":"2024-04-25T08:30:30.306196+0000","last_peered":"2024-04-25T08:30:30.306196+0000","last_clean":"2024-04-25T08:30:30.306196+0000","last_became_active":"2024-04-25T08:30:30.304493+0000","last_became_peered":"2024-04-25T08:30:30.304493+0000","last_unstale":"2024-04-25T08:30:30.306196+0000","last_undegraded":"2024-04-25T08:30:30.306196+0000","last_fullsized":"2024-04-25T08:30:30.306196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330098+0000","last_change":"2024-04-25T08:30:31.326518+0000","last_active":"2024-04-25T08:30:31.330098+0000","last_peered":"2024-04-25T08:30:31.330098+0000","last_clean":"2024-04-25T08:30:31.330098+0000","last_became_active":"2024-04-25T08:30:31.326241+0000","last_became_peered":"2024-04-25T08:30:31.326241+0000","last_unstale":"2024-04-25T08:30:31.330098+0000","last_undegraded":"2024-04-25T08:30:31.330098+0000","last_fullsized":"2024-04-25T08:30:31.330098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310441+0000","last_change":"2024-04-25T08:30:30.294767+0000","last_active":"2024-04-25T08:30:31.310441+0000","last_peered":"2024-04-25T08:30:31.310441+0000","last_clean":"2024-04-25T08:30:31.310441+0000","last_became_active":"2024-04-25T08:30:30.294475+0000","last_became_peered":"2024-04-25T08:30:30.294475+0000","last_unstale":"2024-04-25T08:30:31.310441+0000","last_undegraded":"2024-04-25T08:30:31.310441+0000","last_fullsized":"2024-04-25T08:30:31.310441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291037+0000","last_change":"2024-04-25T08:30:30.291037+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291037+0000","last_undegraded":"2024-04-25T08:30:30.291037+0000","last_fullsized":"2024-04-25T08:30:30.291037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.342812+0000","last_change":"2024-04-25T08:30:30.297207+0000","last_active":"2024-04-25T08:30:31.342812+0000","last_peered":"2024-04-25T08:30:31.342812+0000","last_clean":"2024-04-25T08:30:31.342812+0000","last_became_active":"2024-04-25T08:30:30.296922+0000","last_became_peered":"2024-04-25T08:30:30.296922+0000","last_unstale":"2024-04-25T08:30:31.342812+0000","last_undegraded":"2024-04-25T08:30:31.342812+0000","last_fullsized":"2024-04-25T08:30:31.342812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:13.711704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.297011+0000","last_change":"2024-04-25T08:30:30.297011+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.297011+0000","last_undegraded":"2024-04-25T08:30:30.297011+0000","last_fullsized":"2024-04-25T08:30:30.297011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340937+0000","last_change":"2024-04-25T08:30:30.297702+0000","last_active":"2024-04-25T08:30:31.340937+0000","last_peered":"2024-04-25T08:30:31.340937+0000","last_clean":"2024-04-25T08:30:31.340937+0000","last_became_active":"2024-04-25T08:30:30.297420+0000","last_became_peered":"2024-04-25T08:30:30.297420+0000","last_unstale":"2024-04-25T08:30:31.340937+0000","last_undegraded":"2024-04-25T08:30:31.340937+0000","last_fullsized":"2024-04-25T08:30:31.340937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.137643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341074+0000","last_change":"2024-04-25T08:30:31.337230+0000","last_active":"2024-04-25T08:30:31.341074+0000","last_peered":"2024-04-25T08:30:31.341074+0000","last_clean":"2024-04-25T08:30:31.341074+0000","last_became_active":"2024-04-25T08:30:31.336785+0000","last_became_peered":"2024-04-25T08:30:31.336785+0000","last_unstale":"2024-04-25T08:30:31.341074+0000","last_undegraded":"2024-04-25T08:30:31.341074+0000","last_fullsized":"2024-04-25T08:30:31.341074+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:28.421465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301349+0000","last_change":"2024-04-25T08:30:30.296836+0000","last_active":"2024-04-25T08:30:30.301349+0000","last_peered":"2024-04-25T08:30:30.301349+0000","last_clean":"2024-04-25T08:30:30.301349+0000","last_became_active":"2024-04-25T08:30:30.296490+0000","last_became_peered":"2024-04-25T08:30:30.296490+0000","last_unstale":"2024-04-25T08:30:30.301349+0000","last_undegraded":"2024-04-25T08:30:30.301349+0000","last_fullsized":"2024-04-25T08:30:30.301349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289132+0000","last_change":"2024-04-25T08:30:30.289132+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289132+0000","last_undegraded":"2024-04-25T08:30:30.289132+0000","last_fullsized":"2024-04-25T08:30:30.289132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.300070+0000","last_change":"2024-04-25T08:30:30.297703+0000","last_active":"2024-04-25T08:30:30.300070+0000","last_peered":"2024-04-25T08:30:30.300070+0000","last_clean":"2024-04-25T08:30:30.300070+0000","last_became_active":"2024-04-25T08:30:30.297424+0000","last_became_peered":"2024-04-25T08:30:30.297424+0000","last_unstale":"2024-04-25T08:30:30.300070+0000","last_undegraded":"2024-04-25T08:30:30.300070+0000","last_fullsized":"2024-04-25T08:30:30.300070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.320688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286213+0000","last_change":"2024-04-25T08:30:30.286213+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286213+0000","last_undegraded":"2024-04-25T08:30:30.286213+0000","last_fullsized":"2024-04-25T08:30:30.286213+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305672+0000","last_change":"2024-04-25T08:30:30.303603+0000","last_active":"2024-04-25T08:30:30.305672+0000","last_peered":"2024-04-25T08:30:30.305672+0000","last_clean":"2024-04-25T08:30:30.305672+0000","last_became_active":"2024-04-25T08:30:30.303322+0000","last_became_peered":"2024-04-25T08:30:30.303322+0000","last_unstale":"2024-04-25T08:30:30.305672+0000","last_undegraded":"2024-04-25T08:30:30.305672+0000","last_fullsized":"2024-04-25T08:30:30.305672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330062+0000","last_change":"2024-04-25T08:30:31.325746+0000","last_active":"2024-04-25T08:30:31.330062+0000","last_peered":"2024-04-25T08:30:31.330062+0000","last_clean":"2024-04-25T08:30:31.330062+0000","last_became_active":"2024-04-25T08:30:31.325449+0000","last_became_peered":"2024-04-25T08:30:31.325449+0000","last_unstale":"2024-04-25T08:30:31.330062+0000","last_undegraded":"2024-04-25T08:30:31.330062+0000","last_fullsized":"2024-04-25T08:30:31.330062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295651+0000","last_change":"2024-04-25T08:30:30.293002+0000","last_active":"2024-04-25T08:30:30.295651+0000","last_peered":"2024-04-25T08:30:30.295651+0000","last_clean":"2024-04-25T08:30:30.295651+0000","last_became_active":"2024-04-25T08:30:30.292702+0000","last_became_peered":"2024-04-25T08:30:30.292702+0000","last_unstale":"2024-04-25T08:30:30.295651+0000","last_undegraded":"2024-04-25T08:30:30.295651+0000","last_fullsized":"2024-04-25T08:30:30.295651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.414247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287883+0000","last_change":"2024-04-25T08:30:30.287883+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287883+0000","last_undegraded":"2024-04-25T08:30:30.287883+0000","last_fullsized":"2024-04-25T08:30:30.287883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295421+0000","last_change":"2024-04-25T08:30:30.292431+0000","last_active":"2024-04-25T08:30:30.295421+0000","last_peered":"2024-04-25T08:30:30.295421+0000","last_clean":"2024-04-25T08:30:30.295421+0000","last_became_active":"2024-04-25T08:30:30.292078+0000","last_became_peered":"2024-04-25T08:30:30.292078+0000","last_unstale":"2024-04-25T08:30:30.295421+0000","last_undegraded":"2024-04-25T08:30:30.295421+0000","last_fullsized":"2024-04-25T08:30:30.295421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:06.079852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335419+0000","last_change":"2024-04-25T08:30:31.331176+0000","last_active":"2024-04-25T08:30:31.335419+0000","last_peered":"2024-04-25T08:30:31.335419+0000","last_clean":"2024-04-25T08:30:31.335419+0000","last_became_active":"2024-04-25T08:30:31.330823+0000","last_became_peered":"2024-04-25T08:30:31.330823+0000","last_unstale":"2024-04-25T08:30:31.335419+0000","last_undegraded":"2024-04-25T08:30:31.335419+0000","last_fullsized":"2024-04-25T08:30:31.335419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:21.292627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311697+0000","last_change":"2024-04-25T08:30:30.299966+0000","last_active":"2024-04-25T08:30:31.311697+0000","last_peered":"2024-04-25T08:30:31.311697+0000","last_clean":"2024-04-25T08:30:31.311697+0000","last_became_active":"2024-04-25T08:30:30.299699+0000","last_became_peered":"2024-04-25T08:30:30.299699+0000","last_unstale":"2024-04-25T08:30:31.311697+0000","last_undegraded":"2024-04-25T08:30:31.311697+0000","last_fullsized":"2024-04-25T08:30:31.311697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289401+0000","last_change":"2024-04-25T08:30:30.289401+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289401+0000","last_undegraded":"2024-04-25T08:30:30.289401+0000","last_fullsized":"2024-04-25T08:30:30.289401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.297186+0000","last_change":"2024-04-25T08:30:30.293467+0000","last_active":"2024-04-25T08:30:30.297186+0000","last_peered":"2024-04-25T08:30:30.297186+0000","last_clean":"2024-04-25T08:30:30.297186+0000","last_became_active":"2024-04-25T08:30:30.293179+0000","last_became_peered":"2024-04-25T08:30:30.293179+0000","last_unstale":"2024-04-25T08:30:30.297186+0000","last_undegraded":"2024-04-25T08:30:30.297186+0000","last_fullsized":"2024-04-25T08:30:30.297186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.629362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.296797+0000","last_change":"2024-04-25T08:30:30.296797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.296797+0000","last_undegraded":"2024-04-25T08:30:30.296797+0000","last_fullsized":"2024-04-25T08:30:30.296797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340294+0000","last_change":"2024-04-25T08:30:30.301760+0000","last_active":"2024-04-25T08:30:31.340294+0000","last_peered":"2024-04-25T08:30:31.340294+0000","last_clean":"2024-04-25T08:30:31.340294+0000","last_became_active":"2024-04-25T08:30:30.301331+0000","last_became_peered":"2024-04-25T08:30:30.301331+0000","last_unstale":"2024-04-25T08:30:31.340294+0000","last_undegraded":"2024-04-25T08:30:31.340294+0000","last_fullsized":"2024-04-25T08:30:31.340294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323304+0000","last_change":"2024-04-25T08:30:31.320048+0000","last_active":"2024-04-25T08:30:31.323304+0000","last_peered":"2024-04-25T08:30:31.323304+0000","last_clean":"2024-04-25T08:30:31.323304+0000","last_became_active":"2024-04-25T08:30:31.319495+0000","last_became_peered":"2024-04-25T08:30:31.319495+0000","last_unstale":"2024-04-25T08:30:31.323304+0000","last_undegraded":"2024-04-25T08:30:31.323304+0000","last_fullsized":"2024-04-25T08:30:31.323304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:05.728351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.299425+0000","last_change":"2024-04-25T08:30:30.296903+0000","last_active":"2024-04-25T08:30:30.299425+0000","last_peered":"2024-04-25T08:30:30.299425+0000","last_clean":"2024-04-25T08:30:30.299425+0000","last_became_active":"2024-04-25T08:30:30.296623+0000","last_became_peered":"2024-04-25T08:30:30.296623+0000","last_unstale":"2024-04-25T08:30:30.299425+0000","last_undegraded":"2024-04-25T08:30:30.299425+0000","last_fullsized":"2024-04-25T08:30:30.299425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:03.007004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293539+0000","last_change":"2024-04-25T08:30:30.293539+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293539+0000","last_undegraded":"2024-04-25T08:30:30.293539+0000","last_fullsized":"2024-04-25T08:30:30.293539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305561+0000","last_change":"2024-04-25T08:30:30.303229+0000","last_active":"2024-04-25T08:30:30.305561+0000","last_peered":"2024-04-25T08:30:30.305561+0000","last_clean":"2024-04-25T08:30:30.305561+0000","last_became_active":"2024-04-25T08:30:30.302894+0000","last_became_peered":"2024-04-25T08:30:30.302894+0000","last_unstale":"2024-04-25T08:30:30.305561+0000","last_undegraded":"2024-04-25T08:30:30.305561+0000","last_fullsized":"2024-04-25T08:30:30.305561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288107+0000","last_change":"2024-04-25T08:30:30.288107+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288107+0000","last_undegraded":"2024-04-25T08:30:30.288107+0000","last_fullsized":"2024-04-25T08:30:30.288107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301519+0000","last_change":"2024-04-25T08:30:30.300280+0000","last_active":"2024-04-25T08:30:30.301519+0000","last_peered":"2024-04-25T08:30:30.301519+0000","last_clean":"2024-04-25T08:30:30.301519+0000","last_became_active":"2024-04-25T08:30:30.299985+0000","last_became_peered":"2024-04-25T08:30:30.299985+0000","last_unstale":"2024-04-25T08:30:30.301519+0000","last_undegraded":"2024-04-25T08:30:30.301519+0000","last_fullsized":"2024-04-25T08:30:30.301519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292115+0000","last_change":"2024-04-25T08:30:30.292115+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292115+0000","last_undegraded":"2024-04-25T08:30:30.292115+0000","last_fullsized":"2024-04-25T08:30:30.292115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.312138+0000","last_change":"2024-04-25T08:30:30.293212+0000","last_active":"2024-04-25T08:30:31.312138+0000","last_peered":"2024-04-25T08:30:31.312138+0000","last_clean":"2024-04-25T08:30:31.312138+0000","last_became_active":"2024-04-25T08:30:30.292931+0000","last_became_peered":"2024-04-25T08:30:30.292931+0000","last_unstale":"2024-04-25T08:30:31.312138+0000","last_undegraded":"2024-04-25T08:30:31.312138+0000","last_fullsized":"2024-04-25T08:30:31.312138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287931+0000","last_change":"2024-04-25T08:30:30.287931+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287931+0000","last_undegraded":"2024-04-25T08:30:30.287931+0000","last_fullsized":"2024-04-25T08:30:30.287931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.302154+0000","last_change":"2024-04-25T08:30:30.298769+0000","last_active":"2024-04-25T08:30:30.302154+0000","last_peered":"2024-04-25T08:30:30.302154+0000","last_clean":"2024-04-25T08:30:30.302154+0000","last_became_active":"2024-04-25T08:30:30.298489+0000","last_became_peered":"2024-04-25T08:30:30.298489+0000","last_unstale":"2024-04-25T08:30:30.302154+0000","last_undegraded":"2024-04-25T08:30:30.302154+0000","last_fullsized":"2024-04-25T08:30:30.302154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323460+0000","last_change":"2024-04-25T08:30:31.318757+0000","last_active":"2024-04-25T08:30:31.323460+0000","last_peered":"2024-04-25T08:30:31.323460+0000","last_clean":"2024-04-25T08:30:31.323460+0000","last_became_active":"2024-04-25T08:30:31.318482+0000","last_became_peered":"2024-04-25T08:30:31.318482+0000","last_unstale":"2024-04-25T08:30:31.323460+0000","last_undegraded":"2024-04-25T08:30:31.323460+0000","last_fullsized":"2024-04-25T08:30:31.323460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.716923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.296818+0000","last_change":"2024-04-25T08:30:30.295325+0000","last_active":"2024-04-25T08:30:30.296818+0000","last_peered":"2024-04-25T08:30:30.296818+0000","last_clean":"2024-04-25T08:30:30.296818+0000","last_became_active":"2024-04-25T08:30:30.295030+0000","last_became_peered":"2024-04-25T08:30:30.295030+0000","last_unstale":"2024-04-25T08:30:30.296818+0000","last_undegraded":"2024-04-25T08:30:30.296818+0000","last_fullsized":"2024-04-25T08:30:30.296818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341677+0000","last_change":"2024-04-25T08:30:31.340453+0000","last_active":"2024-04-25T08:30:31.341677+0000","last_peered":"2024-04-25T08:30:31.341677+0000","last_clean":"2024-04-25T08:30:31.341677+0000","last_became_active":"2024-04-25T08:30:31.340188+0000","last_became_peered":"2024-04-25T08:30:31.340188+0000","last_unstale":"2024-04-25T08:30:31.341677+0000","last_undegraded":"2024-04-25T08:30:31.341677+0000","last_fullsized":"2024-04-25T08:30:31.341677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311406+0000","last_change":"2024-04-25T08:30:30.300697+0000","last_active":"2024-04-25T08:30:31.311406+0000","last_peered":"2024-04-25T08:30:31.311406+0000","last_clean":"2024-04-25T08:30:31.311406+0000","last_became_active":"2024-04-25T08:30:30.300410+0000","last_became_peered":"2024-04-25T08:30:30.300410+0000","last_unstale":"2024-04-25T08:30:31.311406+0000","last_undegraded":"2024-04-25T08:30:31.311406+0000","last_fullsized":"2024-04-25T08:30:31.311406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:38.298492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310971+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:30:31.310971+0000","last_peered":"2024-04-25T08:30:31.310971+0000","last_clean":"2024-04-25T08:30:31.310971+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:30:31.310971+0000","last_undegraded":"2024-04-25T08:30:31.310971+0000","last_fullsized":"2024-04-25T08:30:31.310971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":763701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:30:34.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:30:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.800+0000 7f7a53876640 1 Processor -- start 2024-04-25T08:30:34.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.800+0000 7f7a53876640 1 -- start start 2024-04-25T08:30:34.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a53876640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a53876640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a4c0d1020 0x7f7a4c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a53876640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0b97a0 con 0x7f7a4c15dbe0 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a53876640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c15dfc0 con 0x7f7a4c0d1020 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a53876640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0d1a00 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a4c0d1020 0x7f7a4c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7a4c0d1020 0x7f7a4c0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45934/0 (socket says 172.21.15.60:45934) 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 -- 172.21.15.60:0/3082548480 learned_addr learned my addr 172.21.15.60:0/3082548480 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a50e11640 1 --2- 172.21.15.60:0/3082548480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 -- 172.21.15.60:0/3082548480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c0b2cb0 msgr2=0x7f7a4c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 --2- 172.21.15.60:0/3082548480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 -- 172.21.15.60:0/3082548480 >> v1:172.21.15.100:6789/0 conn(0x7f7a4c15dbe0 legacy=0x7f7a4c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.801+0000 7f7a51e13640 1 -- 172.21.15.60:0/3082548480 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0d1c50 con 0x7f7a4c0d1020 2024-04-25T08:30:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.802+0000 7f7a51e13640 1 --2- 172.21.15.60:0/3082548480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0d2c70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009e50 tx=0x7f7a4402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729d704131f57399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.802+0000 7f7a43fff640 1 -- 172.21.15.60:0/3082548480 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44036070 con 0x7f7a4c0d1020 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.802+0000 7f7a43fff640 1 -- 172.21.15.60:0/3082548480 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a44031600 con 0x7f7a4c0d1020 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.802+0000 7f7a43fff640 1 -- 172.21.15.60:0/3082548480 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a440319d0 con 0x7f7a4c0d1020 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.802+0000 7f7a53876640 1 -- 172.21.15.60:0/3082548480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0d1020 msgr2=0x7f7a4c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.802+0000 7f7a53876640 1 --2- 172.21.15.60:0/3082548480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0d2c70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009e50 tx=0x7f7a4402a710 comp rx=0 tx=0).stop 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.803+0000 7f7a53876640 1 -- 172.21.15.60:0/3082548480 shutdown_connections 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.803+0000 7f7a53876640 1 --2- 172.21.15.60:0/3082548480 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0d2c70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.803+0000 7f7a53876640 1 --2- 172.21.15.60:0/3082548480 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.803+0000 7f7a53876640 1 -- 172.21.15.60:0/3082548480 >> 172.21.15.60:0/3082548480 conn(0x7f7a4c061a20 msgr2=0x7f7a4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.803+0000 7f7a53876640 1 -- 172.21.15.60:0/3082548480 shutdown_connections 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.803+0000 7f7a53876640 1 -- 172.21.15.60:0/3082548480 wait complete. 2024-04-25T08:30:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.804+0000 7f7a53876640 1 Processor -- start 2024-04-25T08:30:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.804+0000 7f7a53876640 1 -- start start 2024-04-25T08:30:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c15dbe0 0x7f7a4c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0b92a0 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0d2210 con 0x7f7a4c0d1020 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0b9420 con 0x7f7a4c15dbe0 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45942/0 (socket says 172.21.15.60:45942) 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 -- 172.21.15.60:0/4052672742 learned_addr learned my addr 172.21.15.60:0/4052672742 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51e13640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c15dbe0 0x7f7a4c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a50e11640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 -- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c15dbe0 msgr2=0x7f7a4c0caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c15dbe0 0x7f7a4c0caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 -- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c0d1020 msgr2=0x7f7a4c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 -- 172.21.15.60:0/4052672742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0cd520 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a51612640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0b59d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c3c0 tx=0x7f7a4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a480052d0 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48005490 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c0cd800 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.805+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48007260 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.806+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c149340 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.807+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c073990 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.807+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a48007400 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.807+0000 7f7a41ffb640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7a3006b4d0 0x7f7a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:34.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.807+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a48089800 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.808+0000 7f7a50e11640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7a3006b4d0 0x7f7a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:34.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.808+0000 7f7a50e11640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7a3006b4d0 0x7f7a3006d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0ca4d0 tx=0x7f7a3c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.812+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a48057e10 con 0x7f7a4c0b2cb0 2024-04-25T08:30:34.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.987+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a4c0b5f10 con 0x7f7a3006b4d0 2024-04-25T08:30:34.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.993+0000 7f7a41ffb640 1 -- 172.21.15.60:0/4052672742 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248860 (secure 0 0 0) 0x7f7a4c0b5f10 con 0x7f7a3006b4d0 2024-04-25T08:30:34.993 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:34.994 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:30:34.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7a3006b4d0 msgr2=0x7f7a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7a3006b4d0 0x7f7a3006d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0ca4d0 tx=0x7f7a3c008040 comp rx=0 tx=0).stop 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 msgr2=0x7f7a4c0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0b59d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c3c0 tx=0x7f7a4800c880 comp rx=0 tx=0).stop 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 shutdown_connections 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7a4c15dbe0 0x7f7a4c0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7a3006b4d0 0x7f7a3006d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c0d1020 0x7f7a4c0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 --2- 172.21.15.60:0/4052672742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a4c0b2cb0 0x7f7a4c0b59d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 >> 172.21.15.60:0/4052672742 conn(0x7f7a4c061a20 msgr2=0x7f7a4c061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 shutdown_connections 2024-04-25T08:30:34.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:34.996+0000 7f7a53876640 1 -- 172.21.15.60:0/4052672742 wait complete. 2024-04-25T08:30:35.004 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-04-25T08:30:33.413583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237196,"kb_used_data":4396,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737524,"statfs":{"total":773094113280,"available":772851224576,"internally_reserved":0,"allocated":4501504,"data_stored":2587401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.132042"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292388+0000","last_change":"2024-04-25T08:30:30.292388+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292388+0000","last_undegraded":"2024-04-25T08:30:30.292388+0000","last_fullsized":"2024-04-25T08:30:30.292388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293940+0000","last_change":"2024-04-25T08:30:30.293940+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293940+0000","last_undegraded":"2024-04-25T08:30:30.293940+0000","last_fullsized":"2024-04-25T08:30:30.293940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287912+0000","last_change":"2024-04-25T08:30:30.287912+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287912+0000","last_undegraded":"2024-04-25T08:30:30.287912+0000","last_fullsized":"2024-04-25T08:30:30.287912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286932+0000","last_change":"2024-04-25T08:30:30.286932+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286932+0000","last_undegraded":"2024-04-25T08:30:30.286932+0000","last_fullsized":"2024-04-25T08:30:30.286932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286790+0000","last_change":"2024-04-25T08:30:30.286790+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286790+0000","last_undegraded":"2024-04-25T08:30:30.286790+0000","last_fullsized":"2024-04-25T08:30:30.286790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292036+0000","last_change":"2024-04-25T08:30:30.292036+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292036+0000","last_undegraded":"2024-04-25T08:30:30.292036+0000","last_fullsized":"2024-04-25T08:30:30.292036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341093+0000","last_change":"2024-04-25T08:30:31.339672+0000","last_active":"2024-04-25T08:30:31.341093+0000","last_peered":"2024-04-25T08:30:31.341093+0000","last_clean":"2024-04-25T08:30:31.341093+0000","last_became_active":"2024-04-25T08:30:31.339395+0000","last_became_peered":"2024-04-25T08:30:31.339395+0000","last_unstale":"2024-04-25T08:30:31.341093+0000","last_undegraded":"2024-04-25T08:30:31.341093+0000","last_fullsized":"2024-04-25T08:30:31.341093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.788878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289376+0000","last_change":"2024-04-25T08:30:30.289376+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289376+0000","last_undegraded":"2024-04-25T08:30:30.289376+0000","last_fullsized":"2024-04-25T08:30:30.289376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287333+0000","last_change":"2024-04-25T08:30:30.287333+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287333+0000","last_undegraded":"2024-04-25T08:30:30.287333+0000","last_fullsized":"2024-04-25T08:30:30.287333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.336655+0000","last_change":"2024-04-25T08:30:31.331300+0000","last_active":"2024-04-25T08:30:31.336655+0000","last_peered":"2024-04-25T08:30:31.336655+0000","last_clean":"2024-04-25T08:30:31.336655+0000","last_became_active":"2024-04-25T08:30:31.330974+0000","last_became_peered":"2024-04-25T08:30:31.330974+0000","last_unstale":"2024-04-25T08:30:31.336655+0000","last_undegraded":"2024-04-25T08:30:31.336655+0000","last_fullsized":"2024-04-25T08:30:31.336655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:43.081894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340959+0000","last_change":"2024-04-25T08:30:31.336378+0000","last_active":"2024-04-25T08:30:31.340959+0000","last_peered":"2024-04-25T08:30:31.340959+0000","last_clean":"2024-04-25T08:30:31.340959+0000","last_became_active":"2024-04-25T08:30:31.335915+0000","last_became_peered":"2024-04-25T08:30:31.335915+0000","last_unstale":"2024-04-25T08:30:31.340959+0000","last_undegraded":"2024-04-25T08:30:31.340959+0000","last_fullsized":"2024-04-25T08:30:31.340959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:47.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287710+0000","last_change":"2024-04-25T08:30:30.287710+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287710+0000","last_undegraded":"2024-04-25T08:30:30.287710+0000","last_fullsized":"2024-04-25T08:30:30.287710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337039+0000","last_change":"2024-04-25T08:30:31.331560+0000","last_active":"2024-04-25T08:30:31.337039+0000","last_peered":"2024-04-25T08:30:31.337039+0000","last_clean":"2024-04-25T08:30:31.337039+0000","last_became_active":"2024-04-25T08:30:31.331207+0000","last_became_peered":"2024-04-25T08:30:31.331207+0000","last_unstale":"2024-04-25T08:30:31.337039+0000","last_undegraded":"2024-04-25T08:30:31.337039+0000","last_fullsized":"2024-04-25T08:30:31.337039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:12.355397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288425+0000","last_change":"2024-04-25T08:30:30.288425+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288425+0000","last_undegraded":"2024-04-25T08:30:30.288425+0000","last_fullsized":"2024-04-25T08:30:30.288425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340097+0000","last_change":"2024-04-25T08:30:31.334369+0000","last_active":"2024-04-25T08:30:31.340097+0000","last_peered":"2024-04-25T08:30:31.340097+0000","last_clean":"2024-04-25T08:30:31.340097+0000","last_became_active":"2024-04-25T08:30:31.334071+0000","last_became_peered":"2024-04-25T08:30:31.334071+0000","last_unstale":"2024-04-25T08:30:31.340097+0000","last_undegraded":"2024-04-25T08:30:31.340097+0000","last_fullsized":"2024-04-25T08:30:31.340097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.453764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332186+0000","last_change":"2024-04-25T08:30:31.331292+0000","last_active":"2024-04-25T08:30:31.332186+0000","last_peered":"2024-04-25T08:30:31.332186+0000","last_clean":"2024-04-25T08:30:31.332186+0000","last_became_active":"2024-04-25T08:30:31.330966+0000","last_became_peered":"2024-04-25T08:30:31.330966+0000","last_unstale":"2024-04-25T08:30:31.332186+0000","last_undegraded":"2024-04-25T08:30:31.332186+0000","last_fullsized":"2024-04-25T08:30:31.332186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332626+0000","last_change":"2024-04-25T08:30:31.331334+0000","last_active":"2024-04-25T08:30:31.332626+0000","last_peered":"2024-04-25T08:30:31.332626+0000","last_clean":"2024-04-25T08:30:31.332626+0000","last_became_active":"2024-04-25T08:30:31.331048+0000","last_became_peered":"2024-04-25T08:30:31.331048+0000","last_unstale":"2024-04-25T08:30:31.332626+0000","last_undegraded":"2024-04-25T08:30:31.332626+0000","last_fullsized":"2024-04-25T08:30:31.332626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325080+0000","last_change":"2024-04-25T08:30:31.322980+0000","last_active":"2024-04-25T08:30:31.325080+0000","last_peered":"2024-04-25T08:30:31.325080+0000","last_clean":"2024-04-25T08:30:31.325080+0000","last_became_active":"2024-04-25T08:30:31.322718+0000","last_became_peered":"2024-04-25T08:30:31.322718+0000","last_unstale":"2024-04-25T08:30:31.325080+0000","last_undegraded":"2024-04-25T08:30:31.325080+0000","last_fullsized":"2024-04-25T08:30:31.325080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:34.413297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286631+0000","last_change":"2024-04-25T08:30:30.286631+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286631+0000","last_undegraded":"2024-04-25T08:30:30.286631+0000","last_fullsized":"2024-04-25T08:30:30.286631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323347+0000","last_change":"2024-04-25T08:30:31.321932+0000","last_active":"2024-04-25T08:30:31.323347+0000","last_peered":"2024-04-25T08:30:31.323347+0000","last_clean":"2024-04-25T08:30:31.323347+0000","last_became_active":"2024-04-25T08:30:31.321696+0000","last_became_peered":"2024-04-25T08:30:31.321696+0000","last_unstale":"2024-04-25T08:30:31.323347+0000","last_undegraded":"2024-04-25T08:30:31.323347+0000","last_fullsized":"2024-04-25T08:30:31.323347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:39.300333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337070+0000","last_change":"2024-04-25T08:30:31.330998+0000","last_active":"2024-04-25T08:30:31.337070+0000","last_peered":"2024-04-25T08:30:31.337070+0000","last_clean":"2024-04-25T08:30:31.337070+0000","last_became_active":"2024-04-25T08:30:31.330539+0000","last_became_peered":"2024-04-25T08:30:31.330539+0000","last_unstale":"2024-04-25T08:30:31.337070+0000","last_undegraded":"2024-04-25T08:30:31.337070+0000","last_fullsized":"2024-04-25T08:30:31.337070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287479+0000","last_change":"2024-04-25T08:30:30.287479+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287479+0000","last_undegraded":"2024-04-25T08:30:30.287479+0000","last_fullsized":"2024-04-25T08:30:30.287479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323285+0000","last_change":"2024-04-25T08:30:31.320712+0000","last_active":"2024-04-25T08:30:31.323285+0000","last_peered":"2024-04-25T08:30:31.323285+0000","last_clean":"2024-04-25T08:30:31.323285+0000","last_became_active":"2024-04-25T08:30:31.320446+0000","last_became_peered":"2024-04-25T08:30:31.320446+0000","last_unstale":"2024-04-25T08:30:31.323285+0000","last_undegraded":"2024-04-25T08:30:31.323285+0000","last_fullsized":"2024-04-25T08:30:31.323285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.070232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286827+0000","last_change":"2024-04-25T08:30:30.286827+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286827+0000","last_undegraded":"2024-04-25T08:30:30.286827+0000","last_fullsized":"2024-04-25T08:30:30.286827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.327272+0000","last_change":"2024-04-25T08:30:31.324771+0000","last_active":"2024-04-25T08:30:31.327272+0000","last_peered":"2024-04-25T08:30:31.327272+0000","last_clean":"2024-04-25T08:30:31.327272+0000","last_became_active":"2024-04-25T08:30:31.324509+0000","last_became_peered":"2024-04-25T08:30:31.324509+0000","last_unstale":"2024-04-25T08:30:31.327272+0000","last_undegraded":"2024-04-25T08:30:31.327272+0000","last_fullsized":"2024-04-25T08:30:31.327272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:32.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.298366+0000","last_change":"2024-04-25T08:30:30.298366+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.298366+0000","last_undegraded":"2024-04-25T08:30:30.298366+0000","last_fullsized":"2024-04-25T08:30:30.298366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.319706+0000","last_change":"2024-04-25T08:30:31.316804+0000","last_active":"2024-04-25T08:30:31.319706+0000","last_peered":"2024-04-25T08:30:31.319706+0000","last_clean":"2024-04-25T08:30:31.319706+0000","last_became_active":"2024-04-25T08:30:31.316503+0000","last_became_peered":"2024-04-25T08:30:31.316503+0000","last_unstale":"2024-04-25T08:30:31.319706+0000","last_undegraded":"2024-04-25T08:30:31.319706+0000","last_fullsized":"2024-04-25T08:30:31.319706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:43.729410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.339771+0000","last_change":"2024-04-25T08:30:31.331474+0000","last_active":"2024-04-25T08:30:31.339771+0000","last_peered":"2024-04-25T08:30:31.339771+0000","last_clean":"2024-04-25T08:30:31.339771+0000","last_became_active":"2024-04-25T08:30:31.331058+0000","last_became_peered":"2024-04-25T08:30:31.331058+0000","last_unstale":"2024-04-25T08:30:31.339771+0000","last_undegraded":"2024-04-25T08:30:31.339771+0000","last_fullsized":"2024-04-25T08:30:31.339771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320679+0000","last_change":"2024-04-25T08:30:31.319410+0000","last_active":"2024-04-25T08:30:31.320679+0000","last_peered":"2024-04-25T08:30:31.320679+0000","last_clean":"2024-04-25T08:30:31.320679+0000","last_became_active":"2024-04-25T08:30:31.318913+0000","last_became_peered":"2024-04-25T08:30:31.318913+0000","last_unstale":"2024-04-25T08:30:31.320679+0000","last_undegraded":"2024-04-25T08:30:31.320679+0000","last_fullsized":"2024-04-25T08:30:31.320679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:14.713545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320655+0000","last_change":"2024-04-25T08:30:31.318909+0000","last_active":"2024-04-25T08:30:31.320655+0000","last_peered":"2024-04-25T08:30:31.320655+0000","last_clean":"2024-04-25T08:30:31.320655+0000","last_became_active":"2024-04-25T08:30:31.318639+0000","last_became_peered":"2024-04-25T08:30:31.318639+0000","last_unstale":"2024-04-25T08:30:31.320655+0000","last_undegraded":"2024-04-25T08:30:31.320655+0000","last_fullsized":"2024-04-25T08:30:31.320655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:50.624325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323302+0000","last_change":"2024-04-25T08:30:31.321924+0000","last_active":"2024-04-25T08:30:31.323302+0000","last_peered":"2024-04-25T08:30:31.323302+0000","last_clean":"2024-04-25T08:30:31.323302+0000","last_became_active":"2024-04-25T08:30:31.321434+0000","last_became_peered":"2024-04-25T08:30:31.321434+0000","last_unstale":"2024-04-25T08:30:31.323302+0000","last_undegraded":"2024-04-25T08:30:31.323302+0000","last_fullsized":"2024-04-25T08:30:31.323302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289797+0000","last_change":"2024-04-25T08:30:30.289797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289797+0000","last_undegraded":"2024-04-25T08:30:30.289797+0000","last_fullsized":"2024-04-25T08:30:30.289797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338955+0000","last_change":"2024-04-25T08:30:31.332758+0000","last_active":"2024-04-25T08:30:31.338955+0000","last_peered":"2024-04-25T08:30:31.338955+0000","last_clean":"2024-04-25T08:30:31.338955+0000","last_became_active":"2024-04-25T08:30:31.332458+0000","last_became_peered":"2024-04-25T08:30:31.332458+0000","last_unstale":"2024-04-25T08:30:31.338955+0000","last_undegraded":"2024-04-25T08:30:31.338955+0000","last_fullsized":"2024-04-25T08:30:31.338955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340572+0000","last_change":"2024-04-25T08:30:30.294755+0000","last_active":"2024-04-25T08:30:31.340572+0000","last_peered":"2024-04-25T08:30:31.340572+0000","last_clean":"2024-04-25T08:30:31.340572+0000","last_became_active":"2024-04-25T08:30:30.293812+0000","last_became_peered":"2024-04-25T08:30:30.293812+0000","last_unstale":"2024-04-25T08:30:31.340572+0000","last_undegraded":"2024-04-25T08:30:31.340572+0000","last_fullsized":"2024-04-25T08:30:31.340572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.299433+0000","last_change":"2024-04-25T08:30:30.299433+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.299433+0000","last_undegraded":"2024-04-25T08:30:30.299433+0000","last_fullsized":"2024-04-25T08:30:30.299433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304327+0000","last_change":"2024-04-25T08:30:30.301932+0000","last_active":"2024-04-25T08:30:30.304327+0000","last_peered":"2024-04-25T08:30:30.304327+0000","last_clean":"2024-04-25T08:30:30.304327+0000","last_became_active":"2024-04-25T08:30:30.301633+0000","last_became_peered":"2024-04-25T08:30:30.301633+0000","last_unstale":"2024-04-25T08:30:30.304327+0000","last_undegraded":"2024-04-25T08:30:30.304327+0000","last_fullsized":"2024-04-25T08:30:30.304327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286830+0000","last_change":"2024-04-25T08:30:30.286830+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286830+0000","last_undegraded":"2024-04-25T08:30:30.286830+0000","last_fullsized":"2024-04-25T08:30:30.286830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306192+0000","last_change":"2024-04-25T08:30:30.304552+0000","last_active":"2024-04-25T08:30:30.306192+0000","last_peered":"2024-04-25T08:30:30.306192+0000","last_clean":"2024-04-25T08:30:30.306192+0000","last_became_active":"2024-04-25T08:30:30.304168+0000","last_became_peered":"2024-04-25T08:30:30.304168+0000","last_unstale":"2024-04-25T08:30:30.306192+0000","last_undegraded":"2024-04-25T08:30:30.306192+0000","last_fullsized":"2024-04-25T08:30:30.306192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292899+0000","last_change":"2024-04-25T08:30:30.292899+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292899+0000","last_undegraded":"2024-04-25T08:30:30.292899+0000","last_fullsized":"2024-04-25T08:30:30.292899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304321+0000","last_change":"2024-04-25T08:30:30.301217+0000","last_active":"2024-04-25T08:30:30.304321+0000","last_peered":"2024-04-25T08:30:30.304321+0000","last_clean":"2024-04-25T08:30:30.304321+0000","last_became_active":"2024-04-25T08:30:30.300899+0000","last_became_peered":"2024-04-25T08:30:30.300899+0000","last_unstale":"2024-04-25T08:30:30.304321+0000","last_undegraded":"2024-04-25T08:30:30.304321+0000","last_fullsized":"2024-04-25T08:30:30.304321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289501+0000","last_change":"2024-04-25T08:30:30.289501+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289501+0000","last_undegraded":"2024-04-25T08:30:30.289501+0000","last_fullsized":"2024-04-25T08:30:30.289501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338973+0000","last_change":"2024-04-25T08:30:30.302842+0000","last_active":"2024-04-25T08:30:31.338973+0000","last_peered":"2024-04-25T08:30:31.338973+0000","last_clean":"2024-04-25T08:30:31.338973+0000","last_became_active":"2024-04-25T08:30:30.302540+0000","last_became_peered":"2024-04-25T08:30:30.302540+0000","last_unstale":"2024-04-25T08:30:31.338973+0000","last_undegraded":"2024-04-25T08:30:31.338973+0000","last_fullsized":"2024-04-25T08:30:31.338973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.946206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335411+0000","last_change":"2024-04-25T08:30:31.330857+0000","last_active":"2024-04-25T08:30:31.335411+0000","last_peered":"2024-04-25T08:30:31.335411+0000","last_clean":"2024-04-25T08:30:31.335411+0000","last_became_active":"2024-04-25T08:30:31.329563+0000","last_became_peered":"2024-04-25T08:30:31.329563+0000","last_unstale":"2024-04-25T08:30:31.335411+0000","last_undegraded":"2024-04-25T08:30:31.335411+0000","last_fullsized":"2024-04-25T08:30:31.335411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341386+0000","last_change":"2024-04-25T08:30:30.298067+0000","last_active":"2024-04-25T08:30:31.341386+0000","last_peered":"2024-04-25T08:30:31.341386+0000","last_clean":"2024-04-25T08:30:31.341386+0000","last_became_active":"2024-04-25T08:30:30.297718+0000","last_became_peered":"2024-04-25T08:30:30.297718+0000","last_unstale":"2024-04-25T08:30:31.341386+0000","last_undegraded":"2024-04-25T08:30:31.341386+0000","last_fullsized":"2024-04-25T08:30:31.341386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287002+0000","last_change":"2024-04-25T08:30:30.287002+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287002+0000","last_undegraded":"2024-04-25T08:30:30.287002+0000","last_fullsized":"2024-04-25T08:30:30.287002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306219+0000","last_change":"2024-04-25T08:30:30.305406+0000","last_active":"2024-04-25T08:30:30.306219+0000","last_peered":"2024-04-25T08:30:30.306219+0000","last_clean":"2024-04-25T08:30:30.306219+0000","last_became_active":"2024-04-25T08:30:30.305041+0000","last_became_peered":"2024-04-25T08:30:30.305041+0000","last_unstale":"2024-04-25T08:30:30.306219+0000","last_undegraded":"2024-04-25T08:30:30.306219+0000","last_fullsized":"2024-04-25T08:30:30.306219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291475+0000","last_change":"2024-04-25T08:30:30.291475+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291475+0000","last_undegraded":"2024-04-25T08:30:30.291475+0000","last_fullsized":"2024-04-25T08:30:30.291475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295501+0000","last_change":"2024-04-25T08:30:30.290419+0000","last_active":"2024-04-25T08:30:30.295501+0000","last_peered":"2024-04-25T08:30:30.295501+0000","last_clean":"2024-04-25T08:30:30.295501+0000","last_became_active":"2024-04-25T08:30:30.290143+0000","last_became_peered":"2024-04-25T08:30:30.290143+0000","last_unstale":"2024-04-25T08:30:30.295501+0000","last_undegraded":"2024-04-25T08:30:30.295501+0000","last_fullsized":"2024-04-25T08:30:30.295501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:19.910492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286356+0000","last_change":"2024-04-25T08:30:30.286356+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286356+0000","last_undegraded":"2024-04-25T08:30:30.286356+0000","last_fullsized":"2024-04-25T08:30:30.286356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305495+0000","last_change":"2024-04-25T08:30:30.302163+0000","last_active":"2024-04-25T08:30:30.305495+0000","last_peered":"2024-04-25T08:30:30.305495+0000","last_clean":"2024-04-25T08:30:30.305495+0000","last_became_active":"2024-04-25T08:30:30.301848+0000","last_became_peered":"2024-04-25T08:30:30.301848+0000","last_unstale":"2024-04-25T08:30:30.305495+0000","last_undegraded":"2024-04-25T08:30:30.305495+0000","last_fullsized":"2024-04-25T08:30:30.305495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:50.715081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323281+0000","last_change":"2024-04-25T08:30:31.319799+0000","last_active":"2024-04-25T08:30:31.323281+0000","last_peered":"2024-04-25T08:30:31.323281+0000","last_clean":"2024-04-25T08:30:31.323281+0000","last_became_active":"2024-04-25T08:30:31.319152+0000","last_became_peered":"2024-04-25T08:30:31.319152+0000","last_unstale":"2024-04-25T08:30:31.323281+0000","last_undegraded":"2024-04-25T08:30:31.323281+0000","last_fullsized":"2024-04-25T08:30:31.323281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:11.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325859+0000","last_change":"2024-04-25T08:30:30.299038+0000","last_active":"2024-04-25T08:30:31.325859+0000","last_peered":"2024-04-25T08:30:31.325859+0000","last_clean":"2024-04-25T08:30:31.325859+0000","last_became_active":"2024-04-25T08:30:30.298782+0000","last_became_peered":"2024-04-25T08:30:30.298782+0000","last_unstale":"2024-04-25T08:30:31.325859+0000","last_undegraded":"2024-04-25T08:30:31.325859+0000","last_fullsized":"2024-04-25T08:30:31.325859+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:47.771125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287963+0000","last_change":"2024-04-25T08:30:30.287963+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287963+0000","last_undegraded":"2024-04-25T08:30:30.287963+0000","last_fullsized":"2024-04-25T08:30:30.287963+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341021+0000","last_change":"2024-04-25T08:30:30.301686+0000","last_active":"2024-04-25T08:30:31.341021+0000","last_peered":"2024-04-25T08:30:31.341021+0000","last_clean":"2024-04-25T08:30:31.341021+0000","last_became_active":"2024-04-25T08:30:30.301391+0000","last_became_peered":"2024-04-25T08:30:30.301391+0000","last_unstale":"2024-04-25T08:30:31.341021+0000","last_undegraded":"2024-04-25T08:30:31.341021+0000","last_fullsized":"2024-04-25T08:30:31.341021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.852169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288660+0000","last_change":"2024-04-25T08:30:30.288660+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288660+0000","last_undegraded":"2024-04-25T08:30:30.288660+0000","last_fullsized":"2024-04-25T08:30:30.288660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305486+0000","last_change":"2024-04-25T08:30:30.303220+0000","last_active":"2024-04-25T08:30:30.305486+0000","last_peered":"2024-04-25T08:30:30.305486+0000","last_clean":"2024-04-25T08:30:30.305486+0000","last_became_active":"2024-04-25T08:30:30.302914+0000","last_became_peered":"2024-04-25T08:30:30.302914+0000","last_unstale":"2024-04-25T08:30:30.305486+0000","last_undegraded":"2024-04-25T08:30:30.305486+0000","last_fullsized":"2024-04-25T08:30:30.305486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:10.215234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289039+0000","last_change":"2024-04-25T08:30:30.289039+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289039+0000","last_undegraded":"2024-04-25T08:30:30.289039+0000","last_fullsized":"2024-04-25T08:30:30.289039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304930+0000","last_change":"2024-04-25T08:30:30.301164+0000","last_active":"2024-04-25T08:30:30.304930+0000","last_peered":"2024-04-25T08:30:30.304930+0000","last_clean":"2024-04-25T08:30:30.304930+0000","last_became_active":"2024-04-25T08:30:30.300887+0000","last_became_peered":"2024-04-25T08:30:30.300887+0000","last_unstale":"2024-04-25T08:30:30.304930+0000","last_undegraded":"2024-04-25T08:30:30.304930+0000","last_fullsized":"2024-04-25T08:30:30.304930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.290447+0000","last_change":"2024-04-25T08:30:30.290447+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.290447+0000","last_undegraded":"2024-04-25T08:30:30.290447+0000","last_fullsized":"2024-04-25T08:30:30.290447+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306196+0000","last_change":"2024-04-25T08:30:30.304789+0000","last_active":"2024-04-25T08:30:30.306196+0000","last_peered":"2024-04-25T08:30:30.306196+0000","last_clean":"2024-04-25T08:30:30.306196+0000","last_became_active":"2024-04-25T08:30:30.304493+0000","last_became_peered":"2024-04-25T08:30:30.304493+0000","last_unstale":"2024-04-25T08:30:30.306196+0000","last_undegraded":"2024-04-25T08:30:30.306196+0000","last_fullsized":"2024-04-25T08:30:30.306196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330098+0000","last_change":"2024-04-25T08:30:31.326518+0000","last_active":"2024-04-25T08:30:31.330098+0000","last_peered":"2024-04-25T08:30:31.330098+0000","last_clean":"2024-04-25T08:30:31.330098+0000","last_became_active":"2024-04-25T08:30:31.326241+0000","last_became_peered":"2024-04-25T08:30:31.326241+0000","last_unstale":"2024-04-25T08:30:31.330098+0000","last_undegraded":"2024-04-25T08:30:31.330098+0000","last_fullsized":"2024-04-25T08:30:31.330098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310441+0000","last_change":"2024-04-25T08:30:30.294767+0000","last_active":"2024-04-25T08:30:31.310441+0000","last_peered":"2024-04-25T08:30:31.310441+0000","last_clean":"2024-04-25T08:30:31.310441+0000","last_became_active":"2024-04-25T08:30:30.294475+0000","last_became_peered":"2024-04-25T08:30:30.294475+0000","last_unstale":"2024-04-25T08:30:31.310441+0000","last_undegraded":"2024-04-25T08:30:31.310441+0000","last_fullsized":"2024-04-25T08:30:31.310441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291037+0000","last_change":"2024-04-25T08:30:30.291037+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291037+0000","last_undegraded":"2024-04-25T08:30:30.291037+0000","last_fullsized":"2024-04-25T08:30:30.291037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.342812+0000","last_change":"2024-04-25T08:30:30.297207+0000","last_active":"2024-04-25T08:30:31.342812+0000","last_peered":"2024-04-25T08:30:31.342812+0000","last_clean":"2024-04-25T08:30:31.342812+0000","last_became_active":"2024-04-25T08:30:30.296922+0000","last_became_peered":"2024-04-25T08:30:30.296922+0000","last_unstale":"2024-04-25T08:30:31.342812+0000","last_undegraded":"2024-04-25T08:30:31.342812+0000","last_fullsized":"2024-04-25T08:30:31.342812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:13.711704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.297011+0000","last_change":"2024-04-25T08:30:30.297011+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.297011+0000","last_undegraded":"2024-04-25T08:30:30.297011+0000","last_fullsized":"2024-04-25T08:30:30.297011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340937+0000","last_change":"2024-04-25T08:30:30.297702+0000","last_active":"2024-04-25T08:30:31.340937+0000","last_peered":"2024-04-25T08:30:31.340937+0000","last_clean":"2024-04-25T08:30:31.340937+0000","last_became_active":"2024-04-25T08:30:30.297420+0000","last_became_peered":"2024-04-25T08:30:30.297420+0000","last_unstale":"2024-04-25T08:30:31.340937+0000","last_undegraded":"2024-04-25T08:30:31.340937+0000","last_fullsized":"2024-04-25T08:30:31.340937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.137643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341074+0000","last_change":"2024-04-25T08:30:31.337230+0000","last_active":"2024-04-25T08:30:31.341074+0000","last_peered":"2024-04-25T08:30:31.341074+0000","last_clean":"2024-04-25T08:30:31.341074+0000","last_became_active":"2024-04-25T08:30:31.336785+0000","last_became_peered":"2024-04-25T08:30:31.336785+0000","last_unstale":"2024-04-25T08:30:31.341074+0000","last_undegraded":"2024-04-25T08:30:31.341074+0000","last_fullsized":"2024-04-25T08:30:31.341074+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:28.421465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301349+0000","last_change":"2024-04-25T08:30:30.296836+0000","last_active":"2024-04-25T08:30:30.301349+0000","last_peered":"2024-04-25T08:30:30.301349+0000","last_clean":"2024-04-25T08:30:30.301349+0000","last_became_active":"2024-04-25T08:30:30.296490+0000","last_became_peered":"2024-04-25T08:30:30.296490+0000","last_unstale":"2024-04-25T08:30:30.301349+0000","last_undegraded":"2024-04-25T08:30:30.301349+0000","last_fullsized":"2024-04-25T08:30:30.301349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289132+0000","last_change":"2024-04-25T08:30:30.289132+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289132+0000","last_undegraded":"2024-04-25T08:30:30.289132+0000","last_fullsized":"2024-04-25T08:30:30.289132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.300070+0000","last_change":"2024-04-25T08:30:30.297703+0000","last_active":"2024-04-25T08:30:30.300070+0000","last_peered":"2024-04-25T08:30:30.300070+0000","last_clean":"2024-04-25T08:30:30.300070+0000","last_became_active":"2024-04-25T08:30:30.297424+0000","last_became_peered":"2024-04-25T08:30:30.297424+0000","last_unstale":"2024-04-25T08:30:30.300070+0000","last_undegraded":"2024-04-25T08:30:30.300070+0000","last_fullsized":"2024-04-25T08:30:30.300070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.320688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286213+0000","last_change":"2024-04-25T08:30:30.286213+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286213+0000","last_undegraded":"2024-04-25T08:30:30.286213+0000","last_fullsized":"2024-04-25T08:30:30.286213+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305672+0000","last_change":"2024-04-25T08:30:30.303603+0000","last_active":"2024-04-25T08:30:30.305672+0000","last_peered":"2024-04-25T08:30:30.305672+0000","last_clean":"2024-04-25T08:30:30.305672+0000","last_became_active":"2024-04-25T08:30:30.303322+0000","last_became_peered":"2024-04-25T08:30:30.303322+0000","last_unstale":"2024-04-25T08:30:30.305672+0000","last_undegraded":"2024-04-25T08:30:30.305672+0000","last_fullsized":"2024-04-25T08:30:30.305672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330062+0000","last_change":"2024-04-25T08:30:31.325746+0000","last_active":"2024-04-25T08:30:31.330062+0000","last_peered":"2024-04-25T08:30:31.330062+0000","last_clean":"2024-04-25T08:30:31.330062+0000","last_became_active":"2024-04-25T08:30:31.325449+0000","last_became_peered":"2024-04-25T08:30:31.325449+0000","last_unstale":"2024-04-25T08:30:31.330062+0000","last_undegraded":"2024-04-25T08:30:31.330062+0000","last_fullsized":"2024-04-25T08:30:31.330062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295651+0000","last_change":"2024-04-25T08:30:30.293002+0000","last_active":"2024-04-25T08:30:30.295651+0000","last_peered":"2024-04-25T08:30:30.295651+0000","last_clean":"2024-04-25T08:30:30.295651+0000","last_became_active":"2024-04-25T08:30:30.292702+0000","last_became_peered":"2024-04-25T08:30:30.292702+0000","last_unstale":"2024-04-25T08:30:30.295651+0000","last_undegraded":"2024-04-25T08:30:30.295651+0000","last_fullsized":"2024-04-25T08:30:30.295651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.414247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287883+0000","last_change":"2024-04-25T08:30:30.287883+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287883+0000","last_undegraded":"2024-04-25T08:30:30.287883+0000","last_fullsized":"2024-04-25T08:30:30.287883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295421+0000","last_change":"2024-04-25T08:30:30.292431+0000","last_active":"2024-04-25T08:30:30.295421+0000","last_peered":"2024-04-25T08:30:30.295421+0000","last_clean":"2024-04-25T08:30:30.295421+0000","last_became_active":"2024-04-25T08:30:30.292078+0000","last_became_peered":"2024-04-25T08:30:30.292078+0000","last_unstale":"2024-04-25T08:30:30.295421+0000","last_undegraded":"2024-04-25T08:30:30.295421+0000","last_fullsized":"2024-04-25T08:30:30.295421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:06.079852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335419+0000","last_change":"2024-04-25T08:30:31.331176+0000","last_active":"2024-04-25T08:30:31.335419+0000","last_peered":"2024-04-25T08:30:31.335419+0000","last_clean":"2024-04-25T08:30:31.335419+0000","last_became_active":"2024-04-25T08:30:31.330823+0000","last_became_peered":"2024-04-25T08:30:31.330823+0000","last_unstale":"2024-04-25T08:30:31.335419+0000","last_undegraded":"2024-04-25T08:30:31.335419+0000","last_fullsized":"2024-04-25T08:30:31.335419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:21.292627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311697+0000","last_change":"2024-04-25T08:30:30.299966+0000","last_active":"2024-04-25T08:30:31.311697+0000","last_peered":"2024-04-25T08:30:31.311697+0000","last_clean":"2024-04-25T08:30:31.311697+0000","last_became_active":"2024-04-25T08:30:30.299699+0000","last_became_peered":"2024-04-25T08:30:30.299699+0000","last_unstale":"2024-04-25T08:30:31.311697+0000","last_undegraded":"2024-04-25T08:30:31.311697+0000","last_fullsized":"2024-04-25T08:30:31.311697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289401+0000","last_change":"2024-04-25T08:30:30.289401+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289401+0000","last_undegraded":"2024-04-25T08:30:30.289401+0000","last_fullsized":"2024-04-25T08:30:30.289401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.297186+0000","last_change":"2024-04-25T08:30:30.293467+0000","last_active":"2024-04-25T08:30:30.297186+0000","last_peered":"2024-04-25T08:30:30.297186+0000","last_clean":"2024-04-25T08:30:30.297186+0000","last_became_active":"2024-04-25T08:30:30.293179+0000","last_became_peered":"2024-04-25T08:30:30.293179+0000","last_unstale":"2024-04-25T08:30:30.297186+0000","last_undegraded":"2024-04-25T08:30:30.297186+0000","last_fullsized":"2024-04-25T08:30:30.297186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.629362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.296797+0000","last_change":"2024-04-25T08:30:30.296797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.296797+0000","last_undegraded":"2024-04-25T08:30:30.296797+0000","last_fullsized":"2024-04-25T08:30:30.296797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340294+0000","last_change":"2024-04-25T08:30:30.301760+0000","last_active":"2024-04-25T08:30:31.340294+0000","last_peered":"2024-04-25T08:30:31.340294+0000","last_clean":"2024-04-25T08:30:31.340294+0000","last_became_active":"2024-04-25T08:30:30.301331+0000","last_became_peered":"2024-04-25T08:30:30.301331+0000","last_unstale":"2024-04-25T08:30:31.340294+0000","last_undegraded":"2024-04-25T08:30:31.340294+0000","last_fullsized":"2024-04-25T08:30:31.340294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323304+0000","last_change":"2024-04-25T08:30:31.320048+0000","last_active":"2024-04-25T08:30:31.323304+0000","last_peered":"2024-04-25T08:30:31.323304+0000","last_clean":"2024-04-25T08:30:31.323304+0000","last_became_active":"2024-04-25T08:30:31.319495+0000","last_became_peered":"2024-04-25T08:30:31.319495+0000","last_unstale":"2024-04-25T08:30:31.323304+0000","last_undegraded":"2024-04-25T08:30:31.323304+0000","last_fullsized":"2024-04-25T08:30:31.323304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:05.728351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.299425+0000","last_change":"2024-04-25T08:30:30.296903+0000","last_active":"2024-04-25T08:30:30.299425+0000","last_peered":"2024-04-25T08:30:30.299425+0000","last_clean":"2024-04-25T08:30:30.299425+0000","last_became_active":"2024-04-25T08:30:30.296623+0000","last_became_peered":"2024-04-25T08:30:30.296623+0000","last_unstale":"2024-04-25T08:30:30.299425+0000","last_undegraded":"2024-04-25T08:30:30.299425+0000","last_fullsized":"2024-04-25T08:30:30.299425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:03.007004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293539+0000","last_change":"2024-04-25T08:30:30.293539+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293539+0000","last_undegraded":"2024-04-25T08:30:30.293539+0000","last_fullsized":"2024-04-25T08:30:30.293539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305561+0000","last_change":"2024-04-25T08:30:30.303229+0000","last_active":"2024-04-25T08:30:30.305561+0000","last_peered":"2024-04-25T08:30:30.305561+0000","last_clean":"2024-04-25T08:30:30.305561+0000","last_became_active":"2024-04-25T08:30:30.302894+0000","last_became_peered":"2024-04-25T08:30:30.302894+0000","last_unstale":"2024-04-25T08:30:30.305561+0000","last_undegraded":"2024-04-25T08:30:30.305561+0000","last_fullsized":"2024-04-25T08:30:30.305561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288107+0000","last_change":"2024-04-25T08:30:30.288107+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288107+0000","last_undegraded":"2024-04-25T08:30:30.288107+0000","last_fullsized":"2024-04-25T08:30:30.288107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301519+0000","last_change":"2024-04-25T08:30:30.300280+0000","last_active":"2024-04-25T08:30:30.301519+0000","last_peered":"2024-04-25T08:30:30.301519+0000","last_clean":"2024-04-25T08:30:30.301519+0000","last_became_active":"2024-04-25T08:30:30.299985+0000","last_became_peered":"2024-04-25T08:30:30.299985+0000","last_unstale":"2024-04-25T08:30:30.301519+0000","last_undegraded":"2024-04-25T08:30:30.301519+0000","last_fullsized":"2024-04-25T08:30:30.301519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292115+0000","last_change":"2024-04-25T08:30:30.292115+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292115+0000","last_undegraded":"2024-04-25T08:30:30.292115+0000","last_fullsized":"2024-04-25T08:30:30.292115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.312138+0000","last_change":"2024-04-25T08:30:30.293212+0000","last_active":"2024-04-25T08:30:31.312138+0000","last_peered":"2024-04-25T08:30:31.312138+0000","last_clean":"2024-04-25T08:30:31.312138+0000","last_became_active":"2024-04-25T08:30:30.292931+0000","last_became_peered":"2024-04-25T08:30:30.292931+0000","last_unstale":"2024-04-25T08:30:31.312138+0000","last_undegraded":"2024-04-25T08:30:31.312138+0000","last_fullsized":"2024-04-25T08:30:31.312138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287931+0000","last_change":"2024-04-25T08:30:30.287931+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287931+0000","last_undegraded":"2024-04-25T08:30:30.287931+0000","last_fullsized":"2024-04-25T08:30:30.287931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.302154+0000","last_change":"2024-04-25T08:30:30.298769+0000","last_active":"2024-04-25T08:30:30.302154+0000","last_peered":"2024-04-25T08:30:30.302154+0000","last_clean":"2024-04-25T08:30:30.302154+0000","last_became_active":"2024-04-25T08:30:30.298489+0000","last_became_peered":"2024-04-25T08:30:30.298489+0000","last_unstale":"2024-04-25T08:30:30.302154+0000","last_undegraded":"2024-04-25T08:30:30.302154+0000","last_fullsized":"2024-04-25T08:30:30.302154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323460+0000","last_change":"2024-04-25T08:30:31.318757+0000","last_active":"2024-04-25T08:30:31.323460+0000","last_peered":"2024-04-25T08:30:31.323460+0000","last_clean":"2024-04-25T08:30:31.323460+0000","last_became_active":"2024-04-25T08:30:31.318482+0000","last_became_peered":"2024-04-25T08:30:31.318482+0000","last_unstale":"2024-04-25T08:30:31.323460+0000","last_undegraded":"2024-04-25T08:30:31.323460+0000","last_fullsized":"2024-04-25T08:30:31.323460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.716923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.296818+0000","last_change":"2024-04-25T08:30:30.295325+0000","last_active":"2024-04-25T08:30:30.296818+0000","last_peered":"2024-04-25T08:30:30.296818+0000","last_clean":"2024-04-25T08:30:30.296818+0000","last_became_active":"2024-04-25T08:30:30.295030+0000","last_became_peered":"2024-04-25T08:30:30.295030+0000","last_unstale":"2024-04-25T08:30:30.296818+0000","last_undegraded":"2024-04-25T08:30:30.296818+0000","last_fullsized":"2024-04-25T08:30:30.296818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341677+0000","last_change":"2024-04-25T08:30:31.340453+0000","last_active":"2024-04-25T08:30:31.341677+0000","last_peered":"2024-04-25T08:30:31.341677+0000","last_clean":"2024-04-25T08:30:31.341677+0000","last_became_active":"2024-04-25T08:30:31.340188+0000","last_became_peered":"2024-04-25T08:30:31.340188+0000","last_unstale":"2024-04-25T08:30:31.341677+0000","last_undegraded":"2024-04-25T08:30:31.341677+0000","last_fullsized":"2024-04-25T08:30:31.341677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311406+0000","last_change":"2024-04-25T08:30:30.300697+0000","last_active":"2024-04-25T08:30:31.311406+0000","last_peered":"2024-04-25T08:30:31.311406+0000","last_clean":"2024-04-25T08:30:31.311406+0000","last_became_active":"2024-04-25T08:30:30.300410+0000","last_became_peered":"2024-04-25T08:30:30.300410+0000","last_unstale":"2024-04-25T08:30:31.311406+0000","last_undegraded":"2024-04-25T08:30:31.311406+0000","last_fullsized":"2024-04-25T08:30:31.311406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:38.298492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310971+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:30:31.310971+0000","last_peered":"2024-04-25T08:30:31.310971+0000","last_clean":"2024-04-25T08:30:31.310971+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:30:31.310971+0000","last_undegraded":"2024-04-25T08:30:31.310971+0000","last_fullsized":"2024-04-25T08:30:31.310971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":763701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:30:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-25T08:30:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.292388+0000', 'last_change': '2024-04-25T08:30:30.292388+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.292388+0000', 'last_undegraded': '2024-04-25T08:30:30.292388+0000', 'last_fullsized': '2024-04-25T08:30:30.292388+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-25T08:30:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.293940+0000', 'last_change': '2024-04-25T08:30:30.293940+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.293940+0000', 'last_undegraded': '2024-04-25T08:30:30.293940+0000', 'last_fullsized': '2024-04-25T08:30:30.293940+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-25T08:30:35.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287912+0000', 'last_change': '2024-04-25T08:30:30.287912+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287912+0000', 'last_undegraded': '2024-04-25T08:30:30.287912+0000', 'last_fullsized': '2024-04-25T08:30:30.287912+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286932+0000', 'last_change': '2024-04-25T08:30:30.286932+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286932+0000', 'last_undegraded': '2024-04-25T08:30:30.286932+0000', 'last_fullsized': '2024-04-25T08:30:30.286932+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286790+0000', 'last_change': '2024-04-25T08:30:30.286790+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286790+0000', 'last_undegraded': '2024-04-25T08:30:30.286790+0000', 'last_fullsized': '2024-04-25T08:30:30.286790+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.292036+0000', 'last_change': '2024-04-25T08:30:30.292036+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.292036+0000', 'last_undegraded': '2024-04-25T08:30:30.292036+0000', 'last_fullsized': '2024-04-25T08:30:30.292036+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.289376+0000', 'last_change': '2024-04-25T08:30:30.289376+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.289376+0000', 'last_undegraded': '2024-04-25T08:30:30.289376+0000', 'last_fullsized': '2024-04-25T08:30:30.289376+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287333+0000', 'last_change': '2024-04-25T08:30:30.287333+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287333+0000', 'last_undegraded': '2024-04-25T08:30:30.287333+0000', 'last_fullsized': '2024-04-25T08:30:30.287333+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-25T08:30:35.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287710+0000', 'last_change': '2024-04-25T08:30:30.287710+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287710+0000', 'last_undegraded': '2024-04-25T08:30:30.287710+0000', 'last_fullsized': '2024-04-25T08:30:30.287710+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.288425+0000', 'last_change': '2024-04-25T08:30:30.288425+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.288425+0000', 'last_undegraded': '2024-04-25T08:30:30.288425+0000', 'last_fullsized': '2024-04-25T08:30:30.288425+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286631+0000', 'last_change': '2024-04-25T08:30:30.286631+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286631+0000', 'last_undegraded': '2024-04-25T08:30:30.286631+0000', 'last_fullsized': '2024-04-25T08:30:30.286631+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287479+0000', 'last_change': '2024-04-25T08:30:30.287479+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287479+0000', 'last_undegraded': '2024-04-25T08:30:30.287479+0000', 'last_fullsized': '2024-04-25T08:30:30.287479+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286827+0000', 'last_change': '2024-04-25T08:30:30.286827+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286827+0000', 'last_undegraded': '2024-04-25T08:30:30.286827+0000', 'last_fullsized': '2024-04-25T08:30:30.286827+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.298366+0000', 'last_change': '2024-04-25T08:30:30.298366+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.298366+0000', 'last_undegraded': '2024-04-25T08:30:30.298366+0000', 'last_fullsized': '2024-04-25T08:30:30.298366+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-25T08:30:35.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.289797+0000', 'last_change': '2024-04-25T08:30:30.289797+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.289797+0000', 'last_undegraded': '2024-04-25T08:30:30.289797+0000', 'last_fullsized': '2024-04-25T08:30:30.289797+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.299433+0000', 'last_change': '2024-04-25T08:30:30.299433+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.299433+0000', 'last_undegraded': '2024-04-25T08:30:30.299433+0000', 'last_fullsized': '2024-04-25T08:30:30.299433+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286830+0000', 'last_change': '2024-04-25T08:30:30.286830+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286830+0000', 'last_undegraded': '2024-04-25T08:30:30.286830+0000', 'last_fullsized': '2024-04-25T08:30:30.286830+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.292899+0000', 'last_change': '2024-04-25T08:30:30.292899+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.292899+0000', 'last_undegraded': '2024-04-25T08:30:30.292899+0000', 'last_fullsized': '2024-04-25T08:30:30.292899+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.289501+0000', 'last_change': '2024-04-25T08:30:30.289501+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.289501+0000', 'last_undegraded': '2024-04-25T08:30:30.289501+0000', 'last_fullsized': '2024-04-25T08:30:30.289501+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-25T08:30:35.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287002+0000', 'last_change': '2024-04-25T08:30:30.287002+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287002+0000', 'last_undegraded': '2024-04-25T08:30:30.287002+0000', 'last_fullsized': '2024-04-25T08:30:30.287002+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.291475+0000', 'last_change': '2024-04-25T08:30:30.291475+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.291475+0000', 'last_undegraded': '2024-04-25T08:30:30.291475+0000', 'last_fullsized': '2024-04-25T08:30:30.291475+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286356+0000', 'last_change': '2024-04-25T08:30:30.286356+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286356+0000', 'last_undegraded': '2024-04-25T08:30:30.286356+0000', 'last_fullsized': '2024-04-25T08:30:30.286356+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287963+0000', 'last_change': '2024-04-25T08:30:30.287963+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287963+0000', 'last_undegraded': '2024-04-25T08:30:30.287963+0000', 'last_fullsized': '2024-04-25T08:30:30.287963+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.288660+0000', 'last_change': '2024-04-25T08:30:30.288660+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.288660+0000', 'last_undegraded': '2024-04-25T08:30:30.288660+0000', 'last_fullsized': '2024-04-25T08:30:30.288660+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.289039+0000', 'last_change': '2024-04-25T08:30:30.289039+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.289039+0000', 'last_undegraded': '2024-04-25T08:30:30.289039+0000', 'last_fullsized': '2024-04-25T08:30:30.289039+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.290447+0000', 'last_change': '2024-04-25T08:30:30.290447+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.290447+0000', 'last_undegraded': '2024-04-25T08:30:30.290447+0000', 'last_fullsized': '2024-04-25T08:30:30.290447+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.291037+0000', 'last_change': '2024-04-25T08:30:30.291037+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.291037+0000', 'last_undegraded': '2024-04-25T08:30:30.291037+0000', 'last_fullsized': '2024-04-25T08:30:30.291037+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.297011+0000', 'last_change': '2024-04-25T08:30:30.297011+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.297011+0000', 'last_undegraded': '2024-04-25T08:30:30.297011+0000', 'last_fullsized': '2024-04-25T08:30:30.297011+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.289132+0000', 'last_change': '2024-04-25T08:30:30.289132+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.289132+0000', 'last_undegraded': '2024-04-25T08:30:30.289132+0000', 'last_fullsized': '2024-04-25T08:30:30.289132+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.286213+0000', 'last_change': '2024-04-25T08:30:30.286213+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.286213+0000', 'last_undegraded': '2024-04-25T08:30:30.286213+0000', 'last_fullsized': '2024-04-25T08:30:30.286213+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-25T08:30:35.012 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287883+0000', 'last_change': '2024-04-25T08:30:30.287883+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287883+0000', 'last_undegraded': '2024-04-25T08:30:30.287883+0000', 'last_fullsized': '2024-04-25T08:30:30.287883+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.289401+0000', 'last_change': '2024-04-25T08:30:30.289401+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.289401+0000', 'last_undegraded': '2024-04-25T08:30:30.289401+0000', 'last_fullsized': '2024-04-25T08:30:30.289401+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.296797+0000', 'last_change': '2024-04-25T08:30:30.296797+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.296797+0000', 'last_undegraded': '2024-04-25T08:30:30.296797+0000', 'last_fullsized': '2024-04-25T08:30:30.296797+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.293539+0000', 'last_change': '2024-04-25T08:30:30.293539+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.293539+0000', 'last_undegraded': '2024-04-25T08:30:30.293539+0000', 'last_fullsized': '2024-04-25T08:30:30.293539+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.288107+0000', 'last_change': '2024-04-25T08:30:30.288107+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.288107+0000', 'last_undegraded': '2024-04-25T08:30:30.288107+0000', 'last_fullsized': '2024-04-25T08:30:30.288107+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.292115+0000', 'last_change': '2024-04-25T08:30:30.292115+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.292115+0000', 'last_undegraded': '2024-04-25T08:30:30.292115+0000', 'last_fullsized': '2024-04-25T08:30:30.292115+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-25T08:30:35.013 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:30:30.287931+0000', 'last_change': '2024-04-25T08:30:30.287931+0000', 'last_active': '2024-04-25T08:30:30.277894+0000', 'last_peered': '2024-04-25T08:30:30.277894+0000', 'last_clean': '2024-04-25T08:30:30.277894+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:30:30.287931+0000', 'last_undegraded': '2024-04-25T08:30:30.287931+0000', 'last_fullsized': '2024-04-25T08:30:30.287931+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'last_clean_scrub_stamp': '2024-04-25T08:30:30.277894+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:30:35.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:30:35.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.146+0000 7f4da8888640 1 Processor -- start 2024-04-25T08:30:35.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.146+0000 7f4da8888640 1 -- start start 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da8888640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4da41602e0 0x7f4da40ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da8888640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4da415f5f0 0x7f4da415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da8888640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da40cd710 con 0x7f4da415ba00 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da8888640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da415ff80 con 0x7f4da415f5f0 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da8888640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da4099b70 con 0x7f4da41602e0 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da2d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4da415f5f0 0x7f4da415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4da415ba00 0x7f4da409a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53508/0 (socket says 172.21.15.60:53508) 2024-04-25T08:30:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.147+0000 7f4da259c640 1 -- 172.21.15.60:0/3985929867 learned_addr learned my addr 172.21.15.60:0/3985929867 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:35.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da1d9b640 1 --2- 172.21.15.60:0/3985929867 >> v2:172.21.15.100:3300/0 conn(0x7f4da41602e0 0x7f4da40ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da2d9d640 1 -- 172.21.15.60:0/3985929867 >> v2:172.21.15.100:3300/0 conn(0x7f4da41602e0 msgr2=0x7f4da40ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:35.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da2d9d640 1 --2- 172.21.15.60:0/3985929867 >> v2:172.21.15.100:3300/0 conn(0x7f4da41602e0 0x7f4da40ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da2d9d640 1 -- 172.21.15.60:0/3985929867 >> v1:172.21.15.60:6789/0 conn(0x7f4da415ba00 legacy=0x7f4da409a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da2d9d640 1 -- 172.21.15.60:0/3985929867 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4099e50 con 0x7f4da415f5f0 2024-04-25T08:30:35.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da159a640 1 -- 172.21.15.60:0/3985929867 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846280244 0 0) 0x7f4d9c001110 con 0x7f4da415ba00 2024-04-25T08:30:35.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da2d9d640 1 --2- 172.21.15.60:0/3985929867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da415f5f0 0x7f4da415fa40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4da40b7830 tx=0x7f4d9402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6613a749904cb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:35.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.148+0000 7f4da159a640 1 -- 172.21.15.60:0/3985929867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94007740 con 0x7f4da415f5f0 2024-04-25T08:30:35.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da159a640 1 -- 172.21.15.60:0/3985929867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d940078e0 con 0x7f4da415f5f0 2024-04-25T08:30:35.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da159a640 1 -- 172.21.15.60:0/3985929867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94007cb0 con 0x7f4da415f5f0 2024-04-25T08:30:35.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 -- 172.21.15.60:0/3985929867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da415f5f0 msgr2=0x7f4da415fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 --2- 172.21.15.60:0/3985929867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da415f5f0 0x7f4da415fa40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4da40b7830 tx=0x7f4d9402a560 comp rx=0 tx=0).stop 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 -- 172.21.15.60:0/3985929867 shutdown_connections 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 --2- 172.21.15.60:0/3985929867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da415f5f0 0x7f4da415fa40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 --2- 172.21.15.60:0/3985929867 >> v2:172.21.15.100:3300/0 conn(0x7f4da41602e0 0x7f4da40ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 -- 172.21.15.60:0/3985929867 >> 172.21.15.60:0/3985929867 conn(0x7f4da40b3270 msgr2=0x7f4da40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 -- 172.21.15.60:0/3985929867 shutdown_connections 2024-04-25T08:30:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.149+0000 7f4da8888640 1 -- 172.21.15.60:0/3985929867 wait complete. 2024-04-25T08:30:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 Processor -- start 2024-04-25T08:30:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 -- start start 2024-04-25T08:30:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4da415ba00 0x7f4da40b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da415f5f0 0x7f4da40b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 0x7f4da40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da415bde0 con 0x7f4da41602e0 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40cd2e0 con 0x7f4da415f5f0 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da8888640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40b1920 con 0x7f4da415ba00 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 0x7f4da40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 0x7f4da40c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45958/0 (socket says 172.21.15.60:45958) 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.151+0000 7f4da2d9d640 1 -- 172.21.15.60:0/1911540259 learned_addr learned my addr 172.21.15.60:0/1911540259 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da1d9b640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da415f5f0 0x7f4da40b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da259c640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4da415ba00 0x7f4da40b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da2d9d640 1 -- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4da415ba00 msgr2=0x7f4da40b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da2d9d640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4da415ba00 0x7f4da40b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da2d9d640 1 -- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da415f5f0 msgr2=0x7f4da40b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da2d9d640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da415f5f0 0x7f4da40b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da2d9d640 1 -- 172.21.15.60:0/1911540259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40b1640 con 0x7f4da41602e0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da1d9b640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da415f5f0 0x7f4da40b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da2d9d640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 0x7f4da40c9670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d9402ab50 tx=0x7f4d94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94040070 con 0x7f4da41602e0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94031c90 con 0x7f4da41602e0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da40c8b90 con 0x7f4da41602e0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94045680 con 0x7f4da41602e0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.152+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40c9150 con 0x7f4da41602e0 2024-04-25T08:30:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.153+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da4067be0 con 0x7f4da41602e0 2024-04-25T08:30:35.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.154+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d94039040 con 0x7f4da41602e0 2024-04-25T08:30:35.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.154+0000 7f4d9affd640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d8406b4d0 0x7f4d8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.154+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d940abd80 con 0x7f4da41602e0 2024-04-25T08:30:35.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.155+0000 7f4da259c640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d8406b4d0 0x7f4d8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.155+0000 7f4da259c640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d8406b4d0 0x7f4d8406d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c000f60 tx=0x7f4d9c007970 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.159+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4da4067be0 con 0x7f4da41602e0 2024-04-25T08:30:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.406+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4da40c9350 con 0x7f4da41602e0 2024-04-25T08:30:35.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.407+0000 7f4d9affd640 1 -- 172.21.15.60:0/1911540259 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1258 (secure 0 0 0) 0x7f4da40c9350 con 0x7f4da41602e0 2024-04-25T08:30:35.404 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:35.404 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":96,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":60},{"state_name":"creating+peering","count":37}],"num_pgs":97,"num_pools":3,"num_objects":12,"data_bytes":591062,"bytes_used":242888704,"bytes_avail":772851224576,"bytes_total":773094113280,"inactive_pgs_ratio":0.38144329190254211,"write_bytes_sec":743,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5379}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T08:30:19.411054+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:30:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d8406b4d0 msgr2=0x7f4d8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d8406b4d0 0x7f4d8406d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c000f60 tx=0x7f4d9c007970 comp rx=0 tx=0).stop 2024-04-25T08:30:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 msgr2=0x7f4da40c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 0x7f4da40c9670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d9402ab50 tx=0x7f4d94004a40 comp rx=0 tx=0).stop 2024-04-25T08:30:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 shutdown_connections 2024-04-25T08:30:35.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4da41602e0 0x7f4da40c9670 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4d8406b4d0 0x7f4d8406d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4da415f5f0 0x7f4da40b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 --2- 172.21.15.60:0/1911540259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4da415ba00 0x7f4da40b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.408+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 >> 172.21.15.60:0/1911540259 conn(0x7f4da40b3270 msgr2=0x7f4da40c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:35.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.409+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 shutdown_connections 2024-04-25T08:30:35.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.409+0000 7f4da8888640 1 -- 172.21.15.60:0/1911540259 wait complete. 2024-04-25T08:30:35.417 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:30:35.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:30:35.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.520+0000 7fe45caac640 1 Processor -- start 2024-04-25T08:30:35.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.520+0000 7fe45caac640 1 -- start start 2024-04-25T08:30:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.521+0000 7fe45caac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe458158220 0x7fe45815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.521+0000 7fe45caac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe4580af720 0x7fe4580afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.521+0000 7fe45caac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45815f180 con 0x7fe4581617b0 2024-04-25T08:30:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.521+0000 7fe45caac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580b3670 con 0x7fe458158220 2024-04-25T08:30:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.521+0000 7fe45caac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe458070a10 con 0x7fe4580af720 2024-04-25T08:30:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.521+0000 7fe45659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe458158220 0x7fe45815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe45659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe458158220 0x7fe45815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45960/0 (socket says 172.21.15.60:45960) 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe45659c640 1 -- 172.21.15.60:0/1690761902 learned_addr learned my addr 172.21.15.60:0/1690761902 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe455d9b640 1 --2- 172.21.15.60:0/1690761902 >> v2:172.21.15.100:3300/0 conn(0x7fe4580af720 0x7fe4580afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe45659c640 1 -- 172.21.15.60:0/1690761902 >> v2:172.21.15.100:3300/0 conn(0x7fe4580af720 msgr2=0x7fe4580afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe45659c640 1 --2- 172.21.15.60:0/1690761902 >> v2:172.21.15.100:3300/0 conn(0x7fe4580af720 0x7fe4580afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe45659c640 1 -- 172.21.15.60:0/1690761902 >> v1:172.21.15.100:6789/0 conn(0x7fe4581617b0 legacy=0x7fe458161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.522+0000 7fe45659c640 1 -- 172.21.15.60:0/1690761902 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45806fe00 con 0x7fe458158220 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45659c640 1 --2- 172.21.15.60:0/1690761902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe458158220 0x7fe45815ec40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4580cfc70 tx=0x7fe45002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74461cef7d3fc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45559a640 1 -- 172.21.15.60:0/1690761902 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450037070 con 0x7fe458158220 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45559a640 1 -- 172.21.15.60:0/1690761902 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450032440 con 0x7fe458158220 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45559a640 1 -- 172.21.15.60:0/1690761902 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450032810 con 0x7fe458158220 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 -- 172.21.15.60:0/1690761902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe458158220 msgr2=0x7fe45815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 --2- 172.21.15.60:0/1690761902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe458158220 0x7fe45815ec40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe4580cfc70 tx=0x7fe45002a270 comp rx=0 tx=0).stop 2024-04-25T08:30:35.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 -- 172.21.15.60:0/1690761902 shutdown_connections 2024-04-25T08:30:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 --2- 172.21.15.60:0/1690761902 >> v2:172.21.15.100:3300/0 conn(0x7fe4580af720 0x7fe4580afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 --2- 172.21.15.60:0/1690761902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe458158220 0x7fe45815ec40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 -- 172.21.15.60:0/1690761902 >> 172.21.15.60:0/1690761902 conn(0x7fe4580b2370 msgr2=0x7fe4580b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 -- 172.21.15.60:0/1690761902 shutdown_connections 2024-04-25T08:30:35.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.523+0000 7fe45caac640 1 -- 172.21.15.60:0/1690761902 wait complete. 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 Processor -- start 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 -- start start 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580af720 0x7fe45815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe458158220 0x7fe4581591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 0x7fe458159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580706a0 con 0x7fe4581617b0 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580703c0 con 0x7fe4580af720 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe45caac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe458158600 con 0x7fe458158220 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.525+0000 7fe456d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 0x7fe458159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 0x7fe458159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45970/0 (socket says 172.21.15.60:45970) 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 -- 172.21.15.60:0/974273313 learned_addr learned my addr 172.21.15.60:0/974273313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe45659c640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580af720 0x7fe45815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe455d9b640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe458158220 0x7fe4581591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 -- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe458158220 msgr2=0x7fe4581591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe458158220 0x7fe4581591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 -- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580af720 msgr2=0x7fe45815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580af720 0x7fe45815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 -- 172.21.15.60:0/974273313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458159d90 con 0x7fe4581617b0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe455d9b640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe458158220 0x7fe4581591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe456d9d640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 0x7fe458159730 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c7a0 tx=0x7fe44800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448003b70 con 0x7fe4581617b0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe448003d30 con 0x7fe4581617b0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45815a070 con 0x7fe4581617b0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800aa90 con 0x7fe4581617b0 2024-04-25T08:30:35.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.526+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45815a630 con 0x7fe4581617b0 2024-04-25T08:30:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.527+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458073990 con 0x7fe4581617b0 2024-04-25T08:30:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.528+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe44800ac30 con 0x7fe4581617b0 2024-04-25T08:30:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.528+0000 7fe44effd640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe43806b4d0 0x7fe43806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:35.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.528+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe44808a240 con 0x7fe4581617b0 2024-04-25T08:30:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.528+0000 7fe45659c640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe43806b4d0 0x7fe43806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:35.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.529+0000 7fe45659c640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe43806b4d0 0x7fe43806d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe450034000 tx=0x7fe45002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:35.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.532+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe45815f180 con 0x7fe4581617b0 2024-04-25T08:30:35.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.709+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe458072300 con 0x7fe43806b4d0 2024-04-25T08:30:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.715+0000 7fe44effd640 1 -- 172.21.15.60:0/974273313 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248860 (secure 0 0 0) 0x7fe458072300 con 0x7fe43806b4d0 2024-04-25T08:30:35.715 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:35.716 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.718+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe43806b4d0 msgr2=0x7fe43806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.718+0000 7fe45caac640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe43806b4d0 0x7fe43806d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe450034000 tx=0x7fe45002e000 comp rx=0 tx=0).stop 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.718+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 msgr2=0x7fe458159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.718+0000 7fe45caac640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 0x7fe458159730 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c7a0 tx=0x7fe44800cc60 comp rx=0 tx=0).stop 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 shutdown_connections 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4581617b0 0x7fe458159730 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe458158220 0x7fe4581591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe43806b4d0 0x7fe43806d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 --2- 172.21.15.60:0/974273313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4580af720 0x7fe45815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 >> 172.21.15.60:0/974273313 conn(0x7fe4580b2370 msgr2=0x7fe45815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:35.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 shutdown_connections 2024-04-25T08:30:35.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:35.719+0000 7fe45caac640 1 -- 172.21.15.60:0/974273313 wait complete. 2024-04-25T08:30:35.726 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-04-25T08:30:35.413883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591062,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237196,"kb_used_data":4396,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737524,"statfs":{"total":773094113280,"available":772851224576,"internally_reserved":0,"allocated":4501504,"data_stored":2587401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000927"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292388+0000","last_change":"2024-04-25T08:30:30.292388+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292388+0000","last_undegraded":"2024-04-25T08:30:30.292388+0000","last_fullsized":"2024-04-25T08:30:30.292388+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293940+0000","last_change":"2024-04-25T08:30:30.293940+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293940+0000","last_undegraded":"2024-04-25T08:30:30.293940+0000","last_fullsized":"2024-04-25T08:30:30.293940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287912+0000","last_change":"2024-04-25T08:30:30.287912+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287912+0000","last_undegraded":"2024-04-25T08:30:30.287912+0000","last_fullsized":"2024-04-25T08:30:30.287912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286932+0000","last_change":"2024-04-25T08:30:30.286932+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286932+0000","last_undegraded":"2024-04-25T08:30:30.286932+0000","last_fullsized":"2024-04-25T08:30:30.286932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286790+0000","last_change":"2024-04-25T08:30:30.286790+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286790+0000","last_undegraded":"2024-04-25T08:30:30.286790+0000","last_fullsized":"2024-04-25T08:30:30.286790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292036+0000","last_change":"2024-04-25T08:30:30.292036+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292036+0000","last_undegraded":"2024-04-25T08:30:30.292036+0000","last_fullsized":"2024-04-25T08:30:30.292036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341093+0000","last_change":"2024-04-25T08:30:31.339672+0000","last_active":"2024-04-25T08:30:31.341093+0000","last_peered":"2024-04-25T08:30:31.341093+0000","last_clean":"2024-04-25T08:30:31.341093+0000","last_became_active":"2024-04-25T08:30:31.339395+0000","last_became_peered":"2024-04-25T08:30:31.339395+0000","last_unstale":"2024-04-25T08:30:31.341093+0000","last_undegraded":"2024-04-25T08:30:31.341093+0000","last_fullsized":"2024-04-25T08:30:31.341093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.788878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289376+0000","last_change":"2024-04-25T08:30:30.289376+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289376+0000","last_undegraded":"2024-04-25T08:30:30.289376+0000","last_fullsized":"2024-04-25T08:30:30.289376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287333+0000","last_change":"2024-04-25T08:30:30.287333+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287333+0000","last_undegraded":"2024-04-25T08:30:30.287333+0000","last_fullsized":"2024-04-25T08:30:30.287333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.336655+0000","last_change":"2024-04-25T08:30:31.331300+0000","last_active":"2024-04-25T08:30:31.336655+0000","last_peered":"2024-04-25T08:30:31.336655+0000","last_clean":"2024-04-25T08:30:31.336655+0000","last_became_active":"2024-04-25T08:30:31.330974+0000","last_became_peered":"2024-04-25T08:30:31.330974+0000","last_unstale":"2024-04-25T08:30:31.336655+0000","last_undegraded":"2024-04-25T08:30:31.336655+0000","last_fullsized":"2024-04-25T08:30:31.336655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:43.081894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340959+0000","last_change":"2024-04-25T08:30:31.336378+0000","last_active":"2024-04-25T08:30:31.340959+0000","last_peered":"2024-04-25T08:30:31.340959+0000","last_clean":"2024-04-25T08:30:31.340959+0000","last_became_active":"2024-04-25T08:30:31.335915+0000","last_became_peered":"2024-04-25T08:30:31.335915+0000","last_unstale":"2024-04-25T08:30:31.340959+0000","last_undegraded":"2024-04-25T08:30:31.340959+0000","last_fullsized":"2024-04-25T08:30:31.340959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:47.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287710+0000","last_change":"2024-04-25T08:30:30.287710+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287710+0000","last_undegraded":"2024-04-25T08:30:30.287710+0000","last_fullsized":"2024-04-25T08:30:30.287710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337039+0000","last_change":"2024-04-25T08:30:31.331560+0000","last_active":"2024-04-25T08:30:31.337039+0000","last_peered":"2024-04-25T08:30:31.337039+0000","last_clean":"2024-04-25T08:30:31.337039+0000","last_became_active":"2024-04-25T08:30:31.331207+0000","last_became_peered":"2024-04-25T08:30:31.331207+0000","last_unstale":"2024-04-25T08:30:31.337039+0000","last_undegraded":"2024-04-25T08:30:31.337039+0000","last_fullsized":"2024-04-25T08:30:31.337039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:12.355397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288425+0000","last_change":"2024-04-25T08:30:30.288425+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288425+0000","last_undegraded":"2024-04-25T08:30:30.288425+0000","last_fullsized":"2024-04-25T08:30:30.288425+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340097+0000","last_change":"2024-04-25T08:30:31.334369+0000","last_active":"2024-04-25T08:30:31.340097+0000","last_peered":"2024-04-25T08:30:31.340097+0000","last_clean":"2024-04-25T08:30:31.340097+0000","last_became_active":"2024-04-25T08:30:31.334071+0000","last_became_peered":"2024-04-25T08:30:31.334071+0000","last_unstale":"2024-04-25T08:30:31.340097+0000","last_undegraded":"2024-04-25T08:30:31.340097+0000","last_fullsized":"2024-04-25T08:30:31.340097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.453764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332186+0000","last_change":"2024-04-25T08:30:31.331292+0000","last_active":"2024-04-25T08:30:31.332186+0000","last_peered":"2024-04-25T08:30:31.332186+0000","last_clean":"2024-04-25T08:30:31.332186+0000","last_became_active":"2024-04-25T08:30:31.330966+0000","last_became_peered":"2024-04-25T08:30:31.330966+0000","last_unstale":"2024-04-25T08:30:31.332186+0000","last_undegraded":"2024-04-25T08:30:31.332186+0000","last_fullsized":"2024-04-25T08:30:31.332186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.332626+0000","last_change":"2024-04-25T08:30:31.331334+0000","last_active":"2024-04-25T08:30:31.332626+0000","last_peered":"2024-04-25T08:30:31.332626+0000","last_clean":"2024-04-25T08:30:31.332626+0000","last_became_active":"2024-04-25T08:30:31.331048+0000","last_became_peered":"2024-04-25T08:30:31.331048+0000","last_unstale":"2024-04-25T08:30:31.332626+0000","last_undegraded":"2024-04-25T08:30:31.332626+0000","last_fullsized":"2024-04-25T08:30:31.332626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325080+0000","last_change":"2024-04-25T08:30:31.322980+0000","last_active":"2024-04-25T08:30:31.325080+0000","last_peered":"2024-04-25T08:30:31.325080+0000","last_clean":"2024-04-25T08:30:31.325080+0000","last_became_active":"2024-04-25T08:30:31.322718+0000","last_became_peered":"2024-04-25T08:30:31.322718+0000","last_unstale":"2024-04-25T08:30:31.325080+0000","last_undegraded":"2024-04-25T08:30:31.325080+0000","last_fullsized":"2024-04-25T08:30:31.325080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:34.413297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286631+0000","last_change":"2024-04-25T08:30:30.286631+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286631+0000","last_undegraded":"2024-04-25T08:30:30.286631+0000","last_fullsized":"2024-04-25T08:30:30.286631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323347+0000","last_change":"2024-04-25T08:30:31.321932+0000","last_active":"2024-04-25T08:30:31.323347+0000","last_peered":"2024-04-25T08:30:31.323347+0000","last_clean":"2024-04-25T08:30:31.323347+0000","last_became_active":"2024-04-25T08:30:31.321696+0000","last_became_peered":"2024-04-25T08:30:31.321696+0000","last_unstale":"2024-04-25T08:30:31.323347+0000","last_undegraded":"2024-04-25T08:30:31.323347+0000","last_fullsized":"2024-04-25T08:30:31.323347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:39.300333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.337070+0000","last_change":"2024-04-25T08:30:31.330998+0000","last_active":"2024-04-25T08:30:31.337070+0000","last_peered":"2024-04-25T08:30:31.337070+0000","last_clean":"2024-04-25T08:30:31.337070+0000","last_became_active":"2024-04-25T08:30:31.330539+0000","last_became_peered":"2024-04-25T08:30:31.330539+0000","last_unstale":"2024-04-25T08:30:31.337070+0000","last_undegraded":"2024-04-25T08:30:31.337070+0000","last_fullsized":"2024-04-25T08:30:31.337070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287479+0000","last_change":"2024-04-25T08:30:30.287479+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287479+0000","last_undegraded":"2024-04-25T08:30:30.287479+0000","last_fullsized":"2024-04-25T08:30:30.287479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323285+0000","last_change":"2024-04-25T08:30:31.320712+0000","last_active":"2024-04-25T08:30:31.323285+0000","last_peered":"2024-04-25T08:30:31.323285+0000","last_clean":"2024-04-25T08:30:31.323285+0000","last_became_active":"2024-04-25T08:30:31.320446+0000","last_became_peered":"2024-04-25T08:30:31.320446+0000","last_unstale":"2024-04-25T08:30:31.323285+0000","last_undegraded":"2024-04-25T08:30:31.323285+0000","last_fullsized":"2024-04-25T08:30:31.323285+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.070232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286827+0000","last_change":"2024-04-25T08:30:30.286827+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286827+0000","last_undegraded":"2024-04-25T08:30:30.286827+0000","last_fullsized":"2024-04-25T08:30:30.286827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.327272+0000","last_change":"2024-04-25T08:30:31.324771+0000","last_active":"2024-04-25T08:30:31.327272+0000","last_peered":"2024-04-25T08:30:31.327272+0000","last_clean":"2024-04-25T08:30:31.327272+0000","last_became_active":"2024-04-25T08:30:31.324509+0000","last_became_peered":"2024-04-25T08:30:31.324509+0000","last_unstale":"2024-04-25T08:30:31.327272+0000","last_undegraded":"2024-04-25T08:30:31.327272+0000","last_fullsized":"2024-04-25T08:30:31.327272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:32.590177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.298366+0000","last_change":"2024-04-25T08:30:30.298366+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.298366+0000","last_undegraded":"2024-04-25T08:30:30.298366+0000","last_fullsized":"2024-04-25T08:30:30.298366+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.319706+0000","last_change":"2024-04-25T08:30:31.316804+0000","last_active":"2024-04-25T08:30:31.319706+0000","last_peered":"2024-04-25T08:30:31.319706+0000","last_clean":"2024-04-25T08:30:31.319706+0000","last_became_active":"2024-04-25T08:30:31.316503+0000","last_became_peered":"2024-04-25T08:30:31.316503+0000","last_unstale":"2024-04-25T08:30:31.319706+0000","last_undegraded":"2024-04-25T08:30:31.319706+0000","last_fullsized":"2024-04-25T08:30:31.319706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:43.729410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.339771+0000","last_change":"2024-04-25T08:30:31.331474+0000","last_active":"2024-04-25T08:30:31.339771+0000","last_peered":"2024-04-25T08:30:31.339771+0000","last_clean":"2024-04-25T08:30:31.339771+0000","last_became_active":"2024-04-25T08:30:31.331058+0000","last_became_peered":"2024-04-25T08:30:31.331058+0000","last_unstale":"2024-04-25T08:30:31.339771+0000","last_undegraded":"2024-04-25T08:30:31.339771+0000","last_fullsized":"2024-04-25T08:30:31.339771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320679+0000","last_change":"2024-04-25T08:30:31.319410+0000","last_active":"2024-04-25T08:30:31.320679+0000","last_peered":"2024-04-25T08:30:31.320679+0000","last_clean":"2024-04-25T08:30:31.320679+0000","last_became_active":"2024-04-25T08:30:31.318913+0000","last_became_peered":"2024-04-25T08:30:31.318913+0000","last_unstale":"2024-04-25T08:30:31.320679+0000","last_undegraded":"2024-04-25T08:30:31.320679+0000","last_fullsized":"2024-04-25T08:30:31.320679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:14.713545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.320655+0000","last_change":"2024-04-25T08:30:31.318909+0000","last_active":"2024-04-25T08:30:31.320655+0000","last_peered":"2024-04-25T08:30:31.320655+0000","last_clean":"2024-04-25T08:30:31.320655+0000","last_became_active":"2024-04-25T08:30:31.318639+0000","last_became_peered":"2024-04-25T08:30:31.318639+0000","last_unstale":"2024-04-25T08:30:31.320655+0000","last_undegraded":"2024-04-25T08:30:31.320655+0000","last_fullsized":"2024-04-25T08:30:31.320655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:50.624325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323302+0000","last_change":"2024-04-25T08:30:31.321924+0000","last_active":"2024-04-25T08:30:31.323302+0000","last_peered":"2024-04-25T08:30:31.323302+0000","last_clean":"2024-04-25T08:30:31.323302+0000","last_became_active":"2024-04-25T08:30:31.321434+0000","last_became_peered":"2024-04-25T08:30:31.321434+0000","last_unstale":"2024-04-25T08:30:31.323302+0000","last_undegraded":"2024-04-25T08:30:31.323302+0000","last_fullsized":"2024-04-25T08:30:31.323302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289797+0000","last_change":"2024-04-25T08:30:30.289797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289797+0000","last_undegraded":"2024-04-25T08:30:30.289797+0000","last_fullsized":"2024-04-25T08:30:30.289797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338955+0000","last_change":"2024-04-25T08:30:31.332758+0000","last_active":"2024-04-25T08:30:31.338955+0000","last_peered":"2024-04-25T08:30:31.338955+0000","last_clean":"2024-04-25T08:30:31.338955+0000","last_became_active":"2024-04-25T08:30:31.332458+0000","last_became_peered":"2024-04-25T08:30:31.332458+0000","last_unstale":"2024-04-25T08:30:31.338955+0000","last_undegraded":"2024-04-25T08:30:31.338955+0000","last_fullsized":"2024-04-25T08:30:31.338955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340572+0000","last_change":"2024-04-25T08:30:30.294755+0000","last_active":"2024-04-25T08:30:31.340572+0000","last_peered":"2024-04-25T08:30:31.340572+0000","last_clean":"2024-04-25T08:30:31.340572+0000","last_became_active":"2024-04-25T08:30:30.293812+0000","last_became_peered":"2024-04-25T08:30:30.293812+0000","last_unstale":"2024-04-25T08:30:31.340572+0000","last_undegraded":"2024-04-25T08:30:31.340572+0000","last_fullsized":"2024-04-25T08:30:31.340572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.299433+0000","last_change":"2024-04-25T08:30:30.299433+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.299433+0000","last_undegraded":"2024-04-25T08:30:30.299433+0000","last_fullsized":"2024-04-25T08:30:30.299433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304327+0000","last_change":"2024-04-25T08:30:30.301932+0000","last_active":"2024-04-25T08:30:30.304327+0000","last_peered":"2024-04-25T08:30:30.304327+0000","last_clean":"2024-04-25T08:30:30.304327+0000","last_became_active":"2024-04-25T08:30:30.301633+0000","last_became_peered":"2024-04-25T08:30:30.301633+0000","last_unstale":"2024-04-25T08:30:30.304327+0000","last_undegraded":"2024-04-25T08:30:30.304327+0000","last_fullsized":"2024-04-25T08:30:30.304327+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286830+0000","last_change":"2024-04-25T08:30:30.286830+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286830+0000","last_undegraded":"2024-04-25T08:30:30.286830+0000","last_fullsized":"2024-04-25T08:30:30.286830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306192+0000","last_change":"2024-04-25T08:30:30.304552+0000","last_active":"2024-04-25T08:30:30.306192+0000","last_peered":"2024-04-25T08:30:30.306192+0000","last_clean":"2024-04-25T08:30:30.306192+0000","last_became_active":"2024-04-25T08:30:30.304168+0000","last_became_peered":"2024-04-25T08:30:30.304168+0000","last_unstale":"2024-04-25T08:30:30.306192+0000","last_undegraded":"2024-04-25T08:30:30.306192+0000","last_fullsized":"2024-04-25T08:30:30.306192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292899+0000","last_change":"2024-04-25T08:30:30.292899+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292899+0000","last_undegraded":"2024-04-25T08:30:30.292899+0000","last_fullsized":"2024-04-25T08:30:30.292899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304321+0000","last_change":"2024-04-25T08:30:30.301217+0000","last_active":"2024-04-25T08:30:30.304321+0000","last_peered":"2024-04-25T08:30:30.304321+0000","last_clean":"2024-04-25T08:30:30.304321+0000","last_became_active":"2024-04-25T08:30:30.300899+0000","last_became_peered":"2024-04-25T08:30:30.300899+0000","last_unstale":"2024-04-25T08:30:30.304321+0000","last_undegraded":"2024-04-25T08:30:30.304321+0000","last_fullsized":"2024-04-25T08:30:30.304321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289501+0000","last_change":"2024-04-25T08:30:30.289501+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289501+0000","last_undegraded":"2024-04-25T08:30:30.289501+0000","last_fullsized":"2024-04-25T08:30:30.289501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.338973+0000","last_change":"2024-04-25T08:30:30.302842+0000","last_active":"2024-04-25T08:30:31.338973+0000","last_peered":"2024-04-25T08:30:31.338973+0000","last_clean":"2024-04-25T08:30:31.338973+0000","last_became_active":"2024-04-25T08:30:30.302540+0000","last_became_peered":"2024-04-25T08:30:30.302540+0000","last_unstale":"2024-04-25T08:30:31.338973+0000","last_undegraded":"2024-04-25T08:30:31.338973+0000","last_fullsized":"2024-04-25T08:30:31.338973+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.946206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335411+0000","last_change":"2024-04-25T08:30:31.330857+0000","last_active":"2024-04-25T08:30:31.335411+0000","last_peered":"2024-04-25T08:30:31.335411+0000","last_clean":"2024-04-25T08:30:31.335411+0000","last_became_active":"2024-04-25T08:30:31.329563+0000","last_became_peered":"2024-04-25T08:30:31.329563+0000","last_unstale":"2024-04-25T08:30:31.335411+0000","last_undegraded":"2024-04-25T08:30:31.335411+0000","last_fullsized":"2024-04-25T08:30:31.335411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341386+0000","last_change":"2024-04-25T08:30:30.298067+0000","last_active":"2024-04-25T08:30:31.341386+0000","last_peered":"2024-04-25T08:30:31.341386+0000","last_clean":"2024-04-25T08:30:31.341386+0000","last_became_active":"2024-04-25T08:30:30.297718+0000","last_became_peered":"2024-04-25T08:30:30.297718+0000","last_unstale":"2024-04-25T08:30:31.341386+0000","last_undegraded":"2024-04-25T08:30:31.341386+0000","last_fullsized":"2024-04-25T08:30:31.341386+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287002+0000","last_change":"2024-04-25T08:30:30.287002+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287002+0000","last_undegraded":"2024-04-25T08:30:30.287002+0000","last_fullsized":"2024-04-25T08:30:30.287002+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306219+0000","last_change":"2024-04-25T08:30:30.305406+0000","last_active":"2024-04-25T08:30:30.306219+0000","last_peered":"2024-04-25T08:30:30.306219+0000","last_clean":"2024-04-25T08:30:30.306219+0000","last_became_active":"2024-04-25T08:30:30.305041+0000","last_became_peered":"2024-04-25T08:30:30.305041+0000","last_unstale":"2024-04-25T08:30:30.306219+0000","last_undegraded":"2024-04-25T08:30:30.306219+0000","last_fullsized":"2024-04-25T08:30:30.306219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291475+0000","last_change":"2024-04-25T08:30:30.291475+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291475+0000","last_undegraded":"2024-04-25T08:30:30.291475+0000","last_fullsized":"2024-04-25T08:30:30.291475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295501+0000","last_change":"2024-04-25T08:30:30.290419+0000","last_active":"2024-04-25T08:30:30.295501+0000","last_peered":"2024-04-25T08:30:30.295501+0000","last_clean":"2024-04-25T08:30:30.295501+0000","last_became_active":"2024-04-25T08:30:30.290143+0000","last_became_peered":"2024-04-25T08:30:30.290143+0000","last_unstale":"2024-04-25T08:30:30.295501+0000","last_undegraded":"2024-04-25T08:30:30.295501+0000","last_fullsized":"2024-04-25T08:30:30.295501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:19.910492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286356+0000","last_change":"2024-04-25T08:30:30.286356+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286356+0000","last_undegraded":"2024-04-25T08:30:30.286356+0000","last_fullsized":"2024-04-25T08:30:30.286356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305495+0000","last_change":"2024-04-25T08:30:30.302163+0000","last_active":"2024-04-25T08:30:30.305495+0000","last_peered":"2024-04-25T08:30:30.305495+0000","last_clean":"2024-04-25T08:30:30.305495+0000","last_became_active":"2024-04-25T08:30:30.301848+0000","last_became_peered":"2024-04-25T08:30:30.301848+0000","last_unstale":"2024-04-25T08:30:30.305495+0000","last_undegraded":"2024-04-25T08:30:30.305495+0000","last_fullsized":"2024-04-25T08:30:30.305495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:50.715081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323281+0000","last_change":"2024-04-25T08:30:31.319799+0000","last_active":"2024-04-25T08:30:31.323281+0000","last_peered":"2024-04-25T08:30:31.323281+0000","last_clean":"2024-04-25T08:30:31.323281+0000","last_became_active":"2024-04-25T08:30:31.319152+0000","last_became_peered":"2024-04-25T08:30:31.319152+0000","last_unstale":"2024-04-25T08:30:31.323281+0000","last_undegraded":"2024-04-25T08:30:31.323281+0000","last_fullsized":"2024-04-25T08:30:31.323281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:11.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.325859+0000","last_change":"2024-04-25T08:30:30.299038+0000","last_active":"2024-04-25T08:30:31.325859+0000","last_peered":"2024-04-25T08:30:31.325859+0000","last_clean":"2024-04-25T08:30:31.325859+0000","last_became_active":"2024-04-25T08:30:30.298782+0000","last_became_peered":"2024-04-25T08:30:30.298782+0000","last_unstale":"2024-04-25T08:30:31.325859+0000","last_undegraded":"2024-04-25T08:30:31.325859+0000","last_fullsized":"2024-04-25T08:30:31.325859+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:47.771125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287963+0000","last_change":"2024-04-25T08:30:30.287963+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287963+0000","last_undegraded":"2024-04-25T08:30:30.287963+0000","last_fullsized":"2024-04-25T08:30:30.287963+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341021+0000","last_change":"2024-04-25T08:30:30.301686+0000","last_active":"2024-04-25T08:30:31.341021+0000","last_peered":"2024-04-25T08:30:31.341021+0000","last_clean":"2024-04-25T08:30:31.341021+0000","last_became_active":"2024-04-25T08:30:30.301391+0000","last_became_peered":"2024-04-25T08:30:30.301391+0000","last_unstale":"2024-04-25T08:30:31.341021+0000","last_undegraded":"2024-04-25T08:30:31.341021+0000","last_fullsized":"2024-04-25T08:30:31.341021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.852169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288660+0000","last_change":"2024-04-25T08:30:30.288660+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288660+0000","last_undegraded":"2024-04-25T08:30:30.288660+0000","last_fullsized":"2024-04-25T08:30:30.288660+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305486+0000","last_change":"2024-04-25T08:30:30.303220+0000","last_active":"2024-04-25T08:30:30.305486+0000","last_peered":"2024-04-25T08:30:30.305486+0000","last_clean":"2024-04-25T08:30:30.305486+0000","last_became_active":"2024-04-25T08:30:30.302914+0000","last_became_peered":"2024-04-25T08:30:30.302914+0000","last_unstale":"2024-04-25T08:30:30.305486+0000","last_undegraded":"2024-04-25T08:30:30.305486+0000","last_fullsized":"2024-04-25T08:30:30.305486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:10.215234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289039+0000","last_change":"2024-04-25T08:30:30.289039+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289039+0000","last_undegraded":"2024-04-25T08:30:30.289039+0000","last_fullsized":"2024-04-25T08:30:30.289039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.304930+0000","last_change":"2024-04-25T08:30:30.301164+0000","last_active":"2024-04-25T08:30:30.304930+0000","last_peered":"2024-04-25T08:30:30.304930+0000","last_clean":"2024-04-25T08:30:30.304930+0000","last_became_active":"2024-04-25T08:30:30.300887+0000","last_became_peered":"2024-04-25T08:30:30.300887+0000","last_unstale":"2024-04-25T08:30:30.304930+0000","last_undegraded":"2024-04-25T08:30:30.304930+0000","last_fullsized":"2024-04-25T08:30:30.304930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.290447+0000","last_change":"2024-04-25T08:30:30.290447+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.290447+0000","last_undegraded":"2024-04-25T08:30:30.290447+0000","last_fullsized":"2024-04-25T08:30:30.290447+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.306196+0000","last_change":"2024-04-25T08:30:30.304789+0000","last_active":"2024-04-25T08:30:30.306196+0000","last_peered":"2024-04-25T08:30:30.306196+0000","last_clean":"2024-04-25T08:30:30.306196+0000","last_became_active":"2024-04-25T08:30:30.304493+0000","last_became_peered":"2024-04-25T08:30:30.304493+0000","last_unstale":"2024-04-25T08:30:30.306196+0000","last_undegraded":"2024-04-25T08:30:30.306196+0000","last_fullsized":"2024-04-25T08:30:30.306196+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330098+0000","last_change":"2024-04-25T08:30:31.326518+0000","last_active":"2024-04-25T08:30:31.330098+0000","last_peered":"2024-04-25T08:30:31.330098+0000","last_clean":"2024-04-25T08:30:31.330098+0000","last_became_active":"2024-04-25T08:30:31.326241+0000","last_became_peered":"2024-04-25T08:30:31.326241+0000","last_unstale":"2024-04-25T08:30:31.330098+0000","last_undegraded":"2024-04-25T08:30:31.330098+0000","last_fullsized":"2024-04-25T08:30:31.330098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310441+0000","last_change":"2024-04-25T08:30:30.294767+0000","last_active":"2024-04-25T08:30:31.310441+0000","last_peered":"2024-04-25T08:30:31.310441+0000","last_clean":"2024-04-25T08:30:31.310441+0000","last_became_active":"2024-04-25T08:30:30.294475+0000","last_became_peered":"2024-04-25T08:30:30.294475+0000","last_unstale":"2024-04-25T08:30:31.310441+0000","last_undegraded":"2024-04-25T08:30:31.310441+0000","last_fullsized":"2024-04-25T08:30:31.310441+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.291037+0000","last_change":"2024-04-25T08:30:30.291037+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.291037+0000","last_undegraded":"2024-04-25T08:30:30.291037+0000","last_fullsized":"2024-04-25T08:30:30.291037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.342812+0000","last_change":"2024-04-25T08:30:30.297207+0000","last_active":"2024-04-25T08:30:31.342812+0000","last_peered":"2024-04-25T08:30:31.342812+0000","last_clean":"2024-04-25T08:30:31.342812+0000","last_became_active":"2024-04-25T08:30:30.296922+0000","last_became_peered":"2024-04-25T08:30:30.296922+0000","last_unstale":"2024-04-25T08:30:31.342812+0000","last_undegraded":"2024-04-25T08:30:31.342812+0000","last_fullsized":"2024-04-25T08:30:31.342812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:13.711704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.297011+0000","last_change":"2024-04-25T08:30:30.297011+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.297011+0000","last_undegraded":"2024-04-25T08:30:30.297011+0000","last_fullsized":"2024-04-25T08:30:30.297011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340937+0000","last_change":"2024-04-25T08:30:30.297702+0000","last_active":"2024-04-25T08:30:31.340937+0000","last_peered":"2024-04-25T08:30:31.340937+0000","last_clean":"2024-04-25T08:30:31.340937+0000","last_became_active":"2024-04-25T08:30:30.297420+0000","last_became_peered":"2024-04-25T08:30:30.297420+0000","last_unstale":"2024-04-25T08:30:31.340937+0000","last_undegraded":"2024-04-25T08:30:31.340937+0000","last_fullsized":"2024-04-25T08:30:31.340937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.137643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341074+0000","last_change":"2024-04-25T08:30:31.337230+0000","last_active":"2024-04-25T08:30:31.341074+0000","last_peered":"2024-04-25T08:30:31.341074+0000","last_clean":"2024-04-25T08:30:31.341074+0000","last_became_active":"2024-04-25T08:30:31.336785+0000","last_became_peered":"2024-04-25T08:30:31.336785+0000","last_unstale":"2024-04-25T08:30:31.341074+0000","last_undegraded":"2024-04-25T08:30:31.341074+0000","last_fullsized":"2024-04-25T08:30:31.341074+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:28.421465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301349+0000","last_change":"2024-04-25T08:30:30.296836+0000","last_active":"2024-04-25T08:30:30.301349+0000","last_peered":"2024-04-25T08:30:30.301349+0000","last_clean":"2024-04-25T08:30:30.301349+0000","last_became_active":"2024-04-25T08:30:30.296490+0000","last_became_peered":"2024-04-25T08:30:30.296490+0000","last_unstale":"2024-04-25T08:30:30.301349+0000","last_undegraded":"2024-04-25T08:30:30.301349+0000","last_fullsized":"2024-04-25T08:30:30.301349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289132+0000","last_change":"2024-04-25T08:30:30.289132+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289132+0000","last_undegraded":"2024-04-25T08:30:30.289132+0000","last_fullsized":"2024-04-25T08:30:30.289132+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.300070+0000","last_change":"2024-04-25T08:30:30.297703+0000","last_active":"2024-04-25T08:30:30.300070+0000","last_peered":"2024-04-25T08:30:30.300070+0000","last_clean":"2024-04-25T08:30:30.300070+0000","last_became_active":"2024-04-25T08:30:30.297424+0000","last_became_peered":"2024-04-25T08:30:30.297424+0000","last_unstale":"2024-04-25T08:30:30.300070+0000","last_undegraded":"2024-04-25T08:30:30.300070+0000","last_fullsized":"2024-04-25T08:30:30.300070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.320688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.286213+0000","last_change":"2024-04-25T08:30:30.286213+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.286213+0000","last_undegraded":"2024-04-25T08:30:30.286213+0000","last_fullsized":"2024-04-25T08:30:30.286213+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305672+0000","last_change":"2024-04-25T08:30:30.303603+0000","last_active":"2024-04-25T08:30:30.305672+0000","last_peered":"2024-04-25T08:30:30.305672+0000","last_clean":"2024-04-25T08:30:30.305672+0000","last_became_active":"2024-04-25T08:30:30.303322+0000","last_became_peered":"2024-04-25T08:30:30.303322+0000","last_unstale":"2024-04-25T08:30:30.305672+0000","last_undegraded":"2024-04-25T08:30:30.305672+0000","last_fullsized":"2024-04-25T08:30:30.305672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.330062+0000","last_change":"2024-04-25T08:30:31.325746+0000","last_active":"2024-04-25T08:30:31.330062+0000","last_peered":"2024-04-25T08:30:31.330062+0000","last_clean":"2024-04-25T08:30:31.330062+0000","last_became_active":"2024-04-25T08:30:31.325449+0000","last_became_peered":"2024-04-25T08:30:31.325449+0000","last_unstale":"2024-04-25T08:30:31.330062+0000","last_undegraded":"2024-04-25T08:30:31.330062+0000","last_fullsized":"2024-04-25T08:30:31.330062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295651+0000","last_change":"2024-04-25T08:30:30.293002+0000","last_active":"2024-04-25T08:30:30.295651+0000","last_peered":"2024-04-25T08:30:30.295651+0000","last_clean":"2024-04-25T08:30:30.295651+0000","last_became_active":"2024-04-25T08:30:30.292702+0000","last_became_peered":"2024-04-25T08:30:30.292702+0000","last_unstale":"2024-04-25T08:30:30.295651+0000","last_undegraded":"2024-04-25T08:30:30.295651+0000","last_fullsized":"2024-04-25T08:30:30.295651+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.414247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287883+0000","last_change":"2024-04-25T08:30:30.287883+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287883+0000","last_undegraded":"2024-04-25T08:30:30.287883+0000","last_fullsized":"2024-04-25T08:30:30.287883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.295421+0000","last_change":"2024-04-25T08:30:30.292431+0000","last_active":"2024-04-25T08:30:30.295421+0000","last_peered":"2024-04-25T08:30:30.295421+0000","last_clean":"2024-04-25T08:30:30.295421+0000","last_became_active":"2024-04-25T08:30:30.292078+0000","last_became_peered":"2024-04-25T08:30:30.292078+0000","last_unstale":"2024-04-25T08:30:30.295421+0000","last_undegraded":"2024-04-25T08:30:30.295421+0000","last_fullsized":"2024-04-25T08:30:30.295421+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:06.079852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.335419+0000","last_change":"2024-04-25T08:30:31.331176+0000","last_active":"2024-04-25T08:30:31.335419+0000","last_peered":"2024-04-25T08:30:31.335419+0000","last_clean":"2024-04-25T08:30:31.335419+0000","last_became_active":"2024-04-25T08:30:31.330823+0000","last_became_peered":"2024-04-25T08:30:31.330823+0000","last_unstale":"2024-04-25T08:30:31.335419+0000","last_undegraded":"2024-04-25T08:30:31.335419+0000","last_fullsized":"2024-04-25T08:30:31.335419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:21.292627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311697+0000","last_change":"2024-04-25T08:30:30.299966+0000","last_active":"2024-04-25T08:30:31.311697+0000","last_peered":"2024-04-25T08:30:31.311697+0000","last_clean":"2024-04-25T08:30:31.311697+0000","last_became_active":"2024-04-25T08:30:30.299699+0000","last_became_peered":"2024-04-25T08:30:30.299699+0000","last_unstale":"2024-04-25T08:30:31.311697+0000","last_undegraded":"2024-04-25T08:30:31.311697+0000","last_fullsized":"2024-04-25T08:30:31.311697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.289401+0000","last_change":"2024-04-25T08:30:30.289401+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.289401+0000","last_undegraded":"2024-04-25T08:30:30.289401+0000","last_fullsized":"2024-04-25T08:30:30.289401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.297186+0000","last_change":"2024-04-25T08:30:30.293467+0000","last_active":"2024-04-25T08:30:30.297186+0000","last_peered":"2024-04-25T08:30:30.297186+0000","last_clean":"2024-04-25T08:30:30.297186+0000","last_became_active":"2024-04-25T08:30:30.293179+0000","last_became_peered":"2024-04-25T08:30:30.293179+0000","last_unstale":"2024-04-25T08:30:30.297186+0000","last_undegraded":"2024-04-25T08:30:30.297186+0000","last_fullsized":"2024-04-25T08:30:30.297186+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:10.629362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.296797+0000","last_change":"2024-04-25T08:30:30.296797+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.296797+0000","last_undegraded":"2024-04-25T08:30:30.296797+0000","last_fullsized":"2024-04-25T08:30:30.296797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.340294+0000","last_change":"2024-04-25T08:30:30.301760+0000","last_active":"2024-04-25T08:30:31.340294+0000","last_peered":"2024-04-25T08:30:31.340294+0000","last_clean":"2024-04-25T08:30:31.340294+0000","last_became_active":"2024-04-25T08:30:30.301331+0000","last_became_peered":"2024-04-25T08:30:30.301331+0000","last_unstale":"2024-04-25T08:30:31.340294+0000","last_undegraded":"2024-04-25T08:30:31.340294+0000","last_fullsized":"2024-04-25T08:30:31.340294+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323304+0000","last_change":"2024-04-25T08:30:31.320048+0000","last_active":"2024-04-25T08:30:31.323304+0000","last_peered":"2024-04-25T08:30:31.323304+0000","last_clean":"2024-04-25T08:30:31.323304+0000","last_became_active":"2024-04-25T08:30:31.319495+0000","last_became_peered":"2024-04-25T08:30:31.319495+0000","last_unstale":"2024-04-25T08:30:31.323304+0000","last_undegraded":"2024-04-25T08:30:31.323304+0000","last_fullsized":"2024-04-25T08:30:31.323304+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:05.728351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.299425+0000","last_change":"2024-04-25T08:30:30.296903+0000","last_active":"2024-04-25T08:30:30.299425+0000","last_peered":"2024-04-25T08:30:30.299425+0000","last_clean":"2024-04-25T08:30:30.299425+0000","last_became_active":"2024-04-25T08:30:30.296623+0000","last_became_peered":"2024-04-25T08:30:30.296623+0000","last_unstale":"2024-04-25T08:30:30.299425+0000","last_undegraded":"2024-04-25T08:30:30.299425+0000","last_fullsized":"2024-04-25T08:30:30.299425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:03.007004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.293539+0000","last_change":"2024-04-25T08:30:30.293539+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.293539+0000","last_undegraded":"2024-04-25T08:30:30.293539+0000","last_fullsized":"2024-04-25T08:30:30.293539+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.305561+0000","last_change":"2024-04-25T08:30:30.303229+0000","last_active":"2024-04-25T08:30:30.305561+0000","last_peered":"2024-04-25T08:30:30.305561+0000","last_clean":"2024-04-25T08:30:30.305561+0000","last_became_active":"2024-04-25T08:30:30.302894+0000","last_became_peered":"2024-04-25T08:30:30.302894+0000","last_unstale":"2024-04-25T08:30:30.305561+0000","last_undegraded":"2024-04-25T08:30:30.305561+0000","last_fullsized":"2024-04-25T08:30:30.305561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.288107+0000","last_change":"2024-04-25T08:30:30.288107+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.288107+0000","last_undegraded":"2024-04-25T08:30:30.288107+0000","last_fullsized":"2024-04-25T08:30:30.288107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.301519+0000","last_change":"2024-04-25T08:30:30.300280+0000","last_active":"2024-04-25T08:30:30.301519+0000","last_peered":"2024-04-25T08:30:30.301519+0000","last_clean":"2024-04-25T08:30:30.301519+0000","last_became_active":"2024-04-25T08:30:30.299985+0000","last_became_peered":"2024-04-25T08:30:30.299985+0000","last_unstale":"2024-04-25T08:30:30.301519+0000","last_undegraded":"2024-04-25T08:30:30.301519+0000","last_fullsized":"2024-04-25T08:30:30.301519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:06.658964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.292115+0000","last_change":"2024-04-25T08:30:30.292115+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.292115+0000","last_undegraded":"2024-04-25T08:30:30.292115+0000","last_fullsized":"2024-04-25T08:30:30.292115+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.312138+0000","last_change":"2024-04-25T08:30:30.293212+0000","last_active":"2024-04-25T08:30:31.312138+0000","last_peered":"2024-04-25T08:30:31.312138+0000","last_clean":"2024-04-25T08:30:31.312138+0000","last_became_active":"2024-04-25T08:30:30.292931+0000","last_became_peered":"2024-04-25T08:30:30.292931+0000","last_unstale":"2024-04-25T08:30:31.312138+0000","last_undegraded":"2024-04-25T08:30:31.312138+0000","last_fullsized":"2024-04-25T08:30:31.312138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T08:30:30.287931+0000","last_change":"2024-04-25T08:30:30.287931+0000","last_active":"2024-04-25T08:30:30.277894+0000","last_peered":"2024-04-25T08:30:30.277894+0000","last_clean":"2024-04-25T08:30:30.277894+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:30:30.287931+0000","last_undegraded":"2024-04-25T08:30:30.287931+0000","last_fullsized":"2024-04-25T08:30:30.287931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.302154+0000","last_change":"2024-04-25T08:30:30.298769+0000","last_active":"2024-04-25T08:30:30.302154+0000","last_peered":"2024-04-25T08:30:30.302154+0000","last_clean":"2024-04-25T08:30:30.302154+0000","last_became_active":"2024-04-25T08:30:30.298489+0000","last_became_peered":"2024-04-25T08:30:30.298489+0000","last_unstale":"2024-04-25T08:30:30.302154+0000","last_undegraded":"2024-04-25T08:30:30.302154+0000","last_fullsized":"2024-04-25T08:30:30.302154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:18.827612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.323460+0000","last_change":"2024-04-25T08:30:31.318757+0000","last_active":"2024-04-25T08:30:31.323460+0000","last_peered":"2024-04-25T08:30:31.323460+0000","last_clean":"2024-04-25T08:30:31.323460+0000","last_became_active":"2024-04-25T08:30:31.318482+0000","last_became_peered":"2024-04-25T08:30:31.318482+0000","last_unstale":"2024-04-25T08:30:31.323460+0000","last_undegraded":"2024-04-25T08:30:31.323460+0000","last_fullsized":"2024-04-25T08:30:31.323460+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:51.716923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T08:30:30.296818+0000","last_change":"2024-04-25T08:30:30.295325+0000","last_active":"2024-04-25T08:30:30.296818+0000","last_peered":"2024-04-25T08:30:30.296818+0000","last_clean":"2024-04-25T08:30:30.296818+0000","last_became_active":"2024-04-25T08:30:30.295030+0000","last_became_peered":"2024-04-25T08:30:30.295030+0000","last_unstale":"2024-04-25T08:30:30.296818+0000","last_undegraded":"2024-04-25T08:30:30.296818+0000","last_fullsized":"2024-04-25T08:30:30.296818+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.341677+0000","last_change":"2024-04-25T08:30:31.340453+0000","last_active":"2024-04-25T08:30:31.341677+0000","last_peered":"2024-04-25T08:30:31.341677+0000","last_clean":"2024-04-25T08:30:31.341677+0000","last_became_active":"2024-04-25T08:30:31.340188+0000","last_became_peered":"2024-04-25T08:30:31.340188+0000","last_unstale":"2024-04-25T08:30:31.341677+0000","last_undegraded":"2024-04-25T08:30:31.341677+0000","last_fullsized":"2024-04-25T08:30:31.341677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.311406+0000","last_change":"2024-04-25T08:30:30.300697+0000","last_active":"2024-04-25T08:30:31.311406+0000","last_peered":"2024-04-25T08:30:31.311406+0000","last_clean":"2024-04-25T08:30:31.311406+0000","last_became_active":"2024-04-25T08:30:30.300410+0000","last_became_peered":"2024-04-25T08:30:30.300410+0000","last_unstale":"2024-04-25T08:30:31.311406+0000","last_undegraded":"2024-04-25T08:30:31.311406+0000","last_fullsized":"2024-04-25T08:30:31.311406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:38.298492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:30:31.310971+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:30:31.310971+0000","last_peered":"2024-04-25T08:30:31.310971+0000","last_clean":"2024-04-25T08:30:31.310971+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:30:31.310971+0000","last_undegraded":"2024-04-25T08:30:31.310971+0000","last_fullsized":"2024-04-25T08:30:31.310971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":694,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1015808,"data_stored":770368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":180000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":173333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":763701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:30:36.271 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:30:36.274+0000 7f689a696640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:30:38.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:30:38.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.843+0000 7f5108ae1640 1 Processor -- start 2024-04-25T08:30:38.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.843+0000 7f5108ae1640 1 -- start start 2024-04-25T08:30:38.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f5108ae1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5104154ee0 0x7f51040cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:38.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f5108ae1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51040cfcf0 0x7f51040cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f5108ae1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5104071480 con 0x7f51040ce3c0 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f5108ae1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51040ce830 con 0x7f51040cfcf0 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f5108ae1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5104072090 con 0x7f5104154ee0 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51040cfcf0 0x7f51040cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f51040cfcf0 0x7f51040cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45986/0 (socket says 172.21.15.60:45986) 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 -- 172.21.15.60:0/1636416354 learned_addr learned my addr 172.21.15.60:0/1636416354 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 -- 172.21.15.60:0/1636416354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5104154ee0 msgr2=0x7f51040cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 --2- 172.21.15.60:0/1636416354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5104154ee0 0x7f51040cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 -- 172.21.15.60:0/1636416354 >> v1:172.21.15.100:6789/0 conn(0x7f51040ce3c0 legacy=0x7f510415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:38.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.844+0000 7f50fbfff640 1 -- 172.21.15.60:0/1636416354 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5104071760 con 0x7f51040cfcf0 2024-04-25T08:30:38.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.845+0000 7f50fbfff640 1 --2- 172.21.15.60:0/1636416354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51040cfcf0 0x7f51040cde80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009870 tx=0x7f50ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a05d48a017370d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:38.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.845+0000 7f5101d9b640 1 -- 172.21.15.60:0/1636416354 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec037070 con 0x7f51040cfcf0 2024-04-25T08:30:38.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.845+0000 7f5101d9b640 1 -- 172.21.15.60:0/1636416354 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec032440 con 0x7f51040cfcf0 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.845+0000 7f5101d9b640 1 -- 172.21.15.60:0/1636416354 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec032810 con 0x7f51040cfcf0 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 -- 172.21.15.60:0/1636416354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51040cfcf0 msgr2=0x7f51040cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 --2- 172.21.15.60:0/1636416354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51040cfcf0 0x7f51040cde80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009870 tx=0x7f50ec02a210 comp rx=0 tx=0).stop 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 -- 172.21.15.60:0/1636416354 shutdown_connections 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 --2- 172.21.15.60:0/1636416354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f51040cfcf0 0x7f51040cde80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 --2- 172.21.15.60:0/1636416354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5104154ee0 0x7f51040cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 -- 172.21.15.60:0/1636416354 >> 172.21.15.60:0/1636416354 conn(0x7f510406abe0 msgr2=0x7f510406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 -- 172.21.15.60:0/1636416354 shutdown_connections 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.846+0000 7f5108ae1640 1 -- 172.21.15.60:0/1636416354 wait complete. 2024-04-25T08:30:38.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.847+0000 7f5108ae1640 1 Processor -- start 2024-04-25T08:30:38.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.847+0000 7f5108ae1640 1 -- start start 2024-04-25T08:30:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5108ae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51040ce3c0 0x7f510413a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5108ae1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51040cfcf0 0x7f510414f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5108ae1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 0x7f510414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5108ae1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5104071230 con 0x7f5104154ee0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5108ae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5104071d20 con 0x7f51040ce3c0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5108ae1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51041552c0 con 0x7f51040cfcf0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 0x7f510414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 0x7f510414f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45994/0 (socket says 172.21.15.60:45994) 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 -- 172.21.15.60:0/2104686294 learned_addr learned my addr 172.21.15.60:0/2104686294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f510259c640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51040ce3c0 0x7f510413a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f50fbfff640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51040cfcf0 0x7f510414f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 -- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51040cfcf0 msgr2=0x7f510414f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51040cfcf0 0x7f510414f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 -- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51040ce3c0 msgr2=0x7f510413a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51040ce3c0 0x7f510413a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.848+0000 7f5102d9d640 1 -- 172.21.15.60:0/2104686294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510413a5f0 con 0x7f5104154ee0 2024-04-25T08:30:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.849+0000 7f5102d9d640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 0x7f510414f5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c7b0 tx=0x7f50fc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.849+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc017070 con 0x7f5104154ee0 2024-04-25T08:30:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.849+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc003da0 con 0x7f5104154ee0 2024-04-25T08:30:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.849+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510414fbe0 con 0x7f5104154ee0 2024-04-25T08:30:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.849+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0052b0 con 0x7f5104154ee0 2024-04-25T08:30:38.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.849+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51041501a0 con 0x7f5104154ee0 2024-04-25T08:30:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.850+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5104073990 con 0x7f5104154ee0 2024-04-25T08:30:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.850+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50fc005450 con 0x7f5104154ee0 2024-04-25T08:30:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.851+0000 7f50faffd640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50e406b4d0 0x7f50e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.851+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f50fc089950 con 0x7f5104154ee0 2024-04-25T08:30:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.851+0000 7f510259c640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50e406b4d0 0x7f50e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:38.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.851+0000 7f510259c640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50e406b4d0 0x7f50e406d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50f40098e0 tx=0x7f50f4009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:38.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:38.855+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50fc057f70 con 0x7f5104154ee0 2024-04-25T08:30:39.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.030+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f510406d360 con 0x7f50e406b4d0 2024-04-25T08:30:39.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.037+0000 7f50faffd640 1 -- 172.21.15.60:0/2104686294 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251935 (secure 0 0 0) 0x7f510406d360 con 0x7f50e406b4d0 2024-04-25T08:30:39.037 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:39.038 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:30:39.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.039+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50e406b4d0 msgr2=0x7f50e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.039+0000 7f5108ae1640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50e406b4d0 0x7f50e406d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50f40098e0 tx=0x7f50f4009060 comp rx=0 tx=0).stop 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.039+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 msgr2=0x7f510414f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.039+0000 7f5108ae1640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 0x7f510414f5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c7b0 tx=0x7f50fc00cc70 comp rx=0 tx=0).stop 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 shutdown_connections 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5104154ee0 0x7f510414f5a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f51040cfcf0 0x7f510414f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50e406b4d0 0x7f50e406d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 --2- 172.21.15.60:0/2104686294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51040ce3c0 0x7f510413a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 >> 172.21.15.60:0/2104686294 conn(0x7f510406abe0 msgr2=0x7f510415d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 shutdown_connections 2024-04-25T08:30:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.040+0000 7f5108ae1640 1 -- 172.21.15.60:0/2104686294 wait complete. 2024-04-25T08:30:39.048 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-25T08:30:37.414156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237568,"kb_used_data":4712,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737152,"statfs":{"total":773094113280,"available":772850843648,"internally_reserved":0,"allocated":4825088,"data_stored":2736428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.132514"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312716+0000","last_change":"2024-04-25T08:30:31.320396+0000","last_active":"2024-04-25T08:30:32.312716+0000","last_peered":"2024-04-25T08:30:32.312716+0000","last_clean":"2024-04-25T08:30:32.312716+0000","last_became_active":"2024-04-25T08:30:31.320073+0000","last_became_peered":"2024-04-25T08:30:31.320073+0000","last_unstale":"2024-04-25T08:30:32.312716+0000","last_undegraded":"2024-04-25T08:30:32.312716+0000","last_fullsized":"2024-04-25T08:30:32.312716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:49.483225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312492+0000","last_change":"2024-04-25T08:30:31.323235+0000","last_active":"2024-04-25T08:30:32.312492+0000","last_peered":"2024-04-25T08:30:32.312492+0000","last_clean":"2024-04-25T08:30:32.312492+0000","last_became_active":"2024-04-25T08:30:31.322921+0000","last_became_peered":"2024-04-25T08:30:31.322921+0000","last_unstale":"2024-04-25T08:30:32.312492+0000","last_undegraded":"2024-04-25T08:30:32.312492+0000","last_fullsized":"2024-04-25T08:30:32.312492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:35.340416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516753+0000","last_change":"2024-04-25T08:30:31.332051+0000","last_active":"2024-04-25T08:30:32.516753+0000","last_peered":"2024-04-25T08:30:32.516753+0000","last_clean":"2024-04-25T08:30:32.516753+0000","last_became_active":"2024-04-25T08:30:31.331764+0000","last_became_peered":"2024-04-25T08:30:31.331764+0000","last_unstale":"2024-04-25T08:30:32.516753+0000","last_undegraded":"2024-04-25T08:30:32.516753+0000","last_fullsized":"2024-04-25T08:30:32.516753+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327327+0000","last_change":"2024-04-25T08:30:31.326816+0000","last_active":"2024-04-25T08:30:32.327327+0000","last_peered":"2024-04-25T08:30:32.327327+0000","last_clean":"2024-04-25T08:30:32.327327+0000","last_became_active":"2024-04-25T08:30:31.325238+0000","last_became_peered":"2024-04-25T08:30:31.325238+0000","last_unstale":"2024-04-25T08:30:32.327327+0000","last_undegraded":"2024-04-25T08:30:32.327327+0000","last_fullsized":"2024-04-25T08:30:32.327327+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:28.515627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327155+0000","last_change":"2024-04-25T08:30:31.325941+0000","last_active":"2024-04-25T08:30:32.327155+0000","last_peered":"2024-04-25T08:30:32.327155+0000","last_clean":"2024-04-25T08:30:32.327155+0000","last_became_active":"2024-04-25T08:30:31.325499+0000","last_became_peered":"2024-04-25T08:30:31.325499+0000","last_unstale":"2024-04-25T08:30:32.327155+0000","last_undegraded":"2024-04-25T08:30:32.327155+0000","last_fullsized":"2024-04-25T08:30:32.327155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:07.660805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517429+0000","last_change":"2024-04-25T08:30:31.319662+0000","last_active":"2024-04-25T08:30:32.517429+0000","last_peered":"2024-04-25T08:30:32.517429+0000","last_clean":"2024-04-25T08:30:32.517429+0000","last_became_active":"2024-04-25T08:30:31.319340+0000","last_became_peered":"2024-04-25T08:30:31.319340+0000","last_unstale":"2024-04-25T08:30:32.517429+0000","last_undegraded":"2024-04-25T08:30:32.517429+0000","last_fullsized":"2024-04-25T08:30:32.517429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329406+0000","last_change":"2024-04-25T08:30:31.339672+0000","last_active":"2024-04-25T08:30:32.329406+0000","last_peered":"2024-04-25T08:30:32.329406+0000","last_clean":"2024-04-25T08:30:32.329406+0000","last_became_active":"2024-04-25T08:30:31.339395+0000","last_became_peered":"2024-04-25T08:30:31.339395+0000","last_unstale":"2024-04-25T08:30:32.329406+0000","last_undegraded":"2024-04-25T08:30:32.329406+0000","last_fullsized":"2024-04-25T08:30:32.329406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:35.340416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312299+0000","last_change":"2024-04-25T08:30:31.333392+0000","last_active":"2024-04-25T08:30:32.312299+0000","last_peered":"2024-04-25T08:30:32.312299+0000","last_clean":"2024-04-25T08:30:32.312299+0000","last_became_active":"2024-04-25T08:30:31.333113+0000","last_became_peered":"2024-04-25T08:30:31.333113+0000","last_unstale":"2024-04-25T08:30:32.312299+0000","last_undegraded":"2024-04-25T08:30:32.312299+0000","last_fullsized":"2024-04-25T08:30:32.312299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:28.421465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517140+0000","last_change":"2024-04-25T08:30:31.332492+0000","last_active":"2024-04-25T08:30:32.517140+0000","last_peered":"2024-04-25T08:30:32.517140+0000","last_clean":"2024-04-25T08:30:32.517140+0000","last_became_active":"2024-04-25T08:30:31.332181+0000","last_became_peered":"2024-04-25T08:30:31.332181+0000","last_unstale":"2024-04-25T08:30:32.517140+0000","last_undegraded":"2024-04-25T08:30:32.517140+0000","last_fullsized":"2024-04-25T08:30:32.517140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329929+0000","last_change":"2024-04-25T08:30:31.331300+0000","last_active":"2024-04-25T08:30:32.329929+0000","last_peered":"2024-04-25T08:30:32.329929+0000","last_clean":"2024-04-25T08:30:32.329929+0000","last_became_active":"2024-04-25T08:30:31.330974+0000","last_became_peered":"2024-04-25T08:30:31.330974+0000","last_unstale":"2024-04-25T08:30:32.329929+0000","last_undegraded":"2024-04-25T08:30:32.329929+0000","last_fullsized":"2024-04-25T08:30:32.329929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.944652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329905+0000","last_change":"2024-04-25T08:30:31.336378+0000","last_active":"2024-04-25T08:30:32.329905+0000","last_peered":"2024-04-25T08:30:32.329905+0000","last_clean":"2024-04-25T08:30:32.329905+0000","last_became_active":"2024-04-25T08:30:31.335915+0000","last_became_peered":"2024-04-25T08:30:31.335915+0000","last_unstale":"2024-04-25T08:30:32.329905+0000","last_undegraded":"2024-04-25T08:30:32.329905+0000","last_fullsized":"2024-04-25T08:30:32.329905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:11.485571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516992+0000","last_change":"2024-04-25T08:30:31.326118+0000","last_active":"2024-04-25T08:30:32.516992+0000","last_peered":"2024-04-25T08:30:32.516992+0000","last_clean":"2024-04-25T08:30:32.516992+0000","last_became_active":"2024-04-25T08:30:31.325849+0000","last_became_peered":"2024-04-25T08:30:31.325849+0000","last_unstale":"2024-04-25T08:30:32.516992+0000","last_undegraded":"2024-04-25T08:30:32.516992+0000","last_fullsized":"2024-04-25T08:30:32.516992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329508+0000","last_change":"2024-04-25T08:30:31.331560+0000","last_active":"2024-04-25T08:30:32.329508+0000","last_peered":"2024-04-25T08:30:32.329508+0000","last_clean":"2024-04-25T08:30:32.329508+0000","last_became_active":"2024-04-25T08:30:31.331207+0000","last_became_peered":"2024-04-25T08:30:31.331207+0000","last_unstale":"2024-04-25T08:30:32.329508+0000","last_undegraded":"2024-04-25T08:30:32.329508+0000","last_fullsized":"2024-04-25T08:30:32.329508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:12.807489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313905+0000","last_change":"2024-04-25T08:30:31.325188+0000","last_active":"2024-04-25T08:30:32.313905+0000","last_peered":"2024-04-25T08:30:32.313905+0000","last_clean":"2024-04-25T08:30:32.313905+0000","last_became_active":"2024-04-25T08:30:31.324899+0000","last_became_peered":"2024-04-25T08:30:31.324899+0000","last_unstale":"2024-04-25T08:30:32.313905+0000","last_undegraded":"2024-04-25T08:30:32.313905+0000","last_fullsized":"2024-04-25T08:30:32.313905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329445+0000","last_change":"2024-04-25T08:30:31.334369+0000","last_active":"2024-04-25T08:30:32.329445+0000","last_peered":"2024-04-25T08:30:32.329445+0000","last_clean":"2024-04-25T08:30:32.329445+0000","last_became_active":"2024-04-25T08:30:31.334071+0000","last_became_peered":"2024-04-25T08:30:31.334071+0000","last_unstale":"2024-04-25T08:30:32.329445+0000","last_undegraded":"2024-04-25T08:30:32.329445+0000","last_fullsized":"2024-04-25T08:30:32.329445+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:36.031710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517034+0000","last_change":"2024-04-25T08:30:31.331292+0000","last_active":"2024-04-25T08:30:32.517034+0000","last_peered":"2024-04-25T08:30:32.517034+0000","last_clean":"2024-04-25T08:30:32.517034+0000","last_became_active":"2024-04-25T08:30:31.330966+0000","last_became_peered":"2024-04-25T08:30:31.330966+0000","last_unstale":"2024-04-25T08:30:32.517034+0000","last_undegraded":"2024-04-25T08:30:32.517034+0000","last_fullsized":"2024-04-25T08:30:32.517034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:41.489831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517719+0000","last_change":"2024-04-25T08:30:31.331334+0000","last_active":"2024-04-25T08:30:32.517719+0000","last_peered":"2024-04-25T08:30:32.517719+0000","last_clean":"2024-04-25T08:30:32.517719+0000","last_became_active":"2024-04-25T08:30:31.331048+0000","last_became_peered":"2024-04-25T08:30:31.331048+0000","last_unstale":"2024-04-25T08:30:32.517719+0000","last_undegraded":"2024-04-25T08:30:32.517719+0000","last_fullsized":"2024-04-25T08:30:32.517719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313434+0000","last_change":"2024-04-25T08:30:31.322980+0000","last_active":"2024-04-25T08:30:32.313434+0000","last_peered":"2024-04-25T08:30:32.313434+0000","last_clean":"2024-04-25T08:30:32.313434+0000","last_became_active":"2024-04-25T08:30:31.322718+0000","last_became_peered":"2024-04-25T08:30:31.322718+0000","last_unstale":"2024-04-25T08:30:32.313434+0000","last_undegraded":"2024-04-25T08:30:32.313434+0000","last_fullsized":"2024-04-25T08:30:32.313434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:11.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327301+0000","last_change":"2024-04-25T08:30:31.319026+0000","last_active":"2024-04-25T08:30:32.327301+0000","last_peered":"2024-04-25T08:30:32.327301+0000","last_clean":"2024-04-25T08:30:32.327301+0000","last_became_active":"2024-04-25T08:30:31.318754+0000","last_became_peered":"2024-04-25T08:30:31.318754+0000","last_unstale":"2024-04-25T08:30:32.327301+0000","last_undegraded":"2024-04-25T08:30:32.327301+0000","last_fullsized":"2024-04-25T08:30:32.327301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:19.829453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313856+0000","last_change":"2024-04-25T08:30:31.321932+0000","last_active":"2024-04-25T08:30:32.313856+0000","last_peered":"2024-04-25T08:30:32.313856+0000","last_clean":"2024-04-25T08:30:32.313856+0000","last_became_active":"2024-04-25T08:30:31.321696+0000","last_became_peered":"2024-04-25T08:30:31.321696+0000","last_unstale":"2024-04-25T08:30:32.313856+0000","last_undegraded":"2024-04-25T08:30:32.313856+0000","last_fullsized":"2024-04-25T08:30:32.313856+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329547+0000","last_change":"2024-04-25T08:30:31.330998+0000","last_active":"2024-04-25T08:30:32.329547+0000","last_peered":"2024-04-25T08:30:32.329547+0000","last_clean":"2024-04-25T08:30:32.329547+0000","last_became_active":"2024-04-25T08:30:31.330539+0000","last_became_peered":"2024-04-25T08:30:31.330539+0000","last_unstale":"2024-04-25T08:30:32.329547+0000","last_undegraded":"2024-04-25T08:30:32.329547+0000","last_fullsized":"2024-04-25T08:30:32.329547+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:24.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313519+0000","last_change":"2024-04-25T08:30:31.327486+0000","last_active":"2024-04-25T08:30:32.313519+0000","last_peered":"2024-04-25T08:30:32.313519+0000","last_clean":"2024-04-25T08:30:32.313519+0000","last_became_active":"2024-04-25T08:30:31.327231+0000","last_became_peered":"2024-04-25T08:30:31.327231+0000","last_unstale":"2024-04-25T08:30:32.313519+0000","last_undegraded":"2024-04-25T08:30:32.313519+0000","last_fullsized":"2024-04-25T08:30:32.313519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517118+0000","last_change":"2024-04-25T08:30:31.320712+0000","last_active":"2024-04-25T08:30:32.517118+0000","last_peered":"2024-04-25T08:30:32.517118+0000","last_clean":"2024-04-25T08:30:32.517118+0000","last_became_active":"2024-04-25T08:30:31.320446+0000","last_became_peered":"2024-04-25T08:30:31.320446+0000","last_unstale":"2024-04-25T08:30:32.517118+0000","last_undegraded":"2024-04-25T08:30:32.517118+0000","last_fullsized":"2024-04-25T08:30:32.517118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:21.292627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327017+0000","last_change":"2024-04-25T08:30:31.315389+0000","last_active":"2024-04-25T08:30:32.327017+0000","last_peered":"2024-04-25T08:30:32.327017+0000","last_clean":"2024-04-25T08:30:32.327017+0000","last_became_active":"2024-04-25T08:30:31.315111+0000","last_became_peered":"2024-04-25T08:30:31.315111+0000","last_unstale":"2024-04-25T08:30:32.327017+0000","last_undegraded":"2024-04-25T08:30:32.327017+0000","last_fullsized":"2024-04-25T08:30:32.327017+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:20.912333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.314073+0000","last_change":"2024-04-25T08:30:31.324771+0000","last_active":"2024-04-25T08:30:32.314073+0000","last_peered":"2024-04-25T08:30:32.314073+0000","last_clean":"2024-04-25T08:30:32.314073+0000","last_became_active":"2024-04-25T08:30:31.324509+0000","last_became_peered":"2024-04-25T08:30:31.324509+0000","last_unstale":"2024-04-25T08:30:32.314073+0000","last_undegraded":"2024-04-25T08:30:32.314073+0000","last_fullsized":"2024-04-25T08:30:32.314073+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517750+0000","last_change":"2024-04-25T08:30:31.333408+0000","last_active":"2024-04-25T08:30:32.517750+0000","last_peered":"2024-04-25T08:30:32.517750+0000","last_clean":"2024-04-25T08:30:32.517750+0000","last_became_active":"2024-04-25T08:30:31.333123+0000","last_became_peered":"2024-04-25T08:30:31.333123+0000","last_unstale":"2024-04-25T08:30:32.517750+0000","last_undegraded":"2024-04-25T08:30:32.517750+0000","last_fullsized":"2024-04-25T08:30:32.517750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.314039+0000","last_change":"2024-04-25T08:30:31.316804+0000","last_active":"2024-04-25T08:30:32.314039+0000","last_peered":"2024-04-25T08:30:32.314039+0000","last_clean":"2024-04-25T08:30:32.314039+0000","last_became_active":"2024-04-25T08:30:31.316503+0000","last_became_peered":"2024-04-25T08:30:31.316503+0000","last_unstale":"2024-04-25T08:30:32.314039+0000","last_undegraded":"2024-04-25T08:30:32.314039+0000","last_fullsized":"2024-04-25T08:30:32.314039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329797+0000","last_change":"2024-04-25T08:30:31.331474+0000","last_active":"2024-04-25T08:30:32.329797+0000","last_peered":"2024-04-25T08:30:32.329797+0000","last_clean":"2024-04-25T08:30:32.329797+0000","last_became_active":"2024-04-25T08:30:31.331058+0000","last_became_peered":"2024-04-25T08:30:31.331058+0000","last_unstale":"2024-04-25T08:30:32.329797+0000","last_undegraded":"2024-04-25T08:30:32.329797+0000","last_fullsized":"2024-04-25T08:30:32.329797+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:58.505644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313671+0000","last_change":"2024-04-25T08:30:31.319410+0000","last_active":"2024-04-25T08:30:32.313671+0000","last_peered":"2024-04-25T08:30:32.313671+0000","last_clean":"2024-04-25T08:30:32.313671+0000","last_became_active":"2024-04-25T08:30:31.318913+0000","last_became_peered":"2024-04-25T08:30:31.318913+0000","last_unstale":"2024-04-25T08:30:32.313671+0000","last_undegraded":"2024-04-25T08:30:32.313671+0000","last_fullsized":"2024-04-25T08:30:32.313671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.070232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313506+0000","last_change":"2024-04-25T08:30:31.318909+0000","last_active":"2024-04-25T08:30:32.313506+0000","last_peered":"2024-04-25T08:30:32.313506+0000","last_clean":"2024-04-25T08:30:32.313506+0000","last_became_active":"2024-04-25T08:30:31.318639+0000","last_became_peered":"2024-04-25T08:30:31.318639+0000","last_unstale":"2024-04-25T08:30:32.313506+0000","last_undegraded":"2024-04-25T08:30:32.313506+0000","last_fullsized":"2024-04-25T08:30:32.313506+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:05.728351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517238+0000","last_change":"2024-04-25T08:30:31.321924+0000","last_active":"2024-04-25T08:30:32.517238+0000","last_peered":"2024-04-25T08:30:32.517238+0000","last_clean":"2024-04-25T08:30:32.517238+0000","last_became_active":"2024-04-25T08:30:31.321434+0000","last_became_peered":"2024-04-25T08:30:31.321434+0000","last_unstale":"2024-04-25T08:30:32.517238+0000","last_undegraded":"2024-04-25T08:30:32.517238+0000","last_fullsized":"2024-04-25T08:30:32.517238+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516925+0000","last_change":"2024-04-25T08:30:31.328203+0000","last_active":"2024-04-25T08:30:32.516925+0000","last_peered":"2024-04-25T08:30:32.516925+0000","last_clean":"2024-04-25T08:30:32.516925+0000","last_became_active":"2024-04-25T08:30:31.327748+0000","last_became_peered":"2024-04-25T08:30:31.327748+0000","last_unstale":"2024-04-25T08:30:32.516925+0000","last_undegraded":"2024-04-25T08:30:32.516925+0000","last_fullsized":"2024-04-25T08:30:32.516925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517374+0000","last_change":"2024-04-25T08:30:31.332758+0000","last_active":"2024-04-25T08:30:32.517374+0000","last_peered":"2024-04-25T08:30:32.517374+0000","last_clean":"2024-04-25T08:30:32.517374+0000","last_became_active":"2024-04-25T08:30:31.332458+0000","last_became_peered":"2024-04-25T08:30:31.332458+0000","last_unstale":"2024-04-25T08:30:32.517374+0000","last_undegraded":"2024-04-25T08:30:32.517374+0000","last_fullsized":"2024-04-25T08:30:32.517374+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:12.355397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.314105+0000","last_change":"2024-04-25T08:30:30.294755+0000","last_active":"2024-04-25T08:30:32.314105+0000","last_peered":"2024-04-25T08:30:32.314105+0000","last_clean":"2024-04-25T08:30:32.314105+0000","last_became_active":"2024-04-25T08:30:30.293812+0000","last_became_peered":"2024-04-25T08:30:30.293812+0000","last_unstale":"2024-04-25T08:30:32.314105+0000","last_undegraded":"2024-04-25T08:30:32.314105+0000","last_fullsized":"2024-04-25T08:30:32.314105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517511+0000","last_change":"2024-04-25T08:30:31.334510+0000","last_active":"2024-04-25T08:30:32.517511+0000","last_peered":"2024-04-25T08:30:32.517511+0000","last_clean":"2024-04-25T08:30:32.517511+0000","last_became_active":"2024-04-25T08:30:31.334224+0000","last_became_peered":"2024-04-25T08:30:31.334224+0000","last_unstale":"2024-04-25T08:30:32.517511+0000","last_undegraded":"2024-04-25T08:30:32.517511+0000","last_fullsized":"2024-04-25T08:30:32.517511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516953+0000","last_change":"2024-04-25T08:30:30.301932+0000","last_active":"2024-04-25T08:30:32.516953+0000","last_peered":"2024-04-25T08:30:32.516953+0000","last_clean":"2024-04-25T08:30:32.516953+0000","last_became_active":"2024-04-25T08:30:30.301633+0000","last_became_peered":"2024-04-25T08:30:30.301633+0000","last_unstale":"2024-04-25T08:30:32.516953+0000","last_undegraded":"2024-04-25T08:30:32.516953+0000","last_fullsized":"2024-04-25T08:30:32.516953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313828+0000","last_change":"2024-04-25T08:30:31.321934+0000","last_active":"2024-04-25T08:30:32.313828+0000","last_peered":"2024-04-25T08:30:32.313828+0000","last_clean":"2024-04-25T08:30:32.313828+0000","last_became_active":"2024-04-25T08:30:31.321537+0000","last_became_peered":"2024-04-25T08:30:31.321537+0000","last_unstale":"2024-04-25T08:30:32.313828+0000","last_undegraded":"2024-04-25T08:30:32.313828+0000","last_fullsized":"2024-04-25T08:30:32.313828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:17.316522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517772+0000","last_change":"2024-04-25T08:30:30.304552+0000","last_active":"2024-04-25T08:30:32.517772+0000","last_peered":"2024-04-25T08:30:32.517772+0000","last_clean":"2024-04-25T08:30:32.517772+0000","last_became_active":"2024-04-25T08:30:30.304168+0000","last_became_peered":"2024-04-25T08:30:30.304168+0000","last_unstale":"2024-04-25T08:30:32.517772+0000","last_undegraded":"2024-04-25T08:30:32.517772+0000","last_fullsized":"2024-04-25T08:30:32.517772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:13.711704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312716+0000","last_change":"2024-04-25T08:30:31.327204+0000","last_active":"2024-04-25T08:30:32.312716+0000","last_peered":"2024-04-25T08:30:32.312716+0000","last_clean":"2024-04-25T08:30:32.312716+0000","last_became_active":"2024-04-25T08:30:31.326149+0000","last_became_peered":"2024-04-25T08:30:31.326149+0000","last_unstale":"2024-04-25T08:30:32.312716+0000","last_undegraded":"2024-04-25T08:30:32.312716+0000","last_fullsized":"2024-04-25T08:30:32.312716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:24.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313838+0000","last_change":"2024-04-25T08:30:30.301217+0000","last_active":"2024-04-25T08:30:32.313838+0000","last_peered":"2024-04-25T08:30:32.313838+0000","last_clean":"2024-04-25T08:30:32.313838+0000","last_became_active":"2024-04-25T08:30:30.300899+0000","last_became_peered":"2024-04-25T08:30:30.300899+0000","last_unstale":"2024-04-25T08:30:32.313838+0000","last_undegraded":"2024-04-25T08:30:32.313838+0000","last_fullsized":"2024-04-25T08:30:32.313838+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:49.622484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516679+0000","last_change":"2024-04-25T08:30:31.322185+0000","last_active":"2024-04-25T08:30:32.516679+0000","last_peered":"2024-04-25T08:30:32.516679+0000","last_clean":"2024-04-25T08:30:32.516679+0000","last_became_active":"2024-04-25T08:30:31.321488+0000","last_became_peered":"2024-04-25T08:30:31.321488+0000","last_unstale":"2024-04-25T08:30:32.516679+0000","last_undegraded":"2024-04-25T08:30:32.516679+0000","last_fullsized":"2024-04-25T08:30:32.516679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329683+0000","last_change":"2024-04-25T08:30:30.302842+0000","last_active":"2024-04-25T08:30:32.329683+0000","last_peered":"2024-04-25T08:30:32.329683+0000","last_clean":"2024-04-25T08:30:32.329683+0000","last_became_active":"2024-04-25T08:30:30.302540+0000","last_became_peered":"2024-04-25T08:30:30.302540+0000","last_unstale":"2024-04-25T08:30:32.329683+0000","last_undegraded":"2024-04-25T08:30:32.329683+0000","last_fullsized":"2024-04-25T08:30:32.329683+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.946206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329749+0000","last_change":"2024-04-25T08:30:31.330857+0000","last_active":"2024-04-25T08:30:32.329749+0000","last_peered":"2024-04-25T08:30:32.329749+0000","last_clean":"2024-04-25T08:30:32.329749+0000","last_became_active":"2024-04-25T08:30:31.329563+0000","last_became_peered":"2024-04-25T08:30:31.329563+0000","last_unstale":"2024-04-25T08:30:32.329749+0000","last_undegraded":"2024-04-25T08:30:32.329749+0000","last_fullsized":"2024-04-25T08:30:32.329749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.173287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517091+0000","last_change":"2024-04-25T08:30:30.298067+0000","last_active":"2024-04-25T08:30:32.517091+0000","last_peered":"2024-04-25T08:30:32.517091+0000","last_clean":"2024-04-25T08:30:32.517091+0000","last_became_active":"2024-04-25T08:30:30.297718+0000","last_became_peered":"2024-04-25T08:30:30.297718+0000","last_unstale":"2024-04-25T08:30:32.517091+0000","last_undegraded":"2024-04-25T08:30:32.517091+0000","last_fullsized":"2024-04-25T08:30:32.517091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:31.588336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313687+0000","last_change":"2024-04-25T08:30:31.335069+0000","last_active":"2024-04-25T08:30:32.313687+0000","last_peered":"2024-04-25T08:30:32.313687+0000","last_clean":"2024-04-25T08:30:32.313687+0000","last_became_active":"2024-04-25T08:30:31.333204+0000","last_became_peered":"2024-04-25T08:30:31.333204+0000","last_unstale":"2024-04-25T08:30:32.313687+0000","last_undegraded":"2024-04-25T08:30:32.313687+0000","last_fullsized":"2024-04-25T08:30:32.313687+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517266+0000","last_change":"2024-04-25T08:30:30.305406+0000","last_active":"2024-04-25T08:30:32.517266+0000","last_peered":"2024-04-25T08:30:32.517266+0000","last_clean":"2024-04-25T08:30:32.517266+0000","last_became_active":"2024-04-25T08:30:30.305041+0000","last_became_peered":"2024-04-25T08:30:30.305041+0000","last_unstale":"2024-04-25T08:30:32.517266+0000","last_undegraded":"2024-04-25T08:30:32.517266+0000","last_fullsized":"2024-04-25T08:30:32.517266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:42.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312379+0000","last_change":"2024-04-25T08:30:31.327750+0000","last_active":"2024-04-25T08:30:32.312379+0000","last_peered":"2024-04-25T08:30:32.312379+0000","last_clean":"2024-04-25T08:30:32.312379+0000","last_became_active":"2024-04-25T08:30:31.327450+0000","last_became_peered":"2024-04-25T08:30:31.327450+0000","last_unstale":"2024-04-25T08:30:32.312379+0000","last_undegraded":"2024-04-25T08:30:32.312379+0000","last_fullsized":"2024-04-25T08:30:32.312379+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.788878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313025+0000","last_change":"2024-04-25T08:30:30.290419+0000","last_active":"2024-04-25T08:30:32.313025+0000","last_peered":"2024-04-25T08:30:32.313025+0000","last_clean":"2024-04-25T08:30:32.313025+0000","last_became_active":"2024-04-25T08:30:30.290143+0000","last_became_peered":"2024-04-25T08:30:30.290143+0000","last_unstale":"2024-04-25T08:30:32.313025+0000","last_undegraded":"2024-04-25T08:30:32.313025+0000","last_fullsized":"2024-04-25T08:30:32.313025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:42.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327338+0000","last_change":"2024-04-25T08:30:31.326464+0000","last_active":"2024-04-25T08:30:32.327338+0000","last_peered":"2024-04-25T08:30:32.327338+0000","last_clean":"2024-04-25T08:30:32.327338+0000","last_became_active":"2024-04-25T08:30:31.325229+0000","last_became_peered":"2024-04-25T08:30:31.325229+0000","last_unstale":"2024-04-25T08:30:32.327338+0000","last_undegraded":"2024-04-25T08:30:32.327338+0000","last_fullsized":"2024-04-25T08:30:32.327338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:37.281007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312810+0000","last_change":"2024-04-25T08:30:30.302163+0000","last_active":"2024-04-25T08:30:32.312810+0000","last_peered":"2024-04-25T08:30:32.312810+0000","last_clean":"2024-04-25T08:30:32.312810+0000","last_became_active":"2024-04-25T08:30:30.301848+0000","last_became_peered":"2024-04-25T08:30:30.301848+0000","last_unstale":"2024-04-25T08:30:32.312810+0000","last_undegraded":"2024-04-25T08:30:32.312810+0000","last_fullsized":"2024-04-25T08:30:32.312810+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:43.946206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517387+0000","last_change":"2024-04-25T08:30:31.319799+0000","last_active":"2024-04-25T08:30:32.517387+0000","last_peered":"2024-04-25T08:30:32.517387+0000","last_clean":"2024-04-25T08:30:32.517387+0000","last_became_active":"2024-04-25T08:30:31.319152+0000","last_became_peered":"2024-04-25T08:30:31.319152+0000","last_unstale":"2024-04-25T08:30:32.517387+0000","last_undegraded":"2024-04-25T08:30:32.517387+0000","last_fullsized":"2024-04-25T08:30:32.517387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:47.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329975+0000","last_change":"2024-04-25T08:30:30.299038+0000","last_active":"2024-04-25T08:30:32.329975+0000","last_peered":"2024-04-25T08:30:32.329975+0000","last_clean":"2024-04-25T08:30:32.329975+0000","last_became_active":"2024-04-25T08:30:30.298782+0000","last_became_peered":"2024-04-25T08:30:30.298782+0000","last_unstale":"2024-04-25T08:30:32.329975+0000","last_undegraded":"2024-04-25T08:30:32.329975+0000","last_fullsized":"2024-04-25T08:30:32.329975+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:47.771125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516892+0000","last_change":"2024-04-25T08:30:31.327455+0000","last_active":"2024-04-25T08:30:32.516892+0000","last_peered":"2024-04-25T08:30:32.516892+0000","last_clean":"2024-04-25T08:30:32.516892+0000","last_became_active":"2024-04-25T08:30:31.327181+0000","last_became_peered":"2024-04-25T08:30:31.327181+0000","last_unstale":"2024-04-25T08:30:32.516892+0000","last_undegraded":"2024-04-25T08:30:32.516892+0000","last_fullsized":"2024-04-25T08:30:32.516892+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:44.275363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329954+0000","last_change":"2024-04-25T08:30:30.301686+0000","last_active":"2024-04-25T08:30:32.329954+0000","last_peered":"2024-04-25T08:30:32.329954+0000","last_clean":"2024-04-25T08:30:32.329954+0000","last_became_active":"2024-04-25T08:30:30.301391+0000","last_became_peered":"2024-04-25T08:30:30.301391+0000","last_unstale":"2024-04-25T08:30:32.329954+0000","last_undegraded":"2024-04-25T08:30:32.329954+0000","last_fullsized":"2024-04-25T08:30:32.329954+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:21.852169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313221+0000","last_change":"2024-04-25T08:30:31.325786+0000","last_active":"2024-04-25T08:30:32.313221+0000","last_peered":"2024-04-25T08:30:32.313221+0000","last_clean":"2024-04-25T08:30:32.313221+0000","last_became_active":"2024-04-25T08:30:31.325528+0000","last_became_peered":"2024-04-25T08:30:31.325528+0000","last_unstale":"2024-04-25T08:30:32.313221+0000","last_undegraded":"2024-04-25T08:30:32.313221+0000","last_fullsized":"2024-04-25T08:30:32.313221+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:44.948047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312486+0000","last_change":"2024-04-25T08:30:30.303220+0000","last_active":"2024-04-25T08:30:32.312486+0000","last_peered":"2024-04-25T08:30:32.312486+0000","last_clean":"2024-04-25T08:30:32.312486+0000","last_became_active":"2024-04-25T08:30:30.302914+0000","last_became_peered":"2024-04-25T08:30:30.302914+0000","last_unstale":"2024-04-25T08:30:32.312486+0000","last_undegraded":"2024-04-25T08:30:32.312486+0000","last_fullsized":"2024-04-25T08:30:32.312486+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:47.771125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516679+0000","last_change":"2024-04-25T08:30:31.331291+0000","last_active":"2024-04-25T08:30:32.516679+0000","last_peered":"2024-04-25T08:30:32.516679+0000","last_clean":"2024-04-25T08:30:32.516679+0000","last_became_active":"2024-04-25T08:30:31.330930+0000","last_became_peered":"2024-04-25T08:30:31.330930+0000","last_unstale":"2024-04-25T08:30:32.516679+0000","last_undegraded":"2024-04-25T08:30:32.516679+0000","last_fullsized":"2024-04-25T08:30:32.516679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:33.070232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312586+0000","last_change":"2024-04-25T08:30:30.301164+0000","last_active":"2024-04-25T08:30:32.312586+0000","last_peered":"2024-04-25T08:30:32.312586+0000","last_clean":"2024-04-25T08:30:32.312586+0000","last_became_active":"2024-04-25T08:30:30.300887+0000","last_became_peered":"2024-04-25T08:30:30.300887+0000","last_unstale":"2024-04-25T08:30:32.312586+0000","last_undegraded":"2024-04-25T08:30:32.312586+0000","last_fullsized":"2024-04-25T08:30:32.312586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:04.726510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312280+0000","last_change":"2024-04-25T08:30:31.329294+0000","last_active":"2024-04-25T08:30:32.312280+0000","last_peered":"2024-04-25T08:30:32.312280+0000","last_clean":"2024-04-25T08:30:32.312280+0000","last_became_active":"2024-04-25T08:30:31.328999+0000","last_became_peered":"2024-04-25T08:30:31.328999+0000","last_unstale":"2024-04-25T08:30:32.312280+0000","last_undegraded":"2024-04-25T08:30:32.312280+0000","last_fullsized":"2024-04-25T08:30:32.312280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:47.159559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517520+0000","last_change":"2024-04-25T08:30:30.304789+0000","last_active":"2024-04-25T08:30:32.517520+0000","last_peered":"2024-04-25T08:30:32.517520+0000","last_clean":"2024-04-25T08:30:32.517520+0000","last_became_active":"2024-04-25T08:30:30.304493+0000","last_became_peered":"2024-04-25T08:30:30.304493+0000","last_unstale":"2024-04-25T08:30:32.517520+0000","last_undegraded":"2024-04-25T08:30:32.517520+0000","last_fullsized":"2024-04-25T08:30:32.517520+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:49.622484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517222+0000","last_change":"2024-04-25T08:30:31.326518+0000","last_active":"2024-04-25T08:30:32.517222+0000","last_peered":"2024-04-25T08:30:32.517222+0000","last_clean":"2024-04-25T08:30:32.517222+0000","last_became_active":"2024-04-25T08:30:31.326241+0000","last_became_peered":"2024-04-25T08:30:31.326241+0000","last_unstale":"2024-04-25T08:30:32.517222+0000","last_undegraded":"2024-04-25T08:30:32.517222+0000","last_fullsized":"2024-04-25T08:30:32.517222+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:43.081894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313507+0000","last_change":"2024-04-25T08:30:30.294767+0000","last_active":"2024-04-25T08:30:32.313507+0000","last_peered":"2024-04-25T08:30:32.313507+0000","last_clean":"2024-04-25T08:30:32.313507+0000","last_became_active":"2024-04-25T08:30:30.294475+0000","last_became_peered":"2024-04-25T08:30:30.294475+0000","last_unstale":"2024-04-25T08:30:32.313507+0000","last_undegraded":"2024-04-25T08:30:32.313507+0000","last_fullsized":"2024-04-25T08:30:32.313507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.513786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312464+0000","last_change":"2024-04-25T08:30:31.320322+0000","last_active":"2024-04-25T08:30:32.312464+0000","last_peered":"2024-04-25T08:30:32.312464+0000","last_clean":"2024-04-25T08:30:32.312464+0000","last_became_active":"2024-04-25T08:30:31.319997+0000","last_became_peered":"2024-04-25T08:30:31.319997+0000","last_unstale":"2024-04-25T08:30:32.312464+0000","last_undegraded":"2024-04-25T08:30:32.312464+0000","last_fullsized":"2024-04-25T08:30:32.312464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:05.804396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517022+0000","last_change":"2024-04-25T08:30:30.297207+0000","last_active":"2024-04-25T08:30:32.517022+0000","last_peered":"2024-04-25T08:30:32.517022+0000","last_clean":"2024-04-25T08:30:32.517022+0000","last_became_active":"2024-04-25T08:30:30.296922+0000","last_became_peered":"2024-04-25T08:30:30.296922+0000","last_unstale":"2024-04-25T08:30:32.517022+0000","last_undegraded":"2024-04-25T08:30:32.517022+0000","last_fullsized":"2024-04-25T08:30:32.517022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:13.711704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312438+0000","last_change":"2024-04-25T08:30:31.326706+0000","last_active":"2024-04-25T08:30:32.312438+0000","last_peered":"2024-04-25T08:30:32.312438+0000","last_clean":"2024-04-25T08:30:32.312438+0000","last_became_active":"2024-04-25T08:30:31.326417+0000","last_became_peered":"2024-04-25T08:30:31.326417+0000","last_unstale":"2024-04-25T08:30:32.312438+0000","last_undegraded":"2024-04-25T08:30:32.312438+0000","last_fullsized":"2024-04-25T08:30:32.312438+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:37.793977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329802+0000","last_change":"2024-04-25T08:30:30.297702+0000","last_active":"2024-04-25T08:30:32.329802+0000","last_peered":"2024-04-25T08:30:32.329802+0000","last_clean":"2024-04-25T08:30:32.329802+0000","last_became_active":"2024-04-25T08:30:30.297420+0000","last_became_peered":"2024-04-25T08:30:30.297420+0000","last_unstale":"2024-04-25T08:30:32.329802+0000","last_undegraded":"2024-04-25T08:30:32.329802+0000","last_fullsized":"2024-04-25T08:30:32.329802+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:48.137643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329845+0000","last_change":"2024-04-25T08:30:31.337230+0000","last_active":"2024-04-25T08:30:32.329845+0000","last_peered":"2024-04-25T08:30:32.329845+0000","last_clean":"2024-04-25T08:30:32.329845+0000","last_became_active":"2024-04-25T08:30:31.336785+0000","last_became_peered":"2024-04-25T08:30:31.336785+0000","last_unstale":"2024-04-25T08:30:32.329845+0000","last_undegraded":"2024-04-25T08:30:32.329845+0000","last_fullsized":"2024-04-25T08:30:32.329845+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:37.289322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313812+0000","last_change":"2024-04-25T08:30:30.296836+0000","last_active":"2024-04-25T08:30:32.313812+0000","last_peered":"2024-04-25T08:30:32.313812+0000","last_clean":"2024-04-25T08:30:32.313812+0000","last_became_active":"2024-04-25T08:30:30.296490+0000","last_became_peered":"2024-04-25T08:30:30.296490+0000","last_unstale":"2024-04-25T08:30:32.313812+0000","last_undegraded":"2024-04-25T08:30:32.313812+0000","last_fullsized":"2024-04-25T08:30:32.313812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313437+0000","last_change":"2024-04-25T08:30:31.326115+0000","last_active":"2024-04-25T08:30:32.313437+0000","last_peered":"2024-04-25T08:30:32.313437+0000","last_clean":"2024-04-25T08:30:32.313437+0000","last_became_active":"2024-04-25T08:30:31.324502+0000","last_became_peered":"2024-04-25T08:30:31.324502+0000","last_unstale":"2024-04-25T08:30:32.313437+0000","last_undegraded":"2024-04-25T08:30:32.313437+0000","last_fullsized":"2024-04-25T08:30:32.313437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327295+0000","last_change":"2024-04-25T08:30:30.297703+0000","last_active":"2024-04-25T08:30:32.327295+0000","last_peered":"2024-04-25T08:30:32.327295+0000","last_clean":"2024-04-25T08:30:32.327295+0000","last_became_active":"2024-04-25T08:30:30.297424+0000","last_became_peered":"2024-04-25T08:30:30.297424+0000","last_unstale":"2024-04-25T08:30:32.327295+0000","last_undegraded":"2024-04-25T08:30:32.327295+0000","last_fullsized":"2024-04-25T08:30:32.327295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:26.490521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327446+0000","last_change":"2024-04-25T08:30:31.319156+0000","last_active":"2024-04-25T08:30:32.327446+0000","last_peered":"2024-04-25T08:30:32.327446+0000","last_clean":"2024-04-25T08:30:32.327446+0000","last_became_active":"2024-04-25T08:30:31.318880+0000","last_became_peered":"2024-04-25T08:30:31.318880+0000","last_unstale":"2024-04-25T08:30:32.327446+0000","last_undegraded":"2024-04-25T08:30:32.327446+0000","last_fullsized":"2024-04-25T08:30:32.327446+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:19.675135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517237+0000","last_change":"2024-04-25T08:30:30.303603+0000","last_active":"2024-04-25T08:30:32.517237+0000","last_peered":"2024-04-25T08:30:32.517237+0000","last_clean":"2024-04-25T08:30:32.517237+0000","last_became_active":"2024-04-25T08:30:30.303322+0000","last_became_peered":"2024-04-25T08:30:30.303322+0000","last_unstale":"2024-04-25T08:30:32.517237+0000","last_undegraded":"2024-04-25T08:30:32.517237+0000","last_fullsized":"2024-04-25T08:30:32.517237+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:18.673294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517371+0000","last_change":"2024-04-25T08:30:31.325746+0000","last_active":"2024-04-25T08:30:32.517371+0000","last_peered":"2024-04-25T08:30:32.517371+0000","last_clean":"2024-04-25T08:30:32.517371+0000","last_became_active":"2024-04-25T08:30:31.325449+0000","last_became_peered":"2024-04-25T08:30:31.325449+0000","last_unstale":"2024-04-25T08:30:32.517371+0000","last_undegraded":"2024-04-25T08:30:32.517371+0000","last_fullsized":"2024-04-25T08:30:32.517371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:23.453764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327124+0000","last_change":"2024-04-25T08:30:30.293002+0000","last_active":"2024-04-25T08:30:32.327124+0000","last_peered":"2024-04-25T08:30:32.327124+0000","last_clean":"2024-04-25T08:30:32.327124+0000","last_became_active":"2024-04-25T08:30:30.292702+0000","last_became_peered":"2024-04-25T08:30:30.292702+0000","last_unstale":"2024-04-25T08:30:32.327124+0000","last_undegraded":"2024-04-25T08:30:32.327124+0000","last_fullsized":"2024-04-25T08:30:32.327124+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:03.357097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516794+0000","last_change":"2024-04-25T08:30:31.323250+0000","last_active":"2024-04-25T08:30:32.516794+0000","last_peered":"2024-04-25T08:30:32.516794+0000","last_clean":"2024-04-25T08:30:32.516794+0000","last_became_active":"2024-04-25T08:30:31.322945+0000","last_became_peered":"2024-04-25T08:30:31.322945+0000","last_unstale":"2024-04-25T08:30:32.516794+0000","last_undegraded":"2024-04-25T08:30:32.516794+0000","last_fullsized":"2024-04-25T08:30:32.516794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:22.854010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327077+0000","last_change":"2024-04-25T08:30:30.292431+0000","last_active":"2024-04-25T08:30:32.327077+0000","last_peered":"2024-04-25T08:30:32.327077+0000","last_clean":"2024-04-25T08:30:32.327077+0000","last_became_active":"2024-04-25T08:30:30.292078+0000","last_became_peered":"2024-04-25T08:30:30.292078+0000","last_unstale":"2024-04-25T08:30:32.327077+0000","last_undegraded":"2024-04-25T08:30:32.327077+0000","last_fullsized":"2024-04-25T08:30:32.327077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:57.453111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329578+0000","last_change":"2024-04-25T08:30:31.331176+0000","last_active":"2024-04-25T08:30:32.329578+0000","last_peered":"2024-04-25T08:30:32.329578+0000","last_clean":"2024-04-25T08:30:32.329578+0000","last_became_active":"2024-04-25T08:30:31.330823+0000","last_became_peered":"2024-04-25T08:30:31.330823+0000","last_unstale":"2024-04-25T08:30:32.329578+0000","last_undegraded":"2024-04-25T08:30:32.329578+0000","last_fullsized":"2024-04-25T08:30:32.329578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:49.483225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517240+0000","last_change":"2024-04-25T08:30:30.299966+0000","last_active":"2024-04-25T08:30:32.517240+0000","last_peered":"2024-04-25T08:30:32.517240+0000","last_clean":"2024-04-25T08:30:32.517240+0000","last_became_active":"2024-04-25T08:30:30.299699+0000","last_became_peered":"2024-04-25T08:30:30.299699+0000","last_unstale":"2024-04-25T08:30:32.517240+0000","last_undegraded":"2024-04-25T08:30:32.517240+0000","last_fullsized":"2024-04-25T08:30:32.517240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.411456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312511+0000","last_change":"2024-04-25T08:30:31.327023+0000","last_active":"2024-04-25T08:30:32.312511+0000","last_peered":"2024-04-25T08:30:32.312511+0000","last_clean":"2024-04-25T08:30:32.312511+0000","last_became_active":"2024-04-25T08:30:31.326729+0000","last_became_peered":"2024-04-25T08:30:31.326729+0000","last_unstale":"2024-04-25T08:30:32.312511+0000","last_undegraded":"2024-04-25T08:30:32.312511+0000","last_fullsized":"2024-04-25T08:30:32.312511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:36.031710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.326958+0000","last_change":"2024-04-25T08:30:30.293467+0000","last_active":"2024-04-25T08:30:32.326958+0000","last_peered":"2024-04-25T08:30:32.326958+0000","last_clean":"2024-04-25T08:30:32.326958+0000","last_became_active":"2024-04-25T08:30:30.293179+0000","last_became_peered":"2024-04-25T08:30:30.293179+0000","last_unstale":"2024-04-25T08:30:32.326958+0000","last_undegraded":"2024-04-25T08:30:32.326958+0000","last_fullsized":"2024-04-25T08:30:32.326958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:28.603390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517145+0000","last_change":"2024-04-25T08:30:31.333948+0000","last_active":"2024-04-25T08:30:32.517145+0000","last_peered":"2024-04-25T08:30:32.517145+0000","last_clean":"2024-04-25T08:30:32.517145+0000","last_became_active":"2024-04-25T08:30:31.333521+0000","last_became_peered":"2024-04-25T08:30:31.333521+0000","last_unstale":"2024-04-25T08:30:32.517145+0000","last_undegraded":"2024-04-25T08:30:32.517145+0000","last_fullsized":"2024-04-25T08:30:32.517145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329939+0000","last_change":"2024-04-25T08:30:30.301760+0000","last_active":"2024-04-25T08:30:32.329939+0000","last_peered":"2024-04-25T08:30:32.329939+0000","last_clean":"2024-04-25T08:30:32.329939+0000","last_became_active":"2024-04-25T08:30:30.301331+0000","last_became_peered":"2024-04-25T08:30:30.301331+0000","last_unstale":"2024-04-25T08:30:32.329939+0000","last_undegraded":"2024-04-25T08:30:32.329939+0000","last_fullsized":"2024-04-25T08:30:32.329939+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:43.273522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517526+0000","last_change":"2024-04-25T08:30:31.320048+0000","last_active":"2024-04-25T08:30:32.517526+0000","last_peered":"2024-04-25T08:30:32.517526+0000","last_clean":"2024-04-25T08:30:32.517526+0000","last_became_active":"2024-04-25T08:30:31.319495+0000","last_became_peered":"2024-04-25T08:30:31.319495+0000","last_unstale":"2024-04-25T08:30:32.517526+0000","last_undegraded":"2024-04-25T08:30:32.517526+0000","last_fullsized":"2024-04-25T08:30:32.517526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:28.421465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.327332+0000","last_change":"2024-04-25T08:30:30.296903+0000","last_active":"2024-04-25T08:30:32.327332+0000","last_peered":"2024-04-25T08:30:32.327332+0000","last_clean":"2024-04-25T08:30:32.327332+0000","last_became_active":"2024-04-25T08:30:30.296623+0000","last_became_peered":"2024-04-25T08:30:30.296623+0000","last_unstale":"2024-04-25T08:30:32.327332+0000","last_undegraded":"2024-04-25T08:30:32.327332+0000","last_fullsized":"2024-04-25T08:30:32.327332+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:27.714670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312527+0000","last_change":"2024-04-25T08:30:31.323886+0000","last_active":"2024-04-25T08:30:32.312527+0000","last_peered":"2024-04-25T08:30:32.312527+0000","last_clean":"2024-04-25T08:30:32.312527+0000","last_became_active":"2024-04-25T08:30:31.323603+0000","last_became_peered":"2024-04-25T08:30:31.323603+0000","last_unstale":"2024-04-25T08:30:32.312527+0000","last_undegraded":"2024-04-25T08:30:32.312527+0000","last_fullsized":"2024-04-25T08:30:32.312527+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:12.807489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516845+0000","last_change":"2024-04-25T08:30:30.303229+0000","last_active":"2024-04-25T08:30:32.516845+0000","last_peered":"2024-04-25T08:30:32.516845+0000","last_clean":"2024-04-25T08:30:32.516845+0000","last_became_active":"2024-04-25T08:30:30.302894+0000","last_became_peered":"2024-04-25T08:30:30.302894+0000","last_unstale":"2024-04-25T08:30:32.516845+0000","last_undegraded":"2024-04-25T08:30:32.516845+0000","last_fullsized":"2024-04-25T08:30:32.516845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.516857+0000","last_change":"2024-04-25T08:30:31.322460+0000","last_active":"2024-04-25T08:30:32.516857+0000","last_peered":"2024-04-25T08:30:32.516857+0000","last_clean":"2024-04-25T08:30:32.516857+0000","last_became_active":"2024-04-25T08:30:31.321874+0000","last_became_peered":"2024-04-25T08:30:31.321874+0000","last_unstale":"2024-04-25T08:30:32.516857+0000","last_undegraded":"2024-04-25T08:30:32.516857+0000","last_fullsized":"2024-04-25T08:30:32.516857+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517101+0000","last_change":"2024-04-25T08:30:30.300280+0000","last_active":"2024-04-25T08:30:32.517101+0000","last_peered":"2024-04-25T08:30:32.517101+0000","last_clean":"2024-04-25T08:30:32.517101+0000","last_became_active":"2024-04-25T08:30:30.299985+0000","last_became_peered":"2024-04-25T08:30:30.299985+0000","last_unstale":"2024-04-25T08:30:32.517101+0000","last_undegraded":"2024-04-25T08:30:32.517101+0000","last_fullsized":"2024-04-25T08:30:32.517101+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:42.727568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517170+0000","last_change":"2024-04-25T08:30:31.320823+0000","last_active":"2024-04-25T08:30:32.517170+0000","last_peered":"2024-04-25T08:30:32.517170+0000","last_clean":"2024-04-25T08:30:32.517170+0000","last_became_active":"2024-04-25T08:30:31.320165+0000","last_became_peered":"2024-04-25T08:30:31.320165+0000","last_unstale":"2024-04-25T08:30:32.517170+0000","last_undegraded":"2024-04-25T08:30:32.517170+0000","last_fullsized":"2024-04-25T08:30:32.517170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.139485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313600+0000","last_change":"2024-04-25T08:30:30.293212+0000","last_active":"2024-04-25T08:30:32.313600+0000","last_peered":"2024-04-25T08:30:32.313600+0000","last_clean":"2024-04-25T08:30:32.313600+0000","last_became_active":"2024-04-25T08:30:30.292931+0000","last_became_peered":"2024-04-25T08:30:30.292931+0000","last_unstale":"2024-04-25T08:30:32.313600+0000","last_undegraded":"2024-04-25T08:30:32.313600+0000","last_fullsized":"2024-04-25T08:30:32.313600+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313256+0000","last_change":"2024-04-25T08:30:31.323017+0000","last_active":"2024-04-25T08:30:32.313256+0000","last_peered":"2024-04-25T08:30:32.313256+0000","last_clean":"2024-04-25T08:30:32.313256+0000","last_became_active":"2024-04-25T08:30:31.322734+0000","last_became_peered":"2024-04-25T08:30:31.322734+0000","last_unstale":"2024-04-25T08:30:32.313256+0000","last_undegraded":"2024-04-25T08:30:32.313256+0000","last_fullsized":"2024-04-25T08:30:32.313256+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:48.772966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.313003+0000","last_change":"2024-04-25T08:30:30.298769+0000","last_active":"2024-04-25T08:30:32.313003+0000","last_peered":"2024-04-25T08:30:32.313003+0000","last_clean":"2024-04-25T08:30:32.313003+0000","last_became_active":"2024-04-25T08:30:30.298489+0000","last_became_peered":"2024-04-25T08:30:30.298489+0000","last_unstale":"2024-04-25T08:30:32.313003+0000","last_undegraded":"2024-04-25T08:30:32.313003+0000","last_fullsized":"2024-04-25T08:30:32.313003+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:36.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517518+0000","last_change":"2024-04-25T08:30:31.318757+0000","last_active":"2024-04-25T08:30:32.517518+0000","last_peered":"2024-04-25T08:30:32.517518+0000","last_clean":"2024-04-25T08:30:32.517518+0000","last_became_active":"2024-04-25T08:30:31.318482+0000","last_became_peered":"2024-04-25T08:30:31.318482+0000","last_unstale":"2024-04-25T08:30:32.517518+0000","last_undegraded":"2024-04-25T08:30:32.517518+0000","last_fullsized":"2024-04-25T08:30:32.517518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:45.788878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.312297+0000","last_change":"2024-04-25T08:30:30.295325+0000","last_active":"2024-04-25T08:30:32.312297+0000","last_peered":"2024-04-25T08:30:32.312297+0000","last_clean":"2024-04-25T08:30:32.312297+0000","last_became_active":"2024-04-25T08:30:30.295030+0000","last_became_peered":"2024-04-25T08:30:30.295030+0000","last_unstale":"2024-04-25T08:30:32.312297+0000","last_undegraded":"2024-04-25T08:30:32.312297+0000","last_fullsized":"2024-04-25T08:30:32.312297+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:32.068390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.329735+0000","last_change":"2024-04-25T08:30:31.340453+0000","last_active":"2024-04-25T08:30:32.329735+0000","last_peered":"2024-04-25T08:30:32.329735+0000","last_clean":"2024-04-25T08:30:32.329735+0000","last_became_active":"2024-04-25T08:30:31.340188+0000","last_became_peered":"2024-04-25T08:30:31.340188+0000","last_unstale":"2024-04-25T08:30:32.329735+0000","last_undegraded":"2024-04-25T08:30:32.329735+0000","last_fullsized":"2024-04-25T08:30:32.329735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:30.277894+0000","last_clean_scrub_stamp":"2024-04-25T08:30:30.277894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:33.153920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517395+0000","last_change":"2024-04-25T08:30:30.300697+0000","last_active":"2024-04-25T08:30:32.517395+0000","last_peered":"2024-04-25T08:30:32.517395+0000","last_clean":"2024-04-25T08:30:32.517395+0000","last_became_active":"2024-04-25T08:30:30.300410+0000","last_became_peered":"2024-04-25T08:30:30.300410+0000","last_unstale":"2024-04-25T08:30:32.517395+0000","last_undegraded":"2024-04-25T08:30:32.517395+0000","last_fullsized":"2024-04-25T08:30:32.517395+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:30:29.276053+0000","last_clean_scrub_stamp":"2024-04-25T08:30:29.276053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:38.298492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:30:32.517504+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:30:32.517504+0000","last_peered":"2024-04-25T08:30:32.517504+0000","last_clean":"2024-04-25T08:30:32.517504+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:30:32.517504+0000","last_undegraded":"2024-04-25T08:30:32.517504+0000","last_fullsized":"2024-04-25T08:30:32.517504+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1056768,"data_stored":785909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":194441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":193971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":194463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":462848,"data_stored":195505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":450560,"data_stored":193929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":450560,"data_stored":193949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":784261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:30:39.051 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:30:39.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:30:39.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.199+0000 7f5d3988f640 1 Processor -- start 2024-04-25T08:30:39.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.199+0000 7f5d3988f640 1 -- start start 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d3988f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5d340ced80 0x7f5d340d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d3988f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d34161710 0x7f5d340d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d3988f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3415e000 con 0x7f5d3415dc20 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d3988f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d340d1de0 con 0x7f5d340ced80 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d3988f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34161c60 con 0x7f5d34161710 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d327fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5d340ced80 0x7f5d340d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d32ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5d3415dc20 0x7f5d340ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53522/0 (socket says 172.21.15.60:53522) 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d32ffd640 1 -- 172.21.15.60:0/4126261569 learned_addr learned my addr 172.21.15.60:0/4126261569 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:39.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d337fe640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d34161710 0x7f5d340d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d327fc640 1 -- 172.21.15.60:0/4126261569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d34161710 msgr2=0x7f5d340d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d327fc640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d34161710 0x7f5d340d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.200+0000 7f5d327fc640 1 -- 172.21.15.60:0/4126261569 >> v1:172.21.15.60:6789/0 conn(0x7f5d3415dc20 legacy=0x7f5d340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d31ffb640 1 -- 172.21.15.60:0/4126261569 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103465917 0 0) 0x7f5d3415e000 con 0x7f5d3415dc20 2024-04-25T08:30:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d327fc640 1 -- 172.21.15.60:0/4126261569 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34161f40 con 0x7f5d340ced80 2024-04-25T08:30:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d337fe640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d34161710 0x7f5d340d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:39.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d327fc640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d340d22e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5d240097a0 tx=0x7f5d2402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f9e41161f13ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:39.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d31ffb640 1 -- 172.21.15.60:0/4126261569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24037070 con 0x7f5d340ced80 2024-04-25T08:30:39.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d31ffb640 1 -- 172.21.15.60:0/4126261569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d24032440 con 0x7f5d340ced80 2024-04-25T08:30:39.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.201+0000 7f5d31ffb640 1 -- 172.21.15.60:0/4126261569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24032810 con 0x7f5d340ced80 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 -- 172.21.15.60:0/4126261569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 msgr2=0x7f5d340d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d340d22e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5d240097a0 tx=0x7f5d2402a180 comp rx=0 tx=0).stop 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 -- 172.21.15.60:0/4126261569 shutdown_connections 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d34161710 0x7f5d340d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 --2- 172.21.15.60:0/4126261569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d340d22e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 -- 172.21.15.60:0/4126261569 >> 172.21.15.60:0/4126261569 conn(0x7f5d340d0420 msgr2=0x7f5d340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 -- 172.21.15.60:0/4126261569 shutdown_connections 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.202+0000 7f5d3988f640 1 -- 172.21.15.60:0/4126261569 wait complete. 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.203+0000 7f5d3988f640 1 Processor -- start 2024-04-25T08:30:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.203+0000 7f5d3988f640 1 -- start start 2024-04-25T08:30:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d3988f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d34160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d3988f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d3415dc20 0x7f5d341537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d3988f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d34161710 0x7f5d3415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d3988f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340b5e50 con 0x7f5d340ced80 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d3988f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34162500 con 0x7f5d34161710 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d3988f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340b5fd0 con 0x7f5d3415dc20 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d34160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d34160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46010/0 (socket says 172.21.15.60:46010) 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 -- 172.21.15.60:0/1560691952 learned_addr learned my addr 172.21.15.60:0/1560691952 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d327fc640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d3415dc20 0x7f5d341537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d337fe640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d34161710 0x7f5d3415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 -- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d3415dc20 msgr2=0x7f5d341537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d3415dc20 0x7f5d341537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 -- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d34161710 msgr2=0x7f5d3415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d34161710 0x7f5d3415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d32ffd640 1 -- 172.21.15.60:0/1560691952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34161040 con 0x7f5d340ced80 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.204+0000 7f5d327fc640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d3415dc20 0x7f5d341537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:39.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.205+0000 7f5d32ffd640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d34160b00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005880 tx=0x7f5d1c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.205+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017070 con 0x7f5d340ced80 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.205+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c005ce0 con 0x7f5d340ced80 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.205+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c007270 con 0x7f5d340ced80 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.205+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d340c87a0 con 0x7f5d340ced80 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.205+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d340c8d60 con 0x7f5d340ced80 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.206+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d340d3490 con 0x7f5d340ced80 2024-04-25T08:30:39.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.206+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d1c01a070 con 0x7f5d340ced80 2024-04-25T08:30:39.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.207+0000 7f5d2bfff640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d1406b4d0 0x7f5d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.207+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d1c0895c0 con 0x7f5d340ced80 2024-04-25T08:30:39.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.207+0000 7f5d327fc640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d1406b4d0 0x7f5d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.207+0000 7f5d327fc640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d1406b4d0 0x7f5d1406d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d24034000 tx=0x7f5d2402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:39.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.211+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d340d3490 con 0x7f5d340ced80 2024-04-25T08:30:39.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.461+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5d340d16b0 con 0x7f5d340ced80 2024-04-25T08:30:39.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.463+0000 7f5d2bfff640 1 -- 172.21.15.60:0/1560691952 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5d1c057c50 con 0x7f5d340ced80 2024-04-25T08:30:39.460 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:30:39.460 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:30:39.461 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:30:39.461 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:30:39.461 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:30:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d1406b4d0 msgr2=0x7f5d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d1406b4d0 0x7f5d1406d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d24034000 tx=0x7f5d2402e000 comp rx=0 tx=0).stop 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 msgr2=0x7f5d34160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d34160b00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005880 tx=0x7f5d1c00c9b0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 shutdown_connections 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d1406b4d0 0x7f5d1406d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d34161710 0x7f5d3415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d3415dc20 0x7f5d341537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 --2- 172.21.15.60:0/1560691952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d340ced80 0x7f5d34160b00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 >> 172.21.15.60:0/1560691952 conn(0x7f5d340d0420 msgr2=0x7f5d340b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 shutdown_connections 2024-04-25T08:30:39.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.465+0000 7f5d3988f640 1 -- 172.21.15.60:0/1560691952 wait complete. 2024-04-25T08:30:39.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:30:39.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 Processor -- start 2024-04-25T08:30:39.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 -- start start 2024-04-25T08:30:39.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1c815dbe0 0x7fe1c80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b2cb0 0x7fe1c80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80b97a0 con 0x7fe1c80b92a0 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c815dfc0 con 0x7fe1c815dbe0 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.578+0000 7fe1cd2c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80d1a00 con 0x7fe1c80b2cb0 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c67fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b2cb0 0x7fe1c80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe1c815dbe0 0x7fe1c80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c77fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe1c80b92a0 0x7fe1c80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:35038/0 (socket says 172.21.15.60:35038) 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c77fe640 1 -- 172.21.15.60:0/2569847114 learned_addr learned my addr 172.21.15.60:0/2569847114 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c6ffd640 1 -- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b2cb0 msgr2=0x7fe1c80d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c6ffd640 1 --2- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b2cb0 0x7fe1c80d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c6ffd640 1 -- 172.21.15.60:0/2569847114 >> v1:172.21.15.100:6789/0 conn(0x7fe1c80b92a0 legacy=0x7fe1c80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.579+0000 7fe1c6ffd640 1 -- 172.21.15.60:0/2569847114 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80d1c50 con 0x7fe1c815dbe0 2024-04-25T08:30:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.580+0000 7fe1c6ffd640 1 --2- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c815dbe0 0x7fe1c80d0910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80097a0 tx=0x7fe1b802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b530a15fc2a58ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:39.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.580+0000 7fe1c5ffb640 1 -- 172.21.15.60:0/2569847114 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8037070 con 0x7fe1c815dbe0 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.580+0000 7fe1c5ffb640 1 -- 172.21.15.60:0/2569847114 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8032440 con 0x7fe1c815dbe0 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.580+0000 7fe1c5ffb640 1 -- 172.21.15.60:0/2569847114 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8032810 con 0x7fe1c815dbe0 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c815dbe0 msgr2=0x7fe1c80d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c815dbe0 0x7fe1c80d0910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80097a0 tx=0x7fe1b802a180 comp rx=0 tx=0).stop 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/2569847114 shutdown_connections 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b2cb0 0x7fe1c80d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/2569847114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c815dbe0 0x7fe1c80d0910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/2569847114 >> 172.21.15.60:0/2569847114 conn(0x7fe1c8061a20 msgr2=0x7fe1c8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/2569847114 shutdown_connections 2024-04-25T08:30:39.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.581+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/2569847114 wait complete. 2024-04-25T08:30:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.582+0000 7fe1cd2c4640 1 Processor -- start 2024-04-25T08:30:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.582+0000 7fe1cd2c4640 1 -- start start 2024-04-25T08:30:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1cd2c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c80b2cb0 0x7fe1c80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1cd2c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b92a0 0x7fe1c814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1cd2c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 0x7fe1c815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1cd2c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80b6ab0 con 0x7fe1c815dbe0 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1cd2c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80d2210 con 0x7fe1c80b2cb0 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1cd2c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80b6c30 con 0x7fe1c80b92a0 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 0x7fe1c815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 0x7fe1c815f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46020/0 (socket says 172.21.15.60:46020) 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 -- 172.21.15.60:0/4078316669 learned_addr learned my addr 172.21.15.60:0/4078316669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c67fc640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b92a0 0x7fe1c814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c6ffd640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c80b2cb0 0x7fe1c80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 -- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b92a0 msgr2=0x7fe1c814e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b92a0 0x7fe1c814e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 -- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c80b2cb0 msgr2=0x7fe1c80b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c80b2cb0 0x7fe1c80b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 -- 172.21.15.60:0/4078316669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8150960 con 0x7fe1c815dbe0 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.583+0000 7fe1c77fe640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 0x7fe1c815f1a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000ca30 tx=0x7fe1c000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.584+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0017070 con 0x7fe1c815dbe0 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.584+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000abd0 con 0x7fe1c815dbe0 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.584+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c8150c40 con 0x7fe1c815dbe0 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.584+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0012400 con 0x7fe1c815dbe0 2024-04-25T08:30:39.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.584+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c80cc1e0 con 0x7fe1c815dbe0 2024-04-25T08:30:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.585+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c8151030 con 0x7fe1c815dbe0 2024-04-25T08:30:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.585+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1c001a070 con 0x7fe1c815dbe0 2024-04-25T08:30:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.586+0000 7fe1af7fe640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1b406b4d0 0x7fe1b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.586+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1c0089ff0 con 0x7fe1c815dbe0 2024-04-25T08:30:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.586+0000 7fe1c6ffd640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1b406b4d0 0x7fe1b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:39.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.587+0000 7fe1c6ffd640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1b406b4d0 0x7fe1b406d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8034000 tx=0x7fe1b802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:39.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.590+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1c0058690 con 0x7fe1c815dbe0 2024-04-25T08:30:39.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.812+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c8150e10 con 0x7fe1c815dbe0 2024-04-25T08:30:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.813+0000 7fe1af7fe640 1 -- 172.21.15.60:0/4078316669 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe1c005c730 con 0x7fe1c815dbe0 2024-04-25T08:30:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:30:39.810 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:30:30.652688+0000","modified":"2024-04-25T08:30:32.670840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2407433231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2407433231},{"type":"v1","addr":"172.21.15.60:6835","nonce":2407433231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:30:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 15 2024-04-25T08:30:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1b406b4d0 msgr2=0x7fe1b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1b406b4d0 0x7fe1b406d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8034000 tx=0x7fe1b802e000 comp rx=0 tx=0).stop 2024-04-25T08:30:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 msgr2=0x7fe1c815f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 0x7fe1c815f1a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000ca30 tx=0x7fe1c000cef0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 shutdown_connections 2024-04-25T08:30:39.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1c815dbe0 0x7fe1c815f1a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1c80b92a0 0x7fe1c814e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1b406b4d0 0x7fe1b406d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 --2- 172.21.15.60:0/4078316669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1c80b2cb0 0x7fe1c80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 >> 172.21.15.60:0/4078316669 conn(0x7fe1c8061a20 msgr2=0x7fe1c8061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 shutdown_connections 2024-04-25T08:30:39.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:39.815+0000 7fe1cd2c4640 1 -- 172.21.15.60:0/4078316669 wait complete. 2024-04-25T08:30:40.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:30:30.652688+0000', 'modified': '2024-04-25T08:30:32.670840+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5379': {'gid': 5379, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/2407433231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 2407433231}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 2407433231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:30:40.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:30:40.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:30:40.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 Processor -- start 2024-04-25T08:30:40.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 -- start start 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1bc015bb50 0x7f1bc00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bc00c70d0 0x7f1bc00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00c75e0 con 0x7f1bc00cfb40 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc00c8680 con 0x7f1bc00c70d0 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.333+0000 7f1bc5f63640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc00c78c0 con 0x7f1bc015bb50 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bc00c70d0 0x7f1bc00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1bc00c70d0 0x7f1bc00cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46028/0 (socket says 172.21.15.60:46028) 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 -- 172.21.15.60:0/1367210386 learned_addr learned my addr 172.21.15.60:0/1367210386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/1367210386 >> v2:172.21.15.100:3300/0 conn(0x7f1bc015bb50 0x7f1bc00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 -- 172.21.15.60:0/1367210386 >> v2:172.21.15.100:3300/0 conn(0x7f1bc015bb50 msgr2=0x7f1bc00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/1367210386 >> v2:172.21.15.100:3300/0 conn(0x7f1bc015bb50 0x7f1bc00c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 -- 172.21.15.60:0/1367210386 >> v1:172.21.15.100:6789/0 conn(0x7f1bc00cfb40 legacy=0x7f1bc0070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.334+0000 7f1bbeffd640 1 -- 172.21.15.60:0/1367210386 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00c7ba0 con 0x7f1bc00c70d0 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/1367210386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc00cf600 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0070ab0 tx=0x7f1bb402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd347bb08c7a236f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bbe7fc640 1 -- 172.21.15.60:0/1367210386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4037070 con 0x7f1bc00c70d0 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bbe7fc640 1 -- 172.21.15.60:0/1367210386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb402ae50 con 0x7f1bc00c70d0 2024-04-25T08:30:40.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bbe7fc640 1 -- 172.21.15.60:0/1367210386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4032860 con 0x7f1bc00c70d0 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 -- 172.21.15.60:0/1367210386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 msgr2=0x7f1bc00cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/1367210386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc00cf600 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0070ab0 tx=0x7f1bb402a300 comp rx=0 tx=0).stop 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 -- 172.21.15.60:0/1367210386 shutdown_connections 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/1367210386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc00cf600 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/1367210386 >> v2:172.21.15.100:3300/0 conn(0x7f1bc015bb50 0x7f1bc00c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 -- 172.21.15.60:0/1367210386 >> 172.21.15.60:0/1367210386 conn(0x7f1bc006f500 msgr2=0x7f1bc006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 -- 172.21.15.60:0/1367210386 shutdown_connections 2024-04-25T08:30:40.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.335+0000 7f1bc5f63640 1 -- 172.21.15.60:0/1367210386 wait complete. 2024-04-25T08:30:40.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 Processor -- start 2024-04-25T08:30:40.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 -- start start 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00cfb40 0x7f1bc01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015bb50 0x7f1bc00cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015f6d0 con 0x7f1bc00c70d0 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00c8160 con 0x7f1bc015bb50 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.337+0000 7f1bc5f63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015f850 con 0x7f1bc00cfb40 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc0151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46044/0 (socket says 172.21.15.60:46044) 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3673732717 learned_addr learned my addr 172.21.15.60:0/3673732717 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbffff640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015bb50 0x7f1bc00cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00cfb40 msgr2=0x7f1bc01565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00cfb40 0x7f1bc01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015bb50 msgr2=0x7f1bc00cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015bb50 0x7f1bc00cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3673732717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc01579d0 con 0x7f1bc00c70d0 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc0151f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e870 tx=0x7f1bb000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1ba7fff640 1 -- 172.21.15.60:0/3673732717 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0017070 con 0x7f1bc00c70d0 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1ba7fff640 1 -- 172.21.15.60:0/3673732717 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb000c940 con 0x7f1bc00c70d0 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc0157cb0 con 0x7f1bc00c70d0 2024-04-25T08:30:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1ba7fff640 1 -- 172.21.15.60:0/3673732717 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000cd30 con 0x7f1bc00c70d0 2024-04-25T08:30:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.338+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc00ca8a0 con 0x7f1bc00c70d0 2024-04-25T08:30:40.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.340+0000 7f1ba7fff640 1 -- 172.21.15.60:0/3673732717 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bb001a070 con 0x7f1bc00c70d0 2024-04-25T08:30:40.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.340+0000 7f1ba7fff640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1bac06b4d0 0x7f1bac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.340+0000 7f1ba7fff640 1 -- 172.21.15.60:0/3673732717 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1bb008a610 con 0x7f1bc00c70d0 2024-04-25T08:30:40.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.340+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1bac06b4d0 0x7f1bac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.341+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1bac06b4d0 0x7f1bac06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4034000 tx=0x7f1bb402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.341+0000 7f1bc8c6d740 1 Processor -- start 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.341+0000 7f1bc8c6d740 1 -- start start 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db431610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db44f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db406f10 0x5651db44d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651db369a10 con 0x5651db431230 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651db312b30 con 0x5651db406f10 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651db365c50 con 0x5651db44ead0 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db431610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db431610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46058/0 (socket says 172.21.15.60:46058) 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3639553351 learned_addr learned my addr 172.21.15.60:0/3639553351 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db44f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbffff640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db406f10 0x5651db44d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 msgr2=0x5651db44f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db44f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db406f10 msgr2=0x5651db44d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db406f10 0x5651db44d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/3639553351 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651db327e10 con 0x5651db431230 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbffff640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db406f10 0x5651db44d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.342+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db44f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db431610 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0055060 tx=0x7f1bb0058870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361becb9ee365691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1ba57fa640 1 -- 172.21.15.60:0/3639553351 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0021050 con 0x5651db431230 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1ba57fa640 1 -- 172.21.15.60:0/3639553351 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb005cd50 con 0x5651db431230 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1ba57fa640 1 -- 172.21.15.60:0/3639553351 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0061310 con 0x5651db431230 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/3639553351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 msgr2=0x5651db431610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db431610 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0055060 tx=0x7f1bb0058870 comp rx=0 tx=0).stop 2024-04-25T08:30:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/3639553351 shutdown_connections 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db406f10 0x5651db44d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db44f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.343+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/3639553351 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db431610 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.344+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/3639553351 >> 172.21.15.60:0/3639553351 conn(0x5651db403840 msgr2=0x5651db44c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.344+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/3639553351 shutdown_connections 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.344+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/3639553351 wait complete. 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.344+0000 7f1bc8c6d740 1 -- start start 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.344+0000 7f1bc8c6d740 1 Processor -- start 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db432d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db42ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 0x5651db42f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651db407cb0 con 0x5651db4332a0 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651db3260c0 con 0x5651db431230 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651db401770 con 0x5651db44ead0 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bc8c6d740 10 client.0 _ll_register_callbacks cb 0x5651db319ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 0x5651db42f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 0x5651db42f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46064/0 (socket says 172.21.15.60:46064) 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 -- 172.21.15.60:0/2874799227 learned_addr learned my addr 172.21.15.60:0/2874799227 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db432d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db42ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 -- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 msgr2=0x5651db42ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db42ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 -- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 msgr2=0x5651db432d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db432d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 -- 172.21.15.60:0/2874799227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651db07e2d0 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1bbffff640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 0x5651db42f100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800f4e0 tx=0x7f1bb800f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6351b95987ae5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.345+0000 7f1b9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2874799227 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8006b50 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2874799227 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1bb8006cf0 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bb005cd50 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2874799227 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb8021050 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 10 client.5577 fetch_fsmap learned FSMap version 15 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5651db1486b0 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2874799227 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb005cd50 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2874799227 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bb8024020 con 0x5651db4332a0 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 10 client.5577 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 10 client.5577 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:30:40.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 20 client.5577 populate_metadata read hostname 'smithi060' 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db45d150 0x5651db45d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 4 client.5577 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651db3c8920 con 0x5651db45d150 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.346+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db45d150 0x5651db45d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db45d150 0x5651db45d530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a331ab6b977a06 server_cookie=d8ff488ddbef121f in_seq=0 out_seq=0 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1b9e7fc640 10 client.5577 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2874799227 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651db3c8920 con 0x5651db45d150 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1b9e7fc640 10 client.5577 handle_command_reply: tid=0 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 client.5577 shutdown 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 msgr2=0x5651db42f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 0x5651db42f100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800f4e0 tx=0x7f1bb800f9a0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 shutdown_connections 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db45d150 0x5651db45d530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db4332a0 0x5651db42f100 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db44ead0 0x5651db42ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2874799227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db432d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1b9e7fc640 0 client.5577 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:30:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 >> 172.21.15.60:0/2874799227 conn(0x5651db406f10 msgr2=0x5651db44f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 shutdown_connections 2024-04-25T08:30:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.347+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2874799227 wait complete. 2024-04-25T08:30:40.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.348+0000 7f1bc8c6d740 20 client.5577 trim_cache size 0 max 16384 2024-04-25T08:30:40.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.363+0000 7f1bc8c6d740 1 Processor -- start 2024-04-25T08:30:40.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.363+0000 7f1bc8c6d740 1 -- start start 2024-04-25T08:30:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db443900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db4082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 0x5651db45b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01fe2f0 con 0x5651db44ead0 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01fd4f0 con 0x5651db431230 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc00c9ae0 con 0x5651db4332a0 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 0x5651db45b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bbffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 0x5651db45b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46070/0 (socket says 172.21.15.60:46070) 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.364+0000 7f1bbffff640 1 -- 172.21.15.60:0/626895575 learned_addr learned my addr 172.21.15.60:0/626895575 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bbffff640 1 -- 172.21.15.60:0/626895575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 msgr2=0x5651db4082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bbffff640 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db4082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bbffff640 1 -- 172.21.15.60:0/626895575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 msgr2=0x5651db443900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bbffff640 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db443900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bbffff640 1 -- 172.21.15.60:0/626895575 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651db42fe10 con 0x5651db44ead0 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bbffff640 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 0x5651db45b0b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800f460 tx=0x7f1bb8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85385c1459ae1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1ba4ff9640 1 -- 172.21.15.60:0/626895575 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80146b0 con 0x5651db44ead0 2024-04-25T08:30:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1ba4ff9640 1 -- 172.21.15.60:0/626895575 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8006b50 con 0x5651db44ead0 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1ba4ff9640 1 -- 172.21.15.60:0/626895575 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8003b90 con 0x5651db44ead0 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/626895575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 msgr2=0x5651db45b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.365+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 0x5651db45b0b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800f460 tx=0x7f1bb8012000 comp rx=0 tx=0).stop 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/626895575 shutdown_connections 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db44ead0 0x5651db45b0b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db4082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/626895575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db431230 0x5651db443900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/626895575 >> 172.21.15.60:0/626895575 conn(0x5651db406f10 msgr2=0x5651db44ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/626895575 shutdown_connections 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/626895575 wait complete. 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 -- start start 2024-04-25T08:30:40.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.366+0000 7f1bc8c6d740 1 Processor -- start 2024-04-25T08:30:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db4b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db45b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db44ead0 0x5651db45b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651db434480 con 0x5651db431230 2024-04-25T08:30:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015d760 con 0x5651db44ead0 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5651db434d20 con 0x5651db4332a0 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bc8c6d740 10 client.0 _ll_register_callbacks cb 0x5651db401c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db4b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db4b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46082/0 (socket says 172.21.15.60:46082) 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/2378652595 learned_addr learned my addr 172.21.15.60:0/2378652595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbffff640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db44ead0 0x5651db45b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db45b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 msgr2=0x5651db45b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db45b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db44ead0 msgr2=0x5651db45b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db44ead0 0x5651db45b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbf7fe640 1 -- 172.21.15.60:0/2378652595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651db42e0c0 con 0x5651db431230 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.367+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db45b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:30:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bbf7fe640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db4b6930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0055060 tx=0x7f1bb0088880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7058ae53dec618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2378652595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0021040 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2378652595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1bb0055270 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bb8006b50 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2378652595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bb0061310 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 10 client.5583 fetch_fsmap learned FSMap version 15 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5651db42ec40 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2378652595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8006b50 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2378652595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1bb006cbc0 con 0x5651db431230 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 10 client.5583 fetch_fsmap finished waiting for FSMap version 15 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 10 client.5583 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 20 client.5583 populate_metadata read hostname 'smithi060' 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db400c10 0x5651db4b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 4 client.5583 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:30:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5651db44df20 con 0x5651db400c10 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db400c10 0x5651db4b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1b9e7fc640 10 client.5583 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.368+0000 7f1bbeffd640 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db400c10 0x5651db4b7260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9c6ec589e74fa8b server_cookie=5a3321083cb04914 in_seq=0 out_seq=0 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1b9e7fc640 1 -- 172.21.15.60:0/2378652595 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5651db44df20 con 0x5651db400c10 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1b9e7fc640 10 client.5583 handle_command_reply: tid=0 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 client.5583 shutdown 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 msgr2=0x5651db4b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db4b6930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0055060 tx=0x7f1bb0088880 comp rx=0 tx=0).stop 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 shutdown_connections 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5651db400c10 0x5651db4b7260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5651db44ead0 0x5651db45b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5651db4332a0 0x5651db45b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1b9e7fc640 0 client.5583 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:30:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 --2- 172.21.15.60:0/2378652595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5651db431230 0x5651db4b6930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 >> 172.21.15.60:0/2378652595 conn(0x5651db406f10 msgr2=0x5651db442ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 shutdown_connections 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 1 -- 172.21.15.60:0/2378652595 wait complete. 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.369+0000 7f1bc8c6d740 20 client.5583 trim_cache size 0 max 16384 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5379, 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:30:40.373 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.0499353859999996, 2024-04-25T08:30:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 15, 2024-04-25T08:30:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 39, 2024-04-25T08:30:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 39, 2024-04-25T08:30:40.374 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 15.953476554 2024-04-25T08:30:40.374 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1bac06b4d0 msgr2=0x7f1bac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1bac06b4d0 0x7f1bac06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4034000 tx=0x7f1bb402c000 comp rx=0 tx=0).stop 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 msgr2=0x7f1bc0151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc0151f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e870 tx=0x7f1bb000ed30 comp rx=0 tx=0).stop 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 shutdown_connections 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1bac06b4d0 0x7f1bac06d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bc015bb50 0x7f1bc00cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bc00cfb40 0x7f1bc01565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 --2- 172.21.15.60:0/3673732717 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bc00c70d0 0x7f1bc0151f00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 >> 172.21.15.60:0/3673732717 conn(0x7f1bc006f500 msgr2=0x7f1bc00b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 shutdown_connections 2024-04-25T08:30:40.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.370+0000 7f1bc5f63640 1 -- 172.21.15.60:0/3673732717 wait complete. 2024-04-25T08:30:40.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:30:40.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.481+0000 7f47378e1640 1 Processor -- start 2024-04-25T08:30:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.481+0000 7f47378e1640 1 -- start start 2024-04-25T08:30:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.482+0000 7f47378e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 0x7f4730168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.482+0000 7f47378e1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47300c8030 con 0x7f473015da70 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.482+0000 7f47378e1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47300c8310 con 0x7f47300d0170 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.482+0000 7f47378e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300c85f0 con 0x7f47300ce8d0 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.482+0000 7f473567d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f473015da70 0x7f47300c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53534/0 (socket says 172.21.15.60:53534) 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.482+0000 7f473567d640 1 -- 172.21.15.60:0/2533551380 learned_addr learned my addr 172.21.15.60:0/2533551380 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217847277 0 0) 0x7f47300c8030 con 0x7f473015da70 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4718003520 con 0x7f473015da70 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4735e7e640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 0x7f4730168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1884820978 0 0) 0x7f4718003520 con 0x7f473015da70 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 msgr2=0x7f4730168690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 0x7f4730168690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 >> v1:172.21.15.100:6789/0 conn(0x7f47300d0170 legacy=0x7f47300ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473006bb30 con 0x7f473015da70 2024-04-25T08:30:40.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f4728001670 con 0x7f473015da70 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 >> v1:172.21.15.60:6789/0 conn(0x7f473015da70 legacy=0x7f47300c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 0x7f47180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4718003f60 0x7f4718004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718004d40 0x7f47180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300c85f0 con 0x7f4718003f60 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471800baa0 con 0x7f4718004d40 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471800bca0 con 0x7f47300ce8d0 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4718003f60 0x7f4718004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f473567d640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 0x7f47180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4735e7e640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718004d40 0x7f47180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4734e7c640 1 -- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 msgr2=0x7f47180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f47300ce8d0 0x7f47180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4734e7c640 1 -- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718004d40 msgr2=0x7f47180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718004d40 0x7f47180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.483+0000 7f4734e7c640 1 -- 172.21.15.60:0/2533551380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718003520 con 0x7f4718003f60 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.484+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4718003f60 0x7f4718004800 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4724001750 tx=0x7f4718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec74d21ba14c9321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.484+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724001370 con 0x7f4718003f60 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.484+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47240052d0 con 0x7f4718003f60 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.484+0000 7f472ffff640 1 -- 172.21.15.60:0/2533551380 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472402abe0 con 0x7f4718003f60 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f47378e1640 1 -- 172.21.15.60:0/2533551380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4718003f60 msgr2=0x7f4718004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f47378e1640 1 --2- 172.21.15.60:0/2533551380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4718003f60 0x7f4718004800 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4724001750 tx=0x7f4718005830 comp rx=0 tx=0).stop 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f4735e7e640 1 -- 172.21.15.60:0/2533551380 reap_dead start 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f47378e1640 1 -- 172.21.15.60:0/2533551380 shutdown_connections 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f47378e1640 1 -- 172.21.15.60:0/2533551380 >> 172.21.15.60:0/2533551380 conn(0x7f47300b33e0 msgr2=0x7f47300b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f47378e1640 1 -- 172.21.15.60:0/2533551380 shutdown_connections 2024-04-25T08:30:40.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.485+0000 7f47378e1640 1 -- 172.21.15.60:0/2533551380 wait complete. 2024-04-25T08:30:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.486+0000 7f47378e1640 1 Processor -- start 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.486+0000 7f47378e1640 1 -- start start 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f47378e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718003f60 0x7f47301592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f47378e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 0x7f473014b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f47378e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f473015fff0 0x7f473014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f47378e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473006d340 con 0x7f473015fb70 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f47378e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473015de50 con 0x7f4718003f60 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f47378e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473006d4c0 con 0x7f473015fff0 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 0x7f473014b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 0x7f473014b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46108/0 (socket says 172.21.15.60:46108) 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 -- 172.21.15.60:0/2730225721 learned_addr learned my addr 172.21.15.60:0/2730225721 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f473567d640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718003f60 0x7f47301592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4735e7e640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f473015fff0 0x7f473014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 -- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f473015fff0 msgr2=0x7f473014bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f473015fff0 0x7f473014bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 -- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718003f60 msgr2=0x7f47301592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718003f60 0x7f47301592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 -- 172.21.15.60:0/2730225721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730160550 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.487+0000 7f4734e7c640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 0x7f473014b640 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4724005040 tx=0x7f4724001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.488+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724042070 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.488+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472402be50 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.488+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4730160830 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.488+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724037040 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.488+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4730160df0 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.489+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4730073990 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.489+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4724045070 con 0x7f473015fb70 2024-04-25T08:30:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.490+0000 7f472dffb640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f471c06b500 0x7f471c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.490+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47240a9a20 con 0x7f473015fb70 2024-04-25T08:30:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.490+0000 7f473567d640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f471c06b500 0x7f471c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.491+0000 7f473567d640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f471c06b500 0x7f471c06d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4728007d30 tx=0x7f47280078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.494+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47240780b0 con 0x7f473015fb70 2024-04-25T08:30:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.711+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f47300ced20 con 0x7f473015fb70 2024-04-25T08:30:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.712+0000 7f472dffb640 1 -- 172.21.15.60:0/2730225721 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f472407c150 con 0x7f473015fb70 2024-04-25T08:30:40.709 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:30:40.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f471c06b500 msgr2=0x7f471c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f471c06b500 0x7f471c06d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4728007d30 tx=0x7f47280078f0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 msgr2=0x7f473014b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 0x7f473014b640 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4724005040 tx=0x7f4724001120 comp rx=0 tx=0).stop 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 shutdown_connections 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f473015fff0 0x7f473014bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f473015fb70 0x7f473014b640 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f471c06b500 0x7f471c06d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 --2- 172.21.15.60:0/2730225721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4718003f60 0x7f47301592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 >> 172.21.15.60:0/2730225721 conn(0x7f47300b33e0 msgr2=0x7f473015ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 shutdown_connections 2024-04-25T08:30:40.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:40.714+0000 7f47378e1640 1 -- 172.21.15.60:0/2730225721 wait complete. 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:30:40.722 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:30:40.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:40.723 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:30:40.738 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:30:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:30:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:30:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.738 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42670sec preferred_lft 42670sec 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.739 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:30:40.740 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:30:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:30:40.741 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:30:40.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:40.741 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:30:40.800 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:30:40.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:40.802 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:30:40.858 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:30:40.858 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:30:40.858 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:30:40.858 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:30:40.858 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:30:40.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:40.859 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:30:40.859 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:30:40.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:30:40.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:30:40.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:30:40.859 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:30:40.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:40.944 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:30:40.959 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:30:40.959 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:30:40.959 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:30:40.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:40.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:30:41.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:30:41.023 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:30:41.078 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:30:41.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:30:41.078 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:30:41.132 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:30:41.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:41.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:30:41.215 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:30:41.215 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:30:41.215 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:30:41.224 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:30:41.224 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:30:41.224 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:30:41.224 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:30:41.224 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:30:41.229 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:30:41.230 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:30:41.231 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:30:41.231 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:30:41.231 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:30:41.231 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:30:41.231 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:30:41.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:30:41.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:41.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:30:41.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:41.298 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:30:41.358 INFO:teuthology.orchestra.run.smithi060.stdout:client5592 2024-04-25T08:30:41.359 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:41.359 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:41.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T08:30:41.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.563+0000 7fe5a6a8f640 1 Processor -- start 2024-04-25T08:30:41.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.563+0000 7fe5a6a8f640 1 -- start start 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a6a8f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe5a01452a0 0x7fe5a006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a6a8f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a00cccc0 con 0x7fe5a01446d0 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a6a8f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a00cc140 con 0x7fe5a0144d50 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a6a8f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00b2b20 con 0x7fe5a01452a0 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a482b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe5a01446d0 0x7fe5a00b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53536/0 (socket says 172.21.15.60:53536) 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a482b640 1 -- 172.21.15.60:0/1477478582 learned_addr learned my addr 172.21.15.60:0/1477478582 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe5a502c640 1 --2- 172.21.15.60:0/1477478582 >> v2:172.21.15.100:3300/0 conn(0x7fe5a01452a0 0x7fe5a006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196510174 0 0) 0x7fe5a00cccc0 con 0x7fe5a01446d0 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe590003520 con 0x7fe5a01446d0 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3565409868 0 0) 0x7fe590003520 con 0x7fe5a01446d0 2024-04-25T08:30:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 >> v2:172.21.15.100:3300/0 conn(0x7fe5a01452a0 msgr2=0x7fe5a006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 --2- 172.21.15.60:0/1477478582 >> v2:172.21.15.100:3300/0 conn(0x7fe5a01452a0 0x7fe5a006dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 >> v1:172.21.15.100:6789/0 conn(0x7fe5a0144d50 legacy=0x7fe5a006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00cc420 con 0x7fe5a01446d0 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe594001670 con 0x7fe5a01446d0 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 >> v1:172.21.15.60:6789/0 conn(0x7fe5a01446d0 legacy=0x7fe5a00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5900039b0 0x7fe590003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.564+0000 7fe59f7fe640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe590004800 0x7fe590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59f7fe640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5900051c0 0x7fe59000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00b2b20 con 0x7fe590004800 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59000c060 con 0x7fe5900039b0 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59000c260 con 0x7fe5900051c0 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe590004800 0x7fe590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe5a482b640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5900039b0 0x7fe590003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe5a502c640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5900051c0 0x7fe59000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59ffff640 1 -- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5900051c0 msgr2=0x7fe59000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5900051c0 0x7fe59000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59ffff640 1 -- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5900039b0 msgr2=0x7fe590003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5900039b0 0x7fe590003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe59ffff640 1 -- 172.21.15.60:0/1477478582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590003520 con 0x7fe590004800 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe5a502c640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5900051c0 0x7fe59000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.565+0000 7fe5a502c640 1 -- 172.21.15.60:0/1477478582 reap_dead start 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe590004800 0x7fe590004c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe58c001750 tx=0x7fe590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe62b2e80a2f18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c001370 con 0x7fe590004800 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c0052d0 con 0x7fe590004800 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe59f7fe640 1 -- 172.21.15.60:0/1477478582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c02abc0 con 0x7fe590004800 2024-04-25T08:30:41.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1477478582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe590004800 msgr2=0x7fe590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe590004800 0x7fe590004c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe58c001750 tx=0x7fe590005830 comp rx=0 tx=0).stop 2024-04-25T08:30:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1477478582 shutdown_connections 2024-04-25T08:30:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1477478582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe590004800 0x7fe590004c80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1477478582 >> 172.21.15.60:0/1477478582 conn(0x7fe5a006abe0 msgr2=0x7fe5a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1477478582 shutdown_connections 2024-04-25T08:30:41.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.566+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1477478582 wait complete. 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 Processor -- start 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 -- start start 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe590004800 0x7fe5a00a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 0x7fe5a00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5a00ac030 0x7fe5a00ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00cc9e0 con 0x7fe5a00b11d0 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a006d400 con 0x7fe590004800 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.568+0000 7fe5a6a8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0144ab0 con 0x7fe5a00ac030 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 0x7fe5a00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 0x7fe5a00b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46136/0 (socket says 172.21.15.60:46136) 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 -- 172.21.15.60:0/1867210017 learned_addr learned my addr 172.21.15.60:0/1867210017 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe5a482b640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe590004800 0x7fe5a00a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe5a502c640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5a00ac030 0x7fe5a00ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 -- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5a00ac030 msgr2=0x7fe5a00ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:41.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5a00ac030 0x7fe5a00ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 -- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe590004800 msgr2=0x7fe5a00a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe590004800 0x7fe5a00a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 -- 172.21.15.60:0/1867210017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00b1c90 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59ffff640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 0x7fe5a00b1630 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe58c001870 tx=0x7fe58c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c037040 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c005480 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c001370 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a01315f0 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.569+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a0131bb0 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.570+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a0073a00 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.571+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe58c043020 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.571+0000 7fe59d7fa640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe58406b500 0x7fe58406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.571+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe58c0a9740 con 0x7fe5a00b11d0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.571+0000 7fe5a482b640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe58406b500 0x7fe58406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:30:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.572+0000 7fe5a482b640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe58406b500 0x7fe58406d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe594001050 tx=0x7fe5940078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:30:41.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.576+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe58c04a020 con 0x7fe5a00b11d0 2024-04-25T08:30:41.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:41.794+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5a006e720 con 0x7fe5a00b11d0 2024-04-25T08:30:42.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.480+0000 7fe59d7fa640 1 -- 172.21.15.60:0/1867210017 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v16) v1 ==== 167+0+0 (secure 0 0 0) 0x7fe58c077d50 con 0x7fe5a00b11d0 2024-04-25T08:30:42.478 INFO:teuthology.orchestra.run.smithi060.stderr:enabled new snapshots 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.483+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe58406b500 msgr2=0x7fe58406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.483+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe58406b500 0x7fe58406d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe594001050 tx=0x7fe5940078f0 comp rx=0 tx=0).stop 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.483+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 msgr2=0x7fe5a00b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.483+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 0x7fe5a00b1630 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe58c001870 tx=0x7fe58c001120 comp rx=0 tx=0).stop 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.483+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 shutdown_connections 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5a00ac030 0x7fe5a00ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5a00b11d0 0x7fe5a00b1630 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe58406b500 0x7fe58406d9c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:42.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 --2- 172.21.15.60:0/1867210017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe590004800 0x7fe5a00a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:30:42.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 >> 172.21.15.60:0/1867210017 conn(0x7fe5a006abe0 msgr2=0x7fe5a006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:30:42.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 shutdown_connections 2024-04-25T08:30:42.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:42.484+0000 7fe5a6a8f640 1 -- 172.21.15.60:0/1867210017 wait complete. 2024-04-25T08:30:42.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:42.494 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T08:30:42.517 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:30:42.517 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> n = 2000 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> if 3 > 0: 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2024-04-25T08:30:42.575 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> try: 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2024-04-25T08:30:42.576 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(3): 2024-04-25T08:30:42.577 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T08:30:42.577 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:30:42.577 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2024-04-25T08:30:42.577 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2024-04-25T08:30:42.577 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2024-04-25T08:30:42.577 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:30:55.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:55.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:30:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:30:59.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:59.357 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:30:59.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:30:59.588 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T08:31:25.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:25.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:26.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:26.096 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:31:26.165 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:31:26.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T08:31:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.294+0000 7faa60e9a640 1 Processor -- start 2024-04-25T08:31:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.294+0000 7faa60e9a640 1 -- start start 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa60e9a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa5c15dbb0 0x7faa5c06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa60e9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa5c06c300 0x7faa5c06db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa60e9a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c0cae60 con 0x7faa5c0cbf00 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa60e9a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c0cb140 con 0x7faa5c15dbb0 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa60e9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c0cb420 con 0x7faa5c06c300 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa5c15dbb0 0x7faa5c06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa5c15dbb0 0x7faa5c06fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49018/0 (socket says 172.21.15.60:49018) 2024-04-25T08:31:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 -- 172.21.15.60:0/3824220475 learned_addr learned my addr 172.21.15.60:0/3824220475 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5ad9d640 1 --2- 172.21.15.60:0/3824220475 >> v2:172.21.15.100:3300/0 conn(0x7faa5c06c300 0x7faa5c06db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 -- 172.21.15.60:0/3824220475 >> v2:172.21.15.100:3300/0 conn(0x7faa5c06c300 msgr2=0x7faa5c06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 --2- 172.21.15.60:0/3824220475 >> v2:172.21.15.100:3300/0 conn(0x7faa5c06c300 0x7faa5c06db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 -- 172.21.15.60:0/3824220475 >> v1:172.21.15.100:6789/0 conn(0x7faa5c0cbf00 legacy=0x7faa5c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.295+0000 7faa5a59c640 1 -- 172.21.15.60:0/3824220475 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c0cb700 con 0x7faa5c15dbb0 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.296+0000 7faa5a59c640 1 --2- 172.21.15.60:0/3824220475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06fea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faa5c061e00 tx=0x7faa4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1aa3956d69889d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.296+0000 7faa59d9b640 1 -- 172.21.15.60:0/3824220475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c037070 con 0x7faa5c15dbb0 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.296+0000 7faa59d9b640 1 -- 172.21.15.60:0/3824220475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c032440 con 0x7faa5c15dbb0 2024-04-25T08:31:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.296+0000 7faa59d9b640 1 -- 172.21.15.60:0/3824220475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c032810 con 0x7faa5c15dbb0 2024-04-25T08:31:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 -- 172.21.15.60:0/3824220475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 msgr2=0x7faa5c06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 --2- 172.21.15.60:0/3824220475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06fea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faa5c061e00 tx=0x7faa4c02a270 comp rx=0 tx=0).stop 2024-04-25T08:31:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 -- 172.21.15.60:0/3824220475 shutdown_connections 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 --2- 172.21.15.60:0/3824220475 >> v2:172.21.15.100:3300/0 conn(0x7faa5c06c300 0x7faa5c06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 --2- 172.21.15.60:0/3824220475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06fea0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 -- 172.21.15.60:0/3824220475 >> 172.21.15.60:0/3824220475 conn(0x7faa5c071af0 msgr2=0x7faa5c071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 -- 172.21.15.60:0/3824220475 shutdown_connections 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.297+0000 7faa60e9a640 1 -- 172.21.15.60:0/3824220475 wait complete. 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.298+0000 7faa60e9a640 1 Processor -- start 2024-04-25T08:31:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.298+0000 7faa60e9a640 1 -- start start 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa60e9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 0x7faa5c1484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa60e9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa5c0cbf00 0x7faa5c14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa60e9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa60e9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0cab10 con 0x7faa5c15dbb0 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa60e9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c15df90 con 0x7faa5c06c300 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa60e9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c0703e0 con 0x7faa5c0cbf00 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa5a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 0x7faa5c1484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa5ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa5a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 0x7faa5c1484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42020/0 (socket says 172.21.15.60:42020) 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa5a59c640 1 -- 172.21.15.60:0/1767675791 learned_addr learned my addr 172.21.15.60:0/1767675791 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.299+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa5c0cbf00 0x7faa5c14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa5a59c640 1 -- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa5c0cbf00 msgr2=0x7faa5c14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa5a59c640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa5c0cbf00 0x7faa5c14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa5a59c640 1 -- 172.21.15.60:0/1767675791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 msgr2=0x7faa5c06ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa5a59c640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa5a59c640 1 -- 172.21.15.60:0/1767675791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c14bc10 con 0x7faa5c06c300 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa5a59c640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 0x7faa5c1484a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faa4c02a2b0 tx=0x7faa4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa537fe640 1 -- 172.21.15.60:0/1767675791 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c041030 con 0x7faa5c06c300 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa537fe640 1 -- 172.21.15.60:0/1767675791 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c030440 con 0x7faa5c06c300 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c0c7220 con 0x7faa5c06c300 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa537fe640 1 -- 172.21.15.60:0/1767675791 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c037040 con 0x7faa5c06c300 2024-04-25T08:31:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.300+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c0c9380 con 0x7faa5c06c300 2024-04-25T08:31:26.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.302+0000 7faa537fe640 1 -- 172.21.15.60:0/1767675791 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa4c044070 con 0x7faa5c06c300 2024-04-25T08:31:26.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.302+0000 7faa537fe640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa3c06b4d0 0x7faa3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.302+0000 7faa537fe640 1 -- 172.21.15.60:0/1767675791 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7faa4c0aa910 con 0x7faa5c06c300 2024-04-25T08:31:26.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.303+0000 7faa51d9b640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa3c06b4d0 0x7faa3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.303+0000 7faa51d9b640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa3c06b4d0 0x7faa3c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0b16d0 tx=0x7faa44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.303+0000 7faa63b97740 1 Processor -- start 2024-04-25T08:31:26.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.303+0000 7faa63b97740 1 -- start start 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa63b97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 0x55d5f08e4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa63b97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f0846b90 con 0x55d5f0920370 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f07efe40 con 0x55d5f08e3ca0 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5f0842c30 con 0x55d5f090e760 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 0x55d5f08e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 0x55d5f08e4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49032/0 (socket says 172.21.15.60:49032) 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa51d9b640 1 -- 172.21.15.60:0/4096607726 learned_addr learned my addr 172.21.15.60:0/4096607726 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa5ad9d640 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.304+0000 7faa5a59c640 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa51d9b640 1 -- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 msgr2=0x55d5f090ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa51d9b640 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa51d9b640 1 -- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 msgr2=0x55d5f08e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa51d9b640 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa51d9b640 1 -- 172.21.15.60:0/4096607726 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5f06257d0 con 0x55d5f0920370 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa5ad9d640 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa51d9b640 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 0x55d5f08e4080 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faa4400fba0 tx=0x7faa44012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fb35de092e814b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa3bfff640 1 -- 172.21.15.60:0/4096607726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44026070 con 0x55d5f0920370 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa3bfff640 1 -- 172.21.15.60:0/4096607726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa44007030 con 0x55d5f0920370 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.305+0000 7faa3bfff640 1 -- 172.21.15.60:0/4096607726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa440073e0 con 0x55d5f0920370 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 -- 172.21.15.60:0/4096607726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 msgr2=0x55d5f08e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 0x55d5f08e4080 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faa4400fba0 tx=0x7faa44012000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 -- 172.21.15.60:0/4096607726 shutdown_connections 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f0920370 0x55d5f08e4080 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 --2- 172.21.15.60:0/4096607726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 -- 172.21.15.60:0/4096607726 >> 172.21.15.60:0/4096607726 conn(0x55d5f08ab870 msgr2=0x55d5f0929760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 -- 172.21.15.60:0/4096607726 shutdown_connections 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 -- 172.21.15.60:0/4096607726 wait complete. 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 -- start start 2024-04-25T08:31:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.306+0000 7faa63b97740 1 Processor -- start 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f09108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0922460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f090b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f08033e0 con 0x55d5f090e760 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f090c2b0 con 0x55d5f08e3ca0 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5f0910e20 con 0x55d5f0920370 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa63b97740 10 client.0 _ll_register_callbacks cb 0x55d5f0802db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.307+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0922460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0922460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49038/0 (socket says 172.21.15.60:49038) 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 -- 172.21.15.60:0/1334032408 learned_addr learned my addr 172.21.15.60:0/1334032408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa5a59c640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f09108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa5ad9d640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f090b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 -- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 msgr2=0x55d5f090b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f090b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 -- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 msgr2=0x55d5f09108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f09108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 -- 172.21.15.60:0/1334032408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5f051b0f0 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa51d9b640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0922460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa4400fb40 tx=0x7faa44000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60f8458f1388441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa397fa640 1 -- 172.21.15.60:0/1334032408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4402f020 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa397fa640 1 -- 172.21.15.60:0/1334032408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7faa44020620 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa44007030 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa397fa640 1 -- 172.21.15.60:0/1334032408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa44039050 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa63b97740 10 client.5610 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d5f0685f60 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.308+0000 7faa397fa640 1 -- 172.21.15.60:0/1334032408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa44007030 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa397fa640 1 -- 172.21.15.60:0/1334032408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faa4403c020 con 0x55d5f090e760 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa63b97740 10 client.5610 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:26.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa63b97740 10 client.5610 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa63b97740 20 client.5610 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa63b97740 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f090b720 0x55d5f08fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa63b97740 4 client.5610 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5f08540d0 con 0x55d5f090b720 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa5a59c640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f090b720 0x55d5f08fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa397fa640 10 client.5610 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.309+0000 7faa5a59c640 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f090b720 0x55d5f08fbbd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f669c06759d35d server_cookie=42e3387d1b979657 in_seq=0 out_seq=0 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa397fa640 1 -- 172.21.15.60:0/1334032408 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d5f08540d0 con 0x55d5f090b720 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa397fa640 10 client.5610 handle_command_reply: tid=0 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 client.5610 shutdown 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 msgr2=0x55d5f0922460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0922460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa4400fb40 tx=0x7faa44000ed0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 shutdown_connections 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f090b720 0x55d5f08fbbd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f090b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0922460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 --2- 172.21.15.60:0/1334032408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f09108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 >> 172.21.15.60:0/1334032408 conn(0x55d5f08ab870 msgr2=0x55d5f090c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa397fa640 0 client.5610 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 shutdown_connections 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 1 -- 172.21.15.60:0/1334032408 wait complete. 2024-04-25T08:31:26.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.310+0000 7faa63b97740 20 client.5610 trim_cache size 0 max 16384 2024-04-25T08:31:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 Processor -- start 2024-04-25T08:31:26.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 -- start start 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 0x55d5f0909480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f0920370 0x55d5f090c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c2004d0 con 0x55d5f08e3ca0 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c1ff6a0 con 0x55d5f0920370 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c1499c0 con 0x55d5f090e760 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa5a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 0x55d5f0909480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa5a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 0x55d5f0909480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49052/0 (socket says 172.21.15.60:49052) 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa5a59c640 1 -- 172.21.15.60:0/364476243 learned_addr learned my addr 172.21.15.60:0/364476243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa5ad9d640 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f0920370 0x55d5f090c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.325+0000 7faa51d9b640 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa5a59c640 1 -- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 msgr2=0x55d5f090c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa5a59c640 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa5a59c640 1 -- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f0920370 msgr2=0x55d5f090c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa5a59c640 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f0920370 0x55d5f090c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa5a59c640 1 -- 172.21.15.60:0/364476243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5f04f41b0 con 0x55d5f08e3ca0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa5a59c640 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 0x55d5f0909480 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa4c078fc0 tx=0x7faa4c075350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec0d8ef12c2ab86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa3b7fe640 1 -- 172.21.15.60:0/364476243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c081690 con 0x55d5f08e3ca0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa3b7fe640 1 -- 172.21.15.60:0/364476243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c0770d0 con 0x55d5f08e3ca0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.326+0000 7faa3b7fe640 1 -- 172.21.15.60:0/364476243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0756a0 con 0x55d5f08e3ca0 2024-04-25T08:31:26.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 -- 172.21.15.60:0/364476243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 msgr2=0x55d5f0909480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 0x55d5f0909480 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa4c078fc0 tx=0x7faa4c075350 comp rx=0 tx=0).stop 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 -- 172.21.15.60:0/364476243 shutdown_connections 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f0920370 0x55d5f090c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f090e760 0x55d5f090c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 --2- 172.21.15.60:0/364476243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f08e3ca0 0x55d5f0909480 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 -- 172.21.15.60:0/364476243 >> 172.21.15.60:0/364476243 conn(0x55d5f08ab870 msgr2=0x55d5f08d9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 -- 172.21.15.60:0/364476243 shutdown_connections 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 -- 172.21.15.60:0/364476243 wait complete. 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 -- start start 2024-04-25T08:31:26.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.327+0000 7faa63b97740 1 Processor -- start 2024-04-25T08:31:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0994e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f091f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c149fc0 con 0x55d5f090e760 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f092d690 con 0x55d5f08e3ca0 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d5f0910670 con 0x55d5f0920370 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa63b97740 10 client.0 _ll_register_callbacks cb 0x7faa44024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0994e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa51d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0994e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49056/0 (socket says 172.21.15.60:49056) 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa51d9b640 1 -- 172.21.15.60:0/3105209210 learned_addr learned my addr 172.21.15.60:0/3105209210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa5ad9d640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f091f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.328+0000 7faa5a59c640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa51d9b640 1 -- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 msgr2=0x55d5f091f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa51d9b640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f091f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa51d9b640 1 -- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 msgr2=0x55d5f08e3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa51d9b640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa51d9b640 1 -- 172.21.15.60:0/3105209210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5f090d160 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa51d9b640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0994e20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faa4400fb40 tx=0x7faa4401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a271a3071b4868d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa397fa640 1 -- 172.21.15.60:0/3105209210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4402f070 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa397fa640 1 -- 172.21.15.60:0/3105209210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7faa44020b70 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa4c0770d0 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa397fa640 1 -- 172.21.15.60:0/3105209210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa44026040 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 10 client.5616 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d5f08e3640 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa397fa640 1 -- 172.21.15.60:0/3105209210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c0770d0 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa397fa640 1 -- 172.21.15.60:0/3105209210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faa4403c070 con 0x55d5f090e760 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 10 client.5616 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:26.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 10 client.5616 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 20 client.5616 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f091f630 0x55d5f0920770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 4 client.5616 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.329+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55d5f084e420 con 0x55d5f091f630 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.330+0000 7faa5a59c640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f091f630 0x55d5f0920770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.330+0000 7faa397fa640 10 client.5616 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.330+0000 7faa5a59c640 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f091f630 0x55d5f0920770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53cf661cdb35b7f3 server_cookie=91e39d2fa53d1f6b in_seq=0 out_seq=0 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.330+0000 7faa397fa640 1 -- 172.21.15.60:0/3105209210 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17915 (crc 0 0 0) 0x55d5f084e420 con 0x55d5f091f630 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.330+0000 7faa397fa640 10 client.5616 handle_command_reply: tid=0 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.330+0000 7faa63b97740 1 client.5616 shutdown 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 msgr2=0x55d5f0994e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0994e20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faa4400fb40 tx=0x7faa4401e000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 shutdown_connections 2024-04-25T08:31:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d5f091f630 0x55d5f0920770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d5f0920370 0x55d5f091f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d5f090e760 0x55d5f0994e20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 --2- 172.21.15.60:0/3105209210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d5f08e3ca0 0x55d5f08e3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa397fa640 0 client.5616 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 >> 172.21.15.60:0/3105209210 conn(0x55d5f08ab870 msgr2=0x55d5f090c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 shutdown_connections 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 1 -- 172.21.15.60:0/3105209210 wait complete. 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa63b97740 20 client.5616 trim_cache size 0 max 16384 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:31:26.333 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2078, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2078, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 520803, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 35219206, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 9, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.300630341, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.175437960, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.043709750, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.068660438, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2076, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.388760590, 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000187264 2024-04-25T08:31:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000139, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000069 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 970, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 983, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 245586, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 20899450, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:31:26.335 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.193209802, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.121527125, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.024183802, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.039280616, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 982, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.221151443, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000225205 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 107216 2024-04-25T08:31:26.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 28516, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 50954, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 9295398, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 52800924, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 2.902916301, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 2.211775298, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.938362366, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.052120351, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 50954, 2024-04-25T08:31:26.337 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.213424215, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000023814 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 48296, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.008278117, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000171 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 116528, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 18208 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2024-04-25T08:31:26.338 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1189, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 17.422327509, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.014652924 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2075, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.224631967, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000590184 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 25267, 2024-04-25T08:31:26.339 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 25267, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 25267, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 467.419286222, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.018499199 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 12, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 19866, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 19866, 2024-04-25T08:31:26.340 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 19866, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 75, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 75, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 2, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 45725, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 45323, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 111, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2024-04-25T08:31:26.341 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:31:26.342 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 912, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 4032, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 3672, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 3272, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 61, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 13874 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.343 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 2000, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 1, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 4125, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 1927, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 2224, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2024-04-25T08:31:26.344 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "evlrg": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 24423, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 24423, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 33, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2024-04-25T08:31:26.345 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 33, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "segmjr": 3, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 95873232, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 0, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2492, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 132.963128536, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.053355990 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 0 2024-04-25T08:31:26.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 2014, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 2014, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 0, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 13, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 13, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 0, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 19866, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 19867, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 1, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 75, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 8186, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 8111, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 114416, 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 215396 2024-04-25T08:31:26.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 28077, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 25267, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 3, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2000, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.347906597, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000673953 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 29, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.004536469, 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000156429 2024-04-25T08:31:26.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 6000, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.953934304, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001158989 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 7001, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.135681435, 2024-04-25T08:31:26.349 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000876400 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.350 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000040191, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000040191 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002939576, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002939576 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.207533556, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.207533556 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.352 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.461151876, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.182643984 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2224, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 434.412748594, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.195329473 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.353 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.050807589, 2024-04-25T08:31:26.354 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.050807589 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000260343, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000260343 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.355 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2024-04-25T08:31:26.356 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8000, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 16.841745692, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002105218 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 1, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 0, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 18479, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 18479, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 43, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mdthresh_evicted": 0 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:31:26.358 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:31:26.359 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:31:26.360 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 16680330, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 6248, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 440, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 5, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 5520, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 78, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-04-25T08:31:26.361 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 21308111, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 884018, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 4653, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 92169601, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 37, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 4653, 2024-04-25T08:31:26.362 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4653, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 21.022805131, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.004518118 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4653, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 4743 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 4653, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 51, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 4602, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 0, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 11, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 51, 2024-04-25T08:31:26.363 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 2543, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 32, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 1927, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 33, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 22, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2024-04-25T08:31:26.364 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2024-04-25T08:31:26.365 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 41, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 3, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 8, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 8, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 96, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 0, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 28 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 1, 2024-04-25T08:31:26.366 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 32, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed_ops": 1927, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 1927, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.367 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 31559, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 7645772, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 31559, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 31559, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 7645772, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 4653, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 92379644, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4653, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 92379644, 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:26.369 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 4653, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 4653, 2024-04-25T08:31:26.370 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4653, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 4653, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 1927, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 210043, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 1927, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:31:26.371 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 51, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 210043, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x5595b54826e0": { 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 84, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 24423, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 91678928, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 24423, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:31:26.372 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 2492, 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 91678844, 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.373 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:31:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa3c06b4d0 msgr2=0x7faa3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa3c06b4d0 0x7faa3c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0b16d0 tx=0x7faa44009040 comp rx=0 tx=0).stop 2024-04-25T08:31:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 msgr2=0x7faa5c1484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 0x7faa5c1484a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faa4c02a2b0 tx=0x7faa4c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 shutdown_connections 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa5c15dbb0 0x7faa5c06ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa5c0cbf00 0x7faa5c14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa3c06b4d0 0x7faa3c06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 --2- 172.21.15.60:0/1767675791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa5c06c300 0x7faa5c1484a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 >> 172.21.15.60:0/1767675791 conn(0x7faa5c071af0 msgr2=0x7faa5c15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.331+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 shutdown_connections 2024-04-25T08:31:26.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.332+0000 7faa60e9a640 1 -- 172.21.15.60:0/1767675791 wait complete. 2024-04-25T08:31:26.376 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T08:31:26.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:26.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T08:31:26.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.485+0000 7fcb0a807640 1 Processor -- start 2024-04-25T08:31:26.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.485+0000 7fcb0a807640 1 -- start start 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.485+0000 7fcb0a807640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb0415b7d0 0x7fcb040cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.485+0000 7fcb0a807640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb0406c9e0 0x7fcb040b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.485+0000 7fcb0a807640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0406cf50 con 0x7fcb040b5620 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb0a807640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb040ce530 con 0x7fcb0415b7d0 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb0a807640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040ce810 con 0x7fcb0406c9e0 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb0415b7d0 0x7fcb040cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb0415b7d0 0x7fcb040cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49068/0 (socket says 172.21.15.60:49068) 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 -- 172.21.15.60:0/472742595 learned_addr learned my addr 172.21.15.60:0/472742595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb08da4640 1 --2- 172.21.15.60:0/472742595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb0406c9e0 0x7fcb040b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 -- 172.21.15.60:0/472742595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb0406c9e0 msgr2=0x7fcb040b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 --2- 172.21.15.60:0/472742595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb0406c9e0 0x7fcb040b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 -- 172.21.15.60:0/472742595 >> v1:172.21.15.100:6789/0 conn(0x7fcb040b5620 legacy=0x7fcb0406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.486+0000 7fcb03fff640 1 -- 172.21.15.60:0/472742595 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040ceaf0 con 0x7fcb0415b7d0 2024-04-25T08:31:26.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb03fff640 1 --2- 172.21.15.60:0/472742595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040cf010 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009870 tx=0x7fcaf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989ef66ef610712e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb02ffd640 1 -- 172.21.15.60:0/472742595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4037070 con 0x7fcb0415b7d0 2024-04-25T08:31:26.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb02ffd640 1 -- 172.21.15.60:0/472742595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4032440 con 0x7fcb0415b7d0 2024-04-25T08:31:26.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb02ffd640 1 -- 172.21.15.60:0/472742595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4032810 con 0x7fcb0415b7d0 2024-04-25T08:31:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 -- 172.21.15.60:0/472742595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 msgr2=0x7fcb040cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 --2- 172.21.15.60:0/472742595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040cf010 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009870 tx=0x7fcaf402a270 comp rx=0 tx=0).stop 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 -- 172.21.15.60:0/472742595 shutdown_connections 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 --2- 172.21.15.60:0/472742595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb0406c9e0 0x7fcb040b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 --2- 172.21.15.60:0/472742595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040cf010 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 -- 172.21.15.60:0/472742595 >> 172.21.15.60:0/472742595 conn(0x7fcb040d0800 msgr2=0x7fcb040d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 -- 172.21.15.60:0/472742595 shutdown_connections 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.487+0000 7fcb0a807640 1 -- 172.21.15.60:0/472742595 wait complete. 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 Processor -- start 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 -- start start 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb0406c9e0 0x7fcb040c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb040b5620 0x7fcb040d2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040cf6d0 con 0x7fcb0415b7d0 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040cf550 con 0x7fcb0406c9e0 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.489+0000 7fcb0a807640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0415bbb0 con 0x7fcb040b5620 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49078/0 (socket says 172.21.15.60:49078) 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 -- 172.21.15.60:0/1903676918 learned_addr learned my addr 172.21.15.60:0/1903676918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb0406c9e0 0x7fcb040c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb037fe640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb040b5620 0x7fcb040d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 -- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb040b5620 msgr2=0x7fcb040d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb040b5620 0x7fcb040d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 -- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb0406c9e0 msgr2=0x7fcb040c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb0406c9e0 0x7fcb040c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 -- 172.21.15.60:0/1903676918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04148410 con 0x7fcb0415b7d0 2024-04-25T08:31:26.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb0406c9e0 0x7fcb040c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb08da4640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040b4550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007d90 tx=0x7fcafc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb00ff9640 1 -- 172.21.15.60:0/1903676918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc014420 con 0x7fcb0415b7d0 2024-04-25T08:31:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb00ff9640 1 -- 172.21.15.60:0/1903676918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc019070 con 0x7fcb0415b7d0 2024-04-25T08:31:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb041486f0 con 0x7fcb0415b7d0 2024-04-25T08:31:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb00ff9640 1 -- 172.21.15.60:0/1903676918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc014820 con 0x7fcb0415b7d0 2024-04-25T08:31:26.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.490+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb04148cb0 con 0x7fcb0415b7d0 2024-04-25T08:31:26.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.492+0000 7fcb00ff9640 1 -- 172.21.15.60:0/1903676918 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcafc0149c0 con 0x7fcb0415b7d0 2024-04-25T08:31:26.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.492+0000 7fcb00ff9640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.492+0000 7fcb00ff9640 1 -- 172.21.15.60:0/1903676918 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcafc08aa60 con 0x7fcb0415b7d0 2024-04-25T08:31:26.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.492+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.493+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaf006b4d0 0x7fcaf006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4034000 tx=0x7fcaf402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.493+0000 7fcb0d5e2740 1 Processor -- start 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.493+0000 7fcb0d5e2740 1 -- start start 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f83df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f83e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f80e370 0x55f77f811b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77f776bc0 con 0x55f77f83b710 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77f71fe70 con 0x55f77f83baf0 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f77f772c60 con 0x55f77f80e370 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f83df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f83df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49084/0 (socket says 172.21.15.60:49084) 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 -- 172.21.15.60:0/1530413252 learned_addr learned my addr 172.21.15.60:0/1530413252 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb037fe640 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f83e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb08da4640 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f80e370 0x55f77f811b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 -- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f80e370 msgr2=0x55f77f811b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f80e370 0x55f77f811b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 -- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 msgr2=0x55f77f83e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f83e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 -- 172.21.15.60:0/1530413252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f77f740ad0 con 0x55f77f83b710 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.494+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f83df20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcaf402a2b0 tx=0x7fcaf402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e35c92be17f6403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcae9ffb640 1 -- 172.21.15.60:0/1530413252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4037040 con 0x55f77f83b710 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcae9ffb640 1 -- 172.21.15.60:0/1530413252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf404d070 con 0x55f77f83b710 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcae9ffb640 1 -- 172.21.15.60:0/1530413252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf403c840 con 0x55f77f83b710 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1530413252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 msgr2=0x55f77f83df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f83df20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcaf402a2b0 tx=0x7fcaf402c000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1530413252 shutdown_connections 2024-04-25T08:31:26.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f80e370 0x55f77f811b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f83e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1530413252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f83df20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1530413252 >> 172.21.15.60:0/1530413252 conn(0x55f77f8105e0 msgr2=0x55f77f858f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1530413252 shutdown_connections 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.495+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1530413252 wait complete. 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.496+0000 7fcb0d5e2740 1 -- start start 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.496+0000 7fcb0d5e2740 1 Processor -- start 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f859920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f8329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f854270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77f813790 con 0x55f77f83b710 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77f7154e0 con 0x55f77f83baf0 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f77f813a70 con 0x55f77f85a050 2024-04-25T08:31:26.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb0d5e2740 10 client.0 _ll_register_callbacks cb 0x55f77f732de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f859920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f8329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f859920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49098/0 (socket says 172.21.15.60:49098) 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb08da4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f854270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f8329c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42106/0 (socket says 172.21.15.60:42106) 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 -- 172.21.15.60:0/1842107044 learned_addr learned my addr 172.21.15.60:0/1842107044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 -- 172.21.15.60:0/1842107044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 msgr2=0x55f77f854270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f854270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 -- 172.21.15.60:0/1842107044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 msgr2=0x55f77f8329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f8329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 -- 172.21.15.60:0/1842107044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f77f557940 con 0x55f77f83b710 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcb03fff640 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f859920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009990 tx=0x7fcaf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f703b07e4f097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.497+0000 7fcae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcae2ffd640 1 -- 172.21.15.60:0/1842107044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4056060 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcae2ffd640 1 -- 172.21.15.60:0/1842107044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcaf4003a20 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaf404d070 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcae2ffd640 1 -- 172.21.15.60:0/1842107044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf405c050 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 10 client.5628 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f77f5b80d0 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcae2ffd640 1 -- 172.21.15.60:0/1842107044 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf404d070 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcae2ffd640 1 -- 172.21.15.60:0/1842107044 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcaf405f020 con 0x55f77f83b710 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 10 client.5628 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 10 client.5628 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 20 client.5628 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f8547b0 0x55f77f851940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 4 client.5628 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:31:26.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f77f784100 con 0x55f77f8547b0 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.498+0000 7fcb037fe640 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f8547b0 0x55f77f851940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcae2ffd640 10 client.5628 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb037fe640 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f8547b0 0x55f77f851940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a00b2b87fd833 server_cookie=453f928b8c5f0c74 in_seq=0 out_seq=0 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcae2ffd640 1 -- 172.21.15.60:0/1842107044 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f77f784100 con 0x55f77f8547b0 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcae2ffd640 10 client.5628 handle_command_reply: tid=0 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 client.5628 shutdown 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 msgr2=0x55f77f859920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f859920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009990 tx=0x7fcaf4004730 comp rx=0 tx=0).stop 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 shutdown_connections 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f8547b0 0x55f77f851940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f854270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcae2ffd640 0 client.5628 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83baf0 0x55f77f8329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/1842107044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83b710 0x55f77f859920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 >> 172.21.15.60:0/1842107044 conn(0x55f77f80e370 msgr2=0x55f77f80f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 shutdown_connections 2024-04-25T08:31:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.499+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/1842107044 wait complete. 2024-04-25T08:31:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.500+0000 7fcb0d5e2740 20 client.5628 trim_cache size 0 max 16384 2024-04-25T08:31:26.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 Processor -- start 2024-04-25T08:31:26.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 -- start start 2024-04-25T08:31:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f841670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f8be2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04200540 con 0x55f77f83baf0 2024-04-25T08:31:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041ff740 con 0x55f77f83b710 2024-04-25T08:31:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.515+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04147370 con 0x55f77f85a050 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f841670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f841670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49110/0 (socket says 172.21.15.60:49110) 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 -- 172.21.15.60:0/2318697305 learned_addr learned my addr 172.21.15.60:0/2318697305 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 -- 172.21.15.60:0/2318697305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 msgr2=0x55f77f8be2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f8be2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 -- 172.21.15.60:0/2318697305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 msgr2=0x55f77f8590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 -- 172.21.15.60:0/2318697305 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f77f44b230 con 0x55f77f83baf0 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcb037fe640 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f841670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000f60 tx=0x7fcaf8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980cc3077b7f9e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcae97fa640 1 -- 172.21.15.60:0/2318697305 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8001cb0 con 0x55f77f83baf0 2024-04-25T08:31:26.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcae97fa640 1 -- 172.21.15.60:0/2318697305 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8023070 con 0x55f77f83baf0 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.516+0000 7fcae97fa640 1 -- 172.21.15.60:0/2318697305 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8005350 con 0x55f77f83baf0 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/2318697305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 msgr2=0x55f77f841670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f841670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000f60 tx=0x7fcaf8009040 comp rx=0 tx=0).stop 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/2318697305 shutdown_connections 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f8be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f841670 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/2318697305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/2318697305 >> 172.21.15.60:0/2318697305 conn(0x55f77f80e370 msgr2=0x55f77f84fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/2318697305 shutdown_connections 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/2318697305 wait complete. 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 -- start start 2024-04-25T08:31:26.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.517+0000 7fcb0d5e2740 1 Processor -- start 2024-04-25T08:31:26.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f86b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f838f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb041ff140 con 0x55f77f83baf0 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f77f84ebc0 con 0x55f77f83b710 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f77f86bab0 con 0x55f77f85a050 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb0d5e2740 10 client.0 _ll_register_callbacks cb 0x7fcaf8001260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f86b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f86b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49116/0 (socket says 172.21.15.60:49116) 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.518+0000 7fcb037fe640 1 -- 172.21.15.60:0/3519308494 learned_addr learned my addr 172.21.15.60:0/3519308494 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb08da4640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f838f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb03fff640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb037fe640 1 -- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 msgr2=0x55f77f838f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb037fe640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f838f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb037fe640 1 -- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 msgr2=0x55f77f8bd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb037fe640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8bd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb037fe640 1 -- 172.21.15.60:0/3519308494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f77f851560 con 0x55f77f83baf0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb03fff640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8bd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb08da4640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f838f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb037fe640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f86b190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcaf801f000 tx=0x7fcaf403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9cadb8b0330647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcae2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcae2ffd640 1 -- 172.21.15.60:0/3519308494 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80179e0 con 0x55f77f83baf0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcae2ffd640 1 -- 172.21.15.60:0/3519308494 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcaf8017b80 con 0x55f77f83baf0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaf8023070 con 0x55f77f83baf0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcae2ffd640 1 -- 172.21.15.60:0/3519308494 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf8017d80 con 0x55f77f83baf0 2024-04-25T08:31:26.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 10 client.5634 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f77f86b6d0 con 0x55f77f83baf0 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcae2ffd640 1 -- 172.21.15.60:0/3519308494 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8023070 con 0x55f77f83baf0 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcae2ffd640 1 -- 172.21.15.60:0/3519308494 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcaf80067d0 con 0x55f77f83baf0 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 10 client.5634 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 10 client.5634 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 20 client.5634 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f839d90 0x55f77f83a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 4 client.5634 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.519+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55f77f77e450 con 0x55f77f839d90 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.520+0000 7fcb03fff640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f839d90 0x55f77f83a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.520+0000 7fcae2ffd640 10 client.5634 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.520+0000 7fcb03fff640 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f839d90 0x55f77f83a170 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9904369fe32e12c0 server_cookie=a405bdced9cc3ea0 in_seq=0 out_seq=0 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcae2ffd640 1 -- 172.21.15.60:0/3519308494 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f77f77e450 con 0x55f77f839d90 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcae2ffd640 10 client.5634 handle_command_reply: tid=0 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 client.5634 shutdown 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 msgr2=0x55f77f86b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f86b190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcaf801f000 tx=0x7fcaf403c000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 shutdown_connections 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55f77f839d90 0x55f77f83a170 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f77f85a050 0x55f77f838f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f77f83baf0 0x55f77f86b190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 --2- 172.21.15.60:0/3519308494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f77f83b710 0x55f77f8bd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 >> 172.21.15.60:0/3519308494 conn(0x55f77f80e370 msgr2=0x55f77f87b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcae2ffd640 0 client.5634 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 shutdown_connections 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 1 -- 172.21.15.60:0/3519308494 wait complete. 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.525+0000 7fcb0d5e2740 20 client.5634 trim_cache size 0 max 16384 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "f6d239eb-f7d1-4f90-a96e-ebbb84f7ffea", 2024-04-25T08:31:26.524 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:31:26.525 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:31:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaf006b4d0 msgr2=0x7fcaf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaf006b4d0 0x7fcaf006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4034000 tx=0x7fcaf402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 msgr2=0x7fcb040b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040b4550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007d90 tx=0x7fcafc00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 shutdown_connections 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb0415b7d0 0x7fcb040b4550 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb040b5620 0x7fcb040d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 --2- 172.21.15.60:0/1903676918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb0406c9e0 0x7fcb040c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 >> 172.21.15.60:0/1903676918 conn(0x7fcb040d0800 msgr2=0x7fcb0415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 shutdown_connections 2024-04-25T08:31:26.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.526+0000 7fcb0a807640 1 -- 172.21.15.60:0/1903676918 wait complete. 2024-04-25T08:31:26.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:26.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.667+0000 7f878e01c640 1 Processor -- start 2024-04-25T08:31:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.667+0000 7f878e01c640 1 -- start start 2024-04-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.668+0000 7f878e01c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8788162200 0x7f87880d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f878e01c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87880ce8a0 con 0x7f87880ce330 2024-04-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f878e01c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87880ceb80 con 0x7f878806b2f0 2024-04-25T08:31:26.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f878e01c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880cf4b0 con 0x7f8788162200 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f8787fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f878806b2f0 0x7f87880cadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:58516/0 (socket says 172.21.15.60:58516) 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f8787fff640 1 -- 172.21.15.60:0/1251901494 learned_addr learned my addr 172.21.15.60:0/1251901494 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87877fe640 1 --2- 172.21.15.60:0/1251901494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8788162200 0x7f87880d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87867fc640 1 -- 172.21.15.60:0/1251901494 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121631895 0 0) 0x7f87880ceb80 con 0x7f878806b2f0 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87867fc640 1 -- 172.21.15.60:0/1251901494 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8770003520 con 0x7f878806b2f0 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87877fe640 1 -- 172.21.15.60:0/1251901494 >> v1:172.21.15.100:6789/0 conn(0x7f878806b2f0 legacy=0x7f87880cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87867fc640 1 -- 172.21.15.60:0/1251901494 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078473492 0 0) 0x7f87880ce8a0 con 0x7f87880ce330 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87877fe640 1 -- 172.21.15.60:0/1251901494 >> v1:172.21.15.60:6789/0 conn(0x7f87880ce330 legacy=0x7f878806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.669+0000 7f87877fe640 1 -- 172.21.15.60:0/1251901494 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880cee60 con 0x7f8788162200 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.670+0000 7f87877fe640 1 --2- 172.21.15.60:0/1251901494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8788162200 0x7f87880d0200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87780097d0 tx=0x7f877802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8546b7f3b6951b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f87867fc640 1 -- 172.21.15.60:0/1251901494 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778037070 con 0x7f8788162200 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f87867fc640 1 -- 172.21.15.60:0/1251901494 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778032440 con 0x7f8788162200 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f87867fc640 1 -- 172.21.15.60:0/1251901494 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87780327f0 con 0x7f8788162200 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f878e01c640 1 -- 172.21.15.60:0/1251901494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8788162200 msgr2=0x7f87880d0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f878e01c640 1 --2- 172.21.15.60:0/1251901494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8788162200 0x7f87880d0200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f87780097d0 tx=0x7f877802a290 comp rx=0 tx=0).stop 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f878e01c640 1 -- 172.21.15.60:0/1251901494 shutdown_connections 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f878e01c640 1 --2- 172.21.15.60:0/1251901494 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8788162200 0x7f87880d0200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.671+0000 7f878e01c640 1 -- 172.21.15.60:0/1251901494 >> 172.21.15.60:0/1251901494 conn(0x7f87880b2390 msgr2=0x7f87880b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.672+0000 7f878e01c640 1 -- 172.21.15.60:0/1251901494 shutdown_connections 2024-04-25T08:31:26.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.672+0000 7f878e01c640 1 -- 172.21.15.60:0/1251901494 wait complete. 2024-04-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.673+0000 7f878e01c640 1 Processor -- start 2024-04-25T08:31:26.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.674+0000 7f878e01c640 1 -- start start 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f878e01c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 0x7f87880b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f878e01c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87880ce330 0x7f878815d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f878e01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8788162200 0x7f87880c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 0x7f87880b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 0x7f87880b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42128/0 (socket says 172.21.15.60:42128) 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f87877fe640 1 -- 172.21.15.60:0/2473555306 learned_addr learned my addr 172.21.15.60:0/2473555306 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f8787fff640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8788162200 0x7f87880c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f8786ffd640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87880ce330 0x7f878815d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878806ba40 con 0x7f8788162200 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878806b8c0 con 0x7f878806b2f0 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.675+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87881625e0 con 0x7f87880ce330 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f87877fe640 1 -- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87880ce330 msgr2=0x7f878815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f87877fe640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87880ce330 0x7f878815d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f87877fe640 1 -- 172.21.15.60:0/2473555306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8788162200 msgr2=0x7f87880c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f87877fe640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8788162200 0x7f87880c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f87877fe640 1 -- 172.21.15.60:0/2473555306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880b38c0 con 0x7f878806b2f0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f87877fe640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 0x7f87880b4920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8778035040 tx=0x7f87780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.676+0000 7f876ffff640 1 -- 172.21.15.60:0/2473555306 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778041020 con 0x7f878806b2f0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.677+0000 7f876ffff640 1 -- 172.21.15.60:0/2473555306 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877803d6e0 con 0x7f878806b2f0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.677+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87880b3b60 con 0x7f878806b2f0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.677+0000 7f876ffff640 1 -- 172.21.15.60:0/2473555306 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778037040 con 0x7f878806b2f0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.677+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87880b4120 con 0x7f878806b2f0 2024-04-25T08:31:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.679+0000 7f876ffff640 1 -- 172.21.15.60:0/2473555306 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8778044020 con 0x7f878806b2f0 2024-04-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.679+0000 7f876ffff640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.680+0000 7f876ffff640 1 -- 172.21.15.60:0/2473555306 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f87780aa990 con 0x7f878806b2f0 2024-04-25T08:31:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.680+0000 7f8786ffd640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.680+0000 7f8786ffd640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f877406b4d0 0x7f877406d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f878815dab0 tx=0x7f877c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.681+0000 7f8790df7740 1 Processor -- start 2024-04-25T08:31:26.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.682+0000 7f8790df7740 1 -- start start 2024-04-25T08:31:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.683+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cbf3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.683+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.683+0000 7f8790df7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbc3770 0x562a7cbc6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8790df7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7cb2cda0 con 0x562a7cbc3770 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cbf3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:49196/0 (socket says 172.21.15.60:49196) 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 -- 172.21.15.60:0/1583744629 learned_addr learned my addr 172.21.15.60:0/1583744629 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cbf3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42140/0 (socket says 172.21.15.60:42140) 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8787fff640 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbc3770 0x562a7cbc6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7cad5b80 con 0x562a7cbf0b10 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a7cb28e40 con 0x562a7cbf0ef0 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 -- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 msgr2=0x562a7cbf3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cbf3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 -- 172.21.15.60:0/1583744629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbc3770 msgr2=0x562a7cbc6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbc3770 0x562a7cbc6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8786ffd640 1 -- 172.21.15.60:0/1583744629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a7cadff10 con 0x562a7cbf0ef0 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.684+0000 7f8787fff640 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbc3770 0x562a7cbc6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.685+0000 7f87877fe640 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cbf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.685+0000 7f8786ffd640 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf3860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f877c007f70 tx=0x7f877c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a39d7868e342c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.685+0000 7f876d7fa640 1 -- 172.21.15.60:0/1583744629 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c025070 con 0x562a7cbf0ef0 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.685+0000 7f876d7fa640 1 -- 172.21.15.60:0/1583744629 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c01b050 con 0x562a7cbf0ef0 2024-04-25T08:31:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.685+0000 7f876d7fa640 1 -- 172.21.15.60:0/1583744629 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c014e20 con 0x562a7cbf0ef0 2024-04-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 msgr2=0x562a7cbf3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf3860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f877c007f70 tx=0x7f877c012000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 shutdown_connections 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbc3770 0x562a7cbc6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf3860 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 --2- 172.21.15.60:0/1583744629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cbf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.686+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 >> 172.21.15.60:0/1583744629 conn(0x562a7cbc59e0 msgr2=0x562a7cc0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.687+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 shutdown_connections 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.687+0000 7f8790df7740 1 -- 172.21.15.60:0/1583744629 wait complete. 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.687+0000 7f8790df7740 1 -- start start 2024-04-25T08:31:26.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.687+0000 7f8790df7740 1 Processor -- start 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f8790df7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 0x562a7cbe77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cc0f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cc0f990 0x562a7cbee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f8790df7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7cbc8b90 con 0x562a7cbf0b10 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f87877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 0x562a7cbe77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f87877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 0x562a7cbe77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49136/0 (socket says 172.21.15.60:49136) 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f87877fe640 1 -- 172.21.15.60:0/2344459136 learned_addr learned my addr 172.21.15.60:0/2344459136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f8786ffd640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cc0f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.689+0000 7f8787fff640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cc0f990 0x562a7cbee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7cacb4b0 con 0x562a7cc0f990 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a7cbc8e70 con 0x562a7cbf0ef0 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f8790df7740 10 client.0 _ll_register_callbacks cb 0x562a7cb0b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87877fe640 1 -- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 msgr2=0x562a7cc0f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87877fe640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cc0f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87877fe640 1 -- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cc0f990 msgr2=0x562a7cbee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87877fe640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cc0f990 0x562a7cbee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87877fe640 1 -- 172.21.15.60:0/2344459136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a7c90b7d0 con 0x562a7cbf0b10 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87877fe640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 0x562a7cbe77e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8778075500 tx=0x7f8778075400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e276cb2bc051b781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.690+0000 7f87667fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f87667fc640 1 -- 172.21.15.60:0/2344459136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87780a6020 con 0x562a7cbf0b10 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f877c01b050 con 0x562a7cbf0b10 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f87667fc640 1 -- 172.21.15.60:0/2344459136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f877803ddd0 con 0x562a7cbf0b10 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f87667fc640 1 -- 172.21.15.60:0/2344459136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8778081700 con 0x562a7cbf0b10 2024-04-25T08:31:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 10 client.5646 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x562a7c8413f0 con 0x562a7cbf0b10 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f87667fc640 1 -- 172.21.15.60:0/2344459136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c01b050 con 0x562a7cbf0b10 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f87667fc640 1 -- 172.21.15.60:0/2344459136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f877803a030 con 0x562a7cbf0b10 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 10 client.5646 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 10 client.5646 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 20 client.5646 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbebf00 0x562a7cbec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 4 client.5646 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a7cb3a2e0 con 0x562a7cbebf00 2024-04-25T08:31:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.691+0000 7f8786ffd640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbebf00 0x562a7cbec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.692+0000 7f8786ffd640 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbebf00 0x562a7cbec300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4477489654a10a2f server_cookie=81d0817c388a328b in_seq=0 out_seq=0 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.692+0000 7f87667fc640 10 client.5646 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f87667fc640 1 -- 172.21.15.60:0/2344459136 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a7cb3a2e0 con 0x562a7cbebf00 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f87667fc640 10 client.5646 handle_command_reply: tid=0 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 client.5646 shutdown 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 msgr2=0x562a7cbe77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 0x562a7cbe77e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8778075500 tx=0x7f8778075400 comp rx=0 tx=0).stop 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 shutdown_connections 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbebf00 0x562a7cbec300 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cc0f990 0x562a7cbee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f87667fc640 0 client.5646 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cc0f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 --2- 172.21.15.60:0/2344459136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0b10 0x562a7cbe77e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 >> 172.21.15.60:0/2344459136 conn(0x562a7cbc3770 msgr2=0x562a7cbef190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.693+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 shutdown_connections 2024-04-25T08:31:26.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.694+0000 7f8790df7740 1 -- 172.21.15.60:0/2344459136 wait complete. 2024-04-25T08:31:26.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.694+0000 7f8790df7740 20 client.5646 trim_cache size 0 max 16384 2024-04-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.722+0000 7f8790df7740 1 Processor -- start 2024-04-25T08:31:26.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.722+0000 7f8790df7740 1 -- start start 2024-04-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.723+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 0x562a7cbec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.723+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc0ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8790df7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0ef0 0x562a7cc0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8786ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc0ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 0x562a7cbec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 0x562a7cbec2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:49220/0 (socket says 172.21.15.60:49220) 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8786ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc0ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42160/0 (socket says 172.21.15.60:42160) 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 -- 172.21.15.60:0/2688423430 learned_addr learned my addr 172.21.15.60:0/2688423430 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87882045b0 con 0x562a7cbf0ef0 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880c8600 con 0x562a7cbf0b10 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880c7a00 con 0x562a7cbebf00 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f8787fff640 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0ef0 0x562a7cc0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 -- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 msgr2=0x562a7cc0ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc0ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 -- 172.21.15.60:0/2688423430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0ef0 msgr2=0x562a7cc0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0ef0 0x562a7cc0b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 -- 172.21.15.60:0/2688423430 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a7cc0f7c0 con 0x562a7cbebf00 2024-04-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.724+0000 7f87877fe640 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 0x562a7cbec2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87780818c0 tx=0x7f8778090000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2692955fa8cfb6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.725+0000 7f876cff9640 1 -- 172.21.15.60:0/2688423430 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87780a6040 con 0x562a7cbebf00 2024-04-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.725+0000 7f876cff9640 1 -- 172.21.15.60:0/2688423430 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877803ddf0 con 0x562a7cbebf00 2024-04-25T08:31:26.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.725+0000 7f876cff9640 1 -- 172.21.15.60:0/2688423430 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778078730 con 0x562a7cbebf00 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 msgr2=0x562a7cbec2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 0x562a7cbec2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87780818c0 tx=0x7f8778090000 comp rx=0 tx=0).stop 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 shutdown_connections 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cbf0ef0 0x562a7cc0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc0ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 --2- 172.21.15.60:0/2688423430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbebf00 0x562a7cbec2e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 >> 172.21.15.60:0/2688423430 conn(0x562a7cbc3770 msgr2=0x562a7cbe8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 shutdown_connections 2024-04-25T08:31:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.726+0000 7f8790df7740 1 -- 172.21.15.60:0/2688423430 wait complete. 2024-04-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.727+0000 7f8790df7740 1 -- start start 2024-04-25T08:31:26.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.727+0000 7f8790df7740 1 Processor -- start 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8790df7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8790df7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cc0f990 0x562a7cbc2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8786ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8786ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:49222/0 (socket says 172.21.15.60:49222) 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f87877fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc763f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:42176/0 (socket says 172.21.15.60:42176) 2024-04-25T08:31:26.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8786ffd640 1 -- 172.21.15.60:0/3720706776 learned_addr learned my addr 172.21.15.60:0/3720706776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7cbedf60 con 0x562a7cc0f990 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880c8000 con 0x562a7cbf0b10 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.729+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a7cbbf420 con 0x562a7cbf0ef0 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8787fff640 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cc0f990 0x562a7cbc2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8786ffd640 1 -- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 msgr2=0x562a7cc763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8786ffd640 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8786ffd640 1 -- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cc0f990 msgr2=0x562a7cbc2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8786ffd640 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cc0f990 0x562a7cbc2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8786ffd640 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a7c8010f0 con 0x562a7cbf0ef0 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8790df7740 10 client.0 _ll_register_callbacks cb 0x7f877c017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f877803ddf0 con 0x562a7cbf0ef0 2024-04-25T08:31:26.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f8786ffd640 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf69c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f877c00ffd0 tx=0x7f877c001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=785620465b6b2b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:26.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.730+0000 7f87667fc640 10 client.5000 ms_handle_connect on v2:172.21.15.100:3301/0 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f87667fc640 1 -- 172.21.15.60:0/3720706776 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c025040 con 0x562a7cbf0ef0 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f87667fc640 1 -- 172.21.15.60:0/3720706776 <== mon.2 v2:172.21.15.100:3301/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f877c0304c0 con 0x562a7cbf0ef0 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f87667fc640 1 -- 172.21.15.60:0/3720706776 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f877c030700 con 0x562a7cbf0ef0 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f8790df7740 10 client.5000 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({fsmap=16}) v3 -- 0x562a7cbef2a0 con 0x562a7cbf0ef0 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f87667fc640 1 -- 172.21.15.60:0/3720706776 <== mon.2 v2:172.21.15.100:3301/0 4 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f877c02b040 con 0x562a7cbf0ef0 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f8790df7740 10 client.5000 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:26.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f8790df7740 10 client.5000 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f8790df7740 20 client.5000 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.731+0000 7f8790df7740 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbefa40 0x562a7cbc2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.732+0000 7f87877fe640 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbefa40 0x562a7cbc2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.732+0000 7f8790df7740 4 client.5000 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.732+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562a7cb34630 con 0x562a7cbefa40 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.732+0000 7f87667fc640 10 client.5000 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.732+0000 7f87877fe640 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbefa40 0x562a7cbc2b40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6e01d762c130fa server_cookie=7a5ae5a5e1cc19bc in_seq=0 out_seq=0 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f87667fc640 1 -- 172.21.15.60:0/3720706776 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x562a7cb34630 con 0x562a7cbefa40 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f87667fc640 10 client.5000 handle_command_reply: tid=0 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 client.5000 shutdown 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 msgr2=0x562a7cbf69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf69c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f877c00ffd0 tx=0x7f877c001080 comp rx=0 tx=0).stop 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 shutdown_connections 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x562a7cbefa40 0x562a7cbc2b40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a7cc0f990 0x562a7cbc2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a7cbf0ef0 0x562a7cbf69c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 --2- 172.21.15.60:0/3720706776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a7cbf0b10 0x562a7cc763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f87667fc640 0 client.5000 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.733+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 >> 172.21.15.60:0/3720706776 conn(0x562a7cbc3770 msgr2=0x562a7cbe8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.734+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 shutdown_connections 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.734+0000 7f8790df7740 1 -- 172.21.15.60:0/3720706776 wait complete. 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.734+0000 7f8790df7740 20 client.5000 trim_cache size 0 max 16384 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (599 inodes in the stack)", 2024-04-25T08:31:26.733 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout: "f6d239eb-f7d1-4f90-a96e-ebbb84f7ffea": { 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "f6d239eb-f7d1-4f90-a96e-ebbb84f7ffea", 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:26.734 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f877406b4d0 msgr2=0x7f877406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f877406b4d0 0x7f877406d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f878815dab0 tx=0x7f877c009040 comp rx=0 tx=0).stop 2024-04-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 msgr2=0x7f87880b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:26.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 0x7f87880b4920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8778035040 tx=0x7f87780046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 shutdown_connections 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8788162200 0x7f87880c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87880ce330 0x7f878815d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 --2- 172.21.15.60:0/2473555306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f878806b2f0 0x7f87880b4920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.735+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 >> 172.21.15.60:0/2473555306 conn(0x7f87880b2390 msgr2=0x7f878806faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.736+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 shutdown_connections 2024-04-25T08:31:26.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:26.736+0000 7f878e01c640 1 -- 172.21.15.60:0/2473555306 wait complete. 2024-04-25T08:31:26.747 INFO:tasks.cephfs.filesystem:scrub status for tag:f6d239eb-f7d1-4f90-a96e-ebbb84f7ffea - {'path': '/', 'tag': 'f6d239eb-f7d1-4f90-a96e-ebbb84f7ffea', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T08:31:55.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:55.940 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:56.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:56.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T08:31:56.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.867+0000 7f960dd70640 1 Processor -- start 2024-04-25T08:31:56.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.867+0000 7f960dd70640 1 -- start start 2024-04-25T08:31:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f960dd70640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f960815dac0 0x7f96080d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f960dd70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96080b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f960dd70640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080d2420 con 0x7f96080d1f30 2024-04-25T08:31:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f960dd70640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080d3030 con 0x7f960815dac0 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f960dd70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96080d2700 con 0x7f96080d0ab0 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f9606ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f96080d1f30 0x7f96080d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50906/0 (socket says 172.21.15.60:50906) 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f9606ffd640 1 -- 172.21.15.60:0/604395104 learned_addr learned my addr 172.21.15.60:0/604395104 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f96077fe640 1 --2- 172.21.15.60:0/604395104 >> v2:172.21.15.100:3300/0 conn(0x7f960815dac0 0x7f96080d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f9607fff640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96080b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f96067fc640 1 -- 172.21.15.60:0/604395104 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136462199 0 0) 0x7f96080d2420 con 0x7f96080d1f30 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.868+0000 7f96067fc640 1 -- 172.21.15.60:0/604395104 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f0003520 con 0x7f96080d1f30 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.869+0000 7f96077fe640 1 -- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 msgr2=0x7f96080b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.869+0000 7f96077fe640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96080b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.869+0000 7f96077fe640 1 -- 172.21.15.60:0/604395104 >> v1:172.21.15.60:6789/0 conn(0x7f96080d1f30 legacy=0x7f96080d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.869+0000 7f96077fe640 1 -- 172.21.15.60:0/604395104 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080d29e0 con 0x7f960815dac0 2024-04-25T08:31:56.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.869+0000 7f9607fff640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96080b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.869+0000 7f96077fe640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960815dac0 0x7f96080d19f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95f80097a0 tx=0x7f95f802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac2dc69131f271b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f96067fc640 1 -- 172.21.15.60:0/604395104 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8037070 con 0x7f960815dac0 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f96067fc640 1 -- 172.21.15.60:0/604395104 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8032440 con 0x7f960815dac0 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f96067fc640 1 -- 172.21.15.60:0/604395104 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f80327f0 con 0x7f960815dac0 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 -- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960815dac0 msgr2=0x7f96080d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960815dac0 0x7f96080d19f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95f80097a0 tx=0x7f95f802a210 comp rx=0 tx=0).stop 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 -- 172.21.15.60:0/604395104 shutdown_connections 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96080b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 --2- 172.21.15.60:0/604395104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f960815dac0 0x7f96080d19f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 -- 172.21.15.60:0/604395104 >> 172.21.15.60:0/604395104 conn(0x7f9608061c30 msgr2=0x7f960806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 -- 172.21.15.60:0/604395104 shutdown_connections 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.870+0000 7f960dd70640 1 -- 172.21.15.60:0/604395104 wait complete. 2024-04-25T08:31:56.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.871+0000 7f960dd70640 1 Processor -- start 2024-04-25T08:31:56.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 -- start start 2024-04-25T08:31:56.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96081f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96080d1f30 0x7f96080cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 0x7f96081f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080b97d0 con 0x7f960815dac0 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960815dea0 con 0x7f96080d1f30 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.872+0000 7f960dd70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96080b9950 con 0x7f96080d0ab0 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 0x7f96081f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 0x7f96081f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56426/0 (socket says 172.21.15.60:56426) 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 -- 172.21.15.60:0/3200980535 learned_addr learned my addr 172.21.15.60:0/3200980535 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9606ffd640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96080d1f30 0x7f96080cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f96077fe640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96081f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 -- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 msgr2=0x7f96081f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96081f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 -- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96080d1f30 msgr2=0x7f96080cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96080d1f30 0x7f96080cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 -- 172.21.15.60:0/3200980535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96081468c0 con 0x7f960815dac0 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f9607fff640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 0x7f96081f2c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9600009eb0 tx=0x7f960000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f95effff640 1 -- 172.21.15.60:0/3200980535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600018020 con 0x7f960815dac0 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f95effff640 1 -- 172.21.15.60:0/3200980535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600014480 con 0x7f960815dac0 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f95effff640 1 -- 172.21.15.60:0/3200980535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600014850 con 0x7f960815dac0 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.873+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9608145b00 con 0x7f960815dac0 2024-04-25T08:31:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.874+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96081460c0 con 0x7f960815dac0 2024-04-25T08:31:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.875+0000 7f95effff640 1 -- 172.21.15.60:0/3200980535 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9600005d10 con 0x7f960815dac0 2024-04-25T08:31:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.876+0000 7f95effff640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.876+0000 7f95effff640 1 -- 172.21.15.60:0/3200980535 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f960008b030 con 0x7f960815dac0 2024-04-25T08:31:56.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.876+0000 7f96077fe640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.876+0000 7f96077fe640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95f406b4d0 0x7f95f406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95f8034000 tx=0x7f95f802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.876+0000 7f9610a77740 1 Processor -- start 2024-04-25T08:31:56.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 -- start start 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982b10 0x5647fa985320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982ef0 0x5647fa985860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 0x5647fa958f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fa8beda0 con 0x5647fa955770 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fa867b80 con 0x5647fa982b10 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.877+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647fa8bae40 con 0x5647fa982ef0 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 0x5647fa958f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 0x5647fa958f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56434/0 (socket says 172.21.15.60:56434) 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 -- 172.21.15.60:0/1758098048 learned_addr learned my addr 172.21.15.60:0/1758098048 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f96077fe640 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982b10 0x5647fa985320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9606ffd640 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982ef0 0x5647fa985860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 -- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982ef0 msgr2=0x5647fa985860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982ef0 0x5647fa985860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 -- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982b10 msgr2=0x5647fa985320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982b10 0x5647fa985320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 -- 172.21.15.60:0/1758098048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647fa871f10 con 0x5647fa955770 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f9607fff640 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 0x5647fa958f80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f96000617c0 tx=0x7f9600059750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e137407a62c600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f95ed7fa640 1 -- 172.21.15.60:0/1758098048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600087070 con 0x5647fa955770 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f95ed7fa640 1 -- 172.21.15.60:0/1758098048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600057990 con 0x5647fa955770 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.878+0000 7f95ed7fa640 1 -- 172.21.15.60:0/1758098048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600071070 con 0x5647fa955770 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 -- 172.21.15.60:0/1758098048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 msgr2=0x5647fa958f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 0x5647fa958f80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f96000617c0 tx=0x7f9600059750 comp rx=0 tx=0).stop 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 -- 172.21.15.60:0/1758098048 shutdown_connections 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa955770 0x5647fa958f80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982ef0 0x5647fa985860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 --2- 172.21.15.60:0/1758098048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982b10 0x5647fa985320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 -- 172.21.15.60:0/1758098048 >> 172.21.15.60:0/1758098048 conn(0x5647fa9579e0 msgr2=0x5647fa9a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 -- 172.21.15.60:0/1758098048 shutdown_connections 2024-04-25T08:31:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.879+0000 7f9610a77740 1 -- 172.21.15.60:0/1758098048 wait complete. 2024-04-25T08:31:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 -- start start 2024-04-25T08:31:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 Processor -- start 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa9a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fa95ab90 con 0x5647fa9a1ca0 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fa85d4b0 con 0x5647fa982ef0 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647fa95ae70 con 0x5647fa982b10 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.880+0000 7f9610a77740 10 client.0 _ll_register_callbacks cb 0x5647fa89d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:56.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56442/0 (socket says 172.21.15.60:56442) 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 -- 172.21.15.60:0/2291693208 learned_addr learned my addr 172.21.15.60:0/2291693208 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f96077fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa9a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9606ffd640 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 -- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 msgr2=0x5647fa9a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa9a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 -- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 msgr2=0x5647fa9b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 -- 172.21.15.60:0/2291693208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647fa69d7d0 con 0x5647fa9a1ca0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9607fff640 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3fb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f96000617c0 tx=0x7f9600013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3351badea84097c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f95e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f95e67fc640 1 -- 172.21.15.60:0/2291693208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600087030 con 0x5647fa9a1ca0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f95e67fc640 1 -- 172.21.15.60:0/2291693208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f9600074c90 con 0x5647fa9a1ca0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9600057990 con 0x5647fa9a1ca0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f95e67fc640 1 -- 172.21.15.60:0/2291693208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9600071040 con 0x5647fa9a1ca0 2024-04-25T08:31:56.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9610a77740 10 client.5664 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.881+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5647fa5d33f0 con 0x5647fa9a1ca0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f95e67fc640 1 -- 172.21.15.60:0/2291693208 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600057990 con 0x5647fa9a1ca0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f95e67fc640 1 -- 172.21.15.60:0/2291693208 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9600058ae0 con 0x5647fa9a1ca0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f9610a77740 10 client.5664 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f9610a77740 10 client.5664 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f9610a77740 20 client.5664 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f9610a77740 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9798f0 0x5647fa9be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f9610a77740 4 client.5664 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647fa8cc2e0 con 0x5647fa9798f0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f96077fe640 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9798f0 0x5647fa9be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f96077fe640 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9798f0 0x5647fa9be770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d4995daff07aba server_cookie=8141213d30d822c2 in_seq=0 out_seq=0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.882+0000 7f95e67fc640 10 client.5664 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f95e67fc640 1 -- 172.21.15.60:0/2291693208 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647fa8cc2e0 con 0x5647fa9798f0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f95e67fc640 10 client.5664 handle_command_reply: tid=0 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 client.5664 shutdown 2024-04-25T08:31:56.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 msgr2=0x5647fa9b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3fb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f96000617c0 tx=0x7f9600013000 comp rx=0 tx=0).stop 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 shutdown_connections 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9798f0 0x5647fa9be770 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3fb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f95e67fc640 0 client.5664 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 --2- 172.21.15.60:0/2291693208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa9a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 >> 172.21.15.60:0/2291693208 conn(0x5647fa955770 msgr2=0x5647fa981190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 shutdown_connections 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 1 -- 172.21.15.60:0/2291693208 wait complete. 2024-04-25T08:31:56.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.883+0000 7f9610a77740 20 client.5664 trim_cache size 0 max 16384 2024-04-25T08:31:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.898+0000 7f9610a77740 1 Processor -- start 2024-04-25T08:31:56.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.898+0000 7f9610a77740 1 -- start start 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa95a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9610a77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 0x5647fa985450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9610a77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9608200240 con 0x5647fa982ef0 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081ff400 con 0x5647fa9a1ca0 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96080b5600 con 0x5647fa982b10 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9606ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 0x5647fa985450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9606ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 0x5647fa985450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56458/0 (socket says 172.21.15.60:56458) 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.899+0000 7f9606ffd640 1 -- 172.21.15.60:0/482184790 learned_addr learned my addr 172.21.15.60:0/482184790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f96077fe640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa95a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9606ffd640 1 -- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 msgr2=0x5647fa95a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9607fff640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9606ffd640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa95a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9606ffd640 1 -- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa9a1ca0 msgr2=0x5647fa9b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9606ffd640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9606ffd640 1 -- 172.21.15.60:0/482184790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647fa9a1ad0 con 0x5647fa982ef0 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f96077fe640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa95a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9607fff640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f9606ffd640 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 0x5647fa985450 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007690 tx=0x7f95fc007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4525c6fc51ece88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f95ecff9640 1 -- 172.21.15.60:0/482184790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0018e0 con 0x5647fa982ef0 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f95ecff9640 1 -- 172.21.15.60:0/482184790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc001a80 con 0x5647fa982ef0 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.900+0000 7f95ecff9640 1 -- 172.21.15.60:0/482184790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc016420 con 0x5647fa982ef0 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 -- 172.21.15.60:0/482184790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 msgr2=0x5647fa985450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 0x5647fa985450 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f95fc007690 tx=0x7f95fc007b80 comp rx=0 tx=0).stop 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 -- 172.21.15.60:0/482184790 shutdown_connections 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa9a1ca0 0x5647fa9b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982ef0 0x5647fa985450 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 --2- 172.21.15.60:0/482184790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa982b10 0x5647fa95a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 -- 172.21.15.60:0/482184790 >> 172.21.15.60:0/482184790 conn(0x5647fa955770 msgr2=0x5647fa95ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 -- 172.21.15.60:0/482184790 shutdown_connections 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 -- 172.21.15.60:0/482184790 wait complete. 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 -- start start 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.901+0000 7f9610a77740 1 Processor -- start 2024-04-25T08:31:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 0x5647faa09e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9c2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa9a1ca0 0x5647fa955e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647fa97ae70 con 0x5647fa982b10 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081fee00 con 0x5647fa982ef0 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647fa979cd0 con 0x5647fa9a1ca0 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9610a77740 10 client.0 _ll_register_callbacks cb 0x7f9600009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f96077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 0x5647faa09e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f96077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 0x5647faa09e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56466/0 (socket says 172.21.15.60:56466) 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f96077fe640 1 -- 172.21.15.60:0/154323558 learned_addr learned my addr 172.21.15.60:0/154323558 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9606ffd640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.902+0000 7f9607fff640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa9a1ca0 0x5647fa955e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f96077fe640 1 -- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa9a1ca0 msgr2=0x5647fa955e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f96077fe640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa9a1ca0 0x5647fa955e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f96077fe640 1 -- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 msgr2=0x5647fa9c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f96077fe640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9c2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f96077fe640 1 -- 172.21.15.60:0/154323558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647fa5930f0 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f96077fe640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 0x5647faa09e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f95f80098c0 tx=0x7f95f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0256c4d71450b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f95e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f95e67fc640 1 -- 172.21.15.60:0/154323558 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8037040 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f95e67fc640 1 -- 172.21.15.60:0/154323558 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f95f804f4d0 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95fc001a80 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f95e67fc640 1 -- 172.21.15.60:0/154323558 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95f805e050 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 10 client.5670 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x5647fa9815a0 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f95e67fc640 1 -- 172.21.15.60:0/154323558 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc001a80 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f95e67fc640 1 -- 172.21.15.60:0/154323558 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f95f8061020 con 0x5647fa982b10 2024-04-25T08:31:56.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 10 client.5670 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 10 client.5670 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 20 client.5670 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9a3a70 0x5647fa981d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 4 client.5670 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.903+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647fa8c6630 con 0x5647fa9a3a70 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9606ffd640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9a3a70 0x5647fa981d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f95e67fc640 10 client.5670 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9606ffd640 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9a3a70 0x5647fa981d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4546aee9b1f159a7 server_cookie=6952584294f93f00 in_seq=0 out_seq=0 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f95e67fc640 1 -- 172.21.15.60:0/154323558 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5647fa8c6630 con 0x5647fa9a3a70 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f95e67fc640 10 client.5670 handle_command_reply: tid=0 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 client.5670 shutdown 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 msgr2=0x5647faa09e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 0x5647faa09e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f95f80098c0 tx=0x7f95f8004730 comp rx=0 tx=0).stop 2024-04-25T08:31:56.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 shutdown_connections 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x5647fa9a3a70 0x5647fa981d30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647fa9a1ca0 0x5647fa955e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647fa982ef0 0x5647fa9c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f95e67fc640 0 client.5670 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 --2- 172.21.15.60:0/154323558 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647fa982b10 0x5647faa09e60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 >> 172.21.15.60:0/154323558 conn(0x5647fa955770 msgr2=0x5647fa9b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 shutdown_connections 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.904+0000 7f9610a77740 1 -- 172.21.15.60:0/154323558 wait complete. 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f9610a77740 20 client.5670 trim_cache size 0 max 16384 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:31:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:31:56.908 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:31:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95f406b4d0 msgr2=0x7f95f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95f406b4d0 0x7f95f406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f95f8034000 tx=0x7f95f802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 msgr2=0x7f96081f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 0x7f96081f2c60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9600009eb0 tx=0x7f960000c490 comp rx=0 tx=0).stop 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 shutdown_connections 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f960815dac0 0x7f96081f2c60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96080d1f30 0x7f96080cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 --2- 172.21.15.60:0/3200980535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96080d0ab0 0x7f96081f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 >> 172.21.15.60:0/3200980535 conn(0x7f9608061c30 msgr2=0x7f96080c7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 shutdown_connections 2024-04-25T08:31:56.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:56.905+0000 7f960dd70640 1 -- 172.21.15.60:0/3200980535 wait complete. 2024-04-25T08:31:56.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:31:56.913 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:31:56.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.019+0000 7f4b4572b640 1 Processor -- start 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.019+0000 7f4b4572b640 1 -- start start 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b4572b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b4015dc70 0x7f4b400af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b4572b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4006b2f0 0x7f4b400ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b4572b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b400b56e0 con 0x7f4b4006f190 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b4572b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b400b3590 con 0x7f4b4015dc70 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b4572b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400b37e0 con 0x7f4b4006b2f0 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b3e7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4b4006f190 0x7f4b4006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50912/0 (socket says 172.21.15.60:50912) 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/1002677019 learned_addr learned my addr 172.21.15.60:0/1002677019 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b3effd640 1 --2- 172.21.15.60:0/1002677019 >> v2:172.21.15.100:3300/0 conn(0x7f4b4015dc70 0x7f4b400af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4006b2f0 0x7f4b400ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b3dffb640 1 -- 172.21.15.60:0/1002677019 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320043005 0 0) 0x7f4b400b56e0 con 0x7f4b4006f190 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.020+0000 7f4b3dffb640 1 -- 172.21.15.60:0/1002677019 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b28003520 con 0x7f4b4006f190 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3dffb640 1 -- 172.21.15.60:0/1002677019 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4005826537 0 0) 0x7f4b28003520 con 0x7f4b4006f190 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3effd640 1 -- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4006b2f0 msgr2=0x7f4b400ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3effd640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4006b2f0 0x7f4b400ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3effd640 1 -- 172.21.15.60:0/1002677019 >> v1:172.21.15.60:6789/0 conn(0x7f4b4006f190 legacy=0x7f4b4006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3effd640 1 -- 172.21.15.60:0/1002677019 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400b3ac0 con 0x7f4b4015dc70 2024-04-25T08:31:57.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4006b2f0 0x7f4b400ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.021+0000 7f4b3effd640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4015dc70 0x7f4b400af6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097a0 tx=0x7f4b3002a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37c6fcb4f30dcbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b3dffb640 1 -- 172.21.15.60:0/1002677019 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30037070 con 0x7f4b4015dc70 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b3dffb640 1 -- 172.21.15.60:0/1002677019 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3002aa90 con 0x7f4b4015dc70 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b3dffb640 1 -- 172.21.15.60:0/1002677019 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34001670 con 0x7f4b4015dc70 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 -- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4015dc70 msgr2=0x7f4b400af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4015dc70 0x7f4b400af6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4b300097a0 tx=0x7f4b3002a000 comp rx=0 tx=0).stop 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 -- 172.21.15.60:0/1002677019 shutdown_connections 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4006b2f0 0x7f4b400ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 --2- 172.21.15.60:0/1002677019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4015dc70 0x7f4b400af6c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 -- 172.21.15.60:0/1002677019 >> 172.21.15.60:0/1002677019 conn(0x7f4b400b5af0 msgr2=0x7f4b400b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 -- 172.21.15.60:0/1002677019 shutdown_connections 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.022+0000 7f4b4572b640 1 -- 172.21.15.60:0/1002677019 wait complete. 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 Processor -- start 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 -- start start 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 0x7f4b401576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4006f190 0x7f4b4014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4015dc70 0x7f4b4009cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b400701d0 con 0x7f4b4006b2f0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4015e050 con 0x7f4b4006f190 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b4572b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40070350 con 0x7f4b4015dc70 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b3effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 0x7f4b401576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b3effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 0x7f4b401576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56476/0 (socket says 172.21.15.60:56476) 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.024+0000 7f4b3effd640 1 -- 172.21.15.60:0/999478663 learned_addr learned my addr 172.21.15.60:0/999478663 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4015dc70 0x7f4b4009cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4006f190 0x7f4b4014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3effd640 1 -- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4015dc70 msgr2=0x7f4b4009cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3effd640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4015dc70 0x7f4b4009cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3effd640 1 -- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4006f190 msgr2=0x7f4b4014e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3effd640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4006f190 0x7f4b4014e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3effd640 1 -- 172.21.15.60:0/999478663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4015b700 con 0x7f4b4006b2f0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b3effd640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 0x7f4b401576f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b30035040 tx=0x7f4b300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b277fe640 1 -- 172.21.15.60:0/999478663 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30042020 con 0x7f4b4006b2f0 2024-04-25T08:31:57.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b277fe640 1 -- 172.21.15.60:0/999478663 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3003d670 con 0x7f4b4006b2f0 2024-04-25T08:31:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b277fe640 1 -- 172.21.15.60:0/999478663 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30037040 con 0x7f4b4006b2f0 2024-04-25T08:31:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4015b9e0 con 0x7f4b4006b2f0 2024-04-25T08:31:57.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.025+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4015bfa0 con 0x7f4b4006b2f0 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.027+0000 7f4b277fe640 1 -- 172.21.15.60:0/999478663 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b30045020 con 0x7f4b4006b2f0 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.027+0000 7f4b277fe640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.027+0000 7f4b277fe640 1 -- 172.21.15.60:0/999478663 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f4b300aabf0 con 0x7f4b4006b2f0 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.027+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.028+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4b400aff00 tx=0x7f4b34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.028+0000 7f4b4842a740 1 Processor -- start 2024-04-25T08:31:57.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.028+0000 7f4b4842a740 1 -- start start 2024-04-25T08:31:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d560e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 0x55d81d55f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d589760 0x55d81d589be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b4842a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81d4c1b90 con 0x55d81d59b370 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81d46ae40 con 0x55d81d55eca0 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81d4bdc30 con 0x55d81d589760 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 0x55d81d55f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 0x55d81d55f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56482/0 (socket says 172.21.15.60:56482) 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/2854988440 learned_addr learned my addr 172.21.15.60:0/2854988440 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3effd640 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d560e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d589760 0x55d81d589be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d589760 msgr2=0x55d81d589be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d589760 0x55d81d589be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 msgr2=0x55d81d560e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d560e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/2854988440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81d2a07d0 con 0x55d81d59b370 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.029+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d589760 0x55d81d589be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 0x55d81d55f080 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005b90 tx=0x7f4b34010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acefdb12ba72896d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b24ff9640 1 -- 172.21.15.60:0/2854988440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3400dbb0 con 0x55d81d59b370 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b24ff9640 1 -- 172.21.15.60:0/2854988440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3400dd70 con 0x55d81d59b370 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b24ff9640 1 -- 172.21.15.60:0/2854988440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34020690 con 0x55d81d59b370 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 -- 172.21.15.60:0/2854988440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 msgr2=0x55d81d55f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 0x55d81d55f080 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005b90 tx=0x7f4b34010000 comp rx=0 tx=0).stop 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 -- 172.21.15.60:0/2854988440 shutdown_connections 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d589760 0x55d81d589be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d59b370 0x55d81d55f080 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 --2- 172.21.15.60:0/2854988440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d560e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 -- 172.21.15.60:0/2854988440 >> 172.21.15.60:0/2854988440 conn(0x55d81d526870 msgr2=0x55d81d5a4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 -- 172.21.15.60:0/2854988440 shutdown_connections 2024-04-25T08:31:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.030+0000 7f4b4842a740 1 -- 172.21.15.60:0/2854988440 wait complete. 2024-04-25T08:31:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.031+0000 7f4b4842a740 1 -- start start 2024-04-25T08:31:57.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.031+0000 7f4b4842a740 1 Processor -- start 2024-04-25T08:31:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d5b7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d58a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d58ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81d47e3e0 con 0x55d81d589760 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81d5872b0 con 0x55d81d59b370 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81d5b5cb0 con 0x55d81d55eca0 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b4842a740 10 client.0 _ll_register_callbacks cb 0x55d81d47ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d58a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d58a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56496/0 (socket says 172.21.15.60:56496) 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3726346832 learned_addr learned my addr 172.21.15.60:0/3726346832 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d58ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d5b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 msgr2=0x55d81d5b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d5b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 msgr2=0x55d81d58ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d58ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3726346832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d81d1960f0 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d5b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d58a610 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005880 tx=0x7f4b34012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a590d532f90f7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.032+0000 7f4b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3726346832 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b34030060 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3726346832 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f4b34001610 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b3400dd70 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3726346832 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b34036050 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 10 client.5682 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d81d300f60 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3726346832 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3400dd70 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3726346832 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b34039020 con 0x55d81d589760 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 10 client.5682 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 10 client.5682 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 20 client.5682 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d59ca40 0x55d81d5b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 4 client.5682 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d81d4cf0d0 con 0x55d81d59ca40 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.033+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d59ca40 0x55d81d5b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b1dffb640 10 client.5682 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d59ca40 0x55d81d5b4e40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fab728203295a5e server_cookie=8a3bcd6f0f6b886c in_seq=0 out_seq=0 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3726346832 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d81d4cf0d0 con 0x55d81d59ca40 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b1dffb640 10 client.5682 handle_command_reply: tid=0 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 client.5682 shutdown 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 msgr2=0x55d81d58a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d58a610 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005880 tx=0x7f4b34012000 comp rx=0 tx=0).stop 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 shutdown_connections 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d59ca40 0x55d81d5b4e40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d58ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d58a610 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3726346832 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d5b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 >> 172.21.15.60:0/3726346832 conn(0x55d81d526870 msgr2=0x55d81d587460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b1dffb640 0 client.5682 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 shutdown_connections 2024-04-25T08:31:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 1 -- 172.21.15.60:0/3726346832 wait complete. 2024-04-25T08:31:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.034+0000 7f4b4842a740 20 client.5682 trim_cache size 0 max 16384 2024-04-25T08:31:57.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 Processor -- start 2024-04-25T08:31:57.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 -- start start 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d588b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d5b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d5a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81d58cc60 con 0x55d81d589760 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b40200480 con 0x55d81d59b370 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b401ff680 con 0x55d81d55eca0 2024-04-25T08:31:57.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.049+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d5b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d5b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56506/0 (socket says 172.21.15.60:56506) 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3251539788 learned_addr learned my addr 172.21.15.60:0/3251539788 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3251539788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 msgr2=0x55d81d588b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d588b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3251539788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 msgr2=0x55d81d5a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d5a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3251539788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81d16f1b0 con 0x55d81d589760 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d5b5030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b34007730 tx=0x7f4b3401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a43ba3140167e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b1ffff640 1 -- 172.21.15.60:0/3251539788 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3403f070 con 0x55d81d589760 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b1ffff640 1 -- 172.21.15.60:0/3251539788 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b34020a10 con 0x55d81d589760 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.050+0000 7f4b1ffff640 1 -- 172.21.15.60:0/3251539788 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3402b410 con 0x55d81d589760 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 -- 172.21.15.60:0/3251539788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 msgr2=0x55d81d5b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d5b5030 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b34007730 tx=0x7f4b3401c000 comp rx=0 tx=0).stop 2024-04-25T08:31:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 -- 172.21.15.60:0/3251539788 shutdown_connections 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d59b370 0x55d81d5a78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d5b5030 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3251539788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d55eca0 0x55d81d588b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 -- 172.21.15.60:0/3251539788 >> 172.21.15.60:0/3251539788 conn(0x55d81d526870 msgr2=0x55d81d579060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 -- 172.21.15.60:0/3251539788 shutdown_connections 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 -- 172.21.15.60:0/3251539788 wait complete. 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 -- start start 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.051+0000 7f4b4842a740 1 Processor -- start 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d5be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d588b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d59b370 0x55d81d589050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b401ff080 con 0x55d81d589760 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4009bfc0 con 0x55d81d55eca0 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d81d55da90 con 0x55d81d59b370 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b4842a740 10 client.0 _ll_register_callbacks cb 0x7f4b34005a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:57.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d588b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d588b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56520/0 (socket says 172.21.15.60:56520) 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3881856300 learned_addr learned my addr 172.21.15.60:0/3881856300 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d59b370 msgr2=0x55d81d589050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3f7fe640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d59b370 0x55d81d589050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d5be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d59b370 0x55d81d589050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 msgr2=0x55d81d5be400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d5be400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.052+0000 7f4b3e7fc640 1 -- 172.21.15.60:0/3881856300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d81d59a890 con 0x55d81d589760 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b3e7fc640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d588b10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005880 tx=0x7f4b340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6307d015a27231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3881856300 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3403f040 con 0x55d81d589760 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3881856300 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f4b34046c80 con 0x55d81d589760 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d81d4c9420 con 0x55d81d589760 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3881856300 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b3404d050 con 0x55d81d589760 2024-04-25T08:31:57.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 10 client.5688 fetch_fsmap learned FSMap version 16 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d81d55d7d0 con 0x55d81d589760 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3881856300 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d81d4c9420 con 0x55d81d589760 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3881856300 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b34050020 con 0x55d81d589760 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 10 client.5688 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 10 client.5688 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 20 client.5688 populate_metadata read hostname 'smithi060' 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d55d320 0x55d81d5b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 4 client.5688 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 --> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55d81d447e60 con 0x55d81d55d320 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.053+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d55d320 0x55d81d5b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b1dffb640 10 client.5688 ms_handle_connect on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b3effd640 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d55d320 0x55d81d5b5030 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5d6c1c2d5494e8 server_cookie=d006ca491b7d39f9 in_seq=0 out_seq=0 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b1dffb640 1 -- 172.21.15.60:0/3881856300 <== mds.0 v2:172.21.15.60:6833/2407433231 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17932 (crc 0 0 0) 0x55d81d447e60 con 0x55d81d55d320 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b1dffb640 10 client.5688 handle_command_reply: tid=0 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 client.5688 shutdown 2024-04-25T08:31:57.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 msgr2=0x55d81d588b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d588b10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005880 tx=0x7f4b340057f0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 shutdown_connections 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:6833/2407433231,v1:172.21.15.60:6835/2407433231] conn(0x55d81d55d320 0x55d81d5b5030 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d81d59b370 0x55d81d589050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b1dffb640 0 client.5688 ms_handle_reset on v2:172.21.15.60:6833/2407433231 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d81d589760 0x55d81d588b10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 --2- 172.21.15.60:0/3881856300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d81d55eca0 0x55d81d5be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 >> 172.21.15.60:0/3881856300 conn(0x55d81d526870 msgr2=0x55d81d5a4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 shutdown_connections 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 1 -- 172.21.15.60:0/3881856300 wait complete. 2024-04-25T08:31:57.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.054+0000 7f4b4842a740 20 client.5688 trim_cache size 0 max 16384 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 3711, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 3711, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 935653, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 36469243, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 17, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 6, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.411027431, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.216940984, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.070673605, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.101795874, 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:31:57.063 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3709, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.482488107, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000130085 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 10, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000874, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000087 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2588, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2607, 2024-04-25T08:31:57.064 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 656664, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 22038594, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.303256325, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.166005967, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.046735430, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.071950348, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2606, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.332225061, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000127484 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.065 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 118016 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 29364, 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 51770, 2024-04-25T08:31:57.066 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 9719766, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 53272863, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 2.960020687, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 2.230491039, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.953061835, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.071866189, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 51770, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.297171984, 2024-04-25T08:31:57.067 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000025056 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 48299, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.008280886, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000171 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 120848, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 25232 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2024-04-25T08:31:57.068 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2622, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 17.827052240, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.006799028 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3984, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.487954498, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000373482 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 25267, 2024-04-25T08:31:57.069 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 25267, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 25267, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 467.419286222, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.018499199 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 12, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 17866, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 17866, 2024-04-25T08:31:57.070 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 17866, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 2, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 2, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 2, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 45726, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 45324, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 111, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2024-04-25T08:31:57.071 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 13, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T08:31:57.072 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 2000, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 912, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 4032, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 3672, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 3272, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 62, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 13874 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2024-04-25T08:31:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 4125, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 3927, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 2224, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2024-04-25T08:31:57.074 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "evlrg": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 26431, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 26431, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 35, 2024-04-25T08:31:57.075 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 35, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "segmjr": 4, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 97088678, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 0, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2497, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 133.002754945, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.053265020 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 0 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2024-04-25T08:31:57.076 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 14, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 2016, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 2002, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 13, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 13, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 0, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 17866, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 19867, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 2001, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 2, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 8186, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 8184, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 101824, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 215396 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 28077, 2024-04-25T08:31:57.077 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 25267, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 5, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2000, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.347906597, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000673953 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 29, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.004536469, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000156429 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2024-04-25T08:31:57.078 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 6000, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.953934304, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001158989 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 7001, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.135681435, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000876400 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.080 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000040191, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000040191 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002939576, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002939576 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.081 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.207533556, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.207533556 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.082 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.461151876, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.182643984 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2224, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 434.412748594, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.195329473 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.083 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.050807589, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.050807589 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000260343, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000260343 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2024-04-25T08:31:57.085 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.086 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8000, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 16.841745692, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002105218 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2024-04-25T08:31:57.087 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 1, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 18479, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 18479, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 73, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "mdthresh_evicted": 0 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:31:57.088 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:31:57.089 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:31:57.090 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 250233, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 92, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 440, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 5, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 5624, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 80, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-04-25T08:31:57.091 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 13453133, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 661175, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 8695, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 93599795, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 37, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 8695, 2024-04-25T08:31:57.092 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8695, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 31.892704342, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003667936 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8695, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 8787 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 8695, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 2074, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 6621, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 0, 2024-04-25T08:31:57.093 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 11, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 59, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 2554, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 36, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 1927, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 2034, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 2015, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 22, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:31:57.094 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2024-04-25T08:31:57.095 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 43, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 5, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 8, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 8, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2024-04-25T08:31:57.096 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 100, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 0, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 29 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 1, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.097 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 32, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed_ops": 3927, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 3927, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 35658, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 8526989, 2024-04-25T08:31:57.098 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 35658, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 35658, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 8526989, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:31:57.099 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 8695, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 94011838, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 8695, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 94011838, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:57.100 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 8695, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 8695, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 8695, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 8695, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:57.101 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 3927, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 412043, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 3927, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 57, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 412043, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:31:57.102 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x5595b54826e0": { 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 26431, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 92894374, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 26431, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 2497, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 92894374, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.103 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:31:57.104 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b2c06b4d0 msgr2=0x7f4b2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4b400aff00 tx=0x7f4b34009040 comp rx=0 tx=0).stop 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 msgr2=0x7f4b401576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 0x7f4b401576f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b30035040 tx=0x7f4b300046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 shutdown_connections 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b4015dc70 0x7f4b4009cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b2c06b4d0 0x7f4b2c06d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b4006f190 0x7f4b4014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 --2- 172.21.15.60:0/999478663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b4006b2f0 0x7f4b401576f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 >> 172.21.15.60:0/999478663 conn(0x7f4b400b5af0 msgr2=0x7f4b400b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 shutdown_connections 2024-04-25T08:31:57.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.055+0000 7f4b4572b640 1 -- 172.21.15.60:0/999478663 wait complete. 2024-04-25T08:31:57.106 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:31:57.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:31:57.139 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:31:57.189 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:31:57.189 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:31:57.190 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:57.205 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:31:57.206 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:57.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:57.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.406+0000 7f2655d09640 1 Processor -- start 2024-04-25T08:31:57.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.406+0000 7f2655d09640 1 -- start start 2024-04-25T08:31:57.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f2655d09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 0x7f26500d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f2655d09640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26500b9940 con 0x7f26500d0900 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f2655d09640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26500d1df0 con 0x7f265015dd30 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f2655d09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d0d70 con 0x7f26500d2e90 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f264ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f26500d0900 0x7f26500b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50928/0 (socket says 172.21.15.60:50928) 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f264ffff640 1 -- 172.21.15.60:0/1175457754 learned_addr learned my addr 172.21.15.60:0/1175457754 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412093529 0 0) 0x7f26500b9940 con 0x7f26500d0900 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f264effd640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 0x7f26500d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.407+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2638003520 con 0x7f26500d0900 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 306620950 0 0) 0x7f2638003520 con 0x7f26500d0900 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 msgr2=0x7f26500d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 0x7f26500d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 >> v1:172.21.15.100:6789/0 conn(0x7f265015dd30 legacy=0x7f26500b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500d20d0 con 0x7f26500d0900 2024-04-25T08:31:57.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f26480019f0 con 0x7f26500d0900 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 >> v1:172.21.15.60:6789/0 conn(0x7f26500d0900 legacy=0x7f26500b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26380039e0 0x7f2638003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 0x7f2638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2638004d40 0x7f26380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d0d70 con 0x7f26380039e0 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263800baf0 con 0x7f2638004d40 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263800bcf0 con 0x7f26500d2e90 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264f7fe640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26380039e0 0x7f2638003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264effd640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 0x7f2638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264ffff640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2638004d40 0x7f26380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264f7fe640 1 -- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 msgr2=0x7f2638004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264f7fe640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26500d2e90 0x7f2638004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264f7fe640 1 -- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2638004d40 msgr2=0x7f26380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264f7fe640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2638004d40 0x7f26380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.408+0000 7f264f7fe640 1 -- 172.21.15.60:0/1175457754 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638003520 con 0x7f26380039e0 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.409+0000 7f264f7fe640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26380039e0 0x7f2638003e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2640001750 tx=0x7f2638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1795765eccc72e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.409+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640001370 con 0x7f26380039e0 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.409+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26400052d0 con 0x7f26380039e0 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.409+0000 7f264e7fc640 1 -- 172.21.15.60:0/1175457754 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264002ab20 con 0x7f26380039e0 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f2655d09640 1 -- 172.21.15.60:0/1175457754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26380039e0 msgr2=0x7f2638003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f2655d09640 1 --2- 172.21.15.60:0/1175457754 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26380039e0 0x7f2638003e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2640001750 tx=0x7f2638005830 comp rx=0 tx=0).stop 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f264ffff640 1 -- 172.21.15.60:0/1175457754 reap_dead start 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f2655d09640 1 -- 172.21.15.60:0/1175457754 shutdown_connections 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f2655d09640 1 -- 172.21.15.60:0/1175457754 >> 172.21.15.60:0/1175457754 conn(0x7f2650061c30 msgr2=0x7f265006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f2655d09640 1 -- 172.21.15.60:0/1175457754 shutdown_connections 2024-04-25T08:31:57.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.410+0000 7f2655d09640 1 -- 172.21.15.60:0/1175457754 wait complete. 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.411+0000 7f2655d09640 1 Processor -- start 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.411+0000 7f2655d09640 1 -- start start 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f2655d09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26380039e0 0x7f265014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f2655d09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26501f6e00 0x7f26500c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f2655d09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 0x7f26500b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f2655d09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d2970 con 0x7f26500b6a00 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f2655d09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d2690 con 0x7f26501f6e00 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f2655d09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26500d1a10 con 0x7f26380039e0 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 0x7f26500b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 0x7f26500b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56542/0 (socket says 172.21.15.60:56542) 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 -- 172.21.15.60:0/771688806 learned_addr learned my addr 172.21.15.60:0/771688806 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264effd640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26501f6e00 0x7f26500c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264f7fe640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26380039e0 0x7f265014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 -- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26380039e0 msgr2=0x7f265014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26380039e0 0x7f265014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 -- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26501f6e00 msgr2=0x7f26500c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26501f6e00 0x7f26500c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 -- 172.21.15.60:0/771688806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265014acb0 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.412+0000 7f264ffff640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 0x7f26500b6e80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2648001560 tx=0x7f2648007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.413+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264800ab60 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.413+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2648019070 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.413+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26501f73c0 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.413+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26480133f0 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.413+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26501f7980 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.414+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26500d03c0 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.414+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2648013590 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.415+0000 7f2637fff640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f263c06b500 0x7f263c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.415+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f264808a7d0 con 0x7f26500b6a00 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.415+0000 7f264f7fe640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f263c06b500 0x7f263c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.415+0000 7f264f7fe640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f263c06b500 0x7f263c06d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2640032000 tx=0x7f264002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.419+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26500d03c0 con 0x7f26500b6a00 2024-04-25T08:31:57.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.599+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2650073990 con 0x7f26500b6a00 2024-04-25T08:31:57.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.600+0000 7f2637fff640 1 -- 172.21.15.60:0/771688806 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f265014acb0 con 0x7f26500b6a00 2024-04-25T08:31:57.597 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:31:57.597 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":43,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:31:34.080345+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f263c06b500 msgr2=0x7f263c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f263c06b500 0x7f263c06d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2640032000 tx=0x7f264002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 msgr2=0x7f26500b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 0x7f26500b6e80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2648001560 tx=0x7f2648007a60 comp rx=0 tx=0).stop 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 shutdown_connections 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26500b6a00 0x7f26500b6e80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f263c06b500 0x7f263c06d9c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26501f6e00 0x7f26500c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 --2- 172.21.15.60:0/771688806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26380039e0 0x7f265014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 >> 172.21.15.60:0/771688806 conn(0x7f2650061c30 msgr2=0x7f2650160430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 shutdown_connections 2024-04-25T08:31:57.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.602+0000 7f2655d09640 1 -- 172.21.15.60:0/771688806 wait complete. 2024-04-25T08:31:57.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:30:29.214789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:30:29.624978+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:31:57.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.717+0000 7f0da3b69640 1 Processor -- start 2024-04-25T08:31:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.717+0000 7f0da3b69640 1 -- start start 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da3b69640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d9c0cc4f0 0x7f0d9c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da3b69640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d9c15b980 0x7f0d9c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da3b69640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9c072280 con 0x7f0d9c0cdb20 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da3b69640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9c071270 con 0x7f0d9c15b980 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da3b69640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9c15bd80 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da2106640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d9c15b980 0x7f0d9c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da2106640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d9c15b980 0x7f0d9c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56552/0 (socket says 172.21.15.60:56552) 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da2106640 1 -- 172.21.15.60:0/776970164 learned_addr learned my addr 172.21.15.60:0/776970164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.718+0000 7f0da1905640 1 --2- 172.21.15.60:0/776970164 >> v2:172.21.15.100:3300/0 conn(0x7f0d9c0cc4f0 0x7f0d9c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da2106640 1 -- 172.21.15.60:0/776970164 >> v2:172.21.15.100:3300/0 conn(0x7f0d9c0cc4f0 msgr2=0x7f0d9c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da2106640 1 --2- 172.21.15.60:0/776970164 >> v2:172.21.15.100:3300/0 conn(0x7f0d9c0cc4f0 0x7f0d9c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da2106640 1 -- 172.21.15.60:0/776970164 >> v1:172.21.15.100:6789/0 conn(0x7f0d9c0cdb20 legacy=0x7f0d9c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da2106640 1 -- 172.21.15.60:0/776970164 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c0714c0 con 0x7f0d9c15b980 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da2106640 1 --2- 172.21.15.60:0/776970164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c15b980 0x7f0d9c0cdfb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0cd290 tx=0x7f0d8c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0796d83c653225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da0903640 1 -- 172.21.15.60:0/776970164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00b2e0 con 0x7f0d9c15b980 2024-04-25T08:31:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da0903640 1 -- 172.21.15.60:0/776970164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c037070 con 0x7f0d9c15b980 2024-04-25T08:31:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.719+0000 7f0da0903640 1 -- 172.21.15.60:0/776970164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00b480 con 0x7f0d9c15b980 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 -- 172.21.15.60:0/776970164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c15b980 msgr2=0x7f0d9c0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 --2- 172.21.15.60:0/776970164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c15b980 0x7f0d9c0cdfb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0cd290 tx=0x7f0d8c02c940 comp rx=0 tx=0).stop 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 -- 172.21.15.60:0/776970164 shutdown_connections 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 --2- 172.21.15.60:0/776970164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c15b980 0x7f0d9c0cdfb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 --2- 172.21.15.60:0/776970164 >> v2:172.21.15.100:3300/0 conn(0x7f0d9c0cc4f0 0x7f0d9c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 -- 172.21.15.60:0/776970164 >> 172.21.15.60:0/776970164 conn(0x7f0d9c06abe0 msgr2=0x7f0d9c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 -- 172.21.15.60:0/776970164 shutdown_connections 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.720+0000 7f0da3b69640 1 -- 172.21.15.60:0/776970164 wait complete. 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.721+0000 7f0da3b69640 1 Processor -- start 2024-04-25T08:31:57.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.721+0000 7f0da3b69640 1 -- start start 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da3b69640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 0x7f0d9c150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da3b69640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c0cdb20 0x7f0d9c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da3b69640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d9c15b980 0x7f0d9c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da3b69640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c071d60 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da3b69640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c071a80 con 0x7f0d9c0cdb20 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da3b69640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c0ae500 con 0x7f0d9c15b980 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 0x7f0d9c150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 0x7f0d9c150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56560/0 (socket says 172.21.15.60:56560) 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 -- 172.21.15.60:0/1179171654 learned_addr learned my addr 172.21.15.60:0/1179171654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da2106640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d9c15b980 0x7f0d9c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1104640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c0cdb20 0x7f0d9c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 -- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d9c15b980 msgr2=0x7f0d9c0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d9c15b980 0x7f0d9c0b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 -- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c0cdb20 msgr2=0x7f0d9c1505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.722+0000 7f0da1905640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c0cdb20 0x7f0d9c1505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0da1905640 1 -- 172.21.15.60:0/1179171654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c1e5110 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0da1104640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c0cdb20 0x7f0d9c1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0da1905640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 0x7f0d9c150cd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d5d0 tx=0x7f0d9400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d940125f0 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d94019070 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c1e4350 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d940129f0 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.723+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c1e4910 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.724+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c073a00 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.724+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d94012b90 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.725+0000 7f0d9a7fc640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d8806b4d0 0x7f0d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.725+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f0d9408a740 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.725+0000 7f0da1104640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d8806b4d0 0x7f0d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.725+0000 7f0da1104640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d8806b4d0 0x7f0d8806d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d90007fd0 tx=0x7f0d90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.729+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d94059490 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.907+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d9c06d3a0 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.907+0000 7f0d9a7fc640 1 -- 172.21.15.60:0/1179171654 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d94058d90 con 0x7f0d9c0cc4f0 2024-04-25T08:31:57.904 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d8806b4d0 msgr2=0x7f0d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d8806b4d0 0x7f0d8806d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0d90007fd0 tx=0x7f0d90009040 comp rx=0 tx=0).stop 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 msgr2=0x7f0d9c150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 0x7f0d9c150cd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d5d0 tx=0x7f0d9400da90 comp rx=0 tx=0).stop 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 shutdown_connections 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d9c15b980 0x7f0d9c0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d8806b4d0 0x7f0d8806d990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d9c0cdb20 0x7f0d9c1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 --2- 172.21.15.60:0/1179171654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d9c0cc4f0 0x7f0d9c150cd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 >> 172.21.15.60:0/1179171654 conn(0x7f0d9c06abe0 msgr2=0x7f0d9c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 shutdown_connections 2024-04-25T08:31:57.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:57.909+0000 7f0da3b69640 1 -- 172.21.15.60:0/1179171654 wait complete. 2024-04-25T08:31:57.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:31:58.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.024+0000 7feffcb29640 1 Processor -- start 2024-04-25T08:31:58.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.024+0000 7feffcb29640 1 -- start start 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feffcb29640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feff81446d0 0x7feff80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feffcb29640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80cccc0 con 0x7feff806e8a0 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feffcb29640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80d0460 con 0x7feff8144d50 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feffcb29640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff80b2b20 con 0x7feff81446d0 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feff81446d0 0x7feff80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff6d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feff806e8a0 0x7feff80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50942/0 (socket says 172.21.15.60:50942) 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff6d9d640 1 -- 172.21.15.60:0/1557124425 learned_addr learned my addr 172.21.15.60:0/1557124425 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff659c640 1 -- 172.21.15.60:0/1557124425 >> v1:172.21.15.100:6789/0 conn(0x7feff8144d50 legacy=0x7feff80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff659c640 1 -- 172.21.15.60:0/1557124425 >> v1:172.21.15.60:6789/0 conn(0x7feff806e8a0 legacy=0x7feff80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff559a640 1 -- 172.21.15.60:0/1557124425 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440294939 0 0) 0x7feff80cccc0 con 0x7feff806e8a0 2024-04-25T08:31:58.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.025+0000 7feff659c640 1 -- 172.21.15.60:0/1557124425 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80cc140 con 0x7feff81446d0 2024-04-25T08:31:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.026+0000 7feff659c640 1 --2- 172.21.15.60:0/1557124425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff80b3f20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097a0 tx=0x7fefe802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68af074fd3577f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.026+0000 7feff559a640 1 -- 172.21.15.60:0/1557124425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8037070 con 0x7feff81446d0 2024-04-25T08:31:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.026+0000 7feff559a640 1 -- 172.21.15.60:0/1557124425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8032440 con 0x7feff81446d0 2024-04-25T08:31:58.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.026+0000 7feff559a640 1 -- 172.21.15.60:0/1557124425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8032810 con 0x7feff81446d0 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 -- 172.21.15.60:0/1557124425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 msgr2=0x7feff80b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 --2- 172.21.15.60:0/1557124425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff80b3f20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097a0 tx=0x7fefe802a230 comp rx=0 tx=0).stop 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 -- 172.21.15.60:0/1557124425 shutdown_connections 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 --2- 172.21.15.60:0/1557124425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff80b3f20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 -- 172.21.15.60:0/1557124425 >> 172.21.15.60:0/1557124425 conn(0x7feff806abe0 msgr2=0x7feff806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 -- 172.21.15.60:0/1557124425 shutdown_connections 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.027+0000 7feffcb29640 1 -- 172.21.15.60:0/1557124425 wait complete. 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.028+0000 7feffcb29640 1 Processor -- start 2024-04-25T08:31:58.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.028+0000 7feffcb29640 1 -- start start 2024-04-25T08:31:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feffcb29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff806e8a0 0x7feff813b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feffcb29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feffcb29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8144d50 0x7feff8139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feffcb29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cc700 con 0x7feff81446d0 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feffcb29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff806d400 con 0x7feff8144d50 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feffcb29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cc9e0 con 0x7feff806e8a0 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56588/0 (socket says 172.21.15.60:56588) 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 -- 172.21.15.60:0/3303307267 learned_addr learned my addr 172.21.15.60:0/3303307267 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff659c640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff806e8a0 0x7feff813b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff6d9d640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8144d50 0x7feff8139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 -- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff806e8a0 msgr2=0x7feff813b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff806e8a0 0x7feff813b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 -- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8144d50 msgr2=0x7feff8139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8144d50 0x7feff8139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 -- 172.21.15.60:0/3303307267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff815b240 con 0x7feff81446d0 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.029+0000 7feff5d9b640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff815ad00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c430 tx=0x7fefec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.030+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec017070 con 0x7feff81446d0 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.030+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec0052d0 con 0x7feff81446d0 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.030+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff813a670 con 0x7feff81446d0 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.030+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec003830 con 0x7feff81446d0 2024-04-25T08:31:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.030+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff813ac30 con 0x7feff81446d0 2024-04-25T08:31:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.031+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8073a00 con 0x7feff81446d0 2024-04-25T08:31:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.031+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefec01a070 con 0x7feff81446d0 2024-04-25T08:31:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.032+0000 7fefe6ffd640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fefd806b4d0 0x7fefd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.032+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fefec089690 con 0x7feff81446d0 2024-04-25T08:31:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.032+0000 7feff659c640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fefd806b4d0 0x7fefd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.032+0000 7feff659c640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fefd806b4d0 0x7fefd806d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fefe8034000 tx=0x7fefe802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.036+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feff815b240 con 0x7feff81446d0 2024-04-25T08:31:58.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.218+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feff80afb10 con 0x7feff81446d0 2024-04-25T08:31:58.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.219+0000 7fefe6ffd640 1 -- 172.21.15.60:0/3303307267 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefec087e20 con 0x7feff81446d0 2024-04-25T08:31:58.215 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:31:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fefd806b4d0 msgr2=0x7fefd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fefd806b4d0 0x7fefd806d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fefe8034000 tx=0x7fefe802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 msgr2=0x7feff815ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff815ad00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c430 tx=0x7fefec00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 shutdown_connections 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fefd806b4d0 0x7fefd806d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8144d50 0x7feff8139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feff81446d0 0x7feff815ad00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 --2- 172.21.15.60:0/3303307267 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feff806e8a0 0x7feff813b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 >> 172.21.15.60:0/3303307267 conn(0x7feff806abe0 msgr2=0x7feff806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 shutdown_connections 2024-04-25T08:31:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.221+0000 7feffcb29640 1 -- 172.21.15.60:0/3303307267 wait complete. 2024-04-25T08:31:58.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:31:58.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.336+0000 7fef8c529640 1 Processor -- start 2024-04-25T08:31:58.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.336+0000 7fef8c529640 1 -- start start 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef8c529640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef840b5620 0x7fef840b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef8c529640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8406cf50 con 0x7fef8415b7d0 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef8c529640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef840ce530 con 0x7fef8406c2d0 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef8c529640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef840ce810 con 0x7fef840b5620 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef89ac4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef840b5620 0x7fef840b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef8a2c5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef8415b7d0 0x7fef840cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50952/0 (socket says 172.21.15.60:50952) 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef8a2c5640 1 -- 172.21.15.60:0/1227750050 learned_addr learned my addr 172.21.15.60:0/1227750050 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef89ac4640 1 -- 172.21.15.60:0/1227750050 >> v1:172.21.15.100:6789/0 conn(0x7fef8406c2d0 legacy=0x7fef8406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef89ac4640 1 -- 172.21.15.60:0/1227750050 >> v1:172.21.15.60:6789/0 conn(0x7fef8415b7d0 legacy=0x7fef840cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.337+0000 7fef89ac4640 1 -- 172.21.15.60:0/1227750050 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840ceaf0 con 0x7fef840b5620 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.338+0000 7fef89ac4640 1 --2- 172.21.15.60:0/1227750050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef840b5ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef740097a0 tx=0x7fef7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9529ef2593bdb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.338+0000 7fef892c3640 1 -- 172.21.15.60:0/1227750050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74037070 con 0x7fef840b5620 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.338+0000 7fef892c3640 1 -- 172.21.15.60:0/1227750050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef74032440 con 0x7fef840b5620 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.338+0000 7fef892c3640 1 -- 172.21.15.60:0/1227750050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74032810 con 0x7fef840b5620 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 -- 172.21.15.60:0/1227750050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 msgr2=0x7fef840b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 --2- 172.21.15.60:0/1227750050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef840b5ab0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef740097a0 tx=0x7fef7402a230 comp rx=0 tx=0).stop 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 -- 172.21.15.60:0/1227750050 shutdown_connections 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 --2- 172.21.15.60:0/1227750050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef840b5ab0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 -- 172.21.15.60:0/1227750050 >> 172.21.15.60:0/1227750050 conn(0x7fef840d0800 msgr2=0x7fef840d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 -- 172.21.15.60:0/1227750050 shutdown_connections 2024-04-25T08:31:58.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.339+0000 7fef8c529640 1 -- 172.21.15.60:0/1227750050 wait complete. 2024-04-25T08:31:58.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.340+0000 7fef8c529640 1 Processor -- start 2024-04-25T08:31:58.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.340+0000 7fef8c529640 1 -- start start 2024-04-25T08:31:58.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8c529640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8406c2d0 0x7fef841f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8c529640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef841f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8c529640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8415b7d0 0x7fef8415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8c529640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8415bd30 con 0x7fef840b5620 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8c529640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8415bbb0 con 0x7fef8406c2d0 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8c529640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef840d1f40 con 0x7fef8415b7d0 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef841f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef841f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56608/0 (socket says 172.21.15.60:56608) 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 -- 172.21.15.60:0/1800454999 learned_addr learned my addr 172.21.15.60:0/1800454999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8a2c5640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8406c2d0 0x7fef841f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8aac6640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8415b7d0 0x7fef8415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 -- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8415b7d0 msgr2=0x7fef8415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8415b7d0 0x7fef8415fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 -- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8406c2d0 msgr2=0x7fef841f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8406c2d0 0x7fef841f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 -- 172.21.15.60:0/1800454999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8414b7a0 con 0x7fef840b5620 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef8a2c5640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8406c2d0 0x7fef841f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef89ac4640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef841f67f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fef74035040 tx=0x7fef740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.341+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74037040 con 0x7fef840b5620 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.342+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef74030770 con 0x7fef840b5620 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.342+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8414ba80 con 0x7fef840b5620 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.342+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef74030b80 con 0x7fef840b5620 2024-04-25T08:31:58.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.342+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8414c040 con 0x7fef840b5620 2024-04-25T08:31:58.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.342+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef840d3980 con 0x7fef840b5620 2024-04-25T08:31:58.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.343+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef74044020 con 0x7fef840b5620 2024-04-25T08:31:58.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.343+0000 7fef7affd640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.344+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fef740aa9a0 con 0x7fef840b5620 2024-04-25T08:31:58.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.344+0000 7fef8a2c5640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.344+0000 7fef8a2c5640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef6806b4d0 0x7fef6806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef80005880 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.348+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef74032e20 con 0x7fef840b5620 2024-04-25T08:31:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.530+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef840739e0 con 0x7fef840b5620 2024-04-25T08:31:58.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.531+0000 7fef7affd640 1 -- 172.21.15.60:0/1800454999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef740313a0 con 0x7fef840b5620 2024-04-25T08:31:58.527 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:31:58.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef6806b4d0 msgr2=0x7fef6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef6806b4d0 0x7fef6806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef80005880 tx=0x7fef80009040 comp rx=0 tx=0).stop 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 msgr2=0x7fef841f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef841f67f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fef74035040 tx=0x7fef740046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 shutdown_connections 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8415b7d0 0x7fef8415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef840b5620 0x7fef841f67f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 --2- 172.21.15.60:0/1800454999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8406c2d0 0x7fef841f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 >> 172.21.15.60:0/1800454999 conn(0x7fef840d0800 msgr2=0x7fef8415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 shutdown_connections 2024-04-25T08:31:58.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.533+0000 7fef8c529640 1 -- 172.21.15.60:0/1800454999 wait complete. 2024-04-25T08:31:58.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:31:58.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.645+0000 7f21b5dd5640 1 Processor -- start 2024-04-25T08:31:58.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.645+0000 7f21b5dd5640 1 -- start start 2024-04-25T08:31:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.645+0000 7f21b5dd5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21b00ce230 0x7f21b00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.645+0000 7f21b5dd5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b0072280 con 0x7f21b015b270 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.645+0000 7f21b5dd5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b00714c0 con 0x7f21b00cc4f0 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.645+0000 7f21b5dd5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b0071270 con 0x7f21b00ce230 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21affff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f21b015b270 0x7f21b015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50960/0 (socket says 172.21.15.60:50960) 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21affff640 1 -- 172.21.15.60:0/2580523433 learned_addr learned my addr 172.21.15.60:0/2580523433 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21aeffd640 1 --2- 172.21.15.60:0/2580523433 >> v2:172.21.15.100:3300/0 conn(0x7f21b00ce230 0x7f21b00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715895013 0 0) 0x7f21b0072280 con 0x7f21b015b270 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f219c003520 con 0x7f21b015b270 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799209716 0 0) 0x7f21b00714c0 con 0x7f21b00cc4f0 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b0072280 con 0x7f21b00cc4f0 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 548558943 0 0) 0x7f219c003520 con 0x7f21b015b270 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21aeffd640 1 -- 172.21.15.60:0/2580523433 >> v1:172.21.15.100:6789/0 conn(0x7f21b00cc4f0 legacy=0x7f21b00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21aeffd640 1 -- 172.21.15.60:0/2580523433 >> v1:172.21.15.60:6789/0 conn(0x7f21b015b270 legacy=0x7f21b015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.646+0000 7f21aeffd640 1 -- 172.21.15.60:0/2580523433 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b00717a0 con 0x7f21b00ce230 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.647+0000 7f21aeffd640 1 --2- 172.21.15.60:0/2580523433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ce230 0x7f21b00cc8d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f21a00097a0 tx=0x7f21a002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee38c30ce83bcf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.647+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0037070 con 0x7f21b00ce230 2024-04-25T08:31:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.647+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a002ac50 con 0x7f21b00ce230 2024-04-25T08:31:58.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.647+0000 7f21ae7fc640 1 -- 172.21.15.60:0/2580523433 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80019f0 con 0x7f21b00ce230 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 -- 172.21.15.60:0/2580523433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ce230 msgr2=0x7f21b00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/2580523433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ce230 0x7f21b00cc8d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f21a00097a0 tx=0x7f21a002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 -- 172.21.15.60:0/2580523433 shutdown_connections 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/2580523433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00ce230 0x7f21b00cc8d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 -- 172.21.15.60:0/2580523433 >> 172.21.15.60:0/2580523433 conn(0x7f21b006abe0 msgr2=0x7f21b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 -- 172.21.15.60:0/2580523433 shutdown_connections 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.648+0000 7f21b5dd5640 1 -- 172.21.15.60:0/2580523433 wait complete. 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.649+0000 7f21b5dd5640 1 Processor -- start 2024-04-25T08:31:58.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.649+0000 7f21b5dd5640 1 -- start start 2024-04-25T08:31:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00cc4f0 0x7f21b00adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ce230 0x7f21b01ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 0x7f21b0145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b006e8a0 con 0x7f21b015b270 2024-04-25T08:31:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b0071d60 con 0x7f21b00cc4f0 2024-04-25T08:31:58.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00ae500 con 0x7f21b00ce230 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 0x7f21b0145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 0x7f21b0145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56610/0 (socket says 172.21.15.60:56610) 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 -- 172.21.15.60:0/3103809073 learned_addr learned my addr 172.21.15.60:0/3103809073 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21af7fe640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00cc4f0 0x7f21b00adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21aeffd640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ce230 0x7f21b01ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 -- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ce230 msgr2=0x7f21b01ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ce230 0x7f21b01ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 -- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00cc4f0 msgr2=0x7f21b00adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00cc4f0 0x7f21b00adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 -- 172.21.15.60:0/3103809073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b01eee10 con 0x7f21b015b270 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21af7fe640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00cc4f0 0x7f21b00adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21affff640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 0x7f21b0145c70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f21a8001350 tx=0x7f21a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8017070 con 0x7f21b015b270 2024-04-25T08:31:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a8012400 con 0x7f21b015b270 2024-04-25T08:31:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b0146520 con 0x7f21b015b270 2024-04-25T08:31:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.650+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8012830 con 0x7f21b015b270 2024-04-25T08:31:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.651+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b00a8820 con 0x7f21b015b270 2024-04-25T08:31:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.651+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b0073a00 con 0x7f21b015b270 2024-04-25T08:31:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.652+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21a801a070 con 0x7f21b015b270 2024-04-25T08:31:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.652+0000 7f21a7fff640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f219006b4d0 0x7f219006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.652+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f21a808a250 con 0x7f21b015b270 2024-04-25T08:31:58.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.653+0000 7f21af7fe640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f219006b4d0 0x7f219006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.653+0000 7f21af7fe640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f219006b4d0 0x7f219006d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21980011e0 tx=0x7f2198009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.657+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21a80588c0 con 0x7f21b015b270 2024-04-25T08:31:58.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.848+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f21b006c250 con 0x7f21b015b270 2024-04-25T08:31:58.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.848+0000 7f21a7fff640 1 -- 172.21.15.60:0/3103809073 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f21a805c960 con 0x7f21b015b270 2024-04-25T08:31:58.845 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f219006b4d0 msgr2=0x7f219006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f219006b4d0 0x7f219006d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21980011e0 tx=0x7f2198009040 comp rx=0 tx=0).stop 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 msgr2=0x7f21b0145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 0x7f21b0145c70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f21a8001350 tx=0x7f21a800cf90 comp rx=0 tx=0).stop 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 shutdown_connections 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21b015b270 0x7f21b0145c70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21b00ce230 0x7f21b01ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f219006b4d0 0x7f219006d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 --2- 172.21.15.60:0/3103809073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21b00cc4f0 0x7f21b00adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 >> 172.21.15.60:0/3103809073 conn(0x7f21b006abe0 msgr2=0x7f21b00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 shutdown_connections 2024-04-25T08:31:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:58.850+0000 7f21b5dd5640 1 -- 172.21.15.60:0/3103809073 wait complete. 2024-04-25T08:31:58.857 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:31:58.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:58.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:31:58.894 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:31:58.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:58.894 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:31:58.950 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:31:58.950 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:58.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:31:58.968 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:31:58.968 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:59.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:31:59.006 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:31:59.006 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:59.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:31:59.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.151+0000 7fdb3c7d5640 1 Processor -- start 2024-04-25T08:31:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.151+0000 7fdb3c7d5640 1 -- start start 2024-04-25T08:31:59.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.152+0000 7fdb3c7d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 0x7fdb3406c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.152+0000 7fdb3c7d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3415dc20 0x7fdb340ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.152+0000 7fdb3c7d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3406b0e0 con 0x7fdb3406f790 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.152+0000 7fdb3c7d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3406b3c0 con 0x7fdb340b2e80 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.152+0000 7fdb3c7d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3406b6a0 con 0x7fdb3415dc20 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3a571640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 0x7fdb3406c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3ad72640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3415dc20 0x7fdb340ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb39d70640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fdb3406f790 0x7fdb340b3260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:41912/0 (socket says 172.21.15.60:41912) 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb39d70640 1 -- 172.21.15.60:0/2229953685 learned_addr learned my addr 172.21.15.60:0/2229953685 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3a571640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 0x7fdb3406c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:43528/0 (socket says 172.21.15.60:43528) 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3ad72640 1 -- 172.21.15.60:0/2229953685 >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 msgr2=0x7fdb3406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3ad72640 1 --2- 172.21.15.60:0/2229953685 >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 0x7fdb3406c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3ad72640 1 -- 172.21.15.60:0/2229953685 >> v1:172.21.15.100:6789/0 conn(0x7fdb3406f790 legacy=0x7fdb340b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3956f640 1 -- 172.21.15.60:0/2229953685 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174252267 0 0) 0x7fdb3406b0e0 con 0x7fdb3406f790 2024-04-25T08:31:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3ad72640 1 -- 172.21.15.60:0/2229953685 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3406b980 con 0x7fdb3415dc20 2024-04-25T08:31:59.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.153+0000 7fdb3a571640 1 --2- 172.21.15.60:0/2229953685 >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 0x7fdb3406c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.154+0000 7fdb3ad72640 1 --2- 172.21.15.60:0/2229953685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3415dc20 0x7fdb340ace60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0097a0 tx=0x7fdb2c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34475a269fc27b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.154+0000 7fdb3956f640 1 -- 172.21.15.60:0/2229953685 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c037070 con 0x7fdb3415dc20 2024-04-25T08:31:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.154+0000 7fdb3956f640 1 -- 172.21.15.60:0/2229953685 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c032440 con 0x7fdb3415dc20 2024-04-25T08:31:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.154+0000 7fdb3956f640 1 -- 172.21.15.60:0/2229953685 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c032810 con 0x7fdb3415dc20 2024-04-25T08:31:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.154+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/2229953685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3415dc20 msgr2=0x7fdb340ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.154+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/2229953685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3415dc20 0x7fdb340ace60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0097a0 tx=0x7fdb2c02a180 comp rx=0 tx=0).stop 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.155+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/2229953685 shutdown_connections 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.155+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/2229953685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3415dc20 0x7fdb340ace60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.155+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/2229953685 >> v2:172.21.15.100:3300/0 conn(0x7fdb340b2e80 0x7fdb3406c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.155+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/2229953685 >> 172.21.15.60:0/2229953685 conn(0x7fdb340cfe10 msgr2=0x7fdb340d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.155+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/2229953685 shutdown_connections 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.155+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/2229953685 wait complete. 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 Processor -- start 2024-04-25T08:31:59.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 -- start start 2024-04-25T08:31:59.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3406f790 0x7fdb340cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 0x7fdb3414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3415dc20 0x7fdb340adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3406ad60 con 0x7fdb340b2e80 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3406abe0 con 0x7fdb3415dc20 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.156+0000 7fdb3c7d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3406dc00 con 0x7fdb3406f790 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 0x7fdb3414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 0x7fdb3414b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56624/0 (socket says 172.21.15.60:56624) 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 -- 172.21.15.60:0/120156274 learned_addr learned my addr 172.21.15.60:0/120156274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb3ad72640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3415dc20 0x7fdb340adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb3a571640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3406f790 0x7fdb340cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 -- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3406f790 msgr2=0x7fdb340cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3406f790 0x7fdb340cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 -- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3415dc20 msgr2=0x7fdb340adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3415dc20 0x7fdb340adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 -- 172.21.15.60:0/120156274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3415faf0 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb3ad72640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3415dc20 0x7fdb340adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb39d70640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 0x7fdb3414b970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800c4d0 tx=0x7fdb2800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28018070 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28007500 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3415fdd0 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28004a20 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.157+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb34160390 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.158+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb34073990 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.159+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb2801b070 con 0x7fdb340b2e80 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.159+0000 7fdb32ffd640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.159+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdb28089790 con 0x7fdb340b2e80 2024-04-25T08:31:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.159+0000 7fdb3a571640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.160+0000 7fdb3a571640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009bd0 tx=0x7fdb2400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.164+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb280583c0 con 0x7fdb340b2e80 2024-04-25T08:31:59.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.384+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb340746d0 con 0x7fdb340b2e80 2024-04-25T08:31:59.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.384+0000 7fdb32ffd640 1 -- 172.21.15.60:0/120156274 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fdb28057cc0 con 0x7fdb340b2e80 2024-04-25T08:31:59.381 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:31:59.381 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:30:30.652688+0000","modified":"2024-04-25T08:30:41.796339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2407433231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2407433231},{"type":"v1","addr":"172.21.15.60:6835","nonce":2407433231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:31:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2024-04-25T08:31:59.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 msgr2=0x7fdb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009bd0 tx=0x7fdb2400b040 comp rx=0 tx=0).stop 2024-04-25T08:31:59.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 msgr2=0x7fdb3414b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 0x7fdb3414b970 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800c4d0 tx=0x7fdb2800c990 comp rx=0 tx=0).stop 2024-04-25T08:31:59.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 shutdown_connections 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3415dc20 0x7fdb340adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb340b2e80 0x7fdb3414b970 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 --2- 172.21.15.60:0/120156274 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3406f790 0x7fdb340cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 >> 172.21.15.60:0/120156274 conn(0x7fdb340cfe10 msgr2=0x7fdb34072370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 shutdown_connections 2024-04-25T08:31:59.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.386+0000 7fdb3c7d5640 1 -- 172.21.15.60:0/120156274 wait complete. 2024-04-25T08:31:59.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.498+0000 7ff428d48640 1 Processor -- start 2024-04-25T08:31:59.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.498+0000 7ff428d48640 1 -- start start 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff428d48640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4240b5f00 0x7ff4240ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff428d48640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240cec00 con 0x7ff42415db60 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff428d48640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240ceee0 con 0x7ff4240d0b10 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff428d48640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240ce930 con 0x7ff4240b5f00 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff422d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4240b5f00 0x7ff4240ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff42259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff42415db60 0x7ff4240d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50970/0 (socket says 172.21.15.60:50970) 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.499+0000 7ff42259c640 1 -- 172.21.15.60:0/2769119111 learned_addr learned my addr 172.21.15.60:0/2769119111 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.500+0000 7ff422d9d640 1 -- 172.21.15.60:0/2769119111 >> v1:172.21.15.100:6789/0 conn(0x7ff4240d0b10 legacy=0x7ff4240d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:59.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.500+0000 7ff422d9d640 1 -- 172.21.15.60:0/2769119111 >> v1:172.21.15.60:6789/0 conn(0x7ff42415db60 legacy=0x7ff4240d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.500+0000 7ff422d9d640 1 -- 172.21.15.60:0/2769119111 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240cf1c0 con 0x7ff4240b5f00 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.500+0000 7ff42159a640 1 -- 172.21.15.60:0/2769119111 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608570253 0 0) 0x7ff4240cec00 con 0x7ff42415db60 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.500+0000 7ff422d9d640 1 --2- 172.21.15.60:0/2769119111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240b5f00 0x7ff4240ce3f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff414009dc0 tx=0x7ff41402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab64b61f9451809a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.500+0000 7ff42159a640 1 -- 172.21.15.60:0/2769119111 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414007740 con 0x7ff4240b5f00 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff42159a640 1 -- 172.21.15.60:0/2769119111 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140078e0 con 0x7ff4240b5f00 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff42159a640 1 -- 172.21.15.60:0/2769119111 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414007cb0 con 0x7ff4240b5f00 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 -- 172.21.15.60:0/2769119111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240b5f00 msgr2=0x7ff4240ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 --2- 172.21.15.60:0/2769119111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240b5f00 0x7ff4240ce3f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff414009dc0 tx=0x7ff41402a640 comp rx=0 tx=0).stop 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 -- 172.21.15.60:0/2769119111 shutdown_connections 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 --2- 172.21.15.60:0/2769119111 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4240b5f00 0x7ff4240ce3f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 -- 172.21.15.60:0/2769119111 >> 172.21.15.60:0/2769119111 conn(0x7ff424061600 msgr2=0x7ff424061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 -- 172.21.15.60:0/2769119111 shutdown_connections 2024-04-25T08:31:59.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.501+0000 7ff428d48640 1 -- 172.21.15.60:0/2769119111 wait complete. 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.502+0000 7ff428d48640 1 Processor -- start 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 -- start start 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240b5f00 0x7ff4240c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240d0b10 0x7ff42415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 0x7ff424151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42415df40 con 0x7ff42415db60 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240b6450 con 0x7ff4240d0b10 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff428d48640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42415e0c0 con 0x7ff4240b5f00 2024-04-25T08:31:59.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 0x7ff424151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 0x7ff424151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56640/0 (socket says 172.21.15.60:56640) 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.503+0000 7ff422d9d640 1 -- 172.21.15.60:0/4287054069 learned_addr learned my addr 172.21.15.60:0/4287054069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff42259c640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240b5f00 0x7ff4240c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff422d9d640 1 -- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240b5f00 msgr2=0x7ff4240c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff422d9d640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240b5f00 0x7ff4240c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff421d9b640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240d0b10 0x7ff42415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff422d9d640 1 -- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240d0b10 msgr2=0x7ff42415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff422d9d640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240d0b10 0x7ff42415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff422d9d640 1 -- 172.21.15.60:0/4287054069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42415ed20 con 0x7ff42415db60 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff42259c640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240b5f00 0x7ff4240c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff421d9b640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240d0b10 0x7ff42415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff422d9d640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 0x7ff424151b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ee0 tx=0x7ff414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414040070 con 0x7ff42415db60 2024-04-25T08:31:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414031bc0 con 0x7ff42415db60 2024-04-25T08:31:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42415f2e0 con 0x7ff42415db60 2024-04-25T08:31:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414045460 con 0x7ff42415db60 2024-04-25T08:31:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.504+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42415f8a0 con 0x7ff42415db60 2024-04-25T08:31:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.505+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff424073270 con 0x7ff42415db60 2024-04-25T08:31:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.506+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff414039040 con 0x7ff42415db60 2024-04-25T08:31:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.506+0000 7ff41affd640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff40406b500 0x7ff40406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.506+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ff4140abc90 con 0x7ff42415db60 2024-04-25T08:31:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.506+0000 7ff42259c640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff40406b500 0x7ff40406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.507+0000 7ff42259c640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff40406b500 0x7ff40406d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.510+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff41402ae20 con 0x7ff42415db60 2024-04-25T08:31:59.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.691+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff42415f110 con 0x7ff42415db60 2024-04-25T08:31:59.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.692+0000 7ff41affd640 1 -- 172.21.15.60:0/4287054069 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16988 (secure 0 0 0) 0x7ff41407aaa0 con 0x7ff42415db60 2024-04-25T08:31:59.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:31:59.689 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":43,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:31:34.080345+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff40406b500 msgr2=0x7ff40406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff40406b500 0x7ff40406d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c009040 comp rx=0 tx=0).stop 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 msgr2=0x7ff424151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 0x7ff424151b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ee0 tx=0x7ff414004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 shutdown_connections 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff42415db60 0x7ff424151b50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff40406b500 0x7ff40406d9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4240d0b10 0x7ff42415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 --2- 172.21.15.60:0/4287054069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4240b5f00 0x7ff4240c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 >> 172.21.15.60:0/4287054069 conn(0x7ff424061600 msgr2=0x7ff42415d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 shutdown_connections 2024-04-25T08:31:59.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.694+0000 7ff428d48640 1 -- 172.21.15.60:0/4287054069 wait complete. 2024-04-25T08:31:59.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:30:29.214789+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:30:29.624978+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:31:59.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:59.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.809+0000 7f90e38a3640 1 Processor -- start 2024-04-25T08:31:59.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.809+0000 7f90e38a3640 1 -- start start 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e38a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90dc15b7d0 0x7f90dc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e38a3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90dc06cf50 con 0x7f90dc0b5620 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e38a3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90dc0ce530 con 0x7f90dc06c9e0 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e38a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90dc0ce810 con 0x7f90dc15b7d0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e163f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f90dc15b7d0 0x7f90dc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e0e3e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f90dc0b5620 0x7f90dc06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50974/0 (socket says 172.21.15.60:50974) 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e0e3e640 1 -- 172.21.15.60:0/1472663692 learned_addr learned my addr 172.21.15.60:0/1472663692 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e163f640 1 -- 172.21.15.60:0/1472663692 >> v1:172.21.15.100:6789/0 conn(0x7f90dc06c9e0 legacy=0x7f90dc0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90d3fff640 1 -- 172.21.15.60:0/1472663692 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795656609 0 0) 0x7f90dc06cf50 con 0x7f90dc0b5620 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e163f640 1 -- 172.21.15.60:0/1472663692 >> v1:172.21.15.60:6789/0 conn(0x7f90dc0b5620 legacy=0x7f90dc06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.810+0000 7f90e163f640 1 -- 172.21.15.60:0/1472663692 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc0ceaf0 con 0x7f90dc15b7d0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.811+0000 7f90e163f640 1 --2- 172.21.15.60:0/1472663692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc15b7d0 0x7f90dc0cf010 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f90d80097a0 tx=0x7f90d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba8ef551151716f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.811+0000 7f90d3fff640 1 -- 172.21.15.60:0/1472663692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8037070 con 0x7f90dc15b7d0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.811+0000 7f90d3fff640 1 -- 172.21.15.60:0/1472663692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d8032440 con 0x7f90dc15b7d0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.811+0000 7f90d3fff640 1 -- 172.21.15.60:0/1472663692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8032810 con 0x7f90dc15b7d0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.811+0000 7f90e38a3640 1 -- 172.21.15.60:0/1472663692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc15b7d0 msgr2=0x7f90dc0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.811+0000 7f90e38a3640 1 --2- 172.21.15.60:0/1472663692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc15b7d0 0x7f90dc0cf010 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f90d80097a0 tx=0x7f90d802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.812+0000 7f90e38a3640 1 -- 172.21.15.60:0/1472663692 shutdown_connections 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.812+0000 7f90e38a3640 1 --2- 172.21.15.60:0/1472663692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc15b7d0 0x7f90dc0cf010 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.812+0000 7f90e38a3640 1 -- 172.21.15.60:0/1472663692 >> 172.21.15.60:0/1472663692 conn(0x7f90dc0d0800 msgr2=0x7f90dc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.812+0000 7f90e38a3640 1 -- 172.21.15.60:0/1472663692 shutdown_connections 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.812+0000 7f90e38a3640 1 -- 172.21.15.60:0/1472663692 wait complete. 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.813+0000 7f90e38a3640 1 Processor -- start 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.813+0000 7f90e38a3640 1 -- start start 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90dc06c9e0 0x7f90dc1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 0x7f90dc0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90dc15b7d0 0x7f90dc0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc0cf6d0 con 0x7f90dc0b5620 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc0cf550 con 0x7f90dc06c9e0 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc15bbb0 con 0x7f90dc15b7d0 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 0x7f90dc0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 0x7f90dc0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56646/0 (socket says 172.21.15.60:56646) 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 -- 172.21.15.60:0/2602258867 learned_addr learned my addr 172.21.15.60:0/2602258867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e163f640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90dc06c9e0 0x7f90dc1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e1e40640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90dc15b7d0 0x7f90dc0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 -- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90dc15b7d0 msgr2=0x7f90dc0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90dc15b7d0 0x7f90dc0b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 -- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90dc06c9e0 msgr2=0x7f90dc1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90dc06c9e0 0x7f90dc1594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 -- 172.21.15.60:0/2602258867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc1599f0 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e0e3e640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 0x7f90dc0c8840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c4d0 tx=0x7f90cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc018070 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90cc007500 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90dc0c9220 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.814+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc004a20 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.815+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc0c9ac0 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.815+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90dc073a00 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.816+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90cc01b070 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.816+0000 7f90d1ffb640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.817+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f90cc089630 con 0x7f90dc0b5620 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.817+0000 7f90e163f640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.817+0000 7f90e163f640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f90c006b4d0 0x7f90c006d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f90d8034000 tx=0x7f90d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.821+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90cc058380 con 0x7f90dc0b5620 2024-04-25T08:31:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:31:59.999+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90dc0b9470 con 0x7f90dc0b5620 2024-04-25T08:31:59.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.000+0000 7f90d1ffb640 1 -- 172.21.15.60:0/2602258867 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90cc057c80 con 0x7f90dc0b5620 2024-04-25T08:31:59.997 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f90c006b4d0 msgr2=0x7f90c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f90c006b4d0 0x7f90c006d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f90d8034000 tx=0x7f90d802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 msgr2=0x7f90dc0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 0x7f90dc0c8840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c4d0 tx=0x7f90cc00c990 comp rx=0 tx=0).stop 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 shutdown_connections 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f90dc15b7d0 0x7f90dc0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f90dc0b5620 0x7f90dc0c8840 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 --2- 172.21.15.60:0/2602258867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90dc06c9e0 0x7f90dc1594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 >> 172.21.15.60:0/2602258867 conn(0x7f90dc0d0800 msgr2=0x7f90dc0d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 shutdown_connections 2024-04-25T08:31:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.002+0000 7f90e38a3640 1 -- 172.21.15.60:0/2602258867 wait complete. 2024-04-25T08:32:00.009 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.116+0000 7f456c322640 1 Processor -- start 2024-04-25T08:32:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.116+0000 7f456c322640 1 -- start start 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456c322640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f456415b7d0 0x7f45640cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456c322640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45640b5620 0x7f45640b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456c322640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456406cf50 con 0x7f456406c2d0 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456c322640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45640b5fc0 con 0x7f45640b5620 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456c322640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45640ce530 con 0x7f456415b7d0 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f45698bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45640b5620 0x7f45640b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456a8bf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f456406c2d0 0x7f456406c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50988/0 (socket says 172.21.15.60:50988) 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456a8bf640 1 -- 172.21.15.60:0/1017421095 learned_addr learned my addr 172.21.15.60:0/1017421095 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f456a0be640 1 --2- 172.21.15.60:0/1017421095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f456415b7d0 0x7f45640cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f45698bd640 1 -- 172.21.15.60:0/1017421095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f456415b7d0 msgr2=0x7f45640cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f45698bd640 1 --2- 172.21.15.60:0/1017421095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f456415b7d0 0x7f45640cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f45698bd640 1 -- 172.21.15.60:0/1017421095 >> v1:172.21.15.60:6789/0 conn(0x7f456406c2d0 legacy=0x7f456406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f45690bc640 1 -- 172.21.15.60:0/1017421095 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473292284 0 0) 0x7f456406cf50 con 0x7f456406c2d0 2024-04-25T08:32:00.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.117+0000 7f45698bd640 1 -- 172.21.15.60:0/1017421095 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45640ce810 con 0x7f45640b5620 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.118+0000 7f45698bd640 1 --2- 172.21.15.60:0/1017421095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45640b5620 0x7f45640b5a80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45540097a0 tx=0x7f455402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751e020db5bb826d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.118+0000 7f45690bc640 1 -- 172.21.15.60:0/1017421095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554037070 con 0x7f45640b5620 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.118+0000 7f45690bc640 1 -- 172.21.15.60:0/1017421095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554032440 con 0x7f45640b5620 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.118+0000 7f45690bc640 1 -- 172.21.15.60:0/1017421095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554032810 con 0x7f45640b5620 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 -- 172.21.15.60:0/1017421095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45640b5620 msgr2=0x7f45640b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 --2- 172.21.15.60:0/1017421095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45640b5620 0x7f45640b5a80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45540097a0 tx=0x7f455402a180 comp rx=0 tx=0).stop 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 -- 172.21.15.60:0/1017421095 shutdown_connections 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 --2- 172.21.15.60:0/1017421095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45640b5620 0x7f45640b5a80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 --2- 172.21.15.60:0/1017421095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f456415b7d0 0x7f45640cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 -- 172.21.15.60:0/1017421095 >> 172.21.15.60:0/1017421095 conn(0x7f45640d0800 msgr2=0x7f45640d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 -- 172.21.15.60:0/1017421095 shutdown_connections 2024-04-25T08:32:00.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.119+0000 7f456c322640 1 -- 172.21.15.60:0/1017421095 wait complete. 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.120+0000 7f456c322640 1 Processor -- start 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.120+0000 7f456c322640 1 -- start start 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456c322640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f456406c2d0 0x7f4564159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456c322640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45640b5620 0x7f45640b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456c322640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 0x7f45640b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456c322640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45640cf550 con 0x7f456415b7d0 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456c322640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456406d230 con 0x7f456406c2d0 2024-04-25T08:32:00.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456c322640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45640cf6d0 con 0x7f45640b5620 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 0x7f45640b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 0x7f45640b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56652/0 (socket says 172.21.15.60:56652) 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 -- 172.21.15.60:0/4018823997 learned_addr learned my addr 172.21.15.60:0/4018823997 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a0be640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f456406c2d0 0x7f4564159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f45698bd640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45640b5620 0x7f45640b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 -- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45640b5620 msgr2=0x7f45640b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45640b5620 0x7f45640b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 -- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f456406c2d0 msgr2=0x7f4564159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f456406c2d0 0x7f4564159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 -- 172.21.15.60:0/4018823997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45641f6be0 con 0x7f456415b7d0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.121+0000 7f456a8bf640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 0x7f45640b2d50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c800 tx=0x7f455c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.122+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c018070 con 0x7f456415b7d0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.122+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c004d80 con 0x7f456415b7d0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.122+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45641f6ec0 con 0x7f456415b7d0 2024-04-25T08:32:00.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.122+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c00a880 con 0x7f456415b7d0 2024-04-25T08:32:00.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.122+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45641f7480 con 0x7f456415b7d0 2024-04-25T08:32:00.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.123+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4564073a00 con 0x7f456415b7d0 2024-04-25T08:32:00.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.124+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f455c01b070 con 0x7f456415b7d0 2024-04-25T08:32:00.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.124+0000 7f455affd640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f454806b500 0x7f454806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.124+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f455c089880 con 0x7f456415b7d0 2024-04-25T08:32:00.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.124+0000 7f456a0be640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f454806b500 0x7f454806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.125+0000 7f456a0be640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f454806b500 0x7f454806d9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45640b34b0 tx=0x7f4560008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.129+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f455c057fb0 con 0x7f456415b7d0 2024-04-25T08:32:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.309+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45640cf270 con 0x7f456415b7d0 2024-04-25T08:32:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.309+0000 7f455affd640 1 -- 172.21.15.60:0/4018823997 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f455c05c050 con 0x7f456415b7d0 2024-04-25T08:32:00.306 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f454806b500 msgr2=0x7f454806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f454806b500 0x7f454806d9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45640b34b0 tx=0x7f4560008040 comp rx=0 tx=0).stop 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 msgr2=0x7f45640b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 0x7f45640b2d50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c800 tx=0x7f455c00ccc0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 shutdown_connections 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f456415b7d0 0x7f45640b2d50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45640b5620 0x7f45640b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f454806b500 0x7f454806d9c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 --2- 172.21.15.60:0/4018823997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f456406c2d0 0x7f4564159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 >> 172.21.15.60:0/4018823997 conn(0x7f45640d0800 msgr2=0x7f45640d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 shutdown_connections 2024-04-25T08:32:00.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.311+0000 7f456c322640 1 -- 172.21.15.60:0/4018823997 wait complete. 2024-04-25T08:32:00.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:00.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 Processor -- start 2024-04-25T08:32:00.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 -- start start 2024-04-25T08:32:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c301610f0 0x7f4c3006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4c300ce7b0 0x7f4c30161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c300c6960 con 0x7f4c301616f0 2024-04-25T08:32:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c301614d0 con 0x7f4c301610f0 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c35ccb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c30162260 con 0x7f4c300ce7b0 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.426+0000 7f4c2f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4c301610f0 0x7f4c3006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4c301616f0 0x7f4c300ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50992/0 (socket says 172.21.15.60:50992) 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2effd640 1 -- 172.21.15.60:0/1531086493 learned_addr learned my addr 172.21.15.60:0/1531086493 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2f7fe640 1 --2- 172.21.15.60:0/1531086493 >> v2:172.21.15.60:3300/0 conn(0x7f4c301610f0 0x7f4c3006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56666/0 (socket says 172.21.15.60:56666) 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2ffff640 1 --2- 172.21.15.60:0/1531086493 >> v2:172.21.15.100:3300/0 conn(0x7f4c300ce7b0 0x7f4c30161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2f7fe640 1 -- 172.21.15.60:0/1531086493 >> v2:172.21.15.100:3300/0 conn(0x7f4c300ce7b0 msgr2=0x7f4c30161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2f7fe640 1 --2- 172.21.15.60:0/1531086493 >> v2:172.21.15.100:3300/0 conn(0x7f4c300ce7b0 0x7f4c30161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2f7fe640 1 -- 172.21.15.60:0/1531086493 >> v1:172.21.15.60:6789/0 conn(0x7f4c301616f0 legacy=0x7f4c300ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2f7fe640 1 -- 172.21.15.60:0/1531086493 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30162540 con 0x7f4c301610f0 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.427+0000 7f4c2f7fe640 1 --2- 172.21.15.60:0/1531086493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c3006c1c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b33e0 tx=0x7f4c2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2e2c6e0618f69b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c2e7fc640 1 -- 172.21.15.60:0/1531086493 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20007740 con 0x7f4c301610f0 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c2e7fc640 1 -- 172.21.15.60:0/1531086493 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c200078e0 con 0x7f4c301610f0 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c2e7fc640 1 -- 172.21.15.60:0/1531086493 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20007cb0 con 0x7f4c301610f0 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 -- 172.21.15.60:0/1531086493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 msgr2=0x7f4c3006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/1531086493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c3006c1c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b33e0 tx=0x7f4c2002a680 comp rx=0 tx=0).stop 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 -- 172.21.15.60:0/1531086493 shutdown_connections 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/1531086493 >> v2:172.21.15.100:3300/0 conn(0x7f4c300ce7b0 0x7f4c30161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/1531086493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c3006c1c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 -- 172.21.15.60:0/1531086493 >> 172.21.15.60:0/1531086493 conn(0x7f4c3006f500 msgr2=0x7f4c3006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 -- 172.21.15.60:0/1531086493 shutdown_connections 2024-04-25T08:32:00.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.428+0000 7f4c35ccb640 1 -- 172.21.15.60:0/1531086493 wait complete. 2024-04-25T08:32:00.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.429+0000 7f4c35ccb640 1 Processor -- start 2024-04-25T08:32:00.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 -- start start 2024-04-25T08:32:00.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c300ce7b0 0x7f4c301f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c300b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c301616f0 0x7f4c300b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c300cf690 con 0x7f4c301610f0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c300cfce0 con 0x7f4c301616f0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.430+0000 7f4c35ccb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3006c700 con 0x7f4c300ce7b0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c300b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c300b6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56672/0 (socket says 172.21.15.60:56672) 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 -- 172.21.15.60:0/3355424368 learned_addr learned my addr 172.21.15.60:0/3355424368 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2f7fe640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c300ce7b0 0x7f4c301f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2ffff640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c301616f0 0x7f4c300b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 -- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c300ce7b0 msgr2=0x7f4c301f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c300ce7b0 0x7f4c301f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 -- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c301616f0 msgr2=0x7f4c300b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c301616f0 0x7f4c300b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 -- 172.21.15.60:0/3355424368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300b5330 con 0x7f4c301610f0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c2effd640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c300b6710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c3c0 tx=0x7f4c2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24017070 con 0x7f4c301610f0 2024-04-25T08:32:00.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c24005ce0 con 0x7f4c301610f0 2024-04-25T08:32:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c240072f0 con 0x7f4c301610f0 2024-04-25T08:32:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c300b5610 con 0x7f4c301610f0 2024-04-25T08:32:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.431+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c300b5bd0 con 0x7f4c301610f0 2024-04-25T08:32:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.432+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c30073a00 con 0x7f4c301610f0 2024-04-25T08:32:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.433+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c2401a070 con 0x7f4c301610f0 2024-04-25T08:32:00.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.433+0000 7f4c17fff640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1c06b4d0 0x7f4c1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.433+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f4c24089660 con 0x7f4c301610f0 2024-04-25T08:32:00.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.434+0000 7f4c2f7fe640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1c06b4d0 0x7f4c1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.434+0000 7f4c2f7fe640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1c06b4d0 0x7f4c1c06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c20034000 tx=0x7f4c2002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.438+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c24057d60 con 0x7f4c301610f0 2024-04-25T08:32:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.618+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c300cf150 con 0x7f4c301610f0 2024-04-25T08:32:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.618+0000 7f4c17fff640 1 -- 172.21.15.60:0/3355424368 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c2405be00 con 0x7f4c301610f0 2024-04-25T08:32:00.615 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:32:00.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1c06b4d0 msgr2=0x7f4c1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1c06b4d0 0x7f4c1c06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c20034000 tx=0x7f4c2002f000 comp rx=0 tx=0).stop 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 msgr2=0x7f4c300b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c300b6710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400c3c0 tx=0x7f4c2400c880 comp rx=0 tx=0).stop 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 shutdown_connections 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1c06b4d0 0x7f4c1c06d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c301616f0 0x7f4c300b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c301610f0 0x7f4c300b6710 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 --2- 172.21.15.60:0/3355424368 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c300ce7b0 0x7f4c301f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 >> 172.21.15.60:0/3355424368 conn(0x7f4c3006f500 msgr2=0x7f4c300b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 shutdown_connections 2024-04-25T08:32:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.620+0000 7f4c35ccb640 1 -- 172.21.15.60:0/3355424368 wait complete. 2024-04-25T08:32:00.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.731+0000 7f10974f5640 1 Processor -- start 2024-04-25T08:32:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.731+0000 7f10974f5640 1 -- start start 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f10974f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f109015da70 0x7f1090161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f10974f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 0x7f10900d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f10974f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090162020 con 0x7f10900b9730 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f10974f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10901625e0 con 0x7f109015da70 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f10974f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1090162300 con 0x7f1090161b80 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1094a90640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f10900b9730 0x7f10900ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51006/0 (socket says 172.21.15.60:51006) 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1094a90640 1 -- 172.21.15.60:0/328774165 learned_addr learned my addr 172.21.15.60:0/328774165 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342385001 0 0) 0x7f1090162020 con 0x7f10900b9730 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1095291640 1 --2- 172.21.15.60:0/328774165 >> v2:172.21.15.100:3300/0 conn(0x7f109015da70 0x7f1090161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1095a92640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 0x7f10900d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c003520 con 0x7f10900b9730 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 655863729 0 0) 0x7f107c003520 con 0x7f10900b9730 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 msgr2=0x7f10900d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 0x7f10900d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 >> v2:172.21.15.100:3300/0 conn(0x7f109015da70 msgr2=0x7f1090161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 --2- 172.21.15.60:0/328774165 >> v2:172.21.15.100:3300/0 conn(0x7f109015da70 0x7f1090161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900d1af0 con 0x7f10900b9730 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1080001670 con 0x7f10900b9730 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 >> v1:172.21.15.60:6789/0 conn(0x7f10900b9730 legacy=0x7f10900ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.732+0000 7f1087fff640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107c0039b0 0x7f107c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1087fff640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 0x7f107c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1087fff640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f107c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10901625e0 con 0x7f107c004d40 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1090162300 con 0x7f107c0039b0 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00b9c0 con 0x7f1090161b80 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095a92640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f107c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095291640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107c0039b0 0x7f107c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1094a90640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 0x7f107c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095a92640 1 -- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 msgr2=0x7f107c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095a92640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090161b80 0x7f107c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095a92640 1 -- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107c0039b0 msgr2=0x7f107c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095a92640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f107c0039b0 0x7f107c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.733+0000 7f1095a92640 1 -- 172.21.15.60:0/328774165 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c003520 con 0x7f107c004d40 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f1095a92640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f107c0051c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f10900b6060 tx=0x7f107c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4093f46160d38fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088037070 con 0x7f107c004d40 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108802e050 con 0x7f107c004d40 2024-04-25T08:32:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f1087fff640 1 -- 172.21.15.60:0/328774165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088009c10 con 0x7f107c004d40 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f10974f5640 1 -- 172.21.15.60:0/328774165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 msgr2=0x7f107c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f10974f5640 1 --2- 172.21.15.60:0/328774165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f107c0051c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f10900b6060 tx=0x7f107c005830 comp rx=0 tx=0).stop 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f1095a92640 1 -- 172.21.15.60:0/328774165 reap_dead start 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f10974f5640 1 -- 172.21.15.60:0/328774165 shutdown_connections 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.734+0000 7f10974f5640 1 -- 172.21.15.60:0/328774165 >> 172.21.15.60:0/328774165 conn(0x7f1090061a20 msgr2=0x7f1090061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.735+0000 7f10974f5640 1 -- 172.21.15.60:0/328774165 shutdown_connections 2024-04-25T08:32:00.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.735+0000 7f10974f5640 1 -- 172.21.15.60:0/328774165 wait complete. 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 Processor -- start 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 -- start start 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f10900c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090155eb0 0x7f10900c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10900c9af0 0x7f1090157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10900ced60 con 0x7f107c004d40 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109015de50 con 0x7f10900c9af0 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f10974f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10900ceee0 con 0x7f1090155eb0 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.736+0000 7f1095291640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f10900c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f10900c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56698/0 (socket says 172.21.15.60:56698) 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 -- 172.21.15.60:0/2257775901 learned_addr learned my addr 172.21.15.60:0/2257775901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1094a90640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090155eb0 0x7f10900c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095a92640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10900c9af0 0x7f1090157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 -- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090155eb0 msgr2=0x7f10900c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090155eb0 0x7f10900c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 -- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10900c9af0 msgr2=0x7f1090157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10900c9af0 0x7f1090157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 -- 172.21.15.60:0/2257775901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900c8750 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1095291640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f10900c8210 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f108c011990 tx=0x7f108c011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00fad0 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c00fc70 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10900c9ff0 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c014440 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.737+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10900c7a10 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.738+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1090073990 con 0x7f107c004d40 2024-04-25T08:32:00.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.739+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f108c012040 con 0x7f107c004d40 2024-04-25T08:32:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.739+0000 7f1085ffb640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f107406b500 0x7f107406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.739+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f108c08aa10 con 0x7f107c004d40 2024-04-25T08:32:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.739+0000 7f1094a90640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f107406b500 0x7f107406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.740+0000 7f1094a90640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f107406b500 0x7f107406d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f109015ccc0 tx=0x7f1080009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.744+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10900c9ff0 con 0x7f107c004d40 2024-04-25T08:32:00.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.960+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10900d1570 con 0x7f107c004d40 2024-04-25T08:32:00.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.960+0000 7f1085ffb640 1 -- 172.21.15.60:0/2257775901 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f108c019030 con 0x7f107c004d40 2024-04-25T08:32:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:00.957 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:30:30.652688+0000","modified":"2024-04-25T08:30:41.796339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2407433231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2407433231},{"type":"v1","addr":"172.21.15.60:6835","nonce":2407433231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:32:00.957 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2024-04-25T08:32:00.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f107406b500 msgr2=0x7f107406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f107406b500 0x7f107406d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f109015ccc0 tx=0x7f1080009040 comp rx=0 tx=0).stop 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 msgr2=0x7f10900c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f10900c8210 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f108c011990 tx=0x7f108c011e50 comp rx=0 tx=0).stop 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 shutdown_connections 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f107406b500 0x7f107406d9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10900c9af0 0x7f1090157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1090155eb0 0x7f10900c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 --2- 172.21.15.60:0/2257775901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f107c004d40 0x7f10900c8210 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.962+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 >> 172.21.15.60:0/2257775901 conn(0x7f1090061a20 msgr2=0x7f10900d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.963+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 shutdown_connections 2024-04-25T08:32:00.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:00.963+0000 7f10974f5640 1 -- 172.21.15.60:0/2257775901 wait complete. 2024-04-25T08:32:00.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 Processor -- start 2024-04-25T08:32:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 -- start start 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f811015dbe0 0x7f81100d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f81100b92a0 0x7f81100d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100b97a0 con 0x7f81100b2cb0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811015dfc0 con 0x7f81100b92a0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.074+0000 7f8115d6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100d1a00 con 0x7f811015dbe0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f81100b2cb0 0x7f81100b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51018/0 (socket says 172.21.15.60:51018) 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810effd640 1 -- 172.21.15.60:0/2194798633 learned_addr learned my addr 172.21.15.60:0/2194798633 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810ffff640 1 --2- 172.21.15.60:0/2194798633 >> v2:172.21.15.100:3300/0 conn(0x7f81100b92a0 0x7f81100d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810f7fe640 1 --2- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f811015dbe0 0x7f81100d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810e7fc640 1 -- 172.21.15.60:0/2194798633 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401764634 0 0) 0x7f81100b97a0 con 0x7f81100b2cb0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810e7fc640 1 -- 172.21.15.60:0/2194798633 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f8003520 con 0x7f81100b2cb0 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810ffff640 1 -- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f811015dbe0 msgr2=0x7f81100d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810ffff640 1 --2- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f811015dbe0 0x7f81100d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810ffff640 1 -- 172.21.15.60:0/2194798633 >> v1:172.21.15.60:6789/0 conn(0x7f81100b2cb0 legacy=0x7f81100b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.075+0000 7f810ffff640 1 -- 172.21.15.60:0/2194798633 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100d1c50 con 0x7f81100b92a0 2024-04-25T08:32:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.076+0000 7f810ffff640 1 --2- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b92a0 0x7f81100d0e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8108009d90 tx=0x7f810802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64405189e454aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.076+0000 7f810e7fc640 1 -- 172.21.15.60:0/2194798633 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108007740 con 0x7f81100b92a0 2024-04-25T08:32:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.076+0000 7f810e7fc640 1 -- 172.21.15.60:0/2194798633 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81080078e0 con 0x7f81100b92a0 2024-04-25T08:32:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.076+0000 7f810e7fc640 1 -- 172.21.15.60:0/2194798633 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108007c90 con 0x7f81100b92a0 2024-04-25T08:32:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 -- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b92a0 msgr2=0x7f81100d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b92a0 0x7f81100d0e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8108009d90 tx=0x7f810802a660 comp rx=0 tx=0).stop 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 -- 172.21.15.60:0/2194798633 shutdown_connections 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b92a0 0x7f81100d0e50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2194798633 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f811015dbe0 0x7f81100d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 -- 172.21.15.60:0/2194798633 >> 172.21.15.60:0/2194798633 conn(0x7f8110061a20 msgr2=0x7f8110061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 -- 172.21.15.60:0/2194798633 shutdown_connections 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.077+0000 7f8115d6a640 1 -- 172.21.15.60:0/2194798633 wait complete. 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.078+0000 7f8115d6a640 1 Processor -- start 2024-04-25T08:32:01.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.078+0000 7f8115d6a640 1 -- start start 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f8115d6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b2cb0 0x7f81100cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f8115d6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81100b92a0 0x7f8110159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f8115d6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 0x7f811015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f8115d6a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b6ab0 con 0x7f811015dbe0 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f8115d6a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100d2210 con 0x7f81100b2cb0 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f8115d6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b6c30 con 0x7f81100b92a0 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 0x7f811015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 0x7f811015a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56706/0 (socket says 172.21.15.60:56706) 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 -- 172.21.15.60:0/2044288979 learned_addr learned my addr 172.21.15.60:0/2044288979 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810effd640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81100b92a0 0x7f8110159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810f7fe640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b2cb0 0x7f81100cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 -- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81100b92a0 msgr2=0x7f8110159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81100b92a0 0x7f8110159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 -- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b2cb0 msgr2=0x7f81100cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b2cb0 0x7f81100cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 -- 172.21.15.60:0/2044288979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100b5c40 con 0x7f811015dbe0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f810ffff640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 0x7f811015a320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8108009eb0 tx=0x7f8108004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.079+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108040070 con 0x7f811015dbe0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.080+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108031c60 con 0x7f811015dbe0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.080+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81100caaf0 con 0x7f811015dbe0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.080+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108009450 con 0x7f811015dbe0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.080+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100cb0b0 con 0x7f811015dbe0 2024-04-25T08:32:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.080+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8110073990 con 0x7f811015dbe0 2024-04-25T08:32:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.081+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8108039040 con 0x7f811015dbe0 2024-04-25T08:32:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.081+0000 7f80f7fff640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80fc06b4d0 0x7f80fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.081+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f81080ab480 con 0x7f811015dbe0 2024-04-25T08:32:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.082+0000 7f810f7fe640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80fc06b4d0 0x7f80fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.082+0000 7f810f7fe640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80fc06b4d0 0x7f80fc06d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8100009a30 tx=0x7f8100008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.086+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f810807a0c0 con 0x7f811015dbe0 2024-04-25T08:32:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.269+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81100cdd20 con 0x7f811015dbe0 2024-04-25T08:32:01.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.271+0000 7f80f7fff640 1 -- 172.21.15.60:0/2044288979 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f81080799c0 con 0x7f811015dbe0 2024-04-25T08:32:01.267 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:01.267 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":43,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:31:34.080345+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80fc06b4d0 msgr2=0x7f80fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80fc06b4d0 0x7f80fc06d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8100009a30 tx=0x7f8100008040 comp rx=0 tx=0).stop 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 msgr2=0x7f811015a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 0x7f811015a320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8108009eb0 tx=0x7f8108004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 shutdown_connections 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f811015dbe0 0x7f811015a320 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f81100b92a0 0x7f8110159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80fc06b4d0 0x7f80fc06d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 --2- 172.21.15.60:0/2044288979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81100b2cb0 0x7f81100cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 >> 172.21.15.60:0/2044288979 conn(0x7f8110061a20 msgr2=0x7f81100b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 shutdown_connections 2024-04-25T08:32:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.273+0000 7f8115d6a640 1 -- 172.21.15.60:0/2044288979 wait complete. 2024-04-25T08:32:01.280 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:32:01.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.385+0000 7faa27c7c640 1 Processor -- start 2024-04-25T08:32:01.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.385+0000 7faa27c7c640 1 -- start start 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa27c7c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa2015da70 0x7faa20161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa27c7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa20161b80 0x7faa200d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa27c7c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa20162020 con 0x7faa200b9730 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa27c7c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa201625e0 con 0x7faa2015da70 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa27c7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa20162300 con 0x7faa20161b80 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25a18640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faa2015da70 0x7faa20161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25217640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa200b9730 0x7faa200ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51024/0 (socket says 172.21.15.60:51024) 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25217640 1 -- 172.21.15.60:0/1665305731 learned_addr learned my addr 172.21.15.60:0/1665305731 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa26219640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa20161b80 0x7faa200d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25a18640 1 -- 172.21.15.60:0/1665305731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa20161b80 msgr2=0x7faa200d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25a18640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa20161b80 0x7faa200d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25a18640 1 -- 172.21.15.60:0/1665305731 >> v1:172.21.15.60:6789/0 conn(0x7faa200b9730 legacy=0x7faa200ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa24a16640 1 -- 172.21.15.60:0/1665305731 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929301145 0 0) 0x7faa20162020 con 0x7faa200b9730 2024-04-25T08:32:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa25a18640 1 -- 172.21.15.60:0/1665305731 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200d1af0 con 0x7faa2015da70 2024-04-25T08:32:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.386+0000 7faa26219640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa20161b80 0x7faa200d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa25a18640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2015da70 0x7faa20161640 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7faa180097a0 tx=0x7faa1802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd6f58673c99e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa24a16640 1 -- 172.21.15.60:0/1665305731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18037070 con 0x7faa2015da70 2024-04-25T08:32:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa24a16640 1 -- 172.21.15.60:0/1665305731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1802ae50 con 0x7faa2015da70 2024-04-25T08:32:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa24a16640 1 -- 172.21.15.60:0/1665305731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18032660 con 0x7faa2015da70 2024-04-25T08:32:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa27c7c640 1 -- 172.21.15.60:0/1665305731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2015da70 msgr2=0x7faa20161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa27c7c640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2015da70 0x7faa20161640 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7faa180097a0 tx=0x7faa1802a100 comp rx=0 tx=0).stop 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa27c7c640 1 -- 172.21.15.60:0/1665305731 shutdown_connections 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa27c7c640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa20161b80 0x7faa200d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa27c7c640 1 --2- 172.21.15.60:0/1665305731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa2015da70 0x7faa20161640 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.387+0000 7faa27c7c640 1 -- 172.21.15.60:0/1665305731 >> 172.21.15.60:0/1665305731 conn(0x7faa20061a20 msgr2=0x7faa20061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.388+0000 7faa27c7c640 1 -- 172.21.15.60:0/1665305731 shutdown_connections 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.388+0000 7faa27c7c640 1 -- 172.21.15.60:0/1665305731 wait complete. 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 Processor -- start 2024-04-25T08:32:01.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 -- start start 2024-04-25T08:32:01.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2015da70 0x7faa201f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa20161b80 0x7faa20160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 0x7faa2015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200ced60 con 0x7faa20158f50 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2015de50 con 0x7faa20161b80 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.389+0000 7faa27c7c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200ceee0 con 0x7faa2015da70 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 0x7faa2015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 0x7faa2015c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56726/0 (socket says 172.21.15.60:56726) 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 -- 172.21.15.60:0/2776227546 learned_addr learned my addr 172.21.15.60:0/2776227546 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa25a18640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2015da70 0x7faa201f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa25217640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa20161b80 0x7faa20160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 -- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2015da70 msgr2=0x7faa201f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2015da70 0x7faa201f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 -- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa20161b80 msgr2=0x7faa20160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa20161b80 0x7faa20160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 -- 172.21.15.60:0/2776227546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa201606b0 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa25217640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa20161b80 0x7faa20160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa26219640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 0x7faa2015c130 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7faa1000cd20 tx=0x7faa1000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000f680 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1000f840 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa201f3bb0 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1000fc30 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.390+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa201f4170 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.391+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20073990 con 0x7faa20158f50 2024-04-25T08:32:01.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.392+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa100213f0 con 0x7faa20158f50 2024-04-25T08:32:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.392+0000 7faa1e7fc640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa0c06b500 0x7faa0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.392+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7faa1008b0c0 con 0x7faa20158f50 2024-04-25T08:32:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.392+0000 7faa25a18640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa0c06b500 0x7faa0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.393+0000 7faa25a18640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa0c06b500 0x7faa0c06d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faa18034000 tx=0x7faa1802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.397+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa10014030 con 0x7faa20158f50 2024-04-25T08:32:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.613+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa2015dfd0 con 0x7faa20158f50 2024-04-25T08:32:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.614+0000 7faa1e7fc640 1 -- 172.21.15.60:0/2776227546 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7faa10059730 con 0x7faa20158f50 2024-04-25T08:32:01.611 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:01.611 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:30:30.652688+0000","modified":"2024-04-25T08:30:41.796339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2407433231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2407433231},{"type":"v1","addr":"172.21.15.60:6835","nonce":2407433231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:32:01.611 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa0c06b500 msgr2=0x7faa0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa0c06b500 0x7faa0c06d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faa18034000 tx=0x7faa1802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 msgr2=0x7faa2015c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 0x7faa2015c130 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7faa1000cd20 tx=0x7faa1000a650 comp rx=0 tx=0).stop 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 shutdown_connections 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa20158f50 0x7faa2015c130 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa0c06b500 0x7faa0c06d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa20161b80 0x7faa20160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 --2- 172.21.15.60:0/2776227546 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa2015da70 0x7faa201f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 >> 172.21.15.60:0/2776227546 conn(0x7faa20061a20 msgr2=0x7faa200b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 shutdown_connections 2024-04-25T08:32:01.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.616+0000 7faa27c7c640 1 -- 172.21.15.60:0/2776227546 wait complete. 2024-04-25T08:32:01.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.727+0000 7fef93742640 1 Processor -- start 2024-04-25T08:32:01.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.728+0000 7fef93742640 1 -- start start 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.728+0000 7fef93742640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef8c15dc00 0x7fef8c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.728+0000 7fef93742640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 0x7fef8c0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.728+0000 7fef93742640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c15dfe0 con 0x7fef8c0d1c00 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.728+0000 7fef93742640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8c0d2060 con 0x7fef8c15dc00 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.728+0000 7fef93742640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0d00c0 con 0x7fef8c0ceb20 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef91cdf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef8c0d1c00 0x7fef8c0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33572/0 (socket says 172.21.15.60:33572) 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef91cdf640 1 -- 172.21.15.60:0/2388076409 learned_addr learned my addr 172.21.15.60:0/2388076409 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef90cdd640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 0x7fef8c0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef914de640 1 --2- 172.21.15.60:0/2388076409 >> v2:172.21.15.100:3300/0 conn(0x7fef8c15dc00 0x7fef8c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434032308 0 0) 0x7fef84001110 con 0x7fef8c0d1c00 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef74003520 con 0x7fef8c0d1c00 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372166338 0 0) 0x7fef74003520 con 0x7fef8c0d1c00 2024-04-25T08:32:01.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 msgr2=0x7fef8c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 0x7fef8c0d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 >> v2:172.21.15.100:3300/0 conn(0x7fef8c15dc00 msgr2=0x7fef8c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 --2- 172.21.15.60:0/2388076409 >> v2:172.21.15.100:3300/0 conn(0x7fef8c15dc00 0x7fef8c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0d03a0 con 0x7fef8c0d1c00 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fef84001840 con 0x7fef8c0d1c00 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 >> v1:172.21.15.60:6789/0 conn(0x7fef8c0d1c00 legacy=0x7fef8c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 0x7fef740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74003f30 0x7fef74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef74004d40 0x7fef740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0d2060 con 0x7fef74004d40 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0d00c0 con 0x7fef74003f30 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7400b9c0 con 0x7fef8c0ceb20 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef91cdf640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef74004d40 0x7fef740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef90cdd640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74003f30 0x7fef74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.729+0000 7fef914de640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 0x7fef740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef91cdf640 1 -- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 msgr2=0x7fef740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef91cdf640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c0ceb20 0x7fef740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef91cdf640 1 -- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74003f30 msgr2=0x7fef74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef91cdf640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74003f30 0x7fef74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef91cdf640 1 -- 172.21.15.60:0/2388076409 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74003520 con 0x7fef74004d40 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef90cdd640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74003f30 0x7fef74004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.730+0000 7fef91cdf640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef74004d40 0x7fef740051c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c3e0 tx=0x7fef74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f95ad2c990aec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84032020 con 0x7fef74004d40 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84005af0 con 0x7fef74004d40 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef8bfff640 1 -- 172.21.15.60:0/2388076409 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8400c0e0 con 0x7fef74004d40 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef93742640 1 -- 172.21.15.60:0/2388076409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef74004d40 msgr2=0x7fef740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef93742640 1 --2- 172.21.15.60:0/2388076409 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef74004d40 0x7fef740051c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c3e0 tx=0x7fef74005830 comp rx=0 tx=0).stop 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef91cdf640 1 -- 172.21.15.60:0/2388076409 reap_dead start 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef93742640 1 -- 172.21.15.60:0/2388076409 shutdown_connections 2024-04-25T08:32:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef93742640 1 -- 172.21.15.60:0/2388076409 >> 172.21.15.60:0/2388076409 conn(0x7fef8c06f5d0 msgr2=0x7fef8c06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef93742640 1 -- 172.21.15.60:0/2388076409 shutdown_connections 2024-04-25T08:32:01.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.731+0000 7fef93742640 1 -- 172.21.15.60:0/2388076409 wait complete. 2024-04-25T08:32:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.732+0000 7fef93742640 1 Processor -- start 2024-04-25T08:32:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.732+0000 7fef93742640 1 -- start start 2024-04-25T08:32:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef93742640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74004d40 0x7fef8c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef93742640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c14c1c0 0x7fef8c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef93742640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 0x7fef8c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef93742640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cc130 con 0x7fef8c06d8a0 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef93742640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cc720 con 0x7fef74004d40 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef93742640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cfd90 con 0x7fef8c14c1c0 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef91cdf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 0x7fef8c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef91cdf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 0x7fef8c0b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42758/0 (socket says 172.21.15.60:42758) 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef91cdf640 1 -- 172.21.15.60:0/3055295569 learned_addr learned my addr 172.21.15.60:0/3055295569 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef90cdd640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c14c1c0 0x7fef8c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.733+0000 7fef914de640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74004d40 0x7fef8c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef91cdf640 1 -- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c14c1c0 msgr2=0x7fef8c06d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef91cdf640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c14c1c0 0x7fef8c06d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef91cdf640 1 -- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74004d40 msgr2=0x7fef8c0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef91cdf640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74004d40 0x7fef8c0b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef91cdf640 1 -- 172.21.15.60:0/3055295569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0b50f0 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef91cdf640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 0x7fef8c0b44e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef84031040 tx=0x7fef84001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84032030 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef840076b0 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c0b53d0 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80009690 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.734+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c0b5990 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.735+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c073990 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.736+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef84035020 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.736+0000 7fef89ffb640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef7806b4d0 0x7fef7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.736+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fef840aa110 con 0x7fef8c06d8a0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.736+0000 7fef914de640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef7806b4d0 0x7fef7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.737+0000 7fef914de640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef7806b4d0 0x7fef7806d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef7c014510 tx=0x7fef7c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.740+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef8c15dfe0 con 0x7fef8c06d8a0 2024-04-25T08:32:01.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.959+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef8c0c66c0 con 0x7fef8c06d8a0 2024-04-25T08:32:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.959+0000 7fef89ffb640 1 -- 172.21.15.60:0/3055295569 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fef840786f0 con 0x7fef8c06d8a0 2024-04-25T08:32:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:01.956 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:30:30.652688+0000","modified":"2024-04-25T08:30:41.796339+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/2407433231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2407433231},{"type":"v1","addr":"172.21.15.60:6835","nonce":2407433231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:32:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 16 2024-04-25T08:32:01.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef7806b4d0 msgr2=0x7fef7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef7806b4d0 0x7fef7806d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fef7c014510 tx=0x7fef7c00b040 comp rx=0 tx=0).stop 2024-04-25T08:32:01.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 msgr2=0x7fef8c0b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 0x7fef8c0b44e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef84031040 tx=0x7fef84001c10 comp rx=0 tx=0).stop 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 shutdown_connections 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef8c06d8a0 0x7fef8c0b44e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef8c14c1c0 0x7fef8c06d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef7806b4d0 0x7fef7806d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 --2- 172.21.15.60:0/3055295569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef74004d40 0x7fef8c0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.961+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 >> 172.21.15.60:0/3055295569 conn(0x7fef8c06f5d0 msgr2=0x7fef8c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.962+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 shutdown_connections 2024-04-25T08:32:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:01.962+0000 7fef93742640 1 -- 172.21.15.60:0/3055295569 wait complete. 2024-04-25T08:32:01.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:02.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.073+0000 7f39a5790640 1 Processor -- start 2024-04-25T08:32:02.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.073+0000 7f39a5790640 1 -- start start 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f39a5790640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f39a5790640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d1ca0 0x7f39a00c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f39a5790640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a00d20e0 con 0x7f39a00c7140 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f39a5790640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39a015f2e0 con 0x7f39a00d1ca0 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f39a5790640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00c7970 con 0x7f39a015ef00 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f39a00d1ca0 0x7f39a00c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f39a00c7140 0x7f39a015f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33584/0 (socket says 172.21.15.60:33584) 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399f7fe640 1 -- 172.21.15.60:0/1212780766 learned_addr learned my addr 172.21.15.60:0/1212780766 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:02.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399effd640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399e7fc640 1 -- 172.21.15.60:0/1212780766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 msgr2=0x7f39a00d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399e7fc640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399e7fc640 1 -- 172.21.15.60:0/1212780766 >> v1:172.21.15.60:6789/0 conn(0x7f39a00c7140 legacy=0x7f39a015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399e7fc640 1 -- 172.21.15.60:0/1212780766 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00c7c50 con 0x7f39a00d1ca0 2024-04-25T08:32:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399dffb640 1 -- 172.21.15.60:0/1212780766 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947576032 0 0) 0x7f39a00d20e0 con 0x7f39a00c7140 2024-04-25T08:32:02.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.074+0000 7f399effd640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:02.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.075+0000 7f399e7fc640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d1ca0 0x7f39a00c6c00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f39940097a0 tx=0x7f399402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1564a253ae283fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.075+0000 7f399dffb640 1 -- 172.21.15.60:0/1212780766 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994037070 con 0x7f39a00d1ca0 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.075+0000 7f399dffb640 1 -- 172.21.15.60:0/1212780766 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994032440 con 0x7f39a00d1ca0 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.075+0000 7f399dffb640 1 -- 172.21.15.60:0/1212780766 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994032810 con 0x7f39a00d1ca0 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 -- 172.21.15.60:0/1212780766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d1ca0 msgr2=0x7f39a00c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d1ca0 0x7f39a00c6c00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f39940097a0 tx=0x7f399402a210 comp rx=0 tx=0).stop 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 -- 172.21.15.60:0/1212780766 shutdown_connections 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00d1ca0 0x7f39a00c6c00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 --2- 172.21.15.60:0/1212780766 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 -- 172.21.15.60:0/1212780766 >> 172.21.15.60:0/1212780766 conn(0x7f39a006f5d0 msgr2=0x7f39a006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 -- 172.21.15.60:0/1212780766 shutdown_connections 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.076+0000 7f39a5790640 1 -- 172.21.15.60:0/1212780766 wait complete. 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.077+0000 7f39a5790640 1 Processor -- start 2024-04-25T08:32:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.077+0000 7f39a5790640 1 -- start start 2024-04-25T08:32:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f39a5790640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 0x7f39a00b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f39a5790640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d1ca0 0x7f39a00b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f39a5790640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f39a5790640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00c84f0 con 0x7f39a00c7140 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f39a5790640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00c8210 con 0x7f39a00d1ca0 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f39a5790640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a00c7560 con 0x7f39a015ef00 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 0x7f39a00b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 0x7f39a00b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42780/0 (socket says 172.21.15.60:42780) 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 -- 172.21.15.60:0/1384957950 learned_addr learned my addr 172.21.15.60:0/1384957950 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399f7fe640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399e7fc640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d1ca0 0x7f39a00b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 -- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 msgr2=0x7f39a00b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 -- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d1ca0 msgr2=0x7f39a00b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d1ca0 0x7f39a00b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 -- 172.21.15.60:0/1384957950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00b4ac0 con 0x7f39a00c7140 2024-04-25T08:32:02.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399e7fc640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d1ca0 0x7f39a00b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399f7fe640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.078+0000 7f399effd640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 0x7f39a00b44e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f399800cf60 tx=0x7f399800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.079+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399801a070 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.079+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399800d840 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.079+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a00b4da0 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.079+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399800dc10 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.079+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a006bcf0 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.080+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a0073990 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.080+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f399801d070 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.081+0000 7f39877fe640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398c06b4d0 0x7f398c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.081+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f399808abf0 con 0x7f39a00c7140 2024-04-25T08:32:02.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.081+0000 7f399e7fc640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398c06b4d0 0x7f398c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.081+0000 7f399e7fc640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398c06b4d0 0x7f398c06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.085+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3998059240 con 0x7f39a00c7140 2024-04-25T08:32:02.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.271+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39a015e440 con 0x7f39a00c7140 2024-04-25T08:32:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.272+0000 7f39877fe640 1 -- 172.21.15.60:0/1384957950 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f399805d2e0 con 0x7f39a00c7140 2024-04-25T08:32:02.269 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:02.269 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":43,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:31:34.080345+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:30:29.214789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:30:29.624978+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398c06b4d0 msgr2=0x7f398c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398c06b4d0 0x7f398c06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3994034000 tx=0x7f399402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 msgr2=0x7f39a00b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 0x7f39a00b44e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f399800cf60 tx=0x7f399800cf90 comp rx=0 tx=0).stop 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 shutdown_connections 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f39a015ef00 0x7f39a00b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398c06b4d0 0x7f398c06d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39a00d1ca0 0x7f39a00b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 --2- 172.21.15.60:0/1384957950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f39a00c7140 0x7f39a00b44e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 >> 172.21.15.60:0/1384957950 conn(0x7f39a006f5d0 msgr2=0x7f39a00cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 shutdown_connections 2024-04-25T08:32:02.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.274+0000 7f39a5790640 1 -- 172.21.15.60:0/1384957950 wait complete. 2024-04-25T08:32:02.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:32:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.387+0000 7fce29812640 1 Processor -- start 2024-04-25T08:32:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.387+0000 7fce29812640 1 -- start start 2024-04-25T08:32:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce29812640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce2415dac0 0x7fce240d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce29812640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce240d0ab0 0x7fce240b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce29812640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce240d2420 con 0x7fce240d1f30 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce29812640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce240d3030 con 0x7fce2415dac0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce29812640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240d2700 con 0x7fce240d0ab0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce2415dac0 0x7fce240d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fce2415dac0 0x7fce240d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42796/0 (socket says 172.21.15.60:42796) 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 -- 172.21.15.60:0/2444025882 learned_addr learned my addr 172.21.15.60:0/2444025882 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce237fe640 1 --2- 172.21.15.60:0/2444025882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce240d0ab0 0x7fce240b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 -- 172.21.15.60:0/2444025882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce240d0ab0 msgr2=0x7fce240b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 --2- 172.21.15.60:0/2444025882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce240d0ab0 0x7fce240b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 -- 172.21.15.60:0/2444025882 >> v1:172.21.15.100:6789/0 conn(0x7fce240d1f30 legacy=0x7fce240d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.388+0000 7fce22ffd640 1 -- 172.21.15.60:0/2444025882 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240d29e0 con 0x7fce2415dac0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.389+0000 7fce22ffd640 1 --2- 172.21.15.60:0/2444025882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2415dac0 0x7fce240d19f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce14009870 tx=0x7fce1402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e71c5e5a5295e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.389+0000 7fce227fc640 1 -- 172.21.15.60:0/2444025882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14037070 con 0x7fce2415dac0 2024-04-25T08:32:02.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.389+0000 7fce227fc640 1 -- 172.21.15.60:0/2444025882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1402ae50 con 0x7fce2415dac0 2024-04-25T08:32:02.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.389+0000 7fce227fc640 1 -- 172.21.15.60:0/2444025882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce14032860 con 0x7fce2415dac0 2024-04-25T08:32:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 -- 172.21.15.60:0/2444025882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2415dac0 msgr2=0x7fce240d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 --2- 172.21.15.60:0/2444025882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2415dac0 0x7fce240d19f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce14009870 tx=0x7fce1402a300 comp rx=0 tx=0).stop 2024-04-25T08:32:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 -- 172.21.15.60:0/2444025882 shutdown_connections 2024-04-25T08:32:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 --2- 172.21.15.60:0/2444025882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce240d0ab0 0x7fce240b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 --2- 172.21.15.60:0/2444025882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce2415dac0 0x7fce240d19f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 -- 172.21.15.60:0/2444025882 >> 172.21.15.60:0/2444025882 conn(0x7fce24061c30 msgr2=0x7fce2406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 -- 172.21.15.60:0/2444025882 shutdown_connections 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.390+0000 7fce29812640 1 -- 172.21.15.60:0/2444025882 wait complete. 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.391+0000 7fce29812640 1 Processor -- start 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.391+0000 7fce29812640 1 -- start start 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce240d0ab0 0x7fce2415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 0x7fce241f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce2415dac0 0x7fce241f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b97d0 con 0x7fce240d1f30 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2415dea0 con 0x7fce240d0ab0 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240b9950 con 0x7fce2415dac0 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 0x7fce241f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 0x7fce241f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42798/0 (socket says 172.21.15.60:42798) 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 -- 172.21.15.60:0/999776134 learned_addr learned my addr 172.21.15.60:0/999776134 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce237fe640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce2415dac0 0x7fce241f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce22ffd640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce240d0ab0 0x7fce2415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 -- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce2415dac0 msgr2=0x7fce241f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce2415dac0 0x7fce241f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 -- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce240d0ab0 msgr2=0x7fce2415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce240d0ab0 0x7fce2415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 -- 172.21.15.60:0/999776134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240b3da0 con 0x7fce240d1f30 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce237fe640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce2415dac0 0x7fce241f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce1bfff640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 0x7fce241f6290 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c400 tx=0x7fce0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c0052d0 con 0x7fce240d1f30 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c0054b0 con 0x7fce240d1f30 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce240b4080 con 0x7fce240d1f30 2024-04-25T08:32:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c006a40 con 0x7fce240d1f30 2024-04-25T08:32:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.392+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce240b4640 con 0x7fce240d1f30 2024-04-25T08:32:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.393+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce24073a00 con 0x7fce240d1f30 2024-04-25T08:32:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.394+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce0c006be0 con 0x7fce240d1f30 2024-04-25T08:32:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.394+0000 7fce20ff9640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fce0406b4d0 0x7fce0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.394+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fce0c089910 con 0x7fce240d1f30 2024-04-25T08:32:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.395+0000 7fce22ffd640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fce0406b4d0 0x7fce0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.395+0000 7fce22ffd640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fce0406b4d0 0x7fce0406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fce14034000 tx=0x7fce1402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.399+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce0c058540 con 0x7fce240d1f30 2024-04-25T08:32:02.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.620+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fce240b4810 con 0x7fce240d1f30 2024-04-25T08:32:02.617 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:32:02.621+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:32:02.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.622+0000 7fce20ff9640 1 -- 172.21.15.60:0/999776134 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fce0c057e40 con 0x7fce240d1f30 2024-04-25T08:32:02.618 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fce0406b4d0 msgr2=0x7fce0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fce0406b4d0 0x7fce0406d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fce14034000 tx=0x7fce1402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 msgr2=0x7fce241f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 0x7fce241f6290 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c400 tx=0x7fce0c00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 shutdown_connections 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce2415dac0 0x7fce241f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fce240d1f30 0x7fce241f6290 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fce0406b4d0 0x7fce0406d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 --2- 172.21.15.60:0/999776134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce240d0ab0 0x7fce2415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 >> 172.21.15.60:0/999776134 conn(0x7fce24061c30 msgr2=0x7fce240b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 shutdown_connections 2024-04-25T08:32:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.625+0000 7fce29812640 1 -- 172.21.15.60:0/999776134 wait complete. 2024-04-25T08:32:02.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:32:02.661 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:32:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 Processor -- start 2024-04-25T08:32:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 -- start start 2024-04-25T08:32:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f416c15bb20 0x7f416c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f416c0d00e0 0x7f416c0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c0cb190 con 0x7f416c0ccdd0 2024-04-25T08:32:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c0cb470 con 0x7f416c0d00e0 2024-04-25T08:32:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.741+0000 7f4171f70640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c0cb750 con 0x7f416c15bb20 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f416c0d00e0 0x7f416c0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f416c0d00e0 0x7f416c0ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42816/0 (socket says 172.21.15.60:42816) 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 -- 172.21.15.60:0/488480906 learned_addr learned my addr 172.21.15.60:0/488480906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416b7fe640 1 --2- 172.21.15.60:0/488480906 >> v2:172.21.15.100:3300/0 conn(0x7f416c15bb20 0x7f416c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 -- 172.21.15.60:0/488480906 >> v2:172.21.15.100:3300/0 conn(0x7f416c15bb20 msgr2=0x7f416c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 --2- 172.21.15.60:0/488480906 >> v2:172.21.15.100:3300/0 conn(0x7f416c15bb20 0x7f416c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 -- 172.21.15.60:0/488480906 >> v1:172.21.15.100:6789/0 conn(0x7f416c0ccdd0 legacy=0x7f416c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416bfff640 1 -- 172.21.15.60:0/488480906 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0cba30 con 0x7f416c0d00e0 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.742+0000 7f416b7fe640 1 --2- 172.21.15.60:0/488480906 >> v2:172.21.15.100:3300/0 conn(0x7f416c15bb20 0x7f416c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f416bfff640 1 --2- 172.21.15.60:0/488480906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c0d00e0 0x7f416c0ad600 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f416c0d1800 tx=0x7f416402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6816372f773d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f416a7fc640 1 -- 172.21.15.60:0/488480906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164036070 con 0x7f416c0d00e0 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f416a7fc640 1 -- 172.21.15.60:0/488480906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164031600 con 0x7f416c0d00e0 2024-04-25T08:32:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f416a7fc640 1 -- 172.21.15.60:0/488480906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640319d0 con 0x7f416c0d00e0 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 -- 172.21.15.60:0/488480906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c0d00e0 msgr2=0x7f416c0ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 --2- 172.21.15.60:0/488480906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c0d00e0 0x7f416c0ad600 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f416c0d1800 tx=0x7f416402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 -- 172.21.15.60:0/488480906 shutdown_connections 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 --2- 172.21.15.60:0/488480906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c0d00e0 0x7f416c0ad600 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 --2- 172.21.15.60:0/488480906 >> v2:172.21.15.100:3300/0 conn(0x7f416c15bb20 0x7f416c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 -- 172.21.15.60:0/488480906 >> 172.21.15.60:0/488480906 conn(0x7f416c0b2390 msgr2=0x7f416c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 -- 172.21.15.60:0/488480906 shutdown_connections 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.743+0000 7f4171f70640 1 -- 172.21.15.60:0/488480906 wait complete. 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 Processor -- start 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 -- start start 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0ccdd0 0x7f416c0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c0d00e0 0x7f416c0b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 0x7f416c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c15bf00 con 0x7f416c15bb20 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0cc770 con 0x7f416c0ccdd0 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.745+0000 7f4171f70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c15c080 con 0x7f416c0d00e0 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 0x7f416c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 0x7f416c0b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42824/0 (socket says 172.21.15.60:42824) 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 -- 172.21.15.60:0/3834927407 learned_addr learned my addr 172.21.15.60:0/3834927407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416affd640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c0d00e0 0x7f416c0b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416b7fe640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0ccdd0 0x7f416c0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 -- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c0d00e0 msgr2=0x7f416c0b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c0d00e0 0x7f416c0b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 -- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0ccdd0 msgr2=0x7f416c0c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0ccdd0 0x7f416c0c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 -- 172.21.15.60:0/3834927407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0c6b60 con 0x7f416c15bb20 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416b7fe640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0ccdd0 0x7f416c0c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f416bfff640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 0x7f416c0b3c20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4164009f40 tx=0x7f4164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164036050 con 0x7f416c15bb20 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164042070 con 0x7f416c15bb20 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c0c6e40 con 0x7f416c15bb20 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164009510 con 0x7f416c15bb20 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.746+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c0c7400 con 0x7f416c15bb20 2024-04-25T08:32:02.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.747+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c0cf2d0 con 0x7f416c15bb20 2024-04-25T08:32:02.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.748+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4164007bd0 con 0x7f416c15bb20 2024-04-25T08:32:02.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.748+0000 7f4153fff640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f415806b4d0 0x7f415806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.748+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f41640ab1f0 con 0x7f416c15bb20 2024-04-25T08:32:02.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.748+0000 7f416b7fe640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f415806b4d0 0x7f415806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.749+0000 7f416b7fe640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f415806b4d0 0x7f415806d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f415c006fd0 tx=0x7f415c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.753+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f416c0cf2d0 con 0x7f416c15bb20 2024-04-25T08:32:02.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:02.973+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f416c0739e0 con 0x7f416c15bb20 2024-04-25T08:32:03.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.623+0000 7f4153fff640 1 -- 172.21.15.60:0/3834927407 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4164079f00 con 0x7f416c15bb20 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f415806b4d0 msgr2=0x7f415806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f415806b4d0 0x7f415806d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f415c006fd0 tx=0x7f415c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 msgr2=0x7f416c0b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 0x7f416c0b3c20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4164009f40 tx=0x7f4164004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 shutdown_connections 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f416c15bb20 0x7f416c0b3c20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f416c0d00e0 0x7f416c0b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f415806b4d0 0x7f415806d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 --2- 172.21.15.60:0/3834927407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f416c0ccdd0 0x7f416c0c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 >> 172.21.15.60:0/3834927407 conn(0x7f416c0b2390 msgr2=0x7f416c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 shutdown_connections 2024-04-25T08:32:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.626+0000 7f4171f70640 1 -- 172.21.15.60:0/3834927407 wait complete. 2024-04-25T08:32:03.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:32:03.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 Processor -- start 2024-04-25T08:32:03.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 -- start start 2024-04-25T08:32:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f607c06c330 0x7f607c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f607c06c7e0 0x7f607c0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c0d1d50 con 0x7f607c158230 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607c158610 con 0x7f607c06c330 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.795+0000 7f608124c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607c06bea0 con 0x7f607c06c7e0 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f6073fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f607c06c330 0x7f607c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f607ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f607c158230 0x7f607c0d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33598/0 (socket says 172.21.15.60:33598) 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f607ad9d640 1 -- 172.21.15.60:0/380802435 learned_addr learned my addr 172.21.15.60:0/380802435 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f607b59e640 1 --2- 172.21.15.60:0/380802435 >> v2:172.21.15.100:3300/0 conn(0x7f607c06c7e0 0x7f607c0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f6073fff640 1 -- 172.21.15.60:0/380802435 >> v2:172.21.15.100:3300/0 conn(0x7f607c06c7e0 msgr2=0x7f607c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f6073fff640 1 --2- 172.21.15.60:0/380802435 >> v2:172.21.15.100:3300/0 conn(0x7f607c06c7e0 0x7f607c0b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f6073fff640 1 -- 172.21.15.60:0/380802435 >> v1:172.21.15.60:6789/0 conn(0x7f607c158230 legacy=0x7f607c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f607a59c640 1 -- 172.21.15.60:0/380802435 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383063044 0 0) 0x7f607c0d1d50 con 0x7f607c158230 2024-04-25T08:32:03.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.796+0000 7f6073fff640 1 -- 172.21.15.60:0/380802435 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0d2020 con 0x7f607c06c330 2024-04-25T08:32:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.797+0000 7f6073fff640 1 --2- 172.21.15.60:0/380802435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c0b4e60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f607c0cdd80 tx=0x7f606402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a372a0b4676144e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.797+0000 7f607a59c640 1 -- 172.21.15.60:0/380802435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064037070 con 0x7f607c06c330 2024-04-25T08:32:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.797+0000 7f607a59c640 1 -- 172.21.15.60:0/380802435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6064032440 con 0x7f607c06c330 2024-04-25T08:32:03.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.797+0000 7f607a59c640 1 -- 172.21.15.60:0/380802435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064032810 con 0x7f607c06c330 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.797+0000 7f608124c640 1 -- 172.21.15.60:0/380802435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 msgr2=0x7f607c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.797+0000 7f608124c640 1 --2- 172.21.15.60:0/380802435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c0b4e60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f607c0cdd80 tx=0x7f606402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.798+0000 7f608124c640 1 -- 172.21.15.60:0/380802435 shutdown_connections 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.798+0000 7f608124c640 1 --2- 172.21.15.60:0/380802435 >> v2:172.21.15.100:3300/0 conn(0x7f607c06c7e0 0x7f607c0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.798+0000 7f608124c640 1 --2- 172.21.15.60:0/380802435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c0b4e60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.798+0000 7f608124c640 1 -- 172.21.15.60:0/380802435 >> 172.21.15.60:0/380802435 conn(0x7f607c061600 msgr2=0x7f607c0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:03.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.798+0000 7f608124c640 1 -- 172.21.15.60:0/380802435 shutdown_connections 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.798+0000 7f608124c640 1 -- 172.21.15.60:0/380802435 wait complete. 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.799+0000 7f608124c640 1 Processor -- start 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.799+0000 7f608124c640 1 -- start start 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f608124c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f608124c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f607c06c7e0 0x7f607c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f608124c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f607c158230 0x7f607c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f608124c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0d25e0 con 0x7f607c06c330 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f608124c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c06bb30 con 0x7f607c06c7e0 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f608124c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c15e320 con 0x7f607c158230 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42842/0 (socket says 172.21.15.60:42842) 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 -- 172.21.15.60:0/26529342 learned_addr learned my addr 172.21.15.60:0/26529342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:03.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f6073fff640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f607c06c7e0 0x7f607c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607b59e640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f607c158230 0x7f607c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 -- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f607c158230 msgr2=0x7f607c15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f607c158230 0x7f607c15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 -- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f607c06c7e0 msgr2=0x7f607c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f607c06c7e0 0x7f607c0cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 -- 172.21.15.60:0/26529342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c15a270 con 0x7f607c06c330 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.800+0000 7f607ad9d640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c156920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c4d0 tx=0x7f606c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.801+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c018070 con 0x7f607c06c330 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.801+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c007500 con 0x7f607c06c330 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.801+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c004a20 con 0x7f607c06c330 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.801+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c1e4380 con 0x7f607c06c330 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.801+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c0cd160 con 0x7f607c06c330 2024-04-25T08:32:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.802+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c15a440 con 0x7f607c06c330 2024-04-25T08:32:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.802+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f606c01b070 con 0x7f607c06c330 2024-04-25T08:32:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.803+0000 7f60737fe640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f605c06b4d0 0x7f605c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.803+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f606c089640 con 0x7f607c06c330 2024-04-25T08:32:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.803+0000 7f6073fff640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f605c06b4d0 0x7f605c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.803+0000 7f6073fff640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f605c06b4d0 0x7f605c06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6064034000 tx=0x7f606402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:03.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.807+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f606c057c70 con 0x7f607c06c330 2024-04-25T08:32:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:03.991+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f607c073a00 con 0x7f607c06c330 2024-04-25T08:32:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.625+0000 7f60737fe640 1 -- 172.21.15.60:0/26529342 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f606c05bd10 con 0x7f607c06c330 2024-04-25T08:32:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:32:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.628+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f605c06b4d0 msgr2=0x7f605c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.628+0000 7f608124c640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f605c06b4d0 0x7f605c06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6064034000 tx=0x7f606402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.628+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 msgr2=0x7f607c156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.628+0000 7f608124c640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c156920 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c4d0 tx=0x7f606c00c990 comp rx=0 tx=0).stop 2024-04-25T08:32:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 shutdown_connections 2024-04-25T08:32:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f607c158230 0x7f607c15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f605c06b4d0 0x7f605c06d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f607c06c7e0 0x7f607c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 --2- 172.21.15.60:0/26529342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f607c06c330 0x7f607c156920 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 >> 172.21.15.60:0/26529342 conn(0x7f607c061600 msgr2=0x7f607c15d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 shutdown_connections 2024-04-25T08:32:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.629+0000 7f608124c640 1 -- 172.21.15.60:0/26529342 wait complete. 2024-04-25T08:32:04.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:32:04.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.753+0000 7f95d5e1f640 1 Processor -- start 2024-04-25T08:32:04.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.753+0000 7f95d5e1f640 1 -- start start 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95d5e1f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f95d00ced80 0x7f95d00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95d5e1f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95d0161710 0x7f95d00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95d5e1f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d015e000 con 0x7f95d015dc20 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95d5e1f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d00d1de0 con 0x7f95d0161710 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95d5e1f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d0161c60 con 0x7f95d00ced80 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95cffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95d0161710 0x7f95d00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:04.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95cffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95d0161710 0x7f95d00d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42844/0 (socket says 172.21.15.60:42844) 2024-04-25T08:32:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95cffff640 1 -- 172.21.15.60:0/543168565 learned_addr learned my addr 172.21.15.60:0/543168565 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95ceffd640 1 --2- 172.21.15.60:0/543168565 >> v2:172.21.15.100:3300/0 conn(0x7f95d00ced80 0x7f95d00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.754+0000 7f95cffff640 1 -- 172.21.15.60:0/543168565 >> v2:172.21.15.100:3300/0 conn(0x7f95d00ced80 msgr2=0x7f95d00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95cffff640 1 --2- 172.21.15.60:0/543168565 >> v2:172.21.15.100:3300/0 conn(0x7f95d00ced80 0x7f95d00d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95cffff640 1 -- 172.21.15.60:0/543168565 >> v1:172.21.15.100:6789/0 conn(0x7f95d015dc20 legacy=0x7f95d00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:04.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95cffff640 1 -- 172.21.15.60:0/543168565 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0161f40 con 0x7f95d0161710 2024-04-25T08:32:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95cffff640 1 --2- 172.21.15.60:0/543168565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d0161710 0x7f95d00d2820 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95d006da90 tx=0x7f95c802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50f133fcd38bb89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95ce7fc640 1 -- 172.21.15.60:0/543168565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8036070 con 0x7f95d0161710 2024-04-25T08:32:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95ce7fc640 1 -- 172.21.15.60:0/543168565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c8031600 con 0x7f95d0161710 2024-04-25T08:32:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.755+0000 7f95ce7fc640 1 -- 172.21.15.60:0/543168565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80319d0 con 0x7f95d0161710 2024-04-25T08:32:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 -- 172.21.15.60:0/543168565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d0161710 msgr2=0x7f95d00d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/543168565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d0161710 0x7f95d00d2820 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95d006da90 tx=0x7f95c802a730 comp rx=0 tx=0).stop 2024-04-25T08:32:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 -- 172.21.15.60:0/543168565 shutdown_connections 2024-04-25T08:32:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/543168565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d0161710 0x7f95d00d2820 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/543168565 >> v2:172.21.15.100:3300/0 conn(0x7f95d00ced80 0x7f95d00d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 -- 172.21.15.60:0/543168565 >> 172.21.15.60:0/543168565 conn(0x7f95d00d0420 msgr2=0x7f95d00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 -- 172.21.15.60:0/543168565 shutdown_connections 2024-04-25T08:32:04.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.756+0000 7f95d5e1f640 1 -- 172.21.15.60:0/543168565 wait complete. 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.757+0000 7f95d5e1f640 1 Processor -- start 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.757+0000 7f95d5e1f640 1 -- start start 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95d5e1f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95d00ced80 0x7f95d00c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95d5e1f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 0x7f95d01e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95d5e1f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d0161710 0x7f95d014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95d5e1f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00b5e50 con 0x7f95d015dc20 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95d5e1f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0162500 con 0x7f95d0161710 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95d5e1f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00b5fd0 con 0x7f95d00ced80 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 0x7f95d01e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 0x7f95d01e8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42846/0 (socket says 172.21.15.60:42846) 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.758+0000 7f95ceffd640 1 -- 172.21.15.60:0/3785135245 learned_addr learned my addr 172.21.15.60:0/3785135245 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95ceffd640 1 -- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95d00ced80 msgr2=0x7f95d00c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95cffff640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d0161710 0x7f95d014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95cf7fe640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95d00ced80 0x7f95d00c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95ceffd640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95d00ced80 0x7f95d00c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95ceffd640 1 -- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d0161710 msgr2=0x7f95d014ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95ceffd640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d0161710 0x7f95d014ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95ceffd640 1 -- 172.21.15.60:0/3785135245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d014f460 con 0x7f95d015dc20 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95cf7fe640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95d00ced80 0x7f95d00c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95cffff640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d0161710 0x7f95d014ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95ceffd640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 0x7f95d01e8960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f95c400d5f0 tx=0x7f95c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40125f0 con 0x7f95d015dc20 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4019070 con 0x7f95d015dc20 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d01f3d30 con 0x7f95d015dc20 2024-04-25T08:32:04.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40129d0 con 0x7f95d015dc20 2024-04-25T08:32:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.759+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d01f42b0 con 0x7f95d015dc20 2024-04-25T08:32:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.760+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d0073a00 con 0x7f95d015dc20 2024-04-25T08:32:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.760+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95c4012b70 con 0x7f95d015dc20 2024-04-25T08:32:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.761+0000 7f95b7fff640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95bc06b4d0 0x7f95bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.761+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f95c408a640 con 0x7f95d015dc20 2024-04-25T08:32:04.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.761+0000 7f95cf7fe640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95bc06b4d0 0x7f95bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:04.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.762+0000 7f95cf7fe640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95bc06b4d0 0x7f95bc06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f95c00015e0 tx=0x7f95c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:04.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.766+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95c4059470 con 0x7f95d015dc20 2024-04-25T08:32:04.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:04.946+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f95d00d16b0 con 0x7f95d015dc20 2024-04-25T08:32:05.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.626+0000 7f95b7fff640 1 -- 172.21.15.60:0/3785135245 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f95c4058d70 con 0x7f95d015dc20 2024-04-25T08:32:05.623 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:32:05.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95bc06b4d0 msgr2=0x7f95bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95bc06b4d0 0x7f95bc06d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f95c00015e0 tx=0x7f95c0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:05.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 msgr2=0x7f95d01e8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 0x7f95d01e8960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f95c400d5f0 tx=0x7f95c400dab0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 shutdown_connections 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95bc06b4d0 0x7f95bc06d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95d0161710 0x7f95d014ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d015dc20 0x7f95d01e8960 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 --2- 172.21.15.60:0/3785135245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95d00ced80 0x7f95d00c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 >> 172.21.15.60:0/3785135245 conn(0x7f95d00d0420 msgr2=0x7f95d015ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 shutdown_connections 2024-04-25T08:32:05.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.630+0000 7f95d5e1f640 1 -- 172.21.15.60:0/3785135245 wait complete. 2024-04-25T08:32:05.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:32:05.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.749+0000 7f8cdaee1640 1 Processor -- start 2024-04-25T08:32:05.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.749+0000 7f8cdaee1640 1 -- start start 2024-04-25T08:32:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.749+0000 7f8cdaee1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cd40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.749+0000 7f8cdaee1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd40cf1d0 con 0x7f8cd4161240 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.749+0000 7f8cdaee1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd4161620 con 0x7f8cd4161890 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.749+0000 7f8cdaee1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40b7120 con 0x7f8cd40b5bb0 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd8c7d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8cd4161240 0x7f8cd40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33608/0 (socket says 172.21.15.60:33608) 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd8c7d640 1 -- 172.21.15.60:0/1640298785 learned_addr learned my addr 172.21.15.60:0/1640298785 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd947e640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cd40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582384589 0 0) 0x7f8cd40cf1d0 con 0x7f8cd4161240 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc4003520 con 0x7f8cd4161240 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3871645822 0 0) 0x7f8cc4003520 con 0x7f8cd4161240 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 msgr2=0x7f8cd40b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cd40b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 >> v1:172.21.15.100:6789/0 conn(0x7f8cd4161890 legacy=0x7f8cd40b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4161e00 con 0x7f8cd4161240 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f8cc8001670 con 0x7f8cd4161240 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 >> v1:172.21.15.60:6789/0 conn(0x7f8cd4161240 legacy=0x7f8cd40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc40039e0 0x7f8cc4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cc4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cc40050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400b9c0 con 0x7f8cc4004390 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400bb70 con 0x7f8cc40039e0 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400bd70 con 0x7f8cd40b5bb0 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cc4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd8c7d640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc40039e0 0x7f8cc4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.750+0000 7f8cd947e640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cc40050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd3fff640 1 -- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 msgr2=0x7f8cc40050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cc40050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd3fff640 1 -- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc40039e0 msgr2=0x7f8cc4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cc40039e0 0x7f8cc4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd3fff640 1 -- 172.21.15.60:0/1640298785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4003520 con 0x7f8cc4004390 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd947e640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b5bb0 0x7f8cc40050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cc4004860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0001860 tx=0x7f8cc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944acddaa4f8b450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0037070 con 0x7f8cc4004390 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc002abf0 con 0x7f8cc4004390 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.751+0000 7f8cd37fe640 1 -- 172.21.15.60:0/1640298785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0001430 con 0x7f8cc4004390 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cdaee1640 1 -- 172.21.15.60:0/1640298785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 msgr2=0x7f8cc4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/1640298785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cc4004860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0001860 tx=0x7f8cc4005830 comp rx=0 tx=0).stop 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cd947e640 1 -- 172.21.15.60:0/1640298785 reap_dead start 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cdaee1640 1 -- 172.21.15.60:0/1640298785 shutdown_connections 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cdaee1640 1 -- 172.21.15.60:0/1640298785 >> 172.21.15.60:0/1640298785 conn(0x7f8cd40d04a0 msgr2=0x7f8cd40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cdaee1640 1 -- 172.21.15.60:0/1640298785 shutdown_connections 2024-04-25T08:32:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.752+0000 7f8cdaee1640 1 -- 172.21.15.60:0/1640298785 wait complete. 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.753+0000 7f8cdaee1640 1 Processor -- start 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.753+0000 7f8cdaee1640 1 -- start start 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cdaee1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cd4151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cdaee1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b89e0 0x7f8cd41f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cdaee1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd40b8e60 0x7f8cd41f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cdaee1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41626a0 con 0x7f8cc4004390 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cdaee1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41623c0 con 0x7f8cd40b8e60 2024-04-25T08:32:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cdaee1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40d15f0 con 0x7f8cd40b89e0 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cd8c7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cd4151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cd8c7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cd4151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42870/0 (socket says 172.21.15.60:42870) 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cd8c7d640 1 -- 172.21.15.60:0/3536496002 learned_addr learned my addr 172.21.15.60:0/3536496002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b89e0 0x7f8cd41f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.754+0000 7f8cd947e640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd40b8e60 0x7f8cd41f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd8c7d640 1 -- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b89e0 msgr2=0x7f8cd41f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd8c7d640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b89e0 0x7f8cd41f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd8c7d640 1 -- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd40b8e60 msgr2=0x7f8cd41f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd8c7d640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd40b8e60 0x7f8cd41f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd8c7d640 1 -- 172.21.15.60:0/3536496002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40b7810 con 0x7f8cc4004390 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd947e640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd40b8e60 0x7f8cd41f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd8c7d640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cd4151fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8007370 tx=0x7f8cc8007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc801a070 con 0x7f8cc4004390 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc800f8e0 con 0x7f8cc4004390 2024-04-25T08:32:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd40b7af0 con 0x7f8cc4004390 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800fcb0 con 0x7f8cc4004390 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.755+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd40caf00 con 0x7f8cc4004390 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.756+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40d3810 con 0x7f8cc4004390 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.756+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cc801d070 con 0x7f8cc4004390 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.757+0000 7f8cd17fa640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8cb806b4d0 0x7f8cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.757+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8cc8089850 con 0x7f8cc4004390 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.757+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8cb806b4d0 0x7f8cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.758+0000 7f8cd3fff640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8cb806b4d0 0x7f8cb806d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0034000 tx=0x7f8cc0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:05.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.762+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cd40d3810 con 0x7f8cc4004390 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.944+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8cd4073a00 con 0x7f8cc4004390 2024-04-25T08:32:05.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.948+0000 7f8cd17fa640 1 -- 172.21.15.60:0/3536496002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8cc8058280 con 0x7f8cc4004390 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8cb806b4d0 msgr2=0x7f8cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8cb806b4d0 0x7f8cb806d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0034000 tx=0x7f8cc0009030 comp rx=0 tx=0).stop 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 msgr2=0x7f8cd4151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cd4151fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8007370 tx=0x7f8cc8007830 comp rx=0 tx=0).stop 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 shutdown_connections 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8cb806b4d0 0x7f8cb806d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8cd40b8e60 0x7f8cd41f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8cd40b89e0 0x7f8cd41f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 --2- 172.21.15.60:0/3536496002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8cc4004390 0x7f8cd4151fa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 >> 172.21.15.60:0/3536496002 conn(0x7f8cd40d04a0 msgr2=0x7f8cd415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 shutdown_connections 2024-04-25T08:32:05.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:05.950+0000 7f8cdaee1640 1 -- 172.21.15.60:0/3536496002 wait complete. 2024-04-25T08:32:05.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T08:32:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.063+0000 7f5cda161640 1 Processor -- start 2024-04-25T08:32:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.063+0000 7f5cda161640 1 -- start start 2024-04-25T08:32:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cda161640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cd415db70 0x7f5cd4162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cda161640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5cd40b6cb0 0x7f5cd40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cda161640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd4162690 con 0x7f5cd40b5a60 2024-04-25T08:32:06.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cda161640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40d3800 con 0x7f5cd415db70 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cda161640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40ceee0 con 0x7f5cd40b6cb0 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cd415db70 0x7f5cd4162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5cd415db70 0x7f5cd4162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42886/0 (socket says 172.21.15.60:42886) 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1200092974 learned_addr learned my addr 172.21.15.60:0/1200092974 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1200092974 >> v2:172.21.15.100:3300/0 conn(0x7f5cd40b6cb0 msgr2=0x7f5cd40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 --2- 172.21.15.60:0/1200092974 >> v2:172.21.15.100:3300/0 conn(0x7f5cd40b6cb0 0x7f5cd40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1200092974 >> v1:172.21.15.100:6789/0 conn(0x7f5cd40b5a60 legacy=0x7f5cd406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:06.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.064+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1200092974 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40d1d10 con 0x7f5cd415db70 2024-04-25T08:32:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cd37fe640 1 --2- 172.21.15.60:0/1200092974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd415db70 0x7f5cd4162150 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40d3980 tx=0x7f5cc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2649b5694923d5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cd27fc640 1 -- 172.21.15.60:0/1200092974 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4037070 con 0x7f5cd415db70 2024-04-25T08:32:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cd27fc640 1 -- 172.21.15.60:0/1200092974 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4032440 con 0x7f5cd415db70 2024-04-25T08:32:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cd27fc640 1 -- 172.21.15.60:0/1200092974 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4032810 con 0x7f5cd415db70 2024-04-25T08:32:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 -- 172.21.15.60:0/1200092974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd415db70 msgr2=0x7f5cd4162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 --2- 172.21.15.60:0/1200092974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd415db70 0x7f5cd4162150 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40d3980 tx=0x7f5cc402a180 comp rx=0 tx=0).stop 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 -- 172.21.15.60:0/1200092974 shutdown_connections 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 --2- 172.21.15.60:0/1200092974 >> v2:172.21.15.100:3300/0 conn(0x7f5cd40b6cb0 0x7f5cd40b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 --2- 172.21.15.60:0/1200092974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd415db70 0x7f5cd4162150 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 -- 172.21.15.60:0/1200092974 >> 172.21.15.60:0/1200092974 conn(0x7f5cd40d0520 msgr2=0x7f5cd40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 -- 172.21.15.60:0/1200092974 shutdown_connections 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.065+0000 7f5cda161640 1 -- 172.21.15.60:0/1200092974 wait complete. 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.067+0000 7f5cda161640 1 Processor -- start 2024-04-25T08:32:06.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.067+0000 7f5cda161640 1 -- start start 2024-04-25T08:32:06.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.067+0000 7f5cda161640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 0x7f5cd40c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.067+0000 7f5cda161640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd40b6cb0 0x7f5cd40aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cda161640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cd415db70 0x7f5cd415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cda161640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd415df50 con 0x7f5cd40b5a60 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cda161640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40d22d0 con 0x7f5cd40b6cb0 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cda161640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40ce530 con 0x7f5cd415db70 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 0x7f5cd40c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 0x7f5cd40c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42898/0 (socket says 172.21.15.60:42898) 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1609826124 learned_addr learned my addr 172.21.15.60:0/1609826124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd2ffd640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd40b6cb0 0x7f5cd40aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd3fff640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cd415db70 0x7f5cd415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cd415db70 msgr2=0x7f5cd415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cd415db70 0x7f5cd415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd40b6cb0 msgr2=0x7f5cd40aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd40b6cb0 0x7f5cd40aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 -- 172.21.15.60:0/1609826124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40aadd0 con 0x7f5cd40b5a60 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd2ffd640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd40b6cb0 0x7f5cd40aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd3fff640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cd415db70 0x7f5cd415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cd37fe640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 0x7f5cd40c9f70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4000c00 tx=0x7f5cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4041070 con 0x7f5cd40b5a60 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc403c870 con 0x7f5cd40b5a60 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40cbc70 con 0x7f5cd40b5a60 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4037040 con 0x7f5cd40b5a60 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.068+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40cc230 con 0x7f5cd40b5a60 2024-04-25T08:32:06.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.069+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd4073990 con 0x7f5cd40b5a60 2024-04-25T08:32:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.070+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cc4044070 con 0x7f5cd40b5a60 2024-04-25T08:32:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.070+0000 7f5cbbfff640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5cc006b4d0 0x7f5cc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.070+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5cc40aabb0 con 0x7f5cd40b5a60 2024-04-25T08:32:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.071+0000 7f5cd2ffd640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5cc006b4d0 0x7f5cc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.071+0000 7f5cd2ffd640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5cc006b4d0 0x7f5cc006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40cba60 tx=0x7f5cc8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:06.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.075+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cc4079d50 con 0x7f5cd40b5a60 2024-04-25T08:32:06.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.327+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f5cd40d1820 con 0x7f5cd40b5a60 2024-04-25T08:32:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.628+0000 7f5cbbfff640 1 -- 172.21.15.60:0/1609826124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v149) v1 ==== 148+0+0 (secure 0 0 0) 0x7f5cc4079650 con 0x7f5cd40b5a60 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5cc006b4d0 msgr2=0x7f5cc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5cc006b4d0 0x7f5cc006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40cba60 tx=0x7f5cc8008040 comp rx=0 tx=0).stop 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 msgr2=0x7f5cd40c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 0x7f5cd40c9f70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4000c00 tx=0x7f5cc40046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 shutdown_connections 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5cd415db70 0x7f5cd415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5cc006b4d0 0x7f5cc006d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cd40b6cb0 0x7f5cd40aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 --2- 172.21.15.60:0/1609826124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cd40b5a60 0x7f5cd40c9f70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 >> 172.21.15.60:0/1609826124 conn(0x7f5cd40d0520 msgr2=0x7f5cd40d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 shutdown_connections 2024-04-25T08:32:06.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.630+0000 7f5cda161640 1 -- 172.21.15.60:0/1609826124 wait complete. 2024-04-25T08:32:06.638 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T08:32:06.638 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:32:06.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:06.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.743+0000 7f6ac349a640 1 Processor -- start 2024-04-25T08:32:06.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.744+0000 7f6ac349a640 1 -- start start 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.744+0000 7f6ac349a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 0x7f6abc165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.744+0000 7f6ac349a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc071480 con 0x7f6abc0cfec0 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.744+0000 7f6ac349a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc071760 con 0x7f6abc154ee0 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.744+0000 7f6ac349a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc072090 con 0x7f6abc0ce590 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6ac0a35640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6abc0cfec0 0x7f6abc0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33616/0 (socket says 172.21.15.60:33616) 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6ac0a35640 1 -- 172.21.15.60:0/824749394 learned_addr learned my addr 172.21.15.60:0/824749394 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6ac1a37640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 0x7f6abc165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382754175 0 0) 0x7f6abc071480 con 0x7f6abc0cfec0 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aa4003520 con 0x7f6abc0cfec0 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313870522 0 0) 0x7f6aa4003520 con 0x7f6abc0cfec0 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 msgr2=0x7f6abc165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 0x7f6abc165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 >> v1:172.21.15.100:6789/0 conn(0x7f6abc154ee0 legacy=0x7f6abc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc071a40 con 0x7f6abc0cfec0 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6ab0001670 con 0x7f6abc0cfec0 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 >> v1:172.21.15.60:6789/0 conn(0x7f6abc0cfec0 legacy=0x7f6abc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa40039e0 0x7f6aa4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa4004390 0x7f6aa4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 0x7f6aa40050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa400b9c0 con 0x7f6aa40039e0 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa400bb70 con 0x7f6aa4004390 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa400df80 con 0x7f6abc0ce590 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6ac1236640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa40039e0 0x7f6aa4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6ac0a35640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa4004390 0x7f6aa4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.745+0000 7f6ac1a37640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 0x7f6aa40050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.746+0000 7f6ac1236640 1 -- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 msgr2=0x7f6aa40050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.746+0000 7f6ac1236640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc0ce590 0x7f6aa40050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.746+0000 7f6ac1236640 1 -- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa4004390 msgr2=0x7f6aa4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:06.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.746+0000 7f6ac1236640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa4004390 0x7f6aa4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.746+0000 7f6ac1236640 1 -- 172.21.15.60:0/824749394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4003520 con 0x7f6aa40039e0 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.746+0000 7f6ac1236640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa40039e0 0x7f6aa4003dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4001860 tx=0x7f6aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54cd1893770b568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4037070 con 0x7f6aa40039e0 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab402ac30 con 0x7f6aa40039e0 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6abbfff640 1 -- 172.21.15.60:0/824749394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4001430 con 0x7f6aa40039e0 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac349a640 1 -- 172.21.15.60:0/824749394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa40039e0 msgr2=0x7f6aa4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac349a640 1 --2- 172.21.15.60:0/824749394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6aa40039e0 0x7f6aa4003dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4001860 tx=0x7f6aa4005830 comp rx=0 tx=0).stop 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac1a37640 1 -- 172.21.15.60:0/824749394 reap_dead start 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac349a640 1 -- 172.21.15.60:0/824749394 shutdown_connections 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac349a640 1 -- 172.21.15.60:0/824749394 >> 172.21.15.60:0/824749394 conn(0x7f6abc06abe0 msgr2=0x7f6abc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac349a640 1 -- 172.21.15.60:0/824749394 shutdown_connections 2024-04-25T08:32:06.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.747+0000 7f6ac349a640 1 -- 172.21.15.60:0/824749394 wait complete. 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.748+0000 7f6ac349a640 1 Processor -- start 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.748+0000 7f6ac349a640 1 -- start start 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac349a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa40039e0 0x7f6abc155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac349a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 0x7f6abc15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac349a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc13fe50 0x7f6abc1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac349a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1552c0 con 0x7f6abc143750 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac349a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc071230 con 0x7f6aa40039e0 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac349a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc155440 con 0x7f6abc13fe50 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac0a35640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 0x7f6abc15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac0a35640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 0x7f6abc15d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42914/0 (socket says 172.21.15.60:42914) 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac0a35640 1 -- 172.21.15.60:0/107151860 learned_addr learned my addr 172.21.15.60:0/107151860 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:06.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac1a37640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc13fe50 0x7f6abc1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac0a35640 1 -- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc13fe50 msgr2=0x7f6abc1402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac1236640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa40039e0 0x7f6abc155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.749+0000 7f6ac0a35640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc13fe50 0x7f6abc1402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac0a35640 1 -- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa40039e0 msgr2=0x7f6abc155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac0a35640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa40039e0 0x7f6abc155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac0a35640 1 -- 172.21.15.60:0/107151860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc155e70 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac1236640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa40039e0 0x7f6abc155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac0a35640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 0x7f6abc15d080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0007320 tx=0x7f6ab00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab001a070 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab000f8e0 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc156150 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000fcb0 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.750+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc140cd0 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.751+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc06e4f0 con 0x7f6abc143750 2024-04-25T08:32:06.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.751+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ab001d070 con 0x7f6abc143750 2024-04-25T08:32:06.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.752+0000 7f6ab9ffb640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6aa806b4d0 0x7f6aa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:06.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.752+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6ab0089900 con 0x7f6abc143750 2024-04-25T08:32:06.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.752+0000 7f6ac1236640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6aa806b4d0 0x7f6aa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:06.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.753+0000 7f6ac1236640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6aa806b4d0 0x7f6aa806d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4034000 tx=0x7f6ab4009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:06.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.756+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6abc06e4f0 con 0x7f6abc143750 2024-04-25T08:32:06.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.938+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6abc073990 con 0x7f6abc143750 2024-04-25T08:32:06.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.938+0000 7f6ab9ffb640 1 -- 172.21.15.60:0/107151860 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13281 (secure 0 0 0) 0x7f6ab0058980 con 0x7f6abc143750 2024-04-25T08:32:06.935 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:06.935 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":46,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:05.626519+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/2407433231":"2024-04-26T08:32:02.621886+0000","172.21.15.60:6833/2407433231":"2024-04-26T08:32:02.621886+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6aa806b4d0 msgr2=0x7f6aa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6aa806b4d0 0x7f6aa806d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4034000 tx=0x7f6ab4009030 comp rx=0 tx=0).stop 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 msgr2=0x7f6abc15d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 0x7f6abc15d080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0007320 tx=0x7f6ab00077e0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 shutdown_connections 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6abc13fe50 0x7f6abc1402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6abc143750 0x7f6abc15d080 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6aa806b4d0 0x7f6aa806d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 --2- 172.21.15.60:0/107151860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6aa40039e0 0x7f6abc155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:06.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.940+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 >> 172.21.15.60:0/107151860 conn(0x7f6abc06abe0 msgr2=0x7f6abc0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:06.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.941+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 shutdown_connections 2024-04-25T08:32:06.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:06.941+0000 7f6ac349a640 1 -- 172.21.15.60:0/107151860 wait complete. 2024-04-25T08:32:06.948 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:32:06.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.056+0000 7f1d3394c640 1 Processor -- start 2024-04-25T08:32:07.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.056+0000 7f1d3394c640 1 -- start start 2024-04-25T08:32:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d3394c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d2c0ced80 0x7f1d2c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d3394c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1d2c161710 0x7f1d2c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d3394c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c15e000 con 0x7f1d2c15dc20 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d3394c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c0d1de0 con 0x7f1d2c0ced80 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d3394c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c161c60 con 0x7f1d2c161710 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d2c0ced80 0x7f1d2c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d2c0ced80 0x7f1d2c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42928/0 (socket says 172.21.15.60:42928) 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 -- 172.21.15.60:0/2815184146 learned_addr learned my addr 172.21.15.60:0/2815184146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d31ee9640 1 --2- 172.21.15.60:0/2815184146 >> v2:172.21.15.100:3300/0 conn(0x7f1d2c161710 0x7f1d2c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 -- 172.21.15.60:0/2815184146 >> v2:172.21.15.100:3300/0 conn(0x7f1d2c161710 msgr2=0x7f1d2c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 --2- 172.21.15.60:0/2815184146 >> v2:172.21.15.100:3300/0 conn(0x7f1d2c161710 0x7f1d2c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 -- 172.21.15.60:0/2815184146 >> v1:172.21.15.100:6789/0 conn(0x7f1d2c15dc20 legacy=0x7f1d2c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d30ee7640 1 -- 172.21.15.60:0/2815184146 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c161f40 con 0x7f1d2c0ced80 2024-04-25T08:32:07.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.057+0000 7f1d31ee9640 1 --2- 172.21.15.60:0/2815184146 >> v2:172.21.15.100:3300/0 conn(0x7f1d2c161710 0x7f1d2c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d30ee7640 1 --2- 172.21.15.60:0/2815184146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c0d22e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0b6570 tx=0x7f1d2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b027fa7160d00df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d2bfff640 1 -- 172.21.15.60:0/2815184146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20037070 con 0x7f1d2c0ced80 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d2bfff640 1 -- 172.21.15.60:0/2815184146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d20032440 con 0x7f1d2c0ced80 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d2bfff640 1 -- 172.21.15.60:0/2815184146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20032810 con 0x7f1d2c0ced80 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d3394c640 1 -- 172.21.15.60:0/2815184146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 msgr2=0x7f1d2c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2815184146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c0d22e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0b6570 tx=0x7f1d2002a230 comp rx=0 tx=0).stop 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d3394c640 1 -- 172.21.15.60:0/2815184146 shutdown_connections 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2815184146 >> v2:172.21.15.100:3300/0 conn(0x7f1d2c161710 0x7f1d2c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.058+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2815184146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c0d22e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.059+0000 7f1d3394c640 1 -- 172.21.15.60:0/2815184146 >> 172.21.15.60:0/2815184146 conn(0x7f1d2c0d0420 msgr2=0x7f1d2c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.059+0000 7f1d3394c640 1 -- 172.21.15.60:0/2815184146 shutdown_connections 2024-04-25T08:32:07.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.059+0000 7f1d3394c640 1 -- 172.21.15.60:0/2815184146 wait complete. 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.060+0000 7f1d3394c640 1 Processor -- start 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.060+0000 7f1d3394c640 1 -- start start 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d2c15dc20 0x7f1d2c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d2c161710 0x7f1d2c0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b5e50 con 0x7f1d2c0ced80 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c162500 con 0x7f1d2c161710 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b5fd0 con 0x7f1d2c15dc20 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42934/0 (socket says 172.21.15.60:42934) 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 -- 172.21.15.60:0/2406526534 learned_addr learned my addr 172.21.15.60:0/2406526534 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d31ee9640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d2c161710 0x7f1d2c0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d30ee7640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d2c15dc20 0x7f1d2c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 -- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d2c15dc20 msgr2=0x7f1d2c0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d2c15dc20 0x7f1d2c0c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 -- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d2c161710 msgr2=0x7f1d2c0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d2c161710 0x7f1d2c0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 -- 172.21.15.60:0/2406526534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0c8790 con 0x7f1d2c0ced80 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d316e8640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c152040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00ba80 tx=0x7f1d1c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00cc00 con 0x7f1d2c0ced80 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c00cda0 con 0x7f1d2c0ced80 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c0c8a70 con 0x7f1d2c0ced80 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.061+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c003bb0 con 0x7f1d2c0ced80 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.062+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c0c9310 con 0x7f1d2c0ced80 2024-04-25T08:32:07.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.062+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c0c8e60 con 0x7f1d2c0ced80 2024-04-25T08:32:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.063+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d1c014440 con 0x7f1d2c0ced80 2024-04-25T08:32:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.063+0000 7f1d29ffb640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.063+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1d1c089f40 con 0x7f1d2c0ced80 2024-04-25T08:32:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.064+0000 7f1d30ee7640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.064+0000 7f1d30ee7640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d20034000 tx=0x7f1d2002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.068+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d1c0588e0 con 0x7f1d2c0ced80 2024-04-25T08:32:07.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.248+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d2c0b93c0 con 0x7f1d2c0ced80 2024-04-25T08:32:07.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.248+0000 7f1d29ffb640 1 -- 172.21.15.60:0/2406526534 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d2c0b93c0 con 0x7f1d2c0ced80 2024-04-25T08:32:07.245 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:32:07.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 msgr2=0x7f1d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d20034000 tx=0x7f1d2002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 msgr2=0x7f1d2c152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c152040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00ba80 tx=0x7f1d1c00bf40 comp rx=0 tx=0).stop 2024-04-25T08:32:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 shutdown_connections 2024-04-25T08:32:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d2c161710 0x7f1d2c0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d2c15dc20 0x7f1d2c0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 --2- 172.21.15.60:0/2406526534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d2c0ced80 0x7f1d2c152040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 >> 172.21.15.60:0/2406526534 conn(0x7f1d2c0d0420 msgr2=0x7f1d2c0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 shutdown_connections 2024-04-25T08:32:07.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.250+0000 7f1d3394c640 1 -- 172.21.15.60:0/2406526534 wait complete. 2024-04-25T08:32:07.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T08:32:07.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.363+0000 7f2737df4640 1 Processor -- start 2024-04-25T08:32:07.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.363+0000 7f2737df4640 1 -- start start 2024-04-25T08:32:07.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2737df4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 0x7f27300d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2737df4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f27300b2cb0 0x7f27300d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2737df4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27300b97a0 con 0x7f27300b92a0 2024-04-25T08:32:07.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2737df4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273015dfc0 con 0x7f27300b2cb0 2024-04-25T08:32:07.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2737df4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27300d1a00 con 0x7f273015dbe0 2024-04-25T08:32:07.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2736391640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f27300b92a0 0x7f27300b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33624/0 (socket says 172.21.15.60:33624) 2024-04-25T08:32:07.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2736391640 1 -- 172.21.15.60:0/1256526349 learned_addr learned my addr 172.21.15.60:0/1256526349 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924572986 0 0) 0x7f27300b97a0 con 0x7f27300b92a0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f273538f640 1 --2- 172.21.15.60:0/1256526349 >> v2:172.21.15.100:3300/0 conn(0x7f27300b2cb0 0x7f27300d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2735b90640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 0x7f27300d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.364+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2718003520 con 0x7f27300b92a0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1608115339 0 0) 0x7f2718003520 con 0x7f27300b92a0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 msgr2=0x7f27300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 0x7f27300d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 >> v2:172.21.15.100:3300/0 conn(0x7f27300b2cb0 msgr2=0x7f27300d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 --2- 172.21.15.60:0/1256526349 >> v2:172.21.15.100:3300/0 conn(0x7f27300b2cb0 0x7f27300d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27300d1c50 con 0x7f27300b92a0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f2728001670 con 0x7f27300b92a0 2024-04-25T08:32:07.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 >> v1:172.21.15.60:6789/0 conn(0x7f27300b92a0 legacy=0x7f27300b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27180039b0 0x7f2718003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2718004800 0x7f2718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 0x7f27180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300d1a00 con 0x7f2718004800 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271800b9c0 con 0x7f27180039b0 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271800bb40 con 0x7f273015dbe0 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f273538f640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2718004800 0x7f2718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2735b90640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27180039b0 0x7f2718003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f2736391640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 0x7f27180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f273538f640 1 -- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 msgr2=0x7f27180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f273538f640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f273015dbe0 0x7f27180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f273538f640 1 -- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27180039b0 msgr2=0x7f2718003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f273538f640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27180039b0 0x7f2718003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.365+0000 7f273538f640 1 -- 172.21.15.60:0/1256526349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2718003520 con 0x7f2718004800 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.366+0000 7f273538f640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2718004800 0x7f2718004c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2724009ac0 tx=0x7f2718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1363e81fbcf343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.366+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724036070 con 0x7f2718004800 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.366+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2724031400 con 0x7f2718004800 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.366+0000 7f2734b8e640 1 -- 172.21.15.60:0/1256526349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724009690 con 0x7f2718004800 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2737df4640 1 -- 172.21.15.60:0/1256526349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2718004800 msgr2=0x7f2718004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2737df4640 1 --2- 172.21.15.60:0/1256526349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2718004800 0x7f2718004c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2724009ac0 tx=0x7f2718005830 comp rx=0 tx=0).stop 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2736391640 1 -- 172.21.15.60:0/1256526349 reap_dead start 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2737df4640 1 -- 172.21.15.60:0/1256526349 shutdown_connections 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2737df4640 1 -- 172.21.15.60:0/1256526349 >> 172.21.15.60:0/1256526349 conn(0x7f2730061a20 msgr2=0x7f2730061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2737df4640 1 -- 172.21.15.60:0/1256526349 shutdown_connections 2024-04-25T08:32:07.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.367+0000 7f2737df4640 1 -- 172.21.15.60:0/1256526349 wait complete. 2024-04-25T08:32:07.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.368+0000 7f2737df4640 1 Processor -- start 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.368+0000 7f2737df4640 1 -- start start 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2737df4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2718004800 0x7f2730156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2737df4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 0x7f27301530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2737df4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27300b52f0 0x7f27300b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2737df4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300b6ab0 con 0x7f27301537c0 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2737df4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300d2210 con 0x7f27300b52f0 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2737df4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27300b6c30 con 0x7f2718004800 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 0x7f27301530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 0x7f27301530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42952/0 (socket says 172.21.15.60:42952) 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 -- 172.21.15.60:0/2730489216 learned_addr learned my addr 172.21.15.60:0/2730489216 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2735b90640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2718004800 0x7f2730156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2736391640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27300b52f0 0x7f27300b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 -- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2718004800 msgr2=0x7f2730156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2718004800 0x7f2730156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 -- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27300b52f0 msgr2=0x7f27300b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27300b52f0 0x7f27300b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 -- 172.21.15.60:0/2730489216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2730156600 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f2735b90640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2718004800 0x7f2730156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f273538f640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 0x7f27301530a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f272402abe0 tx=0x7f2724004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724040020 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.369+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2724003890 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.370+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724036040 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.370+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27301568e0 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.370+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273015dfc0 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.371+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27300b5cb0 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.371+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2724043020 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.371+0000 7f272e7fc640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f271c06b500 0x7f271c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.372+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f27240a9aa0 con 0x7f27301537c0 2024-04-25T08:32:07.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.372+0000 7f2735b90640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f271c06b500 0x7f271c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.372+0000 7f2735b90640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f271c06b500 0x7f271c06d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2720013300 tx=0x7f272000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.376+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2724078b20 con 0x7f27301537c0 2024-04-25T08:32:07.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:07.632+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f2730073990 con 0x7f27301537c0 2024-04-25T08:32:08.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.633+0000 7f272e7fc640 1 -- 172.21.15.60:0/2730489216 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v151) v1 ==== 146+0+0 (secure 0 0 0) 0x7f2724078420 con 0x7f27301537c0 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f271c06b500 msgr2=0x7f271c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f271c06b500 0x7f271c06d9c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2720013300 tx=0x7f272000b040 comp rx=0 tx=0).stop 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 msgr2=0x7f27301530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 0x7f27301530a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f272402abe0 tx=0x7f2724004730 comp rx=0 tx=0).stop 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 shutdown_connections 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f271c06b500 0x7f271c06d9c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27300b52f0 0x7f27300b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f27301537c0 0x7f27301530a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 --2- 172.21.15.60:0/2730489216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2718004800 0x7f2730156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 >> 172.21.15.60:0/2730489216 conn(0x7f2730061a20 msgr2=0x7f273015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 shutdown_connections 2024-04-25T08:32:08.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.636+0000 7f2737df4640 1 -- 172.21.15.60:0/2730489216 wait complete. 2024-04-25T08:32:08.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:32:08.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 Processor -- start 2024-04-25T08:32:08.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 -- start start 2024-04-25T08:32:08.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f05480d1f30 0x7f05480d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05480d34c0 0x7f05480d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480d3030 con 0x7f05480d1f30 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480d2420 con 0x7f05480d34c0 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.761+0000 7f054dd1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480d2700 con 0x7f054815dac0 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f05480d1f30 0x7f05480d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f05480d1f30 0x7f05480d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42966/0 (socket says 172.21.15.60:42966) 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 -- 172.21.15.60:0/1605817079 learned_addr learned my addr 172.21.15.60:0/1605817079 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f05477fe640 1 --2- 172.21.15.60:0/1605817079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0547fff640 1 --2- 172.21.15.60:0/1605817079 >> v2:172.21.15.100:3300/0 conn(0x7f05480d34c0 0x7f05480d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 -- 172.21.15.60:0/1605817079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 msgr2=0x7f05480d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 --2- 172.21.15.60:0/1605817079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 -- 172.21.15.60:0/1605817079 >> v2:172.21.15.100:3300/0 conn(0x7f05480d34c0 msgr2=0x7f05480d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 --2- 172.21.15.60:0/1605817079 >> v2:172.21.15.100:3300/0 conn(0x7f05480d34c0 0x7f05480d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.762+0000 7f0546ffd640 1 -- 172.21.15.60:0/1605817079 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480d29e0 con 0x7f05480d1f30 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f0546ffd640 1 --2- 172.21.15.60:0/1605817079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f05480d03a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f054806cc00 tx=0x7f053c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b50c27c9b929e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f05467fc640 1 -- 172.21.15.60:0/1605817079 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c004030 con 0x7f05480d1f30 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f05467fc640 1 -- 172.21.15.60:0/1605817079 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c0041d0 con 0x7f05480d1f30 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f05467fc640 1 -- 172.21.15.60:0/1605817079 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c003610 con 0x7f05480d1f30 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 -- 172.21.15.60:0/1605817079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 msgr2=0x7f05480d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 --2- 172.21.15.60:0/1605817079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f05480d03a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f054806cc00 tx=0x7f053c02a250 comp rx=0 tx=0).stop 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 -- 172.21.15.60:0/1605817079 shutdown_connections 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 --2- 172.21.15.60:0/1605817079 >> v2:172.21.15.100:3300/0 conn(0x7f05480d34c0 0x7f05480d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 --2- 172.21.15.60:0/1605817079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f05480d03a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 --2- 172.21.15.60:0/1605817079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 -- 172.21.15.60:0/1605817079 >> 172.21.15.60:0/1605817079 conn(0x7f0548061c30 msgr2=0x7f054806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:08.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 -- 172.21.15.60:0/1605817079 shutdown_connections 2024-04-25T08:32:08.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.763+0000 7f054dd1d640 1 -- 172.21.15.60:0/1605817079 wait complete. 2024-04-25T08:32:08.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 Processor -- start 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 -- start start 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f054815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05480d34c0 0x7f05480b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480b97d0 con 0x7f05480d1f30 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815dea0 con 0x7f05480d34c0 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.765+0000 7f054dd1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480b9950 con 0x7f054815dac0 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f054815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f054815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42968/0 (socket says 172.21.15.60:42968) 2024-04-25T08:32:08.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 -- 172.21.15.60:0/3834563167 learned_addr learned my addr 172.21.15.60:0/3834563167 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f0546ffd640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05480d34c0 0x7f05480b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f0547fff640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 -- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 msgr2=0x7f05480b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 -- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05480d34c0 msgr2=0x7f05480b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05480d34c0 0x7f05480b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 -- 172.21.15.60:0/3834563167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480cc680 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f0546ffd640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05480d34c0 0x7f05480b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f05477fe640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f054815e0b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f053800d5f0 tx=0x7f053800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380125f0 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538019070 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05480cc960 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380129d0 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.766+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05480ccf20 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.767+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548073a00 con 0x7f05480d1f30 2024-04-25T08:32:08.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.768+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0538012b70 con 0x7f05480d1f30 2024-04-25T08:32:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.768+0000 7f052ffff640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f053406b4d0 0x7f053406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.768+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f053808a370 con 0x7f05480d1f30 2024-04-25T08:32:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.768+0000 7f0546ffd640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f053406b4d0 0x7f053406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:08.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.769+0000 7f0546ffd640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f053406b4d0 0x7f053406d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f053c033000 tx=0x7f053c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:08.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.772+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05380593f0 con 0x7f05480d1f30 2024-04-25T08:32:08.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.950+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05480b2440 con 0x7f05480d1f30 2024-04-25T08:32:08.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.954+0000 7f052ffff640 1 -- 172.21.15.60:0/3834563167 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0538058cf0 con 0x7f05480d1f30 2024-04-25T08:32:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.955+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f053406b4d0 msgr2=0x7f053406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f053406b4d0 0x7f053406d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f053c033000 tx=0x7f053c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 msgr2=0x7f054815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:08.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f054815e0b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f053800d5f0 tx=0x7f053800dab0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 shutdown_connections 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f054815dac0 0x7f05480b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f053406b4d0 0x7f053406d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05480d34c0 0x7f05480b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 --2- 172.21.15.60:0/3834563167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05480d1f30 0x7f054815e0b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 >> 172.21.15.60:0/3834563167 conn(0x7f0548061c30 msgr2=0x7f05480b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 shutdown_connections 2024-04-25T08:32:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:08.956+0000 7f054dd1d640 1 -- 172.21.15.60:0/3834563167 wait complete. 2024-04-25T08:32:08.963 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:32:08.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:09.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:09.050 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:32:09.050 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:09.050 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:09.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:32:09.066 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:32:09.066 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:09.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:32:09.085 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:09.085 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:09.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:32:09.121 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:32:09.121 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:09.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:32:09.142 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:32:09.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.264+0000 7fe41bf63640 1 Processor -- start 2024-04-25T08:32:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.265+0000 7fe41bf63640 1 -- start start 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.265+0000 7fe41bf63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe41415dbe0 0x7fe4140d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.265+0000 7fe41bf63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.265+0000 7fe41bf63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140b97a0 con 0x7fe4140b2cb0 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.265+0000 7fe41bf63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41415dfc0 con 0x7fe41415dbe0 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.265+0000 7fe41bf63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140d1a00 con 0x7fe4140b92a0 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe41415dbe0 0x7fe4140d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe41415dbe0 0x7fe4140d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42976/0 (socket says 172.21.15.60:42976) 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 -- 172.21.15.60:0/4081347809 learned_addr learned my addr 172.21.15.60:0/4081347809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe41a500640 1 --2- 172.21.15.60:0/4081347809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 -- 172.21.15.60:0/4081347809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 msgr2=0x7fe4140d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 --2- 172.21.15.60:0/4081347809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 -- 172.21.15.60:0/4081347809 >> v1:172.21.15.100:6789/0 conn(0x7fe4140b2cb0 legacy=0x7fe4140b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.266+0000 7fe419cff640 1 -- 172.21.15.60:0/4081347809 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140d1c50 con 0x7fe41415dbe0 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe419cff640 1 --2- 172.21.15.60:0/4081347809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41415dbe0 0x7fe4140d0910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe410009870 tx=0x7fe41002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6932e70978b37c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe418cfd640 1 -- 172.21.15.60:0/4081347809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410037070 con 0x7fe41415dbe0 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe418cfd640 1 -- 172.21.15.60:0/4081347809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410032440 con 0x7fe41415dbe0 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe418cfd640 1 -- 172.21.15.60:0/4081347809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410032810 con 0x7fe41415dbe0 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe41bf63640 1 -- 172.21.15.60:0/4081347809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41415dbe0 msgr2=0x7fe4140d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe41bf63640 1 --2- 172.21.15.60:0/4081347809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41415dbe0 0x7fe4140d0910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe410009870 tx=0x7fe41002a270 comp rx=0 tx=0).stop 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe41bf63640 1 -- 172.21.15.60:0/4081347809 shutdown_connections 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe41bf63640 1 --2- 172.21.15.60:0/4081347809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe41bf63640 1 --2- 172.21.15.60:0/4081347809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41415dbe0 0x7fe4140d0910 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.267+0000 7fe41bf63640 1 -- 172.21.15.60:0/4081347809 >> 172.21.15.60:0/4081347809 conn(0x7fe414061a20 msgr2=0x7fe414061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:09.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.268+0000 7fe41bf63640 1 -- 172.21.15.60:0/4081347809 shutdown_connections 2024-04-25T08:32:09.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.268+0000 7fe41bf63640 1 -- 172.21.15.60:0/4081347809 wait complete. 2024-04-25T08:32:09.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 Processor -- start 2024-04-25T08:32:09.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 -- start start 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 0x7fe4140c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41415dbe0 0x7fe4141f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140b6ab0 con 0x7fe4140b2cb0 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140d2210 con 0x7fe41415dbe0 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.269+0000 7fe41bf63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140b6c30 con 0x7fe4140b92a0 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 0x7fe4140c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 0x7fe4140c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42992/0 (socket says 172.21.15.60:42992) 2024-04-25T08:32:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 -- 172.21.15.60:0/1558928890 learned_addr learned my addr 172.21.15.60:0/1558928890 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 -- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 msgr2=0x7fe4140cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 -- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41415dbe0 msgr2=0x7fe4141f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe41a500640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41415dbe0 0x7fe4141f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41415dbe0 0x7fe4141f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 -- 172.21.15.60:0/1558928890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4141f4900 con 0x7fe4140b2cb0 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe419cff640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 0x7fe4140c9610 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe410009870 tx=0x7fe4100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410041020 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100306c0 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4140cbd10 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410037040 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.270+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4141f3bc0 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.271+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4141f39b0 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.272+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe410044020 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.272+0000 7fe40a7fc640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.272+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe4100aa890 con 0x7fe4140b2cb0 2024-04-25T08:32:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.272+0000 7fe4194fe640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.273+0000 7fe4194fe640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3f806b4d0 0x7fe3f806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe404000f60 tx=0x7fe4040079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.277+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe41004b030 con 0x7fe4140b2cb0 2024-04-25T08:32:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.496+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4140cc230 con 0x7fe4140b2cb0 2024-04-25T08:32:09.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.496+0000 7fe40a7fc640 1 -- 172.21.15.60:0/1558928890 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fe4141f4900 con 0x7fe4140b2cb0 2024-04-25T08:32:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:09.493 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4823,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/177001298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":177001298},{"type":"v1","addr":"172.21.15.100:6836","nonce":177001298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4826,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2201183413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2201183413},{"type":"v1","addr":"172.21.15.100:6837","nonce":2201183413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3764655471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3764655471},{"type":"v1","addr":"172.21.15.60:6834","nonce":3764655471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5811,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1254965533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1254965533},{"type":"v1","addr":"172.21.15.60:6835","nonce":1254965533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2024-04-25T08:32:09.493 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 18 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3f806b4d0 msgr2=0x7fe3f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3f806b4d0 0x7fe3f806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe404000f60 tx=0x7fe4040079d0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 msgr2=0x7fe4140c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 0x7fe4140c9610 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe410009870 tx=0x7fe4100046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 shutdown_connections 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe41415dbe0 0x7fe4141f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4140b92a0 0x7fe4140cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 --2- 172.21.15.60:0/1558928890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4140b2cb0 0x7fe4140c9610 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 >> 172.21.15.60:0/1558928890 conn(0x7fe414061a20 msgr2=0x7fe41415c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 shutdown_connections 2024-04-25T08:32:09.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.498+0000 7fe41bf63640 1 -- 172.21.15.60:0/1558928890 wait complete. 2024-04-25T08:32:09.506 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:32:09.506 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:32:09.506 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:32:09.506 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:09.507 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:32:09.507 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:32:09.507 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:32:09.507 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:09.507 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:32:09.507 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:32:09.507 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:32:09.507 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:09.508 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:32:09.508 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:32:09.508 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:32:09.508 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:09.516 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:32:09.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:32:09.517 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:32:09.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:32:09.520 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:32:09.520 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:32:09.520 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:32:09.521 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:32:09.558 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:32:09.559 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:09.559 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:32:09.559 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:09.562 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:32:09.562 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:09.563 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:32:09.563 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:09.564 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:32:09.635 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:32:09.635 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:32:09.641 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:32:09.642 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:32:09.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.673+0000 7f2b5c7b0640 1 Processor -- start 2024-04-25T08:32:09.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.673+0000 7f2b5c7b0640 1 -- start start 2024-04-25T08:32:09.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.674+0000 7f2b5c7b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b541610f0 0x7f2b5406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.674+0000 7f2b5c7b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b541616f0 0x7f2b54161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.674+0000 7f2b5ad4d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2b540ce0a0 0x7f2b540d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33648/0 (socket says 172.21.15.60:33648) 2024-04-25T08:32:09.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.674+0000 7f2b5ad4d640 1 -- 172.21.15.60:0/2433861828 learned_addr learned my addr 172.21.15.60:0/2433861828 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:09.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.674+0000 7f2b5c7b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b540c6960 con 0x7f2b540ce0a0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b541614d0 con 0x7f2b541610f0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54162260 con 0x7f2b541616f0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b59d4b640 1 --2- 172.21.15.60:0/2433861828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b541616f0 0x7f2b54161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b5a54c640 1 --2- 172.21.15.60:0/2433861828 >> v2:172.21.15.100:3300/0 conn(0x7f2b541610f0 0x7f2b5406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b5954a640 1 -- 172.21.15.60:0/2433861828 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351169227 0 0) 0x7f2b540c6960 con 0x7f2b540ce0a0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b5954a640 1 -- 172.21.15.60:0/2433861828 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b40003520 con 0x7f2b540ce0a0 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b59d4b640 1 -- 172.21.15.60:0/2433861828 >> v2:172.21.15.100:3300/0 conn(0x7f2b541610f0 msgr2=0x7f2b5406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b59d4b640 1 --2- 172.21.15.60:0/2433861828 >> v2:172.21.15.100:3300/0 conn(0x7f2b541610f0 0x7f2b5406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b59d4b640 1 -- 172.21.15.60:0/2433861828 >> v1:172.21.15.60:6789/0 conn(0x7f2b540ce0a0 legacy=0x7f2b540d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.675+0000 7f2b59d4b640 1 -- 172.21.15.60:0/2433861828 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54162540 con 0x7f2b541616f0 2024-04-25T08:32:09.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.676+0000 7f2b59d4b640 1 --2- 172.21.15.60:0/2433861828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b541616f0 0x7f2b54161b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b500097a0 tx=0x7f2b5002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300edf4d0091a7da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.676+0000 7f2b5954a640 1 -- 172.21.15.60:0/2433861828 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50037070 con 0x7f2b541616f0 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.676+0000 7f2b5954a640 1 -- 172.21.15.60:0/2433861828 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50032440 con 0x7f2b541616f0 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.676+0000 7f2b5954a640 1 -- 172.21.15.60:0/2433861828 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50032810 con 0x7f2b541616f0 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.676+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b541616f0 msgr2=0x7f2b54161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.676+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/2433861828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b541616f0 0x7f2b54161b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b500097a0 tx=0x7f2b5002a180 comp rx=0 tx=0).stop 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.677+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 shutdown_connections 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.677+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/2433861828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b541616f0 0x7f2b54161b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.677+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/2433861828 >> v2:172.21.15.100:3300/0 conn(0x7f2b541610f0 0x7f2b5406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.677+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 >> 172.21.15.60:0/2433861828 conn(0x7f2b5406f500 msgr2=0x7f2b5406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.677+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 shutdown_connections 2024-04-25T08:32:09.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.677+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/2433861828 wait complete. 2024-04-25T08:32:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.678+0000 7f2b5c7b0640 1 Processor -- start 2024-04-25T08:32:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.678+0000 7f2b5c7b0640 1 -- start start 2024-04-25T08:32:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b540ce0a0 0x7f2b541536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b541610f0 0x7f2b5415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 0x7f2b5415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540cf690 con 0x7f2b541616f0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540cfce0 con 0x7f2b541610f0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5406c700 con 0x7f2b540ce0a0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 0x7f2b5415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 0x7f2b5415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43034/0 (socket says 172.21.15.60:43034) 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 -- 172.21.15.60:0/933930525 learned_addr learned my addr 172.21.15.60:0/933930525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b59d4b640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b541610f0 0x7f2b5415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5a54c640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b540ce0a0 0x7f2b541536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 -- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b540ce0a0 msgr2=0x7f2b541536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b540ce0a0 0x7f2b541536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 -- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b541610f0 msgr2=0x7f2b5415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b541610f0 0x7f2b5415c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 -- 172.21.15.60:0/933930525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540b5f70 con 0x7f2b541616f0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5ad4d640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 0x7f2b5415c6f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00ca20 tx=0x7f2b4c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c017070 con 0x7f2b541616f0 2024-04-25T08:32:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b540b6210 con 0x7f2b541616f0 2024-04-25T08:32:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c00abd0 con 0x7f2b541616f0 2024-04-25T08:32:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c012400 con 0x7f2b541616f0 2024-04-25T08:32:09.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.679+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b540b67d0 con 0x7f2b541616f0 2024-04-25T08:32:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.681+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b540b3130 con 0x7f2b541616f0 2024-04-25T08:32:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.681+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b4c01a070 con 0x7f2b541616f0 2024-04-25T08:32:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.682+0000 7f2b46ffd640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b3806b4d0 0x7f2b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.682+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2b4c089ea0 con 0x7f2b541616f0 2024-04-25T08:32:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.682+0000 7f2b5a54c640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b3806b4d0 0x7f2b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.682+0000 7f2b5a54c640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b3806b4d0 0x7f2b3806d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009ad0 tx=0x7f2b48008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.686+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b540b3130 con 0x7f2b541616f0 2024-04-25T08:32:09.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:09.877+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2b54073a00 con 0x7f2b541616f0 2024-04-25T08:32:10.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.971+0000 7f2b46ffd640 1 -- 172.21.15.60:0/933930525 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2b4c058e10 con 0x7f2b541616f0 2024-04-25T08:32:10.967 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:32:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b3806b4d0 msgr2=0x7f2b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b3806b4d0 0x7f2b3806d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009ad0 tx=0x7f2b48008040 comp rx=0 tx=0).stop 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 msgr2=0x7f2b5415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 0x7f2b5415c6f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00ca20 tx=0x7f2b4c00cee0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 shutdown_connections 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541616f0 0x7f2b5415c6f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b3806b4d0 0x7f2b3806d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b541610f0 0x7f2b5415c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 --2- 172.21.15.60:0/933930525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b540ce0a0 0x7f2b541536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 >> 172.21.15.60:0/933930525 conn(0x7f2b5406f500 msgr2=0x7f2b54071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 shutdown_connections 2024-04-25T08:32:10.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:10.974+0000 7f2b5c7b0640 1 -- 172.21.15.60:0/933930525 wait complete. 2024-04-25T08:32:10.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:32:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.094+0000 7f7e667ce640 1 Processor -- start 2024-04-25T08:32:11.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.094+0000 7f7e667ce640 1 -- start start 2024-04-25T08:32:11.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e667ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7e6015dbe0 0x7f7e600d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e667ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 0x7f7e600d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e667ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e600b97a0 con 0x7f7e600b2cb0 2024-04-25T08:32:11.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e667ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6015dfc0 con 0x7f7e6015dbe0 2024-04-25T08:32:11.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e667ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600d1a00 con 0x7f7e600b92a0 2024-04-25T08:32:11.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e5f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7e600b2cb0 0x7f7e600b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33658/0 (socket says 172.21.15.60:33658) 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.095+0000 7f7e5f7fe640 1 -- 172.21.15.60:0/3592212045 learned_addr learned my addr 172.21.15.60:0/3592212045 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936502158 0 0) 0x7f7e600b97a0 con 0x7f7e600b2cb0 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5ffff640 1 --2- 172.21.15.60:0/3592212045 >> v2:172.21.15.100:3300/0 conn(0x7f7e6015dbe0 0x7f7e600d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e64d6b640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 0x7f7e600d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e48003520 con 0x7f7e600b2cb0 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1122404413 0 0) 0x7f7e48003520 con 0x7f7e600b2cb0 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 msgr2=0x7f7e600d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 0x7f7e600d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 >> v2:172.21.15.100:3300/0 conn(0x7f7e6015dbe0 msgr2=0x7f7e600d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 --2- 172.21.15.60:0/3592212045 >> v2:172.21.15.100:3300/0 conn(0x7f7e6015dbe0 0x7f7e600d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600d1c50 con 0x7f7e600b2cb0 2024-04-25T08:32:11.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f7e54001670 con 0x7f7e600b2cb0 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 >> v1:172.21.15.60:6789/0 conn(0x7f7e600b2cb0 legacy=0x7f7e600b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 0x7f7e480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600d1a00 con 0x7f7e48004800 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4800b9f0 con 0x7f7e480039b0 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4800bbf0 con 0x7f7e600b92a0 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5f7fe640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e5ffff640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.096+0000 7f7e64d6b640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 0x7f7e480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5f7fe640 1 -- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 msgr2=0x7f7e480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5f7fe640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e600b92a0 0x7f7e480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5f7fe640 1 -- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e480039b0 msgr2=0x7f7e48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5f7fe640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e480039b0 0x7f7e48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5f7fe640 1 -- 172.21.15.60:0/3592212045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48003520 con 0x7f7e48004800 2024-04-25T08:32:11.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5f7fe640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7e600d3930 tx=0x7f7e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277455a0ef72cf4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54007d50 con 0x7f7e48004800 2024-04-25T08:32:11.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e54005af0 con 0x7f7e48004800 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.097+0000 7f7e5effd640 1 -- 172.21.15.60:0/3592212045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54007070 con 0x7f7e48004800 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e667ce640 1 -- 172.21.15.60:0/3592212045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48004800 msgr2=0x7f7e48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3592212045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e48004800 0x7f7e48004c80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7e600d3930 tx=0x7f7e48005830 comp rx=0 tx=0).stop 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e64d6b640 1 -- 172.21.15.60:0/3592212045 reap_dead start 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e667ce640 1 -- 172.21.15.60:0/3592212045 shutdown_connections 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e667ce640 1 -- 172.21.15.60:0/3592212045 >> 172.21.15.60:0/3592212045 conn(0x7f7e60061a20 msgr2=0x7f7e60061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e667ce640 1 -- 172.21.15.60:0/3592212045 shutdown_connections 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.098+0000 7f7e667ce640 1 -- 172.21.15.60:0/3592212045 wait complete. 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.099+0000 7f7e667ce640 1 Processor -- start 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 -- start start 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e48004800 0x7f7e60153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e600b56a0 0x7f7e6015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 0x7f7e60156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600b6ab0 con 0x7f7e60156980 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600d2210 con 0x7f7e600b56a0 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e667ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600b6c30 con 0x7f7e48004800 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e64d6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 0x7f7e60156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e64d6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 0x7f7e60156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43054/0 (socket says 172.21.15.60:43054) 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.100+0000 7f7e64d6b640 1 -- 172.21.15.60:0/3616358693 learned_addr learned my addr 172.21.15.60:0/3616358693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e5ffff640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e48004800 0x7f7e60153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e5f7fe640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e600b56a0 0x7f7e6015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e64d6b640 1 -- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e48004800 msgr2=0x7f7e60153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e64d6b640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e48004800 0x7f7e60153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e64d6b640 1 -- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e600b56a0 msgr2=0x7f7e6015e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e64d6b640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e600b56a0 0x7f7e6015e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e64d6b640 1 -- 172.21.15.60:0/3616358693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6015f220 con 0x7f7e60156980 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e5f7fe640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e600b56a0 0x7f7e6015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e64d6b640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 0x7f7e60156e00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7e600b96d0 tx=0x7f7e58007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580046a0 con 0x7f7e60156980 2024-04-25T08:32:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5801a070 con 0x7f7e60156980 2024-04-25T08:32:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58015640 con 0x7f7e60156980 2024-04-25T08:32:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6015f500 con 0x7f7e60156980 2024-04-25T08:32:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.101+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6015fac0 con 0x7f7e60156980 2024-04-25T08:32:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.102+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e60073990 con 0x7f7e60156980 2024-04-25T08:32:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.103+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e580157e0 con 0x7f7e60156980 2024-04-25T08:32:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.103+0000 7f7e5cff9640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.103+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7e5808a670 con 0x7f7e60156980 2024-04-25T08:32:11.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.103+0000 7f7e5ffff640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.104+0000 7f7e5ffff640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e6015a0b0 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.107+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e58024020 con 0x7f7e60156980 2024-04-25T08:32:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.353+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7e600b8100 con 0x7f7e60156980 2024-04-25T08:32:11.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.353+0000 7f7e5cff9640 1 -- 172.21.15.60:0/3616358693 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7e58059740 con 0x7f7e60156980 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:32:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:32:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:32:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:32:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:32:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:32:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:32:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:32:11.362 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:11.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e4c06b4d0 msgr2=0x7f7e4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e6015a0b0 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2024-04-25T08:32:11.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 msgr2=0x7f7e60156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 0x7f7e60156e00 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7e600b96d0 tx=0x7f7e58007700 comp rx=0 tx=0).stop 2024-04-25T08:32:11.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 shutdown_connections 2024-04-25T08:32:11.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e60156980 0x7f7e60156e00 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e4c06b4d0 0x7f7e4c06d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e600b56a0 0x7f7e6015e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 --2- 172.21.15.60:0/3616358693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e48004800 0x7f7e60153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 >> 172.21.15.60:0/3616358693 conn(0x7f7e60061a20 msgr2=0x7f7e6015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 shutdown_connections 2024-04-25T08:32:11.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.355+0000 7f7e667ce640 1 -- 172.21.15.60:0/3616358693 wait complete. 2024-04-25T08:32:11.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.468+0000 7fc78a215640 1 Processor -- start 2024-04-25T08:32:11.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.469+0000 7fc78a215640 1 -- start start 2024-04-25T08:32:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.469+0000 7fc78a215640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 0x7fc78406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.469+0000 7fc78a215640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 0x7fc784161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.469+0000 7fc78a215640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78415dee0 con 0x7fc78406ba90 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.469+0000 7fc78a215640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78406cd70 con 0x7fc78406bef0 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.469+0000 7fc78a215640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78415e160 con 0x7fc78406c450 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 0x7fc78406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc783fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 0x7fc784161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 0x7fc78406c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:49346/0 (socket says 172.21.15.60:49346) 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 -- 172.21.15.60:0/13441341 learned_addr learned my addr 172.21.15.60:0/13441341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc783fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 0x7fc784161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:48360/0 (socket says 172.21.15.60:48360) 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc782ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc78406ba90 0x7fc7841615b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:50094/0 (socket says 172.21.15.60:50094) 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 -- 172.21.15.60:0/13441341 >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 msgr2=0x7fc784161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 --2- 172.21.15.60:0/13441341 >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 0x7fc784161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 -- 172.21.15.60:0/13441341 >> v1:172.21.15.100:6789/0 conn(0x7fc78406ba90 legacy=0x7fc7841615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc7837fe640 1 -- 172.21.15.60:0/13441341 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78415e440 con 0x7fc78406c450 2024-04-25T08:32:11.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.470+0000 7fc783fff640 1 --2- 172.21.15.60:0/13441341 >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 0x7fc784161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.471+0000 7fc7837fe640 1 --2- 172.21.15.60:0/13441341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 0x7fc78406c830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc774009870 tx=0x7fc77402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b075ed315a80104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.471+0000 7fc7827fc640 1 -- 172.21.15.60:0/13441341 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774037070 con 0x7fc78406c450 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.471+0000 7fc7827fc640 1 -- 172.21.15.60:0/13441341 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774032440 con 0x7fc78406c450 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.471+0000 7fc7827fc640 1 -- 172.21.15.60:0/13441341 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774032810 con 0x7fc78406c450 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 -- 172.21.15.60:0/13441341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 msgr2=0x7fc78406c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 --2- 172.21.15.60:0/13441341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 0x7fc78406c830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc774009870 tx=0x7fc77402a130 comp rx=0 tx=0).stop 2024-04-25T08:32:11.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 -- 172.21.15.60:0/13441341 shutdown_connections 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 --2- 172.21.15.60:0/13441341 >> v2:172.21.15.100:3300/0 conn(0x7fc78406bef0 0x7fc784161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 --2- 172.21.15.60:0/13441341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406c450 0x7fc78406c830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 -- 172.21.15.60:0/13441341 >> 172.21.15.60:0/13441341 conn(0x7fc7840b3450 msgr2=0x7fc7840b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 -- 172.21.15.60:0/13441341 shutdown_connections 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.472+0000 7fc78a215640 1 -- 172.21.15.60:0/13441341 wait complete. 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.473+0000 7fc78a215640 1 Processor -- start 2024-04-25T08:32:11.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.473+0000 7fc78a215640 1 -- start start 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc78a215640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 0x7fc784147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc78a215640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406bef0 0x7fc78415f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc78a215640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78406c450 0x7fc78415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc78a215640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840b0f30 con 0x7fc78406ba90 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc78a215640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78415ea00 con 0x7fc78406c450 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc78a215640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840b10b0 con 0x7fc78406bef0 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 0x7fc784147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 0x7fc784147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43066/0 (socket says 172.21.15.60:43066) 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 -- 172.21.15.60:0/2140161623 learned_addr learned my addr 172.21.15.60:0/2140161623 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc782ffd640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406bef0 0x7fc78415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc783fff640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78406c450 0x7fc78415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 -- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406bef0 msgr2=0x7fc78415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406bef0 0x7fc78415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 -- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78406c450 msgr2=0x7fc78415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78406c450 0x7fc78415f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 -- 172.21.15.60:0/2140161623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784148170 con 0x7fc78406ba90 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.474+0000 7fc7837fe640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 0x7fc784147c30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc774000c00 tx=0x7fc7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.475+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774041020 con 0x7fc78406ba90 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.475+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774030660 con 0x7fc78406ba90 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.475+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7841f3dc0 con 0x7fc78406ba90 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.475+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774037040 con 0x7fc78406ba90 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.475+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7840cd400 con 0x7fc78406ba90 2024-04-25T08:32:11.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.476+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78406de80 con 0x7fc78406ba90 2024-04-25T08:32:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.476+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc774044020 con 0x7fc78406ba90 2024-04-25T08:32:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.477+0000 7fc76bfff640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc77006b4d0 0x7fc77006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.477+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc7740aa7a0 con 0x7fc78406ba90 2024-04-25T08:32:11.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.477+0000 7fc782ffd640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc77006b4d0 0x7fc77006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.477+0000 7fc782ffd640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc77006b4d0 0x7fc77006d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc784159e70 tx=0x7fc778007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.481+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc77404b030 con 0x7fc78406ba90 2024-04-25T08:32:11.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.664+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc784073990 con 0x7fc78406ba90 2024-04-25T08:32:11.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.665+0000 7fc76bfff640 1 -- 172.21.15.60:0/2140161623 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13152 (secure 0 0 0) 0x7fc774079110 con 0x7fc78406ba90 2024-04-25T08:32:11.662 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:11.662 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:10.970550+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc77006b4d0 msgr2=0x7fc77006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc77006b4d0 0x7fc77006d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc784159e70 tx=0x7fc778007e80 comp rx=0 tx=0).stop 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 msgr2=0x7fc784147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 0x7fc784147c30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc774000c00 tx=0x7fc7740046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 shutdown_connections 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc77006b4d0 0x7fc77006d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc78406c450 0x7fc78415f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc78406bef0 0x7fc78415f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 --2- 172.21.15.60:0/2140161623 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc78406ba90 0x7fc784147c30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 >> 172.21.15.60:0/2140161623 conn(0x7fc7840b3450 msgr2=0x7fc784071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 shutdown_connections 2024-04-25T08:32:11.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.667+0000 7fc78a215640 1 -- 172.21.15.60:0/2140161623 wait complete. 2024-04-25T08:32:11.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:32:11.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:11.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.782+0000 7f16a4fd5640 1 Processor -- start 2024-04-25T08:32:11.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.782+0000 7f16a4fd5640 1 -- start start 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f16a4fd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16a0166a10 0x7f16a0168e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f16a4fd5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00afc80 con 0x7f16a00cadb0 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f16a4fd5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00ce4d0 con 0x7f16a015de80 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f16a4fd5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00cb240 con 0x7f16a0166a10 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f169ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f16a0166a10 0x7f16a0168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f169dd9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f16a00cadb0 0x7f16a00cf6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58710/0 (socket says 172.21.15.60:58710) 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.783+0000 7f169dd9b640 1 -- 172.21.15.60:0/1412904414 learned_addr learned my addr 172.21.15.60:0/1412904414 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.784+0000 7f169ed9d640 1 -- 172.21.15.60:0/1412904414 >> v1:172.21.15.100:6789/0 conn(0x7f16a015de80 legacy=0x7f16a00cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.784+0000 7f169ed9d640 1 -- 172.21.15.60:0/1412904414 >> v1:172.21.15.60:6789/0 conn(0x7f16a00cadb0 legacy=0x7f16a00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.784+0000 7f169d59a640 1 -- 172.21.15.60:0/1412904414 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904315784 0 0) 0x7f16a00afc80 con 0x7f16a00cadb0 2024-04-25T08:32:11.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.784+0000 7f169ed9d640 1 -- 172.21.15.60:0/1412904414 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a00ce7b0 con 0x7f16a0166a10 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f169ed9d640 1 --2- 172.21.15.60:0/1412904414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a0166a10 0x7f16a0168e00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1690009d90 tx=0x7f169002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6507ea9ac1589009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f169d59a640 1 -- 172.21.15.60:0/1412904414 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690007740 con 0x7f16a0166a10 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f169d59a640 1 -- 172.21.15.60:0/1412904414 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900078e0 con 0x7f16a0166a10 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f169d59a640 1 -- 172.21.15.60:0/1412904414 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690007cb0 con 0x7f16a0166a10 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1412904414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a0166a10 msgr2=0x7f16a0168e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1412904414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a0166a10 0x7f16a0168e00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1690009d90 tx=0x7f169002a660 comp rx=0 tx=0).stop 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1412904414 shutdown_connections 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1412904414 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a0166a10 0x7f16a0168e00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1412904414 >> 172.21.15.60:0/1412904414 conn(0x7f16a00b2390 msgr2=0x7f16a00b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1412904414 shutdown_connections 2024-04-25T08:32:11.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.785+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1412904414 wait complete. 2024-04-25T08:32:11.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 Processor -- start 2024-04-25T08:32:11.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 -- start start 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16a00cadb0 0x7f16a00c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 0x7f16a00c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0166a10 0x7f16a0159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b2eb0 con 0x7f16a015de80 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a015e260 con 0x7f16a0166a10 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f16a4fd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b3030 con 0x7f16a00cadb0 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.787+0000 7f169dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 0x7f16a00c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 0x7f16a00c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44348/0 (socket says 172.21.15.60:44348) 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 -- 172.21.15.60:0/1416471787 learned_addr learned my addr 172.21.15.60:0/1416471787 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169ed9d640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0166a10 0x7f16a0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 -- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16a00cadb0 msgr2=0x7f16a00c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169e59c640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16a00cadb0 0x7f16a00c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16a00cadb0 0x7f16a00c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 -- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0166a10 msgr2=0x7f16a0159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0166a10 0x7f16a0159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 -- 172.21.15.60:0/1416471787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a00ad7a0 con 0x7f16a015de80 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169ed9d640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0166a10 0x7f16a0159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f169dd9b640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 0x7f16a00c95b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f168800c4d0 tx=0x7f168800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688018070 con 0x7f16a015de80 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16880074e0 con 0x7f16a015de80 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a00ada80 con 0x7f16a015de80 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688004a20 con 0x7f16a015de80 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.788+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a00ae040 con 0x7f16a015de80 2024-04-25T08:32:11.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.789+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0073a00 con 0x7f16a015de80 2024-04-25T08:32:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.790+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f168801b070 con 0x7f16a015de80 2024-04-25T08:32:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.790+0000 7f1696ffd640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f168006b4d0 0x7f168006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.790+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1688089340 con 0x7f16a015de80 2024-04-25T08:32:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.790+0000 7f169e59c640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f168006b4d0 0x7f168006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.788 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.791+0000 7f169e59c640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f168006b4d0 0x7f168006d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1698000f60 tx=0x7f1698009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.794+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1688058380 con 0x7f16a015de80 2024-04-25T08:32:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.981+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16a00ae210 con 0x7f16a015de80 2024-04-25T08:32:11.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.981+0000 7f1696ffd640 1 -- 172.21.15.60:0/1416471787 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1688057c80 con 0x7f16a015de80 2024-04-25T08:32:11.978 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:32:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f168006b4d0 msgr2=0x7f168006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f168006b4d0 0x7f168006d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1698000f60 tx=0x7f1698009040 comp rx=0 tx=0).stop 2024-04-25T08:32:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 msgr2=0x7f16a00c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 0x7f16a00c95b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f168800c4d0 tx=0x7f168800c990 comp rx=0 tx=0).stop 2024-04-25T08:32:11.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 shutdown_connections 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f168006b4d0 0x7f168006d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f16a0166a10 0x7f16a0159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16a015de80 0x7f16a00c95b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 --2- 172.21.15.60:0/1416471787 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f16a00cadb0 0x7f16a00c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 >> 172.21.15.60:0/1416471787 conn(0x7f16a00b2390 msgr2=0x7f16a00cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 shutdown_connections 2024-04-25T08:32:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:11.984+0000 7f16a4fd5640 1 -- 172.21.15.60:0/1416471787 wait complete. 2024-04-25T08:32:11.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:32:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.096+0000 7fb486f29640 1 Processor -- start 2024-04-25T08:32:12.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.096+0000 7fb486f29640 1 -- start start 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb486f29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48015db70 0x7fb480162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb486f29640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4800b6cb0 0x7fb4800b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb486f29640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4800b5a60 0x7fb48006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb486f29640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb480162690 con 0x7fb4800b5a60 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb486f29640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4800ceee0 con 0x7fb4800b6cb0 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb486f29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800d1d10 con 0x7fb48015db70 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4800b5a60 0x7fb48006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb4800b5a60 0x7fb48006c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44358/0 (socket says 172.21.15.60:44358) 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 -- 172.21.15.60:0/3748970261 learned_addr learned my addr 172.21.15.60:0/3748970261 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb484cc5640 1 --2- 172.21.15.60:0/3748970261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48015db70 0x7fb480162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb47ffff640 1 --2- 172.21.15.60:0/3748970261 >> v2:172.21.15.100:3300/0 conn(0x7fb4800b6cb0 0x7fb4800b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 -- 172.21.15.60:0/3748970261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48015db70 msgr2=0x7fb480162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:12.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 --2- 172.21.15.60:0/3748970261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48015db70 0x7fb480162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 -- 172.21.15.60:0/3748970261 >> v2:172.21.15.100:3300/0 conn(0x7fb4800b6cb0 msgr2=0x7fb4800b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:12.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 --2- 172.21.15.60:0/3748970261 >> v2:172.21.15.100:3300/0 conn(0x7fb4800b6cb0 0x7fb4800b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.097+0000 7fb4854c6640 1 -- 172.21.15.60:0/3748970261 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800d1ff0 con 0x7fb4800b5a60 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.098+0000 7fb4854c6640 1 --2- 172.21.15.60:0/3748970261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b5a60 0x7fb48006c2d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb4800d4b60 tx=0x7fb47802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6920ff05460afa86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.098+0000 7fb47f7fe640 1 -- 172.21.15.60:0/3748970261 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478036070 con 0x7fb4800b5a60 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.098+0000 7fb47f7fe640 1 -- 172.21.15.60:0/3748970261 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb478031600 con 0x7fb4800b5a60 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.098+0000 7fb47f7fe640 1 -- 172.21.15.60:0/3748970261 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780319d0 con 0x7fb4800b5a60 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.098+0000 7fb486f29640 1 -- 172.21.15.60:0/3748970261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b5a60 msgr2=0x7fb48006c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.098+0000 7fb486f29640 1 --2- 172.21.15.60:0/3748970261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b5a60 0x7fb48006c2d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb4800d4b60 tx=0x7fb47802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 -- 172.21.15.60:0/3748970261 shutdown_connections 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 --2- 172.21.15.60:0/3748970261 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b5a60 0x7fb48006c2d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 --2- 172.21.15.60:0/3748970261 >> v2:172.21.15.100:3300/0 conn(0x7fb4800b6cb0 0x7fb4800b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 --2- 172.21.15.60:0/3748970261 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48015db70 0x7fb480162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 -- 172.21.15.60:0/3748970261 >> 172.21.15.60:0/3748970261 conn(0x7fb4800d0520 msgr2=0x7fb4800d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 -- 172.21.15.60:0/3748970261 shutdown_connections 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.099+0000 7fb486f29640 1 -- 172.21.15.60:0/3748970261 wait complete. 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.100+0000 7fb486f29640 1 Processor -- start 2024-04-25T08:32:12.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.100+0000 7fb486f29640 1 -- start start 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4800b5a60 0x7fb48015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 0x7fb480160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48015db70 0x7fb48015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800ce530 con 0x7fb4800b6cb0 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48015df50 con 0x7fb48015db70 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800cf1c0 con 0x7fb4800b5a60 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 0x7fb480160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 0x7fb480160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44374/0 (socket says 172.21.15.60:44374) 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 -- 172.21.15.60:0/3641459784 learned_addr learned my addr 172.21.15.60:0/3641459784 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 -- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4800b5a60 msgr2=0x7fb48015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb484cc5640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4800b5a60 0x7fb48015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb4854c6640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48015db70 0x7fb48015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4800b5a60 0x7fb48015a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 -- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48015db70 msgr2=0x7fb48015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48015db70 0x7fb48015bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 -- 172.21.15.60:0/3641459784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480160fa0 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47ffff640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 0x7fb480160a60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00d7d0 tx=0x7fb46c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c009640 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c019070 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48015e320 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c009a00 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.101+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48015e8e0 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.102+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb480073990 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.103+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb46c009ba0 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.103+0000 7fb47d7fa640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb46406b4d0 0x7fb46406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.103+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb46c08a390 con 0x7fb4800b6cb0 2024-04-25T08:32:12.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.104+0000 7fb484cc5640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb46406b4d0 0x7fb46406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.104+0000 7fb484cc5640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb46406b4d0 0x7fb46406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb474006fd0 tx=0x7fb474008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.108+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb46c059460 con 0x7fb4800b6cb0 2024-04-25T08:32:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.289+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb4800d1820 con 0x7fb4800b6cb0 2024-04-25T08:32:12.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.290+0000 7fb47d7fa640 1 -- 172.21.15.60:0/3641459784 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb46c058d60 con 0x7fb4800b6cb0 2024-04-25T08:32:12.286 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:12.286 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:32:12.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:12.287 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:32:12.287 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:32:12.287 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:12.287 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:32:12.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb46406b4d0 msgr2=0x7fb46406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb46406b4d0 0x7fb46406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb474006fd0 tx=0x7fb474008040 comp rx=0 tx=0).stop 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 msgr2=0x7fb480160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 0x7fb480160a60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00d7d0 tx=0x7fb46c00dc90 comp rx=0 tx=0).stop 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 shutdown_connections 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb46406b4d0 0x7fb46406d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48015db70 0x7fb48015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4800b6cb0 0x7fb480160a60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 --2- 172.21.15.60:0/3641459784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb4800b5a60 0x7fb48015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 >> 172.21.15.60:0/3641459784 conn(0x7fb4800d0520 msgr2=0x7fb48006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 shutdown_connections 2024-04-25T08:32:12.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.292+0000 7fb486f29640 1 -- 172.21.15.60:0/3641459784 wait complete. 2024-04-25T08:32:12.299 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:32:12.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:32:12.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.407+0000 7f381e6f7640 1 Processor -- start 2024-04-25T08:32:12.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.407+0000 7f381e6f7640 1 -- start start 2024-04-25T08:32:12.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381e6f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381e6f7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38180cefb0 0x7f38180cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381e6f7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180afc80 con 0x7f38180cf950 2024-04-25T08:32:12.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381e6f7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38180ce4d0 con 0x7f38180cefb0 2024-04-25T08:32:12.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381e6f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180ce7b0 con 0x7f381815de80 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381cc94640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38180cf950 0x7f381806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58720/0 (socket says 172.21.15.60:58720) 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.408+0000 7f381cc94640 1 -- 172.21.15.60:0/2282589739 learned_addr learned my addr 172.21.15.60:0/2282589739 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3817fff640 1 --2- 172.21.15.60:0/2282589739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f38177fe640 1 --2- 172.21.15.60:0/2282589739 >> v2:172.21.15.100:3300/0 conn(0x7f38180cefb0 0x7f38180cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3816ffd640 1 -- 172.21.15.60:0/2282589739 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397917831 0 0) 0x7f38180afc80 con 0x7f38180cf950 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3816ffd640 1 -- 172.21.15.60:0/2282589739 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3800003520 con 0x7f38180cf950 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3817fff640 1 -- 172.21.15.60:0/2282589739 >> v2:172.21.15.100:3300/0 conn(0x7f38180cefb0 msgr2=0x7f38180cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3817fff640 1 --2- 172.21.15.60:0/2282589739 >> v2:172.21.15.100:3300/0 conn(0x7f38180cefb0 0x7f38180cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3817fff640 1 -- 172.21.15.60:0/2282589739 >> v1:172.21.15.60:6789/0 conn(0x7f38180cf950 legacy=0x7f381806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3816ffd640 1 -- 172.21.15.60:0/2282589739 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3864969025 0 0) 0x7f3800003520 con 0x7f38180cf950 2024-04-25T08:32:12.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.409+0000 7f3817fff640 1 -- 172.21.15.60:0/2282589739 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180cea90 con 0x7f381815de80 2024-04-25T08:32:12.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f3817fff640 1 --2- 172.21.15.60:0/2282589739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180cadb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f38080097a0 tx=0x7f380802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1638c15f77927a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f3816ffd640 1 -- 172.21.15.60:0/2282589739 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808037070 con 0x7f381815de80 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f3816ffd640 1 -- 172.21.15.60:0/2282589739 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380802aba0 con 0x7f381815de80 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f3816ffd640 1 -- 172.21.15.60:0/2282589739 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100019f0 con 0x7f381815de80 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f381e6f7640 1 -- 172.21.15.60:0/2282589739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 msgr2=0x7f38180cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f381e6f7640 1 --2- 172.21.15.60:0/2282589739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180cadb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f38080097a0 tx=0x7f380802a110 comp rx=0 tx=0).stop 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f381e6f7640 1 -- 172.21.15.60:0/2282589739 shutdown_connections 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f381e6f7640 1 --2- 172.21.15.60:0/2282589739 >> v2:172.21.15.100:3300/0 conn(0x7f38180cefb0 0x7f38180cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f381e6f7640 1 --2- 172.21.15.60:0/2282589739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180cadb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.410+0000 7f381e6f7640 1 -- 172.21.15.60:0/2282589739 >> 172.21.15.60:0/2282589739 conn(0x7f38180b2390 msgr2=0x7f38180b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.411+0000 7f381e6f7640 1 -- 172.21.15.60:0/2282589739 shutdown_connections 2024-04-25T08:32:12.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.411+0000 7f381e6f7640 1 -- 172.21.15.60:0/2282589739 wait complete. 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.412+0000 7f381e6f7640 1 Processor -- start 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.412+0000 7f381e6f7640 1 -- start start 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 0x7f38181f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180cf950 0x7f38180b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b3030 con 0x7f38180cefb0 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b2eb0 con 0x7f38180cf950 2024-04-25T08:32:12.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180c5600 con 0x7f381815de80 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 0x7f38181f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 0x7f38181f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44384/0 (socket says 172.21.15.60:44384) 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 -- 172.21.15.60:0/1952826791 learned_addr learned my addr 172.21.15.60:0/1952826791 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 -- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 msgr2=0x7f38180b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 -- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180cf950 msgr2=0x7f38180b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180cf950 0x7f38180b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 -- 172.21.15.60:0/1952826791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381815d3b0 con 0x7f38180cefb0 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3817fff640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 0x7f38181f2be0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3808035040 tx=0x7f38080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808041020 con 0x7f38180cefb0 2024-04-25T08:32:12.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380803d620 con 0x7f38180cefb0 2024-04-25T08:32:12.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808037040 con 0x7f38180cefb0 2024-04-25T08:32:12.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180c8380 con 0x7f38180cefb0 2024-04-25T08:32:12.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.413+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180c8940 con 0x7f38180cefb0 2024-04-25T08:32:12.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.414+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818073a00 con 0x7f38180cefb0 2024-04-25T08:32:12.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.415+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3808044020 con 0x7f38180cefb0 2024-04-25T08:32:12.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.415+0000 7f3814ff9640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f380406b4d0 0x7f380406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.415+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f38080aa640 con 0x7f38180cefb0 2024-04-25T08:32:12.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.415+0000 7f38177fe640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f380406b4d0 0x7f380406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.416+0000 7f38177fe640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f380406b4d0 0x7f380406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f380c006a90 tx=0x7f380c006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.420+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f380804b030 con 0x7f38180cefb0 2024-04-25T08:32:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.603+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f38180c8b10 con 0x7f38180cefb0 2024-04-25T08:32:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.973+0000 7f3814ff9640 1 -- 172.21.15.60:0/1952826791 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f38080795e0 con 0x7f38180cefb0 2024-04-25T08:32:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:32:12.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.976+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f380406b4d0 msgr2=0x7f380406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f380406b4d0 0x7f380406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f380c006a90 tx=0x7f380c006a20 comp rx=0 tx=0).stop 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 msgr2=0x7f38181f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 0x7f38181f2be0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3808035040 tx=0x7f38080046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 shutdown_connections 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f381815de80 0x7f38180b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f380406b4d0 0x7f380406d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38180cf950 0x7f38180b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 --2- 172.21.15.60:0/1952826791 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38180cefb0 0x7f38181f2be0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 >> 172.21.15.60:0/1952826791 conn(0x7f38180b2390 msgr2=0x7f38180aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 shutdown_connections 2024-04-25T08:32:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:12.977+0000 7f381e6f7640 1 -- 172.21.15.60:0/1952826791 wait complete. 2024-04-25T08:32:12.989 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:32:13.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.103+0000 7f004fcb3640 1 Processor -- start 2024-04-25T08:32:13.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.103+0000 7f004fcb3640 1 -- start start 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004fcb3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f004815b7d0 0x7f00480cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004fcb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00480b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004fcb3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f004806c2d0 0x7f004806c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004fcb3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00480b5fc0 con 0x7f004815b7d0 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004fcb3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004806cca0 con 0x7f004806c2d0 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004fcb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004806cf50 con 0x7f00480b5620 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f004815b7d0 0x7f00480cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f004815b7d0 0x7f00480cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44390/0 (socket says 172.21.15.60:44390) 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 -- 172.21.15.60:0/2046371469 learned_addr learned my addr 172.21.15.60:0/2046371469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 -- 172.21.15.60:0/2046371469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 msgr2=0x7f00480b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004e250640 1 --2- 172.21.15.60:0/2046371469 >> v2:172.21.15.100:3300/0 conn(0x7f004806c2d0 0x7f004806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004d24e640 1 --2- 172.21.15.60:0/2046371469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00480b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 --2- 172.21.15.60:0/2046371469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00480b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 -- 172.21.15.60:0/2046371469 >> v2:172.21.15.100:3300/0 conn(0x7f004806c2d0 msgr2=0x7f004806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 --2- 172.21.15.60:0/2046371469 >> v2:172.21.15.100:3300/0 conn(0x7f004806c2d0 0x7f004806c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.104+0000 7f004da4f640 1 -- 172.21.15.60:0/2046371469 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480ce530 con 0x7f004815b7d0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.105+0000 7f004da4f640 1 --2- 172.21.15.60:0/2046371469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004815b7d0 0x7f00480cf010 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f004815e340 tx=0x7f004402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0e1a29537a3eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.105+0000 7f004ca4d640 1 -- 172.21.15.60:0/2046371469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044037070 con 0x7f004815b7d0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.105+0000 7f004ca4d640 1 -- 172.21.15.60:0/2046371469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044032440 con 0x7f004815b7d0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.105+0000 7f004ca4d640 1 -- 172.21.15.60:0/2046371469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044032810 con 0x7f004815b7d0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 -- 172.21.15.60:0/2046371469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004815b7d0 msgr2=0x7f00480cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2046371469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004815b7d0 0x7f00480cf010 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f004815e340 tx=0x7f004402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 -- 172.21.15.60:0/2046371469 shutdown_connections 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2046371469 >> v2:172.21.15.100:3300/0 conn(0x7f004806c2d0 0x7f004806c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2046371469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00480b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2046371469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004815b7d0 0x7f00480cf010 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 -- 172.21.15.60:0/2046371469 >> 172.21.15.60:0/2046371469 conn(0x7f00480d0800 msgr2=0x7f00480d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 -- 172.21.15.60:0/2046371469 shutdown_connections 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.106+0000 7f004fcb3640 1 -- 172.21.15.60:0/2046371469 wait complete. 2024-04-25T08:32:13.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.107+0000 7f004fcb3640 1 Processor -- start 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.107+0000 7f004fcb3640 1 -- start start 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004fcb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 0x7f00480cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004fcb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00481f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004fcb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004815b7d0 0x7f00480c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004fcb3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004806d230 con 0x7f004806c2d0 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004fcb3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480ceaf0 con 0x7f004815b7d0 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004fcb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00480cf550 con 0x7f00480b5620 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 0x7f00480cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 0x7f00480cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44392/0 (socket says 172.21.15.60:44392) 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 -- 172.21.15.60:0/2881062484 learned_addr learned my addr 172.21.15.60:0/2881062484 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004d24e640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00481f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 -- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 msgr2=0x7f00481f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00481f2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 -- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004815b7d0 msgr2=0x7f00480c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004e250640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004815b7d0 0x7f00480c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004815b7d0 0x7f00480c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 -- 172.21.15.60:0/2881062484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480cbed0 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f004da4f640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 0x7f00480cb680 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0044000c00 tx=0x7f00440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.108+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044037040 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.109+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044042070 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.109+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00480cc1b0 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.109+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044030bb0 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.109+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00481f3710 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.110+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00480cc5a0 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.110+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0044047460 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.111+0000 7f003e7fc640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f002c06b4d0 0x7f002c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.111+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f00440aae50 con 0x7f004806c2d0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.111+0000 7f004d24e640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f002c06b4d0 0x7f002c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.111+0000 7f004d24e640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f002c06b4d0 0x7f002c06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0038006fd0 tx=0x7f0038008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.115+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0044079d20 con 0x7f004806c2d0 2024-04-25T08:32:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.296+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0048073a00 con 0x7f004806c2d0 2024-04-25T08:32:13.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.979+0000 7f003e7fc640 1 -- 172.21.15.60:0/2881062484 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0044079620 con 0x7f004806c2d0 2024-04-25T08:32:13.975 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.983+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f002c06b4d0 msgr2=0x7f002c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.983+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f002c06b4d0 0x7f002c06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0038006fd0 tx=0x7f0038008040 comp rx=0 tx=0).stop 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.983+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 msgr2=0x7f00480cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.983+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 0x7f00480cb680 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0044000c00 tx=0x7f00440046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 shutdown_connections 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f002c06b4d0 0x7f002c06d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f004815b7d0 0x7f00480c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00480b5620 0x7f00481f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 --2- 172.21.15.60:0/2881062484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f004806c2d0 0x7f00480cb680 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 >> 172.21.15.60:0/2881062484 conn(0x7f00480d0800 msgr2=0x7f00481607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.984+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 shutdown_connections 2024-04-25T08:32:13.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:13.996+0000 7f004fcb3640 1 -- 172.21.15.60:0/2881062484 wait complete. 2024-04-25T08:32:14.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:32:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.124+0000 7f6c4d722640 1 Processor -- start 2024-04-25T08:32:14.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.124+0000 7f6c4d722640 1 -- start start 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c4d722640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c4d722640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c48161710 0x7f6c480cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c4d722640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4815e000 con 0x7f6c480b9330 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c4d722640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c480d1de0 con 0x7f6c48161710 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c4d722640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48161c60 con 0x7f6c4815dc20 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c48161710 0x7f6c480cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c48161710 0x7f6c480cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44398/0 (socket says 172.21.15.60:44398) 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 -- 172.21.15.60:0/2784498047 learned_addr learned my addr 172.21.15.60:0/2784498047 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c46ffd640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 -- 172.21.15.60:0/2784498047 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 msgr2=0x7f6c480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:14.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 -- 172.21.15.60:0/2784498047 >> v1:172.21.15.100:6789/0 conn(0x7f6c480b9330 legacy=0x7f6c4806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c467fc640 1 -- 172.21.15.60:0/2784498047 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48161f40 con 0x7f6c48161710 2024-04-25T08:32:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.125+0000 7f6c46ffd640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.126+0000 7f6c467fc640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c480cebb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009870 tx=0x7f6c3c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29f48a1a440191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.126+0000 7f6c45ffb640 1 -- 172.21.15.60:0/2784498047 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c037070 con 0x7f6c48161710 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.126+0000 7f6c45ffb640 1 -- 172.21.15.60:0/2784498047 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c032440 con 0x7f6c48161710 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.126+0000 7f6c45ffb640 1 -- 172.21.15.60:0/2784498047 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c032810 con 0x7f6c48161710 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 -- 172.21.15.60:0/2784498047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 msgr2=0x7f6c480cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c480cebb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009870 tx=0x7f6c3c02a200 comp rx=0 tx=0).stop 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 -- 172.21.15.60:0/2784498047 shutdown_connections 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c480cebb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 --2- 172.21.15.60:0/2784498047 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 -- 172.21.15.60:0/2784498047 >> 172.21.15.60:0/2784498047 conn(0x7f6c480d0420 msgr2=0x7f6c480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 -- 172.21.15.60:0/2784498047 shutdown_connections 2024-04-25T08:32:14.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.127+0000 7f6c4d722640 1 -- 172.21.15.60:0/2784498047 wait complete. 2024-04-25T08:32:14.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.128+0000 7f6c4d722640 1 Processor -- start 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.128+0000 7f6c4d722640 1 -- start start 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c4d722640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c480b9330 0x7f6c4814c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c4d722640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c4d722640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c481494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c4d722640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480b5e50 con 0x7f6c48161710 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c4d722640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48162500 con 0x7f6c480b9330 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c4d722640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480b5fd0 con 0x7f6c4815dc20 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c481494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c481494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44414/0 (socket says 172.21.15.60:44414) 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 -- 172.21.15.60:0/1033946424 learned_addr learned my addr 172.21.15.60:0/1033946424 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c46ffd640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c480b9330 0x7f6c4814c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c467fc640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 -- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 msgr2=0x7f6c480caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 -- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c480b9330 msgr2=0x7f6c4814c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c480b9330 0x7f6c4814c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.129+0000 7f6c477fe640 1 -- 172.21.15.60:0/1033946424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4814ffb0 con 0x7f6c48161710 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.130+0000 7f6c477fe640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c481494b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b9e0 tx=0x7f6c4000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.130+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003b70 con 0x7f6c48161710 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.130+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40003d10 con 0x7f6c48161710 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.130+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c48150450 con 0x7f6c48161710 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.130+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000cae0 con 0x7f6c48161710 2024-04-25T08:32:14.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.130+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c480cd720 con 0x7f6c48161710 2024-04-25T08:32:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.131+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c480cf940 con 0x7f6c48161710 2024-04-25T08:32:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.131+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c4000cc80 con 0x7f6c48161710 2024-04-25T08:32:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.132+0000 7f6c2f7fe640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c3406b4d0 0x7f6c3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.132+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f6c4008a180 con 0x7f6c48161710 2024-04-25T08:32:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.132+0000 7f6c46ffd640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c3406b4d0 0x7f6c3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.133+0000 7f6c46ffd640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c3406b4d0 0x7f6c3406d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800aac0 tx=0x7f6c38008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.137+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c480cf940 con 0x7f6c48161710 2024-04-25T08:32:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.354+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6c48073a00 con 0x7f6c48161710 2024-04-25T08:32:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.980+0000 7f6c2f7fe640 1 -- 172.21.15.60:0/1033946424 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6c40058760 con 0x7f6c48161710 2024-04-25T08:32:14.977 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T08:32:14.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.986+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c3406b4d0 msgr2=0x7f6c3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.986+0000 7f6c4d722640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c3406b4d0 0x7f6c3406d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800aac0 tx=0x7f6c38008040 comp rx=0 tx=0).stop 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.986+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 msgr2=0x7f6c481494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.986+0000 7f6c4d722640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c481494b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b9e0 tx=0x7f6c4000bea0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.987+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 shutdown_connections 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.987+0000 7f6c4d722640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c48161710 0x7f6c481494b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.987+0000 7f6c4d722640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c4815dc20 0x7f6c480caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.987+0000 7f6c4d722640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c3406b4d0 0x7f6c3406d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.987+0000 7f6c4d722640 1 --2- 172.21.15.60:0/1033946424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c480b9330 0x7f6c4814c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.987+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 >> 172.21.15.60:0/1033946424 conn(0x7f6c480d0420 msgr2=0x7f6c480d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.996+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 shutdown_connections 2024-04-25T08:32:14.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:14.996+0000 7f6c4d722640 1 -- 172.21.15.60:0/1033946424 wait complete. 2024-04-25T08:32:15.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.126+0000 7fe906c8f640 1 Processor -- start 2024-04-25T08:32:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.126+0000 7fe906c8f640 1 -- start start 2024-04-25T08:32:15.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe906c8f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe9000ce8d0 0x7fe900168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe906c8f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000c8030 con 0x7fe90015da70 2024-04-25T08:32:15.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe906c8f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000c8310 con 0x7fe9000d0170 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe906c8f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000c85f0 con 0x7fe9000ce8d0 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe904a2b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe90015da70 0x7fe9000c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58732/0 (socket says 172.21.15.60:58732) 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe904a2b640 1 -- 172.21.15.60:0/3751948660 learned_addr learned my addr 172.21.15.60:0/3751948660 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.127+0000 7fe90522c640 1 --2- 172.21.15.60:0/3751948660 >> v2:172.21.15.100:3300/0 conn(0x7fe9000ce8d0 0x7fe900168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563115821 0 0) 0x7fe9000c8030 con 0x7fe90015da70 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f0003520 con 0x7fe90015da70 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131070917 0 0) 0x7fe9000c8310 con 0x7fe9000d0170 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9000c8030 con 0x7fe9000d0170 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132034038 0 0) 0x7fe8f0003520 con 0x7fe90015da70 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 >> v2:172.21.15.100:3300/0 conn(0x7fe9000ce8d0 msgr2=0x7fe900168690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 --2- 172.21.15.60:0/3751948660 >> v2:172.21.15.100:3300/0 conn(0x7fe9000ce8d0 0x7fe900168690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 >> v1:172.21.15.100:6789/0 conn(0x7fe9000d0170 legacy=0x7fe9000ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90006bb30 con 0x7fe90015da70 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe8f4001670 con 0x7fe90015da70 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 >> v1:172.21.15.60:6789/0 conn(0x7fe90015da70 legacy=0x7fe9000c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe8f0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000c85f0 con 0x7fe8f0003b40 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f000c040 con 0x7fe8f0004800 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f000c240 con 0x7fe8f00051c0 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe8f0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8fffff640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe90522c640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe904a2b640 1 -- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f00051c0 msgr2=0x7fe8f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe904a2b640 1 -- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f0004800 msgr2=0x7fe8f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe90522c640 1 -- 172.21.15.60:0/3751948660 reap_dead start 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe904a2b640 1 -- 172.21.15.60:0/3751948660 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0003520 con 0x7fe8f0003b40 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.128+0000 7fe8fffff640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.129+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe8f0003f40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007fd0 tx=0x7fe8f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59519db89583785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.129+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4007ca0 con 0x7fe8f0003b40 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.129+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f4005af0 con 0x7fe8f0003b40 2024-04-25T08:32:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.129+0000 7fe8ff7fe640 1 -- 172.21.15.60:0/3751948660 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4007070 con 0x7fe8f0003b40 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 -- 172.21.15.60:0/3751948660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 msgr2=0x7fe8f0003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe8f0003f40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007fd0 tx=0x7fe8f0005830 comp rx=0 tx=0).stop 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 -- 172.21.15.60:0/3751948660 shutdown_connections 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3751948660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe8f0003f40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 -- 172.21.15.60:0/3751948660 >> 172.21.15.60:0/3751948660 conn(0x7fe9000b33e0 msgr2=0x7fe9000b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 -- 172.21.15.60:0/3751948660 shutdown_connections 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.130+0000 7fe906c8f640 1 -- 172.21.15.60:0/3751948660 wait complete. 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.131+0000 7fe906c8f640 1 Processor -- start 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.131+0000 7fe906c8f640 1 -- start start 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe906c8f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe9000cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe906c8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe900156970 0x7fe9000b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe906c8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9000b5e50 0x7fe9000b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe906c8f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000ce1c0 con 0x7fe8f0003b40 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe906c8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015de50 con 0x7fe9000b5e50 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe906c8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000ce340 con 0x7fe900156970 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe9000cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe9000cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44440/0 (socket says 172.21.15.60:44440) 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 -- 172.21.15.60:0/3789113234 learned_addr learned my addr 172.21.15.60:0/3789113234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe90522c640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9000b5e50 0x7fe9000b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe8fffff640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe900156970 0x7fe9000b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 -- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe900156970 msgr2=0x7fe9000b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe900156970 0x7fe9000b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 -- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9000b5e50 msgr2=0x7fe9000b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9000b5e50 0x7fe9000b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 -- 172.21.15.60:0/3789113234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000b6bb0 con 0x7fe8f0003b40 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe90522c640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9000b5e50 0x7fe9000b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:15.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.132+0000 7fe904a2b640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe9000cb570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400c250 tx=0x7fe90006bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.133+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4037040 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.133+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f40047e0 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.133+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9000b6e90 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.133+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4001670 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.133+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9000b7450 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.134+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe900073990 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.134+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8f4004bd0 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.134+0000 7fe8fd7fa640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.135+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8f40a9f70 con 0x7fe8f0003b40 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.135+0000 7fe8fffff640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.135+0000 7fe8fffff640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe9000cc7f0 tx=0x7fe8ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.139+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8f4007d40 con 0x7fe8f0003b40 2024-04-25T08:32:15.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.320+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9000cfcf0 con 0x7fe8f0003b40 2024-04-25T08:32:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.321+0000 7fe8fd7fa640 1 -- 172.21.15.60:0/3789113234 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fe8f4078c60 con 0x7fe8f0003b40 2024-04-25T08:32:15.318 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:15.318 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:14.355592+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:15.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e406b4d0 msgr2=0x7fe8e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe9000cc7f0 tx=0x7fe8ec009040 comp rx=0 tx=0).stop 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 msgr2=0x7fe9000cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe9000cb570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400c250 tx=0x7fe90006bd00 comp rx=0 tx=0).stop 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 shutdown_connections 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9000b5e50 0x7fe9000b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe900156970 0x7fe9000b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 --2- 172.21.15.60:0/3789113234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0003b40 0x7fe9000cb570 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 >> 172.21.15.60:0/3789113234 conn(0x7fe9000b33e0 msgr2=0x7fe90006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 shutdown_connections 2024-04-25T08:32:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.323+0000 7fe906c8f640 1 -- 172.21.15.60:0/3789113234 wait complete. 2024-04-25T08:32:15.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:15.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.436+0000 7f7970310640 1 Processor -- start 2024-04-25T08:32:15.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.436+0000 7f7970310640 1 -- start start 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f7970310640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79680d2e90 0x7f79680d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f7970310640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79680d0900 0x7f79680b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f7970310640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680b9940 con 0x7f796815dd30 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f7970310640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680d0d40 con 0x7f79680d2e90 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f7970310640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680d1df0 con 0x7f79680d0900 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796d8ab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79680d2e90 0x7f79680d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796e0ac640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f796815dd30 0x7f79680b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58746/0 (socket says 172.21.15.60:58746) 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796e0ac640 1 -- 172.21.15.60:0/3708730707 learned_addr learned my addr 172.21.15.60:0/3708730707 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796e8ad640 1 --2- 172.21.15.60:0/3708730707 >> v2:172.21.15.100:3300/0 conn(0x7f79680d0900 0x7f79680b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796d8ab640 1 -- 172.21.15.60:0/3708730707 >> v2:172.21.15.100:3300/0 conn(0x7f79680d0900 msgr2=0x7f79680b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796d8ab640 1 --2- 172.21.15.60:0/3708730707 >> v2:172.21.15.100:3300/0 conn(0x7f79680d0900 0x7f79680b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796d8ab640 1 -- 172.21.15.60:0/3708730707 >> v1:172.21.15.60:6789/0 conn(0x7f796815dd30 legacy=0x7f79680b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796d8ab640 1 -- 172.21.15.60:0/3708730707 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680d20d0 con 0x7f79680d2e90 2024-04-25T08:32:15.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.437+0000 7f796d0aa640 1 -- 172.21.15.60:0/3708730707 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544499293 0 0) 0x7f79680b9940 con 0x7f796815dd30 2024-04-25T08:32:15.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.438+0000 7f796d8ab640 1 --2- 172.21.15.60:0/3708730707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d2e90 0x7f79680d03c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f79680ce1d0 tx=0x7f795802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a87f5a03e02993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.438+0000 7f796d0aa640 1 -- 172.21.15.60:0/3708730707 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958037070 con 0x7f79680d2e90 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.438+0000 7f796d0aa640 1 -- 172.21.15.60:0/3708730707 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795802add0 con 0x7f79680d2e90 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.438+0000 7f796d0aa640 1 -- 172.21.15.60:0/3708730707 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958032660 con 0x7f79680d2e90 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.438+0000 7f7970310640 1 -- 172.21.15.60:0/3708730707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d2e90 msgr2=0x7f79680d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.438+0000 7f7970310640 1 --2- 172.21.15.60:0/3708730707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d2e90 0x7f79680d03c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f79680ce1d0 tx=0x7f795802a080 comp rx=0 tx=0).stop 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.439+0000 7f7970310640 1 -- 172.21.15.60:0/3708730707 shutdown_connections 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.439+0000 7f7970310640 1 --2- 172.21.15.60:0/3708730707 >> v2:172.21.15.100:3300/0 conn(0x7f79680d0900 0x7f79680b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.439+0000 7f7970310640 1 --2- 172.21.15.60:0/3708730707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d2e90 0x7f79680d03c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.439+0000 7f7970310640 1 -- 172.21.15.60:0/3708730707 >> 172.21.15.60:0/3708730707 conn(0x7f7968061c30 msgr2=0x7f796806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.439+0000 7f7970310640 1 -- 172.21.15.60:0/3708730707 shutdown_connections 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.439+0000 7f7970310640 1 -- 172.21.15.60:0/3708730707 wait complete. 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 Processor -- start 2024-04-25T08:32:15.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 -- start start 2024-04-25T08:32:15.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 0x7f7968152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79680d2e90 0x7f796815f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f796815dd30 0x7f796815fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680d2970 con 0x7f79680d0900 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680d2690 con 0x7f796815dd30 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.440+0000 7f7970310640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680d1a10 con 0x7f79680d2e90 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 0x7f7968152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 0x7f7968152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44460/0 (socket says 172.21.15.60:44460) 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 -- 172.21.15.60:0/1519695188 learned_addr learned my addr 172.21.15.60:0/1519695188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796d8ab640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79680d2e90 0x7f796815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e8ad640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f796815dd30 0x7f796815fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 -- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79680d2e90 msgr2=0x7f796815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79680d2e90 0x7f796815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 -- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f796815dd30 msgr2=0x7f796815fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f796815dd30 0x7f796815fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 -- 172.21.15.60:0/1519695188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968153d70 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796d8ab640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79680d2e90 0x7f796815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f796e0ac640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 0x7f7968152fd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f796400c430 tx=0x7f796400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640074e0 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79640047e0 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964004bd0 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7968154050 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.441+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7968154b30 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.442+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79680d0000 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.443+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7964011070 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.443+0000 7f795effd640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f794c06b500 0x7f794c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.443+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7964089f50 con 0x7f79680d0900 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.443+0000 7f796d8ab640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f794c06b500 0x7f794c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.444+0000 7f796d8ab640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f794c06b500 0x7f794c06d9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7958034000 tx=0x7f795802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.448+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79680d0000 con 0x7f79680d0900 2024-04-25T08:32:15.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.629+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7968073990 con 0x7f79680d0900 2024-04-25T08:32:15.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.630+0000 7f795effd640 1 -- 172.21.15.60:0/1519695188 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f7964058c40 con 0x7f79680d0900 2024-04-25T08:32:15.626 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:15.626 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:14.355592+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:15.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.631+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f794c06b500 msgr2=0x7f794c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.631+0000 7f7970310640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f794c06b500 0x7f794c06d9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7958034000 tx=0x7f795802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 msgr2=0x7f7968152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 0x7f7968152fd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f796400c430 tx=0x7f796400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 shutdown_connections 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f794c06b500 0x7f794c06d9c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f796815dd30 0x7f796815fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79680d2e90 0x7f796815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 --2- 172.21.15.60:0/1519695188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79680d0900 0x7f7968152fd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 >> 172.21.15.60:0/1519695188 conn(0x7f7968061c30 msgr2=0x7f79680b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 shutdown_connections 2024-04-25T08:32:15.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.632+0000 7f7970310640 1 -- 172.21.15.60:0/1519695188 wait complete. 2024-04-25T08:32:15.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:32:15.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.746+0000 7f50c766b640 1 Processor -- start 2024-04-25T08:32:15.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.746+0000 7f50c766b640 1 -- start start 2024-04-25T08:32:15.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c766b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 0x7f50c00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c766b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 0x7f50c00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c766b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c015e000 con 0x7f50c00b9330 2024-04-25T08:32:15.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c766b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c00d1de0 con 0x7f50c015dc20 2024-04-25T08:32:15.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c766b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0161c60 con 0x7f50c0161710 2024-04-25T08:32:15.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c5407640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 0x7f50c00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.747+0000 7f50c4c06640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 0x7f50c00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c5c08640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f50c00b9330 0x7f50c006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:54334/0 (socket says 172.21.15.60:54334) 2024-04-25T08:32:15.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c5407640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 0x7f50c00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:34882/0 (socket says 172.21.15.60:34882) 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c4c06640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 0x7f50c00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:33486/0 (socket says 172.21.15.60:33486) 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c5c08640 1 -- 172.21.15.60:0/285672739 learned_addr learned my addr 172.21.15.60:0/285672739 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c4c06640 1 -- 172.21.15.60:0/285672739 >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 msgr2=0x7f50c00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c4c06640 1 --2- 172.21.15.60:0/285672739 >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 0x7f50c00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c4c06640 1 -- 172.21.15.60:0/285672739 >> v1:172.21.15.100:6789/0 conn(0x7f50c00b9330 legacy=0x7f50c006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c4c06640 1 -- 172.21.15.60:0/285672739 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0161f40 con 0x7f50c0161710 2024-04-25T08:32:15.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.748+0000 7f50c5407640 1 --2- 172.21.15.60:0/285672739 >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 0x7f50c00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:15.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50c4c06640 1 --2- 172.21.15.60:0/285672739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 0x7f50c00cebb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50b40097a0 tx=0x7f50b402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e321afe1e58980 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50bffff640 1 -- 172.21.15.60:0/285672739 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4037070 con 0x7f50c0161710 2024-04-25T08:32:15.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50bffff640 1 -- 172.21.15.60:0/285672739 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b4032440 con 0x7f50c0161710 2024-04-25T08:32:15.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50bffff640 1 -- 172.21.15.60:0/285672739 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4032810 con 0x7f50c0161710 2024-04-25T08:32:15.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50c766b640 1 -- 172.21.15.60:0/285672739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 msgr2=0x7f50c00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50c766b640 1 --2- 172.21.15.60:0/285672739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 0x7f50c00cebb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50b40097a0 tx=0x7f50b402a180 comp rx=0 tx=0).stop 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50c766b640 1 -- 172.21.15.60:0/285672739 shutdown_connections 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50c766b640 1 --2- 172.21.15.60:0/285672739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c0161710 0x7f50c00cebb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.749+0000 7f50c766b640 1 --2- 172.21.15.60:0/285672739 >> v2:172.21.15.100:3300/0 conn(0x7f50c015dc20 0x7f50c00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.750+0000 7f50c766b640 1 -- 172.21.15.60:0/285672739 >> 172.21.15.60:0/285672739 conn(0x7f50c00d0420 msgr2=0x7f50c00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.750+0000 7f50c766b640 1 -- 172.21.15.60:0/285672739 shutdown_connections 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.750+0000 7f50c766b640 1 -- 172.21.15.60:0/285672739 wait complete. 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.751+0000 7f50c766b640 1 Processor -- start 2024-04-25T08:32:15.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.751+0000 7f50c766b640 1 -- start start 2024-04-25T08:32:15.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.751+0000 7f50c766b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c00b9330 0x7f50c00cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.751+0000 7f50c766b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50c015dc20 0x7f50c01e9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.751+0000 7f50c766b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 0x7f50c01560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c766b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00b5e50 con 0x7f50c0161710 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c766b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0162500 con 0x7f50c015dc20 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c766b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00b5fd0 con 0x7f50c00b9330 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 0x7f50c01560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 0x7f50c01560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44470/0 (socket says 172.21.15.60:44470) 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 -- 172.21.15.60:0/3137045632 learned_addr learned my addr 172.21.15.60:0/3137045632 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5407640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c00b9330 0x7f50c00cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c4c06640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50c015dc20 0x7f50c01e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 -- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c00b9330 msgr2=0x7f50c00cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c00b9330 0x7f50c00cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 -- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50c015dc20 msgr2=0x7f50c01e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50c015dc20 0x7f50c01e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 -- 172.21.15.60:0/3137045632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c015f280 con 0x7f50c0161710 2024-04-25T08:32:15.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c5c08640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 0x7f50c01560d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f50b000caa0 tx=0x7f50b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0017070 con 0x7f50c0161710 2024-04-25T08:32:15.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b000ab40 con 0x7f50c0161710 2024-04-25T08:32:15.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c00cb7d0 con 0x7f50c0161710 2024-04-25T08:32:15.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.752+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0012400 con 0x7f50c0161710 2024-04-25T08:32:15.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.753+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c015f7d0 con 0x7f50c0161710 2024-04-25T08:32:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.754+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c00d16b0 con 0x7f50c0161710 2024-04-25T08:32:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.754+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50b001a070 con 0x7f50c0161710 2024-04-25T08:32:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.754+0000 7f50bdffb640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50ac06b4d0 0x7f50ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.755+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f50b0089f30 con 0x7f50c0161710 2024-04-25T08:32:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.755+0000 7f50c5407640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50ac06b4d0 0x7f50ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.755+0000 7f50c5407640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50ac06b4d0 0x7f50ac06d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009560 tx=0x7f50b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.759+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50b0058530 con 0x7f50c0161710 2024-04-25T08:32:15.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.976+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50c00739e0 con 0x7f50c0161710 2024-04-25T08:32:15.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.992+0000 7f50bdffb640 1 -- 172.21.15.60:0/3137045632 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50b005c5d0 con 0x7f50c0161710 2024-04-25T08:32:15.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50ac06b4d0 msgr2=0x7f50ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50ac06b4d0 0x7f50ac06d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009560 tx=0x7f50b8009190 comp rx=0 tx=0).stop 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 msgr2=0x7f50c01560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 0x7f50c01560d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f50b000caa0 tx=0x7f50b000cf60 comp rx=0 tx=0).stop 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 shutdown_connections 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50c0161710 0x7f50c01560d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50ac06b4d0 0x7f50ac06d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50c015dc20 0x7f50c01e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 --2- 172.21.15.60:0/3137045632 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50c00b9330 0x7f50c00cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 >> 172.21.15.60:0/3137045632 conn(0x7f50c00d0420 msgr2=0x7f50c00d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 shutdown_connections 2024-04-25T08:32:15.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:15.995+0000 7f50c766b640 1 -- 172.21.15.60:0/3137045632 wait complete. 2024-04-25T08:32:16.002 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:32:16.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:32:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.110+0000 7f5e48441640 1 Processor -- start 2024-04-25T08:32:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.110+0000 7f5e48441640 1 -- start start 2024-04-25T08:32:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e48441640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e4015dc20 0x7f5e400ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e48441640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e40161710 0x7f5e400cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e48441640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4015e000 con 0x7f5e400b9330 2024-04-25T08:32:16.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e48441640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400d1de0 con 0x7f5e4015dc20 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e48441640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e40161c60 con 0x7f5e40161710 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e461dd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e4015dc20 0x7f5e400ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e469de640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e400b9330 0x7f5e4006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58756/0 (socket says 172.21.15.60:58756) 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e469de640 1 -- 172.21.15.60:0/4138522080 learned_addr learned my addr 172.21.15.60:0/4138522080 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e459dc640 1 --2- 172.21.15.60:0/4138522080 >> v2:172.21.15.100:3300/0 conn(0x7f5e40161710 0x7f5e400cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e461dd640 1 -- 172.21.15.60:0/4138522080 >> v2:172.21.15.100:3300/0 conn(0x7f5e40161710 msgr2=0x7f5e400cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e461dd640 1 --2- 172.21.15.60:0/4138522080 >> v2:172.21.15.100:3300/0 conn(0x7f5e40161710 0x7f5e400cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e461dd640 1 -- 172.21.15.60:0/4138522080 >> v1:172.21.15.60:6789/0 conn(0x7f5e400b9330 legacy=0x7f5e4006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.111+0000 7f5e461dd640 1 -- 172.21.15.60:0/4138522080 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40161f40 con 0x7f5e4015dc20 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.112+0000 7f5e461dd640 1 --2- 172.21.15.60:0/4138522080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015dc20 0x7f5e400ce670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e4006da90 tx=0x7f5e3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5028d5e81144e5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.112+0000 7f5e451db640 1 -- 172.21.15.60:0/4138522080 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037070 con 0x7f5e4015dc20 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.112+0000 7f5e451db640 1 -- 172.21.15.60:0/4138522080 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c032440 con 0x7f5e4015dc20 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.112+0000 7f5e451db640 1 -- 172.21.15.60:0/4138522080 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c032810 con 0x7f5e4015dc20 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.112+0000 7f5e48441640 1 -- 172.21.15.60:0/4138522080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015dc20 msgr2=0x7f5e400ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.112+0000 7f5e48441640 1 --2- 172.21.15.60:0/4138522080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015dc20 0x7f5e400ce670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e4006da90 tx=0x7f5e3c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.113+0000 7f5e48441640 1 -- 172.21.15.60:0/4138522080 shutdown_connections 2024-04-25T08:32:16.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.113+0000 7f5e48441640 1 --2- 172.21.15.60:0/4138522080 >> v2:172.21.15.100:3300/0 conn(0x7f5e40161710 0x7f5e400cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.113+0000 7f5e48441640 1 --2- 172.21.15.60:0/4138522080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015dc20 0x7f5e400ce670 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.113+0000 7f5e48441640 1 -- 172.21.15.60:0/4138522080 >> 172.21.15.60:0/4138522080 conn(0x7f5e400d0420 msgr2=0x7f5e400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.113+0000 7f5e48441640 1 -- 172.21.15.60:0/4138522080 shutdown_connections 2024-04-25T08:32:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.113+0000 7f5e48441640 1 -- 172.21.15.60:0/4138522080 wait complete. 2024-04-25T08:32:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.114+0000 7f5e48441640 1 Processor -- start 2024-04-25T08:32:16.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.114+0000 7f5e48441640 1 -- start start 2024-04-25T08:32:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.114+0000 7f5e48441640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e400b9330 0x7f5e4015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.114+0000 7f5e48441640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e4015dc20 0x7f5e400b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.114+0000 7f5e48441640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 0x7f5e4015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.114+0000 7f5e48441640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400b5e50 con 0x7f5e40161710 2024-04-25T08:32:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e48441640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40162500 con 0x7f5e400b9330 2024-04-25T08:32:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e48441640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400b5fd0 con 0x7f5e4015dc20 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 0x7f5e4015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 0x7f5e4015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44480/0 (socket says 172.21.15.60:44480) 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 -- 172.21.15.60:0/865098999 learned_addr learned my addr 172.21.15.60:0/865098999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 -- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e4015dc20 msgr2=0x7f5e400b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e4015dc20 0x7f5e400b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e461dd640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e400b9330 0x7f5e4015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 -- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e400b9330 msgr2=0x7f5e4015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e400b9330 0x7f5e4015d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 -- 172.21.15.60:0/865098999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4015ce80 con 0x7f5e40161710 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e469de640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 0x7f5e4015c460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800caa0 tx=0x7f5e3800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38017070 con 0x7f5e40161710 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3800ab40 con 0x7f5e40161710 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4015eec0 con 0x7f5e40161710 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38012400 con 0x7f5e40161710 2024-04-25T08:32:16.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.115+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e401e86e0 con 0x7f5e40161710 2024-04-25T08:32:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.116+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e401e88e0 con 0x7f5e40161710 2024-04-25T08:32:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.117+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e3801a070 con 0x7f5e40161710 2024-04-25T08:32:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.117+0000 7f5e36ffd640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.117+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e38089e50 con 0x7f5e40161710 2024-04-25T08:32:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.118+0000 7f5e461dd640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.118+0000 7f5e461dd640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.122+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e401e88e0 con 0x7f5e40161710 2024-04-25T08:32:16.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.339+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5e4015f3e0 con 0x7f5e40161710 2024-04-25T08:32:16.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:16.998+0000 7f5e36ffd640 1 -- 172.21.15.60:0/865098999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e380584f0 con 0x7f5e40161710 2024-04-25T08:32:16.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 msgr2=0x7f5e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:16.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 msgr2=0x7f5e4015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 0x7f5e4015c460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800caa0 tx=0x7f5e3800cf60 comp rx=0 tx=0).stop 2024-04-25T08:32:16.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 shutdown_connections 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e40161710 0x7f5e4015c460 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e4015dc20 0x7f5e400b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 --2- 172.21.15.60:0/865098999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e400b9330 0x7f5e4015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 >> 172.21.15.60:0/865098999 conn(0x7f5e400d0420 msgr2=0x7f5e400cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 shutdown_connections 2024-04-25T08:32:16.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.001+0000 7f5e48441640 1 -- 172.21.15.60:0/865098999 wait complete. 2024-04-25T08:32:17.011 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:32:17.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.122+0000 7f4845abd640 1 Processor -- start 2024-04-25T08:32:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.122+0000 7f4845abd640 1 -- start start 2024-04-25T08:32:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.122+0000 7f4845abd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f48400b6cb0 0x7f48400b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.122+0000 7f4845abd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400ceee0 con 0x7f484015db70 2024-04-25T08:32:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f4845abd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400d1d10 con 0x7f48400b5a60 2024-04-25T08:32:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f4845abd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400d1ff0 con 0x7f48400b6cb0 2024-04-25T08:32:17.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f48400b6cb0 0x7f48400b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f484015db70 0x7f4840162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58772/0 (socket says 172.21.15.60:58772) 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483f7fe640 1 -- 172.21.15.60:0/2953044660 learned_addr learned my addr 172.21.15.60:0/2953044660 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483e7fc640 1 -- 172.21.15.60:0/2953044660 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641744825 0 0) 0x7f48400ceee0 con 0x7f484015db70 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483e7fc640 1 -- 172.21.15.60:0/2953044660 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4828003520 con 0x7f484015db70 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483effd640 1 -- 172.21.15.60:0/2953044660 >> v1:172.21.15.100:6789/0 conn(0x7f48400b5a60 legacy=0x7f48400d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483effd640 1 -- 172.21.15.60:0/2953044660 >> v1:172.21.15.60:6789/0 conn(0x7f484015db70 legacy=0x7f4840162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.123+0000 7f483effd640 1 -- 172.21.15.60:0/2953044660 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400d22d0 con 0x7f48400b6cb0 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f483effd640 1 --2- 172.21.15.60:0/2953044660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48400b5520 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f48340097a0 tx=0x7f483402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0c15fe80c201a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f483e7fc640 1 -- 172.21.15.60:0/2953044660 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834037070 con 0x7f48400b6cb0 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f483e7fc640 1 -- 172.21.15.60:0/2953044660 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4834032440 con 0x7f48400b6cb0 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f483e7fc640 1 -- 172.21.15.60:0/2953044660 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834032810 con 0x7f48400b6cb0 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f4845abd640 1 -- 172.21.15.60:0/2953044660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 msgr2=0x7f48400b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f4845abd640 1 --2- 172.21.15.60:0/2953044660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48400b5520 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f48340097a0 tx=0x7f483402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f4845abd640 1 -- 172.21.15.60:0/2953044660 shutdown_connections 2024-04-25T08:32:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f4845abd640 1 --2- 172.21.15.60:0/2953044660 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48400b5520 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.124+0000 7f4845abd640 1 -- 172.21.15.60:0/2953044660 >> 172.21.15.60:0/2953044660 conn(0x7f48400d0520 msgr2=0x7f48400d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.125+0000 7f4845abd640 1 -- 172.21.15.60:0/2953044660 shutdown_connections 2024-04-25T08:32:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.125+0000 7f4845abd640 1 -- 172.21.15.60:0/2953044660 wait complete. 2024-04-25T08:32:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.126+0000 7f4845abd640 1 Processor -- start 2024-04-25T08:32:17.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.126+0000 7f4845abd640 1 -- start start 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48400b5a60 0x7f48400aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48401f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f484015db70 0x7f4840160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400cf1c0 con 0x7f48400b6cb0 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400ce530 con 0x7f48400b5a60 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400cf340 con 0x7f484015db70 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48401f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48401f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44498/0 (socket says 172.21.15.60:44498) 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 -- 172.21.15.60:0/2715303472 learned_addr learned my addr 172.21.15.60:0/2715303472 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483f7fe640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48400b5a60 0x7f48400aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483ffff640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f484015db70 0x7f4840160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 -- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f484015db70 msgr2=0x7f4840160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f484015db70 0x7f4840160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 -- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48400b5a60 msgr2=0x7f48400aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48400b5a60 0x7f48400aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 -- 172.21.15.60:0/2715303472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400aa930 con 0x7f48400b6cb0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483ffff640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f484015db70 0x7f4840160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f483effd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48401f3ed0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4834035040 tx=0x7f48340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834037040 con 0x7f48400b6cb0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4834030500 con 0x7f48400b6cb0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48400cc8d0 con 0x7f48400b6cb0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834030910 con 0x7f48400b6cb0 2024-04-25T08:32:17.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.127+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48400cce90 con 0x7f48400b6cb0 2024-04-25T08:32:17.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.128+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840073990 con 0x7f48400b6cb0 2024-04-25T08:32:17.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.129+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4834044020 con 0x7f48400b6cb0 2024-04-25T08:32:17.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.129+0000 7f4827fff640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f482c06b4d0 0x7f482c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.129+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f48340aa6a0 con 0x7f48400b6cb0 2024-04-25T08:32:17.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.130+0000 7f483f7fe640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f482c06b4d0 0x7f482c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.130+0000 7f483f7fe640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f482c06b4d0 0x7f482c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f48300011e0 tx=0x7f4830009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.134+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4834079380 con 0x7f48400b6cb0 2024-04-25T08:32:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.353+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48400b5840 con 0x7f48400b6cb0 2024-04-25T08:32:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.353+0000 7f4827fff640 1 -- 172.21.15.60:0/2715303472 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f4834078c80 con 0x7f48400b6cb0 2024-04-25T08:32:17.350 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:17.350 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:17.350 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f482c06b4d0 msgr2=0x7f482c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f482c06b4d0 0x7f482c06d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f48300011e0 tx=0x7f4830009040 comp rx=0 tx=0).stop 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 msgr2=0x7f48401f3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48401f3ed0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4834035040 tx=0x7f48340046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 shutdown_connections 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f484015db70 0x7f4840160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48400b6cb0 0x7f48401f3ed0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f482c06b4d0 0x7f482c06d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 --2- 172.21.15.60:0/2715303472 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48400b5a60 0x7f48400aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 >> 172.21.15.60:0/2715303472 conn(0x7f48400d0520 msgr2=0x7f484015c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 shutdown_connections 2024-04-25T08:32:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.355+0000 7f4845abd640 1 -- 172.21.15.60:0/2715303472 wait complete. 2024-04-25T08:32:17.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:17.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.467+0000 7f1cc3dc8640 1 Processor -- start 2024-04-25T08:32:17.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc3dc8640 1 -- start start 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc3dc8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1cbc158220 0x7f1cbc15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc3dc8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1cbc0af720 0x7f1cbc0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc3dc8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc15f180 con 0x7f1cbc1617b0 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc3dc8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc0b3670 con 0x7f1cbc0af720 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc3dc8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc070a10 con 0x7f1cbc158220 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.468+0000 7f1cc1363640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1cbc0af720 0x7f1cbc0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1cbc0af720 0x7f1cbc0afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44500/0 (socket says 172.21.15.60:44500) 2024-04-25T08:32:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 -- 172.21.15.60:0/2467258452 learned_addr learned my addr 172.21.15.60:0/2467258452 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1b64640 1 --2- 172.21.15.60:0/2467258452 >> v2:172.21.15.100:3300/0 conn(0x7f1cbc158220 0x7f1cbc15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 -- 172.21.15.60:0/2467258452 >> v2:172.21.15.100:3300/0 conn(0x7f1cbc158220 msgr2=0x7f1cbc15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 --2- 172.21.15.60:0/2467258452 >> v2:172.21.15.100:3300/0 conn(0x7f1cbc158220 0x7f1cbc15ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 -- 172.21.15.60:0/2467258452 >> v1:172.21.15.100:6789/0 conn(0x7f1cbc1617b0 legacy=0x7f1cbc161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:17.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 -- 172.21.15.60:0/2467258452 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc06fe00 con 0x7f1cbc0af720 2024-04-25T08:32:17.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.469+0000 7f1cc1363640 1 --2- 172.21.15.60:0/2467258452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc0afba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06ba30 tx=0x7f1cac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e45be4c50a8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc0b62640 1 -- 172.21.15.60:0/2467258452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac037070 con 0x7f1cbc0af720 2024-04-25T08:32:17.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc0b62640 1 -- 172.21.15.60:0/2467258452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac032440 con 0x7f1cbc0af720 2024-04-25T08:32:17.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc0b62640 1 -- 172.21.15.60:0/2467258452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac032810 con 0x7f1cbc0af720 2024-04-25T08:32:17.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/2467258452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 msgr2=0x7f1cbc0afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/2467258452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc0afba0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06ba30 tx=0x7f1cac02a230 comp rx=0 tx=0).stop 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/2467258452 shutdown_connections 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/2467258452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc0afba0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/2467258452 >> v2:172.21.15.100:3300/0 conn(0x7f1cbc158220 0x7f1cbc15ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/2467258452 >> 172.21.15.60:0/2467258452 conn(0x7f1cbc0b2370 msgr2=0x7f1cbc0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/2467258452 shutdown_connections 2024-04-25T08:32:17.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.470+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/2467258452 wait complete. 2024-04-25T08:32:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.471+0000 7f1cc3dc8640 1 Processor -- start 2024-04-25T08:32:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 -- start start 2024-04-25T08:32:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cbc158220 0x7f1cbc15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1617b0 0x7f1cbc158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0706a0 con 0x7f1cbc0af720 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0703c0 con 0x7f1cbc1617b0 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc3dc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc158600 con 0x7f1cbc158220 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc1b64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc1b64640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc14b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44510/0 (socket says 172.21.15.60:44510) 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc1b64640 1 -- 172.21.15.60:0/4041989229 learned_addr learned my addr 172.21.15.60:0/4041989229 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc2365640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1617b0 0x7f1cbc158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.472+0000 7f1cc1363640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cbc158220 0x7f1cbc15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1b64640 1 -- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cbc158220 msgr2=0x7f1cbc15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1b64640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cbc158220 0x7f1cbc15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1b64640 1 -- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1617b0 msgr2=0x7f1cbc158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1b64640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1617b0 0x7f1cbc158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1b64640 1 -- 172.21.15.60:0/4041989229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc159240 con 0x7f1cbc0af720 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1363640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cbc158220 0x7f1cbc15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc1b64640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc14b0c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800d800 tx=0x7f1cb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800b6c0 con 0x7f1cbc0af720 2024-04-25T08:32:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8019070 con 0x7f1cbc0af720 2024-04-25T08:32:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800ba80 con 0x7f1cbc0af720 2024-04-25T08:32:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc159520 con 0x7f1cbc0af720 2024-04-25T08:32:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.473+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc159ae0 con 0x7f1cbc0af720 2024-04-25T08:32:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.474+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc073990 con 0x7f1cbc0af720 2024-04-25T08:32:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.474+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cb800bc20 con 0x7f1cbc0af720 2024-04-25T08:32:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.475+0000 7f1cb27fc640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b4d0 0x7f1ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.475+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cb808a840 con 0x7f1cbc0af720 2024-04-25T08:32:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.475+0000 7f1cc1363640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b4d0 0x7f1ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.476+0000 7f1cc1363640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b4d0 0x7f1ca006d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1cac034000 tx=0x7f1cac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.479+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cb8059530 con 0x7f1cbc0af720 2024-04-25T08:32:17.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.661+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cbc15dd70 con 0x7f1cbc0af720 2024-04-25T08:32:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.662+0000 7f1cb27fc640 1 -- 172.21.15.60:0/4041989229 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f1cb8058e30 con 0x7f1cbc0af720 2024-04-25T08:32:17.659 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:17.659 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:15.989621+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b4d0 msgr2=0x7f1ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b4d0 0x7f1ca006d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1cac034000 tx=0x7f1cac02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 msgr2=0x7f1cbc14b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc14b0c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800d800 tx=0x7f1cb800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 shutdown_connections 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1ca006b4d0 0x7f1ca006d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cbc1617b0 0x7f1cbc158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cbc158220 0x7f1cbc15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 --2- 172.21.15.60:0/4041989229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cbc0af720 0x7f1cbc14b0c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 >> 172.21.15.60:0/4041989229 conn(0x7f1cbc0b2370 msgr2=0x7f1cbc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 shutdown_connections 2024-04-25T08:32:17.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.664+0000 7f1cc3dc8640 1 -- 172.21.15.60:0/4041989229 wait complete. 2024-04-25T08:32:17.672 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:32:17.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:32:17.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.776+0000 7f79c5f29640 1 Processor -- start 2024-04-25T08:32:17.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.776+0000 7f79c5f29640 1 -- start start 2024-04-25T08:32:17.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79c5f29640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79c00b6cb0 0x7f79c00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79c5f29640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79c00b5a60 0x7f79c00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79c5f29640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00ceee0 con 0x7f79c015db70 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79c5f29640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00d1d10 con 0x7f79c00b6cb0 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79c5f29640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00d1ff0 con 0x7f79c00b5a60 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79beffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79c00b6cb0 0x7f79c00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79bf7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79c015db70 0x7f79c0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58774/0 (socket says 172.21.15.60:58774) 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79bf7fe640 1 -- 172.21.15.60:0/1650512849 learned_addr learned my addr 172.21.15.60:0/1650512849 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79be7fc640 1 -- 172.21.15.60:0/1650512849 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123186166 0 0) 0x7f79c00ceee0 con 0x7f79c015db70 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79bffff640 1 --2- 172.21.15.60:0/1650512849 >> v2:172.21.15.100:3300/0 conn(0x7f79c00b5a60 0x7f79c00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79be7fc640 1 -- 172.21.15.60:0/1650512849 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a8003520 con 0x7f79c015db70 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79beffd640 1 -- 172.21.15.60:0/1650512849 >> v2:172.21.15.100:3300/0 conn(0x7f79c00b5a60 msgr2=0x7f79c00d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79beffd640 1 --2- 172.21.15.60:0/1650512849 >> v2:172.21.15.100:3300/0 conn(0x7f79c00b5a60 0x7f79c00d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79beffd640 1 -- 172.21.15.60:0/1650512849 >> v1:172.21.15.60:6789/0 conn(0x7f79c015db70 legacy=0x7f79c0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79beffd640 1 -- 172.21.15.60:0/1650512849 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00d22d0 con 0x7f79c00b6cb0 2024-04-25T08:32:17.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.777+0000 7f79bffff640 1 --2- 172.21.15.60:0/1650512849 >> v2:172.21.15.100:3300/0 conn(0x7f79c00b5a60 0x7f79c00d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.778+0000 7f79beffd640 1 --2- 172.21.15.60:0/1650512849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c00b6cb0 0x7f79c00b5520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f79c00cfa10 tx=0x7f79b002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebda6260eda396a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.778+0000 7f79be7fc640 1 -- 172.21.15.60:0/1650512849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0037070 con 0x7f79c00b6cb0 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.778+0000 7f79be7fc640 1 -- 172.21.15.60:0/1650512849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0032440 con 0x7f79c00b6cb0 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.778+0000 7f79be7fc640 1 -- 172.21.15.60:0/1650512849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b002ae60 con 0x7f79c00b6cb0 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 -- 172.21.15.60:0/1650512849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c00b6cb0 msgr2=0x7f79c00b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 --2- 172.21.15.60:0/1650512849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c00b6cb0 0x7f79c00b5520 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f79c00cfa10 tx=0x7f79b002a110 comp rx=0 tx=0).stop 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 -- 172.21.15.60:0/1650512849 shutdown_connections 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 --2- 172.21.15.60:0/1650512849 >> v2:172.21.15.100:3300/0 conn(0x7f79c00b5a60 0x7f79c00d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 --2- 172.21.15.60:0/1650512849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c00b6cb0 0x7f79c00b5520 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 -- 172.21.15.60:0/1650512849 >> 172.21.15.60:0/1650512849 conn(0x7f79c00d0520 msgr2=0x7f79c00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 -- 172.21.15.60:0/1650512849 shutdown_connections 2024-04-25T08:32:17.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.779+0000 7f79c5f29640 1 -- 172.21.15.60:0/1650512849 wait complete. 2024-04-25T08:32:17.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.780+0000 7f79c5f29640 1 Processor -- start 2024-04-25T08:32:17.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.780+0000 7f79c5f29640 1 -- start start 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79c5f29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c00b5a60 0x7f79c015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79c5f29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79c00b6cb0 0x7f79c00cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79c5f29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 0x7f79c0143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79c5f29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00cf1c0 con 0x7f79c015db70 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79c5f29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00ce530 con 0x7f79c00b6cb0 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79c5f29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00cf340 con 0x7f79c00b5a60 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 0x7f79c0143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 0x7f79c0143cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44518/0 (socket says 172.21.15.60:44518) 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 -- 172.21.15.60:0/2743755283 learned_addr learned my addr 172.21.15.60:0/2743755283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79beffd640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79c00b6cb0 0x7f79c00cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bf7fe640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c00b5a60 0x7f79c015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 -- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c00b5a60 msgr2=0x7f79c015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c00b5a60 0x7f79c015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 -- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79c00b6cb0 msgr2=0x7f79c00cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79c00b6cb0 0x7f79c00cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 -- 172.21.15.60:0/2743755283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00c9cc0 con 0x7f79c015db70 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bf7fe640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c00b5a60 0x7f79c015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79beffd640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79c00b6cb0 0x7f79c00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79bffff640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 0x7f79c0143cf0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f79b800cef0 tx=0x7f79b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b801a070 con 0x7f79c015db70 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.781+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b800d860 con 0x7f79c015db70 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.782+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c00c9fa0 con 0x7f79c015db70 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.782+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b800dc50 con 0x7f79c015db70 2024-04-25T08:32:17.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.782+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c00cc670 con 0x7f79c015db70 2024-04-25T08:32:17.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.783+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c0073990 con 0x7f79c015db70 2024-04-25T08:32:17.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.783+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79b801d070 con 0x7f79c015db70 2024-04-25T08:32:17.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.783+0000 7f79a7fff640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79ac06b4d0 0x7f79ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.783+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79b808ac60 con 0x7f79c015db70 2024-04-25T08:32:17.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.784+0000 7f79bf7fe640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79ac06b4d0 0x7f79ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.784+0000 7f79bf7fe640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79ac06b4d0 0x7f79ac06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005880 tx=0x7f79b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.788+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79b8059a10 con 0x7f79c015db70 2024-04-25T08:32:17.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.966+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79c00d1820 con 0x7f79ac06b4d0 2024-04-25T08:32:17.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.972+0000 7f79a7fff640 1 -- 172.21.15.60:0/2743755283 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251900 (secure 0 0 0) 0x7f79c00d1820 con 0x7f79ac06b4d0 2024-04-25T08:32:17.971 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79ac06b4d0 msgr2=0x7f79ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79ac06b4d0 0x7f79ac06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005880 tx=0x7f79b4009040 comp rx=0 tx=0).stop 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 msgr2=0x7f79c0143cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 0x7f79c0143cf0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f79b800cef0 tx=0x7f79b800a710 comp rx=0 tx=0).stop 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 shutdown_connections 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79c015db70 0x7f79c0143cf0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79ac06b4d0 0x7f79ac06d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79c00b6cb0 0x7f79c00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 --2- 172.21.15.60:0/2743755283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79c00b5a60 0x7f79c015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 >> 172.21.15.60:0/2743755283 conn(0x7f79c00d0520 msgr2=0x7f79c00b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 shutdown_connections 2024-04-25T08:32:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:17.975+0000 7f79c5f29640 1 -- 172.21.15.60:0/2743755283 wait complete. 2024-04-25T08:32:17.983 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T08:32:17.436944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297224,"kb_used_data":6148,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754677496,"statfs":{"total":773094113280,"available":772789755904,"internally_reserved":0,"allocated":6295552,"data_stored":3451962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.457082"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991183+0000","last_change":"2024-04-25T08:32:14.989814+0000","last_active":"2024-04-25T08:32:14.991183+0000","last_peered":"2024-04-25T08:32:14.991183+0000","last_clean":"2024-04-25T08:32:14.991183+0000","last_became_active":"2024-04-25T08:32:14.989516+0000","last_became_peered":"2024-04-25T08:32:14.989516+0000","last_unstale":"2024-04-25T08:32:14.991183+0000","last_undegraded":"2024-04-25T08:32:14.991183+0000","last_fullsized":"2024-04-25T08:32:14.991183+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:33.347596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997071+0000","last_change":"2024-04-25T08:32:15.005789+0000","last_active":"2024-04-25T08:32:15.997071+0000","last_peered":"2024-04-25T08:32:15.997071+0000","last_clean":"2024-04-25T08:32:15.997071+0000","last_became_active":"2024-04-25T08:32:15.005466+0000","last_became_peered":"2024-04-25T08:32:15.005466+0000","last_unstale":"2024-04-25T08:32:15.997071+0000","last_undegraded":"2024-04-25T08:32:15.997071+0000","last_fullsized":"2024-04-25T08:32:15.997071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.523775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019573+0000","last_change":"2024-04-25T08:32:15.014835+0000","last_active":"2024-04-25T08:32:15.019573+0000","last_peered":"2024-04-25T08:32:15.019573+0000","last_clean":"2024-04-25T08:32:15.019573+0000","last_became_active":"2024-04-25T08:32:15.014327+0000","last_became_peered":"2024-04-25T08:32:15.014327+0000","last_unstale":"2024-04-25T08:32:15.019573+0000","last_undegraded":"2024-04-25T08:32:15.019573+0000","last_fullsized":"2024-04-25T08:32:15.019573+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:53.309146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996846+0000","last_change":"2024-04-25T08:32:15.008520+0000","last_active":"2024-04-25T08:32:15.996846+0000","last_peered":"2024-04-25T08:32:15.996846+0000","last_clean":"2024-04-25T08:32:15.996846+0000","last_became_active":"2024-04-25T08:32:15.008240+0000","last_became_peered":"2024-04-25T08:32:15.008240+0000","last_unstale":"2024-04-25T08:32:15.996846+0000","last_undegraded":"2024-04-25T08:32:15.996846+0000","last_fullsized":"2024-04-25T08:32:15.996846+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.002973+0000","last_change":"2024-04-25T08:32:15.001367+0000","last_active":"2024-04-25T08:32:15.002973+0000","last_peered":"2024-04-25T08:32:15.002973+0000","last_clean":"2024-04-25T08:32:15.002973+0000","last_became_active":"2024-04-25T08:32:15.001094+0000","last_became_peered":"2024-04-25T08:32:15.001094+0000","last_unstale":"2024-04-25T08:32:15.002973+0000","last_undegraded":"2024-04-25T08:32:15.002973+0000","last_fullsized":"2024-04-25T08:32:15.002973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:33.872454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013950+0000","last_change":"2024-04-25T08:32:15.000513+0000","last_active":"2024-04-25T08:32:15.013950+0000","last_peered":"2024-04-25T08:32:15.013950+0000","last_clean":"2024-04-25T08:32:15.013950+0000","last_became_active":"2024-04-25T08:32:15.000226+0000","last_became_peered":"2024-04-25T08:32:15.000226+0000","last_unstale":"2024-04-25T08:32:15.013950+0000","last_undegraded":"2024-04-25T08:32:15.013950+0000","last_fullsized":"2024-04-25T08:32:15.013950+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.006817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020259+0000","last_change":"2024-04-25T08:32:15.014800+0000","last_active":"2024-04-25T08:32:15.020259+0000","last_peered":"2024-04-25T08:32:15.020259+0000","last_clean":"2024-04-25T08:32:15.020259+0000","last_became_active":"2024-04-25T08:32:15.014091+0000","last_became_peered":"2024-04-25T08:32:15.014091+0000","last_unstale":"2024-04-25T08:32:15.020259+0000","last_undegraded":"2024-04-25T08:32:15.020259+0000","last_fullsized":"2024-04-25T08:32:15.020259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:31.735360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021075+0000","last_change":"2024-04-25T08:32:15.018706+0000","last_active":"2024-04-25T08:32:15.021075+0000","last_peered":"2024-04-25T08:32:15.021075+0000","last_clean":"2024-04-25T08:32:15.021075+0000","last_became_active":"2024-04-25T08:32:15.017501+0000","last_became_peered":"2024-04-25T08:32:15.017501+0000","last_unstale":"2024-04-25T08:32:15.021075+0000","last_undegraded":"2024-04-25T08:32:15.021075+0000","last_fullsized":"2024-04-25T08:32:15.021075+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.455629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021529+0000","last_change":"2024-04-25T08:32:15.020851+0000","last_active":"2024-04-25T08:32:15.021529+0000","last_peered":"2024-04-25T08:32:15.021529+0000","last_clean":"2024-04-25T08:32:15.021529+0000","last_became_active":"2024-04-25T08:32:15.020559+0000","last_became_peered":"2024-04-25T08:32:15.020559+0000","last_unstale":"2024-04-25T08:32:15.021529+0000","last_undegraded":"2024-04-25T08:32:15.021529+0000","last_fullsized":"2024-04-25T08:32:15.021529+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:23.034958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.997094+0000","last_change":"2024-04-25T08:32:14.995267+0000","last_active":"2024-04-25T08:32:14.997094+0000","last_peered":"2024-04-25T08:32:14.997094+0000","last_clean":"2024-04-25T08:32:14.997094+0000","last_became_active":"2024-04-25T08:32:14.994913+0000","last_became_peered":"2024-04-25T08:32:14.994913+0000","last_unstale":"2024-04-25T08:32:14.997094+0000","last_undegraded":"2024-04-25T08:32:14.997094+0000","last_fullsized":"2024-04-25T08:32:14.997094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:12.242989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.011049+0000","last_change":"2024-04-25T08:32:15.008055+0000","last_active":"2024-04-25T08:32:15.011049+0000","last_peered":"2024-04-25T08:32:15.011049+0000","last_clean":"2024-04-25T08:32:15.011049+0000","last_became_active":"2024-04-25T08:32:15.007769+0000","last_became_peered":"2024-04-25T08:32:15.007769+0000","last_unstale":"2024-04-25T08:32:15.011049+0000","last_undegraded":"2024-04-25T08:32:15.011049+0000","last_fullsized":"2024-04-25T08:32:15.011049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:49.128390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999892+0000","last_change":"2024-04-25T08:32:14.993958+0000","last_active":"2024-04-25T08:32:14.999892+0000","last_peered":"2024-04-25T08:32:14.999892+0000","last_clean":"2024-04-25T08:32:14.999892+0000","last_became_active":"2024-04-25T08:32:14.993676+0000","last_became_peered":"2024-04-25T08:32:14.993676+0000","last_unstale":"2024-04-25T08:32:14.999892+0000","last_undegraded":"2024-04-25T08:32:14.999892+0000","last_fullsized":"2024-04-25T08:32:14.999892+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.683820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.995284+0000","last_change":"2024-04-25T08:32:14.994425+0000","last_active":"2024-04-25T08:32:14.995284+0000","last_peered":"2024-04-25T08:32:14.995284+0000","last_clean":"2024-04-25T08:32:14.995284+0000","last_became_active":"2024-04-25T08:32:14.994114+0000","last_became_peered":"2024-04-25T08:32:14.994114+0000","last_unstale":"2024-04-25T08:32:14.995284+0000","last_undegraded":"2024-04-25T08:32:14.995284+0000","last_fullsized":"2024-04-25T08:32:14.995284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:33.872454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013023+0000","last_change":"2024-04-25T08:32:15.010589+0000","last_active":"2024-04-25T08:32:15.013023+0000","last_peered":"2024-04-25T08:32:15.013023+0000","last_clean":"2024-04-25T08:32:15.013023+0000","last_became_active":"2024-04-25T08:32:15.010314+0000","last_became_peered":"2024-04-25T08:32:15.010314+0000","last_unstale":"2024-04-25T08:32:15.013023+0000","last_undegraded":"2024-04-25T08:32:15.013023+0000","last_fullsized":"2024-04-25T08:32:15.013023+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:42.145872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997049+0000","last_change":"2024-04-25T08:32:15.002598+0000","last_active":"2024-04-25T08:32:15.997049+0000","last_peered":"2024-04-25T08:32:15.997049+0000","last_clean":"2024-04-25T08:32:15.997049+0000","last_became_active":"2024-04-25T08:32:15.002296+0000","last_became_peered":"2024-04-25T08:32:15.002296+0000","last_unstale":"2024-04-25T08:32:15.997049+0000","last_undegraded":"2024-04-25T08:32:15.997049+0000","last_fullsized":"2024-04-25T08:32:15.997049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.997054+0000","last_change":"2024-04-25T08:32:14.995980+0000","last_active":"2024-04-25T08:32:14.997054+0000","last_peered":"2024-04-25T08:32:14.997054+0000","last_clean":"2024-04-25T08:32:14.997054+0000","last_became_active":"2024-04-25T08:32:14.995702+0000","last_became_peered":"2024-04-25T08:32:14.995702+0000","last_unstale":"2024-04-25T08:32:14.997054+0000","last_undegraded":"2024-04-25T08:32:14.997054+0000","last_fullsized":"2024-04-25T08:32:14.997054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:50.340059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020129+0000","last_change":"2024-04-25T08:32:15.017186+0000","last_active":"2024-04-25T08:32:15.020129+0000","last_peered":"2024-04-25T08:32:15.020129+0000","last_clean":"2024-04-25T08:32:15.020129+0000","last_became_active":"2024-04-25T08:32:15.015982+0000","last_became_peered":"2024-04-25T08:32:15.015982+0000","last_unstale":"2024-04-25T08:32:15.020129+0000","last_undegraded":"2024-04-25T08:32:15.020129+0000","last_fullsized":"2024-04-25T08:32:15.020129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997206+0000","last_change":"2024-04-25T08:32:15.006063+0000","last_active":"2024-04-25T08:32:15.997206+0000","last_peered":"2024-04-25T08:32:15.997206+0000","last_clean":"2024-04-25T08:32:15.997206+0000","last_became_active":"2024-04-25T08:32:15.005802+0000","last_became_peered":"2024-04-25T08:32:15.005802+0000","last_unstale":"2024-04-25T08:32:15.997206+0000","last_undegraded":"2024-04-25T08:32:15.997206+0000","last_fullsized":"2024-04-25T08:32:15.997206+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.455629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.014537+0000","last_change":"2024-04-25T08:32:15.011361+0000","last_active":"2024-04-25T08:32:15.014537+0000","last_peered":"2024-04-25T08:32:15.014537+0000","last_clean":"2024-04-25T08:32:15.014537+0000","last_became_active":"2024-04-25T08:32:15.011101+0000","last_became_peered":"2024-04-25T08:32:15.011101+0000","last_unstale":"2024-04-25T08:32:15.014537+0000","last_undegraded":"2024-04-25T08:32:15.014537+0000","last_fullsized":"2024-04-25T08:32:15.014537+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:12.991306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999253+0000","last_change":"2024-04-25T08:32:14.994236+0000","last_active":"2024-04-25T08:32:15.999253+0000","last_peered":"2024-04-25T08:32:15.999253+0000","last_clean":"2024-04-25T08:32:15.999253+0000","last_became_active":"2024-04-25T08:32:14.993972+0000","last_became_peered":"2024-04-25T08:32:14.993972+0000","last_unstale":"2024-04-25T08:32:15.999253+0000","last_undegraded":"2024-04-25T08:32:15.999253+0000","last_fullsized":"2024-04-25T08:32:15.999253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:43.786558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999274+0000","last_change":"2024-04-25T08:32:14.998730+0000","last_active":"2024-04-25T08:32:15.999274+0000","last_peered":"2024-04-25T08:32:15.999274+0000","last_clean":"2024-04-25T08:32:15.999274+0000","last_became_active":"2024-04-25T08:32:14.998447+0000","last_became_peered":"2024-04-25T08:32:14.998447+0000","last_unstale":"2024-04-25T08:32:15.999274+0000","last_undegraded":"2024-04-25T08:32:15.999274+0000","last_fullsized":"2024-04-25T08:32:15.999274+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:48.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021004+0000","last_change":"2024-04-25T08:32:15.017636+0000","last_active":"2024-04-25T08:32:15.021004+0000","last_peered":"2024-04-25T08:32:15.021004+0000","last_clean":"2024-04-25T08:32:15.021004+0000","last_became_active":"2024-04-25T08:32:15.017307+0000","last_became_peered":"2024-04-25T08:32:15.017307+0000","last_unstale":"2024-04-25T08:32:15.021004+0000","last_undegraded":"2024-04-25T08:32:15.021004+0000","last_fullsized":"2024-04-25T08:32:15.021004+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:42.411147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010993+0000","last_change":"2024-04-25T08:32:15.007297+0000","last_active":"2024-04-25T08:32:15.010993+0000","last_peered":"2024-04-25T08:32:15.010993+0000","last_clean":"2024-04-25T08:32:15.010993+0000","last_became_active":"2024-04-25T08:32:15.007014+0000","last_became_peered":"2024-04-25T08:32:15.007014+0000","last_unstale":"2024-04-25T08:32:15.010993+0000","last_undegraded":"2024-04-25T08:32:15.010993+0000","last_fullsized":"2024-04-25T08:32:15.010993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:16.595926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010939+0000","last_change":"2024-04-25T08:32:15.008795+0000","last_active":"2024-04-25T08:32:15.010939+0000","last_peered":"2024-04-25T08:32:15.010939+0000","last_clean":"2024-04-25T08:32:15.010939+0000","last_became_active":"2024-04-25T08:32:15.008463+0000","last_became_peered":"2024-04-25T08:32:15.008463+0000","last_unstale":"2024-04-25T08:32:15.010939+0000","last_undegraded":"2024-04-25T08:32:15.010939+0000","last_fullsized":"2024-04-25T08:32:15.010939+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.261984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020078+0000","last_change":"2024-04-25T08:32:15.016656+0000","last_active":"2024-04-25T08:32:15.020078+0000","last_peered":"2024-04-25T08:32:15.020078+0000","last_clean":"2024-04-25T08:32:15.020078+0000","last_became_active":"2024-04-25T08:32:15.015669+0000","last_became_peered":"2024-04-25T08:32:15.015669+0000","last_unstale":"2024-04-25T08:32:15.020078+0000","last_undegraded":"2024-04-25T08:32:15.020078+0000","last_fullsized":"2024-04-25T08:32:15.020078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020385+0000","last_change":"2024-04-25T08:32:15.011847+0000","last_active":"2024-04-25T08:32:15.020385+0000","last_peered":"2024-04-25T08:32:15.020385+0000","last_clean":"2024-04-25T08:32:15.020385+0000","last_became_active":"2024-04-25T08:32:15.011108+0000","last_became_peered":"2024-04-25T08:32:15.011108+0000","last_unstale":"2024-04-25T08:32:15.020385+0000","last_undegraded":"2024-04-25T08:32:15.020385+0000","last_fullsized":"2024-04-25T08:32:15.020385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.013735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.016482+0000","last_change":"2024-04-25T08:32:15.012556+0000","last_active":"2024-04-25T08:32:15.016482+0000","last_peered":"2024-04-25T08:32:15.016482+0000","last_clean":"2024-04-25T08:32:15.016482+0000","last_became_active":"2024-04-25T08:32:15.012282+0000","last_became_peered":"2024-04-25T08:32:15.012282+0000","last_unstale":"2024-04-25T08:32:15.016482+0000","last_undegraded":"2024-04-25T08:32:15.016482+0000","last_fullsized":"2024-04-25T08:32:15.016482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:28.834471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999349+0000","last_change":"2024-04-25T08:32:15.003108+0000","last_active":"2024-04-25T08:32:15.999349+0000","last_peered":"2024-04-25T08:32:15.999349+0000","last_clean":"2024-04-25T08:32:15.999349+0000","last_became_active":"2024-04-25T08:32:15.002850+0000","last_became_peered":"2024-04-25T08:32:15.002850+0000","last_unstale":"2024-04-25T08:32:15.999349+0000","last_undegraded":"2024-04-25T08:32:15.999349+0000","last_fullsized":"2024-04-25T08:32:15.999349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:42.145872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020315+0000","last_change":"2024-04-25T08:32:15.016978+0000","last_active":"2024-04-25T08:32:15.020315+0000","last_peered":"2024-04-25T08:32:15.020315+0000","last_clean":"2024-04-25T08:32:15.020315+0000","last_became_active":"2024-04-25T08:32:15.013597+0000","last_became_peered":"2024-04-25T08:32:15.013597+0000","last_unstale":"2024-04-25T08:32:15.020315+0000","last_undegraded":"2024-04-25T08:32:15.020315+0000","last_fullsized":"2024-04-25T08:32:15.020315+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997139+0000","last_change":"2024-04-25T08:32:15.014990+0000","last_active":"2024-04-25T08:32:15.997139+0000","last_peered":"2024-04-25T08:32:15.997139+0000","last_clean":"2024-04-25T08:32:15.997139+0000","last_became_active":"2024-04-25T08:32:15.014698+0000","last_became_peered":"2024-04-25T08:32:15.014698+0000","last_unstale":"2024-04-25T08:32:15.997139+0000","last_undegraded":"2024-04-25T08:32:15.997139+0000","last_fullsized":"2024-04-25T08:32:15.997139+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:42.411147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020279+0000","last_change":"2024-04-25T08:32:15.014991+0000","last_active":"2024-04-25T08:32:15.020279+0000","last_peered":"2024-04-25T08:32:15.020279+0000","last_clean":"2024-04-25T08:32:15.020279+0000","last_became_active":"2024-04-25T08:32:15.014358+0000","last_became_peered":"2024-04-25T08:32:15.014358+0000","last_unstale":"2024-04-25T08:32:15.020279+0000","last_undegraded":"2024-04-25T08:32:15.020279+0000","last_fullsized":"2024-04-25T08:32:15.020279+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.683820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999815+0000","last_change":"2024-04-25T08:32:14.998608+0000","last_active":"2024-04-25T08:32:14.999815+0000","last_peered":"2024-04-25T08:32:14.999815+0000","last_clean":"2024-04-25T08:32:14.999815+0000","last_became_active":"2024-04-25T08:32:14.997988+0000","last_became_peered":"2024-04-25T08:32:14.997988+0000","last_unstale":"2024-04-25T08:32:14.999815+0000","last_undegraded":"2024-04-25T08:32:14.999815+0000","last_fullsized":"2024-04-25T08:32:14.999815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996593+0000","last_change":"2024-04-25T08:32:15.004050+0000","last_active":"2024-04-25T08:32:15.996593+0000","last_peered":"2024-04-25T08:32:15.996593+0000","last_clean":"2024-04-25T08:32:15.996593+0000","last_became_active":"2024-04-25T08:32:15.003017+0000","last_became_peered":"2024-04-25T08:32:15.003017+0000","last_unstale":"2024-04-25T08:32:15.996593+0000","last_undegraded":"2024-04-25T08:32:15.996593+0000","last_fullsized":"2024-04-25T08:32:15.996593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999318+0000","last_change":"2024-04-25T08:32:13.993725+0000","last_active":"2024-04-25T08:32:15.999318+0000","last_peered":"2024-04-25T08:32:15.999318+0000","last_clean":"2024-04-25T08:32:15.999318+0000","last_became_active":"2024-04-25T08:32:13.993433+0000","last_became_peered":"2024-04-25T08:32:13.993433+0000","last_unstale":"2024-04-25T08:32:15.999318+0000","last_undegraded":"2024-04-25T08:32:15.999318+0000","last_fullsized":"2024-04-25T08:32:15.999318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:49.338054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999598+0000","last_change":"2024-04-25T08:32:14.996622+0000","last_active":"2024-04-25T08:32:14.999598+0000","last_peered":"2024-04-25T08:32:14.999598+0000","last_clean":"2024-04-25T08:32:14.999598+0000","last_became_active":"2024-04-25T08:32:14.996344+0000","last_became_peered":"2024-04-25T08:32:14.996344+0000","last_unstale":"2024-04-25T08:32:14.999598+0000","last_undegraded":"2024-04-25T08:32:14.999598+0000","last_fullsized":"2024-04-25T08:32:14.999598+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021333+0000","last_change":"2024-04-25T08:32:13.995941+0000","last_active":"2024-04-25T08:32:15.021333+0000","last_peered":"2024-04-25T08:32:15.021333+0000","last_clean":"2024-04-25T08:32:15.021333+0000","last_became_active":"2024-04-25T08:32:13.995664+0000","last_became_peered":"2024-04-25T08:32:13.995664+0000","last_unstale":"2024-04-25T08:32:15.021333+0000","last_undegraded":"2024-04-25T08:32:15.021333+0000","last_fullsized":"2024-04-25T08:32:15.021333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:01.905014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999346+0000","last_change":"2024-04-25T08:32:14.998847+0000","last_active":"2024-04-25T08:32:15.999346+0000","last_peered":"2024-04-25T08:32:15.999346+0000","last_clean":"2024-04-25T08:32:15.999346+0000","last_became_active":"2024-04-25T08:32:14.998567+0000","last_became_peered":"2024-04-25T08:32:14.998567+0000","last_unstale":"2024-04-25T08:32:15.999346+0000","last_undegraded":"2024-04-25T08:32:15.999346+0000","last_fullsized":"2024-04-25T08:32:15.999346+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:12.991306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.014286+0000","last_change":"2024-04-25T08:32:13.997194+0000","last_active":"2024-04-25T08:32:15.014286+0000","last_peered":"2024-04-25T08:32:15.014286+0000","last_clean":"2024-04-25T08:32:15.014286+0000","last_became_active":"2024-04-25T08:32:13.996924+0000","last_became_peered":"2024-04-25T08:32:13.996924+0000","last_unstale":"2024-04-25T08:32:15.014286+0000","last_undegraded":"2024-04-25T08:32:15.014286+0000","last_fullsized":"2024-04-25T08:32:15.014286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:29.994423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004914+0000","last_change":"2024-04-25T08:32:15.000366+0000","last_active":"2024-04-25T08:32:15.004914+0000","last_peered":"2024-04-25T08:32:15.004914+0000","last_clean":"2024-04-25T08:32:15.004914+0000","last_became_active":"2024-04-25T08:32:14.999937+0000","last_became_peered":"2024-04-25T08:32:14.999937+0000","last_unstale":"2024-04-25T08:32:15.004914+0000","last_undegraded":"2024-04-25T08:32:15.004914+0000","last_fullsized":"2024-04-25T08:32:15.004914+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020784+0000","last_change":"2024-04-25T08:32:13.995763+0000","last_active":"2024-04-25T08:32:15.020784+0000","last_peered":"2024-04-25T08:32:15.020784+0000","last_clean":"2024-04-25T08:32:15.020784+0000","last_became_active":"2024-04-25T08:32:13.995462+0000","last_became_peered":"2024-04-25T08:32:13.995462+0000","last_unstale":"2024-04-25T08:32:15.020784+0000","last_undegraded":"2024-04-25T08:32:15.020784+0000","last_fullsized":"2024-04-25T08:32:15.020784+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:33.681815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.012043+0000","last_change":"2024-04-25T08:32:15.011521+0000","last_active":"2024-04-25T08:32:15.012043+0000","last_peered":"2024-04-25T08:32:15.012043+0000","last_clean":"2024-04-25T08:32:15.012043+0000","last_became_active":"2024-04-25T08:32:15.011221+0000","last_became_peered":"2024-04-25T08:32:15.011221+0000","last_unstale":"2024-04-25T08:32:15.012043+0000","last_undegraded":"2024-04-25T08:32:15.012043+0000","last_fullsized":"2024-04-25T08:32:15.012043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:50.340059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010738+0000","last_change":"2024-04-25T08:32:14.004026+0000","last_active":"2024-04-25T08:32:15.010738+0000","last_peered":"2024-04-25T08:32:15.010738+0000","last_clean":"2024-04-25T08:32:15.010738+0000","last_became_active":"2024-04-25T08:32:14.003739+0000","last_became_peered":"2024-04-25T08:32:14.003739+0000","last_unstale":"2024-04-25T08:32:15.010738+0000","last_undegraded":"2024-04-25T08:32:15.010738+0000","last_fullsized":"2024-04-25T08:32:15.010738+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:52.448640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997136+0000","last_change":"2024-04-25T08:32:15.017225+0000","last_active":"2024-04-25T08:32:15.997136+0000","last_peered":"2024-04-25T08:32:15.997136+0000","last_clean":"2024-04-25T08:32:15.997136+0000","last_became_active":"2024-04-25T08:32:15.016947+0000","last_became_peered":"2024-04-25T08:32:15.016947+0000","last_unstale":"2024-04-25T08:32:15.997136+0000","last_undegraded":"2024-04-25T08:32:15.997136+0000","last_fullsized":"2024-04-25T08:32:15.997136+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.013735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.009155+0000","last_change":"2024-04-25T08:32:13.994959+0000","last_active":"2024-04-25T08:32:15.009155+0000","last_peered":"2024-04-25T08:32:15.009155+0000","last_clean":"2024-04-25T08:32:15.009155+0000","last_became_active":"2024-04-25T08:32:13.994695+0000","last_became_peered":"2024-04-25T08:32:13.994695+0000","last_unstale":"2024-04-25T08:32:15.009155+0000","last_undegraded":"2024-04-25T08:32:15.009155+0000","last_fullsized":"2024-04-25T08:32:15.009155+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:29.994423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.017604+0000","last_change":"2024-04-25T08:32:15.011614+0000","last_active":"2024-04-25T08:32:15.017604+0000","last_peered":"2024-04-25T08:32:15.017604+0000","last_clean":"2024-04-25T08:32:15.017604+0000","last_became_active":"2024-04-25T08:32:15.010727+0000","last_became_peered":"2024-04-25T08:32:15.010727+0000","last_unstale":"2024-04-25T08:32:15.017604+0000","last_undegraded":"2024-04-25T08:32:15.017604+0000","last_fullsized":"2024-04-25T08:32:15.017604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.523775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.998990+0000","last_change":"2024-04-25T08:32:13.995416+0000","last_active":"2024-04-25T08:32:15.998990+0000","last_peered":"2024-04-25T08:32:15.998990+0000","last_clean":"2024-04-25T08:32:15.998990+0000","last_became_active":"2024-04-25T08:32:13.994915+0000","last_became_peered":"2024-04-25T08:32:13.994915+0000","last_unstale":"2024-04-25T08:32:15.998990+0000","last_undegraded":"2024-04-25T08:32:15.998990+0000","last_fullsized":"2024-04-25T08:32:15.998990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:32.345591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996916+0000","last_change":"2024-04-25T08:32:15.017295+0000","last_active":"2024-04-25T08:32:15.996916+0000","last_peered":"2024-04-25T08:32:15.996916+0000","last_clean":"2024-04-25T08:32:15.996916+0000","last_became_active":"2024-04-25T08:32:15.016971+0000","last_became_peered":"2024-04-25T08:32:15.016971+0000","last_unstale":"2024-04-25T08:32:15.996916+0000","last_undegraded":"2024-04-25T08:32:15.996916+0000","last_fullsized":"2024-04-25T08:32:15.996916+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.993046+0000","last_change":"2024-04-25T08:32:13.994678+0000","last_active":"2024-04-25T08:32:14.993046+0000","last_peered":"2024-04-25T08:32:14.993046+0000","last_clean":"2024-04-25T08:32:14.993046+0000","last_became_active":"2024-04-25T08:32:13.994295+0000","last_became_peered":"2024-04-25T08:32:13.994295+0000","last_unstale":"2024-04-25T08:32:14.993046+0000","last_undegraded":"2024-04-25T08:32:14.993046+0000","last_fullsized":"2024-04-25T08:32:14.993046+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:39.881857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.000546+0000","last_change":"2024-04-25T08:32:14.999446+0000","last_active":"2024-04-25T08:32:15.000546+0000","last_peered":"2024-04-25T08:32:15.000546+0000","last_clean":"2024-04-25T08:32:15.000546+0000","last_became_active":"2024-04-25T08:32:14.999128+0000","last_became_peered":"2024-04-25T08:32:14.999128+0000","last_unstale":"2024-04-25T08:32:15.000546+0000","last_undegraded":"2024-04-25T08:32:15.000546+0000","last_fullsized":"2024-04-25T08:32:15.000546+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:22.112330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997026+0000","last_change":"2024-04-25T08:32:13.993849+0000","last_active":"2024-04-25T08:32:15.997026+0000","last_peered":"2024-04-25T08:32:15.997026+0000","last_clean":"2024-04-25T08:32:15.997026+0000","last_became_active":"2024-04-25T08:32:13.993579+0000","last_became_peered":"2024-04-25T08:32:13.993579+0000","last_unstale":"2024-04-25T08:32:15.997026+0000","last_undegraded":"2024-04-25T08:32:15.997026+0000","last_fullsized":"2024-04-25T08:32:15.997026+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.106774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.998875+0000","last_change":"2024-04-25T08:32:14.995372+0000","last_active":"2024-04-25T08:32:14.998875+0000","last_peered":"2024-04-25T08:32:14.998875+0000","last_clean":"2024-04-25T08:32:14.998875+0000","last_became_active":"2024-04-25T08:32:14.995091+0000","last_became_peered":"2024-04-25T08:32:14.995091+0000","last_unstale":"2024-04-25T08:32:14.998875+0000","last_undegraded":"2024-04-25T08:32:14.998875+0000","last_fullsized":"2024-04-25T08:32:14.998875+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992316+0000","last_change":"2024-04-25T08:32:13.997478+0000","last_active":"2024-04-25T08:32:14.992316+0000","last_peered":"2024-04-25T08:32:14.992316+0000","last_clean":"2024-04-25T08:32:14.992316+0000","last_became_active":"2024-04-25T08:32:13.997221+0000","last_became_peered":"2024-04-25T08:32:13.997221+0000","last_unstale":"2024-04-25T08:32:14.992316+0000","last_undegraded":"2024-04-25T08:32:14.992316+0000","last_fullsized":"2024-04-25T08:32:14.992316+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:32.345591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010613+0000","last_change":"2024-04-25T08:32:14.998600+0000","last_active":"2024-04-25T08:32:15.010613+0000","last_peered":"2024-04-25T08:32:15.010613+0000","last_clean":"2024-04-25T08:32:15.010613+0000","last_became_active":"2024-04-25T08:32:14.998292+0000","last_became_peered":"2024-04-25T08:32:14.998292+0000","last_unstale":"2024-04-25T08:32:15.010613+0000","last_undegraded":"2024-04-25T08:32:15.010613+0000","last_fullsized":"2024-04-25T08:32:15.010613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:31.455212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997732+0000","last_change":"2024-04-25T08:32:13.999888+0000","last_active":"2024-04-25T08:32:15.997732+0000","last_peered":"2024-04-25T08:32:15.997732+0000","last_clean":"2024-04-25T08:32:15.997732+0000","last_became_active":"2024-04-25T08:32:13.999679+0000","last_became_peered":"2024-04-25T08:32:13.999679+0000","last_unstale":"2024-04-25T08:32:15.997732+0000","last_undegraded":"2024-04-25T08:32:15.997732+0000","last_fullsized":"2024-04-25T08:32:15.997732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:30.453207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.008928+0000","last_change":"2024-04-25T08:32:15.008169+0000","last_active":"2024-04-25T08:32:15.008928+0000","last_peered":"2024-04-25T08:32:15.008928+0000","last_clean":"2024-04-25T08:32:15.008928+0000","last_became_active":"2024-04-25T08:32:15.007807+0000","last_became_peered":"2024-04-25T08:32:15.007807+0000","last_unstale":"2024-04-25T08:32:15.008928+0000","last_undegraded":"2024-04-25T08:32:15.008928+0000","last_fullsized":"2024-04-25T08:32:15.008928+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:12.242989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021151+0000","last_change":"2024-04-25T08:32:14.001704+0000","last_active":"2024-04-25T08:32:15.021151+0000","last_peered":"2024-04-25T08:32:15.021151+0000","last_clean":"2024-04-25T08:32:15.021151+0000","last_became_active":"2024-04-25T08:32:14.001419+0000","last_became_peered":"2024-04-25T08:32:14.001419+0000","last_unstale":"2024-04-25T08:32:15.021151+0000","last_undegraded":"2024-04-25T08:32:15.021151+0000","last_fullsized":"2024-04-25T08:32:15.021151+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:52.307141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.002162+0000","last_change":"2024-04-25T08:32:15.001570+0000","last_active":"2024-04-25T08:32:15.002162+0000","last_peered":"2024-04-25T08:32:15.002162+0000","last_clean":"2024-04-25T08:32:15.002162+0000","last_became_active":"2024-04-25T08:32:15.001161+0000","last_became_peered":"2024-04-25T08:32:15.001161+0000","last_unstale":"2024-04-25T08:32:15.002162+0000","last_undegraded":"2024-04-25T08:32:15.002162+0000","last_fullsized":"2024-04-25T08:32:15.002162+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.523775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010844+0000","last_change":"2024-04-25T08:32:13.999313+0000","last_active":"2024-04-25T08:32:15.010844+0000","last_peered":"2024-04-25T08:32:15.010844+0000","last_clean":"2024-04-25T08:32:15.010844+0000","last_became_active":"2024-04-25T08:32:13.998989+0000","last_became_peered":"2024-04-25T08:32:13.998989+0000","last_unstale":"2024-04-25T08:32:15.010844+0000","last_undegraded":"2024-04-25T08:32:15.010844+0000","last_fullsized":"2024-04-25T08:32:15.010844+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:39.004812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019600+0000","last_change":"2024-04-25T08:32:15.015741+0000","last_active":"2024-04-25T08:32:15.019600+0000","last_peered":"2024-04-25T08:32:15.019600+0000","last_clean":"2024-04-25T08:32:15.019600+0000","last_became_active":"2024-04-25T08:32:15.015478+0000","last_became_peered":"2024-04-25T08:32:15.015478+0000","last_unstale":"2024-04-25T08:32:15.019600+0000","last_undegraded":"2024-04-25T08:32:15.019600+0000","last_fullsized":"2024-04-25T08:32:15.019600+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:32.556937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999154+0000","last_change":"2024-04-25T08:32:13.992863+0000","last_active":"2024-04-25T08:32:15.999154+0000","last_peered":"2024-04-25T08:32:15.999154+0000","last_clean":"2024-04-25T08:32:15.999154+0000","last_became_active":"2024-04-25T08:32:13.992571+0000","last_became_peered":"2024-04-25T08:32:13.992571+0000","last_unstale":"2024-04-25T08:32:15.999154+0000","last_undegraded":"2024-04-25T08:32:15.999154+0000","last_fullsized":"2024-04-25T08:32:15.999154+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:29.994423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.017342+0000","last_change":"2024-04-25T08:32:15.011383+0000","last_active":"2024-04-25T08:32:15.017342+0000","last_peered":"2024-04-25T08:32:15.017342+0000","last_clean":"2024-04-25T08:32:15.017342+0000","last_became_active":"2024-04-25T08:32:15.010350+0000","last_became_peered":"2024-04-25T08:32:15.010350+0000","last_unstale":"2024-04-25T08:32:15.017342+0000","last_undegraded":"2024-04-25T08:32:15.017342+0000","last_fullsized":"2024-04-25T08:32:15.017342+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997191+0000","last_change":"2024-04-25T08:32:13.994543+0000","last_active":"2024-04-25T08:32:15.997191+0000","last_peered":"2024-04-25T08:32:15.997191+0000","last_clean":"2024-04-25T08:32:15.997191+0000","last_became_active":"2024-04-25T08:32:13.994262+0000","last_became_peered":"2024-04-25T08:32:13.994262+0000","last_unstale":"2024-04-25T08:32:15.997191+0000","last_undegraded":"2024-04-25T08:32:15.997191+0000","last_fullsized":"2024-04-25T08:32:15.997191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.463887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020110+0000","last_change":"2024-04-25T08:32:15.016936+0000","last_active":"2024-04-25T08:32:15.020110+0000","last_peered":"2024-04-25T08:32:15.020110+0000","last_clean":"2024-04-25T08:32:15.020110+0000","last_became_active":"2024-04-25T08:32:15.016300+0000","last_became_peered":"2024-04-25T08:32:15.016300+0000","last_unstale":"2024-04-25T08:32:15.020110+0000","last_undegraded":"2024-04-25T08:32:15.020110+0000","last_fullsized":"2024-04-25T08:32:15.020110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019921+0000","last_change":"2024-04-25T08:32:13.999954+0000","last_active":"2024-04-25T08:32:15.019921+0000","last_peered":"2024-04-25T08:32:15.019921+0000","last_clean":"2024-04-25T08:32:15.019921+0000","last_became_active":"2024-04-25T08:32:13.999683+0000","last_became_peered":"2024-04-25T08:32:13.999683+0000","last_unstale":"2024-04-25T08:32:15.019921+0000","last_undegraded":"2024-04-25T08:32:15.019921+0000","last_fullsized":"2024-04-25T08:32:15.019921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:41.143867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.003419+0000","last_change":"2024-04-25T08:32:14.996914+0000","last_active":"2024-04-25T08:32:15.003419+0000","last_peered":"2024-04-25T08:32:15.003419+0000","last_clean":"2024-04-25T08:32:15.003419+0000","last_became_active":"2024-04-25T08:32:14.996613+0000","last_became_peered":"2024-04-25T08:32:14.996613+0000","last_unstale":"2024-04-25T08:32:15.003419+0000","last_undegraded":"2024-04-25T08:32:15.003419+0000","last_fullsized":"2024-04-25T08:32:15.003419+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:32.556937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.015428+0000","last_change":"2024-04-25T08:32:14.002719+0000","last_active":"2024-04-25T08:32:15.015428+0000","last_peered":"2024-04-25T08:32:15.015428+0000","last_clean":"2024-04-25T08:32:15.015428+0000","last_became_active":"2024-04-25T08:32:14.002413+0000","last_became_peered":"2024-04-25T08:32:14.002413+0000","last_unstale":"2024-04-25T08:32:15.015428+0000","last_undegraded":"2024-04-25T08:32:15.015428+0000","last_fullsized":"2024-04-25T08:32:15.015428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.463887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.016641+0000","last_change":"2024-04-25T08:32:15.015333+0000","last_active":"2024-04-25T08:32:15.016641+0000","last_peered":"2024-04-25T08:32:15.016641+0000","last_clean":"2024-04-25T08:32:15.016641+0000","last_became_active":"2024-04-25T08:32:15.014936+0000","last_became_peered":"2024-04-25T08:32:15.014936+0000","last_unstale":"2024-04-25T08:32:15.016641+0000","last_undegraded":"2024-04-25T08:32:15.016641+0000","last_fullsized":"2024-04-25T08:32:15.016641+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:10.108779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.011344+0000","last_change":"2024-04-25T08:32:14.000838+0000","last_active":"2024-04-25T08:32:15.011344+0000","last_peered":"2024-04-25T08:32:15.011344+0000","last_clean":"2024-04-25T08:32:15.011344+0000","last_became_active":"2024-04-25T08:32:14.000552+0000","last_became_peered":"2024-04-25T08:32:14.000552+0000","last_unstale":"2024-04-25T08:32:15.011344+0000","last_undegraded":"2024-04-25T08:32:15.011344+0000","last_fullsized":"2024-04-25T08:32:15.011344+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:05.432255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.012055+0000","last_change":"2024-04-25T08:32:15.011514+0000","last_active":"2024-04-25T08:32:15.012055+0000","last_peered":"2024-04-25T08:32:15.012055+0000","last_clean":"2024-04-25T08:32:15.012055+0000","last_became_active":"2024-04-25T08:32:15.011210+0000","last_became_peered":"2024-04-25T08:32:15.011210+0000","last_unstale":"2024-04-25T08:32:15.012055+0000","last_undegraded":"2024-04-25T08:32:15.012055+0000","last_fullsized":"2024-04-25T08:32:15.012055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:40.883862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999177+0000","last_change":"2024-04-25T08:32:14.000451+0000","last_active":"2024-04-25T08:32:15.999177+0000","last_peered":"2024-04-25T08:32:15.999177+0000","last_clean":"2024-04-25T08:32:15.999177+0000","last_became_active":"2024-04-25T08:32:14.000266+0000","last_became_peered":"2024-04-25T08:32:14.000266+0000","last_unstale":"2024-04-25T08:32:15.999177+0000","last_undegraded":"2024-04-25T08:32:15.999177+0000","last_fullsized":"2024-04-25T08:32:15.999177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:21.110324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992314+0000","last_change":"2024-04-25T08:32:14.990582+0000","last_active":"2024-04-25T08:32:14.992314+0000","last_peered":"2024-04-25T08:32:14.992314+0000","last_clean":"2024-04-25T08:32:14.992314+0000","last_became_active":"2024-04-25T08:32:14.990316+0000","last_became_peered":"2024-04-25T08:32:14.990316+0000","last_unstale":"2024-04-25T08:32:14.992314+0000","last_undegraded":"2024-04-25T08:32:14.992314+0000","last_fullsized":"2024-04-25T08:32:14.992314+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:22.112330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999388+0000","last_change":"2024-04-25T08:32:13.995332+0000","last_active":"2024-04-25T08:32:15.999388+0000","last_peered":"2024-04-25T08:32:15.999388+0000","last_clean":"2024-04-25T08:32:15.999388+0000","last_became_active":"2024-04-25T08:32:13.995053+0000","last_became_peered":"2024-04-25T08:32:13.995053+0000","last_unstale":"2024-04-25T08:32:15.999388+0000","last_undegraded":"2024-04-25T08:32:15.999388+0000","last_fullsized":"2024-04-25T08:32:15.999388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:32.870449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.998837+0000","last_change":"2024-04-25T08:32:14.995212+0000","last_active":"2024-04-25T08:32:14.998837+0000","last_peered":"2024-04-25T08:32:14.998837+0000","last_clean":"2024-04-25T08:32:14.998837+0000","last_became_active":"2024-04-25T08:32:14.994919+0000","last_became_peered":"2024-04-25T08:32:14.994919+0000","last_unstale":"2024-04-25T08:32:14.998837+0000","last_undegraded":"2024-04-25T08:32:14.998837+0000","last_fullsized":"2024-04-25T08:32:14.998837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:32.556937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.014024+0000","last_change":"2024-04-25T08:32:13.992513+0000","last_active":"2024-04-25T08:32:15.014024+0000","last_peered":"2024-04-25T08:32:15.014024+0000","last_clean":"2024-04-25T08:32:15.014024+0000","last_became_active":"2024-04-25T08:32:13.992055+0000","last_became_peered":"2024-04-25T08:32:13.992055+0000","last_unstale":"2024-04-25T08:32:15.014024+0000","last_undegraded":"2024-04-25T08:32:15.014024+0000","last_fullsized":"2024-04-25T08:32:15.014024+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:37.885848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999209+0000","last_change":"2024-04-25T08:32:14.995057+0000","last_active":"2024-04-25T08:32:15.999209+0000","last_peered":"2024-04-25T08:32:15.999209+0000","last_clean":"2024-04-25T08:32:15.999209+0000","last_became_active":"2024-04-25T08:32:14.994709+0000","last_became_peered":"2024-04-25T08:32:14.994709+0000","last_unstale":"2024-04-25T08:32:15.999209+0000","last_undegraded":"2024-04-25T08:32:15.999209+0000","last_fullsized":"2024-04-25T08:32:15.999209+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.261984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992035+0000","last_change":"2024-04-25T08:32:13.998848+0000","last_active":"2024-04-25T08:32:14.992035+0000","last_peered":"2024-04-25T08:32:14.992035+0000","last_clean":"2024-04-25T08:32:14.992035+0000","last_became_active":"2024-04-25T08:32:13.998570+0000","last_became_peered":"2024-04-25T08:32:13.998570+0000","last_unstale":"2024-04-25T08:32:14.992035+0000","last_undegraded":"2024-04-25T08:32:14.992035+0000","last_fullsized":"2024-04-25T08:32:14.992035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:14.901524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013005+0000","last_change":"2024-04-25T08:32:15.009976+0000","last_active":"2024-04-25T08:32:15.013005+0000","last_peered":"2024-04-25T08:32:15.013005+0000","last_clean":"2024-04-25T08:32:15.013005+0000","last_became_active":"2024-04-25T08:32:15.009373+0000","last_became_peered":"2024-04-25T08:32:15.009373+0000","last_unstale":"2024-04-25T08:32:15.013005+0000","last_undegraded":"2024-04-25T08:32:15.013005+0000","last_fullsized":"2024-04-25T08:32:15.013005+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:43.786558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.994276+0000","last_change":"2024-04-25T08:32:14.001519+0000","last_active":"2024-04-25T08:32:14.994276+0000","last_peered":"2024-04-25T08:32:14.994276+0000","last_clean":"2024-04-25T08:32:14.994276+0000","last_became_active":"2024-04-25T08:32:14.001174+0000","last_became_peered":"2024-04-25T08:32:14.001174+0000","last_unstale":"2024-04-25T08:32:14.994276+0000","last_undegraded":"2024-04-25T08:32:14.994276+0000","last_fullsized":"2024-04-25T08:32:14.994276+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:27.832466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.997374+0000","last_change":"2024-04-25T08:32:14.996743+0000","last_active":"2024-04-25T08:32:14.997374+0000","last_peered":"2024-04-25T08:32:14.997374+0000","last_clean":"2024-04-25T08:32:14.997374+0000","last_became_active":"2024-04-25T08:32:14.996453+0000","last_became_peered":"2024-04-25T08:32:14.996453+0000","last_unstale":"2024-04-25T08:32:14.997374+0000","last_undegraded":"2024-04-25T08:32:14.997374+0000","last_fullsized":"2024-04-25T08:32:14.997374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:40.883862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992598+0000","last_change":"2024-04-25T08:32:13.998203+0000","last_active":"2024-04-25T08:32:14.992598+0000","last_peered":"2024-04-25T08:32:14.992598+0000","last_clean":"2024-04-25T08:32:14.992598+0000","last_became_active":"2024-04-25T08:32:13.997887+0000","last_became_peered":"2024-04-25T08:32:13.997887+0000","last_unstale":"2024-04-25T08:32:14.992598+0000","last_undegraded":"2024-04-25T08:32:14.992598+0000","last_fullsized":"2024-04-25T08:32:14.992598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:29.631313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999795+0000","last_change":"2024-04-25T08:32:14.998485+0000","last_active":"2024-04-25T08:32:14.999795+0000","last_peered":"2024-04-25T08:32:14.999795+0000","last_clean":"2024-04-25T08:32:14.999795+0000","last_became_active":"2024-04-25T08:32:14.997709+0000","last_became_peered":"2024-04-25T08:32:14.997709+0000","last_unstale":"2024-04-25T08:32:14.999795+0000","last_undegraded":"2024-04-25T08:32:14.999795+0000","last_fullsized":"2024-04-25T08:32:14.999795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999359+0000","last_change":"2024-04-25T08:32:13.994688+0000","last_active":"2024-04-25T08:32:15.999359+0000","last_peered":"2024-04-25T08:32:15.999359+0000","last_clean":"2024-04-25T08:32:15.999359+0000","last_became_active":"2024-04-25T08:32:13.994430+0000","last_became_peered":"2024-04-25T08:32:13.994430+0000","last_unstale":"2024-04-25T08:32:15.999359+0000","last_undegraded":"2024-04-25T08:32:15.999359+0000","last_fullsized":"2024-04-25T08:32:15.999359+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:11.240984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013913+0000","last_change":"2024-04-25T08:32:14.999331+0000","last_active":"2024-04-25T08:32:15.013913+0000","last_peered":"2024-04-25T08:32:15.013913+0000","last_clean":"2024-04-25T08:32:15.013913+0000","last_became_active":"2024-04-25T08:32:14.999030+0000","last_became_peered":"2024-04-25T08:32:14.999030+0000","last_unstale":"2024-04-25T08:32:15.013913+0000","last_undegraded":"2024-04-25T08:32:15.013913+0000","last_fullsized":"2024-04-25T08:32:15.013913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:53.450645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997733+0000","last_change":"2024-04-25T08:32:14.001260+0000","last_active":"2024-04-25T08:32:15.997733+0000","last_peered":"2024-04-25T08:32:15.997733+0000","last_clean":"2024-04-25T08:32:15.997733+0000","last_became_active":"2024-04-25T08:32:14.000935+0000","last_became_peered":"2024-04-25T08:32:14.000935+0000","last_unstale":"2024-04-25T08:32:15.997733+0000","last_undegraded":"2024-04-25T08:32:15.997733+0000","last_fullsized":"2024-04-25T08:32:15.997733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:44.259979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004890+0000","last_change":"2024-04-25T08:32:14.997044+0000","last_active":"2024-04-25T08:32:15.004890+0000","last_peered":"2024-04-25T08:32:15.004890+0000","last_clean":"2024-04-25T08:32:15.004890+0000","last_became_active":"2024-04-25T08:32:14.996712+0000","last_became_peered":"2024-04-25T08:32:14.996712+0000","last_unstale":"2024-04-25T08:32:15.004890+0000","last_undegraded":"2024-04-25T08:32:15.004890+0000","last_fullsized":"2024-04-25T08:32:15.004890+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.994876+0000","last_change":"2024-04-25T08:32:13.994985+0000","last_active":"2024-04-25T08:32:14.994876+0000","last_peered":"2024-04-25T08:32:14.994876+0000","last_clean":"2024-04-25T08:32:14.994876+0000","last_became_active":"2024-04-25T08:32:13.994706+0000","last_became_peered":"2024-04-25T08:32:13.994706+0000","last_unstale":"2024-04-25T08:32:14.994876+0000","last_undegraded":"2024-04-25T08:32:14.994876+0000","last_fullsized":"2024-04-25T08:32:14.994876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.106774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019581+0000","last_change":"2024-04-25T08:32:15.013896+0000","last_active":"2024-04-25T08:32:15.019581+0000","last_peered":"2024-04-25T08:32:15.019581+0000","last_clean":"2024-04-25T08:32:15.019581+0000","last_became_active":"2024-04-25T08:32:15.013638+0000","last_became_peered":"2024-04-25T08:32:15.013638+0000","last_unstale":"2024-04-25T08:32:15.019581+0000","last_undegraded":"2024-04-25T08:32:15.019581+0000","last_fullsized":"2024-04-25T08:32:15.019581+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:15.903529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991867+0000","last_change":"2024-04-25T08:32:13.999061+0000","last_active":"2024-04-25T08:32:14.991867+0000","last_peered":"2024-04-25T08:32:14.991867+0000","last_clean":"2024-04-25T08:32:14.991867+0000","last_became_active":"2024-04-25T08:32:13.998788+0000","last_became_peered":"2024-04-25T08:32:13.998788+0000","last_unstale":"2024-04-25T08:32:14.991867+0000","last_undegraded":"2024-04-25T08:32:14.991867+0000","last_fullsized":"2024-04-25T08:32:14.991867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:11.989300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004367+0000","last_change":"2024-04-25T08:32:15.002659+0000","last_active":"2024-04-25T08:32:15.004367+0000","last_peered":"2024-04-25T08:32:15.004367+0000","last_clean":"2024-04-25T08:32:15.004367+0000","last_became_active":"2024-04-25T08:32:15.002396+0000","last_became_peered":"2024-04-25T08:32:15.002396+0000","last_unstale":"2024-04-25T08:32:15.004367+0000","last_undegraded":"2024-04-25T08:32:15.004367+0000","last_fullsized":"2024-04-25T08:32:15.004367+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.013735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992201+0000","last_change":"2024-04-25T08:32:13.997172+0000","last_active":"2024-04-25T08:32:14.992201+0000","last_peered":"2024-04-25T08:32:14.992201+0000","last_clean":"2024-04-25T08:32:14.992201+0000","last_became_active":"2024-04-25T08:32:13.996883+0000","last_became_peered":"2024-04-25T08:32:13.996883+0000","last_unstale":"2024-04-25T08:32:14.992201+0000","last_undegraded":"2024-04-25T08:32:14.992201+0000","last_fullsized":"2024-04-25T08:32:14.992201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.733354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013029+0000","last_change":"2024-04-25T08:32:15.010218+0000","last_active":"2024-04-25T08:32:15.013029+0000","last_peered":"2024-04-25T08:32:15.013029+0000","last_clean":"2024-04-25T08:32:15.013029+0000","last_became_active":"2024-04-25T08:32:15.009696+0000","last_became_peered":"2024-04-25T08:32:15.009696+0000","last_unstale":"2024-04-25T08:32:15.013029+0000","last_undegraded":"2024-04-25T08:32:15.013029+0000","last_fullsized":"2024-04-25T08:32:15.013029+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:48.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991437+0000","last_change":"2024-04-25T08:32:13.996157+0000","last_active":"2024-04-25T08:32:14.991437+0000","last_peered":"2024-04-25T08:32:14.991437+0000","last_clean":"2024-04-25T08:32:14.991437+0000","last_became_active":"2024-04-25T08:32:13.995870+0000","last_became_peered":"2024-04-25T08:32:13.995870+0000","last_unstale":"2024-04-25T08:32:14.991437+0000","last_undegraded":"2024-04-25T08:32:14.991437+0000","last_fullsized":"2024-04-25T08:32:14.991437+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:01.905014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004825+0000","last_change":"2024-04-25T08:32:15.000611+0000","last_active":"2024-04-25T08:32:15.004825+0000","last_peered":"2024-04-25T08:32:15.004825+0000","last_clean":"2024-04-25T08:32:15.004825+0000","last_became_active":"2024-04-25T08:32:14.999876+0000","last_became_peered":"2024-04-25T08:32:14.999876+0000","last_unstale":"2024-04-25T08:32:15.004825+0000","last_undegraded":"2024-04-25T08:32:15.004825+0000","last_fullsized":"2024-04-25T08:32:15.004825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.986377+0000","last_change":"2024-04-25T08:32:13.990048+0000","last_active":"2024-04-25T08:32:14.986377+0000","last_peered":"2024-04-25T08:32:14.986377+0000","last_clean":"2024-04-25T08:32:14.986377+0000","last_became_active":"2024-04-25T08:32:13.989722+0000","last_became_peered":"2024-04-25T08:32:13.989722+0000","last_unstale":"2024-04-25T08:32:14.986377+0000","last_undegraded":"2024-04-25T08:32:14.986377+0000","last_fullsized":"2024-04-25T08:32:14.986377+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:52.307141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999166+0000","last_change":"2024-04-25T08:32:14.992261+0000","last_active":"2024-04-25T08:32:15.999166+0000","last_peered":"2024-04-25T08:32:15.999166+0000","last_clean":"2024-04-25T08:32:15.999166+0000","last_became_active":"2024-04-25T08:32:14.991947+0000","last_became_peered":"2024-04-25T08:32:14.991947+0000","last_unstale":"2024-04-25T08:32:15.999166+0000","last_undegraded":"2024-04-25T08:32:15.999166+0000","last_fullsized":"2024-04-25T08:32:15.999166+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:31.455212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991104+0000","last_change":"2024-04-25T08:32:13.996123+0000","last_active":"2024-04-25T08:32:14.991104+0000","last_peered":"2024-04-25T08:32:14.991104+0000","last_clean":"2024-04-25T08:32:14.991104+0000","last_became_active":"2024-04-25T08:32:13.995711+0000","last_became_peered":"2024-04-25T08:32:13.995711+0000","last_unstale":"2024-04-25T08:32:14.991104+0000","last_undegraded":"2024-04-25T08:32:14.991104+0000","last_fullsized":"2024-04-25T08:32:14.991104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:49.338054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996592+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:32:15.996592+0000","last_peered":"2024-04-25T08:32:15.996592+0000","last_clean":"2024-04-25T08:32:15.996592+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:32:15.996592+0000","last_undegraded":"2024-04-25T08:32:15.996592+0000","last_fullsized":"2024-04-25T08:32:15.996592+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1241088,"data_stored":876841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35988,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335852,"statfs":{"total":96636764160,"available":96599912448,"internally_reserved":0,"allocated":659456,"data_stored":293693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":286349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35956,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335884,"statfs":{"total":96636764160,"available":96599945216,"internally_reserved":0,"allocated":626688,"data_stored":279182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":655360,"data_stored":287983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":626688,"data_stored":279182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35956,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335884,"statfs":{"total":96636764160,"available":96599945216,"internally_reserved":0,"allocated":626688,"data_stored":279182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40696,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331144,"statfs":{"total":96636764160,"available":96595091456,"internally_reserved":0,"allocated":1220608,"data_stored":869550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:32:17.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:32:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.128+0000 7f9dcdf30640 1 Processor -- start 2024-04-25T08:32:18.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.128+0000 7f9dcdf30640 1 -- start start 2024-04-25T08:32:18.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dcdf30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dc80cc8e0 0x7f9dc81625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dcdf30640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9dc80d1810 0x7f9dc80d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dcdf30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80cce50 con 0x7f9dc815dc90 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dcdf30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80d21c0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dcdf30640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc8161af0 con 0x7f9dc80d1810 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9dc80cc8e0 0x7f9dc81625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9dc815dc90 0x7f9dc815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58780/0 (socket says 172.21.15.60:58780) 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc77fe640 1 -- 172.21.15.60:0/4011922954 learned_addr learned my addr 172.21.15.60:0/4011922954 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc6ffd640 1 -- 172.21.15.60:0/4011922954 >> v2:172.21.15.100:3300/0 conn(0x7f9dc80d1810 msgr2=0x7f9dc80d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/4011922954 >> v2:172.21.15.100:3300/0 conn(0x7f9dc80d1810 0x7f9dc80d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc6ffd640 1 -- 172.21.15.60:0/4011922954 >> v1:172.21.15.60:6789/0 conn(0x7f9dc815dc90 legacy=0x7f9dc815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc6ffd640 1 -- 172.21.15.60:0/4011922954 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8161dd0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.129+0000 7f9dc67fc640 1 -- 172.21.15.60:0/4011922954 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543463377 0 0) 0x7f9dc80cce50 con 0x7f9dc815dc90 2024-04-25T08:32:18.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.130+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/4011922954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc81625d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cdbb0 tx=0x7f9dbc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49896e1b1512f6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.130+0000 7f9dc67fc640 1 -- 172.21.15.60:0/4011922954 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc037070 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.130+0000 7f9dc67fc640 1 -- 172.21.15.60:0/4011922954 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc032440 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.130+0000 7f9dc67fc640 1 -- 172.21.15.60:0/4011922954 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc02ae60 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 -- 172.21.15.60:0/4011922954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 msgr2=0x7f9dc81625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/4011922954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc81625d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80cdbb0 tx=0x7f9dbc02a110 comp rx=0 tx=0).stop 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 -- 172.21.15.60:0/4011922954 shutdown_connections 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/4011922954 >> v2:172.21.15.100:3300/0 conn(0x7f9dc80d1810 0x7f9dc80d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/4011922954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc81625d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 -- 172.21.15.60:0/4011922954 >> 172.21.15.60:0/4011922954 conn(0x7f9dc806f590 msgr2=0x7f9dc806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 -- 172.21.15.60:0/4011922954 shutdown_connections 2024-04-25T08:32:18.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.131+0000 7f9dcdf30640 1 -- 172.21.15.60:0/4011922954 wait complete. 2024-04-25T08:32:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.132+0000 7f9dcdf30640 1 Processor -- start 2024-04-25T08:32:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.132+0000 7f9dcdf30640 1 -- start start 2024-04-25T08:32:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dcdf30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dcdf30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc80d1810 0x7f9dc80c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dcdf30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc815dc90 0x7f9dc81f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dcdf30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815e070 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dcdf30640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cd130 con 0x7f9dc815dc90 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dcdf30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc815e1f0 con 0x7f9dc80d1810 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc80b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44530/0 (socket says 172.21.15.60:44530) 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 -- 172.21.15.60:0/2698094553 learned_addr learned my addr 172.21.15.60:0/2698094553 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc7fff640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc815dc90 0x7f9dc81f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc80d1810 0x7f9dc80c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 -- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc80d1810 msgr2=0x7f9dc80c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc80d1810 0x7f9dc80c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 -- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc815dc90 msgr2=0x7f9dc81f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc815dc90 0x7f9dc81f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 -- 172.21.15.60:0/2698094553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80c5ae0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.133+0000 7f9dc77fe640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc80b0470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c430 tx=0x7f9db800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.134+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80074e0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.134+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db80047e0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.134+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004bd0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.134+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc80c5dc0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.134+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc81f41c0 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.135+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc80d1e20 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.135+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9db8011070 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.135+0000 7f9daffff640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9db406b4d0 0x7f9db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.136+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9db8089f80 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.136+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9db406b4d0 0x7f9db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.136+0000 7f9dc6ffd640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9db406b4d0 0x7f9db406d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc034000 tx=0x7f9dbc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.140+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dc80d1e20 con 0x7f9dc80cc8e0 2024-04-25T08:32:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.318+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dc81625d0 con 0x7f9db406b4d0 2024-04-25T08:32:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.325+0000 7f9daffff640 1 -- 172.21.15.60:0/2698094553 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251900 (secure 0 0 0) 0x7f9dc81625d0 con 0x7f9db406b4d0 2024-04-25T08:32:18.324 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:18.325 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9db406b4d0 msgr2=0x7f9db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9db406b4d0 0x7f9db406d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc034000 tx=0x7f9dbc02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 msgr2=0x7f9dc80b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc80b0470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c430 tx=0x7f9db800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 shutdown_connections 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9db406b4d0 0x7f9db406d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dc815dc90 0x7f9dc81f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9dc80d1810 0x7f9dc80c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 --2- 172.21.15.60:0/2698094553 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9dc80cc8e0 0x7f9dc80b0470 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 >> 172.21.15.60:0/2698094553 conn(0x7f9dc806f590 msgr2=0x7f9dc815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 shutdown_connections 2024-04-25T08:32:18.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.328+0000 7f9dcdf30640 1 -- 172.21.15.60:0/2698094553 wait complete. 2024-04-25T08:32:18.336 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-04-25T08:32:17.436944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297224,"kb_used_data":6148,"kb_used_omap":12,"kb_used_meta":290931,"kb_avail":754677496,"statfs":{"total":773094113280,"available":772789755904,"internally_reserved":0,"allocated":6295552,"data_stored":3451962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":297913904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.457082"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991183+0000","last_change":"2024-04-25T08:32:14.989814+0000","last_active":"2024-04-25T08:32:14.991183+0000","last_peered":"2024-04-25T08:32:14.991183+0000","last_clean":"2024-04-25T08:32:14.991183+0000","last_became_active":"2024-04-25T08:32:14.989516+0000","last_became_peered":"2024-04-25T08:32:14.989516+0000","last_unstale":"2024-04-25T08:32:14.991183+0000","last_undegraded":"2024-04-25T08:32:14.991183+0000","last_fullsized":"2024-04-25T08:32:14.991183+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:33.347596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997071+0000","last_change":"2024-04-25T08:32:15.005789+0000","last_active":"2024-04-25T08:32:15.997071+0000","last_peered":"2024-04-25T08:32:15.997071+0000","last_clean":"2024-04-25T08:32:15.997071+0000","last_became_active":"2024-04-25T08:32:15.005466+0000","last_became_peered":"2024-04-25T08:32:15.005466+0000","last_unstale":"2024-04-25T08:32:15.997071+0000","last_undegraded":"2024-04-25T08:32:15.997071+0000","last_fullsized":"2024-04-25T08:32:15.997071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.523775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019573+0000","last_change":"2024-04-25T08:32:15.014835+0000","last_active":"2024-04-25T08:32:15.019573+0000","last_peered":"2024-04-25T08:32:15.019573+0000","last_clean":"2024-04-25T08:32:15.019573+0000","last_became_active":"2024-04-25T08:32:15.014327+0000","last_became_peered":"2024-04-25T08:32:15.014327+0000","last_unstale":"2024-04-25T08:32:15.019573+0000","last_undegraded":"2024-04-25T08:32:15.019573+0000","last_fullsized":"2024-04-25T08:32:15.019573+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:53.309146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996846+0000","last_change":"2024-04-25T08:32:15.008520+0000","last_active":"2024-04-25T08:32:15.996846+0000","last_peered":"2024-04-25T08:32:15.996846+0000","last_clean":"2024-04-25T08:32:15.996846+0000","last_became_active":"2024-04-25T08:32:15.008240+0000","last_became_peered":"2024-04-25T08:32:15.008240+0000","last_unstale":"2024-04-25T08:32:15.996846+0000","last_undegraded":"2024-04-25T08:32:15.996846+0000","last_fullsized":"2024-04-25T08:32:15.996846+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.002973+0000","last_change":"2024-04-25T08:32:15.001367+0000","last_active":"2024-04-25T08:32:15.002973+0000","last_peered":"2024-04-25T08:32:15.002973+0000","last_clean":"2024-04-25T08:32:15.002973+0000","last_became_active":"2024-04-25T08:32:15.001094+0000","last_became_peered":"2024-04-25T08:32:15.001094+0000","last_unstale":"2024-04-25T08:32:15.002973+0000","last_undegraded":"2024-04-25T08:32:15.002973+0000","last_fullsized":"2024-04-25T08:32:15.002973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:33.872454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013950+0000","last_change":"2024-04-25T08:32:15.000513+0000","last_active":"2024-04-25T08:32:15.013950+0000","last_peered":"2024-04-25T08:32:15.013950+0000","last_clean":"2024-04-25T08:32:15.013950+0000","last_became_active":"2024-04-25T08:32:15.000226+0000","last_became_peered":"2024-04-25T08:32:15.000226+0000","last_unstale":"2024-04-25T08:32:15.013950+0000","last_undegraded":"2024-04-25T08:32:15.013950+0000","last_fullsized":"2024-04-25T08:32:15.013950+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:40.006817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020259+0000","last_change":"2024-04-25T08:32:15.014800+0000","last_active":"2024-04-25T08:32:15.020259+0000","last_peered":"2024-04-25T08:32:15.020259+0000","last_clean":"2024-04-25T08:32:15.020259+0000","last_became_active":"2024-04-25T08:32:15.014091+0000","last_became_peered":"2024-04-25T08:32:15.014091+0000","last_unstale":"2024-04-25T08:32:15.020259+0000","last_undegraded":"2024-04-25T08:32:15.020259+0000","last_fullsized":"2024-04-25T08:32:15.020259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:31.735360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021075+0000","last_change":"2024-04-25T08:32:15.018706+0000","last_active":"2024-04-25T08:32:15.021075+0000","last_peered":"2024-04-25T08:32:15.021075+0000","last_clean":"2024-04-25T08:32:15.021075+0000","last_became_active":"2024-04-25T08:32:15.017501+0000","last_became_peered":"2024-04-25T08:32:15.017501+0000","last_unstale":"2024-04-25T08:32:15.021075+0000","last_undegraded":"2024-04-25T08:32:15.021075+0000","last_fullsized":"2024-04-25T08:32:15.021075+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.455629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021529+0000","last_change":"2024-04-25T08:32:15.020851+0000","last_active":"2024-04-25T08:32:15.021529+0000","last_peered":"2024-04-25T08:32:15.021529+0000","last_clean":"2024-04-25T08:32:15.021529+0000","last_became_active":"2024-04-25T08:32:15.020559+0000","last_became_peered":"2024-04-25T08:32:15.020559+0000","last_unstale":"2024-04-25T08:32:15.021529+0000","last_undegraded":"2024-04-25T08:32:15.021529+0000","last_fullsized":"2024-04-25T08:32:15.021529+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:23.034958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.997094+0000","last_change":"2024-04-25T08:32:14.995267+0000","last_active":"2024-04-25T08:32:14.997094+0000","last_peered":"2024-04-25T08:32:14.997094+0000","last_clean":"2024-04-25T08:32:14.997094+0000","last_became_active":"2024-04-25T08:32:14.994913+0000","last_became_peered":"2024-04-25T08:32:14.994913+0000","last_unstale":"2024-04-25T08:32:14.997094+0000","last_undegraded":"2024-04-25T08:32:14.997094+0000","last_fullsized":"2024-04-25T08:32:14.997094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:12.242989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.011049+0000","last_change":"2024-04-25T08:32:15.008055+0000","last_active":"2024-04-25T08:32:15.011049+0000","last_peered":"2024-04-25T08:32:15.011049+0000","last_clean":"2024-04-25T08:32:15.011049+0000","last_became_active":"2024-04-25T08:32:15.007769+0000","last_became_peered":"2024-04-25T08:32:15.007769+0000","last_unstale":"2024-04-25T08:32:15.011049+0000","last_undegraded":"2024-04-25T08:32:15.011049+0000","last_fullsized":"2024-04-25T08:32:15.011049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:49.128390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999892+0000","last_change":"2024-04-25T08:32:14.993958+0000","last_active":"2024-04-25T08:32:14.999892+0000","last_peered":"2024-04-25T08:32:14.999892+0000","last_clean":"2024-04-25T08:32:14.999892+0000","last_became_active":"2024-04-25T08:32:14.993676+0000","last_became_peered":"2024-04-25T08:32:14.993676+0000","last_unstale":"2024-04-25T08:32:14.999892+0000","last_undegraded":"2024-04-25T08:32:14.999892+0000","last_fullsized":"2024-04-25T08:32:14.999892+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.683820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.995284+0000","last_change":"2024-04-25T08:32:14.994425+0000","last_active":"2024-04-25T08:32:14.995284+0000","last_peered":"2024-04-25T08:32:14.995284+0000","last_clean":"2024-04-25T08:32:14.995284+0000","last_became_active":"2024-04-25T08:32:14.994114+0000","last_became_peered":"2024-04-25T08:32:14.994114+0000","last_unstale":"2024-04-25T08:32:14.995284+0000","last_undegraded":"2024-04-25T08:32:14.995284+0000","last_fullsized":"2024-04-25T08:32:14.995284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:33.872454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013023+0000","last_change":"2024-04-25T08:32:15.010589+0000","last_active":"2024-04-25T08:32:15.013023+0000","last_peered":"2024-04-25T08:32:15.013023+0000","last_clean":"2024-04-25T08:32:15.013023+0000","last_became_active":"2024-04-25T08:32:15.010314+0000","last_became_peered":"2024-04-25T08:32:15.010314+0000","last_unstale":"2024-04-25T08:32:15.013023+0000","last_undegraded":"2024-04-25T08:32:15.013023+0000","last_fullsized":"2024-04-25T08:32:15.013023+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:42.145872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997049+0000","last_change":"2024-04-25T08:32:15.002598+0000","last_active":"2024-04-25T08:32:15.997049+0000","last_peered":"2024-04-25T08:32:15.997049+0000","last_clean":"2024-04-25T08:32:15.997049+0000","last_became_active":"2024-04-25T08:32:15.002296+0000","last_became_peered":"2024-04-25T08:32:15.002296+0000","last_unstale":"2024-04-25T08:32:15.997049+0000","last_undegraded":"2024-04-25T08:32:15.997049+0000","last_fullsized":"2024-04-25T08:32:15.997049+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.997054+0000","last_change":"2024-04-25T08:32:14.995980+0000","last_active":"2024-04-25T08:32:14.997054+0000","last_peered":"2024-04-25T08:32:14.997054+0000","last_clean":"2024-04-25T08:32:14.997054+0000","last_became_active":"2024-04-25T08:32:14.995702+0000","last_became_peered":"2024-04-25T08:32:14.995702+0000","last_unstale":"2024-04-25T08:32:14.997054+0000","last_undegraded":"2024-04-25T08:32:14.997054+0000","last_fullsized":"2024-04-25T08:32:14.997054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:50.340059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020129+0000","last_change":"2024-04-25T08:32:15.017186+0000","last_active":"2024-04-25T08:32:15.020129+0000","last_peered":"2024-04-25T08:32:15.020129+0000","last_clean":"2024-04-25T08:32:15.020129+0000","last_became_active":"2024-04-25T08:32:15.015982+0000","last_became_peered":"2024-04-25T08:32:15.015982+0000","last_unstale":"2024-04-25T08:32:15.020129+0000","last_undegraded":"2024-04-25T08:32:15.020129+0000","last_fullsized":"2024-04-25T08:32:15.020129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997206+0000","last_change":"2024-04-25T08:32:15.006063+0000","last_active":"2024-04-25T08:32:15.997206+0000","last_peered":"2024-04-25T08:32:15.997206+0000","last_clean":"2024-04-25T08:32:15.997206+0000","last_became_active":"2024-04-25T08:32:15.005802+0000","last_became_peered":"2024-04-25T08:32:15.005802+0000","last_unstale":"2024-04-25T08:32:15.997206+0000","last_undegraded":"2024-04-25T08:32:15.997206+0000","last_fullsized":"2024-04-25T08:32:15.997206+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:42.455629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.014537+0000","last_change":"2024-04-25T08:32:15.011361+0000","last_active":"2024-04-25T08:32:15.014537+0000","last_peered":"2024-04-25T08:32:15.014537+0000","last_clean":"2024-04-25T08:32:15.014537+0000","last_became_active":"2024-04-25T08:32:15.011101+0000","last_became_peered":"2024-04-25T08:32:15.011101+0000","last_unstale":"2024-04-25T08:32:15.014537+0000","last_undegraded":"2024-04-25T08:32:15.014537+0000","last_fullsized":"2024-04-25T08:32:15.014537+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:12.991306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999253+0000","last_change":"2024-04-25T08:32:14.994236+0000","last_active":"2024-04-25T08:32:15.999253+0000","last_peered":"2024-04-25T08:32:15.999253+0000","last_clean":"2024-04-25T08:32:15.999253+0000","last_became_active":"2024-04-25T08:32:14.993972+0000","last_became_peered":"2024-04-25T08:32:14.993972+0000","last_unstale":"2024-04-25T08:32:15.999253+0000","last_undegraded":"2024-04-25T08:32:15.999253+0000","last_fullsized":"2024-04-25T08:32:15.999253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:43.786558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999274+0000","last_change":"2024-04-25T08:32:14.998730+0000","last_active":"2024-04-25T08:32:15.999274+0000","last_peered":"2024-04-25T08:32:15.999274+0000","last_clean":"2024-04-25T08:32:15.999274+0000","last_became_active":"2024-04-25T08:32:14.998447+0000","last_became_peered":"2024-04-25T08:32:14.998447+0000","last_unstale":"2024-04-25T08:32:15.999274+0000","last_undegraded":"2024-04-25T08:32:15.999274+0000","last_fullsized":"2024-04-25T08:32:15.999274+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:48.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021004+0000","last_change":"2024-04-25T08:32:15.017636+0000","last_active":"2024-04-25T08:32:15.021004+0000","last_peered":"2024-04-25T08:32:15.021004+0000","last_clean":"2024-04-25T08:32:15.021004+0000","last_became_active":"2024-04-25T08:32:15.017307+0000","last_became_peered":"2024-04-25T08:32:15.017307+0000","last_unstale":"2024-04-25T08:32:15.021004+0000","last_undegraded":"2024-04-25T08:32:15.021004+0000","last_fullsized":"2024-04-25T08:32:15.021004+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:42.411147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010993+0000","last_change":"2024-04-25T08:32:15.007297+0000","last_active":"2024-04-25T08:32:15.010993+0000","last_peered":"2024-04-25T08:32:15.010993+0000","last_clean":"2024-04-25T08:32:15.010993+0000","last_became_active":"2024-04-25T08:32:15.007014+0000","last_became_peered":"2024-04-25T08:32:15.007014+0000","last_unstale":"2024-04-25T08:32:15.010993+0000","last_undegraded":"2024-04-25T08:32:15.010993+0000","last_fullsized":"2024-04-25T08:32:15.010993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:16.595926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010939+0000","last_change":"2024-04-25T08:32:15.008795+0000","last_active":"2024-04-25T08:32:15.010939+0000","last_peered":"2024-04-25T08:32:15.010939+0000","last_clean":"2024-04-25T08:32:15.010939+0000","last_became_active":"2024-04-25T08:32:15.008463+0000","last_became_peered":"2024-04-25T08:32:15.008463+0000","last_unstale":"2024-04-25T08:32:15.010939+0000","last_undegraded":"2024-04-25T08:32:15.010939+0000","last_fullsized":"2024-04-25T08:32:15.010939+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.261984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020078+0000","last_change":"2024-04-25T08:32:15.016656+0000","last_active":"2024-04-25T08:32:15.020078+0000","last_peered":"2024-04-25T08:32:15.020078+0000","last_clean":"2024-04-25T08:32:15.020078+0000","last_became_active":"2024-04-25T08:32:15.015669+0000","last_became_peered":"2024-04-25T08:32:15.015669+0000","last_unstale":"2024-04-25T08:32:15.020078+0000","last_undegraded":"2024-04-25T08:32:15.020078+0000","last_fullsized":"2024-04-25T08:32:15.020078+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020385+0000","last_change":"2024-04-25T08:32:15.011847+0000","last_active":"2024-04-25T08:32:15.020385+0000","last_peered":"2024-04-25T08:32:15.020385+0000","last_clean":"2024-04-25T08:32:15.020385+0000","last_became_active":"2024-04-25T08:32:15.011108+0000","last_became_peered":"2024-04-25T08:32:15.011108+0000","last_unstale":"2024-04-25T08:32:15.020385+0000","last_undegraded":"2024-04-25T08:32:15.020385+0000","last_fullsized":"2024-04-25T08:32:15.020385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.013735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.016482+0000","last_change":"2024-04-25T08:32:15.012556+0000","last_active":"2024-04-25T08:32:15.016482+0000","last_peered":"2024-04-25T08:32:15.016482+0000","last_clean":"2024-04-25T08:32:15.016482+0000","last_became_active":"2024-04-25T08:32:15.012282+0000","last_became_peered":"2024-04-25T08:32:15.012282+0000","last_unstale":"2024-04-25T08:32:15.016482+0000","last_undegraded":"2024-04-25T08:32:15.016482+0000","last_fullsized":"2024-04-25T08:32:15.016482+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:28.834471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999349+0000","last_change":"2024-04-25T08:32:15.003108+0000","last_active":"2024-04-25T08:32:15.999349+0000","last_peered":"2024-04-25T08:32:15.999349+0000","last_clean":"2024-04-25T08:32:15.999349+0000","last_became_active":"2024-04-25T08:32:15.002850+0000","last_became_peered":"2024-04-25T08:32:15.002850+0000","last_unstale":"2024-04-25T08:32:15.999349+0000","last_undegraded":"2024-04-25T08:32:15.999349+0000","last_fullsized":"2024-04-25T08:32:15.999349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:42.145872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020315+0000","last_change":"2024-04-25T08:32:15.016978+0000","last_active":"2024-04-25T08:32:15.020315+0000","last_peered":"2024-04-25T08:32:15.020315+0000","last_clean":"2024-04-25T08:32:15.020315+0000","last_became_active":"2024-04-25T08:32:15.013597+0000","last_became_peered":"2024-04-25T08:32:15.013597+0000","last_unstale":"2024-04-25T08:32:15.020315+0000","last_undegraded":"2024-04-25T08:32:15.020315+0000","last_fullsized":"2024-04-25T08:32:15.020315+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997139+0000","last_change":"2024-04-25T08:32:15.014990+0000","last_active":"2024-04-25T08:32:15.997139+0000","last_peered":"2024-04-25T08:32:15.997139+0000","last_clean":"2024-04-25T08:32:15.997139+0000","last_became_active":"2024-04-25T08:32:15.014698+0000","last_became_peered":"2024-04-25T08:32:15.014698+0000","last_unstale":"2024-04-25T08:32:15.997139+0000","last_undegraded":"2024-04-25T08:32:15.997139+0000","last_fullsized":"2024-04-25T08:32:15.997139+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:42.411147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020279+0000","last_change":"2024-04-25T08:32:15.014991+0000","last_active":"2024-04-25T08:32:15.020279+0000","last_peered":"2024-04-25T08:32:15.020279+0000","last_clean":"2024-04-25T08:32:15.020279+0000","last_became_active":"2024-04-25T08:32:15.014358+0000","last_became_peered":"2024-04-25T08:32:15.014358+0000","last_unstale":"2024-04-25T08:32:15.020279+0000","last_undegraded":"2024-04-25T08:32:15.020279+0000","last_fullsized":"2024-04-25T08:32:15.020279+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.683820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999815+0000","last_change":"2024-04-25T08:32:14.998608+0000","last_active":"2024-04-25T08:32:14.999815+0000","last_peered":"2024-04-25T08:32:14.999815+0000","last_clean":"2024-04-25T08:32:14.999815+0000","last_became_active":"2024-04-25T08:32:14.997988+0000","last_became_peered":"2024-04-25T08:32:14.997988+0000","last_unstale":"2024-04-25T08:32:14.999815+0000","last_undegraded":"2024-04-25T08:32:14.999815+0000","last_fullsized":"2024-04-25T08:32:14.999815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996593+0000","last_change":"2024-04-25T08:32:15.004050+0000","last_active":"2024-04-25T08:32:15.996593+0000","last_peered":"2024-04-25T08:32:15.996593+0000","last_clean":"2024-04-25T08:32:15.996593+0000","last_became_active":"2024-04-25T08:32:15.003017+0000","last_became_peered":"2024-04-25T08:32:15.003017+0000","last_unstale":"2024-04-25T08:32:15.996593+0000","last_undegraded":"2024-04-25T08:32:15.996593+0000","last_fullsized":"2024-04-25T08:32:15.996593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999318+0000","last_change":"2024-04-25T08:32:13.993725+0000","last_active":"2024-04-25T08:32:15.999318+0000","last_peered":"2024-04-25T08:32:15.999318+0000","last_clean":"2024-04-25T08:32:15.999318+0000","last_became_active":"2024-04-25T08:32:13.993433+0000","last_became_peered":"2024-04-25T08:32:13.993433+0000","last_unstale":"2024-04-25T08:32:15.999318+0000","last_undegraded":"2024-04-25T08:32:15.999318+0000","last_fullsized":"2024-04-25T08:32:15.999318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:49.338054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999598+0000","last_change":"2024-04-25T08:32:14.996622+0000","last_active":"2024-04-25T08:32:14.999598+0000","last_peered":"2024-04-25T08:32:14.999598+0000","last_clean":"2024-04-25T08:32:14.999598+0000","last_became_active":"2024-04-25T08:32:14.996344+0000","last_became_peered":"2024-04-25T08:32:14.996344+0000","last_unstale":"2024-04-25T08:32:14.999598+0000","last_undegraded":"2024-04-25T08:32:14.999598+0000","last_fullsized":"2024-04-25T08:32:14.999598+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021333+0000","last_change":"2024-04-25T08:32:13.995941+0000","last_active":"2024-04-25T08:32:15.021333+0000","last_peered":"2024-04-25T08:32:15.021333+0000","last_clean":"2024-04-25T08:32:15.021333+0000","last_became_active":"2024-04-25T08:32:13.995664+0000","last_became_peered":"2024-04-25T08:32:13.995664+0000","last_unstale":"2024-04-25T08:32:15.021333+0000","last_undegraded":"2024-04-25T08:32:15.021333+0000","last_fullsized":"2024-04-25T08:32:15.021333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:01.905014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999346+0000","last_change":"2024-04-25T08:32:14.998847+0000","last_active":"2024-04-25T08:32:15.999346+0000","last_peered":"2024-04-25T08:32:15.999346+0000","last_clean":"2024-04-25T08:32:15.999346+0000","last_became_active":"2024-04-25T08:32:14.998567+0000","last_became_peered":"2024-04-25T08:32:14.998567+0000","last_unstale":"2024-04-25T08:32:15.999346+0000","last_undegraded":"2024-04-25T08:32:15.999346+0000","last_fullsized":"2024-04-25T08:32:15.999346+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:12.991306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.014286+0000","last_change":"2024-04-25T08:32:13.997194+0000","last_active":"2024-04-25T08:32:15.014286+0000","last_peered":"2024-04-25T08:32:15.014286+0000","last_clean":"2024-04-25T08:32:15.014286+0000","last_became_active":"2024-04-25T08:32:13.996924+0000","last_became_peered":"2024-04-25T08:32:13.996924+0000","last_unstale":"2024-04-25T08:32:15.014286+0000","last_undegraded":"2024-04-25T08:32:15.014286+0000","last_fullsized":"2024-04-25T08:32:15.014286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:29.994423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004914+0000","last_change":"2024-04-25T08:32:15.000366+0000","last_active":"2024-04-25T08:32:15.004914+0000","last_peered":"2024-04-25T08:32:15.004914+0000","last_clean":"2024-04-25T08:32:15.004914+0000","last_became_active":"2024-04-25T08:32:14.999937+0000","last_became_peered":"2024-04-25T08:32:14.999937+0000","last_unstale":"2024-04-25T08:32:15.004914+0000","last_undegraded":"2024-04-25T08:32:15.004914+0000","last_fullsized":"2024-04-25T08:32:15.004914+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020784+0000","last_change":"2024-04-25T08:32:13.995763+0000","last_active":"2024-04-25T08:32:15.020784+0000","last_peered":"2024-04-25T08:32:15.020784+0000","last_clean":"2024-04-25T08:32:15.020784+0000","last_became_active":"2024-04-25T08:32:13.995462+0000","last_became_peered":"2024-04-25T08:32:13.995462+0000","last_unstale":"2024-04-25T08:32:15.020784+0000","last_undegraded":"2024-04-25T08:32:15.020784+0000","last_fullsized":"2024-04-25T08:32:15.020784+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:33.681815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.012043+0000","last_change":"2024-04-25T08:32:15.011521+0000","last_active":"2024-04-25T08:32:15.012043+0000","last_peered":"2024-04-25T08:32:15.012043+0000","last_clean":"2024-04-25T08:32:15.012043+0000","last_became_active":"2024-04-25T08:32:15.011221+0000","last_became_peered":"2024-04-25T08:32:15.011221+0000","last_unstale":"2024-04-25T08:32:15.012043+0000","last_undegraded":"2024-04-25T08:32:15.012043+0000","last_fullsized":"2024-04-25T08:32:15.012043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:50.340059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010738+0000","last_change":"2024-04-25T08:32:14.004026+0000","last_active":"2024-04-25T08:32:15.010738+0000","last_peered":"2024-04-25T08:32:15.010738+0000","last_clean":"2024-04-25T08:32:15.010738+0000","last_became_active":"2024-04-25T08:32:14.003739+0000","last_became_peered":"2024-04-25T08:32:14.003739+0000","last_unstale":"2024-04-25T08:32:15.010738+0000","last_undegraded":"2024-04-25T08:32:15.010738+0000","last_fullsized":"2024-04-25T08:32:15.010738+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:52.448640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997136+0000","last_change":"2024-04-25T08:32:15.017225+0000","last_active":"2024-04-25T08:32:15.997136+0000","last_peered":"2024-04-25T08:32:15.997136+0000","last_clean":"2024-04-25T08:32:15.997136+0000","last_became_active":"2024-04-25T08:32:15.016947+0000","last_became_peered":"2024-04-25T08:32:15.016947+0000","last_unstale":"2024-04-25T08:32:15.997136+0000","last_undegraded":"2024-04-25T08:32:15.997136+0000","last_fullsized":"2024-04-25T08:32:15.997136+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.013735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.009155+0000","last_change":"2024-04-25T08:32:13.994959+0000","last_active":"2024-04-25T08:32:15.009155+0000","last_peered":"2024-04-25T08:32:15.009155+0000","last_clean":"2024-04-25T08:32:15.009155+0000","last_became_active":"2024-04-25T08:32:13.994695+0000","last_became_peered":"2024-04-25T08:32:13.994695+0000","last_unstale":"2024-04-25T08:32:15.009155+0000","last_undegraded":"2024-04-25T08:32:15.009155+0000","last_fullsized":"2024-04-25T08:32:15.009155+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:29.994423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.017604+0000","last_change":"2024-04-25T08:32:15.011614+0000","last_active":"2024-04-25T08:32:15.017604+0000","last_peered":"2024-04-25T08:32:15.017604+0000","last_clean":"2024-04-25T08:32:15.017604+0000","last_became_active":"2024-04-25T08:32:15.010727+0000","last_became_peered":"2024-04-25T08:32:15.010727+0000","last_unstale":"2024-04-25T08:32:15.017604+0000","last_undegraded":"2024-04-25T08:32:15.017604+0000","last_fullsized":"2024-04-25T08:32:15.017604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.523775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.998990+0000","last_change":"2024-04-25T08:32:13.995416+0000","last_active":"2024-04-25T08:32:15.998990+0000","last_peered":"2024-04-25T08:32:15.998990+0000","last_clean":"2024-04-25T08:32:15.998990+0000","last_became_active":"2024-04-25T08:32:13.994915+0000","last_became_peered":"2024-04-25T08:32:13.994915+0000","last_unstale":"2024-04-25T08:32:15.998990+0000","last_undegraded":"2024-04-25T08:32:15.998990+0000","last_fullsized":"2024-04-25T08:32:15.998990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:32.345591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996916+0000","last_change":"2024-04-25T08:32:15.017295+0000","last_active":"2024-04-25T08:32:15.996916+0000","last_peered":"2024-04-25T08:32:15.996916+0000","last_clean":"2024-04-25T08:32:15.996916+0000","last_became_active":"2024-04-25T08:32:15.016971+0000","last_became_peered":"2024-04-25T08:32:15.016971+0000","last_unstale":"2024-04-25T08:32:15.996916+0000","last_undegraded":"2024-04-25T08:32:15.996916+0000","last_fullsized":"2024-04-25T08:32:15.996916+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.993046+0000","last_change":"2024-04-25T08:32:13.994678+0000","last_active":"2024-04-25T08:32:14.993046+0000","last_peered":"2024-04-25T08:32:14.993046+0000","last_clean":"2024-04-25T08:32:14.993046+0000","last_became_active":"2024-04-25T08:32:13.994295+0000","last_became_peered":"2024-04-25T08:32:13.994295+0000","last_unstale":"2024-04-25T08:32:14.993046+0000","last_undegraded":"2024-04-25T08:32:14.993046+0000","last_fullsized":"2024-04-25T08:32:14.993046+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:39.881857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.000546+0000","last_change":"2024-04-25T08:32:14.999446+0000","last_active":"2024-04-25T08:32:15.000546+0000","last_peered":"2024-04-25T08:32:15.000546+0000","last_clean":"2024-04-25T08:32:15.000546+0000","last_became_active":"2024-04-25T08:32:14.999128+0000","last_became_peered":"2024-04-25T08:32:14.999128+0000","last_unstale":"2024-04-25T08:32:15.000546+0000","last_undegraded":"2024-04-25T08:32:15.000546+0000","last_fullsized":"2024-04-25T08:32:15.000546+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:22.112330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997026+0000","last_change":"2024-04-25T08:32:13.993849+0000","last_active":"2024-04-25T08:32:15.997026+0000","last_peered":"2024-04-25T08:32:15.997026+0000","last_clean":"2024-04-25T08:32:15.997026+0000","last_became_active":"2024-04-25T08:32:13.993579+0000","last_became_peered":"2024-04-25T08:32:13.993579+0000","last_unstale":"2024-04-25T08:32:15.997026+0000","last_undegraded":"2024-04-25T08:32:15.997026+0000","last_fullsized":"2024-04-25T08:32:15.997026+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.106774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.998875+0000","last_change":"2024-04-25T08:32:14.995372+0000","last_active":"2024-04-25T08:32:14.998875+0000","last_peered":"2024-04-25T08:32:14.998875+0000","last_clean":"2024-04-25T08:32:14.998875+0000","last_became_active":"2024-04-25T08:32:14.995091+0000","last_became_peered":"2024-04-25T08:32:14.995091+0000","last_unstale":"2024-04-25T08:32:14.998875+0000","last_undegraded":"2024-04-25T08:32:14.998875+0000","last_fullsized":"2024-04-25T08:32:14.998875+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992316+0000","last_change":"2024-04-25T08:32:13.997478+0000","last_active":"2024-04-25T08:32:14.992316+0000","last_peered":"2024-04-25T08:32:14.992316+0000","last_clean":"2024-04-25T08:32:14.992316+0000","last_became_active":"2024-04-25T08:32:13.997221+0000","last_became_peered":"2024-04-25T08:32:13.997221+0000","last_unstale":"2024-04-25T08:32:14.992316+0000","last_undegraded":"2024-04-25T08:32:14.992316+0000","last_fullsized":"2024-04-25T08:32:14.992316+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:32.345591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010613+0000","last_change":"2024-04-25T08:32:14.998600+0000","last_active":"2024-04-25T08:32:15.010613+0000","last_peered":"2024-04-25T08:32:15.010613+0000","last_clean":"2024-04-25T08:32:15.010613+0000","last_became_active":"2024-04-25T08:32:14.998292+0000","last_became_peered":"2024-04-25T08:32:14.998292+0000","last_unstale":"2024-04-25T08:32:15.010613+0000","last_undegraded":"2024-04-25T08:32:15.010613+0000","last_fullsized":"2024-04-25T08:32:15.010613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:31.455212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997732+0000","last_change":"2024-04-25T08:32:13.999888+0000","last_active":"2024-04-25T08:32:15.997732+0000","last_peered":"2024-04-25T08:32:15.997732+0000","last_clean":"2024-04-25T08:32:15.997732+0000","last_became_active":"2024-04-25T08:32:13.999679+0000","last_became_peered":"2024-04-25T08:32:13.999679+0000","last_unstale":"2024-04-25T08:32:15.997732+0000","last_undegraded":"2024-04-25T08:32:15.997732+0000","last_fullsized":"2024-04-25T08:32:15.997732+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:30.453207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.008928+0000","last_change":"2024-04-25T08:32:15.008169+0000","last_active":"2024-04-25T08:32:15.008928+0000","last_peered":"2024-04-25T08:32:15.008928+0000","last_clean":"2024-04-25T08:32:15.008928+0000","last_became_active":"2024-04-25T08:32:15.007807+0000","last_became_peered":"2024-04-25T08:32:15.007807+0000","last_unstale":"2024-04-25T08:32:15.008928+0000","last_undegraded":"2024-04-25T08:32:15.008928+0000","last_fullsized":"2024-04-25T08:32:15.008928+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:12.242989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.021151+0000","last_change":"2024-04-25T08:32:14.001704+0000","last_active":"2024-04-25T08:32:15.021151+0000","last_peered":"2024-04-25T08:32:15.021151+0000","last_clean":"2024-04-25T08:32:15.021151+0000","last_became_active":"2024-04-25T08:32:14.001419+0000","last_became_peered":"2024-04-25T08:32:14.001419+0000","last_unstale":"2024-04-25T08:32:15.021151+0000","last_undegraded":"2024-04-25T08:32:15.021151+0000","last_fullsized":"2024-04-25T08:32:15.021151+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:52.307141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.002162+0000","last_change":"2024-04-25T08:32:15.001570+0000","last_active":"2024-04-25T08:32:15.002162+0000","last_peered":"2024-04-25T08:32:15.002162+0000","last_clean":"2024-04-25T08:32:15.002162+0000","last_became_active":"2024-04-25T08:32:15.001161+0000","last_became_peered":"2024-04-25T08:32:15.001161+0000","last_unstale":"2024-04-25T08:32:15.002162+0000","last_undegraded":"2024-04-25T08:32:15.002162+0000","last_fullsized":"2024-04-25T08:32:15.002162+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:53.523775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.010844+0000","last_change":"2024-04-25T08:32:13.999313+0000","last_active":"2024-04-25T08:32:15.010844+0000","last_peered":"2024-04-25T08:32:15.010844+0000","last_clean":"2024-04-25T08:32:15.010844+0000","last_became_active":"2024-04-25T08:32:13.998989+0000","last_became_peered":"2024-04-25T08:32:13.998989+0000","last_unstale":"2024-04-25T08:32:15.010844+0000","last_undegraded":"2024-04-25T08:32:15.010844+0000","last_fullsized":"2024-04-25T08:32:15.010844+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:39.004812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019600+0000","last_change":"2024-04-25T08:32:15.015741+0000","last_active":"2024-04-25T08:32:15.019600+0000","last_peered":"2024-04-25T08:32:15.019600+0000","last_clean":"2024-04-25T08:32:15.019600+0000","last_became_active":"2024-04-25T08:32:15.015478+0000","last_became_peered":"2024-04-25T08:32:15.015478+0000","last_unstale":"2024-04-25T08:32:15.019600+0000","last_undegraded":"2024-04-25T08:32:15.019600+0000","last_fullsized":"2024-04-25T08:32:15.019600+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:32.556937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999154+0000","last_change":"2024-04-25T08:32:13.992863+0000","last_active":"2024-04-25T08:32:15.999154+0000","last_peered":"2024-04-25T08:32:15.999154+0000","last_clean":"2024-04-25T08:32:15.999154+0000","last_became_active":"2024-04-25T08:32:13.992571+0000","last_became_peered":"2024-04-25T08:32:13.992571+0000","last_unstale":"2024-04-25T08:32:15.999154+0000","last_undegraded":"2024-04-25T08:32:15.999154+0000","last_fullsized":"2024-04-25T08:32:15.999154+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:29.994423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.017342+0000","last_change":"2024-04-25T08:32:15.011383+0000","last_active":"2024-04-25T08:32:15.017342+0000","last_peered":"2024-04-25T08:32:15.017342+0000","last_clean":"2024-04-25T08:32:15.017342+0000","last_became_active":"2024-04-25T08:32:15.010350+0000","last_became_peered":"2024-04-25T08:32:15.010350+0000","last_unstale":"2024-04-25T08:32:15.017342+0000","last_undegraded":"2024-04-25T08:32:15.017342+0000","last_fullsized":"2024-04-25T08:32:15.017342+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.498478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997191+0000","last_change":"2024-04-25T08:32:13.994543+0000","last_active":"2024-04-25T08:32:15.997191+0000","last_peered":"2024-04-25T08:32:15.997191+0000","last_clean":"2024-04-25T08:32:15.997191+0000","last_became_active":"2024-04-25T08:32:13.994262+0000","last_became_peered":"2024-04-25T08:32:13.994262+0000","last_unstale":"2024-04-25T08:32:15.997191+0000","last_undegraded":"2024-04-25T08:32:15.997191+0000","last_fullsized":"2024-04-25T08:32:15.997191+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.463887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.020110+0000","last_change":"2024-04-25T08:32:15.016936+0000","last_active":"2024-04-25T08:32:15.020110+0000","last_peered":"2024-04-25T08:32:15.020110+0000","last_clean":"2024-04-25T08:32:15.020110+0000","last_became_active":"2024-04-25T08:32:15.016300+0000","last_became_peered":"2024-04-25T08:32:15.016300+0000","last_unstale":"2024-04-25T08:32:15.020110+0000","last_undegraded":"2024-04-25T08:32:15.020110+0000","last_fullsized":"2024-04-25T08:32:15.020110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:10.345756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019921+0000","last_change":"2024-04-25T08:32:13.999954+0000","last_active":"2024-04-25T08:32:15.019921+0000","last_peered":"2024-04-25T08:32:15.019921+0000","last_clean":"2024-04-25T08:32:15.019921+0000","last_became_active":"2024-04-25T08:32:13.999683+0000","last_became_peered":"2024-04-25T08:32:13.999683+0000","last_unstale":"2024-04-25T08:32:15.019921+0000","last_undegraded":"2024-04-25T08:32:15.019921+0000","last_fullsized":"2024-04-25T08:32:15.019921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:41.143867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.003419+0000","last_change":"2024-04-25T08:32:14.996914+0000","last_active":"2024-04-25T08:32:15.003419+0000","last_peered":"2024-04-25T08:32:15.003419+0000","last_clean":"2024-04-25T08:32:15.003419+0000","last_became_active":"2024-04-25T08:32:14.996613+0000","last_became_peered":"2024-04-25T08:32:14.996613+0000","last_unstale":"2024-04-25T08:32:15.003419+0000","last_undegraded":"2024-04-25T08:32:15.003419+0000","last_fullsized":"2024-04-25T08:32:15.003419+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:32.556937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.015428+0000","last_change":"2024-04-25T08:32:14.002719+0000","last_active":"2024-04-25T08:32:15.015428+0000","last_peered":"2024-04-25T08:32:15.015428+0000","last_clean":"2024-04-25T08:32:15.015428+0000","last_became_active":"2024-04-25T08:32:14.002413+0000","last_became_peered":"2024-04-25T08:32:14.002413+0000","last_unstale":"2024-04-25T08:32:15.015428+0000","last_undegraded":"2024-04-25T08:32:15.015428+0000","last_fullsized":"2024-04-25T08:32:15.015428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:31.463887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.016641+0000","last_change":"2024-04-25T08:32:15.015333+0000","last_active":"2024-04-25T08:32:15.016641+0000","last_peered":"2024-04-25T08:32:15.016641+0000","last_clean":"2024-04-25T08:32:15.016641+0000","last_became_active":"2024-04-25T08:32:15.014936+0000","last_became_peered":"2024-04-25T08:32:15.014936+0000","last_unstale":"2024-04-25T08:32:15.016641+0000","last_undegraded":"2024-04-25T08:32:15.016641+0000","last_fullsized":"2024-04-25T08:32:15.016641+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:10.108779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.011344+0000","last_change":"2024-04-25T08:32:14.000838+0000","last_active":"2024-04-25T08:32:15.011344+0000","last_peered":"2024-04-25T08:32:15.011344+0000","last_clean":"2024-04-25T08:32:15.011344+0000","last_became_active":"2024-04-25T08:32:14.000552+0000","last_became_peered":"2024-04-25T08:32:14.000552+0000","last_unstale":"2024-04-25T08:32:15.011344+0000","last_undegraded":"2024-04-25T08:32:15.011344+0000","last_fullsized":"2024-04-25T08:32:15.011344+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:05.432255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.012055+0000","last_change":"2024-04-25T08:32:15.011514+0000","last_active":"2024-04-25T08:32:15.012055+0000","last_peered":"2024-04-25T08:32:15.012055+0000","last_clean":"2024-04-25T08:32:15.012055+0000","last_became_active":"2024-04-25T08:32:15.011210+0000","last_became_peered":"2024-04-25T08:32:15.011210+0000","last_unstale":"2024-04-25T08:32:15.012055+0000","last_undegraded":"2024-04-25T08:32:15.012055+0000","last_fullsized":"2024-04-25T08:32:15.012055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:40.883862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999177+0000","last_change":"2024-04-25T08:32:14.000451+0000","last_active":"2024-04-25T08:32:15.999177+0000","last_peered":"2024-04-25T08:32:15.999177+0000","last_clean":"2024-04-25T08:32:15.999177+0000","last_became_active":"2024-04-25T08:32:14.000266+0000","last_became_peered":"2024-04-25T08:32:14.000266+0000","last_unstale":"2024-04-25T08:32:15.999177+0000","last_undegraded":"2024-04-25T08:32:15.999177+0000","last_fullsized":"2024-04-25T08:32:15.999177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:21.110324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992314+0000","last_change":"2024-04-25T08:32:14.990582+0000","last_active":"2024-04-25T08:32:14.992314+0000","last_peered":"2024-04-25T08:32:14.992314+0000","last_clean":"2024-04-25T08:32:14.992314+0000","last_became_active":"2024-04-25T08:32:14.990316+0000","last_became_peered":"2024-04-25T08:32:14.990316+0000","last_unstale":"2024-04-25T08:32:14.992314+0000","last_undegraded":"2024-04-25T08:32:14.992314+0000","last_fullsized":"2024-04-25T08:32:14.992314+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:22.112330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999388+0000","last_change":"2024-04-25T08:32:13.995332+0000","last_active":"2024-04-25T08:32:15.999388+0000","last_peered":"2024-04-25T08:32:15.999388+0000","last_clean":"2024-04-25T08:32:15.999388+0000","last_became_active":"2024-04-25T08:32:13.995053+0000","last_became_peered":"2024-04-25T08:32:13.995053+0000","last_unstale":"2024-04-25T08:32:15.999388+0000","last_undegraded":"2024-04-25T08:32:15.999388+0000","last_fullsized":"2024-04-25T08:32:15.999388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:32.870449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.998837+0000","last_change":"2024-04-25T08:32:14.995212+0000","last_active":"2024-04-25T08:32:14.998837+0000","last_peered":"2024-04-25T08:32:14.998837+0000","last_clean":"2024-04-25T08:32:14.998837+0000","last_became_active":"2024-04-25T08:32:14.994919+0000","last_became_peered":"2024-04-25T08:32:14.994919+0000","last_unstale":"2024-04-25T08:32:14.998837+0000","last_undegraded":"2024-04-25T08:32:14.998837+0000","last_fullsized":"2024-04-25T08:32:14.998837+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:32.556937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.014024+0000","last_change":"2024-04-25T08:32:13.992513+0000","last_active":"2024-04-25T08:32:15.014024+0000","last_peered":"2024-04-25T08:32:15.014024+0000","last_clean":"2024-04-25T08:32:15.014024+0000","last_became_active":"2024-04-25T08:32:13.992055+0000","last_became_peered":"2024-04-25T08:32:13.992055+0000","last_unstale":"2024-04-25T08:32:15.014024+0000","last_undegraded":"2024-04-25T08:32:15.014024+0000","last_fullsized":"2024-04-25T08:32:15.014024+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:37.885848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999209+0000","last_change":"2024-04-25T08:32:14.995057+0000","last_active":"2024-04-25T08:32:15.999209+0000","last_peered":"2024-04-25T08:32:15.999209+0000","last_clean":"2024-04-25T08:32:15.999209+0000","last_became_active":"2024-04-25T08:32:14.994709+0000","last_became_peered":"2024-04-25T08:32:14.994709+0000","last_unstale":"2024-04-25T08:32:15.999209+0000","last_undegraded":"2024-04-25T08:32:15.999209+0000","last_fullsized":"2024-04-25T08:32:15.999209+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:45.261984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992035+0000","last_change":"2024-04-25T08:32:13.998848+0000","last_active":"2024-04-25T08:32:14.992035+0000","last_peered":"2024-04-25T08:32:14.992035+0000","last_clean":"2024-04-25T08:32:14.992035+0000","last_became_active":"2024-04-25T08:32:13.998570+0000","last_became_peered":"2024-04-25T08:32:13.998570+0000","last_unstale":"2024-04-25T08:32:14.992035+0000","last_undegraded":"2024-04-25T08:32:14.992035+0000","last_fullsized":"2024-04-25T08:32:14.992035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:14.901524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013005+0000","last_change":"2024-04-25T08:32:15.009976+0000","last_active":"2024-04-25T08:32:15.013005+0000","last_peered":"2024-04-25T08:32:15.013005+0000","last_clean":"2024-04-25T08:32:15.013005+0000","last_became_active":"2024-04-25T08:32:15.009373+0000","last_became_peered":"2024-04-25T08:32:15.009373+0000","last_unstale":"2024-04-25T08:32:15.013005+0000","last_undegraded":"2024-04-25T08:32:15.013005+0000","last_fullsized":"2024-04-25T08:32:15.013005+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:43.786558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.994276+0000","last_change":"2024-04-25T08:32:14.001519+0000","last_active":"2024-04-25T08:32:14.994276+0000","last_peered":"2024-04-25T08:32:14.994276+0000","last_clean":"2024-04-25T08:32:14.994276+0000","last_became_active":"2024-04-25T08:32:14.001174+0000","last_became_peered":"2024-04-25T08:32:14.001174+0000","last_unstale":"2024-04-25T08:32:14.994276+0000","last_undegraded":"2024-04-25T08:32:14.994276+0000","last_fullsized":"2024-04-25T08:32:14.994276+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:27.832466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.997374+0000","last_change":"2024-04-25T08:32:14.996743+0000","last_active":"2024-04-25T08:32:14.997374+0000","last_peered":"2024-04-25T08:32:14.997374+0000","last_clean":"2024-04-25T08:32:14.997374+0000","last_became_active":"2024-04-25T08:32:14.996453+0000","last_became_peered":"2024-04-25T08:32:14.996453+0000","last_unstale":"2024-04-25T08:32:14.997374+0000","last_undegraded":"2024-04-25T08:32:14.997374+0000","last_fullsized":"2024-04-25T08:32:14.997374+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:40.883862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992598+0000","last_change":"2024-04-25T08:32:13.998203+0000","last_active":"2024-04-25T08:32:14.992598+0000","last_peered":"2024-04-25T08:32:14.992598+0000","last_clean":"2024-04-25T08:32:14.992598+0000","last_became_active":"2024-04-25T08:32:13.997887+0000","last_became_peered":"2024-04-25T08:32:13.997887+0000","last_unstale":"2024-04-25T08:32:14.992598+0000","last_undegraded":"2024-04-25T08:32:14.992598+0000","last_fullsized":"2024-04-25T08:32:14.992598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:29.631313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.999795+0000","last_change":"2024-04-25T08:32:14.998485+0000","last_active":"2024-04-25T08:32:14.999795+0000","last_peered":"2024-04-25T08:32:14.999795+0000","last_clean":"2024-04-25T08:32:14.999795+0000","last_became_active":"2024-04-25T08:32:14.997709+0000","last_became_peered":"2024-04-25T08:32:14.997709+0000","last_unstale":"2024-04-25T08:32:14.999795+0000","last_undegraded":"2024-04-25T08:32:14.999795+0000","last_fullsized":"2024-04-25T08:32:14.999795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999359+0000","last_change":"2024-04-25T08:32:13.994688+0000","last_active":"2024-04-25T08:32:15.999359+0000","last_peered":"2024-04-25T08:32:15.999359+0000","last_clean":"2024-04-25T08:32:15.999359+0000","last_became_active":"2024-04-25T08:32:13.994430+0000","last_became_peered":"2024-04-25T08:32:13.994430+0000","last_unstale":"2024-04-25T08:32:15.999359+0000","last_undegraded":"2024-04-25T08:32:15.999359+0000","last_fullsized":"2024-04-25T08:32:15.999359+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:11.240984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013913+0000","last_change":"2024-04-25T08:32:14.999331+0000","last_active":"2024-04-25T08:32:15.013913+0000","last_peered":"2024-04-25T08:32:15.013913+0000","last_clean":"2024-04-25T08:32:15.013913+0000","last_became_active":"2024-04-25T08:32:14.999030+0000","last_became_peered":"2024-04-25T08:32:14.999030+0000","last_unstale":"2024-04-25T08:32:15.013913+0000","last_undegraded":"2024-04-25T08:32:15.013913+0000","last_fullsized":"2024-04-25T08:32:15.013913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:53.450645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.997733+0000","last_change":"2024-04-25T08:32:14.001260+0000","last_active":"2024-04-25T08:32:15.997733+0000","last_peered":"2024-04-25T08:32:15.997733+0000","last_clean":"2024-04-25T08:32:15.997733+0000","last_became_active":"2024-04-25T08:32:14.000935+0000","last_became_peered":"2024-04-25T08:32:14.000935+0000","last_unstale":"2024-04-25T08:32:15.997733+0000","last_undegraded":"2024-04-25T08:32:15.997733+0000","last_fullsized":"2024-04-25T08:32:15.997733+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:44.259979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004890+0000","last_change":"2024-04-25T08:32:14.997044+0000","last_active":"2024-04-25T08:32:15.004890+0000","last_peered":"2024-04-25T08:32:15.004890+0000","last_clean":"2024-04-25T08:32:15.004890+0000","last_became_active":"2024-04-25T08:32:14.996712+0000","last_became_peered":"2024-04-25T08:32:14.996712+0000","last_unstale":"2024-04-25T08:32:15.004890+0000","last_undegraded":"2024-04-25T08:32:15.004890+0000","last_fullsized":"2024-04-25T08:32:15.004890+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:57.648330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.994876+0000","last_change":"2024-04-25T08:32:13.994985+0000","last_active":"2024-04-25T08:32:14.994876+0000","last_peered":"2024-04-25T08:32:14.994876+0000","last_clean":"2024-04-25T08:32:14.994876+0000","last_became_active":"2024-04-25T08:32:13.994706+0000","last_became_peered":"2024-04-25T08:32:13.994706+0000","last_unstale":"2024-04-25T08:32:14.994876+0000","last_undegraded":"2024-04-25T08:32:14.994876+0000","last_fullsized":"2024-04-25T08:32:14.994876+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.106774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.019581+0000","last_change":"2024-04-25T08:32:15.013896+0000","last_active":"2024-04-25T08:32:15.019581+0000","last_peered":"2024-04-25T08:32:15.019581+0000","last_clean":"2024-04-25T08:32:15.019581+0000","last_became_active":"2024-04-25T08:32:15.013638+0000","last_became_peered":"2024-04-25T08:32:15.013638+0000","last_unstale":"2024-04-25T08:32:15.019581+0000","last_undegraded":"2024-04-25T08:32:15.019581+0000","last_fullsized":"2024-04-25T08:32:15.019581+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:15.903529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991867+0000","last_change":"2024-04-25T08:32:13.999061+0000","last_active":"2024-04-25T08:32:14.991867+0000","last_peered":"2024-04-25T08:32:14.991867+0000","last_clean":"2024-04-25T08:32:14.991867+0000","last_became_active":"2024-04-25T08:32:13.998788+0000","last_became_peered":"2024-04-25T08:32:13.998788+0000","last_unstale":"2024-04-25T08:32:14.991867+0000","last_undegraded":"2024-04-25T08:32:14.991867+0000","last_fullsized":"2024-04-25T08:32:14.991867+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:11.989300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004367+0000","last_change":"2024-04-25T08:32:15.002659+0000","last_active":"2024-04-25T08:32:15.004367+0000","last_peered":"2024-04-25T08:32:15.004367+0000","last_clean":"2024-04-25T08:32:15.004367+0000","last_became_active":"2024-04-25T08:32:15.002396+0000","last_became_peered":"2024-04-25T08:32:15.002396+0000","last_unstale":"2024-04-25T08:32:15.004367+0000","last_undegraded":"2024-04-25T08:32:15.004367+0000","last_fullsized":"2024-04-25T08:32:15.004367+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:06.013735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.992201+0000","last_change":"2024-04-25T08:32:13.997172+0000","last_active":"2024-04-25T08:32:14.992201+0000","last_peered":"2024-04-25T08:32:14.992201+0000","last_clean":"2024-04-25T08:32:14.992201+0000","last_became_active":"2024-04-25T08:32:13.996883+0000","last_became_peered":"2024-04-25T08:32:13.996883+0000","last_unstale":"2024-04-25T08:32:14.992201+0000","last_undegraded":"2024-04-25T08:32:14.992201+0000","last_fullsized":"2024-04-25T08:32:14.992201+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:30.733354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.013029+0000","last_change":"2024-04-25T08:32:15.010218+0000","last_active":"2024-04-25T08:32:15.013029+0000","last_peered":"2024-04-25T08:32:15.013029+0000","last_clean":"2024-04-25T08:32:15.013029+0000","last_became_active":"2024-04-25T08:32:15.009696+0000","last_became_peered":"2024-04-25T08:32:15.009696+0000","last_unstale":"2024-04-25T08:32:15.013029+0000","last_undegraded":"2024-04-25T08:32:15.013029+0000","last_fullsized":"2024-04-25T08:32:15.013029+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:48.073422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991437+0000","last_change":"2024-04-25T08:32:13.996157+0000","last_active":"2024-04-25T08:32:14.991437+0000","last_peered":"2024-04-25T08:32:14.991437+0000","last_clean":"2024-04-25T08:32:14.991437+0000","last_became_active":"2024-04-25T08:32:13.995870+0000","last_became_peered":"2024-04-25T08:32:13.995870+0000","last_unstale":"2024-04-25T08:32:14.991437+0000","last_undegraded":"2024-04-25T08:32:14.991437+0000","last_fullsized":"2024-04-25T08:32:14.991437+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:01.905014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.004825+0000","last_change":"2024-04-25T08:32:15.000611+0000","last_active":"2024-04-25T08:32:15.004825+0000","last_peered":"2024-04-25T08:32:15.004825+0000","last_clean":"2024-04-25T08:32:15.004825+0000","last_became_active":"2024-04-25T08:32:14.999876+0000","last_became_peered":"2024-04-25T08:32:14.999876+0000","last_unstale":"2024-04-25T08:32:15.004825+0000","last_undegraded":"2024-04-25T08:32:15.004825+0000","last_fullsized":"2024-04-25T08:32:15.004825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:31.522085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.986377+0000","last_change":"2024-04-25T08:32:13.990048+0000","last_active":"2024-04-25T08:32:14.986377+0000","last_peered":"2024-04-25T08:32:14.986377+0000","last_clean":"2024-04-25T08:32:14.986377+0000","last_became_active":"2024-04-25T08:32:13.989722+0000","last_became_peered":"2024-04-25T08:32:13.989722+0000","last_unstale":"2024-04-25T08:32:14.986377+0000","last_undegraded":"2024-04-25T08:32:14.986377+0000","last_fullsized":"2024-04-25T08:32:14.986377+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:33:52.307141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.999166+0000","last_change":"2024-04-25T08:32:14.992261+0000","last_active":"2024-04-25T08:32:15.999166+0000","last_peered":"2024-04-25T08:32:15.999166+0000","last_clean":"2024-04-25T08:32:15.999166+0000","last_became_active":"2024-04-25T08:32:14.991947+0000","last_became_peered":"2024-04-25T08:32:14.991947+0000","last_unstale":"2024-04-25T08:32:15.999166+0000","last_undegraded":"2024-04-25T08:32:15.999166+0000","last_fullsized":"2024-04-25T08:32:15.999166+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:13.975185+0000","last_clean_scrub_stamp":"2024-04-25T08:32:13.975185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:31.455212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:32:14.991104+0000","last_change":"2024-04-25T08:32:13.996123+0000","last_active":"2024-04-25T08:32:14.991104+0000","last_peered":"2024-04-25T08:32:14.991104+0000","last_clean":"2024-04-25T08:32:14.991104+0000","last_became_active":"2024-04-25T08:32:13.995711+0000","last_became_peered":"2024-04-25T08:32:13.995711+0000","last_unstale":"2024-04-25T08:32:14.991104+0000","last_undegraded":"2024-04-25T08:32:14.991104+0000","last_fullsized":"2024-04-25T08:32:14.991104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:12.973179+0000","last_clean_scrub_stamp":"2024-04-25T08:32:12.973179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:49.338054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:32:15.996592+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:32:15.996592+0000","last_peered":"2024-04-25T08:32:15.996592+0000","last_clean":"2024-04-25T08:32:15.996592+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:32:15.996592+0000","last_undegraded":"2024-04-25T08:32:15.996592+0000","last_fullsized":"2024-04-25T08:32:15.996592+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36556,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335284,"statfs":{"total":96636764160,"available":96599330816,"internally_reserved":0,"allocated":1241088,"data_stored":876841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35988,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335852,"statfs":{"total":96636764160,"available":96599912448,"internally_reserved":0,"allocated":659456,"data_stored":293693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":286349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35956,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335884,"statfs":{"total":96636764160,"available":96599945216,"internally_reserved":0,"allocated":626688,"data_stored":279182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":655360,"data_stored":287983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40112,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331728,"statfs":{"total":96636764160,"available":96595689472,"internally_reserved":0,"allocated":626688,"data_stored":279182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35956,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335884,"statfs":{"total":96636764160,"available":96599945216,"internally_reserved":0,"allocated":626688,"data_stored":279182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40696,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331144,"statfs":{"total":96636764160,"available":96595091456,"internally_reserved":0,"allocated":1220608,"data_stored":869550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:32:18.339 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:32:18.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:32:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.488+0000 7faeed8ea640 1 Processor -- start 2024-04-25T08:32:18.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.488+0000 7faeed8ea640 1 -- start start 2024-04-25T08:32:18.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faeed8ea640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faee80d34c0 0x7faee80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faeed8ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee80d08e0 0x7faee80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faeed8ea640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80d2420 con 0x7faee815dac0 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faeed8ea640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee80d3030 con 0x7faee80d34c0 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faeed8ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80d0d20 con 0x7faee80d08e0 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faee80d34c0 0x7faee80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faee80d34c0 0x7faee80d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44540/0 (socket says 172.21.15.60:44540) 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 -- 172.21.15.60:0/2564205470 learned_addr learned my addr 172.21.15.60:0/2564205470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faee77fe640 1 --2- 172.21.15.60:0/2564205470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee80d08e0 0x7faee80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 -- 172.21.15.60:0/2564205470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee80d08e0 msgr2=0x7faee80b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 --2- 172.21.15.60:0/2564205470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee80d08e0 0x7faee80b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 -- 172.21.15.60:0/2564205470 >> v1:172.21.15.100:6789/0 conn(0x7faee815dac0 legacy=0x7faee80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.489+0000 7faedffff640 1 -- 172.21.15.60:0/2564205470 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80d2700 con 0x7faee80d34c0 2024-04-25T08:32:18.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.490+0000 7faedffff640 1 --2- 172.21.15.60:0/2564205470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d34c0 0x7faee80d03a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7faed0009850 tx=0x7faed002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50eb3c0dc3f864b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.490+0000 7faee67fc640 1 -- 172.21.15.60:0/2564205470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0004230 con 0x7faee80d34c0 2024-04-25T08:32:18.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.490+0000 7faee67fc640 1 -- 172.21.15.60:0/2564205470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00043d0 con 0x7faee80d34c0 2024-04-25T08:32:18.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.490+0000 7faee67fc640 1 -- 172.21.15.60:0/2564205470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0003810 con 0x7faee80d34c0 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 -- 172.21.15.60:0/2564205470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d34c0 msgr2=0x7faee80d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 --2- 172.21.15.60:0/2564205470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d34c0 0x7faee80d03a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7faed0009850 tx=0x7faed002a2c0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 -- 172.21.15.60:0/2564205470 shutdown_connections 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 --2- 172.21.15.60:0/2564205470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d34c0 0x7faee80d03a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 --2- 172.21.15.60:0/2564205470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee80d08e0 0x7faee80b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 -- 172.21.15.60:0/2564205470 >> 172.21.15.60:0/2564205470 conn(0x7faee8061c30 msgr2=0x7faee806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 -- 172.21.15.60:0/2564205470 shutdown_connections 2024-04-25T08:32:18.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.491+0000 7faeed8ea640 1 -- 172.21.15.60:0/2564205470 wait complete. 2024-04-25T08:32:18.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.492+0000 7faeed8ea640 1 Processor -- start 2024-04-25T08:32:18.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.492+0000 7faeed8ea640 1 -- start start 2024-04-25T08:32:18.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 0x7faee8152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee80d34c0 0x7faee80cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee815dac0 0x7faee80c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80d2100 con 0x7faee80d08e0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80d2cc0 con 0x7faee80d34c0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee815dea0 con 0x7faee815dac0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 0x7faee8152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 0x7faee8152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44550/0 (socket says 172.21.15.60:44550) 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 -- 172.21.15.60:0/4293972530 learned_addr learned my addr 172.21.15.60:0/4293972530 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faedffff640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee80d34c0 0x7faee80cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee77fe640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee815dac0 0x7faee80c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 -- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee815dac0 msgr2=0x7faee80c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee815dac0 0x7faee80c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 -- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee80d34c0 msgr2=0x7faee80cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee80d34c0 0x7faee80cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 -- 172.21.15.60:0/4293972530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80cd730 con 0x7faee80d08e0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee77fe640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee815dac0 0x7faee80c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee6ffd640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 0x7faee8152570 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faed800c3c0 tx=0x7faed800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80052d0 con 0x7faee80d08e0 2024-04-25T08:32:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8005490 con 0x7faee80d08e0 2024-04-25T08:32:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee81f73d0 con 0x7faee80d08e0 2024-04-25T08:32:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8007260 con 0x7faee80d08e0 2024-04-25T08:32:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.493+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee81f7990 con 0x7faee80d08e0 2024-04-25T08:32:18.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.494+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee8073a00 con 0x7faee80d08e0 2024-04-25T08:32:18.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.495+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faed8007400 con 0x7faee80d08e0 2024-04-25T08:32:18.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.495+0000 7faee4ff9640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.495+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faed80897c0 con 0x7faee80d08e0 2024-04-25T08:32:18.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.496+0000 7faedffff640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.496+0000 7faedffff640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faed0033000 tx=0x7faed002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.500+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faed8057e60 con 0x7faee80d08e0 2024-04-25T08:32:18.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.748+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faee80b7150 con 0x7faee80d08e0 2024-04-25T08:32:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.750+0000 7faee4ff9640 1 -- 172.21.15.60:0/4293972530 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7faed805ce20 con 0x7faee80d08e0 2024-04-25T08:32:18.747 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:32:18.747 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:32:18.747 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:32:18.747 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:32:18.747 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:32:18.748 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faec806b4d0 msgr2=0x7faec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faed0033000 tx=0x7faed002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 msgr2=0x7faee8152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 0x7faee8152570 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7faed800c3c0 tx=0x7faed800c880 comp rx=0 tx=0).stop 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 shutdown_connections 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faee815dac0 0x7faee80c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faee80d34c0 0x7faee80cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 --2- 172.21.15.60:0/4293972530 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faee80d08e0 0x7faee8152570 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.753+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 >> 172.21.15.60:0/4293972530 conn(0x7faee8061c30 msgr2=0x7faee815edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.754+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 shutdown_connections 2024-04-25T08:32:18.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.754+0000 7faeed8ea640 1 -- 172.21.15.60:0/4293972530 wait complete. 2024-04-25T08:32:18.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:18.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.865+0000 7f692857e640 1 Processor -- start 2024-04-25T08:32:18.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.865+0000 7f692857e640 1 -- start start 2024-04-25T08:32:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692857e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f692015db70 0x7f6920162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692857e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6920162690 con 0x7f69200b5c30 2024-04-25T08:32:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692857e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200ceee0 con 0x7f69200b6cb0 2024-04-25T08:32:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692857e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69200d1d10 con 0x7f692015db70 2024-04-25T08:32:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692631a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f692015db70 0x7f6920162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f6926b1b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f69200b5c30 0x7f69200d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58796/0 (socket says 172.21.15.60:58796) 2024-04-25T08:32:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f6926b1b640 1 -- 172.21.15.60:0/2411139197 learned_addr learned my addr 172.21.15.60:0/2411139197 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692631a640 1 -- 172.21.15.60:0/2411139197 >> v1:172.21.15.100:6789/0 conn(0x7f69200b6cb0 legacy=0x7f69200b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692631a640 1 -- 172.21.15.60:0/2411139197 >> v1:172.21.15.60:6789/0 conn(0x7f69200b5c30 legacy=0x7f69200d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.866+0000 7f692631a640 1 -- 172.21.15.60:0/2411139197 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200d1ff0 con 0x7f692015db70 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.867+0000 7f692631a640 1 --2- 172.21.15.60:0/2411139197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f6920162150 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f691c0097a0 tx=0x7f691c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcfa53373908443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.867+0000 7f6925318640 1 -- 172.21.15.60:0/2411139197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c037070 con 0x7f692015db70 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.867+0000 7f6925318640 1 -- 172.21.15.60:0/2411139197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c032440 con 0x7f692015db70 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.867+0000 7f6925318640 1 -- 172.21.15.60:0/2411139197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c032810 con 0x7f692015db70 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.867+0000 7f692857e640 1 -- 172.21.15.60:0/2411139197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 msgr2=0x7f6920162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.867+0000 7f692857e640 1 --2- 172.21.15.60:0/2411139197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f6920162150 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f691c0097a0 tx=0x7f691c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.868+0000 7f692857e640 1 -- 172.21.15.60:0/2411139197 shutdown_connections 2024-04-25T08:32:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.868+0000 7f692857e640 1 --2- 172.21.15.60:0/2411139197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f6920162150 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.868+0000 7f692857e640 1 -- 172.21.15.60:0/2411139197 >> 172.21.15.60:0/2411139197 conn(0x7f69200d0520 msgr2=0x7f69200d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.868+0000 7f692857e640 1 -- 172.21.15.60:0/2411139197 shutdown_connections 2024-04-25T08:32:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.868+0000 7f692857e640 1 -- 172.21.15.60:0/2411139197 wait complete. 2024-04-25T08:32:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.869+0000 7f692857e640 1 Processor -- start 2024-04-25T08:32:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.869+0000 7f692857e640 1 -- start start 2024-04-25T08:32:18.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69200b5c30 0x7f69200ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69200b6cb0 0x7f6920158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f69200b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200ce530 con 0x7f692015db70 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692015df50 con 0x7f69200b5c30 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200cf1c0 con 0x7f69200b6cb0 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f69200b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f69200b89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44554/0 (socket says 172.21.15.60:44554) 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 -- 172.21.15.60:0/3595346511 learned_addr learned my addr 172.21.15.60:0/3595346511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692631a640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69200b5c30 0x7f69200ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6925b19640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69200b6cb0 0x7f6920158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 -- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69200b6cb0 msgr2=0x7f6920158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69200b6cb0 0x7f6920158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 -- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69200b5c30 msgr2=0x7f69200ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69200b5c30 0x7f69200ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 -- 172.21.15.60:0/3595346511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200ac690 con 0x7f692015db70 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692631a640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69200b5c30 0x7f69200ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6926b1b640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f69200b89e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f691800cb40 tx=0x7f691800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918017070 con 0x7f692015db70 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691800aca0 con 0x7f692015db70 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69200aa350 con 0x7f692015db70 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918012600 con 0x7f692015db70 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.870+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69200aabf0 con 0x7f692015db70 2024-04-25T08:32:18.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.871+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69200cf7d0 con 0x7f692015db70 2024-04-25T08:32:18.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.872+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f691801a070 con 0x7f692015db70 2024-04-25T08:32:18.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.872+0000 7f6916ffd640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f690406b4d0 0x7f690406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.873+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6918089f80 con 0x7f692015db70 2024-04-25T08:32:18.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.873+0000 7f692631a640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f690406b4d0 0x7f690406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.873+0000 7f692631a640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f690406b4d0 0x7f690406d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f691c034000 tx=0x7f691c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:18.877+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69200cf7d0 con 0x7f692015db70 2024-04-25T08:32:19.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.099+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6920073990 con 0x7f692015db70 2024-04-25T08:32:19.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.099+0000 7f6916ffd640 1 -- 172.21.15.60:0/3595346511 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6918058610 con 0x7f692015db70 2024-04-25T08:32:19.096 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:19.096 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:19.096 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.101+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f690406b4d0 msgr2=0x7f690406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.101+0000 7f692857e640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f690406b4d0 0x7f690406d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f691c034000 tx=0x7f691c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:19.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.101+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 msgr2=0x7f69200b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.101+0000 7f692857e640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f69200b89e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f691800cb40 tx=0x7f691800a060 comp rx=0 tx=0).stop 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 shutdown_connections 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f692015db70 0x7f69200b89e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69200b6cb0 0x7f6920158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f690406b4d0 0x7f690406d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 --2- 172.21.15.60:0/3595346511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69200b5c30 0x7f69200ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 >> 172.21.15.60:0/3595346511 conn(0x7f69200d0520 msgr2=0x7f692015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 shutdown_connections 2024-04-25T08:32:19.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.102+0000 7f692857e640 1 -- 172.21.15.60:0/3595346511 wait complete. 2024-04-25T08:32:19.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:32:14.355898+0000', 'modified': '2024-04-25T08:32:16.340529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5886}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5886': {'gid': 5886, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/1977192273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 1977192273}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1977192273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:32:19.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:32:19.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:32:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.625+0000 7fccea64a640 1 Processor -- start 2024-04-25T08:32:19.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.625+0000 7fccea64a640 1 -- start start 2024-04-25T08:32:19.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fccea64a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce40b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fccea64a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcce4162450 0x7fcce40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fccea64a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce40cf320 con 0x7fcce415b980 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fccea64a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce40cf040 con 0x7fcce4162450 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fccea64a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4161970 con 0x7fcce415b5a0 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcce4162450 0x7fcce40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcce4162450 0x7fcce40cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44560/0 (socket says 172.21.15.60:44560) 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 -- 172.21.15.60:0/689291165 learned_addr learned my addr 172.21.15.60:0/689291165 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce3fff640 1 --2- 172.21.15.60:0/689291165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce40b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 -- 172.21.15.60:0/689291165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 msgr2=0x7fcce40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 --2- 172.21.15.60:0/689291165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce40b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 -- 172.21.15.60:0/689291165 >> v1:172.21.15.100:6789/0 conn(0x7fcce415b980 legacy=0x7fcce40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.626+0000 7fcce8be7640 1 -- 172.21.15.60:0/689291165 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4161c50 con 0x7fcce4162450 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.627+0000 7fcce8be7640 1 --2- 172.21.15.60:0/689291165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce4162450 0x7fcce40cea20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009e50 tx=0x7fccdc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f92201c66ad465e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.627+0000 7fcce2ffd640 1 -- 172.21.15.60:0/689291165 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc036070 con 0x7fcce4162450 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.627+0000 7fcce2ffd640 1 -- 172.21.15.60:0/689291165 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc031600 con 0x7fcce4162450 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.627+0000 7fcce2ffd640 1 -- 172.21.15.60:0/689291165 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0319d0 con 0x7fcce4162450 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 -- 172.21.15.60:0/689291165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce4162450 msgr2=0x7fcce40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 --2- 172.21.15.60:0/689291165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce4162450 0x7fcce40cea20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009e50 tx=0x7fccdc02a710 comp rx=0 tx=0).stop 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 -- 172.21.15.60:0/689291165 shutdown_connections 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 --2- 172.21.15.60:0/689291165 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce4162450 0x7fcce40cea20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 --2- 172.21.15.60:0/689291165 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce40b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 -- 172.21.15.60:0/689291165 >> 172.21.15.60:0/689291165 conn(0x7fcce40d0530 msgr2=0x7fcce40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 -- 172.21.15.60:0/689291165 shutdown_connections 2024-04-25T08:32:19.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.628+0000 7fccea64a640 1 -- 172.21.15.60:0/689291165 wait complete. 2024-04-25T08:32:19.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.629+0000 7fccea64a640 1 Processor -- start 2024-04-25T08:32:19.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.629+0000 7fccea64a640 1 -- start start 2024-04-25T08:32:19.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fccea64a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fccea64a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 0x7fcce40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fccea64a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcce4162450 0x7fcce40cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fccea64a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40b5bf0 con 0x7fcce415b980 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fccea64a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40b5a70 con 0x7fcce4162450 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fccea64a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40b5d70 con 0x7fcce415b5a0 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 0x7fcce40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 0x7fcce40ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44562/0 (socket says 172.21.15.60:44562) 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 -- 172.21.15.60:0/2180913335 learned_addr learned my addr 172.21.15.60:0/2180913335 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce3fff640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce8be7640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcce4162450 0x7fcce40cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 -- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 msgr2=0x7fcce415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 -- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcce4162450 msgr2=0x7fcce40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcce4162450 0x7fcce40cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 -- 172.21.15.60:0/2180913335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4147160 con 0x7fcce415b980 2024-04-25T08:32:19.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.630+0000 7fcce37fe640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 0x7fcce40ccdc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c400 tx=0x7fccd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.631+0000 7fcce0ff9640 1 -- 172.21.15.60:0/2180913335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80052d0 con 0x7fcce415b980 2024-04-25T08:32:19.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.631+0000 7fcce0ff9640 1 -- 172.21.15.60:0/2180913335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd80054b0 con 0x7fcce415b980 2024-04-25T08:32:19.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.631+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce40cd840 con 0x7fcce415b980 2024-04-25T08:32:19.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.631+0000 7fcce0ff9640 1 -- 172.21.15.60:0/2180913335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8006a60 con 0x7fcce415b980 2024-04-25T08:32:19.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.631+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce4146680 con 0x7fcce415b980 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.632+0000 7fcce0ff9640 1 -- 172.21.15.60:0/2180913335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccd8006c00 con 0x7fcce415b980 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.633+0000 7fcce0ff9640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccd006b4d0 0x7fccd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.633+0000 7fcce0ff9640 1 -- 172.21.15.60:0/2180913335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccd80898c0 con 0x7fcce415b980 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.633+0000 7fcce3fff640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccd006b4d0 0x7fccd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 Processor -- start 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcce3fff640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccd006b4d0 0x7fccd006d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009a10 tx=0x7fccd4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 -- start start 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x560320ff9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560321017020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320fcef10 0x560321015190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560320f31a10 con 0x560321016ad0 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560320edab30 con 0x560320ff9230 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.634+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560320f2dc50 con 0x560320fcef10 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560321017020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560321017020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44576/0 (socket says 172.21.15.60:44576) 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 -- 172.21.15.60:0/1973064393 learned_addr learned my addr 172.21.15.60:0/1973064393 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce3fff640 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x560320ff9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce8be7640 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320fcef10 0x560321015190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 -- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320fcef10 msgr2=0x560321015190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320fcef10 0x560321015190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 -- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 msgr2=0x560320ff9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x560320ff9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 -- 172.21.15.60:0/1973064393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560320eefe10 con 0x560321016ad0 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce3fff640 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x560320ff9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fcce37fe640 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560321017020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fccd8001080 tx=0x7fccd8057ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedc0f96879a3bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fccc9ffb640 1 -- 172.21.15.60:0/1973064393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8056120 con 0x560321016ad0 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fccc9ffb640 1 -- 172.21.15.60:0/1973064393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8054490 con 0x560321016ad0 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.635+0000 7fccc9ffb640 1 -- 172.21.15.60:0/1973064393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8057490 con 0x560321016ad0 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 -- 172.21.15.60:0/1973064393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 msgr2=0x560321017020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560321017020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fccd8001080 tx=0x7fccd8057ee0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 -- 172.21.15.60:0/1973064393 shutdown_connections 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320fcef10 0x560321015190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560321017020 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 --2- 172.21.15.60:0/1973064393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x560320ff9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 -- 172.21.15.60:0/1973064393 >> 172.21.15.60:0/1973064393 conn(0x560320fcb840 msgr2=0x560321014040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 -- 172.21.15.60:0/1973064393 shutdown_connections 2024-04-25T08:32:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 -- 172.21.15.60:0/1973064393 wait complete. 2024-04-25T08:32:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.636+0000 7fcced428740 1 -- start start 2024-04-25T08:32:19.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 Processor -- start 2024-04-25T08:32:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x56032100d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560320ffd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 0x560320fc8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560320f31a10 con 0x560320fc84a0 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560320eee0c0 con 0x560320ff9230 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560320fcfcb0 con 0x560321016ad0 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.637+0000 7fcced428740 10 client.0 _ll_register_callbacks cb 0x560320ee1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 0x560320fc8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 0x560320fc8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44584/0 (socket says 172.21.15.60:44584) 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 -- 172.21.15.60:0/3926157262 learned_addr learned my addr 172.21.15.60:0/3926157262 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce37fe640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560320ffd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce3fff640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x56032100d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 -- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 msgr2=0x560320ffd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560320ffd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 -- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 msgr2=0x56032100d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x56032100d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 -- 172.21.15.60:0/3926157262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560320c462d0 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcce8be7640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 0x560320fc8920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009f70 tx=0x7fccdc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c527260805ca7108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fccc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fccc2ffd640 1 -- 172.21.15.60:0/3926157262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc036040 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fccc2ffd640 1 -- 172.21.15.60:0/3926157262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccdc031d10 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccd8054490 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fccc2ffd640 1 -- 172.21.15.60:0/3926157262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccdc04d050 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcced428740 10 client.6036 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.638+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560320d106b0 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fccc2ffd640 1 -- 172.21.15.60:0/3926157262 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8054490 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fccc2ffd640 1 -- 172.21.15.60:0/3926157262 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fccdc050020 con 0x560320fc84a0 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcced428740 10 client.6036 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcced428740 10 client.6036 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcced428740 20 client.6036 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcced428740 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560320ffb880 0x560320ffbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcced428740 4 client.6036 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:19.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560320f90920 con 0x560320ffb880 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcce3fff640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560320ffb880 0x560320ffbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fccc2ffd640 10 client.6036 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.639+0000 7fcce3fff640 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560320ffb880 0x560320ffbc60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9afcd9f830dd072 server_cookie=e2d3adf7e339aad8 in_seq=0 out_seq=0 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fccc2ffd640 1 -- 172.21.15.60:0/3926157262 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560320f90920 con 0x560320ffb880 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fccc2ffd640 10 client.6036 handle_command_reply: tid=0 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 client.6036 shutdown 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 msgr2=0x560320fc8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 0x560320fc8920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009f70 tx=0x7fccdc004ab0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 shutdown_connections 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320fc84a0 0x560320fc8920 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560320ffb880 0x560320ffbc60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560320ffd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 --2- 172.21.15.60:0/3926157262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320ff9230 0x56032100d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 >> 172.21.15.60:0/3926157262 conn(0x560320fcef10 msgr2=0x5603210138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fccc2ffd640 0 client.6036 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:19.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 shutdown_connections 2024-04-25T08:32:19.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 1 -- 172.21.15.60:0/3926157262 wait complete. 2024-04-25T08:32:19.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.640+0000 7fcced428740 20 client.6036 trim_cache size 0 max 16384 2024-04-25T08:32:19.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.656+0000 7fcced428740 1 Processor -- start 2024-04-25T08:32:19.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.656+0000 7fcced428740 1 -- start start 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x560320fc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320ff9230 0x56032100c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcced428740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560320fea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcced428740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4200450 con 0x560321016ad0 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41ff5f0 con 0x560320fc84a0 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41569c0 con 0x560320ff9230 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560320fea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560320fea590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44586/0 (socket says 172.21.15.60:44586) 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 -- 172.21.15.60:0/1563925214 learned_addr learned my addr 172.21.15.60:0/1563925214 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce3fff640 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x560320fc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce37fe640 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320ff9230 0x56032100c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 -- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320ff9230 msgr2=0x56032100c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320ff9230 0x56032100c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 -- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 msgr2=0x560320fc8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x560320fc8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 -- 172.21.15.60:0/1563925214 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560320ff9980 con 0x560321016ad0 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fcce8be7640 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560320fea590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0071f0 tx=0x7fccdc03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa64ae06c6542a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.657+0000 7fccc97fa640 1 -- 172.21.15.60:0/1563925214 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc056070 con 0x560321016ad0 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fccc97fa640 1 -- 172.21.15.60:0/1563925214 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc031d10 con 0x560321016ad0 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fccc97fa640 1 -- 172.21.15.60:0/1563925214 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc036040 con 0x560321016ad0 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 -- 172.21.15.60:0/1563925214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 msgr2=0x560320fea590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560320fea590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0071f0 tx=0x7fccdc03d000 comp rx=0 tx=0).stop 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 -- 172.21.15.60:0/1563925214 shutdown_connections 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560321016ad0 0x560320fea590 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560320ff9230 0x56032100c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 --2- 172.21.15.60:0/1563925214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x560320fc8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 -- 172.21.15.60:0/1563925214 >> 172.21.15.60:0/1563925214 conn(0x560320fcef10 msgr2=0x560321036bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 -- 172.21.15.60:0/1563925214 shutdown_connections 2024-04-25T08:32:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.658+0000 7fcced428740 1 -- 172.21.15.60:0/1563925214 wait complete. 2024-04-25T08:32:19.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.659+0000 7fcced428740 1 -- start start 2024-04-25T08:32:19.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.659+0000 7fcced428740 1 Processor -- start 2024-04-25T08:32:19.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x56032102d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 0x56032107e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560321009ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032100bc90 con 0x560320ff9230 2024-04-25T08:32:19.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41feff0 con 0x560320fc84a0 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56032100b0f0 con 0x560321016ad0 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 10 client.0 _ll_register_callbacks cb 0x560320fc9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 0x56032107e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 0x56032107e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44596/0 (socket says 172.21.15.60:44596) 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x56032102d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 -- 172.21.15.60:0/4083972187 learned_addr learned my addr 172.21.15.60:0/4083972187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce8be7640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560321009ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 -- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 msgr2=0x560321009ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560321009ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 -- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 msgr2=0x56032102d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x56032102d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 -- 172.21.15.60:0/4083972187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560320ffc180 con 0x560320ff9230 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce3fff640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x56032102d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:19.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcce37fe640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 0x56032107e050 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fccd8001f00 tx=0x7fccd8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aecfc5e6da6a245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fccc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fccc2ffd640 1 -- 172.21.15.60:0/4083972187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd807b060 con 0x560320ff9230 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fccc2ffd640 1 -- 172.21.15.60:0/4083972187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccd8057270 con 0x560320ff9230 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.660+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccdc031d10 con 0x560320ff9230 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fccc2ffd640 1 -- 172.21.15.60:0/4083972187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd8081050 con 0x560320ff9230 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 10 client.6042 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x56032102d8e0 con 0x560320ff9230 2024-04-25T08:32:19.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fccc2ffd640 1 -- 172.21.15.60:0/4083972187 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc031d10 con 0x560320ff9230 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fccc2ffd640 1 -- 172.21.15.60:0/4083972187 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fccd80811f0 con 0x560320ff9230 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 10 client.6042 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 10 client.6042 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 20 client.6042 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560321026800 0x560321026be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 4 client.6042 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560320f3ef50 con 0x560321026800 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcce3fff640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560321026800 0x560321026be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fccc2ffd640 10 client.6042 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:19.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fcce3fff640 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560321026800 0x560321026be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1780238ddd3c4ace server_cookie=a993b6de9b4972f3 in_seq=0 out_seq=0 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fccc2ffd640 1 -- 172.21.15.60:0/4083972187 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560320f3ef50 con 0x560321026800 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.661+0000 7fccc2ffd640 10 client.6042 handle_command_reply: tid=0 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 client.6042 shutdown 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 msgr2=0x56032107e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 0x56032107e050 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fccd8001f00 tx=0x7fccd8007f70 comp rx=0 tx=0).stop 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 shutdown_connections 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560321026800 0x560321026be0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560321016ad0 0x560321009ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560320ff9230 0x56032107e050 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccc2ffd640 0 client.6042 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 --2- 172.21.15.60:0/4083972187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560320fc84a0 0x56032102d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 >> 172.21.15.60:0/4083972187 conn(0x560320fcef10 msgr2=0x560320ffc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 shutdown_connections 2024-04-25T08:32:19.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 1 -- 172.21.15.60:0/4083972187 wait complete. 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fcced428740 20 client.6042 trim_cache size 0 max 16384 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 5886, 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.6807525180000003, 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 22, 2024-04-25T08:32:19.666 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 55, 2024-04-25T08:32:19.667 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 55, 2024-04-25T08:32:19.667 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 10.020930365 2024-04-25T08:32:19.667 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccd006b4d0 msgr2=0x7fccd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccd006b4d0 0x7fccd006d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009a10 tx=0x7fccd4008040 comp rx=0 tx=0).stop 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 msgr2=0x7fcce40ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 0x7fcce40ccdc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c400 tx=0x7fccd800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 shutdown_connections 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fccd006b4d0 0x7fccd006d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcce4162450 0x7fcce40cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcce415b980 0x7fcce40ccdc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 --2- 172.21.15.60:0/2180913335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcce415b5a0 0x7fcce415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.662+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 >> 172.21.15.60:0/2180913335 conn(0x7fcce40d0530 msgr2=0x7fcce40b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.663+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 shutdown_connections 2024-04-25T08:32:19.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.663+0000 7fccea64a640 1 -- 172.21.15.60:0/2180913335 wait complete. 2024-04-25T08:32:19.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:32:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.774+0000 7fa1232b6640 1 Processor -- start 2024-04-25T08:32:19.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.774+0000 7fa1232b6640 1 -- start start 2024-04-25T08:32:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa1232b6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa11c0b3af0 0x7fa11c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa1232b6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa11c0cf340 0x7fa11c0cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa1232b6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0c7380 con 0x7fa11c15b8f0 2024-04-25T08:32:19.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa1232b6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0d03c0 con 0x7fa11c0cf340 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa1232b6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0c7150 con 0x7fa11c0b3af0 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa121853640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa11c0cf340 0x7fa11c0cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa121853640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa11c0cf340 0x7fa11c0cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44612/0 (socket says 172.21.15.60:44612) 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa121052640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa11c15b8f0 0x7fa11c0b33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58812/0 (socket says 172.21.15.60:58812) 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.775+0000 7fa121853640 1 -- 172.21.15.60:0/3134239101 learned_addr learned my addr 172.21.15.60:0/3134239101 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.776+0000 7fa120851640 1 --2- 172.21.15.60:0/3134239101 >> v2:172.21.15.100:3300/0 conn(0x7fa11c0b3af0 0x7fa11c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.776+0000 7fa121853640 1 -- 172.21.15.60:0/3134239101 >> v2:172.21.15.100:3300/0 conn(0x7fa11c0b3af0 msgr2=0x7fa11c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.776+0000 7fa121853640 1 --2- 172.21.15.60:0/3134239101 >> v2:172.21.15.100:3300/0 conn(0x7fa11c0b3af0 0x7fa11c0c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.776+0000 7fa121853640 1 -- 172.21.15.60:0/3134239101 >> v1:172.21.15.60:6789/0 conn(0x7fa11c15b8f0 legacy=0x7fa11c0b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.776+0000 7fa121853640 1 -- 172.21.15.60:0/3134239101 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0c7660 con 0x7fa11c0cf340 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.776+0000 7fa121853640 1 --2- 172.21.15.60:0/3134239101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c0cbee0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0b4300 tx=0x7fa10c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b7745c73cb66c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa11bfff640 1 -- 172.21.15.60:0/3134239101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c036070 con 0x7fa11c0cf340 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa11bfff640 1 -- 172.21.15.60:0/3134239101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c031600 con 0x7fa11c0cf340 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa11bfff640 1 -- 172.21.15.60:0/3134239101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0319f0 con 0x7fa11c0cf340 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 -- 172.21.15.60:0/3134239101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 msgr2=0x7fa11c0cbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3134239101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c0cbee0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0b4300 tx=0x7fa10c02a7c0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 -- 172.21.15.60:0/3134239101 shutdown_connections 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3134239101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c0cbee0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3134239101 >> v2:172.21.15.100:3300/0 conn(0x7fa11c0b3af0 0x7fa11c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 -- 172.21.15.60:0/3134239101 >> 172.21.15.60:0/3134239101 conn(0x7fa11c0d1800 msgr2=0x7fa11c0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 -- 172.21.15.60:0/3134239101 shutdown_connections 2024-04-25T08:32:19.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.777+0000 7fa1232b6640 1 -- 172.21.15.60:0/3134239101 wait complete. 2024-04-25T08:32:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.778+0000 7fa1232b6640 1 Processor -- start 2024-04-25T08:32:19.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 -- start start 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa11c0b3af0 0x7fa11c15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c15b8f0 0x7fa11c0b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c7c20 con 0x7fa11c0cf340 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c7f90 con 0x7fa11c15b8f0 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa1232b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c15bcd0 con 0x7fa11c0b3af0 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa120851640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa120851640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c14dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44622/0 (socket says 172.21.15.60:44622) 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.779+0000 7fa120851640 1 -- 172.21.15.60:0/3179479084 learned_addr learned my addr 172.21.15.60:0/3179479084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa121052640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa11c0b3af0 0x7fa11c15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa120851640 1 -- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa11c0b3af0 msgr2=0x7fa11c15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa120851640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa11c0b3af0 0x7fa11c15e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa120851640 1 -- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c15b8f0 msgr2=0x7fa11c0b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa121853640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c15b8f0 0x7fa11c0b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa120851640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c15b8f0 0x7fa11c0b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa120851640 1 -- 172.21.15.60:0/3179479084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0b65c0 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa120851640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c14dea0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c820 tx=0x7fa11000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110015070 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa110004240 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11000ce10 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c0b68a0 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.780+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c14e3e0 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.781+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c0b6f60 con 0x7fa11c0cf340 2024-04-25T08:32:19.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.782+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa110007830 con 0x7fa11c0cf340 2024-04-25T08:32:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.782+0000 7fa119ffb640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa10806b4d0 0x7fa10806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.782+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa11008a270 con 0x7fa11c0cf340 2024-04-25T08:32:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.782+0000 7fa121052640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa10806b4d0 0x7fa10806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.783+0000 7fa121052640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa10806b4d0 0x7fa10806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0ce910 tx=0x7fa114009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:19.787+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa110058f60 con 0x7fa11c0cf340 2024-04-25T08:32:20.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.003+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa11c073990 con 0x7fa11c0cf340 2024-04-25T08:32:20.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.004+0000 7fa119ffb640 1 -- 172.21.15.60:0/3179479084 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa110058860 con 0x7fa11c0cf340 2024-04-25T08:32:20.000 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:32:20.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa10806b4d0 msgr2=0x7fa10806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa10806b4d0 0x7fa10806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0ce910 tx=0x7fa114009040 comp rx=0 tx=0).stop 2024-04-25T08:32:20.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 msgr2=0x7fa11c14dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c14dea0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c820 tx=0x7fa11000cce0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 shutdown_connections 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa10806b4d0 0x7fa10806d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.006+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa11c15b8f0 0x7fa11c0b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.007+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa11c0cf340 0x7fa11c14dea0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.007+0000 7fa1232b6640 1 --2- 172.21.15.60:0/3179479084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa11c0b3af0 0x7fa11c15e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.007+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 >> 172.21.15.60:0/3179479084 conn(0x7fa11c0d1800 msgr2=0x7fa11c156060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.007+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 shutdown_connections 2024-04-25T08:32:20.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20.007+0000 7fa1232b6640 1 -- 172.21.15.60:0/3179479084 wait complete. 2024-04-25T08:32:20.014 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:32:20.014 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:32:20.014 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:32:20.015 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:32:20.015 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:20.015 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:32:20.015 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:32:20.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:32:20.015 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:20.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.015 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:32:20.031 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:20.031 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:20.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:20.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.032 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.033 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:32:20.034 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:20.034 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:32:20.034 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:20.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.034 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:32:20.095 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:32:20.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.097 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:32:20.153 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:32:20.153 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:32:20.153 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:32:20.153 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:32:20.153 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:32:20.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.153 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:32:20.153 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:32:20.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:32:20.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:32:20.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:32:20.154 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:32:20.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.239 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:32:20.254 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:32:20.254 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:32:20.254 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:20.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.255 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:32:20.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:20.319 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:20.375 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:20.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:20.375 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:20.430 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:32:20.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:32:20.512 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:32:20.513 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:32:20.513 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:32:20.521 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:32:20.521 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:32:20.521 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:32:20.521 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:32:20.521 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:32:20.526 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:32:20.526 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:32:20.526 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:32:20.526 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:32:20.527 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:32:20.529 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:32:20.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:20.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.594 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:32:20.643 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:32:20.647+0000 7fe78c355640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:32:20.654 INFO:teuthology.orchestra.run.smithi060.stdout:client5656 2024-04-25T08:32:20.654 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:20.654 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:20.709 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:20.709 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T08:32:20.710 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T08:32:20.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.710 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> import json 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:32:20.769 DEBUG:teuthology.orchestra.run.smithi060:> try: 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> except OSError as e: 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> sys.exit(e.errno) 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> print(json.dumps( 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> indent=2)) 2024-04-25T08:32:20.770 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:32:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:20 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T08:32:20.887 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T08:32:20.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:20.888 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T08:32:20.944 INFO:teuthology.orchestra.run.smithi060.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T08:32:21.095 INFO:teuthology.orchestra.run.smithi060.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T08:32:23.644 INFO:teuthology.orchestra.run.smithi060.stderr:Updating files: 54% (760/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T08:32:23.649 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T08:32:23.649 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T08:32:23.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.793+0000 7fa1d0fbd640 1 Processor -- start 2024-04-25T08:32:23.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.793+0000 7fa1d0fbd640 1 -- start start 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1d0fbd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc15ba00 0x7fa1cc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1d0fbd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1cc15f5f0 0x7fa1cc15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1d0fbd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1cc0cd710 con 0x7fa1cc1602e0 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1d0fbd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1cc0cd4b0 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1d0fbd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc099b70 con 0x7fa1cc15ba00 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1cad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1cc15f5f0 0x7fa1cc15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1cad9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1cc15f5f0 0x7fa1cc15faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44898/0 (socket says 172.21.15.60:44898) 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.794+0000 7fa1cad9d640 1 -- 172.21.15.60:0/2346222197 learned_addr learned my addr 172.21.15.60:0/2346222197 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:23.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.795+0000 7fa1ca59c640 1 --2- 172.21.15.60:0/2346222197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc15ba00 0x7fa1cc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.795+0000 7fa1cad9d640 1 -- 172.21.15.60:0/2346222197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc15ba00 msgr2=0x7fa1cc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.795+0000 7fa1cad9d640 1 --2- 172.21.15.60:0/2346222197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc15ba00 0x7fa1cc09a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.795+0000 7fa1cad9d640 1 -- 172.21.15.60:0/2346222197 >> v1:172.21.15.100:6789/0 conn(0x7fa1cc1602e0 legacy=0x7fa1cc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.795+0000 7fa1cad9d640 1 -- 172.21.15.60:0/2346222197 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc099e50 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.795+0000 7fa1cad9d640 1 --2- 172.21.15.60:0/2346222197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc15faa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009e50 tx=0x7fa1bc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e536a62170251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1c959a640 1 -- 172.21.15.60:0/2346222197 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc036070 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1c959a640 1 -- 172.21.15.60:0/2346222197 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc031600 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1c959a640 1 -- 172.21.15.60:0/2346222197 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0319d0 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/2346222197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 msgr2=0x7fa1cc15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/2346222197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc15faa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009e50 tx=0x7fa1bc02a5f0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/2346222197 shutdown_connections 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/2346222197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc15faa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/2346222197 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc15ba00 0x7fa1cc09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/2346222197 >> 172.21.15.60:0/2346222197 conn(0x7fa1cc0b3270 msgr2=0x7fa1cc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/2346222197 shutdown_connections 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.796+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/2346222197 wait complete. 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 Processor -- start 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 -- start start 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc15ba00 0x7fa1cc1f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc1602e0 0x7fa1cc0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc15bde0 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc09ae70 con 0x7fa1cc15ba00 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1d0fbd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1cc153300 con 0x7fa1cc1602e0 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1c9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1c9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44902/0 (socket says 172.21.15.60:44902) 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.798+0000 7fa1c9d9b640 1 -- 172.21.15.60:0/3327033063 learned_addr learned my addr 172.21.15.60:0/3327033063 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c9d9b640 1 -- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc1602e0 msgr2=0x7fa1cc0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c9d9b640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc1602e0 0x7fa1cc0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c9d9b640 1 -- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc15ba00 msgr2=0x7fa1cc1f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c9d9b640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc15ba00 0x7fa1cc1f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c9d9b640 1 -- 172.21.15.60:0/3327033063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc0c9150 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c9d9b640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc152dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c400 tx=0x7fa1b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4017070 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b40052d0 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1cc0c9430 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4003830 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.799+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1cc0b4490 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.800+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1cc073990 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.801+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1b401a070 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.801+0000 7fa1c2ffd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1ac06b4d0 0x7fa1ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.801+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1b40896f0 con 0x7fa1cc15f5f0 2024-04-25T08:32:23.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.801+0000 7fa1ca59c640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1ac06b4d0 0x7fa1ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.802+0000 7fa1ca59c640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1ac06b4d0 0x7fa1ac06d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0ca410 tx=0x7fa1c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:23.806+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1cc0cd710 con 0x7fa1cc15f5f0 2024-04-25T08:32:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.021+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1cc0b4660 con 0x7fa1cc15f5f0 2024-04-25T08:32:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.022+0000 7fa1c2ffd640 1 -- 172.21.15.60:0/3327033063 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa1b40583e0 con 0x7fa1cc15f5f0 2024-04-25T08:32:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:24.018 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:24.018 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.023+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1ac06b4d0 msgr2=0x7fa1ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.023+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1ac06b4d0 0x7fa1ac06d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0ca410 tx=0x7fa1c4008040 comp rx=0 tx=0).stop 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.023+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 msgr2=0x7fa1cc152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.023+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc152dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c400 tx=0x7fa1b400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 shutdown_connections 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa1cc1602e0 0x7fa1cc0c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1cc15f5f0 0x7fa1cc152dc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1ac06b4d0 0x7fa1ac06d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 --2- 172.21.15.60:0/3327033063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1cc15ba00 0x7fa1cc1f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 >> 172.21.15.60:0/3327033063 conn(0x7fa1cc0b3270 msgr2=0x7fa1cc0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 shutdown_connections 2024-04-25T08:32:24.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.024+0000 7fa1d0fbd640 1 -- 172.21.15.60:0/3327033063 wait complete. 2024-04-25T08:32:24.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T08:32:24.137 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:24.138 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": -2 2024-04-25T08:32:24.138 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:24.146 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T08:32:24.146 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T08:32:24.147 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:24.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T08:32:24.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.254+0000 7fba89789640 1 Processor -- start 2024-04-25T08:32:24.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.254+0000 7fba89789640 1 -- start start 2024-04-25T08:32:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba89789640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba8415a9f0 0x7fba840c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba89789640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba840cdbf0 0x7fba840ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba89789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cd520 0x7fba840ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba89789640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840b0020 con 0x7fba840cdbf0 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba89789640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba840cb5a0 con 0x7fba8415a9f0 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba89789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840c6940 con 0x7fba840cd520 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba827fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba840cdbf0 0x7fba840ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba827fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fba840cdbf0 0x7fba840ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44904/0 (socket says 172.21.15.60:44904) 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.255+0000 7fba827fc640 1 -- 172.21.15.60:0/867196238 learned_addr learned my addr 172.21.15.60:0/867196238 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba82ffd640 1 --2- 172.21.15.60:0/867196238 >> v2:172.21.15.100:3300/0 conn(0x7fba8415a9f0 0x7fba840c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba837fe640 1 --2- 172.21.15.60:0/867196238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cd520 0x7fba840ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba827fc640 1 -- 172.21.15.60:0/867196238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cd520 msgr2=0x7fba840ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba827fc640 1 --2- 172.21.15.60:0/867196238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cd520 0x7fba840ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba827fc640 1 -- 172.21.15.60:0/867196238 >> v2:172.21.15.100:3300/0 conn(0x7fba8415a9f0 msgr2=0x7fba840c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba827fc640 1 --2- 172.21.15.60:0/867196238 >> v2:172.21.15.100:3300/0 conn(0x7fba8415a9f0 0x7fba840c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba827fc640 1 -- 172.21.15.60:0/867196238 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8415add0 con 0x7fba840cdbf0 2024-04-25T08:32:24.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.256+0000 7fba827fc640 1 --2- 172.21.15.60:0/867196238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba840cdbf0 0x7fba840ccfe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fba840c77e0 tx=0x7fba6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3046a7a39c35d494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba81ffb640 1 -- 172.21.15.60:0/867196238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037070 con 0x7fba840cdbf0 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba81ffb640 1 -- 172.21.15.60:0/867196238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c032440 con 0x7fba840cdbf0 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba81ffb640 1 -- 172.21.15.60:0/867196238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c032810 con 0x7fba840cdbf0 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 -- 172.21.15.60:0/867196238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba840cdbf0 msgr2=0x7fba840ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 --2- 172.21.15.60:0/867196238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba840cdbf0 0x7fba840ccfe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fba840c77e0 tx=0x7fba6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 -- 172.21.15.60:0/867196238 shutdown_connections 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 --2- 172.21.15.60:0/867196238 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cd520 0x7fba840ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 --2- 172.21.15.60:0/867196238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba840cdbf0 0x7fba840ccfe0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 --2- 172.21.15.60:0/867196238 >> v2:172.21.15.100:3300/0 conn(0x7fba8415a9f0 0x7fba840c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 -- 172.21.15.60:0/867196238 >> 172.21.15.60:0/867196238 conn(0x7fba840b2390 msgr2=0x7fba840b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 -- 172.21.15.60:0/867196238 shutdown_connections 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.257+0000 7fba89789640 1 -- 172.21.15.60:0/867196238 wait complete. 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 Processor -- start 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 -- start start 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba840cd520 0x7fba841579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cdbf0 0x7fba8415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 0x7fba84149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840cb230 con 0x7fba8415a9f0 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840caf50 con 0x7fba840cd520 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba89789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840ce070 con 0x7fba840cdbf0 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 0x7fba84149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 0x7fba84149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44906/0 (socket says 172.21.15.60:44906) 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba837fe640 1 -- 172.21.15.60:0/663787353 learned_addr learned my addr 172.21.15.60:0/663787353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba82ffd640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba840cd520 0x7fba841579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.259+0000 7fba827fc640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cdbf0 0x7fba8415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba837fe640 1 -- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cdbf0 msgr2=0x7fba8415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba837fe640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cdbf0 0x7fba8415dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba837fe640 1 -- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba840cd520 msgr2=0x7fba841579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba837fe640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba840cd520 0x7fba841579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba837fe640 1 -- 172.21.15.60:0/663787353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84157f30 con 0x7fba8415a9f0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba837fe640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 0x7fba84149f60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fba74007e60 tx=0x7fba7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba7b7fe640 1 -- 172.21.15.60:0/663787353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74014620 con 0x7fba8415a9f0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba7b7fe640 1 -- 172.21.15.60:0/663787353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74019070 con 0x7fba8415a9f0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8415e260 con 0x7fba8415a9f0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba7b7fe640 1 -- 172.21.15.60:0/663787353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74014a20 con 0x7fba8415a9f0 2024-04-25T08:32:24.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.260+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba84158770 con 0x7fba8415a9f0 2024-04-25T08:32:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.262+0000 7fba7b7fe640 1 -- 172.21.15.60:0/663787353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba74014bc0 con 0x7fba8415a9f0 2024-04-25T08:32:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.262+0000 7fba7b7fe640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.262+0000 7fba7b7fe640 1 -- 172.21.15.60:0/663787353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba7408aa60 con 0x7fba8415a9f0 2024-04-25T08:32:24.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.262+0000 7fba82ffd640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.263+0000 7fba82ffd640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0079a0 tx=0x7fba7c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.263+0000 7fba8c569740 1 Processor -- start 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.263+0000 7fba8c569740 1 -- start start 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b026b0 0x55bcc1b02a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba8c569740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1ad8390 0x55bcc1b1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba8c569740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1a3ac60 con 0x55bcc1b1ff50 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc19e3ee0 con 0x55bcc1b026b0 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1a36d00 con 0x55bcc1ad8390 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b204a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44918/0 (socket says 172.21.15.60:44918) 2024-04-25T08:32:24.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 -- 172.21.15.60:0/449714518 learned_addr learned my addr 172.21.15.60:0/449714518 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba82ffd640 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b026b0 0x55bcc1b02a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba837fe640 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1ad8390 0x55bcc1b1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 -- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1ad8390 msgr2=0x55bcc1b1e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1ad8390 0x55bcc1b1e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 -- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b026b0 msgr2=0x55bcc1b02a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b026b0 0x55bcc1b02a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 -- 172.21.15.60:0/449714518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc181b9f0 con 0x55bcc1b1ff50 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba827fc640 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b204a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0090b0 tx=0x7fba6c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a3c7844673e468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba78ff9640 1 -- 172.21.15.60:0/449714518 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c043070 con 0x55bcc1b1ff50 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba78ff9640 1 -- 172.21.15.60:0/449714518 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c0326c0 con 0x55bcc1b1ff50 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.264+0000 7fba78ff9640 1 -- 172.21.15.60:0/449714518 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037040 con 0x55bcc1b1ff50 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 -- 172.21.15.60:0/449714518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 msgr2=0x55bcc1b204a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b204a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0090b0 tx=0x7fba6c02c000 comp rx=0 tx=0).stop 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 -- 172.21.15.60:0/449714518 shutdown_connections 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1ad8390 0x55bcc1b1e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b204a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 --2- 172.21.15.60:0/449714518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b026b0 0x55bcc1b02a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 -- 172.21.15.60:0/449714518 >> 172.21.15.60:0/449714518 conn(0x55bcc1ad4cc0 msgr2=0x55bcc1b1d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 -- 172.21.15.60:0/449714518 shutdown_connections 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.265+0000 7fba8c569740 1 -- 172.21.15.60:0/449714518 wait complete. 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.266+0000 7fba8c569740 1 -- start start 2024-04-25T08:32:24.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.266+0000 7fba8c569740 1 Processor -- start 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 0x55bcc1b1bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b1ff50 0x55bcc1b2d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b2de90 0x55bcc1b06070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1ad9130 con 0x55bcc1b026b0 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc19f7470 con 0x55bcc1b2de90 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1ad2bf0 con 0x55bcc1b1ff50 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 10 client.0 _ll_register_callbacks cb 0x55bcc19c37d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 0x55bcc1b1bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 0x55bcc1b1bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44932/0 (socket says 172.21.15.60:44932) 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 -- 172.21.15.60:0/958432440 learned_addr learned my addr 172.21.15.60:0/958432440 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba827fc640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b1ff50 0x55bcc1b2d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba837fe640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b2de90 0x55bcc1b06070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 -- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b1ff50 msgr2=0x55bcc1b2d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b1ff50 0x55bcc1b2d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 -- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b2de90 msgr2=0x55bcc1b06070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b2de90 0x55bcc1b06070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 -- 172.21.15.60:0/958432440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcc170f2e0 con 0x55bcc1b026b0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba837fe640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b2de90 0x55bcc1b06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba82ffd640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 0x55bcc1b1bea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fba7c016a30 tx=0x7fba7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e038582d256c3c23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba61ffb640 1 -- 172.21.15.60:0/958432440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c026070 con 0x55bcc1b026b0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba61ffb640 1 -- 172.21.15.60:0/958432440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba7c010e40 con 0x55bcc1b026b0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.267+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba6c0326c0 con 0x55bcc1b026b0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba61ffb640 1 -- 172.21.15.60:0/958432440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba7c031050 con 0x55bcc1b026b0 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 10 client.6069 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:24.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55bcc187c180 con 0x55bcc1b026b0 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba61ffb640 1 -- 172.21.15.60:0/958432440 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c0326c0 con 0x55bcc1b026b0 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba61ffb640 1 -- 172.21.15.60:0/958432440 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fba7c034020 con 0x55bcc1b026b0 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 10 client.6069 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 10 client.6069 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 20 client.6069 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b03f60 0x55bcc1b04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 4 client.6069 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcc1a481a0 con 0x55bcc1b03f60 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba827fc640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b03f60 0x55bcc1b04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba61ffb640 10 client.6069 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.268+0000 7fba827fc640 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b03f60 0x55bcc1b04370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbddd79353749fa0 server_cookie=260d803ee887f98b in_seq=0 out_seq=0 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba61ffb640 1 -- 172.21.15.60:0/958432440 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bcc1a481a0 con 0x55bcc1b03f60 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba61ffb640 10 client.6069 handle_command_reply: tid=0 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 client.6069 shutdown 2024-04-25T08:32:24.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 msgr2=0x55bcc1b1bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 0x55bcc1b1bea0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fba7c016a30 tx=0x7fba7c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 shutdown_connections 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b2de90 0x55bcc1b06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b03f60 0x55bcc1b04370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b1ff50 0x55bcc1b2d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 --2- 172.21.15.60:0/958432440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b026b0 0x55bcc1b1bea0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 >> 172.21.15.60:0/958432440 conn(0x55bcc1ad8390 msgr2=0x55bcc1b1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba61ffb640 0 client.6069 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 shutdown_connections 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 1 -- 172.21.15.60:0/958432440 wait complete. 2024-04-25T08:32:24.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.269+0000 7fba8c569740 20 client.6069 trim_cache size 0 max 16384 2024-04-25T08:32:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.284+0000 7fba8c569740 1 Processor -- start 2024-04-25T08:32:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.284+0000 7fba8c569740 1 -- start start 2024-04-25T08:32:24.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b20cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba8c569740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1ad8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba8c569740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba842001d0 con 0x55bcc1b2de90 2024-04-25T08:32:24.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba841ff390 con 0x55bcc1b1ff50 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8414d5a0 con 0x55bcc1b026b0 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1ad8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1ad8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44938/0 (socket says 172.21.15.60:44938) 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba837fe640 1 -- 172.21.15.60:0/203452428 learned_addr learned my addr 172.21.15.60:0/203452428 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba827fc640 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b20cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba82ffd640 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba837fe640 1 -- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 msgr2=0x55bcc1b17a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba837fe640 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b17a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.285+0000 7fba837fe640 1 -- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 msgr2=0x55bcc1b20cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba837fe640 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b20cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba837fe640 1 -- 172.21.15.60:0/203452428 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc16e83a0 con 0x55bcc1b2de90 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba82ffd640 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b17a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba837fe640 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1ad8960 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fba74058e00 tx=0x7fba740553a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c92aae3aff1406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba63fff640 1 -- 172.21.15.60:0/203452428 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74087070 con 0x55bcc1b2de90 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba63fff640 1 -- 172.21.15.60:0/203452428 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74057400 con 0x55bcc1b2de90 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba63fff640 1 -- 172.21.15.60:0/203452428 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740612d0 con 0x55bcc1b2de90 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 -- 172.21.15.60:0/203452428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 msgr2=0x55bcc1ad8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1ad8960 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fba74058e00 tx=0x7fba740553a0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 -- 172.21.15.60:0/203452428 shutdown_connections 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1ad8960 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b20cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 --2- 172.21.15.60:0/203452428 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b17a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 -- 172.21.15.60:0/203452428 >> 172.21.15.60:0/203452428 conn(0x55bcc1ad8390 msgr2=0x55bcc1b403c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 -- 172.21.15.60:0/203452428 shutdown_connections 2024-04-25T08:32:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.286+0000 7fba8c569740 1 -- 172.21.15.60:0/203452428 wait complete. 2024-04-25T08:32:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.287+0000 7fba8c569740 1 -- start start 2024-04-25T08:32:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.287+0000 7fba8c569740 1 Processor -- start 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b05fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b29ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1b8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1b40ad0 con 0x55bcc1b2de90 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba841fed90 con 0x55bcc1b1ff50 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bcc1ace9e0 con 0x55bcc1b026b0 2024-04-25T08:32:24.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 10 client.0 _ll_register_callbacks cb 0x55bcc1ad30d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1b8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1b8bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44952/0 (socket says 172.21.15.60:44952) 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba82ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b05fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 -- 172.21.15.60:0/4078267208 learned_addr learned my addr 172.21.15.60:0/4078267208 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba827fc640 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b29ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 -- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 msgr2=0x55bcc1b05fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b05fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 -- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 msgr2=0x55bcc1b29ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b29ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 -- 172.21.15.60:0/4078267208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcc1b2a430 con 0x55bcc1b2de90 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba837fe640 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1b8bea0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fba74088e50 tx=0x7fba74007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b7f8e26b8a30e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba61ffb640 1 -- 172.21.15.60:0/4078267208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74074070 con 0x55bcc1b2de90 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba61ffb640 1 -- 172.21.15.60:0/4078267208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba740614d0 con 0x55bcc1b2de90 2024-04-25T08:32:24.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.288+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba74057400 con 0x55bcc1b2de90 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba61ffb640 1 -- 172.21.15.60:0/4078267208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba74087070 con 0x55bcc1b2de90 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 10 client.6075 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55bcc1b21480 con 0x55bcc1b2de90 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba61ffb640 1 -- 172.21.15.60:0/4078267208 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74057400 con 0x55bcc1b2de90 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba61ffb640 1 -- 172.21.15.60:0/4078267208 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fba74058550 con 0x55bcc1b2de90 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 10 client.6075 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 20 client.6075 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b88a90 0x55bcc1ad6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 4 client.6075 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55bcc1a424f0 con 0x55bcc1b88a90 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba82ffd640 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b88a90 0x55bcc1ad6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba61ffb640 10 client.6075 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.289+0000 7fba82ffd640 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b88a90 0x55bcc1ad6260 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef673d9a8861a5dd server_cookie=9e1d711b4566322e in_seq=0 out_seq=0 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba61ffb640 1 -- 172.21.15.60:0/4078267208 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55bcc1a424f0 con 0x55bcc1b88a90 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba61ffb640 10 client.6075 handle_command_reply: tid=0 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 client.6075 shutdown 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 msgr2=0x55bcc1b8bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1b8bea0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fba74088e50 tx=0x7fba74007f50 comp rx=0 tx=0).stop 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 shutdown_connections 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bcc1b2de90 0x55bcc1b8bea0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bcc1b1ff50 0x55bcc1b29ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55bcc1b88a90 0x55bcc1ad6260 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 --2- 172.21.15.60:0/4078267208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bcc1b026b0 0x55bcc1b05fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba61ffb640 0 client.6075 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 >> 172.21.15.60:0/4078267208 conn(0x55bcc1ad8390 msgr2=0x55bcc1b37800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 shutdown_connections 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 1 -- 172.21.15.60:0/4078267208 wait complete. 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba8c569740 20 client.6075 trim_cache size 0 max 16384 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": -2 2024-04-25T08:32:24.293 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:24.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba6406b4d0 msgr2=0x7fba6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0079a0 tx=0x7fba7c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 msgr2=0x7fba84149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 0x7fba84149f60 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fba74007e60 tx=0x7fba7400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 shutdown_connections 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba8415a9f0 0x7fba84149f60 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fba840cdbf0 0x7fba8415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 --2- 172.21.15.60:0/663787353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba840cd520 0x7fba841579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 >> 172.21.15.60:0/663787353 conn(0x7fba840b2390 msgr2=0x7fba8406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 shutdown_connections 2024-04-25T08:32:24.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.290+0000 7fba89789640 1 -- 172.21.15.60:0/663787353 wait complete. 2024-04-25T08:32:24.298 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T08:32:24.298 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:32:24.298 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:32:24.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.403+0000 7f48841a5640 1 Processor -- start 2024-04-25T08:32:24.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.403+0000 7f48841a5640 1 -- start start 2024-04-25T08:32:24.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f48841a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f48841a5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f487c0d0ad0 0x7f487c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f48841a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487c0d1df0 con 0x7f487c0b9770 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f48841a5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c0d20d0 con 0x7f487c0d0ad0 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f48841a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0d23b0 con 0x7f487c15dd30 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f487c0d0ad0 0x7f487c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f487c0d0ad0 0x7f487c0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44968/0 (socket says 172.21.15.60:44968) 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 -- 172.21.15.60:0/4004587279 learned_addr learned my addr 172.21.15.60:0/4004587279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4881f41640 1 --2- 172.21.15.60:0/4004587279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 -- 172.21.15.60:0/4004587279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 msgr2=0x7f487c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 --2- 172.21.15.60:0/4004587279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 -- 172.21.15.60:0/4004587279 >> v1:172.21.15.100:6789/0 conn(0x7f487c0b9770 legacy=0x7f487c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.404+0000 7f4882742640 1 -- 172.21.15.60:0/4004587279 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c0d2690 con 0x7f487c0d0ad0 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.405+0000 7f4882742640 1 --2- 172.21.15.60:0/4004587279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0d0ad0 0x7f487c0b6a40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4874009e50 tx=0x7f487402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73197d9b50cea0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.405+0000 7f4880f3f640 1 -- 172.21.15.60:0/4004587279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874036070 con 0x7f487c0d0ad0 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.405+0000 7f4880f3f640 1 -- 172.21.15.60:0/4004587279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874031600 con 0x7f487c0d0ad0 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.405+0000 7f4880f3f640 1 -- 172.21.15.60:0/4004587279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740319d0 con 0x7f487c0d0ad0 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.405+0000 7f48841a5640 1 -- 172.21.15.60:0/4004587279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0d0ad0 msgr2=0x7f487c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.405+0000 7f48841a5640 1 --2- 172.21.15.60:0/4004587279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0d0ad0 0x7f487c0b6a40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4874009e50 tx=0x7f487402a630 comp rx=0 tx=0).stop 2024-04-25T08:32:24.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.406+0000 7f48841a5640 1 -- 172.21.15.60:0/4004587279 shutdown_connections 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.406+0000 7f48841a5640 1 --2- 172.21.15.60:0/4004587279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0d0ad0 0x7f487c0b6a40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.406+0000 7f48841a5640 1 --2- 172.21.15.60:0/4004587279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.406+0000 7f48841a5640 1 -- 172.21.15.60:0/4004587279 >> 172.21.15.60:0/4004587279 conn(0x7f487c061c30 msgr2=0x7f487c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.406+0000 7f48841a5640 1 -- 172.21.15.60:0/4004587279 shutdown_connections 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.406+0000 7f48841a5640 1 -- 172.21.15.60:0/4004587279 wait complete. 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.407+0000 7f48841a5640 1 Processor -- start 2024-04-25T08:32:24.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.407+0000 7f48841a5640 1 -- start start 2024-04-25T08:32:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 0x7f487c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c0d0ad0 0x7f487c14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0d1b90 con 0x7f487c0b9770 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0d1a10 con 0x7f487c0d0ad0 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0d2e90 con 0x7f487c15dd30 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 0x7f487c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 0x7f487c0cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:44976/0 (socket says 172.21.15.60:44976) 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 -- 172.21.15.60:0/1606975189 learned_addr learned my addr 172.21.15.60:0/1606975189 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4882742640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881740640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c0d0ad0 0x7f487c14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 -- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 msgr2=0x7f487c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 -- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c0d0ad0 msgr2=0x7f487c14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c0d0ad0 0x7f487c14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 -- 172.21.15.60:0/1606975189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c15eca0 con 0x7f487c0b9770 2024-04-25T08:32:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f4881f41640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 0x7f487c0cd580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f487800d620 tx=0x7f487800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800b620 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878019070 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c15ef80 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.408+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800ba00 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.409+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c15f540 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.409+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c073990 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.410+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f487800bba0 con 0x7f487c0b9770 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.410+0000 7f48727fc640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f486006b500 0x7f486006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.410+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f487808a5e0 con 0x7f487c0b9770 2024-04-25T08:32:24.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.410+0000 7f4881740640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f486006b500 0x7f486006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.411+0000 7f4881740640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f486006b500 0x7f486006d9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f487c1f6c20 tx=0x7f486c007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.415+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4878058c80 con 0x7f487c0b9770 2024-04-25T08:32:24.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.631+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f487c15d2a0 con 0x7f487c0b9770 2024-04-25T08:32:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.632+0000 7f48727fc640 1 -- 172.21.15.60:0/1606975189 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f487805cd20 con 0x7f487c0b9770 2024-04-25T08:32:24.628 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:24.628 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:24.628 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f486006b500 msgr2=0x7f486006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f486006b500 0x7f486006d9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f487c1f6c20 tx=0x7f486c007e80 comp rx=0 tx=0).stop 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 msgr2=0x7f487c0cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 0x7f487c0cd580 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f487800d620 tx=0x7f487800dae0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 shutdown_connections 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f487c15dd30 0x7f487c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f486006b500 0x7f486006d9c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f487c0d0ad0 0x7f487c14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 --2- 172.21.15.60:0/1606975189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f487c0b9770 0x7f487c0cd580 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 >> 172.21.15.60:0/1606975189 conn(0x7f487c061c30 msgr2=0x7f487c0c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 shutdown_connections 2024-04-25T08:32:24.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.634+0000 7f48841a5640 1 -- 172.21.15.60:0/1606975189 wait complete. 2024-04-25T08:32:24.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:32:24.751 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:24.751 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:24.751 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:24.761 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:24.761 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:32:24.761 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:24.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:32:24.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.868+0000 7f79abbd5640 1 Processor -- start 2024-04-25T08:32:24.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.869+0000 7f79abbd5640 1 -- start start 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.869+0000 7f79abbd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79a40ce230 0x7f79a40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.869+0000 7f79abbd5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a4072280 con 0x7f79a415b270 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.869+0000 7f79abbd5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a40714c0 con 0x7f79a40cc4f0 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.869+0000 7f79abbd5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a4071270 con 0x7f79a40ce230 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79a9170640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79a40ce230 0x7f79a40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79aa172640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f79a415b270 0x7f79a415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45522/0 (socket says 172.21.15.60:45522) 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79aa172640 1 -- 172.21.15.60:0/1242814487 learned_addr learned my addr 172.21.15.60:0/1242814487 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:24.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79a9170640 1 -- 172.21.15.60:0/1242814487 >> v1:172.21.15.100:6789/0 conn(0x7f79a40cc4f0 legacy=0x7f79a40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79a9170640 1 -- 172.21.15.60:0/1242814487 >> v1:172.21.15.60:6789/0 conn(0x7f79a415b270 legacy=0x7f79a415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79a9170640 1 -- 172.21.15.60:0/1242814487 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40717a0 con 0x7f79a40ce230 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.870+0000 7f79a9170640 1 --2- 172.21.15.60:0/1242814487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40ce230 0x7f79a40cc8d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f79940097a0 tx=0x7f799402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca8322b7a4b742e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79a896f640 1 -- 172.21.15.60:0/1242814487 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994037070 con 0x7f79a40ce230 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79a896f640 1 -- 172.21.15.60:0/1242814487 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994032440 con 0x7f79a40ce230 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79a896f640 1 -- 172.21.15.60:0/1242814487 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994032810 con 0x7f79a40ce230 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 -- 172.21.15.60:0/1242814487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40ce230 msgr2=0x7f79a40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1242814487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40ce230 0x7f79a40cc8d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f79940097a0 tx=0x7f799402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 -- 172.21.15.60:0/1242814487 shutdown_connections 2024-04-25T08:32:24.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1242814487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40ce230 0x7f79a40cc8d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 -- 172.21.15.60:0/1242814487 >> 172.21.15.60:0/1242814487 conn(0x7f79a406abe0 msgr2=0x7f79a406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 -- 172.21.15.60:0/1242814487 shutdown_connections 2024-04-25T08:32:24.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.871+0000 7f79abbd5640 1 -- 172.21.15.60:0/1242814487 wait complete. 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.872+0000 7f79abbd5640 1 Processor -- start 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 -- start start 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 0x7f79a41e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a40ce230 0x7f79a4150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a415b270 0x7f79a41e3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a406e8a0 con 0x7f79a40cc4f0 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4071d60 con 0x7f79a415b270 2024-04-25T08:32:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79abbd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a40ae500 con 0x7f79a40ce230 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79a9971640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 0x7f79a41e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79a9971640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 0x7f79a41e4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45008/0 (socket says 172.21.15.60:45008) 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.873+0000 7f79a9971640 1 -- 172.21.15.60:0/1853930662 learned_addr learned my addr 172.21.15.60:0/1853930662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9170640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a40ce230 0x7f79a4150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79aa172640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a415b270 0x7f79a41e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9971640 1 -- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a40ce230 msgr2=0x7f79a4150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9971640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a40ce230 0x7f79a4150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9971640 1 -- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a415b270 msgr2=0x7f79a41e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9971640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a415b270 0x7f79a41e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9971640 1 -- 172.21.15.60:0/1853930662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a41e53c0 con 0x7f79a40cc4f0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79a9971640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 0x7f79a41e4e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f79a000b420 tx=0x7f79a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f799a7fc640 1 -- 172.21.15.60:0/1853930662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0011830 con 0x7f79a40cc4f0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f799a7fc640 1 -- 172.21.15.60:0/1853930662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0016070 con 0x7f79a40cc4f0 2024-04-25T08:32:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a41e3de0 con 0x7f79a40cc4f0 2024-04-25T08:32:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f799a7fc640 1 -- 172.21.15.60:0/1853930662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0011c10 con 0x7f79a40cc4f0 2024-04-25T08:32:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.874+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a41e4680 con 0x7f79a40cc4f0 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.875+0000 7f799a7fc640 1 -- 172.21.15.60:0/1853930662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79a001b460 con 0x7f79a40cc4f0 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.876+0000 7f799a7fc640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f798806b4d0 0x7f798806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.876+0000 7f799a7fc640 1 -- 172.21.15.60:0/1853930662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79a0089da0 con 0x7f79a40cc4f0 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.876+0000 7f79a9170640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f798806b4d0 0x7f798806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.877+0000 7f79a9170640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f798806b4d0 0x7f798806d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7994034000 tx=0x7f799402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.877+0000 7f79ae9b1740 1 Processor -- start 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.877+0000 7f79ae9b1740 1 -- start start 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.877+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c0032f10 0x5558c0035070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c00332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c005de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558bff64fe0 con 0x5558c005d9d0 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558bff96d20 con 0x5558c006f5e0 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558bff3ffa0 con 0x5558c0032f10 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c005de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c005de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45018/0 (socket says 172.21.15.60:45018) 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 -- 172.21.15.60:0/4273625314 learned_addr learned my addr 172.21.15.60:0/4273625314 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79a9971640 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c0032f10 0x5558c0035070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79a9170640 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c00332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 -- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c0032f10 msgr2=0x5558c0035070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c0032f10 0x5558c0035070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 -- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 msgr2=0x5558c00332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c00332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 -- 172.21.15.60:0/4273625314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558bfd77b00 con 0x5558c005d9d0 2024-04-25T08:32:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79aa172640 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c005de50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f799c002120 tx=0x7f799c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5b6eb31016485e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79877fe640 1 -- 172.21.15.60:0/4273625314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c018070 con 0x5558c005d9d0 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79877fe640 1 -- 172.21.15.60:0/4273625314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c013670 con 0x5558c005d9d0 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.878+0000 7f79877fe640 1 -- 172.21.15.60:0/4273625314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c013a40 con 0x5558c005d9d0 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 -- 172.21.15.60:0/4273625314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 msgr2=0x5558c005de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c005de50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f799c002120 tx=0x7f799c00d520 comp rx=0 tx=0).stop 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 -- 172.21.15.60:0/4273625314 shutdown_connections 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c005de50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c00332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/4273625314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c0032f10 0x5558c0035070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 -- 172.21.15.60:0/4273625314 >> 172.21.15.60:0/4273625314 conn(0x5558bfffb100 msgr2=0x5558c00789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 -- 172.21.15.60:0/4273625314 shutdown_connections 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.879+0000 7f79ae9b1740 1 -- 172.21.15.60:0/4273625314 wait complete. 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.880+0000 7f79ae9b1740 1 -- start start 2024-04-25T08:32:24.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.880+0000 7f79ae9b1740 1 Processor -- start 2024-04-25T08:32:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 0x5558c004f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c0070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c0070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558bff35750 con 0x5558c0032f10 2024-04-25T08:32:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558bff92dc0 con 0x5558c006f5e0 2024-04-25T08:32:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c0074670 con 0x5558c005d9d0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 10 client.0 _ll_register_callbacks cb 0x5558bff4dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 0x5558c004f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 0x5558c004f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45026/0 (socket says 172.21.15.60:45026) 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 -- 172.21.15.60:0/2177895249 learned_addr learned my addr 172.21.15.60:0/2177895249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79aa172640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c0070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9170640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 -- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 msgr2=0x5558c0070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c0070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 -- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 msgr2=0x5558c0070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c0070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 -- 172.21.15.60:0/2177895249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558bfc6b3f0 con 0x5558c0032f10 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9170640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c0070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79a9971640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 0x5558c004f3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f79a0060600 tx=0x7f79a005c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87bb37c809e5818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f7984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f7984ff9640 1 -- 172.21.15.60:0/2177895249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a005c170 con 0x5558c0032f10 2024-04-25T08:32:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f7984ff9640 1 -- 172.21.15.60:0/2177895249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79a005c310 con 0x5558c0032f10 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.881+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f799c013670 con 0x5558c0032f10 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f7984ff9640 1 -- 172.21.15.60:0/2177895249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a0081b30 con 0x5558c0032f10 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 10 client.6096 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5558bfdd8290 con 0x5558c0032f10 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f7984ff9640 1 -- 172.21.15.60:0/2177895249 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c013670 con 0x5558c0032f10 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f7984ff9640 1 -- 172.21.15.60:0/2177895249 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79a0057870 con 0x5558c0032f10 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 10 client.6096 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 20 client.6096 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0060250 0x5558c008bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 4 client.6096 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558bffa4260 con 0x5558c0060250 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79a9170640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0060250 0x5558c008bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f7984ff9640 10 client.6096 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.882+0000 7f79a9170640 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0060250 0x5558c008bb60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd6281b026fc3c9 server_cookie=8397afb628c53f22 in_seq=0 out_seq=0 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f7984ff9640 1 -- 172.21.15.60:0/2177895249 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558bffa4260 con 0x5558c0060250 2024-04-25T08:32:24.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f7984ff9640 10 client.6096 handle_command_reply: tid=0 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 client.6096 shutdown 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 msgr2=0x5558c004f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 0x5558c004f3b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f79a0060600 tx=0x7f79a005c570 comp rx=0 tx=0).stop 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 shutdown_connections 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c006f5e0 0x5558c0070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0060250 0x5558c008bb60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c0070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2177895249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c0032f10 0x5558c004f3b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f7984ff9640 0 client.6096 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 >> 172.21.15.60:0/2177895249 conn(0x5558bfffb100 msgr2=0x5558c0079b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 shutdown_connections 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2177895249 wait complete. 2024-04-25T08:32:24.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.883+0000 7f79ae9b1740 20 client.6096 trim_cache size 0 max 16384 2024-04-25T08:32:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 Processor -- start 2024-04-25T08:32:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 -- start start 2024-04-25T08:32:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c002d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c004e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c006f5e0 0x5558c005dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41fe350 con 0x5558c005d9d0 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41fd4d0 con 0x5558c0032f10 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a414e740 con 0x5558c006f5e0 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.899+0000 7f79a9170640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c004e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c004e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45028/0 (socket says 172.21.15.60:45028) 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 -- 172.21.15.60:0/692303179 learned_addr learned my addr 172.21.15.60:0/692303179 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79aa172640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c006f5e0 0x5558c005dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9971640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c002d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 -- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c006f5e0 msgr2=0x5558c005dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c006f5e0 0x5558c005dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 -- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 msgr2=0x5558c002d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c002d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 -- 172.21.15.60:0/692303179 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558bfc444b0 con 0x5558c005d9d0 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9971640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c002d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79aa172640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c006f5e0 0x5558c005dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f79a9170640 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c004e850 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7994009560 tx=0x7f799402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cea6a18f04277f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f7986ffd640 1 -- 172.21.15.60:0/692303179 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994052070 con 0x5558c005d9d0 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f7986ffd640 1 -- 172.21.15.60:0/692303179 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994004300 con 0x5558c005d9d0 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.900+0000 7f7986ffd640 1 -- 172.21.15.60:0/692303179 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994037040 con 0x5558c005d9d0 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 -- 172.21.15.60:0/692303179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 msgr2=0x5558c004e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c004e850 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7994009560 tx=0x7f799402c000 comp rx=0 tx=0).stop 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 -- 172.21.15.60:0/692303179 shutdown_connections 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c006f5e0 0x5558c005dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c005d9d0 0x5558c004e850 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/692303179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c002d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 -- 172.21.15.60:0/692303179 >> 172.21.15.60:0/692303179 conn(0x5558bfffb100 msgr2=0x5558c0079b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 -- 172.21.15.60:0/692303179 shutdown_connections 2024-04-25T08:32:24.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 -- 172.21.15.60:0/692303179 wait complete. 2024-04-25T08:32:24.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 -- start start 2024-04-25T08:32:24.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.901+0000 7f79ae9b1740 1 Processor -- start 2024-04-25T08:32:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c004dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c004e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 0x5558c006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558c00720a0 con 0x5558c006f5e0 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a41fcf60 con 0x5558c0032f10 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558c00e3730 con 0x5558c005d9d0 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.902+0000 7f79ae9b1740 10 client.0 _ll_register_callbacks cb 0x7f799c001940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 0x5558c006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 0x5558c006f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45042/0 (socket says 172.21.15.60:45042) 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 -- 172.21.15.60:0/2391717216 learned_addr learned my addr 172.21.15.60:0/2391717216 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79a9971640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c004dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79a9170640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c004e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 -- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 msgr2=0x5558c004e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c004e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 -- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 msgr2=0x5558c004dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c004dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 -- 172.21.15.60:0/2391717216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558c0032180 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79a9170640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c004e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79aa172640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 0x5558c006f9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f799c005c00 tx=0x7f799c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3888dd53e0d2b392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f7984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f7984ff9640 1 -- 172.21.15.60:0/2391717216 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c025070 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f7984ff9640 1 -- 172.21.15.60:0/2391717216 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f799c013c70 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7994004300 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f7984ff9640 1 -- 172.21.15.60:0/2391717216 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799c02f050 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79ae9b1740 10 client.6102 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5558c00e3470 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.903+0000 7f7984ff9640 1 -- 172.21.15.60:0/2391717216 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994004300 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f7984ff9640 1 -- 172.21.15.60:0/2391717216 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f799c032020 con 0x5558c006f5e0 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79ae9b1740 10 client.6102 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79ae9b1740 10 client.6102 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79ae9b1740 20 client.6102 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:24.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0091f10 0x5558c006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79ae9b1740 4 client.6102 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x5558bff9e5b0 con 0x5558c0091f10 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79a9971640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0091f10 0x5558c006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f7984ff9640 10 client.6102 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.904+0000 7f79a9971640 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0091f10 0x5558c006ff00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977473fa5f051ccf server_cookie=46adae974e8d0bbe in_seq=0 out_seq=0 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f7984ff9640 1 -- 172.21.15.60:0/2391717216 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5558bff9e5b0 con 0x5558c0091f10 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f7984ff9640 10 client.6102 handle_command_reply: tid=0 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 client.6102 shutdown 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 msgr2=0x5558c006f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 0x5558c006f9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f799c005c00 tx=0x7f799c001250 comp rx=0 tx=0).stop 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 shutdown_connections 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558c006f5e0 0x5558c006f9c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5558c0091f10 0x5558c006ff00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558c005d9d0 0x5558c004e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 --2- 172.21.15.60:0/2391717216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558c0032f10 0x5558c004dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f7984ff9640 0 client.6102 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 >> 172.21.15.60:0/2391717216 conn(0x5558bfffb100 msgr2=0x5558c009b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 shutdown_connections 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 1 -- 172.21.15.60:0/2391717216 wait complete. 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.905+0000 7f79ae9b1740 20 client.6102 trim_cache size 0 max 16384 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "e31d8522-50f6-40d6-9890-5449856da7fc", 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:24.907 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f798806b4d0 msgr2=0x7f798806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f798806b4d0 0x7f798806d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7994034000 tx=0x7f799402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 msgr2=0x7f79a41e4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 0x7f79a41e4e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f79a000b420 tx=0x7f79a000b8e0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 shutdown_connections 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f798806b4d0 0x7f798806d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a415b270 0x7f79a41e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a40ce230 0x7f79a4150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 --2- 172.21.15.60:0/1853930662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a40cc4f0 0x7f79a41e4e80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 >> 172.21.15.60:0/1853930662 conn(0x7f79a406abe0 msgr2=0x7f79a406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 shutdown_connections 2024-04-25T08:32:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:24.906+0000 7f79abbd5640 1 -- 172.21.15.60:0/1853930662 wait complete. 2024-04-25T08:32:24.912 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'e31d8522-50f6-40d6-9890-5449856da7fc', 'mode': 'asynchronous'}' 2024-04-25T08:32:24.913 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:32:24.913 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:32:24.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:25.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 Processor -- start 2024-04-25T08:32:25.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 -- start start 2024-04-25T08:32:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 0x7fb03c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b92a0 0x7fb03c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0b97a0 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c15dfc0 con 0x7fb03c0b92a0 2024-04-25T08:32:25.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.018+0000 7fb0422fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0d1a00 con 0x7fb03c15dbe0 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 0x7fb03c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb040899640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b92a0 0x7fb03c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03b7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb03c0b2cb0 0x7fb03c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57058/0 (socket says 172.21.15.60:57058) 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 0x7fb03c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:44174/0 (socket says 172.21.15.60:44174) 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03b7fe640 1 -- 172.21.15.60:0/2037361592 learned_addr learned my addr 172.21.15.60:0/2037361592 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03bfff640 1 -- 172.21.15.60:0/2037361592 >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b92a0 msgr2=0x7fb03c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03bfff640 1 --2- 172.21.15.60:0/2037361592 >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b92a0 0x7fb03c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03bfff640 1 -- 172.21.15.60:0/2037361592 >> v1:172.21.15.100:6789/0 conn(0x7fb03c0b2cb0 legacy=0x7fb03c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb03bfff640 1 -- 172.21.15.60:0/2037361592 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0d1c50 con 0x7fb03c15dbe0 2024-04-25T08:32:25.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.019+0000 7fb040899640 1 --2- 172.21.15.60:0/2037361592 >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b92a0 0x7fb03c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb03bfff640 1 --2- 172.21.15.60:0/2037361592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 0x7fb03c0d0910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0097a0 tx=0x7fb02c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e96f530571b7505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb03affd640 1 -- 172.21.15.60:0/2037361592 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c037070 con 0x7fb03c15dbe0 2024-04-25T08:32:25.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb03affd640 1 -- 172.21.15.60:0/2037361592 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c032440 con 0x7fb03c15dbe0 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb03affd640 1 -- 172.21.15.60:0/2037361592 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c032810 con 0x7fb03c15dbe0 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb0422fc640 1 -- 172.21.15.60:0/2037361592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 msgr2=0x7fb03c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb0422fc640 1 --2- 172.21.15.60:0/2037361592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 0x7fb03c0d0910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0097a0 tx=0x7fb02c02a180 comp rx=0 tx=0).stop 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.020+0000 7fb0422fc640 1 -- 172.21.15.60:0/2037361592 shutdown_connections 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.021+0000 7fb0422fc640 1 --2- 172.21.15.60:0/2037361592 >> v2:172.21.15.100:3300/0 conn(0x7fb03c0b92a0 0x7fb03c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.021+0000 7fb0422fc640 1 --2- 172.21.15.60:0/2037361592 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c15dbe0 0x7fb03c0d0910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.021+0000 7fb0422fc640 1 -- 172.21.15.60:0/2037361592 >> 172.21.15.60:0/2037361592 conn(0x7fb03c061a20 msgr2=0x7fb03c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.021+0000 7fb0422fc640 1 -- 172.21.15.60:0/2037361592 shutdown_connections 2024-04-25T08:32:25.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.021+0000 7fb0422fc640 1 -- 172.21.15.60:0/2037361592 wait complete. 2024-04-25T08:32:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.022+0000 7fb0422fc640 1 Processor -- start 2024-04-25T08:32:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.022+0000 7fb0422fc640 1 -- start start 2024-04-25T08:32:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb0422fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 0x7fb03c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb0422fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b92a0 0x7fb03c1f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb0422fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15dbe0 0x7fb03c0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb0422fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0b6ab0 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb0422fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0d2210 con 0x7fb03c15dbe0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb0422fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0b6c30 con 0x7fb03c0b92a0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 0x7fb03c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 0x7fb03c154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45054/0 (socket says 172.21.15.60:45054) 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 -- 172.21.15.60:0/3997988723 learned_addr learned my addr 172.21.15.60:0/3997988723 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03b7fe640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b92a0 0x7fb03c1f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb040899640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15dbe0 0x7fb03c0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 -- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b92a0 msgr2=0x7fb03c1f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b92a0 0x7fb03c1f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 -- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15dbe0 msgr2=0x7fb03c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15dbe0 0x7fb03c0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 -- 172.21.15.60:0/3997988723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0b47c0 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb040899640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15dbe0 0x7fb03c0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb03bfff640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 0x7fb03c154600 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb02c035040 tx=0x7fb02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.023+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c037040 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.024+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c030500 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.024+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c0b4a80 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.024+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c030910 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.024+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c0b5040 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.024+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c06d0a0 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.025+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb02c044020 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.025+0000 7fb038ff9640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb02806b4d0 0x7fb02806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.026+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb02c0aa620 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.026+0000 7fb03b7fe640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb02806b4d0 0x7fb02806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.026+0000 7fb03b7fe640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb02806b4d0 0x7fb02806d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0cda20 tx=0x7fb030009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.030+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb02c079320 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.247+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb03c073990 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.248+0000 7fb038ff9640 1 -- 172.21.15.60:0/3997988723 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fb02c078c20 con 0x7fb03c0b2cb0 2024-04-25T08:32:25.244 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:25.244 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:25.244 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:25.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.249+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb02806b4d0 msgr2=0x7fb02806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.249+0000 7fb0422fc640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb02806b4d0 0x7fb02806d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0cda20 tx=0x7fb030009040 comp rx=0 tx=0).stop 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 msgr2=0x7fb03c154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 0x7fb03c154600 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb02c035040 tx=0x7fb02c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 shutdown_connections 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb02806b4d0 0x7fb02806d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb03c15dbe0 0x7fb03c0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb03c0b92a0 0x7fb03c1f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 --2- 172.21.15.60:0/3997988723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb03c0b2cb0 0x7fb03c154600 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 >> 172.21.15.60:0/3997988723 conn(0x7fb03c061a20 msgr2=0x7fb03c15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 shutdown_connections 2024-04-25T08:32:25.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.250+0000 7fb0422fc640 1 -- 172.21.15.60:0/3997988723 wait complete. 2024-04-25T08:32:25.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:32:25.367 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:25.367 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:25.367 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:25.375 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:25.376 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:32:25.376 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:25.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:32:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.484+0000 7fa5e8e41640 1 Processor -- start 2024-04-25T08:32:25.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.484+0000 7fa5e8e41640 1 -- start start 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e8e41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5e4144d50 0x7fa5e406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e8e41640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5e41452a0 0x7fa5e406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e8e41640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40cccc0 con 0x7fa5e41446d0 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e8e41640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40b2b20 con 0x7fa5e41452a0 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e8e41640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e406d400 con 0x7fa5e4144d50 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e2d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa5e41452a0 0x7fa5e406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e259c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5e41446d0 0x7fa5e40b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45528/0 (socket says 172.21.15.60:45528) 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e259c640 1 -- 172.21.15.60:0/3955509994 learned_addr learned my addr 172.21.15.60:0/3955509994 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:25.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e1d9b640 1 -- 172.21.15.60:0/3955509994 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361860106 0 0) 0x7fa5e40cccc0 con 0x7fa5e41446d0 2024-04-25T08:32:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/3955509994 >> v2:172.21.15.100:3300/0 conn(0x7fa5e4144d50 0x7fa5e406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e1d9b640 1 -- 172.21.15.60:0/3955509994 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d0003520 con 0x7fa5e41446d0 2024-04-25T08:32:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/3955509994 >> v2:172.21.15.100:3300/0 conn(0x7fa5e4144d50 msgr2=0x7fa5e406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/3955509994 >> v2:172.21.15.100:3300/0 conn(0x7fa5e4144d50 0x7fa5e406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/3955509994 >> v1:172.21.15.60:6789/0 conn(0x7fa5e41446d0 legacy=0x7fa5e40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.485+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/3955509994 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40cc140 con 0x7fa5e41452a0 2024-04-25T08:32:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.486+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/3955509994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41452a0 0x7fa5e406ddd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4071c70 tx=0x7fa5dc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff6adf21854e63b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.486+0000 7fa5e1d9b640 1 -- 172.21.15.60:0/3955509994 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc007740 con 0x7fa5e41452a0 2024-04-25T08:32:25.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.486+0000 7fa5e1d9b640 1 -- 172.21.15.60:0/3955509994 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc0078e0 con 0x7fa5e41452a0 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.486+0000 7fa5e1d9b640 1 -- 172.21.15.60:0/3955509994 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc007cb0 con 0x7fa5e41452a0 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 -- 172.21.15.60:0/3955509994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41452a0 msgr2=0x7fa5e406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/3955509994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41452a0 0x7fa5e406ddd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4071c70 tx=0x7fa5dc02a680 comp rx=0 tx=0).stop 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 -- 172.21.15.60:0/3955509994 shutdown_connections 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/3955509994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41452a0 0x7fa5e406ddd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/3955509994 >> v2:172.21.15.100:3300/0 conn(0x7fa5e4144d50 0x7fa5e406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 -- 172.21.15.60:0/3955509994 >> 172.21.15.60:0/3955509994 conn(0x7fa5e406abe0 msgr2=0x7fa5e406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 -- 172.21.15.60:0/3955509994 shutdown_connections 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.487+0000 7fa5e8e41640 1 -- 172.21.15.60:0/3955509994 wait complete. 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.488+0000 7fa5e8e41640 1 Processor -- start 2024-04-25T08:32:25.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.488+0000 7fa5e8e41640 1 -- start start 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e8e41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 0x7fa5e40cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e8e41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e4144d50 0x7fa5e40c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e8e41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5e41452a0 0x7fa5e40b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e8e41640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40cc9e0 con 0x7fa5e41446d0 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e8e41640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40cc700 con 0x7fa5e4144d50 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e8e41640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4144ab0 con 0x7fa5e41452a0 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 0x7fa5e40cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 0x7fa5e40cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45076/0 (socket says 172.21.15.60:45076) 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 -- 172.21.15.60:0/2470433851 learned_addr learned my addr 172.21.15.60:0/2470433851 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e4144d50 0x7fa5e40c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 -- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5e41452a0 msgr2=0x7fa5e40b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5e41452a0 0x7fa5e40b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5e41452a0 0x7fa5e40b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 -- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e4144d50 msgr2=0x7fa5e40c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e4144d50 0x7fa5e40c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5e259c640 1 -- 172.21.15.60:0/2470433851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40cde20 con 0x7fa5e41446d0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.489+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e4144d50 0x7fa5e40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.490+0000 7fa5e259c640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 0x7fa5e40cd6e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c390 tx=0x7fa5d400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.490+0000 7fa5daffd640 1 -- 172.21.15.60:0/2470433851 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4017070 con 0x7fa5e41446d0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.490+0000 7fa5daffd640 1 -- 172.21.15.60:0/2470433851 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4005ce0 con 0x7fa5e41446d0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.490+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e40ce100 con 0x7fa5e41446d0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.490+0000 7fa5daffd640 1 -- 172.21.15.60:0/2470433851 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40072f0 con 0x7fa5e41446d0 2024-04-25T08:32:25.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.490+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e40ce6c0 con 0x7fa5e41446d0 2024-04-25T08:32:25.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.491+0000 7fa5daffd640 1 -- 172.21.15.60:0/2470433851 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5d401a070 con 0x7fa5e41446d0 2024-04-25T08:32:25.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.492+0000 7fa5daffd640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa5c406b4d0 0x7fa5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.492+0000 7fa5daffd640 1 -- 172.21.15.60:0/2470433851 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5d4089710 con 0x7fa5e41446d0 2024-04-25T08:32:25.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.492+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa5c406b4d0 0x7fa5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.492+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa5c406b4d0 0x7fa5c406d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0c60 tx=0x7fa5cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.493+0000 7fa5ebc17740 1 Processor -- start 2024-04-25T08:32:25.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.493+0000 7fa5ebc17740 1 -- start start 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cc4e9e60 con 0x5569cc5ccf60 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cc493120 con 0x5569cc587130 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569cc4e5f00 con 0x5569cc5adaf0 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45092/0 (socket says 172.21.15.60:45092) 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 -- 172.21.15.60:0/2039030044 learned_addr learned my addr 172.21.15.60:0/2039030044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 -- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 msgr2=0x5569cc5ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 -- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 msgr2=0x5569cc5cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 -- 172.21.15.60:0/2039030044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569cc2caf00 con 0x5569cc5ccf60 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5e259c640 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5cb1b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4000f30 tx=0x7fa5d4057e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f3331a337413e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5c3fff640 1 -- 172.21.15.60:0/2039030044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4055e30 con 0x5569cc5ccf60 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5c3fff640 1 -- 172.21.15.60:0/2039030044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4056090 con 0x5569cc5ccf60 2024-04-25T08:32:25.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.494+0000 7fa5c3fff640 1 -- 172.21.15.60:0/2039030044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4054400 con 0x5569cc5ccf60 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 -- 172.21.15.60:0/2039030044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 msgr2=0x5569cc5cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5cb1b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4000f30 tx=0x7fa5d4057e50 comp rx=0 tx=0).stop 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 -- 172.21.15.60:0/2039030044 shutdown_connections 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/2039030044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5cb1b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 -- 172.21.15.60:0/2039030044 >> 172.21.15.60:0/2039030044 conn(0x5569cc54df70 msgr2=0x5569cc584150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 -- 172.21.15.60:0/2039030044 shutdown_connections 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.495+0000 7fa5ebc17740 1 -- 172.21.15.60:0/2039030044 wait complete. 2024-04-25T08:32:25.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.496+0000 7fa5ebc17740 1 -- start start 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.496+0000 7fa5ebc17740 1 Processor -- start 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc587130 0x5569cc584b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc5adaf0 0x5569cc5b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cc5cc760 con 0x5569cc5ccf60 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cc488cd0 con 0x5569cc5adaf0 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569cc5cc480 con 0x5569cc587130 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 10 client.0 _ll_register_callbacks cb 0x5569cc1fe2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45096/0 (socket says 172.21.15.60:45096) 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/1826997775 learned_addr learned my addr 172.21.15.60:0/1826997775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e259c640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc587130 0x5569cc584b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc5adaf0 0x5569cc5b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc587130 msgr2=0x5569cc584b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc587130 0x5569cc584b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc5adaf0 msgr2=0x5569cc5b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc5adaf0 0x5569cc5b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/1826997775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569cc1be7f0 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5b0780 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc02ac70 tx=0x7fa5dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292aee01de34801b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5c17fa640 1 -- 172.21.15.60:0/1826997775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc009450 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5c17fa640 1 -- 172.21.15.60:0/1826997775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5dc046c30 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.497+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5d4056090 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5c17fa640 1 -- 172.21.15.60:0/1826997775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5dc046e30 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 10 client.6120 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5569cc32b690 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5c17fa640 1 -- 172.21.15.60:0/1826997775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4056090 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5c17fa640 1 -- 172.21.15.60:0/1826997775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa5dc04d050 con 0x5569cc5ccf60 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 10 client.6120 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:25.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 10 client.6120 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 20 client.6120 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5aa9b0 0x5569cc5df290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 4 client.6120 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569cc4f73a0 con 0x5569cc5aa9b0 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5e259c640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5aa9b0 0x5569cc5df290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5c17fa640 10 client.6120 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.498+0000 7fa5e259c640 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5aa9b0 0x5569cc5df290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80944b850dbc555 server_cookie=ff4e7a5d75510f8b in_seq=0 out_seq=0 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5c17fa640 1 -- 172.21.15.60:0/1826997775 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569cc4f73a0 con 0x5569cc5aa9b0 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5c17fa640 10 client.6120 handle_command_reply: tid=0 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 client.6120 shutdown 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 msgr2=0x5569cc5b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5b0780 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc02ac70 tx=0x7fa5dc004ab0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 shutdown_connections 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5ccf60 0x5569cc5b0780 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc5adaf0 0x5569cc5b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5aa9b0 0x5569cc5df290 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1826997775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc587130 0x5569cc584b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5c17fa640 0 client.6120 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 >> 172.21.15.60:0/1826997775 conn(0x5569cc54df70 msgr2=0x5569cc5ac5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 shutdown_connections 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1826997775 wait complete. 2024-04-25T08:32:25.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.499+0000 7fa5ebc17740 20 client.6120 trim_cache size 0 max 16384 2024-04-25T08:32:25.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 Processor -- start 2024-04-25T08:32:25.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 -- start start 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc54df70 0x5569cc5b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 0x5569cc5afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e42002e0 con 0x5569cc587130 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41ff4c0 con 0x5569cc54df70 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41fe8f0 con 0x5569cc5adaf0 2024-04-25T08:32:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 0x5569cc5afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5dbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 0x5569cc5afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45102/0 (socket says 172.21.15.60:45102) 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.515+0000 7fa5dbfff640 1 -- 172.21.15.60:0/1921679813 learned_addr learned my addr 172.21.15.60:0/1921679813 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5dbfff640 1 -- 172.21.15.60:0/1921679813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 msgr2=0x5569cc5b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5dbfff640 1 -- 172.21.15.60:0/1921679813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc54df70 msgr2=0x5569cc5b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc54df70 0x5569cc5b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5dbfff640 1 -- 172.21.15.60:0/1921679813 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569cc1978b0 con 0x5569cc587130 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 0x5569cc5afaf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00dc20 tx=0x7fa5cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a3dd286f8e7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5c37fe640 1 -- 172.21.15.60:0/1921679813 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc019070 con 0x5569cc587130 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5c37fe640 1 -- 172.21.15.60:0/1921679813 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc01a040 con 0x5569cc587130 2024-04-25T08:32:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.516+0000 7fa5c37fe640 1 -- 172.21.15.60:0/1921679813 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00de90 con 0x5569cc587130 2024-04-25T08:32:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1921679813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 msgr2=0x5569cc5afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 0x5569cc5afaf0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00dc20 tx=0x7fa5cc013000 comp rx=0 tx=0).stop 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1921679813 shutdown_connections 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc5adaf0 0x5569cc5b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc587130 0x5569cc5afaf0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/1921679813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc54df70 0x5569cc5b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1921679813 >> 172.21.15.60:0/1921679813 conn(0x5569cc584c60 msgr2=0x5569cc5ee230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1921679813 shutdown_connections 2024-04-25T08:32:25.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 -- 172.21.15.60:0/1921679813 wait complete. 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 -- start start 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.517+0000 7fa5ebc17740 1 Processor -- start 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc54df70 0x5569cc5de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5de750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 0x5569cc639840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569cc5ccde0 con 0x5569cc5adaf0 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41feef0 con 0x5569cc587130 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569cc5c5cd0 con 0x5569cc54df70 2024-04-25T08:32:25.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5ebc17740 10 client.0 _ll_register_callbacks cb 0x7fa5d40540b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 0x5569cc639840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5e2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 0x5569cc639840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45118/0 (socket says 172.21.15.60:45118) 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/4130634055 learned_addr learned my addr 172.21.15.60:0/4130634055 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5dbfff640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.518+0000 7fa5e259c640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc54df70 0x5569cc5de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc54df70 msgr2=0x5569cc5de210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc54df70 0x5569cc5de210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 msgr2=0x5569cc5de750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5de750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e2d9d640 1 -- 172.21.15.60:0/4130634055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569cc636260 con 0x5569cc5adaf0 2024-04-25T08:32:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e259c640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc54df70 0x5569cc5de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e2d9d640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 0x5569cc639840 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc007030 tx=0x7fa5dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294e18c7e7422f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:25.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5c17fa640 1 -- 172.21.15.60:0/4130634055 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc009450 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5c17fa640 1 -- 172.21.15.60:0/4130634055 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5dc003ba0 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5cc01a040 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5c17fa640 1 -- 172.21.15.60:0/4130634055 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5dc05e050 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 10 client.6126 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5569cc5ac270 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5c17fa640 1 -- 172.21.15.60:0/4130634055 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc01a040 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5c17fa640 1 -- 172.21.15.60:0/4130634055 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa5dc061020 con 0x5569cc5adaf0 2024-04-25T08:32:25.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 10 client.6126 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 10 client.6126 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 20 client.6126 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5ccf60 0x5569cc639d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 4 client.6126 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x5569cc4f16f0 con 0x5569cc5ccf60 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.519+0000 7fa5e259c640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5ccf60 0x5569cc639d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5c17fa640 10 client.6126 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5e259c640 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5ccf60 0x5569cc639d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748b598aff1bc2f8 server_cookie=16e78abf1aee731 in_seq=0 out_seq=0 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5c17fa640 1 -- 172.21.15.60:0/4130634055 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5569cc4f16f0 con 0x5569cc5ccf60 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5c17fa640 10 client.6126 handle_command_reply: tid=0 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5ebc17740 1 client.6126 shutdown 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 msgr2=0x5569cc639840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.520+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 0x5569cc639840 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc007030 tx=0x7fa5dc004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 shutdown_connections 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5569cc5adaf0 0x5569cc639840 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569cc587130 0x5569cc5de750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5569cc5ccf60 0x5569cc639d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 --2- 172.21.15.60:0/4130634055 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5569cc54df70 0x5569cc5de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 >> 172.21.15.60:0/4130634055 conn(0x5569cc584c60 msgr2=0x5569cc5acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5c17fa640 0 client.6126 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 shutdown_connections 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 1 -- 172.21.15.60:0/4130634055 wait complete. 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5ebc17740 20 client.6126 trim_cache size 0 max 16384 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "f2166c11-2568-4c63-95b5-d52bcc34f34e", 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:25.527 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa5c406b4d0 msgr2=0x7fa5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa5c406b4d0 0x7fa5c406d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40b0c60 tx=0x7fa5cc008040 comp rx=0 tx=0).stop 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 msgr2=0x7fa5e40cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 0x7fa5e40cd6e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c390 tx=0x7fa5d400c850 comp rx=0 tx=0).stop 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 shutdown_connections 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5e41452a0 0x7fa5e40b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa5c406b4d0 0x7fa5c406d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5e4144d50 0x7fa5e40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 --2- 172.21.15.60:0/2470433851 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5e41446d0 0x7fa5e40cd6e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 >> 172.21.15.60:0/2470433851 conn(0x7fa5e406abe0 msgr2=0x7fa5e4143620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 shutdown_connections 2024-04-25T08:32:25.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.521+0000 7fa5e8e41640 1 -- 172.21.15.60:0/2470433851 wait complete. 2024-04-25T08:32:25.530 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'f2166c11-2568-4c63-95b5-d52bcc34f34e', 'mode': 'asynchronous'}' 2024-04-25T08:32:25.530 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T08:32:25.530 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:32:25.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.635+0000 7f33914db640 1 Processor -- start 2024-04-25T08:32:25.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.635+0000 7f33914db640 1 -- start start 2024-04-25T08:32:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f33914db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c15db70 0x7f338c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f33914db640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f338c0b5c30 0x7f338c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f33914db640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c162690 con 0x7f338c0b6cb0 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f33914db640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c0ceee0 con 0x7f338c0b5c30 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f33914db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0d1d10 con 0x7f338c15db70 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f338c0b5c30 0x7f338c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f338c0b6cb0 0x7f338c0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45538/0 (socket says 172.21.15.60:45538) 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338a7fc640 1 -- 172.21.15.60:0/443977569 learned_addr learned my addr 172.21.15.60:0/443977569 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338affd640 1 --2- 172.21.15.60:0/443977569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c15db70 0x7f338c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338b7fe640 1 -- 172.21.15.60:0/443977569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c15db70 msgr2=0x7f338c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338b7fe640 1 --2- 172.21.15.60:0/443977569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c15db70 0x7f338c162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338b7fe640 1 -- 172.21.15.60:0/443977569 >> v1:172.21.15.60:6789/0 conn(0x7f338c0b6cb0 legacy=0x7f338c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f338b7fe640 1 -- 172.21.15.60:0/443977569 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c0d1ff0 con 0x7f338c0b5c30 2024-04-25T08:32:25.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.636+0000 7f3389ffb640 1 -- 172.21.15.60:0/443977569 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802915413 0 0) 0x7f338c162690 con 0x7f338c0b6cb0 2024-04-25T08:32:25.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.637+0000 7f338b7fe640 1 --2- 172.21.15.60:0/443977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c0b5c30 0x7f338c0d27f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f337c009d90 tx=0x7f337c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df213f2402ef21c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.637+0000 7f3389ffb640 1 -- 172.21.15.60:0/443977569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c031600 con 0x7f338c0b5c30 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.637+0000 7f3389ffb640 1 -- 172.21.15.60:0/443977569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c0317a0 con 0x7f338c0b5c30 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.637+0000 7f3389ffb640 1 -- 172.21.15.60:0/443977569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c031b70 con 0x7f338c0b5c30 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 -- 172.21.15.60:0/443977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c0b5c30 msgr2=0x7f338c0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 --2- 172.21.15.60:0/443977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c0b5c30 0x7f338c0d27f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f337c009d90 tx=0x7f337c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 -- 172.21.15.60:0/443977569 shutdown_connections 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 --2- 172.21.15.60:0/443977569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c0b5c30 0x7f338c0d27f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 --2- 172.21.15.60:0/443977569 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c15db70 0x7f338c162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 -- 172.21.15.60:0/443977569 >> 172.21.15.60:0/443977569 conn(0x7f338c0d0520 msgr2=0x7f338c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 -- 172.21.15.60:0/443977569 shutdown_connections 2024-04-25T08:32:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.638+0000 7f33914db640 1 -- 172.21.15.60:0/443977569 wait complete. 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.639+0000 7f33914db640 1 Processor -- start 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.639+0000 7f33914db640 1 -- start start 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f33914db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c0b5c30 0x7f338c160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f33914db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338c0b6cb0 0x7f338c15f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f33914db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 0x7f338c1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f33914db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0ce530 con 0x7f338c15db70 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f33914db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c15df50 con 0x7f338c0b6cb0 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f33914db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0cf1c0 con 0x7f338c0b5c30 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 0x7f338c1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 0x7f338c1f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45138/0 (socket says 172.21.15.60:45138) 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 -- 172.21.15.60:0/4258732069 learned_addr learned my addr 172.21.15.60:0/4258732069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338affd640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c0b5c30 0x7f338c160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338a7fc640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338c0b6cb0 0x7f338c15f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 -- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c0b5c30 msgr2=0x7f338c160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c0b5c30 0x7f338c160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 -- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338c0b6cb0 msgr2=0x7f338c15f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338c0b6cb0 0x7f338c15f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 -- 172.21.15.60:0/4258732069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c15e560 con 0x7f338c15db70 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.640+0000 7f338b7fe640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 0x7f338c1f3d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f337c009d90 tx=0x7f337c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.641+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c040070 con 0x7f338c15db70 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.641+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c030b60 con 0x7f338c15db70 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.641+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c009450 con 0x7f338c15db70 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.641+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c0cc3c0 con 0x7f338c15db70 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.641+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c0cc980 con 0x7f338c15db70 2024-04-25T08:32:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.642+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c073990 con 0x7f338c15db70 2024-04-25T08:32:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.642+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f337c039040 con 0x7f338c15db70 2024-04-25T08:32:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.643+0000 7f33837fe640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f336c06b4d0 0x7f336c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.643+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f337c0ab5a0 con 0x7f338c15db70 2024-04-25T08:32:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.643+0000 7f338affd640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f336c06b4d0 0x7f336c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.643+0000 7f338affd640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f336c06b4d0 0x7f336c06d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3384009a30 tx=0x7f3384008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.647+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f337c07a320 con 0x7f338c15db70 2024-04-25T08:32:25.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.866+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f338c0d1820 con 0x7f338c15db70 2024-04-25T08:32:25.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.867+0000 7f33837fe640 1 -- 172.21.15.60:0/4258732069 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f337c079c20 con 0x7f338c15db70 2024-04-25T08:32:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:25.863 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:25.864 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f336c06b4d0 msgr2=0x7f336c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f336c06b4d0 0x7f336c06d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3384009a30 tx=0x7f3384008040 comp rx=0 tx=0).stop 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 msgr2=0x7f338c1f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 0x7f338c1f3d90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f337c009d90 tx=0x7f337c004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 shutdown_connections 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f338c15db70 0x7f338c1f3d90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f336c06b4d0 0x7f336c06d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f338c0b6cb0 0x7f338c15f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 --2- 172.21.15.60:0/4258732069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f338c0b5c30 0x7f338c160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 >> 172.21.15.60:0/4258732069 conn(0x7f338c0d0520 msgr2=0x7f338c156080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 shutdown_connections 2024-04-25T08:32:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:25.869+0000 7f33914db640 1 -- 172.21.15.60:0/4258732069 wait complete. 2024-04-25T08:32:25.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T08:32:25.985 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:25.985 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:25.985 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:25.994 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:25.994 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T08:32:25.995 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:25.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T08:32:26.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.103+0000 7fc96d9a9640 1 Processor -- start 2024-04-25T08:32:26.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.103+0000 7fc96d9a9640 1 -- start start 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc96d9a9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc9680ced80 0x7fc9680d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc96d9a9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc968161710 0x7fc9680d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc96d9a9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96815e000 con 0x7fc96815dc20 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc96d9a9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680d1de0 con 0x7fc9680ced80 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc96d9a9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc968161c60 con 0x7fc968161710 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc966ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc96815dc20 0x7fc9680ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45554/0 (socket says 172.21.15.60:45554) 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc966ffd640 1 -- 172.21.15.60:0/2832315737 learned_addr learned my addr 172.21.15.60:0/2832315737 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.104+0000 7fc95ffff640 1 --2- 172.21.15.60:0/2832315737 >> v2:172.21.15.60:3300/0 conn(0x7fc9680ced80 0x7fc9680d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc9667fc640 1 -- 172.21.15.60:0/2832315737 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560170185 0 0) 0x7fc96815e000 con 0x7fc96815dc20 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc9667fc640 1 -- 172.21.15.60:0/2832315737 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc954003520 con 0x7fc96815dc20 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc9677fe640 1 --2- 172.21.15.60:0/2832315737 >> v2:172.21.15.100:3300/0 conn(0x7fc968161710 0x7fc9680d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc95ffff640 1 -- 172.21.15.60:0/2832315737 >> v2:172.21.15.100:3300/0 conn(0x7fc968161710 msgr2=0x7fc9680d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc95ffff640 1 --2- 172.21.15.60:0/2832315737 >> v2:172.21.15.100:3300/0 conn(0x7fc968161710 0x7fc9680d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc95ffff640 1 -- 172.21.15.60:0/2832315737 >> v1:172.21.15.60:6789/0 conn(0x7fc96815dc20 legacy=0x7fc9680ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc95ffff640 1 -- 172.21.15.60:0/2832315737 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968161f40 con 0x7fc9680ced80 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc9677fe640 1 --2- 172.21.15.60:0/2832315737 >> v2:172.21.15.100:3300/0 conn(0x7fc968161710 0x7fc9680d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.105+0000 7fc95ffff640 1 --2- 172.21.15.60:0/2832315737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680ced80 0x7fc9680d22e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc9680b6570 tx=0x7fc95002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ffeb805b7f9d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc9667fc640 1 -- 172.21.15.60:0/2832315737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950037070 con 0x7fc9680ced80 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc9667fc640 1 -- 172.21.15.60:0/2832315737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950032440 con 0x7fc9680ced80 2024-04-25T08:32:26.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc9667fc640 1 -- 172.21.15.60:0/2832315737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950032810 con 0x7fc9680ced80 2024-04-25T08:32:26.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 -- 172.21.15.60:0/2832315737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680ced80 msgr2=0x7fc9680d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/2832315737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680ced80 0x7fc9680d22e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc9680b6570 tx=0x7fc95002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 -- 172.21.15.60:0/2832315737 shutdown_connections 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/2832315737 >> v2:172.21.15.100:3300/0 conn(0x7fc968161710 0x7fc9680d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/2832315737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9680ced80 0x7fc9680d22e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 -- 172.21.15.60:0/2832315737 >> 172.21.15.60:0/2832315737 conn(0x7fc9680d0420 msgr2=0x7fc9680d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 -- 172.21.15.60:0/2832315737 shutdown_connections 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.106+0000 7fc96d9a9640 1 -- 172.21.15.60:0/2832315737 wait complete. 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.107+0000 7fc96d9a9640 1 Processor -- start 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.107+0000 7fc96d9a9640 1 -- start start 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc96d9a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9680ced80 0x7fc968155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc96d9a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 0x7fc96815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc96d9a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc968161710 0x7fc968155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc96d9a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680b5e50 con 0x7fc96815dc20 2024-04-25T08:32:26.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc96d9a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968162500 con 0x7fc9680ced80 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc96d9a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680b5fd0 con 0x7fc968161710 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc95ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 0x7fc96815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc95ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 0x7fc96815ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45168/0 (socket says 172.21.15.60:45168) 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.108+0000 7fc95ffff640 1 -- 172.21.15.60:0/3703788115 learned_addr learned my addr 172.21.15.60:0/3703788115 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc966ffd640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9680ced80 0x7fc968155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc9677fe640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc968161710 0x7fc968155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc95ffff640 1 -- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc968161710 msgr2=0x7fc968155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc95ffff640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc968161710 0x7fc968155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc95ffff640 1 -- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9680ced80 msgr2=0x7fc968155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc95ffff640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9680ced80 0x7fc968155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc95ffff640 1 -- 172.21.15.60:0/3703788115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680c7770 con 0x7fc96815dc20 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc966ffd640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9680ced80 0x7fc968155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc95ffff640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 0x7fc96815ec00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc950000c00 tx=0x7fc9500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc964ff9640 1 -- 172.21.15.60:0/3703788115 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950037040 con 0x7fc96815dc20 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc964ff9640 1 -- 172.21.15.60:0/3703788115 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500304c0 con 0x7fc96815dc20 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9680c7a50 con 0x7fc96815dc20 2024-04-25T08:32:26.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc964ff9640 1 -- 172.21.15.60:0/3703788115 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950030890 con 0x7fc96815dc20 2024-04-25T08:32:26.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.109+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9680c8010 con 0x7fc96815dc20 2024-04-25T08:32:26.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.110+0000 7fc964ff9640 1 -- 172.21.15.60:0/3703788115 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc950044020 con 0x7fc96815dc20 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.111+0000 7fc964ff9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc94806b500 0x7fc94806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.111+0000 7fc964ff9640 1 -- 172.21.15.60:0/3703788115 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc9500aa6b0 con 0x7fc96815dc20 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.111+0000 7fc966ffd640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc94806b500 0x7fc94806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.112+0000 7fc966ffd640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc94806b500 0x7fc94806d9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc958005880 tx=0x7fc958009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.112+0000 7fc970789740 1 Processor -- start 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.112+0000 7fc970789740 1 -- start start 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d723c0 0x55cca5d74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc970789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 0x55cca5d48450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc970789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca5cada50 con 0x55cca5d44c40 2024-04-25T08:32:26.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca5c56d40 con 0x55cca5d723c0 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cca5ca9af0 con 0x55cca5d71fe0 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 0x55cca5d48450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 0x55cca5d48450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45170/0 (socket says 172.21.15.60:45170) 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 -- 172.21.15.60:0/2904010741 learned_addr learned my addr 172.21.15.60:0/2904010741 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc966ffd640 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc95ffff640 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d723c0 0x55cca5d74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 -- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 msgr2=0x55cca5d747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 -- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d723c0 msgr2=0x55cca5d74d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d723c0 0x55cca5d74d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.113+0000 7fc9677fe640 1 -- 172.21.15.60:0/2904010741 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cca5a8e690 con 0x55cca5d44c40 2024-04-25T08:32:26.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.114+0000 7fc9677fe640 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 0x55cca5d48450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc960007ee0 tx=0x7fc96000d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29c52baff884f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.114+0000 7fc95d7fa640 1 -- 172.21.15.60:0/2904010741 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000f860 con 0x55cca5d44c40 2024-04-25T08:32:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.114+0000 7fc95d7fa640 1 -- 172.21.15.60:0/2904010741 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96000fa00 con 0x55cca5d44c40 2024-04-25T08:32:26.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.114+0000 7fc95d7fa640 1 -- 172.21.15.60:0/2904010741 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000fe10 con 0x55cca5d44c40 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 -- 172.21.15.60:0/2904010741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 msgr2=0x55cca5d48450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 0x55cca5d48450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc960007ee0 tx=0x7fc96000d740 comp rx=0 tx=0).stop 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 -- 172.21.15.60:0/2904010741 shutdown_connections 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d44c40 0x55cca5d48450 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d723c0 0x55cca5d74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 --2- 172.21.15.60:0/2904010741 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 -- 172.21.15.60:0/2904010741 >> 172.21.15.60:0/2904010741 conn(0x55cca5d46eb0 msgr2=0x55cca5d8f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 -- 172.21.15.60:0/2904010741 shutdown_connections 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 -- 172.21.15.60:0/2904010741 wait complete. 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 -- start start 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.115+0000 7fc970789740 1 Processor -- start 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d91c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5d76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d76cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca5d4a060 con 0x55cca5d723c0 2024-04-25T08:32:26.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca5c4c340 con 0x55cca5d6bdc0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cca5d4a340 con 0x55cca5d71fe0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc970789740 10 client.0 _ll_register_callbacks cb 0x55cca5c69cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc95ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5d76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc95ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5d76790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45174/0 (socket says 172.21.15.60:45174) 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc95ffff640 1 -- 172.21.15.60:0/643136235 learned_addr learned my addr 172.21.15.60:0/643136235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc9677fe640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d76cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.116+0000 7fc966ffd640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d91c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc95ffff640 1 -- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 msgr2=0x55cca5d91c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc95ffff640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d91c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc95ffff640 1 -- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 msgr2=0x55cca5d76cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc95ffff640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d76cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc95ffff640 1 -- 172.21.15.60:0/643136235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cca5aeee20 con 0x55cca5d723c0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc95ffff640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5d76790 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc95002a790 tx=0x7fc950079440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdbc8f34560855e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc9467fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc9467fc640 1 -- 172.21.15.60:0/643136235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950081310 con 0x55cca5d723c0 2024-04-25T08:32:26.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc9467fc640 1 -- 172.21.15.60:0/643136235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc950075270 con 0x55cca5d723c0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc96000fa00 con 0x55cca5d723c0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc9467fc640 1 -- 172.21.15.60:0/643136235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc950080e80 con 0x55cca5d723c0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc970789740 10 client.6150 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55cca5981f80 con 0x55cca5d723c0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc9467fc640 1 -- 172.21.15.60:0/643136235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96000fa00 con 0x55cca5d723c0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc9467fc640 1 -- 172.21.15.60:0/643136235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc95003a070 con 0x55cca5d723c0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.117+0000 7fc970789740 10 client.6150 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc970789740 10 client.6150 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc970789740 20 client.6150 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc970789740 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5d6f490 0x55cca5dad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc970789740 4 client.6150 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cca5cbaf90 con 0x55cca5d6f490 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc966ffd640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5d6f490 0x55cca5dad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc9467fc640 10 client.6150 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.118+0000 7fc966ffd640 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5d6f490 0x55cca5dad470 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50125bf30a5d45f9 server_cookie=33c0c22b1914d9d0 in_seq=0 out_seq=0 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc9467fc640 1 -- 172.21.15.60:0/643136235 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cca5cbaf90 con 0x55cca5d6f490 2024-04-25T08:32:26.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc9467fc640 10 client.6150 handle_command_reply: tid=0 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 client.6150 shutdown 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 msgr2=0x55cca5d76790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5d76790 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc95002a790 tx=0x7fc950079440 comp rx=0 tx=0).stop 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 shutdown_connections 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d76cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5d76790 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5d6f490 0x55cca5dad470 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 --2- 172.21.15.60:0/643136235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d71fe0 0x55cca5d91c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 >> 172.21.15.60:0/643136235 conn(0x55cca5d44c40 msgr2=0x55cca5d71340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc9467fc640 0 client.6150 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 shutdown_connections 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 1 -- 172.21.15.60:0/643136235 wait complete. 2024-04-25T08:32:26.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.119+0000 7fc970789740 20 client.6150 trim_cache size 0 max 16384 2024-04-25T08:32:26.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.135+0000 7fc970789740 1 Processor -- start 2024-04-25T08:32:26.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.135+0000 7fc970789740 1 -- start start 2024-04-25T08:32:26.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d43f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc970789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 0x55cca5df4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d723c0 0x55cca5d75740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc970789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968200260 con 0x55cca5d71fe0 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9681ff460 con 0x55cca5d6bdc0 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9681fe8e0 con 0x55cca5d723c0 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 0x55cca5df4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 0x55cca5df4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45182/0 (socket says 172.21.15.60:45182) 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 -- 172.21.15.60:0/1135683144 learned_addr learned my addr 172.21.15.60:0/1135683144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc9677fe640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d723c0 0x55cca5d75740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc966ffd640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d43f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 -- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d723c0 msgr2=0x55cca5d75740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d723c0 0x55cca5d75740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 -- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 msgr2=0x55cca5d43f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d43f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 -- 172.21.15.60:0/1135683144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cca5d934e0 con 0x55cca5d71fe0 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc9677fe640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d723c0 0x55cca5d75740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc966ffd640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d43f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95ffff640 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 0x55cca5df4f30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc95002a1e0 tx=0x7fc950008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d49d439bf1653e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95cff9640 1 -- 172.21.15.60:0/1135683144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950094070 con 0x55cca5d71fe0 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95cff9640 1 -- 172.21.15.60:0/1135683144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950081070 con 0x55cca5d71fe0 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.136+0000 7fc95cff9640 1 -- 172.21.15.60:0/1135683144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950081310 con 0x55cca5d71fe0 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 -- 172.21.15.60:0/1135683144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 msgr2=0x55cca5df4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 0x55cca5df4f30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc95002a1e0 tx=0x7fc950008000 comp rx=0 tx=0).stop 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 -- 172.21.15.60:0/1135683144 shutdown_connections 2024-04-25T08:32:26.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d723c0 0x55cca5d75740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d71fe0 0x55cca5df4f30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 --2- 172.21.15.60:0/1135683144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d6bdc0 0x55cca5d43f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 -- 172.21.15.60:0/1135683144 >> 172.21.15.60:0/1135683144 conn(0x55cca5d44c40 msgr2=0x55cca5d747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 -- 172.21.15.60:0/1135683144 shutdown_connections 2024-04-25T08:32:26.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.137+0000 7fc970789740 1 -- 172.21.15.60:0/1135683144 wait complete. 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.138+0000 7fc970789740 1 -- start start 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.138+0000 7fc970789740 1 Processor -- start 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d6bdc0 0x55cca5d85b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d71fe0 0x55cca5de1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5db1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca5d4a740 con 0x55cca5d723c0 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9681fee60 con 0x55cca5d71fe0 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cca5d6b750 con 0x55cca5d6bdc0 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc970789740 10 client.0 _ll_register_callbacks cb 0x7fc950009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5db1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5db1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45190/0 (socket says 172.21.15.60:45190) 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 -- 172.21.15.60:0/858899799 learned_addr learned my addr 172.21.15.60:0/858899799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc95ffff640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d71fe0 0x55cca5de1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc966ffd640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d6bdc0 0x55cca5d85b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 -- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d6bdc0 msgr2=0x55cca5d85b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d6bdc0 0x55cca5d85b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 -- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d71fe0 msgr2=0x55cca5de1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d71fe0 0x55cca5de1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 -- 172.21.15.60:0/858899799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cca5d6a6b0 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9677fe640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5db1490 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc960007fd0 tx=0x7fc960010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d1601e86b5be66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9467fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.139+0000 7fc9467fc640 1 -- 172.21.15.60:0/858899799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960029070 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc9467fc640 1 -- 172.21.15.60:0/858899799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc960018480 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc950081070 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc9467fc640 1 -- 172.21.15.60:0/858899799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc960007840 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 10 client.6156 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55cca5d86480 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc9467fc640 1 -- 172.21.15.60:0/858899799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950081070 con 0x55cca5d723c0 2024-04-25T08:32:26.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc9467fc640 1 -- 172.21.15.60:0/858899799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc96000c040 con 0x55cca5d723c0 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 10 client.6156 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 10 client.6156 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 20 client.6156 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5db1b60 0x55cca5dfb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 4 client.6156 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55cca5cb52e0 con 0x55cca5db1b60 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc966ffd640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5db1b60 0x55cca5dfb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc9467fc640 10 client.6156 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.140+0000 7fc966ffd640 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5db1b60 0x55cca5dfb4a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ecc738c0668947 server_cookie=88ae8f77d5e1f6c9 in_seq=0 out_seq=0 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc9467fc640 1 -- 172.21.15.60:0/858899799 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cca5cb52e0 con 0x55cca5db1b60 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc9467fc640 10 client.6156 handle_command_reply: tid=0 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 client.6156 shutdown 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 msgr2=0x55cca5db1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5db1490 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc960007fd0 tx=0x7fc960010000 comp rx=0 tx=0).stop 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 shutdown_connections 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cca5d723c0 0x55cca5db1490 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cca5d71fe0 0x55cca5de1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55cca5db1b60 0x55cca5dfb4a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 --2- 172.21.15.60:0/858899799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cca5d6bdc0 0x55cca5d85b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc9467fc640 0 client.6156 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 >> 172.21.15.60:0/858899799 conn(0x55cca5d44c40 msgr2=0x55cca5d43d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 shutdown_connections 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 1 -- 172.21.15.60:0/858899799 wait complete. 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.141+0000 7fc970789740 20 client.6156 trim_cache size 0 max 16384 2024-04-25T08:32:26.143 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:26.144 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:26.144 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b34414b0-dcf0-42a8-a5bd-0feed5c216b0", 2024-04-25T08:32:26.144 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:26.144 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc94806b500 msgr2=0x7fc94806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc94806b500 0x7fc94806d9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc958005880 tx=0x7fc958009040 comp rx=0 tx=0).stop 2024-04-25T08:32:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 msgr2=0x7fc96815ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 0x7fc96815ec00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc950000c00 tx=0x7fc9500046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 shutdown_connections 2024-04-25T08:32:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc968161710 0x7fc968155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc96815dc20 0x7fc96815ec00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc94806b500 0x7fc94806d9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 --2- 172.21.15.60:0/3703788115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9680ced80 0x7fc968155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 >> 172.21.15.60:0/3703788115 conn(0x7fc9680d0420 msgr2=0x7fc9680d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 shutdown_connections 2024-04-25T08:32:26.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.142+0000 7fc96d9a9640 1 -- 172.21.15.60:0/3703788115 wait complete. 2024-04-25T08:32:26.150 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'b34414b0-dcf0-42a8-a5bd-0feed5c216b0', 'mode': 'asynchronous'}' 2024-04-25T08:32:26.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:26.150 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T08:32:26.224 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T08:32:26.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.369+0000 7f03c0253640 1 Processor -- start 2024-04-25T08:32:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.370+0000 7f03c0253640 1 -- start start 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.370+0000 7f03c0253640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03b80d1120 0x7f03b809a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.370+0000 7f03c0253640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b806c220 con 0x7f03b806ce10 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.370+0000 7f03c0253640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b80d0080 con 0x7f03b815dc70 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.370+0000 7f03c0253640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b80d0c90 con 0x7f03b80d1120 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03be7f0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f03b80d1120 0x7f03b809a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03bd7ee640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f03b806ce10 0x7f03b80cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45560/0 (socket says 172.21.15.60:45560) 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03bd7ee640 1 -- 172.21.15.60:0/3988167540 learned_addr learned my addr 172.21.15.60:0/3988167540 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03be7f0640 1 -- 172.21.15.60:0/3988167540 >> v1:172.21.15.100:6789/0 conn(0x7f03b815dc70 legacy=0x7f03b806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03be7f0640 1 -- 172.21.15.60:0/3988167540 >> v1:172.21.15.60:6789/0 conn(0x7f03b806ce10 legacy=0x7f03b80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03be7f0640 1 -- 172.21.15.60:0/3988167540 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80d0360 con 0x7f03b80d1120 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.371+0000 7f03be7f0640 1 --2- 172.21.15.60:0/3988167540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b809a450 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009d90 tx=0x7f03b002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3bcb92a27831d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03bcfed640 1 -- 172.21.15.60:0/3988167540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0007740 con 0x7f03b80d1120 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03bcfed640 1 -- 172.21.15.60:0/3988167540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b00078e0 con 0x7f03b80d1120 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03bcfed640 1 -- 172.21.15.60:0/3988167540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0007cb0 con 0x7f03b80d1120 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 -- 172.21.15.60:0/3988167540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 msgr2=0x7f03b809a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 --2- 172.21.15.60:0/3988167540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b809a450 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009d90 tx=0x7f03b002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 -- 172.21.15.60:0/3988167540 shutdown_connections 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 --2- 172.21.15.60:0/3988167540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b809a450 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 -- 172.21.15.60:0/3988167540 >> 172.21.15.60:0/3988167540 conn(0x7f03b80b3270 msgr2=0x7f03b80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 -- 172.21.15.60:0/3988167540 shutdown_connections 2024-04-25T08:32:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.372+0000 7f03c0253640 1 -- 172.21.15.60:0/3988167540 wait complete. 2024-04-25T08:32:26.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.373+0000 7f03c0253640 1 Processor -- start 2024-04-25T08:32:26.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 -- start start 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03b806ce10 0x7f03b814cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03b815dc70 0x7f03b80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b815e050 con 0x7f03b80d1120 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80d0920 con 0x7f03b806ce10 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03c0253640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80cf9c0 con 0x7f03b815dc70 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03bd7ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03bd7ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b8158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45200/0 (socket says 172.21.15.60:45200) 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.374+0000 7f03bd7ee640 1 -- 172.21.15.60:0/15143582 learned_addr learned my addr 172.21.15.60:0/15143582 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03be7f0640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03b815dc70 0x7f03b80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bdfef640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03b806ce10 0x7f03b814cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bd7ee640 1 -- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03b815dc70 msgr2=0x7f03b80b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bd7ee640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03b815dc70 0x7f03b80b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bd7ee640 1 -- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03b806ce10 msgr2=0x7f03b814cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bd7ee640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03b806ce10 0x7f03b814cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bd7ee640 1 -- 172.21.15.60:0/15143582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80c5430 con 0x7f03b80d1120 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03bd7ee640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b8158170 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c4d0 tx=0x7f03a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8017070 con 0x7f03b80d1120 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8005ce0 con 0x7f03b80d1120 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8007270 con 0x7f03b80d1120 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b80c5710 con 0x7f03b80d1120 2024-04-25T08:32:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.375+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b81586b0 con 0x7f03b80d1120 2024-04-25T08:32:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.376+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b8073990 con 0x7f03b80d1120 2024-04-25T08:32:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.376+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03a801a070 con 0x7f03b80d1120 2024-04-25T08:32:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.377+0000 7f03ae7fc640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f039c06b500 0x7f039c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.377+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03a8089610 con 0x7f03b80d1120 2024-04-25T08:32:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.377+0000 7f03bdfef640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f039c06b500 0x7f039c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.378+0000 7f03bdfef640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f039c06b500 0x7f039c06d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001500 tx=0x7f03b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.381+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03a8057cb0 con 0x7f03b80d1120 2024-04-25T08:32:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.601+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03b80cfdc0 con 0x7f03b80d1120 2024-04-25T08:32:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.601+0000 7f03ae7fc640 1 -- 172.21.15.60:0/15143582 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f03a805bd50 con 0x7f03b80d1120 2024-04-25T08:32:26.598 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:26.598 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f039c06b500 msgr2=0x7f039c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f039c06b500 0x7f039c06d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001500 tx=0x7f03b4009040 comp rx=0 tx=0).stop 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 msgr2=0x7f03b8158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b8158170 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c4d0 tx=0x7f03a800c990 comp rx=0 tx=0).stop 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 shutdown_connections 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f03b815dc70 0x7f03b80b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03b80d1120 0x7f03b8158170 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f039c06b500 0x7f039c06d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 --2- 172.21.15.60:0/15143582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03b806ce10 0x7f03b814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 >> 172.21.15.60:0/15143582 conn(0x7f03b80b3270 msgr2=0x7f03b80cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 shutdown_connections 2024-04-25T08:32:26.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.603+0000 7f03c0253640 1 -- 172.21.15.60:0/15143582 wait complete. 2024-04-25T08:32:26.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T08:32:26.720 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:26.720 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:26.729 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:26.730 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:26.730 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:26.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:26.745 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T08:32:26.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:32:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:32:26.811 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814745 s, 129 MB/s 2024-04-25T08:32:26.811 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T08:32:26.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.956+0000 7ffb7dc58640 1 Processor -- start 2024-04-25T08:32:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.956+0000 7ffb7dc58640 1 -- start start 2024-04-25T08:32:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb7dc58640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffb780d2e90 0x7ffb780d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb7dc58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb780d0900 0x7ffb780b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb7dc58640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb780b9940 con 0x7ffb7815dd30 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb7dc58640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb780d0d40 con 0x7ffb780d2e90 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb7dc58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d1df0 con 0x7ffb780d0900 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb777fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb7815dd30 0x7ffb780b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:45568/0 (socket says 172.21.15.60:45568) 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb777fe640 1 -- 172.21.15.60:0/365296708 learned_addr learned my addr 172.21.15.60:0/365296708 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118549163 0 0) 0x7ffb780b9940 con 0x7ffb7815dd30 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb60003520 con 0x7ffb7815dd30 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb76ffd640 1 --2- 172.21.15.60:0/365296708 >> v2:172.21.15.100:3300/0 conn(0x7ffb780d2e90 0x7ffb780d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb77fff640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb780d0900 0x7ffb780b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 467988773 0 0) 0x7ffb60003520 con 0x7ffb7815dd30 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb780d0900 msgr2=0x7ffb780b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb780d0900 0x7ffb780b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 >> v2:172.21.15.100:3300/0 conn(0x7ffb780d2e90 msgr2=0x7ffb780d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 --2- 172.21.15.60:0/365296708 >> v2:172.21.15.100:3300/0 conn(0x7ffb780d2e90 0x7ffb780d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.957+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780d20d0 con 0x7ffb7815dd30 2024-04-25T08:32:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ffb68001670 con 0x7ffb7815dd30 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb76ffd640 1 --2- 172.21.15.60:0/365296708 >> v2:172.21.15.100:3300/0 conn(0x7ffb780d2e90 0x7ffb780d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 >> v1:172.21.15.60:6789/0 conn(0x7ffb7815dd30 legacy=0x7ffb780b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb600039b0 0x7ffb60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb60004800 0x7ffb60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d0d40 con 0x7ffb600039b0 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d1df0 con 0x7ffb600051c0 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6000c090 con 0x7ffb60004800 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb77fff640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb780d0900 0x7ffb780b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb777fe640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb600039b0 0x7ffb60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb76ffd640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb60004800 0x7ffb60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb77fff640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb777fe640 1 -- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb60004800 msgr2=0x7ffb60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb777fe640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb60004800 0x7ffb60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb777fe640 1 -- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb600051c0 msgr2=0x7ffb6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb777fe640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb777fe640 1 -- 172.21.15.60:0/365296708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60003520 con 0x7ffb600039b0 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.958+0000 7ffb77fff640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb600051c0 0x7ffb6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb777fe640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb600039b0 0x7ffb60003dd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800c290 tx=0x7ffb60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494d4c17be7dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68007d50 con 0x7ffb600039b0 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb68005af0 con 0x7ffb600039b0 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb767fc640 1 -- 172.21.15.60:0/365296708 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68007070 con 0x7ffb600039b0 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb7dc58640 1 -- 172.21.15.60:0/365296708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb600039b0 msgr2=0x7ffb60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/365296708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb600039b0 0x7ffb60003dd0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800c290 tx=0x7ffb60005830 comp rx=0 tx=0).stop 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.959+0000 7ffb77fff640 1 -- 172.21.15.60:0/365296708 reap_dead start 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.960+0000 7ffb7dc58640 1 -- 172.21.15.60:0/365296708 shutdown_connections 2024-04-25T08:32:26.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.960+0000 7ffb7dc58640 1 -- 172.21.15.60:0/365296708 >> 172.21.15.60:0/365296708 conn(0x7ffb78061c30 msgr2=0x7ffb7806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.960+0000 7ffb7dc58640 1 -- 172.21.15.60:0/365296708 shutdown_connections 2024-04-25T08:32:26.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.960+0000 7ffb7dc58640 1 -- 172.21.15.60:0/365296708 wait complete. 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.961+0000 7ffb7dc58640 1 Processor -- start 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.961+0000 7ffb7dc58640 1 -- start start 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb600039b0 0x7ffb780b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78154c20 0x7ffb780b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 0x7ffb780b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d2970 con 0x7ffb780b5600 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d2690 con 0x7ffb78154c20 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d1a10 con 0x7ffb600039b0 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 0x7ffb780b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 0x7ffb780b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45210/0 (socket says 172.21.15.60:45210) 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 -- 172.21.15.60:0/2028678571 learned_addr learned my addr 172.21.15.60:0/2028678571 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:26.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb777fe640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb600039b0 0x7ffb780b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb76ffd640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78154c20 0x7ffb780b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 -- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb600039b0 msgr2=0x7ffb780b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb600039b0 0x7ffb780b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 -- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78154c20 msgr2=0x7ffb780b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78154c20 0x7ffb780b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 -- 172.21.15.60:0/2028678571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78155400 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb77fff640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 0x7ffb780b3650 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb78073050 tx=0x7ffb70007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001a070 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb70015430 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb781556e0 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70015820 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.962+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb781550a0 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.963+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb78073990 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.964+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb7001d070 con 0x7ffb780b5600 2024-04-25T08:32:26.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.964+0000 7ffb5ffff640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffb6406b500 0x7ffb6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.964+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb70089e80 con 0x7ffb780b5600 2024-04-25T08:32:26.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.965+0000 7ffb777fe640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffb6406b500 0x7ffb6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.965+0000 7ffb777fe640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffb6406b500 0x7ffb6406d9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffb68033000 tx=0x7ffb68009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:26.969+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb70012070 con 0x7ffb780b5600 2024-04-25T08:32:27.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.188+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb780b7390 con 0x7ffb780b5600 2024-04-25T08:32:27.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.188+0000 7ffb5ffff640 1 -- 172.21.15.60:0/2028678571 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ffb70058c00 con 0x7ffb780b5600 2024-04-25T08:32:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:27.185 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:16.340529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:27.185 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 22 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffb6406b500 msgr2=0x7ffb6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffb6406b500 0x7ffb6406d9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffb68033000 tx=0x7ffb68009030 comp rx=0 tx=0).stop 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 msgr2=0x7ffb780b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 0x7ffb780b3650 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb78073050 tx=0x7ffb70007700 comp rx=0 tx=0).stop 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 shutdown_connections 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb780b5600 0x7ffb780b3650 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffb6406b500 0x7ffb6406d9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb78154c20 0x7ffb780b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 --2- 172.21.15.60:0/2028678571 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb600039b0 0x7ffb780b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 >> 172.21.15.60:0/2028678571 conn(0x7ffb78061c30 msgr2=0x7ffb7815e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 shutdown_connections 2024-04-25T08:32:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.190+0000 7ffb7dc58640 1 -- 172.21.15.60:0/2028678571 wait complete. 2024-04-25T08:32:27.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T08:32:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:27.308 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:27.308 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:27.318 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:27.318 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:32:27.319 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:32:27.319 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T08:32:27.319 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:32:27.319 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T08:32:27.319 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:32:27.386 INFO:teuthology.orchestra.run.smithi060.stdout:1099511630186 2024-04-25T08:32:27.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:27.395 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:27.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:32:27.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.542+0000 7f963e530640 1 Processor -- start 2024-04-25T08:32:27.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.542+0000 7f963e530640 1 -- start start 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963e530640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f963815b7d0 0x7f96380cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963e530640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963806c9e0 0x7f96380b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963e530640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963806cf50 con 0x7f96380b5620 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963e530640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96380ce530 con 0x7f963815b7d0 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963e530640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96380ce810 con 0x7f963806c9e0 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f9637fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f963815b7d0 0x7f96380cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963cacd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963806c9e0 0x7f96380b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f96377fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f96380b5620 0x7f963806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57074/0 (socket says 172.21.15.60:57074) 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f96377fe640 1 -- 172.21.15.60:0/879752296 learned_addr learned my addr 172.21.15.60:0/879752296 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f9637fff640 1 -- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963806c9e0 msgr2=0x7f96380b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f9637fff640 1 --2- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963806c9e0 0x7f96380b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f9637fff640 1 -- 172.21.15.60:0/879752296 >> v1:172.21.15.100:6789/0 conn(0x7f96380b5620 legacy=0x7f963806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f9637fff640 1 -- 172.21.15.60:0/879752296 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380ceaf0 con 0x7f963815b7d0 2024-04-25T08:32:27.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.543+0000 7f963cacd640 1 --2- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963806c9e0 0x7f96380b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.544+0000 7f9637fff640 1 --2- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963815b7d0 0x7f96380cf010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96200097a0 tx=0x7f962002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7fed579371c40c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.544+0000 7f9636ffd640 1 -- 172.21.15.60:0/879752296 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620037070 con 0x7f963815b7d0 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.544+0000 7f9636ffd640 1 -- 172.21.15.60:0/879752296 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620032440 con 0x7f963815b7d0 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.544+0000 7f9636ffd640 1 -- 172.21.15.60:0/879752296 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620032810 con 0x7f963815b7d0 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 -- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963815b7d0 msgr2=0x7f96380cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 --2- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963815b7d0 0x7f96380cf010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96200097a0 tx=0x7f962002a180 comp rx=0 tx=0).stop 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 -- 172.21.15.60:0/879752296 shutdown_connections 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 --2- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963806c9e0 0x7f96380b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 --2- 172.21.15.60:0/879752296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963815b7d0 0x7f96380cf010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 -- 172.21.15.60:0/879752296 >> 172.21.15.60:0/879752296 conn(0x7f96380d0800 msgr2=0x7f96380d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 -- 172.21.15.60:0/879752296 shutdown_connections 2024-04-25T08:32:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.545+0000 7f963e530640 1 -- 172.21.15.60:0/879752296 wait complete. 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.546+0000 7f963e530640 1 Processor -- start 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.546+0000 7f963e530640 1 -- start start 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963e530640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963806c9e0 0x7f9638148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963e530640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 0x7f96380b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963e530640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963815b7d0 0x7f96381f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963e530640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380cf6d0 con 0x7f96380b5620 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963e530640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380cf550 con 0x7f963806c9e0 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963e530640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963815bbb0 con 0x7f963815b7d0 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 0x7f96380b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 0x7f96380b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45226/0 (socket says 172.21.15.60:45226) 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 -- 172.21.15.60:0/3826092130 learned_addr learned my addr 172.21.15.60:0/3826092130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f9637fff640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963806c9e0 0x7f9638148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f963cacd640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963815b7d0 0x7f96381f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 -- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963815b7d0 msgr2=0x7f96381f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963815b7d0 0x7f96381f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 -- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963806c9e0 msgr2=0x7f9638148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963806c9e0 0x7f9638148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.547+0000 7f96377fe640 1 -- 172.21.15.60:0/3826092130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380b7540 con 0x7f96380b5620 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.548+0000 7f96377fe640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 0x7f96380b7000 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f962800c400 tx=0x7f962800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.548+0000 7f9634ff9640 1 -- 172.21.15.60:0/3826092130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628017070 con 0x7f96380b5620 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.548+0000 7f9634ff9640 1 -- 172.21.15.60:0/3826092130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9628005ce0 con 0x7f96380b5620 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.548+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96380b7810 con 0x7f96380b5620 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.548+0000 7f9634ff9640 1 -- 172.21.15.60:0/3826092130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628007310 con 0x7f96380b5620 2024-04-25T08:32:27.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.548+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96380b80b0 con 0x7f96380b5620 2024-04-25T08:32:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.549+0000 7f9634ff9640 1 -- 172.21.15.60:0/3826092130 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f962801a070 con 0x7f96380b5620 2024-04-25T08:32:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.550+0000 7f9634ff9640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f961806b4d0 0x7f961806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.550+0000 7f9634ff9640 1 -- 172.21.15.60:0/3826092130 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9628089710 con 0x7f96380b5620 2024-04-25T08:32:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.550+0000 7f9637fff640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f961806b4d0 0x7f961806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.551+0000 7f9637fff640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f961806b4d0 0x7f961806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9620034000 tx=0x7f962002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.551+0000 7f9641234740 1 Processor -- start 2024-04-25T08:32:27.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.551+0000 7f9641234740 1 -- start start 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9641234740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d8062072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d8062077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d8061d7700 0x55d8061daf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9641234740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d806140e00 con 0x55d806204aa0 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8060e9d20 con 0x55d806204e80 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d80613cea0 con 0x55d8061d7700 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d8062072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d8062072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45240/0 (socket says 172.21.15.60:45240) 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 -- 172.21.15.60:0/2299149512 learned_addr learned my addr 172.21.15.60:0/2299149512 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f96377fe640 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d8062077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f963cacd640 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d8061d7700 0x55d8061daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 -- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d8061d7700 msgr2=0x55d8061daf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d8061d7700 0x55d8061daf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 -- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 msgr2=0x55d8062077f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d8062077f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 -- 172.21.15.60:0/2299149512 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d80610bdf0 con 0x55d806204aa0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f96377fe640 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d8062077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f9637fff640 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d8062072b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9620035040 tx=0x7f962002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c3ed8f15342a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f962dffb640 1 -- 172.21.15.60:0/2299149512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962004e070 con 0x55d806204aa0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f962dffb640 1 -- 172.21.15.60:0/2299149512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620032c30 con 0x55d806204aa0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.552+0000 7f962dffb640 1 -- 172.21.15.60:0/2299149512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620037040 con 0x55d806204aa0 2024-04-25T08:32:27.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 -- 172.21.15.60:0/2299149512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 msgr2=0x55d8062072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d8062072b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9620035040 tx=0x7f962002c000 comp rx=0 tx=0).stop 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 -- 172.21.15.60:0/2299149512 shutdown_connections 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d8061d7700 0x55d8061daf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d8062077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 --2- 172.21.15.60:0/2299149512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d8062072b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 -- 172.21.15.60:0/2299149512 >> 172.21.15.60:0/2299149512 conn(0x55d8061d9970 msgr2=0x55d8062222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 -- 172.21.15.60:0/2299149512 shutdown_connections 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.553+0000 7f9641234740 1 -- 172.21.15.60:0/2299149512 wait complete. 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.554+0000 7f9641234740 1 -- start start 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.554+0000 7f9641234740 1 Processor -- start 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d806236670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d806224b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8061faa10 0x55d8061faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8061dcb20 con 0x55d806204aa0 2024-04-25T08:32:27.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8060df590 con 0x55d8061faa10 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8061dce00 con 0x55d806204e80 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 10 client.0 _ll_register_callbacks cb 0x55d8060ebe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d806236670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d806236670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45246/0 (socket says 172.21.15.60:45246) 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 -- 172.21.15.60:0/2244844963 learned_addr learned my addr 172.21.15.60:0/2244844963 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f963cacd640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8061faa10 0x55d8061faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f96377fe640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d806224b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 -- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 msgr2=0x55d806224b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d806224b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 -- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8061faa10 msgr2=0x55d8061faf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8061faa10 0x55d8061faf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 -- 172.21.15.60:0/2244844963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d805e553f0 con 0x55d806204aa0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9637fff640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d806236670 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9620035040 tx=0x7f9620004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ebe27c9518fce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9616ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:27.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9616ffd640 1 -- 172.21.15.60:0/2244844963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962004e070 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9616ffd640 1 -- 172.21.15.60:0/2244844963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9620053c50 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.555+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9620032c30 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9616ffd640 1 -- 172.21.15.60:0/2244844963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f962005a050 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 10 client.6180 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55d805f1f7d0 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9616ffd640 1 -- 172.21.15.60:0/2244844963 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620032c30 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9616ffd640 1 -- 172.21.15.60:0/2244844963 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f962005d020 con 0x55d806204aa0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 10 client.6180 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 10 client.6180 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 20 client.6180 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806225440 0x55d806225820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 4 client.6180 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d80614e340 con 0x55d806225440 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f96377fe640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806225440 0x55d806225820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f96377fe640 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806225440 0x55d806225820 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddabd090ff9f54e server_cookie=fc9250fbe1c0320e in_seq=0 out_seq=0 2024-04-25T08:32:27.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.556+0000 7f9616ffd640 10 client.6180 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9616ffd640 1 -- 172.21.15.60:0/2244844963 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d80614e340 con 0x55d806225440 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9616ffd640 10 client.6180 handle_command_reply: tid=0 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 client.6180 shutdown 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 msgr2=0x55d806236670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d806236670 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9620035040 tx=0x7f9620004730 comp rx=0 tx=0).stop 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 shutdown_connections 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8061faa10 0x55d8061faf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806225440 0x55d806225820 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d806224b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 --2- 172.21.15.60:0/2244844963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d806204aa0 0x55d806236670 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9616ffd640 0 client.6180 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 >> 172.21.15.60:0/2244844963 conn(0x55d8061d7700 msgr2=0x55d8061daf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 shutdown_connections 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 1 -- 172.21.15.60:0/2244844963 wait complete. 2024-04-25T08:32:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.557+0000 7f9641234740 20 client.6180 trim_cache size 0 max 16384 2024-04-25T08:32:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.569+0000 7f9641234740 1 Processor -- start 2024-04-25T08:32:27.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.569+0000 7f9641234740 1 -- start start 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9641234740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d8061d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204aa0 0x55d8061d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d8061d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9641234740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96382003a0 con 0x55d8061faa10 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96381ff5d0 con 0x55d806204aa0 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96380ccc40 con 0x55d806204e80 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d8061d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d8061d81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45248/0 (socket says 172.21.15.60:45248) 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 -- 172.21.15.60:0/2221146202 learned_addr learned my addr 172.21.15.60:0/2221146202 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 -- 172.21.15.60:0/2221146202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 msgr2=0x55d8061d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d8061d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 -- 172.21.15.60:0/2221146202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204aa0 msgr2=0x55d8061d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204aa0 0x55d8061d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 -- 172.21.15.60:0/2221146202 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d80621f2e0 con 0x55d8061faa10 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f9637fff640 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d8061d81f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f962002a6f0 tx=0x7f962000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faeba06e9b39009b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f962d7fa640 1 -- 172.21.15.60:0/2221146202 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620037040 con 0x55d8061faa10 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.570+0000 7f962d7fa640 1 -- 172.21.15.60:0/2221146202 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620032c30 con 0x55d8061faa10 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f962d7fa640 1 -- 172.21.15.60:0/2221146202 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620052630 con 0x55d8061faa10 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 -- 172.21.15.60:0/2221146202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 msgr2=0x55d8061d81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d8061d81f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f962002a6f0 tx=0x7f962000b050 comp rx=0 tx=0).stop 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 -- 172.21.15.60:0/2221146202 shutdown_connections 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204e80 0x55d8061d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204aa0 0x55d8061d8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 --2- 172.21.15.60:0/2221146202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d8061d81f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 -- 172.21.15.60:0/2221146202 >> 172.21.15.60:0/2221146202 conn(0x55d8061d7700 msgr2=0x55d806225e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 -- 172.21.15.60:0/2221146202 shutdown_connections 2024-04-25T08:32:27.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.571+0000 7f9641234740 1 -- 172.21.15.60:0/2221146202 wait complete. 2024-04-25T08:32:27.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.572+0000 7f9641234740 1 -- start start 2024-04-25T08:32:27.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.572+0000 7f9641234740 1 Processor -- start 2024-04-25T08:32:27.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.572+0000 7f9641234740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d806209dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204aa0 0x55d80620a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d806203a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8061ddd30 con 0x55d8061faa10 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96380cd240 con 0x55d806204e80 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d806226530 con 0x55d806204aa0 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 10 client.0 _ll_register_callbacks cb 0x7f96300095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d806209dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d806209dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45252/0 (socket says 172.21.15.60:45252) 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 -- 172.21.15.60:0/2347787432 learned_addr learned my addr 172.21.15.60:0/2347787432 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f96377fe640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204aa0 0x55d80620a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f963cacd640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d806203a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 -- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204aa0 msgr2=0x55d80620a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204aa0 0x55d80620a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 -- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 msgr2=0x55d806203a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d806203a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 -- 172.21.15.60:0/2347787432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d806207820 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f96377fe640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204aa0 0x55d80620a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9637fff640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d806209dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9620035040 tx=0x7f9620008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26648b098c36b7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9616ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9616ffd640 1 -- 172.21.15.60:0/2347787432 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620037040 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9616ffd640 1 -- 172.21.15.60:0/2347787432 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f962003cbc0 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9620032c30 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9616ffd640 1 -- 172.21.15.60:0/2347787432 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9620074050 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 10 client.6186 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.573+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55d806203510 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9616ffd640 1 -- 172.21.15.60:0/2347787432 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9620032c30 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9616ffd640 1 -- 172.21.15.60:0/2347787432 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9620077020 con 0x55d8061faa10 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 10 client.6186 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 10 client.6186 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 20 client.6186 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:27.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806273460 0x55d806203f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 4 client.6186 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d806148690 con 0x55d806273460 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f96377fe640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806273460 0x55d806203f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9616ffd640 10 client.6186 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f96377fe640 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806273460 0x55d806203f70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6323f1980d091e1 server_cookie=9d5e5743c3ef729f in_seq=0 out_seq=0 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9616ffd640 1 -- 172.21.15.60:0/2347787432 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d806148690 con 0x55d806273460 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9616ffd640 10 client.6186 handle_command_reply: tid=0 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.574+0000 7f9641234740 1 client.6186 shutdown 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 msgr2=0x55d806209dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d806209dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9620035040 tx=0x7f9620008030 comp rx=0 tx=0).stop 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 shutdown_connections 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d806204e80 0x55d806203a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d806273460 0x55d806203f70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d806204aa0 0x55d80620a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 --2- 172.21.15.60:0/2347787432 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8061faa10 0x55d806209dd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 >> 172.21.15.60:0/2347787432 conn(0x55d8061d7700 msgr2=0x55d8061d81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9616ffd640 0 client.6186 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 shutdown_connections 2024-04-25T08:32:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 1 -- 172.21.15.60:0/2347787432 wait complete. 2024-04-25T08:32:27.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f9641234740 20 client.6186 trim_cache size 0 max 16384 2024-04-25T08:32:27.578 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f961806b4d0 msgr2=0x7f961806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f961806b4d0 0x7f961806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9620034000 tx=0x7f962002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 msgr2=0x7f96380b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 0x7f96380b7000 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f962800c400 tx=0x7f962800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 shutdown_connections 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f963815b7d0 0x7f96381f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96380b5620 0x7f96380b7000 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f961806b4d0 0x7f961806d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 --2- 172.21.15.60:0/3826092130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f963806c9e0 0x7f9638148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 >> 172.21.15.60:0/3826092130 conn(0x7f96380d0800 msgr2=0x7f96380d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 shutdown_connections 2024-04-25T08:32:27.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.575+0000 7f963e530640 1 -- 172.21.15.60:0/3826092130 wait complete. 2024-04-25T08:32:27.583 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096a.00000000 parent 2024-04-25T08:32:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.629+0000 7f7f3a60ef40 1 Processor -- start 2024-04-25T08:32:27.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.629+0000 7f7f3a60ef40 1 -- start start 2024-04-25T08:32:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f3a60ef40 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557653818b30 0x5576538970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f3a60ef40 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x5576537fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f3a60ef40 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f3a60ef40 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557653818f40 con 0x5576538a4010 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f3a60ef40 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557653776df0 con 0x5576538a3c30 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f3a60ef40 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576537fe1e0 con 0x557653818b30 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45276/0 (socket says 172.21.15.60:45276) 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 -- 172.21.15.60:0/3855621662 learned_addr learned my addr 172.21.15.60:0/3855621662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f374bf640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557653818b30 0x5576538970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f36cbe640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x5576537fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 -- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557653818b30 msgr2=0x5576538970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557653818b30 0x5576538970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 -- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 msgr2=0x5576537fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x5576537fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f37cc0640 1 -- 172.21.15.60:0/3855621662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576536fcc50 con 0x5576538a4010 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f374bf640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557653818b30 0x5576538970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.630+0000 7f7f36cbe640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x5576537fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 0x5576537f9f50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c480 tx=0x7f7f3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df7bdde6fc6a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f35cbc640 1 -- 172.21.15.60:0/3855621662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f300115f0 con 0x5576538a4010 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f35cbc640 1 -- 172.21.15.60:0/3855621662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f30011790 con 0x5576538a4010 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f35cbc640 1 -- 172.21.15.60:0/3855621662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011b80 con 0x5576538a4010 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/3855621662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 msgr2=0x5576537f9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 0x5576537f9f50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c480 tx=0x7f7f3000c940 comp rx=0 tx=0).stop 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/3855621662 shutdown_connections 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x5576537fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/3855621662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x557653818b30 0x5576538970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/3855621662 >> 172.21.15.60:0/3855621662 conn(0x5576537ce9b0 msgr2=0x557653899f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/3855621662 shutdown_connections 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.631+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/3855621662 wait complete. 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.632+0000 7f7f3a60ef40 1 Processor -- start 2024-04-25T08:32:27.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.632+0000 7f7f3a60ef40 1 -- start start 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557653818b30 0x5576537d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x557653893c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576538975f0 con 0x557653818b30 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576537d2620 con 0x5576538a3c30 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576538a5ab0 con 0x5576538a4010 2024-04-25T08:32:27.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557653818b30 0x5576537d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557653818b30 0x5576537d0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45278/0 (socket says 172.21.15.60:45278) 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 -- 172.21.15.60:0/1859722891 learned_addr learned my addr 172.21.15.60:0/1859722891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f36cbe640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x557653893c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 -- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5576538a4010 msgr2=0x5576537f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 -- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 msgr2=0x557653893c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x557653893c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 -- 172.21.15.60:0/1859722891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557653894150 con 0x557653818b30 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f36cbe640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5576538a3c30 0x557653893c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:27.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5576538a4010 0x5576537f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f374bf640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557653818b30 0x5576537d0cb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7f28001cb0 tx=0x7f7f2800a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f34cba640 1 -- 172.21.15.60:0/1859722891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800e640 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f34cba640 1 -- 172.21.15.60:0/1859722891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800e7e0 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f34cba640 1 -- 172.21.15.60:0/1859722891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800eb90 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.633+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5576537d12d0 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.634+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576538198e0 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.635+0000 7f7f34cba640 1 -- 172.21.15.60:0/1859722891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f28009030 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.635+0000 7f7f34cba640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f24069eb0 0x7f7f2406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.635+0000 7f7f34cba640 1 -- 172.21.15.60:0/1859722891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f28089250 con 0x557653818b30 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.635+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x5576538a5f70 0x5576538a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.636+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 --> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] -- osd_op(unknown.0.0:1 8.11 8:88e5eea5:::1000000096a.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v9 -- 0x5576538a68d0 con 0x5576538a5f70 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.636+0000 7f7f36cbe640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f24069eb0 0x7f7f2406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.636+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x5576538a5f70 0x5576538a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.636+0000 7f7f36cbe640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f24069eb0 0x7f7f2406c370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c056000 tx=0x7f7f2c050000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.636+0000 7f7f37cc0640 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x5576538a5f70 0x5576538a6390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f37cc0640 1 -- 172.21.15.60:0/1859722891 <== osd.5 v2:172.21.15.100:6800/301366941 1 ==== osd_op_reply(1 1000000096a.00000000 [rmxattr] v55'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f7f30011b80 con 0x5576538a5f70 2024-04-25T08:32:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x5576538a5f70 msgr2=0x5576538a6390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x5576538a5f70 0x5576538a6390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f24069eb0 msgr2=0x7f7f2406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f24069eb0 0x7f7f2406c370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c056000 tx=0x7f7f2c050000 comp rx=0 tx=0).stop 2024-04-25T08:32:27.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557653818b30 msgr2=0x5576537d0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 --2- 172.21.15.60:0/1859722891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557653818b30 0x5576537d0cb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7f28001cb0 tx=0x7f7f2800a6b0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f37cc0640 1 -- 172.21.15.60:0/1859722891 reap_dead start 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 shutdown_connections 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 >> 172.21.15.60:0/1859722891 conn(0x5576537ce9b0 msgr2=0x557653899f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 shutdown_connections 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.638+0000 7f7f3a60ef40 1 -- 172.21.15.60:0/1859722891 wait complete. 2024-04-25T08:32:27.638 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T08:32:27.638 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:27.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T08:32:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.785+0000 7fd3084b6640 1 Processor -- start 2024-04-25T08:32:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.785+0000 7fd3084b6640 1 -- start start 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd3084b6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd30015dc20 0x7fd3000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd3084b6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3000d32f0 0x7fd3000cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd3084b6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30015e000 con 0x7fd300161710 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd3084b6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3000d1de0 con 0x7fd3000d32f0 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd3084b6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd300161c60 con 0x7fd30015dc20 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3000d32f0 0x7fd3000cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd3000d32f0 0x7fd3000cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45280/0 (socket says 172.21.15.60:45280) 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 -- 172.21.15.60:0/931981459 learned_addr learned my addr 172.21.15.60:0/931981459 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306252640 1 --2- 172.21.15.60:0/931981459 >> v2:172.21.15.100:3300/0 conn(0x7fd30015dc20 0x7fd3000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 -- 172.21.15.60:0/931981459 >> v2:172.21.15.100:3300/0 conn(0x7fd30015dc20 msgr2=0x7fd3000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 --2- 172.21.15.60:0/931981459 >> v2:172.21.15.100:3300/0 conn(0x7fd30015dc20 0x7fd3000ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 -- 172.21.15.60:0/931981459 >> v1:172.21.15.100:6789/0 conn(0x7fd300161710 legacy=0x7fd30006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306a53640 1 -- 172.21.15.60:0/931981459 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300161f40 con 0x7fd3000d32f0 2024-04-25T08:32:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.786+0000 7fd306252640 1 --2- 172.21.15.60:0/931981459 >> v2:172.21.15.100:3300/0 conn(0x7fd30015dc20 0x7fd3000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.787+0000 7fd306a53640 1 --2- 172.21.15.60:0/931981459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3000d32f0 0x7fd3000cebb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd3000d1520 tx=0x7fd2f002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11463fd070f67724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.787+0000 7fd305250640 1 -- 172.21.15.60:0/931981459 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0036070 con 0x7fd3000d32f0 2024-04-25T08:32:27.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.787+0000 7fd305250640 1 -- 172.21.15.60:0/931981459 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0031600 con 0x7fd3000d32f0 2024-04-25T08:32:27.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.787+0000 7fd305250640 1 -- 172.21.15.60:0/931981459 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00319d0 con 0x7fd3000d32f0 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 -- 172.21.15.60:0/931981459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3000d32f0 msgr2=0x7fd3000cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 --2- 172.21.15.60:0/931981459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3000d32f0 0x7fd3000cebb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd3000d1520 tx=0x7fd2f002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 -- 172.21.15.60:0/931981459 shutdown_connections 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 --2- 172.21.15.60:0/931981459 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3000d32f0 0x7fd3000cebb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 --2- 172.21.15.60:0/931981459 >> v2:172.21.15.100:3300/0 conn(0x7fd30015dc20 0x7fd3000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 -- 172.21.15.60:0/931981459 >> 172.21.15.60:0/931981459 conn(0x7fd3000d0420 msgr2=0x7fd3000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 -- 172.21.15.60:0/931981459 shutdown_connections 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.788+0000 7fd3084b6640 1 -- 172.21.15.60:0/931981459 wait complete. 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.789+0000 7fd3084b6640 1 Processor -- start 2024-04-25T08:32:27.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.789+0000 7fd3084b6640 1 -- start start 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd3084b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3000d32f0 0x7fd3000cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd3084b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd30015dc20 0x7fd3001f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd3084b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 0x7fd3001f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd3084b6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b5e50 con 0x7fd300161710 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd3084b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300162500 con 0x7fd3000d32f0 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd3084b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3000b5fd0 con 0x7fd30015dc20 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 0x7fd3001f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 0x7fd3001f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45292/0 (socket says 172.21.15.60:45292) 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 -- 172.21.15.60:0/158613620 learned_addr learned my addr 172.21.15.60:0/158613620 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306252640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3000d32f0 0x7fd3000cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd305a51640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd30015dc20 0x7fd3001f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 -- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd30015dc20 msgr2=0x7fd3001f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd30015dc20 0x7fd3001f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 -- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3000d32f0 msgr2=0x7fd3000cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3000d32f0 0x7fd3000cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd306a53640 1 -- 172.21.15.60:0/158613620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3001f4860 con 0x7fd300161710 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.790+0000 7fd305a51640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd30015dc20 0x7fd3001f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.791+0000 7fd306a53640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 0x7fd3001f71d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009f40 tx=0x7fd2f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.791+0000 7fd2feffd640 1 -- 172.21.15.60:0/158613620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0036050 con 0x7fd300161710 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.791+0000 7fd2feffd640 1 -- 172.21.15.60:0/158613620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0042070 con 0x7fd300161710 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.791+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30015e370 con 0x7fd300161710 2024-04-25T08:32:27.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.791+0000 7fd2feffd640 1 -- 172.21.15.60:0/158613620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0009510 con 0x7fd300161710 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.791+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30015e930 con 0x7fd300161710 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.792+0000 7fd2feffd640 1 -- 172.21.15.60:0/158613620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2f0007bf0 con 0x7fd300161710 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.793+0000 7fd2feffd640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2ec06b4d0 0x7fd2ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.793+0000 7fd2feffd640 1 -- 172.21.15.60:0/158613620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2f00ab340 con 0x7fd300161710 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.793+0000 7fd306252640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2ec06b4d0 0x7fd2ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.794+0000 7fd306252640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2ec06b4d0 0x7fd2ec06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b67f0 tx=0x7fd2f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.794+0000 7fd30b1bf740 1 Processor -- start 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.794+0000 7fd30b1bf740 1 -- start start 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd207d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd07c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5abc40ee0 con 0x55c5abd08140 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5abbe9c50 con 0x55c5abd00360 2024-04-25T08:32:27.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5abc3cf80 con 0x55c5abcdf360 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45300/0 (socket says 172.21.15.60:45300) 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 -- 172.21.15.60:0/900462349 learned_addr learned my addr 172.21.15.60:0/900462349 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306252640 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd207d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd305a51640 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd07c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 -- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 msgr2=0x55c5abd07c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd07c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 -- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 msgr2=0x55c5abd207d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd207d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 -- 172.21.15.60:0/900462349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5aba21c00 con 0x55c5abd08140 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd306a53640 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdd7c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0079820 tx=0x7fd2f00793d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2fed2b223a9e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd2e7fff640 1 -- 172.21.15.60:0/900462349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0031600 con 0x55c5abd08140 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd2e7fff640 1 -- 172.21.15.60:0/900462349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0037040 con 0x55c5abd08140 2024-04-25T08:32:27.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.795+0000 7fd2e7fff640 1 -- 172.21.15.60:0/900462349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0079f90 con 0x55c5abd08140 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 -- 172.21.15.60:0/900462349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 msgr2=0x55c5abcdd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdd7c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0079820 tx=0x7fd2f00793d0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 -- 172.21.15.60:0/900462349 shutdown_connections 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdd7c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd07c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/900462349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd207d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 -- 172.21.15.60:0/900462349 >> 172.21.15.60:0/900462349 conn(0x55c5abca4810 msgr2=0x55c5abd22c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 -- 172.21.15.60:0/900462349 shutdown_connections 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.796+0000 7fd30b1bf740 1 -- 172.21.15.60:0/900462349 wait complete. 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.797+0000 7fd30b1bf740 1 -- start start 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.797+0000 7fd30b1bf740 1 Processor -- start 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 0x55c5abd36590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd08140 0x55c5abd36ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5abc40ee0 con 0x55c5abd00360 2024-04-25T08:32:27.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5abbdf340 con 0x55c5abd08140 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5abcdd330 con 0x55c5abcdf360 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 10 client.0 _ll_register_callbacks cb 0x55c5ab954fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 0x55c5abd36590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 0x55c5abd36590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45310/0 (socket says 172.21.15.60:45310) 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 -- 172.21.15.60:0/1733039756 learned_addr learned my addr 172.21.15.60:0/1733039756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd306a53640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd08140 0x55c5abd36ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd306252640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 -- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 msgr2=0x55c5abd1b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 -- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd08140 msgr2=0x55c5abd36ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd08140 0x55c5abd36ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 -- 172.21.15.60:0/1733039756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ab9154f0 con 0x55c5abd00360 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd305a51640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 0x55c5abd36590 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b8e30 tx=0x7fd2f400f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55d951d705e8df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd2e57fa640 1 -- 172.21.15.60:0/1733039756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4004220 con 0x55c5abd00360 2024-04-25T08:32:27.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd2e57fa640 1 -- 172.21.15.60:0/1733039756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2f40043c0 con 0x55c5abd00360 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.798+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f0037040 con 0x55c5abd00360 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd2e57fa640 1 -- 172.21.15.60:0/1733039756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f4022050 con 0x55c5abd00360 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 10 client.6210 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c5aba82390 con 0x55c5abd00360 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd2e57fa640 1 -- 172.21.15.60:0/1733039756 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0037040 con 0x55c5abd00360 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd2e57fa640 1 -- 172.21.15.60:0/1733039756 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd2f4025020 con 0x55c5abd00360 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 10 client.6210 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 10 client.6210 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 20 client.6210 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd1a160 0x55c5abd08da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 4 client.6210 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5abc4e420 con 0x55c5abd1a160 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd306252640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd1a160 0x55c5abd08da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd2e57fa640 10 client.6210 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.799+0000 7fd306252640 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd1a160 0x55c5abd08da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfeb561c271aefea server_cookie=32a118731214a179 in_seq=0 out_seq=0 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd2e57fa640 1 -- 172.21.15.60:0/1733039756 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5abc4e420 con 0x55c5abd1a160 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd2e57fa640 10 client.6210 handle_command_reply: tid=0 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 client.6210 shutdown 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 msgr2=0x55c5abd36590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 0x55c5abd36590 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b8e30 tx=0x7fd2f400f950 comp rx=0 tx=0).stop 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 shutdown_connections 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd08140 0x55c5abd36ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd00360 0x55c5abd36590 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd1a160 0x55c5abd08da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/1733039756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd2e57fa640 0 client.6210 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 >> 172.21.15.60:0/1733039756 conn(0x55c5abca4810 msgr2=0x55c5abcdc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 shutdown_connections 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 1 -- 172.21.15.60:0/1733039756 wait complete. 2024-04-25T08:32:27.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.800+0000 7fd30b1bf740 20 client.6210 trim_cache size 0 max 16384 2024-04-25T08:32:27.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 Processor -- start 2024-04-25T08:32:27.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 -- start start 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3002004f0 con 0x55c5abd08140 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001ff6f0 con 0x55c5abd00360 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30015fec0 con 0x55c5abcdf360 2024-04-25T08:32:27.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45314/0 (socket says 172.21.15.60:45314) 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.816+0000 7fd306a53640 1 -- 172.21.15.60:0/2023715163 learned_addr learned my addr 172.21.15.60:0/2023715163 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd306a53640 1 -- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 msgr2=0x55c5abd1c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd306a53640 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd306a53640 1 -- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 msgr2=0x55c5abd45dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd305a51640 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd306a53640 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd45dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd306a53640 1 -- 172.21.15.60:0/2023715163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ab8ee670 con 0x55c5abd08140 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd306a53640 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0079820 tx=0x7fd2f0049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4874a656a2d7a96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd2e77fe640 1 -- 172.21.15.60:0/2023715163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0096070 con 0x55c5abd08140 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd2e77fe640 1 -- 172.21.15.60:0/2023715163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0079f90 con 0x55c5abd08140 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.817+0000 7fd2e77fe640 1 -- 172.21.15.60:0/2023715163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0079890 con 0x55c5abd08140 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2023715163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 msgr2=0x55c5abcdb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0079820 tx=0x7fd2f0049000 comp rx=0 tx=0).stop 2024-04-25T08:32:27.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2023715163 shutdown_connections 2024-04-25T08:32:27.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd45dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2023715163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd1c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2023715163 >> 172.21.15.60:0/2023715163 conn(0x55c5abca4810 msgr2=0x55c5abcdc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2023715163 shutdown_connections 2024-04-25T08:32:27.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2023715163 wait complete. 2024-04-25T08:32:27.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 -- start start 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.818+0000 7fd30b1bf740 1 Processor -- start 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd45dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001ff0f0 con 0x55c5abd08140 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5abcde9f0 con 0x55c5abd00360 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5abd20ab0 con 0x55c5abcdf360 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd30b1bf740 10 client.0 _ll_register_callbacks cb 0x7fd2f00092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd306a53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45330/0 (socket says 172.21.15.60:45330) 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd306a53640 1 -- 172.21.15.60:0/2666930756 learned_addr learned my addr 172.21.15.60:0/2666930756 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd305a51640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.819+0000 7fd306252640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd45dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306a53640 1 -- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 msgr2=0x55c5abd45dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306a53640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd45dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306a53640 1 -- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 msgr2=0x55c5abd8c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306a53640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd8c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306a53640 1 -- 172.21.15.60:0/2666930756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5abd1c710 con 0x55c5abd08140 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306a53640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0079820 tx=0x7fd2f0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b9b7d086c23e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd2e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd2e57fa640 1 -- 172.21.15.60:0/2666930756 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0096030 con 0x55c5abd08140 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd2e57fa640 1 -- 172.21.15.60:0/2666930756 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2f0077b90 con 0x55c5abd08140 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f0079f90 con 0x55c5abd08140 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd2e57fa640 1 -- 172.21.15.60:0/2666930756 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f0031600 con 0x55c5abd08140 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 10 client.6216 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c5abd1c710 con 0x55c5abd08140 2024-04-25T08:32:27.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd2e57fa640 1 -- 172.21.15.60:0/2666930756 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0079f90 con 0x55c5abd08140 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd2e57fa640 1 -- 172.21.15.60:0/2666930756 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd2f00982c0 con 0x55c5abd08140 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 10 client.6216 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 10 client.6216 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 20 client.6216 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd196f0 0x55c5abcdbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 4 client.6216 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55c5abc48770 con 0x55c5abd196f0 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.820+0000 7fd306252640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd196f0 0x55c5abcdbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd306252640 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd196f0 0x55c5abcdbdb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370449c518fdddad server_cookie=88292355e8fb9012 in_seq=0 out_seq=0 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd2e57fa640 10 client.6216 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd2e57fa640 1 -- 172.21.15.60:0/2666930756 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c5abc48770 con 0x55c5abd196f0 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd2e57fa640 10 client.6216 handle_command_reply: tid=0 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 client.6216 shutdown 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 msgr2=0x55c5abcdb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0079820 tx=0x7fd2f0009810 comp rx=0 tx=0).stop 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 shutdown_connections 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c5abd08140 0x55c5abcdb870 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c5abd00360 0x55c5abd8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55c5abd196f0 0x55c5abcdbdb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 --2- 172.21.15.60:0/2666930756 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55c5abcdf360 0x55c5abd45dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd2e57fa640 0 client.6216 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 >> 172.21.15.60:0/2666930756 conn(0x55c5abca4810 msgr2=0x55c5abcdac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.824 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:32:27.821+0000 7fe78834d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096a [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v446 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5656=0-4194304@1} caps={5656=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5656 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562dcf3fb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096a:[<0x10000000001/new_file_0 v446>,<0x10000000000/ceph-qa-suite v38>,<0x1/scrub_test_path v9>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:32:27.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 shutdown_connections 2024-04-25T08:32:27.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd30b1bf740 1 -- 172.21.15.60:0/2666930756 wait complete. 2024-04-25T08:32:27.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd30b1bf740 20 client.6216 trim_cache size 0 max 16384 2024-04-25T08:32:27.825 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "09d41767-a0b8-4beb-a52b-48a8e837ba2d", 2024-04-25T08:32:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:27.825 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2ec06b4d0 msgr2=0x7fd2ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2ec06b4d0 0x7fd2ec06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd3000b67f0 tx=0x7fd2f8008040 comp rx=0 tx=0).stop 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 msgr2=0x7fd3001f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 0x7fd3001f71d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009f40 tx=0x7fd2f0004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 shutdown_connections 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd300161710 0x7fd3001f71d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd30015dc20 0x7fd3001f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2ec06b4d0 0x7fd2ec06d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 --2- 172.21.15.60:0/158613620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3000d32f0 0x7fd3000cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 >> 172.21.15.60:0/158613620 conn(0x7fd3000d0420 msgr2=0x7fd30015c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 shutdown_connections 2024-04-25T08:32:27.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.822+0000 7fd3084b6640 1 -- 172.21.15.60:0/158613620 wait complete. 2024-04-25T08:32:27.829 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '09d41767-a0b8-4beb-a52b-48a8e837ba2d', 'mode': 'asynchronous'}' 2024-04-25T08:32:27.829 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:32:27.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:32:27.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.935+0000 7f2b90ed2640 1 Processor -- start 2024-04-25T08:32:27.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.935+0000 7f2b90ed2640 1 -- start start 2024-04-25T08:32:27.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.936+0000 7f2b90ed2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b8c0ced80 0x7f2b8c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.936+0000 7f2b90ed2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b8c161710 0x7f2b8c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.936+0000 7f2b90ed2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c15e000 con 0x7f2b8c15dc20 2024-04-25T08:32:27.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.936+0000 7f2b90ed2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c0d1de0 con 0x7f2b8c0ced80 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.936+0000 7f2b90ed2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c161c60 con 0x7f2b8c161710 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.936+0000 7f2b89d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b8c0ced80 0x7f2b8c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b89d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2b8c0ced80 0x7f2b8c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45342/0 (socket says 172.21.15.60:45342) 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b89d9b640 1 -- 172.21.15.60:0/3194109407 learned_addr learned my addr 172.21.15.60:0/3194109407 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/3194109407 >> v2:172.21.15.100:3300/0 conn(0x7f2b8c161710 0x7f2b8c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b89d9b640 1 -- 172.21.15.60:0/3194109407 >> v2:172.21.15.100:3300/0 conn(0x7f2b8c161710 msgr2=0x7f2b8c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/3194109407 >> v2:172.21.15.100:3300/0 conn(0x7f2b8c161710 0x7f2b8c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b89d9b640 1 -- 172.21.15.60:0/3194109407 >> v1:172.21.15.100:6789/0 conn(0x7f2b8c15dc20 legacy=0x7f2b8c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.937+0000 7f2b89d9b640 1 -- 172.21.15.60:0/3194109407 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c161f40 con 0x7f2b8c0ced80 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/3194109407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c0ced80 0x7f2b8c0d22e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0b6570 tx=0x7f2b7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a667a76124c3207 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b8959a640 1 -- 172.21.15.60:0/3194109407 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74037070 con 0x7f2b8c0ced80 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b8959a640 1 -- 172.21.15.60:0/3194109407 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74032440 con 0x7f2b8c0ced80 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b8959a640 1 -- 172.21.15.60:0/3194109407 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74032810 con 0x7f2b8c0ced80 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 -- 172.21.15.60:0/3194109407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c0ced80 msgr2=0x7f2b8c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/3194109407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c0ced80 0x7f2b8c0d22e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0b6570 tx=0x7f2b7402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 -- 172.21.15.60:0/3194109407 shutdown_connections 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/3194109407 >> v2:172.21.15.100:3300/0 conn(0x7f2b8c161710 0x7f2b8c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/3194109407 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c0ced80 0x7f2b8c0d22e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 -- 172.21.15.60:0/3194109407 >> 172.21.15.60:0/3194109407 conn(0x7f2b8c0d0420 msgr2=0x7f2b8c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 -- 172.21.15.60:0/3194109407 shutdown_connections 2024-04-25T08:32:27.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.938+0000 7f2b90ed2640 1 -- 172.21.15.60:0/3194109407 wait complete. 2024-04-25T08:32:27.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 Processor -- start 2024-04-25T08:32:27.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 -- start start 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b8c0ced80 0x7f2b8c148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b8c15dc20 0x7f2b8c0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 0x7f2b8c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0b5e50 con 0x7f2b8c161710 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c162500 con 0x7f2b8c15dc20 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.940+0000 7f2b90ed2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0b5fd0 con 0x7f2b8c0ced80 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 0x7f2b8c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 0x7f2b8c1587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45352/0 (socket says 172.21.15.60:45352) 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 -- 172.21.15.60:0/2212426523 learned_addr learned my addr 172.21.15.60:0/2212426523 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b8c15dc20 0x7f2b8c0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b8c0ced80 0x7f2b8c148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 -- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b8c0ced80 msgr2=0x7f2b8c148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b8c0ced80 0x7f2b8c148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 -- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b8c15dc20 msgr2=0x7f2b8c0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b8c15dc20 0x7f2b8c0c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 -- 172.21.15.60:0/2212426523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c15a650 con 0x7f2b8c161710 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b8c15dc20 0x7f2b8c0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 0x7f2b8c1587b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00d4c0 tx=0x7f2b7c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b82ffd640 1 -- 172.21.15.60:0/2212426523 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0125f0 con 0x7f2b8c161710 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b82ffd640 1 -- 172.21.15.60:0/2212426523 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c019070 con 0x7f2b8c161710 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b82ffd640 1 -- 172.21.15.60:0/2212426523 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c0129f0 con 0x7f2b8c161710 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c15a930 con 0x7f2b8c161710 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.941+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c15aef0 con 0x7f2b8c161710 2024-04-25T08:32:27.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.943+0000 7f2b82ffd640 1 -- 172.21.15.60:0/2212426523 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b7c012b90 con 0x7f2b8c161710 2024-04-25T08:32:27.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.943+0000 7f2b82ffd640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b6c06b4d0 0x7f2b6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.943+0000 7f2b82ffd640 1 -- 172.21.15.60:0/2212426523 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b7c08a5f0 con 0x7f2b8c161710 2024-04-25T08:32:27.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.943+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b6c06b4d0 0x7f2b6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.944+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b6c06b4d0 0x7f2b6c06d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2b84007fd0 tx=0x7f2b84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.944+0000 7f2b93bd8740 1 Processor -- start 2024-04-25T08:32:27.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.944+0000 7f2b93bd8740 1 -- start start 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca254b10 0x560dca257320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca257860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca227770 0x560dca22af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dca190da0 con 0x560dca254ef0 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dca139b80 con 0x560dca227770 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560dca18ce40 con 0x560dca254b10 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b89d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca257860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b89d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca257860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45354/0 (socket says 172.21.15.60:45354) 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b89d9b640 1 -- 172.21.15.60:0/4277548067 learned_addr learned my addr 172.21.15.60:0/4277548067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca227770 0x560dca22af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca254b10 0x560dca257320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.945+0000 7f2b89d9b640 1 -- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca254b10 msgr2=0x560dca257320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca254b10 0x560dca257320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b89d9b640 1 -- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca227770 msgr2=0x560dca22af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca227770 0x560dca22af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b89d9b640 1 -- 172.21.15.60:0/4277548067 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dca143f10 con 0x560dca254ef0 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca257860 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2b740090b0 tx=0x7f2b7402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef02d5650a08257f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b6bfff640 1 -- 172.21.15.60:0/4277548067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74041070 con 0x560dca254ef0 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b6bfff640 1 -- 172.21.15.60:0/4277548067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b740326c0 con 0x560dca254ef0 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b6bfff640 1 -- 172.21.15.60:0/4277548067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74037040 con 0x560dca254ef0 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b93bd8740 1 -- 172.21.15.60:0/4277548067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 msgr2=0x560dca257860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.946+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca257860 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2b740090b0 tx=0x7f2b7402c000 comp rx=0 tx=0).stop 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 -- 172.21.15.60:0/4277548067 shutdown_connections 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca227770 0x560dca22af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca257860 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/4277548067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca254b10 0x560dca257320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 -- 172.21.15.60:0/4277548067 >> 172.21.15.60:0/4277548067 conn(0x560dca2299e0 msgr2=0x560dca272320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 -- 172.21.15.60:0/4277548067 shutdown_connections 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 -- 172.21.15.60:0/4277548067 wait complete. 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 -- start start 2024-04-25T08:32:27.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.947+0000 7f2b93bd8740 1 Processor -- start 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca25b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca276f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca274dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dca22cb90 con 0x560dca254b10 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dca12f4b0 con 0x560dca254ef0 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560dca22ce70 con 0x560dca274940 2024-04-25T08:32:27.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b93bd8740 10 client.0 _ll_register_callbacks cb 0x560dca16f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b8a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca25b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b8a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca25b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45360/0 (socket says 172.21.15.60:45360) 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b8a59c640 1 -- 172.21.15.60:0/2894497207 learned_addr learned my addr 172.21.15.60:0/2894497207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca276f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.948+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca274dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b8a59c640 1 -- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 msgr2=0x560dca274dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca274dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b8a59c640 1 -- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 msgr2=0x560dca276f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca276f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b8a59c640 1 -- 172.21.15.60:0/2894497207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dc9f6f7d0 con 0x560dca254b10 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca25b340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2b84014480 tx=0x7f2b84001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2cfa31890b08bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b697fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b697fa640 1 -- 172.21.15.60:0/2894497207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84024020 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b697fa640 1 -- 172.21.15.60:0/2894497207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b84020600 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b740326c0 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b697fa640 1 -- 172.21.15.60:0/2894497207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b8402e050 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 10 client.6228 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560dc9ea53f0 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b697fa640 1 -- 172.21.15.60:0/2894497207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b740326c0 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b697fa640 1 -- 172.21.15.60:0/2894497207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b84031020 con 0x560dca254b10 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 10 client.6228 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 20 client.6228 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca258320 0x560dca258740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 4 client.6228 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.949+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dca19e2e0 con 0x560dca258320 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca258320 0x560dca258740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca258320 0x560dca258740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec7d78a6910f13b server_cookie=1fd52baec8f98fe4 in_seq=0 out_seq=0 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b697fa640 10 client.6228 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b697fa640 1 -- 172.21.15.60:0/2894497207 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560dca19e2e0 con 0x560dca258320 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b697fa640 10 client.6228 handle_command_reply: tid=0 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b93bd8740 1 client.6228 shutdown 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 msgr2=0x560dca25b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.950+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca25b340 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2b84014480 tx=0x7f2b84001250 comp rx=0 tx=0).stop 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 shutdown_connections 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca258320 0x560dca258740 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca274dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca276f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2894497207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca25b340 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b697fa640 0 client.6228 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 >> 172.21.15.60:0/2894497207 conn(0x560dca227770 msgr2=0x560dca228420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 shutdown_connections 2024-04-25T08:32:27.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2894497207 wait complete. 2024-04-25T08:32:27.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.951+0000 7f2b93bd8740 20 client.6228 trim_cache size 0 max 16384 2024-04-25T08:32:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.966+0000 7f2b93bd8740 1 Processor -- start 2024-04-25T08:32:27.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.966+0000 7f2b93bd8740 1 -- start start 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254b10 0x560dca26f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca22cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca2267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c200600 con 0x560dca254ef0 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1ff830 con 0x560dca254b10 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0c9ac0 con 0x560dca274940 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca22cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca22cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45362/0 (socket says 172.21.15.60:45362) 2024-04-25T08:32:27.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 -- 172.21.15.60:0/2907218173 learned_addr learned my addr 172.21.15.60:0/2907218173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254b10 0x560dca26f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 -- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 msgr2=0x560dca2267f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca2267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca2267f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 -- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254b10 msgr2=0x560dca26f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254b10 0x560dca26f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 -- 172.21.15.60:0/2907218173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dca275130 con 0x560dca254ef0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254b10 0x560dca26f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca2267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca22cdd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009930 tx=0x7f2b7403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b143c844f09e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b6b7fe640 1 -- 172.21.15.60:0/2907218173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74037040 con 0x560dca254ef0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b6b7fe640 1 -- 172.21.15.60:0/2907218173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b740325e0 con 0x560dca254ef0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.967+0000 7f2b6b7fe640 1 -- 172.21.15.60:0/2907218173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74041040 con 0x560dca254ef0 2024-04-25T08:32:27.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2907218173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 msgr2=0x560dca22cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca22cdd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009930 tx=0x7f2b7403f040 comp rx=0 tx=0).stop 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2907218173 shutdown_connections 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca2267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254ef0 0x560dca22cdd0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/2907218173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254b10 0x560dca26f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2907218173 >> 172.21.15.60:0/2907218173 conn(0x560dca227770 msgr2=0x560dca2540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2907218173 shutdown_connections 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 -- 172.21.15.60:0/2907218173 wait complete. 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 -- start start 2024-04-25T08:32:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.968+0000 7f2b93bd8740 1 Processor -- start 2024-04-25T08:32:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca2bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca275b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca227e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1ff230 con 0x560dca254b10 2024-04-25T08:32:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dca294a60 con 0x560dca254ef0 2024-04-25T08:32:27.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560dca228850 con 0x560dca274940 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b93bd8740 10 client.0 _ll_register_callbacks cb 0x7f2b74035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b8a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca2bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b8a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca2bf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45366/0 (socket says 172.21.15.60:45366) 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.969+0000 7f2b8a59c640 1 -- 172.21.15.60:0/1553112227 learned_addr learned my addr 172.21.15.60:0/1553112227 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca275b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca227e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8a59c640 1 -- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 msgr2=0x560dca227e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca227e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8a59c640 1 -- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 msgr2=0x560dca275b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca275b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8a59c640 1 -- 172.21.15.60:0/1553112227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dc9e650f0 con 0x560dca254b10 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca275b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8ad9d640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca227e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b8a59c640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca2bf5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009010 tx=0x7f2b84014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9031a963180aff11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b697fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b697fa640 1 -- 172.21.15.60:0/1553112227 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84024040 con 0x560dca254b10 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b697fa640 1 -- 172.21.15.60:0/1553112227 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b84020cb0 con 0x560dca254b10 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b740325e0 con 0x560dca254b10 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b697fa640 1 -- 172.21.15.60:0/1553112227 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b8403f050 con 0x560dca254b10 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 10 client.6234 fetch_fsmap learned FSMap version 22 2024-04-25T08:32:27.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x560dca273020 con 0x560dca254b10 2024-04-25T08:32:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b697fa640 1 -- 172.21.15.60:0/1553112227 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b740325e0 con 0x560dca254b10 2024-04-25T08:32:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b697fa640 1 -- 172.21.15.60:0/1553112227 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b84042020 con 0x560dca254b10 2024-04-25T08:32:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 10 client.6234 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:32:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 10 client.6234 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:27.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 20 client.6234 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:27.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.970+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca2c2460 0x560dca253210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 4 client.6234 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:32:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560dca198630 con 0x560dca2c2460 2024-04-25T08:32:27.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca2c2460 0x560dca253210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b697fa640 10 client.6234 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b89d9b640 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca2c2460 0x560dca253210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff0115e544c397b server_cookie=178e0047793104fb in_seq=0 out_seq=0 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b697fa640 1 -- 172.21.15.60:0/1553112227 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x560dca198630 con 0x560dca2c2460 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b697fa640 10 client.6234 handle_command_reply: tid=0 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 client.6234 shutdown 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 msgr2=0x560dca2bf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca2bf5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009010 tx=0x7f2b84014000 comp rx=0 tx=0).stop 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 shutdown_connections 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560dca2c2460 0x560dca253210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560dca274940 0x560dca227e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560dca254ef0 0x560dca275b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 --2- 172.21.15.60:0/1553112227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560dca254b10 0x560dca2bf5a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b697fa640 0 client.6234 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 >> 172.21.15.60:0/1553112227 conn(0x560dca227770 msgr2=0x560dca22cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 shutdown_connections 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 1 -- 172.21.15.60:0/1553112227 wait complete. 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.971+0000 7f2b93bd8740 20 client.6234 trim_cache size 0 max 16384 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout: "09d41767-a0b8-4beb-a52b-48a8e837ba2d": { 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "09d41767-a0b8-4beb-a52b-48a8e837ba2d", 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "b34414b0-dcf0-42a8-a5bd-0feed5c216b0": { 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "b34414b0-dcf0-42a8-a5bd-0feed5c216b0", 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "e31d8522-50f6-40d6-9890-5449856da7fc": { 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "e31d8522-50f6-40d6-9890-5449856da7fc", 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "f2166c11-2568-4c63-95b5-d52bcc34f34e": { 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T08:32:27.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "f2166c11-2568-4c63-95b5-d52bcc34f34e", 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b6c06b4d0 msgr2=0x7f2b6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b6c06b4d0 0x7f2b6c06d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2b84007fd0 tx=0x7f2b84009040 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 msgr2=0x7f2b8c1587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 0x7f2b8c1587b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00d4c0 tx=0x7f2b7c00d980 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 shutdown_connections 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b8c161710 0x7f2b8c1587b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b6c06b4d0 0x7f2b6c06d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b8c15dc20 0x7f2b8c0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 --2- 172.21.15.60:0/2212426523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b8c0ced80 0x7f2b8c148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 >> 172.21.15.60:0/2212426523 conn(0x7f2b8c0d0420 msgr2=0x7f2b8c0b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 shutdown_connections 2024-04-25T08:32:27.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:27.972+0000 7f2b90ed2640 1 -- 172.21.15.60:0/2212426523 wait complete. 2024-04-25T08:32:31.024 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:32:31.028+0000 7fcb3a3bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:32:32.983 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:32:32.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:32:33.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.104+0000 7f9091590640 1 Processor -- start 2024-04-25T08:32:33.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.104+0000 7f9091590640 1 -- start start 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9091590640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f908c161240 0x7f908c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9091590640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f908c161890 0x7f908c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9091590640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c0b54a0 0x7f908c0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9091590640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c0b7120 con 0x7f908c161890 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9091590640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c161620 con 0x7f908c161240 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9091590640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c161e00 con 0x7f908c0b54a0 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f908c161890 0x7f908c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f908c161890 0x7f908c0ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49282/0 (socket says 172.21.15.60:49282) 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 -- 172.21.15.60:0/3276076503 learned_addr learned my addr 172.21.15.60:0/3276076503 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f908affd640 1 --2- 172.21.15.60:0/3276076503 >> v2:172.21.15.100:3300/0 conn(0x7f908c161240 0x7f908c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f908b7fe640 1 --2- 172.21.15.60:0/3276076503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c0b54a0 0x7f908c0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 -- 172.21.15.60:0/3276076503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c0b54a0 msgr2=0x7f908c0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 --2- 172.21.15.60:0/3276076503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c0b54a0 0x7f908c0b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 -- 172.21.15.60:0/3276076503 >> v2:172.21.15.100:3300/0 conn(0x7f908c161240 msgr2=0x7f908c0b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 --2- 172.21.15.60:0/3276076503 >> v2:172.21.15.100:3300/0 conn(0x7f908c161240 0x7f908c0b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.105+0000 7f9083fff640 1 -- 172.21.15.60:0/3276076503 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c1620e0 con 0x7f908c161890 2024-04-25T08:32:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.106+0000 7f9083fff640 1 --2- 172.21.15.60:0/3276076503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0ceac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f908c0d0400 tx=0x7f907402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2512ea19e5436132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.106+0000 7f908a7fc640 1 -- 172.21.15.60:0/3276076503 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074037070 con 0x7f908c161890 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.106+0000 7f908a7fc640 1 -- 172.21.15.60:0/3276076503 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074032440 con 0x7f908c161890 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.106+0000 7f908a7fc640 1 -- 172.21.15.60:0/3276076503 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074032810 con 0x7f908c161890 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 -- 172.21.15.60:0/3276076503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 msgr2=0x7f908c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 --2- 172.21.15.60:0/3276076503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0ceac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f908c0d0400 tx=0x7f907402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 -- 172.21.15.60:0/3276076503 shutdown_connections 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 --2- 172.21.15.60:0/3276076503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c0b54a0 0x7f908c0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 --2- 172.21.15.60:0/3276076503 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0ceac0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 --2- 172.21.15.60:0/3276076503 >> v2:172.21.15.100:3300/0 conn(0x7f908c161240 0x7f908c0b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 -- 172.21.15.60:0/3276076503 >> 172.21.15.60:0/3276076503 conn(0x7f908c0d04a0 msgr2=0x7f908c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 -- 172.21.15.60:0/3276076503 shutdown_connections 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.107+0000 7f9091590640 1 -- 172.21.15.60:0/3276076503 wait complete. 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.108+0000 7f9091590640 1 Processor -- start 2024-04-25T08:32:33.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.108+0000 7f9091590640 1 -- start start 2024-04-25T08:32:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9091590640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f908c0b54a0 0x7f908c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9091590640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c161240 0x7f908c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.107 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9091590640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9091590640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0d15f0 con 0x7f908c161890 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9091590640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c1626a0 con 0x7f908c0b54a0 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9091590640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0cf000 con 0x7f908c161240 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49290/0 (socket says 172.21.15.60:49290) 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 -- 172.21.15.60:0/2257565399 learned_addr learned my addr 172.21.15.60:0/2257565399 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908affd640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f908c0b54a0 0x7f908c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f9083fff640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c161240 0x7f908c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 -- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c161240 msgr2=0x7f908c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c161240 0x7f908c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 -- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f908c0b54a0 msgr2=0x7f908c158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f908c0b54a0 0x7f908c158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.109+0000 7f908b7fe640 1 -- 172.21.15.60:0/2257565399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c159000 con 0x7f908c161890 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.110+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0b39c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f908400ef00 tx=0x7f908400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.110+0000 7f90837fe640 1 -- 172.21.15.60:0/2257565399 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084018020 con 0x7f908c161890 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.110+0000 7f90837fe640 1 -- 172.21.15.60:0/2257565399 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084014400 con 0x7f908c161890 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.110+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c0cb8e0 con 0x7f908c161890 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.110+0000 7f90837fe640 1 -- 172.21.15.60:0/2257565399 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840147f0 con 0x7f908c161890 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.110+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c0c79d0 con 0x7f908c161890 2024-04-25T08:32:33.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.111+0000 7f90837fe640 1 -- 172.21.15.60:0/2257565399 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9084005d10 con 0x7f908c161890 2024-04-25T08:32:33.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.112+0000 7f90837fe640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f906c06b4d0 0x7f906c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.112+0000 7f90837fe640 1 -- 172.21.15.60:0/2257565399 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f908408a8e0 con 0x7f908c161890 2024-04-25T08:32:33.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.112+0000 7f908affd640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f906c06b4d0 0x7f906c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.113+0000 7f908affd640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f906c06b4d0 0x7f906c06d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f907c009b40 tx=0x7f907c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.113+0000 7f9094374740 1 Processor -- start 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.113+0000 7f9094374740 1 -- start start 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9094374740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c02a07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04ef0 0x560c02a07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029d7770 0x560c029daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9094374740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c02940da0 con 0x560c02a04b10 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c028e9b80 con 0x560c029d7770 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c0293ce40 con 0x560c02a04ef0 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c02a07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c02a07320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49302/0 (socket says 172.21.15.60:49302) 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 -- 172.21.15.60:0/2672257830 learned_addr learned my addr 172.21.15.60:0/2672257830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f9083fff640 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04ef0 0x560c02a07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.113 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029d7770 0x560c029daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 -- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04ef0 msgr2=0x560c02a07860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04ef0 0x560c02a07860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 -- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029d7770 msgr2=0x560c029daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.114+0000 7f908affd640 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029d7770 0x560c029daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f908affd640 1 -- 172.21.15.60:0/2672257830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c028f3f10 con 0x560c02a04b10 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f908affd640 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c02a07320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f907c006a90 tx=0x7f907c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c4620034ecf8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f9080ff9640 1 -- 172.21.15.60:0/2672257830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c01b070 con 0x560c02a04b10 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f9080ff9640 1 -- 172.21.15.60:0/2672257830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c027070 con 0x560c02a04b10 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f9080ff9640 1 -- 172.21.15.60:0/2672257830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c003a40 con 0x560c02a04b10 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f9094374740 1 -- 172.21.15.60:0/2672257830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 msgr2=0x560c02a07320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.115+0000 7f9094374740 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c02a07320 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f907c006a90 tx=0x7f907c014000 comp rx=0 tx=0).stop 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 -- 172.21.15.60:0/2672257830 shutdown_connections 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029d7770 0x560c029daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04ef0 0x560c02a07860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 --2- 172.21.15.60:0/2672257830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c02a07320 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 -- 172.21.15.60:0/2672257830 >> 172.21.15.60:0/2672257830 conn(0x560c029d99e0 msgr2=0x560c02a22320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 -- 172.21.15.60:0/2672257830 shutdown_connections 2024-04-25T08:32:33.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 -- 172.21.15.60:0/2672257830 wait complete. 2024-04-25T08:32:33.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 -- start start 2024-04-25T08:32:33.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.116+0000 7f9094374740 1 Processor -- start 2024-04-25T08:32:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c02a26020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c02a0b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 0x560c02a32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c029dcb90 con 0x560c029fb850 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c028df4b0 con 0x560c02a04ef0 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c029dce70 con 0x560c02a04b10 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9094374740 10 client.0 _ll_register_callbacks cb 0x560c0291f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f908b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 0x560c02a32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f908b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 0x560c02a32000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49304/0 (socket says 172.21.15.60:49304) 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f908b7fe640 1 -- 172.21.15.60:0/625498636 learned_addr learned my addr 172.21.15.60:0/625498636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f9083fff640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c02a0b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.117+0000 7f908affd640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c02a26020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f908b7fe640 1 -- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 msgr2=0x560c02a26020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f908b7fe640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c02a26020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f908b7fe640 1 -- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 msgr2=0x560c02a0b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f908b7fe640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c02a0b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f908b7fe640 1 -- 172.21.15.60:0/625498636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c0271f7d0 con 0x560c029fb850 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f908b7fe640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 0x560c02a32000 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f90840553e0 tx=0x7f90840589a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edee307ba9acfc64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9069ffb640 1 -- 172.21.15.60:0/625498636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084061350 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9069ffb640 1 -- 172.21.15.60:0/625498636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90840583d0 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f907c027070 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9069ffb640 1 -- 172.21.15.60:0/625498636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9084077050 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 10 client.6246 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560c026553f0 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9069ffb640 1 -- 172.21.15.60:0/625498636 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c027070 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9069ffb640 1 -- 172.21.15.60:0/625498636 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9084054cd0 con 0x560c029fb850 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 10 client.6246 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 10 client.6246 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 20 client.6246 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c029d6ad0 0x560c02a23450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 4 client.6246 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.118+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c0294e2e0 con 0x560c029d6ad0 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.119+0000 7f908affd640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c029d6ad0 0x560c02a23450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.119+0000 7f908affd640 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c029d6ad0 0x560c02a23450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4956da54a4d5abc0 server_cookie=64d2ea0e12be81b6 in_seq=0 out_seq=0 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.119+0000 7f9069ffb640 10 client.6246 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.119+0000 7f9069ffb640 1 -- 172.21.15.60:0/625498636 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c0294e2e0 con 0x560c029d6ad0 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.119+0000 7f9069ffb640 10 client.6246 handle_command_reply: tid=0 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.119+0000 7f9094374740 1 client.6246 shutdown 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 msgr2=0x560c02a32000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 0x560c02a32000 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f90840553e0 tx=0x7f90840589a0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 shutdown_connections 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c029fb850 0x560c02a32000 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c02a0b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c029d6ad0 0x560c02a23450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 --2- 172.21.15.60:0/625498636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c02a26020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 >> 172.21.15.60:0/625498636 conn(0x560c029d7770 msgr2=0x560c029dc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9069ffb640 0 client.6246 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 shutdown_connections 2024-04-25T08:32:33.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 1 -- 172.21.15.60:0/625498636 wait complete. 2024-04-25T08:32:33.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.120+0000 7f9094374740 20 client.6246 trim_cache size 0 max 16384 2024-04-25T08:32:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.135+0000 7f9094374740 1 Processor -- start 2024-04-25T08:32:33.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.135+0000 7f9094374740 1 -- start start 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c029fb850 0x560c02a093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9094374740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c029daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c029db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9094374740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c200540 con 0x560c02a04b10 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c1ff700 con 0x560c02a04ef0 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0c9250 con 0x560c029fb850 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c029daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c029daf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49312/0 (socket says 172.21.15.60:49312) 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 -- 172.21.15.60:0/4250144252 learned_addr learned my addr 172.21.15.60:0/4250144252 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f908b7fe640 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c029db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f908affd640 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c029fb850 0x560c02a093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 -- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c029fb850 msgr2=0x560c02a093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c029fb850 0x560c02a093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 -- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 msgr2=0x560c029db4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c029db4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 -- 172.21.15.60:0/4250144252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c026150f0 con 0x560c02a04b10 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f908b7fe640 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c029db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f9083fff640 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c029daf80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9074009960 tx=0x7f907402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1c10b89cabef60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f906bfff640 1 -- 172.21.15.60:0/4250144252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907404b070 con 0x560c02a04b10 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f906bfff640 1 -- 172.21.15.60:0/4250144252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074032780 con 0x560c02a04b10 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.136+0000 7f906bfff640 1 -- 172.21.15.60:0/4250144252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074037040 con 0x560c02a04b10 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 -- 172.21.15.60:0/4250144252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 msgr2=0x560c029daf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c029daf80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9074009960 tx=0x7f907402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 -- 172.21.15.60:0/4250144252 shutdown_connections 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c02a04ef0 0x560c029db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04b10 0x560c029daf80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 --2- 172.21.15.60:0/4250144252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c029fb850 0x560c02a093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 -- 172.21.15.60:0/4250144252 >> 172.21.15.60:0/4250144252 conn(0x560c029d7770 msgr2=0x560c02a0b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 -- 172.21.15.60:0/4250144252 shutdown_connections 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.137+0000 7f9094374740 1 -- 172.21.15.60:0/4250144252 wait complete. 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 -- start start 2024-04-25T08:32:33.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 Processor -- start 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029fb850 0x560c02a8c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c029d7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 0x560c029daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c02a09e00 con 0x560c02a04ef0 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c1ff100 con 0x560c029fb850 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c02902520 con 0x560c02a04b10 2024-04-25T08:32:33.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.138+0000 7f9094374740 10 client.0 _ll_register_callbacks cb 0x7f907c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 0x560c029daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 0x560c029daf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49326/0 (socket says 172.21.15.60:49326) 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 -- 172.21.15.60:0/2255739083 learned_addr learned my addr 172.21.15.60:0/2255739083 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029fb850 0x560c02a8c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9083fff640 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c029d7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 -- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 msgr2=0x560c029d7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c029d7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 -- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029fb850 msgr2=0x560c02a8c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029fb850 0x560c02a8c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 -- 172.21.15.60:0/2255739083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c02a31150 con 0x560c02a04ef0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f908b7fe640 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 0x560c029daf80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f908400b980 tx=0x7f9084055220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42517362d9980c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9069ffb640 1 -- 172.21.15.60:0/2255739083 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840583d0 con 0x560c02a04ef0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9069ffb640 1 -- 172.21.15.60:0/2255739083 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9084058570 con 0x560c02a04ef0 2024-04-25T08:32:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9074032780 con 0x560c02a04ef0 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9069ffb640 1 -- 172.21.15.60:0/2255739083 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9084066020 con 0x560c02a04ef0 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9094374740 10 client.6252 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.139+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560c02a8cd50 con 0x560c02a04ef0 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9069ffb640 1 -- 172.21.15.60:0/2255739083 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074032780 con 0x560c02a04ef0 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9069ffb640 1 -- 172.21.15.60:0/2255739083 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9084082100 con 0x560c02a04ef0 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 10 client.6252 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 10 client.6252 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 20 client.6252 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c02a75110 0x560c029db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 4 client.6252 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c02948630 con 0x560c02a75110 2024-04-25T08:32:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f908affd640 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c02a75110 0x560c029db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9069ffb640 10 client.6252 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f908affd640 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c02a75110 0x560c029db4c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b44314ca6f0b2d2 server_cookie=fa05db32c4e04857 in_seq=0 out_seq=0 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9069ffb640 1 -- 172.21.15.60:0/2255739083 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c02948630 con 0x560c02a75110 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9069ffb640 10 client.6252 handle_command_reply: tid=0 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 client.6252 shutdown 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 msgr2=0x560c029daf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 0x560c029daf80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f908400b980 tx=0x7f9084055220 comp rx=0 tx=0).stop 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 shutdown_connections 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560c02a04ef0 0x560c029daf80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x560c02a75110 0x560c029db4c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560c02a04b10 0x560c029d7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 --2- 172.21.15.60:0/2255739083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c029fb850 0x560c02a8c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9069ffb640 0 client.6252 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.140+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 >> 172.21.15.60:0/2255739083 conn(0x560c029d7770 msgr2=0x560c029d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 shutdown_connections 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9094374740 1 -- 172.21.15.60:0/2255739083 wait complete. 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9094374740 20 client.6252 trim_cache size 0 max 16384 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:32:33.144 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f906c06b4d0 msgr2=0x7f906c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f906c06b4d0 0x7f906c06d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f907c009b40 tx=0x7f907c009190 comp rx=0 tx=0).stop 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 msgr2=0x7f908c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0b39c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f908400ef00 tx=0x7f908400b480 comp rx=0 tx=0).stop 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 shutdown_connections 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f908c161890 0x7f908c0b39c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f908c161240 0x7f908c0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f906c06b4d0 0x7f906c06d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 --2- 172.21.15.60:0/2257565399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f908c0b54a0 0x7f908c158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 >> 172.21.15.60:0/2257565399 conn(0x7f908c0d04a0 msgr2=0x7f908c0b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 shutdown_connections 2024-04-25T08:32:33.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.141+0000 7f9091590640 1 -- 172.21.15.60:0/2257565399 wait complete. 2024-04-25T08:32:33.148 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:32:33.148 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:33.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:32:33.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.254+0000 7f5f140b3640 1 Processor -- start 2024-04-25T08:32:33.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.255+0000 7f5f140b3640 1 -- start start 2024-04-25T08:32:33.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.255+0000 7f5f140b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f0c15bb50 0x7f5f0c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.255+0000 7f5f140b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.255+0000 7f5f140b3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c0c75e0 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.255+0000 7f5f140b3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c0c8680 con 0x7f5f0c15bb50 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.255+0000 7f5f140b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0c78c0 con 0x7f5f0c0cfd10 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f1164e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5f0c0c70d0 0x7f5f0c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55792/0 (socket says 172.21.15.60:55792) 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f1164e640 1 -- 172.21.15.60:0/3051210614 learned_addr learned my addr 172.21.15.60:0/3051210614 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3051210614 >> v2:172.21.15.100:3300/0 conn(0x7f5f0c15bb50 0x7f5f0c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f12650640 1 --2- 172.21.15.60:0/3051210614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f10e4d640 1 -- 172.21.15.60:0/3051210614 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267920789 0 0) 0x7f5f0c0c75e0 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f10e4d640 1 -- 172.21.15.60:0/3051210614 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef8003520 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f12650640 1 -- 172.21.15.60:0/3051210614 >> v2:172.21.15.100:3300/0 conn(0x7f5f0c15bb50 msgr2=0x7f5f0c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f12650640 1 --2- 172.21.15.60:0/3051210614 >> v2:172.21.15.100:3300/0 conn(0x7f5f0c15bb50 0x7f5f0c0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f12650640 1 -- 172.21.15.60:0/3051210614 >> v1:172.21.15.60:6789/0 conn(0x7f5f0c0c70d0 legacy=0x7f5f0c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.256+0000 7f5f12650640 1 -- 172.21.15.60:0/3051210614 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0c7ba0 con 0x7f5f0c0cfd10 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f12650640 1 --2- 172.21.15.60:0/3051210614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c0cbf20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009d90 tx=0x7f5f0402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2100794648328562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f10e4d640 1 -- 172.21.15.60:0/3051210614 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04007740 con 0x7f5f0c0cfd10 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f10e4d640 1 -- 172.21.15.60:0/3051210614 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f040078e0 con 0x7f5f0c0cfd10 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f10e4d640 1 -- 172.21.15.60:0/3051210614 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04007c90 con 0x7f5f0c0cfd10 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f140b3640 1 -- 172.21.15.60:0/3051210614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 msgr2=0x7f5f0c0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3051210614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c0cbf20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009d90 tx=0x7f5f0402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f140b3640 1 -- 172.21.15.60:0/3051210614 shutdown_connections 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3051210614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c0cbf20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.257+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3051210614 >> v2:172.21.15.100:3300/0 conn(0x7f5f0c15bb50 0x7f5f0c0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.258+0000 7f5f140b3640 1 -- 172.21.15.60:0/3051210614 >> 172.21.15.60:0/3051210614 conn(0x7f5f0c06f500 msgr2=0x7f5f0c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.258+0000 7f5f140b3640 1 -- 172.21.15.60:0/3051210614 shutdown_connections 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.258+0000 7f5f140b3640 1 -- 172.21.15.60:0/3051210614 wait complete. 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 Processor -- start 2024-04-25T08:32:33.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 -- start start 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 0x7f5f0c14e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c15bb50 0x7f5f0c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15f6d0 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0c8160 con 0x7f5f0c15bb50 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.259+0000 7f5f140b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15f850 con 0x7f5f0c0cfd10 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 0x7f5f0c14e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 0x7f5f0c14e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49334/0 (socket says 172.21.15.60:49334) 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3361165211 learned_addr learned my addr 172.21.15.60:0/3361165211 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f12650640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c15bb50 0x7f5f0c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 msgr2=0x7f5f0c15da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c15bb50 msgr2=0x7f5f0c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c15bb50 0x7f5f0c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3361165211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c15c9a0 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 0x7f5f0c14e4a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e870 tx=0x7f5f0800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f027fc640 1 -- 172.21.15.60:0/3361165211 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08017070 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f027fc640 1 -- 172.21.15.60:0/3361165211 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0800c940 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c15cce0 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f027fc640 1 -- 172.21.15.60:0/3361165211 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0800cd30 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.260+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c15d520 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.262+0000 7f5f027fc640 1 -- 172.21.15.60:0/3361165211 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f0801a070 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.262+0000 7f5f027fc640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ef006b4d0 0x7f5ef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.262+0000 7f5f027fc640 1 -- 172.21.15.60:0/3361165211 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f0808a1e0 con 0x7f5f0c0c70d0 2024-04-25T08:32:33.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.263+0000 7f5f1164e640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ef006b4d0 0x7f5ef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.263+0000 7f5f1164e640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ef006b4d0 0x7f5ef006d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5efc000f60 tx=0x7f5efc0078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.263+0000 7f5f16dbc740 1 Processor -- start 2024-04-25T08:32:33.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.263+0000 7f5f16dbc740 1 -- start start 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 0x557785083f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557784fe9e00 con 0x557785080700 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557784f92d20 con 0x5577850ade80 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557784fe5ea0 con 0x5577850adaa0 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 0x557785083f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 0x557785083f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49342/0 (socket says 172.21.15.60:49342) 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 -- 172.21.15.60:0/1998073331 learned_addr learned my addr 172.21.15.60:0/1998073331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f1164e640 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 -- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 msgr2=0x5577850b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 -- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 msgr2=0x5577850b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.264+0000 7f5f12650640 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5f12650640 1 -- 172.21.15.60:0/1998073331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557784fb4df0 con 0x557785080700 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5f12650640 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 0x557785083f10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009eb0 tx=0x7f5f0402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa63254d1ceccef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5eef7fe640 1 -- 172.21.15.60:0/1998073331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f040077c0 con 0x557785080700 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5eef7fe640 1 -- 172.21.15.60:0/1998073331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f04007960 con 0x557785080700 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5eef7fe640 1 -- 172.21.15.60:0/1998073331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04007d50 con 0x557785080700 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1998073331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 msgr2=0x557785083f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.265+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 0x557785083f10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009eb0 tx=0x7f5f0402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1998073331 shutdown_connections 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x557785080700 0x557785083f10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1998073331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1998073331 >> 172.21.15.60:0/1998073331 conn(0x557785082970 msgr2=0x5577850cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1998073331 shutdown_connections 2024-04-25T08:32:33.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1998073331 wait complete. 2024-04-25T08:32:33.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 -- start start 2024-04-25T08:32:33.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.266+0000 7f5f16dbc740 1 Processor -- start 2024-04-25T08:32:33.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 0x5577850a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850a4d70 0x5577850a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x557785085b20 con 0x5577850adaa0 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x557784f88590 con 0x5577850ade80 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x557785085e00 con 0x5577850a4d70 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f16dbc740 10 client.0 _ll_register_callbacks cb 0x557784f94e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f11e4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 0x5577850a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f11e4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 0x5577850a6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49354/0 (socket says 172.21.15.60:49354) 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3185802850 learned_addr learned my addr 172.21.15.60:0/3185802850 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f1164e640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f12650640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850a4d70 0x5577850a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.267+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850a4d70 msgr2=0x5577850a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850a4d70 0x5577850a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 msgr2=0x5577850a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f11e4f640 1 -- 172.21.15.60:0/3185802850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557784cfe3f0 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f12650640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850a4d70 0x5577850a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f1164e640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 0x5577850a6030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5f08058560 tx=0x7f5f08059020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc985142f5d1968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5eecff9640 1 -- 172.21.15.60:0/3185802850 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08021030 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5eecff9640 1 -- 172.21.15.60:0/3185802850 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f08070380 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f04007960 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5eecff9640 1 -- 172.21.15.60:0/3185802850 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f08060f70 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 10 client.6264 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x557784dc87d0 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5eecff9640 1 -- 172.21.15.60:0/3185802850 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f04007960 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5eecff9640 1 -- 172.21.15.60:0/3185802850 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f08024060 con 0x5577850adaa0 2024-04-25T08:32:33.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 10 client.6264 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 10 client.6264 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 20 client.6264 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5577850def90 0x5577850cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 4 client.6264 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557784ff7340 con 0x5577850def90 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.268+0000 7f5f1164e640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5577850def90 0x5577850cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5eecff9640 10 client.6264 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5f1164e640 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5577850def90 0x5577850cd3f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357d3557887debf5 server_cookie=808f734b11790418 in_seq=0 out_seq=0 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5eecff9640 1 -- 172.21.15.60:0/3185802850 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557784ff7340 con 0x5577850def90 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5eecff9640 10 client.6264 handle_command_reply: tid=0 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5f16dbc740 1 client.6264 shutdown 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 msgr2=0x5577850a6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.269+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 0x5577850a6030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5f08058560 tx=0x7f5f08059020 comp rx=0 tx=0).stop 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 shutdown_connections 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5577850def90 0x5577850cd3f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850a4d70 0x5577850a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850ade80 0x5577850a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5eecff9640 0 client.6264 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/3185802850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850adaa0 0x5577850a6030 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 >> 172.21.15.60:0/3185802850 conn(0x557785080700 msgr2=0x5577850ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 shutdown_connections 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 1 -- 172.21.15.60:0/3185802850 wait complete. 2024-04-25T08:32:33.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.270+0000 7f5f16dbc740 20 client.6264 trim_cache size 0 max 16384 2024-04-25T08:32:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 Processor -- start 2024-04-25T08:32:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 -- start start 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 0x55778507f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850adaa0 0x5577850a3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850ade80 0x5577850a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1fdfd0 con 0x5577850a4d70 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1fd1d0 con 0x5577850adaa0 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0b5240 con 0x5577850ade80 2024-04-25T08:32:33.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f11e4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 0x55778507f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f11e4f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 0x55778507f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49366/0 (socket says 172.21.15.60:49366) 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.282+0000 7f5f11e4f640 1 -- 172.21.15.60:0/1267069643 learned_addr learned my addr 172.21.15.60:0/1267069643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f11e4f640 1 -- 172.21.15.60:0/1267069643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850ade80 msgr2=0x5577850a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850ade80 0x5577850a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f11e4f640 1 -- 172.21.15.60:0/1267069643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850adaa0 msgr2=0x5577850a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850adaa0 0x5577850a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f11e4f640 1 -- 172.21.15.60:0/1267069643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577850ef820 con 0x5577850a4d70 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 0x55778507f8c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5f0808a1b0 tx=0x7f5f0805a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dc7b3a42cd5939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5eeeffd640 1 -- 172.21.15.60:0/1267069643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08021070 con 0x5577850a4d70 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5eeeffd640 1 -- 172.21.15.60:0/1267069643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08070180 con 0x5577850a4d70 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5eeeffd640 1 -- 172.21.15.60:0/1267069643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08059b90 con 0x5577850a4d70 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1267069643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 msgr2=0x55778507f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 0x55778507f8c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5f0808a1b0 tx=0x7f5f0805a000 comp rx=0 tx=0).stop 2024-04-25T08:32:33.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1267069643 shutdown_connections 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.283+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850ade80 0x5577850a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850adaa0 0x5577850a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1267069643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850a4d70 0x55778507f8c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1267069643 >> 172.21.15.60:0/1267069643 conn(0x557785080700 msgr2=0x5577850a5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1267069643 shutdown_connections 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1267069643 wait complete. 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 -- start start 2024-04-25T08:32:33.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.284+0000 7f5f16dbc740 1 Processor -- start 2024-04-25T08:32:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850a4d70 0x55778513d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 0x5577850a6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0c95c0 con 0x5577850ade80 2024-04-25T08:32:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577850dafe0 con 0x5577850a4d70 2024-04-25T08:32:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5577850cd330 con 0x5577850adaa0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f16dbc740 10 client.0 _ll_register_callbacks cb 0x7f5ef80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f12650640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 0x5577850a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f12650640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 0x5577850a6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49382/0 (socket says 172.21.15.60:49382) 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f12650640 1 -- 172.21.15.60:0/1246749210 learned_addr learned my addr 172.21.15.60:0/1246749210 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f1164e640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.285+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850a4d70 0x55778513d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f12650640 1 -- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 msgr2=0x5577850a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f12650640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f12650640 1 -- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850a4d70 msgr2=0x55778513d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f12650640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850a4d70 0x55778513d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f12650640 1 -- 172.21.15.60:0/1246749210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577850810e0 con 0x5577850ade80 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f1164e640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f12650640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 0x5577850a6380 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f0402ace0 tx=0x7f5f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9142e28b3744fdb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5eecff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5eecff9640 1 -- 172.21.15.60:0/1246749210 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04009a30 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5eecff9640 1 -- 172.21.15.60:0/1246749210 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f04003ba0 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f08070180 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5eecff9640 1 -- 172.21.15.60:0/1246749210 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0405a050 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 10 client.6270 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5577850cd100 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5eecff9640 1 -- 172.21.15.60:0/1246749210 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08070180 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5eecff9640 1 -- 172.21.15.60:0/1246749210 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f0405d020 con 0x5577850ade80 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 10 client.6270 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 10 client.6270 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 20 client.6270 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x557785087b10 0x5577850cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 4 client.6270 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557784ff1690 con 0x557785087b10 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.286+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x557785087b10 0x5577850cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5eecff9640 10 client.6270 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f11e4f640 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x557785087b10 0x5577850cc3e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b0e9742e99fb1d5 server_cookie=99530cb2d183ad8f in_seq=0 out_seq=0 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5eecff9640 1 -- 172.21.15.60:0/1246749210 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x557784ff1690 con 0x557785087b10 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5eecff9640 10 client.6270 handle_command_reply: tid=0 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 client.6270 shutdown 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 msgr2=0x5577850a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 0x5577850a6380 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5f0402ace0 tx=0x7f5f04004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 shutdown_connections 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5577850ade80 0x5577850a6380 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x557785087b10 0x5577850cc3e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5577850adaa0 0x5577850a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5eecff9640 0 client.6270 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 --2- 172.21.15.60:0/1246749210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5577850a4d70 0x55778513d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 >> 172.21.15.60:0/1246749210 conn(0x557785080700 msgr2=0x5577850a70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 shutdown_connections 2024-04-25T08:32:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 1 -- 172.21.15.60:0/1246749210 wait complete. 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.287+0000 7f5f16dbc740 20 client.6270 trim_cache size 0 max 16384 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 2897141188, 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511630186, 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:33.290 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:32:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ef006b4d0 msgr2=0x7f5ef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ef006b4d0 0x7f5ef006d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5efc000f60 tx=0x7f5efc0078f0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 msgr2=0x7f5f0c14e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 0x7f5f0c14e4a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e870 tx=0x7f5f0800ed30 comp rx=0 tx=0).stop 2024-04-25T08:32:33.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 shutdown_connections 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ef006b4d0 0x7f5ef006d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f0c15bb50 0x7f5f0c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f0c0cfd10 0x7f5f0c15da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 --2- 172.21.15.60:0/3361165211 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f0c0c70d0 0x7f5f0c14e4a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 >> 172.21.15.60:0/3361165211 conn(0x7f5f0c06f500 msgr2=0x7f5f0c0b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 shutdown_connections 2024-04-25T08:32:33.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.288+0000 7f5f140b3640 1 -- 172.21.15.60:0/3361165211 wait complete. 2024-04-25T08:32:33.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.399+0000 7f7197770640 1 Processor -- start 2024-04-25T08:32:33.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.399+0000 7f7197770640 1 -- start start 2024-04-25T08:32:33.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7197770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 0x7f7190168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7197770640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71900ceee0 con 0x7f71900b6cb0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7197770640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71900d1d10 con 0x7f719015db70 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7197770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71900d3800 con 0x7f71900b5c30 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7194d0b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f71900b6cb0 0x7f71900b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55802/0 (socket says 172.21.15.60:55802) 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7194d0b640 1 -- 172.21.15.60:0/2337026973 learned_addr learned my addr 172.21.15.60:0/2337026973 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7195d0d640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 0x7f7190168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713717477 0 0) 0x7f71900ceee0 con 0x7f71900b6cb0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f717c003520 con 0x7f71900b6cb0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 718810080 0 0) 0x7f717c003520 con 0x7f71900b6cb0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 msgr2=0x7f7190168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 0x7f7190168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 >> v1:172.21.15.100:6789/0 conn(0x7f719015db70 legacy=0x7f7190162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900d1ff0 con 0x7f71900b6cb0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f7180001670 con 0x7f71900b6cb0 2024-04-25T08:32:33.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 >> v1:172.21.15.60:6789/0 conn(0x7f71900b6cb0 legacy=0x7f71900b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f717c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 0x7f717c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.400+0000 7f7187fff640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f717c004d40 0x7f717c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71900d3800 con 0x7f717c0039e0 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c00b9f0 con 0x7f717c004d40 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c00bbf0 con 0x7f71900b5c30 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f719550c640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f717c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f7194d0b640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 0x7f717c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f7195d0d640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f717c004d40 0x7f717c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f719550c640 1 -- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 msgr2=0x7f717c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f719550c640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900b5c30 0x7f717c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f719550c640 1 -- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f717c004d40 msgr2=0x7f717c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f719550c640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f717c004d40 0x7f717c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f719550c640 1 -- 172.21.15.60:0/2337026973 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c003520 con 0x7f717c0039e0 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.401+0000 7f7195d0d640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f717c004d40 0x7f717c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:33.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f719550c640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f717c003dc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f718c001860 tx=0x7f717c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7491d6e25357ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c037070 con 0x7f717c0039e0 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c02ab60 con 0x7f717c0039e0 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7187fff640 1 -- 172.21.15.60:0/2337026973 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c001430 con 0x7f717c0039e0 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7197770640 1 -- 172.21.15.60:0/2337026973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 msgr2=0x7f717c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7197770640 1 --2- 172.21.15.60:0/2337026973 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f717c003dc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f718c001860 tx=0x7f717c005830 comp rx=0 tx=0).stop 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7195d0d640 1 -- 172.21.15.60:0/2337026973 reap_dead start 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7197770640 1 -- 172.21.15.60:0/2337026973 shutdown_connections 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7197770640 1 -- 172.21.15.60:0/2337026973 >> 172.21.15.60:0/2337026973 conn(0x7f71900d0520 msgr2=0x7f71900d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7197770640 1 -- 172.21.15.60:0/2337026973 shutdown_connections 2024-04-25T08:32:33.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.402+0000 7f7197770640 1 -- 172.21.15.60:0/2337026973 wait complete. 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 Processor -- start 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 -- start start 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f719014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900ccb00 0x7f71900caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71900cb430 0x7f71901f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71900ce530 con 0x7f717c0039e0 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719015df50 con 0x7f71900cb430 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f7197770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71900cf1c0 con 0x7f71900ccb00 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f719550c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f719014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f719550c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f719014cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49412/0 (socket says 172.21.15.60:49412) 2024-04-25T08:32:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.404+0000 7f719550c640 1 -- 172.21.15.60:0/716558844 learned_addr learned my addr 172.21.15.60:0/716558844 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7195d0d640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71900cb430 0x7f71901f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7194d0b640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900ccb00 0x7f71900caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f719550c640 1 -- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900ccb00 msgr2=0x7f71900caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f719550c640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900ccb00 0x7f71900caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f719550c640 1 -- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71900cb430 msgr2=0x7f71901f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f719550c640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71900cb430 0x7f71901f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f719550c640 1 -- 172.21.15.60:0/716558844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900aa600 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f719550c640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f719014cda0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f718c0018c0 tx=0x7f718c001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c040020 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c005070 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c037040 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71900cba60 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.405+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71900cc020 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.406+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7190073990 con 0x7f717c0039e0 2024-04-25T08:32:33.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.407+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f718c043020 con 0x7f717c0039e0 2024-04-25T08:32:33.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.407+0000 7f7185ffb640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f717406b4d0 0x7f717406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.407+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f718c0aa520 con 0x7f717c0039e0 2024-04-25T08:32:33.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.407+0000 7f7194d0b640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f717406b4d0 0x7f717406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.408+0000 7f7194d0b640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f717406b4d0 0x7f717406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71900d09e0 tx=0x7f7180009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.411+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f718c04a030 con 0x7f717c0039e0 2024-04-25T08:32:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.629+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71900ca400 con 0x7f717c0039e0 2024-04-25T08:32:33.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.630+0000 7f7185ffb640 1 -- 172.21.15.60:0/716558844 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f718c04c300 con 0x7f717c0039e0 2024-04-25T08:32:33.626 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:33.626 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:31.022346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:33.627 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-04-25T08:32:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f717406b4d0 msgr2=0x7f717406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f717406b4d0 0x7f717406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71900d09e0 tx=0x7f7180009040 comp rx=0 tx=0).stop 2024-04-25T08:32:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 msgr2=0x7f719014cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f719014cda0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f718c0018c0 tx=0x7f718c001230 comp rx=0 tx=0).stop 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 shutdown_connections 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f717406b4d0 0x7f717406d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71900cb430 0x7f71901f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f71900ccb00 0x7f71900caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 --2- 172.21.15.60:0/716558844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f717c0039e0 0x7f719014cda0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 >> 172.21.15.60:0/716558844 conn(0x7f71900d0520 msgr2=0x7f719015cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 shutdown_connections 2024-04-25T08:32:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.632+0000 7f7197770640 1 -- 172.21.15.60:0/716558844 wait complete. 2024-04-25T08:32:33.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2897141188 2024-04-25T08:32:33.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.746+0000 7f6e5444a640 1 Processor -- start 2024-04-25T08:32:33.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.746+0000 7f6e5444a640 1 -- start start 2024-04-25T08:32:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.747+0000 7f6e5444a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e4c0cf600 0x7f6e4c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.747+0000 7f6e5444a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c0c75e0 con 0x7f6e4c15bb50 2024-04-25T08:32:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.747+0000 7f6e521e6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6e4c15bb50 0x7f6e4c0c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55808/0 (socket says 172.21.15.60:55808) 2024-04-25T08:32:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.747+0000 7f6e521e6640 1 -- 172.21.15.60:0/1308164673 learned_addr learned my addr 172.21.15.60:0/1308164673 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c0c78c0 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c8680 con 0x7f6e4c0cf600 2024-04-25T08:32:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e519e5640 1 --2- 172.21.15.60:0/1308164673 >> v2:172.21.15.100:3300/0 conn(0x7f6e4c0cf600 0x7f6e4c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e511e4640 1 -- 172.21.15.60:0/1308164673 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941226321 0 0) 0x7f6e4c0c75e0 con 0x7f6e4c15bb50 2024-04-25T08:32:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e511e4640 1 -- 172.21.15.60:0/1308164673 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e34003520 con 0x7f6e4c15bb50 2024-04-25T08:32:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e519e5640 1 -- 172.21.15.60:0/1308164673 >> v1:172.21.15.100:6789/0 conn(0x7f6e4c0cbf20 legacy=0x7f6e4c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e519e5640 1 -- 172.21.15.60:0/1308164673 >> v1:172.21.15.60:6789/0 conn(0x7f6e4c15bb50 legacy=0x7f6e4c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.748+0000 7f6e519e5640 1 -- 172.21.15.60:0/1308164673 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0c7ba0 con 0x7f6e4c0cf600 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.749+0000 7f6e519e5640 1 --2- 172.21.15.60:0/1308164673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0cfa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e400097a0 tx=0x7f6e4002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ead5b2553fed89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.749+0000 7f6e511e4640 1 -- 172.21.15.60:0/1308164673 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40037070 con 0x7f6e4c0cf600 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.749+0000 7f6e511e4640 1 -- 172.21.15.60:0/1308164673 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40032640 con 0x7f6e4c0cf600 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.749+0000 7f6e511e4640 1 -- 172.21.15.60:0/1308164673 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40032a10 con 0x7f6e4c0cf600 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.749+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 msgr2=0x7f6e4c0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.749+0000 7f6e5444a640 1 --2- 172.21.15.60:0/1308164673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0cfa40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e400097a0 tx=0x7f6e4002a2c0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.750+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 shutdown_connections 2024-04-25T08:32:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.750+0000 7f6e5444a640 1 --2- 172.21.15.60:0/1308164673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0cfa40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.750+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 >> 172.21.15.60:0/1308164673 conn(0x7f6e4c06f500 msgr2=0x7f6e4c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.750+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 shutdown_connections 2024-04-25T08:32:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.750+0000 7f6e5444a640 1 -- 172.21.15.60:0/1308164673 wait complete. 2024-04-25T08:32:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.751+0000 7f6e5444a640 1 Processor -- start 2024-04-25T08:32:33.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.751+0000 7f6e5444a640 1 -- start start 2024-04-25T08:32:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 0x7f6e4c14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c15bb50 0x7f6e4c0b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c8160 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0cff80 con 0x7f6e4c0cf600 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c72a0 con 0x7f6e4c15bb50 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 0x7f6e4c14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 0x7f6e4c14f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49426/0 (socket says 172.21.15.60:49426) 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 -- 172.21.15.60:0/115887030 learned_addr learned my addr 172.21.15.60:0/115887030 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e519e5640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e529e7640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c15bb50 0x7f6e4c0b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 -- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c15bb50 msgr2=0x7f6e4c0b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c15bb50 0x7f6e4c0b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 -- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 msgr2=0x7f6e4c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 -- 172.21.15.60:0/115887030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0b5090 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e519e5640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e529e7640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c15bb50 0x7f6e4c0b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:33.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e521e6640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 0x7f6e4c14f320 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400cbc0 tx=0x7f6e4400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e4affd640 1 -- 172.21.15.60:0/115887030 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44017070 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e4affd640 1 -- 172.21.15.60:0/115887030 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4400ad40 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0b5370 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e4affd640 1 -- 172.21.15.60:0/115887030 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44012620 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.752+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c0b5930 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.754+0000 7f6e4affd640 1 -- 172.21.15.60:0/115887030 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e4401a070 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.754+0000 7f6e4affd640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e3806b4d0 0x7f6e3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.754+0000 7f6e4affd640 1 -- 172.21.15.60:0/115887030 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e4408a160 con 0x7f6e4c0cbf20 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.754+0000 7f6e519e5640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e3806b4d0 0x7f6e3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.755+0000 7f6e519e5640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e3806b4d0 0x7f6e3806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6e40034000 tx=0x7f6e4002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.755+0000 7f6e57229740 1 Processor -- start 2024-04-25T08:32:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 -- start start 2024-04-25T08:32:33.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f980790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f980cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f950be0 0x56041f9543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041f8b9ac0 con 0x56041f97df80 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041f862db0 con 0x56041f950be0 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56041f8b5b60 con 0x56041f97e360 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e521e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f980790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e521e6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f980790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49442/0 (socket says 172.21.15.60:49442) 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.756+0000 7f6e521e6640 1 -- 172.21.15.60:0/1422297084 learned_addr learned my addr 172.21.15.60:0/1422297084 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e519e5640 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f980cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e529e7640 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f950be0 0x56041f9543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e521e6640 1 -- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 msgr2=0x56041f980cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e521e6640 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f980cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e521e6640 1 -- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f950be0 msgr2=0x56041f9543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e521e6640 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f950be0 0x56041f9543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e521e6640 1 -- 172.21.15.60:0/1422297084 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56041f69a710 con 0x56041f97df80 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e519e5640 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f980cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e521e6640 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f980790 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400afd0 tx=0x7f6e440588a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3216c38a20d9ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e33fff640 1 -- 172.21.15.60:0/1422297084 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44056ae0 con 0x56041f97df80 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e33fff640 1 -- 172.21.15.60:0/1422297084 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44054e50 con 0x56041f97df80 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.757+0000 7f6e33fff640 1 -- 172.21.15.60:0/1422297084 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44060c80 con 0x56041f97df80 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 -- 172.21.15.60:0/1422297084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 msgr2=0x56041f980790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f980790 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400afd0 tx=0x7f6e440588a0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 -- 172.21.15.60:0/1422297084 shutdown_connections 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f950be0 0x56041f9543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f980cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 --2- 172.21.15.60:0/1422297084 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f980790 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 -- 172.21.15.60:0/1422297084 >> 172.21.15.60:0/1422297084 conn(0x56041f952e50 msgr2=0x56041f99b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 -- 172.21.15.60:0/1422297084 shutdown_connections 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 -- 172.21.15.60:0/1422297084 wait complete. 2024-04-25T08:32:33.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 -- start start 2024-04-25T08:32:33.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.758+0000 7f6e57229740 1 Processor -- start 2024-04-25T08:32:33.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f9753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f99e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 0x56041f97d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041f956000 con 0x56041f97d230 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041f8584e0 con 0x56041f97df80 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56041f9562e0 con 0x56041f97e360 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e57229740 10 client.0 _ll_register_callbacks cb 0x56041f5cdaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e529e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 0x56041f97d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e529e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 0x56041f97d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49456/0 (socket says 172.21.15.60:49456) 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e529e7640 1 -- 172.21.15.60:0/1888162773 learned_addr learned my addr 172.21.15.60:0/1888162773 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.759+0000 7f6e521e6640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f9753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e519e5640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f99e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e529e7640 1 -- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 msgr2=0x56041f99e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e529e7640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f99e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e529e7640 1 -- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 msgr2=0x56041f9753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e529e7640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f9753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e529e7640 1 -- 172.21.15.60:0/1888162773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56041f6faea0 con 0x56041f97d230 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e529e7640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 0x56041f97d6b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c001e30 tx=0x7f6e3c00f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ced7507dfbd678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e317fa640 1 -- 172.21.15.60:0/1888162773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0050d0 con 0x56041f97d230 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e317fa640 1 -- 172.21.15.60:0/1888162773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e3c005270 con 0x56041f97d230 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e44054e50 con 0x56041f97d230 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e317fa640 1 -- 172.21.15.60:0/1888162773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e3c005470 con 0x56041f97d230 2024-04-25T08:32:33.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e57229740 10 client.6288 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56041f58e000 con 0x56041f97d230 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.760+0000 7f6e317fa640 1 -- 172.21.15.60:0/1888162773 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44054e50 con 0x56041f97d230 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e317fa640 1 -- 172.21.15.60:0/1888162773 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e3c023080 con 0x56041f97d230 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e57229740 10 client.6288 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e57229740 10 client.6288 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e57229740 20 client.6288 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:33.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e57229740 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f99cba0 0x56041f99cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e57229740 4 client.6288 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56041f8c7000 con 0x56041f99cba0 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e521e6640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f99cba0 0x56041f99cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e317fa640 10 client.6288 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e521e6640 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f99cba0 0x56041f99cf80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f48d24a4ed2529 server_cookie=3e40320258267f60 in_seq=0 out_seq=0 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.761+0000 7f6e317fa640 1 -- 172.21.15.60:0/1888162773 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56041f8c7000 con 0x56041f99cba0 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e317fa640 10 client.6288 handle_command_reply: tid=0 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 client.6288 shutdown 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 msgr2=0x56041f97d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 0x56041f97d6b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c001e30 tx=0x7f6e3c00f660 comp rx=0 tx=0).stop 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 shutdown_connections 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97d230 0x56041f97d6b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f99cba0 0x56041f99cf80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041f99e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 --2- 172.21.15.60:0/1888162773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f9753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 >> 172.21.15.60:0/1888162773 conn(0x56041f950be0 msgr2=0x56041f97c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e317fa640 0 client.6288 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 shutdown_connections 2024-04-25T08:32:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 1 -- 172.21.15.60:0/1888162773 wait complete. 2024-04-25T08:32:33.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.762+0000 7f6e57229740 20 client.6288 trim_cache size 0 max 16384 2024-04-25T08:32:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.774+0000 7f6e57229740 1 Processor -- start 2024-04-25T08:32:33.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.774+0000 7f6e57229740 1 -- start start 2024-04-25T08:32:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97d230 0x56041f9846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f954c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e57229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 0x56041f97c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e57229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1fd410 con 0x56041f97e360 2024-04-25T08:32:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0ca910 con 0x56041f97df80 2024-04-25T08:32:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0caf10 con 0x56041f97d230 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 0x56041f97c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 0x56041f97c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49458/0 (socket says 172.21.15.60:49458) 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 -- 172.21.15.60:0/2489164105 learned_addr learned my addr 172.21.15.60:0/2489164105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e521e6640 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97d230 0x56041f9846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e519e5640 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f954c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 -- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97d230 msgr2=0x56041f9846f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97d230 0x56041f9846f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 -- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 msgr2=0x56041f954c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f954c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 -- 172.21.15.60:0/2489164105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56041f99fc20 con 0x56041f97e360 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e529e7640 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 0x56041f97c600 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0016f0 tx=0x7f6e3c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e88533ff60c216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e337fe640 1 -- 172.21.15.60:0/2489164105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0052d0 con 0x56041f97e360 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e337fe640 1 -- 172.21.15.60:0/2489164105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c018070 con 0x56041f97e360 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.775+0000 7f6e337fe640 1 -- 172.21.15.60:0/2489164105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c003c70 con 0x56041f97e360 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 -- 172.21.15.60:0/2489164105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 msgr2=0x56041f97c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 0x56041f97c600 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0016f0 tx=0x7f6e3c00d000 comp rx=0 tx=0).stop 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 -- 172.21.15.60:0/2489164105 shutdown_connections 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97e360 0x56041f97c600 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97df80 0x56041f954c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 --2- 172.21.15.60:0/2489164105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97d230 0x56041f9846f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 -- 172.21.15.60:0/2489164105 >> 172.21.15.60:0/2489164105 conn(0x56041f950be0 msgr2=0x56041f99b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 -- 172.21.15.60:0/2489164105 shutdown_connections 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.776+0000 7f6e57229740 1 -- 172.21.15.60:0/2489164105 wait complete. 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.777+0000 7f6e57229740 1 -- start start 2024-04-25T08:32:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.777+0000 7f6e57229740 1 Processor -- start 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.777+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97d230 0x56041f9a3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.777+0000 7f6e57229740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f9519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.777+0000 7f6e57229740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041fa05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041f99e670 con 0x56041f97df80 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041f99f100 con 0x56041f97d230 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56041f9abd60 con 0x56041f97e360 2024-04-25T08:32:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 10 client.0 _ll_register_callbacks cb 0x7f6e44054a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f9519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f9519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49470/0 (socket says 172.21.15.60:49470) 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 -- 172.21.15.60:0/3639718830 learned_addr learned my addr 172.21.15.60:0/3639718830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e521e6640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97d230 0x56041f9a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e529e7640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041fa05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 -- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 msgr2=0x56041fa05d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041fa05d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 -- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97d230 msgr2=0x56041f9a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97d230 0x56041f9a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 -- 172.21.15.60:0/3639718830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56041fa06290 con 0x56041f97df80 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e529e7640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041fa05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e519e5640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f9519d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002a8b0 tx=0x7f6e40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c38217d618481a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e317fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e317fa640 1 -- 172.21.15.60:0/3639718830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40037040 con 0x56041f97df80 2024-04-25T08:32:33.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e317fa640 1 -- 172.21.15.60:0/3639718830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e40003810 con 0x56041f97df80 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e3c018070 con 0x56041f97df80 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e317fa640 1 -- 172.21.15.60:0/3639718830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e40057050 con 0x56041f97df80 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 10 client.6294 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.778+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x56041f9514b0 con 0x56041f97df80 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e317fa640 1 -- 172.21.15.60:0/3639718830 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c018070 con 0x56041f97df80 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e317fa640 1 -- 172.21.15.60:0/3639718830 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e4005a020 con 0x56041f97df80 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 10 client.6294 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 10 client.6294 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 20 client.6294 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f9a25e0 0x56041f972630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 4 client.6294 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2897141188}] 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2897141188}) v1 -- 0x56041f8c1350 con 0x56041f9a25e0 2024-04-25T08:32:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e521e6640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f9a25e0 0x56041f972630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e317fa640 10 client.6294 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e521e6640 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f9a25e0 0x56041f972630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704f020664e1a4d3 server_cookie=f3e75a3ee4121f88 in_seq=0 out_seq=0 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e317fa640 1 -- 172.21.15.60:0/3639718830 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x56041f8c1350 con 0x56041f9a25e0 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e317fa640 10 client.6294 handle_command_reply: tid=0 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 1 client.6294 shutdown 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 msgr2=0x56041f9519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.779+0000 7f6e57229740 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f9519d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6e4002a8b0 tx=0x7f6e40004730 comp rx=0 tx=0).stop 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 shutdown_connections 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x56041f9a25e0 0x56041f972630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56041f97e360 0x56041fa05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56041f97df80 0x56041f9519d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 --2- 172.21.15.60:0/3639718830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56041f97d230 0x56041f9a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e317fa640 0 client.6294 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 >> 172.21.15.60:0/3639718830 conn(0x56041f950be0 msgr2=0x56041f99e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 shutdown_connections 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 1 -- 172.21.15.60:0/3639718830 wait complete. 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e57229740 20 client.6294 trim_cache size 0 max 16384 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e3806b4d0 msgr2=0x7f6e3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e3806b4d0 0x7f6e3806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6e40034000 tx=0x7f6e4002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 msgr2=0x7f6e4c14f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 0x7f6e4c14f320 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400cbc0 tx=0x7f6e4400a430 comp rx=0 tx=0).stop 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 shutdown_connections 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c15bb50 0x7f6e4c0b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e3806b4d0 0x7f6e3806d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c0cf600 0x7f6e4c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 --2- 172.21.15.60:0/115887030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c0cbf20 0x7f6e4c14f320 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 >> 172.21.15.60:0/115887030 conn(0x7f6e4c06f500 msgr2=0x7f6e4c0b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 shutdown_connections 2024-04-25T08:32:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.780+0000 7f6e5444a640 1 -- 172.21.15.60:0/115887030 wait complete. 2024-04-25T08:32:33.787 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:32:33.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:33.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.891+0000 7f24fc69e640 1 Processor -- start 2024-04-25T08:32:33.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.891+0000 7f24fc69e640 1 -- start start 2024-04-25T08:32:33.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.892+0000 7f24fc69e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f415b9c0 0x7f24f415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.892+0000 7f24fc69e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24f4072d30 0x7f24f406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.892+0000 7f24fc69e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f40cfa60 con 0x7f24f406abe0 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.892+0000 7f24fc69e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f40731a0 con 0x7f24f4072d30 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.892+0000 7f24fc69e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f406b580 con 0x7f24f415b9c0 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24f9c39640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f24f406abe0 0x7f24f4072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55824/0 (socket says 172.21.15.60:55824) 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24f9c39640 1 -- 172.21.15.60:0/2823132449 learned_addr learned my addr 172.21.15.60:0/2823132449 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24fa43a640 1 --2- 172.21.15.60:0/2823132449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f415b9c0 0x7f24f415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24fac3b640 1 --2- 172.21.15.60:0/2823132449 >> v2:172.21.15.100:3300/0 conn(0x7f24f4072d30 0x7f24f406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24f9438640 1 -- 172.21.15.60:0/2823132449 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072955921 0 0) 0x7f24f40cfa60 con 0x7f24f406abe0 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24f9438640 1 -- 172.21.15.60:0/2823132449 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24dc003520 con 0x7f24f406abe0 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24fa43a640 1 -- 172.21.15.60:0/2823132449 >> v2:172.21.15.100:3300/0 conn(0x7f24f4072d30 msgr2=0x7f24f406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24fa43a640 1 --2- 172.21.15.60:0/2823132449 >> v2:172.21.15.100:3300/0 conn(0x7f24f4072d30 0x7f24f406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24fa43a640 1 -- 172.21.15.60:0/2823132449 >> v1:172.21.15.60:6789/0 conn(0x7f24f406abe0 legacy=0x7f24f4072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.893+0000 7f24fa43a640 1 -- 172.21.15.60:0/2823132449 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40cfd40 con 0x7f24f415b9c0 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.894+0000 7f24fa43a640 1 --2- 172.21.15.60:0/2823132449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f415b9c0 0x7f24f415bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d90 tx=0x7f24ec02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0a7ba6ce4ea113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.894+0000 7f24f9438640 1 -- 172.21.15.60:0/2823132449 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec007740 con 0x7f24f415b9c0 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.894+0000 7f24f9438640 1 -- 172.21.15.60:0/2823132449 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec0078e0 con 0x7f24f415b9c0 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.894+0000 7f24f9438640 1 -- 172.21.15.60:0/2823132449 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec007cb0 con 0x7f24f415b9c0 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 -- 172.21.15.60:0/2823132449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f415b9c0 msgr2=0x7f24f415bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2823132449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f415b9c0 0x7f24f415bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d90 tx=0x7f24ec02a660 comp rx=0 tx=0).stop 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 -- 172.21.15.60:0/2823132449 shutdown_connections 2024-04-25T08:32:33.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2823132449 >> v2:172.21.15.100:3300/0 conn(0x7f24f4072d30 0x7f24f406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2823132449 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f415b9c0 0x7f24f415bda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 -- 172.21.15.60:0/2823132449 >> 172.21.15.60:0/2823132449 conn(0x7f24f40b2370 msgr2=0x7f24f40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 -- 172.21.15.60:0/2823132449 shutdown_connections 2024-04-25T08:32:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.895+0000 7f24fc69e640 1 -- 172.21.15.60:0/2823132449 wait complete. 2024-04-25T08:32:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.896+0000 7f24fc69e640 1 Processor -- start 2024-04-25T08:32:33.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.896+0000 7f24fc69e640 1 -- start start 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 0x7f24f413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f4072d30 0x7f24f413d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24f415b9c0 0x7f24f413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40cf710 con 0x7f24f406abe0 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40d0300 con 0x7f24f415b9c0 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f415c2e0 con 0x7f24f4072d30 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 0x7f24f413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 0x7f24f413f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49482/0 (socket says 172.21.15.60:49482) 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 -- 172.21.15.60:0/2698431352 learned_addr learned my addr 172.21.15.60:0/2698431352 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:33.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24f9c39640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f4072d30 0x7f24f413d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fac3b640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24f415b9c0 0x7f24f413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 -- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f4072d30 msgr2=0x7f24f413d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f4072d30 0x7f24f413d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 -- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24f415b9c0 msgr2=0x7f24f413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24f415b9c0 0x7f24f413dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 -- 172.21.15.60:0/2698431352 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f413e8a0 con 0x7f24f406abe0 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fac3b640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24f415b9c0 0x7f24f413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24f9c39640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f4072d30 0x7f24f413d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fa43a640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 0x7f24f413f940 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d90 tx=0x7f24ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec03f070 con 0x7f24f406abe0 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec031c40 con 0x7f24f406abe0 2024-04-25T08:32:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.897+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f413eb80 con 0x7f24f406abe0 2024-04-25T08:32:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.898+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009450 con 0x7f24f406abe0 2024-04-25T08:32:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.898+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f413f140 con 0x7f24f406abe0 2024-04-25T08:32:33.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.899+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f4073990 con 0x7f24f406abe0 2024-04-25T08:32:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.899+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24ec044460 con 0x7f24f406abe0 2024-04-25T08:32:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.900+0000 7f24f2ffd640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24e006b4d0 0x7f24e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.900+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24ec0abc70 con 0x7f24f406abe0 2024-04-25T08:32:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.900+0000 7f24f9c39640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24e006b4d0 0x7f24e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.900+0000 7f24f9c39640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24e006b4d0 0x7f24e006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f24e8007fd0 tx=0x7f24e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:33.904+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24ec07a960 con 0x7f24f406abe0 2024-04-25T08:32:34.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.121+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24f406b1b0 con 0x7f24f406abe0 2024-04-25T08:32:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.122+0000 7f24f2ffd640 1 -- 172.21.15.60:0/2698431352 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f24ec07a260 con 0x7f24f406abe0 2024-04-25T08:32:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:34.118 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:31.022346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:34.118 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-04-25T08:32:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24e006b4d0 msgr2=0x7f24e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24e006b4d0 0x7f24e006d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f24e8007fd0 tx=0x7f24e8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 msgr2=0x7f24f413f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 0x7f24f413f940 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009d90 tx=0x7f24ec004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 shutdown_connections 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24e006b4d0 0x7f24e006d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24f415b9c0 0x7f24f413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24f4072d30 0x7f24f413d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 --2- 172.21.15.60:0/2698431352 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24f406abe0 0x7f24f413f940 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 >> 172.21.15.60:0/2698431352 conn(0x7f24f40b2370 msgr2=0x7f24f415dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 shutdown_connections 2024-04-25T08:32:34.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.124+0000 7f24fc69e640 1 -- 172.21.15.60:0/2698431352 wait complete. 2024-04-25T08:32:34.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T08:32:34.239 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:34.239 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:34.239 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:34.248 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:34.248 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:34.248 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T08:32:34.249 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T08:32:34.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:34.249 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T08:32:34.265 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:32:34.265 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:32:34.265 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T08:32:34.265 DEBUG:teuthology.orchestra.run.smithi060:> import json 2024-04-25T08:32:34.265 DEBUG:teuthology.orchestra.run.smithi060:> import sys 2024-04-25T08:32:34.265 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> try: 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> except OSError as e: 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> sys.exit(e.errno) 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> print(json.dumps( 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> indent=2)) 2024-04-25T08:32:34.266 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:32:34.374 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_mode": 16877, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_ino": 1099511627777, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_dev": 42, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_nlink": 16, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_uid": 1000, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_gid": 1316, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_size": 18, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_atime": 1714033940.9485567, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_mtime": 1714033946.8045027, 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout: "st_ctime": 1714033946.8045027 2024-04-25T08:32:34.375 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:34.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:34.383 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T08:32:34.383 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T08:32:34.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.527+0000 7fc5ddc3a640 1 Processor -- start 2024-04-25T08:32:34.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5ddc3a640 1 -- start start 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5ddc3a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5d81602e0 0x7fc5d80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5ddc3a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80cd710 con 0x7fc5d815ba00 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5ddc3a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d8099b70 con 0x7fc5d815f5f0 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5ddc3a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d8099e50 con 0x7fc5d81602e0 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5d77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc5d815ba00 0x7fc5d809a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55830/0 (socket says 172.21.15.60:55830) 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.528+0000 7fc5d77fe640 1 -- 172.21.15.60:0/149390404 learned_addr learned my addr 172.21.15.60:0/149390404 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d6ffd640 1 --2- 172.21.15.60:0/149390404 >> v2:172.21.15.100:3300/0 conn(0x7fc5d81602e0 0x7fc5d80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788638743 0 0) 0x7fc5c8001110 con 0x7fc5d815ba00 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c0003520 con 0x7fc5d815ba00 2024-04-25T08:32:34.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3664497824 0 0) 0x7fc5c0003520 con 0x7fc5d815ba00 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 >> v2:172.21.15.100:3300/0 conn(0x7fc5d81602e0 msgr2=0x7fc5d80ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 --2- 172.21.15.60:0/149390404 >> v2:172.21.15.100:3300/0 conn(0x7fc5d81602e0 0x7fc5d80ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 >> v1:172.21.15.100:6789/0 conn(0x7fc5d815f5f0 legacy=0x7fc5d80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d809a130 con 0x7fc5d815ba00 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fc5c8001840 con 0x7fc5d815ba00 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d6ffd640 1 --2- 172.21.15.60:0/149390404 >> v2:172.21.15.100:3300/0 conn(0x7fc5d81602e0 0x7fc5d80ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 >> v1:172.21.15.60:6789/0 conn(0x7fc5d815ba00 legacy=0x7fc5d809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5c00039b0 0x7fc5c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c0004800 0x7fc5c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5c00051c0 0x7fc5c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8099e50 con 0x7fc5c00039b0 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000c110 con 0x7fc5c0004800 2024-04-25T08:32:34.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.529+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c000c310 con 0x7fc5c00051c0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5c00039b0 0x7fc5c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d6ffd640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c0004800 0x7fc5c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d7fff640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5c00051c0 0x7fc5c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d77fe640 1 -- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5c00051c0 msgr2=0x7fc5c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5c00051c0 0x7fc5c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d77fe640 1 -- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c0004800 msgr2=0x7fc5c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c0004800 0x7fc5c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d7fff640 1 -- 172.21.15.60:0/149390404 reap_dead start 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.530+0000 7fc5d77fe640 1 -- 172.21.15.60:0/149390404 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c0003520 con 0x7fc5c00039b0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5c00039b0 0x7fc5c0003d90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c4c0 tx=0x7fc5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9249689fd9c47fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800c0e0 con 0x7fc5c00039b0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8034070 con 0x7fc5c00039b0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5d67fc640 1 -- 172.21.15.60:0/149390404 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005d10 con 0x7fc5c00039b0 2024-04-25T08:32:34.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/149390404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5c00039b0 msgr2=0x7fc5c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5c00039b0 0x7fc5c0003d90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c4c0 tx=0x7fc5c0005830 comp rx=0 tx=0).stop 2024-04-25T08:32:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/149390404 shutdown_connections 2024-04-25T08:32:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/149390404 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5c00039b0 0x7fc5c0003d90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/149390404 >> 172.21.15.60:0/149390404 conn(0x7fc5d80b3270 msgr2=0x7fc5d80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/149390404 shutdown_connections 2024-04-25T08:32:34.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.531+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/149390404 wait complete. 2024-04-25T08:32:34.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.533+0000 7fc5ddc3a640 1 Processor -- start 2024-04-25T08:32:34.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.533+0000 7fc5ddc3a640 1 -- start start 2024-04-25T08:32:34.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c00039b0 0x7fc5d81555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5d80c75e0 0x7fc5d80c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 0x7fc5d80c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815fbc0 con 0x7fc5d80c7a60 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815fa40 con 0x7fc5c00039b0 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815fd40 con 0x7fc5d80c75e0 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 0x7fc5d80c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 0x7fc5d80c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49508/0 (socket says 172.21.15.60:49508) 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 -- 172.21.15.60:0/1311526308 learned_addr learned my addr 172.21.15.60:0/1311526308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d6ffd640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5d80c75e0 0x7fc5d80c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c00039b0 0x7fc5d81555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 -- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5d80c75e0 msgr2=0x7fc5d80c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5d80c75e0 0x7fc5d80c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 -- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c00039b0 msgr2=0x7fc5d81555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c00039b0 0x7fc5d81555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 -- 172.21.15.60:0/1311526308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8155b20 con 0x7fc5d80c7a60 2024-04-25T08:32:34.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c00039b0 0x7fc5d81555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5d7fff640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 0x7fc5d80c5d40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0001c50 tx=0x7fc5d0002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0003d80 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d00050d0 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00054c0 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d80b62d0 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.534+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d80b6890 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.535+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d8073990 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.536+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5d0015070 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.536+0000 7fc5bffff640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc5c406b500 0x7fc5c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.536+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5d008a040 con 0x7fc5d80c7a60 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.537+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc5c406b500 0x7fc5c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.537+0000 7fc5d77fe640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc5c406b500 0x7fc5c406d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8033000 tx=0x7fc5c8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.541+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5d001d060 con 0x7fc5d80c7a60 2024-04-25T08:32:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.755+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5d80c7f00 con 0x7fc5d80c7a60 2024-04-25T08:32:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.756+0000 7fc5bffff640 1 -- 172.21.15.60:0/1311526308 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc5d0058d30 con 0x7fc5d80c7a60 2024-04-25T08:32:34.752 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:34.752 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:31.022346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:34.752 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 23 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc5c406b500 msgr2=0x7fc5c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc5c406b500 0x7fc5c406d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8033000 tx=0x7fc5c8009030 comp rx=0 tx=0).stop 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 msgr2=0x7fc5d80c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 0x7fc5d80c5d40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0001c50 tx=0x7fc5d0002110 comp rx=0 tx=0).stop 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 shutdown_connections 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc5d80c7a60 0x7fc5d80c5d40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5d80c75e0 0x7fc5d80c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc5c406b500 0x7fc5c406d9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 --2- 172.21.15.60:0/1311526308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5c00039b0 0x7fc5d81555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.757+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 >> 172.21.15.60:0/1311526308 conn(0x7fc5d80b3270 msgr2=0x7fc5d80ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.758+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 shutdown_connections 2024-04-25T08:32:34.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.758+0000 7fc5ddc3a640 1 -- 172.21.15.60:0/1311526308 wait complete. 2024-04-25T08:32:34.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T08:32:34.870 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:34.870 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": -2 2024-04-25T08:32:34.871 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:34.879 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T08:32:34.879 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T08:32:34.879 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:34.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T08:32:34.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.987+0000 7fb85bd30640 1 Processor -- start 2024-04-25T08:32:34.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.987+0000 7fb85bd30640 1 -- start start 2024-04-25T08:32:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85bd30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8540b5c40 0x7fb8540ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85bd30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb85415b980 0x7fb8540ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85bd30640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8540cf320 con 0x7fb85415b5a0 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85bd30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8540ced90 con 0x7fb85415b980 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85bd30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8540cf040 con 0x7fb8540b5c40 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb85415b980 0x7fb8540ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb85415b980 0x7fb8540ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49522/0 (socket says 172.21.15.60:49522) 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 -- 172.21.15.60:0/652661534 learned_addr learned my addr 172.21.15.60:0/652661534 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb8592cb640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8540b5c40 0x7fb8540ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 -- 172.21.15.60:0/652661534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8540b5c40 msgr2=0x7fb8540ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8540b5c40 0x7fb8540ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 -- 172.21.15.60:0/652661534 >> v1:172.21.15.100:6789/0 conn(0x7fb85415b5a0 legacy=0x7fb8540b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb85a2cd640 1 -- 172.21.15.60:0/652661534 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854161970 con 0x7fb85415b980 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.988+0000 7fb8592cb640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8540b5c40 0x7fb8540ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.989+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb85415b980 0x7fb8540ce850 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009e50 tx=0x7fb84c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aabd6af0488eec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.989+0000 7fb858aca640 1 -- 172.21.15.60:0/652661534 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c036070 con 0x7fb85415b980 2024-04-25T08:32:34.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.989+0000 7fb858aca640 1 -- 172.21.15.60:0/652661534 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c031600 con 0x7fb85415b980 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.989+0000 7fb858aca640 1 -- 172.21.15.60:0/652661534 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0319d0 con 0x7fb85415b980 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 -- 172.21.15.60:0/652661534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb85415b980 msgr2=0x7fb8540ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb85415b980 0x7fb8540ce850 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009e50 tx=0x7fb84c02a710 comp rx=0 tx=0).stop 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 -- 172.21.15.60:0/652661534 shutdown_connections 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb85415b980 0x7fb8540ce850 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 --2- 172.21.15.60:0/652661534 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8540b5c40 0x7fb8540ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 -- 172.21.15.60:0/652661534 >> 172.21.15.60:0/652661534 conn(0x7fb8540d0530 msgr2=0x7fb8540d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 -- 172.21.15.60:0/652661534 shutdown_connections 2024-04-25T08:32:34.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.990+0000 7fb85bd30640 1 -- 172.21.15.60:0/652661534 wait complete. 2024-04-25T08:32:34.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.991+0000 7fb85bd30640 1 Processor -- start 2024-04-25T08:32:34.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.991+0000 7fb85bd30640 1 -- start start 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85bd30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 0x7fb85414fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85bd30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb85415b5a0 0x7fb8540b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85bd30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb85415b980 0x7fb8540b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85bd30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb854162450 con 0x7fb8540b5c40 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85bd30640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb854161f30 con 0x7fb85415b980 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85bd30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8541625d0 con 0x7fb85415b5a0 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 0x7fb85414fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 0x7fb85414fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49524/0 (socket says 172.21.15.60:49524) 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 -- 172.21.15.60:0/271273166 learned_addr learned my addr 172.21.15.60:0/271273166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb85415b980 0x7fb8540b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb8592cb640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb85415b5a0 0x7fb8540b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 -- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb85415b5a0 msgr2=0x7fb8540b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb85415b5a0 0x7fb8540b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 -- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb85415b980 msgr2=0x7fb8540b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb85415b980 0x7fb8540b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 -- 172.21.15.60:0/271273166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8540cadd0 con 0x7fb8540b5c40 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb8592cb640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb85415b5a0 0x7fb8540b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.992+0000 7fb859acc640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 0x7fb85414fa30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c400 tx=0x7fb85000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.993+0000 7fb84a7fc640 1 -- 172.21.15.60:0/271273166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500052d0 con 0x7fb8540b5c40 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.993+0000 7fb84a7fc640 1 -- 172.21.15.60:0/271273166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850005490 con 0x7fb8540b5c40 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.993+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8540cb0b0 con 0x7fb8540b5c40 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.993+0000 7fb84a7fc640 1 -- 172.21.15.60:0/271273166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850007260 con 0x7fb8540b5c40 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.993+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8540cb670 con 0x7fb8540b5c40 2024-04-25T08:32:34.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.994+0000 7fb84a7fc640 1 -- 172.21.15.60:0/271273166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb850007400 con 0x7fb8540b5c40 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.995+0000 7fb84a7fc640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb83806b4d0 0x7fb83806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.995+0000 7fb84a7fc640 1 -- 172.21.15.60:0/271273166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb850089860 con 0x7fb8540b5c40 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.995+0000 7fb8592cb640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb83806b4d0 0x7fb83806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.996+0000 7fb8592cb640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb83806b4d0 0x7fb83806d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb8541f83e0 tx=0x7fb844008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.996+0000 7fb85ea34740 1 Processor -- start 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.996+0000 7fb85ea34740 1 -- start start 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 0x55eea3217a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea32354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea31ed390 0x55eea3233610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85ea34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea314fc60 con 0x55eea32176b0 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea30f8ee0 con 0x55eea3234f50 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eea314bd00 con 0x55eea31ed390 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 0x55eea3217a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 0x55eea3217a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49526/0 (socket says 172.21.15.60:49526) 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 -- 172.21.15.60:0/1515073088 learned_addr learned my addr 172.21.15.60:0/1515073088 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb8592cb640 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea32354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea31ed390 0x55eea3233610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 -- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea31ed390 msgr2=0x55eea3233610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea31ed390 0x55eea3233610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 -- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 msgr2=0x55eea32354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea32354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 -- 172.21.15.60:0/1515073088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea2f309f0 con 0x55eea32176b0 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb859acc640 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 0x55eea3217a90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb850087c00 tx=0x7fb85000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688cb54b39737c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb8377fe640 1 -- 172.21.15.60:0/1515073088 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850056200 con 0x55eea32176b0 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb8377fe640 1 -- 172.21.15.60:0/1515073088 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85005a040 con 0x55eea32176b0 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.997+0000 7fb8377fe640 1 -- 172.21.15.60:0/1515073088 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85005e070 con 0x55eea32176b0 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 -- 172.21.15.60:0/1515073088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 msgr2=0x55eea3217a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 0x55eea3217a90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb850087c00 tx=0x7fb85000a000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 -- 172.21.15.60:0/1515073088 shutdown_connections 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea31ed390 0x55eea3233610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea32354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 --2- 172.21.15.60:0/1515073088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea32176b0 0x55eea3217a90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 -- 172.21.15.60:0/1515073088 >> 172.21.15.60:0/1515073088 conn(0x55eea31e9cc0 msgr2=0x55eea32324c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 -- 172.21.15.60:0/1515073088 shutdown_connections 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 -- 172.21.15.60:0/1515073088 wait complete. 2024-04-25T08:32:35.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.998+0000 7fb85ea34740 1 -- start start 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 Processor -- start 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea32052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea3233880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3202230 0x55eea3233dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea31ee130 con 0x55eea3234f50 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea310c470 con 0x55eea3202230 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:34.999+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eea31e7bf0 con 0x55eea32176b0 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb85ea34740 10 client.0 _ll_register_callbacks cb 0x55eea30d87d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea3233880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea3233880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49540/0 (socket says 172.21.15.60:49540) 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 -- 172.21.15.60:0/2686954540 learned_addr learned my addr 172.21.15.60:0/2686954540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb859acc640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea32052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3202230 0x55eea3233dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 -- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 msgr2=0x55eea32052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea32052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 -- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3202230 msgr2=0x55eea3233dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3202230 0x55eea3233dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 -- 172.21.15.60:0/2686954540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eea2e242e0 con 0x55eea3234f50 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3202230 0x55eea3233dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb8592cb640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea3233880 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb844015720 tx=0x7fb844004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b489df8e3d435112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb834ff9640 1 -- 172.21.15.60:0/2686954540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844026020 con 0x55eea3234f50 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb834ff9640 1 -- 172.21.15.60:0/2686954540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb844016a70 con 0x55eea3234f50 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb85005a040 con 0x55eea3234f50 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb834ff9640 1 -- 172.21.15.60:0/2686954540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb844030050 con 0x55eea3234f50 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb85ea34740 10 client.6318 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:35.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.000+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55eea2f91180 con 0x55eea3234f50 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb834ff9640 1 -- 172.21.15.60:0/2686954540 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85005a040 con 0x55eea3234f50 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb834ff9640 1 -- 172.21.15.60:0/2686954540 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb844033020 con 0x55eea3234f50 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb85ea34740 10 client.6318 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb85ea34740 10 client.6318 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb85ea34740 20 client.6318 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea3218840 0x55eea3218c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb85ea34740 4 client.6318 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eea315d1a0 con 0x55eea3218840 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb859acc640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea3218840 0x55eea3218c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb834ff9640 10 client.6318 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.001+0000 7fb859acc640 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea3218840 0x55eea3218c50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d107047a0355202a server_cookie=9f048bdfcae67d68 in_seq=0 out_seq=0 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb834ff9640 1 -- 172.21.15.60:0/2686954540 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eea315d1a0 con 0x55eea3218840 2024-04-25T08:32:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb834ff9640 10 client.6318 handle_command_reply: tid=0 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 client.6318 shutdown 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 msgr2=0x55eea3233880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea3233880 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb844015720 tx=0x7fb844004730 comp rx=0 tx=0).stop 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 shutdown_connections 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3202230 0x55eea3233dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea3233880 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea3218840 0x55eea3218c50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2686954540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea32052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb834ff9640 0 client.6318 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 >> 172.21.15.60:0/2686954540 conn(0x55eea31ed390 msgr2=0x55eea3235a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 shutdown_connections 2024-04-25T08:32:35.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 1 -- 172.21.15.60:0/2686954540 wait complete. 2024-04-25T08:32:35.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.002+0000 7fb85ea34740 20 client.6318 trim_cache size 0 max 16384 2024-04-25T08:32:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.017+0000 7fb85ea34740 1 Processor -- start 2024-04-25T08:32:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.017+0000 7fb85ea34740 1 -- start start 2024-04-25T08:32:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea3202230 0x55eea32339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea32176b0 0x55eea3235a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea322bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85ea34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb854200600 con 0x55eea3234f50 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8541ff7b0 con 0x55eea32176b0 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8540ccd70 con 0x55eea3202230 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea322bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea322bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49546/0 (socket says 172.21.15.60:49546) 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 -- 172.21.15.60:0/2977433182 learned_addr learned my addr 172.21.15.60:0/2977433182 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb8592cb640 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea32176b0 0x55eea3235a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb859acc640 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea3202230 0x55eea32339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 -- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea3202230 msgr2=0x55eea32339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea3202230 0x55eea32339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 -- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea32176b0 msgr2=0x55eea3235a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea32176b0 0x55eea3235a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 -- 172.21.15.60:0/2977433182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eea2dfd3a0 con 0x55eea3234f50 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.018+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea322bdb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb84c034040 tx=0x7fb84c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a20015bcdf38994 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb836ffd640 1 -- 172.21.15.60:0/2977433182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c04a070 con 0x55eea3234f50 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb836ffd640 1 -- 172.21.15.60:0/2977433182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c03c4a0 con 0x55eea3234f50 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb836ffd640 1 -- 172.21.15.60:0/2977433182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c036040 con 0x55eea3234f50 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 -- 172.21.15.60:0/2977433182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 msgr2=0x55eea322bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea322bdb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb84c034040 tx=0x7fb84c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 -- 172.21.15.60:0/2977433182 shutdown_connections 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3234f50 0x55eea322bdb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea32176b0 0x55eea3235a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 --2- 172.21.15.60:0/2977433182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea3202230 0x55eea32339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 -- 172.21.15.60:0/2977433182 >> 172.21.15.60:0/2977433182 conn(0x55eea31ed390 msgr2=0x55eea324c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 -- 172.21.15.60:0/2977433182 shutdown_connections 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.019+0000 7fb85ea34740 1 -- 172.21.15.60:0/2977433182 wait complete. 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.020+0000 7fb85ea34740 1 -- start start 2024-04-25T08:32:35.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.020+0000 7fb85ea34740 1 Processor -- start 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 0x55eea3280460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea321aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea3235a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eea31ed770 con 0x55eea3202230 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8540cd370 con 0x55eea3234f50 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eea31ee020 con 0x55eea32176b0 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85ea34740 10 client.0 _ll_register_callbacks cb 0x55eea31e80d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 0x55eea3280460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 0x55eea3280460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49548/0 (socket says 172.21.15.60:49548) 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 -- 172.21.15.60:0/999037691 learned_addr learned my addr 172.21.15.60:0/999037691 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea3235a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb8592cb640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea321aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 -- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 msgr2=0x55eea321aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea321aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 -- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 msgr2=0x55eea3235a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea3235a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 -- 172.21.15.60:0/999037691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eea321b400 con 0x55eea3202230 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb85a2cd640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea3235a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb859acc640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 0x55eea3280460 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb850057e90 tx=0x7fb85005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40e63c7693bbaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.021+0000 7fb834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb834ff9640 1 -- 172.21.15.60:0/999037691 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850073070 con 0x55eea3202230 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb834ff9640 1 -- 172.21.15.60:0/999037691 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb85005e070 con 0x55eea3202230 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb84c03c4a0 con 0x55eea3202230 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb834ff9640 1 -- 172.21.15.60:0/999037691 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb85005e290 con 0x55eea3202230 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 10 client.6324 fetch_fsmap learned FSMap version 23 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x55eea3216080 con 0x55eea3202230 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb834ff9640 1 -- 172.21.15.60:0/999037691 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c03c4a0 con 0x55eea3202230 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb834ff9640 1 -- 172.21.15.60:0/999037691 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb85006fbe0 con 0x55eea3202230 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 10 client.6324 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 10 client.6324 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 20 client.6324 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea321a4e0 0x55eea3235f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 4 client.6324 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55eea31574f0 con 0x55eea321a4e0 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb8592cb640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea321a4e0 0x55eea3235f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb834ff9640 10 client.6324 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.022+0000 7fb8592cb640 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea321a4e0 0x55eea3235f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7538e9c302141bda server_cookie=992580fa75dae73 in_seq=0 out_seq=0 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb834ff9640 1 -- 172.21.15.60:0/999037691 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55eea31574f0 con 0x55eea321a4e0 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb834ff9640 10 client.6324 handle_command_reply: tid=0 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 client.6324 shutdown 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 msgr2=0x55eea3280460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 0x55eea3280460 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb850057e90 tx=0x7fb85005b000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 shutdown_connections 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eea3234f50 0x55eea3235a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eea321a4e0 0x55eea3235f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eea32176b0 0x55eea321aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb834ff9640 0 client.6324 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 --2- 172.21.15.60:0/999037691 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eea3202230 0x55eea3280460 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 >> 172.21.15.60:0/999037691 conn(0x55eea31ed390 msgr2=0x55eea322bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 shutdown_connections 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 1 -- 172.21.15.60:0/999037691 wait complete. 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.023+0000 7fb85ea34740 20 client.6324 trim_cache size 0 max 16384 2024-04-25T08:32:35.027 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:35.027 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": -2 2024-04-25T08:32:35.027 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb83806b4d0 msgr2=0x7fb83806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb83806b4d0 0x7fb83806d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb8541f83e0 tx=0x7fb844008040 comp rx=0 tx=0).stop 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 msgr2=0x7fb85414fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 0x7fb85414fa30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c400 tx=0x7fb85000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 shutdown_connections 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb83806b4d0 0x7fb83806d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb85415b980 0x7fb8540b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb85415b5a0 0x7fb8540b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 --2- 172.21.15.60:0/271273166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8540b5c40 0x7fb85414fa30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 >> 172.21.15.60:0/271273166 conn(0x7fb8540d0530 msgr2=0x7fb8540b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 shutdown_connections 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.024+0000 7fb85bd30640 1 -- 172.21.15.60:0/271273166 wait complete. 2024-04-25T08:32:35.030 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T08:32:35.031 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:32:35.031 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:35.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:32:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.139+0000 7f6aee49e640 1 Processor -- start 2024-04-25T08:32:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.139+0000 7f6aee49e640 1 -- start start 2024-04-25T08:32:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aee49e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae815dac0 0x7f6ae80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aee49e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ae80d1f30 0x7f6ae80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aee49e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae80d2420 con 0x7f6ae80d34c0 2024-04-25T08:32:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aee49e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae80d3030 con 0x7f6ae80d1f30 2024-04-25T08:32:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aee49e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80d2700 con 0x7f6ae815dac0 2024-04-25T08:32:35.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aeca3b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6ae80d34c0 0x7f6ae80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38056/0 (socket says 172.21.15.60:38056) 2024-04-25T08:32:35.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6aeca3b640 1 -- 172.21.15.60:0/315702185 learned_addr learned my addr 172.21.15.60:0/315702185 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:35.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6ae7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae815dac0 0x7f6ae80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.140+0000 7f6ae77fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ae80d1f30 0x7f6ae80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae77fe640 1 -- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae815dac0 msgr2=0x7f6ae80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae815dac0 0x7f6ae80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae77fe640 1 -- 172.21.15.60:0/315702185 >> v1:172.21.15.100:6789/0 conn(0x7f6ae80d34c0 legacy=0x7f6ae80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae77fe640 1 -- 172.21.15.60:0/315702185 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80d29e0 con 0x7f6ae80d1f30 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae6ffd640 1 -- 172.21.15.60:0/315702185 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165460798 0 0) 0x7f6ae80d2420 con 0x7f6ae80d34c0 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae815dac0 0x7f6ae80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.141+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae80d03a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009790 tx=0x7f6ad802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34372656d0905302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6ae6ffd640 1 -- 172.21.15.60:0/315702185 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8037070 con 0x7f6ae80d1f30 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6ae6ffd640 1 -- 172.21.15.60:0/315702185 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8032440 con 0x7f6ae80d1f30 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6ae6ffd640 1 -- 172.21.15.60:0/315702185 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8032810 con 0x7f6ae80d1f30 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 -- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 msgr2=0x7f6ae80d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 --2- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae80d03a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009790 tx=0x7f6ad802a210 comp rx=0 tx=0).stop 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 -- 172.21.15.60:0/315702185 shutdown_connections 2024-04-25T08:32:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 --2- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae80d03a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 --2- 172.21.15.60:0/315702185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae815dac0 0x7f6ae80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 -- 172.21.15.60:0/315702185 >> 172.21.15.60:0/315702185 conn(0x7f6ae8061c30 msgr2=0x7f6ae806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 -- 172.21.15.60:0/315702185 shutdown_connections 2024-04-25T08:32:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.142+0000 7f6aee49e640 1 -- 172.21.15.60:0/315702185 wait complete. 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.143+0000 7f6aee49e640 1 Processor -- start 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.143+0000 7f6aee49e640 1 -- start start 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aee49e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aee49e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae80d34c0 0x7f6ae80c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aee49e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 0x7f6ae80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aee49e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80b97d0 con 0x7f6ae815dac0 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aee49e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae815dea0 con 0x7f6ae80d1f30 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aee49e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80b9950 con 0x7f6ae80d34c0 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aeca3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 0x7f6ae80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aeca3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 0x7f6ae80cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49560/0 (socket says 172.21.15.60:49560) 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3609853555 learned_addr learned my addr 172.21.15.60:0/3609853555 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae80d34c0 0x7f6ae80c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.144+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae80d34c0 msgr2=0x7f6ae80c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae80d34c0 0x7f6ae80c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 msgr2=0x7f6ae8160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae8160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3609853555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8153090 con 0x7f6ae815dac0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 0x7f6ae80cd420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c7b0 tx=0x7f6ae000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6ae4ff9640 1 -- 172.21.15.60:0/3609853555 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0018070 con 0x7f6ae815dac0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6ae4ff9640 1 -- 172.21.15.60:0/3609853555 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000a5e0 con 0x7f6ae815dac0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae81f3490 con 0x7f6ae815dac0 2024-04-25T08:32:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6ae4ff9640 1 -- 172.21.15.60:0/3609853555 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000aa10 con 0x7f6ae815dac0 2024-04-25T08:32:35.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.145+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae80c8630 con 0x7f6ae815dac0 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.147+0000 7f6ae4ff9640 1 -- 172.21.15.60:0/3609853555 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ae000abb0 con 0x7f6ae815dac0 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.147+0000 7f6ae4ff9640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ad406b4d0 0x7f6ad406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.147+0000 7f6ae4ff9640 1 -- 172.21.15.60:0/3609853555 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ae0089940 con 0x7f6ae815dac0 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.148+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ad406b4d0 0x7f6ad406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.148+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ad406b4d0 0x7f6ad406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009560 tx=0x7f6adc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.148+0000 7f6af11a7740 1 Processor -- start 2024-04-25T08:32:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.148+0000 7f6af11a7740 1 -- start start 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa469d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46d75e0 0x55ffa469b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6af11a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa45ccfe0 con 0x55ffa46c59d0 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa45fed20 con 0x55ffa46d75e0 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffa45a7fa0 con 0x55ffa469af10 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49576/0 (socket says 172.21.15.60:49576) 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 -- 172.21.15.60:0/1502903209 learned_addr learned my addr 172.21.15.60:0/1502903209 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa469d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46d75e0 0x55ffa469b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 -- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 msgr2=0x55ffa469d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa469d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 -- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46d75e0 msgr2=0x55ffa469b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46d75e0 0x55ffa469b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.149+0000 7f6aeca3b640 1 -- 172.21.15.60:0/1502903209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffa43dfb00 con 0x55ffa46c59d0 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa469d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c5e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0087c50 tx=0x7f6ae00578c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c334cfa276cfe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6acdffb640 1 -- 172.21.15.60:0/1502903209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00605c0 con 0x55ffa46c59d0 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6acdffb640 1 -- 172.21.15.60:0/1502903209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0056000 con 0x55ffa46c59d0 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6acdffb640 1 -- 172.21.15.60:0/1502903209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00545d0 con 0x55ffa46c59d0 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 -- 172.21.15.60:0/1502903209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 msgr2=0x55ffa46c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c5e50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0087c50 tx=0x7f6ae00578c0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 -- 172.21.15.60:0/1502903209 shutdown_connections 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c5e50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46d75e0 0x55ffa469b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1502903209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa469d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 -- 172.21.15.60:0/1502903209 >> 172.21.15.60:0/1502903209 conn(0x55ffa4663100 msgr2=0x55ffa46e09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 -- 172.21.15.60:0/1502903209 shutdown_connections 2024-04-25T08:32:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.150+0000 7f6af11a7740 1 -- 172.21.15.60:0/1502903209 wait complete. 2024-04-25T08:32:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.151+0000 7f6af11a7740 1 -- start start 2024-04-25T08:32:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.151+0000 7f6af11a7740 1 Processor -- start 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa459d750 con 0x55ffa46c59d0 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa45fadc0 con 0x55ffa469af10 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffa46f1e20 con 0x55ffa46d75e0 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6af11a7740 10 client.0 _ll_register_callbacks cb 0x55ffa45b5ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46dd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49578/0 (socket says 172.21.15.60:49578) 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1950647745 learned_addr learned my addr 172.21.15.60:0/1950647745 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 msgr2=0x55ffa46dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46dd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 msgr2=0x55ffa46b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1950647745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffa42d33f0 con 0x55ffa46c59d0 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.152+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46dd1d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8035040 tx=0x7f6ad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b836cb7e49fb5837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ac6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/1950647745 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8041030 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/1950647745 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ad8006250 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ae0056000 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/1950647745 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad804d050 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 10 client.6333 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ffa4440290 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/1950647745 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0056000 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/1950647745 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ad8050020 con 0x55ffa46c59d0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 10 client.6333 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 10 client.6333 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 20 client.6333 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa46fbdc0 0x55ffa46fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 4 client.6333 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffa460c260 con 0x55ffa46fbdc0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.153+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa46fbdc0 0x55ffa46fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6ac6ffd640 10 client.6333 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa46fbdc0 0x55ffa46fe1b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d075be1d025531b3 server_cookie=2ae2e3c1802289b9 in_seq=0 out_seq=0 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/1950647745 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffa460c260 con 0x55ffa46fbdc0 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6ac6ffd640 10 client.6333 handle_command_reply: tid=0 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 client.6333 shutdown 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 msgr2=0x55ffa46dd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46dd1d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8035040 tx=0x7f6ad8004730 comp rx=0 tx=0).stop 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 shutdown_connections 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa46fbdc0 0x55ffa46fe1b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46dd1d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1950647745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6ac6ffd640 0 client.6333 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.154+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 >> 172.21.15.60:0/1950647745 conn(0x55ffa4663100 msgr2=0x55ffa46e3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.155+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 shutdown_connections 2024-04-25T08:32:35.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.155+0000 7f6af11a7740 1 -- 172.21.15.60:0/1950647745 wait complete. 2024-04-25T08:32:35.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.155+0000 7f6af11a7740 20 client.6333 trim_cache size 0 max 16384 2024-04-25T08:32:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 Processor -- start 2024-04-25T08:32:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 -- start start 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae82002d0 con 0x55ffa46c59d0 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.170+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81ff470 con 0x55ffa469af10 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81467e0 con 0x55ffa46d75e0 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49594/0 (socket says 172.21.15.60:49594) 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1251819874 learned_addr learned my addr 172.21.15.60:0/1251819874 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 msgr2=0x55ffa46c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 msgr2=0x55ffa46debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 -- 172.21.15.60:0/1251819874 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffa42ac4b0 con 0x55ffa46c59d0 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c85b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80062b0 tx=0x7f6ad803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9350731c402b3e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6acd7fa640 1 -- 172.21.15.60:0/1251819874 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8041040 con 0x55ffa46c59d0 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6acd7fa640 1 -- 172.21.15.60:0/1251819874 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8006830 con 0x55ffa46c59d0 2024-04-25T08:32:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.171+0000 7f6acd7fa640 1 -- 172.21.15.60:0/1251819874 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8037040 con 0x55ffa46c59d0 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 -- 172.21.15.60:0/1251819874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 msgr2=0x55ffa46c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c85b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80062b0 tx=0x7f6ad803d000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 -- 172.21.15.60:0/1251819874 shutdown_connections 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa46d75e0 0x55ffa46c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46c59d0 0x55ffa46c85b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 --2- 172.21.15.60:0/1251819874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 -- 172.21.15.60:0/1251819874 >> 172.21.15.60:0/1251819874 conn(0x55ffa4663100 msgr2=0x55ffa469bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 -- 172.21.15.60:0/1251819874 shutdown_connections 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 -- 172.21.15.60:0/1251819874 wait complete. 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 -- start start 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.172+0000 7f6af11a7740 1 Processor -- start 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46c59d0 0x55ffa46c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 0x55ffa46c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa46ddb30 con 0x55ffa46d75e0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81feef0 con 0x55ffa46c59d0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffa4700a60 con 0x55ffa469af10 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6af11a7740 10 client.0 _ll_register_callbacks cb 0x7f6ad8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6aeca3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 0x55ffa46c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6aeca3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 0x55ffa46c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49610/0 (socket says 172.21.15.60:49610) 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3008724908 learned_addr learned my addr 172.21.15.60:0/3008724908 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.173+0000 7f6ae77fe640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46c59d0 0x55ffa46c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 msgr2=0x55ffa46debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46c59d0 msgr2=0x55ffa46c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46c59d0 0x55ffa46c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6aeca3b640 1 -- 172.21.15.60:0/3008724908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffa46fa460 con 0x55ffa46d75e0 2024-04-25T08:32:35.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6aeca3b640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 0x55ffa46c4ca0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0014040 tx=0x7f6ae005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3cfda5e9701349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6ac6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/3008724908 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae007b060 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/3008724908 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ae0056260 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ad8006830 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/3008724908 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ae0081050 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 10 client.6339 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55ffa46c63a0 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/3008724908 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8006830 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/3008724908 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ae0075240 con 0x55ffa46d75e0 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 10 client.6339 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 10 client.6339 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 20 client.6339 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa4695d30 0x55ffa46996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 4 client.6339 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T08:32:35.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.174+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55ffa46065b0 con 0x55ffa4695d30 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa4695d30 0x55ffa46996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6ac6ffd640 10 client.6339 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6ae7fff640 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa4695d30 0x55ffa46996d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5669f595360dca server_cookie=6bcaf2f883011c25 in_seq=0 out_seq=0 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6ac6ffd640 1 -- 172.21.15.60:0/3008724908 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ffa46065b0 con 0x55ffa4695d30 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6ac6ffd640 10 client.6339 handle_command_reply: tid=0 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6af11a7740 1 client.6339 shutdown 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 msgr2=0x55ffa46c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.175+0000 7f6af11a7740 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 0x55ffa46c4ca0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0014040 tx=0x7f6ae005b000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 shutdown_connections 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ffa46d75e0 0x55ffa46c4ca0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ffa46c59d0 0x55ffa46c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55ffa4695d30 0x55ffa46996d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 --2- 172.21.15.60:0/3008724908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ffa469af10 0x55ffa46debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 >> 172.21.15.60:0/3008724908 conn(0x55ffa4663100 msgr2=0x55ffa46c85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6ac6ffd640 0 client.6339 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 shutdown_connections 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 1 -- 172.21.15.60:0/3008724908 wait complete. 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6af11a7740 20 client.6339 trim_cache size 0 max 16384 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "a5827676-2461-4422-af56-4bb0a384300e", 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:35.179 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:35.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ad406b4d0 msgr2=0x7f6ad406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ad406b4d0 0x7f6ad406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009560 tx=0x7f6adc009190 comp rx=0 tx=0).stop 2024-04-25T08:32:35.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 msgr2=0x7f6ae80cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 0x7f6ae80cd420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c7b0 tx=0x7f6ae000cc70 comp rx=0 tx=0).stop 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 shutdown_connections 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ae815dac0 0x7f6ae80cd420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ae80d34c0 0x7f6ae80c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ad406b4d0 0x7f6ad406d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 --2- 172.21.15.60:0/3609853555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ae80d1f30 0x7f6ae8160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 >> 172.21.15.60:0/3609853555 conn(0x7f6ae8061c30 msgr2=0x7f6ae80b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 shutdown_connections 2024-04-25T08:32:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.176+0000 7f6aee49e640 1 -- 172.21.15.60:0/3609853555 wait complete. 2024-04-25T08:32:35.183 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'a5827676-2461-4422-af56-4bb0a384300e', 'mode': 'asynchronous'}' 2024-04-25T08:32:35.183 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:32:35.183 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:35.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:32:35.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.291+0000 7fde6bb96640 1 Processor -- start 2024-04-25T08:32:35.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.291+0000 7fde6bb96640 1 -- start start 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde6bb96640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde6415b7d0 0x7fde640cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde6bb96640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde640b5620 0x7fde640b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde6bb96640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6406cf50 con 0x7fde6406c2d0 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde6bb96640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640ce530 con 0x7fde640b5620 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde6bb96640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640ce810 con 0x7fde6415b7d0 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde640b5620 0x7fde640b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde640b5620 0x7fde640b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49614/0 (socket says 172.21.15.60:49614) 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 -- 172.21.15.60:0/4198936489 learned_addr learned my addr 172.21.15.60:0/4198936489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69932640 1 --2- 172.21.15.60:0/4198936489 >> v2:172.21.15.100:3300/0 conn(0x7fde6415b7d0 0x7fde640cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 -- 172.21.15.60:0/4198936489 >> v2:172.21.15.100:3300/0 conn(0x7fde6415b7d0 msgr2=0x7fde640cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 --2- 172.21.15.60:0/4198936489 >> v2:172.21.15.100:3300/0 conn(0x7fde6415b7d0 0x7fde640cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 -- 172.21.15.60:0/4198936489 >> v1:172.21.15.100:6789/0 conn(0x7fde6406c2d0 legacy=0x7fde6406c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.292+0000 7fde69131640 1 -- 172.21.15.60:0/4198936489 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640ceaf0 con 0x7fde640b5620 2024-04-25T08:32:35.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.293+0000 7fde69131640 1 --2- 172.21.15.60:0/4198936489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde640b5620 0x7fde640b5ab0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fde6408c0d0 tx=0x7fde5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90345e92fb8aeef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.293+0000 7fde68930640 1 -- 172.21.15.60:0/4198936489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58037070 con 0x7fde640b5620 2024-04-25T08:32:35.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.293+0000 7fde68930640 1 -- 172.21.15.60:0/4198936489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58032440 con 0x7fde640b5620 2024-04-25T08:32:35.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.293+0000 7fde68930640 1 -- 172.21.15.60:0/4198936489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58032810 con 0x7fde640b5620 2024-04-25T08:32:35.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 -- 172.21.15.60:0/4198936489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde640b5620 msgr2=0x7fde640b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 --2- 172.21.15.60:0/4198936489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde640b5620 0x7fde640b5ab0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fde6408c0d0 tx=0x7fde5802a270 comp rx=0 tx=0).stop 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 -- 172.21.15.60:0/4198936489 shutdown_connections 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 --2- 172.21.15.60:0/4198936489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde640b5620 0x7fde640b5ab0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 --2- 172.21.15.60:0/4198936489 >> v2:172.21.15.100:3300/0 conn(0x7fde6415b7d0 0x7fde640cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 -- 172.21.15.60:0/4198936489 >> 172.21.15.60:0/4198936489 conn(0x7fde640d0800 msgr2=0x7fde640d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 -- 172.21.15.60:0/4198936489 shutdown_connections 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.294+0000 7fde6bb96640 1 -- 172.21.15.60:0/4198936489 wait complete. 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.295+0000 7fde6bb96640 1 Processor -- start 2024-04-25T08:32:35.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.295+0000 7fde6bb96640 1 -- start start 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6bb96640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 0x7fde64154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6bb96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde640b5620 0x7fde640b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6bb96640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde6415b7d0 0x7fde640b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6bb96640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640cf6d0 con 0x7fde6406c2d0 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6bb96640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640cf550 con 0x7fde640b5620 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6bb96640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415bbb0 con 0x7fde6415b7d0 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 0x7fde64154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 0x7fde64154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49624/0 (socket says 172.21.15.60:49624) 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 -- 172.21.15.60:0/672692295 learned_addr learned my addr 172.21.15.60:0/672692295 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde6a133640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde6415b7d0 0x7fde640b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69131640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde640b5620 0x7fde640b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 -- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde6415b7d0 msgr2=0x7fde640b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde6415b7d0 0x7fde640b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 -- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde640b5620 msgr2=0x7fde640b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde640b5620 0x7fde640b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 -- 172.21.15.60:0/672692295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640ca060 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.296+0000 7fde69932640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 0x7fde64154690 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde54007e70 tx=0x7fde5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.297+0000 7fde627fc640 1 -- 172.21.15.60:0/672692295 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54014620 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.297+0000 7fde627fc640 1 -- 172.21.15.60:0/672692295 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54019070 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.297+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde640ca340 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.297+0000 7fde627fc640 1 -- 172.21.15.60:0/672692295 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54014a00 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.297+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde640c9330 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.298+0000 7fde627fc640 1 -- 172.21.15.60:0/672692295 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde54014ba0 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.298+0000 7fde627fc640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fde5006b4d0 0x7fde5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.299+0000 7fde627fc640 1 -- 172.21.15.60:0/672692295 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde5408ab70 con 0x7fde6406c2d0 2024-04-25T08:32:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.299+0000 7fde69131640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fde5006b4d0 0x7fde5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.299+0000 7fde69131640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fde5006b4d0 0x7fde5006d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fde58034000 tx=0x7fde5802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 Processor -- start 2024-04-25T08:32:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 -- start start 2024-04-25T08:32:35.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 0x558695a4c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a49590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558695968e60 con 0x558695a06130 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558695912120 con 0x558695a4bf60 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558695964f00 con 0x558695a2caf0 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde69131640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 0x558695a4c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde69131640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 0x558695a4c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49632/0 (socket says 172.21.15.60:49632) 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde69131640 1 -- 172.21.15.60:0/55975369 learned_addr learned my addr 172.21.15.60:0/55975369 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde6a133640 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a49590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.300+0000 7fde69932640 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde69131640 1 -- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 msgr2=0x558695a49590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde69131640 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a49590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde69131640 1 -- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 msgr2=0x558695a4a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde69131640 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a4a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde69131640 1 -- 172.21.15.60:0/55975369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558695749f00 con 0x558695a06130 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde6a133640 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a49590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde69131640 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 0x558695a4c340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde58035040 tx=0x7fde5802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e903c33848011fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde4b7fe640 1 -- 172.21.15.60:0/55975369 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5804e070 con 0x558695a06130 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde4b7fe640 1 -- 172.21.15.60:0/55975369 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58032e00 con 0x558695a06130 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.301+0000 7fde4b7fe640 1 -- 172.21.15.60:0/55975369 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58037040 con 0x558695a06130 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 -- 172.21.15.60:0/55975369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 msgr2=0x558695a4c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 0x558695a4c340 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde58035040 tx=0x7fde5802c000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 -- 172.21.15.60:0/55975369 shutdown_connections 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a49590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a06130 0x558695a4c340 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 --2- 172.21.15.60:0/55975369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a4a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 -- 172.21.15.60:0/55975369 >> 172.21.15.60:0/55975369 conn(0x5586959ccf70 msgr2=0x558695a03150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 -- 172.21.15.60:0/55975369 shutdown_connections 2024-04-25T08:32:35.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 -- 172.21.15.60:0/55975369 wait complete. 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 -- start start 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.302+0000 7fde6e970740 1 Processor -- start 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a2fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 0x558695a30190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558695a4b760 con 0x558695a2caf0 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558695907cd0 con 0x558695a4bf60 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558695a4b480 con 0x558695a06130 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6e970740 10 client.0 _ll_register_callbacks cb 0x55869567d2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde69131640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 0x558695a30190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde69131640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 0x558695a30190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49636/0 (socket says 172.21.15.60:49636) 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde69131640 1 -- 172.21.15.60:0/2469082993 learned_addr learned my addr 172.21.15.60:0/2469082993 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde69932640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a2fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.303+0000 7fde6a133640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde69131640 1 -- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 msgr2=0x558695a2fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde69131640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a2fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde69131640 1 -- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 msgr2=0x558695a5aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde69131640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a5aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde69131640 1 -- 172.21.15.60:0/2469082993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55869563d7f0 con 0x558695a2caf0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde69131640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 0x558695a30190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fde580090b0 tx=0x7fde58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a08732dd99a5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde48ff9640 1 -- 172.21.15.60:0/2469082993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5804e070 con 0x558695a2caf0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde58032e00 con 0x558695a2caf0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde48ff9640 1 -- 172.21.15.60:0/2469082993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde5803ea80 con 0x558695a2caf0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde48ff9640 1 -- 172.21.15.60:0/2469082993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde58059050 con 0x558695a2caf0 2024-04-25T08:32:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 10 client.6351 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5586957aa690 con 0x558695a2caf0 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde48ff9640 1 -- 172.21.15.60:0/2469082993 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58032e00 con 0x558695a2caf0 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde48ff9640 1 -- 172.21.15.60:0/2469082993 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde5805c020 con 0x558695a2caf0 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 10 client.6351 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 10 client.6351 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 20 client.6351 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.304+0000 7fde6e970740 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a03520 0x558695a03900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde6e970740 4 client.6351 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586959763a0 con 0x558695a03520 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde69932640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a03520 0x558695a03900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde48ff9640 10 client.6351 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde69932640 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a03520 0x558695a03900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250e9a702f409daf server_cookie=5d287eb3a68c2230 in_seq=0 out_seq=0 2024-04-25T08:32:35.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde48ff9640 1 -- 172.21.15.60:0/2469082993 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586959763a0 con 0x558695a03520 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde48ff9640 10 client.6351 handle_command_reply: tid=0 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.305+0000 7fde6e970740 1 client.6351 shutdown 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 msgr2=0x558695a30190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 0x558695a30190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fde580090b0 tx=0x7fde58004730 comp rx=0 tx=0).stop 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 shutdown_connections 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a4bf60 0x558695a5aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a2caf0 0x558695a30190 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a03520 0x558695a03900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 --2- 172.21.15.60:0/2469082993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a2fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde48ff9640 0 client.6351 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 >> 172.21.15.60:0/2469082993 conn(0x5586959ccf70 msgr2=0x558695a29f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 shutdown_connections 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 1 -- 172.21.15.60:0/2469082993 wait complete. 2024-04-25T08:32:35.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.306+0000 7fde6e970740 20 client.6351 trim_cache size 0 max 16384 2024-04-25T08:32:35.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.321+0000 7fde6e970740 1 Processor -- start 2024-04-25T08:32:35.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.321+0000 7fde6e970740 1 -- start start 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a2caf0 0x558695aaaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6e970740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6e970740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64200500 con 0x558695a4bf60 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641ff6e0 con 0x558695a2caf0 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde641feae0 con 0x558695a06130 2024-04-25T08:32:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49642/0 (socket says 172.21.15.60:49642) 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 -- 172.21.15.60:0/2545253464 learned_addr learned my addr 172.21.15.60:0/2545253464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde69131640 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a2caf0 0x558695aaaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde69932640 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 -- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 msgr2=0x558695a30b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a30b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 -- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a2caf0 msgr2=0x558695aaaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a2caf0 0x558695aaaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde6a133640 1 -- 172.21.15.60:0/2545253464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586956168b0 con 0x558695a4bf60 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.322+0000 7fde69131640 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a2caf0 0x558695aaaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6a133640 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2bee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005460 tx=0x7fde5c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab1aa7cc199ac94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde4affd640 1 -- 172.21.15.60:0/2545253464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c007030 con 0x558695a4bf60 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde4affd640 1 -- 172.21.15.60:0/2545253464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c0071d0 con 0x558695a4bf60 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde4affd640 1 -- 172.21.15.60:0/2545253464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0073c0 con 0x558695a4bf60 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 -- 172.21.15.60:0/2545253464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 msgr2=0x558695a2bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2bee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005460 tx=0x7fde5c0039c0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 -- 172.21.15.60:0/2545253464 shutdown_connections 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2bee0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a2caf0 0x558695aaaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 --2- 172.21.15.60:0/2545253464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a06130 0x558695a30b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 -- 172.21.15.60:0/2545253464 >> 172.21.15.60:0/2545253464 conn(0x5586959ccf70 msgr2=0x558695a2b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 -- 172.21.15.60:0/2545253464 shutdown_connections 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.323+0000 7fde6e970740 1 -- 172.21.15.60:0/2545253464 wait complete. 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.324+0000 7fde6e970740 1 -- start start 2024-04-25T08:32:35.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.324+0000 7fde6e970740 1 Processor -- start 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a06130 0x558695a5e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558695a5dbc0 con 0x558695a4bf60 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641ff0e0 con 0x558695a06130 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558695a00510 con 0x558695a2caf0 2024-04-25T08:32:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6e970740 10 client.0 _ll_register_callbacks cb 0x7fde58035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49656/0 (socket says 172.21.15.60:49656) 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 -- 172.21.15.60:0/3150204839 learned_addr learned my addr 172.21.15.60:0/3150204839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde69932640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a06130 0x558695a5e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde69131640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 -- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 msgr2=0x558695a2bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a2bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 -- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a06130 msgr2=0x558695a5e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a06130 0x558695a5e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 -- 172.21.15.60:0/3150204839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558695a47830 con 0x558695a4bf60 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde6a133640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2c420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fde5c000f60 tx=0x7fde5c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a384d5bcbdc7f2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.325+0000 7fde48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde48ff9640 1 -- 172.21.15.60:0/3150204839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c027060 con 0x558695a4bf60 2024-04-25T08:32:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde48ff9640 1 -- 172.21.15.60:0/3150204839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde5c016480 con 0x558695a4bf60 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde5c0071d0 con 0x558695a4bf60 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde48ff9640 1 -- 172.21.15.60:0/3150204839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5c007030 con 0x558695a4bf60 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 10 client.6357 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x558695a979c0 con 0x558695a4bf60 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde48ff9640 1 -- 172.21.15.60:0/3150204839 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c0071d0 con 0x558695a4bf60 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde48ff9640 1 -- 172.21.15.60:0/3150204839 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde5c0166c0 con 0x558695a4bf60 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 10 client.6357 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 10 client.6357 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 20 client.6357 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a5a990 0x5586959ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 4 client.6357 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x5586959706f0 con 0x558695a5a990 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde69932640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a5a990 0x5586959ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde69932640 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a5a990 0x5586959ff000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b0100f7d2050b8 server_cookie=cb9c24c5c1f712ee in_seq=0 out_seq=0 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.326+0000 7fde48ff9640 10 client.6357 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde48ff9640 1 -- 172.21.15.60:0/3150204839 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5586959706f0 con 0x558695a5a990 2024-04-25T08:32:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde48ff9640 10 client.6357 handle_command_reply: tid=0 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 client.6357 shutdown 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 msgr2=0x558695a2c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2c420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fde5c000f60 tx=0x7fde5c000ed0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 shutdown_connections 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558695a4bf60 0x558695a2c420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558695a5a990 0x5586959ff000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558695a2caf0 0x558695a2bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 --2- 172.21.15.60:0/3150204839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558695a06130 0x558695a5e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 >> 172.21.15.60:0/3150204839 conn(0x5586959ccf70 msgr2=0x558695a2b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde48ff9640 0 client.6357 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.327+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 shutdown_connections 2024-04-25T08:32:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6e970740 1 -- 172.21.15.60:0/3150204839 wait complete. 2024-04-25T08:32:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6e970740 20 client.6357 trim_cache size 0 max 16384 2024-04-25T08:32:35.332 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:35.332 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:35.332 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "dfc6d281-a2dc-4974-8185-c2bc9b553cd0", 2024-04-25T08:32:35.332 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:35.332 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fde5006b4d0 msgr2=0x7fde5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fde5006b4d0 0x7fde5006d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fde58034000 tx=0x7fde5802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 msgr2=0x7fde64154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 0x7fde64154690 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde54007e70 tx=0x7fde5400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 shutdown_connections 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde6415b7d0 0x7fde640b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fde5006b4d0 0x7fde5006d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde640b5620 0x7fde640b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 --2- 172.21.15.60:0/672692295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde6406c2d0 0x7fde64154690 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 >> 172.21.15.60:0/672692295 conn(0x7fde640d0800 msgr2=0x7fde6415afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 shutdown_connections 2024-04-25T08:32:35.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.328+0000 7fde6bb96640 1 -- 172.21.15.60:0/672692295 wait complete. 2024-04-25T08:32:35.336 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'dfc6d281-a2dc-4974-8185-c2bc9b553cd0', 'mode': 'asynchronous'}' 2024-04-25T08:32:35.336 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T08:32:35.336 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:35.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T08:32:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.443+0000 7f072c59d640 1 Processor -- start 2024-04-25T08:32:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.443+0000 7f072c59d640 1 -- start start 2024-04-25T08:32:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f072c59d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07240cf600 0x7f07240cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f072c59d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07240c75e0 con 0x7f07240cbf20 2024-04-25T08:32:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f072c59d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07240c78c0 con 0x7f072415bb50 2024-04-25T08:32:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f072c59d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07240c8680 con 0x7f07240cf600 2024-04-25T08:32:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f072ab3a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f07240cbf20 0x7f07240cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55842/0 (socket says 172.21.15.60:55842) 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f072ab3a640 1 -- 172.21.15.60:0/3978082226 learned_addr learned my addr 172.21.15.60:0/3978082226 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.444+0000 7f0729b38640 1 --2- 172.21.15.60:0/3978082226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07240cf600 0x7f07240cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.445+0000 7f0729337640 1 -- 172.21.15.60:0/3978082226 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273276657 0 0) 0x7f07240c75e0 con 0x7f07240cbf20 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.445+0000 7f0729337640 1 -- 172.21.15.60:0/3978082226 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0710003520 con 0x7f07240cbf20 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.445+0000 7f0729337640 1 -- 172.21.15.60:0/3978082226 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1314410731 0 0) 0x7f0710003520 con 0x7f07240cbf20 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.445+0000 7f0729b38640 1 -- 172.21.15.60:0/3978082226 >> v1:172.21.15.100:6789/0 conn(0x7f072415bb50 legacy=0x7f07240c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.445+0000 7f0729b38640 1 -- 172.21.15.60:0/3978082226 >> v1:172.21.15.60:6789/0 conn(0x7f07240cbf20 legacy=0x7f07240cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.445+0000 7f0729b38640 1 -- 172.21.15.60:0/3978082226 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240c7ba0 con 0x7f07240cf600 2024-04-25T08:32:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f0729b38640 1 --2- 172.21.15.60:0/3978082226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07240cf600 0x7f07240cfa40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07140097d0 tx=0x7f071402a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7dfe69099f5e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f0729337640 1 -- 172.21.15.60:0/3978082226 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714037070 con 0x7f07240cf600 2024-04-25T08:32:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f0729337640 1 -- 172.21.15.60:0/3978082226 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071402abd0 con 0x7f07240cf600 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f0729337640 1 -- 172.21.15.60:0/3978082226 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200019f0 con 0x7f07240cf600 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 -- 172.21.15.60:0/3978082226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07240cf600 msgr2=0x7f07240cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 --2- 172.21.15.60:0/3978082226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07240cf600 0x7f07240cfa40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07140097d0 tx=0x7f071402a140 comp rx=0 tx=0).stop 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 -- 172.21.15.60:0/3978082226 shutdown_connections 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 --2- 172.21.15.60:0/3978082226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07240cf600 0x7f07240cfa40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 -- 172.21.15.60:0/3978082226 >> 172.21.15.60:0/3978082226 conn(0x7f072406f500 msgr2=0x7f072406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 -- 172.21.15.60:0/3978082226 shutdown_connections 2024-04-25T08:32:35.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.446+0000 7f072c59d640 1 -- 172.21.15.60:0/3978082226 wait complete. 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 Processor -- start 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 -- start start 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 0x7f07240ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07240cf600 0x7f072414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f072415bb50 0x7f07240c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07240c8160 con 0x7f07240cbf20 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07240cff80 con 0x7f07240cf600 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072c59d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07240c72a0 con 0x7f072415bb50 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.448+0000 7f072a339640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 0x7f07240ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 0x7f07240ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49668/0 (socket says 172.21.15.60:49668) 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 -- 172.21.15.60:0/1778651232 learned_addr learned my addr 172.21.15.60:0/1778651232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072ab3a640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f072415bb50 0x7f07240c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f0729b38640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07240cf600 0x7f072414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 -- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f072415bb50 msgr2=0x7f07240c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f072415bb50 0x7f07240c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 -- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07240cf600 msgr2=0x7f072414dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07240cf600 0x7f072414dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 -- 172.21.15.60:0/1778651232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240c8fb0 con 0x7f07240cbf20 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072a339640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 0x7f07240ca050 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c540 tx=0x7f071c00ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f071affd640 1 -- 172.21.15.60:0/1778651232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c017070 con 0x7f07240cbf20 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f071affd640 1 -- 172.21.15.60:0/1778651232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c005480 con 0x7f07240cbf20 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f071affd640 1 -- 172.21.15.60:0/1778651232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c003830 con 0x7f07240cbf20 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07240c9290 con 0x7f07240cbf20 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.449+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07240c9850 con 0x7f07240cbf20 2024-04-25T08:32:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.451+0000 7f071affd640 1 -- 172.21.15.60:0/1778651232 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f071c01a070 con 0x7f07240cbf20 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.451+0000 7f071affd640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f070806b4d0 0x7f070806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.451+0000 7f071affd640 1 -- 172.21.15.60:0/1778651232 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f071c089840 con 0x7f07240cbf20 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.451+0000 7f0729b38640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f070806b4d0 0x7f070806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.452+0000 7f0729b38640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f070806b4d0 0x7f070806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0714034000 tx=0x7f071402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.452+0000 7f072f377740 1 Processor -- start 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 -- start start 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebccfe0 0x556daebcf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebcd3c0 0x556daebcfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 0x556daeba3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556daeb08a50 con 0x556daeb9fc40 2024-04-25T08:32:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556daeab1d40 con 0x556daebccfe0 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556daeb04af0 con 0x556daebcd3c0 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072ab3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 0x556daeba3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072ab3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 0x556daeba3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49676/0 (socket says 172.21.15.60:49676) 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f072ab3a640 1 -- 172.21.15.60:0/2459856519 learned_addr learned my addr 172.21.15.60:0/2459856519 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.453+0000 7f0729b38640 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebcd3c0 0x556daebcfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072a339640 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebccfe0 0x556daebcf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072ab3a640 1 -- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebcd3c0 msgr2=0x556daebcfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072ab3a640 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebcd3c0 0x556daebcfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072ab3a640 1 -- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebccfe0 msgr2=0x556daebcf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072ab3a640 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebccfe0 0x556daebcf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072ab3a640 1 -- 172.21.15.60:0/2459856519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dae8e9690 con 0x556daeb9fc40 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f072ab3a640 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 0x556daeba3450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0720004f00 tx=0x7f0720001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c819bc9b039111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f0707fff640 1 -- 172.21.15.60:0/2459856519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200078b0 con 0x556daeb9fc40 2024-04-25T08:32:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f0707fff640 1 -- 172.21.15.60:0/2459856519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0720018070 con 0x556daeb9fc40 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.454+0000 7f0707fff640 1 -- 172.21.15.60:0/2459856519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200135f0 con 0x556daeb9fc40 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 -- 172.21.15.60:0/2459856519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 msgr2=0x556daeba3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 0x556daeba3450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0720004f00 tx=0x7f0720001960 comp rx=0 tx=0).stop 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 -- 172.21.15.60:0/2459856519 shutdown_connections 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daeb9fc40 0x556daeba3450 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebcd3c0 0x556daebcfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 --2- 172.21.15.60:0/2459856519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebccfe0 0x556daebcf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 -- 172.21.15.60:0/2459856519 >> 172.21.15.60:0/2459856519 conn(0x556daeba1eb0 msgr2=0x556daebea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 -- 172.21.15.60:0/2459856519 shutdown_connections 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 -- 172.21.15.60:0/2459856519 wait complete. 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 -- start start 2024-04-25T08:32:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.455+0000 7f072f377740 1 Processor -- start 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daebc7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebedf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 0x556daeba08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556daeba5060 con 0x556daebee460 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556daeaa7340 con 0x556daebcd3c0 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556daeba5340 con 0x556daebccfe0 2024-04-25T08:32:35.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072f377740 10 client.0 _ll_register_callbacks cb 0x556daeac4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.456+0000 7f072ab3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 0x556daeba08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 0x556daeba08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49684/0 (socket says 172.21.15.60:49684) 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 -- 172.21.15.60:0/987076770 learned_addr learned my addr 172.21.15.60:0/987076770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072a339640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daebc7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f0729b38640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebedf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 -- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 msgr2=0x556daebc7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daebc7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 -- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 msgr2=0x556daebedf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebedf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 -- 172.21.15.60:0/987076770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dae949e20 con 0x556daebee460 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072ab3a640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 0x556daeba08f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0720013b70 tx=0x7f0720012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4889cef58025e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f07057fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f07057fa640 1 -- 172.21.15.60:0/987076770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072001c450 con 0x556daebee460 2024-04-25T08:32:35.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f07057fa640 1 -- 172.21.15.60:0/987076770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f072001c5f0 con 0x556daebee460 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0720018070 con 0x556daebee460 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f07057fa640 1 -- 172.21.15.60:0/987076770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f072002e050 con 0x556daebee460 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072f377740 10 client.6369 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556dae7dcf80 con 0x556daebee460 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.457+0000 7f07057fa640 1 -- 172.21.15.60:0/987076770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0720018070 con 0x556daebee460 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f07057fa640 1 -- 172.21.15.60:0/987076770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0720031020 con 0x556daebee460 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072f377740 10 client.6369 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072f377740 10 client.6369 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072f377740 20 client.6369 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072f377740 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebc80e0 0x556daebc84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072f377740 4 client.6369 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556daeb15f90 con 0x556daebc80e0 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072a339640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebc80e0 0x556daebc84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f07057fa640 10 client.6369 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f072a339640 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebc80e0 0x556daebc84f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e010a0be471368e4 server_cookie=56ae056c9b15765f in_seq=0 out_seq=0 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f07057fa640 1 -- 172.21.15.60:0/987076770 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556daeb15f90 con 0x556daebc80e0 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.458+0000 7f07057fa640 10 client.6369 handle_command_reply: tid=0 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 client.6369 shutdown 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 msgr2=0x556daeba08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 0x556daeba08f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0720013b70 tx=0x7f0720012000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 shutdown_connections 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebee460 0x556daeba08f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebedf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebc80e0 0x556daebc84f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 --2- 172.21.15.60:0/987076770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daebc7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 >> 172.21.15.60:0/987076770 conn(0x556daeb9fc40 msgr2=0x556daebcc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f07057fa640 0 client.6369 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 shutdown_connections 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 1 -- 172.21.15.60:0/987076770 wait complete. 2024-04-25T08:32:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.459+0000 7f072f377740 20 client.6369 trim_cache size 0 max 16384 2024-04-25T08:32:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.474+0000 7f072f377740 1 Processor -- start 2024-04-25T08:32:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.474+0000 7f072f377740 1 -- start start 2024-04-25T08:32:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daeba4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f072f377740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 0x556daebe9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f072f377740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07241fe230 con 0x556daebcd3c0 2024-04-25T08:32:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07241fd410 con 0x556daebee460 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07240b5070 con 0x556daebccfe0 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 0x556daebe9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 0x556daebe9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49686/0 (socket says 172.21.15.60:49686) 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 -- 172.21.15.60:0/1157396576 learned_addr learned my addr 172.21.15.60:0/1157396576 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 -- 172.21.15.60:0/1157396576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 msgr2=0x556daeba4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daeba4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 -- 172.21.15.60:0/1157396576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebee460 msgr2=0x556daebed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.475+0000 7f0729b38640 1 -- 172.21.15.60:0/1157396576 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556daeac8bc0 con 0x556daebcd3c0 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f0729b38640 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 0x556daebe9cd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0714040040 tx=0x7f071402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83f26c8131ea6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f07077fe640 1 -- 172.21.15.60:0/1157396576 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714037040 con 0x556daebcd3c0 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f07077fe640 1 -- 172.21.15.60:0/1157396576 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714004460 con 0x556daebcd3c0 2024-04-25T08:32:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f07077fe640 1 -- 172.21.15.60:0/1157396576 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714003830 con 0x556daebcd3c0 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 -- 172.21.15.60:0/1157396576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 msgr2=0x556daebe9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 0x556daebe9cd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0714040040 tx=0x7f071402c000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 -- 172.21.15.60:0/1157396576 shutdown_connections 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebcd3c0 0x556daebe9cd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 --2- 172.21.15.60:0/1157396576 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebccfe0 0x556daeba4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 -- 172.21.15.60:0/1157396576 >> 172.21.15.60:0/1157396576 conn(0x556daeb9fc40 msgr2=0x556daeba5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 -- 172.21.15.60:0/1157396576 shutdown_connections 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.476+0000 7f072f377740 1 -- 172.21.15.60:0/1157396576 wait complete. 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.477+0000 7f072f377740 1 -- start start 2024-04-25T08:32:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.477+0000 7f072f377740 1 Processor -- start 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 0x556daebe7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebc5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556daeba02d0 con 0x556daebccfe0 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07241fce10 con 0x556daebcd3c0 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556daeb9f390 con 0x556daebee460 2024-04-25T08:32:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072f377740 10 client.0 _ll_register_callbacks cb 0x7f0720004d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 0x556daebe7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 0x556daebe7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49700/0 (socket says 172.21.15.60:49700) 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 -- 172.21.15.60:0/439584799 learned_addr learned my addr 172.21.15.60:0/439584799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072ab3a640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f0729b38640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebc5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 -- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebee460 msgr2=0x556daebed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 -- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 msgr2=0x556daebc5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebc5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 -- 172.21.15.60:0/439584799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556daec59930 con 0x556daebccfe0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f072a339640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 0x556daebe7180 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f071c05c030 tx=0x7f071c0011d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d88b6a65efd55c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.478+0000 7f07057fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f07057fa640 1 -- 172.21.15.60:0/439584799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c054440 con 0x556daebccfe0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f07057fa640 1 -- 172.21.15.60:0/439584799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f071c082d80 con 0x556daebccfe0 2024-04-25T08:32:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0714004460 con 0x556daebccfe0 2024-04-25T08:32:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f07057fa640 1 -- 172.21.15.60:0/439584799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f071c0545e0 con 0x556daebccfe0 2024-04-25T08:32:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 10 client.6375 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556daeba3ed0 con 0x556daebccfe0 2024-04-25T08:32:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f07057fa640 1 -- 172.21.15.60:0/439584799 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714004460 con 0x556daebccfe0 2024-04-25T08:32:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f07057fa640 1 -- 172.21.15.60:0/439584799 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f071c060030 con 0x556daebccfe0 2024-04-25T08:32:35.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 10 client.6375 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 10 client.6375 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 20 client.6375 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebede70 0x556daebc8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 4 client.6375 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T08:32:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x556daeb102e0 con 0x556daebede70 2024-04-25T08:32:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f0729b38640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebede70 0x556daebc8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f07057fa640 10 client.6375 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.479+0000 7f0729b38640 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebede70 0x556daebc8be0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c79c74a6c301014 server_cookie=38a53e94eecb1379 in_seq=0 out_seq=0 2024-04-25T08:32:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f07057fa640 1 -- 172.21.15.60:0/439584799 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556daeb102e0 con 0x556daebede70 2024-04-25T08:32:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f07057fa640 10 client.6375 handle_command_reply: tid=0 2024-04-25T08:32:35.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 client.6375 shutdown 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 msgr2=0x556daebe7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 0x556daebe7180 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f071c05c030 tx=0x7f071c0011d0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 shutdown_connections 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556daebede70 0x556daebc8be0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556daebee460 0x556daebed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556daebcd3c0 0x556daebc5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 --2- 172.21.15.60:0/439584799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556daebccfe0 0x556daebe7180 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f07057fa640 0 client.6375 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 >> 172.21.15.60:0/439584799 conn(0x556daeb9fc40 msgr2=0x556daebc7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 shutdown_connections 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 1 -- 172.21.15.60:0/439584799 wait complete. 2024-04-25T08:32:35.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.480+0000 7f072f377740 20 client.6375 trim_cache size 0 max 16384 2024-04-25T08:32:35.485 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:35.485 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:35.485 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "bb5b1f2e-e235-4a9e-8ca6-ee83a15dee41", 2024-04-25T08:32:35.485 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:35.485 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f070806b4d0 msgr2=0x7f070806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f070806b4d0 0x7f070806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0714034000 tx=0x7f071402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 msgr2=0x7f07240ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 0x7f07240ca050 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c540 tx=0x7f071c00ca00 comp rx=0 tx=0).stop 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 shutdown_connections 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f072415bb50 0x7f07240c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f070806b4d0 0x7f070806d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07240cf600 0x7f072414dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 --2- 172.21.15.60:0/1778651232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07240cbf20 0x7f07240ca050 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 >> 172.21.15.60:0/1778651232 conn(0x7f072406f500 msgr2=0x7f07240d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 shutdown_connections 2024-04-25T08:32:35.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.481+0000 7f072c59d640 1 -- 172.21.15.60:0/1778651232 wait complete. 2024-04-25T08:32:35.488 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'bb5b1f2e-e235-4a9e-8ca6-ee83a15dee41', 'mode': 'asynchronous'}' 2024-04-25T08:32:35.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:35.489 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T08:32:35.505 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T08:32:35.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:35.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.650+0000 7fbffc658640 1 Processor -- start 2024-04-25T08:32:35.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.650+0000 7fbffc658640 1 -- start start 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffc658640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbff415dbe0 0x7fbff40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffc658640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40b6220 con 0x7fbff40b92a0 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffc658640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40b97a0 con 0x7fbff40b2cb0 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffc658640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff415dfc0 con 0x7fbff415dbe0 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffabf5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbff40b92a0 0x7fbff40b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55850/0 (socket says 172.21.15.60:55850) 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffabf5640 1 -- 172.21.15.60:0/464002389 learned_addr learned my addr 172.21.15.60:0/464002389 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.651+0000 7fbffa3f4640 1 --2- 172.21.15.60:0/464002389 >> v2:172.21.15.100:3300/0 conn(0x7fbff415dbe0 0x7fbff40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.652+0000 7fbff93f2640 1 -- 172.21.15.60:0/464002389 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516398554 0 0) 0x7fbff40b6220 con 0x7fbff40b92a0 2024-04-25T08:32:35.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.652+0000 7fbff93f2640 1 -- 172.21.15.60:0/464002389 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe0003520 con 0x7fbff40b92a0 2024-04-25T08:32:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.652+0000 7fbff93f2640 1 -- 172.21.15.60:0/464002389 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397966431 0 0) 0x7fbfe0003520 con 0x7fbff40b92a0 2024-04-25T08:32:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.652+0000 7fbffa3f4640 1 -- 172.21.15.60:0/464002389 >> v1:172.21.15.100:6789/0 conn(0x7fbff40b2cb0 legacy=0x7fbff40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.652+0000 7fbffa3f4640 1 -- 172.21.15.60:0/464002389 >> v1:172.21.15.60:6789/0 conn(0x7fbff40b92a0 legacy=0x7fbff40b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.652+0000 7fbffa3f4640 1 -- 172.21.15.60:0/464002389 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40d1c50 con 0x7fbff415dbe0 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffa3f4640 1 --2- 172.21.15.60:0/464002389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff415dbe0 0x7fbff40d0910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbff00097a0 tx=0x7fbff002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445988dd12ebf397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbff93f2640 1 -- 172.21.15.60:0/464002389 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0037070 con 0x7fbff415dbe0 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbff93f2640 1 -- 172.21.15.60:0/464002389 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff002aad0 con 0x7fbff415dbe0 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbff93f2640 1 -- 172.21.15.60:0/464002389 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0019f0 con 0x7fbff415dbe0 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffc658640 1 -- 172.21.15.60:0/464002389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff415dbe0 msgr2=0x7fbff40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffc658640 1 --2- 172.21.15.60:0/464002389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff415dbe0 0x7fbff40d0910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbff00097a0 tx=0x7fbff002a040 comp rx=0 tx=0).stop 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffc658640 1 -- 172.21.15.60:0/464002389 shutdown_connections 2024-04-25T08:32:35.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffc658640 1 --2- 172.21.15.60:0/464002389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff415dbe0 0x7fbff40d0910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffc658640 1 -- 172.21.15.60:0/464002389 >> 172.21.15.60:0/464002389 conn(0x7fbff4061a20 msgr2=0x7fbff4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.653+0000 7fbffc658640 1 -- 172.21.15.60:0/464002389 shutdown_connections 2024-04-25T08:32:35.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.654+0000 7fbffc658640 1 -- 172.21.15.60:0/464002389 wait complete. 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 Processor -- start 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 -- start start 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 0x7fbff40c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff40b92a0 0x7fbff40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff415dbe0 0x7fbff41e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40d2210 con 0x7fbff40b2cb0 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40d1a00 con 0x7fbff40b92a0 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.655+0000 7fbffc658640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff40b6ab0 con 0x7fbff415dbe0 2024-04-25T08:32:35.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 0x7fbff40c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 0x7fbff40c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49716/0 (socket says 172.21.15.60:49716) 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 -- 172.21.15.60:0/2423219481 learned_addr learned my addr 172.21.15.60:0/2423219481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffabf5640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff415dbe0 0x7fbff41e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbff9bf3640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff40b92a0 0x7fbff40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 -- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff415dbe0 msgr2=0x7fbff41e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff415dbe0 0x7fbff41e60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 -- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff40b92a0 msgr2=0x7fbff40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff40b92a0 0x7fbff40cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 -- 172.21.15.60:0/2423219481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40cd0a0 con 0x7fbff40b2cb0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffa3f4640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 0x7fbff40c9810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbff0035040 tx=0x7fbff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0037040 con 0x7fbff40b2cb0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0004030 con 0x7fbff40b2cb0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff40cd380 con 0x7fbff40b2cb0 2024-04-25T08:32:35.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004400 con 0x7fbff40b2cb0 2024-04-25T08:32:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.656+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff4154280 con 0x7fbff40b2cb0 2024-04-25T08:32:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.657+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff40b3d40 con 0x7fbff40b2cb0 2024-04-25T08:32:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.658+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbff003a040 con 0x7fbff40b2cb0 2024-04-25T08:32:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.658+0000 7fbfeaffd640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbfd806b4d0 0x7fbfd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.659+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbff00aa650 con 0x7fbff40b2cb0 2024-04-25T08:32:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.659+0000 7fbff9bf3640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbfd806b4d0 0x7fbfd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.659+0000 7fbff9bf3640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbfd806b4d0 0x7fbfd806d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbff41f7440 tx=0x7fbfe4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.663+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbff0042020 con 0x7fbff40b2cb0 2024-04-25T08:32:35.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.878+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbff4073990 con 0x7fbff40b2cb0 2024-04-25T08:32:35.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.878+0000 7fbfeaffd640 1 -- 172.21.15.60:0/2423219481 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fbff00793e0 con 0x7fbff40b2cb0 2024-04-25T08:32:35.875 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:35.875 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:35.021589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:35.875 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-04-25T08:32:35.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbfd806b4d0 msgr2=0x7fbfd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbfd806b4d0 0x7fbfd806d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbff41f7440 tx=0x7fbfe4009040 comp rx=0 tx=0).stop 2024-04-25T08:32:35.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 msgr2=0x7fbff40c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 0x7fbff40c9810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbff0035040 tx=0x7fbff00046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 shutdown_connections 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff415dbe0 0x7fbff41e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbfd806b4d0 0x7fbfd806d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff40b92a0 0x7fbff40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 --2- 172.21.15.60:0/2423219481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff40b2cb0 0x7fbff40c9810 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 >> 172.21.15.60:0/2423219481 conn(0x7fbff4061a20 msgr2=0x7fbff415ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 shutdown_connections 2024-04-25T08:32:35.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:35.880+0000 7fbffc658640 1 -- 172.21.15.60:0/2423219481 wait complete. 2024-04-25T08:32:35.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T08:32:35.997 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:35.997 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:35.998 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:36.006 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:36.007 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:36.007 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:36.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:36.024 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T08:32:36.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:32:36.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:32:36.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787827 s, 133 MB/s 2024-04-25T08:32:36.088 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T08:32:36.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.232+0000 7fcac3a93640 1 Processor -- start 2024-04-25T08:32:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.232+0000 7fcac3a93640 1 -- start start 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac3a93640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcabc15db70 0x7fcabc162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac3a93640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcabc0b6cb0 0x7fcabc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac3a93640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc162690 con 0x7fcabc0b5a60 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac3a93640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcabc0d3800 con 0x7fcabc15db70 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac3a93640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcabc0ceee0 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac182f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcabc15db70 0x7fcabc162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac182f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcabc15db70 0x7fcabc162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49726/0 (socket says 172.21.15.60:49726) 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.233+0000 7fcac182f640 1 -- 172.21.15.60:0/1985288264 learned_addr learned my addr 172.21.15.60:0/1985288264 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac182f640 1 -- 172.21.15.60:0/1985288264 >> v2:172.21.15.100:3300/0 conn(0x7fcabc0b6cb0 msgr2=0x7fcabc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac182f640 1 --2- 172.21.15.60:0/1985288264 >> v2:172.21.15.100:3300/0 conn(0x7fcabc0b6cb0 0x7fcabc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac182f640 1 -- 172.21.15.60:0/1985288264 >> v1:172.21.15.100:6789/0 conn(0x7fcabc0b5a60 legacy=0x7fcabc06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac182f640 1 -- 172.21.15.60:0/1985288264 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0d1d10 con 0x7fcabc15db70 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac182f640 1 --2- 172.21.15.60:0/1985288264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc15db70 0x7fcabc162150 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0d3980 tx=0x7fcab802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19a0b4cca52c0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac082d640 1 -- 172.21.15.60:0/1985288264 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8037070 con 0x7fcabc15db70 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac082d640 1 -- 172.21.15.60:0/1985288264 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8032440 con 0x7fcabc15db70 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.234+0000 7fcac082d640 1 -- 172.21.15.60:0/1985288264 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8032810 con 0x7fcabc15db70 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 -- 172.21.15.60:0/1985288264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc15db70 msgr2=0x7fcabc162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 --2- 172.21.15.60:0/1985288264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc15db70 0x7fcabc162150 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0d3980 tx=0x7fcab802a180 comp rx=0 tx=0).stop 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 -- 172.21.15.60:0/1985288264 shutdown_connections 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 --2- 172.21.15.60:0/1985288264 >> v2:172.21.15.100:3300/0 conn(0x7fcabc0b6cb0 0x7fcabc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 --2- 172.21.15.60:0/1985288264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc15db70 0x7fcabc162150 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 -- 172.21.15.60:0/1985288264 >> 172.21.15.60:0/1985288264 conn(0x7fcabc0d0520 msgr2=0x7fcabc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 -- 172.21.15.60:0/1985288264 shutdown_connections 2024-04-25T08:32:36.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.235+0000 7fcac3a93640 1 -- 172.21.15.60:0/1985288264 wait complete. 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.236+0000 7fcac3a93640 1 Processor -- start 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.236+0000 7fcac3a93640 1 -- start start 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac3a93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcabc0b5a60 0x7fcabc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac3a93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 0x7fcabc0aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac3a93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcabc15db70 0x7fcabc15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac3a93640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc15df50 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac3a93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0d22d0 con 0x7fcabc0b5a60 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac3a93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0ce530 con 0x7fcabc15db70 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 0x7fcabc0aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 0x7fcabc0aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49728/0 (socket says 172.21.15.60:49728) 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 -- 172.21.15.60:0/2368507430 learned_addr learned my addr 172.21.15.60:0/2368507430 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac182f640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcabc0b5a60 0x7fcabc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac2030640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcabc15db70 0x7fcabc15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 -- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcabc15db70 msgr2=0x7fcabc15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcabc15db70 0x7fcabc15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 -- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcabc0b5a60 msgr2=0x7fcabc0c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcabc0b5a60 0x7fcabc0c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac102e640 1 -- 172.21.15.60:0/2368507430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0aadd0 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.237+0000 7fcac182f640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcabc0b5a60 0x7fcabc0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.238+0000 7fcac102e640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 0x7fcabc0aa350 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b450 tx=0x7fcaac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.238+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac015020 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.238+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac011830 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.238+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc0cbc70 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.238+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac011c20 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.238+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc0cc230 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.239+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc073990 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.239+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcaac018020 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.239+0000 7fcab27fc640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaa006b4d0 0x7fcaa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.240+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcaac08a280 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.240+0000 7fcac182f640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaa006b4d0 0x7fcaa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.240+0000 7fcac182f640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaa006b4d0 0x7fcaa006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcab8034000 tx=0x7fcab802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.244+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcaac01f020 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.464+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcabc0d1820 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.464+0000 7fcab27fc640 1 -- 172.21.15.60:0/2368507430 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcaac0172c0 con 0x7fcabc0b6cb0 2024-04-25T08:32:36.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:36.461 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:35.021589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:36.461 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaa006b4d0 msgr2=0x7fcaa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaa006b4d0 0x7fcaa006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcab8034000 tx=0x7fcab802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 msgr2=0x7fcabc0aa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 0x7fcabc0aa350 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b450 tx=0x7fcaac00b910 comp rx=0 tx=0).stop 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 shutdown_connections 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcabc15db70 0x7fcabc15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcabc0b6cb0 0x7fcabc0aa350 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcaa006b4d0 0x7fcaa006d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 --2- 172.21.15.60:0/2368507430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcabc0b5a60 0x7fcabc0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 >> 172.21.15.60:0/2368507430 conn(0x7fcabc0d0520 msgr2=0x7fcabc0d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 shutdown_connections 2024-04-25T08:32:36.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.467+0000 7fcac3a93640 1 -- 172.21.15.60:0/2368507430 wait complete. 2024-04-25T08:32:36.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T08:32:36.584 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:36.584 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:36.584 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:36.593 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:36.594 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:32:36.594 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:32:36.594 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T08:32:36.594 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:32:36.594 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T08:32:36.594 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:32:36.660 INFO:teuthology.orchestra.run.smithi060.stdout:1099511630188 2024-04-25T08:32:36.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:36.669 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:36.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:32:36.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.816+0000 7f0acdf7e640 1 Processor -- start 2024-04-25T08:32:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.816+0000 7f0acdf7e640 1 -- start start 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0acdf7e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0ac80ced80 0x7f0ac80d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0acdf7e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac815e000 con 0x7f0ac815dc20 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0acdf7e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac80d1de0 con 0x7f0ac8161710 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0acdf7e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac8161c60 con 0x7f0ac80ced80 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0ac77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0ac815dc20 0x7f0ac80ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55866/0 (socket says 172.21.15.60:55866) 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0ac77fe640 1 -- 172.21.15.60:0/3466039902 learned_addr learned my addr 172.21.15.60:0/3466039902 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3466039902 >> v2:172.21.15.100:3300/0 conn(0x7f0ac80ced80 0x7f0ac80d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998730216 0 0) 0x7f0ac815e000 con 0x7f0ac815dc20 2024-04-25T08:32:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.817+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab0003520 con 0x7f0ac815dc20 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 97078275 0 0) 0x7f0ab0003520 con 0x7f0ac815dc20 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 >> v2:172.21.15.100:3300/0 conn(0x7f0ac80ced80 msgr2=0x7f0ac80d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 --2- 172.21.15.60:0/3466039902 >> v2:172.21.15.100:3300/0 conn(0x7f0ac80ced80 0x7f0ac80d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 >> v1:172.21.15.100:6789/0 conn(0x7f0ac8161710 legacy=0x7f0ac80d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8161f40 con 0x7f0ac815dc20 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f0ab8001670 con 0x7f0ac815dc20 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3466039902 >> v2:172.21.15.100:3300/0 conn(0x7f0ac80ced80 0x7f0ac80d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 >> v1:172.21.15.60:6789/0 conn(0x7f0ac815dc20 legacy=0x7f0ac80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab00039b0 0x7f0ab0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab0004800 0x7f0ab0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab00051c0 0x7f0ab000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8161c60 con 0x7f0ab00051c0 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab000c060 con 0x7f0ab00039b0 2024-04-25T08:32:36.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab000c260 con 0x7f0ab0004800 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab00051c0 0x7f0ab000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab00039b0 0x7f0ab0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab0004800 0x7f0ab0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac7fff640 1 -- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab0004800 msgr2=0x7f0ab0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab0004800 0x7f0ab0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac7fff640 1 -- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab00039b0 msgr2=0x7f0ab0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab00039b0 0x7f0ab0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac7fff640 1 -- 172.21.15.60:0/3466039902 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0003520 con 0x7f0ab00051c0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.818+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ab00039b0 0x7f0ab0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.819+0000 7f0ac7fff640 1 -- 172.21.15.60:0/3466039902 reap_dead start 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.819+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab00051c0 0x7f0ab000b9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0001e50 tx=0x7f0ab0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92258f137746777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.819+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8007d10 con 0x7f0ab00051c0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.819+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0003970 con 0x7f0ab00051c0 2024-04-25T08:32:36.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.819+0000 7f0ac67fc640 1 -- 172.21.15.60:0/3466039902 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0001a70 con 0x7f0ab00051c0 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 -- 172.21.15.60:0/3466039902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab00051c0 msgr2=0x7f0ab000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab00051c0 0x7f0ab000b9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0001e50 tx=0x7f0ab0005830 comp rx=0 tx=0).stop 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 -- 172.21.15.60:0/3466039902 shutdown_connections 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/3466039902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab00051c0 0x7f0ab000b9c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 -- 172.21.15.60:0/3466039902 >> 172.21.15.60:0/3466039902 conn(0x7f0ac80d0420 msgr2=0x7f0ac80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 -- 172.21.15.60:0/3466039902 shutdown_connections 2024-04-25T08:32:36.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.820+0000 7f0acdf7e640 1 -- 172.21.15.60:0/3466039902 wait complete. 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.821+0000 7f0acdf7e640 1 Processor -- start 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.821+0000 7f0acdf7e640 1 -- start start 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0acdf7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab00051c0 0x7f0ac815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0acdf7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 0x7f0ac80ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0acdf7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac80cd2f0 0x7f0ac80d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0acdf7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80b5e50 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0acdf7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8162500 con 0x7f0ac80cd2f0 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0acdf7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80b5fd0 con 0x7f0ab00051c0 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 0x7f0ac80ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 0x7f0ac80ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49750/0 (socket says 172.21.15.60:49750) 2024-04-25T08:32:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2916109834 learned_addr learned my addr 172.21.15.60:0/2916109834 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac80cd2f0 0x7f0ac80d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab00051c0 0x7f0ac815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab00051c0 msgr2=0x7f0ac815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab00051c0 0x7f0ac815ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac80cd2f0 msgr2=0x7f0ac80d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac80cd2f0 0x7f0ac80d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2916109834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80cd920 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac80cd2f0 0x7f0ac80d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab00051c0 0x7f0ac815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.822+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 0x7f0ac80ccdb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0abc011a90 tx=0x7f0abc011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.823+0000 7f0aaffff640 1 -- 172.21.15.60:0/2916109834 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc00fc10 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.823+0000 7f0aaffff640 1 -- 172.21.15.60:0/2916109834 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc00fdb0 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.823+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac80aa5d0 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.823+0000 7f0aaffff640 1 -- 172.21.15.60:0/2916109834 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc004260 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.823+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac80aab90 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.825+0000 7f0aaffff640 1 -- 172.21.15.60:0/2916109834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0abc012040 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.825+0000 7f0aaffff640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.825+0000 7f0aaffff640 1 -- 172.21.15.60:0/2916109834 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0abc08a830 con 0x7f0ac80ab0b0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.825+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.826+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ab406b4d0 0x7f0ab406d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8007cb0 tx=0x7f0ab8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.826+0000 7f0ad0c82740 1 Processor -- start 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.826+0000 7f0ad0c82740 1 -- start start 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593e80 0x55eaaf5967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf566700 0x55eaaf569f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf4cfe00 con 0x55eaaf593aa0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf478d20 con 0x55eaaf566700 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf4cbea0 con 0x55eaaf593e80 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49760/0 (socket says 172.21.15.60:49760) 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 -- 172.21.15.60:0/3491399708 learned_addr learned my addr 172.21.15.60:0/3491399708 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593e80 0x55eaaf5967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf566700 0x55eaaf569f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 -- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593e80 msgr2=0x55eaaf5967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593e80 0x55eaaf5967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 -- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf566700 msgr2=0x55eaaf569f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf566700 0x55eaaf569f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 -- 172.21.15.60:0/3491399708 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaaf49adf0 con 0x55eaaf593aa0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf566700 0x55eaaf569f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5962b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8001e90 tx=0x7f0ab8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf867232a94fed64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0aad7fa640 1 -- 172.21.15.60:0/3491399708 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8021450 con 0x55eaaf593aa0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0aad7fa640 1 -- 172.21.15.60:0/3491399708 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8026070 con 0x55eaaf593aa0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.827+0000 7f0aad7fa640 1 -- 172.21.15.60:0/3491399708 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80217f0 con 0x55eaaf593aa0 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3491399708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 msgr2=0x55eaaf5962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5962b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8001e90 tx=0x7f0ab8014000 comp rx=0 tx=0).stop 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3491399708 shutdown_connections 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf566700 0x55eaaf569f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593e80 0x55eaaf5967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3491399708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5962b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3491399708 >> 172.21.15.60:0/3491399708 conn(0x55eaaf568970 msgr2=0x55eaaf5b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3491399708 shutdown_connections 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.828+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3491399708 wait complete. 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.829+0000 7f0ad0c82740 1 -- start start 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.829+0000 7f0ad0c82740 1 Processor -- start 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.829+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593aa0 0x55eaaf5973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.829+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf5917d0 0x55eaaf59a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf56bb20 con 0x55eaaf593e80 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf46e590 con 0x55eaaf5917d0 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf56be00 con 0x55eaaf593aa0 2024-04-25T08:32:36.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 10 client.0 _ll_register_callbacks cb 0x55eaaf47ae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49774/0 (socket says 172.21.15.60:49774) 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2020578308 learned_addr learned my addr 172.21.15.60:0/2020578308 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593aa0 0x55eaaf5973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf5917d0 0x55eaaf59a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593aa0 msgr2=0x55eaaf5973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593aa0 0x55eaaf5973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf5917d0 msgr2=0x55eaaf59a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf5917d0 0x55eaaf59a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/2020578308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaaf1e43f0 con 0x55eaaf593e80 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599dd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0abc088c60 tx=0x7f0abc0596f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c3ce96675479d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0aa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0aa67fc640 1 -- 172.21.15.60:0/2020578308 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc070020 con 0x55eaaf593e80 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0aa67fc640 1 -- 172.21.15.60:0/2020578308 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0abc0571b0 con 0x55eaaf593e80 2024-04-25T08:32:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ab8026070 con 0x55eaaf593e80 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0aa67fc640 1 -- 172.21.15.60:0/2020578308 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0abc022040 con 0x55eaaf593e80 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 10 client.6399 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.830+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55eaaf2ae7d0 con 0x55eaaf593e80 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0aa67fc640 1 -- 172.21.15.60:0/2020578308 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8026070 con 0x55eaaf593e80 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0aa67fc640 1 -- 172.21.15.60:0/2020578308 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0abc059110 con 0x55eaaf593e80 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ad0c82740 10 client.6399 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ad0c82740 10 client.6399 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ad0c82740 20 client.6399 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf57fda0 0x55eaaf599330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ad0c82740 4 client.6399 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaaf4dd340 con 0x55eaaf57fda0 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf57fda0 0x55eaaf599330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf57fda0 0x55eaaf599330 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8807a52f3fbd90 server_cookie=948acbdef6746f5f in_seq=0 out_seq=0 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.831+0000 7f0aa67fc640 10 client.6399 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0aa67fc640 1 -- 172.21.15.60:0/2020578308 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eaaf4dd340 con 0x55eaaf57fda0 2024-04-25T08:32:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0aa67fc640 10 client.6399 handle_command_reply: tid=0 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 client.6399 shutdown 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 msgr2=0x55eaaf599dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599dd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0abc088c60 tx=0x7f0abc0596f0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 shutdown_connections 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf5917d0 0x55eaaf59a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599dd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf57fda0 0x55eaaf599330 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/2020578308 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf593aa0 0x55eaaf5973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0aa67fc640 0 client.6399 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 >> 172.21.15.60:0/2020578308 conn(0x55eaaf566700 msgr2=0x55eaaf5673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 shutdown_connections 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 1 -- 172.21.15.60:0/2020578308 wait complete. 2024-04-25T08:32:36.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.832+0000 7f0ad0c82740 20 client.6399 trim_cache size 0 max 16384 2024-04-25T08:32:36.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 Processor -- start 2024-04-25T08:32:36.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 -- start start 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf567290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593aa0 0x55eaaf599a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8200bd0 con 0x55eaaf593e80 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81ffda0 con 0x55eaaf593aa0 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.844+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81f4170 con 0x55eaaf5917d0 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49784/0 (socket says 172.21.15.60:49784) 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 -- 172.21.15.60:0/4134817508 learned_addr learned my addr 172.21.15.60:0/4134817508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 -- 172.21.15.60:0/4134817508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 msgr2=0x55eaaf567290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf567290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 -- 172.21.15.60:0/4134817508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593aa0 msgr2=0x55eaaf599a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593aa0 0x55eaaf599a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 -- 172.21.15.60:0/4134817508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaaf562690 con 0x55eaaf593e80 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599f50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0001f40 tx=0x7f0ac002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997d89c89bce1817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0aacff9640 1 -- 172.21.15.60:0/4134817508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0003b70 con 0x55eaaf593e80 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0aacff9640 1 -- 172.21.15.60:0/4134817508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0003d10 con 0x55eaaf593e80 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.845+0000 7f0aacff9640 1 -- 172.21.15.60:0/4134817508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00052f0 con 0x55eaaf593e80 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 -- 172.21.15.60:0/4134817508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 msgr2=0x55eaaf599f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599f50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0001f40 tx=0x7f0ac002c000 comp rx=0 tx=0).stop 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 -- 172.21.15.60:0/4134817508 shutdown_connections 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593e80 0x55eaaf599f50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593aa0 0x55eaaf599a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/4134817508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf567290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 -- 172.21.15.60:0/4134817508 >> 172.21.15.60:0/4134817508 conn(0x55eaaf566700 msgr2=0x55eaaf5656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 -- 172.21.15.60:0/4134817508 shutdown_connections 2024-04-25T08:32:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 -- 172.21.15.60:0/4134817508 wait complete. 2024-04-25T08:32:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 -- start start 2024-04-25T08:32:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.846+0000 7f0ad0c82740 1 Processor -- start 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf562020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593e80 0x55eaaf57e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf5ae5b0 con 0x55eaaf593aa0 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81ff7a0 con 0x55eaaf593e80 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eaaf598c20 con 0x55eaaf5917d0 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ad0c82740 10 client.0 _ll_register_callbacks cb 0x7f0ab8001630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49794/0 (socket says 172.21.15.60:49794) 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/3852480783 learned_addr learned my addr 172.21.15.60:0/3852480783 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ac7fff640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593e80 0x55eaaf57e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.847+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf562020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 msgr2=0x55eaaf562020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf562020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593e80 msgr2=0x55eaaf57e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593e80 0x55eaaf57e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac6ffd640 1 -- 172.21.15.60:0/3852480783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaaf5d49a0 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf562020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac6ffd640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5d4270 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0abc088cb0 tx=0x7f0abc009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11972a9170a524a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0aa67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0aa67fc640 1 -- 172.21.15.60:0/3852480783 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc070040 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0aa67fc640 1 -- 172.21.15.60:0/3852480783 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0abc058500 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ac0003d10 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0aa67fc640 1 -- 172.21.15.60:0/3852480783 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0abc07e050 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 10 client.6405 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55eaaf601630 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0aa67fc640 1 -- 172.21.15.60:0/3852480783 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0003d10 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0aa67fc640 1 -- 172.21.15.60:0/3852480783 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0abc081020 con 0x55eaaf593aa0 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 10 client.6405 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 10 client.6405 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 20 client.6405 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf599a60 0x55eaaf599e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 4 client.6405 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eaaf4d7690 con 0x55eaaf599a60 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.848+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf599a60 0x55eaaf599e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0aa67fc640 10 client.6405 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ac77fe640 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf599a60 0x55eaaf599e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c205a7df7a8d3c9d server_cookie=a5b860e840e5723d in_seq=0 out_seq=0 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0aa67fc640 1 -- 172.21.15.60:0/3852480783 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0ab8026020 con 0x55eaaf599a60 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0aa67fc640 10 client.6405 handle_command_reply: tid=0 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 client.6405 shutdown 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 msgr2=0x55eaaf5d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5d4270 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0abc088cb0 tx=0x7f0abc009390 comp rx=0 tx=0).stop 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 shutdown_connections 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eaaf593e80 0x55eaaf57e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eaaf593aa0 0x55eaaf5d4270 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55eaaf599a60 0x55eaaf599e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 --2- 172.21.15.60:0/3852480783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55eaaf5917d0 0x55eaaf562020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0aa67fc640 0 client.6405 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 >> 172.21.15.60:0/3852480783 conn(0x55eaaf566700 msgr2=0x55eaaf5b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 shutdown_connections 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 1 -- 172.21.15.60:0/3852480783 wait complete. 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.849+0000 7f0ad0c82740 20 client.6405 trim_cache size 0 max 16384 2024-04-25T08:32:36.851 INFO:teuthology.orchestra.run.smithi060.stdout:[] 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ab406b4d0 msgr2=0x7f0ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ab406b4d0 0x7f0ab406d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8007cb0 tx=0x7f0ab8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 msgr2=0x7f0ac80ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 0x7f0ac80ccdb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0abc011a90 tx=0x7f0abc011f50 comp rx=0 tx=0).stop 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 shutdown_connections 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ac80cd2f0 0x7f0ac80d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ac80ab0b0 0x7f0ac80ccdb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 --2- 172.21.15.60:0/2916109834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ab00051c0 0x7f0ac815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 >> 172.21.15.60:0/2916109834 conn(0x7f0ac80d0420 msgr2=0x7f0ac80b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 shutdown_connections 2024-04-25T08:32:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.850+0000 7f0acdf7e640 1 -- 172.21.15.60:0/2916109834 wait complete. 2024-04-25T08:32:36.857 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096c.00000000 parent 2024-04-25T08:32:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.903+0000 7fc6f131af40 1 Processor -- start 2024-04-25T08:32:36.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.903+0000 7fc6f131af40 1 -- start start 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6f131af40 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33f9cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6f131af40 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33fa55ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6f131af40 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33fa563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6f131af40 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f928df0 con 0x55d33f9ab610 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6f131af40 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f92cd50 con 0x55d33f97ffb0 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6f131af40 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d33fa5b0f0 con 0x55d33f9ab9f0 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33fa55ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33fa55ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49804/0 (socket says 172.21.15.60:49804) 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 -- 172.21.15.60:0/980296881 learned_addr learned my addr 172.21.15.60:0/980296881 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6ee3d7640 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33f9cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6eebd8640 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33fa563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 -- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 msgr2=0x55d33fa563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33fa563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 -- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 msgr2=0x55d33f9cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33f9cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 -- 172.21.15.60:0/980296881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33f97f8e0 con 0x55d33f9ab610 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6ee3d7640 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33f9cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.904+0000 7fc6edbd6640 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33fa55ea0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4001e90 tx=0x7fc6e400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c53a13cdbb02786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6ecbd4640 1 -- 172.21.15.60:0/980296881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4016070 con 0x55d33f9ab610 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6ecbd4640 1 -- 172.21.15.60:0/980296881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e400d6b0 con 0x55d33f9ab610 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6ecbd4640 1 -- 172.21.15.60:0/980296881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e400daa0 con 0x55d33f9ab610 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 -- 172.21.15.60:0/980296881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 msgr2=0x55d33fa55ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33fa55ea0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4001e90 tx=0x7fc6e400b650 comp rx=0 tx=0).stop 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 -- 172.21.15.60:0/980296881 shutdown_connections 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33fa563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33fa55ea0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 --2- 172.21.15.60:0/980296881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33f9cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 -- 172.21.15.60:0/980296881 >> 172.21.15.60:0/980296881 conn(0x55d33f9809b0 msgr2=0x55d33f981d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 -- 172.21.15.60:0/980296881 shutdown_connections 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.905+0000 7fc6f131af40 1 -- 172.21.15.60:0/980296881 wait complete. 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 Processor -- start 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 -- start start 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33fa4f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33faf2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33faf3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33f97f600 con 0x55d33f9ab610 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33fa58ba0 con 0x55d33f97ffb0 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6f131af40 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d33fa578b0 con 0x55d33f9ab9f0 2024-04-25T08:32:36.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6edbd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33faf2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6edbd6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33faf2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49820/0 (socket says 172.21.15.60:49820) 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.906+0000 7fc6edbd6640 1 -- 172.21.15.60:0/77989386 learned_addr learned my addr 172.21.15.60:0/77989386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6eebd8640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33faf3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6ee3d7640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33fa4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6edbd6640 1 -- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 msgr2=0x55d33faf3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6edbd6640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d33f9ab9f0 0x55d33faf3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6edbd6640 1 -- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 msgr2=0x55d33fa4f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6edbd6640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33fa4f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6edbd6640 1 -- 172.21.15.60:0/77989386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33fae93b0 con 0x55d33f9ab610 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6ee3d7640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d33f97ffb0 0x55d33fa4f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6edbd6640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33faf2de0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b690 tx=0x7fc6e4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6df7fe640 1 -- 172.21.15.60:0/77989386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4021020 con 0x55d33f9ab610 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6df7fe640 1 -- 172.21.15.60:0/77989386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e400be80 con 0x55d33f9ab610 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6df7fe640 1 -- 172.21.15.60:0/77989386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4016030 con 0x55d33f9ab610 2024-04-25T08:32:36.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d33f9c30c0 con 0x55d33f9ab610 2024-04-25T08:32:36.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.907+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d33fa509c0 con 0x55d33f9ab610 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.908+0000 7fc6df7fe640 1 -- 172.21.15.60:0/77989386 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6e4024020 con 0x55d33f9ab610 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.909+0000 7fc6df7fe640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6d0069eb0 0x7fc6d006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.909+0000 7fc6df7fe640 1 -- 172.21.15.60:0/77989386 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6e408a120 con 0x55d33f9ab610 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.909+0000 7fc6f131af40 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x55d33fa4f9c0 0x55d33fae9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.909+0000 7fc6ee3d7640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6d0069eb0 0x7fc6d006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.909+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 --> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] -- osd_op(unknown.0.0:1 8.2b 8:d716f75e:::1000000096c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v9 -- 0x55d33f980550 con 0x55d33fa4f9c0 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.909+0000 7fc6eebd8640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x55d33fa4f9c0 0x55d33fae9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.910+0000 7fc6ee3d7640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6d0069eb0 0x7fc6d006c370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55d33f984190 tx=0x7fc6e0000bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.910+0000 7fc6eebd8640 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x55d33fa4f9c0 0x55d33fae9890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6eebd8640 1 -- 172.21.15.60:0/77989386 <== osd.6 v2:172.21.15.100:6816/4212964990 1 ==== osd_op_reply(1 1000000096c.00000000 [rmxattr] v55'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc6e8057030 con 0x55d33fa4f9c0 2024-04-25T08:32:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x55d33fa4f9c0 msgr2=0x55d33fae9890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x55d33fa4f9c0 0x55d33fae9890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6d0069eb0 msgr2=0x7fc6d006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6d0069eb0 0x7fc6d006c370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55d33f984190 tx=0x7fc6e0000bc0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 msgr2=0x55d33faf2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 --2- 172.21.15.60:0/77989386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d33f9ab610 0x55d33faf2de0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b690 tx=0x7fc6e4000ed0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6eebd8640 1 -- 172.21.15.60:0/77989386 reap_dead start 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 shutdown_connections 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 >> 172.21.15.60:0/77989386 conn(0x55d33f9809b0 msgr2=0x55d33f984450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 shutdown_connections 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:36.911+0000 7fc6f131af40 1 -- 172.21.15.60:0/77989386 wait complete. 2024-04-25T08:32:36.911 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T08:32:36.911 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:36.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T08:32:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.058+0000 7f8e8edd5640 1 Processor -- start 2024-04-25T08:32:37.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8edd5640 1 -- start start 2024-04-25T08:32:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8edd5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e8815a9f0 0x7f8e880c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8edd5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e880c6940 con 0x7f8e880cd6f0 2024-04-25T08:32:37.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8edd5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8815add0 con 0x7f8e880cdbf0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8edd5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e880b0020 con 0x7f8e8815a9f0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8d372640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e880cd6f0 0x7f8e880cba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55876/0 (socket says 172.21.15.60:55876) 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.059+0000 7f8e8d372640 1 -- 172.21.15.60:0/2065510030 learned_addr learned my addr 172.21.15.60:0/2065510030 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/2065510030 >> v2:172.21.15.100:3300/0 conn(0x7f8e8815a9f0 0x7f8e880c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114595838 0 0) 0x7f8e80001490 con 0x7f8e880cd6f0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e70003520 con 0x7f8e880cd6f0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749103334 0 0) 0x7f8e8815add0 con 0x7f8e880cdbf0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e80001490 con 0x7f8e880cdbf0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2065510030 >> v1:172.21.15.100:6789/0 conn(0x7f8e880cdbf0 legacy=0x7f8e880ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2788207143 0 0) 0x7f8e70003520 con 0x7f8e880cd6f0 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2065510030 >> v1:172.21.15.60:6789/0 conn(0x7f8e880cd6f0 legacy=0x7f8e880cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.060+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2065510030 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880c6660 con 0x7f8e8815a9f0 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/2065510030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e880c6010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e780097a0 tx=0x7f8e7802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b9e618ec7b62762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78037070 con 0x7f8e8815a9f0 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7802ac50 con 0x7f8e8815a9f0 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e877fe640 1 -- 172.21.15.60:0/2065510030 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80001bc0 con 0x7f8e8815a9f0 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2065510030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 msgr2=0x7f8e880c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2065510030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e880c6010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e780097a0 tx=0x7f8e7802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2065510030 shutdown_connections 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.061+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2065510030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e880c6010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.062+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2065510030 >> 172.21.15.60:0/2065510030 conn(0x7f8e880b2390 msgr2=0x7f8e880b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.062+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2065510030 shutdown_connections 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.062+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2065510030 wait complete. 2024-04-25T08:32:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.063+0000 7f8e8edd5640 1 Processor -- start 2024-04-25T08:32:37.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.063+0000 7f8e8edd5640 1 -- start start 2024-04-25T08:32:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 0x7f8e881577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880cdbf0 0x7f8e880c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e88143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880cb230 con 0x7f8e880cd6f0 2024-04-25T08:32:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880cb5a0 con 0x7f8e8815a9f0 2024-04-25T08:32:37.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880ce070 con 0x7f8e880cdbf0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 0x7f8e881577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 0x7f8e881577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49828/0 (socket says 172.21.15.60:49828) 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2529898008 learned_addr learned my addr 172.21.15.60:0/2529898008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8d372640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e88143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e87fff640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880cdbf0 0x7f8e880c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880cdbf0 msgr2=0x7f8e880c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880cdbf0 0x7f8e880c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 msgr2=0x7f8e88143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e88143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 -- 172.21.15.60:0/2529898008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e88157d00 con 0x7f8e880cd6f0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 0x7f8e881577c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8e78035040 tx=0x7f8e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e857fa640 1 -- 172.21.15.60:0/2529898008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78041020 con 0x7f8e880cd6f0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e857fa640 1 -- 172.21.15.60:0/2529898008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7803d420 con 0x7f8e880cd6f0 2024-04-25T08:32:37.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e880c7d70 con 0x7f8e880cd6f0 2024-04-25T08:32:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e857fa640 1 -- 172.21.15.60:0/2529898008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78037040 con 0x7f8e880cd6f0 2024-04-25T08:32:37.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.064+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e88156ce0 con 0x7f8e880cd6f0 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.066+0000 7f8e857fa640 1 -- 172.21.15.60:0/2529898008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e78044020 con 0x7f8e880cd6f0 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.067+0000 7f8e857fa640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b500 0x7f8e7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.067+0000 7f8e857fa640 1 -- 172.21.15.60:0/2529898008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e780aa4f0 con 0x7f8e880cd6f0 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.067+0000 7f8e87fff640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b500 0x7f8e7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.067+0000 7f8e87fff640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b500 0x7f8e7406d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8e881f7060 tx=0x7f8e7c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.067+0000 7f8e91bbc740 1 Processor -- start 2024-04-25T08:32:37.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 -- start start 2024-04-25T08:32:37.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbad2360 0x55d2cbaf27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbad9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 0x55d2cbaaf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cba12ee0 con 0x55d2cbada140 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cb9bbc50 con 0x55d2cbab1360 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2cba0ef80 con 0x55d2cbad2360 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e8d372640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 0x55d2cbaaf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e8d372640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 0x55d2cbaaf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49834/0 (socket says 172.21.15.60:49834) 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e8d372640 1 -- 172.21.15.60:0/2192975785 learned_addr learned my addr 172.21.15.60:0/2192975785 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbad2360 0x55d2cbaf27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.068+0000 7f8e87fff640 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbad9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e8d372640 1 -- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbad2360 msgr2=0x55d2cbaf27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e8d372640 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbad2360 0x55d2cbaf27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e8d372640 1 -- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 msgr2=0x55d2cbad9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e8d372640 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbad9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e8d372640 1 -- 172.21.15.60:0/2192975785 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2cb7f3c00 con 0x55d2cbada140 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e87fff640 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbad9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:37.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e8d372640 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 0x55d2cbaaf7c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8e80001d30 tx=0x7f8e8000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a998f6324c36824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e6e7fc640 1 -- 172.21.15.60:0/2192975785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80019070 con 0x55d2cbada140 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e6e7fc640 1 -- 172.21.15.60:0/2192975785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e800072e0 con 0x55d2cbada140 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e6e7fc640 1 -- 172.21.15.60:0/2192975785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e800076b0 con 0x55d2cbada140 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.069+0000 7f8e91bbc740 1 -- 172.21.15.60:0/2192975785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 msgr2=0x55d2cbaaf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 0x55d2cbaaf7c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8e80001d30 tx=0x7f8e8000e6c0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 -- 172.21.15.60:0/2192975785 shutdown_connections 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbada140 0x55d2cbaaf7c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbad9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/2192975785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbad2360 0x55d2cbaf27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 -- 172.21.15.60:0/2192975785 >> 172.21.15.60:0/2192975785 conn(0x55d2cba76810 msgr2=0x55d2cbaf4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 -- 172.21.15.60:0/2192975785 shutdown_connections 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 -- 172.21.15.60:0/2192975785 wait complete. 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 -- start start 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.070+0000 7f8e91bbc740 1 Processor -- start 2024-04-25T08:32:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cbaaf330 con 0x55d2cbad2360 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cb9b1340 con 0x55d2cbab1360 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2cbaaa1a0 con 0x55d2cbada140 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e91bbc740 10 client.0 _ll_register_callbacks cb 0x55d2cb726fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49838/0 (socket says 172.21.15.60:49838) 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e87fff640 1 -- 172.21.15.60:0/188831370 learned_addr learned my addr 172.21.15.60:0/188831370 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e8d372640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e87fff640 1 -- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 msgr2=0x55d2cbaebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.071+0000 7f8e87fff640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e87fff640 1 -- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 msgr2=0x55d2cbaed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e87fff640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e87fff640 1 -- 172.21.15.60:0/188831370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2cb6e74f0 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e87fff640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad1440 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0011e0 tx=0x7f8e7c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4505405ea3764436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e677fe640 1 -- 172.21.15.60:0/188831370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c027060 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e677fe640 1 -- 172.21.15.60:0/188831370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e7c010040 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e800072e0 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e677fe640 1 -- 172.21.15.60:0/188831370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7c0039c0 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 10 client.6426 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d2cb854390 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e677fe640 1 -- 172.21.15.60:0/188831370 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e800072e0 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e677fe640 1 -- 172.21.15.60:0/188831370 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e7c003b60 con 0x55d2cbad2360 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 10 client.6426 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 10 client.6426 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 20 client.6426 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbad9470 0x55d2cbb04bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 4 client.6426 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.072+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2cba20420 con 0x55d2cbad9470 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbad9470 0x55d2cbb04bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e677fe640 10 client.6426 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbad9470 0x55d2cbb04bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50e2254d1aad69a server_cookie=a054ce1c3dfe99be in_seq=0 out_seq=0 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e677fe640 1 -- 172.21.15.60:0/188831370 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2cba20420 con 0x55d2cbad9470 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e677fe640 10 client.6426 handle_command_reply: tid=0 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e91bbc740 1 client.6426 shutdown 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 msgr2=0x55d2cbad1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.073+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad1440 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0011e0 tx=0x7f8e7c000ed0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 shutdown_connections 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbad9470 0x55d2cbb04bd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad1440 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e677fe640 0 client.6426 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/188831370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 >> 172.21.15.60:0/188831370 conn(0x55d2cba76810 msgr2=0x55d2cbaaeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 shutdown_connections 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 1 -- 172.21.15.60:0/188831370 wait complete. 2024-04-25T08:32:37.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.074+0000 7f8e91bbc740 20 client.6426 trim_cache size 0 max 16384 2024-04-25T08:32:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.089+0000 7f8e91bbc740 1 Processor -- start 2024-04-25T08:32:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.089+0000 7f8e91bbc740 1 -- start start 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.089+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbab1360 0x55d2cbb03b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.089+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbadba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.089+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbada140 0x55d2cbadbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e882004a0 con 0x55d2cbad2360 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e881f7270 con 0x55d2cbada140 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880c9630 con 0x55d2cbab1360 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbadba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbadba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49854/0 (socket says 172.21.15.60:49854) 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 -- 172.21.15.60:0/744125304 learned_addr learned my addr 172.21.15.60:0/744125304 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 -- 172.21.15.60:0/744125304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbab1360 msgr2=0x55d2cbb03b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbab1360 0x55d2cbb03b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 -- 172.21.15.60:0/744125304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbada140 msgr2=0x55d2cbadbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbada140 0x55d2cbadbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 -- 172.21.15.60:0/744125304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2cb6c0670 con 0x55d2cbad2360 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e87fff640 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbadba50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0053a0 tx=0x7f8e7c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44c50aacb59eaa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e6dffb640 1 -- 172.21.15.60:0/744125304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0134c0 con 0x55d2cbad2360 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e6dffb640 1 -- 172.21.15.60:0/744125304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c003b60 con 0x55d2cbad2360 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.090+0000 7f8e6dffb640 1 -- 172.21.15.60:0/744125304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c013720 con 0x55d2cbad2360 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 -- 172.21.15.60:0/744125304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 msgr2=0x55d2cbadba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbadba50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0053a0 tx=0x7f8e7c01f000 comp rx=0 tx=0).stop 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 -- 172.21.15.60:0/744125304 shutdown_connections 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbada140 0x55d2cbadbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbadba50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/744125304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbab1360 0x55d2cbb03b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 -- 172.21.15.60:0/744125304 >> 172.21.15.60:0/744125304 conn(0x55d2cba76810 msgr2=0x55d2cbaa51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 -- 172.21.15.60:0/744125304 shutdown_connections 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 -- 172.21.15.60:0/744125304 wait complete. 2024-04-25T08:32:37.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 -- start start 2024-04-25T08:32:37.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.091+0000 7f8e91bbc740 1 Processor -- start 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaadb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaaf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880c9c30 con 0x55d2cbad2360 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cba15380 con 0x55d2cbab1360 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2cbaae520 con 0x55d2cbada140 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e91bbc740 10 client.0 _ll_register_callbacks cb 0x7f8e800021b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:37.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49870/0 (socket says 172.21.15.60:49870) 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e87fff640 1 -- 172.21.15.60:0/3662888692 learned_addr learned my addr 172.21.15.60:0/3662888692 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaadb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.092+0000 7f8e8d372640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaaf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e87fff640 1 -- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 msgr2=0x55d2cbaaf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e87fff640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaaf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e87fff640 1 -- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 msgr2=0x55d2cbaadb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e87fff640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaadb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e87fff640 1 -- 172.21.15.60:0/3662888692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2cbacfb10 con 0x55d2cbad2360 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaadb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e87fff640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad84d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02f040 tx=0x7f8e7c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ce3fac80371d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e677fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e677fe640 1 -- 172.21.15.60:0/3662888692 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c03a070 con 0x55d2cbad2360 2024-04-25T08:32:37.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e677fe640 1 -- 172.21.15.60:0/3662888692 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e7c010040 con 0x55d2cbad2360 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e7c003b60 con 0x55d2cbad2360 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e677fe640 1 -- 172.21.15.60:0/3662888692 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7c013930 con 0x55d2cbad2360 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 10 client.6432 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55d2cbaddc40 con 0x55d2cbad2360 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e677fe640 1 -- 172.21.15.60:0/3662888692 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c003b60 con 0x55d2cbad2360 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e677fe640 1 -- 172.21.15.60:0/3662888692 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e7c013ad0 con 0x55d2cbad2360 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 10 client.6432 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 10 client.6432 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 20 client.6432 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.093+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbb03b70 0x55d2cbaf6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 4 client.6432 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x55d2cba1a770 con 0x55d2cbb03b70 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbb03b70 0x55d2cbaf6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e677fe640 10 client.6432 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e8cb71640 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbb03b70 0x55d2cbaf6910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8780ac8c03bca1b server_cookie=3465e577c9e175bc in_seq=0 out_seq=0 2024-04-25T08:32:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e677fe640 1 -- 172.21.15.60:0/3662888692 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d2cba1a770 con 0x55d2cbb03b70 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e677fe640 10 client.6432 handle_command_reply: tid=0 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 client.6432 shutdown 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 msgr2=0x55d2cbad84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad84d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02f040 tx=0x7f8e7c005000 comp rx=0 tx=0).stop 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 shutdown_connections 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55d2cbb03b70 0x55d2cbaf6910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55d2cbada140 0x55d2cbaaf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e677fe640 0 client.6432 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d2cbad2360 0x55d2cbad84d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 --2- 172.21.15.60:0/3662888692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2cbab1360 0x55d2cbaadb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.094+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 >> 172.21.15.60:0/3662888692 conn(0x55d2cba76810 msgr2=0x55d2cbad5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 shutdown_connections 2024-04-25T08:32:37.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e91bbc740 1 -- 172.21.15.60:0/3662888692 wait complete. 2024-04-25T08:32:37.097 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:32:37.095+0000 7fe78834d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096c [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v469 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5656=0-4194304@1} caps={5656=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5656 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x562dcf3fc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096c:[<0x10000000001/new_file_1 v469>,<0x10000000000/ceph-qa-suite v53>,<0x1/scrub_test_path v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:32:37.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e91bbc740 20 client.6432 trim_cache size 0 max 16384 2024-04-25T08:32:37.097 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:37.097 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:32:37.097 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "b3ebcc8c-8122-47c6-a2bb-c03b218a3fa4", 2024-04-25T08:32:37.097 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:32:37.097 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b500 msgr2=0x7f8e7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b500 0x7f8e7406d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8e881f7060 tx=0x7f8e7c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 msgr2=0x7f8e881577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 0x7f8e881577c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8e78035040 tx=0x7f8e780046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 shutdown_connections 2024-04-25T08:32:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b500 0x7f8e7406d9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e8815a9f0 0x7f8e88143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880cdbf0 0x7f8e880c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 --2- 172.21.15.60:0/2529898008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e880cd6f0 0x7f8e881577c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 >> 172.21.15.60:0/2529898008 conn(0x7f8e880b2390 msgr2=0x7f8e88159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 shutdown_connections 2024-04-25T08:32:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.095+0000 7f8e8edd5640 1 -- 172.21.15.60:0/2529898008 wait complete. 2024-04-25T08:32:37.102 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': 'b3ebcc8c-8122-47c6-a2bb-c03b218a3fa4', 'mode': 'asynchronous'}' 2024-04-25T08:32:37.102 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:32:37.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:32:37.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.208+0000 7fe528ad8640 1 Processor -- start 2024-04-25T08:32:37.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe528ad8640 1 -- start start 2024-04-25T08:32:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe528ad8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe52415dc20 0x7fe5240ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe528ad8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe5240d32f0 0x7fe5240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe528ad8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52415e000 con 0x7fe524161710 2024-04-25T08:32:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe528ad8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5240d1de0 con 0x7fe52415dc20 2024-04-25T08:32:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe528ad8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe524161c60 con 0x7fe5240d32f0 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe52259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe52415dc20 0x7fe5240ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe51bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe524161710 0x7fe52406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:55880/0 (socket says 172.21.15.60:55880) 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.209+0000 7fe51bfff640 1 -- 172.21.15.60:0/2213133927 learned_addr learned my addr 172.21.15.60:0/2213133927 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe52259c640 1 -- 172.21.15.60:0/2213133927 >> v2:172.21.15.100:3300/0 conn(0x7fe5240d32f0 msgr2=0x7fe5240cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe522d9d640 1 --2- 172.21.15.60:0/2213133927 >> v2:172.21.15.100:3300/0 conn(0x7fe5240d32f0 0x7fe5240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe52259c640 1 --2- 172.21.15.60:0/2213133927 >> v2:172.21.15.100:3300/0 conn(0x7fe5240d32f0 0x7fe5240cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe52259c640 1 -- 172.21.15.60:0/2213133927 >> v1:172.21.15.60:6789/0 conn(0x7fe524161710 legacy=0x7fe52406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe521d9b640 1 -- 172.21.15.60:0/2213133927 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603962225 0 0) 0x7fe52415e000 con 0x7fe524161710 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe52259c640 1 -- 172.21.15.60:0/2213133927 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524161f40 con 0x7fe52415dc20 2024-04-25T08:32:37.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.210+0000 7fe522d9d640 1 --2- 172.21.15.60:0/2213133927 >> v2:172.21.15.100:3300/0 conn(0x7fe5240d32f0 0x7fe5240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:37.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe52259c640 1 --2- 172.21.15.60:0/2213133927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52415dc20 0x7fe5240ce670 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe5240b6910 tx=0x7fe51402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ddacb57f7724e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe521d9b640 1 -- 172.21.15.60:0/2213133927 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514037070 con 0x7fe52415dc20 2024-04-25T08:32:37.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe521d9b640 1 -- 172.21.15.60:0/2213133927 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514032440 con 0x7fe52415dc20 2024-04-25T08:32:37.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe521d9b640 1 -- 172.21.15.60:0/2213133927 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514032810 con 0x7fe52415dc20 2024-04-25T08:32:37.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 -- 172.21.15.60:0/2213133927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52415dc20 msgr2=0x7fe5240ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 --2- 172.21.15.60:0/2213133927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52415dc20 0x7fe5240ce670 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe5240b6910 tx=0x7fe51402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 -- 172.21.15.60:0/2213133927 shutdown_connections 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 --2- 172.21.15.60:0/2213133927 >> v2:172.21.15.100:3300/0 conn(0x7fe5240d32f0 0x7fe5240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 --2- 172.21.15.60:0/2213133927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe52415dc20 0x7fe5240ce670 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 -- 172.21.15.60:0/2213133927 >> 172.21.15.60:0/2213133927 conn(0x7fe5240d0420 msgr2=0x7fe5240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 -- 172.21.15.60:0/2213133927 shutdown_connections 2024-04-25T08:32:37.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.211+0000 7fe528ad8640 1 -- 172.21.15.60:0/2213133927 wait complete. 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 Processor -- start 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 -- start start 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 0x7fe5240b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe52415dc20 0x7fe524156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe524161710 0x7fe524160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b5e50 con 0x7fe5240d32f0 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe524162500 con 0x7fe524161710 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe528ad8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b5fd0 con 0x7fe52415dc20 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe52259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 0x7fe5240b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.213+0000 7fe52259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 0x7fe5240b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49892/0 (socket says 172.21.15.60:49892) 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 -- 172.21.15.60:0/1465350275 learned_addr learned my addr 172.21.15.60:0/1465350275 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 -- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe52415dc20 msgr2=0x7fe524156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe52415dc20 0x7fe524156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 -- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe524161710 msgr2=0x7fe524160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe524161710 0x7fe524160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 -- 172.21.15.60:0/1465350275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240b17f0 con 0x7fe5240d32f0 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe52259c640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 0x7fe5240b48f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe514009890 tx=0x7fe5140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe51affd640 1 -- 172.21.15.60:0/1465350275 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514037040 con 0x7fe5240d32f0 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe51affd640 1 -- 172.21.15.60:0/1465350275 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514042070 con 0x7fe5240d32f0 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5240b1ad0 con 0x7fe5240d32f0 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe51affd640 1 -- 172.21.15.60:0/1465350275 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514030a70 con 0x7fe5240d32f0 2024-04-25T08:32:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.214+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5240b2090 con 0x7fe5240d32f0 2024-04-25T08:32:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.216+0000 7fe51affd640 1 -- 172.21.15.60:0/1465350275 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe514030c10 con 0x7fe5240d32f0 2024-04-25T08:32:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.216+0000 7fe51affd640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe50406b4d0 0x7fe50406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.216+0000 7fe51affd640 1 -- 172.21.15.60:0/1465350275 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5140aae60 con 0x7fe5240d32f0 2024-04-25T08:32:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.216+0000 7fe51bfff640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe50406b4d0 0x7fe50406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.217+0000 7fe51bfff640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe50406b4d0 0x7fe50406d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe50c007fd0 tx=0x7fe50c007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.217+0000 7fe52b7d6740 1 Processor -- start 2024-04-25T08:32:37.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.217+0000 7fe52b7d6740 1 -- start start 2024-04-25T08:32:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fcf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fcf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 0x55aeb7fa2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7f08da0 con 0x55aeb7f9f770 2024-04-25T08:32:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7eb1b80 con 0x55aeb7fccb10 2024-04-25T08:32:37.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7f04e40 con 0x55aeb7fccef0 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 0x55aeb7fa2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 0x55aeb7fa2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49894/0 (socket says 172.21.15.60:49894) 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 -- 172.21.15.60:0/1896429636 learned_addr learned my addr 172.21.15.60:0/1896429636 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe51bfff640 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fcf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52259c640 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fcf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 -- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 msgr2=0x55aeb7fcf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fcf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 -- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 msgr2=0x55aeb7fcf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fcf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 -- 172.21.15.60:0/1896429636 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb7ebbf10 con 0x55aeb7f9f770 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe52259c640 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fcf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.218+0000 7fe522d9d640 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 0x55aeb7fa2f80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe51c008da0 tx=0x7fe51c00d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f688b74d395573ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe503fff640 1 -- 172.21.15.60:0/1896429636 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c018680 con 0x55aeb7f9f770 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe503fff640 1 -- 172.21.15.60:0/1896429636 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c018840 con 0x55aeb7f9f770 2024-04-25T08:32:37.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe503fff640 1 -- 172.21.15.60:0/1896429636 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c018c70 con 0x55aeb7f9f770 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1896429636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 msgr2=0x55aeb7fa2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 0x55aeb7fa2f80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe51c008da0 tx=0x7fe51c00d6e0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1896429636 shutdown_connections 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7f9f770 0x55aeb7fa2f80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fcf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1896429636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fcf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1896429636 >> 172.21.15.60:0/1896429636 conn(0x55aeb7fa19e0 msgr2=0x55aeb7fea320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1896429636 shutdown_connections 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.219+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1896429636 wait complete. 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.220+0000 7fe52b7d6740 1 -- start start 2024-04-25T08:32:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.220+0000 7fe52b7d6740 1 Processor -- start 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fc2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb7fc27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7fa4b90 con 0x55aeb7fd1820 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7ea74b0 con 0x55aeb7fccb10 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7fa4e70 con 0x55aeb7fccef0 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52b7d6740 10 client.0 _ll_register_callbacks cb 0x55aeb7ee7180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb7fc27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb7fc27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49904/0 (socket says 172.21.15.60:49904) 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 -- 172.21.15.60:0/3331853263 learned_addr learned my addr 172.21.15.60:0/3331853263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe51bfff640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fc2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe52259c640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 -- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 msgr2=0x55aeb7fc2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fc2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 -- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 msgr2=0x55aeb7fee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 -- 172.21.15.60:0/3331853263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeb7ce77d0 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.221+0000 7fe522d9d640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb7fc27b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00dbe0 tx=0x7fe51c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd902ba63219f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe5017fa640 1 -- 172.21.15.60:0/3331853263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c022430 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe5017fa640 1 -- 172.21.15.60:0/3331853263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe51c0225d0 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe51c018840 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe5017fa640 1 -- 172.21.15.60:0/3331853263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51c02f050 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 10 client.6447 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55aeb7c1d3f0 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe5017fa640 1 -- 172.21.15.60:0/3331853263 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c018840 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe5017fa640 1 -- 172.21.15.60:0/3331853263 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe51c032020 con 0x55aeb7fd1820 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 10 client.6447 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 10 client.6447 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:37.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 20 client.6447 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb7fa4460 0x55aeb7feb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 4 client.6447 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aeb7f162e0 con 0x55aeb7fa4460 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.222+0000 7fe52259c640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb7fa4460 0x55aeb7feb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52259c640 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb7fa4460 0x55aeb7feb450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6ab7da88f16637 server_cookie=71a36c18798f8564 in_seq=0 out_seq=0 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe5017fa640 10 client.6447 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe5017fa640 1 -- 172.21.15.60:0/3331853263 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aeb7f162e0 con 0x55aeb7fa4460 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe5017fa640 10 client.6447 handle_command_reply: tid=0 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 client.6447 shutdown 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 msgr2=0x55aeb7fc27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb7fc27b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00dbe0 tx=0x7fe51c005db0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 shutdown_connections 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb7fc27b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb7fa4460 0x55aeb7feb450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccef0 0x55aeb7fc2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe5017fa640 0 client.6447 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/3331853263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccb10 0x55aeb7fee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 >> 172.21.15.60:0/3331853263 conn(0x55aeb7f9f770 msgr2=0x55aeb7fcbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 shutdown_connections 2024-04-25T08:32:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.223+0000 7fe52b7d6740 1 -- 172.21.15.60:0/3331853263 wait complete. 2024-04-25T08:32:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.224+0000 7fe52b7d6740 20 client.6447 trim_cache size 0 max 16384 2024-04-25T08:32:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.239+0000 7fe52b7d6740 1 Processor -- start 2024-04-25T08:32:37.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.239+0000 7fe52b7d6740 1 -- start start 2024-04-25T08:32:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 0x55aeb7fec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb7fa6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fd1820 0x55aeb7fcb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe524200350 con 0x55aeb7fccb10 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5241ff510 con 0x55aeb7fccef0 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240cb860 con 0x55aeb7fd1820 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 0x55aeb7fec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 0x55aeb7fec670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49920/0 (socket says 172.21.15.60:49920) 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 -- 172.21.15.60:0/4212520996 learned_addr learned my addr 172.21.15.60:0/4212520996 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 -- 172.21.15.60:0/4212520996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fd1820 msgr2=0x55aeb7fcb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fd1820 0x55aeb7fcb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 -- 172.21.15.60:0/4212520996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 msgr2=0x55aeb7fa6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb7fa6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 -- 172.21.15.60:0/4212520996 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb7bdd0f0 con 0x55aeb7fccb10 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe52259c640 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 0x55aeb7fec670 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe514009860 tx=0x7fe51407c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93b1cd38ad9fef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe5037fe640 1 -- 172.21.15.60:0/4212520996 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140a7070 con 0x55aeb7fccb10 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe5037fe640 1 -- 172.21.15.60:0/4212520996 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5140794e0 con 0x55aeb7fccb10 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.240+0000 7fe5037fe640 1 -- 172.21.15.60:0/4212520996 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514091030 con 0x55aeb7fccb10 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 -- 172.21.15.60:0/4212520996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 msgr2=0x55aeb7fec670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 0x55aeb7fec670 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe514009860 tx=0x7fe51407c040 comp rx=0 tx=0).stop 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 -- 172.21.15.60:0/4212520996 shutdown_connections 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fd1820 0x55aeb7fcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb7fa6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/4212520996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fccb10 0x55aeb7fec670 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 -- 172.21.15.60:0/4212520996 >> 172.21.15.60:0/4212520996 conn(0x55aeb7f9f770 msgr2=0x55aeb7fe6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 -- 172.21.15.60:0/4212520996 shutdown_connections 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 -- 172.21.15.60:0/4212520996 wait complete. 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.241+0000 7fe52b7d6740 1 -- start start 2024-04-25T08:32:37.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 Processor -- start 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccb10 0x55aeb7f9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb8057480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb80579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7f9fd50 con 0x55aeb7fd1820 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240cbe60 con 0x55aeb7fccef0 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aeb7fd1ed0 con 0x55aeb7fccb10 2024-04-25T08:32:37.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe52b7d6740 10 client.0 _ll_register_callbacks cb 0x7fe524062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.242+0000 7fe522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb80579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb80579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49924/0 (socket says 172.21.15.60:49924) 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 -- 172.21.15.60:0/1826024437 learned_addr learned my addr 172.21.15.60:0/1826024437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe52259c640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccb10 0x55aeb7f9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe51bfff640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb8057480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 -- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccb10 msgr2=0x55aeb7f9ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccb10 0x55aeb7f9ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 -- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 msgr2=0x55aeb8057480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb8057480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 -- 172.21.15.60:0/1826024437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeb7fe9bc0 con 0x55aeb7fd1820 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe522d9d640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb80579c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000f80 tx=0x7fe51c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589e19b984debd9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe5017fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe5017fa640 1 -- 172.21.15.60:0/1826024437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c018840 con 0x55aeb7fd1820 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe5017fa640 1 -- 172.21.15.60:0/1826024437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe51c0189e0 con 0x55aeb7fd1820 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5140794e0 con 0x55aeb7fd1820 2024-04-25T08:32:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe5017fa640 1 -- 172.21.15.60:0/1826024437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe51c041050 con 0x55aeb7fd1820 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe52b7d6740 10 client.6453 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55aeb7fe9e30 con 0x55aeb7fd1820 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.243+0000 7fe5017fa640 1 -- 172.21.15.60:0/1826024437 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5140794e0 con 0x55aeb7fd1820 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe5017fa640 1 -- 172.21.15.60:0/1826024437 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe51c044020 con 0x55aeb7fd1820 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 10 client.6453 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 10 client.6453 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 20 client.6453 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb800cdb0 0x55aeb7fc0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 4 client.6453 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aeb7f10630 con 0x55aeb800cdb0 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52259c640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb800cdb0 0x55aeb7fc0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52259c640 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb800cdb0 0x55aeb7fc0ed0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57627682251c6975 server_cookie=9d53690ba8dfee64 in_seq=0 out_seq=0 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe5017fa640 10 client.6453 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe5017fa640 1 -- 172.21.15.60:0/1826024437 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x55aeb7f10630 con 0x55aeb800cdb0 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe5017fa640 10 client.6453 handle_command_reply: tid=0 2024-04-25T08:32:37.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 client.6453 shutdown 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 msgr2=0x55aeb80579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb80579c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000f80 tx=0x7fe51c004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 shutdown_connections 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55aeb7fd1820 0x55aeb80579c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aeb7fccef0 0x55aeb8057480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x55aeb800cdb0 0x55aeb7fc0ed0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 --2- 172.21.15.60:0/1826024437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55aeb7fccb10 0x55aeb7f9ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe5017fa640 0 client.6453 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.244+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 >> 172.21.15.60:0/1826024437 conn(0x55aeb7f9f770 msgr2=0x55aeb7fcb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 shutdown_connections 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe52b7d6740 1 -- 172.21.15.60:0/1826024437 wait complete. 2024-04-25T08:32:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe52b7d6740 20 client.6453 trim_cache size 0 max 16384 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "a5827676-2461-4422-af56-4bb0a384300e": { 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "a5827676-2461-4422-af56-4bb0a384300e", 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "b3ebcc8c-8122-47c6-a2bb-c03b218a3fa4": { 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "b3ebcc8c-8122-47c6-a2bb-c03b218a3fa4", 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "bb5b1f2e-e235-4a9e-8ca6-ee83a15dee41": { 2024-04-25T08:32:37.247 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/", 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "bb5b1f2e-e235-4a9e-8ca6-ee83a15dee41", 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: "dfc6d281-a2dc-4974-8185-c2bc9b553cd0": { 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "dfc6d281-a2dc-4974-8185-c2bc9b553cd0", 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "" 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:37.248 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe50406b4d0 msgr2=0x7fe50406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe50406b4d0 0x7fe50406d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe50c007fd0 tx=0x7fe50c007960 comp rx=0 tx=0).stop 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 msgr2=0x7fe5240b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 0x7fe5240b48f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe514009890 tx=0x7fe5140046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 shutdown_connections 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe50406b4d0 0x7fe50406d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe524161710 0x7fe524160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe52415dc20 0x7fe524156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 --2- 172.21.15.60:0/1465350275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5240d32f0 0x7fe5240b48f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 >> 172.21.15.60:0/1465350275 conn(0x7fe5240d0420 msgr2=0x7fe5240b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 shutdown_connections 2024-04-25T08:32:37.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:37.245+0000 7fe528ad8640 1 -- 172.21.15.60:0/1465350275 wait complete. 2024-04-25T08:32:42.253 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:32:42.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:32:42.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.372+0000 7f314fce1640 1 Processor -- start 2024-04-25T08:32:42.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.373+0000 7f314fce1640 1 -- start start 2024-04-25T08:32:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.373+0000 7f314fce1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481616f0 0x7f3148161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.373+0000 7f314fce1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314806c8d0 con 0x7f31481610f0 2024-04-25T08:32:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.373+0000 7f314fce1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31480c6960 con 0x7f31480ce0a0 2024-04-25T08:32:42.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.373+0000 7f314fce1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31481614d0 con 0x7f31481616f0 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314da7d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f31481610f0 0x7f314806c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39638/0 (socket says 172.21.15.60:39638) 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314da7d640 1 -- 172.21.15.60:0/2761040465 learned_addr learned my addr 172.21.15.60:0/2761040465 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314d27c640 1 --2- 172.21.15.60:0/2761040465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481616f0 0x7f3148161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314ca7b640 1 -- 172.21.15.60:0/2761040465 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285787913 0 0) 0x7f314806c8d0 con 0x7f31481610f0 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314ca7b640 1 -- 172.21.15.60:0/2761040465 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3130003520 con 0x7f31481610f0 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314ca7b640 1 -- 172.21.15.60:0/2761040465 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 481557567 0 0) 0x7f3130003520 con 0x7f31481610f0 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314d27c640 1 -- 172.21.15.60:0/2761040465 >> v1:172.21.15.100:6789/0 conn(0x7f31480ce0a0 legacy=0x7f31480d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314d27c640 1 -- 172.21.15.60:0/2761040465 >> v1:172.21.15.60:6789/0 conn(0x7f31481610f0 legacy=0x7f314806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.374+0000 7f314d27c640 1 -- 172.21.15.60:0/2761040465 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148162260 con 0x7f31481616f0 2024-04-25T08:32:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.375+0000 7f314d27c640 1 --2- 172.21.15.60:0/2761040465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481616f0 0x7f3148161b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f313c0097d0 tx=0x7f313c02a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f60695868029d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.375+0000 7f314ca7b640 1 -- 172.21.15.60:0/2761040465 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c037070 con 0x7f31481616f0 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.375+0000 7f314ca7b640 1 -- 172.21.15.60:0/2761040465 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c02ab40 con 0x7f31481616f0 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.375+0000 7f314ca7b640 1 -- 172.21.15.60:0/2761040465 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31400019f0 con 0x7f31481616f0 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 -- 172.21.15.60:0/2761040465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481616f0 msgr2=0x7f3148161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 --2- 172.21.15.60:0/2761040465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481616f0 0x7f3148161b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f313c0097d0 tx=0x7f313c02a0b0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 -- 172.21.15.60:0/2761040465 shutdown_connections 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 --2- 172.21.15.60:0/2761040465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481616f0 0x7f3148161b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 -- 172.21.15.60:0/2761040465 >> 172.21.15.60:0/2761040465 conn(0x7f314806f500 msgr2=0x7f314806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 -- 172.21.15.60:0/2761040465 shutdown_connections 2024-04-25T08:32:42.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.376+0000 7f314fce1640 1 -- 172.21.15.60:0/2761040465 wait complete. 2024-04-25T08:32:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.377+0000 7f314fce1640 1 Processor -- start 2024-04-25T08:32:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.377+0000 7f314fce1640 1 -- start start 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314fce1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31480ce0a0 0x7f31481f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314fce1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481610f0 0x7f31481597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314fce1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 0x7f31481531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314fce1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480cfce0 con 0x7f31481616f0 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314fce1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31480cfa00 con 0x7f31480ce0a0 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314fce1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31480cf690 con 0x7f31481610f0 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 0x7f31481531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 0x7f31481531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41884/0 (socket says 172.21.15.60:41884) 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 -- 172.21.15.60:0/1751851730 learned_addr learned my addr 172.21.15.60:0/1751851730 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314da7d640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31480ce0a0 0x7f31481f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314d27c640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481610f0 0x7f31481597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 -- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481610f0 msgr2=0x7f31481597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481610f0 0x7f31481597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 -- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31480ce0a0 msgr2=0x7f31481f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31480ce0a0 0x7f31481f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.378+0000 7f314e27e640 1 -- 172.21.15.60:0/1751851730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31481f3c40 con 0x7f31481616f0 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.379+0000 7f314e27e640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 0x7f31481531a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f313800c480 tx=0x7f313800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.379+0000 7f31467fc640 1 -- 172.21.15.60:0/1751851730 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138017070 con 0x7f31481616f0 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.379+0000 7f31467fc640 1 -- 172.21.15.60:0/1751851730 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138005480 con 0x7f31481616f0 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.379+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31481f3f20 con 0x7f31481616f0 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.379+0000 7f31467fc640 1 -- 172.21.15.60:0/1751851730 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138003830 con 0x7f31481616f0 2024-04-25T08:32:42.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.379+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31481f6cd0 con 0x7f31481616f0 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.380+0000 7f31467fc640 1 -- 172.21.15.60:0/1751851730 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f313801a070 con 0x7f31481616f0 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.381+0000 7f31467fc640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f313406b4d0 0x7f313406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.381+0000 7f31467fc640 1 -- 172.21.15.60:0/1751851730 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3138089600 con 0x7f31481616f0 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.381+0000 7f314da7d640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f313406b4d0 0x7f313406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.382+0000 7f314da7d640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f313406b4d0 0x7f313406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31400012e0 tx=0x7f3140001ad0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.382+0000 7f31529e9740 1 Processor -- start 2024-04-25T08:32:42.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.382+0000 7f31529e9740 1 -- start start 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112d96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f31529e9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 0x556112d69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f31529e9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112ccfda0 con 0x556112d66770 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112c78b80 con 0x556112d93b10 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112ccbe40 con 0x556112d93ef0 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 0x556112d69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 0x556112d69f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41890/0 (socket says 172.21.15.60:41890) 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 -- 172.21.15.60:0/3381194301 learned_addr learned my addr 172.21.15.60:0/3381194301 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314d27c640 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314da7d640 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112d96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 -- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 msgr2=0x556112d96860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d96860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 -- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 msgr2=0x556112d96320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112d96320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 -- 172.21.15.60:0/3381194301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556112c82f10 con 0x556112d66770 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f314e27e640 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 0x556112d69f80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f313805ff20 tx=0x7f313805bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cf33d88b6f2a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.383+0000 7f312f7fe640 1 -- 172.21.15.60:0/3381194301 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138087070 con 0x556112d66770 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f312f7fe640 1 -- 172.21.15.60:0/3381194301 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31380560f0 con 0x556112d66770 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f312f7fe640 1 -- 172.21.15.60:0/3381194301 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313805ffc0 con 0x556112d66770 2024-04-25T08:32:42.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 -- 172.21.15.60:0/3381194301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 msgr2=0x556112d69f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 0x556112d69f80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f313805ff20 tx=0x7f313805bfd0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 -- 172.21.15.60:0/3381194301 shutdown_connections 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d66770 0x556112d69f80 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 --2- 172.21.15.60:0/3381194301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112d96320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 -- 172.21.15.60:0/3381194301 >> 172.21.15.60:0/3381194301 conn(0x556112d689e0 msgr2=0x556112db1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 -- 172.21.15.60:0/3381194301 shutdown_connections 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.384+0000 7f31529e9740 1 -- 172.21.15.60:0/3381194301 wait complete. 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.385+0000 7f31529e9740 1 -- start start 2024-04-25T08:32:42.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.385+0000 7f31529e9740 1 Processor -- start 2024-04-25T08:32:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93b10 0x556112d6b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 0x556112db3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112db4000 0x556112dc1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112d6bb90 con 0x556112d93ef0 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112c6e4b0 con 0x556112db4000 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112d6be70 con 0x556112d93b10 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f31529e9740 10 client.0 _ll_register_callbacks cb 0x556112cae180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 0x556112db3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 0x556112db3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41898/0 (socket says 172.21.15.60:41898) 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 -- 172.21.15.60:0/260768948 learned_addr learned my addr 172.21.15.60:0/260768948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314da7d640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93b10 0x556112d6b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 -- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93b10 msgr2=0x556112d6b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93b10 0x556112d6b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314e27e640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112db4000 0x556112dc1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 -- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112db4000 msgr2=0x556112dc1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112db4000 0x556112dc1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314d27c640 1 -- 172.21.15.60:0/260768948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556112aae7d0 con 0x556112d93ef0 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.386+0000 7f314da7d640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93b10 0x556112d6b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f314d27c640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 0x556112db3ac0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f313c009720 tx=0x7f313c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca8fb70c185ede7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f312cff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f312cff9640 1 -- 172.21.15.60:0/260768948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c037040 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f312cff9640 1 -- 172.21.15.60:0/260768948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f313c047c30 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31380560f0 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f312cff9640 1 -- 172.21.15.60:0/260768948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f313c047e30 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 10 client.6465 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5561129e43f0 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f312cff9640 1 -- 172.21.15.60:0/260768948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31380560f0 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f312cff9640 1 -- 172.21.15.60:0/260768948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f313c032850 con 0x556112d93ef0 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 10 client.6465 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 10 client.6465 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 20 client.6465 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112dc4ae0 0x556112dc4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 4 client.6465 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556112cdd2e0 con 0x556112dc4ae0 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.387+0000 7f314da7d640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112dc4ae0 0x556112dc4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f312cff9640 10 client.6465 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f314da7d640 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112dc4ae0 0x556112dc4f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459a6594209d66c6 server_cookie=873f489e7a766f97 in_seq=0 out_seq=0 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f312cff9640 1 -- 172.21.15.60:0/260768948 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556112cdd2e0 con 0x556112dc4ae0 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f312cff9640 10 client.6465 handle_command_reply: tid=0 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 client.6465 shutdown 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 msgr2=0x556112db3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 0x556112db3ac0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f313c009720 tx=0x7f313c004730 comp rx=0 tx=0).stop 2024-04-25T08:32:42.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 shutdown_connections 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112db4000 0x556112dc1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93ef0 0x556112db3ac0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112dc4ae0 0x556112dc4f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 --2- 172.21.15.60:0/260768948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93b10 0x556112d6b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f312cff9640 0 client.6465 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.388+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 >> 172.21.15.60:0/260768948 conn(0x556112d66770 msgr2=0x556112d67420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.389+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 shutdown_connections 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.389+0000 7f31529e9740 1 -- 172.21.15.60:0/260768948 wait complete. 2024-04-25T08:32:42.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.389+0000 7f31529e9740 20 client.6465 trim_cache size 0 max 16384 2024-04-25T08:32:42.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.404+0000 7f31529e9740 1 Processor -- start 2024-04-25T08:32:42.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.404+0000 7f31529e9740 1 -- start start 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112dd4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d93310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f31529e9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 0x556112db4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f31529e9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31482003a0 con 0x556112db4000 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31481ff520 con 0x556112d93b10 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3148158570 con 0x556112d93ef0 2024-04-25T08:32:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 0x556112db4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 0x556112db4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41912/0 (socket says 172.21.15.60:41912) 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 -- 172.21.15.60:0/2068677733 learned_addr learned my addr 172.21.15.60:0/2068677733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314d27c640 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d93310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314da7d640 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112dd4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 -- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 msgr2=0x556112d93310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d93310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 -- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 msgr2=0x556112dd4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112dd4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 -- 172.21.15.60:0/2068677733 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561129a40f0 con 0x556112db4000 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314da7d640 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112dd4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f314e27e640 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 0x556112db4970 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3138087a80 tx=0x7f3138013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df39ad6a0a6d15c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.405+0000 7f312effd640 1 -- 172.21.15.60:0/2068677733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138087040 con 0x556112db4000 2024-04-25T08:32:42.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f312effd640 1 -- 172.21.15.60:0/2068677733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138054460 con 0x556112db4000 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f312effd640 1 -- 172.21.15.60:0/2068677733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138060200 con 0x556112db4000 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 -- 172.21.15.60:0/2068677733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 msgr2=0x556112db4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 0x556112db4970 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3138087a80 tx=0x7f3138013000 comp rx=0 tx=0).stop 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 -- 172.21.15.60:0/2068677733 shutdown_connections 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112db4000 0x556112db4970 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112d93ef0 0x556112d93310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 --2- 172.21.15.60:0/2068677733 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93b10 0x556112dd4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 -- 172.21.15.60:0/2068677733 >> 172.21.15.60:0/2068677733 conn(0x556112d66770 msgr2=0x556112db1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 -- 172.21.15.60:0/2068677733 shutdown_connections 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.406+0000 7f31529e9740 1 -- 172.21.15.60:0/2068677733 wait complete. 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.407+0000 7f31529e9740 1 -- start start 2024-04-25T08:32:42.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.407+0000 7f31529e9740 1 Processor -- start 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 0x556112db4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93ef0 0x556112d8a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112db4000 0x556112d8a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556112d999a0 con 0x556112d93b10 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31481fefa0 con 0x556112d93ef0 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x556112c70010 con 0x556112db4000 2024-04-25T08:32:42.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 10 client.0 _ll_register_callbacks cb 0x7f313c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 0x556112db4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 0x556112db4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41922/0 (socket says 172.21.15.60:41922) 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 -- 172.21.15.60:0/2809289357 learned_addr learned my addr 172.21.15.60:0/2809289357 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314d27c640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93ef0 0x556112d8a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314e27e640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112db4000 0x556112d8a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 -- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112db4000 msgr2=0x556112d8a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112db4000 0x556112d8a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 -- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93ef0 msgr2=0x556112d8a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93ef0 0x556112d8a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 -- 172.21.15.60:0/2809289357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556112d935d0 con 0x556112d93b10 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f314da7d640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 0x556112db4c20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3140001560 tx=0x7f3140012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e29997f41bf3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f312cff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f312cff9640 1 -- 172.21.15.60:0/2809289357 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140013940 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f312cff9640 1 -- 172.21.15.60:0/2809289357 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f314002fc50 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.408+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3138054460 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f312cff9640 1 -- 172.21.15.60:0/2809289357 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f314002fe50 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 10 client.6471 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556112dcb490 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f312cff9640 1 -- 172.21.15.60:0/2809289357 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138054460 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f312cff9640 1 -- 172.21.15.60:0/2809289357 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3140036050 con 0x556112d93b10 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 10 client.6471 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 10 client.6471 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 20 client.6471 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112d67490 0x556112e1afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 4 client.6471 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556112cd7630 con 0x556112d67490 2024-04-25T08:32:42.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f314d27c640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112d67490 0x556112e1afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f312cff9640 10 client.6471 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.409+0000 7f314d27c640 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112d67490 0x556112e1afe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8364de91990f1ce1 server_cookie=7b70f2e4faf32c0f in_seq=0 out_seq=0 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f312cff9640 1 -- 172.21.15.60:0/2809289357 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556112cd7630 con 0x556112d67490 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f312cff9640 10 client.6471 handle_command_reply: tid=0 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 client.6471 shutdown 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 msgr2=0x556112db4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 0x556112db4c20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3140001560 tx=0x7f3140012000 comp rx=0 tx=0).stop 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 shutdown_connections 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x556112d67490 0x556112e1afe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x556112db4000 0x556112d8a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556112d93ef0 0x556112d8a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f312cff9640 0 client.6471 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 --2- 172.21.15.60:0/2809289357 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556112d93b10 0x556112db4c20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 >> 172.21.15.60:0/2809289357 conn(0x556112d66770 msgr2=0x556112d61db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 shutdown_connections 2024-04-25T08:32:42.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 1 -- 172.21.15.60:0/2809289357 wait complete. 2024-04-25T08:32:42.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f31529e9740 20 client.6471 trim_cache size 0 max 16384 2024-04-25T08:32:42.413 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:32:42.413 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:32:42.413 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f313406b4d0 msgr2=0x7f313406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f314fce1640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f313406b4d0 0x7f313406d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31400012e0 tx=0x7f3140001ad0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 msgr2=0x7f31481531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.410+0000 7f314fce1640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 0x7f31481531a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f313800c480 tx=0x7f313800c940 comp rx=0 tx=0).stop 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 shutdown_connections 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31481616f0 0x7f31481531a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31481610f0 0x7f31481597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f313406b4d0 0x7f313406d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 --2- 172.21.15.60:0/1751851730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31480ce0a0 0x7f31481f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 >> 172.21.15.60:0/1751851730 conn(0x7f314806f500 msgr2=0x7f3148155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 shutdown_connections 2024-04-25T08:32:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.411+0000 7f314fce1640 1 -- 172.21.15.60:0/1751851730 wait complete. 2024-04-25T08:32:42.418 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:32:42.418 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:32:42.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:32:42.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.525+0000 7fb3692dd640 1 Processor -- start 2024-04-25T08:32:42.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.525+0000 7fb3692dd640 1 -- start start 2024-04-25T08:32:42.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3692dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 0x7fb3640d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3692dd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb364161710 0x7fb3640d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3692dd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36415e000 con 0x7fb36415dc20 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3692dd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3640d1de0 con 0x7fb364161710 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3692dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb364161c60 con 0x7fb3640ced80 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb362ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb36415dc20 0x7fb3640ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39640/0 (socket says 172.21.15.60:39640) 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb362ffd640 1 -- 172.21.15.60:0/2435174883 learned_addr learned my addr 172.21.15.60:0/2435174883 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3627fc640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 0x7fb3640d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635130458 0 0) 0x7fb36415e000 con 0x7fb36415dc20 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3637fe640 1 --2- 172.21.15.60:0/2435174883 >> v2:172.21.15.100:3300/0 conn(0x7fb364161710 0x7fb3640d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb34c003520 con 0x7fb36415dc20 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 763375755 0 0) 0x7fb34c003520 con 0x7fb36415dc20 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 msgr2=0x7fb3640d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 0x7fb3640d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 >> v2:172.21.15.100:3300/0 conn(0x7fb364161710 msgr2=0x7fb3640d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 --2- 172.21.15.60:0/2435174883 >> v2:172.21.15.100:3300/0 conn(0x7fb364161710 0x7fb3640d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364161f40 con 0x7fb36415dc20 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb354001670 con 0x7fb36415dc20 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 >> v1:172.21.15.60:6789/0 conn(0x7fb36415dc20 legacy=0x7fb3640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 0x7fb34c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb34c003f30 0x7fb34c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c004d40 0x7fb34c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640d1de0 con 0x7fb34c004d40 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb364161c60 con 0x7fb34c003f30 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c00b9c0 con 0x7fb3640ced80 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.526+0000 7fb3637fe640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c004d40 0x7fb34c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3627fc640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb34c003f30 0x7fb34c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb362ffd640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 0x7fb34c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3637fe640 1 -- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 msgr2=0x7fb34c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3637fe640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3640ced80 0x7fb34c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3637fe640 1 -- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb34c003f30 msgr2=0x7fb34c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3637fe640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb34c003f30 0x7fb34c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3637fe640 1 -- 172.21.15.60:0/2435174883 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c003520 con 0x7fb34c004d40 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.527+0000 7fb3637fe640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c004d40 0x7fb34c0051c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009fc0 tx=0x7fb34c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3605dd851e0dd228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c009c10 con 0x7fb34c004d40 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c036070 con 0x7fb34c004d40 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb361ffb640 1 -- 172.21.15.60:0/2435174883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c02d070 con 0x7fb34c004d40 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3692dd640 1 -- 172.21.15.60:0/2435174883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c004d40 msgr2=0x7fb34c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3692dd640 1 --2- 172.21.15.60:0/2435174883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb34c004d40 0x7fb34c0051c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009fc0 tx=0x7fb34c005830 comp rx=0 tx=0).stop 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3637fe640 1 -- 172.21.15.60:0/2435174883 reap_dead start 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3692dd640 1 -- 172.21.15.60:0/2435174883 shutdown_connections 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3692dd640 1 -- 172.21.15.60:0/2435174883 >> 172.21.15.60:0/2435174883 conn(0x7fb3640d0420 msgr2=0x7fb3640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3692dd640 1 -- 172.21.15.60:0/2435174883 shutdown_connections 2024-04-25T08:32:42.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.528+0000 7fb3692dd640 1 -- 172.21.15.60:0/2435174883 wait complete. 2024-04-25T08:32:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.529+0000 7fb3692dd640 1 Processor -- start 2024-04-25T08:32:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.529+0000 7fb3692dd640 1 -- start start 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3692dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb364154ee0 0x7fb364157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3692dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 0x7fb3640d2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3692dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640d2e50 0x7fb364155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3692dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640b5e50 con 0x7fb3641578c0 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3692dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb364162500 con 0x7fb3640d2e50 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3692dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640b5fd0 con 0x7fb364154ee0 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 0x7fb3640d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 0x7fb3640d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41940/0 (socket says 172.21.15.60:41940) 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.530+0000 7fb3627fc640 1 -- 172.21.15.60:0/1056546561 learned_addr learned my addr 172.21.15.60:0/1056546561 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb364154ee0 0x7fb364157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3637fe640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640d2e50 0x7fb364155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3627fc640 1 -- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb364154ee0 msgr2=0x7fb364157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3627fc640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb364154ee0 0x7fb364157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3627fc640 1 -- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640d2e50 msgr2=0x7fb364155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3627fc640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640d2e50 0x7fb364155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3627fc640 1 -- 172.21.15.60:0/1056546561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36415f3c0 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb364154ee0 0x7fb364157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3637fe640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640d2e50 0x7fb364155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3627fc640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 0x7fb3640d2910 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb358011910 tx=0x7fb358011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb34b7fe640 1 -- 172.21.15.60:0/1056546561 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35800f9d0 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb34b7fe640 1 -- 172.21.15.60:0/1056546561 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35800fb70 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36415f6a0 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb34b7fe640 1 -- 172.21.15.60:0/1056546561 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358014440 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.531+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36415fc60 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.533+0000 7fb34b7fe640 1 -- 172.21.15.60:0/1056546561 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb358012040 con 0x7fb3641578c0 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.533+0000 7fb34b7fe640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb35006b500 0x7fb35006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.533+0000 7fb34b7fe640 1 -- 172.21.15.60:0/1056546561 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb35808a8c0 con 0x7fb3641578c0 2024-04-25T08:32:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.533+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb35006b500 0x7fb35006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.534+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb35006b500 0x7fb35006d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb354004840 tx=0x7fb354009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.534+0000 7fb36bfdb740 1 Processor -- start 2024-04-25T08:32:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.534+0000 7fb36bfdb740 1 -- start start 2024-04-25T08:32:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d0e80 0x558ac08d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08a3700 0x558ac08a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac080ce00 con 0x558ac08d0aa0 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac07b5d20 con 0x558ac08d0e80 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ac0808ea0 con 0x558ac08a3700 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41942/0 (socket says 172.21.15.60:41942) 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 -- 172.21.15.60:0/3352158591 learned_addr learned my addr 172.21.15.60:0/3352158591 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb3637fe640 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08a3700 0x558ac08a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb3627fc640 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d0e80 0x558ac08d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 -- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08a3700 msgr2=0x558ac08a6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08a3700 0x558ac08a6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 -- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d0e80 msgr2=0x558ac08d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d0e80 0x558ac08d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 -- 172.21.15.60:0/3352158591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ac07d7df0 con 0x558ac08d0aa0 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb3627fc640 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d0e80 0x558ac08d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.535+0000 7fb362ffd640 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d32b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb354007e00 tx=0x7fb354013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99aedcdce93bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb348ff9640 1 -- 172.21.15.60:0/3352158591 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354026070 con 0x558ac08d0aa0 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb348ff9640 1 -- 172.21.15.60:0/3352158591 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35401c050 con 0x558ac08d0aa0 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb348ff9640 1 -- 172.21.15.60:0/3352158591 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354021610 con 0x558ac08d0aa0 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 -- 172.21.15.60:0/3352158591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 msgr2=0x558ac08d32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d32b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb354007e00 tx=0x7fb354013000 comp rx=0 tx=0).stop 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 -- 172.21.15.60:0/3352158591 shutdown_connections 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08a3700 0x558ac08a6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d0e80 0x558ac08d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/3352158591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d32b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 -- 172.21.15.60:0/3352158591 >> 172.21.15.60:0/3352158591 conn(0x558ac08a5970 msgr2=0x558ac08ee2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 -- 172.21.15.60:0/3352158591 shutdown_connections 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.536+0000 7fb36bfdb740 1 -- 172.21.15.60:0/3352158591 wait complete. 2024-04-25T08:32:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.537+0000 7fb36bfdb740 1 -- start start 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.537+0000 7fb36bfdb740 1 Processor -- start 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac08a8b20 con 0x558ac08d0aa0 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac07ab590 con 0x558ac08d5510 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ac08a8e00 con 0x558ac08d0e80 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb36bfdb740 10 client.0 _ll_register_callbacks cb 0x558ac07b7e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41958/0 (socket says 172.21.15.60:41958) 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 -- 172.21.15.60:0/2942619247 learned_addr learned my addr 172.21.15.60:0/2942619247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb3627fc640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb3637fe640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 -- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 msgr2=0x558ac08d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 -- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 msgr2=0x558ac08d4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08d4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 -- 172.21.15.60:0/2942619247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ac05213f0 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb362ffd640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08cf960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x558ac08ab000 tx=0x7fb364162110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e36dc542e9c031b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb341ffb640 1 -- 172.21.15.60:0/2942619247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354030060 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.538+0000 7fb341ffb640 1 -- 172.21.15.60:0/2942619247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb354022d20 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb35401c050 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb341ffb640 1 -- 172.21.15.60:0/2942619247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb354036050 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 10 client.6483 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x558ac05eb7d0 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb341ffb640 1 -- 172.21.15.60:0/2942619247 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35401c050 con 0x558ac08d0aa0 2024-04-25T08:32:42.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb341ffb640 1 -- 172.21.15.60:0/2942619247 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb35402b410 con 0x558ac08d0aa0 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 10 client.6483 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 10 client.6483 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 20 client.6483 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac08ed620 0x558ac08ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 4 client.6483 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ac081a340 con 0x558ac08ed620 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb3627fc640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac08ed620 0x558ac08ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb341ffb640 10 client.6483 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.539+0000 7fb3627fc640 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac08ed620 0x558ac08ef3e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83e5caeeb77c5e3 server_cookie=96b3c8918f9cd4af in_seq=0 out_seq=0 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb341ffb640 1 -- 172.21.15.60:0/2942619247 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ac081a340 con 0x558ac08ed620 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb341ffb640 10 client.6483 handle_command_reply: tid=0 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 client.6483 shutdown 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 msgr2=0x558ac08cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08cf960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x558ac08ab000 tx=0x7fb364162110 comp rx=0 tx=0).stop 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 shutdown_connections 2024-04-25T08:32:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08d4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac08ed620 0x558ac08ef3e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/2942619247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08cf960 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb341ffb640 0 client.6483 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 >> 172.21.15.60:0/2942619247 conn(0x558ac08a3700 msgr2=0x558ac08eca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 shutdown_connections 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 1 -- 172.21.15.60:0/2942619247 wait complete. 2024-04-25T08:32:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.540+0000 7fb36bfdb740 20 client.6483 trim_cache size 0 max 16384 2024-04-25T08:32:42.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.552+0000 7fb36bfdb740 1 Processor -- start 2024-04-25T08:32:42.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.552+0000 7fb36bfdb740 1 -- start start 2024-04-25T08:32:42.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac089eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3642010f0 con 0x558ac08d0aa0 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3642002f0 con 0x558ac08d5510 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3641f4790 con 0x558ac08d0e80 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac089eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac089eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41972/0 (socket says 172.21.15.60:41972) 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 -- 172.21.15.60:0/1965124746 learned_addr learned my addr 172.21.15.60:0/1965124746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 -- 172.21.15.60:0/1965124746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 msgr2=0x558ac08a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 -- 172.21.15.60:0/1965124746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 msgr2=0x558ac08a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 -- 172.21.15.60:0/1965124746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ac08e5640 con 0x558ac08d0aa0 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac089eea0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb354001050 tx=0x7fb35402a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a18f9f2d34ea1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb343fff640 1 -- 172.21.15.60:0/1965124746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354026040 con 0x558ac08d0aa0 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb343fff640 1 -- 172.21.15.60:0/1965124746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35401c070 con 0x558ac08d0aa0 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.553+0000 7fb343fff640 1 -- 172.21.15.60:0/1965124746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354022980 con 0x558ac08d0aa0 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1965124746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 msgr2=0x558ac089eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac089eea0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb354001050 tx=0x7fb35402a000 comp rx=0 tx=0).stop 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1965124746 shutdown_connections 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac08a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1965124746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac089eea0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1965124746 >> 172.21.15.60:0/1965124746 conn(0x558ac08a3700 msgr2=0x558ac08e8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1965124746 shutdown_connections 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1965124746 wait complete. 2024-04-25T08:32:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 -- start start 2024-04-25T08:32:42.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.554+0000 7fb36bfdb740 1 Processor -- start 2024-04-25T08:32:42.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08efd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac0956070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ac0840710 con 0x558ac08d0aa0 2024-04-25T08:32:42.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3641ffdf0 con 0x558ac08d5510 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ac08f6890 con 0x558ac08d0e80 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb36bfdb740 10 client.0 _ll_register_callbacks cb 0x7fb354018040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb362ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41974/0 (socket says 172.21.15.60:41974) 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb362ffd640 1 -- 172.21.15.60:0/1262606291 learned_addr learned my addr 172.21.15.60:0/1262606291 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb3627fc640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.555+0000 7fb3637fe640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac0956070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb362ffd640 1 -- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 msgr2=0x558ac08efd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08efd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb362ffd640 1 -- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 msgr2=0x558ac0956070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac0956070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb362ffd640 1 -- 172.21.15.60:0/1262606291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ac08d6e50 con 0x558ac08d0aa0 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb362ffd640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d6910 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb3540078c0 tx=0x7fb35400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e851743ed63358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb341ffb640 1 -- 172.21.15.60:0/1262606291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354026070 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb341ffb640 1 -- 172.21.15.60:0/1262606291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb354022b80 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb35401c070 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb341ffb640 1 -- 172.21.15.60:0/1262606291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb35404a050 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 10 client.6489 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x558ac09019b0 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb341ffb640 1 -- 172.21.15.60:0/1262606291 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35401c070 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb341ffb640 1 -- 172.21.15.60:0/1262606291 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb35404d020 con 0x558ac08d0aa0 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 10 client.6489 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 10 client.6489 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 20 client.6489 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac09565b0 0x558ac09593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 4 client.6489 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558ac0814690 con 0x558ac09565b0 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.556+0000 7fb3627fc640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac09565b0 0x558ac09593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb3627fc640 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac09565b0 0x558ac09593c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b3697444d312b76 server_cookie=fd8b3940088e22b5 in_seq=0 out_seq=0 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb341ffb640 10 client.6489 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb341ffb640 1 -- 172.21.15.60:0/1262606291 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+167 (crc 0 0 0) 0x558ac0814690 con 0x558ac09565b0 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb341ffb640 10 client.6489 handle_command_reply: tid=0 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 client.6489 shutdown 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 msgr2=0x558ac08d6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d6910 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb3540078c0 tx=0x7fb35400c000 comp rx=0 tx=0).stop 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 shutdown_connections 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558ac08d5510 0x558ac0956070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x558ac09565b0 0x558ac09593c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558ac08d0e80 0x558ac08efd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 --2- 172.21.15.60:0/1262606291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558ac08d0aa0 0x558ac08d6910 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb341ffb640 0 client.6489 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 >> 172.21.15.60:0/1262606291 conn(0x558ac08a3700 msgr2=0x558ac08d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 shutdown_connections 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 1 -- 172.21.15.60:0/1262606291 wait complete. 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.557+0000 7fb36bfdb740 20 client.6489 trim_cache size 0 max 16384 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stdout: "damage_type": "backtrace", 2024-04-25T08:32:42.560 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 152601341, 2024-04-25T08:32:42.561 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 1099511630188, 2024-04-25T08:32:42.561 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T08:32:42.561 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:32:42.561 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb35006b500 msgr2=0x7fb35006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb35006b500 0x7fb35006d9c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb354004840 tx=0x7fb354009040 comp rx=0 tx=0).stop 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 msgr2=0x7fb3640d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 0x7fb3640d2910 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb358011910 tx=0x7fb358011dd0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 shutdown_connections 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb35006b500 0x7fb35006d9c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3640d2e50 0x7fb364155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3641578c0 0x7fb3640d2910 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 --2- 172.21.15.60:0/1056546561 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb364154ee0 0x7fb364157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 >> 172.21.15.60:0/1056546561 conn(0x7fb3640d0420 msgr2=0x7fb3640b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 shutdown_connections 2024-04-25T08:32:42.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.558+0000 7fb3692dd640 1 -- 172.21.15.60:0/1056546561 wait complete. 2024-04-25T08:32:42.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.669+0000 7fc00dd5c640 1 Processor -- start 2024-04-25T08:32:42.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.669+0000 7fc00dd5c640 1 -- start start 2024-04-25T08:32:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc00dd5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 0x7fc00815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc00dd5c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc0080af720 0x7fc0080afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc00dd5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00815f180 con 0x7fc0081617b0 2024-04-25T08:32:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc00dd5c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0080b3670 con 0x7fc0080af720 2024-04-25T08:32:42.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc00dd5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc008070a10 con 0x7fc008158220 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc007fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc0081617b0 0x7fc008161c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39642/0 (socket says 172.21.15.60:39642) 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.670+0000 7fc007fff640 1 -- 172.21.15.60:0/1153203410 learned_addr learned my addr 172.21.15.60:0/1153203410 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0077fe640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 0x7fc00815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566213169 0 0) 0x7fc000001490 con 0x7fc0081617b0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1153203410 >> v2:172.21.15.100:3300/0 conn(0x7fc0080af720 0x7fc0080afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff0003520 con 0x7fc0081617b0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222602555 0 0) 0x7fbff0003520 con 0x7fc0081617b0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 msgr2=0x7fc00815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 0x7fc00815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 >> v2:172.21.15.100:3300/0 conn(0x7fc0080af720 msgr2=0x7fc0080afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 --2- 172.21.15.60:0/1153203410 >> v2:172.21.15.100:3300/0 conn(0x7fc0080af720 0x7fc0080afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00806fe00 con 0x7fc0081617b0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fc000001bc0 con 0x7fc0081617b0 2024-04-25T08:32:42.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 >> v1:172.21.15.60:6789/0 conn(0x7fc0081617b0 legacy=0x7fc008161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 0x7fbff00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0003f30 0x7fbff0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc008070a10 con 0x7fbff0003f30 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000b9c0 con 0x7fbff0004d40 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000bb40 con 0x7fc008158220 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0003f30 0x7fbff0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc0077fe640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 0x7fbff00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc007fff640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 -- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 msgr2=0x7fbff00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc008158220 0x7fbff00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 -- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff0004d40 msgr2=0x7fbff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.671+0000 7fc006ffd640 1 -- 172.21.15.60:0/1153203410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0003520 con 0x7fbff0003f30 2024-04-25T08:32:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.672+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0003f30 0x7fbff0004800 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009ac0 tx=0x7fbff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e9082cca8dc84c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.672+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc036070 con 0x7fbff0003f30 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.672+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc031400 con 0x7fbff0003f30 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.672+0000 7fc0067fc640 1 -- 172.21.15.60:0/1153203410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc009690 con 0x7fbff0003f30 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1153203410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0003f30 msgr2=0x7fbff0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1153203410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbff0003f30 0x7fbff0004800 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009ac0 tx=0x7fbff0005830 comp rx=0 tx=0).stop 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc007fff640 1 -- 172.21.15.60:0/1153203410 reap_dead start 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1153203410 shutdown_connections 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1153203410 >> 172.21.15.60:0/1153203410 conn(0x7fc0080b2370 msgr2=0x7fc0080b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1153203410 shutdown_connections 2024-04-25T08:32:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.673+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1153203410 wait complete. 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.674+0000 7fc00dd5c640 1 Processor -- start 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.674+0000 7fc00dd5c640 1 -- start start 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc00dd5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0003f30 0x7fc00814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc00dd5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 0x7fc008154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc00dd5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc008154ba0 0x7fc008159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc00dd5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080706a0 con 0x7fc00815a610 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc00dd5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080703c0 con 0x7fc008154ba0 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc00dd5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc008158600 con 0x7fbff0003f30 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 0x7fc008154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 0x7fc008154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42000/0 (socket says 172.21.15.60:42000) 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 -- 172.21.15.60:0/1269133833 learned_addr learned my addr 172.21.15.60:0/1269133833 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc0077fe640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0003f30 0x7fc00814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc007fff640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc008154ba0 0x7fc008159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 -- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0003f30 msgr2=0x7fc00814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0003f30 0x7fc00814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 -- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc008154ba0 msgr2=0x7fc008159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc008154ba0 0x7fc008159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.675+0000 7fc006ffd640 1 -- 172.21.15.60:0/1269133833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00814c710 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.676+0000 7fc006ffd640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 0x7fc008154660 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009be0 tx=0x7fbffc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.676+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc040020 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.676+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc003c60 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.676+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc036040 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.676+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0080b1680 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.676+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc008159160 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.677+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc008073990 con 0x7fc00815a610 2024-04-25T08:32:42.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.677+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbffc043020 con 0x7fc00815a610 2024-04-25T08:32:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.678+0000 7fbfeffff640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbff406b4d0 0x7fbff406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.678+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbffc0aa470 con 0x7fc00815a610 2024-04-25T08:32:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.678+0000 7fc0077fe640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbff406b4d0 0x7fbff406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:42.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.678+0000 7fc0077fe640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbff406b4d0 0x7fbff406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc008161610 tx=0x7fbff8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.682+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbffc04a070 con 0x7fc00815a610 2024-04-25T08:32:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.902+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0080b3bc0 con 0x7fc00815a610 2024-04-25T08:32:42.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.903+0000 7fbfeffff640 1 -- 172.21.15.60:0/1269133833 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc00814c710 con 0x7fc00815a610 2024-04-25T08:32:42.899 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:42.899 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:35.021589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":8,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:42.899 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 24 2024-04-25T08:32:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbff406b4d0 msgr2=0x7fbff406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbff406b4d0 0x7fbff406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc008161610 tx=0x7fbff8008040 comp rx=0 tx=0).stop 2024-04-25T08:32:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 msgr2=0x7fc008154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 0x7fc008154660 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009be0 tx=0x7fbffc004730 comp rx=0 tx=0).stop 2024-04-25T08:32:42.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 shutdown_connections 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbff406b4d0 0x7fbff406d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc008154ba0 0x7fc008159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc00815a610 0x7fc008154660 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 --2- 172.21.15.60:0/1269133833 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbff0003f30 0x7fc00814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 >> 172.21.15.60:0/1269133833 conn(0x7fc0080b2370 msgr2=0x7fc0080b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 shutdown_connections 2024-04-25T08:32:42.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:42.905+0000 7fc00dd5c640 1 -- 172.21.15.60:0/1269133833 wait complete. 2024-04-25T08:32:42.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 152601341 2024-04-25T08:32:43.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.019+0000 7f2ee179b640 1 Processor -- start 2024-04-25T08:32:43.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.019+0000 7f2ee179b640 1 -- start start 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.020+0000 7f2ee179b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc0b5bb0 0x7f2edc0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.020+0000 7f2ee179b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc0cf1d0 con 0x7f2edc161890 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.020+0000 7f2ee179b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc161620 con 0x7f2edc161240 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.020+0000 7f2ee179b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0b7120 con 0x7f2edc0b5bb0 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.020+0000 7f2eda7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2edc161890 0x7f2edc0b54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39646/0 (socket says 172.21.15.60:39646) 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.020+0000 7f2eda7fc640 1 -- 172.21.15.60:0/1476631652 learned_addr learned my addr 172.21.15.60:0/1476631652 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/1476631652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc0b5bb0 0x7f2edc0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2ed9ffb640 1 -- 172.21.15.60:0/1476631652 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536413399 0 0) 0x7f2edc0cf1d0 con 0x7f2edc161890 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2ed9ffb640 1 -- 172.21.15.60:0/1476631652 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec8003520 con 0x7f2edc161890 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2ed9ffb640 1 -- 172.21.15.60:0/1476631652 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773392567 0 0) 0x7f2edc161620 con 0x7f2edc161240 2024-04-25T08:32:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2edb7fe640 1 -- 172.21.15.60:0/1476631652 >> v1:172.21.15.100:6789/0 conn(0x7f2edc161240 legacy=0x7f2edc0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2edb7fe640 1 -- 172.21.15.60:0/1476631652 >> v1:172.21.15.60:6789/0 conn(0x7f2edc161890 legacy=0x7f2edc0b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.021+0000 7f2edb7fe640 1 -- 172.21.15.60:0/1476631652 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc161e00 con 0x7f2edc0b5bb0 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/1476631652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc0b5bb0 0x7f2edc0b6be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009dc0 tx=0x7f2ecc02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf5e00da6bcbc0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ed9ffb640 1 -- 172.21.15.60:0/1476631652 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc035020 con 0x7f2edc0b5bb0 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ed9ffb640 1 -- 172.21.15.60:0/1476631652 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc007540 con 0x7f2edc0b5bb0 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ed9ffb640 1 -- 172.21.15.60:0/1476631652 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40015f0 con 0x7f2edc0b5bb0 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ee179b640 1 -- 172.21.15.60:0/1476631652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc0b5bb0 msgr2=0x7f2edc0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ee179b640 1 --2- 172.21.15.60:0/1476631652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc0b5bb0 0x7f2edc0b6be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009dc0 tx=0x7f2ecc02a640 comp rx=0 tx=0).stop 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ee179b640 1 -- 172.21.15.60:0/1476631652 shutdown_connections 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ee179b640 1 --2- 172.21.15.60:0/1476631652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc0b5bb0 0x7f2edc0b6be0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ee179b640 1 -- 172.21.15.60:0/1476631652 >> 172.21.15.60:0/1476631652 conn(0x7f2edc0d04a0 msgr2=0x7f2edc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.022+0000 7f2ee179b640 1 -- 172.21.15.60:0/1476631652 shutdown_connections 2024-04-25T08:32:43.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.023+0000 7f2ee179b640 1 -- 172.21.15.60:0/1476631652 wait complete. 2024-04-25T08:32:43.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.024+0000 7f2ee179b640 1 Processor -- start 2024-04-25T08:32:43.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.024+0000 7f2ee179b640 1 -- start start 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2edc0b5bb0 0x7f2edc14eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc161240 0x7f2edc0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 0x7f2edc15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc1626a0 con 0x7f2edc161890 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc1623c0 con 0x7f2edc0b5bb0 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0d15f0 con 0x7f2edc161240 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 0x7f2edc15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 0x7f2edc15f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42012/0 (socket says 172.21.15.60:42012) 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 -- 172.21.15.60:0/3948716378 learned_addr learned my addr 172.21.15.60:0/3948716378 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc161240 0x7f2edc0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edaffd640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2edc0b5bb0 0x7f2edc14eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 -- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc161240 msgr2=0x7f2edc0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc161240 0x7f2edc0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 -- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2edc0b5bb0 msgr2=0x7f2edc14eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2edc0b5bb0 0x7f2edc14eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 -- 172.21.15.60:0/3948716378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc159df0 con 0x7f2edc161890 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 0x7f2edc15f880 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ee0 tx=0x7f2ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ed37fe640 1 -- 172.21.15.60:0/3948716378 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc03f070 con 0x7f2edc161890 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ed37fe640 1 -- 172.21.15.60:0/3948716378 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc031bc0 con 0x7f2edc161890 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ed37fe640 1 -- 172.21.15.60:0/3948716378 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc035070 con 0x7f2edc161890 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc15a0d0 con 0x7f2edc161890 2024-04-25T08:32:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.025+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc15a690 con 0x7f2edc161890 2024-04-25T08:32:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.027+0000 7f2ed37fe640 1 -- 172.21.15.60:0/3948716378 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ecc042070 con 0x7f2edc161890 2024-04-25T08:32:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.027+0000 7f2ed37fe640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ebc06b4d0 0x7f2ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.027+0000 7f2ed37fe640 1 -- 172.21.15.60:0/3948716378 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ecc0ab370 con 0x7f2edc161890 2024-04-25T08:32:43.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.028+0000 7f2edaffd640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ebc06b4d0 0x7f2ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.028+0000 7f2edaffd640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ebc06b4d0 0x7f2ebc06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007fd0 tx=0x7f2ed4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.028+0000 7f2ee4578740 1 Processor -- start 2024-04-25T08:32:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.028+0000 7f2ee4578740 1 -- start start 2024-04-25T08:32:43.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 0x5625d5fcd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ffda30 0x5625d5ffcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffd2f0 0x5625d5fd2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2ee4578740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d5f36010 con 0x5625d601a910 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d5ed5180 con 0x5625d5ffda30 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625d5f320b0 con 0x5625d5ffd2f0 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 0x5625d5fcd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 0x5625d5fcd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42026/0 (socket says 172.21.15.60:42026) 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2edaffd640 1 -- 172.21.15.60:0/1423844567 learned_addr learned my addr 172.21.15.60:0/1423844567 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ffda30 0x5625d5ffcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.029+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffd2f0 0x5625d5fd2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edaffd640 1 -- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffd2f0 msgr2=0x5625d5fd2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edaffd640 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffd2f0 0x5625d5fd2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edaffd640 1 -- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ffda30 msgr2=0x5625d5ffcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edaffd640 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ffda30 0x5625d5ffcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edaffd640 1 -- 172.21.15.60:0/1423844567 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625d5d16900 con 0x5625d601a910 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffd2f0 0x5625d5fd2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2edaffd640 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 0x5625d5fcd530 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005260 tx=0x7f2ed4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83f89b3c5d8bacf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2ed0ff9640 1 -- 172.21.15.60:0/1423844567 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4005450 con 0x5625d601a910 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2ed0ff9640 1 -- 172.21.15.60:0/1423844567 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4003800 con 0x5625d601a910 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.030+0000 7f2ed0ff9640 1 -- 172.21.15.60:0/1423844567 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4003bd0 con 0x5625d601a910 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 -- 172.21.15.60:0/1423844567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 msgr2=0x5625d5fcd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 0x5625d5fcd530 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005260 tx=0x7f2ed4011000 comp rx=0 tx=0).stop 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 -- 172.21.15.60:0/1423844567 shutdown_connections 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffd2f0 0x5625d5fd2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ffda30 0x5625d5ffcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1423844567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d601a910 0x5625d5fcd530 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 -- 172.21.15.60:0/1423844567 >> 172.21.15.60:0/1423844567 conn(0x5625d5fcf7a0 msgr2=0x5625d6017db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 -- 172.21.15.60:0/1423844567 shutdown_connections 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 -- 172.21.15.60:0/1423844567 wait complete. 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 -- start start 2024-04-25T08:32:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.031+0000 7f2ee4578740 1 Processor -- start 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d5ffff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d601a910 0x5625d6019880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ff8a00 0x5625d602bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d5fd16c0 con 0x5625d5ffda30 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d5ed2c50 con 0x5625d5ff8a00 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625d5fd0ab0 con 0x5625d601a910 2024-04-25T08:32:43.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2ee4578740 10 client.0 _ll_register_callbacks cb 0x5625d5eda1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d5ffff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d5ffff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42034/0 (socket says 172.21.15.60:42034) 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2edaffd640 1 -- 172.21.15.60:0/1056613563 learned_addr learned my addr 172.21.15.60:0/1056613563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d601a910 0x5625d6019880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.032+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ff8a00 0x5625d602bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edaffd640 1 -- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d601a910 msgr2=0x5625d6019880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edaffd640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d601a910 0x5625d6019880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edaffd640 1 -- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ff8a00 msgr2=0x5625d602bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edaffd640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ff8a00 0x5625d602bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edaffd640 1 -- 172.21.15.60:0/1056613563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625d5c0a1f0 con 0x5625d5ffda30 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ff8a00 0x5625d602bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2edaffd640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d5ffff70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4003c40 tx=0x7f2ed4007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879fc152f41db1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/1056613563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4021860 con 0x5625d5ffda30 2024-04-25T08:32:43.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/1056613563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ed4021a00 con 0x5625d5ffda30 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ed4003800 con 0x5625d5ffda30 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/1056613563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ed4031050 con 0x5625d5ffda30 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 10 client.6507 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5625d5d77090 con 0x5625d5ffda30 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/1056613563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4003800 con 0x5625d5ffda30 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/1056613563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ed4034020 con 0x5625d5ffda30 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 10 client.6507 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 10 client.6507 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 20 client.6507 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d5ffe280 0x5625d5ffe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 4 client.6507 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.033+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625d5f43550 con 0x5625d5ffe280 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.034+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d5ffe280 0x5625d5ffe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.034+0000 7f2eb9ffb640 10 client.6507 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.034+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d5ffe280 0x5625d5ffe660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5a34fc9b349d1f server_cookie=75c0fc484ae3c77d in_seq=0 out_seq=0 2024-04-25T08:32:43.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.034+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/1056613563 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625d5f43550 con 0x5625d5ffe280 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.034+0000 7f2eb9ffb640 10 client.6507 handle_command_reply: tid=0 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.034+0000 7f2ee4578740 1 client.6507 shutdown 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 msgr2=0x5625d5ffff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d5ffff70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4003c40 tx=0x7f2ed4007ab0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 shutdown_connections 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d5ff8a00 0x5625d602bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d5ffe280 0x5625d5ffe660 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d601a910 0x5625d6019880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 --2- 172.21.15.60:0/1056613563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d5ffff70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2eb9ffb640 0 client.6507 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 >> 172.21.15.60:0/1056613563 conn(0x5625d5ffd2f0 msgr2=0x5625d6017db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 shutdown_connections 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 1 -- 172.21.15.60:0/1056613563 wait complete. 2024-04-25T08:32:43.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.035+0000 7f2ee4578740 20 client.6507 trim_cache size 0 max 16384 2024-04-25T08:32:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 Processor -- start 2024-04-25T08:32:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 -- start start 2024-04-25T08:32:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 0x5625d6018040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffda30 0x5625d602a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d602aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc1ff3c0 con 0x5625d5ff8a00 2024-04-25T08:32:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c8cd0 con 0x5625d601a910 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.047+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0c92d0 con 0x5625d5ffda30 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 0x5625d6018040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 0x5625d6018040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42044/0 (socket says 172.21.15.60:42044) 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 -- 172.21.15.60:0/943692532 learned_addr learned my addr 172.21.15.60:0/943692532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffda30 0x5625d602a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 -- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffda30 msgr2=0x5625d602a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d602aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffda30 0x5625d602a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 -- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 msgr2=0x5625d602aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d602aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 -- 172.21.15.60:0/943692532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625d5be3370 con 0x5625d5ff8a00 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffda30 0x5625d602a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2edaffd640 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 0x5625d6018040 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4003ef0 tx=0x7f2ed4025070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc0629b58146e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2ebbfff640 1 -- 172.21.15.60:0/943692532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4039070 con 0x5625d5ff8a00 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2ebbfff640 1 -- 172.21.15.60:0/943692532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4003800 con 0x5625d5ff8a00 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.048+0000 7f2ebbfff640 1 -- 172.21.15.60:0/943692532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4021880 con 0x5625d5ff8a00 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 -- 172.21.15.60:0/943692532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 msgr2=0x5625d6018040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 0x5625d6018040 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4003ef0 tx=0x7f2ed4025070 comp rx=0 tx=0).stop 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 -- 172.21.15.60:0/943692532 shutdown_connections 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d602aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ffda30 0x5625d602a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 --2- 172.21.15.60:0/943692532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ff8a00 0x5625d6018040 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 -- 172.21.15.60:0/943692532 >> 172.21.15.60:0/943692532 conn(0x5625d5ffd2f0 msgr2=0x5625d6019690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 -- 172.21.15.60:0/943692532 shutdown_connections 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 -- 172.21.15.60:0/943692532 wait complete. 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 -- start start 2024-04-25T08:32:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.049+0000 7f2ee4578740 1 Processor -- start 2024-04-25T08:32:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ff8a00 0x5625d602a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d602aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d600f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d6000a80 con 0x5625d5ffda30 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d6019dc0 con 0x5625d601a910 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625d602c600 con 0x5625d5ff8a00 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2ee4578740 10 client.0 _ll_register_callbacks cb 0x7f2ed40013d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d602aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d602aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42058/0 (socket says 172.21.15.60:42058) 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 -- 172.21.15.60:0/774869815 learned_addr learned my addr 172.21.15.60:0/774869815 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2edaffd640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ff8a00 0x5625d602a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d600f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 -- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ff8a00 msgr2=0x5625d602a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ff8a00 0x5625d602a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 -- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 msgr2=0x5625d600f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d600f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2eda7fc640 1 -- 172.21.15.60:0/774869815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625d6017ae0 con 0x5625d5ffda30 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.050+0000 7f2edb7fe640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d600f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:43.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eda7fc640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d602aa30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4000f60 tx=0x7f2ec4001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aea57ac729c63b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/774869815 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4005050 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/774869815 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ec4024c60 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ed4003800 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/774869815 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ec4024e60 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 10 client.6513 fetch_fsmap learned FSMap version 24 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5625d5ff7a00 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/774869815 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4003800 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/774869815 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ec402b050 con 0x5625d5ffda30 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 10 client.6513 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 10 client.6513 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 20 client.6513 populate_metadata read hostname 'smithi060' 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d6010180 0x5625d60826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 4 client.6513 mds_command: new command op to 5886 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 152601341}] 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 --> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] -- command(tid 0: {"prefix": "damage rm", "damage_id": 152601341}) v1 -- 0x5625d5f3d8a0 con 0x5625d6010180 2024-04-25T08:32:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.051+0000 7f2edaffd640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d6010180 0x5625d60826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2eb9ffb640 10 client.6513 ms_handle_connect on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2edaffd640 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d6010180 0x5625d60826d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949c03a8cafaedd5 server_cookie=f570fdc8c48b0500 in_seq=0 out_seq=0 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2eb9ffb640 1 -- 172.21.15.60:0/774869815 <== mds.0 v2:172.21.15.60:6833/1977192273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x5625d5f3d8a0 con 0x5625d6010180 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2eb9ffb640 10 client.6513 handle_command_reply: tid=0 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 client.6513 shutdown 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 msgr2=0x5625d602aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d602aa30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4000f60 tx=0x7f2ec4001700 comp rx=0 tx=0).stop 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 shutdown_connections 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625d601a910 0x5625d600f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5625d5ffda30 0x5625d602aa30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.60:6833/1977192273,v1:172.21.15.60:6835/1977192273] conn(0x5625d6010180 0x5625d60826d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 --2- 172.21.15.60:0/774869815 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5625d5ff8a00 0x5625d602a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2eb9ffb640 0 client.6513 ms_handle_reset on v2:172.21.15.60:6833/1977192273 2024-04-25T08:32:43.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 >> 172.21.15.60:0/774869815 conn(0x5625d5ffd2f0 msgr2=0x5625d6018040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 shutdown_connections 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 1 -- 172.21.15.60:0/774869815 wait complete. 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.052+0000 7f2ee4578740 20 client.6513 trim_cache size 0 max 16384 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ebc06b4d0 msgr2=0x7f2ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ebc06b4d0 0x7f2ebc06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4007fd0 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 msgr2=0x7f2edc15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 0x7f2edc15f880 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009ee0 tx=0x7f2ecc004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 shutdown_connections 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2edc161890 0x7f2edc15f880 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2edc161240 0x7f2edc0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ebc06b4d0 0x7f2ebc06d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 --2- 172.21.15.60:0/3948716378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2edc0b5bb0 0x7f2edc14eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 >> 172.21.15.60:0/3948716378 conn(0x7f2edc0d04a0 msgr2=0x7f2edc15e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 shutdown_connections 2024-04-25T08:32:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.053+0000 7f2ee179b640 1 -- 172.21.15.60:0/3948716378 wait complete. 2024-04-25T08:32:43.056 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:32:43.059+0000 7fcb3a3bb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:32:43.060 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:32:43.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:43.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.165+0000 7f93f077e640 1 Processor -- start 2024-04-25T08:32:43.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.165+0000 7f93f077e640 1 -- start start 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93f077e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e815dc70 0x7f93e80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93f077e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93e809a450 0x7f93e80cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93f077e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e806c220 con 0x7f93e806c700 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93f077e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e809a8b0 con 0x7f93e809a450 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93f077e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80d0c90 con 0x7f93e815dc70 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93edd19640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93e809a450 0x7f93e80cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93eed1b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f93e806c700 0x7f93e80cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39662/0 (socket says 172.21.15.60:39662) 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93eed1b640 1 -- 172.21.15.60:0/2454282917 learned_addr learned my addr 172.21.15.60:0/2454282917 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93ee51a640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e815dc70 0x7f93e80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93edd19640 1 -- 172.21.15.60:0/2454282917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e815dc70 msgr2=0x7f93e80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93edd19640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e815dc70 0x7f93e80d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93edd19640 1 -- 172.21.15.60:0/2454282917 >> v1:172.21.15.60:6789/0 conn(0x7f93e806c700 legacy=0x7f93e80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93edd19640 1 -- 172.21.15.60:0/2454282917 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80d0080 con 0x7f93e809a450 2024-04-25T08:32:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93ed518640 1 -- 172.21.15.60:0/2454282917 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251688242 0 0) 0x7f93e806c220 con 0x7f93e806c700 2024-04-25T08:32:43.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.166+0000 7f93ee51a640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e815dc70 0x7f93e80d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:43.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.167+0000 7f93edd19640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cf9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097a0 tx=0x7f93dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d764f700ed9856b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.167+0000 7f93ed518640 1 -- 172.21.15.60:0/2454282917 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc037070 con 0x7f93e809a450 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.167+0000 7f93ed518640 1 -- 172.21.15.60:0/2454282917 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc032440 con 0x7f93e809a450 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.167+0000 7f93ed518640 1 -- 172.21.15.60:0/2454282917 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc032810 con 0x7f93e809a450 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 -- 172.21.15.60:0/2454282917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 msgr2=0x7f93e80cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cf9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097a0 tx=0x7f93dc02a180 comp rx=0 tx=0).stop 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 -- 172.21.15.60:0/2454282917 shutdown_connections 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cf9c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 --2- 172.21.15.60:0/2454282917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e815dc70 0x7f93e80d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 -- 172.21.15.60:0/2454282917 >> 172.21.15.60:0/2454282917 conn(0x7f93e80b3270 msgr2=0x7f93e80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 -- 172.21.15.60:0/2454282917 shutdown_connections 2024-04-25T08:32:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.168+0000 7f93f077e640 1 -- 172.21.15.60:0/2454282917 wait complete. 2024-04-25T08:32:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.169+0000 7f93f077e640 1 Processor -- start 2024-04-25T08:32:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.169+0000 7f93f077e640 1 -- start start 2024-04-25T08:32:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93f077e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e806c700 0x7f93e815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93f077e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93f077e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e815dc70 0x7f93e80cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93f077e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80d0920 con 0x7f93e809a450 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93f077e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80d0640 con 0x7f93e815dc70 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93f077e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e815e050 con 0x7f93e806c700 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42076/0 (socket says 172.21.15.60:42076) 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 -- 172.21.15.60:0/2127587268 learned_addr learned my addr 172.21.15.60:0/2127587268 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93ee51a640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e806c700 0x7f93e815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93eed1b640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e815dc70 0x7f93e80cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 -- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e806c700 msgr2=0x7f93e815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e806c700 0x7f93e815adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 -- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e815dc70 msgr2=0x7f93e80cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e815dc70 0x7f93e80cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 -- 172.21.15.60:0/2127587268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80c9d00 con 0x7f93e809a450 2024-04-25T08:32:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.170+0000 7f93edd19640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cbdb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f93dc035040 tx=0x7f93dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.171+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc037040 con 0x7f93e809a450 2024-04-25T08:32:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.171+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc030770 con 0x7f93e809a450 2024-04-25T08:32:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.171+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e815f450 con 0x7f93e809a450 2024-04-25T08:32:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.171+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc030b80 con 0x7f93e809a450 2024-04-25T08:32:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.171+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e815fa10 con 0x7f93e809a450 2024-04-25T08:32:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.172+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e8073990 con 0x7f93e809a450 2024-04-25T08:32:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.172+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93dc044020 con 0x7f93e809a450 2024-04-25T08:32:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.173+0000 7f93e6ffd640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f93d406b500 0x7f93d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.173+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93dc0aa8c0 con 0x7f93e809a450 2024-04-25T08:32:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.173+0000 7f93ee51a640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f93d406b500 0x7f93d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.173+0000 7f93ee51a640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f93d406b500 0x7f93d406d9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f93e000ab60 tx=0x7f93e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.177+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93e80c9d00 con 0x7f93e809a450 2024-04-25T08:32:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.393+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93e80b40a0 con 0x7f93e809a450 2024-04-25T08:32:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.393+0000 7f93e6ffd640 1 -- 172.21.15.60:0/2127587268 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f93dc032e20 con 0x7f93e809a450 2024-04-25T08:32:43.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:43.390 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:43.022243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:43.390 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 25 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f93d406b500 msgr2=0x7f93d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f93d406b500 0x7f93d406d9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f93e000ab60 tx=0x7f93e0008040 comp rx=0 tx=0).stop 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 msgr2=0x7f93e80cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cbdb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f93dc035040 tx=0x7f93dc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 shutdown_connections 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f93d406b500 0x7f93d406d9c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93e815dc70 0x7f93e80cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93e809a450 0x7f93e80cbdb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 --2- 172.21.15.60:0/2127587268 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93e806c700 0x7f93e815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.395+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 >> 172.21.15.60:0/2127587268 conn(0x7f93e80b3270 msgr2=0x7f93e8070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.396+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 shutdown_connections 2024-04-25T08:32:43.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.396+0000 7f93f077e640 1 -- 172.21.15.60:0/2127587268 wait complete. 2024-04-25T08:32:43.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T08:32:43.511 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:32:43.511 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:32:43.511 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:32:43.520 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:32:43.520 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:32:43.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:43.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:43.611 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:32:43.612 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:43.612 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:43.628 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:43.628 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:43.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:43.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.827+0000 7f154404d640 1 Processor -- start 2024-04-25T08:32:43.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f154404d640 1 -- start start 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f154404d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f153c15ba00 0x7f153c09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f154404d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f153c1602e0 0x7f153c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f154404d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c0cd710 con 0x7f153c15f5f0 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f154404d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c099b70 con 0x7f153c15ba00 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f154404d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c099e50 con 0x7f153c1602e0 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f1541de9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f153c15ba00 0x7f153c09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.828+0000 7f1541de9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f153c15ba00 0x7f153c09a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42086/0 (socket says 172.21.15.60:42086) 2024-04-25T08:32:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f1541de9640 1 -- 172.21.15.60:0/907490436 learned_addr learned my addr 172.21.15.60:0/907490436 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f1541de9640 1 -- 172.21.15.60:0/907490436 >> v2:172.21.15.100:3300/0 conn(0x7f153c1602e0 msgr2=0x7f153c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f15415e8640 1 --2- 172.21.15.60:0/907490436 >> v2:172.21.15.100:3300/0 conn(0x7f153c1602e0 0x7f153c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f1541de9640 1 --2- 172.21.15.60:0/907490436 >> v2:172.21.15.100:3300/0 conn(0x7f153c1602e0 0x7f153c0ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f1541de9640 1 -- 172.21.15.60:0/907490436 >> v1:172.21.15.100:6789/0 conn(0x7f153c15f5f0 legacy=0x7f153c0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f1541de9640 1 -- 172.21.15.60:0/907490436 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c09a130 con 0x7f153c15ba00 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.829+0000 7f1541de9640 1 --2- 172.21.15.60:0/907490436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c15ba00 0x7f153c09a930 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b7830 tx=0x7f153402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea35794b10a5a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f1540de7640 1 -- 172.21.15.60:0/907490436 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534037070 con 0x7f153c15ba00 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f1540de7640 1 -- 172.21.15.60:0/907490436 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153402ae20 con 0x7f153c15ba00 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f1540de7640 1 -- 172.21.15.60:0/907490436 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534032660 con 0x7f153c15ba00 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 -- 172.21.15.60:0/907490436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c15ba00 msgr2=0x7f153c09a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 --2- 172.21.15.60:0/907490436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c15ba00 0x7f153c09a930 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b7830 tx=0x7f153402a110 comp rx=0 tx=0).stop 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 -- 172.21.15.60:0/907490436 shutdown_connections 2024-04-25T08:32:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 --2- 172.21.15.60:0/907490436 >> v2:172.21.15.100:3300/0 conn(0x7f153c1602e0 0x7f153c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 --2- 172.21.15.60:0/907490436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c15ba00 0x7f153c09a930 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 -- 172.21.15.60:0/907490436 >> 172.21.15.60:0/907490436 conn(0x7f153c0b3270 msgr2=0x7f153c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 -- 172.21.15.60:0/907490436 shutdown_connections 2024-04-25T08:32:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.830+0000 7f154404d640 1 -- 172.21.15.60:0/907490436 wait complete. 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 Processor -- start 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 -- start start 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153c15ba00 0x7f153c14caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153c15f5f0 0x7f153c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 0x7f153c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15bde0 con 0x7f153c1602e0 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0cd2e0 con 0x7f153c15ba00 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f154404d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c06c5f0 con 0x7f153c15f5f0 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f15425ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 0x7f153c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f15425ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 0x7f153c06d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42094/0 (socket says 172.21.15.60:42094) 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.832+0000 7f15425ea640 1 -- 172.21.15.60:0/3804225486 learned_addr learned my addr 172.21.15.60:0/3804225486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f1541de9640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153c15ba00 0x7f153c14caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15415e8640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153c15f5f0 0x7f153c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15425ea640 1 -- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153c15f5f0 msgr2=0x7f153c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15425ea640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153c15f5f0 0x7f153c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15425ea640 1 -- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153c15ba00 msgr2=0x7f153c14caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15425ea640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153c15ba00 0x7f153c14caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15425ea640 1 -- 172.21.15.60:0/3804225486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c06c8d0 con 0x7f153c1602e0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15415e8640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153c15f5f0 0x7f153c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f15425ea640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 0x7f153c06d690 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b970 tx=0x7f152c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c019070 con 0x7f153c1602e0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c00cb40 con 0x7f153c1602e0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c06cbb0 con 0x7f153c1602e0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c014440 con 0x7f153c1602e0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.833+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c06d170 con 0x7f153c1602e0 2024-04-25T08:32:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.834+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c06d370 con 0x7f153c1602e0 2024-04-25T08:32:43.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.835+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f152c01c070 con 0x7f153c1602e0 2024-04-25T08:32:43.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.835+0000 7f153a7fc640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f152806b4d0 0x7f152806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.835+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f152c089e80 con 0x7f153c1602e0 2024-04-25T08:32:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.836+0000 7f1541de9640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f152806b4d0 0x7f152806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.836+0000 7f1541de9640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f152806b4d0 0x7f152806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1534034000 tx=0x7f153402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:43.840+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f153c06d370 con 0x7f153c1602e0 2024-04-25T08:32:44.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.022+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f153c073990 con 0x7f153c1602e0 2024-04-25T08:32:44.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.023+0000 7f153a7fc640 1 -- 172.21.15.60:0/3804225486 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f152c0585b0 con 0x7f153c1602e0 2024-04-25T08:32:44.020 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:44.020 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:15.989621+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f152806b4d0 msgr2=0x7f152806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f152806b4d0 0x7f152806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1534034000 tx=0x7f153402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 msgr2=0x7f153c06d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 0x7f153c06d690 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b970 tx=0x7f152c00be30 comp rx=0 tx=0).stop 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 shutdown_connections 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f153c1602e0 0x7f153c06d690 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f153c15f5f0 0x7f153c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f152806b4d0 0x7f152806d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 --2- 172.21.15.60:0/3804225486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f153c15ba00 0x7f153c14caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 >> 172.21.15.60:0/3804225486 conn(0x7f153c0b3270 msgr2=0x7f153c15e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 shutdown_connections 2024-04-25T08:32:44.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.025+0000 7f154404d640 1 -- 172.21.15.60:0/3804225486 wait complete. 2024-04-25T08:32:44.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:32:12.634751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:32:13.328038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:32:44.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:44.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.140+0000 7f3bf195c640 1 Processor -- start 2024-04-25T08:32:44.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.141+0000 7f3bf195c640 1 -- start start 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.141+0000 7f3bf195c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3bec0b5620 0x7f3bec0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.141+0000 7f3bf195c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec06c2d0 0x7f3bec06c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.141+0000 7f3bf195c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec06cc80 con 0x7f3bec15b7d0 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.141+0000 7f3bf195c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec06cf50 con 0x7f3bec0b5620 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.141+0000 7f3bf195c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0ce530 con 0x7f3bec06c2d0 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3bea7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3bec0b5620 0x7f3bec0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec06c2d0 0x7f3bec06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beaffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3bec15b7d0 0x7f3bec0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47364/0 (socket says 172.21.15.60:47364) 2024-04-25T08:32:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beaffd640 1 -- 172.21.15.60:0/857969683 learned_addr learned my addr 172.21.15.60:0/857969683 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beb7fe640 1 -- 172.21.15.60:0/857969683 >> v2:172.21.15.100:3300/0 conn(0x7f3bec0b5620 msgr2=0x7f3bec0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beb7fe640 1 --2- 172.21.15.60:0/857969683 >> v2:172.21.15.100:3300/0 conn(0x7f3bec0b5620 0x7f3bec0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beb7fe640 1 -- 172.21.15.60:0/857969683 >> v1:172.21.15.100:6789/0 conn(0x7f3bec15b7d0 legacy=0x7f3bec0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.142+0000 7f3beb7fe640 1 -- 172.21.15.60:0/857969683 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0ce810 con 0x7f3bec06c2d0 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3beb7fe640 1 --2- 172.21.15.60:0/857969683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec06c2d0 0x7f3bec06c740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009d90 tx=0x7f3bdc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6849b45ddf5a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3be9ffb640 1 -- 172.21.15.60:0/857969683 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007740 con 0x7f3bec06c2d0 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3be9ffb640 1 -- 172.21.15.60:0/857969683 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0078e0 con 0x7f3bec06c2d0 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3be9ffb640 1 -- 172.21.15.60:0/857969683 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007cb0 con 0x7f3bec06c2d0 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 -- 172.21.15.60:0/857969683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec06c2d0 msgr2=0x7f3bec06c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 --2- 172.21.15.60:0/857969683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec06c2d0 0x7f3bec06c740 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009d90 tx=0x7f3bdc02a660 comp rx=0 tx=0).stop 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 -- 172.21.15.60:0/857969683 shutdown_connections 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 --2- 172.21.15.60:0/857969683 >> v2:172.21.15.100:3300/0 conn(0x7f3bec0b5620 0x7f3bec0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 --2- 172.21.15.60:0/857969683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec06c2d0 0x7f3bec06c740 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 -- 172.21.15.60:0/857969683 >> 172.21.15.60:0/857969683 conn(0x7f3bec0d0800 msgr2=0x7f3bec0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.143+0000 7f3bf195c640 1 -- 172.21.15.60:0/857969683 shutdown_connections 2024-04-25T08:32:44.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.144+0000 7f3bf195c640 1 -- 172.21.15.60:0/857969683 wait complete. 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 Processor -- start 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 -- start start 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bec06c2d0 0x7f3bec157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 0x7f3bec0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec15b7d0 0x7f3bec0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec15bbb0 con 0x7f3bec0b5620 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec06d230 con 0x7f3bec06c2d0 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.145+0000 7f3bf195c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec15bd30 con 0x7f3bec15b7d0 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 0x7f3bec0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 0x7f3bec0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42102/0 (socket says 172.21.15.60:42102) 2024-04-25T08:32:44.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 -- 172.21.15.60:0/3873622440 learned_addr learned my addr 172.21.15.60:0/3873622440 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3beaffd640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bec06c2d0 0x7f3bec157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3beb7fe640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec15b7d0 0x7f3bec0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 -- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec15b7d0 msgr2=0x7f3bec0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec15b7d0 0x7f3bec0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 -- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bec06c2d0 msgr2=0x7f3bec157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bec06c2d0 0x7f3bec157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 -- 172.21.15.60:0/3873622440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0b4e50 con 0x7f3bec0b5620 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bea7fc640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 0x7f3bec0cb630 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9b0 tx=0x7f3be000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0017070 con 0x7f3bec0b5620 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000c9a0 con 0x7f3bec0b5620 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0ce2b0 con 0x7f3bec0b5620 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.146+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000cd90 con 0x7f3bec0b5620 2024-04-25T08:32:44.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.147+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0b4890 con 0x7f3bec0b5620 2024-04-25T08:32:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.147+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec073a00 con 0x7f3bec0b5620 2024-04-25T08:32:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.148+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3be001a070 con 0x7f3bec0b5620 2024-04-25T08:32:44.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.148+0000 7f3bd3fff640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bd806b4d0 0x7f3bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.148+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3be008a210 con 0x7f3bec0b5620 2024-04-25T08:32:44.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.149+0000 7f3beaffd640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bd806b4d0 0x7f3bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.149+0000 7f3beaffd640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bd806b4d0 0x7f3bd806d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0c82b0 tx=0x7f3be4007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.153+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3be0058940 con 0x7f3bec0b5620 2024-04-25T08:32:44.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.331+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bec15c200 con 0x7f3bec0b5620 2024-04-25T08:32:44.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.332+0000 7f3bd3fff640 1 -- 172.21.15.60:0/3873622440 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3be005c9e0 con 0x7f3bec0b5620 2024-04-25T08:32:44.328 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:32:44.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bd806b4d0 msgr2=0x7f3bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bd806b4d0 0x7f3bd806d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0c82b0 tx=0x7f3be4007e80 comp rx=0 tx=0).stop 2024-04-25T08:32:44.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 msgr2=0x7f3bec0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 0x7f3bec0cb630 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9b0 tx=0x7f3be000ee70 comp rx=0 tx=0).stop 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 shutdown_connections 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bec15b7d0 0x7f3bec0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bec0b5620 0x7f3bec0cb630 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bd806b4d0 0x7f3bd806d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 --2- 172.21.15.60:0/3873622440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bec06c2d0 0x7f3bec157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 >> 172.21.15.60:0/3873622440 conn(0x7f3bec0d0800 msgr2=0x7f3bec15d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 shutdown_connections 2024-04-25T08:32:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.334+0000 7f3bf195c640 1 -- 172.21.15.60:0/3873622440 wait complete. 2024-04-25T08:32:44.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.448+0000 7f7329815640 1 Processor -- start 2024-04-25T08:32:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.448+0000 7f7329815640 1 -- start start 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7329815640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 0x7f73240cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7329815640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f732406c9e0 0x7f73240b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7329815640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732406cf50 con 0x7f73240b5620 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7329815640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240b5fc0 con 0x7f732406c9e0 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7329815640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240ce530 con 0x7f732415b7d0 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7322ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 0x7f73240cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f73237fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f732406c9e0 0x7f73240b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7322ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 0x7f73240cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:57304/0 (socket says 172.21.15.60:57304) 2024-04-25T08:32:44.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f7322ffd640 1 -- 172.21.15.60:0/3957593525 learned_addr learned my addr 172.21.15.60:0/3957593525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:44.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.449+0000 7f73227fc640 1 --1- 172.21.15.60:0/3957593525 >> v1:172.21.15.100:6789/0 conn(0x7f73240b5620 0x7f732406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47368/0 (socket says 172.21.15.60:47368) 2024-04-25T08:32:44.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.450+0000 7f7322ffd640 1 -- 172.21.15.60:0/3957593525 >> v2:172.21.15.100:3300/0 conn(0x7f732406c9e0 msgr2=0x7f73240b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:44.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.450+0000 7f7322ffd640 1 --2- 172.21.15.60:0/3957593525 >> v2:172.21.15.100:3300/0 conn(0x7f732406c9e0 0x7f73240b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.450+0000 7f7322ffd640 1 -- 172.21.15.60:0/3957593525 >> v1:172.21.15.100:6789/0 conn(0x7f73240b5620 legacy=0x7f732406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.450+0000 7f7322ffd640 1 -- 172.21.15.60:0/3957593525 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240ce810 con 0x7f732415b7d0 2024-04-25T08:32:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.450+0000 7f7322ffd640 1 --2- 172.21.15.60:0/3957593525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 0x7f73240cf010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7314009870 tx=0x7f731402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe290a4df85a052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.450+0000 7f7321ffb640 1 -- 172.21.15.60:0/3957593525 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314037070 con 0x7f732415b7d0 2024-04-25T08:32:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7321ffb640 1 -- 172.21.15.60:0/3957593525 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314032440 con 0x7f732415b7d0 2024-04-25T08:32:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7321ffb640 1 -- 172.21.15.60:0/3957593525 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314032810 con 0x7f732415b7d0 2024-04-25T08:32:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 -- 172.21.15.60:0/3957593525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 msgr2=0x7f73240cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 --2- 172.21.15.60:0/3957593525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 0x7f73240cf010 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7314009870 tx=0x7f731402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 -- 172.21.15.60:0/3957593525 shutdown_connections 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 --2- 172.21.15.60:0/3957593525 >> v2:172.21.15.100:3300/0 conn(0x7f732406c9e0 0x7f73240b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 --2- 172.21.15.60:0/3957593525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732415b7d0 0x7f73240cf010 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 -- 172.21.15.60:0/3957593525 >> 172.21.15.60:0/3957593525 conn(0x7f73240d0800 msgr2=0x7f73240d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 -- 172.21.15.60:0/3957593525 shutdown_connections 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.451+0000 7f7329815640 1 -- 172.21.15.60:0/3957593525 wait complete. 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.452+0000 7f7329815640 1 Processor -- start 2024-04-25T08:32:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.452+0000 7f7329815640 1 -- start start 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f7329815640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732406c9e0 0x7f73240c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f7329815640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73240b5620 0x7f73241f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f7329815640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 0x7f73241f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f7329815640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240cf550 con 0x7f732415b7d0 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f7329815640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732406d230 con 0x7f73240b5620 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f7329815640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240cf6d0 con 0x7f732406c9e0 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f73237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 0x7f73241f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f73237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 0x7f73241f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42112/0 (socket says 172.21.15.60:42112) 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.453+0000 7f73237fe640 1 -- 172.21.15.60:0/612816745 learned_addr learned my addr 172.21.15.60:0/612816745 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73227fc640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73240b5620 0x7f73241f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f7322ffd640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732406c9e0 0x7f73240c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73237fe640 1 -- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732406c9e0 msgr2=0x7f73240c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73237fe640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732406c9e0 0x7f73240c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73237fe640 1 -- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73240b5620 msgr2=0x7f73241f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73237fe640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73240b5620 0x7f73241f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73237fe640 1 -- 172.21.15.60:0/612816745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240cce20 con 0x7f732415b7d0 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73227fc640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73240b5620 0x7f73241f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f7322ffd640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732406c9e0 0x7f73240c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f73237fe640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 0x7f73241f7150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f731c00efc0 tx=0x7f731c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c018020 con 0x7f732415b7d0 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c009b10 con 0x7f732415b7d0 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73240cd100 con 0x7f732415b7d0 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0146c0 con 0x7f732415b7d0 2024-04-25T08:32:44.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.454+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73240cd6c0 con 0x7f732415b7d0 2024-04-25T08:32:44.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.455+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73240739e0 con 0x7f732415b7d0 2024-04-25T08:32:44.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.456+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f731c005d10 con 0x7f732415b7d0 2024-04-25T08:32:44.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.456+0000 7f730bfff640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f731006b4d0 0x7f731006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.456+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f731c08ae50 con 0x7f732415b7d0 2024-04-25T08:32:44.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.456+0000 7f7322ffd640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f731006b4d0 0x7f731006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.457+0000 7f7322ffd640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f731006b4d0 0x7f731006d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7314034000 tx=0x7f731402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.460+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f731c022070 con 0x7f732415b7d0 2024-04-25T08:32:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.638+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73240d0d80 con 0x7f732415b7d0 2024-04-25T08:32:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.638+0000 7f730bfff640 1 -- 172.21.15.60:0/612816745 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f731c010040 con 0x7f732415b7d0 2024-04-25T08:32:44.635 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f731006b4d0 msgr2=0x7f731006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f731006b4d0 0x7f731006d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7314034000 tx=0x7f731402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 msgr2=0x7f73241f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 0x7f73241f7150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f731c00efc0 tx=0x7f731c00c490 comp rx=0 tx=0).stop 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 shutdown_connections 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f732415b7d0 0x7f73241f7150 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f731006b4d0 0x7f731006d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73240b5620 0x7f73241f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 --2- 172.21.15.60:0/612816745 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f732406c9e0 0x7f73240c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.640+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 >> 172.21.15.60:0/612816745 conn(0x7f73240d0800 msgr2=0x7f7324158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.641+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 shutdown_connections 2024-04-25T08:32:44.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.641+0000 7f7329815640 1 -- 172.21.15.60:0/612816745 wait complete. 2024-04-25T08:32:44.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.755+0000 7f80cf914640 1 Processor -- start 2024-04-25T08:32:44.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.755+0000 7f80cf914640 1 -- start start 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cf914640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80ce230 0x7f80c80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cf914640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80c815b270 0x7f80c815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cf914640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c8072280 con 0x7f80c80cc4f0 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cf914640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c8071270 con 0x7f80c815b270 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cf914640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80714c0 con 0x7f80c80ce230 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cdeb1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80c815b270 0x7f80c815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cdeb1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80c815b270 0x7f80c815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42122/0 (socket says 172.21.15.60:42122) 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cdeb1640 1 -- 172.21.15.60:0/2101857700 learned_addr learned my addr 172.21.15.60:0/2101857700 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.756+0000 7f80cceaf640 1 --2- 172.21.15.60:0/2101857700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80ce230 0x7f80c80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.757+0000 7f80cdeb1640 1 -- 172.21.15.60:0/2101857700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80ce230 msgr2=0x7f80c80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.757+0000 7f80cdeb1640 1 --2- 172.21.15.60:0/2101857700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80ce230 0x7f80c80cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.757+0000 7f80cdeb1640 1 -- 172.21.15.60:0/2101857700 >> v1:172.21.15.100:6789/0 conn(0x7f80c80cc4f0 legacy=0x7f80c80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.757+0000 7f80cdeb1640 1 -- 172.21.15.60:0/2101857700 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c80717a0 con 0x7f80c815b270 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.757+0000 7f80cdeb1640 1 --2- 172.21.15.60:0/2101857700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c815b270 0x7f80c815b6c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009870 tx=0x7f80b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0b087a1e878468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.757+0000 7f80c7fff640 1 -- 172.21.15.60:0/2101857700 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8037070 con 0x7f80c815b270 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80c7fff640 1 -- 172.21.15.60:0/2101857700 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8032440 con 0x7f80c815b270 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80c7fff640 1 -- 172.21.15.60:0/2101857700 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8032810 con 0x7f80c815b270 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 -- 172.21.15.60:0/2101857700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c815b270 msgr2=0x7f80c815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 --2- 172.21.15.60:0/2101857700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c815b270 0x7f80c815b6c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009870 tx=0x7f80b802a270 comp rx=0 tx=0).stop 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 -- 172.21.15.60:0/2101857700 shutdown_connections 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 --2- 172.21.15.60:0/2101857700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c815b270 0x7f80c815b6c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 --2- 172.21.15.60:0/2101857700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80ce230 0x7f80c80cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 -- 172.21.15.60:0/2101857700 >> 172.21.15.60:0/2101857700 conn(0x7f80c806abe0 msgr2=0x7f80c806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 -- 172.21.15.60:0/2101857700 shutdown_connections 2024-04-25T08:32:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.758+0000 7f80cf914640 1 -- 172.21.15.60:0/2101857700 wait complete. 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.759+0000 7f80cf914640 1 Processor -- start 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.759+0000 7f80cf914640 1 -- start start 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cf914640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80cc4f0 0x7f80c80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cf914640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 0x7f80c81e75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cf914640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c815b270 0x7f80c8141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cf914640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c815bc00 con 0x7f80c80ce230 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cf914640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8071d60 con 0x7f80c815b270 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cf914640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c815bd80 con 0x7f80c80cc4f0 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 0x7f80c81e75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cceaf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 0x7f80c81e75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42134/0 (socket says 172.21.15.60:42134) 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cceaf640 1 -- 172.21.15.60:0/2754215773 learned_addr learned my addr 172.21.15.60:0/2754215773 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cdeb1640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c815b270 0x7f80c8141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.760+0000 7f80cd6b0640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80cc4f0 0x7f80c80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cceaf640 1 -- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80cc4f0 msgr2=0x7f80c80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cceaf640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80cc4f0 0x7f80c80cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cceaf640 1 -- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c815b270 msgr2=0x7f80c8141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cceaf640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c815b270 0x7f80c8141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cceaf640 1 -- 172.21.15.60:0/2754215773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c80cb5b0 con 0x7f80c80ce230 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cd6b0640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80cc4f0 0x7f80c80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cceaf640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 0x7f80c81e75b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00e9e0 tx=0x7f80bc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc017070 con 0x7f80c80ce230 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc00c9d0 con 0x7f80c80ce230 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c80cb890 con 0x7f80c80ce230 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00cdc0 con 0x7f80c80ce230 2024-04-25T08:32:44.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.761+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c80cb230 con 0x7f80c80ce230 2024-04-25T08:32:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.762+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c8073a00 con 0x7f80c80ce230 2024-04-25T08:32:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.763+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80bc01a070 con 0x7f80c80ce230 2024-04-25T08:32:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.763+0000 7f80c5ffb640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b406b4d0 0x7f80b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.763+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80bc08a740 con 0x7f80c80ce230 2024-04-25T08:32:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.763+0000 7f80cd6b0640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b406b4d0 0x7f80b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.764+0000 7f80cd6b0640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b406b4d0 0x7f80b406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80c0001cd0 tx=0x7f80c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.768+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80bc0594c0 con 0x7f80c80ce230 2024-04-25T08:32:44.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.946+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80c80ce650 con 0x7f80c80ce230 2024-04-25T08:32:44.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.946+0000 7f80c5ffb640 1 -- 172.21.15.60:0/2754215773 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80bc058dc0 con 0x7f80c80ce230 2024-04-25T08:32:44.943 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:32:44.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b406b4d0 msgr2=0x7f80b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b406b4d0 0x7f80b406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80c0001cd0 tx=0x7f80c0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 msgr2=0x7f80c81e75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 0x7f80c81e75b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00e9e0 tx=0x7f80bc00eea0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 shutdown_connections 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b406b4d0 0x7f80b406d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80c815b270 0x7f80c8141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80c80ce230 0x7f80c81e75b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 --2- 172.21.15.60:0/2754215773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80c80cc4f0 0x7f80c80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 >> 172.21.15.60:0/2754215773 conn(0x7f80c806abe0 msgr2=0x7f80c80cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 shutdown_connections 2024-04-25T08:32:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:44.948+0000 7f80cf914640 1 -- 172.21.15.60:0/2754215773 wait complete. 2024-04-25T08:32:44.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:45.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.061+0000 7f5a6fc5a640 1 Processor -- start 2024-04-25T08:32:45.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.061+0000 7f5a6fc5a640 1 -- start start 2024-04-25T08:32:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6fc5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6815b7d0 0x7f5a680cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6fc5a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6806c9e0 0x7f5a680b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6fc5a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6806cf50 con 0x7f5a680b5620 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6fc5a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680b5fc0 con 0x7f5a6806c9e0 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6fc5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680ce530 con 0x7f5a6815b7d0 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6e1f7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6806c9e0 0x7f5a680b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6d1f5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a680b5620 0x7f5a6806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39666/0 (socket says 172.21.15.60:39666) 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6d1f5640 1 -- 172.21.15.60:0/793250720 learned_addr learned my addr 172.21.15.60:0/793250720 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6e1f7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6806c9e0 0x7f5a680b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42146/0 (socket says 172.21.15.60:42146) 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6d9f6640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6815b7d0 0x7f5a680cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/793250720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6815b7d0 msgr2=0x7f5a680cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6e1f7640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6815b7d0 0x7f5a680cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/793250720 >> v1:172.21.15.60:6789/0 conn(0x7f5a680b5620 legacy=0x7f5a6806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/793250720 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680ce810 con 0x7f5a6806c9e0 2024-04-25T08:32:45.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6c9f4640 1 -- 172.21.15.60:0/793250720 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670898152 0 0) 0x7f5a6806cf50 con 0x7f5a680b5620 2024-04-25T08:32:45.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.062+0000 7f5a6d9f6640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6815b7d0 0x7f5a680cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:45.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.063+0000 7f5a6e1f7640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6806c9e0 0x7f5a680b5a80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009d90 tx=0x7f5a6002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea1d3b56a1fe3b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.063+0000 7f5a6c9f4640 1 -- 172.21.15.60:0/793250720 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60007740 con 0x7f5a6806c9e0 2024-04-25T08:32:45.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.063+0000 7f5a6c9f4640 1 -- 172.21.15.60:0/793250720 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a600078e0 con 0x7f5a6806c9e0 2024-04-25T08:32:45.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.063+0000 7f5a6c9f4640 1 -- 172.21.15.60:0/793250720 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60007cb0 con 0x7f5a6806c9e0 2024-04-25T08:32:45.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/793250720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6806c9e0 msgr2=0x7f5a680b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6806c9e0 0x7f5a680b5a80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009d90 tx=0x7f5a6002a660 comp rx=0 tx=0).stop 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/793250720 shutdown_connections 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6806c9e0 0x7f5a680b5a80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/793250720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6815b7d0 0x7f5a680cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/793250720 >> 172.21.15.60:0/793250720 conn(0x7f5a680d0800 msgr2=0x7f5a680d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/793250720 shutdown_connections 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.064+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/793250720 wait complete. 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.065+0000 7f5a6fc5a640 1 Processor -- start 2024-04-25T08:32:45.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.065+0000 7f5a6fc5a640 1 -- start start 2024-04-25T08:32:45.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6fc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6806c9e0 0x7f5a681f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6fc5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b5620 0x7f5a681f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6fc5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 0x7f5a680b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6fc5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cf550 con 0x7f5a6815b7d0 2024-04-25T08:32:45.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6fc5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6806d230 con 0x7f5a6806c9e0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6fc5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cf6d0 con 0x7f5a680b5620 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 0x7f5a680b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 0x7f5a680b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42156/0 (socket says 172.21.15.60:42156) 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/667843510 learned_addr learned my addr 172.21.15.60:0/667843510 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6d9f6640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6806c9e0 0x7f5a681f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6d1f5640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b5620 0x7f5a681f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b5620 msgr2=0x7f5a681f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b5620 0x7f5a681f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6806c9e0 msgr2=0x7f5a681f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6806c9e0 0x7f5a681f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 -- 172.21.15.60:0/667843510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680c8860 con 0x7f5a6815b7d0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.066+0000 7f5a6e1f7640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 0x7f5a680b8120 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009eb0 tx=0x7f5a60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.067+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60040070 con 0x7f5a6815b7d0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.067+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a60031bc0 con 0x7f5a6815b7d0 2024-04-25T08:32:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.067+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60009450 con 0x7f5a6815b7d0 2024-04-25T08:32:45.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.067+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a680c8b40 con 0x7f5a6815b7d0 2024-04-25T08:32:45.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.067+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a680c9100 con 0x7f5a6815b7d0 2024-04-25T08:32:45.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.068+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a68073a00 con 0x7f5a6815b7d0 2024-04-25T08:32:45.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.068+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a60039040 con 0x7f5a6815b7d0 2024-04-25T08:32:45.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.069+0000 7f5a5e7fc640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.069+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a600aba70 con 0x7f5a6815b7d0 2024-04-25T08:32:45.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.069+0000 7f5a6d9f6640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.069+0000 7f5a6d9f6640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006fd0 tx=0x7f5a64008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.073+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a6007a1a0 con 0x7f5a6815b7d0 2024-04-25T08:32:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.264+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5a68160120 con 0x7f5a6815b7d0 2024-04-25T08:32:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.264+0000 7f5a5e7fc640 1 -- 172.21.15.60:0/667843510 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5a6007e240 con 0x7f5a6815b7d0 2024-04-25T08:32:45.260 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:32:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 msgr2=0x7f5a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006fd0 tx=0x7f5a64008040 comp rx=0 tx=0).stop 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 msgr2=0x7f5a680b8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 0x7f5a680b8120 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009eb0 tx=0x7f5a60004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 shutdown_connections 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6815b7d0 0x7f5a680b8120 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a680b5620 0x7f5a681f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 --2- 172.21.15.60:0/667843510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6806c9e0 0x7f5a681f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 >> 172.21.15.60:0/667843510 conn(0x7f5a680d0800 msgr2=0x7f5a68160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 shutdown_connections 2024-04-25T08:32:45.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:45.266+0000 7f5a6fc5a640 1 -- 172.21.15.60:0/667843510 wait complete. 2024-04-25T08:32:45.273 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:32:45.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:45.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:46.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:32:46.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:46.685 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:32:46.700 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:32:46.700 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:46.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:32:46.718 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:46.718 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:46.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:32:46.756 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:32:46.756 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:46.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:32:46.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.901+0000 7f29cc7e0640 1 Processor -- start 2024-04-25T08:32:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.901+0000 7f29cc7e0640 1 -- start start 2024-04-25T08:32:46.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cc7e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c415dd30 0x7f29c40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cc7e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f29c40d0ad0 0x7f29c40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cc7e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c40d1df0 con 0x7f29c40b9770 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cc7e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40d20d0 con 0x7f29c40d0ad0 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cc7e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d23b0 con 0x7f29c415dd30 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29c9d7b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f29c40b9770 0x7f29c40d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39678/0 (socket says 172.21.15.60:39678) 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cad7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f29c40d0ad0 0x7f29c40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29c9d7b640 1 -- 172.21.15.60:0/1456326013 learned_addr learned my addr 172.21.15.60:0/1456326013 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29ca57c640 1 --2- 172.21.15.60:0/1456326013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c415dd30 0x7f29c40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cad7d640 1 -- 172.21.15.60:0/1456326013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c415dd30 msgr2=0x7f29c40b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cad7d640 1 --2- 172.21.15.60:0/1456326013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c415dd30 0x7f29c40b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cad7d640 1 -- 172.21.15.60:0/1456326013 >> v1:172.21.15.60:6789/0 conn(0x7f29c40b9770 legacy=0x7f29c40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29cad7d640 1 -- 172.21.15.60:0/1456326013 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40d2690 con 0x7f29c40d0ad0 2024-04-25T08:32:46.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.902+0000 7f29c957a640 1 -- 172.21.15.60:0/1456326013 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827228354 0 0) 0x7f29c40d1df0 con 0x7f29c40b9770 2024-04-25T08:32:46.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.903+0000 7f29cad7d640 1 --2- 172.21.15.60:0/1456326013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c40d0ad0 0x7f29c40b6a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009d90 tx=0x7f29bc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea380fa7643aff4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.903+0000 7f29c957a640 1 -- 172.21.15.60:0/1456326013 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc031600 con 0x7f29c40d0ad0 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.903+0000 7f29c957a640 1 -- 172.21.15.60:0/1456326013 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc0317a0 con 0x7f29c40d0ad0 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.903+0000 7f29c957a640 1 -- 172.21.15.60:0/1456326013 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc031b70 con 0x7f29c40d0ad0 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 -- 172.21.15.60:0/1456326013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c40d0ad0 msgr2=0x7f29c40b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/1456326013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c40d0ad0 0x7f29c40b6a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009d90 tx=0x7f29bc02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 -- 172.21.15.60:0/1456326013 shutdown_connections 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/1456326013 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c40d0ad0 0x7f29c40b6a40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/1456326013 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c415dd30 0x7f29c40b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 -- 172.21.15.60:0/1456326013 >> 172.21.15.60:0/1456326013 conn(0x7f29c4061c30 msgr2=0x7f29c406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 -- 172.21.15.60:0/1456326013 shutdown_connections 2024-04-25T08:32:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.904+0000 7f29cc7e0640 1 -- 172.21.15.60:0/1456326013 wait complete. 2024-04-25T08:32:46.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.905+0000 7f29cc7e0640 1 Processor -- start 2024-04-25T08:32:46.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.905+0000 7f29cc7e0640 1 -- start start 2024-04-25T08:32:46.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cc7e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29c40b9770 0x7f29c41582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cc7e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c40d0ad0 0x7f29c4159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cc7e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 0x7f29c40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cc7e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d1b90 con 0x7f29c415dd30 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cc7e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d1a10 con 0x7f29c40b9770 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cc7e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40d2e90 con 0x7f29c40d0ad0 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 0x7f29c40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 0x7f29c40b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42164/0 (socket says 172.21.15.60:42164) 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 -- 172.21.15.60:0/4261973140 learned_addr learned my addr 172.21.15.60:0/4261973140 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29c9d7b640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c40d0ad0 0x7f29c4159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29ca57c640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29c40b9770 0x7f29c41582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 -- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c40d0ad0 msgr2=0x7f29c4159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c40d0ad0 0x7f29c4159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 -- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29c40b9770 msgr2=0x7f29c41582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29c40b9770 0x7f29c41582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29cad7d640 1 -- 172.21.15.60:0/4261973140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4158d40 con 0x7f29c415dd30 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.906+0000 7f29ca57c640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29c40b9770 0x7f29c41582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:46.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.907+0000 7f29cad7d640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 0x7f29c40b3150 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009eb0 tx=0x7f29bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.907+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc040070 con 0x7f29c415dd30 2024-04-25T08:32:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.907+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc030b60 con 0x7f29c415dd30 2024-04-25T08:32:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.907+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c4159020 con 0x7f29c415dd30 2024-04-25T08:32:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.907+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc009450 con 0x7f29c415dd30 2024-04-25T08:32:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.907+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c41595e0 con 0x7f29c415dd30 2024-04-25T08:32:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.908+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c4073990 con 0x7f29c415dd30 2024-04-25T08:32:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.908+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29bc039040 con 0x7f29c415dd30 2024-04-25T08:32:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.909+0000 7f29baffd640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29a806b4d0 0x7f29a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.909+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29bc0ab880 con 0x7f29c415dd30 2024-04-25T08:32:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.909+0000 7f29ca57c640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29a806b4d0 0x7f29a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.909+0000 7f29ca57c640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29a806b4d0 0x7f29a806d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f29c40ce0b0 tx=0x7f29c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:46.913+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29bc07a720 con 0x7f29c415dd30 2024-04-25T08:32:47.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.137+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c4160790 con 0x7f29c415dd30 2024-04-25T08:32:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.137+0000 7f29baffd640 1 -- 172.21.15.60:0/4261973140 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f29bc07a020 con 0x7f29c415dd30 2024-04-25T08:32:47.134 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:47.134 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:47.022299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:47.134 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-04-25T08:32:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29a806b4d0 msgr2=0x7f29a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29a806b4d0 0x7f29a806d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f29c40ce0b0 tx=0x7f29c0008040 comp rx=0 tx=0).stop 2024-04-25T08:32:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 msgr2=0x7f29c40b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 0x7f29c40b3150 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009eb0 tx=0x7f29bc004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 shutdown_connections 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29c415dd30 0x7f29c40b3150 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f29c40d0ad0 0x7f29c4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29a806b4d0 0x7f29a806d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 --2- 172.21.15.60:0/4261973140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29c40b9770 0x7f29c41582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 >> 172.21.15.60:0/4261973140 conn(0x7f29c4061c30 msgr2=0x7f29c415cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 shutdown_connections 2024-04-25T08:32:47.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.140+0000 7f29cc7e0640 1 -- 172.21.15.60:0/4261973140 wait complete. 2024-04-25T08:32:47.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:47.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.251+0000 7fa05f25a640 1 Processor -- start 2024-04-25T08:32:47.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.251+0000 7fa05f25a640 1 -- start start 2024-04-25T08:32:47.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05f25a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa05806b1a0 0x7fa05806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05f25a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa05806b580 0x7fa0580cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05f25a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa058161960 con 0x7fa0580cffa0 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05f25a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0580d0430 con 0x7fa05806b1a0 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05f25a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0580cbd50 con 0x7fa05806b580 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05cff6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa05806b1a0 0x7fa05806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05d7f7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa0580cffa0 0x7fa05806e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39684/0 (socket says 172.21.15.60:39684) 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa05d7f7640 1 -- 172.21.15.60:0/4082666725 learned_addr learned my addr 172.21.15.60:0/4082666725 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.252+0000 7fa057fff640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa05806b580 0x7fa0580cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa05cff6640 1 -- 172.21.15.60:0/4082666725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa05806b580 msgr2=0x7fa0580cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa05cff6640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa05806b580 0x7fa0580cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa05cff6640 1 -- 172.21.15.60:0/4082666725 >> v1:172.21.15.60:6789/0 conn(0x7fa0580cffa0 legacy=0x7fa05806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa05cff6640 1 -- 172.21.15.60:0/4082666725 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058161c40 con 0x7fa05806b1a0 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa0577fe640 1 -- 172.21.15.60:0/4082666725 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295069596 0 0) 0x7fa058161960 con 0x7fa0580cffa0 2024-04-25T08:32:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa057fff640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa05806b580 0x7fa0580cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa05cff6640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b1a0 0x7fa05806f9c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa0480097a0 tx=0x7fa04802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8464818a3359633c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa0577fe640 1 -- 172.21.15.60:0/4082666725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048037070 con 0x7fa05806b1a0 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa0577fe640 1 -- 172.21.15.60:0/4082666725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04802ae50 con 0x7fa05806b1a0 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.253+0000 7fa0577fe640 1 -- 172.21.15.60:0/4082666725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048032660 con 0x7fa05806b1a0 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 -- 172.21.15.60:0/4082666725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b1a0 msgr2=0x7fa05806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b1a0 0x7fa05806f9c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa0480097a0 tx=0x7fa04802a100 comp rx=0 tx=0).stop 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 -- 172.21.15.60:0/4082666725 shutdown_connections 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa05806b580 0x7fa0580cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 --2- 172.21.15.60:0/4082666725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b1a0 0x7fa05806f9c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 -- 172.21.15.60:0/4082666725 >> 172.21.15.60:0/4082666725 conn(0x7fa0580b2220 msgr2=0x7fa0580b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 -- 172.21.15.60:0/4082666725 shutdown_connections 2024-04-25T08:32:47.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.254+0000 7fa05f25a640 1 -- 172.21.15.60:0/4082666725 wait complete. 2024-04-25T08:32:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.255+0000 7fa05f25a640 1 Processor -- start 2024-04-25T08:32:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.255+0000 7fa05f25a640 1 -- start start 2024-04-25T08:32:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa05f25a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa05806b1a0 0x7fa0580c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa05f25a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 0x7fa058150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa05f25a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0580cffa0 0x7fa0580b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa05f25a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058162720 con 0x7fa05806b580 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa05f25a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058162200 con 0x7fa05806b1a0 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa05f25a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa058070330 con 0x7fa0580cffa0 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa057fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 0x7fa058150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa057fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 0x7fa058150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42174/0 (socket says 172.21.15.60:42174) 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.256+0000 7fa057fff640 1 -- 172.21.15.60:0/3918054463 learned_addr learned my addr 172.21.15.60:0/3918054463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa05d7f7640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0580cffa0 0x7fa0580b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa057fff640 1 -- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0580cffa0 msgr2=0x7fa0580b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa057fff640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0580cffa0 0x7fa0580b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa05cff6640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa05806b1a0 0x7fa0580c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa057fff640 1 -- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa05806b1a0 msgr2=0x7fa0580c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa057fff640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa05806b1a0 0x7fa0580c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa057fff640 1 -- 172.21.15.60:0/3918054463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580c4ee0 con 0x7fa05806b580 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa05d7f7640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0580cffa0 0x7fa0580b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa057fff640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 0x7fa058150c90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e7e0 tx=0x7fa04c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00f040 con 0x7fa05806b580 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c004030 con 0x7fa05806b580 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0580c51c0 con 0x7fa05806b580 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c004400 con 0x7fa05806b580 2024-04-25T08:32:47.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.257+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0580c5780 con 0x7fa05806b580 2024-04-25T08:32:47.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.258+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa058073990 con 0x7fa05806b580 2024-04-25T08:32:47.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.259+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa04c00c6d0 con 0x7fa05806b580 2024-04-25T08:32:47.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.259+0000 7fa0557fa640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa04406b4d0 0x7fa04406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.259+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa04c08a1a0 con 0x7fa05806b580 2024-04-25T08:32:47.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.259+0000 7fa05cff6640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa04406b4d0 0x7fa04406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.260+0000 7fa05cff6640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa04406b4d0 0x7fa04406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa048034000 tx=0x7fa04802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.263+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa04c017020 con 0x7fa05806b580 2024-04-25T08:32:47.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.444+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0580cc250 con 0x7fa05806b580 2024-04-25T08:32:47.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.446+0000 7fa0557fa640 1 -- 172.21.15.60:0/3918054463 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fa04c058e90 con 0x7fa05806b580 2024-04-25T08:32:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:47.442 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:15.989621+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:47.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.447+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa04406b4d0 msgr2=0x7fa04406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.447+0000 7fa05f25a640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa04406b4d0 0x7fa04406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa048034000 tx=0x7fa04802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.447+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 msgr2=0x7fa058150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.447+0000 7fa05f25a640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 0x7fa058150c90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e7e0 tx=0x7fa04c00eca0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 shutdown_connections 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0580cffa0 0x7fa0580b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa05806b580 0x7fa058150c90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa04406b4d0 0x7fa04406d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 --2- 172.21.15.60:0/3918054463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa05806b1a0 0x7fa0580c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 >> 172.21.15.60:0/3918054463 conn(0x7fa0580b2220 msgr2=0x7fa0580ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 shutdown_connections 2024-04-25T08:32:47.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.448+0000 7fa05f25a640 1 -- 172.21.15.60:0/3918054463 wait complete. 2024-04-25T08:32:47.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:32:12.634751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:32:13.328038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:32:47.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:47.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.563+0000 7f5c5d676640 1 Processor -- start 2024-04-25T08:32:47.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.563+0000 7f5c5d676640 1 -- start start 2024-04-25T08:32:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.563+0000 7f5c5d676640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c580b5c40 0x7f5c580ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c5d676640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580cf320 con 0x7f5c5815b5a0 2024-04-25T08:32:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c5d676640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580cf040 con 0x7f5c5815b980 2024-04-25T08:32:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c5d676640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c58161970 con 0x7f5c580b5c40 2024-04-25T08:32:47.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c567fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5c580b5c40 0x7f5c580ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c56ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5c5815b5a0 0x7f5c580b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39686/0 (socket says 172.21.15.60:39686) 2024-04-25T08:32:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c56ffd640 1 -- 172.21.15.60:0/507833249 learned_addr learned my addr 172.21.15.60:0/507833249 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c567fc640 1 -- 172.21.15.60:0/507833249 >> v1:172.21.15.100:6789/0 conn(0x7f5c5815b980 legacy=0x7f5c580ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c567fc640 1 -- 172.21.15.60:0/507833249 >> v1:172.21.15.60:6789/0 conn(0x7f5c5815b5a0 legacy=0x7f5c580b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c567fc640 1 -- 172.21.15.60:0/507833249 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c58161c50 con 0x7f5c580b5c40 2024-04-25T08:32:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.564+0000 7f5c55ffb640 1 -- 172.21.15.60:0/507833249 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211561137 0 0) 0x7f5c48001110 con 0x7f5c5815b5a0 2024-04-25T08:32:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c567fc640 1 --2- 172.21.15.60:0/507833249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c580b5c40 0x7f5c580ce310 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009790 tx=0x7f5c4c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297031fc6403564b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c55ffb640 1 -- 172.21.15.60:0/507833249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037070 con 0x7f5c580b5c40 2024-04-25T08:32:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c55ffb640 1 -- 172.21.15.60:0/507833249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c032440 con 0x7f5c580b5c40 2024-04-25T08:32:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c55ffb640 1 -- 172.21.15.60:0/507833249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c032810 con 0x7f5c580b5c40 2024-04-25T08:32:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c5d676640 1 -- 172.21.15.60:0/507833249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c580b5c40 msgr2=0x7f5c580ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c5d676640 1 --2- 172.21.15.60:0/507833249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c580b5c40 0x7f5c580ce310 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009790 tx=0x7f5c4c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c5d676640 1 -- 172.21.15.60:0/507833249 shutdown_connections 2024-04-25T08:32:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c5d676640 1 --2- 172.21.15.60:0/507833249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c580b5c40 0x7f5c580ce310 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c5d676640 1 -- 172.21.15.60:0/507833249 >> 172.21.15.60:0/507833249 conn(0x7f5c580d0530 msgr2=0x7f5c580d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.565+0000 7f5c5d676640 1 -- 172.21.15.60:0/507833249 shutdown_connections 2024-04-25T08:32:47.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.566+0000 7f5c5d676640 1 -- 172.21.15.60:0/507833249 wait complete. 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 Processor -- start 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 -- start start 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c580b5c40 0x7f5c5814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 0x7f5c581600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c5815b980 0x7f5c5815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c581625d0 con 0x7f5c5815b5a0 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58162450 con 0x7f5c5815b980 2024-04-25T08:32:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.567+0000 7f5c5d676640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58162750 con 0x7f5c580b5c40 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 0x7f5c581600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 0x7f5c581600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42200/0 (socket says 172.21.15.60:42200) 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 -- 172.21.15.60:0/3882873515 learned_addr learned my addr 172.21.15.60:0/3882873515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c56ffd640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c580b5c40 0x7f5c5814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c577fe640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c5815b980 0x7f5c5815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 -- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c580b5c40 msgr2=0x7f5c5814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c580b5c40 0x7f5c5814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 -- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c5815b980 msgr2=0x7f5c5815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c5815b980 0x7f5c5815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 -- 172.21.15.60:0/3882873515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c581f4b60 con 0x7f5c5815b5a0 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c577fe640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c5815b980 0x7f5c5815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c56ffd640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c580b5c40 0x7f5c5814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c567fc640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 0x7f5c581600f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c035040 tx=0x7f5c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c041020 con 0x7f5c5815b5a0 2024-04-25T08:32:47.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c005da0 con 0x7f5c5815b5a0 2024-04-25T08:32:47.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c037040 con 0x7f5c5815b5a0 2024-04-25T08:32:47.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c581f4f60 con 0x7f5c5815b5a0 2024-04-25T08:32:47.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.568+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c581f5520 con 0x7f5c5815b5a0 2024-04-25T08:32:47.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.569+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c58073a00 con 0x7f5c5815b5a0 2024-04-25T08:32:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.570+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c4c044020 con 0x7f5c5815b5a0 2024-04-25T08:32:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.570+0000 7f5c3f7fe640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c4406b4d0 0x7f5c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.570+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c4c0aaa30 con 0x7f5c5815b5a0 2024-04-25T08:32:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.570+0000 7f5c56ffd640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c4406b4d0 0x7f5c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.571+0000 7f5c56ffd640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c4406b4d0 0x7f5c4406d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c48000f60 tx=0x7f5c48007a30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.575+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c4c04b030 con 0x7f5c5815b5a0 2024-04-25T08:32:47.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.757+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c580d17c0 con 0x7f5c5815b5a0 2024-04-25T08:32:47.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.757+0000 7f5c3f7fe640 1 -- 172.21.15.60:0/3882873515 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c4c079720 con 0x7f5c5815b5a0 2024-04-25T08:32:47.753 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c4406b4d0 msgr2=0x7f5c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c4406b4d0 0x7f5c4406d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c48000f60 tx=0x7f5c48007a30 comp rx=0 tx=0).stop 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 msgr2=0x7f5c581600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 0x7f5c581600f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c035040 tx=0x7f5c4c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 shutdown_connections 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c4406b4d0 0x7f5c4406d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c5815b980 0x7f5c5815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5c5815b5a0 0x7f5c581600f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 --2- 172.21.15.60:0/3882873515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c580b5c40 0x7f5c5814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 >> 172.21.15.60:0/3882873515 conn(0x7f5c580d0530 msgr2=0x7f5c58158450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 shutdown_connections 2024-04-25T08:32:47.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.759+0000 7f5c5d676640 1 -- 172.21.15.60:0/3882873515 wait complete. 2024-04-25T08:32:47.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.873+0000 7f6637b3b640 1 Processor -- start 2024-04-25T08:32:47.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f6637b3b640 1 -- start start 2024-04-25T08:32:47.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f6637b3b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66300cf600 0x7f66300cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f6637b3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 0x7f66300cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f6637b3b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66300c75e0 con 0x7f663015bb50 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f6637b3b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66300c8680 con 0x7f66300cf600 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f6637b3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66300cff80 con 0x7f66300cbf20 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f66358d7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f663015bb50 0x7f66300c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39694/0 (socket says 172.21.15.60:39694) 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.874+0000 7f66358d7640 1 -- 172.21.15.60:0/3578235099 learned_addr learned my addr 172.21.15.60:0/3578235099 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66350d6640 1 --2- 172.21.15.60:0/3578235099 >> v2:172.21.15.100:3300/0 conn(0x7f66300cf600 0x7f66300cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66360d8640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 0x7f66300cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889030690 0 0) 0x7f66300c75e0 con 0x7f663015bb50 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6618003520 con 0x7f663015bb50 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1599871029 0 0) 0x7f6618003520 con 0x7f663015bb50 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 msgr2=0x7f66300cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 0x7f66300cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 >> v2:172.21.15.100:3300/0 conn(0x7f66300cf600 msgr2=0x7f66300cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 --2- 172.21.15.60:0/3578235099 >> v2:172.21.15.100:3300/0 conn(0x7f66300cf600 0x7f66300cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300c78c0 con 0x7f663015bb50 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6628001670 con 0x7f663015bb50 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 >> v1:172.21.15.60:6789/0 conn(0x7f663015bb50 legacy=0x7f66300c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 0x7f66180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6618003f30 0x7f6618004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6618004d40 0x7f66180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66300c8680 con 0x7f6618003f30 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66300cff80 con 0x7f6618004d40 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661800b9f0 con 0x7f66300cbf20 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66350d6640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6618003f30 0x7f6618004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66358d7640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 0x7f66180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.875+0000 7f66360d8640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6618004d40 0x7f66180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66350d6640 1 -- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 msgr2=0x7f66180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66350d6640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66300cbf20 0x7f66180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66350d6640 1 -- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6618004d40 msgr2=0x7f66180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66350d6640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6618004d40 0x7f66180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66350d6640 1 -- 172.21.15.60:0/3578235099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618003520 con 0x7f6618003f30 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66350d6640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6618003f30 0x7f6618004800 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6624009ac0 tx=0x7f6618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc60283248609694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624036070 con 0x7f6618003f30 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624031400 con 0x7f6618003f30 2024-04-25T08:32:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.876+0000 7f66348d5640 1 -- 172.21.15.60:0/3578235099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624009690 con 0x7f6618003f30 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f6637b3b640 1 -- 172.21.15.60:0/3578235099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6618003f30 msgr2=0x7f6618004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f6637b3b640 1 --2- 172.21.15.60:0/3578235099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6618003f30 0x7f6618004800 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6624009ac0 tx=0x7f6618005830 comp rx=0 tx=0).stop 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f66360d8640 1 -- 172.21.15.60:0/3578235099 reap_dead start 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f6637b3b640 1 -- 172.21.15.60:0/3578235099 shutdown_connections 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f6637b3b640 1 -- 172.21.15.60:0/3578235099 >> 172.21.15.60:0/3578235099 conn(0x7f663006f500 msgr2=0x7f663006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f6637b3b640 1 -- 172.21.15.60:0/3578235099 shutdown_connections 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.877+0000 7f6637b3b640 1 -- 172.21.15.60:0/3578235099 wait complete. 2024-04-25T08:32:47.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.878+0000 7f6637b3b640 1 Processor -- start 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.878+0000 7f6637b3b640 1 -- start start 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f6637b3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6618003f30 0x7f66301473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f6637b3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 0x7f66300cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f6637b3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66300c88f0 0x7f66300c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f6637b3b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66300c8160 con 0x7f66300cd380 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f6637b3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66300c7e80 con 0x7f66300c88f0 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f6637b3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66300c72a0 con 0x7f6618003f30 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 0x7f66300cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 0x7f66300cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42212/0 (socket says 172.21.15.60:42212) 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 -- 172.21.15.60:0/1197894463 learned_addr learned my addr 172.21.15.60:0/1197894463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66358d7640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6618003f30 0x7f66301473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66360d8640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66300c88f0 0x7f66300c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 -- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6618003f30 msgr2=0x7f66301473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6618003f30 0x7f66301473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 -- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66300c88f0 msgr2=0x7f66300c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66300c88f0 0x7f66300c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66350d6640 1 -- 172.21.15.60:0/1197894463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300c92b0 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.879+0000 7f66360d8640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66300c88f0 0x7f66300c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.880+0000 7f66350d6640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 0x7f66300cd7e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6624000c00 tx=0x7f6624004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.880+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624040020 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.880+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624003c60 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.880+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66300c9570 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.880+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624036040 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.880+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66300c9b30 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.881+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663015ac80 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.881+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6624043020 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.882+0000 7f662e7fc640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661c06b500 0x7f661c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.882+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66240aa550 con 0x7f66300cd380 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.882+0000 7f66358d7640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661c06b500 0x7f661c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.882+0000 7f66358d7640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661c06b500 0x7f661c06d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f66280078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:47.886+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f663015ac80 con 0x7f66300cd380 2024-04-25T08:32:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.064+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6630073a00 con 0x7f66300cd380 2024-04-25T08:32:48.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.064+0000 7f662e7fc640 1 -- 172.21.15.60:0/1197894463 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f662404a070 con 0x7f66300cd380 2024-04-25T08:32:48.060 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:32:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661c06b500 msgr2=0x7f661c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661c06b500 0x7f661c06d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f66280078f0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 msgr2=0x7f66300cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 0x7f66300cd7e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6624000c00 tx=0x7f6624004730 comp rx=0 tx=0).stop 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 shutdown_connections 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661c06b500 0x7f661c06d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66300c88f0 0x7f66300c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66300cd380 0x7f66300cd7e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 --2- 172.21.15.60:0/1197894463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6618003f30 0x7f66301473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 >> 172.21.15.60:0/1197894463 conn(0x7f663006f500 msgr2=0x7f66300b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 shutdown_connections 2024-04-25T08:32:48.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.066+0000 7f6637b3b640 1 -- 172.21.15.60:0/1197894463 wait complete. 2024-04-25T08:32:48.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:48.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.181+0000 7f7e9aae1640 1 Processor -- start 2024-04-25T08:32:48.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.181+0000 7f7e9aae1640 1 -- start start 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e9aae1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7e9406c9e0 0x7f7e940b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e9aae1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9406cf50 con 0x7f7e940b5620 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e9aae1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e940ce530 con 0x7f7e9415b7d0 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e9aae1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e940ce810 con 0x7f7e9406c9e0 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e93fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7e940b5620 0x7f7e9406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39710/0 (socket says 172.21.15.60:39710) 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e93fff640 1 -- 172.21.15.60:0/3869793491 learned_addr learned my addr 172.21.15.60:0/3869793491 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e9907e640 1 --2- 172.21.15.60:0/3869793491 >> v2:172.21.15.100:3300/0 conn(0x7f7e9406c9e0 0x7f7e940b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934428476 0 0) 0x7f7e9406cf50 con 0x7f7e940b5620 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e84003520 con 0x7f7e940b5620 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3301370857 0 0) 0x7f7e84003520 con 0x7f7e940b5620 2024-04-25T08:32:48.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 >> v2:172.21.15.100:3300/0 conn(0x7f7e9406c9e0 msgr2=0x7f7e940b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 --2- 172.21.15.60:0/3869793491 >> v2:172.21.15.100:3300/0 conn(0x7f7e9406c9e0 0x7f7e940b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 >> v1:172.21.15.100:6789/0 conn(0x7f7e9415b7d0 legacy=0x7f7e940cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e940ceaf0 con 0x7f7e940b5620 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.182+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f7e80001670 con 0x7f7e940b5620 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 >> v1:172.21.15.60:6789/0 conn(0x7f7e940b5620 legacy=0x7f7e9406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e840039b0 0x7f7e84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e84004800 0x7f7e84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e840051c0 0x7f7e8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940ce810 con 0x7f7e84004800 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8400c060 con 0x7f7e840051c0 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8400c260 con 0x7f7e840039b0 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e93fff640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e84004800 0x7f7e84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e9887d640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e840039b0 0x7f7e84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e9907e640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e840051c0 0x7f7e8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e93fff640 1 -- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e840039b0 msgr2=0x7f7e84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e93fff640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e840039b0 0x7f7e84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e93fff640 1 -- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e840051c0 msgr2=0x7f7e8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e93fff640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e840051c0 0x7f7e8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e93fff640 1 -- 172.21.15.60:0/3869793491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84003520 con 0x7f7e84004800 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.183+0000 7f7e9907e640 1 -- 172.21.15.60:0/3869793491 reap_dead start 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e93fff640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e84004800 0x7f7e84004c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c240 tx=0x7f7e84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6f2d3f664bf94b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80007cf0 con 0x7f7e84004800 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80005ad0 con 0x7f7e84004800 2024-04-25T08:32:48.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e937fe640 1 -- 172.21.15.60:0/3869793491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80007070 con 0x7f7e84004800 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e9aae1640 1 -- 172.21.15.60:0/3869793491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e84004800 msgr2=0x7f7e84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e84004800 0x7f7e84004c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c240 tx=0x7f7e84005830 comp rx=0 tx=0).stop 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e9aae1640 1 -- 172.21.15.60:0/3869793491 shutdown_connections 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/3869793491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e84004800 0x7f7e84004c80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e9aae1640 1 -- 172.21.15.60:0/3869793491 >> 172.21.15.60:0/3869793491 conn(0x7f7e940d0800 msgr2=0x7f7e940d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.184+0000 7f7e9aae1640 1 -- 172.21.15.60:0/3869793491 shutdown_connections 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.185+0000 7f7e9aae1640 1 -- 172.21.15.60:0/3869793491 wait complete. 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.186+0000 7f7e9aae1640 1 Processor -- start 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.186+0000 7f7e9aae1640 1 -- start start 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.186+0000 7f7e9aae1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e84004800 0x7f7e941f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.186+0000 7f7e9aae1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 0x7f7e941f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9aae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e940b82b0 0x7f7e940b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9aae1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415bd30 con 0x7f7e9415a030 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9aae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9415bbb0 con 0x7f7e940b82b0 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9aae1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e940b5ab0 con 0x7f7e84004800 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 0x7f7e941f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 0x7f7e941f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42228/0 (socket says 172.21.15.60:42228) 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 -- 172.21.15.60:0/609109848 learned_addr learned my addr 172.21.15.60:0/609109848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9907e640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e940b82b0 0x7f7e940b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9887d640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e84004800 0x7f7e941f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 -- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e84004800 msgr2=0x7f7e941f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e84004800 0x7f7e941f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 -- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e940b82b0 msgr2=0x7f7e940b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e940b82b0 0x7f7e940b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 -- 172.21.15.60:0/609109848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e941f46d0 con 0x7f7e9415a030 2024-04-25T08:32:48.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e93fff640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 0x7f7e941f4190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c360 tx=0x7f7e940cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80037040 con 0x7f7e9415a030 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80001470 con 0x7f7e9415a030 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800047e0 con 0x7f7e9415a030 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e940caf60 con 0x7f7e9415a030 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.187+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e940cb520 con 0x7f7e9415a030 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.188+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e94073a00 con 0x7f7e9415a030 2024-04-25T08:32:48.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.189+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e80043030 con 0x7f7e9415a030 2024-04-25T08:32:48.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.189+0000 7f7e917fa640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e7806b500 0x7f7e7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.190+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e800a96f0 con 0x7f7e9415a030 2024-04-25T08:32:48.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.190+0000 7f7e9887d640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e7806b500 0x7f7e7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.190+0000 7f7e9887d640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e7806b500 0x7f7e7806d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000c00 tx=0x7f7e88009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.194+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e8004a020 con 0x7f7e9415a030 2024-04-25T08:32:48.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.373+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e9415ed90 con 0x7f7e9415a030 2024-04-25T08:32:48.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.374+0000 7f7e917fa640 1 -- 172.21.15.60:0/609109848 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e941f46d0 con 0x7f7e9415a030 2024-04-25T08:32:48.370 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.375+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e7806b500 msgr2=0x7f7e7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.375+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e7806b500 0x7f7e7806d9c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7e88000c00 tx=0x7f7e88009040 comp rx=0 tx=0).stop 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.375+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 msgr2=0x7f7e941f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.375+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 0x7f7e941f4190 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c360 tx=0x7f7e940cecc0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 shutdown_connections 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7e7806b500 0x7f7e7806d9c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e940b82b0 0x7f7e940b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e9415a030 0x7f7e941f4190 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 --2- 172.21.15.60:0/609109848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7e84004800 0x7f7e941f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 >> 172.21.15.60:0/609109848 conn(0x7f7e940d0800 msgr2=0x7f7e940b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 shutdown_connections 2024-04-25T08:32:48.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.376+0000 7f7e9aae1640 1 -- 172.21.15.60:0/609109848 wait complete. 2024-04-25T08:32:48.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.488+0000 7f9e3a86a640 1 Processor -- start 2024-04-25T08:32:48.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.488+0000 7f9e3a86a640 1 -- start start 2024-04-25T08:32:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e3a86a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9e3415dbe0 0x7f9e340d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e3a86a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e340b6220 con 0x7f9e340b2cb0 2024-04-25T08:32:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e3a86a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e340b97a0 con 0x7f9e340b92a0 2024-04-25T08:32:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e3a86a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3415dfc0 con 0x7f9e3415dbe0 2024-04-25T08:32:48.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e33fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9e3415dbe0 0x7f9e340d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e337fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9e340b2cb0 0x7f9e340b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39718/0 (socket says 172.21.15.60:39718) 2024-04-25T08:32:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e337fe640 1 -- 172.21.15.60:0/661959046 learned_addr learned my addr 172.21.15.60:0/661959046 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e33fff640 1 -- 172.21.15.60:0/661959046 >> v1:172.21.15.100:6789/0 conn(0x7f9e340b92a0 legacy=0x7f9e340b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e33fff640 1 -- 172.21.15.60:0/661959046 >> v1:172.21.15.60:6789/0 conn(0x7f9e340b2cb0 legacy=0x7f9e340b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e33fff640 1 -- 172.21.15.60:0/661959046 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340d1c50 con 0x7f9e3415dbe0 2024-04-25T08:32:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.489+0000 7f9e32ffd640 1 -- 172.21.15.60:0/661959046 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778698300 0 0) 0x7f9e340b6220 con 0x7f9e340b2cb0 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.490+0000 7f9e33fff640 1 --2- 172.21.15.60:0/661959046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e3415dbe0 0x7f9e340d0910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9e240097a0 tx=0x7f9e2402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cabba23a9e5a010 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.490+0000 7f9e32ffd640 1 -- 172.21.15.60:0/661959046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24037070 con 0x7f9e3415dbe0 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.490+0000 7f9e32ffd640 1 -- 172.21.15.60:0/661959046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2402ae50 con 0x7f9e3415dbe0 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.490+0000 7f9e32ffd640 1 -- 172.21.15.60:0/661959046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24032660 con 0x7f9e3415dbe0 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.490+0000 7f9e3a86a640 1 -- 172.21.15.60:0/661959046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e3415dbe0 msgr2=0x7f9e340d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.490+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/661959046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e3415dbe0 0x7f9e340d0910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9e240097a0 tx=0x7f9e2402a140 comp rx=0 tx=0).stop 2024-04-25T08:32:48.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.491+0000 7f9e3a86a640 1 -- 172.21.15.60:0/661959046 shutdown_connections 2024-04-25T08:32:48.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.491+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/661959046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e3415dbe0 0x7f9e340d0910 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.491+0000 7f9e3a86a640 1 -- 172.21.15.60:0/661959046 >> 172.21.15.60:0/661959046 conn(0x7f9e34061a20 msgr2=0x7f9e34061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.491+0000 7f9e3a86a640 1 -- 172.21.15.60:0/661959046 shutdown_connections 2024-04-25T08:32:48.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.491+0000 7f9e3a86a640 1 -- 172.21.15.60:0/661959046 wait complete. 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.492+0000 7f9e3a86a640 1 Processor -- start 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.492+0000 7f9e3a86a640 1 -- start start 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e340b2cb0 0x7f9e341539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 0x7f9e340cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e3415dbe0 0x7f9e34156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340d2210 con 0x7f9e340b92a0 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340d1a00 con 0x7f9e3415dbe0 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340b6ab0 con 0x7f9e340b2cb0 2024-04-25T08:32:48.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 0x7f9e340cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 0x7f9e340cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42248/0 (socket says 172.21.15.60:42248) 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 -- 172.21.15.60:0/2473644694 learned_addr learned my addr 172.21.15.60:0/2473644694 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e38e07640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e3415dbe0 0x7f9e34156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e33fff640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e340b2cb0 0x7f9e341539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 -- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e340b2cb0 msgr2=0x7f9e341539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e340b2cb0 0x7f9e341539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 -- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e3415dbe0 msgr2=0x7f9e34156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e3415dbe0 0x7f9e34156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 -- 172.21.15.60:0/2473644694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340cbfe0 con 0x7f9e340b92a0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e337fe640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 0x7f9e340cd360 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c4b0 tx=0x7f9e1c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c018070 con 0x7f9e340b92a0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c007500 con 0x7f9e340b92a0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c004a20 con 0x7f9e340b92a0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e340cc2c0 con 0x7f9e340b92a0 2024-04-25T08:32:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.493+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e340cc880 con 0x7f9e340b92a0 2024-04-25T08:32:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.495+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e34073990 con 0x7f9e340b92a0 2024-04-25T08:32:48.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.495+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e1c004bc0 con 0x7f9e340b92a0 2024-04-25T08:32:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.495+0000 7f9e30ff9640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9e1406b500 0x7f9e1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.496+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e1c089600 con 0x7f9e340b92a0 2024-04-25T08:32:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.496+0000 7f9e33fff640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9e1406b500 0x7f9e1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.496+0000 7f9e33fff640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9e1406b500 0x7f9e1406d9c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9e24034000 tx=0x7f9e2402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.500+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e1c0583b0 con 0x7f9e340b92a0 2024-04-25T08:32:48.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.718+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e341552d0 con 0x7f9e340b92a0 2024-04-25T08:32:48.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.718+0000 7f9e30ff9640 1 -- 172.21.15.60:0/2473644694 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f9e1c057cb0 con 0x7f9e340b92a0 2024-04-25T08:32:48.714 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:48.714 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:47.022299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:48.715 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-04-25T08:32:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9e1406b500 msgr2=0x7f9e1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9e1406b500 0x7f9e1406d9c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9e24034000 tx=0x7f9e2402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 msgr2=0x7f9e340cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 0x7f9e340cd360 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c4b0 tx=0x7f9e1c00c970 comp rx=0 tx=0).stop 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 shutdown_connections 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9e1406b500 0x7f9e1406d9c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9e3415dbe0 0x7f9e34156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e340b92a0 0x7f9e340cd360 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 --2- 172.21.15.60:0/2473644694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9e340b2cb0 0x7f9e341539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 >> 172.21.15.60:0/2473644694 conn(0x7f9e34061a20 msgr2=0x7f9e340d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 shutdown_connections 2024-04-25T08:32:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.720+0000 7f9e3a86a640 1 -- 172.21.15.60:0/2473644694 wait complete. 2024-04-25T08:32:48.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:48.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.831+0000 7f778eecb640 1 Processor -- start 2024-04-25T08:32:48.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.831+0000 7f778eecb640 1 -- start start 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778eecb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f778815dbe0 0x7f77880d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778eecb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f77880b2cb0 0x7f77880d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778eecb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77880b97a0 con 0x7f77880b92a0 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778eecb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778815dfc0 con 0x7f778815dbe0 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778eecb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77880d1a00 con 0x7f77880b2cb0 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778cc67640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f778815dbe0 0x7f77880d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778cc67640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f778815dbe0 0x7f77880d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42256/0 (socket says 172.21.15.60:42256) 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.832+0000 7f778cc67640 1 -- 172.21.15.60:0/2574511628 learned_addr learned my addr 172.21.15.60:0/2574511628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f7787fff640 1 --2- 172.21.15.60:0/2574511628 >> v2:172.21.15.100:3300/0 conn(0x7f77880b2cb0 0x7f77880d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f778cc67640 1 -- 172.21.15.60:0/2574511628 >> v2:172.21.15.100:3300/0 conn(0x7f77880b2cb0 msgr2=0x7f77880d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f778cc67640 1 --2- 172.21.15.60:0/2574511628 >> v2:172.21.15.100:3300/0 conn(0x7f77880b2cb0 0x7f77880d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f778cc67640 1 -- 172.21.15.60:0/2574511628 >> v1:172.21.15.100:6789/0 conn(0x7f77880b92a0 legacy=0x7f77880b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f778cc67640 1 -- 172.21.15.60:0/2574511628 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880d1c50 con 0x7f778815dbe0 2024-04-25T08:32:48.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f7787fff640 1 --2- 172.21.15.60:0/2574511628 >> v2:172.21.15.100:3300/0 conn(0x7f77880b2cb0 0x7f77880d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f778cc67640 1 --2- 172.21.15.60:0/2574511628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778815dbe0 0x7f77880d0910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f778806cff0 tx=0x7f778002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3520a7e6115bf447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.833+0000 7f77877fe640 1 -- 172.21.15.60:0/2574511628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780036070 con 0x7f778815dbe0 2024-04-25T08:32:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f77877fe640 1 -- 172.21.15.60:0/2574511628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7780031600 con 0x7f778815dbe0 2024-04-25T08:32:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f77877fe640 1 -- 172.21.15.60:0/2574511628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77800319d0 con 0x7f778815dbe0 2024-04-25T08:32:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 -- 172.21.15.60:0/2574511628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778815dbe0 msgr2=0x7f77880d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 --2- 172.21.15.60:0/2574511628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778815dbe0 0x7f77880d0910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f778806cff0 tx=0x7f778002a660 comp rx=0 tx=0).stop 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 -- 172.21.15.60:0/2574511628 shutdown_connections 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 --2- 172.21.15.60:0/2574511628 >> v2:172.21.15.100:3300/0 conn(0x7f77880b2cb0 0x7f77880d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 --2- 172.21.15.60:0/2574511628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f778815dbe0 0x7f77880d0910 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 -- 172.21.15.60:0/2574511628 >> 172.21.15.60:0/2574511628 conn(0x7f7788061a20 msgr2=0x7f7788061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 -- 172.21.15.60:0/2574511628 shutdown_connections 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.834+0000 7f778eecb640 1 -- 172.21.15.60:0/2574511628 wait complete. 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.835+0000 7f778eecb640 1 Processor -- start 2024-04-25T08:32:48.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.835+0000 7f778eecb640 1 -- start start 2024-04-25T08:32:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778eecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77880b2cb0 0x7f77880b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778eecb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 0x7f778814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778eecb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f778815dbe0 0x7f77880b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778eecb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880b6ab0 con 0x7f77880b92a0 2024-04-25T08:32:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778eecb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880d2210 con 0x7f778815dbe0 2024-04-25T08:32:48.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778eecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77880b6c30 con 0x7f77880b2cb0 2024-04-25T08:32:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f7787fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 0x7f778814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f7787fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 0x7f778814c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42266/0 (socket says 172.21.15.60:42266) 2024-04-25T08:32:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f7787fff640 1 -- 172.21.15.60:0/3990327379 learned_addr learned my addr 172.21.15.60:0/3990327379 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778cc67640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77880b2cb0 0x7f77880b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.836+0000 7f778d468640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f778815dbe0 0x7f77880b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f7787fff640 1 -- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77880b2cb0 msgr2=0x7f77880b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f7787fff640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77880b2cb0 0x7f77880b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f7787fff640 1 -- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f778815dbe0 msgr2=0x7f77880b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f7787fff640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f778815dbe0 0x7f77880b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f7787fff640 1 -- 172.21.15.60:0/3990327379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77881e8c60 con 0x7f77880b92a0 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f7787fff640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 0x7f778814c1c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f777000c7f0 tx=0x7f777000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770015070 con 0x7f77880b92a0 2024-04-25T08:32:48.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770007a30 con 0x7f77880b92a0 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770007e00 con 0x7f77880b92a0 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77880b6170 con 0x7f77880b92a0 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.837+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77881e9200 con 0x7f77880b92a0 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.838+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7788073990 con 0x7f77880b92a0 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.839+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7770018070 con 0x7f77880b92a0 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.839+0000 7f77857fa640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f776806b4d0 0x7f776806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.839+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777008a210 con 0x7f77880b92a0 2024-04-25T08:32:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.839+0000 7f778cc67640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f776806b4d0 0x7f776806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.840+0000 7f778cc67640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f776806b4d0 0x7f776806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7780033000 tx=0x7f7780009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:48.843+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7770058940 con 0x7f77880b92a0 2024-04-25T08:32:49.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.025+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77880b7130 con 0x7f77880b92a0 2024-04-25T08:32:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.026+0000 7f77857fa640 1 -- 172.21.15.60:0/3990327379 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f777005c9e0 con 0x7f77880b92a0 2024-04-25T08:32:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:49.023 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:15.989621+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f776806b4d0 msgr2=0x7f776806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f776806b4d0 0x7f776806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7780033000 tx=0x7f7780009000 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 msgr2=0x7f778814c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 0x7f778814c1c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f777000c7f0 tx=0x7f777000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 shutdown_connections 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f776806b4d0 0x7f776806d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f778815dbe0 0x7f77880b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77880b92a0 0x7f778814c1c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 --2- 172.21.15.60:0/3990327379 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f77880b2cb0 0x7f77880b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 >> 172.21.15.60:0/3990327379 conn(0x7f7788061a20 msgr2=0x7f778815ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 shutdown_connections 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.028+0000 7f778eecb640 1 -- 172.21.15.60:0/3990327379 wait complete. 2024-04-25T08:32:49.035 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:32:49.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:49.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 Processor -- start 2024-04-25T08:32:49.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 -- start start 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d180b9730 0x7f8d180ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d180ceb90 0x7f8d180d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d180cf040 con 0x7f8d1815da70 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d181625e0 con 0x7f8d180ceb90 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.140+0000 7f8d1ee5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d18161d50 con 0x7f8d180b9730 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1d3fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d180ceb90 0x7f8d180d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1cbf9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8d1815da70 0x7f8d18161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39728/0 (socket says 172.21.15.60:39728) 2024-04-25T08:32:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1cbf9640 1 -- 172.21.15.60:0/2603056589 learned_addr learned my addr 172.21.15.60:0/2603056589 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1d3fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8d180ceb90 0x7f8d180d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42274/0 (socket says 172.21.15.60:42274) 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d17fff640 1 --2- 172.21.15.60:0/2603056589 >> v2:172.21.15.100:3300/0 conn(0x7f8d180b9730 0x7f8d180ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1d3fa640 1 -- 172.21.15.60:0/2603056589 >> v2:172.21.15.100:3300/0 conn(0x7f8d180b9730 msgr2=0x7f8d180ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1d3fa640 1 --2- 172.21.15.60:0/2603056589 >> v2:172.21.15.100:3300/0 conn(0x7f8d180b9730 0x7f8d180ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1d3fa640 1 -- 172.21.15.60:0/2603056589 >> v1:172.21.15.60:6789/0 conn(0x7f8d1815da70 legacy=0x7f8d18161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d1d3fa640 1 -- 172.21.15.60:0/2603056589 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18162020 con 0x7f8d180ceb90 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.141+0000 7f8d177fe640 1 -- 172.21.15.60:0/2603056589 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537322691 0 0) 0x7f8d180cf040 con 0x7f8d1815da70 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1d3fa640 1 --2- 172.21.15.60:0/2603056589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d180ceb90 0x7f8d180d22f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8d180c72a0 tx=0x7f8d1002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c439a77d3962bae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d177fe640 1 -- 172.21.15.60:0/2603056589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10007740 con 0x7f8d180ceb90 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d177fe640 1 -- 172.21.15.60:0/2603056589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d100078e0 con 0x7f8d180ceb90 2024-04-25T08:32:49.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d177fe640 1 -- 172.21.15.60:0/2603056589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10007cb0 con 0x7f8d180ceb90 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/2603056589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d180ceb90 msgr2=0x7f8d180d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/2603056589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d180ceb90 0x7f8d180d22f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8d180c72a0 tx=0x7f8d1002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/2603056589 shutdown_connections 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/2603056589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d180ceb90 0x7f8d180d22f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/2603056589 >> v2:172.21.15.100:3300/0 conn(0x7f8d180b9730 0x7f8d180ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/2603056589 >> 172.21.15.60:0/2603056589 conn(0x7f8d18061a20 msgr2=0x7f8d18061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/2603056589 shutdown_connections 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.142+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/2603056589 wait complete. 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.144+0000 7f8d1ee5d640 1 Processor -- start 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.144+0000 7f8d1ee5d640 1 -- start start 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.144+0000 7f8d1ee5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d180ceb90 0x7f8d180cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.144+0000 7f8d1ee5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 0x7f8d1815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1ee5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d181f78f0 0x7f8d1815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1ee5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180d1dd0 con 0x7f8d1815da70 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1ee5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180d1af0 con 0x7f8d181f78f0 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1ee5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1815de50 con 0x7f8d180ceb90 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 0x7f8d1815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 0x7f8d1815fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42276/0 (socket says 172.21.15.60:42276) 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 -- 172.21.15.60:0/765434514 learned_addr learned my addr 172.21.15.60:0/765434514 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1cbf9640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d180ceb90 0x7f8d180cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1d3fa640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d181f78f0 0x7f8d1815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 -- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d180ceb90 msgr2=0x7f8d180cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d180ceb90 0x7f8d180cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 -- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d181f78f0 msgr2=0x7f8d1815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d181f78f0 0x7f8d1815aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 -- 172.21.15.60:0/765434514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1815e060 con 0x7f8d1815da70 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d17fff640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 0x7f8d1815fc20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00d7d0 tx=0x7f8d0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00b6c0 con 0x7f8d1815da70 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c019070 con 0x7f8d1815da70 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00bac0 con 0x7f8d1815da70 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1815e340 con 0x7f8d1815da70 2024-04-25T08:32:49.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.145+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1815e900 con 0x7f8d1815da70 2024-04-25T08:32:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.146+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d18073990 con 0x7f8d1815da70 2024-04-25T08:32:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.147+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d0c00bc60 con 0x7f8d1815da70 2024-04-25T08:32:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.147+0000 7f8d157fa640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d0406b4d0 0x7f8d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.147+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d0c08a7c0 con 0x7f8d1815da70 2024-04-25T08:32:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.147+0000 7f8d1cbf9640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d0406b4d0 0x7f8d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.148+0000 7f8d1cbf9640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d0406b4d0 0x7f8d0406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8d08000f60 tx=0x7f8d08009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.152+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d0c059540 con 0x7f8d1815da70 2024-04-25T08:32:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.369+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d180d1570 con 0x7f8d1815da70 2024-04-25T08:32:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.369+0000 7f8d157fa640 1 -- 172.21.15.60:0/765434514 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f8d0c058e40 con 0x7f8d1815da70 2024-04-25T08:32:49.366 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:49.366 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:47.022299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:49.366 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d0406b4d0 msgr2=0x7f8d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d0406b4d0 0x7f8d0406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8d08000f60 tx=0x7f8d08009040 comp rx=0 tx=0).stop 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 msgr2=0x7f8d1815fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 0x7f8d1815fc20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00d7d0 tx=0x7f8d0c00dc90 comp rx=0 tx=0).stop 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 shutdown_connections 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8d0406b4d0 0x7f8d0406d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d181f78f0 0x7f8d1815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8d1815da70 0x7f8d1815fc20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 --2- 172.21.15.60:0/765434514 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8d180ceb90 0x7f8d180cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 >> 172.21.15.60:0/765434514 conn(0x7f8d18061a20 msgr2=0x7f8d180741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.371+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 shutdown_connections 2024-04-25T08:32:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.372+0000 7f8d1ee5d640 1 -- 172.21.15.60:0/765434514 wait complete. 2024-04-25T08:32:49.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:49.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.483+0000 7f63b479e640 1 Processor -- start 2024-04-25T08:32:49.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.483+0000 7f63b479e640 1 -- start start 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b479e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63ac15a790 0x7f63ac0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b479e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63ac06fea0 0x7f63ac0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b479e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac072300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b479e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0b3710 con 0x7f63ac15a790 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b479e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0cf0e0 con 0x7f63ac06fea0 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b479e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0cf3c0 con 0x7f63ac0af9c0 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b253a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63ac15a790 0x7f63ac0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b253a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63ac15a790 0x7f63ac0d0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42280/0 (socket says 172.21.15.60:42280) 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.484+0000 7f63b253a640 1 -- 172.21.15.60:0/1494171340 learned_addr learned my addr 172.21.15.60:0/1494171340 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b1d39640 1 --2- 172.21.15.60:0/1494171340 >> v2:172.21.15.100:3300/0 conn(0x7f63ac06fea0 0x7f63ac0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b2d3b640 1 --2- 172.21.15.60:0/1494171340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b253a640 1 -- 172.21.15.60:0/1494171340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 msgr2=0x7f63ac072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b253a640 1 --2- 172.21.15.60:0/1494171340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac072300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b253a640 1 -- 172.21.15.60:0/1494171340 >> v2:172.21.15.100:3300/0 conn(0x7f63ac06fea0 msgr2=0x7f63ac0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b253a640 1 --2- 172.21.15.60:0/1494171340 >> v2:172.21.15.100:3300/0 conn(0x7f63ac06fea0 0x7f63ac0d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b253a640 1 -- 172.21.15.60:0/1494171340 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0cf6a0 con 0x7f63ac15a790 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.485+0000 7f63b253a640 1 --2- 172.21.15.60:0/1494171340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0d0390 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f63ac070780 tx=0x7f63a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24828d14dbeeee2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b1538640 1 -- 172.21.15.60:0/1494171340 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4037070 con 0x7f63ac15a790 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b1538640 1 -- 172.21.15.60:0/1494171340 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4032440 con 0x7f63ac15a790 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b1538640 1 -- 172.21.15.60:0/1494171340 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4032810 con 0x7f63ac15a790 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 -- 172.21.15.60:0/1494171340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 msgr2=0x7f63ac0d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 --2- 172.21.15.60:0/1494171340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0d0390 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f63ac070780 tx=0x7f63a402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 -- 172.21.15.60:0/1494171340 shutdown_connections 2024-04-25T08:32:49.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 --2- 172.21.15.60:0/1494171340 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac072300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 --2- 172.21.15.60:0/1494171340 >> v2:172.21.15.100:3300/0 conn(0x7f63ac06fea0 0x7f63ac0d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 --2- 172.21.15.60:0/1494171340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0d0390 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 -- 172.21.15.60:0/1494171340 >> 172.21.15.60:0/1494171340 conn(0x7f63ac0b2370 msgr2=0x7f63ac0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 -- 172.21.15.60:0/1494171340 shutdown_connections 2024-04-25T08:32:49.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.486+0000 7f63b479e640 1 -- 172.21.15.60:0/1494171340 wait complete. 2024-04-25T08:32:49.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.487+0000 7f63b479e640 1 Processor -- start 2024-04-25T08:32:49.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 -- start start 2024-04-25T08:32:49.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac06fea0 0x7f63ac1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac15ab70 con 0x7f63ac15a790 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0cf980 con 0x7f63ac06fea0 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b479e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac15acf0 con 0x7f63ac0af9c0 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b2d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b2d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42282/0 (socket says 172.21.15.60:42282) 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.488+0000 7f63b2d3b640 1 -- 172.21.15.60:0/518945020 learned_addr learned my addr 172.21.15.60:0/518945020 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b1d39640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b253a640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac06fea0 0x7f63ac1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b2d3b640 1 -- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 msgr2=0x7f63ac1f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b2d3b640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac1f74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b2d3b640 1 -- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac06fea0 msgr2=0x7f63ac1f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b2d3b640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac06fea0 0x7f63ac1f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b2d3b640 1 -- 172.21.15.60:0/518945020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0ae220 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b1d39640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac1f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b253a640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac06fea0 0x7f63ac1f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b2d3b640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0ad2c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f639c007e60 tx=0x7f639c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c014420 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c019070 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac0adf40 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c014820 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.489+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac13f830 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.490+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac067be0 con 0x7f63ac15a790 2024-04-25T08:32:49.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.491+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f639c0149c0 con 0x7f63ac15a790 2024-04-25T08:32:49.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.491+0000 7f63aaffd640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f639806b4d0 0x7f639806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.491+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f639c08abf0 con 0x7f63ac15a790 2024-04-25T08:32:49.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.491+0000 7f63b253a640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f639806b4d0 0x7f639806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.492+0000 7f63b253a640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f639806b4d0 0x7f639806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63a4034000 tx=0x7f63a402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.496+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63ac067be0 con 0x7f63ac15a790 2024-04-25T08:32:49.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.715+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63ac0cfe60 con 0x7f63ac15a790 2024-04-25T08:32:49.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.715+0000 7f63aaffd640 1 -- 172.21.15.60:0/518945020 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f639c0598e0 con 0x7f63ac15a790 2024-04-25T08:32:49.712 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:49.712 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:14.355898+0000","modified":"2024-04-25T08:32:47.022299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5886},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5886":{"gid":5886,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.60:6835/1977192273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1977192273},{"type":"v1","addr":"172.21.15.60:6835","nonce":1977192273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:32:49.712 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 26 2024-04-25T08:32:49.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f639806b4d0 msgr2=0x7f639806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f639806b4d0 0x7f639806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63a4034000 tx=0x7f63a402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 msgr2=0x7f63ac0ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0ad2c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f639c007e60 tx=0x7f639c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 shutdown_connections 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63ac15a790 0x7f63ac0ad2c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63ac0af9c0 0x7f63ac1f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f639806b4d0 0x7f639806d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 --2- 172.21.15.60:0/518945020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63ac06fea0 0x7f63ac1f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 >> 172.21.15.60:0/518945020 conn(0x7f63ac0b2370 msgr2=0x7f63ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 shutdown_connections 2024-04-25T08:32:49.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.717+0000 7f63b479e640 1 -- 172.21.15.60:0/518945020 wait complete. 2024-04-25T08:32:49.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.828+0000 7fc1887a5640 1 Processor -- start 2024-04-25T08:32:49.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.828+0000 7fc1887a5640 1 -- start start 2024-04-25T08:32:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.829+0000 7fc1887a5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc18015dbe0 0x7fc1800d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.829+0000 7fc1887a5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1800b6220 con 0x7fc1800b92a0 2024-04-25T08:32:49.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.829+0000 7fc1887a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1800b97a0 con 0x7fc1800b2cb0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.829+0000 7fc1887a5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18015dfc0 con 0x7fc18015dbe0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.829+0000 7fc186d42640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc1800b92a0 0x7fc1800b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39742/0 (socket says 172.21.15.60:39742) 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.829+0000 7fc186d42640 1 -- 172.21.15.60:0/348368466 learned_addr learned my addr 172.21.15.60:0/348368466 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc186541640 1 --2- 172.21.15.60:0/348368466 >> v2:172.21.15.100:3300/0 conn(0x7fc18015dbe0 0x7fc1800d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329866542 0 0) 0x7fc1800b6220 con 0x7fc1800b92a0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc168003520 con 0x7fc1800b92a0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87877472 0 0) 0x7fc168003520 con 0x7fc1800b92a0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 >> v2:172.21.15.100:3300/0 conn(0x7fc18015dbe0 msgr2=0x7fc1800d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 --2- 172.21.15.60:0/348368466 >> v2:172.21.15.100:3300/0 conn(0x7fc18015dbe0 0x7fc1800d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 >> v1:172.21.15.100:6789/0 conn(0x7fc1800b2cb0 legacy=0x7fc1800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800d1c50 con 0x7fc1800b92a0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fc170001670 con 0x7fc1800b92a0 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 >> v1:172.21.15.60:6789/0 conn(0x7fc1800b92a0 legacy=0x7fc1800b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1680039b0 0x7fc168003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc168004800 0x7fc168004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc16800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16800c130 con 0x7fc1680051c0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16800c2e0 con 0x7fc1680039b0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16800c4e0 con 0x7fc168004800 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc186d42640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc16800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc186541640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1680039b0 0x7fc168003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.830+0000 7fc185d40640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc168004800 0x7fc168004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 -- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc168004800 msgr2=0x7fc168004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc168004800 0x7fc168004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 -- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1680039b0 msgr2=0x7fc168003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1680039b0 0x7fc168003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 -- 172.21.15.60:0/348368466 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc168003520 con 0x7fc1680051c0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 -- 172.21.15.60:0/348368466 reap_dead start 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc186d42640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc16800b9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc17000c210 tx=0x7fc168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6446d21039f897ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170007d10 con 0x7fc1680051c0 2024-04-25T08:32:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1700058d0 con 0x7fc1680051c0 2024-04-25T08:32:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.831+0000 7fc18553f640 1 -- 172.21.15.60:0/348368466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc170005cc0 con 0x7fc1680051c0 2024-04-25T08:32:49.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 -- 172.21.15.60:0/348368466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 msgr2=0x7fc16800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc16800b9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc17000c210 tx=0x7fc168005830 comp rx=0 tx=0).stop 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 -- 172.21.15.60:0/348368466 shutdown_connections 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 --2- 172.21.15.60:0/348368466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc16800b9c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 -- 172.21.15.60:0/348368466 >> 172.21.15.60:0/348368466 conn(0x7fc180061a20 msgr2=0x7fc180061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 -- 172.21.15.60:0/348368466 shutdown_connections 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.832+0000 7fc1887a5640 1 -- 172.21.15.60:0/348368466 wait complete. 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.833+0000 7fc1887a5640 1 Processor -- start 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.833+0000 7fc1887a5640 1 -- start start 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc1887a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc180157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc1887a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc18015efd0 0x7fc1800cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc1887a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1800ccfa0 0x7fc18015ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc1887a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800d2210 con 0x7fc1680051c0 2024-04-25T08:32:49.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc1887a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800d1a00 con 0x7fc1800ccfa0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc1887a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800b6ab0 con 0x7fc18015efd0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc180157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc180157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42296/0 (socket says 172.21.15.60:42296) 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 -- 172.21.15.60:0/2660547901 learned_addr learned my addr 172.21.15.60:0/2660547901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186d42640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1800ccfa0 0x7fc18015ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc185d40640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc18015efd0 0x7fc1800cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 -- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc18015efd0 msgr2=0x7fc1800cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc18015efd0 0x7fc1800cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 -- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1800ccfa0 msgr2=0x7fc18015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1800ccfa0 0x7fc18015ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186541640 1 -- 172.21.15.60:0/2660547901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1801584a0 con 0x7fc1680051c0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.834+0000 7fc186d42640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1800ccfa0 0x7fc18015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.835+0000 7fc186541640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc180157f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc178009fc0 tx=0x7fc17800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.835+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800fea0 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.835+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178015630 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.835+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18015e210 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.835+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178015a20 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.835+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18015e7d0 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.836+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc180073990 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.836+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc178015bc0 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.837+0000 7fc17effd640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc16c06b500 0x7fc16c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.837+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc17808b4c0 con 0x7fc1680051c0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.837+0000 7fc185d40640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc16c06b500 0x7fc16c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.838+0000 7fc185d40640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc16c06b500 0x7fc16c06d9c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc174000c00 tx=0x7fc174009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:49.841+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1801584a0 con 0x7fc1680051c0 2024-04-25T08:32:50.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.022+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1800b8060 con 0x7fc1680051c0 2024-04-25T08:32:50.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.023+0000 7fc17effd640 1 -- 172.21.15.60:0/2660547901 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fc178019030 con 0x7fc1680051c0 2024-04-25T08:32:50.020 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:50.020 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:15.989621+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:12.634751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:13.328038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc16c06b500 msgr2=0x7fc16c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc16c06b500 0x7fc16c06d9c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc174000c00 tx=0x7fc174009040 comp rx=0 tx=0).stop 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 msgr2=0x7fc180157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc180157f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc178009fc0 tx=0x7fc17800f580 comp rx=0 tx=0).stop 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 shutdown_connections 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc16c06b500 0x7fc16c06d9c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1800ccfa0 0x7fc18015ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc18015efd0 0x7fc1800cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 --2- 172.21.15.60:0/2660547901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1680051c0 0x7fc180157f60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.025+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 >> 172.21.15.60:0/2660547901 conn(0x7fc180061a20 msgr2=0x7fc1800b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.026+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 shutdown_connections 2024-04-25T08:32:50.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.026+0000 7fc1887a5640 1 -- 172.21.15.60:0/2660547901 wait complete. 2024-04-25T08:32:50.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:32:50.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.139+0000 7fd5f1f9e640 1 Processor -- start 2024-04-25T08:32:50.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.139+0000 7fd5f1f9e640 1 -- start start 2024-04-25T08:32:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5f1f9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 0x7fd5ec0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5f1f9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec0b5620 0x7fd5ec0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5f1f9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec06cf50 con 0x7fd5ec06c2d0 2024-04-25T08:32:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5f1f9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec0b5fc0 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5f1f9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0ce530 con 0x7fd5ec0b5620 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5eb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 0x7fd5ec0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5eaffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec0b5620 0x7fd5ec0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5eb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 0x7fd5ec0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45668/0 (socket says 172.21.15.60:45668) 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5eb7fe640 1 -- 172.21.15.60:0/3648768355 learned_addr learned my addr 172.21.15.60:0/3648768355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.140+0000 7fd5ebfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd5ec06c2d0 0x7fd5ec06c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:47428/0 (socket says 172.21.15.60:47428) 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5ea7fc640 1 -- 172.21.15.60:0/3648768355 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919463806 0 0) 0x7fd5ec06cf50 con 0x7fd5ec06c2d0 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5ea7fc640 1 -- 172.21.15.60:0/3648768355 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d4003520 con 0x7fd5ec06c2d0 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5eaffd640 1 -- 172.21.15.60:0/3648768355 >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 msgr2=0x7fd5ec0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5eaffd640 1 --2- 172.21.15.60:0/3648768355 >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 0x7fd5ec0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5eaffd640 1 -- 172.21.15.60:0/3648768355 >> v1:172.21.15.100:6789/0 conn(0x7fd5ec06c2d0 legacy=0x7fd5ec06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5eaffd640 1 -- 172.21.15.60:0/3648768355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0ce810 con 0x7fd5ec0b5620 2024-04-25T08:32:50.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5eb7fe640 1 --2- 172.21.15.60:0/3648768355 >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 0x7fd5ec0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.141+0000 7fd5eaffd640 1 --2- 172.21.15.60:0/3648768355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec0b5620 0x7fd5ec0b5a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00097a0 tx=0x7fd5e002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1d331047949eec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5ea7fc640 1 -- 172.21.15.60:0/3648768355 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0037070 con 0x7fd5ec0b5620 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5ea7fc640 1 -- 172.21.15.60:0/3648768355 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0032440 con 0x7fd5ec0b5620 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5ea7fc640 1 -- 172.21.15.60:0/3648768355 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00327f0 con 0x7fd5ec0b5620 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/3648768355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec0b5620 msgr2=0x7fd5ec0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/3648768355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec0b5620 0x7fd5ec0b5a80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00097a0 tx=0x7fd5e002a180 comp rx=0 tx=0).stop 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/3648768355 shutdown_connections 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/3648768355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec0b5620 0x7fd5ec0b5a80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/3648768355 >> v2:172.21.15.100:3300/0 conn(0x7fd5ec15b7d0 0x7fd5ec0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/3648768355 >> 172.21.15.60:0/3648768355 conn(0x7fd5ec0d0800 msgr2=0x7fd5ec0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/3648768355 shutdown_connections 2024-04-25T08:32:50.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.142+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/3648768355 wait complete. 2024-04-25T08:32:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.143+0000 7fd5f1f9e640 1 Processor -- start 2024-04-25T08:32:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.143+0000 7fd5f1f9e640 1 -- start start 2024-04-25T08:32:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5f1f9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec06c2d0 0x7fd5ec0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5f1f9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ec0b5620 0x7fd5ec0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5f1f9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 0x7fd5ec151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5f1f9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0cf550 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5f1f9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec06d230 con 0x7fd5ec0b5620 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5f1f9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0cf6d0 con 0x7fd5ec06c2d0 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 0x7fd5ec151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5ebfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 0x7fd5ec151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42298/0 (socket says 172.21.15.60:42298) 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5ebfff640 1 -- 172.21.15.60:0/2112277230 learned_addr learned my addr 172.21.15.60:0/2112277230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5eb7fe640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec06c2d0 0x7fd5ec0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.144+0000 7fd5eaffd640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ec0b5620 0x7fd5ec0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5ebfff640 1 -- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec06c2d0 msgr2=0x7fd5ec0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5ebfff640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec06c2d0 0x7fd5ec0cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5ebfff640 1 -- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ec0b5620 msgr2=0x7fd5ec0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5ebfff640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ec0b5620 0x7fd5ec0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5ebfff640 1 -- 172.21.15.60:0/2112277230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0cbc30 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5ebfff640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 0x7fd5ec151430 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400c730 tx=0x7fd5e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4017070 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e400ce40 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec0cc8f0 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400a860 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.145+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec0cceb0 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.146+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec073a00 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.147+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5e401a070 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.147+0000 7fd5d3fff640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5d806b4d0 0x7fd5d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.147+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5e4089990 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.147+0000 7fd5eb7fe640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5d806b4d0 0x7fd5d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.148+0000 7fd5eb7fe640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5d806b4d0 0x7fd5d806d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006ca0 tx=0x7fd5dc009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.152+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5ec0cbc30 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.370+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd5ec0d0ff0 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.367 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:32:50.371+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:32:50.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.372+0000 7fd5d3fff640 1 -- 172.21.15.60:0/2112277230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd5e4058030 con 0x7fd5ec15b7d0 2024-04-25T08:32:50.368 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:32:50.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5d806b4d0 msgr2=0x7fd5d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5d806b4d0 0x7fd5d806d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006ca0 tx=0x7fd5dc009240 comp rx=0 tx=0).stop 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 msgr2=0x7fd5ec151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 0x7fd5ec151430 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400c730 tx=0x7fd5e400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 shutdown_connections 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5ec15b7d0 0x7fd5ec151430 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5d806b4d0 0x7fd5d806d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5ec0b5620 0x7fd5ec0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 --2- 172.21.15.60:0/2112277230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5ec06c2d0 0x7fd5ec0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 >> 172.21.15.60:0/2112277230 conn(0x7fd5ec0d0800 msgr2=0x7fd5ec160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 shutdown_connections 2024-04-25T08:32:50.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.375+0000 7fd5f1f9e640 1 -- 172.21.15.60:0/2112277230 wait complete. 2024-04-25T08:32:50.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:32:50.411 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:32:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.491+0000 7f6a57cee640 1 Processor -- start 2024-04-25T08:32:50.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.491+0000 7f6a57cee640 1 -- start start 2024-04-25T08:32:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a57cee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a500d34c0 0x7f6a500d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a57cee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a500b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a57cee640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500d2420 con 0x7f6a5015dac0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a57cee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a500d3030 con 0x7f6a500d34c0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a57cee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500d0d20 con 0x7f6a500d08e0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a55a8a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6a5015dac0 0x7f6a500d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39758/0 (socket says 172.21.15.60:39758) 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a55a8a640 1 -- 172.21.15.60:0/1802345163 learned_addr learned my addr 172.21.15.60:0/1802345163 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280165630 0 0) 0x7f6a500d2420 con 0x7f6a5015dac0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c003520 con 0x7f6a5015dac0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a55289640 1 --2- 172.21.15.60:0/1802345163 >> v2:172.21.15.100:3300/0 conn(0x7f6a500d34c0 0x7f6a500d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a5628b640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a500b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125008333 0 0) 0x7f6a3c003520 con 0x7f6a5015dac0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 msgr2=0x7f6a500b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a500b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 >> v2:172.21.15.100:3300/0 conn(0x7f6a500d34c0 msgr2=0x7f6a500d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 --2- 172.21.15.60:0/1802345163 >> v2:172.21.15.100:3300/0 conn(0x7f6a500d34c0 0x7f6a500d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500d2700 con 0x7f6a5015dac0 2024-04-25T08:32:50.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6a4c001670 con 0x7f6a5015dac0 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.492+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 >> v1:172.21.15.60:6789/0 conn(0x7f6a5015dac0 legacy=0x7f6a500d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a54a88640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a3c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a54a88640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a3c003f30 0x7f6a3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a54a88640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c004d40 0x7f6a3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500d3030 con 0x7f6a3c003f30 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500d0d20 con 0x7f6a3c004d40 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c00baa0 con 0x7f6a500d08e0 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55289640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a3c003f30 0x7f6a3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55a8a640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a3c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a5628b640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c004d40 0x7f6a3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55289640 1 -- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 msgr2=0x7f6a3c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55289640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a3c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55289640 1 -- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c004d40 msgr2=0x7f6a3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55289640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c004d40 0x7f6a3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55289640 1 -- 172.21.15.60:0/1802345163 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c003520 con 0x7f6a3c003f30 2024-04-25T08:32:50.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.493+0000 7f6a55a8a640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500d08e0 0x7f6a3c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a55289640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a3c003f30 0x7f6a3c004800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009ab0 tx=0x7f6a3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dab3c7c33739193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40036070 con 0x7f6a3c003f30 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40005990 con 0x7f6a3c003f30 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a54a88640 1 -- 172.21.15.60:0/1802345163 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40009680 con 0x7f6a3c003f30 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a57cee640 1 -- 172.21.15.60:0/1802345163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a3c003f30 msgr2=0x7f6a3c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1802345163 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a3c003f30 0x7f6a3c004800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009ab0 tx=0x7f6a3c005830 comp rx=0 tx=0).stop 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a5628b640 1 -- 172.21.15.60:0/1802345163 reap_dead start 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a57cee640 1 -- 172.21.15.60:0/1802345163 shutdown_connections 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a57cee640 1 -- 172.21.15.60:0/1802345163 >> 172.21.15.60:0/1802345163 conn(0x7f6a50061c30 msgr2=0x7f6a5006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a57cee640 1 -- 172.21.15.60:0/1802345163 shutdown_connections 2024-04-25T08:32:50.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.494+0000 7f6a57cee640 1 -- 172.21.15.60:0/1802345163 wait complete. 2024-04-25T08:32:50.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.495+0000 7f6a57cee640 1 Processor -- start 2024-04-25T08:32:50.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 -- start start 2024-04-25T08:32:50.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c003f30 0x7f6a501f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 0x7f6a500b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b3580 0x7f6a500b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500d2100 con 0x7f6a500b5620 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500d2cc0 con 0x7f6a3c003f30 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a57cee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5015dea0 con 0x7f6a500b3580 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a55289640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 0x7f6a500b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a55289640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 0x7f6a500b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42334/0 (socket says 172.21.15.60:42334) 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.496+0000 7f6a55289640 1 -- 172.21.15.60:0/1755026876 learned_addr learned my addr 172.21.15.60:0/1755026876 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55a8a640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c003f30 0x7f6a501f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55289640 1 -- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b3580 msgr2=0x7f6a500b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a5628b640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b3580 0x7f6a500b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55289640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b3580 0x7f6a500b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55289640 1 -- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c003f30 msgr2=0x7f6a501f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55289640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c003f30 0x7f6a501f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55289640 1 -- 172.21.15.60:0/1755026876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5015e9a0 con 0x7f6a500b5620 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55a8a640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c003f30 0x7f6a501f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:50.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a5628b640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b3580 0x7f6a500b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a55289640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 0x7f6a500b5aa0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009bd0 tx=0x7f6a40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40041020 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40037040 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5015ec80 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40036070 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.497+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5015f240 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.498+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a50073a00 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.499+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a40044020 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.499+0000 7f6a467fc640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3406b500 0x7f6a3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.499+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6a400aaca0 con 0x7f6a500b5620 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.499+0000 7f6a55a8a640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3406b500 0x7f6a3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.500+0000 7f6a55a8a640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3406b500 0x7f6a3406d9c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c001eb0 tx=0x7f6a4c0014c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.503+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a4004b030 con 0x7f6a500b5620 2024-04-25T08:32:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:50.722+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6a5015afc0 con 0x7f6a500b5620 2024-04-25T08:32:51.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.374+0000 7f6a467fc640 1 -- 172.21.15.60:0/1755026876 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a40079a60 con 0x7f6a500b5620 2024-04-25T08:32:51.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3406b500 msgr2=0x7f6a3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3406b500 0x7f6a3406d9c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c001eb0 tx=0x7f6a4c0014c0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 msgr2=0x7f6a500b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 0x7f6a500b5aa0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009bd0 tx=0x7f6a40004730 comp rx=0 tx=0).stop 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 shutdown_connections 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a500b3580 0x7f6a500b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a500b5620 0x7f6a500b5aa0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a3406b500 0x7f6a3406d9c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 --2- 172.21.15.60:0/1755026876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c003f30 0x7f6a501f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 >> 172.21.15.60:0/1755026876 conn(0x7f6a50061c30 msgr2=0x7f6a500c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 shutdown_connections 2024-04-25T08:32:51.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.376+0000 7f6a57cee640 1 -- 172.21.15.60:0/1755026876 wait complete. 2024-04-25T08:32:51.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:32:51.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.539+0000 7f8714970640 1 Processor -- start 2024-04-25T08:32:51.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.539+0000 7f8714970640 1 -- start start 2024-04-25T08:32:51.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f8714970640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f87100b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f8714970640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871006cf50 con 0x7f871006c2d0 2024-04-25T08:32:51.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f8714970640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87100ce530 con 0x7f871015b7d0 2024-04-25T08:32:51.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f8714970640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b5fc0 con 0x7f87100b5620 2024-04-25T08:32:51.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f870ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f871006c2d0 0x7f871006c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:39766/0 (socket says 172.21.15.60:39766) 2024-04-25T08:32:51.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f870ed9d640 1 -- 172.21.15.60:0/1785478383 learned_addr learned my addr 172.21.15.60:0/1785478383 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662818779 0 0) 0x7f871006cf50 con 0x7f871006c2d0 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc003520 con 0x7f871006c2d0 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.540+0000 7f8707fff640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f87100b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2883339372 0 0) 0x7f86fc003520 con 0x7f871006c2d0 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 msgr2=0x7f87100b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f87100b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 >> v1:172.21.15.100:6789/0 conn(0x7f871015b7d0 legacy=0x7f87100cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100ce810 con 0x7f871006c2d0 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f8707fff640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f87100b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f87080019f0 con 0x7f871006c2d0 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 >> v1:172.21.15.60:6789/0 conn(0x7f871006c2d0 legacy=0x7f871006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86fc0039e0 0x7f86fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f86fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86fc004d40 0x7f86fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc00baf0 con 0x7f86fc004d40 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc00bca0 con 0x7f86fc0039e0 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc00bea0 con 0x7f87100b5620 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870ed9d640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86fc004d40 0x7f86fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f8707fff640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f86fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870e59c640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86fc0039e0 0x7f86fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870ed9d640 1 -- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 msgr2=0x7f86fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870ed9d640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87100b5620 0x7f86fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870ed9d640 1 -- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86fc0039e0 msgr2=0x7f86fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870ed9d640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86fc0039e0 0x7f86fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.541+0000 7f870ed9d640 1 -- 172.21.15.60:0/1785478383 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc003520 con 0x7f86fc004d40 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.542+0000 7f870ed9d640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86fc004d40 0x7f86fc0051c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f870800a840 tx=0x7f86fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc9ed4346c88f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.542+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870800a460 con 0x7f86fc004d40 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.542+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870802f070 con 0x7f86fc004d40 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.542+0000 7f870dd9b640 1 -- 172.21.15.60:0/1785478383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870802bcb0 con 0x7f86fc004d40 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f8714970640 1 -- 172.21.15.60:0/1785478383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86fc004d40 msgr2=0x7f86fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f8714970640 1 --2- 172.21.15.60:0/1785478383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86fc004d40 0x7f86fc0051c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f870800a840 tx=0x7f86fc005830 comp rx=0 tx=0).stop 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f870ed9d640 1 -- 172.21.15.60:0/1785478383 reap_dead start 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f8714970640 1 -- 172.21.15.60:0/1785478383 shutdown_connections 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f8714970640 1 -- 172.21.15.60:0/1785478383 >> 172.21.15.60:0/1785478383 conn(0x7f87100d0800 msgr2=0x7f87100d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f8714970640 1 -- 172.21.15.60:0/1785478383 shutdown_connections 2024-04-25T08:32:51.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.543+0000 7f8714970640 1 -- 172.21.15.60:0/1785478383 wait complete. 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.544+0000 7f8714970640 1 Processor -- start 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.544+0000 7f8714970640 1 -- start start 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8714970640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86fc004d40 0x7f87101f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8714970640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 0x7f871014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8714970640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87100d1ed0 0x7f87100d2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8714970640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871015bbb0 con 0x7f87100c78c0 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8714970640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871006d230 con 0x7f87100d1ed0 2024-04-25T08:32:51.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8714970640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871015bd30 con 0x7f86fc004d40 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 0x7f871014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 0x7f871014be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42348/0 (socket says 172.21.15.60:42348) 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 -- 172.21.15.60:0/1805360188 learned_addr learned my addr 172.21.15.60:0/1805360188 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f870e59c640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86fc004d40 0x7f87101f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f870ed9d640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87100d1ed0 0x7f87100d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 -- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86fc004d40 msgr2=0x7f87101f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86fc004d40 0x7f87101f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 -- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87100d1ed0 msgr2=0x7f87100d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87100d1ed0 0x7f87100d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 -- 172.21.15.60:0/1805360188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100cc6f0 con 0x7f87100c78c0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.545+0000 7f8707fff640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 0x7f871014be90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f86f80115f0 tx=0x7f86f8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.546+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8019070 con 0x7f87100c78c0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.546+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f8004220 con 0x7f87100c78c0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.546+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100cafc0 con 0x7f87100c78c0 2024-04-25T08:32:51.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.546+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f800f040 con 0x7f87100c78c0 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.546+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87100ccc50 con 0x7f87100c78c0 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.547+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87100d2890 con 0x7f87100c78c0 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.547+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86f800f6f0 con 0x7f87100c78c0 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.548+0000 7f8706ffd640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.548+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f86f808a580 con 0x7f87100c78c0 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.548+0000 7f870e59c640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:51.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.548+0000 7f870e59c640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86f006b4d0 0x7f86f006d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f87000017e0 tx=0x7f8700009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:51.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.552+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87100cc6f0 con 0x7f87100c78c0 2024-04-25T08:32:51.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:51.738+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8710073a00 con 0x7f87100c78c0 2024-04-25T08:32:52.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.378+0000 7f8706ffd640 1 -- 172.21.15.60:0/1805360188 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f86f8058c60 con 0x7f87100c78c0 2024-04-25T08:32:52.375 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.382+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86f006b4d0 msgr2=0x7f86f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.382+0000 7f8714970640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86f006b4d0 0x7f86f006d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f87000017e0 tx=0x7f8700009040 comp rx=0 tx=0).stop 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.382+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 msgr2=0x7f871014be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.382+0000 7f8714970640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 0x7f871014be90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f86f80115f0 tx=0x7f86f8011ab0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.382+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 shutdown_connections 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87100d1ed0 0x7f87100d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87100c78c0 0x7f871014be90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 --2- 172.21.15.60:0/1805360188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86fc004d40 0x7f87101f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 >> 172.21.15.60:0/1805360188 conn(0x7f87100d0800 msgr2=0x7f871015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 shutdown_connections 2024-04-25T08:32:52.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.383+0000 7f8714970640 1 -- 172.21.15.60:0/1805360188 wait complete. 2024-04-25T08:32:52.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:32:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.508+0000 7f507cc07640 1 Processor -- start 2024-04-25T08:32:52.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.508+0000 7f507cc07640 1 -- start start 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507cc07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f507815b7d0 0x7f50780cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507cc07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50780b5620 0x7f50780b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507cc07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f507806c2d0 0x7f507806c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507cc07640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50780b5fc0 con 0x7f507815b7d0 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507cc07640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507806cca0 con 0x7f507806c2d0 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507cc07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507806cf50 con 0x7f50780b5620 2024-04-25T08:32:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f507815b7d0 0x7f50780cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f507815b7d0 0x7f50780cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43378/0 (socket says 172.21.15.60:43378) 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 -- 172.21.15.60:0/4244122565 learned_addr learned my addr 172.21.15.60:0/4244122565 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f5076d9d640 1 --2- 172.21.15.60:0/4244122565 >> v2:172.21.15.100:3300/0 conn(0x7f507806c2d0 0x7f507806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f506ffff640 1 --2- 172.21.15.60:0/4244122565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50780b5620 0x7f50780b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 -- 172.21.15.60:0/4244122565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50780b5620 msgr2=0x7f50780b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 --2- 172.21.15.60:0/4244122565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50780b5620 0x7f50780b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 -- 172.21.15.60:0/4244122565 >> v2:172.21.15.100:3300/0 conn(0x7f507806c2d0 msgr2=0x7f507806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 --2- 172.21.15.60:0/4244122565 >> v2:172.21.15.100:3300/0 conn(0x7f507806c2d0 0x7f507806c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f507659c640 1 -- 172.21.15.60:0/4244122565 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780ce530 con 0x7f507815b7d0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.509+0000 7f5076d9d640 1 --2- 172.21.15.60:0/4244122565 >> v2:172.21.15.100:3300/0 conn(0x7f507806c2d0 0x7f507806c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.510+0000 7f507659c640 1 --2- 172.21.15.60:0/4244122565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f507815b7d0 0x7f50780cf010 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f507815e340 tx=0x7f506802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c76a74b91f5c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.510+0000 7f5075d9b640 1 -- 172.21.15.60:0/4244122565 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068037070 con 0x7f507815b7d0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.510+0000 7f5075d9b640 1 -- 172.21.15.60:0/4244122565 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5068032440 con 0x7f507815b7d0 2024-04-25T08:32:52.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.510+0000 7f5075d9b640 1 -- 172.21.15.60:0/4244122565 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068032810 con 0x7f507815b7d0 2024-04-25T08:32:52.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 -- 172.21.15.60:0/4244122565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f507815b7d0 msgr2=0x7f50780cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 --2- 172.21.15.60:0/4244122565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f507815b7d0 0x7f50780cf010 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f507815e340 tx=0x7f506802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 -- 172.21.15.60:0/4244122565 shutdown_connections 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 --2- 172.21.15.60:0/4244122565 >> v2:172.21.15.100:3300/0 conn(0x7f507806c2d0 0x7f507806c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 --2- 172.21.15.60:0/4244122565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50780b5620 0x7f50780b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 --2- 172.21.15.60:0/4244122565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f507815b7d0 0x7f50780cf010 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 -- 172.21.15.60:0/4244122565 >> 172.21.15.60:0/4244122565 conn(0x7f50780d0800 msgr2=0x7f50780d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 -- 172.21.15.60:0/4244122565 shutdown_connections 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.511+0000 7f507cc07640 1 -- 172.21.15.60:0/4244122565 wait complete. 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.512+0000 7f507cc07640 1 Processor -- start 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.512+0000 7f507cc07640 1 -- start start 2024-04-25T08:32:52.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507cc07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f507806c2d0 0x7f507815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507cc07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 0x7f5078150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507cc07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078151120 0x7f50780abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507cc07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507806d230 con 0x7f50780b5620 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507cc07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780ceaf0 con 0x7f5078151120 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507cc07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cf550 con 0x7f507806c2d0 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 0x7f5078150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 0x7f5078150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43390/0 (socket says 172.21.15.60:43390) 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 -- 172.21.15.60:0/3973655012 learned_addr learned my addr 172.21.15.60:0/3973655012 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f507659c640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f507806c2d0 0x7f507815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f5076d9d640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078151120 0x7f50780abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 -- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f507806c2d0 msgr2=0x7f507815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f507806c2d0 0x7f507815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 -- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078151120 msgr2=0x7f50780abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078151120 0x7f50780abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.513+0000 7f506ffff640 1 -- 172.21.15.60:0/3973655012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780ab370 con 0x7f50780b5620 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.514+0000 7f506ffff640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 0x7f5078150be0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f506000c7f0 tx=0x7f506000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.514+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060015070 con 0x7f50780b5620 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.514+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5060004220 con 0x7f50780b5620 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.514+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780aa5b0 con 0x7f50780b5620 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.514+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060007620 con 0x7f50780b5620 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.514+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50780aab70 con 0x7f50780b5620 2024-04-25T08:32:52.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.515+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078073a00 con 0x7f50780b5620 2024-04-25T08:32:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.515+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50600079d0 con 0x7f50780b5620 2024-04-25T08:32:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.516+0000 7f506effd640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f505806b4d0 0x7f505806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.516+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f506008a420 con 0x7f50780b5620 2024-04-25T08:32:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.516+0000 7f507659c640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f505806b4d0 0x7f505806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.516+0000 7f507659c640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f505806b4d0 0x7f505806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5068034000 tx=0x7f506802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.520+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5060058cb0 con 0x7f50780b5620 2024-04-25T08:32:52.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:52.699+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f50780d21c0 con 0x7f50780b5620 2024-04-25T08:32:53.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.380+0000 7f506effd640 1 -- 172.21.15.60:0/3973655012 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f506005cd50 con 0x7f50780b5620 2024-04-25T08:32:53.376 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:32:53.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f505806b4d0 msgr2=0x7f505806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f505806b4d0 0x7f505806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5068034000 tx=0x7f506802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 msgr2=0x7f5078150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 0x7f5078150be0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f506000c7f0 tx=0x7f506000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 shutdown_connections 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f505806b4d0 0x7f505806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5078151120 0x7f50780abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50780b5620 0x7f5078150be0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 --2- 172.21.15.60:0/3973655012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f507806c2d0 0x7f507815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 >> 172.21.15.60:0/3973655012 conn(0x7f50780d0800 msgr2=0x7f50780b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 shutdown_connections 2024-04-25T08:32:53.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.384+0000 7f507cc07640 1 -- 172.21.15.60:0/3973655012 wait complete. 2024-04-25T08:32:53.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:32:53.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.511+0000 7f7b31bfe640 1 Processor -- start 2024-04-25T08:32:53.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.511+0000 7f7b31bfe640 1 -- start start 2024-04-25T08:32:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b31bfe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b2c1610f0 0x7f7b2c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b31bfe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b2c0ce7b0 0x7f7b2c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b31bfe640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c0c6960 con 0x7f7b2c1616f0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b31bfe640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2c1614d0 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b31bfe640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2c162260 con 0x7f7b2c1610f0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b2bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b2c0ce7b0 0x7f7b2c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b2bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7b2c0ce7b0 0x7f7b2c161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43398/0 (socket says 172.21.15.60:43398) 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.512+0000 7f7b2bfff640 1 -- 172.21.15.60:0/2210003548 learned_addr learned my addr 172.21.15.60:0/2210003548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.513+0000 7f7b2b7fe640 1 --2- 172.21.15.60:0/2210003548 >> v2:172.21.15.100:3300/0 conn(0x7f7b2c1610f0 0x7f7b2c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.513+0000 7f7b2bfff640 1 -- 172.21.15.60:0/2210003548 >> v2:172.21.15.100:3300/0 conn(0x7f7b2c1610f0 msgr2=0x7f7b2c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.513+0000 7f7b2bfff640 1 --2- 172.21.15.60:0/2210003548 >> v2:172.21.15.100:3300/0 conn(0x7f7b2c1610f0 0x7f7b2c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.513+0000 7f7b2bfff640 1 -- 172.21.15.60:0/2210003548 >> v1:172.21.15.100:6789/0 conn(0x7f7b2c1616f0 legacy=0x7f7b2c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.513+0000 7f7b2bfff640 1 -- 172.21.15.60:0/2210003548 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c162540 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.513+0000 7f7b2bfff640 1 --2- 172.21.15.60:0/2210003548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c161b50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c071fb0 tx=0x7f7b2402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc2ad9b991f1e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b2a7fc640 1 -- 172.21.15.60:0/2210003548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24036070 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b2a7fc640 1 -- 172.21.15.60:0/2210003548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b24031600 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b2a7fc640 1 -- 172.21.15.60:0/2210003548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240319d0 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 -- 172.21.15.60:0/2210003548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 msgr2=0x7f7b2c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/2210003548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c161b50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c071fb0 tx=0x7f7b2402a730 comp rx=0 tx=0).stop 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 -- 172.21.15.60:0/2210003548 shutdown_connections 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/2210003548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c161b50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/2210003548 >> v2:172.21.15.100:3300/0 conn(0x7f7b2c1610f0 0x7f7b2c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 -- 172.21.15.60:0/2210003548 >> 172.21.15.60:0/2210003548 conn(0x7f7b2c06f500 msgr2=0x7f7b2c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 -- 172.21.15.60:0/2210003548 shutdown_connections 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.514+0000 7f7b31bfe640 1 -- 172.21.15.60:0/2210003548 wait complete. 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.515+0000 7f7b31bfe640 1 Processor -- start 2024-04-25T08:32:53.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 -- start start 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c1450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2c1610f0 0x7f7b2c1f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2c1616f0 0x7f7b2c1f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0cf690 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0cfce0 con 0x7f7b2c1616f0 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.516+0000 7f7b31bfe640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c06c700 con 0x7f7b2c1610f0 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c1450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c1450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43402/0 (socket says 172.21.15.60:43402) 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 -- 172.21.15.60:0/3542208408 learned_addr learned my addr 172.21.15.60:0/3542208408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:53.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2bfff640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2c1616f0 0x7f7b2c1f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2affd640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2c1610f0 0x7f7b2c1f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 -- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2c1610f0 msgr2=0x7f7b2c1f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2c1610f0 0x7f7b2c1f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 -- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2c1616f0 msgr2=0x7f7b2c1f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2c1616f0 0x7f7b2c1f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 -- 172.21.15.60:0/3542208408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c145600 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2affd640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2c1610f0 0x7f7b2c1f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b2b7fe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c1450c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00e9b0 tx=0x7f7b1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c017070 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c00c9a0 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c15acc0 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c00cd90 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.517+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c15b690 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.518+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c073a00 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.519+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b1c01a070 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.519+0000 7f7b13fff640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b1806b4d0 0x7f7b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.519+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7b1c08a330 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.519+0000 7f7b2affd640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b1806b4d0 0x7f7b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.520+0000 7f7b2affd640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b1806b4d0 0x7f7b1806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0b45d0 tx=0x7f7b20009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.523+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b1c058d60 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.707+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7b2c0cf150 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.710+0000 7f7b13fff640 1 -- 172.21.15.60:0/3542208408 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7b1c05ce00 con 0x7f7b2c0ce7b0 2024-04-25T08:32:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b1806b4d0 msgr2=0x7f7b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b1806b4d0 0x7f7b1806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0b45d0 tx=0x7f7b20009040 comp rx=0 tx=0).stop 2024-04-25T08:32:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 msgr2=0x7f7b2c1450c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c1450c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00e9b0 tx=0x7f7b1c00ee70 comp rx=0 tx=0).stop 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 shutdown_connections 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7b1806b4d0 0x7f7b1806d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b2c1616f0 0x7f7b2c1f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b2c1610f0 0x7f7b2c1f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 --2- 172.21.15.60:0/3542208408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b2c0ce7b0 0x7f7b2c1450c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 >> 172.21.15.60:0/3542208408 conn(0x7f7b2c06f500 msgr2=0x7f7b2c1573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 shutdown_connections 2024-04-25T08:32:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.712+0000 7f7b31bfe640 1 -- 172.21.15.60:0/3542208408 wait complete. 2024-04-25T08:32:53.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T08:32:53.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.823+0000 7f2f63fb0640 1 Processor -- start 2024-04-25T08:32:53.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.824+0000 7f2f63fb0640 1 -- start start 2024-04-25T08:32:53.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.824+0000 7f2f63fb0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f5c15dbe0 0x7f2f5c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.824+0000 7f2f63fb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.824+0000 7f2f63fb0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5c0b97a0 con 0x7f2f5c0b92a0 2024-04-25T08:32:53.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f63fb0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5c15dfc0 con 0x7f2f5c15dbe0 2024-04-25T08:32:53.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f63fb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0d1a00 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6254d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2f5c0b92a0 0x7f2f5c0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48378/0 (socket says 172.21.15.60:48378) 2024-04-25T08:32:53.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6254d640 1 -- 172.21.15.60:0/856740658 learned_addr learned my addr 172.21.15.60:0/856740658 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:53.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f61d4c640 1 --2- 172.21.15.60:0/856740658 >> v2:172.21.15.100:3300/0 conn(0x7f2f5c15dbe0 0x7f2f5c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6154b640 1 --2- 172.21.15.60:0/856740658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f60d4a640 1 -- 172.21.15.60:0/856740658 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226607648 0 0) 0x7f2f5c0b97a0 con 0x7f2f5c0b92a0 2024-04-25T08:32:53.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f60d4a640 1 -- 172.21.15.60:0/856740658 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f48003520 con 0x7f2f5c0b92a0 2024-04-25T08:32:53.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6154b640 1 -- 172.21.15.60:0/856740658 >> v2:172.21.15.100:3300/0 conn(0x7f2f5c15dbe0 msgr2=0x7f2f5c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:53.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6154b640 1 --2- 172.21.15.60:0/856740658 >> v2:172.21.15.100:3300/0 conn(0x7f2f5c15dbe0 0x7f2f5c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6154b640 1 -- 172.21.15.60:0/856740658 >> v1:172.21.15.60:6789/0 conn(0x7f2f5c0b92a0 legacy=0x7f2f5c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f60d4a640 1 -- 172.21.15.60:0/856740658 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702626781 0 0) 0x7f2f48003520 con 0x7f2f5c0b92a0 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.825+0000 7f2f6154b640 1 -- 172.21.15.60:0/856740658 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0d1c50 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.826+0000 7f2f6154b640 1 --2- 172.21.15.60:0/856740658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0d0e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0097a0 tx=0x7f2f4c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d2f761c3950d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.826+0000 7f2f60d4a640 1 -- 172.21.15.60:0/856740658 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c037070 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.826+0000 7f2f60d4a640 1 -- 172.21.15.60:0/856740658 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c02aba0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.826+0000 7f2f60d4a640 1 -- 172.21.15.60:0/856740658 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f540019f0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 -- 172.21.15.60:0/856740658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b2cb0 msgr2=0x7f2f5c0d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/856740658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0d0e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0097a0 tx=0x7f2f4c02a110 comp rx=0 tx=0).stop 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 -- 172.21.15.60:0/856740658 shutdown_connections 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/856740658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0d0e50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/856740658 >> v2:172.21.15.100:3300/0 conn(0x7f2f5c15dbe0 0x7f2f5c0d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 -- 172.21.15.60:0/856740658 >> 172.21.15.60:0/856740658 conn(0x7f2f5c061a20 msgr2=0x7f2f5c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 -- 172.21.15.60:0/856740658 shutdown_connections 2024-04-25T08:32:53.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.827+0000 7f2f63fb0640 1 -- 172.21.15.60:0/856740658 wait complete. 2024-04-25T08:32:53.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.828+0000 7f2f63fb0640 1 Processor -- start 2024-04-25T08:32:53.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.828+0000 7f2f63fb0640 1 -- start start 2024-04-25T08:32:53.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b92a0 0x7f2f5c0b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c15dbe0 0x7f2f5c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0b6ab0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0d2210 con 0x7f2f5c15dbe0 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0b6c30 con 0x7f2f5c0b92a0 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43406/0 (socket says 172.21.15.60:43406) 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 -- 172.21.15.60:0/3037538589 learned_addr learned my addr 172.21.15.60:0/3037538589 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f6154b640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b92a0 0x7f2f5c0b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f6254d640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c15dbe0 0x7f2f5c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 -- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b92a0 msgr2=0x7f2f5c0b86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b92a0 0x7f2f5c0b86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 -- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c15dbe0 msgr2=0x7f2f5c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c15dbe0 0x7f2f5c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 -- 172.21.15.60:0/3037538589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0b32e0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f6154b640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b92a0 0x7f2f5c0b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f61d4c640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0b4fd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e9b0 tx=0x7f2f5800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58017070 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5800c9a0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c0b35c0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800cd90 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.829+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c0b3b80 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.830+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5c073990 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.831+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f5801a070 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.831+0000 7f2f527fc640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f4006b4d0 0x7f2f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.831+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2f5808a430 con 0x7f2f5c0b2cb0 2024-04-25T08:32:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.832+0000 7f2f6154b640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f4006b4d0 0x7f2f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.832+0000 7f2f6154b640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f4006b4d0 0x7f2f4006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c034000 tx=0x7f2f4c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:53.836+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f58059410 con 0x7f2f5c0b2cb0 2024-04-25T08:32:54.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.085+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f2f5c0b73a0 con 0x7f2f5c0b2cb0 2024-04-25T08:32:54.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.384+0000 7f2f527fc640 1 -- 172.21.15.60:0/3037538589 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v191) v1 ==== 143+0+0 (secure 0 0 0) 0x7f2f58058d10 con 0x7f2f5c0b2cb0 2024-04-25T08:32:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f4006b4d0 msgr2=0x7f2f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f4006b4d0 0x7f2f4006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c034000 tx=0x7f2f4c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 msgr2=0x7f2f5c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0b4fd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e9b0 tx=0x7f2f5800ee70 comp rx=0 tx=0).stop 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 shutdown_connections 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f4006b4d0 0x7f2f4006d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f5c15dbe0 0x7f2f5c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5c0b92a0 0x7f2f5c0b86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 --2- 172.21.15.60:0/3037538589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f5c0b2cb0 0x7f2f5c0b4fd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 >> 172.21.15.60:0/3037538589 conn(0x7f2f5c061a20 msgr2=0x7f2f5c0b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 shutdown_connections 2024-04-25T08:32:54.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.386+0000 7f2f63fb0640 1 -- 172.21.15.60:0/3037538589 wait complete. 2024-04-25T08:32:54.394 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T08:32:54.394 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:32:54.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.501+0000 7f981d9cf640 1 Processor -- start 2024-04-25T08:32:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.501+0000 7f981d9cf640 1 -- start start 2024-04-25T08:32:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f981d9cf640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f981815ba00 0x7f981809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f981d9cf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98181602e0 0x7f98180ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f981d9cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 0x7f981815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f981d9cf640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98180cd710 con 0x7f98181602e0 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f981d9cf640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9818099b70 con 0x7f981815ba00 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f981d9cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9818099e50 con 0x7f981815f5f0 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f98167fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98181602e0 0x7f98180ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f98167fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f98181602e0 0x7f98180ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43408/0 (socket says 172.21.15.60:43408) 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.502+0000 7f98167fc640 1 -- 172.21.15.60:0/1668825360 learned_addr learned my addr 172.21.15.60:0/1668825360 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f98167fc640 1 -- 172.21.15.60:0/1668825360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 msgr2=0x7f981815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f98167fc640 1 --2- 172.21.15.60:0/1668825360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 0x7f981815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f98167fc640 1 -- 172.21.15.60:0/1668825360 >> v2:172.21.15.100:3300/0 conn(0x7f981815ba00 msgr2=0x7f981809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f9816ffd640 1 --2- 172.21.15.60:0/1668825360 >> v2:172.21.15.100:3300/0 conn(0x7f981815ba00 0x7f981809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f98167fc640 1 --2- 172.21.15.60:0/1668825360 >> v2:172.21.15.100:3300/0 conn(0x7f981815ba00 0x7f981809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f98167fc640 1 -- 172.21.15.60:0/1668825360 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981809a130 con 0x7f98181602e0 2024-04-25T08:32:54.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f9816ffd640 1 --2- 172.21.15.60:0/1668825360 >> v2:172.21.15.100:3300/0 conn(0x7f981815ba00 0x7f981809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.503+0000 7f98167fc640 1 --2- 172.21.15.60:0/1668825360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98181602e0 0x7f98180ccda0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f98180b4370 tx=0x7f980c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd07f4a3b694fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f9815ffb640 1 -- 172.21.15.60:0/1668825360 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c037070 con 0x7f98181602e0 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f9815ffb640 1 -- 172.21.15.60:0/1668825360 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c02ae00 con 0x7f98181602e0 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f9815ffb640 1 -- 172.21.15.60:0/1668825360 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c032660 con 0x7f98181602e0 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 -- 172.21.15.60:0/1668825360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98181602e0 msgr2=0x7f98180ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 --2- 172.21.15.60:0/1668825360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98181602e0 0x7f98180ccda0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f98180b4370 tx=0x7f980c02a0b0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 -- 172.21.15.60:0/1668825360 shutdown_connections 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 --2- 172.21.15.60:0/1668825360 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 0x7f981815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 --2- 172.21.15.60:0/1668825360 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f98181602e0 0x7f98180ccda0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 --2- 172.21.15.60:0/1668825360 >> v2:172.21.15.100:3300/0 conn(0x7f981815ba00 0x7f981809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 -- 172.21.15.60:0/1668825360 >> 172.21.15.60:0/1668825360 conn(0x7f98180b3270 msgr2=0x7f98180b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 -- 172.21.15.60:0/1668825360 shutdown_connections 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.504+0000 7f981d9cf640 1 -- 172.21.15.60:0/1668825360 wait complete. 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.505+0000 7f981d9cf640 1 Processor -- start 2024-04-25T08:32:54.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 -- start start 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 0x7f981814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 0x7f981806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98181602e0 0x7f981815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981815bde0 con 0x7f981815ba00 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180cd2e0 con 0x7f98181602e0 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f981d9cf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98180b4310 con 0x7f981815f5f0 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.506+0000 7f9816ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 0x7f981814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 0x7f981814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43418/0 (socket says 172.21.15.60:43418) 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 -- 172.21.15.60:0/3212876954 learned_addr learned my addr 172.21.15.60:0/3212876954 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 -- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 msgr2=0x7f981806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 0x7f981806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 -- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98181602e0 msgr2=0x7f981815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98181602e0 0x7f981815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 -- 172.21.15.60:0/3212876954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180b5600 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f9816ffd640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 0x7f981814ed80 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f981000ce30 tx=0x7f981000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000f890 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981000fa30 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981806d750 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000fe00 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.507+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180b4840 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.508+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9818073990 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.509+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9810018400 con 0x7f981815ba00 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.509+0000 7f97fffff640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f980406b4d0 0x7f980406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.509+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f981008ad70 con 0x7f981815ba00 2024-04-25T08:32:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.509+0000 7f98167fc640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f980406b4d0 0x7f980406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.510+0000 7f98167fc640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f980406b4d0 0x7f980406d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f980c034000 tx=0x7f980c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.514+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9810014030 con 0x7f981815ba00 2024-04-25T08:32:54.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.695+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98180b40a0 con 0x7f981815ba00 2024-04-25T08:32:54.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.696+0000 7f97fffff640 1 -- 172.21.15.60:0/3212876954 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13281 (secure 0 0 0) 0x7f981000ae20 con 0x7f981815ba00 2024-04-25T08:32:54.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:54.692 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:32:53.379883+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1977192273":"2024-04-26T08:32:50.372093+0000","172.21.15.60:6833/1977192273":"2024-04-26T08:32:50.372093+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.697+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f980406b4d0 msgr2=0x7f980406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f980406b4d0 0x7f980406d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f980c034000 tx=0x7f980c02c000 comp rx=0 tx=0).stop 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 msgr2=0x7f981814ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 0x7f981814ed80 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f981000ce30 tx=0x7f981000a650 comp rx=0 tx=0).stop 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 shutdown_connections 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f980406b4d0 0x7f980406d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98181602e0 0x7f981815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f981815f5f0 0x7f981806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 --2- 172.21.15.60:0/3212876954 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f981815ba00 0x7f981814ed80 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 >> 172.21.15.60:0/3212876954 conn(0x7f98180b3270 msgr2=0x7f9818070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 shutdown_connections 2024-04-25T08:32:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.698+0000 7f981d9cf640 1 -- 172.21.15.60:0/3212876954 wait complete. 2024-04-25T08:32:54.705 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:32:54.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:54.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.812+0000 7fc735b34640 1 Processor -- start 2024-04-25T08:32:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.812+0000 7fc735b34640 1 -- start start 2024-04-25T08:32:54.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc735b34640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7301610f0 0x7fc73006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc735b34640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc7301616f0 0x7fc730161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc735b34640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7300c6960 con 0x7fc7300ce0a0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc735b34640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7301614d0 con 0x7fc7301610f0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc735b34640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc730162260 con 0x7fc7301616f0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc72f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7301610f0 0x7fc73006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc72f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7301610f0 0x7fc73006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43434/0 (socket says 172.21.15.60:43434) 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.813+0000 7fc72f7fe640 1 -- 172.21.15.60:0/3947174332 learned_addr learned my addr 172.21.15.60:0/3947174332 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72effd640 1 --2- 172.21.15.60:0/3947174332 >> v2:172.21.15.100:3300/0 conn(0x7fc7301616f0 0x7fc730161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72f7fe640 1 -- 172.21.15.60:0/3947174332 >> v2:172.21.15.100:3300/0 conn(0x7fc7301616f0 msgr2=0x7fc730161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72f7fe640 1 --2- 172.21.15.60:0/3947174332 >> v2:172.21.15.100:3300/0 conn(0x7fc7301616f0 0x7fc730161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72f7fe640 1 -- 172.21.15.60:0/3947174332 >> v1:172.21.15.100:6789/0 conn(0x7fc7300ce0a0 legacy=0x7fc7300d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72f7fe640 1 -- 172.21.15.60:0/3947174332 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730162540 con 0x7fc7301610f0 2024-04-25T08:32:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72effd640 1 --2- 172.21.15.60:0/3947174332 >> v2:172.21.15.100:3300/0 conn(0x7fc7301616f0 0x7fc730161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.814+0000 7fc72f7fe640 1 --2- 172.21.15.60:0/3947174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301610f0 0x7fc73006c1c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc73006f220 tx=0x7fc72002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e164fa0dd34311e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc72e7fc640 1 -- 172.21.15.60:0/3947174332 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720037070 con 0x7fc7301610f0 2024-04-25T08:32:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc72e7fc640 1 -- 172.21.15.60:0/3947174332 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720032440 con 0x7fc7301610f0 2024-04-25T08:32:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc72e7fc640 1 -- 172.21.15.60:0/3947174332 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720032810 con 0x7fc7301610f0 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 -- 172.21.15.60:0/3947174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301610f0 msgr2=0x7fc73006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 --2- 172.21.15.60:0/3947174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301610f0 0x7fc73006c1c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc73006f220 tx=0x7fc72002a230 comp rx=0 tx=0).stop 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 -- 172.21.15.60:0/3947174332 shutdown_connections 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 --2- 172.21.15.60:0/3947174332 >> v2:172.21.15.100:3300/0 conn(0x7fc7301616f0 0x7fc730161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 --2- 172.21.15.60:0/3947174332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301610f0 0x7fc73006c1c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 -- 172.21.15.60:0/3947174332 >> 172.21.15.60:0/3947174332 conn(0x7fc73006f500 msgr2=0x7fc73006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 -- 172.21.15.60:0/3947174332 shutdown_connections 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.815+0000 7fc735b34640 1 -- 172.21.15.60:0/3947174332 wait complete. 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.816+0000 7fc735b34640 1 Processor -- start 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 -- start start 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7300ce0a0 0x7fc7300b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7301610f0 0x7fc730152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 0x7fc7300ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300cf690 con 0x7fc7301616f0 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300cfce0 con 0x7fc7301610f0 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc735b34640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73006c700 con 0x7fc7300ce0a0 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc72ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 0x7fc7300ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc72ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 0x7fc7300ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43448/0 (socket says 172.21.15.60:43448) 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.817+0000 7fc72ffff640 1 -- 172.21.15.60:0/3439383606 learned_addr learned my addr 172.21.15.60:0/3439383606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72f7fe640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7300ce0a0 0x7fc7300b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72effd640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7301610f0 0x7fc730152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72ffff640 1 -- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7300ce0a0 msgr2=0x7fc7300b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72ffff640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7300ce0a0 0x7fc7300b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72ffff640 1 -- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7301610f0 msgr2=0x7fc730152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72ffff640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7301610f0 0x7fc730152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72ffff640 1 -- 172.21.15.60:0/3439383606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300afc40 con 0x7fc7301616f0 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc72ffff640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 0x7fc7300ad110 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c7b0 tx=0x7fc72800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728004b60 con 0x7fc7301616f0 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728004d00 con 0x7fc7301616f0 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7300aff20 con 0x7fc7301616f0 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800aa90 con 0x7fc7301616f0 2024-04-25T08:32:54.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.818+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300b04e0 con 0x7fc7301616f0 2024-04-25T08:32:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.819+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc730073a00 con 0x7fc7301616f0 2024-04-25T08:32:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.820+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc72800ac30 con 0x7fc7301616f0 2024-04-25T08:32:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.820+0000 7fc717fff640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.820+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc72808a140 con 0x7fc7301616f0 2024-04-25T08:32:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.820+0000 7fc72f7fe640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.821+0000 7fc72f7fe640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc71c06b4d0 0x7fc71c06d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc720034000 tx=0x7fc72002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:54.824+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc728020070 con 0x7fc7301616f0 2024-04-25T08:32:55.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.005+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7300cf150 con 0x7fc7301616f0 2024-04-25T08:32:55.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.006+0000 7fc717fff640 1 -- 172.21.15.60:0/3439383606 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc728058b70 con 0x7fc7301616f0 2024-04-25T08:32:55.002 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:32:55.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc71c06b4d0 msgr2=0x7fc71c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc71c06b4d0 0x7fc71c06d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc720034000 tx=0x7fc72002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:55.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 msgr2=0x7fc7300ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 0x7fc7300ad110 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c7b0 tx=0x7fc72800cc70 comp rx=0 tx=0).stop 2024-04-25T08:32:55.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 shutdown_connections 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7301616f0 0x7fc7300ad110 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7301610f0 0x7fc730152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 --2- 172.21.15.60:0/3439383606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7300ce0a0 0x7fc7300b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 >> 172.21.15.60:0/3439383606 conn(0x7fc73006f500 msgr2=0x7fc7300b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 shutdown_connections 2024-04-25T08:32:55.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.008+0000 7fc735b34640 1 -- 172.21.15.60:0/3439383606 wait complete. 2024-04-25T08:32:55.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T08:32:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.141+0000 7f0c1be4c640 1 Processor -- start 2024-04-25T08:32:55.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.141+0000 7f0c1be4c640 1 -- start start 2024-04-25T08:32:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.141+0000 7f0c1be4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1415db70 0x7f0c14162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.141+0000 7f0c1be4c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c140b6cb0 0x7f0c140b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c1be4c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0c140b5a60 0x7f0c1406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c1be4c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c14162690 con 0x7f0c140b6cb0 2024-04-25T08:32:55.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c1be4c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140ceee0 con 0x7f0c140b5a60 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c1be4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140d1d10 con 0x7f0c1415db70 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c140b6cb0 0x7f0c140b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c140b6cb0 0x7f0c140b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43452/0 (socket says 172.21.15.60:43452) 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 -- 172.21.15.60:0/908628070 learned_addr learned my addr 172.21.15.60:0/908628070 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 -- 172.21.15.60:0/908628070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1415db70 msgr2=0x7f0c14162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 --2- 172.21.15.60:0/908628070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1415db70 0x7f0c14162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 -- 172.21.15.60:0/908628070 >> v2:172.21.15.100:3300/0 conn(0x7f0c140b5a60 msgr2=0x7f0c1406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c1a3e9640 1 --2- 172.21.15.60:0/908628070 >> v2:172.21.15.100:3300/0 conn(0x7f0c140b5a60 0x7f0c1406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 --2- 172.21.15.60:0/908628070 >> v2:172.21.15.100:3300/0 conn(0x7f0c140b5a60 0x7f0c1406c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c193e7640 1 -- 172.21.15.60:0/908628070 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140d1ff0 con 0x7f0c140b6cb0 2024-04-25T08:32:55.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.142+0000 7f0c1a3e9640 1 --2- 172.21.15.60:0/908628070 >> v2:172.21.15.100:3300/0 conn(0x7f0c140b5a60 0x7f0c1406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:55.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c193e7640 1 --2- 172.21.15.60:0/908628070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140b6cb0 0x7f0c140b5520 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0c1408bfb0 tx=0x7f0c0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ce27c5adf55ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c18be6640 1 -- 172.21.15.60:0/908628070 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04037070 con 0x7f0c140b6cb0 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c18be6640 1 -- 172.21.15.60:0/908628070 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04032440 con 0x7f0c140b6cb0 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c18be6640 1 -- 172.21.15.60:0/908628070 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04032810 con 0x7f0c140b6cb0 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 -- 172.21.15.60:0/908628070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140b6cb0 msgr2=0x7f0c140b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/908628070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140b6cb0 0x7f0c140b5520 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0c1408bfb0 tx=0x7f0c0402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 -- 172.21.15.60:0/908628070 shutdown_connections 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/908628070 >> v2:172.21.15.100:3300/0 conn(0x7f0c140b5a60 0x7f0c1406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/908628070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140b6cb0 0x7f0c140b5520 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/908628070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1415db70 0x7f0c14162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.143+0000 7f0c1be4c640 1 -- 172.21.15.60:0/908628070 >> 172.21.15.60:0/908628070 conn(0x7f0c140d0520 msgr2=0x7f0c140d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.144+0000 7f0c1be4c640 1 -- 172.21.15.60:0/908628070 shutdown_connections 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.144+0000 7f0c1be4c640 1 -- 172.21.15.60:0/908628070 wait complete. 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 Processor -- start 2024-04-25T08:32:55.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 -- start start 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c140b5a60 0x7f0c140cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140b6cb0 0x7f0c14160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 0x7f0c14156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140ce530 con 0x7f0c1415db70 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415df50 con 0x7f0c140b5a60 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.145+0000 7f0c1be4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140cf1c0 con 0x7f0c140b6cb0 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 0x7f0c14156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 0x7f0c14156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43462/0 (socket says 172.21.15.60:43462) 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 -- 172.21.15.60:0/1787127052 learned_addr learned my addr 172.21.15.60:0/1787127052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c193e7640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140b6cb0 0x7f0c14160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c19be8640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c140b5a60 0x7f0c140cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 -- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140b6cb0 msgr2=0x7f0c14160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140b6cb0 0x7f0c14160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 -- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c140b5a60 msgr2=0x7f0c140cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c140b5a60 0x7f0c140cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 -- 172.21.15.60:0/1787127052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1415f2c0 con 0x7f0c1415db70 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1a3e9640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 0x7f0c14156b20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00cf20 tx=0x7f0c0c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c01a070 con 0x7f0c1415db70 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00d840 con 0x7f0c1415db70 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1415f560 con 0x7f0c1415db70 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00dc30 con 0x7f0c1415db70 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.146+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1415fb20 con 0x7f0c1415db70 2024-04-25T08:32:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.147+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c14073990 con 0x7f0c1415db70 2024-04-25T08:32:55.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.148+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c0c01d070 con 0x7f0c1415db70 2024-04-25T08:32:55.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.148+0000 7f0c0a7fc640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.148+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0c0c08a810 con 0x7f0c1415db70 2024-04-25T08:32:55.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.148+0000 7f0c19be8640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.149+0000 7f0c19be8640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bf806b4d0 0x7f0bf806d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0c1415cc40 tx=0x7f0c10009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.153+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c0c059240 con 0x7f0c1415db70 2024-04-25T08:32:55.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:55.401+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f0c140b6650 con 0x7f0c1415db70 2024-04-25T08:32:56.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:56.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:56.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.389+0000 7f0c0a7fc640 1 -- 172.21.15.60:0/1787127052 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v193) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0c0c05d2e0 con 0x7f0c1415db70 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bf806b4d0 msgr2=0x7f0bf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bf806b4d0 0x7f0bf806d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0c1415cc40 tx=0x7f0c10009060 comp rx=0 tx=0).stop 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 msgr2=0x7f0c14156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 0x7f0c14156b20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00cf20 tx=0x7f0c0c00a7d0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 shutdown_connections 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c1415db70 0x7f0c14156b20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140b6cb0 0x7f0c14160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 --2- 172.21.15.60:0/1787127052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c140b5a60 0x7f0c140cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 >> 172.21.15.60:0/1787127052 conn(0x7f0c140d0520 msgr2=0x7f0c1415cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 shutdown_connections 2024-04-25T08:32:56.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.392+0000 7f0c1be4c640 1 -- 172.21.15.60:0/1787127052 wait complete. 2024-04-25T08:32:56.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:32:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.516+0000 7f25c895c640 1 Processor -- start 2024-04-25T08:32:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.516+0000 7f25c895c640 1 -- start start 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c895c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25c40cc4f0 0x7f25c40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c895c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c4072280 con 0x7f25c415b980 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c895c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c40714c0 con 0x7f25c40cdb20 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c895c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c4071270 con 0x7f25c40cc4f0 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c2d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f25c415b980 0x7f25c40ae500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48380/0 (socket says 172.21.15.60:48380) 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c2d9d640 1 -- 172.21.15.60:0/3375114526 learned_addr learned my addr 172.21.15.60:0/3375114526 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:56.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c259c640 1 --2- 172.21.15.60:0/3375114526 >> v2:172.21.15.100:3300/0 conn(0x7f25c40cc4f0 0x7f25c40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862796484 0 0) 0x7f25c4072280 con 0x7f25c415b980 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.517+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ac003520 con 0x7f25c415b980 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4275301091 0 0) 0x7f25ac003520 con 0x7f25c415b980 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 >> v2:172.21.15.100:3300/0 conn(0x7f25c40cc4f0 msgr2=0x7f25c40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 --2- 172.21.15.60:0/3375114526 >> v2:172.21.15.100:3300/0 conn(0x7f25c40cc4f0 0x7f25c40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 >> v1:172.21.15.100:6789/0 conn(0x7f25c40cdb20 legacy=0x7f25c415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40717a0 con 0x7f25c415b980 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f25b40019f0 con 0x7f25c415b980 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 >> v1:172.21.15.60:6789/0 conn(0x7f25c415b980 legacy=0x7f25c40ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25ac0039b0 0x7f25ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25ac0043b0 0x7f25ac004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac005080 0x7f25ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4071270 con 0x7f25ac0043b0 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac00bfe0 con 0x7f25ac0039b0 2024-04-25T08:32:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac00c1e0 con 0x7f25ac005080 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c1d9b640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25ac0043b0 0x7f25ac004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c259c640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25ac0039b0 0x7f25ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c2d9d640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac005080 0x7f25ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c1d9b640 1 -- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac005080 msgr2=0x7f25ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c1d9b640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac005080 0x7f25ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c1d9b640 1 -- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25ac0039b0 msgr2=0x7f25ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c1d9b640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25ac0039b0 0x7f25ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c1d9b640 1 -- 172.21.15.60:0/3375114526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac003520 con 0x7f25ac0043b0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.518+0000 7f25c2d9d640 1 -- 172.21.15.60:0/3375114526 reap_dead start 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.519+0000 7f25c1d9b640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25ac0043b0 0x7f25ac004880 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f25b8001910 tx=0x7f25ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f0374c17bfb06d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.519+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8037070 con 0x7f25ac0043b0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.519+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b802ac60 con 0x7f25ac0043b0 2024-04-25T08:32:56.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.519+0000 7f25c159a640 1 -- 172.21.15.60:0/3375114526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80014e0 con 0x7f25ac0043b0 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 -- 172.21.15.60:0/3375114526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25ac0043b0 msgr2=0x7f25ac004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25ac0043b0 0x7f25ac004880 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f25b8001910 tx=0x7f25ac005830 comp rx=0 tx=0).stop 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 -- 172.21.15.60:0/3375114526 shutdown_connections 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 --2- 172.21.15.60:0/3375114526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25ac0043b0 0x7f25ac004880 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 -- 172.21.15.60:0/3375114526 >> 172.21.15.60:0/3375114526 conn(0x7f25c406abe0 msgr2=0x7f25c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 -- 172.21.15.60:0/3375114526 shutdown_connections 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.520+0000 7f25c895c640 1 -- 172.21.15.60:0/3375114526 wait complete. 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.521+0000 7f25c895c640 1 Processor -- start 2024-04-25T08:32:56.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.521+0000 7f25c895c640 1 -- start start 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c895c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac0043b0 0x7f25c40cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c895c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c415d5c0 0x7f25c4150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c895c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 0x7f25c41f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c895c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4071d60 con 0x7f25c41f6c10 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c895c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415bd80 con 0x7f25c415d5c0 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c895c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c40cc8d0 con 0x7f25ac0043b0 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 0x7f25c41f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 0x7f25c41f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43474/0 (socket says 172.21.15.60:43474) 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.522+0000 7f25c2d9d640 1 -- 172.21.15.60:0/1281083269 learned_addr learned my addr 172.21.15.60:0/1281083269 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c259c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac0043b0 0x7f25c40cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c1d9b640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c415d5c0 0x7f25c4150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c2d9d640 1 -- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac0043b0 msgr2=0x7f25c40cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c2d9d640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac0043b0 0x7f25c40cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c2d9d640 1 -- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c415d5c0 msgr2=0x7f25c4150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c2d9d640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c415d5c0 0x7f25c4150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c2d9d640 1 -- 172.21.15.60:0/1281083269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c415e020 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c1d9b640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c415d5c0 0x7f25c4150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c2d9d640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 0x7f25c41f7090 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f25c406b930 tx=0x7f25b4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400ac10 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4018070 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c4157d50 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40133f0 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.523+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c4157a50 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.524+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c415db30 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.524+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25b40135b0 con 0x7f25c41f6c10 2024-04-25T08:32:56.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.525+0000 7f25b2ffd640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.525+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f25b408a5f0 con 0x7f25c41f6c10 2024-04-25T08:32:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.525+0000 7f25c259c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:56.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.526+0000 7f25c259c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009a00 tx=0x7f25bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:56.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.529+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25b40595e0 con 0x7f25c41f6c10 2024-04-25T08:32:56.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.709+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25c4073a00 con 0x7f25c41f6c10 2024-04-25T08:32:56.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.712+0000 7f25b2ffd640 1 -- 172.21.15.60:0/1281083269 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f25b4058ee0 con 0x7f25c41f6c10 2024-04-25T08:32:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 msgr2=0x7f25a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009a00 tx=0x7f25bc008040 comp rx=0 tx=0).stop 2024-04-25T08:32:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 msgr2=0x7f25c41f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 0x7f25c41f7090 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f25c406b930 tx=0x7f25b4007bd0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 shutdown_connections 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25c41f6c10 0x7f25c41f7090 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25c415d5c0 0x7f25c4150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 --2- 172.21.15.60:0/1281083269 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25ac0043b0 0x7f25c40cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 >> 172.21.15.60:0/1281083269 conn(0x7f25c406abe0 msgr2=0x7f25c406e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 shutdown_connections 2024-04-25T08:32:56.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:56.714+0000 7f25c895c640 1 -- 172.21.15.60:0/1281083269 wait complete. 2024-04-25T08:32:56.721 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:32:56.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:56.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:56.815 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:32:56.815 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:56.815 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:56.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:32:56.831 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:32:56.831 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:56.849 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:32:56.849 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:32:56.849 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:56.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:32:56.886 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:32:56.886 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:56.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:32:56.906 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:32:56.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:57.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.030+0000 7f2b5b4fc640 1 Processor -- start 2024-04-25T08:32:57.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.030+0000 7f2b5b4fc640 1 -- start start 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b5b4fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415ba00 0x7f2b5409a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b5b4fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b541602e0 0x7f2b540ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b5b4fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b540cd710 con 0x7f2b5415f5f0 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b5b4fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b54099b70 con 0x7f2b541602e0 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b5b4fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54099e50 con 0x7f2b5415ba00 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b59a99640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2b5415f5f0 0x7f2b540c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48386/0 (socket says 172.21.15.60:48386) 2024-04-25T08:32:57.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.031+0000 7f2b59a99640 1 -- 172.21.15.60:0/3017711638 learned_addr learned my addr 172.21.15.60:0/3017711638 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b58a97640 1 --2- 172.21.15.60:0/3017711638 >> v2:172.21.15.100:3300/0 conn(0x7f2b541602e0 0x7f2b540ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b59298640 1 --2- 172.21.15.60:0/3017711638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415ba00 0x7f2b5409a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b53fff640 1 -- 172.21.15.60:0/3017711638 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949507377 0 0) 0x7f2b44001490 con 0x7f2b5415f5f0 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b53fff640 1 -- 172.21.15.60:0/3017711638 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c003520 con 0x7f2b5415f5f0 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b59298640 1 -- 172.21.15.60:0/3017711638 >> v2:172.21.15.100:3300/0 conn(0x7f2b541602e0 msgr2=0x7f2b540ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b59298640 1 --2- 172.21.15.60:0/3017711638 >> v2:172.21.15.100:3300/0 conn(0x7f2b541602e0 0x7f2b540ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b59298640 1 -- 172.21.15.60:0/3017711638 >> v1:172.21.15.60:6789/0 conn(0x7f2b5415f5f0 legacy=0x7f2b540c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b59298640 1 -- 172.21.15.60:0/3017711638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5409a130 con 0x7f2b5415ba00 2024-04-25T08:32:57.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.032+0000 7f2b58a97640 1 --2- 172.21.15.60:0/3017711638 >> v2:172.21.15.100:3300/0 conn(0x7f2b541602e0 0x7f2b540ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b59298640 1 --2- 172.21.15.60:0/3017711638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415ba00 0x7f2b5409a930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0097a0 tx=0x7f2b4c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fe3706c7a486400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b53fff640 1 -- 172.21.15.60:0/3017711638 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c037070 con 0x7f2b5415ba00 2024-04-25T08:32:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b53fff640 1 -- 172.21.15.60:0/3017711638 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c02adb0 con 0x7f2b5415ba00 2024-04-25T08:32:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b53fff640 1 -- 172.21.15.60:0/3017711638 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c032640 con 0x7f2b5415ba00 2024-04-25T08:32:57.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/3017711638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415ba00 msgr2=0x7f2b5409a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/3017711638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415ba00 0x7f2b5409a930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0097a0 tx=0x7f2b4c02a060 comp rx=0 tx=0).stop 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/3017711638 shutdown_connections 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/3017711638 >> v2:172.21.15.100:3300/0 conn(0x7f2b541602e0 0x7f2b540ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/3017711638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415ba00 0x7f2b5409a930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.033+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/3017711638 >> 172.21.15.60:0/3017711638 conn(0x7f2b540b3270 msgr2=0x7f2b540b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.034+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/3017711638 shutdown_connections 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.034+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/3017711638 wait complete. 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 Processor -- start 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 -- start start 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5415ba00 0x7f2b5415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415f5f0 0x7f2b5415d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 0x7f2b540c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5415bde0 con 0x7f2b541602e0 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540cd2e0 con 0x7f2b5415ba00 2024-04-25T08:32:57.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.035+0000 7f2b5b4fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540b4d40 con 0x7f2b5415f5f0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 0x7f2b540c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 0x7f2b540c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43490/0 (socket says 172.21.15.60:43490) 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 -- 172.21.15.60:0/2912287806 learned_addr learned my addr 172.21.15.60:0/2912287806 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b58a97640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415f5f0 0x7f2b5415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59298640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5415ba00 0x7f2b5415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 -- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415f5f0 msgr2=0x7f2b5415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415f5f0 0x7f2b5415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 -- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5415ba00 msgr2=0x7f2b5415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5415ba00 0x7f2b5415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 -- 172.21.15.60:0/2912287806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54158590 con 0x7f2b541602e0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b59a99640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 0x7f2b540c8ba0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400cb10 tx=0x7f2b4400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44017070 con 0x7f2b541602e0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4400ac10 con 0x7f2b541602e0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44012400 con 0x7f2b541602e0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.036+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5415eee0 con 0x7f2b541602e0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.037+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b540b46f0 con 0x7f2b541602e0 2024-04-25T08:32:57.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.037+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b54073990 con 0x7f2b541602e0 2024-04-25T08:32:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.038+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b4401a070 con 0x7f2b541602e0 2024-04-25T08:32:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.038+0000 7f2b51ffb640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b4006b4d0 0x7f2b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.038+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2b44089e30 con 0x7f2b541602e0 2024-04-25T08:32:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.039+0000 7f2b59298640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b4006b4d0 0x7f2b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.039+0000 7f2b59298640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b4006b4d0 0x7f2b4006d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c034000 tx=0x7f2b4c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:57.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.043+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b540cd710 con 0x7f2b541602e0 2024-04-25T08:32:57.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.260+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b540ced30 con 0x7f2b541602e0 2024-04-25T08:32:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.261+0000 7f2b51ffb640 1 -- 172.21.15.60:0/2912287806 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f2b44058e20 con 0x7f2b541602e0 2024-04-25T08:32:57.257 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:32:57.257 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2522403646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2522403646},{"type":"v1","addr":"172.21.15.100:6835","nonce":2522403646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5560,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3108273436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3108273436},{"type":"v1","addr":"172.21.15.100:6837","nonce":3108273436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5883,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3902409528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3902409528},{"type":"v1","addr":"172.21.15.60:6834","nonce":3902409528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1958948945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1958948945},{"type":"v1","addr":"172.21.15.60:6835","nonce":1958948945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2024-04-25T08:32:57.257 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 28 2024-04-25T08:32:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b4006b4d0 msgr2=0x7f2b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b4006b4d0 0x7f2b4006d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c034000 tx=0x7f2b4c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 msgr2=0x7f2b540c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 0x7f2b540c8ba0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400cb10 tx=0x7f2b4400a060 comp rx=0 tx=0).stop 2024-04-25T08:32:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 shutdown_connections 2024-04-25T08:32:57.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b541602e0 0x7f2b540c8ba0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b5415f5f0 0x7f2b5415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2b4006b4d0 0x7f2b4006d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 --2- 172.21.15.60:0/2912287806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b5415ba00 0x7f2b5415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 >> 172.21.15.60:0/2912287806 conn(0x7f2b540b3270 msgr2=0x7f2b540c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 shutdown_connections 2024-04-25T08:32:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:32:57.263+0000 7f2b5b4fc640 1 -- 172.21.15.60:0/2912287806 wait complete. 2024-04-25T08:32:57.270 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:32:57.271 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:32:57.271 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:32:57.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:57.271 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:32:57.271 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:32:57.271 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:32:57.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:57.271 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:32:57.272 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:32:57.272 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:32:57.272 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:57.272 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:32:57.272 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:32:57.272 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:32:57.272 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:57.280 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:32:57.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:32:57.281 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:32:57.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:32:57.287 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:32:57.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:32:57.323 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:32:57.323 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:57.324 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:32:57.324 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:57.331 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:32:57.331 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:57.397 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:32:57.397 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:32:57.411 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:32:57.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T08:32:57.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T08:32:57.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T08:33:03.288 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:33:03.288 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:33:03.291 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:33:03.292 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:03.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:33:03.374 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:33:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.411+0000 7f02345f5640 1 Processor -- start 2024-04-25T08:33:03.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.411+0000 7f02345f5640 1 -- start start 2024-04-25T08:33:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f02345f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c0ced80 0x7f022c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:03.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f02345f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c15e000 con 0x7f022c161710 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f02345f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c0d1de0 con 0x7f022c15dc20 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f02345f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c161c60 con 0x7f022c0ced80 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f0232b92640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f022c161710 0x7f022c0d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37404/0 (socket says 172.21.15.60:37404) 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f0232b92640 1 -- 172.21.15.60:0/1081441068 learned_addr learned my addr 172.21.15.60:0/1081441068 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f0231b90640 1 --2- 172.21.15.60:0/1081441068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c0ced80 0x7f022c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f023138f640 1 -- 172.21.15.60:0/1081441068 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506626873 0 0) 0x7f022c15e000 con 0x7f022c161710 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f023138f640 1 -- 172.21.15.60:0/1081441068 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0218003520 con 0x7f022c161710 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f023138f640 1 -- 172.21.15.60:0/1081441068 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769639868 0 0) 0x7f022c0d1de0 con 0x7f022c15dc20 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.412+0000 7f0231b90640 1 -- 172.21.15.60:0/1081441068 >> v1:172.21.15.100:6789/0 conn(0x7f022c15dc20 legacy=0x7f022c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.413+0000 7f0231b90640 1 -- 172.21.15.60:0/1081441068 >> v1:172.21.15.60:6789/0 conn(0x7f022c161710 legacy=0x7f022c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:03.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.413+0000 7f0231b90640 1 -- 172.21.15.60:0/1081441068 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c161f40 con 0x7f022c0ced80 2024-04-25T08:33:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.413+0000 7f0231b90640 1 --2- 172.21.15.60:0/1081441068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c0ced80 0x7f022c0d22e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f021c0097d0 tx=0x7f021c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=954c2e9623de9e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f023138f640 1 -- 172.21.15.60:0/1081441068 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c037070 con 0x7f022c0ced80 2024-04-25T08:33:03.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f023138f640 1 -- 172.21.15.60:0/1081441068 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c032440 con 0x7f022c0ced80 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f023138f640 1 -- 172.21.15.60:0/1081441068 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c032810 con 0x7f022c0ced80 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 -- 172.21.15.60:0/1081441068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c0ced80 msgr2=0x7f022c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 --2- 172.21.15.60:0/1081441068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c0ced80 0x7f022c0d22e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f021c0097d0 tx=0x7f021c02a210 comp rx=0 tx=0).stop 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 -- 172.21.15.60:0/1081441068 shutdown_connections 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 --2- 172.21.15.60:0/1081441068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c0ced80 0x7f022c0d22e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 -- 172.21.15.60:0/1081441068 >> 172.21.15.60:0/1081441068 conn(0x7f022c0d0420 msgr2=0x7f022c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 -- 172.21.15.60:0/1081441068 shutdown_connections 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.414+0000 7f02345f5640 1 -- 172.21.15.60:0/1081441068 wait complete. 2024-04-25T08:33:03.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.415+0000 7f02345f5640 1 Processor -- start 2024-04-25T08:33:03.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 -- start start 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f022c0ced80 0x7f022c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c15dc20 0x7f022c160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 0x7f022c0aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b5e50 con 0x7f022c161710 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c162500 con 0x7f022c0ced80 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f02345f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b5fd0 con 0x7f022c15dc20 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f0232b92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 0x7f022c0aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f0232b92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 0x7f022c0aa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43186/0 (socket says 172.21.15.60:43186) 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.416+0000 7f0232b92640 1 -- 172.21.15.60:0/3957928823 learned_addr learned my addr 172.21.15.60:0/3957928823 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:03.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0231b90640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c15dc20 0x7f022c160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232391640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f022c0ced80 0x7f022c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232b92640 1 -- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c15dc20 msgr2=0x7f022c160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232b92640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c15dc20 0x7f022c160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232b92640 1 -- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f022c0ced80 msgr2=0x7f022c0cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232b92640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f022c0ced80 0x7f022c0cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232b92640 1 -- 172.21.15.60:0/3957928823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c161030 con 0x7f022c161710 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232391640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f022c0ced80 0x7f022c0cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0231b90640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c15dc20 0x7f022c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0232b92640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 0x7f022c0aa730 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f022400caa0 tx=0x7f022400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224017070 con 0x7f022c161710 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022400aba0 con 0x7f022c161710 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c0cd430 con 0x7f022c161710 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224012400 con 0x7f022c161710 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.417+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c0cce70 con 0x7f022c161710 2024-04-25T08:33:03.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.418+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c073a00 con 0x7f022c161710 2024-04-25T08:33:03.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.419+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f022401a070 con 0x7f022c161710 2024-04-25T08:33:03.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.419+0000 7f0222ffd640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f021006b4d0 0x7f021006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:03.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.419+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0224089da0 con 0x7f022c161710 2024-04-25T08:33:03.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.419+0000 7f0232391640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f021006b4d0 0x7f021006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:03.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.420+0000 7f0232391640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f021006b4d0 0x7f021006d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0228000f60 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:03.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.424+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0224058da0 con 0x7f022c161710 2024-04-25T08:33:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:03.620+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f022c0d16b0 con 0x7f022c161710 2024-04-25T08:33:04.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T08:33:04.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T08:33:04.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T08:33:04.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.511+0000 7f0222ffd640 1 -- 172.21.15.60:0/3957928823 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) v1 ==== 97+0+0 (secure 0 0 0) 0x7f02240586a0 con 0x7f022c161710 2024-04-25T08:33:04.507 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f021006b4d0 msgr2=0x7f021006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f021006b4d0 0x7f021006d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0228000f60 tx=0x7f0228009040 comp rx=0 tx=0).stop 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 msgr2=0x7f022c0aa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 0x7f022c0aa730 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f022400caa0 tx=0x7f022400cf60 comp rx=0 tx=0).stop 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 shutdown_connections 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f022c161710 0x7f022c0aa730 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f022c15dc20 0x7f022c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f021006b4d0 0x7f021006d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 --2- 172.21.15.60:0/3957928823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f022c0ced80 0x7f022c0cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 >> 172.21.15.60:0/3957928823 conn(0x7f022c0d0420 msgr2=0x7f022c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 shutdown_connections 2024-04-25T08:33:04.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.514+0000 7f02345f5640 1 -- 172.21.15.60:0/3957928823 wait complete. 2024-04-25T08:33:04.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:33:04.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.635+0000 7f97fd81c640 1 Processor -- start 2024-04-25T08:33:04.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.635+0000 7f97fd81c640 1 -- start start 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97fd81c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97f815dd30 0x7f97f80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97fd81c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97f80b9770 0x7f97f80d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97fd81c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80d1df0 con 0x7f97f80d2e90 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97fd81c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80b7a50 con 0x7f97f80b9770 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97fd81c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80d20d0 con 0x7f97f815dd30 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97effff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97f80b9770 0x7f97f80d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:04.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97f77fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f97f80d2e90 0x7f97f80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37416/0 (socket says 172.21.15.60:37416) 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97f77fe640 1 -- 172.21.15.60:0/1281436719 learned_addr learned my addr 172.21.15.60:0/1281436719 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97effff640 1 -- 172.21.15.60:0/1281436719 >> v2:172.21.15.100:3300/0 conn(0x7f97f815dd30 msgr2=0x7f97f80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97f6ffd640 1 --2- 172.21.15.60:0/1281436719 >> v2:172.21.15.100:3300/0 conn(0x7f97f815dd30 0x7f97f80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97effff640 1 --2- 172.21.15.60:0/1281436719 >> v2:172.21.15.100:3300/0 conn(0x7f97f815dd30 0x7f97f80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97f67fc640 1 -- 172.21.15.60:0/1281436719 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143984117 0 0) 0x7f97f80d1df0 con 0x7f97f80d2e90 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97effff640 1 -- 172.21.15.60:0/1281436719 >> v1:172.21.15.60:6789/0 conn(0x7f97f80d2e90 legacy=0x7f97f80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97effff640 1 -- 172.21.15.60:0/1281436719 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80d23b0 con 0x7f97f80b9770 2024-04-25T08:33:04.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.636+0000 7f97f6ffd640 1 --2- 172.21.15.60:0/1281436719 >> v2:172.21.15.100:3300/0 conn(0x7f97f815dd30 0x7f97f80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:04.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.637+0000 7f97effff640 1 --2- 172.21.15.60:0/1281436719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80b9770 0x7f97f80d03c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97f806d000 tx=0x7f97e002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0cd9f6ad40d850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.637+0000 7f97f67fc640 1 -- 172.21.15.60:0/1281436719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0037070 con 0x7f97f80b9770 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.637+0000 7f97f67fc640 1 -- 172.21.15.60:0/1281436719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e002add0 con 0x7f97f80b9770 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.637+0000 7f97f67fc640 1 -- 172.21.15.60:0/1281436719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0032660 con 0x7f97f80b9770 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 -- 172.21.15.60:0/1281436719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80b9770 msgr2=0x7f97f80d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 --2- 172.21.15.60:0/1281436719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80b9770 0x7f97f80d03c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f97f806d000 tx=0x7f97e002a080 comp rx=0 tx=0).stop 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 -- 172.21.15.60:0/1281436719 shutdown_connections 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 --2- 172.21.15.60:0/1281436719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80b9770 0x7f97f80d03c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 --2- 172.21.15.60:0/1281436719 >> v2:172.21.15.100:3300/0 conn(0x7f97f815dd30 0x7f97f80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 -- 172.21.15.60:0/1281436719 >> 172.21.15.60:0/1281436719 conn(0x7f97f8061c30 msgr2=0x7f97f806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 -- 172.21.15.60:0/1281436719 shutdown_connections 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.638+0000 7f97fd81c640 1 -- 172.21.15.60:0/1281436719 wait complete. 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.639+0000 7f97fd81c640 1 Processor -- start 2024-04-25T08:33:04.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.639+0000 7f97fd81c640 1 -- start start 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97f80b9770 0x7f97f81f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 0x7f97f814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97f815dd30 0x7f97f815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80d1a10 con 0x7f97f80d2e90 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80d2970 con 0x7f97f815dd30 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80d1b90 con 0x7f97f80b9770 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 0x7f97f814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 0x7f97f814da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43208/0 (socket says 172.21.15.60:43208) 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 -- 172.21.15.60:0/2955985552 learned_addr learned my addr 172.21.15.60:0/2955985552 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f6ffd640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97f80b9770 0x7f97f81f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f77fe640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97f815dd30 0x7f97f815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:04.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 -- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97f80b9770 msgr2=0x7f97f81f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97f80b9770 0x7f97f81f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 -- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97f815dd30 msgr2=0x7f97f815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97f815dd30 0x7f97f815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 -- 172.21.15.60:0/2955985552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80b4420 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f6ffd640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97f80b9770 0x7f97f81f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f77fe640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97f815dd30 0x7f97f815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97effff640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 0x7f97f814da50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f97e0035040 tx=0x7f97e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0037040 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0005b90 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0006030 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80b4700 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.640+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f80b4fa0 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.642+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f80b48d0 con 0x7f97f80d2e90 2024-04-25T08:33:04.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.642+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97e00061d0 con 0x7f97f80d2e90 2024-04-25T08:33:04.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.642+0000 7f97f4ff9640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97d806b4d0 0x7f97d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:04.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.643+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f97e00aa610 con 0x7f97f80d2e90 2024-04-25T08:33:04.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.643+0000 7f97f6ffd640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97d806b4d0 0x7f97d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:04.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.643+0000 7f97f6ffd640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97d806b4d0 0x7f97d806d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97e8006fd0 tx=0x7f97e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:04.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.647+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97e0079650 con 0x7f97f80d2e90 2024-04-25T08:33:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.896+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f97f80b51a0 con 0x7f97f80d2e90 2024-04-25T08:33:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.897+0000 7f97f4ff9640 1 -- 172.21.15.60:0/2955985552 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f97f80b51a0 con 0x7f97f80d2e90 2024-04-25T08:33:04.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:04.893 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.894 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:33:04.896 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.898 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.900 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:04.901 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:33:04.902 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:33:04.903 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.904 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:33:04.905 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:04.907 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:33:04.907 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:33:04.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97d806b4d0 msgr2=0x7f97d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97d806b4d0 0x7f97d806d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97e8006fd0 tx=0x7f97e8008040 comp rx=0 tx=0).stop 2024-04-25T08:33:04.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 msgr2=0x7f97f814da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:04.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 0x7f97f814da50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f97e0035040 tx=0x7f97e00046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 shutdown_connections 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97d806b4d0 0x7f97d806d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97f815dd30 0x7f97f815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97f80d2e90 0x7f97f814da50 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 --2- 172.21.15.60:0/2955985552 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97f80b9770 0x7f97f81f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 >> 172.21.15.60:0/2955985552 conn(0x7f97f8061c30 msgr2=0x7f97f80d3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 shutdown_connections 2024-04-25T08:33:04.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:04.899+0000 7f97fd81c640 1 -- 172.21.15.60:0/2955985552 wait complete. 2024-04-25T08:33:04.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:05.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.014+0000 7f129424a640 1 Processor -- start 2024-04-25T08:33:05.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.015+0000 7f129424a640 1 -- start start 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.015+0000 7f129424a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f128c15db60 0x7f128c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.015+0000 7f129424a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f128c0b5f00 0x7f128c0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f129424a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c15df40 0x7f128c0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f129424a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c0ce930 con 0x7f128c15db60 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f129424a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c0b6450 con 0x7f128c0b5f00 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f128c15db60 0x7f128c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f129424a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0cec00 con 0x7f128c15df40 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f128c15db60 0x7f128c0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43216/0 (socket says 172.21.15.60:43216) 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 -- 172.21.15.60:0/2435353504 learned_addr learned my addr 172.21.15.60:0/2435353504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f12917e5640 1 --2- 172.21.15.60:0/2435353504 >> v2:172.21.15.100:3300/0 conn(0x7f128c0b5f00 0x7f128c0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f12927e7640 1 --2- 172.21.15.60:0/2435353504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c15df40 0x7f128c0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 -- 172.21.15.60:0/2435353504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c15df40 msgr2=0x7f128c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 --2- 172.21.15.60:0/2435353504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c15df40 0x7f128c0d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 -- 172.21.15.60:0/2435353504 >> v2:172.21.15.100:3300/0 conn(0x7f128c0b5f00 msgr2=0x7f128c0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 --2- 172.21.15.60:0/2435353504 >> v2:172.21.15.100:3300/0 conn(0x7f128c0b5f00 0x7f128c0d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.016+0000 7f1291fe6640 1 -- 172.21.15.60:0/2435353504 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0ceee0 con 0x7f128c15db60 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.017+0000 7f1291fe6640 1 --2- 172.21.15.60:0/2435353504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c0ce3f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f128c0d02f0 tx=0x7f128802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309e3a398588f64a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.017+0000 7f1290fe4640 1 -- 172.21.15.60:0/2435353504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288037070 con 0x7f128c15db60 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.017+0000 7f1290fe4640 1 -- 172.21.15.60:0/2435353504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1288032440 con 0x7f128c15db60 2024-04-25T08:33:05.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.017+0000 7f1290fe4640 1 -- 172.21.15.60:0/2435353504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288032810 con 0x7f128c15db60 2024-04-25T08:33:05.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 -- 172.21.15.60:0/2435353504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 msgr2=0x7f128c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 --2- 172.21.15.60:0/2435353504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c0ce3f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f128c0d02f0 tx=0x7f128802a180 comp rx=0 tx=0).stop 2024-04-25T08:33:05.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 -- 172.21.15.60:0/2435353504 shutdown_connections 2024-04-25T08:33:05.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 --2- 172.21.15.60:0/2435353504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c15df40 0x7f128c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 --2- 172.21.15.60:0/2435353504 >> v2:172.21.15.100:3300/0 conn(0x7f128c0b5f00 0x7f128c0d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 --2- 172.21.15.60:0/2435353504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c0ce3f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 -- 172.21.15.60:0/2435353504 >> 172.21.15.60:0/2435353504 conn(0x7f128c061600 msgr2=0x7f128c061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 -- 172.21.15.60:0/2435353504 shutdown_connections 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.018+0000 7f129424a640 1 -- 172.21.15.60:0/2435353504 wait complete. 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.019+0000 7f129424a640 1 Processor -- start 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.019+0000 7f129424a640 1 -- start start 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c0b5f00 0x7f128c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c15df40 0x7f128c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c161950 con 0x7f128c15db60 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0d1d50 con 0x7f128c15df40 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c161ad0 con 0x7f128c0b5f00 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43228/0 (socket says 172.21.15.60:43228) 2024-04-25T08:33:05.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 -- 172.21.15.60:0/1534684820 learned_addr learned my addr 172.21.15.60:0/1534684820 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f1291fe6640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c0b5f00 0x7f128c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12927e7640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c15df40 0x7f128c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 -- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c0b5f00 msgr2=0x7f128c151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c0b5f00 0x7f128c151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 -- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c15df40 msgr2=0x7f128c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c15df40 0x7f128c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 -- 172.21.15.60:0/1534684820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c161020 con 0x7f128c15db60 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f1291fe6640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c0b5f00 0x7f128c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12917e5640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c160640 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e890 tx=0x7f127c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c017070 con 0x7f128c15db60 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c00c940 con 0x7f128c15db60 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c1f7470 con 0x7f128c15db60 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.020+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c00cd10 con 0x7f128c15db60 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.021+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c151fe0 con 0x7f128c15db60 2024-04-25T08:33:05.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.021+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c073270 con 0x7f128c15db60 2024-04-25T08:33:05.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.022+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f127c01a070 con 0x7f128c15db60 2024-04-25T08:33:05.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.022+0000 7f12827fc640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f127006b4d0 0x7f127006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.022+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f127c08a300 con 0x7f128c15db60 2024-04-25T08:33:05.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.023+0000 7f1291fe6640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f127006b4d0 0x7f127006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.023+0000 7f1291fe6640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f127006b4d0 0x7f127006d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1288034000 tx=0x7f128802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.027+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f127c0593d0 con 0x7f128c15db60 2024-04-25T08:33:05.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.209+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f128c0ce3e0 con 0x7f128c15db60 2024-04-25T08:33:05.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.210+0000 7f12827fc640 1 -- 172.21.15.60:0/1534684820 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f127c058cd0 con 0x7f128c15db60 2024-04-25T08:33:05.206 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:05.207 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":63,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:04.511152+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f127006b4d0 msgr2=0x7f127006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f127006b4d0 0x7f127006d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1288034000 tx=0x7f128802e000 comp rx=0 tx=0).stop 2024-04-25T08:33:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 msgr2=0x7f128c160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c160640 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e890 tx=0x7f127c00ed50 comp rx=0 tx=0).stop 2024-04-25T08:33:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 shutdown_connections 2024-04-25T08:33:05.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f127006b4d0 0x7f127006d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c15df40 0x7f128c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f128c15db60 0x7f128c160640 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 --2- 172.21.15.60:0/1534684820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f128c0b5f00 0x7f128c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 >> 172.21.15.60:0/1534684820 conn(0x7f128c061600 msgr2=0x7f128c161f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 shutdown_connections 2024-04-25T08:33:05.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.212+0000 7f129424a640 1 -- 172.21.15.60:0/1534684820 wait complete. 2024-04-25T08:33:05.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:33:05.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:33:05.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.327+0000 7f15a6df5640 1 Processor -- start 2024-04-25T08:33:05.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.327+0000 7f15a6df5640 1 -- start start 2024-04-25T08:33:05.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a6df5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15a015b7d0 0x7f15a00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a6df5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 0x7f15a00b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a6df5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a006cf50 con 0x7f15a006c2d0 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a6df5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a00b5fc0 con 0x7f15a015b7d0 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a6df5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00ce530 con 0x7f15a00b5620 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a4b91640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15a015b7d0 0x7f15a00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f159ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 0x7f15a00b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a5392640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f15a006c2d0 0x7f15a006c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46614/0 (socket says 172.21.15.60:46614) 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f159ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 0x7f15a00b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51126/0 (socket says 172.21.15.60:51126) 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a5392640 1 -- 172.21.15.60:0/618569072 learned_addr learned my addr 172.21.15.60:0/618569072 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f159ffff640 1 -- 172.21.15.60:0/618569072 >> v2:172.21.15.100:3300/0 conn(0x7f15a015b7d0 msgr2=0x7f15a00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f159ffff640 1 --2- 172.21.15.60:0/618569072 >> v2:172.21.15.100:3300/0 conn(0x7f15a015b7d0 0x7f15a00cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f159ffff640 1 -- 172.21.15.60:0/618569072 >> v1:172.21.15.100:6789/0 conn(0x7f15a006c2d0 legacy=0x7f15a006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f159ffff640 1 -- 172.21.15.60:0/618569072 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a00ce810 con 0x7f15a00b5620 2024-04-25T08:33:05.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.328+0000 7f15a4b91640 1 --2- 172.21.15.60:0/618569072 >> v2:172.21.15.100:3300/0 conn(0x7f15a015b7d0 0x7f15a00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.329+0000 7f159ffff640 1 --2- 172.21.15.60:0/618569072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 0x7f15a00b5a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f158c0097a0 tx=0x7f158c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d07f3348dc069475 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.329+0000 7f159f7fe640 1 -- 172.21.15.60:0/618569072 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c037070 con 0x7f15a00b5620 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.329+0000 7f159f7fe640 1 -- 172.21.15.60:0/618569072 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c032440 con 0x7f15a00b5620 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.329+0000 7f159f7fe640 1 -- 172.21.15.60:0/618569072 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c032810 con 0x7f15a00b5620 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 -- 172.21.15.60:0/618569072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 msgr2=0x7f15a00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 --2- 172.21.15.60:0/618569072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 0x7f15a00b5a80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f158c0097a0 tx=0x7f158c02a180 comp rx=0 tx=0).stop 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 -- 172.21.15.60:0/618569072 shutdown_connections 2024-04-25T08:33:05.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 --2- 172.21.15.60:0/618569072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a00b5620 0x7f15a00b5a80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 --2- 172.21.15.60:0/618569072 >> v2:172.21.15.100:3300/0 conn(0x7f15a015b7d0 0x7f15a00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 -- 172.21.15.60:0/618569072 >> 172.21.15.60:0/618569072 conn(0x7f15a00d0800 msgr2=0x7f15a00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 -- 172.21.15.60:0/618569072 shutdown_connections 2024-04-25T08:33:05.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.330+0000 7f15a6df5640 1 -- 172.21.15.60:0/618569072 wait complete. 2024-04-25T08:33:05.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.331+0000 7f15a6df5640 1 Processor -- start 2024-04-25T08:33:05.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.331+0000 7f15a6df5640 1 -- start start 2024-04-25T08:33:05.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a6df5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a006c2d0 0x7f15a00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a6df5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 0x7f15a0158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a6df5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a015b7d0 0x7f15a01510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a6df5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00cf550 con 0x7f15a00b5620 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a6df5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a006d230 con 0x7f15a006c2d0 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a6df5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00cf6d0 con 0x7f15a015b7d0 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 0x7f15a0158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a4b91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a006c2d0 0x7f15a00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 0x7f15a0158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43242/0 (socket says 172.21.15.60:43242) 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 -- 172.21.15.60:0/2465300081 learned_addr learned my addr 172.21.15.60:0/2465300081 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a5392640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a015b7d0 0x7f15a01510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 -- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a015b7d0 msgr2=0x7f15a01510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a015b7d0 0x7f15a01510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 -- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a006c2d0 msgr2=0x7f15a00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a006c2d0 0x7f15a00cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 -- 172.21.15.60:0/2465300081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0151630 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f15a4b91640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a006c2d0 0x7f15a00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.332+0000 7f159ffff640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 0x7f15a0158ef0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f158c02a1c0 tx=0x7f158c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.333+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c041020 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.333+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c030700 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.333+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a00ccc00 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.333+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c037040 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.333+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a00cd1c0 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.334+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a0073a00 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.334+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f158c044020 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.335+0000 7f159d7fa640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f158406b4d0 0x7f158406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.335+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f158c0aa6e0 con 0x7f15a00b5620 2024-04-25T08:33:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.335+0000 7f15a4b91640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f158406b4d0 0x7f158406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.335+0000 7f15a4b91640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f158406b4d0 0x7f158406d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1594009560 tx=0x7f1594009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.339+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f158c04b030 con 0x7f15a00b5620 2024-04-25T08:33:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.523+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15a015b4d0 con 0x7f15a00b5620 2024-04-25T08:33:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.523+0000 7f159d7fa640 1 -- 172.21.15.60:0/2465300081 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f158c0790d0 con 0x7f15a00b5620 2024-04-25T08:33:05.519 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:33:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f158406b4d0 msgr2=0x7f158406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f158406b4d0 0x7f158406d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1594009560 tx=0x7f1594009190 comp rx=0 tx=0).stop 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 msgr2=0x7f15a0158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 0x7f15a0158ef0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f158c02a1c0 tx=0x7f158c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 shutdown_connections 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15a015b7d0 0x7f15a01510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15a00b5620 0x7f15a0158ef0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f158406b4d0 0x7f158406d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 --2- 172.21.15.60:0/2465300081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15a006c2d0 0x7f15a00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 >> 172.21.15.60:0/2465300081 conn(0x7f15a00d0800 msgr2=0x7f15a00d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 shutdown_connections 2024-04-25T08:33:05.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.525+0000 7f15a6df5640 1 -- 172.21.15.60:0/2465300081 wait complete. 2024-04-25T08:33:05.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:33:05.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 Processor -- start 2024-04-25T08:33:05.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 -- start start 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f4260162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42600b6cb0 0x7f42600b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4260162690 con 0x7f42600b5a60 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600d3800 con 0x7f42600b6cb0 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.638+0000 7f4266be0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600ceee0 con 0x7f426015db70 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426517d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42600b5a60 0x7f426006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37428/0 (socket says 172.21.15.60:37428) 2024-04-25T08:33:05.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426517d640 1 -- 172.21.15.60:0/1356952079 learned_addr learned my addr 172.21.15.60:0/1356952079 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f425ffff640 1 --2- 172.21.15.60:0/1356952079 >> v2:172.21.15.100:3300/0 conn(0x7f42600b6cb0 0x7f42600b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426497c640 1 --2- 172.21.15.60:0/1356952079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f4260162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f425f7fe640 1 -- 172.21.15.60:0/1356952079 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022329612 0 0) 0x7f4260162690 con 0x7f42600b5a60 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f425f7fe640 1 -- 172.21.15.60:0/1356952079 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4248003520 con 0x7f42600b5a60 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f425f7fe640 1 -- 172.21.15.60:0/1356952079 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2955878556 0 0) 0x7f4248003520 con 0x7f42600b5a60 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426497c640 1 -- 172.21.15.60:0/1356952079 >> v2:172.21.15.100:3300/0 conn(0x7f42600b6cb0 msgr2=0x7f42600b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426497c640 1 --2- 172.21.15.60:0/1356952079 >> v2:172.21.15.100:3300/0 conn(0x7f42600b6cb0 0x7f42600b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426497c640 1 -- 172.21.15.60:0/1356952079 >> v1:172.21.15.60:6789/0 conn(0x7f42600b5a60 legacy=0x7f426006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f426497c640 1 -- 172.21.15.60:0/1356952079 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600d1d10 con 0x7f426015db70 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.639+0000 7f425ffff640 1 --2- 172.21.15.60:0/1356952079 >> v2:172.21.15.100:3300/0 conn(0x7f42600b6cb0 0x7f42600b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.640+0000 7f426497c640 1 --2- 172.21.15.60:0/1356952079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f4260162150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4258009d90 tx=0x7f425802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1098b32af006a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.640+0000 7f425f7fe640 1 -- 172.21.15.60:0/1356952079 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4250001670 con 0x7f426015db70 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.640+0000 7f425f7fe640 1 -- 172.21.15.60:0/1356952079 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258007540 con 0x7f426015db70 2024-04-25T08:33:05.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.640+0000 7f425f7fe640 1 -- 172.21.15.60:0/1356952079 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258007910 con 0x7f426015db70 2024-04-25T08:33:05.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 -- 172.21.15.60:0/1356952079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 msgr2=0x7f4260162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 --2- 172.21.15.60:0/1356952079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f4260162150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4258009d90 tx=0x7f425802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 -- 172.21.15.60:0/1356952079 shutdown_connections 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 --2- 172.21.15.60:0/1356952079 >> v2:172.21.15.100:3300/0 conn(0x7f42600b6cb0 0x7f42600b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 --2- 172.21.15.60:0/1356952079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f4260162150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 -- 172.21.15.60:0/1356952079 >> 172.21.15.60:0/1356952079 conn(0x7f42600d0520 msgr2=0x7f42600d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 -- 172.21.15.60:0/1356952079 shutdown_connections 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.641+0000 7f4266be0640 1 -- 172.21.15.60:0/1356952079 wait complete. 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.642+0000 7f4266be0640 1 Processor -- start 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.642+0000 7f4266be0640 1 -- start start 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f4266be0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 0x7f42601566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f4266be0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42600b6cb0 0x7f42600cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f4266be0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f42600cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f4266be0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426015df50 con 0x7f42600b5a60 2024-04-25T08:33:05.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f4266be0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600d22d0 con 0x7f42600b6cb0 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f4266be0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600ce530 con 0x7f426015db70 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 0x7f42601566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 0x7f42601566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43258/0 (socket says 172.21.15.60:43258) 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 -- 172.21.15.60:0/2743497709 learned_addr learned my addr 172.21.15.60:0/2743497709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426517d640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f42600cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f425ffff640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42600b6cb0 0x7f42600cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 -- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 msgr2=0x7f42600cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f42600cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 -- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42600b6cb0 msgr2=0x7f42600cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42600b6cb0 0x7f42600cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.643+0000 7f426497c640 1 -- 172.21.15.60:0/2743497709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260157190 con 0x7f42600b5a60 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.644+0000 7f426497c640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 0x7f42601566d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4258009d90 tx=0x7f4258004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.644+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425803f070 con 0x7f42600b5a60 2024-04-25T08:33:05.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.644+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258031c40 con 0x7f42600b5a60 2024-04-25T08:33:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.644+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4260157470 con 0x7f42600b5a60 2024-04-25T08:33:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.644+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258009450 con 0x7f42600b5a60 2024-04-25T08:33:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.644+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4260156440 con 0x7f42600b5a60 2024-04-25T08:33:05.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.645+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4260073990 con 0x7f42600b5a60 2024-04-25T08:33:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.645+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f425803b410 con 0x7f42600b5a60 2024-04-25T08:33:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.646+0000 7f425d7fa640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f424c06b4d0 0x7f424c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.646+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f42580ab540 con 0x7f42600b5a60 2024-04-25T08:33:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.646+0000 7f425ffff640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f424c06b4d0 0x7f424c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.646+0000 7f425ffff640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f424c06b4d0 0x7f424c06d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4254009560 tx=0x7f4254009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.650+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4258079fc0 con 0x7f42600b5a60 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.832+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f426015d320 con 0x7f42600b5a60 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.832+0000 7f425d7fa640 1 -- 172.21.15.60:0/2743497709 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+65 (secure 0 0 0) 0x7f425807e060 con 0x7f42600b5a60 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:33:05.829 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:33:05.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:05.830 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f424c06b4d0 msgr2=0x7f424c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f424c06b4d0 0x7f424c06d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4254009560 tx=0x7f4254009190 comp rx=0 tx=0).stop 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 msgr2=0x7f42601566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 0x7f42601566d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4258009d90 tx=0x7f4258004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 shutdown_connections 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f426015db70 0x7f42600cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f424c06b4d0 0x7f424c06d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42600b6cb0 0x7f42600cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 --2- 172.21.15.60:0/2743497709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42600b5a60 0x7f42601566d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.834+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 >> 172.21.15.60:0/2743497709 conn(0x7f42600d0520 msgr2=0x7f42601581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.835+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 shutdown_connections 2024-04-25T08:33:05.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.835+0000 7f4266be0640 1 -- 172.21.15.60:0/2743497709 wait complete. 2024-04-25T08:33:05.841 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:33:05.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:33:05.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.950+0000 7fee15f5c640 1 Processor -- start 2024-04-25T08:33:05.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.950+0000 7fee15f5c640 1 -- start start 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.951+0000 7fee15f5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1006c9e0 0x7fee100b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.951+0000 7fee15f5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1006cf50 con 0x7fee1015b7d0 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.951+0000 7fee15f5c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee100ce530 con 0x7fee100b5620 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.951+0000 7fee15f5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee100ce810 con 0x7fee1006c9e0 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.951+0000 7fee0f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fee1015b7d0 0x7fee100cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37430/0 (socket says 172.21.15.60:37430) 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.951+0000 7fee0f7fe640 1 -- 172.21.15.60:0/995042351 learned_addr learned my addr 172.21.15.60:0/995042351 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0ffff640 1 --2- 172.21.15.60:0/995042351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1006c9e0 0x7fee100b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0e7fc640 1 -- 172.21.15.60:0/995042351 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524382839 0 0) 0x7fee1006cf50 con 0x7fee1015b7d0 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0e7fc640 1 -- 172.21.15.60:0/995042351 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedf8003520 con 0x7fee1015b7d0 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0ffff640 1 -- 172.21.15.60:0/995042351 >> v1:172.21.15.100:6789/0 conn(0x7fee100b5620 legacy=0x7fee1006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0e7fc640 1 -- 172.21.15.60:0/995042351 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919792435 0 0) 0x7fee100ce530 con 0x7fee100b5620 2024-04-25T08:33:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0ffff640 1 -- 172.21.15.60:0/995042351 >> v1:172.21.15.60:6789/0 conn(0x7fee1015b7d0 legacy=0x7fee100cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.952+0000 7fee0ffff640 1 -- 172.21.15.60:0/995042351 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100ceaf0 con 0x7fee1006c9e0 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee0ffff640 1 --2- 172.21.15.60:0/995042351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1006c9e0 0x7fee100b5ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee08009dc0 tx=0x7fee0802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=276302080ca95a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee0e7fc640 1 -- 172.21.15.60:0/995042351 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08007740 con 0x7fee1006c9e0 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee0e7fc640 1 -- 172.21.15.60:0/995042351 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee080078e0 con 0x7fee1006c9e0 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee0e7fc640 1 -- 172.21.15.60:0/995042351 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08007cb0 con 0x7fee1006c9e0 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 -- 172.21.15.60:0/995042351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1006c9e0 msgr2=0x7fee100b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 --2- 172.21.15.60:0/995042351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1006c9e0 0x7fee100b5ab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee08009dc0 tx=0x7fee0802a6d0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 -- 172.21.15.60:0/995042351 shutdown_connections 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 --2- 172.21.15.60:0/995042351 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1006c9e0 0x7fee100b5ab0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 -- 172.21.15.60:0/995042351 >> 172.21.15.60:0/995042351 conn(0x7fee100d0800 msgr2=0x7fee100d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 -- 172.21.15.60:0/995042351 shutdown_connections 2024-04-25T08:33:05.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.953+0000 7fee15f5c640 1 -- 172.21.15.60:0/995042351 wait complete. 2024-04-25T08:33:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 Processor -- start 2024-04-25T08:33:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 -- start start 2024-04-25T08:33:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 0x7fee100ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee100b5620 0x7fee101f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1015b7d0 0x7fee100c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1015bd30 con 0x7fee1006c9e0 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1015bbb0 con 0x7fee100b5620 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.955+0000 7fee15f5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee100d1f40 con 0x7fee1015b7d0 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 0x7fee100ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 0x7fee100ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43264/0 (socket says 172.21.15.60:43264) 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 -- 172.21.15.60:0/2129527888 learned_addr learned my addr 172.21.15.60:0/2129527888 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0ffff640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1015b7d0 0x7fee100c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0effd640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee100b5620 0x7fee101f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 -- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1015b7d0 msgr2=0x7fee100c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1015b7d0 0x7fee100c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 -- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee100b5620 msgr2=0x7fee101f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee100b5620 0x7fee101f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 -- 172.21.15.60:0/2129527888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1015c9f0 con 0x7fee1006c9e0 2024-04-25T08:33:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0effd640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee100b5620 0x7fee101f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee0f7fe640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 0x7fee100ca050 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c400 tx=0x7fee0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00017070 con 0x7fee1006c9e0 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00005d00 con 0x7fee1006c9e0 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00007310 con 0x7fee1006c9e0 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1015ccd0 con 0x7fee1006c9e0 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.956+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1015d290 con 0x7fee1006c9e0 2024-04-25T08:33:05.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.957+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee10073a00 con 0x7fee1006c9e0 2024-04-25T08:33:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.958+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee0001a070 con 0x7fee1006c9e0 2024-04-25T08:33:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.958+0000 7fedf7fff640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fedfc06b4d0 0x7fedfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.958+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fee00089470 con 0x7fee1006c9e0 2024-04-25T08:33:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.958+0000 7fee0effd640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fedfc06b4d0 0x7fedfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:05.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.959+0000 7fee0effd640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fedfc06b4d0 0x7fedfc06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee100b9230 tx=0x7fee04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:05.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:05.963+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee00058450 con 0x7fee1006c9e0 2024-04-25T08:33:06.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.143+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fee10068070 con 0x7fee1006c9e0 2024-04-25T08:33:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.521+0000 7fedf7fff640 1 -- 172.21.15.60:0/2129527888 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7fee00057d50 con 0x7fee1006c9e0 2024-04-25T08:33:06.517 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:33:06.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fedfc06b4d0 msgr2=0x7fedfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:06.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fedfc06b4d0 0x7fedfc06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fee100b9230 tx=0x7fee04009040 comp rx=0 tx=0).stop 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 msgr2=0x7fee100ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 0x7fee100ca050 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c400 tx=0x7fee0000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 shutdown_connections 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fee1015b7d0 0x7fee100c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fedfc06b4d0 0x7fedfc06d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee100b5620 0x7fee101f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 --2- 172.21.15.60:0/2129527888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee1006c9e0 0x7fee100ca050 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.524+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 >> 172.21.15.60:0/2129527888 conn(0x7fee100d0800 msgr2=0x7fee100d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.525+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 shutdown_connections 2024-04-25T08:33:06.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.525+0000 7fee15f5c640 1 -- 172.21.15.60:0/2129527888 wait complete. 2024-04-25T08:33:06.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:33:06.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 Processor -- start 2024-04-25T08:33:06.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 -- start start 2024-04-25T08:33:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 0x7fe230161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe2300ce0a0 0x7fe2300ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23006c8d0 con 0x7fe2301610f0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2301614d0 con 0x7fe2300ce0a0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.656+0000 7fe2351d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300c6960 con 0x7fe2301616f0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2301610f0 0x7fe23006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37432/0 (socket says 172.21.15.60:37432) 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22ed9d640 1 -- 172.21.15.60:0/2397636758 learned_addr learned my addr 172.21.15.60:0/2397636758 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22e59c640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 0x7fe230161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038404034 0 0) 0x7fe23006c8d0 con 0x7fe2301610f0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22f59e640 1 --2- 172.21.15.60:0/2397636758 >> v2:172.21.15.100:3300/0 conn(0x7fe2300ce0a0 0x7fe2300ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe21c003520 con 0x7fe2301610f0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126236608 0 0) 0x7fe21c003520 con 0x7fe2301610f0 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 msgr2=0x7fe230161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 0x7fe230161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 >> v2:172.21.15.100:3300/0 conn(0x7fe2300ce0a0 msgr2=0x7fe2300ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 --2- 172.21.15.60:0/2397636758 >> v2:172.21.15.100:3300/0 conn(0x7fe2300ce0a0 0x7fe2300ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230162260 con 0x7fe2301610f0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe228001670 con 0x7fe2301610f0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 >> v1:172.21.15.60:6789/0 conn(0x7fe2301610f0 legacy=0x7fe23006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c0039b0 0x7fe21c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21c004800 0x7fe21c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 0x7fe21c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301614d0 con 0x7fe21c004800 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300c6960 con 0x7fe21c0039b0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c00b9c0 con 0x7fe2301616f0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22e59c640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21c004800 0x7fe21c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22ed9d640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c0039b0 0x7fe21c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.657+0000 7fe22f59e640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 0x7fe21c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.658+0000 7fe22e59c640 1 -- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 msgr2=0x7fe21c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.658+0000 7fe22e59c640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2301616f0 0x7fe21c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.658+0000 7fe22e59c640 1 -- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c0039b0 msgr2=0x7fe21c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.658+0000 7fe22e59c640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c0039b0 0x7fe21c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.658+0000 7fe22e59c640 1 -- 172.21.15.60:0/2397636758 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c003520 con 0x7fe21c004800 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.658+0000 7fe22e59c640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21c004800 0x7fe21c004c80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe218009ac0 tx=0x7fe21c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4fc1b8d53160cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218036070 con 0x7fe21c004800 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe218031400 con 0x7fe21c004800 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe22dd9b640 1 -- 172.21.15.60:0/2397636758 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218009690 con 0x7fe21c004800 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe2351d5640 1 -- 172.21.15.60:0/2397636758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21c004800 msgr2=0x7fe21c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe2351d5640 1 --2- 172.21.15.60:0/2397636758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe21c004800 0x7fe21c004c80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe218009ac0 tx=0x7fe21c005830 comp rx=0 tx=0).stop 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe22f59e640 1 -- 172.21.15.60:0/2397636758 reap_dead start 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe2351d5640 1 -- 172.21.15.60:0/2397636758 shutdown_connections 2024-04-25T08:33:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe2351d5640 1 -- 172.21.15.60:0/2397636758 >> 172.21.15.60:0/2397636758 conn(0x7fe23006f500 msgr2=0x7fe23006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:06.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe2351d5640 1 -- 172.21.15.60:0/2397636758 shutdown_connections 2024-04-25T08:33:06.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.659+0000 7fe2351d5640 1 -- 172.21.15.60:0/2397636758 wait complete. 2024-04-25T08:33:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 Processor -- start 2024-04-25T08:33:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 -- start start 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c004800 0x7fe230151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 0x7fe2300ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe230152820 0x7fe2300aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300cfce0 con 0x7fe2301522d0 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300cfa00 con 0x7fe21c004800 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.661+0000 7fe2351d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300cf690 con 0x7fe230152820 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 0x7fe2300ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 0x7fe2300ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43286/0 (socket says 172.21.15.60:43286) 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 -- 172.21.15.60:0/3797672242 learned_addr learned my addr 172.21.15.60:0/3797672242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22f59e640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe230152820 0x7fe2300aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22ed9d640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c004800 0x7fe230151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 -- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe230152820 msgr2=0x7fe2300aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe230152820 0x7fe2300aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 -- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c004800 msgr2=0x7fe230151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c004800 0x7fe230151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 -- 172.21.15.60:0/3797672242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2300cd7b0 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22ed9d640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c004800 0x7fe230151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22f59e640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe230152820 0x7fe2300aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe22e59c640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 0x7fe2300ae4d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe218034040 tx=0x7fe218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218040020 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe218003bc0 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2300adca0 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218036040 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.662+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2300cd440 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.663+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe230073a00 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.664+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe218043020 con 0x7fe2301522d0 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.664+0000 7fe2277fe640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.664+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe2180aa590 con 0x7fe2301522d0 2024-04-25T08:33:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.665+0000 7fe22ed9d640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.665+0000 7fe22ed9d640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe21006b500 0x7fe21006d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe228001050 tx=0x7fe228009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:06.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.669+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe21804a030 con 0x7fe2301522d0 2024-04-25T08:33:06.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:06.853+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe2301559f0 con 0x7fe2301522d0 2024-04-25T08:33:07.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.526+0000 7fe2277fe640 1 -- 172.21.15.60:0/3797672242 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe2180044e0 con 0x7fe2301522d0 2024-04-25T08:33:07.523 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe21006b500 msgr2=0x7fe21006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe21006b500 0x7fe21006d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe228001050 tx=0x7fe228009040 comp rx=0 tx=0).stop 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 msgr2=0x7fe2300ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 0x7fe2300ae4d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe218034040 tx=0x7fe218004730 comp rx=0 tx=0).stop 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 shutdown_connections 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe230152820 0x7fe2300aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2301522d0 0x7fe2300ae4d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 --2- 172.21.15.60:0/3797672242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe21c004800 0x7fe230151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 >> 172.21.15.60:0/3797672242 conn(0x7fe23006f500 msgr2=0x7fe230070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 shutdown_connections 2024-04-25T08:33:07.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.530+0000 7fe2351d5640 1 -- 172.21.15.60:0/3797672242 wait complete. 2024-04-25T08:33:07.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:33:07.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.668+0000 7f40f101c640 1 Processor -- start 2024-04-25T08:33:07.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.668+0000 7f40f101c640 1 -- start start 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40f101c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40ec15b7d0 0x7f40ec0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40f101c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40f101c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec06cf50 con 0x7f40ec0b5620 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40f101c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec0ce530 con 0x7f40ec15b7d0 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40f101c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0ce810 con 0x7f40ec06c9e0 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40eb59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40ead9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f40ec15b7d0 0x7f40ec0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40eb59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51214/0 (socket says 172.21.15.60:51214) 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40eb59e640 1 -- 172.21.15.60:0/2259969503 learned_addr learned my addr 172.21.15.60:0/2259969503 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:07.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40e3fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f40ec0b5620 0x7f40ec06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46646/0 (socket says 172.21.15.60:46646) 2024-04-25T08:33:07.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40ea59c640 1 -- 172.21.15.60:0/2259969503 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593886548 0 0) 0x7f40ec06cf50 con 0x7f40ec0b5620 2024-04-25T08:33:07.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40ea59c640 1 -- 172.21.15.60:0/2259969503 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40d8003520 con 0x7f40ec0b5620 2024-04-25T08:33:07.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40eb59e640 1 -- 172.21.15.60:0/2259969503 >> v2:172.21.15.100:3300/0 conn(0x7f40ec15b7d0 msgr2=0x7f40ec0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:07.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.669+0000 7f40eb59e640 1 --2- 172.21.15.60:0/2259969503 >> v2:172.21.15.100:3300/0 conn(0x7f40ec15b7d0 0x7f40ec0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.670+0000 7f40eb59e640 1 -- 172.21.15.60:0/2259969503 >> v1:172.21.15.100:6789/0 conn(0x7f40ec0b5620 legacy=0x7f40ec06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.670+0000 7f40eb59e640 1 -- 172.21.15.60:0/2259969503 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0ceaf0 con 0x7f40ec06c9e0 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.670+0000 7f40ead9d640 1 --2- 172.21.15.60:0/2259969503 >> v2:172.21.15.100:3300/0 conn(0x7f40ec15b7d0 0x7f40ec0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.670+0000 7f40eb59e640 1 --2- 172.21.15.60:0/2259969503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec0b5ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009e50 tx=0x7f40e402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f528405676ab486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.670+0000 7f40ea59c640 1 -- 172.21.15.60:0/2259969503 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4036070 con 0x7f40ec06c9e0 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.670+0000 7f40ea59c640 1 -- 172.21.15.60:0/2259969503 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4031600 con 0x7f40ec06c9e0 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40ea59c640 1 -- 172.21.15.60:0/2259969503 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e40319b0 con 0x7f40ec06c9e0 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 -- 172.21.15.60:0/2259969503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 msgr2=0x7f40ec0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 --2- 172.21.15.60:0/2259969503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec0b5ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009e50 tx=0x7f40e402a710 comp rx=0 tx=0).stop 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 -- 172.21.15.60:0/2259969503 shutdown_connections 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 --2- 172.21.15.60:0/2259969503 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec0b5ab0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 --2- 172.21.15.60:0/2259969503 >> v2:172.21.15.100:3300/0 conn(0x7f40ec15b7d0 0x7f40ec0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 -- 172.21.15.60:0/2259969503 >> 172.21.15.60:0/2259969503 conn(0x7f40ec0d0800 msgr2=0x7f40ec0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 -- 172.21.15.60:0/2259969503 shutdown_connections 2024-04-25T08:33:07.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.671+0000 7f40f101c640 1 -- 172.21.15.60:0/2259969503 wait complete. 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.672+0000 7f40f101c640 1 Processor -- start 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 -- start start 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40ec0b5620 0x7f40ec0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 0x7f40ec0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0cf6d0 con 0x7f40ec15b7d0 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0cf550 con 0x7f40ec0b5620 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40f101c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec15bbb0 con 0x7f40ec06c9e0 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40eb59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 0x7f40ec0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40eb59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 0x7f40ec0cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43302/0 (socket says 172.21.15.60:43302) 2024-04-25T08:33:07.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.673+0000 7f40eb59e640 1 -- 172.21.15.60:0/4196297873 learned_addr learned my addr 172.21.15.60:0/4196297873 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40e3fff640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40ec0b5620 0x7f40ec0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40ead9d640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40eb59e640 1 -- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 msgr2=0x7f40ec14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40eb59e640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40eb59e640 1 -- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40ec0b5620 msgr2=0x7f40ec0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40eb59e640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40ec0b5620 0x7f40ec0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40eb59e640 1 -- 172.21.15.60:0/4196297873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0cb050 con 0x7f40ec15b7d0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40eb59e640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 0x7f40ec0cd710 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f40e4034040 tx=0x7f40e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4036050 con 0x7f40ec15b7d0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4042070 con 0x7f40ec15b7d0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec0c9360 con 0x7f40ec15b7d0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009510 con 0x7f40ec15b7d0 2024-04-25T08:33:07.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.674+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec0c7830 con 0x7f40ec15b7d0 2024-04-25T08:33:07.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.675+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec073a00 con 0x7f40ec15b7d0 2024-04-25T08:33:07.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.676+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40e4007c70 con 0x7f40ec15b7d0 2024-04-25T08:33:07.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.676+0000 7f40e37fe640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f40cc06b500 0x7f40cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:07.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.676+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f40e40aaf90 con 0x7f40ec15b7d0 2024-04-25T08:33:07.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.677+0000 7f40ead9d640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f40cc06b500 0x7f40cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:07.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.677+0000 7f40ead9d640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f40cc06b500 0x7f40cc06d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009560 tx=0x7f40dc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:07.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.681+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40e40796a0 con 0x7f40ec15b7d0 2024-04-25T08:33:07.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:07.897+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f40ec160d20 con 0x7f40ec15b7d0 2024-04-25T08:33:08.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.528+0000 7f40e37fe640 1 -- 172.21.15.60:0/4196297873 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v32) v1 ==== 173+0+0 (secure 0 0 0) 0x7f40e407d740 con 0x7f40ec15b7d0 2024-04-25T08:33:08.524 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T08:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.532+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f40cc06b500 msgr2=0x7f40cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.532+0000 7f40f101c640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f40cc06b500 0x7f40cc06d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009560 tx=0x7f40dc009190 comp rx=0 tx=0).stop 2024-04-25T08:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 msgr2=0x7f40ec0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 0x7f40ec0cd710 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f40e4034040 tx=0x7f40e4004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 shutdown_connections 2024-04-25T08:33:08.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f40ec15b7d0 0x7f40ec0cd710 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f40cc06b500 0x7f40cc06d9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40ec0b5620 0x7f40ec0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 --2- 172.21.15.60:0/4196297873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f40ec06c9e0 0x7f40ec14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.533+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 >> 172.21.15.60:0/4196297873 conn(0x7f40ec0d0800 msgr2=0x7f40ec0b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.534+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 shutdown_connections 2024-04-25T08:33:08.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.535+0000 7f40f101c640 1 -- 172.21.15.60:0/4196297873 wait complete. 2024-04-25T08:33:08.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:08.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.672+0000 7f67c264a640 1 Processor -- start 2024-04-25T08:33:08.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.672+0000 7f67c264a640 1 -- start start 2024-04-25T08:33:08.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c264a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67bc0b6cb0 0x7f67bc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c264a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 0x7f67bc0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c264a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc0ceee0 con 0x7f67bc15db70 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c264a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc0d1d10 con 0x7f67bc0b6cb0 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c264a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0d1ff0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67bc0b6cb0 0x7f67bc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c0be7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 0x7f67bc0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67bc0b6cb0 0x7f67bc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58100/0 (socket says 172.21.15.60:58100) 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bb7fe640 1 -- 172.21.15.60:0/2540697179 learned_addr learned my addr 172.21.15.60:0/2540697179 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67c0be7640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 0x7f67bc0d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51222/0 (socket says 172.21.15.60:51222) 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bbfff640 1 --1- 172.21.15.60:0/2540697179 >> v1:172.21.15.100:6789/0 conn(0x7f67bc15db70 0x7f67bc162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46648/0 (socket says 172.21.15.60:46648) 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bb7fe640 1 -- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 msgr2=0x7f67bc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bb7fe640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 0x7f67bc0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.673+0000 7f67bb7fe640 1 -- 172.21.15.60:0/2540697179 >> v1:172.21.15.100:6789/0 conn(0x7f67bc15db70 legacy=0x7f67bc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67bb7fe640 1 -- 172.21.15.60:0/2540697179 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0d22d0 con 0x7f67bc0b6cb0 2024-04-25T08:33:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67baffd640 1 -- 172.21.15.60:0/2540697179 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928495429 0 0) 0x7f67bc0ceee0 con 0x7f67bc15db70 2024-04-25T08:33:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67c0be7640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 0x7f67bc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67bb7fe640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc0b6cb0 0x7f67bc0b5520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009870 tx=0x7f67ac02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8d79f5d0507fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67baffd640 1 -- 172.21.15.60:0/2540697179 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac037070 con 0x7f67bc0b6cb0 2024-04-25T08:33:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67baffd640 1 -- 172.21.15.60:0/2540697179 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac032440 con 0x7f67bc0b6cb0 2024-04-25T08:33:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.674+0000 7f67baffd640 1 -- 172.21.15.60:0/2540697179 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac032810 con 0x7f67bc0b6cb0 2024-04-25T08:33:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 -- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc0b6cb0 msgr2=0x7f67bc0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc0b6cb0 0x7f67bc0b5520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009870 tx=0x7f67ac02a250 comp rx=0 tx=0).stop 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 -- 172.21.15.60:0/2540697179 shutdown_connections 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc0b6cb0 0x7f67bc0b5520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 --2- 172.21.15.60:0/2540697179 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b5a60 0x7f67bc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 -- 172.21.15.60:0/2540697179 >> 172.21.15.60:0/2540697179 conn(0x7f67bc0d0520 msgr2=0x7f67bc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 -- 172.21.15.60:0/2540697179 shutdown_connections 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.675+0000 7f67c264a640 1 -- 172.21.15.60:0/2540697179 wait complete. 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.676+0000 7f67c264a640 1 Processor -- start 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.676+0000 7f67c264a640 1 -- start start 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c264a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 0x7f67bc158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c264a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b6cb0 0x7f67bc0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c264a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc15db70 0x7f67bc1e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c264a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0cf1c0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c264a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0ce530 con 0x7f67bc15db70 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c264a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0cf340 con 0x7f67bc0b6cb0 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 0x7f67bc158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 0x7f67bc158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43314/0 (socket says 172.21.15.60:43314) 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67bbfff640 1 -- 172.21.15.60:0/2730506226 learned_addr learned my addr 172.21.15.60:0/2730506226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67c0be7640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc15db70 0x7f67bc1e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.677+0000 7f67bb7fe640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b6cb0 0x7f67bc0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67bbfff640 1 -- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b6cb0 msgr2=0x7f67bc0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67bbfff640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b6cb0 0x7f67bc0c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67bbfff640 1 -- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc15db70 msgr2=0x7f67bc1e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67bbfff640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc15db70 0x7f67bc1e6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67bbfff640 1 -- 172.21.15.60:0/2730506226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0c7770 con 0x7f67bc0b5a60 2024-04-25T08:33:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67bbfff640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 0x7f67bc158a20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c460 tx=0x7f67b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00074e0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b00047e0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004bd0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc0c7a50 con 0x7f67bc0b5a60 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.678+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc0c8010 con 0x7f67bc0b5a60 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.679+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc073990 con 0x7f67bc0b5a60 2024-04-25T08:33:08.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.679+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67b0011070 con 0x7f67bc0b5a60 2024-04-25T08:33:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.680+0000 7f67b8ff9640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67a806b4d0 0x7f67a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.680+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67b0089ed0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.680+0000 7f67bb7fe640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67a806b4d0 0x7f67a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.681+0000 7f67bb7fe640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67a806b4d0 0x7f67a806d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f67ac034000 tx=0x7f67ac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.684+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67b0058bc0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.867+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67bc0d1820 con 0x7f67bc0b5a60 2024-04-25T08:33:08.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.868+0000 7f67b8ff9640 1 -- 172.21.15.60:0/2730506226 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f67b00584c0 con 0x7f67bc0b5a60 2024-04-25T08:33:08.864 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:08.864 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":66,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:07.898937+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67a806b4d0 msgr2=0x7f67a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67a806b4d0 0x7f67a806d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f67ac034000 tx=0x7f67ac02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 msgr2=0x7f67bc158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 0x7f67bc158a20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c460 tx=0x7f67b000c860 comp rx=0 tx=0).stop 2024-04-25T08:33:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 shutdown_connections 2024-04-25T08:33:08.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67a806b4d0 0x7f67a806d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67bc15db70 0x7f67bc1e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67bc0b6cb0 0x7f67bc0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 --2- 172.21.15.60:0/2730506226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67bc0b5a60 0x7f67bc158a20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 >> 172.21.15.60:0/2730506226 conn(0x7f67bc0d0520 msgr2=0x7f67bc15ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 shutdown_connections 2024-04-25T08:33:08.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.870+0000 7f67c264a640 1 -- 172.21.15.60:0/2730506226 wait complete. 2024-04-25T08:33:08.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:08.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.983+0000 7f2ea1953640 1 Processor -- start 2024-04-25T08:33:08.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.983+0000 7f2ea1953640 1 -- start start 2024-04-25T08:33:08.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2ea1953640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2e9c0d0ad0 0x7f2e9c168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2ea1953640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0b9940 con 0x7f2e9c15dd30 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2ea1953640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0d1df0 con 0x7f2e9c0d2e90 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2ea1953640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0b7a50 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2e9c15dd30 0x7f2e9c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37436/0 (socket says 172.21.15.60:37436) 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9affd640 1 -- 172.21.15.60:0/3521600220 learned_addr learned my addr 172.21.15.60:0/3521600220 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9b7fe640 1 --2- 172.21.15.60:0/3521600220 >> v2:172.21.15.100:3300/0 conn(0x7f2e9c0d0ad0 0x7f2e9c168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931145235 0 0) 0x7f2e9c0b9940 con 0x7f2e9c15dd30 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e88003520 con 0x7f2e9c15dd30 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047329928 0 0) 0x7f2e9c0d1df0 con 0x7f2e9c0d2e90 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c0b9940 con 0x7f2e9c0d2e90 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9b7fe640 1 -- 172.21.15.60:0/3521600220 >> v1:172.21.15.100:6789/0 conn(0x7f2e9c0d2e90 legacy=0x7f2e9c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9b7fe640 1 -- 172.21.15.60:0/3521600220 >> v1:172.21.15.60:6789/0 conn(0x7f2e9c15dd30 legacy=0x7f2e9c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.984+0000 7f2e9b7fe640 1 -- 172.21.15.60:0/3521600220 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0d20d0 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.985+0000 7f2e9b7fe640 1 --2- 172.21.15.60:0/3521600220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c168760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009d90 tx=0x7f2e9402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a49e2594831521e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.985+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94031600 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.985+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e940317a0 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.985+0000 7f2e9a7fc640 1 -- 172.21.15.60:0/3521600220 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94031b70 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 -- 172.21.15.60:0/3521600220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c0d0ad0 msgr2=0x7f2e9c168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3521600220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c168760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009d90 tx=0x7f2e9402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 -- 172.21.15.60:0/3521600220 shutdown_connections 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3521600220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c168760 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 -- 172.21.15.60:0/3521600220 >> 172.21.15.60:0/3521600220 conn(0x7f2e9c061c30 msgr2=0x7f2e9c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 -- 172.21.15.60:0/3521600220 shutdown_connections 2024-04-25T08:33:08.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.986+0000 7f2ea1953640 1 -- 172.21.15.60:0/3521600220 wait complete. 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.987+0000 7f2ea1953640 1 Processor -- start 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.987+0000 7f2ea1953640 1 -- start start 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2ea1953640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2ea1953640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e9c0d2e90 0x7f2e9c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2ea1953640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c15dd30 0x7f2e9c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2ea1953640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0d2970 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2ea1953640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0d2690 con 0x7f2e9c15dd30 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2ea1953640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0d1a10 con 0x7f2e9c0d2e90 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43330/0 (socket says 172.21.15.60:43330) 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 -- 172.21.15.60:0/3309530144 learned_addr learned my addr 172.21.15.60:0/3309530144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9b7fe640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c15dd30 0x7f2e9c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e93fff640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e9c0d2e90 0x7f2e9c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 -- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e9c0d2e90 msgr2=0x7f2e9c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e9c0d2e90 0x7f2e9c1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 -- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c15dd30 msgr2=0x7f2e9c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c15dd30 0x7f2e9c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.988+0000 7f2e9affd640 1 -- 172.21.15.60:0/3309530144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c158730 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.989+0000 7f2e9affd640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c158050 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c500 tx=0x7f2e8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.989+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c017070 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.989+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c005ce0 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.989+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c158a10 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.989+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0072f0 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.989+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c158fd0 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.990+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c073990 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.990+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e8c01a070 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.991+0000 7f2e98ff9640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e7c06b4d0 0x7f2e7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.991+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e8c089740 con 0x7f2e9c0d0ad0 2024-04-25T08:33:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.991+0000 7f2e93fff640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e7c06b4d0 0x7f2e7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:08.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.991+0000 7f2e93fff640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e7c06b4d0 0x7f2e7c06d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c156c00 tx=0x7f2e84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:08.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:08.995+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e8c057dd0 con 0x7f2e9c0d0ad0 2024-04-25T08:33:09.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.179+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e9c0b2420 con 0x7f2e9c0d0ad0 2024-04-25T08:33:09.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.180+0000 7f2e98ff9640 1 -- 172.21.15.60:0/3309530144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f2e8c05be70 con 0x7f2e9c0d0ad0 2024-04-25T08:33:09.177 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:09.177 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":66,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:07.898937+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e7c06b4d0 msgr2=0x7f2e7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e7c06b4d0 0x7f2e7c06d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c156c00 tx=0x7f2e84009040 comp rx=0 tx=0).stop 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 msgr2=0x7f2e9c158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c158050 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00c500 tx=0x7f2e8c00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 shutdown_connections 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e7c06b4d0 0x7f2e7c06d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e9c15dd30 0x7f2e9c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2e9c0d2e90 0x7f2e9c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 --2- 172.21.15.60:0/3309530144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2e9c0d0ad0 0x7f2e9c158050 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 >> 172.21.15.60:0/3309530144 conn(0x7f2e9c061c30 msgr2=0x7f2e9c0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:09.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 shutdown_connections 2024-04-25T08:33:09.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.183+0000 7f2ea1953640 1 -- 172.21.15.60:0/3309530144 wait complete. 2024-04-25T08:33:09.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:33:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 Processor -- start 2024-04-25T08:33:09.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 -- start start 2024-04-25T08:33:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1394161710 0x7f13940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139415e000 con 0x7f13940b9330 2024-04-25T08:33:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13940d1de0 con 0x7f1394161710 2024-04-25T08:33:09.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.298+0000 7f139a4cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1394161c60 con 0x7f139415dc20 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1394161710 0x7f13940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1394161710 0x7f13940cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43342/0 (socket says 172.21.15.60:43342) 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 -- 172.21.15.60:0/3887306398 learned_addr learned my addr 172.21.15.60:0/3887306398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 -- 172.21.15.60:0/3887306398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 msgr2=0x7f13940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 --2- 172.21.15.60:0/3887306398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 -- 172.21.15.60:0/3887306398 >> v1:172.21.15.60:6789/0 conn(0x7f13940b9330 legacy=0x7f139406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f1392ffd640 1 -- 172.21.15.60:0/3887306398 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020812921 0 0) 0x7f139415e000 con 0x7f13940b9330 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.299+0000 7f13937fe640 1 -- 172.21.15.60:0/3887306398 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394161f40 con 0x7f1394161710 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f13937fe640 1 --2- 172.21.15.60:0/3887306398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f13940cebb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1388009870 tx=0x7f138802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72aa837666c6354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f1392ffd640 1 -- 172.21.15.60:0/3887306398 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388037070 con 0x7f1394161710 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f1392ffd640 1 -- 172.21.15.60:0/3887306398 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1388032440 con 0x7f1394161710 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f1392ffd640 1 -- 172.21.15.60:0/3887306398 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388032810 con 0x7f1394161710 2024-04-25T08:33:09.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 -- 172.21.15.60:0/3887306398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 msgr2=0x7f13940cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 --2- 172.21.15.60:0/3887306398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f13940cebb0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1388009870 tx=0x7f138802a270 comp rx=0 tx=0).stop 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 -- 172.21.15.60:0/3887306398 shutdown_connections 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 --2- 172.21.15.60:0/3887306398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f13940cebb0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 --2- 172.21.15.60:0/3887306398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13940ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 -- 172.21.15.60:0/3887306398 >> 172.21.15.60:0/3887306398 conn(0x7f13940d0420 msgr2=0x7f13940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 -- 172.21.15.60:0/3887306398 shutdown_connections 2024-04-25T08:33:09.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.300+0000 7f139a4cb640 1 -- 172.21.15.60:0/3887306398 wait complete. 2024-04-25T08:33:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.301+0000 7f139a4cb640 1 Processor -- start 2024-04-25T08:33:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 -- start start 2024-04-25T08:33:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13940b9330 0x7f139414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13941e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f1394160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940b5e50 con 0x7f1394161710 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394162500 con 0x7f13940b9330 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f139a4cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13940b5fd0 con 0x7f139415dc20 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f1398a68640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f1394160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.302+0000 7f1398a68640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f1394160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43350/0 (socket says 172.21.15.60:43350) 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 -- 172.21.15.60:0/567666273 learned_addr learned my addr 172.21.15.60:0/567666273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f13937fe640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13941e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1393fff640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13940b9330 0x7f139414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 -- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 msgr2=0x7f13941e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13941e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 -- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13940b9330 msgr2=0x7f139414b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:09.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13940b9330 0x7f139414b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 -- 172.21.15.60:0/567666273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13941e8750 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1393fff640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13940b9330 0x7f139414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1398a68640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f1394160200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f138c00c7b0 tx=0x7f138c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c003b70 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c003d10 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941e8a30 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c005290 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.303+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941e8ff0 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.304+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394073a00 con 0x7f1394161710 2024-04-25T08:33:09.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.305+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f138c005430 con 0x7f1394161710 2024-04-25T08:33:09.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.305+0000 7f1390ff9640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f138006b500 0x7f138006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:09.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.305+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f138c08a110 con 0x7f1394161710 2024-04-25T08:33:09.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.305+0000 7f1393fff640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f138006b500 0x7f138006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:09.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.306+0000 7f1393fff640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f138006b500 0x7f138006d9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1384009890 tx=0x7f1384008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:09.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.310+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f138c058e00 con 0x7f1394161710 2024-04-25T08:33:09.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:09.528+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f13940cbab0 con 0x7f1394161710 2024-04-25T08:33:10.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.100+0000 7f1390ff9640 1 -- 172.21.15.60:0/567666273 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f138c058700 con 0x7f1394161710 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f138006b500 msgr2=0x7f138006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f138006b500 0x7f138006d9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1384009890 tx=0x7f1384008040 comp rx=0 tx=0).stop 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 msgr2=0x7f1394160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f1394160200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f138c00c7b0 tx=0x7f138c00cc70 comp rx=0 tx=0).stop 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 shutdown_connections 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1394161710 0x7f1394160200 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f139415dc20 0x7f13941e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f138006b500 0x7f138006d9c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 --2- 172.21.15.60:0/567666273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13940b9330 0x7f139414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 >> 172.21.15.60:0/567666273 conn(0x7f13940d0420 msgr2=0x7f13940d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 shutdown_connections 2024-04-25T08:33:10.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.103+0000 7f139a4cb640 1 -- 172.21.15.60:0/567666273 wait complete. 2024-04-25T08:33:10.112 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:33:10.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:33:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.226+0000 7fe1e821b640 1 Processor -- start 2024-04-25T08:33:10.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.226+0000 7fe1e821b640 1 -- start start 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e821b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e821b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1e006c9e0 0x7fe1e00b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e821b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e006cf50 con 0x7fe1e00b5620 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e821b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e00b5fc0 con 0x7fe1e006c9e0 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e821b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00ce530 con 0x7fe1e015b7d0 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e67b8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1e006c9e0 0x7fe1e00b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:10.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e57b6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe1e00b5620 0x7fe1e006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37448/0 (socket says 172.21.15.60:37448) 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e57b6640 1 -- 172.21.15.60:0/1269644978 learned_addr learned my addr 172.21.15.60:0/1269644978 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e67b8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe1e006c9e0 0x7fe1e00b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43364/0 (socket says 172.21.15.60:43364) 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e5fb7640 1 --2- 172.21.15.60:0/1269644978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e67b8640 1 -- 172.21.15.60:0/1269644978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 msgr2=0x7fe1e00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e67b8640 1 --2- 172.21.15.60:0/1269644978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e67b8640 1 -- 172.21.15.60:0/1269644978 >> v1:172.21.15.60:6789/0 conn(0x7fe1e00b5620 legacy=0x7fe1e006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e67b8640 1 -- 172.21.15.60:0/1269644978 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00ce810 con 0x7fe1e006c9e0 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.227+0000 7fe1e4fb5640 1 -- 172.21.15.60:0/1269644978 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614404749 0 0) 0x7fe1e006cf50 con 0x7fe1e00b5620 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.228+0000 7fe1e67b8640 1 --2- 172.21.15.60:0/1269644978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e006c9e0 0x7fe1e00b5a80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009d90 tx=0x7fe1d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e450b39d4b5b688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.228+0000 7fe1e4fb5640 1 -- 172.21.15.60:0/1269644978 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8007740 con 0x7fe1e006c9e0 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.228+0000 7fe1e4fb5640 1 -- 172.21.15.60:0/1269644978 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80078e0 con 0x7fe1e006c9e0 2024-04-25T08:33:10.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.228+0000 7fe1e4fb5640 1 -- 172.21.15.60:0/1269644978 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8007cb0 con 0x7fe1e006c9e0 2024-04-25T08:33:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 -- 172.21.15.60:0/1269644978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e006c9e0 msgr2=0x7fe1e00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1269644978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e006c9e0 0x7fe1e00b5a80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009d90 tx=0x7fe1d802a660 comp rx=0 tx=0).stop 2024-04-25T08:33:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 -- 172.21.15.60:0/1269644978 shutdown_connections 2024-04-25T08:33:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1269644978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e006c9e0 0x7fe1e00b5a80 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1269644978 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 -- 172.21.15.60:0/1269644978 >> 172.21.15.60:0/1269644978 conn(0x7fe1e00d0800 msgr2=0x7fe1e00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 -- 172.21.15.60:0/1269644978 shutdown_connections 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.229+0000 7fe1e821b640 1 -- 172.21.15.60:0/1269644978 wait complete. 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.230+0000 7fe1e821b640 1 Processor -- start 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.230+0000 7fe1e821b640 1 -- start start 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e821b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e006c9e0 0x7fe1e01591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e821b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 0x7fe1e01f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e821b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e821b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00cf550 con 0x7fe1e00b5620 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e821b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e006d230 con 0x7fe1e006c9e0 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e821b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e00cf6d0 con 0x7fe1e015b7d0 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 0x7fe1e01f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 0x7fe1e01f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43368/0 (socket says 172.21.15.60:43368) 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 -- 172.21.15.60:0/1208721393 learned_addr learned my addr 172.21.15.60:0/1208721393 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:10.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e67b8640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e5fb7640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e006c9e0 0x7fe1e01591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 -- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 msgr2=0x7fe1e00cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 -- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e006c9e0 msgr2=0x7fe1e01591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e006c9e0 0x7fe1e01591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.231+0000 7fe1e57b6640 1 -- 172.21.15.60:0/1208721393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00caf10 con 0x7fe1e00b5620 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.232+0000 7fe1e57b6640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 0x7fe1e01f3d50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c4d0 tx=0x7fe1d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.232+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0018070 con 0x7fe1e00b5620 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.232+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d0007500 con 0x7fe1e00b5620 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.232+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e00c8de0 con 0x7fe1e00b5620 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.232+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0004a20 con 0x7fe1e00b5620 2024-04-25T08:33:10.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.232+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e00c93a0 con 0x7fe1e00b5620 2024-04-25T08:33:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.233+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e0073a00 con 0x7fe1e00b5620 2024-04-25T08:33:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.233+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1d001b070 con 0x7fe1e00b5620 2024-04-25T08:33:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.234+0000 7fe1d67fc640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1c406b500 0x7fe1c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.234+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1d0089530 con 0x7fe1e00b5620 2024-04-25T08:33:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.234+0000 7fe1e5fb7640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1c406b500 0x7fe1c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:10.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.235+0000 7fe1e5fb7640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1c406b500 0x7fe1c406d9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0079a0 tx=0x7fe1dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:10.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.238+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1d0057c60 con 0x7fe1e00b5620 2024-04-25T08:33:10.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:10.463+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe1e00ce1d0 con 0x7fe1e00b5620 2024-04-25T08:33:11.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.102+0000 7fe1d67fc640 1 -- 172.21.15.60:0/1208721393 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe1d005bd00 con 0x7fe1e00b5620 2024-04-25T08:33:11.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1c406b500 msgr2=0x7fe1c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1c406b500 0x7fe1c406d9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0079a0 tx=0x7fe1dc008040 comp rx=0 tx=0).stop 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 msgr2=0x7fe1e01f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 0x7fe1e01f3d50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c4d0 tx=0x7fe1d000c990 comp rx=0 tx=0).stop 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 shutdown_connections 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe1e015b7d0 0x7fe1e00cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe1e00b5620 0x7fe1e01f3d50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe1c406b500 0x7fe1c406d9c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 --2- 172.21.15.60:0/1208721393 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1e006c9e0 0x7fe1e01591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 >> 172.21.15.60:0/1208721393 conn(0x7fe1e00d0800 msgr2=0x7fe1e006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 shutdown_connections 2024-04-25T08:33:11.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.104+0000 7fe1e821b640 1 -- 172.21.15.60:0/1208721393 wait complete. 2024-04-25T08:33:11.114 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:33:11.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.226+0000 7fe8097e7640 1 Processor -- start 2024-04-25T08:33:11.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.226+0000 7fe8097e7640 1 -- start start 2024-04-25T08:33:11.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8097e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe8040b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8097e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe8040d11b0 0x7fe8040d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8097e7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80415eac0 con 0x7fe80415e100 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8097e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8040d1c00 con 0x7fe8040d11b0 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8097e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80415eda0 con 0x7fe80415dd20 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe7fbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe80415e100 0x7fe8040d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:37464/0 (socket says 172.21.15.60:37464) 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe7fbfff640 1 -- 172.21.15.60:0/3702654694 learned_addr learned my addr 172.21.15.60:0/3702654694 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8037fe640 1 --2- 172.21.15.60:0/3702654694 >> v2:172.21.15.100:3300/0 conn(0x7fe8040d11b0 0x7fe8040d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe802ffd640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe8040b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020600894 0 0) 0x7fe80415eac0 con 0x7fe80415e100 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f0003520 con 0x7fe80415e100 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1918802162 0 0) 0x7fe7f0003520 con 0x7fe80415e100 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 msgr2=0x7fe8040b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe8040b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 >> v2:172.21.15.100:3300/0 conn(0x7fe8040d11b0 msgr2=0x7fe8040d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 --2- 172.21.15.60:0/3702654694 >> v2:172.21.15.100:3300/0 conn(0x7fe8040d11b0 0x7fe8040d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80415f080 con 0x7fe80415e100 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe7ec001670 con 0x7fe80415e100 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8037fe640 1 --2- 172.21.15.60:0/3702654694 >> v2:172.21.15.100:3300/0 conn(0x7fe8040d11b0 0x7fe8040d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 >> v1:172.21.15.60:6789/0 conn(0x7fe80415e100 legacy=0x7fe8040d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe7f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.227+0000 7fe8027fc640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f0003f30 0x7fe7f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe8027fc640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0004d40 0x7fe7f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8040d1c00 con 0x7fe7f0003f30 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80415eda0 con 0x7fe7f0004d40 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f000ba10 con 0x7fe80415dd20 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe7fbfff640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f0003f30 0x7fe7f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe802ffd640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe7f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe8037fe640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0004d40 0x7fe7f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe7fbfff640 1 -- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 msgr2=0x7fe7f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe7fbfff640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe7f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe7fbfff640 1 -- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0004d40 msgr2=0x7fe7f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe7fbfff640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0004d40 0x7fe7f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe7fbfff640 1 -- 172.21.15.60:0/3702654694 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f0003520 con 0x7fe7f0003f30 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.228+0000 7fe802ffd640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe80415dd20 0x7fe7f00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe7fbfff640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f0003f30 0x7fe7f0004800 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c270 tx=0x7fe7f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b9dcc83bb5c5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007d30 con 0x7fe7f0003f30 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec005ad0 con 0x7fe7f0003f30 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8027fc640 1 -- 172.21.15.60:0/3702654694 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec007070 con 0x7fe7f0003f30 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8097e7640 1 -- 172.21.15.60:0/3702654694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f0003f30 msgr2=0x7fe7f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3702654694 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe7f0003f30 0x7fe7f0004800 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c270 tx=0x7fe7f0005830 comp rx=0 tx=0).stop 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8037fe640 1 -- 172.21.15.60:0/3702654694 reap_dead start 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8097e7640 1 -- 172.21.15.60:0/3702654694 shutdown_connections 2024-04-25T08:33:11.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8097e7640 1 -- 172.21.15.60:0/3702654694 >> 172.21.15.60:0/3702654694 conn(0x7fe804061a20 msgr2=0x7fe804061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8097e7640 1 -- 172.21.15.60:0/3702654694 shutdown_connections 2024-04-25T08:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.229+0000 7fe8097e7640 1 -- 172.21.15.60:0/3702654694 wait complete. 2024-04-25T08:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 Processor -- start 2024-04-25T08:33:11.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 -- start start 2024-04-25T08:33:11.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0003f30 0x7fe80415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8040ac220 0x7fe8040b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 0x7fe8040b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80415e6e0 con 0x7fe8040b4e10 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80415e560 con 0x7fe7f0003f30 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.231+0000 7fe8097e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80415e860 con 0x7fe8040ac220 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 0x7fe8040b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 0x7fe8040b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43380/0 (socket says 172.21.15.60:43380) 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 -- 172.21.15.60:0/3704760669 learned_addr learned my addr 172.21.15.60:0/3704760669 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe802ffd640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0003f30 0x7fe80415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe7fbfff640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8040ac220 0x7fe8040b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 -- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8040ac220 msgr2=0x7fe8040b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8040ac220 0x7fe8040b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 -- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0003f30 msgr2=0x7fe80415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0003f30 0x7fe80415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 -- 172.21.15.60:0/3704760669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8040b27f0 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe7fbfff640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8040ac220 0x7fe8040b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8037fe640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 0x7fe8040b1ac0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe8040b33f0 tx=0x7fe7fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc003990 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc01b070 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc003b50 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8040ab740 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.232+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8040abd00 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.233+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe804073990 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.234+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7fc005d10 con 0x7fe8040b4e10 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.234+0000 7fe7fb7fe640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe7e406b4d0 0x7fe7e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.234+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7fc08a9c0 con 0x7fe8040b4e10 2024-04-25T08:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.234+0000 7fe802ffd640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe7e406b4d0 0x7fe7e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.235+0000 7fe802ffd640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe7e406b4d0 0x7fe7e406d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40099e0 tx=0x7fe7f4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.239+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7fc08ce20 con 0x7fe8040b4e10 2024-04-25T08:33:11.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.456+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8040d35e0 con 0x7fe8040b4e10 2024-04-25T08:33:11.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.457+0000 7fe7fb7fe640 1 -- 172.21.15.60:0/3704760669 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fe7fc025020 con 0x7fe8040b4e10 2024-04-25T08:33:11.453 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:11.453 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1884848904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1884848904},{"type":"v1","addr":"172.21.15.60:6835","nonce":1884848904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1120004533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1120004533},{"type":"v1","addr":"172.21.15.100:6835","nonce":1120004533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2249277423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2249277423},{"type":"v1","addr":"172.21.15.100:6837","nonce":2249277423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:10.464973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6834/1590331682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1590331682},{"type":"v1","addr":"172.21.15.60:6834","nonce":1590331682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:11.454 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 35 2024-04-25T08:33:11.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe7e406b4d0 msgr2=0x7fe7e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe7e406b4d0 0x7fe7e406d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40099e0 tx=0x7fe7f4008040 comp rx=0 tx=0).stop 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 msgr2=0x7fe8040b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 0x7fe8040b1ac0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe8040b33f0 tx=0x7fe7fc009f90 comp rx=0 tx=0).stop 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 shutdown_connections 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8040b4e10 0x7fe8040b1ac0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8040ac220 0x7fe8040b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe7e406b4d0 0x7fe7e406d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 --2- 172.21.15.60:0/3704760669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7f0003f30 0x7fe80415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 >> 172.21.15.60:0/3704760669 conn(0x7fe804061a20 msgr2=0x7fe8040c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 shutdown_connections 2024-04-25T08:33:11.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.459+0000 7fe8097e7640 1 -- 172.21.15.60:0/3704760669 wait complete. 2024-04-25T08:33:11.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:11.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.571+0000 7f634c89a640 1 Processor -- start 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.571+0000 7f634c89a640 1 -- start start 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634c89a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634c89a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63480b5a60 0x7f63480d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634c89a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63480ceee0 con 0x7f634815db70 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634c89a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63480d1d10 con 0x7f63480b5a60 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634c89a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63480d1ff0 con 0x7f63480b6cb0 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63480b5a60 0x7f63480d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f63480b5a60 0x7f63480d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43394/0 (socket says 172.21.15.60:43394) 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 -- 172.21.15.60:0/1799178101 learned_addr learned my addr 172.21.15.60:0/1799178101 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6345d9b640 1 --2- 172.21.15.60:0/1799178101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 -- 172.21.15.60:0/1799178101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 msgr2=0x7f63480b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 --2- 172.21.15.60:0/1799178101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 -- 172.21.15.60:0/1799178101 >> v1:172.21.15.100:6789/0 conn(0x7f634815db70 legacy=0x7f6348162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 -- 172.21.15.60:0/1799178101 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63480d22d0 con 0x7f63480b5a60 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f6346d9d640 1 --2- 172.21.15.60:0/1799178101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480d27f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6340009870 tx=0x7f634002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc0319186ee799d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634559a640 1 -- 172.21.15.60:0/1799178101 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340037070 con 0x7f63480b5a60 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634559a640 1 -- 172.21.15.60:0/1799178101 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340032440 con 0x7f63480b5a60 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.573+0000 7f634559a640 1 -- 172.21.15.60:0/1799178101 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340032810 con 0x7f63480b5a60 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 -- 172.21.15.60:0/1799178101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 msgr2=0x7f63480d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 --2- 172.21.15.60:0/1799178101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480d27f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6340009870 tx=0x7f634002a210 comp rx=0 tx=0).stop 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 -- 172.21.15.60:0/1799178101 shutdown_connections 2024-04-25T08:33:11.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 --2- 172.21.15.60:0/1799178101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480d27f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 --2- 172.21.15.60:0/1799178101 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 -- 172.21.15.60:0/1799178101 >> 172.21.15.60:0/1799178101 conn(0x7f63480d0520 msgr2=0x7f63480d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 -- 172.21.15.60:0/1799178101 shutdown_connections 2024-04-25T08:33:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.574+0000 7f634c89a640 1 -- 172.21.15.60:0/1799178101 wait complete. 2024-04-25T08:33:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.575+0000 7f634c89a640 1 Processor -- start 2024-04-25T08:33:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.575+0000 7f634c89a640 1 -- start start 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634c89a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634c89a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634c89a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f634815db70 0x7f63481f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634c89a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63480cf1c0 con 0x7f63480b5a60 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634c89a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63480ce530 con 0x7f634815db70 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634c89a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63480cf340 con 0x7f63480b6cb0 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43404/0 (socket says 172.21.15.60:43404) 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 -- 172.21.15.60:0/2916869498 learned_addr learned my addr 172.21.15.60:0/2916869498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f6345d9b640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f6346d9d640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f634815db70 0x7f63481f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 -- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 msgr2=0x7f63480c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 -- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f634815db70 msgr2=0x7f63481f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f634815db70 0x7f63481f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f634659c640 1 -- 172.21.15.60:0/2916869498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63481f5210 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f6345d9b640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.576+0000 7f6346d9d640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f634815db70 0x7f63481f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.577+0000 7f634659c640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480c84f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f633800ba50 tx=0x7f633800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.577+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338019070 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.577+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633800cc20 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.577+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63480c8f70 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.577+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338014440 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.577+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63481f54b0 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.578+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6348073990 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.578+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f633801c070 con 0x7f63480b5a60 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.579+0000 7f633effd640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f632806b4d0 0x7f632806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.579+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6338089fa0 con 0x7f63480b5a60 2024-04-25T08:33:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.579+0000 7f6345d9b640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f632806b4d0 0x7f632806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.579+0000 7f6345d9b640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f632806b4d0 0x7f632806d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6348157640 tx=0x7f6330008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.583+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63380586d0 con 0x7f63480b5a60 2024-04-25T08:33:11.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.768+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63480b9630 con 0x7f63480b5a60 2024-04-25T08:33:11.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.769+0000 7f633effd640 1 -- 172.21.15.60:0/2916869498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f633805c770 con 0x7f63480b5a60 2024-04-25T08:33:11.765 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:11.765 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":67,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:09.539822+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:11.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f632806b4d0 msgr2=0x7f632806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f632806b4d0 0x7f632806d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6348157640 tx=0x7f6330008040 comp rx=0 tx=0).stop 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 msgr2=0x7f63480c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480c84f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f633800ba50 tx=0x7f633800bf10 comp rx=0 tx=0).stop 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 shutdown_connections 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f632806b4d0 0x7f632806d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f634815db70 0x7f63481f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63480b6cb0 0x7f63480c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 --2- 172.21.15.60:0/2916869498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63480b5a60 0x7f63480c84f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 >> 172.21.15.60:0/2916869498 conn(0x7f63480d0520 msgr2=0x7f634806c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:11.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 shutdown_connections 2024-04-25T08:33:11.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.771+0000 7f634c89a640 1 -- 172.21.15.60:0/2916869498 wait complete. 2024-04-25T08:33:11.778 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:33:11.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:33:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.884+0000 7fbb826cd640 1 Processor -- start 2024-04-25T08:33:11.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.884+0000 7fbb826cd640 1 -- start start 2024-04-25T08:33:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb826cd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbb7c15dc20 0x7fbb7c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb826cd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb826cd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c15e000 con 0x7fbb7c0cec20 2024-04-25T08:33:11.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb826cd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b6bb0 con 0x7fbb7c15dc20 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb826cd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b6030 con 0x7fbb7c0b93a0 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb80c6a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbb7c0cec20 0x7fbb7c06c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59446/0 (socket says 172.21.15.60:59446) 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb80c6a640 1 -- 172.21.15.60:0/2855635026 learned_addr learned my addr 172.21.15.60:0/2855635026 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/2855635026 >> v2:172.21.15.100:3300/0 conn(0x7fbb7c15dc20 0x7fbb7c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb7b7fe640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb7affd640 1 -- 172.21.15.60:0/2855635026 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554911057 0 0) 0x7fbb74001490 con 0x7fbb7c0cec20 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.885+0000 7fbb7affd640 1 -- 172.21.15.60:0/2855635026 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb68003520 con 0x7fbb7c0cec20 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7affd640 1 -- 172.21.15.60:0/2855635026 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102594909 0 0) 0x7fbb68003520 con 0x7fbb7c0cec20 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7bfff640 1 -- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 msgr2=0x7fbb7c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7bfff640 1 -- 172.21.15.60:0/2855635026 >> v1:172.21.15.60:6789/0 conn(0x7fbb7c0cec20 legacy=0x7fbb7c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7bfff640 1 -- 172.21.15.60:0/2855635026 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c161de0 con 0x7fbb7c15dc20 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7b7fe640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:11.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.886+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c1616b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb640097a0 tx=0x7fbb6402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b01c4caf6378381 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb7affd640 1 -- 172.21.15.60:0/2855635026 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64037070 con 0x7fbb7c15dc20 2024-04-25T08:33:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb7affd640 1 -- 172.21.15.60:0/2855635026 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6402aba0 con 0x7fbb7c15dc20 2024-04-25T08:33:11.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb7affd640 1 -- 172.21.15.60:0/2855635026 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74001bc0 con 0x7fbb7c15dc20 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 -- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 msgr2=0x7fbb7c1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c1616b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb640097a0 tx=0x7fbb6402a110 comp rx=0 tx=0).stop 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 -- 172.21.15.60:0/2855635026 shutdown_connections 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 --2- 172.21.15.60:0/2855635026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c1616b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 -- 172.21.15.60:0/2855635026 >> 172.21.15.60:0/2855635026 conn(0x7fbb7c0d0490 msgr2=0x7fbb7c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 -- 172.21.15.60:0/2855635026 shutdown_connections 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.887+0000 7fbb826cd640 1 -- 172.21.15.60:0/2855635026 wait complete. 2024-04-25T08:33:11.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.888+0000 7fbb826cd640 1 Processor -- start 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 -- start start 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 0x7fbb7c149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0d3520 con 0x7fbb7c0cec20 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c1623a0 con 0x7fbb7c15dc20 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb826cd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0d36a0 con 0x7fbb7c0b93a0 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.889+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 0x7fbb7c149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 0x7fbb7c149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56412/0 (socket says 172.21.15.60:56412) 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 -- 172.21.15.60:0/1819583321 learned_addr learned my addr 172.21.15.60:0/1819583321 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 -- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 msgr2=0x7fbb7c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb80c6a640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 -- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 msgr2=0x7fbb7c146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 -- 172.21.15.60:0/1819583321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c14a3b0 con 0x7fbb7c0cec20 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb80c6a640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb7b7fe640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 0x7fbb7c149e70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00e960 tx=0x7fbb6c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c017070 con 0x7fbb7c0cec20 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c00ca00 con 0x7fbb7c0cec20 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00cdd0 con 0x7fbb7c0cec20 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c160ab0 con 0x7fbb7c0cec20 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.890+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c161070 con 0x7fbb7c0cec20 2024-04-25T08:33:11.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.891+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c161730 con 0x7fbb7c0cec20 2024-04-25T08:33:11.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.892+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb6c01a070 con 0x7fbb7c0cec20 2024-04-25T08:33:11.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.892+0000 7fbb78ff9640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:11.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.892+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb6c08a1b0 con 0x7fbb7c0cec20 2024-04-25T08:33:11.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.892+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:11.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.893+0000 7fbb7bfff640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb64034000 tx=0x7fbb6402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:11.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:11.896+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb7c161730 con 0x7fbb7c0cec20 2024-04-25T08:33:12.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.074+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb7c0d1720 con 0x7fbb5c06b4d0 2024-04-25T08:33:12.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.081+0000 7fbb78ff9640 1 -- 172.21.15.60:0/1819583321 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252019 (secure 0 0 0) 0x7fbb7c0d1720 con 0x7fbb5c06b4d0 2024-04-25T08:33:12.079 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:12.081 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb5c06b4d0 msgr2=0x7fbb5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbb64034000 tx=0x7fbb6402e000 comp rx=0 tx=0).stop 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 msgr2=0x7fbb7c149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 0x7fbb7c149e70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00e960 tx=0x7fbb6c00ee20 comp rx=0 tx=0).stop 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 shutdown_connections 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb5c06b4d0 0x7fbb5c06d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb7c15dc20 0x7fbb7c146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb7c0cec20 0x7fbb7c149e70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 --2- 172.21.15.60:0/1819583321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb7c0b93a0 0x7fbb7c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 >> 172.21.15.60:0/1819583321 conn(0x7fbb7c0d0490 msgr2=0x7fbb7c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 shutdown_connections 2024-04-25T08:33:12.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.083+0000 7fbb826cd640 1 -- 172.21.15.60:0/1819583321 wait complete. 2024-04-25T08:33:12.090 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-25T08:33:11.450278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306912,"kb_used_data":7528,"kb_used_omap":12,"kb_used_meta":299251,"kb_avail":754667808,"statfs":{"total":773094113280,"available":772779835392,"internally_reserved":0,"allocated":7708672,"data_stored":4115572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":306433584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.926550"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547374+0000","last_change":"2024-04-25T08:33:08.560273+0000","last_active":"2024-04-25T08:33:09.547374+0000","last_peered":"2024-04-25T08:33:09.547374+0000","last_clean":"2024-04-25T08:33:09.547374+0000","last_became_active":"2024-04-25T08:33:08.560000+0000","last_became_peered":"2024-04-25T08:33:08.560000+0000","last_unstale":"2024-04-25T08:33:09.547374+0000","last_undegraded":"2024-04-25T08:33:09.547374+0000","last_fullsized":"2024-04-25T08:33:09.547374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:45.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547181+0000","last_change":"2024-04-25T08:33:08.556370+0000","last_active":"2024-04-25T08:33:09.547181+0000","last_peered":"2024-04-25T08:33:09.547181+0000","last_clean":"2024-04-25T08:33:09.547181+0000","last_became_active":"2024-04-25T08:33:08.556061+0000","last_became_peered":"2024-04-25T08:33:08.556061+0000","last_unstale":"2024-04-25T08:33:09.547181+0000","last_undegraded":"2024-04-25T08:33:09.547181+0000","last_fullsized":"2024-04-25T08:33:09.547181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:36.862604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864695+0000","last_change":"2024-04-25T08:33:08.553695+0000","last_active":"2024-04-25T08:33:09.864695+0000","last_peered":"2024-04-25T08:33:09.864695+0000","last_clean":"2024-04-25T08:33:09.864695+0000","last_became_active":"2024-04-25T08:33:08.553440+0000","last_became_peered":"2024-04-25T08:33:08.553440+0000","last_unstale":"2024-04-25T08:33:09.864695+0000","last_undegraded":"2024-04-25T08:33:09.864695+0000","last_fullsized":"2024-04-25T08:33:09.864695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551688+0000","last_change":"2024-04-25T08:33:08.554151+0000","last_active":"2024-04-25T08:33:09.551688+0000","last_peered":"2024-04-25T08:33:09.551688+0000","last_clean":"2024-04-25T08:33:09.551688+0000","last_became_active":"2024-04-25T08:33:08.553866+0000","last_became_peered":"2024-04-25T08:33:08.553866+0000","last_unstale":"2024-04-25T08:33:09.551688+0000","last_undegraded":"2024-04-25T08:33:09.551688+0000","last_fullsized":"2024-04-25T08:33:09.551688+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:59.510900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550429+0000","last_change":"2024-04-25T08:33:08.548448+0000","last_active":"2024-04-25T08:33:09.550429+0000","last_peered":"2024-04-25T08:33:09.550429+0000","last_clean":"2024-04-25T08:33:09.550429+0000","last_became_active":"2024-04-25T08:33:08.547964+0000","last_became_peered":"2024-04-25T08:33:08.547964+0000","last_unstale":"2024-04-25T08:33:09.550429+0000","last_undegraded":"2024-04-25T08:33:09.550429+0000","last_fullsized":"2024-04-25T08:33:09.550429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:53.399708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864894+0000","last_change":"2024-04-25T08:33:08.550386+0000","last_active":"2024-04-25T08:33:09.864894+0000","last_peered":"2024-04-25T08:33:09.864894+0000","last_clean":"2024-04-25T08:33:09.864894+0000","last_became_active":"2024-04-25T08:33:08.550106+0000","last_became_peered":"2024-04-25T08:33:08.550106+0000","last_unstale":"2024-04-25T08:33:09.864894+0000","last_undegraded":"2024-04-25T08:33:09.864894+0000","last_fullsized":"2024-04-25T08:33:09.864894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:58.250815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864646+0000","last_change":"2024-04-25T08:33:08.558798+0000","last_active":"2024-04-25T08:33:09.864646+0000","last_peered":"2024-04-25T08:33:09.864646+0000","last_clean":"2024-04-25T08:33:09.864646+0000","last_became_active":"2024-04-25T08:33:08.558484+0000","last_became_peered":"2024-04-25T08:33:08.558484+0000","last_unstale":"2024-04-25T08:33:09.864646+0000","last_undegraded":"2024-04-25T08:33:09.864646+0000","last_fullsized":"2024-04-25T08:33:09.864646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:34.431464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864737+0000","last_change":"2024-04-25T08:33:08.556616+0000","last_active":"2024-04-25T08:33:09.864737+0000","last_peered":"2024-04-25T08:33:09.864737+0000","last_clean":"2024-04-25T08:33:09.864737+0000","last_became_active":"2024-04-25T08:33:08.556366+0000","last_became_peered":"2024-04-25T08:33:08.556366+0000","last_unstale":"2024-04-25T08:33:09.864737+0000","last_undegraded":"2024-04-25T08:33:09.864737+0000","last_fullsized":"2024-04-25T08:33:09.864737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865294+0000","last_change":"2024-04-25T08:33:08.552548+0000","last_active":"2024-04-25T08:33:09.865294+0000","last_peered":"2024-04-25T08:33:09.865294+0000","last_clean":"2024-04-25T08:33:09.865294+0000","last_became_active":"2024-04-25T08:33:08.550323+0000","last_became_peered":"2024-04-25T08:33:08.550323+0000","last_unstale":"2024-04-25T08:33:09.865294+0000","last_undegraded":"2024-04-25T08:33:09.865294+0000","last_fullsized":"2024-04-25T08:33:09.865294+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:25.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865072+0000","last_change":"2024-04-25T08:33:08.558332+0000","last_active":"2024-04-25T08:33:09.865072+0000","last_peered":"2024-04-25T08:33:09.865072+0000","last_clean":"2024-04-25T08:33:09.865072+0000","last_became_active":"2024-04-25T08:33:08.558055+0000","last_became_peered":"2024-04-25T08:33:08.558055+0000","last_unstale":"2024-04-25T08:33:09.865072+0000","last_undegraded":"2024-04-25T08:33:09.865072+0000","last_fullsized":"2024-04-25T08:33:09.865072+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551733+0000","last_change":"2024-04-25T08:33:08.559023+0000","last_active":"2024-04-25T08:33:09.551733+0000","last_peered":"2024-04-25T08:33:09.551733+0000","last_clean":"2024-04-25T08:33:09.551733+0000","last_became_active":"2024-04-25T08:33:08.558776+0000","last_became_peered":"2024-04-25T08:33:08.558776+0000","last_unstale":"2024-04-25T08:33:09.551733+0000","last_undegraded":"2024-04-25T08:33:09.551733+0000","last_fullsized":"2024-04-25T08:33:09.551733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:11.754363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864746+0000","last_change":"2024-04-25T08:33:08.559719+0000","last_active":"2024-04-25T08:33:09.864746+0000","last_peered":"2024-04-25T08:33:09.864746+0000","last_clean":"2024-04-25T08:33:09.864746+0000","last_became_active":"2024-04-25T08:33:08.559436+0000","last_became_peered":"2024-04-25T08:33:08.559436+0000","last_unstale":"2024-04-25T08:33:09.864746+0000","last_undegraded":"2024-04-25T08:33:09.864746+0000","last_fullsized":"2024-04-25T08:33:09.864746+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.057423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864606+0000","last_change":"2024-04-25T08:33:08.548981+0000","last_active":"2024-04-25T08:33:09.864606+0000","last_peered":"2024-04-25T08:33:09.864606+0000","last_clean":"2024-04-25T08:33:09.864606+0000","last_became_active":"2024-04-25T08:33:08.548648+0000","last_became_peered":"2024-04-25T08:33:08.548648+0000","last_unstale":"2024-04-25T08:33:09.864606+0000","last_undegraded":"2024-04-25T08:33:09.864606+0000","last_fullsized":"2024-04-25T08:33:09.864606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:12.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865155+0000","last_change":"2024-04-25T08:33:08.551464+0000","last_active":"2024-04-25T08:33:09.865155+0000","last_peered":"2024-04-25T08:33:09.865155+0000","last_clean":"2024-04-25T08:33:09.865155+0000","last_became_active":"2024-04-25T08:33:08.551191+0000","last_became_peered":"2024-04-25T08:33:08.551191+0000","last_unstale":"2024-04-25T08:33:09.865155+0000","last_undegraded":"2024-04-25T08:33:09.865155+0000","last_fullsized":"2024-04-25T08:33:09.865155+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.057423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865521+0000","last_change":"2024-04-25T08:33:08.554896+0000","last_active":"2024-04-25T08:33:09.865521+0000","last_peered":"2024-04-25T08:33:09.865521+0000","last_clean":"2024-04-25T08:33:09.865521+0000","last_became_active":"2024-04-25T08:33:08.554601+0000","last_became_peered":"2024-04-25T08:33:08.554601+0000","last_unstale":"2024-04-25T08:33:09.865521+0000","last_undegraded":"2024-04-25T08:33:09.865521+0000","last_fullsized":"2024-04-25T08:33:09.865521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:04.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864926+0000","last_change":"2024-04-25T08:33:08.543627+0000","last_active":"2024-04-25T08:33:09.864926+0000","last_peered":"2024-04-25T08:33:09.864926+0000","last_clean":"2024-04-25T08:33:09.864926+0000","last_became_active":"2024-04-25T08:33:08.543363+0000","last_became_peered":"2024-04-25T08:33:08.543363+0000","last_unstale":"2024-04-25T08:33:09.864926+0000","last_undegraded":"2024-04-25T08:33:09.864926+0000","last_fullsized":"2024-04-25T08:33:09.864926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:46.998248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547399+0000","last_change":"2024-04-25T08:33:08.563503+0000","last_active":"2024-04-25T08:33:09.547399+0000","last_peered":"2024-04-25T08:33:09.547399+0000","last_clean":"2024-04-25T08:33:09.547399+0000","last_became_active":"2024-04-25T08:33:08.563198+0000","last_became_peered":"2024-04-25T08:33:08.563198+0000","last_unstale":"2024-04-25T08:33:09.547399+0000","last_undegraded":"2024-04-25T08:33:09.547399+0000","last_fullsized":"2024-04-25T08:33:09.547399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:13.480153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551611+0000","last_change":"2024-04-25T08:33:08.551702+0000","last_active":"2024-04-25T08:33:09.551611+0000","last_peered":"2024-04-25T08:33:09.551611+0000","last_clean":"2024-04-25T08:33:09.551611+0000","last_became_active":"2024-04-25T08:33:08.551430+0000","last_became_peered":"2024-04-25T08:33:08.551430+0000","last_unstale":"2024-04-25T08:33:09.551611+0000","last_undegraded":"2024-04-25T08:33:09.551611+0000","last_fullsized":"2024-04-25T08:33:09.551611+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:09.826039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550847+0000","last_change":"2024-04-25T08:33:08.554954+0000","last_active":"2024-04-25T08:33:09.550847+0000","last_peered":"2024-04-25T08:33:09.550847+0000","last_clean":"2024-04-25T08:33:09.550847+0000","last_became_active":"2024-04-25T08:33:08.554666+0000","last_became_peered":"2024-04-25T08:33:08.554666+0000","last_unstale":"2024-04-25T08:33:09.550847+0000","last_undegraded":"2024-04-25T08:33:09.550847+0000","last_fullsized":"2024-04-25T08:33:09.550847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:45.438238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864472+0000","last_change":"2024-04-25T08:33:08.546368+0000","last_active":"2024-04-25T08:33:09.864472+0000","last_peered":"2024-04-25T08:33:09.864472+0000","last_clean":"2024-04-25T08:33:09.864472+0000","last_became_active":"2024-04-25T08:33:08.546103+0000","last_became_peered":"2024-04-25T08:33:08.546103+0000","last_unstale":"2024-04-25T08:33:09.864472+0000","last_undegraded":"2024-04-25T08:33:09.864472+0000","last_fullsized":"2024-04-25T08:33:09.864472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:25.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864712+0000","last_change":"2024-04-25T08:33:08.556278+0000","last_active":"2024-04-25T08:33:09.864712+0000","last_peered":"2024-04-25T08:33:09.864712+0000","last_clean":"2024-04-25T08:33:09.864712+0000","last_became_active":"2024-04-25T08:33:08.556013+0000","last_became_peered":"2024-04-25T08:33:08.556013+0000","last_unstale":"2024-04-25T08:33:09.864712+0000","last_undegraded":"2024-04-25T08:33:09.864712+0000","last_fullsized":"2024-04-25T08:33:09.864712+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:30.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864605+0000","last_change":"2024-04-25T08:33:08.553832+0000","last_active":"2024-04-25T08:33:09.864605+0000","last_peered":"2024-04-25T08:33:09.864605+0000","last_clean":"2024-04-25T08:33:09.864605+0000","last_became_active":"2024-04-25T08:33:08.553391+0000","last_became_peered":"2024-04-25T08:33:08.553391+0000","last_unstale":"2024-04-25T08:33:09.864605+0000","last_undegraded":"2024-04-25T08:33:09.864605+0000","last_fullsized":"2024-04-25T08:33:09.864605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.805711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864266+0000","last_change":"2024-04-25T08:33:08.560624+0000","last_active":"2024-04-25T08:33:09.864266+0000","last_peered":"2024-04-25T08:33:09.864266+0000","last_clean":"2024-04-25T08:33:09.864266+0000","last_became_active":"2024-04-25T08:33:08.560326+0000","last_became_peered":"2024-04-25T08:33:08.560326+0000","last_unstale":"2024-04-25T08:33:09.864266+0000","last_undegraded":"2024-04-25T08:33:09.864266+0000","last_fullsized":"2024-04-25T08:33:09.864266+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864669+0000","last_change":"2024-04-25T08:33:08.559445+0000","last_active":"2024-04-25T08:33:09.864669+0000","last_peered":"2024-04-25T08:33:09.864669+0000","last_clean":"2024-04-25T08:33:09.864669+0000","last_became_active":"2024-04-25T08:33:08.559161+0000","last_became_peered":"2024-04-25T08:33:08.559161+0000","last_unstale":"2024-04-25T08:33:09.864669+0000","last_undegraded":"2024-04-25T08:33:09.864669+0000","last_fullsized":"2024-04-25T08:33:09.864669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:19.097337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865423+0000","last_change":"2024-04-25T08:33:08.555080+0000","last_active":"2024-04-25T08:33:09.865423+0000","last_peered":"2024-04-25T08:33:09.865423+0000","last_clean":"2024-04-25T08:33:09.865423+0000","last_became_active":"2024-04-25T08:33:08.554825+0000","last_became_peered":"2024-04-25T08:33:08.554825+0000","last_unstale":"2024-04-25T08:33:09.865423+0000","last_undegraded":"2024-04-25T08:33:09.865423+0000","last_fullsized":"2024-04-25T08:33:09.865423+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:04.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865271+0000","last_change":"2024-04-25T08:33:08.554523+0000","last_active":"2024-04-25T08:33:09.865271+0000","last_peered":"2024-04-25T08:33:09.865271+0000","last_clean":"2024-04-25T08:33:09.865271+0000","last_became_active":"2024-04-25T08:33:08.554257+0000","last_became_peered":"2024-04-25T08:33:08.554257+0000","last_unstale":"2024-04-25T08:33:09.865271+0000","last_undegraded":"2024-04-25T08:33:09.865271+0000","last_fullsized":"2024-04-25T08:33:09.865271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:25.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551792+0000","last_change":"2024-04-25T08:33:08.554280+0000","last_active":"2024-04-25T08:33:09.551792+0000","last_peered":"2024-04-25T08:33:09.551792+0000","last_clean":"2024-04-25T08:33:09.551792+0000","last_became_active":"2024-04-25T08:33:08.553988+0000","last_became_peered":"2024-04-25T08:33:08.553988+0000","last_unstale":"2024-04-25T08:33:09.551792+0000","last_undegraded":"2024-04-25T08:33:09.551792+0000","last_fullsized":"2024-04-25T08:33:09.551792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:17.271075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547058+0000","last_change":"2024-04-25T08:33:08.550356+0000","last_active":"2024-04-25T08:33:09.547058+0000","last_peered":"2024-04-25T08:33:09.547058+0000","last_clean":"2024-04-25T08:33:09.547058+0000","last_became_active":"2024-04-25T08:33:08.549045+0000","last_became_peered":"2024-04-25T08:33:08.549045+0000","last_unstale":"2024-04-25T08:33:09.547058+0000","last_undegraded":"2024-04-25T08:33:09.547058+0000","last_fullsized":"2024-04-25T08:33:09.547058+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:45.438238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864937+0000","last_change":"2024-04-25T08:33:08.551934+0000","last_active":"2024-04-25T08:33:09.864937+0000","last_peered":"2024-04-25T08:33:09.864937+0000","last_clean":"2024-04-25T08:33:09.864937+0000","last_became_active":"2024-04-25T08:33:08.551384+0000","last_became_peered":"2024-04-25T08:33:08.551384+0000","last_unstale":"2024-04-25T08:33:09.864937+0000","last_undegraded":"2024-04-25T08:33:09.864937+0000","last_fullsized":"2024-04-25T08:33:09.864937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:12.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864544+0000","last_change":"2024-04-25T08:33:08.557334+0000","last_active":"2024-04-25T08:33:09.864544+0000","last_peered":"2024-04-25T08:33:09.864544+0000","last_clean":"2024-04-25T08:33:09.864544+0000","last_became_active":"2024-04-25T08:33:08.557081+0000","last_became_peered":"2024-04-25T08:33:08.557081+0000","last_unstale":"2024-04-25T08:33:09.864544+0000","last_undegraded":"2024-04-25T08:33:09.864544+0000","last_fullsized":"2024-04-25T08:33:09.864544+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:58.250815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550684+0000","last_change":"2024-04-25T08:33:08.546322+0000","last_active":"2024-04-25T08:33:09.550684+0000","last_peered":"2024-04-25T08:33:09.550684+0000","last_clean":"2024-04-25T08:33:09.550684+0000","last_became_active":"2024-04-25T08:33:08.545614+0000","last_became_peered":"2024-04-25T08:33:08.545614+0000","last_unstale":"2024-04-25T08:33:09.550684+0000","last_undegraded":"2024-04-25T08:33:09.550684+0000","last_fullsized":"2024-04-25T08:33:09.550684+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:35.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864906+0000","last_change":"2024-04-25T08:33:08.551250+0000","last_active":"2024-04-25T08:33:09.864906+0000","last_peered":"2024-04-25T08:33:09.864906+0000","last_clean":"2024-04-25T08:33:09.864906+0000","last_became_active":"2024-04-25T08:33:08.550950+0000","last_became_peered":"2024-04-25T08:33:08.550950+0000","last_unstale":"2024-04-25T08:33:09.864906+0000","last_undegraded":"2024-04-25T08:33:09.864906+0000","last_fullsized":"2024-04-25T08:33:09.864906+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:12.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550781+0000","last_change":"2024-04-25T08:33:08.553945+0000","last_active":"2024-04-25T08:33:09.550781+0000","last_peered":"2024-04-25T08:33:09.550781+0000","last_clean":"2024-04-25T08:33:09.550781+0000","last_became_active":"2024-04-25T08:33:08.550496+0000","last_became_peered":"2024-04-25T08:33:08.550496+0000","last_unstale":"2024-04-25T08:33:09.550781+0000","last_undegraded":"2024-04-25T08:33:09.550781+0000","last_fullsized":"2024-04-25T08:33:09.550781+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.382160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547426+0000","last_change":"2024-04-25T08:33:07.544574+0000","last_active":"2024-04-25T08:33:09.547426+0000","last_peered":"2024-04-25T08:33:09.547426+0000","last_clean":"2024-04-25T08:33:09.547426+0000","last_became_active":"2024-04-25T08:33:07.544311+0000","last_became_peered":"2024-04-25T08:33:07.544311+0000","last_unstale":"2024-04-25T08:33:09.547426+0000","last_undegraded":"2024-04-25T08:33:09.547426+0000","last_fullsized":"2024-04-25T08:33:09.547426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:58.505300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551832+0000","last_change":"2024-04-25T08:33:08.541418+0000","last_active":"2024-04-25T08:33:09.551832+0000","last_peered":"2024-04-25T08:33:09.551832+0000","last_clean":"2024-04-25T08:33:09.551832+0000","last_became_active":"2024-04-25T08:33:08.540238+0000","last_became_peered":"2024-04-25T08:33:08.540238+0000","last_unstale":"2024-04-25T08:33:09.551832+0000","last_undegraded":"2024-04-25T08:33:09.551832+0000","last_fullsized":"2024-04-25T08:33:09.551832+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:42.343594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865231+0000","last_change":"2024-04-25T08:33:07.538223+0000","last_active":"2024-04-25T08:33:09.865231+0000","last_peered":"2024-04-25T08:33:09.865231+0000","last_clean":"2024-04-25T08:33:09.865231+0000","last_became_active":"2024-04-25T08:33:07.537909+0000","last_became_peered":"2024-04-25T08:33:07.537909+0000","last_unstale":"2024-04-25T08:33:09.865231+0000","last_undegraded":"2024-04-25T08:33:09.865231+0000","last_fullsized":"2024-04-25T08:33:09.865231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:30.854905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865069+0000","last_change":"2024-04-25T08:33:08.546649+0000","last_active":"2024-04-25T08:33:09.865069+0000","last_peered":"2024-04-25T08:33:09.865069+0000","last_clean":"2024-04-25T08:33:09.865069+0000","last_became_active":"2024-04-25T08:33:08.546377+0000","last_became_peered":"2024-04-25T08:33:08.546377+0000","last_unstale":"2024-04-25T08:33:09.865069+0000","last_undegraded":"2024-04-25T08:33:09.865069+0000","last_fullsized":"2024-04-25T08:33:09.865069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:30.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865020+0000","last_change":"2024-04-25T08:33:07.543378+0000","last_active":"2024-04-25T08:33:09.865020+0000","last_peered":"2024-04-25T08:33:09.865020+0000","last_clean":"2024-04-25T08:33:09.865020+0000","last_became_active":"2024-04-25T08:33:07.543068+0000","last_became_peered":"2024-04-25T08:33:07.543068+0000","last_unstale":"2024-04-25T08:33:09.865020+0000","last_undegraded":"2024-04-25T08:33:09.865020+0000","last_fullsized":"2024-04-25T08:33:09.865020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:53.492411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864801+0000","last_change":"2024-04-25T08:33:08.555873+0000","last_active":"2024-04-25T08:33:09.864801+0000","last_peered":"2024-04-25T08:33:09.864801+0000","last_clean":"2024-04-25T08:33:09.864801+0000","last_became_active":"2024-04-25T08:33:08.555575+0000","last_became_peered":"2024-04-25T08:33:08.555575+0000","last_unstale":"2024-04-25T08:33:09.864801+0000","last_undegraded":"2024-04-25T08:33:09.864801+0000","last_fullsized":"2024-04-25T08:33:09.864801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:58.250815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864868+0000","last_change":"2024-04-25T08:33:07.544654+0000","last_active":"2024-04-25T08:33:09.864868+0000","last_peered":"2024-04-25T08:33:09.864868+0000","last_clean":"2024-04-25T08:33:09.864868+0000","last_became_active":"2024-04-25T08:33:07.544158+0000","last_became_peered":"2024-04-25T08:33:07.544158+0000","last_unstale":"2024-04-25T08:33:09.864868+0000","last_undegraded":"2024-04-25T08:33:09.864868+0000","last_fullsized":"2024-04-25T08:33:09.864868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:44.778767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865062+0000","last_change":"2024-04-25T08:33:08.554621+0000","last_active":"2024-04-25T08:33:09.865062+0000","last_peered":"2024-04-25T08:33:09.865062+0000","last_clean":"2024-04-25T08:33:09.865062+0000","last_became_active":"2024-04-25T08:33:08.553827+0000","last_became_peered":"2024-04-25T08:33:08.553827+0000","last_unstale":"2024-04-25T08:33:09.865062+0000","last_undegraded":"2024-04-25T08:33:09.865062+0000","last_fullsized":"2024-04-25T08:33:09.865062+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:33.554420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864587+0000","last_change":"2024-04-25T08:33:07.541015+0000","last_active":"2024-04-25T08:33:09.864587+0000","last_peered":"2024-04-25T08:33:09.864587+0000","last_clean":"2024-04-25T08:33:09.864587+0000","last_became_active":"2024-04-25T08:33:07.540688+0000","last_became_peered":"2024-04-25T08:33:07.540688+0000","last_unstale":"2024-04-25T08:33:09.864587+0000","last_undegraded":"2024-04-25T08:33:09.864587+0000","last_fullsized":"2024-04-25T08:33:09.864587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:22.449454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.546864+0000","last_change":"2024-04-25T08:33:08.563441+0000","last_active":"2024-04-25T08:33:09.546864+0000","last_peered":"2024-04-25T08:33:09.546864+0000","last_clean":"2024-04-25T08:33:09.546864+0000","last_became_active":"2024-04-25T08:33:08.563151+0000","last_became_peered":"2024-04-25T08:33:08.563151+0000","last_unstale":"2024-04-25T08:33:09.546864+0000","last_undegraded":"2024-04-25T08:33:09.546864+0000","last_fullsized":"2024-04-25T08:33:09.546864+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:29.540053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864445+0000","last_change":"2024-04-25T08:33:07.546439+0000","last_active":"2024-04-25T08:33:09.864445+0000","last_peered":"2024-04-25T08:33:09.864445+0000","last_clean":"2024-04-25T08:33:09.864445+0000","last_became_active":"2024-04-25T08:33:07.545798+0000","last_became_peered":"2024-04-25T08:33:07.545798+0000","last_unstale":"2024-04-25T08:33:09.864445+0000","last_undegraded":"2024-04-25T08:33:09.864445+0000","last_fullsized":"2024-04-25T08:33:09.864445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:30.854905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550222+0000","last_change":"2024-04-25T08:33:08.552270+0000","last_active":"2024-04-25T08:33:09.550222+0000","last_peered":"2024-04-25T08:33:09.550222+0000","last_clean":"2024-04-25T08:33:09.550222+0000","last_became_active":"2024-04-25T08:33:08.551947+0000","last_became_peered":"2024-04-25T08:33:08.551947+0000","last_unstale":"2024-04-25T08:33:09.550222+0000","last_undegraded":"2024-04-25T08:33:09.550222+0000","last_fullsized":"2024-04-25T08:33:09.550222+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:35.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864273+0000","last_change":"2024-04-25T08:33:07.543221+0000","last_active":"2024-04-25T08:33:09.864273+0000","last_peered":"2024-04-25T08:33:09.864273+0000","last_clean":"2024-04-25T08:33:09.864273+0000","last_became_active":"2024-04-25T08:33:07.542936+0000","last_became_peered":"2024-04-25T08:33:07.542936+0000","last_unstale":"2024-04-25T08:33:09.864273+0000","last_undegraded":"2024-04-25T08:33:09.864273+0000","last_fullsized":"2024-04-25T08:33:09.864273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:56.406524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864361+0000","last_change":"2024-04-25T08:33:08.548472+0000","last_active":"2024-04-25T08:33:09.864361+0000","last_peered":"2024-04-25T08:33:09.864361+0000","last_clean":"2024-04-25T08:33:09.864361+0000","last_became_active":"2024-04-25T08:33:08.548188+0000","last_became_peered":"2024-04-25T08:33:08.548188+0000","last_unstale":"2024-04-25T08:33:09.864361+0000","last_undegraded":"2024-04-25T08:33:09.864361+0000","last_fullsized":"2024-04-25T08:33:09.864361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.057423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550179+0000","last_change":"2024-04-25T08:33:07.544691+0000","last_active":"2024-04-25T08:33:09.550179+0000","last_peered":"2024-04-25T08:33:09.550179+0000","last_clean":"2024-04-25T08:33:09.550179+0000","last_became_active":"2024-04-25T08:33:07.544319+0000","last_became_peered":"2024-04-25T08:33:07.544319+0000","last_unstale":"2024-04-25T08:33:09.550179+0000","last_undegraded":"2024-04-25T08:33:09.550179+0000","last_fullsized":"2024-04-25T08:33:09.550179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:32.068805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547137+0000","last_change":"2024-04-25T08:33:08.552435+0000","last_active":"2024-04-25T08:33:09.547137+0000","last_peered":"2024-04-25T08:33:09.547137+0000","last_clean":"2024-04-25T08:33:09.547137+0000","last_became_active":"2024-04-25T08:33:08.552015+0000","last_became_peered":"2024-04-25T08:33:08.552015+0000","last_unstale":"2024-04-25T08:33:09.547137+0000","last_undegraded":"2024-04-25T08:33:09.547137+0000","last_fullsized":"2024-04-25T08:33:09.547137+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:57.975994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865577+0000","last_change":"2024-04-25T08:33:07.548394+0000","last_active":"2024-04-25T08:33:09.865577+0000","last_peered":"2024-04-25T08:33:09.865577+0000","last_clean":"2024-04-25T08:33:09.865577+0000","last_became_active":"2024-04-25T08:33:07.548134+0000","last_became_peered":"2024-04-25T08:33:07.548134+0000","last_unstale":"2024-04-25T08:33:09.865577+0000","last_undegraded":"2024-04-25T08:33:09.865577+0000","last_fullsized":"2024-04-25T08:33:09.865577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:56.970394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864508+0000","last_change":"2024-04-25T08:33:08.545734+0000","last_active":"2024-04-25T08:33:09.864508+0000","last_peered":"2024-04-25T08:33:09.864508+0000","last_clean":"2024-04-25T08:33:09.864508+0000","last_became_active":"2024-04-25T08:33:08.545249+0000","last_became_peered":"2024-04-25T08:33:08.545249+0000","last_unstale":"2024-04-25T08:33:09.864508+0000","last_undegraded":"2024-04-25T08:33:09.864508+0000","last_fullsized":"2024-04-25T08:33:09.864508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:04.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551468+0000","last_change":"2024-04-25T08:33:07.546575+0000","last_active":"2024-04-25T08:33:09.551468+0000","last_peered":"2024-04-25T08:33:09.551468+0000","last_clean":"2024-04-25T08:33:09.551468+0000","last_became_active":"2024-04-25T08:33:07.546301+0000","last_became_peered":"2024-04-25T08:33:07.546301+0000","last_unstale":"2024-04-25T08:33:09.551468+0000","last_undegraded":"2024-04-25T08:33:09.551468+0000","last_fullsized":"2024-04-25T08:33:09.551468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:34.997631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551170+0000","last_change":"2024-04-25T08:33:08.542944+0000","last_active":"2024-04-25T08:33:09.551170+0000","last_peered":"2024-04-25T08:33:09.551170+0000","last_clean":"2024-04-25T08:33:09.551170+0000","last_became_active":"2024-04-25T08:33:08.542671+0000","last_became_peered":"2024-04-25T08:33:08.542671+0000","last_unstale":"2024-04-25T08:33:09.551170+0000","last_undegraded":"2024-04-25T08:33:09.551170+0000","last_fullsized":"2024-04-25T08:33:09.551170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:36.862604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864871+0000","last_change":"2024-04-25T08:33:07.548502+0000","last_active":"2024-04-25T08:33:09.864871+0000","last_peered":"2024-04-25T08:33:09.864871+0000","last_clean":"2024-04-25T08:33:09.864871+0000","last_became_active":"2024-04-25T08:33:07.548248+0000","last_became_peered":"2024-04-25T08:33:07.548248+0000","last_unstale":"2024-04-25T08:33:09.864871+0000","last_undegraded":"2024-04-25T08:33:09.864871+0000","last_fullsized":"2024-04-25T08:33:09.864871+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:35.857004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551067+0000","last_change":"2024-04-25T08:33:08.546960+0000","last_active":"2024-04-25T08:33:09.551067+0000","last_peered":"2024-04-25T08:33:09.551067+0000","last_clean":"2024-04-25T08:33:09.551067+0000","last_became_active":"2024-04-25T08:33:08.546634+0000","last_became_peered":"2024-04-25T08:33:08.546634+0000","last_unstale":"2024-04-25T08:33:09.551067+0000","last_undegraded":"2024-04-25T08:33:09.551067+0000","last_fullsized":"2024-04-25T08:33:09.551067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:00.494779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547276+0000","last_change":"2024-04-25T08:33:07.541975+0000","last_active":"2024-04-25T08:33:09.547276+0000","last_peered":"2024-04-25T08:33:09.547276+0000","last_clean":"2024-04-25T08:33:09.547276+0000","last_became_active":"2024-04-25T08:33:07.541691+0000","last_became_peered":"2024-04-25T08:33:07.541691+0000","last_unstale":"2024-04-25T08:33:09.547276+0000","last_undegraded":"2024-04-25T08:33:09.547276+0000","last_fullsized":"2024-04-25T08:33:09.547276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:33.833345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550617+0000","last_change":"2024-04-25T08:33:08.549160+0000","last_active":"2024-04-25T08:33:09.550617+0000","last_peered":"2024-04-25T08:33:09.550617+0000","last_clean":"2024-04-25T08:33:09.550617+0000","last_became_active":"2024-04-25T08:33:08.548894+0000","last_became_peered":"2024-04-25T08:33:08.548894+0000","last_unstale":"2024-04-25T08:33:09.550617+0000","last_undegraded":"2024-04-25T08:33:09.550617+0000","last_fullsized":"2024-04-25T08:33:09.550617+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:05.027592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864636+0000","last_change":"2024-04-25T08:33:07.543683+0000","last_active":"2024-04-25T08:33:09.864636+0000","last_peered":"2024-04-25T08:33:09.864636+0000","last_clean":"2024-04-25T08:33:09.864636+0000","last_became_active":"2024-04-25T08:33:07.543154+0000","last_became_peered":"2024-04-25T08:33:07.543154+0000","last_unstale":"2024-04-25T08:33:09.864636+0000","last_undegraded":"2024-04-25T08:33:09.864636+0000","last_fullsized":"2024-04-25T08:33:09.864636+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:52.394108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864433+0000","last_change":"2024-04-25T08:33:08.552996+0000","last_active":"2024-04-25T08:33:09.864433+0000","last_peered":"2024-04-25T08:33:09.864433+0000","last_clean":"2024-04-25T08:33:09.864433+0000","last_became_active":"2024-04-25T08:33:08.550905+0000","last_became_peered":"2024-04-25T08:33:08.550905+0000","last_unstale":"2024-04-25T08:33:09.864433+0000","last_undegraded":"2024-04-25T08:33:09.864433+0000","last_fullsized":"2024-04-25T08:33:09.864433+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:19.097337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864817+0000","last_change":"2024-04-25T08:33:07.549661+0000","last_active":"2024-04-25T08:33:09.864817+0000","last_peered":"2024-04-25T08:33:09.864817+0000","last_clean":"2024-04-25T08:33:09.864817+0000","last_became_active":"2024-04-25T08:33:07.549387+0000","last_became_peered":"2024-04-25T08:33:07.549387+0000","last_unstale":"2024-04-25T08:33:09.864817+0000","last_undegraded":"2024-04-25T08:33:09.864817+0000","last_fullsized":"2024-04-25T08:33:09.864817+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:28.534452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550407+0000","last_change":"2024-04-25T08:33:08.546035+0000","last_active":"2024-04-25T08:33:09.550407+0000","last_peered":"2024-04-25T08:33:09.550407+0000","last_clean":"2024-04-25T08:33:09.550407+0000","last_became_active":"2024-04-25T08:33:08.545678+0000","last_became_peered":"2024-04-25T08:33:08.545678+0000","last_unstale":"2024-04-25T08:33:09.550407+0000","last_undegraded":"2024-04-25T08:33:09.550407+0000","last_fullsized":"2024-04-25T08:33:09.550407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:26.016740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865465+0000","last_change":"2024-04-25T08:33:07.546458+0000","last_active":"2024-04-25T08:33:09.865465+0000","last_peered":"2024-04-25T08:33:09.865465+0000","last_clean":"2024-04-25T08:33:09.865465+0000","last_became_active":"2024-04-25T08:33:07.546172+0000","last_became_peered":"2024-04-25T08:33:07.546172+0000","last_unstale":"2024-04-25T08:33:09.865465+0000","last_undegraded":"2024-04-25T08:33:09.865465+0000","last_fullsized":"2024-04-25T08:33:09.865465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:07.376559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547258+0000","last_change":"2024-04-25T08:33:08.562273+0000","last_active":"2024-04-25T08:33:09.547258+0000","last_peered":"2024-04-25T08:33:09.547258+0000","last_clean":"2024-04-25T08:33:09.547258+0000","last_became_active":"2024-04-25T08:33:08.561973+0000","last_became_peered":"2024-04-25T08:33:08.561973+0000","last_unstale":"2024-04-25T08:33:09.547258+0000","last_undegraded":"2024-04-25T08:33:09.547258+0000","last_fullsized":"2024-04-25T08:33:09.547258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:54.498011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550577+0000","last_change":"2024-04-25T08:33:07.543855+0000","last_active":"2024-04-25T08:33:09.550577+0000","last_peered":"2024-04-25T08:33:09.550577+0000","last_clean":"2024-04-25T08:33:09.550577+0000","last_became_active":"2024-04-25T08:33:07.543538+0000","last_became_peered":"2024-04-25T08:33:07.543538+0000","last_unstale":"2024-04-25T08:33:09.550577+0000","last_undegraded":"2024-04-25T08:33:09.550577+0000","last_fullsized":"2024-04-25T08:33:09.550577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:15.576960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865032+0000","last_change":"2024-04-25T08:33:08.550553+0000","last_active":"2024-04-25T08:33:09.865032+0000","last_peered":"2024-04-25T08:33:09.865032+0000","last_clean":"2024-04-25T08:33:09.865032+0000","last_became_active":"2024-04-25T08:33:08.550268+0000","last_became_peered":"2024-04-25T08:33:08.550268+0000","last_unstale":"2024-04-25T08:33:09.865032+0000","last_undegraded":"2024-04-25T08:33:09.865032+0000","last_fullsized":"2024-04-25T08:33:09.865032+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:30.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865511+0000","last_change":"2024-04-25T08:33:07.549635+0000","last_active":"2024-04-25T08:33:09.865511+0000","last_peered":"2024-04-25T08:33:09.865511+0000","last_clean":"2024-04-25T08:33:09.865511+0000","last_became_active":"2024-04-25T08:33:07.549355+0000","last_became_peered":"2024-04-25T08:33:07.549355+0000","last_unstale":"2024-04-25T08:33:09.865511+0000","last_undegraded":"2024-04-25T08:33:09.865511+0000","last_fullsized":"2024-04-25T08:33:09.865511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:52.394108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551017+0000","last_change":"2024-04-25T08:33:08.542217+0000","last_active":"2024-04-25T08:33:09.551017+0000","last_peered":"2024-04-25T08:33:09.551017+0000","last_clean":"2024-04-25T08:33:09.551017+0000","last_became_active":"2024-04-25T08:33:08.541905+0000","last_became_peered":"2024-04-25T08:33:08.541905+0000","last_unstale":"2024-04-25T08:33:09.551017+0000","last_undegraded":"2024-04-25T08:33:09.551017+0000","last_fullsized":"2024-04-25T08:33:09.551017+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:57.975994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865246+0000","last_change":"2024-04-25T08:33:07.545683+0000","last_active":"2024-04-25T08:33:09.865246+0000","last_peered":"2024-04-25T08:33:09.865246+0000","last_clean":"2024-04-25T08:33:09.865246+0000","last_became_active":"2024-04-25T08:33:07.545403+0000","last_became_peered":"2024-04-25T08:33:07.545403+0000","last_unstale":"2024-04-25T08:33:09.865246+0000","last_undegraded":"2024-04-25T08:33:09.865246+0000","last_fullsized":"2024-04-25T08:33:09.865246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:44.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551372+0000","last_change":"2024-04-25T08:33:08.536670+0000","last_active":"2024-04-25T08:33:09.551372+0000","last_peered":"2024-04-25T08:33:09.551372+0000","last_clean":"2024-04-25T08:33:09.551372+0000","last_became_active":"2024-04-25T08:33:08.536399+0000","last_became_peered":"2024-04-25T08:33:08.536399+0000","last_unstale":"2024-04-25T08:33:09.551372+0000","last_undegraded":"2024-04-25T08:33:09.551372+0000","last_fullsized":"2024-04-25T08:33:09.551372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:34.838945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865380+0000","last_change":"2024-04-25T08:33:07.549848+0000","last_active":"2024-04-25T08:33:09.865380+0000","last_peered":"2024-04-25T08:33:09.865380+0000","last_clean":"2024-04-25T08:33:09.865380+0000","last_became_active":"2024-04-25T08:33:07.549547+0000","last_became_peered":"2024-04-25T08:33:07.549547+0000","last_unstale":"2024-04-25T08:33:09.865380+0000","last_undegraded":"2024-04-25T08:33:09.865380+0000","last_fullsized":"2024-04-25T08:33:09.865380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:53.492411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551221+0000","last_change":"2024-04-25T08:33:08.548240+0000","last_active":"2024-04-25T08:33:09.551221+0000","last_peered":"2024-04-25T08:33:09.551221+0000","last_clean":"2024-04-25T08:33:09.551221+0000","last_became_active":"2024-04-25T08:33:08.547974+0000","last_became_peered":"2024-04-25T08:33:08.547974+0000","last_unstale":"2024-04-25T08:33:09.551221+0000","last_undegraded":"2024-04-25T08:33:09.551221+0000","last_fullsized":"2024-04-25T08:33:09.551221+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:54.498011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864526+0000","last_change":"2024-04-25T08:33:07.539818+0000","last_active":"2024-04-25T08:33:09.864526+0000","last_peered":"2024-04-25T08:33:09.864526+0000","last_clean":"2024-04-25T08:33:09.864526+0000","last_became_active":"2024-04-25T08:33:07.539393+0000","last_became_peered":"2024-04-25T08:33:07.539393+0000","last_unstale":"2024-04-25T08:33:09.864526+0000","last_undegraded":"2024-04-25T08:33:09.864526+0000","last_fullsized":"2024-04-25T08:33:09.864526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:29.040181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865232+0000","last_change":"2024-04-25T08:33:08.550778+0000","last_active":"2024-04-25T08:33:09.865232+0000","last_peered":"2024-04-25T08:33:09.865232+0000","last_clean":"2024-04-25T08:33:09.865232+0000","last_became_active":"2024-04-25T08:33:08.549866+0000","last_became_peered":"2024-04-25T08:33:08.549866+0000","last_unstale":"2024-04-25T08:33:09.865232+0000","last_undegraded":"2024-04-25T08:33:09.865232+0000","last_fullsized":"2024-04-25T08:33:09.865232+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864600+0000","last_change":"2024-04-25T08:33:07.544798+0000","last_active":"2024-04-25T08:33:09.864600+0000","last_peered":"2024-04-25T08:33:09.864600+0000","last_clean":"2024-04-25T08:33:09.864600+0000","last_became_active":"2024-04-25T08:33:07.544391+0000","last_became_peered":"2024-04-25T08:33:07.544391+0000","last_unstale":"2024-04-25T08:33:09.864600+0000","last_undegraded":"2024-04-25T08:33:09.864600+0000","last_fullsized":"2024-04-25T08:33:09.864600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:35.857004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864528+0000","last_change":"2024-04-25T08:33:08.546653+0000","last_active":"2024-04-25T08:33:09.864528+0000","last_peered":"2024-04-25T08:33:09.864528+0000","last_clean":"2024-04-25T08:33:09.864528+0000","last_became_active":"2024-04-25T08:33:08.546363+0000","last_became_peered":"2024-04-25T08:33:08.546363+0000","last_unstale":"2024-04-25T08:33:09.864528+0000","last_undegraded":"2024-04-25T08:33:09.864528+0000","last_fullsized":"2024-04-25T08:33:09.864528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864913+0000","last_change":"2024-04-25T08:33:07.547407+0000","last_active":"2024-04-25T08:33:09.864913+0000","last_peered":"2024-04-25T08:33:09.864913+0000","last_clean":"2024-04-25T08:33:09.864913+0000","last_became_active":"2024-04-25T08:33:07.547116+0000","last_became_peered":"2024-04-25T08:33:07.547116+0000","last_unstale":"2024-04-25T08:33:09.864913+0000","last_undegraded":"2024-04-25T08:33:09.864913+0000","last_fullsized":"2024-04-25T08:33:09.864913+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:44.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550429+0000","last_change":"2024-04-25T08:33:08.550448+0000","last_active":"2024-04-25T08:33:09.550429+0000","last_peered":"2024-04-25T08:33:09.550429+0000","last_clean":"2024-04-25T08:33:09.550429+0000","last_became_active":"2024-04-25T08:33:08.550107+0000","last_became_peered":"2024-04-25T08:33:08.550107+0000","last_unstale":"2024-04-25T08:33:09.550429+0000","last_undegraded":"2024-04-25T08:33:09.550429+0000","last_fullsized":"2024-04-25T08:33:09.550429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:28.833570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865115+0000","last_change":"2024-04-25T08:33:07.542368+0000","last_active":"2024-04-25T08:33:09.865115+0000","last_peered":"2024-04-25T08:33:09.865115+0000","last_clean":"2024-04-25T08:33:09.865115+0000","last_became_active":"2024-04-25T08:33:07.542110+0000","last_became_peered":"2024-04-25T08:33:07.542110+0000","last_unstale":"2024-04-25T08:33:09.865115+0000","last_undegraded":"2024-04-25T08:33:09.865115+0000","last_fullsized":"2024-04-25T08:33:09.865115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:22.449454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864761+0000","last_change":"2024-04-25T08:33:08.542650+0000","last_active":"2024-04-25T08:33:09.864761+0000","last_peered":"2024-04-25T08:33:09.864761+0000","last_clean":"2024-04-25T08:33:09.864761+0000","last_became_active":"2024-04-25T08:33:08.542383+0000","last_became_peered":"2024-04-25T08:33:08.542383+0000","last_unstale":"2024-04-25T08:33:09.864761+0000","last_undegraded":"2024-04-25T08:33:09.864761+0000","last_fullsized":"2024-04-25T08:33:09.864761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865644+0000","last_change":"2024-04-25T08:33:07.546860+0000","last_active":"2024-04-25T08:33:09.865644+0000","last_peered":"2024-04-25T08:33:09.865644+0000","last_clean":"2024-04-25T08:33:09.865644+0000","last_became_active":"2024-04-25T08:33:07.546585+0000","last_became_peered":"2024-04-25T08:33:07.546585+0000","last_unstale":"2024-04-25T08:33:09.865644+0000","last_undegraded":"2024-04-25T08:33:09.865644+0000","last_fullsized":"2024-04-25T08:33:09.865644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:07.376559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865515+0000","last_change":"2024-04-25T08:33:08.557006+0000","last_active":"2024-04-25T08:33:09.865515+0000","last_peered":"2024-04-25T08:33:09.865515+0000","last_clean":"2024-04-25T08:33:09.865515+0000","last_became_active":"2024-04-25T08:33:08.556157+0000","last_became_peered":"2024-04-25T08:33:08.556157+0000","last_unstale":"2024-04-25T08:33:09.865515+0000","last_undegraded":"2024-04-25T08:33:09.865515+0000","last_fullsized":"2024-04-25T08:33:09.865515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:03.656381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865569+0000","last_change":"2024-04-25T08:33:07.546753+0000","last_active":"2024-04-25T08:33:09.865569+0000","last_peered":"2024-04-25T08:33:09.865569+0000","last_clean":"2024-04-25T08:33:09.865569+0000","last_became_active":"2024-04-25T08:33:07.546472+0000","last_became_peered":"2024-04-25T08:33:07.546472+0000","last_unstale":"2024-04-25T08:33:09.865569+0000","last_undegraded":"2024-04-25T08:33:09.865569+0000","last_fullsized":"2024-04-25T08:33:09.865569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:56.970394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865439+0000","last_change":"2024-04-25T08:33:08.552873+0000","last_active":"2024-04-25T08:33:09.865439+0000","last_peered":"2024-04-25T08:33:09.865439+0000","last_clean":"2024-04-25T08:33:09.865439+0000","last_became_active":"2024-04-25T08:33:08.552484+0000","last_became_peered":"2024-04-25T08:33:08.552484+0000","last_unstale":"2024-04-25T08:33:09.865439+0000","last_undegraded":"2024-04-25T08:33:09.865439+0000","last_fullsized":"2024-04-25T08:33:09.865439+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.805711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865150+0000","last_change":"2024-04-25T08:33:07.541388+0000","last_active":"2024-04-25T08:33:09.865150+0000","last_peered":"2024-04-25T08:33:09.865150+0000","last_clean":"2024-04-25T08:33:09.865150+0000","last_became_active":"2024-04-25T08:33:07.541073+0000","last_became_peered":"2024-04-25T08:33:07.541073+0000","last_unstale":"2024-04-25T08:33:09.865150+0000","last_undegraded":"2024-04-25T08:33:09.865150+0000","last_fullsized":"2024-04-25T08:33:09.865150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:12.474553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551237+0000","last_change":"2024-04-25T08:33:08.547913+0000","last_active":"2024-04-25T08:33:09.551237+0000","last_peered":"2024-04-25T08:33:09.551237+0000","last_clean":"2024-04-25T08:33:09.551237+0000","last_became_active":"2024-04-25T08:33:08.547630+0000","last_became_peered":"2024-04-25T08:33:08.547630+0000","last_unstale":"2024-04-25T08:33:09.551237+0000","last_undegraded":"2024-04-25T08:33:09.551237+0000","last_fullsized":"2024-04-25T08:33:09.551237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:45.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547275+0000","last_change":"2024-04-25T08:33:07.543180+0000","last_active":"2024-04-25T08:33:09.547275+0000","last_peered":"2024-04-25T08:33:09.547275+0000","last_clean":"2024-04-25T08:33:09.547275+0000","last_became_active":"2024-04-25T08:33:07.541512+0000","last_became_peered":"2024-04-25T08:33:07.541512+0000","last_unstale":"2024-04-25T08:33:09.547275+0000","last_undegraded":"2024-04-25T08:33:09.547275+0000","last_fullsized":"2024-04-25T08:33:09.547275+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:08.820438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550209+0000","last_change":"2024-04-25T08:33:08.545104+0000","last_active":"2024-04-25T08:33:09.550209+0000","last_peered":"2024-04-25T08:33:09.550209+0000","last_clean":"2024-04-25T08:33:09.550209+0000","last_became_active":"2024-04-25T08:33:08.544837+0000","last_became_peered":"2024-04-25T08:33:08.544837+0000","last_unstale":"2024-04-25T08:33:09.550209+0000","last_undegraded":"2024-04-25T08:33:09.550209+0000","last_fullsized":"2024-04-25T08:33:09.550209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:32.462998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864746+0000","last_change":"2024-04-25T08:33:07.544710+0000","last_active":"2024-04-25T08:33:09.864746+0000","last_peered":"2024-04-25T08:33:09.864746+0000","last_clean":"2024-04-25T08:33:09.864746+0000","last_became_active":"2024-04-25T08:33:07.544440+0000","last_became_peered":"2024-04-25T08:33:07.544440+0000","last_unstale":"2024-04-25T08:33:09.864746+0000","last_undegraded":"2024-04-25T08:33:09.864746+0000","last_fullsized":"2024-04-25T08:33:09.864746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:28.534452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551714+0000","last_change":"2024-04-25T08:33:08.552778+0000","last_active":"2024-04-25T08:33:09.551714+0000","last_peered":"2024-04-25T08:33:09.551714+0000","last_clean":"2024-04-25T08:33:09.551714+0000","last_became_active":"2024-04-25T08:33:08.551945+0000","last_became_peered":"2024-04-25T08:33:08.551945+0000","last_unstale":"2024-04-25T08:33:09.551714+0000","last_undegraded":"2024-04-25T08:33:09.551714+0000","last_fullsized":"2024-04-25T08:33:09.551714+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:03.009613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864544+0000","last_change":"2024-04-25T08:33:07.546678+0000","last_active":"2024-04-25T08:33:09.864544+0000","last_peered":"2024-04-25T08:33:09.864544+0000","last_clean":"2024-04-25T08:33:09.864544+0000","last_became_active":"2024-04-25T08:33:07.546160+0000","last_became_peered":"2024-04-25T08:33:07.546160+0000","last_unstale":"2024-04-25T08:33:09.864544+0000","last_undegraded":"2024-04-25T08:33:09.864544+0000","last_fullsized":"2024-04-25T08:33:09.864544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:18.315174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547010+0000","last_change":"2024-04-25T08:33:08.562844+0000","last_active":"2024-04-25T08:33:09.547010+0000","last_peered":"2024-04-25T08:33:09.547010+0000","last_clean":"2024-04-25T08:33:09.547010+0000","last_became_active":"2024-04-25T08:33:08.562537+0000","last_became_peered":"2024-04-25T08:33:08.562537+0000","last_unstale":"2024-04-25T08:33:09.547010+0000","last_undegraded":"2024-04-25T08:33:09.547010+0000","last_fullsized":"2024-04-25T08:33:09.547010+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.382160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864383+0000","last_change":"2024-04-25T08:33:07.543755+0000","last_active":"2024-04-25T08:33:09.864383+0000","last_peered":"2024-04-25T08:33:09.864383+0000","last_clean":"2024-04-25T08:33:09.864383+0000","last_became_active":"2024-04-25T08:33:07.543397+0000","last_became_peered":"2024-04-25T08:33:07.543397+0000","last_unstale":"2024-04-25T08:33:09.864383+0000","last_undegraded":"2024-04-25T08:33:09.864383+0000","last_fullsized":"2024-04-25T08:33:09.864383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:04.471094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865344+0000","last_change":"2024-04-25T08:33:08.540802+0000","last_active":"2024-04-25T08:33:09.865344+0000","last_peered":"2024-04-25T08:33:09.865344+0000","last_clean":"2024-04-25T08:33:09.865344+0000","last_became_active":"2024-04-25T08:33:08.540500+0000","last_became_peered":"2024-04-25T08:33:08.540500+0000","last_unstale":"2024-04-25T08:33:09.865344+0000","last_undegraded":"2024-04-25T08:33:09.865344+0000","last_fullsized":"2024-04-25T08:33:09.865344+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.382073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.546804+0000","last_change":"2024-04-25T08:33:07.539992+0000","last_active":"2024-04-25T08:33:09.546804+0000","last_peered":"2024-04-25T08:33:09.546804+0000","last_clean":"2024-04-25T08:33:09.546804+0000","last_became_active":"2024-04-25T08:33:07.539652+0000","last_became_peered":"2024-04-25T08:33:07.539652+0000","last_unstale":"2024-04-25T08:33:09.546804+0000","last_undegraded":"2024-04-25T08:33:09.546804+0000","last_fullsized":"2024-04-25T08:33:09.546804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:59.489179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550445+0000","last_change":"2024-04-25T08:33:08.556216+0000","last_active":"2024-04-25T08:33:09.550445+0000","last_peered":"2024-04-25T08:33:09.550445+0000","last_clean":"2024-04-25T08:33:09.550445+0000","last_became_active":"2024-04-25T08:33:08.555940+0000","last_became_peered":"2024-04-25T08:33:08.555940+0000","last_unstale":"2024-04-25T08:33:09.550445+0000","last_undegraded":"2024-04-25T08:33:09.550445+0000","last_fullsized":"2024-04-25T08:33:09.550445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:29.540053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864133+0000","last_change":"2024-04-25T08:33:07.540989+0000","last_active":"2024-04-25T08:33:09.864133+0000","last_peered":"2024-04-25T08:33:09.864133+0000","last_clean":"2024-04-25T08:33:09.864133+0000","last_became_active":"2024-04-25T08:33:07.540635+0000","last_became_peered":"2024-04-25T08:33:07.540635+0000","last_unstale":"2024-04-25T08:33:09.864133+0000","last_undegraded":"2024-04-25T08:33:09.864133+0000","last_fullsized":"2024-04-25T08:33:09.864133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:25.457017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865217+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:09.865217+0000","last_peered":"2024-04-25T08:33:09.865217+0000","last_clean":"2024-04-25T08:33:09.865217+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:09.865217+0000","last_undegraded":"2024-04-25T08:33:09.865217+0000","last_fullsized":"2024-04-25T08:33:09.865217+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":1417216,"data_stored":957396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":811008,"data_stored":366286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":811008,"data_stored":366274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":819200,"data_stored":367864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40292,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331548,"statfs":{"total":96636764160,"available":96595505152,"internally_reserved":0,"allocated":806912,"data_stored":366252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":806912,"data_stored":366252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":835584,"data_stored":368628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45028,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326812,"statfs":{"total":96636764160,"available":96590655488,"internally_reserved":0,"allocated":1400832,"data_stored":956620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:33:12.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:33:12.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 Processor -- start 2024-04-25T08:33:12.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 -- start start 2024-04-25T08:33:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ea815ba00 0x7f2ea809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ea81602e0 0x7f2ea80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80cd710 con 0x7f2ea815f5f0 2024-04-25T08:33:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea8099b70 con 0x7f2ea81602e0 2024-04-25T08:33:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.236+0000 7f2eae589640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea8099e50 con 0x7f2ea815ba00 2024-04-25T08:33:12.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ea81602e0 0x7f2ea80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2ea81602e0 0x7f2ea80ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56416/0 (socket says 172.21.15.60:56416) 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 -- 172.21.15.60:0/2890871167 learned_addr learned my addr 172.21.15.60:0/2890871167 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea7fff640 1 --2- 172.21.15.60:0/2890871167 >> v2:172.21.15.100:3300/0 conn(0x7f2ea815ba00 0x7f2ea809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 -- 172.21.15.60:0/2890871167 >> v2:172.21.15.100:3300/0 conn(0x7f2ea815ba00 msgr2=0x7f2ea809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 --2- 172.21.15.60:0/2890871167 >> v2:172.21.15.100:3300/0 conn(0x7f2ea815ba00 0x7f2ea809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 -- 172.21.15.60:0/2890871167 >> v1:172.21.15.100:6789/0 conn(0x7f2ea815f5f0 legacy=0x7f2ea80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.237+0000 7f2ea77fe640 1 -- 172.21.15.60:0/2890871167 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea809a130 con 0x7f2ea81602e0 2024-04-25T08:33:12.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2ea77fe640 1 --2- 172.21.15.60:0/2890871167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea80ccda0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b4370 tx=0x7f2e9c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814ec1b82de7775e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2ea6ffd640 1 -- 172.21.15.60:0/2890871167 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c037070 con 0x7f2ea81602e0 2024-04-25T08:33:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2ea6ffd640 1 -- 172.21.15.60:0/2890871167 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c02ae20 con 0x7f2ea81602e0 2024-04-25T08:33:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2ea6ffd640 1 -- 172.21.15.60:0/2890871167 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c032660 con 0x7f2ea81602e0 2024-04-25T08:33:12.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 -- 172.21.15.60:0/2890871167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 msgr2=0x7f2ea80ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 --2- 172.21.15.60:0/2890871167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea80ccda0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b4370 tx=0x7f2e9c02a110 comp rx=0 tx=0).stop 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 -- 172.21.15.60:0/2890871167 shutdown_connections 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 --2- 172.21.15.60:0/2890871167 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea80ccda0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 --2- 172.21.15.60:0/2890871167 >> v2:172.21.15.100:3300/0 conn(0x7f2ea815ba00 0x7f2ea809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 -- 172.21.15.60:0/2890871167 >> 172.21.15.60:0/2890871167 conn(0x7f2ea80b3270 msgr2=0x7f2ea80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 -- 172.21.15.60:0/2890871167 shutdown_connections 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.238+0000 7f2eae589640 1 -- 172.21.15.60:0/2890871167 wait complete. 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.239+0000 7f2eae589640 1 Processor -- start 2024-04-25T08:33:12.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 -- start start 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea815ba00 0x7f2ea815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea815f5f0 0x7f2ea8156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea806ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea815bde0 con 0x7f2ea81602e0 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cd2e0 con 0x7f2ea815f5f0 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eae589640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80cb1a0 con 0x7f2ea815ba00 2024-04-25T08:33:12.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.240+0000 7f2eacb26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea806ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea806ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56418/0 (socket says 172.21.15.60:56418) 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 -- 172.21.15.60:0/544834141 learned_addr learned my addr 172.21.15.60:0/544834141 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea77fe640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea815f5f0 0x7f2ea8156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea7fff640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea815ba00 0x7f2ea815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 -- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea815ba00 msgr2=0x7f2ea815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea815ba00 0x7f2ea815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 -- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea815f5f0 msgr2=0x7f2ea8156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea815f5f0 0x7f2ea8156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 -- 172.21.15.60:0/544834141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80cb480 con 0x7f2ea81602e0 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea77fe640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea815f5f0 0x7f2ea8156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea7fff640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea815ba00 0x7f2ea815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eacb26640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea806ce20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000b970 tx=0x7f2ea000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0019070 con 0x7f2ea81602e0 2024-04-25T08:33:12.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea000cb40 con 0x7f2ea81602e0 2024-04-25T08:33:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea80cb760 con 0x7f2ea81602e0 2024-04-25T08:33:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0014440 con 0x7f2ea81602e0 2024-04-25T08:33:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.241+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea80cadc0 con 0x7f2ea81602e0 2024-04-25T08:33:12.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.242+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea8073990 con 0x7f2ea81602e0 2024-04-25T08:33:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.243+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ea001c070 con 0x7f2ea81602e0 2024-04-25T08:33:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.243+0000 7f2ea4ff9640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e9406b4d0 0x7f2e9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.243+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ea008a120 con 0x7f2ea81602e0 2024-04-25T08:33:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.244+0000 7f2ea7fff640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e9406b4d0 0x7f2e9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.244+0000 7f2ea7fff640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e9406b4d0 0x7f2e9406d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006fd0 tx=0x7f2e98008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.248+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ea0058d90 con 0x7f2ea81602e0 2024-04-25T08:33:12.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.425+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ea80b40a0 con 0x7f2e9406b4d0 2024-04-25T08:33:12.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.432+0000 7f2ea4ff9640 1 -- 172.21.15.60:0/544834141 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252019 (secure 0 0 0) 0x7f2ea80b40a0 con 0x7f2e9406b4d0 2024-04-25T08:33:12.429 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:12.431 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:33:12.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e9406b4d0 msgr2=0x7f2e9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e9406b4d0 0x7f2e9406d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006fd0 tx=0x7f2e98008040 comp rx=0 tx=0).stop 2024-04-25T08:33:12.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 msgr2=0x7f2ea806ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea806ce20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000b970 tx=0x7f2ea000be30 comp rx=0 tx=0).stop 2024-04-25T08:33:12.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 shutdown_connections 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ea81602e0 0x7f2ea806ce20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2e9406b4d0 0x7f2e9406d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ea815f5f0 0x7f2ea8156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 --2- 172.21.15.60:0/544834141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ea815ba00 0x7f2ea815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 >> 172.21.15.60:0/544834141 conn(0x7f2ea80b3270 msgr2=0x7f2ea80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 shutdown_connections 2024-04-25T08:33:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.435+0000 7f2eae589640 1 -- 172.21.15.60:0/544834141 wait complete. 2024-04-25T08:33:12.441 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-25T08:33:11.450278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306912,"kb_used_data":7528,"kb_used_omap":12,"kb_used_meta":299251,"kb_avail":754667808,"statfs":{"total":773094113280,"available":772779835392,"internally_reserved":0,"allocated":7708672,"data_stored":4115572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":306433584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.926550"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547374+0000","last_change":"2024-04-25T08:33:08.560273+0000","last_active":"2024-04-25T08:33:09.547374+0000","last_peered":"2024-04-25T08:33:09.547374+0000","last_clean":"2024-04-25T08:33:09.547374+0000","last_became_active":"2024-04-25T08:33:08.560000+0000","last_became_peered":"2024-04-25T08:33:08.560000+0000","last_unstale":"2024-04-25T08:33:09.547374+0000","last_undegraded":"2024-04-25T08:33:09.547374+0000","last_fullsized":"2024-04-25T08:33:09.547374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:45.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547181+0000","last_change":"2024-04-25T08:33:08.556370+0000","last_active":"2024-04-25T08:33:09.547181+0000","last_peered":"2024-04-25T08:33:09.547181+0000","last_clean":"2024-04-25T08:33:09.547181+0000","last_became_active":"2024-04-25T08:33:08.556061+0000","last_became_peered":"2024-04-25T08:33:08.556061+0000","last_unstale":"2024-04-25T08:33:09.547181+0000","last_undegraded":"2024-04-25T08:33:09.547181+0000","last_fullsized":"2024-04-25T08:33:09.547181+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:36.862604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864695+0000","last_change":"2024-04-25T08:33:08.553695+0000","last_active":"2024-04-25T08:33:09.864695+0000","last_peered":"2024-04-25T08:33:09.864695+0000","last_clean":"2024-04-25T08:33:09.864695+0000","last_became_active":"2024-04-25T08:33:08.553440+0000","last_became_peered":"2024-04-25T08:33:08.553440+0000","last_unstale":"2024-04-25T08:33:09.864695+0000","last_undegraded":"2024-04-25T08:33:09.864695+0000","last_fullsized":"2024-04-25T08:33:09.864695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551688+0000","last_change":"2024-04-25T08:33:08.554151+0000","last_active":"2024-04-25T08:33:09.551688+0000","last_peered":"2024-04-25T08:33:09.551688+0000","last_clean":"2024-04-25T08:33:09.551688+0000","last_became_active":"2024-04-25T08:33:08.553866+0000","last_became_peered":"2024-04-25T08:33:08.553866+0000","last_unstale":"2024-04-25T08:33:09.551688+0000","last_undegraded":"2024-04-25T08:33:09.551688+0000","last_fullsized":"2024-04-25T08:33:09.551688+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:59.510900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550429+0000","last_change":"2024-04-25T08:33:08.548448+0000","last_active":"2024-04-25T08:33:09.550429+0000","last_peered":"2024-04-25T08:33:09.550429+0000","last_clean":"2024-04-25T08:33:09.550429+0000","last_became_active":"2024-04-25T08:33:08.547964+0000","last_became_peered":"2024-04-25T08:33:08.547964+0000","last_unstale":"2024-04-25T08:33:09.550429+0000","last_undegraded":"2024-04-25T08:33:09.550429+0000","last_fullsized":"2024-04-25T08:33:09.550429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:53.399708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864894+0000","last_change":"2024-04-25T08:33:08.550386+0000","last_active":"2024-04-25T08:33:09.864894+0000","last_peered":"2024-04-25T08:33:09.864894+0000","last_clean":"2024-04-25T08:33:09.864894+0000","last_became_active":"2024-04-25T08:33:08.550106+0000","last_became_peered":"2024-04-25T08:33:08.550106+0000","last_unstale":"2024-04-25T08:33:09.864894+0000","last_undegraded":"2024-04-25T08:33:09.864894+0000","last_fullsized":"2024-04-25T08:33:09.864894+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:58.250815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864646+0000","last_change":"2024-04-25T08:33:08.558798+0000","last_active":"2024-04-25T08:33:09.864646+0000","last_peered":"2024-04-25T08:33:09.864646+0000","last_clean":"2024-04-25T08:33:09.864646+0000","last_became_active":"2024-04-25T08:33:08.558484+0000","last_became_peered":"2024-04-25T08:33:08.558484+0000","last_unstale":"2024-04-25T08:33:09.864646+0000","last_undegraded":"2024-04-25T08:33:09.864646+0000","last_fullsized":"2024-04-25T08:33:09.864646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:34.431464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864737+0000","last_change":"2024-04-25T08:33:08.556616+0000","last_active":"2024-04-25T08:33:09.864737+0000","last_peered":"2024-04-25T08:33:09.864737+0000","last_clean":"2024-04-25T08:33:09.864737+0000","last_became_active":"2024-04-25T08:33:08.556366+0000","last_became_peered":"2024-04-25T08:33:08.556366+0000","last_unstale":"2024-04-25T08:33:09.864737+0000","last_undegraded":"2024-04-25T08:33:09.864737+0000","last_fullsized":"2024-04-25T08:33:09.864737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865294+0000","last_change":"2024-04-25T08:33:08.552548+0000","last_active":"2024-04-25T08:33:09.865294+0000","last_peered":"2024-04-25T08:33:09.865294+0000","last_clean":"2024-04-25T08:33:09.865294+0000","last_became_active":"2024-04-25T08:33:08.550323+0000","last_became_peered":"2024-04-25T08:33:08.550323+0000","last_unstale":"2024-04-25T08:33:09.865294+0000","last_undegraded":"2024-04-25T08:33:09.865294+0000","last_fullsized":"2024-04-25T08:33:09.865294+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:25.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865072+0000","last_change":"2024-04-25T08:33:08.558332+0000","last_active":"2024-04-25T08:33:09.865072+0000","last_peered":"2024-04-25T08:33:09.865072+0000","last_clean":"2024-04-25T08:33:09.865072+0000","last_became_active":"2024-04-25T08:33:08.558055+0000","last_became_peered":"2024-04-25T08:33:08.558055+0000","last_unstale":"2024-04-25T08:33:09.865072+0000","last_undegraded":"2024-04-25T08:33:09.865072+0000","last_fullsized":"2024-04-25T08:33:09.865072+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551733+0000","last_change":"2024-04-25T08:33:08.559023+0000","last_active":"2024-04-25T08:33:09.551733+0000","last_peered":"2024-04-25T08:33:09.551733+0000","last_clean":"2024-04-25T08:33:09.551733+0000","last_became_active":"2024-04-25T08:33:08.558776+0000","last_became_peered":"2024-04-25T08:33:08.558776+0000","last_unstale":"2024-04-25T08:33:09.551733+0000","last_undegraded":"2024-04-25T08:33:09.551733+0000","last_fullsized":"2024-04-25T08:33:09.551733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:11.754363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864746+0000","last_change":"2024-04-25T08:33:08.559719+0000","last_active":"2024-04-25T08:33:09.864746+0000","last_peered":"2024-04-25T08:33:09.864746+0000","last_clean":"2024-04-25T08:33:09.864746+0000","last_became_active":"2024-04-25T08:33:08.559436+0000","last_became_peered":"2024-04-25T08:33:08.559436+0000","last_unstale":"2024-04-25T08:33:09.864746+0000","last_undegraded":"2024-04-25T08:33:09.864746+0000","last_fullsized":"2024-04-25T08:33:09.864746+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.057423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864606+0000","last_change":"2024-04-25T08:33:08.548981+0000","last_active":"2024-04-25T08:33:09.864606+0000","last_peered":"2024-04-25T08:33:09.864606+0000","last_clean":"2024-04-25T08:33:09.864606+0000","last_became_active":"2024-04-25T08:33:08.548648+0000","last_became_peered":"2024-04-25T08:33:08.548648+0000","last_unstale":"2024-04-25T08:33:09.864606+0000","last_undegraded":"2024-04-25T08:33:09.864606+0000","last_fullsized":"2024-04-25T08:33:09.864606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:12.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865155+0000","last_change":"2024-04-25T08:33:08.551464+0000","last_active":"2024-04-25T08:33:09.865155+0000","last_peered":"2024-04-25T08:33:09.865155+0000","last_clean":"2024-04-25T08:33:09.865155+0000","last_became_active":"2024-04-25T08:33:08.551191+0000","last_became_peered":"2024-04-25T08:33:08.551191+0000","last_unstale":"2024-04-25T08:33:09.865155+0000","last_undegraded":"2024-04-25T08:33:09.865155+0000","last_fullsized":"2024-04-25T08:33:09.865155+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.057423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865521+0000","last_change":"2024-04-25T08:33:08.554896+0000","last_active":"2024-04-25T08:33:09.865521+0000","last_peered":"2024-04-25T08:33:09.865521+0000","last_clean":"2024-04-25T08:33:09.865521+0000","last_became_active":"2024-04-25T08:33:08.554601+0000","last_became_peered":"2024-04-25T08:33:08.554601+0000","last_unstale":"2024-04-25T08:33:09.865521+0000","last_undegraded":"2024-04-25T08:33:09.865521+0000","last_fullsized":"2024-04-25T08:33:09.865521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:04.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864926+0000","last_change":"2024-04-25T08:33:08.543627+0000","last_active":"2024-04-25T08:33:09.864926+0000","last_peered":"2024-04-25T08:33:09.864926+0000","last_clean":"2024-04-25T08:33:09.864926+0000","last_became_active":"2024-04-25T08:33:08.543363+0000","last_became_peered":"2024-04-25T08:33:08.543363+0000","last_unstale":"2024-04-25T08:33:09.864926+0000","last_undegraded":"2024-04-25T08:33:09.864926+0000","last_fullsized":"2024-04-25T08:33:09.864926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:46.998248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547399+0000","last_change":"2024-04-25T08:33:08.563503+0000","last_active":"2024-04-25T08:33:09.547399+0000","last_peered":"2024-04-25T08:33:09.547399+0000","last_clean":"2024-04-25T08:33:09.547399+0000","last_became_active":"2024-04-25T08:33:08.563198+0000","last_became_peered":"2024-04-25T08:33:08.563198+0000","last_unstale":"2024-04-25T08:33:09.547399+0000","last_undegraded":"2024-04-25T08:33:09.547399+0000","last_fullsized":"2024-04-25T08:33:09.547399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:13.480153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551611+0000","last_change":"2024-04-25T08:33:08.551702+0000","last_active":"2024-04-25T08:33:09.551611+0000","last_peered":"2024-04-25T08:33:09.551611+0000","last_clean":"2024-04-25T08:33:09.551611+0000","last_became_active":"2024-04-25T08:33:08.551430+0000","last_became_peered":"2024-04-25T08:33:08.551430+0000","last_unstale":"2024-04-25T08:33:09.551611+0000","last_undegraded":"2024-04-25T08:33:09.551611+0000","last_fullsized":"2024-04-25T08:33:09.551611+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:09.826039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550847+0000","last_change":"2024-04-25T08:33:08.554954+0000","last_active":"2024-04-25T08:33:09.550847+0000","last_peered":"2024-04-25T08:33:09.550847+0000","last_clean":"2024-04-25T08:33:09.550847+0000","last_became_active":"2024-04-25T08:33:08.554666+0000","last_became_peered":"2024-04-25T08:33:08.554666+0000","last_unstale":"2024-04-25T08:33:09.550847+0000","last_undegraded":"2024-04-25T08:33:09.550847+0000","last_fullsized":"2024-04-25T08:33:09.550847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:45.438238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864472+0000","last_change":"2024-04-25T08:33:08.546368+0000","last_active":"2024-04-25T08:33:09.864472+0000","last_peered":"2024-04-25T08:33:09.864472+0000","last_clean":"2024-04-25T08:33:09.864472+0000","last_became_active":"2024-04-25T08:33:08.546103+0000","last_became_peered":"2024-04-25T08:33:08.546103+0000","last_unstale":"2024-04-25T08:33:09.864472+0000","last_undegraded":"2024-04-25T08:33:09.864472+0000","last_fullsized":"2024-04-25T08:33:09.864472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:25.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864712+0000","last_change":"2024-04-25T08:33:08.556278+0000","last_active":"2024-04-25T08:33:09.864712+0000","last_peered":"2024-04-25T08:33:09.864712+0000","last_clean":"2024-04-25T08:33:09.864712+0000","last_became_active":"2024-04-25T08:33:08.556013+0000","last_became_peered":"2024-04-25T08:33:08.556013+0000","last_unstale":"2024-04-25T08:33:09.864712+0000","last_undegraded":"2024-04-25T08:33:09.864712+0000","last_fullsized":"2024-04-25T08:33:09.864712+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:30.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864605+0000","last_change":"2024-04-25T08:33:08.553832+0000","last_active":"2024-04-25T08:33:09.864605+0000","last_peered":"2024-04-25T08:33:09.864605+0000","last_clean":"2024-04-25T08:33:09.864605+0000","last_became_active":"2024-04-25T08:33:08.553391+0000","last_became_peered":"2024-04-25T08:33:08.553391+0000","last_unstale":"2024-04-25T08:33:09.864605+0000","last_undegraded":"2024-04-25T08:33:09.864605+0000","last_fullsized":"2024-04-25T08:33:09.864605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.805711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864266+0000","last_change":"2024-04-25T08:33:08.560624+0000","last_active":"2024-04-25T08:33:09.864266+0000","last_peered":"2024-04-25T08:33:09.864266+0000","last_clean":"2024-04-25T08:33:09.864266+0000","last_became_active":"2024-04-25T08:33:08.560326+0000","last_became_peered":"2024-04-25T08:33:08.560326+0000","last_unstale":"2024-04-25T08:33:09.864266+0000","last_undegraded":"2024-04-25T08:33:09.864266+0000","last_fullsized":"2024-04-25T08:33:09.864266+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864669+0000","last_change":"2024-04-25T08:33:08.559445+0000","last_active":"2024-04-25T08:33:09.864669+0000","last_peered":"2024-04-25T08:33:09.864669+0000","last_clean":"2024-04-25T08:33:09.864669+0000","last_became_active":"2024-04-25T08:33:08.559161+0000","last_became_peered":"2024-04-25T08:33:08.559161+0000","last_unstale":"2024-04-25T08:33:09.864669+0000","last_undegraded":"2024-04-25T08:33:09.864669+0000","last_fullsized":"2024-04-25T08:33:09.864669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:19.097337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865423+0000","last_change":"2024-04-25T08:33:08.555080+0000","last_active":"2024-04-25T08:33:09.865423+0000","last_peered":"2024-04-25T08:33:09.865423+0000","last_clean":"2024-04-25T08:33:09.865423+0000","last_became_active":"2024-04-25T08:33:08.554825+0000","last_became_peered":"2024-04-25T08:33:08.554825+0000","last_unstale":"2024-04-25T08:33:09.865423+0000","last_undegraded":"2024-04-25T08:33:09.865423+0000","last_fullsized":"2024-04-25T08:33:09.865423+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:04.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865271+0000","last_change":"2024-04-25T08:33:08.554523+0000","last_active":"2024-04-25T08:33:09.865271+0000","last_peered":"2024-04-25T08:33:09.865271+0000","last_clean":"2024-04-25T08:33:09.865271+0000","last_became_active":"2024-04-25T08:33:08.554257+0000","last_became_peered":"2024-04-25T08:33:08.554257+0000","last_unstale":"2024-04-25T08:33:09.865271+0000","last_undegraded":"2024-04-25T08:33:09.865271+0000","last_fullsized":"2024-04-25T08:33:09.865271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:25.784503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551792+0000","last_change":"2024-04-25T08:33:08.554280+0000","last_active":"2024-04-25T08:33:09.551792+0000","last_peered":"2024-04-25T08:33:09.551792+0000","last_clean":"2024-04-25T08:33:09.551792+0000","last_became_active":"2024-04-25T08:33:08.553988+0000","last_became_peered":"2024-04-25T08:33:08.553988+0000","last_unstale":"2024-04-25T08:33:09.551792+0000","last_undegraded":"2024-04-25T08:33:09.551792+0000","last_fullsized":"2024-04-25T08:33:09.551792+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:17.271075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547058+0000","last_change":"2024-04-25T08:33:08.550356+0000","last_active":"2024-04-25T08:33:09.547058+0000","last_peered":"2024-04-25T08:33:09.547058+0000","last_clean":"2024-04-25T08:33:09.547058+0000","last_became_active":"2024-04-25T08:33:08.549045+0000","last_became_peered":"2024-04-25T08:33:08.549045+0000","last_unstale":"2024-04-25T08:33:09.547058+0000","last_undegraded":"2024-04-25T08:33:09.547058+0000","last_fullsized":"2024-04-25T08:33:09.547058+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:45.438238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864937+0000","last_change":"2024-04-25T08:33:08.551934+0000","last_active":"2024-04-25T08:33:09.864937+0000","last_peered":"2024-04-25T08:33:09.864937+0000","last_clean":"2024-04-25T08:33:09.864937+0000","last_became_active":"2024-04-25T08:33:08.551384+0000","last_became_peered":"2024-04-25T08:33:08.551384+0000","last_unstale":"2024-04-25T08:33:09.864937+0000","last_undegraded":"2024-04-25T08:33:09.864937+0000","last_fullsized":"2024-04-25T08:33:09.864937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:12.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864544+0000","last_change":"2024-04-25T08:33:08.557334+0000","last_active":"2024-04-25T08:33:09.864544+0000","last_peered":"2024-04-25T08:33:09.864544+0000","last_clean":"2024-04-25T08:33:09.864544+0000","last_became_active":"2024-04-25T08:33:08.557081+0000","last_became_peered":"2024-04-25T08:33:08.557081+0000","last_unstale":"2024-04-25T08:33:09.864544+0000","last_undegraded":"2024-04-25T08:33:09.864544+0000","last_fullsized":"2024-04-25T08:33:09.864544+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:58.250815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550684+0000","last_change":"2024-04-25T08:33:08.546322+0000","last_active":"2024-04-25T08:33:09.550684+0000","last_peered":"2024-04-25T08:33:09.550684+0000","last_clean":"2024-04-25T08:33:09.550684+0000","last_became_active":"2024-04-25T08:33:08.545614+0000","last_became_peered":"2024-04-25T08:33:08.545614+0000","last_unstale":"2024-04-25T08:33:09.550684+0000","last_undegraded":"2024-04-25T08:33:09.550684+0000","last_fullsized":"2024-04-25T08:33:09.550684+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:35.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864906+0000","last_change":"2024-04-25T08:33:08.551250+0000","last_active":"2024-04-25T08:33:09.864906+0000","last_peered":"2024-04-25T08:33:09.864906+0000","last_clean":"2024-04-25T08:33:09.864906+0000","last_became_active":"2024-04-25T08:33:08.550950+0000","last_became_peered":"2024-04-25T08:33:08.550950+0000","last_unstale":"2024-04-25T08:33:09.864906+0000","last_undegraded":"2024-04-25T08:33:09.864906+0000","last_fullsized":"2024-04-25T08:33:09.864906+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:12.069309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550781+0000","last_change":"2024-04-25T08:33:08.553945+0000","last_active":"2024-04-25T08:33:09.550781+0000","last_peered":"2024-04-25T08:33:09.550781+0000","last_clean":"2024-04-25T08:33:09.550781+0000","last_became_active":"2024-04-25T08:33:08.550496+0000","last_became_peered":"2024-04-25T08:33:08.550496+0000","last_unstale":"2024-04-25T08:33:09.550781+0000","last_undegraded":"2024-04-25T08:33:09.550781+0000","last_fullsized":"2024-04-25T08:33:09.550781+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.382160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547426+0000","last_change":"2024-04-25T08:33:07.544574+0000","last_active":"2024-04-25T08:33:09.547426+0000","last_peered":"2024-04-25T08:33:09.547426+0000","last_clean":"2024-04-25T08:33:09.547426+0000","last_became_active":"2024-04-25T08:33:07.544311+0000","last_became_peered":"2024-04-25T08:33:07.544311+0000","last_unstale":"2024-04-25T08:33:09.547426+0000","last_undegraded":"2024-04-25T08:33:09.547426+0000","last_fullsized":"2024-04-25T08:33:09.547426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:58.505300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551832+0000","last_change":"2024-04-25T08:33:08.541418+0000","last_active":"2024-04-25T08:33:09.551832+0000","last_peered":"2024-04-25T08:33:09.551832+0000","last_clean":"2024-04-25T08:33:09.551832+0000","last_became_active":"2024-04-25T08:33:08.540238+0000","last_became_peered":"2024-04-25T08:33:08.540238+0000","last_unstale":"2024-04-25T08:33:09.551832+0000","last_undegraded":"2024-04-25T08:33:09.551832+0000","last_fullsized":"2024-04-25T08:33:09.551832+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:42.343594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865231+0000","last_change":"2024-04-25T08:33:07.538223+0000","last_active":"2024-04-25T08:33:09.865231+0000","last_peered":"2024-04-25T08:33:09.865231+0000","last_clean":"2024-04-25T08:33:09.865231+0000","last_became_active":"2024-04-25T08:33:07.537909+0000","last_became_peered":"2024-04-25T08:33:07.537909+0000","last_unstale":"2024-04-25T08:33:09.865231+0000","last_undegraded":"2024-04-25T08:33:09.865231+0000","last_fullsized":"2024-04-25T08:33:09.865231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:30.854905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865069+0000","last_change":"2024-04-25T08:33:08.546649+0000","last_active":"2024-04-25T08:33:09.865069+0000","last_peered":"2024-04-25T08:33:09.865069+0000","last_clean":"2024-04-25T08:33:09.865069+0000","last_became_active":"2024-04-25T08:33:08.546377+0000","last_became_peered":"2024-04-25T08:33:08.546377+0000","last_unstale":"2024-04-25T08:33:09.865069+0000","last_undegraded":"2024-04-25T08:33:09.865069+0000","last_fullsized":"2024-04-25T08:33:09.865069+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:30.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865020+0000","last_change":"2024-04-25T08:33:07.543378+0000","last_active":"2024-04-25T08:33:09.865020+0000","last_peered":"2024-04-25T08:33:09.865020+0000","last_clean":"2024-04-25T08:33:09.865020+0000","last_became_active":"2024-04-25T08:33:07.543068+0000","last_became_peered":"2024-04-25T08:33:07.543068+0000","last_unstale":"2024-04-25T08:33:09.865020+0000","last_undegraded":"2024-04-25T08:33:09.865020+0000","last_fullsized":"2024-04-25T08:33:09.865020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:53.492411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864801+0000","last_change":"2024-04-25T08:33:08.555873+0000","last_active":"2024-04-25T08:33:09.864801+0000","last_peered":"2024-04-25T08:33:09.864801+0000","last_clean":"2024-04-25T08:33:09.864801+0000","last_became_active":"2024-04-25T08:33:08.555575+0000","last_became_peered":"2024-04-25T08:33:08.555575+0000","last_unstale":"2024-04-25T08:33:09.864801+0000","last_undegraded":"2024-04-25T08:33:09.864801+0000","last_fullsized":"2024-04-25T08:33:09.864801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:58.250815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864868+0000","last_change":"2024-04-25T08:33:07.544654+0000","last_active":"2024-04-25T08:33:09.864868+0000","last_peered":"2024-04-25T08:33:09.864868+0000","last_clean":"2024-04-25T08:33:09.864868+0000","last_became_active":"2024-04-25T08:33:07.544158+0000","last_became_peered":"2024-04-25T08:33:07.544158+0000","last_unstale":"2024-04-25T08:33:09.864868+0000","last_undegraded":"2024-04-25T08:33:09.864868+0000","last_fullsized":"2024-04-25T08:33:09.864868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:44.778767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865062+0000","last_change":"2024-04-25T08:33:08.554621+0000","last_active":"2024-04-25T08:33:09.865062+0000","last_peered":"2024-04-25T08:33:09.865062+0000","last_clean":"2024-04-25T08:33:09.865062+0000","last_became_active":"2024-04-25T08:33:08.553827+0000","last_became_peered":"2024-04-25T08:33:08.553827+0000","last_unstale":"2024-04-25T08:33:09.865062+0000","last_undegraded":"2024-04-25T08:33:09.865062+0000","last_fullsized":"2024-04-25T08:33:09.865062+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:33.554420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864587+0000","last_change":"2024-04-25T08:33:07.541015+0000","last_active":"2024-04-25T08:33:09.864587+0000","last_peered":"2024-04-25T08:33:09.864587+0000","last_clean":"2024-04-25T08:33:09.864587+0000","last_became_active":"2024-04-25T08:33:07.540688+0000","last_became_peered":"2024-04-25T08:33:07.540688+0000","last_unstale":"2024-04-25T08:33:09.864587+0000","last_undegraded":"2024-04-25T08:33:09.864587+0000","last_fullsized":"2024-04-25T08:33:09.864587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:22.449454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.546864+0000","last_change":"2024-04-25T08:33:08.563441+0000","last_active":"2024-04-25T08:33:09.546864+0000","last_peered":"2024-04-25T08:33:09.546864+0000","last_clean":"2024-04-25T08:33:09.546864+0000","last_became_active":"2024-04-25T08:33:08.563151+0000","last_became_peered":"2024-04-25T08:33:08.563151+0000","last_unstale":"2024-04-25T08:33:09.546864+0000","last_undegraded":"2024-04-25T08:33:09.546864+0000","last_fullsized":"2024-04-25T08:33:09.546864+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:29.540053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864445+0000","last_change":"2024-04-25T08:33:07.546439+0000","last_active":"2024-04-25T08:33:09.864445+0000","last_peered":"2024-04-25T08:33:09.864445+0000","last_clean":"2024-04-25T08:33:09.864445+0000","last_became_active":"2024-04-25T08:33:07.545798+0000","last_became_peered":"2024-04-25T08:33:07.545798+0000","last_unstale":"2024-04-25T08:33:09.864445+0000","last_undegraded":"2024-04-25T08:33:09.864445+0000","last_fullsized":"2024-04-25T08:33:09.864445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:30.854905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550222+0000","last_change":"2024-04-25T08:33:08.552270+0000","last_active":"2024-04-25T08:33:09.550222+0000","last_peered":"2024-04-25T08:33:09.550222+0000","last_clean":"2024-04-25T08:33:09.550222+0000","last_became_active":"2024-04-25T08:33:08.551947+0000","last_became_peered":"2024-04-25T08:33:08.551947+0000","last_unstale":"2024-04-25T08:33:09.550222+0000","last_undegraded":"2024-04-25T08:33:09.550222+0000","last_fullsized":"2024-04-25T08:33:09.550222+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:35.852808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864273+0000","last_change":"2024-04-25T08:33:07.543221+0000","last_active":"2024-04-25T08:33:09.864273+0000","last_peered":"2024-04-25T08:33:09.864273+0000","last_clean":"2024-04-25T08:33:09.864273+0000","last_became_active":"2024-04-25T08:33:07.542936+0000","last_became_peered":"2024-04-25T08:33:07.542936+0000","last_unstale":"2024-04-25T08:33:09.864273+0000","last_undegraded":"2024-04-25T08:33:09.864273+0000","last_fullsized":"2024-04-25T08:33:09.864273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:56.406524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864361+0000","last_change":"2024-04-25T08:33:08.548472+0000","last_active":"2024-04-25T08:33:09.864361+0000","last_peered":"2024-04-25T08:33:09.864361+0000","last_clean":"2024-04-25T08:33:09.864361+0000","last_became_active":"2024-04-25T08:33:08.548188+0000","last_became_peered":"2024-04-25T08:33:08.548188+0000","last_unstale":"2024-04-25T08:33:09.864361+0000","last_undegraded":"2024-04-25T08:33:09.864361+0000","last_fullsized":"2024-04-25T08:33:09.864361+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:04.057423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550179+0000","last_change":"2024-04-25T08:33:07.544691+0000","last_active":"2024-04-25T08:33:09.550179+0000","last_peered":"2024-04-25T08:33:09.550179+0000","last_clean":"2024-04-25T08:33:09.550179+0000","last_became_active":"2024-04-25T08:33:07.544319+0000","last_became_peered":"2024-04-25T08:33:07.544319+0000","last_unstale":"2024-04-25T08:33:09.550179+0000","last_undegraded":"2024-04-25T08:33:09.550179+0000","last_fullsized":"2024-04-25T08:33:09.550179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:32.068805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547137+0000","last_change":"2024-04-25T08:33:08.552435+0000","last_active":"2024-04-25T08:33:09.547137+0000","last_peered":"2024-04-25T08:33:09.547137+0000","last_clean":"2024-04-25T08:33:09.547137+0000","last_became_active":"2024-04-25T08:33:08.552015+0000","last_became_peered":"2024-04-25T08:33:08.552015+0000","last_unstale":"2024-04-25T08:33:09.547137+0000","last_undegraded":"2024-04-25T08:33:09.547137+0000","last_fullsized":"2024-04-25T08:33:09.547137+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:57.975994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865577+0000","last_change":"2024-04-25T08:33:07.548394+0000","last_active":"2024-04-25T08:33:09.865577+0000","last_peered":"2024-04-25T08:33:09.865577+0000","last_clean":"2024-04-25T08:33:09.865577+0000","last_became_active":"2024-04-25T08:33:07.548134+0000","last_became_peered":"2024-04-25T08:33:07.548134+0000","last_unstale":"2024-04-25T08:33:09.865577+0000","last_undegraded":"2024-04-25T08:33:09.865577+0000","last_fullsized":"2024-04-25T08:33:09.865577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:56.970394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864508+0000","last_change":"2024-04-25T08:33:08.545734+0000","last_active":"2024-04-25T08:33:09.864508+0000","last_peered":"2024-04-25T08:33:09.864508+0000","last_clean":"2024-04-25T08:33:09.864508+0000","last_became_active":"2024-04-25T08:33:08.545249+0000","last_became_peered":"2024-04-25T08:33:08.545249+0000","last_unstale":"2024-04-25T08:33:09.864508+0000","last_undegraded":"2024-04-25T08:33:09.864508+0000","last_fullsized":"2024-04-25T08:33:09.864508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:04.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551468+0000","last_change":"2024-04-25T08:33:07.546575+0000","last_active":"2024-04-25T08:33:09.551468+0000","last_peered":"2024-04-25T08:33:09.551468+0000","last_clean":"2024-04-25T08:33:09.551468+0000","last_became_active":"2024-04-25T08:33:07.546301+0000","last_became_peered":"2024-04-25T08:33:07.546301+0000","last_unstale":"2024-04-25T08:33:09.551468+0000","last_undegraded":"2024-04-25T08:33:09.551468+0000","last_fullsized":"2024-04-25T08:33:09.551468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:34.997631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551170+0000","last_change":"2024-04-25T08:33:08.542944+0000","last_active":"2024-04-25T08:33:09.551170+0000","last_peered":"2024-04-25T08:33:09.551170+0000","last_clean":"2024-04-25T08:33:09.551170+0000","last_became_active":"2024-04-25T08:33:08.542671+0000","last_became_peered":"2024-04-25T08:33:08.542671+0000","last_unstale":"2024-04-25T08:33:09.551170+0000","last_undegraded":"2024-04-25T08:33:09.551170+0000","last_fullsized":"2024-04-25T08:33:09.551170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:36.862604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864871+0000","last_change":"2024-04-25T08:33:07.548502+0000","last_active":"2024-04-25T08:33:09.864871+0000","last_peered":"2024-04-25T08:33:09.864871+0000","last_clean":"2024-04-25T08:33:09.864871+0000","last_became_active":"2024-04-25T08:33:07.548248+0000","last_became_peered":"2024-04-25T08:33:07.548248+0000","last_unstale":"2024-04-25T08:33:09.864871+0000","last_undegraded":"2024-04-25T08:33:09.864871+0000","last_fullsized":"2024-04-25T08:33:09.864871+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:35.857004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551067+0000","last_change":"2024-04-25T08:33:08.546960+0000","last_active":"2024-04-25T08:33:09.551067+0000","last_peered":"2024-04-25T08:33:09.551067+0000","last_clean":"2024-04-25T08:33:09.551067+0000","last_became_active":"2024-04-25T08:33:08.546634+0000","last_became_peered":"2024-04-25T08:33:08.546634+0000","last_unstale":"2024-04-25T08:33:09.551067+0000","last_undegraded":"2024-04-25T08:33:09.551067+0000","last_fullsized":"2024-04-25T08:33:09.551067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:00.494779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547276+0000","last_change":"2024-04-25T08:33:07.541975+0000","last_active":"2024-04-25T08:33:09.547276+0000","last_peered":"2024-04-25T08:33:09.547276+0000","last_clean":"2024-04-25T08:33:09.547276+0000","last_became_active":"2024-04-25T08:33:07.541691+0000","last_became_peered":"2024-04-25T08:33:07.541691+0000","last_unstale":"2024-04-25T08:33:09.547276+0000","last_undegraded":"2024-04-25T08:33:09.547276+0000","last_fullsized":"2024-04-25T08:33:09.547276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:33.833345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550617+0000","last_change":"2024-04-25T08:33:08.549160+0000","last_active":"2024-04-25T08:33:09.550617+0000","last_peered":"2024-04-25T08:33:09.550617+0000","last_clean":"2024-04-25T08:33:09.550617+0000","last_became_active":"2024-04-25T08:33:08.548894+0000","last_became_peered":"2024-04-25T08:33:08.548894+0000","last_unstale":"2024-04-25T08:33:09.550617+0000","last_undegraded":"2024-04-25T08:33:09.550617+0000","last_fullsized":"2024-04-25T08:33:09.550617+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:05.027592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864636+0000","last_change":"2024-04-25T08:33:07.543683+0000","last_active":"2024-04-25T08:33:09.864636+0000","last_peered":"2024-04-25T08:33:09.864636+0000","last_clean":"2024-04-25T08:33:09.864636+0000","last_became_active":"2024-04-25T08:33:07.543154+0000","last_became_peered":"2024-04-25T08:33:07.543154+0000","last_unstale":"2024-04-25T08:33:09.864636+0000","last_undegraded":"2024-04-25T08:33:09.864636+0000","last_fullsized":"2024-04-25T08:33:09.864636+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:52.394108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864433+0000","last_change":"2024-04-25T08:33:08.552996+0000","last_active":"2024-04-25T08:33:09.864433+0000","last_peered":"2024-04-25T08:33:09.864433+0000","last_clean":"2024-04-25T08:33:09.864433+0000","last_became_active":"2024-04-25T08:33:08.550905+0000","last_became_peered":"2024-04-25T08:33:08.550905+0000","last_unstale":"2024-04-25T08:33:09.864433+0000","last_undegraded":"2024-04-25T08:33:09.864433+0000","last_fullsized":"2024-04-25T08:33:09.864433+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:19.097337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864817+0000","last_change":"2024-04-25T08:33:07.549661+0000","last_active":"2024-04-25T08:33:09.864817+0000","last_peered":"2024-04-25T08:33:09.864817+0000","last_clean":"2024-04-25T08:33:09.864817+0000","last_became_active":"2024-04-25T08:33:07.549387+0000","last_became_peered":"2024-04-25T08:33:07.549387+0000","last_unstale":"2024-04-25T08:33:09.864817+0000","last_undegraded":"2024-04-25T08:33:09.864817+0000","last_fullsized":"2024-04-25T08:33:09.864817+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:28.534452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550407+0000","last_change":"2024-04-25T08:33:08.546035+0000","last_active":"2024-04-25T08:33:09.550407+0000","last_peered":"2024-04-25T08:33:09.550407+0000","last_clean":"2024-04-25T08:33:09.550407+0000","last_became_active":"2024-04-25T08:33:08.545678+0000","last_became_peered":"2024-04-25T08:33:08.545678+0000","last_unstale":"2024-04-25T08:33:09.550407+0000","last_undegraded":"2024-04-25T08:33:09.550407+0000","last_fullsized":"2024-04-25T08:33:09.550407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:26.016740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865465+0000","last_change":"2024-04-25T08:33:07.546458+0000","last_active":"2024-04-25T08:33:09.865465+0000","last_peered":"2024-04-25T08:33:09.865465+0000","last_clean":"2024-04-25T08:33:09.865465+0000","last_became_active":"2024-04-25T08:33:07.546172+0000","last_became_peered":"2024-04-25T08:33:07.546172+0000","last_unstale":"2024-04-25T08:33:09.865465+0000","last_undegraded":"2024-04-25T08:33:09.865465+0000","last_fullsized":"2024-04-25T08:33:09.865465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:07.376559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547258+0000","last_change":"2024-04-25T08:33:08.562273+0000","last_active":"2024-04-25T08:33:09.547258+0000","last_peered":"2024-04-25T08:33:09.547258+0000","last_clean":"2024-04-25T08:33:09.547258+0000","last_became_active":"2024-04-25T08:33:08.561973+0000","last_became_peered":"2024-04-25T08:33:08.561973+0000","last_unstale":"2024-04-25T08:33:09.547258+0000","last_undegraded":"2024-04-25T08:33:09.547258+0000","last_fullsized":"2024-04-25T08:33:09.547258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:54.498011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550577+0000","last_change":"2024-04-25T08:33:07.543855+0000","last_active":"2024-04-25T08:33:09.550577+0000","last_peered":"2024-04-25T08:33:09.550577+0000","last_clean":"2024-04-25T08:33:09.550577+0000","last_became_active":"2024-04-25T08:33:07.543538+0000","last_became_peered":"2024-04-25T08:33:07.543538+0000","last_unstale":"2024-04-25T08:33:09.550577+0000","last_undegraded":"2024-04-25T08:33:09.550577+0000","last_fullsized":"2024-04-25T08:33:09.550577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:15.576960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865032+0000","last_change":"2024-04-25T08:33:08.550553+0000","last_active":"2024-04-25T08:33:09.865032+0000","last_peered":"2024-04-25T08:33:09.865032+0000","last_clean":"2024-04-25T08:33:09.865032+0000","last_became_active":"2024-04-25T08:33:08.550268+0000","last_became_peered":"2024-04-25T08:33:08.550268+0000","last_unstale":"2024-04-25T08:33:09.865032+0000","last_undegraded":"2024-04-25T08:33:09.865032+0000","last_fullsized":"2024-04-25T08:33:09.865032+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:30.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865511+0000","last_change":"2024-04-25T08:33:07.549635+0000","last_active":"2024-04-25T08:33:09.865511+0000","last_peered":"2024-04-25T08:33:09.865511+0000","last_clean":"2024-04-25T08:33:09.865511+0000","last_became_active":"2024-04-25T08:33:07.549355+0000","last_became_peered":"2024-04-25T08:33:07.549355+0000","last_unstale":"2024-04-25T08:33:09.865511+0000","last_undegraded":"2024-04-25T08:33:09.865511+0000","last_fullsized":"2024-04-25T08:33:09.865511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:52.394108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551017+0000","last_change":"2024-04-25T08:33:08.542217+0000","last_active":"2024-04-25T08:33:09.551017+0000","last_peered":"2024-04-25T08:33:09.551017+0000","last_clean":"2024-04-25T08:33:09.551017+0000","last_became_active":"2024-04-25T08:33:08.541905+0000","last_became_peered":"2024-04-25T08:33:08.541905+0000","last_unstale":"2024-04-25T08:33:09.551017+0000","last_undegraded":"2024-04-25T08:33:09.551017+0000","last_fullsized":"2024-04-25T08:33:09.551017+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:57.975994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865246+0000","last_change":"2024-04-25T08:33:07.545683+0000","last_active":"2024-04-25T08:33:09.865246+0000","last_peered":"2024-04-25T08:33:09.865246+0000","last_clean":"2024-04-25T08:33:09.865246+0000","last_became_active":"2024-04-25T08:33:07.545403+0000","last_became_peered":"2024-04-25T08:33:07.545403+0000","last_unstale":"2024-04-25T08:33:09.865246+0000","last_undegraded":"2024-04-25T08:33:09.865246+0000","last_fullsized":"2024-04-25T08:33:09.865246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:44.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551372+0000","last_change":"2024-04-25T08:33:08.536670+0000","last_active":"2024-04-25T08:33:09.551372+0000","last_peered":"2024-04-25T08:33:09.551372+0000","last_clean":"2024-04-25T08:33:09.551372+0000","last_became_active":"2024-04-25T08:33:08.536399+0000","last_became_peered":"2024-04-25T08:33:08.536399+0000","last_unstale":"2024-04-25T08:33:09.551372+0000","last_undegraded":"2024-04-25T08:33:09.551372+0000","last_fullsized":"2024-04-25T08:33:09.551372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:34.838945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865380+0000","last_change":"2024-04-25T08:33:07.549848+0000","last_active":"2024-04-25T08:33:09.865380+0000","last_peered":"2024-04-25T08:33:09.865380+0000","last_clean":"2024-04-25T08:33:09.865380+0000","last_became_active":"2024-04-25T08:33:07.549547+0000","last_became_peered":"2024-04-25T08:33:07.549547+0000","last_unstale":"2024-04-25T08:33:09.865380+0000","last_undegraded":"2024-04-25T08:33:09.865380+0000","last_fullsized":"2024-04-25T08:33:09.865380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:53.492411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551221+0000","last_change":"2024-04-25T08:33:08.548240+0000","last_active":"2024-04-25T08:33:09.551221+0000","last_peered":"2024-04-25T08:33:09.551221+0000","last_clean":"2024-04-25T08:33:09.551221+0000","last_became_active":"2024-04-25T08:33:08.547974+0000","last_became_peered":"2024-04-25T08:33:08.547974+0000","last_unstale":"2024-04-25T08:33:09.551221+0000","last_undegraded":"2024-04-25T08:33:09.551221+0000","last_fullsized":"2024-04-25T08:33:09.551221+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:54.498011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864526+0000","last_change":"2024-04-25T08:33:07.539818+0000","last_active":"2024-04-25T08:33:09.864526+0000","last_peered":"2024-04-25T08:33:09.864526+0000","last_clean":"2024-04-25T08:33:09.864526+0000","last_became_active":"2024-04-25T08:33:07.539393+0000","last_became_peered":"2024-04-25T08:33:07.539393+0000","last_unstale":"2024-04-25T08:33:09.864526+0000","last_undegraded":"2024-04-25T08:33:09.864526+0000","last_fullsized":"2024-04-25T08:33:09.864526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:29.040181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865232+0000","last_change":"2024-04-25T08:33:08.550778+0000","last_active":"2024-04-25T08:33:09.865232+0000","last_peered":"2024-04-25T08:33:09.865232+0000","last_clean":"2024-04-25T08:33:09.865232+0000","last_became_active":"2024-04-25T08:33:08.549866+0000","last_became_peered":"2024-04-25T08:33:08.549866+0000","last_unstale":"2024-04-25T08:33:09.865232+0000","last_undegraded":"2024-04-25T08:33:09.865232+0000","last_fullsized":"2024-04-25T08:33:09.865232+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:00.554067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864600+0000","last_change":"2024-04-25T08:33:07.544798+0000","last_active":"2024-04-25T08:33:09.864600+0000","last_peered":"2024-04-25T08:33:09.864600+0000","last_clean":"2024-04-25T08:33:09.864600+0000","last_became_active":"2024-04-25T08:33:07.544391+0000","last_became_peered":"2024-04-25T08:33:07.544391+0000","last_unstale":"2024-04-25T08:33:09.864600+0000","last_undegraded":"2024-04-25T08:33:09.864600+0000","last_fullsized":"2024-04-25T08:33:09.864600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:35.857004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864528+0000","last_change":"2024-04-25T08:33:08.546653+0000","last_active":"2024-04-25T08:33:09.864528+0000","last_peered":"2024-04-25T08:33:09.864528+0000","last_clean":"2024-04-25T08:33:09.864528+0000","last_became_active":"2024-04-25T08:33:08.546363+0000","last_became_peered":"2024-04-25T08:33:08.546363+0000","last_unstale":"2024-04-25T08:33:09.864528+0000","last_undegraded":"2024-04-25T08:33:09.864528+0000","last_fullsized":"2024-04-25T08:33:09.864528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864913+0000","last_change":"2024-04-25T08:33:07.547407+0000","last_active":"2024-04-25T08:33:09.864913+0000","last_peered":"2024-04-25T08:33:09.864913+0000","last_clean":"2024-04-25T08:33:09.864913+0000","last_became_active":"2024-04-25T08:33:07.547116+0000","last_became_peered":"2024-04-25T08:33:07.547116+0000","last_unstale":"2024-04-25T08:33:09.864913+0000","last_undegraded":"2024-04-25T08:33:09.864913+0000","last_fullsized":"2024-04-25T08:33:09.864913+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:44.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550429+0000","last_change":"2024-04-25T08:33:08.550448+0000","last_active":"2024-04-25T08:33:09.550429+0000","last_peered":"2024-04-25T08:33:09.550429+0000","last_clean":"2024-04-25T08:33:09.550429+0000","last_became_active":"2024-04-25T08:33:08.550107+0000","last_became_peered":"2024-04-25T08:33:08.550107+0000","last_unstale":"2024-04-25T08:33:09.550429+0000","last_undegraded":"2024-04-25T08:33:09.550429+0000","last_fullsized":"2024-04-25T08:33:09.550429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:28.833570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865115+0000","last_change":"2024-04-25T08:33:07.542368+0000","last_active":"2024-04-25T08:33:09.865115+0000","last_peered":"2024-04-25T08:33:09.865115+0000","last_clean":"2024-04-25T08:33:09.865115+0000","last_became_active":"2024-04-25T08:33:07.542110+0000","last_became_peered":"2024-04-25T08:33:07.542110+0000","last_unstale":"2024-04-25T08:33:09.865115+0000","last_undegraded":"2024-04-25T08:33:09.865115+0000","last_fullsized":"2024-04-25T08:33:09.865115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:22.449454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864761+0000","last_change":"2024-04-25T08:33:08.542650+0000","last_active":"2024-04-25T08:33:09.864761+0000","last_peered":"2024-04-25T08:33:09.864761+0000","last_clean":"2024-04-25T08:33:09.864761+0000","last_became_active":"2024-04-25T08:33:08.542383+0000","last_became_peered":"2024-04-25T08:33:08.542383+0000","last_unstale":"2024-04-25T08:33:09.864761+0000","last_undegraded":"2024-04-25T08:33:09.864761+0000","last_fullsized":"2024-04-25T08:33:09.864761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:02.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865644+0000","last_change":"2024-04-25T08:33:07.546860+0000","last_active":"2024-04-25T08:33:09.865644+0000","last_peered":"2024-04-25T08:33:09.865644+0000","last_clean":"2024-04-25T08:33:09.865644+0000","last_became_active":"2024-04-25T08:33:07.546585+0000","last_became_peered":"2024-04-25T08:33:07.546585+0000","last_unstale":"2024-04-25T08:33:09.865644+0000","last_undegraded":"2024-04-25T08:33:09.865644+0000","last_fullsized":"2024-04-25T08:33:09.865644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:07.376559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865515+0000","last_change":"2024-04-25T08:33:08.557006+0000","last_active":"2024-04-25T08:33:09.865515+0000","last_peered":"2024-04-25T08:33:09.865515+0000","last_clean":"2024-04-25T08:33:09.865515+0000","last_became_active":"2024-04-25T08:33:08.556157+0000","last_became_peered":"2024-04-25T08:33:08.556157+0000","last_unstale":"2024-04-25T08:33:09.865515+0000","last_undegraded":"2024-04-25T08:33:09.865515+0000","last_fullsized":"2024-04-25T08:33:09.865515+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:03.656381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865569+0000","last_change":"2024-04-25T08:33:07.546753+0000","last_active":"2024-04-25T08:33:09.865569+0000","last_peered":"2024-04-25T08:33:09.865569+0000","last_clean":"2024-04-25T08:33:09.865569+0000","last_became_active":"2024-04-25T08:33:07.546472+0000","last_became_peered":"2024-04-25T08:33:07.546472+0000","last_unstale":"2024-04-25T08:33:09.865569+0000","last_undegraded":"2024-04-25T08:33:09.865569+0000","last_fullsized":"2024-04-25T08:33:09.865569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:56.970394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865439+0000","last_change":"2024-04-25T08:33:08.552873+0000","last_active":"2024-04-25T08:33:09.865439+0000","last_peered":"2024-04-25T08:33:09.865439+0000","last_clean":"2024-04-25T08:33:09.865439+0000","last_became_active":"2024-04-25T08:33:08.552484+0000","last_became_peered":"2024-04-25T08:33:08.552484+0000","last_unstale":"2024-04-25T08:33:09.865439+0000","last_undegraded":"2024-04-25T08:33:09.865439+0000","last_fullsized":"2024-04-25T08:33:09.865439+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:13.805711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865150+0000","last_change":"2024-04-25T08:33:07.541388+0000","last_active":"2024-04-25T08:33:09.865150+0000","last_peered":"2024-04-25T08:33:09.865150+0000","last_clean":"2024-04-25T08:33:09.865150+0000","last_became_active":"2024-04-25T08:33:07.541073+0000","last_became_peered":"2024-04-25T08:33:07.541073+0000","last_unstale":"2024-04-25T08:33:09.865150+0000","last_undegraded":"2024-04-25T08:33:09.865150+0000","last_fullsized":"2024-04-25T08:33:09.865150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:12.474553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551237+0000","last_change":"2024-04-25T08:33:08.547913+0000","last_active":"2024-04-25T08:33:09.551237+0000","last_peered":"2024-04-25T08:33:09.551237+0000","last_clean":"2024-04-25T08:33:09.551237+0000","last_became_active":"2024-04-25T08:33:08.547630+0000","last_became_peered":"2024-04-25T08:33:08.547630+0000","last_unstale":"2024-04-25T08:33:09.551237+0000","last_undegraded":"2024-04-25T08:33:09.551237+0000","last_fullsized":"2024-04-25T08:33:09.551237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:45.784368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547275+0000","last_change":"2024-04-25T08:33:07.543180+0000","last_active":"2024-04-25T08:33:09.547275+0000","last_peered":"2024-04-25T08:33:09.547275+0000","last_clean":"2024-04-25T08:33:09.547275+0000","last_became_active":"2024-04-25T08:33:07.541512+0000","last_became_peered":"2024-04-25T08:33:07.541512+0000","last_unstale":"2024-04-25T08:33:09.547275+0000","last_undegraded":"2024-04-25T08:33:09.547275+0000","last_fullsized":"2024-04-25T08:33:09.547275+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:08.820438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550209+0000","last_change":"2024-04-25T08:33:08.545104+0000","last_active":"2024-04-25T08:33:09.550209+0000","last_peered":"2024-04-25T08:33:09.550209+0000","last_clean":"2024-04-25T08:33:09.550209+0000","last_became_active":"2024-04-25T08:33:08.544837+0000","last_became_peered":"2024-04-25T08:33:08.544837+0000","last_unstale":"2024-04-25T08:33:09.550209+0000","last_undegraded":"2024-04-25T08:33:09.550209+0000","last_fullsized":"2024-04-25T08:33:09.550209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:32.462998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864746+0000","last_change":"2024-04-25T08:33:07.544710+0000","last_active":"2024-04-25T08:33:09.864746+0000","last_peered":"2024-04-25T08:33:09.864746+0000","last_clean":"2024-04-25T08:33:09.864746+0000","last_became_active":"2024-04-25T08:33:07.544440+0000","last_became_peered":"2024-04-25T08:33:07.544440+0000","last_unstale":"2024-04-25T08:33:09.864746+0000","last_undegraded":"2024-04-25T08:33:09.864746+0000","last_fullsized":"2024-04-25T08:33:09.864746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:28.534452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.551714+0000","last_change":"2024-04-25T08:33:08.552778+0000","last_active":"2024-04-25T08:33:09.551714+0000","last_peered":"2024-04-25T08:33:09.551714+0000","last_clean":"2024-04-25T08:33:09.551714+0000","last_became_active":"2024-04-25T08:33:08.551945+0000","last_became_peered":"2024-04-25T08:33:08.551945+0000","last_unstale":"2024-04-25T08:33:09.551714+0000","last_undegraded":"2024-04-25T08:33:09.551714+0000","last_fullsized":"2024-04-25T08:33:09.551714+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:03.009613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864544+0000","last_change":"2024-04-25T08:33:07.546678+0000","last_active":"2024-04-25T08:33:09.864544+0000","last_peered":"2024-04-25T08:33:09.864544+0000","last_clean":"2024-04-25T08:33:09.864544+0000","last_became_active":"2024-04-25T08:33:07.546160+0000","last_became_peered":"2024-04-25T08:33:07.546160+0000","last_unstale":"2024-04-25T08:33:09.864544+0000","last_undegraded":"2024-04-25T08:33:09.864544+0000","last_fullsized":"2024-04-25T08:33:09.864544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:18.315174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.547010+0000","last_change":"2024-04-25T08:33:08.562844+0000","last_active":"2024-04-25T08:33:09.547010+0000","last_peered":"2024-04-25T08:33:09.547010+0000","last_clean":"2024-04-25T08:33:09.547010+0000","last_became_active":"2024-04-25T08:33:08.562537+0000","last_became_peered":"2024-04-25T08:33:08.562537+0000","last_unstale":"2024-04-25T08:33:09.547010+0000","last_undegraded":"2024-04-25T08:33:09.547010+0000","last_fullsized":"2024-04-25T08:33:09.547010+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:08.382160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864383+0000","last_change":"2024-04-25T08:33:07.543755+0000","last_active":"2024-04-25T08:33:09.864383+0000","last_peered":"2024-04-25T08:33:09.864383+0000","last_clean":"2024-04-25T08:33:09.864383+0000","last_became_active":"2024-04-25T08:33:07.543397+0000","last_became_peered":"2024-04-25T08:33:07.543397+0000","last_unstale":"2024-04-25T08:33:09.864383+0000","last_undegraded":"2024-04-25T08:33:09.864383+0000","last_fullsized":"2024-04-25T08:33:09.864383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:04.471094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865344+0000","last_change":"2024-04-25T08:33:08.540802+0000","last_active":"2024-04-25T08:33:09.865344+0000","last_peered":"2024-04-25T08:33:09.865344+0000","last_clean":"2024-04-25T08:33:09.865344+0000","last_became_active":"2024-04-25T08:33:08.540500+0000","last_became_peered":"2024-04-25T08:33:08.540500+0000","last_unstale":"2024-04-25T08:33:09.865344+0000","last_undegraded":"2024-04-25T08:33:09.865344+0000","last_fullsized":"2024-04-25T08:33:09.865344+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:22.382073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.546804+0000","last_change":"2024-04-25T08:33:07.539992+0000","last_active":"2024-04-25T08:33:09.546804+0000","last_peered":"2024-04-25T08:33:09.546804+0000","last_clean":"2024-04-25T08:33:09.546804+0000","last_became_active":"2024-04-25T08:33:07.539652+0000","last_became_peered":"2024-04-25T08:33:07.539652+0000","last_unstale":"2024-04-25T08:33:09.546804+0000","last_undegraded":"2024-04-25T08:33:09.546804+0000","last_fullsized":"2024-04-25T08:33:09.546804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:59.489179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.550445+0000","last_change":"2024-04-25T08:33:08.556216+0000","last_active":"2024-04-25T08:33:09.550445+0000","last_peered":"2024-04-25T08:33:09.550445+0000","last_clean":"2024-04-25T08:33:09.550445+0000","last_became_active":"2024-04-25T08:33:08.555940+0000","last_became_peered":"2024-04-25T08:33:08.555940+0000","last_unstale":"2024-04-25T08:33:09.550445+0000","last_undegraded":"2024-04-25T08:33:09.550445+0000","last_fullsized":"2024-04-25T08:33:09.550445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:07.522787+0000","last_clean_scrub_stamp":"2024-04-25T08:33:07.522787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:29.540053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.864133+0000","last_change":"2024-04-25T08:33:07.540989+0000","last_active":"2024-04-25T08:33:09.864133+0000","last_peered":"2024-04-25T08:33:09.864133+0000","last_clean":"2024-04-25T08:33:09.864133+0000","last_became_active":"2024-04-25T08:33:07.540635+0000","last_became_peered":"2024-04-25T08:33:07.540635+0000","last_unstale":"2024-04-25T08:33:09.864133+0000","last_undegraded":"2024-04-25T08:33:09.864133+0000","last_fullsized":"2024-04-25T08:33:09.864133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:06.517187+0000","last_clean_scrub_stamp":"2024-04-25T08:33:06.517187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:25.457017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":259,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-25T08:33:09.865217+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:09.865217+0000","last_peered":"2024-04-25T08:33:09.865217+0000","last_clean":"2024-04-25T08:33:09.865217+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:09.865217+0000","last_undegraded":"2024-04-25T08:33:09.865217+0000","last_fullsized":"2024-04-25T08:33:09.865217+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542196,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":1417216,"data_stored":957396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":811008,"data_stored":366286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":811008,"data_stored":366274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36140,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335700,"statfs":{"total":96636764160,"available":96599756800,"internally_reserved":0,"allocated":819200,"data_stored":367864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40292,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331548,"statfs":{"total":96636764160,"available":96595505152,"internally_reserved":0,"allocated":806912,"data_stored":366252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":806912,"data_stored":366252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36164,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335676,"statfs":{"total":96636764160,"available":96599732224,"internally_reserved":0,"allocated":835584,"data_stored":368628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45028,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326812,"statfs":{"total":96636764160,"available":96590655488,"internally_reserved":0,"allocated":1400832,"data_stored":956620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:33:12.444 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:33:12.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:33:12.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.594+0000 7f1df024a640 1 Processor -- start 2024-04-25T08:33:12.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.594+0000 7f1df024a640 1 -- start start 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1df024a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de815dc20 0x7f1de80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1df024a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1de8161710 0x7f1de80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1df024a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1de80b9330 0x7f1de806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1df024a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de815e000 con 0x7f1de80b9330 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1df024a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de80d1de0 con 0x7f1de8161710 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1df024a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de8161c60 con 0x7f1de815dc20 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.595+0000 7f1dee7e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1de80b9330 0x7f1de806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1de80b9330 0x7f1de806bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56420/0 (socket says 172.21.15.60:56420) 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 -- 172.21.15.60:0/1876077940 learned_addr learned my addr 172.21.15.60:0/1876077940 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1ded7e5640 1 --2- 172.21.15.60:0/1876077940 >> v2:172.21.15.100:3300/0 conn(0x7f1de8161710 0x7f1de80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dedfe6640 1 --2- 172.21.15.60:0/1876077940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de815dc20 0x7f1de80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 -- 172.21.15.60:0/1876077940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de815dc20 msgr2=0x7f1de80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 --2- 172.21.15.60:0/1876077940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de815dc20 0x7f1de80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 -- 172.21.15.60:0/1876077940 >> v2:172.21.15.100:3300/0 conn(0x7f1de8161710 msgr2=0x7f1de80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 --2- 172.21.15.60:0/1876077940 >> v2:172.21.15.100:3300/0 conn(0x7f1de8161710 0x7f1de80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.596+0000 7f1dee7e7640 1 -- 172.21.15.60:0/1876077940 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8161f40 con 0x7f1de80b9330 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1dee7e7640 1 --2- 172.21.15.60:0/1876077940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de806bf80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1de806d9a0 tx=0x7f1dd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f07516e16cb320d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1decfe4640 1 -- 172.21.15.60:0/1876077940 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8036070 con 0x7f1de80b9330 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1decfe4640 1 -- 172.21.15.60:0/1876077940 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd8031600 con 0x7f1de80b9330 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1decfe4640 1 -- 172.21.15.60:0/1876077940 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd80319d0 con 0x7f1de80b9330 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 -- 172.21.15.60:0/1876077940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 msgr2=0x7f1de806bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 --2- 172.21.15.60:0/1876077940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de806bf80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1de806d9a0 tx=0x7f1dd802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 -- 172.21.15.60:0/1876077940 shutdown_connections 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 --2- 172.21.15.60:0/1876077940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de806bf80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 --2- 172.21.15.60:0/1876077940 >> v2:172.21.15.100:3300/0 conn(0x7f1de8161710 0x7f1de80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 --2- 172.21.15.60:0/1876077940 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de815dc20 0x7f1de80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 -- 172.21.15.60:0/1876077940 >> 172.21.15.60:0/1876077940 conn(0x7f1de80d0420 msgr2=0x7f1de80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.597+0000 7f1df024a640 1 -- 172.21.15.60:0/1876077940 shutdown_connections 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.598+0000 7f1df024a640 1 -- 172.21.15.60:0/1876077940 wait complete. 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 Processor -- start 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 -- start start 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de81f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1de815dc20 0x7f1de815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de8161710 0x7f1de80ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80b5e50 con 0x7f1de80b9330 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de8162500 con 0x7f1de815dc20 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1df024a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80b5fd0 con 0x7f1de8161710 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1dedfe6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de81f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1dedfe6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de81f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56432/0 (socket says 172.21.15.60:56432) 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.599+0000 7f1dedfe6640 1 -- 172.21.15.60:0/1729762801 learned_addr learned my addr 172.21.15.60:0/1729762801 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1dedfe6640 1 -- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de8161710 msgr2=0x7f1de80ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1dedfe6640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de8161710 0x7f1de80ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1ded7e5640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1de815dc20 0x7f1de815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1dedfe6640 1 -- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1de815dc20 msgr2=0x7f1de815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1dedfe6640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1de815dc20 0x7f1de815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1dedfe6640 1 -- 172.21.15.60:0/1729762801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de815d480 con 0x7f1de80b9330 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1ded7e5640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1de815dc20 0x7f1de815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1dedfe6640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de81f3f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1de000e8a0 tx=0x7f1de000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0017070 con 0x7f1de80b9330 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de000ee90 con 0x7f1de80b9330 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de80aa520 con 0x7f1de80b9330 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de000cb80 con 0x7f1de80b9330 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.600+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de80aaae0 con 0x7f1de80b9330 2024-04-25T08:33:12.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.601+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de8073a00 con 0x7f1de80b9330 2024-04-25T08:33:12.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.602+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1de001a070 con 0x7f1de80b9330 2024-04-25T08:33:12.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.602+0000 7f1de67fc640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1dd406b4d0 0x7f1dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.602+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1de008a1f0 con 0x7f1de80b9330 2024-04-25T08:33:12.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.602+0000 7f1ded7e5640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1dd406b4d0 0x7f1dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.603+0000 7f1ded7e5640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1dd406b4d0 0x7f1dd406d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1de80b2780 tx=0x7f1ddc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.607+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1de815d480 con 0x7f1de80b9330 2024-04-25T08:33:12.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.856+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1de80d16b0 con 0x7f1de80b9330 2024-04-25T08:33:12.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.858+0000 7f1de67fc640 1 -- 172.21.15.60:0/1729762801 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1de0058ee0 con 0x7f1de80b9330 2024-04-25T08:33:12.853 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:33:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:33:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:33:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:33:12.853 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:33:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1dd406b4d0 msgr2=0x7f1dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1dd406b4d0 0x7f1dd406d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1de80b2780 tx=0x7f1ddc008040 comp rx=0 tx=0).stop 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 msgr2=0x7f1de81f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de81f3f60 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1de000e8a0 tx=0x7f1de000ed60 comp rx=0 tx=0).stop 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 shutdown_connections 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1de8161710 0x7f1de80ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1dd406b4d0 0x7f1dd406d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1de815dc20 0x7f1de815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 --2- 172.21.15.60:0/1729762801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de80b9330 0x7f1de81f3f60 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 >> 172.21.15.60:0/1729762801 conn(0x7f1de80d0420 msgr2=0x7f1de80b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 shutdown_connections 2024-04-25T08:33:12.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.861+0000 7f1df024a640 1 -- 172.21.15.60:0/1729762801 wait complete. 2024-04-25T08:33:12.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.973+0000 7f3d1377e640 1 Processor -- start 2024-04-25T08:33:12.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.973+0000 7f3d1377e640 1 -- start start 2024-04-25T08:33:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1377e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d0c15db70 0x7f3d0c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1377e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d0c0b6cb0 0x7f3d0c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1377e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b5a60 0x7f3d0c06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1377e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c162690 con 0x7f3d0c15db70 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1377e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c0ceee0 con 0x7f3d0c0b6cb0 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1377e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0d1d10 con 0x7f3d0c0b5a60 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1151a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d0c15db70 0x7f3d0c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1151a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d0c15db70 0x7f3d0c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56442/0 (socket says 172.21.15.60:56442) 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d1151a640 1 -- 172.21.15.60:0/2009497490 learned_addr learned my addr 172.21.15.60:0/2009497490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d11d1b640 1 --2- 172.21.15.60:0/2009497490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b5a60 0x7f3d0c06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.974+0000 7f3d10d19640 1 --2- 172.21.15.60:0/2009497490 >> v2:172.21.15.100:3300/0 conn(0x7f3d0c0b6cb0 0x7f3d0c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d1151a640 1 -- 172.21.15.60:0/2009497490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b5a60 msgr2=0x7f3d0c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d1151a640 1 --2- 172.21.15.60:0/2009497490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b5a60 0x7f3d0c06c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d1151a640 1 -- 172.21.15.60:0/2009497490 >> v2:172.21.15.100:3300/0 conn(0x7f3d0c0b6cb0 msgr2=0x7f3d0c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d1151a640 1 --2- 172.21.15.60:0/2009497490 >> v2:172.21.15.100:3300/0 conn(0x7f3d0c0b6cb0 0x7f3d0c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d1151a640 1 -- 172.21.15.60:0/2009497490 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0d1ff0 con 0x7f3d0c15db70 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d10d19640 1 --2- 172.21.15.60:0/2009497490 >> v2:172.21.15.100:3300/0 conn(0x7f3d0c0b6cb0 0x7f3d0c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:12.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d1151a640 1 --2- 172.21.15.60:0/2009497490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c162150 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c08c600 tx=0x7f3d0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70be2099e1a5f668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d03fff640 1 -- 172.21.15.60:0/2009497490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04037070 con 0x7f3d0c15db70 2024-04-25T08:33:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.975+0000 7f3d03fff640 1 -- 172.21.15.60:0/2009497490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d04032440 con 0x7f3d0c15db70 2024-04-25T08:33:12.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d03fff640 1 -- 172.21.15.60:0/2009497490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04032810 con 0x7f3d0c15db70 2024-04-25T08:33:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 -- 172.21.15.60:0/2009497490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 msgr2=0x7f3d0c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 --2- 172.21.15.60:0/2009497490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c162150 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c08c600 tx=0x7f3d0402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 -- 172.21.15.60:0/2009497490 shutdown_connections 2024-04-25T08:33:12.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 --2- 172.21.15.60:0/2009497490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b5a60 0x7f3d0c06c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 --2- 172.21.15.60:0/2009497490 >> v2:172.21.15.100:3300/0 conn(0x7f3d0c0b6cb0 0x7f3d0c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 --2- 172.21.15.60:0/2009497490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c162150 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 -- 172.21.15.60:0/2009497490 >> 172.21.15.60:0/2009497490 conn(0x7f3d0c0d0520 msgr2=0x7f3d0c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 -- 172.21.15.60:0/2009497490 shutdown_connections 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.976+0000 7f3d1377e640 1 -- 172.21.15.60:0/2009497490 wait complete. 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.977+0000 7f3d1377e640 1 Processor -- start 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.977+0000 7f3d1377e640 1 -- start start 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d1377e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d0c0b5a60 0x7f3d0c1f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d1377e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b6cb0 0x7f3d0c1f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d1377e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d1377e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0ce530 con 0x7f3d0c15db70 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d1377e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c15df50 con 0x7f3d0c0b5a60 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d1377e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0cf1c0 con 0x7f3d0c0b6cb0 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d11d1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d11d1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56454/0 (socket says 172.21.15.60:56454) 2024-04-25T08:33:12.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.978+0000 7f3d11d1b640 1 -- 172.21.15.60:0/246412149 learned_addr learned my addr 172.21.15.60:0/246412149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d11d1b640 1 -- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b6cb0 msgr2=0x7f3d0c1f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d11d1b640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b6cb0 0x7f3d0c1f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d11d1b640 1 -- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d0c0b5a60 msgr2=0x7f3d0c1f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d11d1b640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d0c0b5a60 0x7f3d0c1f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d11d1b640 1 -- 172.21.15.60:0/246412149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c15eb60 con 0x7f3d0c15db70 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d11d1b640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c151ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3d08007e60 tx=0x7f3d0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08014620 con 0x7f3d0c15db70 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08019070 con 0x7f3d0c15db70 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08014a20 con 0x7f3d0c15db70 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c1f39f0 con 0x7f3d0c15db70 2024-04-25T08:33:12.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.979+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c1f3fb0 con 0x7f3d0c15db70 2024-04-25T08:33:12.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.980+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c073990 con 0x7f3d0c15db70 2024-04-25T08:33:12.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.980+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d08014bc0 con 0x7f3d0c15db70 2024-04-25T08:33:12.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.981+0000 7f3d01ffb640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cf006b4d0 0x7f3cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:12.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.981+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d0808abc0 con 0x7f3d0c15db70 2024-04-25T08:33:12.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.981+0000 7f3d1151a640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cf006b4d0 0x7f3cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:12.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.981+0000 7f3d1151a640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cf006b4d0 0x7f3cf006d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3d04034000 tx=0x7f3d0402c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:12.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:12.985+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d08059940 con 0x7f3d0c15db70 2024-04-25T08:33:13.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.201+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d0c06b210 con 0x7f3d0c15db70 2024-04-25T08:33:13.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.202+0000 7f3d01ffb640 1 -- 172.21.15.60:0/246412149 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f3d08059240 con 0x7f3d0c15db70 2024-04-25T08:33:13.197 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:13.197 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1884848904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1884848904},{"type":"v1","addr":"172.21.15.60:6835","nonce":1884848904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1120004533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1120004533},{"type":"v1","addr":"172.21.15.100:6835","nonce":1120004533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2249277423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2249277423},{"type":"v1","addr":"172.21.15.100:6837","nonce":2249277423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:10.464973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6834/1590331682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1590331682},{"type":"v1","addr":"172.21.15.60:6834","nonce":1590331682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:13.197 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 35 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cf006b4d0 msgr2=0x7f3cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cf006b4d0 0x7f3cf006d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3d04034000 tx=0x7f3d0402c000 comp rx=0 tx=0).stop 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 msgr2=0x7f3d0c151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c151ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3d08007e60 tx=0x7f3d0800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 shutdown_connections 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d0c15db70 0x7f3d0c151ad0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d0c0b6cb0 0x7f3d0c1f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cf006b4d0 0x7f3cf006d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 --2- 172.21.15.60:0/246412149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d0c0b5a60 0x7f3d0c1f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 >> 172.21.15.60:0/246412149 conn(0x7f3d0c0d0520 msgr2=0x7f3d0c0b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 shutdown_connections 2024-04-25T08:33:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.204+0000 7f3d1377e640 1 -- 172.21.15.60:0/246412149 wait complete. 2024-04-25T08:33:13.399 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:33:13.404+0000 7f9e519a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:33:13.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:33:07.899265+0000', 'modified': '2024-04-25T08:33:10.464973+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6699}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6699': {'gid': 6699, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6834/1590331682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 1590331682}, {'type': 'v1', 'addr': '172.21.15.60:6834', 'nonce': 1590331682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:33:13.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:33:13.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:33:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.724+0000 7fa21488f640 1 Processor -- start 2024-04-25T08:33:13.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.724+0000 7fa21488f640 1 -- start start 2024-04-25T08:33:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa21488f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa21015da90 0x7fa210161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa21488f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2100b9750 0x7fa2100ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa21488f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa210162040 con 0x7fa210161ba0 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa21488f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa210162600 con 0x7fa2100b9750 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa21488f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa210162320 con 0x7fa21015da90 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2100b9750 0x7fa2100ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa2100b9750 0x7fa2100ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56456/0 (socket says 172.21.15.60:56456) 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20dd9b640 1 -- 172.21.15.60:0/3416826437 learned_addr learned my addr 172.21.15.60:0/3416826437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20e59c640 1 --2- 172.21.15.60:0/3416826437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa21015da90 0x7fa210161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20dd9b640 1 -- 172.21.15.60:0/3416826437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa21015da90 msgr2=0x7fa210161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/3416826437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa21015da90 0x7fa210161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.725+0000 7fa20dd9b640 1 -- 172.21.15.60:0/3416826437 >> v1:172.21.15.100:6789/0 conn(0x7fa210161ba0 legacy=0x7fa21006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.726+0000 7fa20dd9b640 1 -- 172.21.15.60:0/3416826437 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100d1b10 con 0x7fa2100b9750 2024-04-25T08:33:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.726+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/3416826437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2100b9750 0x7fa2100ce670 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bb929c0c40f063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.726+0000 7fa20d59a640 1 -- 172.21.15.60:0/3416826437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8037070 con 0x7fa2100b9750 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.726+0000 7fa20d59a640 1 -- 172.21.15.60:0/3416826437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8032440 con 0x7fa2100b9750 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.726+0000 7fa20d59a640 1 -- 172.21.15.60:0/3416826437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8032810 con 0x7fa2100b9750 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 -- 172.21.15.60:0/3416826437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2100b9750 msgr2=0x7fa2100ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 --2- 172.21.15.60:0/3416826437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2100b9750 0x7fa2100ce670 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009870 tx=0x7fa1f802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 -- 172.21.15.60:0/3416826437 shutdown_connections 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 --2- 172.21.15.60:0/3416826437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2100b9750 0x7fa2100ce670 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 --2- 172.21.15.60:0/3416826437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa21015da90 0x7fa210161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 -- 172.21.15.60:0/3416826437 >> 172.21.15.60:0/3416826437 conn(0x7fa210061a20 msgr2=0x7fa210061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 -- 172.21.15.60:0/3416826437 shutdown_connections 2024-04-25T08:33:13.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.727+0000 7fa21488f640 1 -- 172.21.15.60:0/3416826437 wait complete. 2024-04-25T08:33:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.728+0000 7fa21488f640 1 Processor -- start 2024-04-25T08:33:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.728+0000 7fa21488f640 1 -- start start 2024-04-25T08:33:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa21488f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 0x7fa2100c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa21488f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa210161ba0 0x7fa2101529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa21488f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2100b50a0 0x7fa2100b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa21488f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100d3320 con 0x7fa21015da90 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa21488f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21015de70 con 0x7fa2100b50a0 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa21488f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2100d34a0 con 0x7fa210161ba0 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 0x7fa2100c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 0x7fa2100c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56462/0 (socket says 172.21.15.60:56462) 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20e59c640 1 -- 172.21.15.60:0/1174837699 learned_addr learned my addr 172.21.15.60:0/1174837699 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa210161ba0 0x7fa2101529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2100b50a0 0x7fa2100b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20e59c640 1 -- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa210161ba0 msgr2=0x7fa2101529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20e59c640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa210161ba0 0x7fa2101529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.729+0000 7fa20e59c640 1 -- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2100b50a0 msgr2=0x7fa2100b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa20e59c640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2100b50a0 0x7fa2100b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa20e59c640 1 -- 172.21.15.60:0/1174837699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100c9970 con 0x7fa21015da90 2024-04-25T08:33:13.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2100b50a0 0x7fa2100b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa20e59c640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 0x7fa2100c9430 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa20800d800 tx=0x7fa20800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa206ffd640 1 -- 172.21.15.60:0/1174837699 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20800b6c0 con 0x7fa21015da90 2024-04-25T08:33:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa206ffd640 1 -- 172.21.15.60:0/1174837699 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208019070 con 0x7fa21015da90 2024-04-25T08:33:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2100c83d0 con 0x7fa21015da90 2024-04-25T08:33:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa206ffd640 1 -- 172.21.15.60:0/1174837699 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20800ba80 con 0x7fa21015da90 2024-04-25T08:33:13.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.730+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2100c8950 con 0x7fa21015da90 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.731+0000 7fa206ffd640 1 -- 172.21.15.60:0/1174837699 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa20800bc20 con 0x7fa21015da90 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.732+0000 7fa206ffd640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.732+0000 7fa206ffd640 1 -- 172.21.15.60:0/1174837699 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa20808a7b0 con 0x7fa21015da90 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.732+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.733+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1f006b4d0 0x7fa1f006d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8034000 tx=0x7fa1f802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.733+0000 7fa21758f740 1 Processor -- start 2024-04-25T08:33:13.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.733+0000 7fa21758f740 1 -- start start 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa21758f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456d9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456f6ad0 0x55fa456f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456aef10 0x55fa456f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa21758f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa45611a10 con 0x55fa456d9230 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa455bab30 con 0x55fa456f6ad0 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa4560dc50 con 0x55fa456aef10 2024-04-25T08:33:13.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456d9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56470/0 (socket says 172.21.15.60:56470) 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 -- 172.21.15.60:0/132558382 learned_addr learned my addr 172.21.15.60:0/132558382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456aef10 0x55fa456f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456f6ad0 0x55fa456f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 -- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456aef10 msgr2=0x55fa456f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456aef10 0x55fa456f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 -- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456f6ad0 msgr2=0x55fa456f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456f6ad0 0x55fa456f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 -- 172.21.15.60:0/132558382 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa455cfe10 con 0x55fa456d9230 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.734+0000 7fa20e59c640 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456d9610 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa208060f70 tx=0x7fa208058820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d8e857c3bef27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa1effff640 1 -- 172.21.15.60:0/132558382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20805cee0 con 0x55fa456d9230 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa1effff640 1 -- 172.21.15.60:0/132558382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2080614a0 con 0x55fa456d9230 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa1effff640 1 -- 172.21.15.60:0/132558382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208057090 con 0x55fa456d9230 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 -- 172.21.15.60:0/132558382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 msgr2=0x55fa456d9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456d9610 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa208060f70 tx=0x7fa208058820 comp rx=0 tx=0).stop 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 -- 172.21.15.60:0/132558382 shutdown_connections 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456aef10 0x55fa456f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456f6ad0 0x55fa456f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 --2- 172.21.15.60:0/132558382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456d9610 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 -- 172.21.15.60:0/132558382 >> 172.21.15.60:0/132558382 conn(0x55fa456ab840 msgr2=0x55fa456f4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 -- 172.21.15.60:0/132558382 shutdown_connections 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.735+0000 7fa21758f740 1 -- 172.21.15.60:0/132558382 wait complete. 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.736+0000 7fa21758f740 1 -- start start 2024-04-25T08:33:13.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.736+0000 7fa21758f740 1 Processor -- start 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa456a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456d7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa456afcb0 con 0x55fa456d9230 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa455ce0c0 con 0x55fa456a8c60 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa456a9770 con 0x55fa456f6ad0 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa21758f740 10 client.0 _ll_register_callbacks cb 0x55fa455c1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456dc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56484/0 (socket says 172.21.15.60:56484) 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 -- 172.21.15.60:0/3021789443 learned_addr learned my addr 172.21.15.60:0/3021789443 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa456a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456d7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 -- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 msgr2=0x55fa456a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa456a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 -- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 msgr2=0x55fa456d7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456d7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20e59c640 1 -- 172.21.15.60:0/3021789443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa453262d0 con 0x55fa456d9230 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa456a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.737+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa20e59c640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456dc330 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa20800d7d0 tx=0x7fa20808a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd105a96087043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa1ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/3021789443 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208088050 con 0x55fa456d9230 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/3021789443 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa20805f0c0 con 0x55fa456d9230 2024-04-25T08:33:13.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2080614a0 con 0x55fa456d9230 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/3021789443 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa20807e050 con 0x55fa456d9230 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 10 client.6828 fetch_fsmap learned FSMap version 35 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55fa453f06b0 con 0x55fa456d9230 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/3021789443 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2080614a0 con 0x55fa456d9230 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/3021789443 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa208013070 con 0x55fa456d9230 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 10 client.6828 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 10 client.6828 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 20 client.6828 populate_metadata read hostname 'smithi060' 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa45704ac0 0x55fa45704ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 4 client.6828 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 --> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa45670920 con 0x55fa45704ac0 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.738+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa45704ac0 0x55fa45704ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa1ed7fa640 10 client.6828 ms_handle_connect on v2:172.21.15.60:6832/1590331682 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa45704ac0 0x55fa45704ea0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2f3f22dcf9f85d server_cookie=42808c8d01161292 in_seq=0 out_seq=0 2024-04-25T08:33:13.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/3021789443 <== mds.0 v2:172.21.15.60:6832/1590331682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa45670920 con 0x55fa45704ac0 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa1ed7fa640 10 client.6828 handle_command_reply: tid=0 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 client.6828 shutdown 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 msgr2=0x55fa456dc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456dc330 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa20800d7d0 tx=0x7fa20808a040 comp rx=0 tx=0).stop 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 shutdown_connections 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa45704ac0 0x55fa45704ea0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa456a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 --2- 172.21.15.60:0/3021789443 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456dc330 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa1ed7fa640 0 client.6828 ms_handle_reset on v2:172.21.15.60:6832/1590331682 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.739+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 >> 172.21.15.60:0/3021789443 conn(0x55fa456aef10 msgr2=0x55fa456f75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.740+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 shutdown_connections 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.740+0000 7fa21758f740 1 -- 172.21.15.60:0/3021789443 wait complete. 2024-04-25T08:33:13.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.740+0000 7fa21758f740 20 client.6828 trim_cache size 0 max 16384 2024-04-25T08:33:13.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 Processor -- start 2024-04-25T08:33:13.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 -- start start 2024-04-25T08:33:13.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa45706d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2101ff6e0 con 0x55fa456d9230 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21015e9a0 con 0x55fa456a8c60 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2101ff130 con 0x55fa456f6ad0 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa20dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa20dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456eaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56494/0 (socket says 172.21.15.60:56494) 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.756+0000 7fa20dd9b640 1 -- 172.21.15.60:0/2469462444 learned_addr learned my addr 172.21.15.60:0/2469462444 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa45706d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20e59c640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20dd9b640 1 -- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 msgr2=0x55fa45706d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa45706d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20dd9b640 1 -- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 msgr2=0x55fa456ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20dd9b640 1 -- 172.21.15.60:0/2469462444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa456d5040 con 0x55fa456d9230 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa45706d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:13.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20e59c640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456eaff0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa1f803e040 tx=0x7fa1f8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10703d44300a24c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa1ef7fe640 1 -- 172.21.15.60:0/2469462444 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8057070 con 0x55fa456d9230 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa1ef7fe640 1 -- 172.21.15.60:0/2469462444 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8032e20 con 0x55fa456d9230 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.757+0000 7fa1ef7fe640 1 -- 172.21.15.60:0/2469462444 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8037040 con 0x55fa456d9230 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 -- 172.21.15.60:0/2469462444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 msgr2=0x55fa456eaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456eaff0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa1f803e040 tx=0x7fa1f8008030 comp rx=0 tx=0).stop 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 -- 172.21.15.60:0/2469462444 shutdown_connections 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456f6ad0 0x55fa45706d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456d9230 0x55fa456eaff0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 --2- 172.21.15.60:0/2469462444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456a8c60 0x55fa456ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 -- 172.21.15.60:0/2469462444 >> 172.21.15.60:0/2469462444 conn(0x55fa456aef10 msgr2=0x55fa456a4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 -- 172.21.15.60:0/2469462444 shutdown_connections 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 -- 172.21.15.60:0/2469462444 wait complete. 2024-04-25T08:33:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 -- start start 2024-04-25T08:33:13.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.758+0000 7fa21758f740 1 Processor -- start 2024-04-25T08:33:13.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456a8c60 0x55fa45761880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456d9230 0x55fa456da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 0x55fa456da9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa456a90e0 con 0x55fa456f6ad0 2024-04-25T08:33:13.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa456afcb0 con 0x55fa456d9230 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa456ec740 con 0x55fa456a8c60 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa21758f740 10 client.0 _ll_register_callbacks cb 0x55fa456a9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 0x55fa456da9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 0x55fa456da9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56498/0 (socket says 172.21.15.60:56498) 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 -- 172.21.15.60:0/2964253790 learned_addr learned my addr 172.21.15.60:0/2964253790 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20e59c640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456a8c60 0x55fa45761880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20dd9b640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456d9230 0x55fa456da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 -- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456a8c60 msgr2=0x55fa45761880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456a8c60 0x55fa45761880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 -- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456d9230 msgr2=0x55fa456da4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.759+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456d9230 0x55fa456da4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa20ed9d640 1 -- 172.21.15.60:0/2964253790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa45707740 con 0x55fa456f6ad0 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa20ed9d640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 0x55fa456da9f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa200001f50 tx=0x7fa200005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502629bd32652071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa1ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/2964253790 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20001a020 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/2964253790 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa200016600 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1f8032e20 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/2964253790 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa200024050 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 10 client.6834 fetch_fsmap learned FSMap version 35 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55fa456eb4c0 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/2964253790 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8032e20 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/2964253790 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa200027020 con 0x55fa456f6ad0 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 10 client.6834 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 10 client.6834 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 20 client.6834 populate_metadata read hostname 'smithi060' 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa456e99d0 0x55fa456e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 4 client.6834 mds_command: new command op to 6699 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:33:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 --> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fa4561ef50 con 0x55fa456e99d0 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.760+0000 7fa20e59c640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa456e99d0 0x55fa456e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa1ed7fa640 10 client.6834 ms_handle_connect on v2:172.21.15.60:6832/1590331682 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa20e59c640 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa456e99d0 0x55fa456e9db0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5597a6d35835d208 server_cookie=c9e010328311e5a9 in_seq=0 out_seq=0 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa1ed7fa640 1 -- 172.21.15.60:0/2964253790 <== mds.0 v2:172.21.15.60:6832/1590331682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fa4561ef50 con 0x55fa456e99d0 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa1ed7fa640 10 client.6834 handle_command_reply: tid=0 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 client.6834 shutdown 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 msgr2=0x55fa456da9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 0x55fa456da9f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa200001f50 tx=0x7fa200005420 comp rx=0 tx=0).stop 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 shutdown_connections 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fa456f6ad0 0x55fa456da9f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fa456d9230 0x55fa456da4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.60:6832/1590331682,v1:172.21.15.60:6834/1590331682] conn(0x55fa456e99d0 0x55fa456e9db0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 --2- 172.21.15.60:0/2964253790 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fa456a8c60 0x55fa45761880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 >> 172.21.15.60:0/2964253790 conn(0x55fa456aef10 msgr2=0x55fa456ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa1ed7fa640 0 client.6834 ms_handle_reset on v2:172.21.15.60:6832/1590331682 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 shutdown_connections 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 1 -- 172.21.15.60:0/2964253790 wait complete. 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.761+0000 7fa21758f740 20 client.6834 trim_cache size 0 max 16384 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6699, 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:33:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 5.2312279979999996, 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 35, 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 67, 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 67, 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 16.357266546999998 2024-04-25T08:33:13.764 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1f006b4d0 msgr2=0x7fa1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1f006b4d0 0x7fa1f006d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8034000 tx=0x7fa1f802e000 comp rx=0 tx=0).stop 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 msgr2=0x7fa2100c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 0x7fa2100c9430 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa20800d800 tx=0x7fa20800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 shutdown_connections 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa1f006b4d0 0x7fa1f006d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2100b50a0 0x7fa2100b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa210161ba0 0x7fa2101529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 --2- 172.21.15.60:0/1174837699 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21015da90 0x7fa2100c9430 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 >> 172.21.15.60:0/1174837699 conn(0x7fa210061a20 msgr2=0x7fa2100b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 shutdown_connections 2024-04-25T08:33:13.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.762+0000 7fa21488f640 1 -- 172.21.15.60:0/1174837699 wait complete. 2024-04-25T08:33:13.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:33:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.873+0000 7f3ed239e640 1 Processor -- start 2024-04-25T08:33:13.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.873+0000 7f3ed239e640 1 -- start start 2024-04-25T08:33:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed239e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ecc15db70 0x7f3ecc162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed239e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed239e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ecc0b5a60 0x7f3ecc06c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed239e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc162690 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed239e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc0ceee0 con 0x7f3ecc15db70 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed239e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0d1d10 con 0x7f3ecc0b6cb0 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ecc0b5a60 0x7f3ecc06c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ecc0b5a60 0x7f3ecc06c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56508/0 (socket says 172.21.15.60:56508) 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 -- 172.21.15.60:0/434739821 learned_addr learned my addr 172.21.15.60:0/434739821 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ecbfff640 1 --2- 172.21.15.60:0/434739821 >> v2:172.21.15.100:3300/0 conn(0x7f3ecc15db70 0x7f3ecc162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ecb7fe640 1 --2- 172.21.15.60:0/434739821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 -- 172.21.15.60:0/434739821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 msgr2=0x7f3ecc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 --2- 172.21.15.60:0/434739821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 -- 172.21.15.60:0/434739821 >> v2:172.21.15.100:3300/0 conn(0x7f3ecc15db70 msgr2=0x7f3ecc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 --2- 172.21.15.60:0/434739821 >> v2:172.21.15.100:3300/0 conn(0x7f3ecc15db70 0x7f3ecc162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.874+0000 7f3ed093b640 1 -- 172.21.15.60:0/434739821 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0d1ff0 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.875+0000 7f3ed093b640 1 --2- 172.21.15.60:0/434739821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc06c2d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0b6570 tx=0x7f3ec402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f7f984547f918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.875+0000 7f3ecaffd640 1 -- 172.21.15.60:0/434739821 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4036070 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.875+0000 7f3ecaffd640 1 -- 172.21.15.60:0/434739821 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4031600 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.875+0000 7f3ecaffd640 1 -- 172.21.15.60:0/434739821 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40319d0 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 -- 172.21.15.60:0/434739821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 msgr2=0x7f3ecc06c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 --2- 172.21.15.60:0/434739821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc06c2d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0b6570 tx=0x7f3ec402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 -- 172.21.15.60:0/434739821 shutdown_connections 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 --2- 172.21.15.60:0/434739821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc06c2d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 --2- 172.21.15.60:0/434739821 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 --2- 172.21.15.60:0/434739821 >> v2:172.21.15.100:3300/0 conn(0x7f3ecc15db70 0x7f3ecc162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 -- 172.21.15.60:0/434739821 >> 172.21.15.60:0/434739821 conn(0x7f3ecc0d0520 msgr2=0x7f3ecc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 -- 172.21.15.60:0/434739821 shutdown_connections 2024-04-25T08:33:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.876+0000 7f3ed239e640 1 -- 172.21.15.60:0/434739821 wait complete. 2024-04-25T08:33:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.877+0000 7f3ed239e640 1 Processor -- start 2024-04-25T08:33:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.877+0000 7f3ed239e640 1 -- start start 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed239e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed239e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc1f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed239e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ecc15db70 0x7f3ecc0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed239e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0ce530 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed239e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc15df50 con 0x7f3ecc15db70 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed239e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0cf1c0 con 0x7f3ecc0b6cb0 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc1558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56516/0 (socket says 172.21.15.60:56516) 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 -- 172.21.15.60:0/3687805324 learned_addr learned my addr 172.21.15.60:0/3687805324 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecb7fe640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc1f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed093b640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ecc15db70 0x7f3ecc0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 -- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 msgr2=0x7f3ecc1f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc1f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 -- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ecc15db70 msgr2=0x7f3ecc0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ecc15db70 0x7f3ecc0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ecbfff640 1 -- 172.21.15.60:0/3687805324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0cc0f0 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.878+0000 7f3ed093b640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ecc15db70 0x7f3ecc0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.879+0000 7f3ecbfff640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc1558e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00e870 tx=0x7f3ebc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.879+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc017070 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.879+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc00c940 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.879+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc160e50 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.879+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00cd30 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.879+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc15e4c0 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.880+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc073990 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.880+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ebc01a070 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.881+0000 7f3ec8ff9640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3eb806b4d0 0x7f3eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.881+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ebc08a1a0 con 0x7f3ecc0b5a60 2024-04-25T08:33:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.881+0000 7f3ecb7fe640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3eb806b4d0 0x7f3eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.881+0000 7f3ecb7fe640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3eb806b4d0 0x7f3eb806d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc1f7ec0 tx=0x7f3ec0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:13.885+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ebc058840 con 0x7f3ecc0b5a60 2024-04-25T08:33:14.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.103+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ecc155e20 con 0x7f3ecc0b5a60 2024-04-25T08:33:14.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.103+0000 7f3ec8ff9640 1 -- 172.21.15.60:0/3687805324 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ebc05c8e0 con 0x7f3ecc0b5a60 2024-04-25T08:33:14.098 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:33:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3eb806b4d0 msgr2=0x7f3eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3eb806b4d0 0x7f3eb806d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc1f7ec0 tx=0x7f3ec0009190 comp rx=0 tx=0).stop 2024-04-25T08:33:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 msgr2=0x7f3ecc1558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:14.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc1558e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00e870 tx=0x7f3ebc00ed30 comp rx=0 tx=0).stop 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 shutdown_connections 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3eb806b4d0 0x7f3eb806d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ecc15db70 0x7f3ecc0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ecc0b6cb0 0x7f3ecc1f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 --2- 172.21.15.60:0/3687805324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ecc0b5a60 0x7f3ecc1558e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 >> 172.21.15.60:0/3687805324 conn(0x7f3ecc0d0520 msgr2=0x7f3ecc0b8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 shutdown_connections 2024-04-25T08:33:14.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:14.105+0000 7f3ed239e640 1 -- 172.21.15.60:0/3687805324 wait complete. 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:33:14.112 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:33:14.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.113 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:33:14.128 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:33:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:33:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:33:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:33:14.128 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42516sec preferred_lft 42516sec 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:14.129 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:33:14.130 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:33:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:14.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.131 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:33:14.191 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:33:14.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.193 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:33:14.248 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:33:14.248 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:33:14.248 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:33:14.248 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:33:14.249 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:33:14.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.249 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:33:14.249 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:33:14.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:33:14.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:33:14.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:33:14.249 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:33:14.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.335 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:33:14.350 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:33:14.350 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:33:14.351 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:33:14.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:33:14.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:33:14.415 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:14.471 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:33:14.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:33:14.471 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:14.526 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:33:14.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:14.609 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:14.609 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:33:14.609 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:33:14.617 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:14.617 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:33:14.617 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:33:14.617 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:33:14.618 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:33:14.623 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:14.623 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:33:14.624 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:33:14.625 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:33:14.626 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:33:14.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:14.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.691 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:33:14.751 INFO:teuthology.orchestra.run.smithi060.stdout:client5732 2024-04-25T08:33:14.752 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:14.752 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:14.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.807 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T08:33:14.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:14.867 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T08:33:14.924 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:33:14.924 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:33:14.924 DEBUG:teuthology.orchestra.run.smithi060:> import stat 2024-04-25T08:33:14.924 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:33:14.924 DEBUG:teuthology.orchestra.run.smithi060:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T08:33:14.924 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:33:15.029 INFO:teuthology.orchestra.run.smithi060.stdout:1099511627776 2024-04-25T08:33:15.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:33:15.038 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:15.039 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:15.094 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:15.094 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:15.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:15.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.296+0000 7f443b948640 1 Processor -- start 2024-04-25T08:33:15.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.297+0000 7f443b948640 1 -- start start 2024-04-25T08:33:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.297+0000 7f443b948640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f443415dc70 0x7f44340d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.297+0000 7f443b948640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 0x7f44340cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.297+0000 7f443b948640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443406c220 con 0x7f443406c700 2024-04-25T08:33:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.297+0000 7f443b948640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443409a8b0 con 0x7f443415dc70 2024-04-25T08:33:15.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.297+0000 7f443b948640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44340d0c90 con 0x7f443409a450 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f4439ee5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f443406c700 0x7f44340cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59456/0 (socket says 172.21.15.60:59456) 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f4439ee5640 1 -- 172.21.15.60:0/1847489184 learned_addr learned my addr 172.21.15.60:0/1847489184 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f44396e4640 1 --2- 172.21.15.60:0/1847489184 >> v2:172.21.15.100:3300/0 conn(0x7f443415dc70 0x7f44340d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207326617 0 0) 0x7f443406c220 con 0x7f443406c700 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f4438ee3640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 0x7f44340cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4420003520 con 0x7f443406c700 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4263469430 0 0) 0x7f4420003520 con 0x7f443406c700 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 msgr2=0x7f44340cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 0x7f44340cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 >> v2:172.21.15.100:3300/0 conn(0x7f443415dc70 msgr2=0x7f44340d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 --2- 172.21.15.60:0/1847489184 >> v2:172.21.15.100:3300/0 conn(0x7f443415dc70 0x7f44340d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340d0080 con 0x7f443406c700 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f442c0019f0 con 0x7f443406c700 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 >> v1:172.21.15.60:6789/0 conn(0x7f443406c700 legacy=0x7f44340cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 0x7f44200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4420003f30 0x7f4420004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4420004d40 0x7f44200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443409a8b0 con 0x7f4420004d40 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44340d0c90 con 0x7f4420003f30 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442000b9c0 con 0x7f443409a450 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f4439ee5640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4420004d40 0x7f44200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f44396e4640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 0x7f44200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.298+0000 7f4438ee3640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4420003f30 0x7f4420004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f4439ee5640 1 -- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 msgr2=0x7f44200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f4439ee5640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f443409a450 0x7f44200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f4439ee5640 1 -- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4420003f30 msgr2=0x7f4420004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f4439ee5640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4420003f30 0x7f4420004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f4439ee5640 1 -- 172.21.15.60:0/1847489184 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420003520 con 0x7f4420004d40 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f4439ee5640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4420004d40 0x7f44200051c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f442c00a920 tx=0x7f4420005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc8ed2819720e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.299+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00a4c0 con 0x7f4420004d40 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c02f070 con 0x7f4420004d40 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f442bfff640 1 -- 172.21.15.60:0/1847489184 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c02bd60 con 0x7f4420004d40 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f443b948640 1 -- 172.21.15.60:0/1847489184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4420004d40 msgr2=0x7f44200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f443b948640 1 --2- 172.21.15.60:0/1847489184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4420004d40 0x7f44200051c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f442c00a920 tx=0x7f4420005830 comp rx=0 tx=0).stop 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f4439ee5640 1 -- 172.21.15.60:0/1847489184 reap_dead start 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f443b948640 1 -- 172.21.15.60:0/1847489184 shutdown_connections 2024-04-25T08:33:15.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f443b948640 1 -- 172.21.15.60:0/1847489184 >> 172.21.15.60:0/1847489184 conn(0x7f44340b3270 msgr2=0x7f44340b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:15.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f443b948640 1 -- 172.21.15.60:0/1847489184 shutdown_connections 2024-04-25T08:33:15.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.300+0000 7f443b948640 1 -- 172.21.15.60:0/1847489184 wait complete. 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.301+0000 7f443b948640 1 Processor -- start 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.301+0000 7f443b948640 1 -- start start 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f443b948640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4420004d40 0x7f443415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f443b948640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340c81b0 0x7f44340b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f443b948640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 0x7f44341f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f443b948640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44340d0920 con 0x7f44340c6440 2024-04-25T08:33:15.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f443b948640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44340d0640 con 0x7f44340c81b0 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f443b948640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443415e050 con 0x7f4420004d40 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f4439ee5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 0x7f44341f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f4439ee5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 0x7f44341f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56524/0 (socket says 172.21.15.60:56524) 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f4439ee5640 1 -- 172.21.15.60:0/3327922157 learned_addr learned my addr 172.21.15.60:0/3327922157 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f44396e4640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4420004d40 0x7f443415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f4438ee3640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340c81b0 0x7f44340b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f4439ee5640 1 -- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4420004d40 msgr2=0x7f443415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.302+0000 7f4439ee5640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4420004d40 0x7f443415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4439ee5640 1 -- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340c81b0 msgr2=0x7f44340b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4439ee5640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340c81b0 0x7f44340b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4439ee5640 1 -- 172.21.15.60:0/3327922157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443415c870 con 0x7f44340c6440 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4438ee3640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340c81b0 0x7f44340b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:15.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4439ee5640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 0x7f44341f3de0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f442c02cae0 tx=0x7f44340d0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c044070 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c02ce50 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44340c53a0 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00a4c0 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.303+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44340c5960 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.304+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4434073990 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.304+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f442c047070 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.305+0000 7f4429ffb640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f441806b4d0 0x7f441806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.305+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f442c0aa420 con 0x7f44340c6440 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.305+0000 7f44396e4640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f441806b4d0 0x7f441806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.306+0000 7f44396e4640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f441806b4d0 0x7f441806d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4430009c40 tx=0x7f4430009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.309+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f442c079130 con 0x7f44340c6440 2024-04-25T08:33:15.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.492+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44340cb880 con 0x7f44340c6440 2024-04-25T08:33:15.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.493+0000 7f4429ffb640 1 -- 172.21.15.60:0/3327922157 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f442c078a30 con 0x7f44340c6440 2024-04-25T08:33:15.488 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:15.489 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":67,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:09.539822+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f441806b4d0 msgr2=0x7f441806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f441806b4d0 0x7f441806d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4430009c40 tx=0x7f4430009190 comp rx=0 tx=0).stop 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 msgr2=0x7f44341f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 0x7f44341f3de0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f442c02cae0 tx=0x7f44340d0c00 comp rx=0 tx=0).stop 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 shutdown_connections 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44340c6440 0x7f44341f3de0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f441806b4d0 0x7f441806d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44340c81b0 0x7f44340b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 --2- 172.21.15.60:0/3327922157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4420004d40 0x7f443415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 >> 172.21.15.60:0/3327922157 conn(0x7f44340b3270 msgr2=0x7f44340b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:15.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 shutdown_connections 2024-04-25T08:33:15.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.496+0000 7f443b948640 1 -- 172.21.15.60:0/3327922157 wait complete. 2024-04-25T08:33:15.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:33:06.174848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:33:06.884806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:33:15.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:33:15.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.612+0000 7f6d45514640 1 Processor -- start 2024-04-25T08:33:15.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d45514640 1 -- start start 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d45514640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6d4015dc20 0x7f6d400ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d45514640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d40161710 0x7f6d400cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d45514640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4015e000 con 0x7f6d400b9330 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d45514640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d400d1de0 con 0x7f6d40161710 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d45514640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d40161c60 con 0x7f6d4015dc20 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d3e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d40161710 0x7f6d400cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d3e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6d40161710 0x7f6d400cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56536/0 (socket says 172.21.15.60:56536) 2024-04-25T08:33:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.613+0000 7f6d3e7fc640 1 -- 172.21.15.60:0/4009242511 learned_addr learned my addr 172.21.15.60:0/4009242511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.614+0000 7f6d3effd640 1 --2- 172.21.15.60:0/4009242511 >> v2:172.21.15.100:3300/0 conn(0x7f6d4015dc20 0x7f6d400ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.614+0000 7f6d3e7fc640 1 -- 172.21.15.60:0/4009242511 >> v2:172.21.15.100:3300/0 conn(0x7f6d4015dc20 msgr2=0x7f6d400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.614+0000 7f6d3e7fc640 1 --2- 172.21.15.60:0/4009242511 >> v2:172.21.15.100:3300/0 conn(0x7f6d4015dc20 0x7f6d400ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.614+0000 7f6d3e7fc640 1 -- 172.21.15.60:0/4009242511 >> v1:172.21.15.100:6789/0 conn(0x7f6d400b9330 legacy=0x7f6d4006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.614+0000 7f6d3e7fc640 1 -- 172.21.15.60:0/4009242511 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40161f40 con 0x7f6d40161710 2024-04-25T08:33:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.614+0000 7f6d3e7fc640 1 --2- 172.21.15.60:0/4009242511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d400cebb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6d400d1520 tx=0x7f6d3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af30dfbaa69fdc4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d3dffb640 1 -- 172.21.15.60:0/4009242511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34037070 con 0x7f6d40161710 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d3dffb640 1 -- 172.21.15.60:0/4009242511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34032440 con 0x7f6d40161710 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d3dffb640 1 -- 172.21.15.60:0/4009242511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34032810 con 0x7f6d40161710 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 -- 172.21.15.60:0/4009242511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 msgr2=0x7f6d400cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 --2- 172.21.15.60:0/4009242511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d400cebb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6d400d1520 tx=0x7f6d3402a270 comp rx=0 tx=0).stop 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 -- 172.21.15.60:0/4009242511 shutdown_connections 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 --2- 172.21.15.60:0/4009242511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d400cebb0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 --2- 172.21.15.60:0/4009242511 >> v2:172.21.15.100:3300/0 conn(0x7f6d4015dc20 0x7f6d400ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 -- 172.21.15.60:0/4009242511 >> 172.21.15.60:0/4009242511 conn(0x7f6d400d0420 msgr2=0x7f6d400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 -- 172.21.15.60:0/4009242511 shutdown_connections 2024-04-25T08:33:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.615+0000 7f6d45514640 1 -- 172.21.15.60:0/4009242511 wait complete. 2024-04-25T08:33:15.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 Processor -- start 2024-04-25T08:33:15.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 -- start start 2024-04-25T08:33:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d400b9330 0x7f6d40158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4015dc20 0x7f6d401e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d4015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b5e50 con 0x7f6d40161710 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40162500 con 0x7f6d400b9330 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.617+0000 7f6d45514640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b5fd0 con 0x7f6d4015dc20 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d4015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d4015ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56538/0 (socket says 172.21.15.60:56538) 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 -- 172.21.15.60:0/1655642052 learned_addr learned my addr 172.21.15.60:0/1655642052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3effd640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d400b9330 0x7f6d40158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3e7fc640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4015dc20 0x7f6d401e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 -- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4015dc20 msgr2=0x7f6d401e7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4015dc20 0x7f6d401e7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 -- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d400b9330 msgr2=0x7f6d40158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d400b9330 0x7f6d40158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 -- 172.21.15.60:0/1655642052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d401e8850 con 0x7f6d40161710 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3effd640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d400b9330 0x7f6d40158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3e7fc640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4015dc20 0x7f6d401e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d3f7fe640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d4015ae80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c730 tx=0x7f6d3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d30003b70 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d30003d30 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d30005310 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d401e7fb0 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.618+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d401e8570 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.619+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40073a00 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.620+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d300054b0 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.620+0000 7f6d277fe640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.621+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6d3008a1a0 con 0x7f6d40161710 2024-04-25T08:33:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.621+0000 7f6d3effd640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.621+0000 7f6d3effd640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009a20 tx=0x7f6d38008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.625+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d401e8850 con 0x7f6d40161710 2024-04-25T08:33:15.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.806+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d400d16b0 con 0x7f6d40161710 2024-04-25T08:33:15.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.807+0000 7f6d277fe640 1 -- 172.21.15.60:0/1655642052 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d30058f00 con 0x7f6d40161710 2024-04-25T08:33:15.802 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d2c06b4d0 msgr2=0x7f6d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009a20 tx=0x7f6d38008040 comp rx=0 tx=0).stop 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 msgr2=0x7f6d4015ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d4015ae80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c730 tx=0x7f6d3000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 shutdown_connections 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d40161710 0x7f6d4015ae80 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6d4015dc20 0x7f6d401e7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6d2c06b4d0 0x7f6d2c06d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 --2- 172.21.15.60:0/1655642052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d400b9330 0x7f6d40158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 >> 172.21.15.60:0/1655642052 conn(0x7f6d400d0420 msgr2=0x7f6d400d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 shutdown_connections 2024-04-25T08:33:15.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.809+0000 7f6d45514640 1 -- 172.21.15.60:0/1655642052 wait complete. 2024-04-25T08:33:15.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:33:15.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 Processor -- start 2024-04-25T08:33:15.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 -- start start 2024-04-25T08:33:15.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f8158230 0x7f38f80d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38f80d1b80 0x7f38f806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80d2020 con 0x7f38f806c870 2024-04-25T08:33:15.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f8158610 con 0x7f38f80d1b80 2024-04-25T08:33:15.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.924+0000 7f38fe45d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f806bea0 con 0x7f38f8158230 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f77fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38f80d1b80 0x7f38f806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f8158230 0x7f38f80d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f8158230 0x7f38f80d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:36218/0 (socket says 172.21.15.60:36218) 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38fc9fa640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f38f806c870 0x7f38f80b4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46084/0 (socket says 172.21.15.60:46084) 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f77fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38f80d1b80 0x7f38f806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:53464/0 (socket says 172.21.15.60:53464) 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38fc9fa640 1 -- 172.21.15.60:0/1099911358 learned_addr learned my addr 172.21.15.60:0/1099911358 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:15.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f77fe640 1 -- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f8158230 msgr2=0x7f38f80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:15.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f77fe640 1 --2- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f8158230 0x7f38f80d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f77fe640 1 -- 172.21.15.60:0/1099911358 >> v1:172.21.15.100:6789/0 conn(0x7f38f806c870 legacy=0x7f38f80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f77fe640 1 -- 172.21.15.60:0/1099911358 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80d2300 con 0x7f38f80d1b80 2024-04-25T08:33:15.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.925+0000 7f38f6ffd640 1 -- 172.21.15.60:0/1099911358 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644528270 0 0) 0x7f38f80d2020 con 0x7f38f806c870 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.926+0000 7f38f77fe640 1 --2- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f80d1b80 0x7f38f806c330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009790 tx=0x7f38ec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d02b17d63ac66c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.926+0000 7f38f6ffd640 1 -- 172.21.15.60:0/1099911358 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec037070 con 0x7f38f80d1b80 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.926+0000 7f38f6ffd640 1 -- 172.21.15.60:0/1099911358 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec032440 con 0x7f38f80d1b80 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.926+0000 7f38f6ffd640 1 -- 172.21.15.60:0/1099911358 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec032810 con 0x7f38f80d1b80 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 -- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f80d1b80 msgr2=0x7f38f806c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f80d1b80 0x7f38f806c330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009790 tx=0x7f38ec02a180 comp rx=0 tx=0).stop 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 -- 172.21.15.60:0/1099911358 shutdown_connections 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f80d1b80 0x7f38f806c330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1099911358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f8158230 0x7f38f80d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 -- 172.21.15.60:0/1099911358 >> 172.21.15.60:0/1099911358 conn(0x7f38f8061600 msgr2=0x7f38f80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 -- 172.21.15.60:0/1099911358 shutdown_connections 2024-04-25T08:33:15.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.927+0000 7f38fe45d640 1 -- 172.21.15.60:0/1099911358 wait complete. 2024-04-25T08:33:15.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.928+0000 7f38fe45d640 1 Processor -- start 2024-04-25T08:33:15.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.928+0000 7f38fe45d640 1 -- start start 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fe45d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 0x7f38f815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fe45d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f80d1b80 0x7f38f80cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fe45d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f8158230 0x7f38f80a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fe45d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f815e320 con 0x7f38f806c870 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fe45d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80d25e0 con 0x7f38f8158230 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fe45d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f815e4a0 con 0x7f38f80d1b80 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 0x7f38f815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 0x7f38f815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56546/0 (socket says 172.21.15.60:56546) 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 -- 172.21.15.60:0/1151815524 learned_addr learned my addr 172.21.15.60:0/1151815524 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38fc9fa640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f8158230 0x7f38f80a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f77fe640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f80d1b80 0x7f38f80cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 -- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f80d1b80 msgr2=0x7f38f80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f80d1b80 0x7f38f80cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 -- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f8158230 msgr2=0x7f38f80a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f8158230 0x7f38f80a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.929+0000 7f38f7fff640 1 -- 172.21.15.60:0/1151815524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80cbe00 con 0x7f38f806c870 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.930+0000 7f38f7fff640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 0x7f38f815b280 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f38e800e9b0 tx=0x7f38e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.930+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8017070 con 0x7f38f806c870 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.930+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e800c9a0 con 0x7f38f806c870 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.930+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e800cd90 con 0x7f38f806c870 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.930+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f80cc0e0 con 0x7f38f806c870 2024-04-25T08:33:15.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.930+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80cc6a0 con 0x7f38f806c870 2024-04-25T08:33:15.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.931+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f806de50 con 0x7f38f806c870 2024-04-25T08:33:15.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.931+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38e801a070 con 0x7f38f806c870 2024-04-25T08:33:15.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.932+0000 7f38f4ff9640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e406b4d0 0x7f38e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:15.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.932+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38e808a670 con 0x7f38f806c870 2024-04-25T08:33:15.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.932+0000 7f38f77fe640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e406b4d0 0x7f38e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:15.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.932+0000 7f38f77fe640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e406b4d0 0x7f38e406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38ec034000 tx=0x7f38ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:15.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:15.936+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38e8058d10 con 0x7f38f806c870 2024-04-25T08:33:16.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.117+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38f8073a00 con 0x7f38f806c870 2024-04-25T08:33:16.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.118+0000 7f38f4ff9640 1 -- 172.21.15.60:0/1151815524 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38e805cdb0 con 0x7f38f806c870 2024-04-25T08:33:16.112 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e406b4d0 msgr2=0x7f38e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e406b4d0 0x7f38e406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38ec034000 tx=0x7f38ec02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 msgr2=0x7f38f815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 0x7f38f815b280 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f38e800e9b0 tx=0x7f38e800ee70 comp rx=0 tx=0).stop 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 shutdown_connections 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e406b4d0 0x7f38e406d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38f8158230 0x7f38f80a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38f80d1b80 0x7f38f80cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 --2- 172.21.15.60:0/1151815524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38f806c870 0x7f38f815b280 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 >> 172.21.15.60:0/1151815524 conn(0x7f38f8061600 msgr2=0x7f38f80b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:16.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 shutdown_connections 2024-04-25T08:33:16.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.120+0000 7f38fe45d640 1 -- 172.21.15.60:0/1151815524 wait complete. 2024-04-25T08:33:16.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:33:16.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.234+0000 7f2f57ce1640 1 Processor -- start 2024-04-25T08:33:16.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.234+0000 7f2f57ce1640 1 -- start start 2024-04-25T08:33:16.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f57ce1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f500cc4f0 0x7f2f500cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f57ce1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f500cdb20 0x7f2f500cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f57ce1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f5015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f57ce1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f50071270 con 0x7f2f500cc4f0 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f57ce1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f50072280 con 0x7f2f500cdb20 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f57ce1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500714c0 con 0x7f2f5015b270 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f55a7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f500cc4f0 0x7f2f500cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f55a7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2f500cc4f0 0x7f2f500cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56550/0 (socket says 172.21.15.60:56550) 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.235+0000 7f2f55a7d640 1 -- 172.21.15.60:0/2500790387 learned_addr learned my addr 172.21.15.60:0/2500790387 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f55a7d640 1 -- 172.21.15.60:0/2500790387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 msgr2=0x7f2f5015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f5627e640 1 --2- 172.21.15.60:0/2500790387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f5015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f5527c640 1 --2- 172.21.15.60:0/2500790387 >> v2:172.21.15.100:3300/0 conn(0x7f2f500cdb20 0x7f2f500cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f55a7d640 1 --2- 172.21.15.60:0/2500790387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f5015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f55a7d640 1 -- 172.21.15.60:0/2500790387 >> v2:172.21.15.100:3300/0 conn(0x7f2f500cdb20 msgr2=0x7f2f500cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f55a7d640 1 --2- 172.21.15.60:0/2500790387 >> v2:172.21.15.100:3300/0 conn(0x7f2f500cdb20 0x7f2f500cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f55a7d640 1 -- 172.21.15.60:0/2500790387 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500717a0 con 0x7f2f500cc4f0 2024-04-25T08:33:16.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f55a7d640 1 --2- 172.21.15.60:0/2500790387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cc8d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2f50158aa0 tx=0x7f2f4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7d96b19e6e7da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.236+0000 7f2f54a7b640 1 -- 172.21.15.60:0/2500790387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48037070 con 0x7f2f500cc4f0 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f54a7b640 1 -- 172.21.15.60:0/2500790387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48032440 con 0x7f2f500cc4f0 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f54a7b640 1 -- 172.21.15.60:0/2500790387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48032810 con 0x7f2f500cc4f0 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 -- 172.21.15.60:0/2500790387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 msgr2=0x7f2f500cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/2500790387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cc8d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f2f50158aa0 tx=0x7f2f4802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 -- 172.21.15.60:0/2500790387 shutdown_connections 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/2500790387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f5015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/2500790387 >> v2:172.21.15.100:3300/0 conn(0x7f2f500cdb20 0x7f2f500cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/2500790387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cc8d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 -- 172.21.15.60:0/2500790387 >> 172.21.15.60:0/2500790387 conn(0x7f2f5006abe0 msgr2=0x7f2f5006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 -- 172.21.15.60:0/2500790387 shutdown_connections 2024-04-25T08:33:16.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.237+0000 7f2f57ce1640 1 -- 172.21.15.60:0/2500790387 wait complete. 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.238+0000 7f2f57ce1640 1 Processor -- start 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.238+0000 7f2f57ce1640 1 -- start start 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f57ce1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f57ce1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500cdb20 0x7f2f500c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f57ce1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f500ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f57ce1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5015bc00 con 0x7f2f500cc4f0 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f57ce1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50071d60 con 0x7f2f500cdb20 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f57ce1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5015bd80 con 0x7f2f5015b270 2024-04-25T08:33:16.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f55a7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f55a7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56552/0 (socket says 172.21.15.60:56552) 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f55a7d640 1 -- 172.21.15.60:0/3388181388 learned_addr learned my addr 172.21.15.60:0/3388181388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f55a7d640 1 -- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 msgr2=0x7f2f500ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f5527c640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500cdb20 0x7f2f500c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.239+0000 7f2f5627e640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f500ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f55a7d640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f500ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f55a7d640 1 -- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500cdb20 msgr2=0x7f2f500c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f55a7d640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500cdb20 0x7f2f500c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f55a7d640 1 -- 172.21.15.60:0/3388181388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500c5990 con 0x7f2f500cc4f0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f5527c640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500cdb20 0x7f2f500c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f5627e640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f500ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f55a7d640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cf980 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2f4802a220 tx=0x7f2f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48037040 con 0x7f2f500cc4f0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48042070 con 0x7f2f500cc4f0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500c5c50 con 0x7f2f500cc4f0 2024-04-25T08:33:16.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48030bb0 con 0x7f2f500cc4f0 2024-04-25T08:33:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.240+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500c6210 con 0x7f2f500cc4f0 2024-04-25T08:33:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.241+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f50073a00 con 0x7f2f500cc4f0 2024-04-25T08:33:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.241+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f48047460 con 0x7f2f500cc4f0 2024-04-25T08:33:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.242+0000 7f2f467fc640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f3406b500 0x7f2f3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.242+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f480ab0b0 con 0x7f2f500cc4f0 2024-04-25T08:33:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.242+0000 7f2f5527c640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f3406b500 0x7f2f3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.243+0000 7f2f5527c640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f3406b500 0x7f2f3406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2f40006fd0 tx=0x7f2f40008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.246+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f48045360 con 0x7f2f500cc4f0 2024-04-25T08:33:16.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.425+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f5006d3a0 con 0x7f2f500cc4f0 2024-04-25T08:33:16.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.426+0000 7f2f467fc640 1 -- 172.21.15.60:0/3388181388 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f48079d00 con 0x7f2f500cc4f0 2024-04-25T08:33:16.421 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f3406b500 msgr2=0x7f2f3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f3406b500 0x7f2f3406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2f40006fd0 tx=0x7f2f40008040 comp rx=0 tx=0).stop 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 msgr2=0x7f2f500cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cf980 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f2f4802a220 tx=0x7f2f480046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 shutdown_connections 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2f5015b270 0x7f2f500ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2f3406b500 0x7f2f3406d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f500cdb20 0x7f2f500c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.427+0000 7f2f57ce1640 1 --2- 172.21.15.60:0/3388181388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f500cc4f0 0x7f2f500cf980 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.428+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 >> 172.21.15.60:0/3388181388 conn(0x7f2f5006abe0 msgr2=0x7f2f50154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.428+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 shutdown_connections 2024-04-25T08:33:16.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.428+0000 7f2f57ce1640 1 -- 172.21.15.60:0/3388181388 wait complete. 2024-04-25T08:33:16.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:33:16.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.541+0000 7ff5021e0640 1 Processor -- start 2024-04-25T08:33:16.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.541+0000 7ff5021e0640 1 -- start start 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff5021e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4fc0cfa40 0x7ff4fc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff5021e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4fc0c75e0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff5021e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4fc0c78c0 con 0x7ff4fc15bb50 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff5021e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4fc0c8680 con 0x7ff4fc0cfa40 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4fbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4fc0cfa40 0x7ff4fc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4faffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff4fc0cf600 0x7ff4fc0cbf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59468/0 (socket says 172.21.15.60:59468) 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4faffd640 1 -- 172.21.15.60:0/956305833 learned_addr learned my addr 172.21.15.60:0/956305833 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4fbfff640 1 --2- 172.21.15.60:0/956305833 >> v2:172.21.15.60:3300/0 conn(0x7ff4fc0cfa40 0x7ff4fc0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56564/0 (socket says 172.21.15.60:56564) 2024-04-25T08:33:16.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4fbfff640 1 -- 172.21.15.60:0/956305833 >> v1:172.21.15.100:6789/0 conn(0x7ff4fc15bb50 legacy=0x7ff4fc0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4fbfff640 1 -- 172.21.15.60:0/956305833 >> v1:172.21.15.60:6789/0 conn(0x7ff4fc0cf600 legacy=0x7ff4fc0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4fbfff640 1 -- 172.21.15.60:0/956305833 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc0c7ba0 con 0x7ff4fc0cfa40 2024-04-25T08:33:16.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.542+0000 7ff4fa7fc640 1 -- 172.21.15.60:0/956305833 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403569284 0 0) 0x7ff4fc0c75e0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.543+0000 7ff4fbfff640 1 --2- 172.21.15.60:0/956305833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cfa40 0x7ff4fc0cc630 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009d90 tx=0x7ff4f402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53389aa005b3452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.543+0000 7ff4fa7fc640 1 -- 172.21.15.60:0/956305833 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4037070 con 0x7ff4fc0cfa40 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.543+0000 7ff4fa7fc640 1 -- 172.21.15.60:0/956305833 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f4007740 con 0x7ff4fc0cfa40 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.543+0000 7ff4fa7fc640 1 -- 172.21.15.60:0/956305833 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4007b10 con 0x7ff4fc0cfa40 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 -- 172.21.15.60:0/956305833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cfa40 msgr2=0x7ff4fc0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 --2- 172.21.15.60:0/956305833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cfa40 0x7ff4fc0cc630 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009d90 tx=0x7ff4f402a780 comp rx=0 tx=0).stop 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 -- 172.21.15.60:0/956305833 shutdown_connections 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 --2- 172.21.15.60:0/956305833 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cfa40 0x7ff4fc0cc630 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 -- 172.21.15.60:0/956305833 >> 172.21.15.60:0/956305833 conn(0x7ff4fc06f500 msgr2=0x7ff4fc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 -- 172.21.15.60:0/956305833 shutdown_connections 2024-04-25T08:33:16.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.544+0000 7ff5021e0640 1 -- 172.21.15.60:0/956305833 wait complete. 2024-04-25T08:33:16.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.545+0000 7ff5021e0640 1 Processor -- start 2024-04-25T08:33:16.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.545+0000 7ff5021e0640 1 -- start start 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff5021e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 0x7ff4fc14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff5021e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4fc0cfa40 0x7ff4fc153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff5021e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4fc15bb50 0x7ff4fc15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff5021e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc0c72a0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff5021e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc0c8160 con 0x7ff4fc15bb50 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff5021e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc0cfe90 con 0x7ff4fc0cfa40 2024-04-25T08:33:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff4fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 0x7ff4fc14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff4fb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 0x7ff4fc14f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56574/0 (socket says 172.21.15.60:56574) 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff4fb7fe640 1 -- 172.21.15.60:0/559433976 learned_addr learned my addr 172.21.15.60:0/559433976 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff4fbfff640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4fc15bb50 0x7ff4fc15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.546+0000 7ff4faffd640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4fc0cfa40 0x7ff4fc153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4fb7fe640 1 -- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4fc0cfa40 msgr2=0x7ff4fc153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4fb7fe640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4fc0cfa40 0x7ff4fc153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4fb7fe640 1 -- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4fc15bb50 msgr2=0x7ff4fc15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4fb7fe640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4fc15bb50 0x7ff4fc15cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4fb7fe640 1 -- 172.21.15.60:0/559433976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc153ac0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4fb7fe640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 0x7ff4fc14f420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c400 tx=0x7ff4ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec017070 con 0x7ff4fc0cf600 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec0052d0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4fc153da0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec003830 con 0x7ff4fc0cf600 2024-04-25T08:33:16.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.547+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4fc14f960 con 0x7ff4fc0cf600 2024-04-25T08:33:16.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.548+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4fc073a00 con 0x7ff4fc0cf600 2024-04-25T08:33:16.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.548+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4ec01a070 con 0x7ff4fc0cf600 2024-04-25T08:33:16.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.549+0000 7ff4e3fff640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4e806b4d0 0x7ff4e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.549+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4ec089640 con 0x7ff4fc0cf600 2024-04-25T08:33:16.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.549+0000 7ff4faffd640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4e806b4d0 0x7ff4e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.550+0000 7ff4faffd640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4e806b4d0 0x7ff4e806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0007f70 tx=0x7ff4f0007f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.554+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4ec057ce0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.746+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff4fc0c6de0 con 0x7ff4fc0cf600 2024-04-25T08:33:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.746+0000 7ff4e3fff640 1 -- 172.21.15.60:0/559433976 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff4ec05bd80 con 0x7ff4fc0cf600 2024-04-25T08:33:16.741 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:33:16.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4e806b4d0 msgr2=0x7ff4e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4e806b4d0 0x7ff4e806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0007f70 tx=0x7ff4f0007f00 comp rx=0 tx=0).stop 2024-04-25T08:33:16.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 msgr2=0x7ff4fc14f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 0x7ff4fc14f420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00c400 tx=0x7ff4ec00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 shutdown_connections 2024-04-25T08:33:16.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4e806b4d0 0x7ff4e806d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4fc15bb50 0x7ff4fc15cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4fc0cfa40 0x7ff4fc153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 --2- 172.21.15.60:0/559433976 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4fc0cf600 0x7ff4fc14f420 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 >> 172.21.15.60:0/559433976 conn(0x7ff4fc06f500 msgr2=0x7ff4fc0d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 shutdown_connections 2024-04-25T08:33:16.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.748+0000 7ff5021e0640 1 -- 172.21.15.60:0/559433976 wait complete. 2024-04-25T08:33:16.754 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:33:16.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:16.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:16.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:33:16.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:16.802 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:33:16.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.971+0000 7f653744d640 1 Processor -- start 2024-04-25T08:33:16.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.971+0000 7f653744d640 1 -- start start 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f653744d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f653006b580 0x7f65300cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f653744d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65300cffa0 0x7f653015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f653744d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6530161960 con 0x7f653006b1a0 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f653744d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65300cbd50 con 0x7f653006b580 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f653744d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65300700d0 con 0x7f65300cffa0 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65351e9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f653006b1a0 0x7f653006f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59478/0 (socket says 172.21.15.60:59478) 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65351e9640 1 -- 172.21.15.60:0/757312820 learned_addr learned my addr 172.21.15.60:0/757312820 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:16.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65349e8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f653006b580 0x7f65300cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65349e8640 1 -- 172.21.15.60:0/757312820 >> v2:172.21.15.100:3300/0 conn(0x7f65300cffa0 msgr2=0x7f653015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65359ea640 1 --2- 172.21.15.60:0/757312820 >> v2:172.21.15.100:3300/0 conn(0x7f65300cffa0 0x7f653015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65349e8640 1 --2- 172.21.15.60:0/757312820 >> v2:172.21.15.100:3300/0 conn(0x7f65300cffa0 0x7f653015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f6527fff640 1 -- 172.21.15.60:0/757312820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234745470 0 0) 0x7f6530161960 con 0x7f653006b1a0 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65349e8640 1 -- 172.21.15.60:0/757312820 >> v1:172.21.15.60:6789/0 conn(0x7f653006b1a0 legacy=0x7f653006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.972+0000 7f65349e8640 1 -- 172.21.15.60:0/757312820 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530161c40 con 0x7f653006b580 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.973+0000 7f65349e8640 1 --2- 172.21.15.60:0/757312820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300cfa60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f65300cbed0 tx=0x7f652002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db43bf370c71dc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.973+0000 7f6527fff640 1 -- 172.21.15.60:0/757312820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520037070 con 0x7f653006b580 2024-04-25T08:33:16.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.973+0000 7f6527fff640 1 -- 172.21.15.60:0/757312820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520032440 con 0x7f653006b580 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.973+0000 7f6527fff640 1 -- 172.21.15.60:0/757312820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652002ae60 con 0x7f653006b580 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 -- 172.21.15.60:0/757312820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 msgr2=0x7f65300cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 --2- 172.21.15.60:0/757312820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300cfa60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f65300cbed0 tx=0x7f652002a110 comp rx=0 tx=0).stop 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 -- 172.21.15.60:0/757312820 shutdown_connections 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 --2- 172.21.15.60:0/757312820 >> v2:172.21.15.100:3300/0 conn(0x7f65300cffa0 0x7f653015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 --2- 172.21.15.60:0/757312820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300cfa60 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 -- 172.21.15.60:0/757312820 >> 172.21.15.60:0/757312820 conn(0x7f65300b2220 msgr2=0x7f65300b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 -- 172.21.15.60:0/757312820 shutdown_connections 2024-04-25T08:33:16.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.974+0000 7f653744d640 1 -- 172.21.15.60:0/757312820 wait complete. 2024-04-25T08:33:16.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.975+0000 7f653744d640 1 Processor -- start 2024-04-25T08:33:16.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.975+0000 7f653744d640 1 -- start start 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f653744d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f653006b1a0 0x7f653014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f653744d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f653744d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65300cffa0 0x7f65300c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f653744d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530162200 con 0x7f653006b580 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f653744d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65300d03f0 con 0x7f653006b1a0 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f653744d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6530162720 con 0x7f65300cffa0 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56588/0 (socket says 172.21.15.60:56588) 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 -- 172.21.15.60:0/3082610002 learned_addr learned my addr 172.21.15.60:0/3082610002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65351e9640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f653006b1a0 0x7f653014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65359ea640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65300cffa0 0x7f65300c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 -- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65300cffa0 msgr2=0x7f65300c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65300cffa0 0x7f65300c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 -- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f653006b1a0 msgr2=0x7f653014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f653006b1a0 0x7f653014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 -- 172.21.15.60:0/3082610002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300b5a20 con 0x7f653006b580 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65351e9640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f653006b1a0 0x7f653014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65359ea640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65300cffa0 0x7f65300c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.976+0000 7f65349e8640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300c54e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6520009860 tx=0x7f65200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.977+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520037040 con 0x7f653006b580 2024-04-25T08:33:16.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.977+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520042070 con 0x7f653006b580 2024-04-25T08:33:16.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.977+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65300c4440 con 0x7f653006b580 2024-04-25T08:33:16.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.977+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520030a50 con 0x7f653006b580 2024-04-25T08:33:16.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.977+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65300c4a00 con 0x7f653006b580 2024-04-25T08:33:16.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.978+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6530073990 con 0x7f653006b580 2024-04-25T08:33:16.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.978+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6520030bf0 con 0x7f653006b580 2024-04-25T08:33:16.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.979+0000 7f6525ffb640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f651406b4d0 0x7f651406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:16.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.979+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65200aae60 con 0x7f653006b580 2024-04-25T08:33:16.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.979+0000 7f65351e9640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f651406b4d0 0x7f651406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:16.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.979+0000 7f65351e9640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f651406b4d0 0x7f651406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f652c000f60 tx=0x7f652c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:16.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:16.983+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6520045360 con 0x7f653006b580 2024-04-25T08:33:17.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.202+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f653006d370 con 0x7f653006b580 2024-04-25T08:33:17.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.202+0000 7f6525ffb640 1 -- 172.21.15.60:0/3082610002 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f6520079b50 con 0x7f653006b580 2024-04-25T08:33:17.197 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:17.197 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1884848904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1884848904},{"type":"v1","addr":"172.21.15.60:6835","nonce":1884848904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1120004533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1120004533},{"type":"v1","addr":"172.21.15.100:6835","nonce":1120004533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2249277423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2249277423},{"type":"v1","addr":"172.21.15.100:6837","nonce":2249277423}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:10.464973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6699},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6699":{"gid":6699,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6834/1590331682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1590331682},{"type":"v1","addr":"172.21.15.60:6834","nonce":1590331682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:17.198 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 35 2024-04-25T08:33:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f651406b4d0 msgr2=0x7f651406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f651406b4d0 0x7f651406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f652c000f60 tx=0x7f652c009040 comp rx=0 tx=0).stop 2024-04-25T08:33:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 msgr2=0x7f65300c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300c54e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6520009860 tx=0x7f65200046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 shutdown_connections 2024-04-25T08:33:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65300cffa0 0x7f65300c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f653006b580 0x7f65300c54e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f651406b4d0 0x7f651406d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 --2- 172.21.15.60:0/3082610002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f653006b1a0 0x7f653014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 >> 172.21.15.60:0/3082610002 conn(0x7f65300b2220 msgr2=0x7f65300b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 shutdown_connections 2024-04-25T08:33:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:17.204+0000 7f653744d640 1 -- 172.21.15.60:0/3082610002 wait complete. 2024-04-25T08:33:17.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:33:17.339 INFO:teuthology.orchestra.run.smithi060.stdout:{"message":"","return_code":0} 2024-04-25T08:33:17.340 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:33:17.340 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:33:17.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:17.341 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:33:17.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:17.341 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:33:17.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:17.342 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:33:17.342 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:17.351 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:33:17.351 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:17.353 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:33:17.353 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:17.363 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:33:17.363 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:17.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T08:33:23.351 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:33:23.351 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:23.351 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T08:33:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.406+0000 7f1e2321cf40 1 Processor -- start 2024-04-25T08:33:23.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.406+0000 7f1e2321cf40 1 -- start start 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e2321cf40 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 0x559e9d966260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e2321cf40 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9da11060 0x559e9d96ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e2321cf40 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d96b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e2321cf40 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9d8e2df0 con 0x559e9d9398a0 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e2321cf40 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9d8e6d50 con 0x559e9da11060 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e2321cf40 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e9d984f30 con 0x559e9da12d10 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 0x559e9d966260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 0x559e9d966260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42602/0 (socket says 172.21.15.60:42602) 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 -- 172.21.15.60:0/3277970333 learned_addr learned my addr 172.21.15.60:0/3277970333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e17fff640 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9da11060 0x559e9d96ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e208ae640 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d96b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 -- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 msgr2=0x559e9d96b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d96b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 -- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9da11060 msgr2=0x559e9d96ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9da11060 0x559e9d96ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e200ad640 1 -- 172.21.15.60:0/3277970333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9d965d20 con 0x559e9d9398a0 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.407+0000 7f1e208ae640 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d96b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e200ad640 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 0x559e9d966260 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1e10001d80 tx=0x7f1e10001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af2dc30d0e178ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e1f0ab640 1 -- 172.21.15.60:0/3277970333 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000bd70 con 0x559e9d9398a0 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e1f0ab640 1 -- 172.21.15.60:0/3277970333 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1000f400 con 0x559e9d9398a0 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e1f0ab640 1 -- 172.21.15.60:0/3277970333 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000f7f0 con 0x559e9d9398a0 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3277970333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 msgr2=0x559e9d966260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 0x559e9d966260 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1e10001d80 tx=0x7f1e10001e70 comp rx=0 tx=0).stop 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3277970333 shutdown_connections 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d96b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.406 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9da11060 0x559e9d96ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3277970333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9d9398a0 0x559e9d966260 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.408+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3277970333 >> 172.21.15.60:0/3277970333 conn(0x559e9d93a9b0 msgr2=0x559e9d939c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.409+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3277970333 shutdown_connections 2024-04-25T08:33:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.409+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3277970333 wait complete. 2024-04-25T08:33:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.409+0000 7f1e2321cf40 1 Processor -- start 2024-04-25T08:33:23.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.409+0000 7f1e2321cf40 1 -- start start 2024-04-25T08:33:23.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e2321cf40 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9d9398a0 0x559e9da05c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e2321cf40 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9da11060 0x559e9d93ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e2321cf40 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d93b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e2321cf40 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9da133e0 con 0x559e9da11060 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e2321cf40 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9d966820 con 0x559e9d9398a0 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e2321cf40 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e9d965400 con 0x559e9da12d10 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9da11060 0x559e9d93ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9da11060 0x559e9d93ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42606/0 (socket says 172.21.15.60:42606) 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 -- 172.21.15.60:0/3521765468 learned_addr learned my addr 172.21.15.60:0/3521765468 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e208ae640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d93b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e200ad640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9d9398a0 0x559e9da05c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 -- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 msgr2=0x559e9d93b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559e9da12d10 0x559e9d93b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 -- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9d9398a0 msgr2=0x559e9da05c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.410+0000 7f1e17fff640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559e9d9398a0 0x559e9da05c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e17fff640 1 -- 172.21.15.60:0/3521765468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9d9656e0 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e17fff640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9da11060 0x559e9d93ad90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c5b0 tx=0x7f1e0800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e1e0a9640 1 -- 172.21.15.60:0/3521765468 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08008070 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e1e0a9640 1 -- 172.21.15.60:0/3521765468 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080106a0 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e1e0a9640 1 -- 172.21.15.60:0/3521765468 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08010a50 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559e9d9659c0 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.411+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e9da061b0 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.413+0000 7f1e1e0a9640 1 -- 172.21.15.60:0/3521765468 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e08010bf0 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.413+0000 7f1e1e0a9640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e00069eb0 0x7f1e0006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.413+0000 7f1e1e0a9640 1 -- 172.21.15.60:0/3521765468 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e080898a0 con 0x559e9da11060 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.413+0000 7f1e200ad640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e00069eb0 0x7f1e0006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.413+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x559e9da0bf30 0x559e9da0c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.413+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 --> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v9 -- 0x559e9da0c850 con 0x559e9da0bf30 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.414+0000 7f1e208ae640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x559e9da0bf30 0x559e9da0c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.414+0000 7f1e200ad640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e00069eb0 0x7f1e0006c370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e10001eb0 tx=0x7f1e10009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.414+0000 7f1e208ae640 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x559e9da0bf30 0x559e9da0c310 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.415+0000 7f1e208ae640 1 -- 172.21.15.60:0/3521765468 <== osd.4 v2:172.21.15.100:6808/2796877898 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v67'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f1e180528d0 con 0x559e9da0bf30 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x559e9da0bf30 msgr2=0x559e9da0c310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x559e9da0bf30 0x559e9da0c310 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e00069eb0 msgr2=0x7f1e0006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1e00069eb0 0x7f1e0006c370 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e10001eb0 tx=0x7f1e10009000 comp rx=0 tx=0).stop 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9da11060 msgr2=0x559e9d93ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 --2- 172.21.15.60:0/3521765468 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559e9da11060 0x559e9d93ad90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c5b0 tx=0x7f1e0800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e208ae640 1 -- 172.21.15.60:0/3521765468 reap_dead start 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 shutdown_connections 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 >> 172.21.15.60:0/3521765468 conn(0x559e9d93a9b0 msgr2=0x559e9d939c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 shutdown_connections 2024-04-25T08:33:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.416+0000 7f1e2321cf40 1 -- 172.21.15.60:0/3521765468 wait complete. 2024-04-25T08:33:23.414 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T08:33:23.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T08:33:23.415 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T08:33:23.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T08:33:23.415 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T08:33:23.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T08:33:23.416 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T08:33:23.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T08:33:23.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7ff28c683640 1 Processor -- start 2024-04-25T08:33:23.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7f35346e4640 1 Processor -- start 2024-04-25T08:33:23.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7f690bd77640 1 Processor -- start 2024-04-25T08:33:23.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7ff28c683640 1 -- start start 2024-04-25T08:33:23.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7f35346e4640 1 -- start start 2024-04-25T08:33:23.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7f670d8db640 1 Processor -- start 2024-04-25T08:33:23.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7f690bd77640 1 -- start start 2024-04-25T08:33:23.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.532+0000 7f670d8db640 1 -- start start 2024-04-25T08:33:23.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.533+0000 7f690bd77640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f69040d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.533+0000 7f35346e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.533+0000 7f690bd77640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690415e000 con 0x7f690415dc20 2024-04-25T08:33:23.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.533+0000 7f6909b13640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f690415dc20 0x7f69040ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35460/0 (socket says 172.21.15.60:35460) 2024-04-25T08:33:23.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f35346e4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0cccc0 con 0x7f352c1452a0 2024-04-25T08:33:23.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f690bd77640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040d1de0 con 0x7f6904161710 2024-04-25T08:33:23.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f6909b13640 1 -- 172.21.15.60:0/657333264 learned_addr learned my addr 172.21.15.60:0/657333264 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:23.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f35346e4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0cc140 con 0x7f352c1446d0 2024-04-25T08:33:23.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f35346e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0b2b20 con 0x7f352c144d50 2024-04-25T08:33:23.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f690bd77640 1 -- 172.21.15.60:0/657333264 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6904161c60 con 0x7f69040ced80 2024-04-25T08:33:23.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f3532c81640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f352c1452a0 0x7f352c06ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35464/0 (socket says 172.21.15.60:35464) 2024-04-25T08:33:23.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f3532c81640 1 -- 172.21.15.60:0/1010887420 learned_addr learned my addr 172.21.15.60:0/1010887420 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28c683640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2840cc4f0 0x7ff2840cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28c683640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2840cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28c683640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff284072280 con 0x7ff2840cdb20 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28c683640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff284071270 con 0x7ff2840cc4f0 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28c683640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840714c0 con 0x7ff28415b980 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436888302 0 0) 0x7f690415e000 con 0x7f690415dc20 2024-04-25T08:33:23.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f0003520 con 0x7f690415dc20 2024-04-25T08:33:23.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f3531c7f640 1 --2- 172.21.15.60:0/1010887420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f6909312640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f69040d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779555823 0 0) 0x7f352c0cccc0 con 0x7f352c1452a0 2024-04-25T08:33:23.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3518007930 con 0x7f352c1452a0 2024-04-25T08:33:23.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff289c1e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff2840cdb20 0x7ff28415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35480/0 (socket says 172.21.15.60:35480) 2024-04-25T08:33:23.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff289c1e640 1 -- 172.21.15.60:0/2360621752 learned_addr learned my addr 172.21.15.60:0/2360621752 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:23.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28ac20640 1 --2- 172.21.15.60:0/2360621752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2840cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f670d8db640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67080cf600 0x7f67080cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590080209 0 0) 0x7f69040d1de0 con 0x7f6904161710 2024-04-25T08:33:23.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f670d8db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7ff28a41f640 1 --2- 172.21.15.60:0/2360621752 >> v2:172.21.15.100:3300/0 conn(0x7ff2840cc4f0 0x7ff2840cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968449506 0 0) 0x7f352c0cc140 con 0x7f352c1446d0 2024-04-25T08:33:23.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f352c0cccc0 con 0x7f352c1446d0 2024-04-25T08:33:23.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f690415e000 con 0x7f6904161710 2024-04-25T08:33:23.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f670d8db640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080c75e0 con 0x7f670815bb50 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.534+0000 7f670d8db640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080c8680 con 0x7f67080cf600 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67080cf600 0x7f67080cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f670d8db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080cff80 con 0x7f67080cbf20 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 930083344 0 0) 0x7f68f0003520 con 0x7f690415dc20 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 msgr2=0x7f69040d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f69040d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 >> v1:172.21.15.100:6789/0 conn(0x7f6904161710 legacy=0x7f69040d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f3531c7f640 1 -- 172.21.15.60:0/1010887420 >> v1:172.21.15.100:6789/0 conn(0x7f352c1446d0 legacy=0x7f352c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f3531c7f640 1 -- 172.21.15.60:0/1010887420 >> v1:172.21.15.60:6789/0 conn(0x7f352c1452a0 legacy=0x7f352c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f3531c7f640 1 -- 172.21.15.60:0/1010887420 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0cc420 con 0x7f352c144d50 2024-04-25T08:33:23.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904161f40 con 0x7f690415dc20 2024-04-25T08:33:23.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6900001670 con 0x7f690415dc20 2024-04-25T08:33:23.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 >> v1:172.21.15.60:6789/0 conn(0x7f690415dc20 legacy=0x7f69040ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68f0003b70 0x7f68f0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f68f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67080cf600 0x7f67080cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58752/0 (socket says 172.21.15.60:58752) 2024-04-25T08:33:23.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 -- 172.21.15.60:0/997110544 learned_addr learned my addr 172.21.15.60:0/997110544 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67077fe640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 -- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 msgr2=0x7f67080cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 -- 172.21.15.60:0/997110544 >> v1:172.21.15.60:6789/0 conn(0x7f670815bb50 legacy=0x7f67080c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6909b13640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68f0003b70 0x7f68f0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67067fc640 1 -- 172.21.15.60:0/997110544 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080c78c0 con 0x7f67080cf600 2024-04-25T08:33:23.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904161c60 con 0x7f68f0004d40 2024-04-25T08:33:23.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000ba70 con 0x7f68f0003b70 2024-04-25T08:33:23.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000bc20 con 0x7f69040ced80 2024-04-25T08:33:23.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6909312640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f68f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f67077fe640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:23.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7ff28941d640 1 -- 172.21.15.60:0/2360621752 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892723061 0 0) 0x7ff284072280 con 0x7ff2840cdb20 2024-04-25T08:33:23.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7ff28ac20640 1 -- 172.21.15.60:0/2360621752 >> v2:172.21.15.100:3300/0 conn(0x7ff2840cc4f0 msgr2=0x7ff2840cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7ff28ac20640 1 --2- 172.21.15.60:0/2360621752 >> v2:172.21.15.100:3300/0 conn(0x7ff2840cc4f0 0x7ff2840cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7ff28ac20640 1 -- 172.21.15.60:0/2360621752 >> v1:172.21.15.60:6789/0 conn(0x7ff2840cdb20 legacy=0x7ff28415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7ff28ac20640 1 -- 172.21.15.60:0/2360621752 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840717a0 con 0x7ff28415b980 2024-04-25T08:33:23.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7ff28a41f640 1 --2- 172.21.15.60:0/2360621752 >> v2:172.21.15.100:3300/0 conn(0x7ff2840cc4f0 0x7ff2840cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:23.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f690a314640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6909b13640 1 -- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 msgr2=0x7f68f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.535+0000 7f6909b13640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f68f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6909b13640 1 -- 172.21.15.60:0/657333264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0004d40 msgr2=0x7f68f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6909b13640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6909b13640 1 -- 172.21.15.60:0/657333264 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904161f40 con 0x7f68f0003b70 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6909312640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040ced80 0x7f68f0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f3531c7f640 1 --2- 172.21.15.60:0/1010887420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c06d890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f351c0097d0 tx=0x7f351c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d24e475ad9486c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7ff28ac20640 1 --2- 172.21.15.60:0/2360621752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2840cdfb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009d90 tx=0x7ff27c02ea70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a25b5664ab9f1a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c037070 con 0x7f352c144d50 2024-04-25T08:33:23.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c032440 con 0x7f352c144d50 2024-04-25T08:33:23.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f353147e640 1 -- 172.21.15.60:0/1010887420 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0327f0 con 0x7f352c144d50 2024-04-25T08:33:23.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f67067fc640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67080cfa40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009870 tx=0x7f66fc02a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5774f73322502fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7ff28941d640 1 -- 172.21.15.60:0/2360621752 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c032830 con 0x7ff28415b980 2024-04-25T08:33:23.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7ff28941d640 1 -- 172.21.15.60:0/2360621752 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c0329d0 con 0x7ff28415b980 2024-04-25T08:33:23.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7ff28941d640 1 -- 172.21.15.60:0/2360621752 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c032da0 con 0x7ff28415b980 2024-04-25T08:33:23.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6705ffb640 1 -- 172.21.15.60:0/997110544 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc037070 con 0x7f67080cf600 2024-04-25T08:33:23.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6705ffb640 1 -- 172.21.15.60:0/997110544 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc032640 con 0x7f67080cf600 2024-04-25T08:33:23.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6705ffb640 1 -- 172.21.15.60:0/997110544 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc032a10 con 0x7f67080cf600 2024-04-25T08:33:23.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.536+0000 7f6909b13640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68f0003b70 0x7f68f0003f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f690000c240 tx=0x7f68f0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777a7d4638e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900007d30 con 0x7f68f0003b70 2024-04-25T08:33:23.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900005af0 con 0x7f68f0003b70 2024-04-25T08:33:23.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f6908b11640 1 -- 172.21.15.60:0/657333264 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900007070 con 0x7f68f0003b70 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 -- 172.21.15.60:0/1010887420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 msgr2=0x7f352c06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 --2- 172.21.15.60:0/1010887420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c06d890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f351c0097d0 tx=0x7f351c02a210 comp rx=0 tx=0).stop 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 -- 172.21.15.60:0/1010887420 shutdown_connections 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 --2- 172.21.15.60:0/1010887420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c06d890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 -- 172.21.15.60:0/1010887420 >> 172.21.15.60:0/1010887420 conn(0x7f352c06abe0 msgr2=0x7f352c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 -- 172.21.15.60:0/1010887420 shutdown_connections 2024-04-25T08:33:23.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f35346e4640 1 -- 172.21.15.60:0/1010887420 wait complete. 2024-04-25T08:33:23.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 -- 172.21.15.60:0/2360621752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 msgr2=0x7ff2840cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 --2- 172.21.15.60:0/2360621752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2840cdfb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009d90 tx=0x7ff27c02ea70 comp rx=0 tx=0).stop 2024-04-25T08:33:23.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f670d8db640 1 -- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 msgr2=0x7f67080cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f670d8db640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67080cfa40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009870 tx=0x7f66fc02a2f0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 -- 172.21.15.60:0/2360621752 shutdown_connections 2024-04-25T08:33:23.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 --2- 172.21.15.60:0/2360621752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2840cdfb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 --2- 172.21.15.60:0/2360621752 >> v2:172.21.15.100:3300/0 conn(0x7ff2840cc4f0 0x7ff2840cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 -- 172.21.15.60:0/2360621752 >> 172.21.15.60:0/2360621752 conn(0x7ff28406abe0 msgr2=0x7ff28406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:23.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f670d8db640 1 -- 172.21.15.60:0/997110544 shutdown_connections 2024-04-25T08:33:23.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 -- 172.21.15.60:0/2360621752 shutdown_connections 2024-04-25T08:33:23.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f670d8db640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7f670d8db640 1 --2- 172.21.15.60:0/997110544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67080cfa40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f670d8db640 1 -- 172.21.15.60:0/997110544 >> 172.21.15.60:0/997110544 conn(0x7f670806f500 msgr2=0x7f670806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:23.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.537+0000 7ff28c683640 1 -- 172.21.15.60:0/2360621752 wait complete. 2024-04-25T08:33:23.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f670d8db640 1 -- 172.21.15.60:0/997110544 shutdown_connections 2024-04-25T08:33:23.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690bd77640 1 -- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68f0003b70 msgr2=0x7f68f0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690bd77640 1 --2- 172.21.15.60:0/657333264 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68f0003b70 0x7f68f0003f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f690000c240 tx=0x7f68f0005830 comp rx=0 tx=0).stop 2024-04-25T08:33:23.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690a314640 1 -- 172.21.15.60:0/657333264 reap_dead start 2024-04-25T08:33:23.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f670d8db640 1 -- 172.21.15.60:0/997110544 wait complete. 2024-04-25T08:33:23.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690bd77640 1 -- 172.21.15.60:0/657333264 shutdown_connections 2024-04-25T08:33:23.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690bd77640 1 -- 172.21.15.60:0/657333264 >> 172.21.15.60:0/657333264 conn(0x7f69040d0420 msgr2=0x7f69040d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:23.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690bd77640 1 -- 172.21.15.60:0/657333264 shutdown_connections 2024-04-25T08:33:23.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.538+0000 7f690bd77640 1 -- 172.21.15.60:0/657333264 wait complete. 2024-04-25T08:33:23.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.539+0000 7f35346e4640 1 Processor -- start 2024-04-25T08:33:23.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.539+0000 7f35346e4640 1 -- start start 2024-04-25T08:33:23.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.539+0000 7f670d8db640 1 Processor -- start 2024-04-25T08:33:23.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.539+0000 7f670d8db640 1 -- start start 2024-04-25T08:33:23.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f690bd77640 1 Processor -- start 2024-04-25T08:33:23.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f690bd77640 1 -- start start 2024-04-25T08:33:23.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7ff28c683640 1 Processor -- start 2024-04-25T08:33:23.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7ff28c683640 1 -- start start 2024-04-25T08:33:23.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f670d8db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f35346e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f352c1446d0 0x7f352c1eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f670d8db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67081e6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f35346e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c1f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f670d8db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670815bb50 0x7f67080b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f670d8db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080c8160 con 0x7f670815bb50 2024-04-25T08:33:23.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f670d8db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080c7e80 con 0x7f67080cf600 2024-04-25T08:33:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f35346e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352c1452a0 0x7f352c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.540+0000 7f670d8db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080c72a0 con 0x7f67080cbf20 2024-04-25T08:33:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40974/0 (socket says 172.21.15.60:40974) 2024-04-25T08:33:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f67077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670815bb50 0x7f67080b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 -- 172.21.15.60:0/182875947 learned_addr learned my addr 172.21.15.60:0/182875947 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f3531c7f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c1f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f3531c7f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c1f0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40986/0 (socket says 172.21.15.60:40986) 2024-04-25T08:33:23.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f67067fc640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67081e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f3532480640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f352c1446d0 0x7f352c1eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f3532480640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f352c1446d0 0x7f352c1eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58780/0 (socket says 172.21.15.60:58780) 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f3531c7f640 1 -- 172.21.15.60:0/3470038983 learned_addr learned my addr 172.21.15.60:0/3470038983 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0cc9e0 con 0x7f352c1452a0 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c06d400 con 0x7f352c1446d0 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f3532c81640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352c1452a0 0x7f352c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c144ab0 con 0x7f352c144d50 2024-04-25T08:33:23.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f690bd77640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 0x7f69041f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f690bd77640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 0x7f690414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6909b13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 0x7f69041f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f690bd77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69040b53a0 0x7f690414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6909b13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 0x7f69041f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42650/0 (socket says 172.21.15.60:42650) 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909312640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 0x7f690414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909312640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 0x7f690414d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40988/0 (socket says 172.21.15.60:40988) 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 -- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 msgr2=0x7f67081e6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67081e6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 -- 172.21.15.60:0/182875947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670815bb50 msgr2=0x7f67080b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670815bb50 0x7f67080b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f6706ffd640 1 -- 172.21.15.60:0/182875947 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670813f140 con 0x7f67080cbf20 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.541+0000 7f67077fe640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670815bb50 0x7f67080b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6706ffd640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080b4790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c3c0 tx=0x7f66f800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f3532c81640 1 -- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 msgr2=0x7f352c1f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f3532c81640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c1f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f3532c81640 1 -- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f352c1446d0 msgr2=0x7f352c1eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f3532c81640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f352c1446d0 0x7f352c1eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80052d0 con 0x7f67080cbf20 2024-04-25T08:33:23.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f3532c81640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c1f0480 con 0x7f352c1452a0 2024-04-25T08:33:23.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28c683640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 0x7ff28415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8005470 con 0x7f67080cbf20 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f690bd77640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b5e50 con 0x7f68f0003b70 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f690bd77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904162500 con 0x7f69040b53a0 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f690bd77640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b5fd0 con 0x7f69040b4f20 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f690a314640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69040b53a0 0x7f690414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f690a314640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69040b53a0 0x7f690414dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58794/0 (socket says 172.21.15.60:58794) 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f690a314640 1 -- 172.21.15.60:0/2530121456 learned_addr learned my addr 172.21.15.60:0/2530121456 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28c683640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2840cdb20 0x7ff2841ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f3532c81640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352c1452a0 0x7f352c15b1a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f352400c9d0 tx=0x7f352400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8007240 con 0x7f67080cbf20 2024-04-25T08:33:23.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67081e6d40 con 0x7f67080cbf20 2024-04-25T08:33:23.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67081e7300 con 0x7f67080cbf20 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28c683640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2841572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28a41f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 0x7ff28415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28a41f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 0x7ff28415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42660/0 (socket says 172.21.15.60:42660) 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff289c1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2840cdb20 0x7ff2841ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28a41f640 1 -- 172.21.15.60:0/453820353 learned_addr learned my addr 172.21.15.60:0/453820353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840ae500 con 0x7ff2840cc4f0 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff284071d60 con 0x7ff2840cdb20 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840ae680 con 0x7ff28415b980 2024-04-25T08:33:23.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909b13640 1 -- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 msgr2=0x7f690414d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909b13640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 0x7f690414d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909b13640 1 -- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69040b53a0 msgr2=0x7f690414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909b13640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69040b53a0 0x7f690414dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909b13640 1 -- 172.21.15.60:0/2530121456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040caba0 con 0x7f68f0003b70 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.542+0000 7f6909312640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 0x7f690414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524017070 con 0x7f352c1452a0 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352400ab70 con 0x7f352c1452a0 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524012400 con 0x7f352c1452a0 2024-04-25T08:33:23.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28ac20640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2841572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c13b580 con 0x7f352c1452a0 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28a41f640 1 -- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 msgr2=0x7ff2841572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28a41f640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2841572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28a41f640 1 -- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2840cdb20 msgr2=0x7ff2841ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28a41f640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2840cdb20 0x7ff2841ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28a41f640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284155590 con 0x7ff2840cc4f0 2024-04-25T08:33:23.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7ff28a41f640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 0x7ff28415c7d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff280008640 tx=0x7ff280008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7f6909b13640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 0x7f69041f33e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6900030000 tx=0x7f6900001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.543+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c13ce80 con 0x7f352c1452a0 2024-04-25T08:33:23.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280019070 con 0x7ff2840cc4f0 2024-04-25T08:33:23.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900042020 con 0x7f68f0003b70 2024-04-25T08:33:23.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff284155850 con 0x7ff2840cc4f0 2024-04-25T08:33:23.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff284155e10 con 0x7ff2840cc4f0 2024-04-25T08:33:23.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67080cf150 con 0x7f67080cbf20 2024-04-25T08:33:23.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69040cae80 con 0x7f68f0003b70 2024-04-25T08:33:23.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280005990 con 0x7ff2840cc4f0 2024-04-25T08:33:23.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900007070 con 0x7f68f0003b70 2024-04-25T08:33:23.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69040cb440 con 0x7f68f0003b70 2024-04-25T08:33:23.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280005d40 con 0x7ff2840cc4f0 2024-04-25T08:33:23.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.544+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900007d30 con 0x7f68f0003b70 2024-04-25T08:33:23.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.545+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66f80073e0 con 0x7f67080cbf20 2024-04-25T08:33:23.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.545+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c073a00 con 0x7f352c1452a0 2024-04-25T08:33:23.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.546+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69040d16b0 con 0x7f68f0003b70 2024-04-25T08:33:23.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.546+0000 7f66effff640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f406b4d0 0x7f66f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.546+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66f8089ab0 con 0x7f67080cbf20 2024-04-25T08:33:23.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.546+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c005290 con 0x7ff2840cc4f0 2024-04-25T08:33:23.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.546+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f352401a070 con 0x7f352c1452a0 2024-04-25T08:33:23.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.546+0000 7f67067fc640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f406b4d0 0x7f66f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7f3522ffd640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f351006b520 0x7f351006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6900045020 con 0x7f68f0003b70 2024-04-25T08:33:23.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3524089f40 con 0x7f352c1452a0 2024-04-25T08:33:23.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7f3532480640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f351006b520 0x7f351006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff28001c070 con 0x7ff2840cc4f0 2024-04-25T08:33:23.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7f67067fc640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f406b4d0 0x7f66f406d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66fc034000 tx=0x7f66fc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.547+0000 7f68fa7fc640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f68e806b4d0 0x7f68e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.548+0000 7f3532480640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f351006b520 0x7f351006d9e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f35280011e0 tx=0x7f3528009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.548+0000 7ff27affd640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff26806b4d0 0x7ff26806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:23.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.548+0000 7f6909312640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f68e806b4d0 0x7f68e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.548+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff2800898d0 con 0x7ff2840cc4f0 2024-04-25T08:33:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.548+0000 7ff289c1e640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff26806b4d0 0x7ff26806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:23.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.548+0000 7f6909312640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f68e806b4d0 0x7f68e806d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f69040d3ad0 tx=0x7f68f4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.549+0000 7ff289c1e640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff26806b4d0 0x7ff26806d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff284156b10 tx=0x7ff274009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:23.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.550+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66f80581c0 con 0x7f67080cbf20 2024-04-25T08:33:23.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.552+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69000765f0 con 0x7f68f0003b70 2024-04-25T08:33:23.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.552+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f352c1f0480 con 0x7f352c1452a0 2024-04-25T08:33:23.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.556+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff280057f70 con 0x7ff2840cc4f0 2024-04-25T08:33:23.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.565+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6900078c50 con 0x7f68f0003b70 2024-04-25T08:33:23.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.803+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f67080cc9b0 con 0x7f67080cbf20 2024-04-25T08:33:23.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.806+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f352c0b1970 con 0x7f352c1452a0 2024-04-25T08:33:23.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.848+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7ff26c005850 con 0x7ff2840cc4f0 2024-04-25T08:33:23.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:23.853+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f69040739e0 con 0x7f68f0003b70 2024-04-25T08:33:23.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T08:33:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.136+0000 7f66effff640 1 -- 172.21.15.60:0/182875947 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 5630 v36) v1 ==== 95+0+0 (secure 0 0 0) 0x7f66f805c260 con 0x7f67080cbf20 2024-04-25T08:33:24.132 INFO:teuthology.orchestra.run.smithi060.stderr:failed mds gid 5630 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f406b4d0 msgr2=0x7f66f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f406b4d0 0x7f66f406d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f66fc034000 tx=0x7f66fc02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 msgr2=0x7f67080b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080b4790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c3c0 tx=0x7f66f800c880 comp rx=0 tx=0).stop 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 shutdown_connections 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670815bb50 0x7f67080b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f406b4d0 0x7f66f406d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67080cf600 0x7f67081e6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 --2- 172.21.15.60:0/182875947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67080cbf20 0x7f67080b4790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 >> 172.21.15.60:0/182875947 conn(0x7f670806f500 msgr2=0x7f67080b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:24.136 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:33:24.139+0000 7fcb353b1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:33:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 shutdown_connections 2024-04-25T08:33:24.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.139+0000 7f670d8db640 1 -- 172.21.15.60:0/182875947 wait complete. 2024-04-25T08:33:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.144+0000 7f3522ffd640 1 -- 172.21.15.60:0/3470038983 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6181 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7f35240585d0 con 0x7f352c1452a0 2024-04-25T08:33:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.144+0000 7ff27affd640 1 -- 172.21.15.60:0/453820353 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6699 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7ff28005c010 con 0x7ff2840cc4f0 2024-04-25T08:33:24.141 INFO:teuthology.orchestra.run.smithi060.stderr:failed mds gid 6181 2024-04-25T08:33:24.141 INFO:teuthology.orchestra.run.smithi060.stderr:failed mds gid 6699 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f351006b520 msgr2=0x7f351006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f351006b520 0x7f351006d9e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f35280011e0 tx=0x7f3528009040 comp rx=0 tx=0).stop 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352c1452a0 msgr2=0x7f352c15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352c1452a0 0x7f352c15b1a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f352400c9d0 tx=0x7f352400ce90 comp rx=0 tx=0).stop 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 shutdown_connections 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f352c1452a0 0x7f352c15b1a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f352c144d50 0x7f352c1f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f351006b520 0x7f351006d9e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 --2- 172.21.15.60:0/3470038983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f352c1446d0 0x7f352c1eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 >> 172.21.15.60:0/3470038983 conn(0x7f352c06abe0 msgr2=0x7f352c1437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 shutdown_connections 2024-04-25T08:33:24.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.147+0000 7f35346e4640 1 -- 172.21.15.60:0/3470038983 wait complete. 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff26806b4d0 msgr2=0x7ff26806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff26806b4d0 0x7ff26806d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff284156b10 tx=0x7ff274009040 comp rx=0 tx=0).stop 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 msgr2=0x7ff28415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 0x7ff28415c7d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff280008640 tx=0x7ff280008730 comp rx=0 tx=0).stop 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 shutdown_connections 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff28415b980 0x7ff2841572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff26806b4d0 0x7ff26806d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2840cdb20 0x7ff2841ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 --2- 172.21.15.60:0/453820353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2840cc4f0 0x7ff28415c7d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 >> 172.21.15.60:0/453820353 conn(0x7ff28406abe0 msgr2=0x7ff28406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 shutdown_connections 2024-04-25T08:33:24.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:24.148+0000 7ff28c683640 1 -- 172.21.15.60:0/453820353 wait complete. 2024-04-25T08:33:24.150 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:33:24.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:33:24.152 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:33:24.153 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:24.158 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:33:24.158 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:33:24.159 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:33:24.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:33:24.161 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:33:24.162 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:24.202 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:33:24.202 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:24.226 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:33:24.237 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:33:24.277 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:33:25.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.146+0000 7f68fa7fc640 1 -- 172.21.15.60:0/2530121456 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6175 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6900078550 con 0x7f68f0003b70 2024-04-25T08:33:25.142 INFO:teuthology.orchestra.run.smithi060.stderr:failed mds gid 6175 2024-04-25T08:33:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f68e806b4d0 msgr2=0x7f68e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f68e806b4d0 0x7f68e806d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f69040d3ad0 tx=0x7f68f4009190 comp rx=0 tx=0).stop 2024-04-25T08:33:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 msgr2=0x7f69041f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 0x7f69041f33e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6900030000 tx=0x7f6900001970 comp rx=0 tx=0).stop 2024-04-25T08:33:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 shutdown_connections 2024-04-25T08:33:25.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f68e806b4d0 0x7f68e806d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69040b53a0 0x7f690414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69040b4f20 0x7f690414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 --2- 172.21.15.60:0/2530121456 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68f0003b70 0x7f69041f33e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 >> 172.21.15.60:0/2530121456 conn(0x7f69040d0420 msgr2=0x7f69040b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:25.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 shutdown_connections 2024-04-25T08:33:25.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.149+0000 7f690bd77640 1 -- 172.21.15.60:0/2530121456 wait complete. 2024-04-25T08:33:25.162 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:33:25.162 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:33:25.165 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:33:25.165 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:25.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:25.243 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:33:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.318+0000 7f74a0ee1640 1 Processor -- start 2024-04-25T08:33:25.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.318+0000 7f74a0ee1640 1 -- start start 2024-04-25T08:33:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f74a0ee1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f749c0b5c30 0x7f749c168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f74a0ee1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0ceee0 con 0x7f749c15db70 2024-04-25T08:33:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f74a0ee1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0d1d10 con 0x7f749c0b6cb0 2024-04-25T08:33:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f74a0ee1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c0d3800 con 0x7f749c0b5c30 2024-04-25T08:33:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f749a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f749c15db70 0x7f749c162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35516/0 (socket says 172.21.15.60:35516) 2024-04-25T08:33:25.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f749a59c640 1 -- 172.21.15.60:0/366465841 learned_addr learned my addr 172.21.15.60:0/366465841 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f749ad9d640 1 --2- 172.21.15.60:0/366465841 >> v2:172.21.15.100:3300/0 conn(0x7f749c0b5c30 0x7f749c168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f7499d9b640 1 -- 172.21.15.60:0/366465841 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778127149 0 0) 0x7f749c0ceee0 con 0x7f749c15db70 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f7499d9b640 1 -- 172.21.15.60:0/366465841 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7488003520 con 0x7f749c15db70 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f7499d9b640 1 -- 172.21.15.60:0/366465841 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118578801 0 0) 0x7f749c0d1d10 con 0x7f749c0b6cb0 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f749ad9d640 1 -- 172.21.15.60:0/366465841 >> v1:172.21.15.100:6789/0 conn(0x7f749c0b6cb0 legacy=0x7f749c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f749ad9d640 1 -- 172.21.15.60:0/366465841 >> v1:172.21.15.60:6789/0 conn(0x7f749c15db70 legacy=0x7f749c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.319+0000 7f749ad9d640 1 -- 172.21.15.60:0/366465841 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0d1ff0 con 0x7f749c0b5c30 2024-04-25T08:33:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.320+0000 7f749ad9d640 1 --2- 172.21.15.60:0/366465841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c0b5c30 0x7f749c168ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f749c0d3980 tx=0x7f749402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ec5cad9dc25c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.320+0000 7f7499d9b640 1 -- 172.21.15.60:0/366465841 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494007740 con 0x7f749c0b5c30 2024-04-25T08:33:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.320+0000 7f7499d9b640 1 -- 172.21.15.60:0/366465841 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74940078e0 con 0x7f749c0b5c30 2024-04-25T08:33:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.320+0000 7f7499d9b640 1 -- 172.21.15.60:0/366465841 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494007c90 con 0x7f749c0b5c30 2024-04-25T08:33:25.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 -- 172.21.15.60:0/366465841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c0b5c30 msgr2=0x7f749c168ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/366465841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c0b5c30 0x7f749c168ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f749c0d3980 tx=0x7f749402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 -- 172.21.15.60:0/366465841 shutdown_connections 2024-04-25T08:33:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/366465841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c0b5c30 0x7f749c168ad0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 -- 172.21.15.60:0/366465841 >> 172.21.15.60:0/366465841 conn(0x7f749c0d0520 msgr2=0x7f749c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 -- 172.21.15.60:0/366465841 shutdown_connections 2024-04-25T08:33:25.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.321+0000 7f74a0ee1640 1 -- 172.21.15.60:0/366465841 wait complete. 2024-04-25T08:33:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.322+0000 7f74a0ee1640 1 Processor -- start 2024-04-25T08:33:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.322+0000 7f74a0ee1640 1 -- start start 2024-04-25T08:33:25.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f74a0ee1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 0x7f749c1f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f74a0ee1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f749c0b6cb0 0x7f749c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f74a0ee1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c15db70 0x7f749c1e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f74a0ee1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0ce530 con 0x7f749c0b5c30 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f74a0ee1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c15df50 con 0x7f749c15db70 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f74a0ee1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0cf1c0 con 0x7f749c0b6cb0 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f749a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 0x7f749c1f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f749a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 0x7f749c1f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42688/0 (socket says 172.21.15.60:42688) 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f749a59c640 1 -- 172.21.15.60:0/229079596 learned_addr learned my addr 172.21.15.60:0/229079596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f7493fff640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f749c0b6cb0 0x7f749c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.323+0000 7f749ad9d640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c15db70 0x7f749c1e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f749a59c640 1 -- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f749c0b6cb0 msgr2=0x7f749c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f749a59c640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f749c0b6cb0 0x7f749c150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f749a59c640 1 -- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c15db70 msgr2=0x7f749c1e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f749a59c640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c15db70 0x7f749c1e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f749a59c640 1 -- 172.21.15.60:0/229079596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c1f81e0 con 0x7f749c0b5c30 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f749a59c640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 0x7f749c1f7ca0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c390 tx=0x7f748c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c017070 con 0x7f749c0b5c30 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c005ce0 con 0x7f749c0b5c30 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c159200 con 0x7f749c0b5c30 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0072f0 con 0x7f749c0b5c30 2024-04-25T08:33:25.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.324+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c158720 con 0x7f749c0b5c30 2024-04-25T08:33:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.325+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c073990 con 0x7f749c0b5c30 2024-04-25T08:33:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.325+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f748c01a070 con 0x7f749c0b5c30 2024-04-25T08:33:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.326+0000 7f7492ffd640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f747c06b500 0x7f747c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.326+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f748c0896e0 con 0x7f749c0b5c30 2024-04-25T08:33:25.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.326+0000 7f7493fff640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f747c06b500 0x7f747c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:25.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.327+0000 7f7493fff640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f747c06b500 0x7f747c06d9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f749c1e98b0 tx=0x7f7484009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:25.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.331+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f748c057d30 con 0x7f749c0b5c30 2024-04-25T08:33:25.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.547+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f749c0b9630 con 0x7f749c0b5c30 2024-04-25T08:33:25.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.547+0000 7f7492ffd640 1 -- 172.21.15.60:0/229079596 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f748c05bdd0 con 0x7f749c0b5c30 2024-04-25T08:33:25.543 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:25.543 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:25.149676+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:25.544 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 39 2024-04-25T08:33:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f747c06b500 msgr2=0x7f747c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f747c06b500 0x7f747c06d9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f749c1e98b0 tx=0x7f7484009040 comp rx=0 tx=0).stop 2024-04-25T08:33:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 msgr2=0x7f749c1f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:25.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 0x7f749c1f7ca0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f748c00c390 tx=0x7f748c00c850 comp rx=0 tx=0).stop 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 shutdown_connections 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f747c06b500 0x7f747c06d9c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f749c15db70 0x7f749c1e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f749c0b6cb0 0x7f749c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 --2- 172.21.15.60:0/229079596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f749c0b5c30 0x7f749c1f7ca0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 >> 172.21.15.60:0/229079596 conn(0x7f749c0d0520 msgr2=0x7f749c0b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 shutdown_connections 2024-04-25T08:33:25.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:25.549+0000 7f74a0ee1640 1 -- 172.21.15.60:0/229079596 wait complete. 2024-04-25T08:33:25.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:33:07.899265+0000', 'modified': '2024-04-25T08:33:25.149676+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6936}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6936': {'gid': 6936, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/1247115122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1247115122}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1247115122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:33:25.959 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6936:up:replay 2024-04-25T08:33:26.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:26.391 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:26.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.077+0000 7fe5101d5640 1 Processor -- start 2024-04-25T08:33:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.078+0000 7fe5101d5640 1 -- start start 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.078+0000 7fe5101d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe50815dbe0 0x7fe5080d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.078+0000 7fe5101d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5080b92a0 0x7fe5080d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.078+0000 7fe5101d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080b97a0 con 0x7fe5080b2cb0 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.078+0000 7fe5101d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50815dfc0 con 0x7fe5080b92a0 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.078+0000 7fe5101d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080d1a00 con 0x7fe50815dbe0 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5080b92a0 0x7fe5080d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe5080b92a0 0x7fe5080d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42692/0 (socket says 172.21.15.60:42692) 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 -- 172.21.15.60:0/2395677224 learned_addr learned my addr 172.21.15.60:0/2395677224 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50df71640 1 --2- 172.21.15.60:0/2395677224 >> v2:172.21.15.100:3300/0 conn(0x7fe50815dbe0 0x7fe5080d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 -- 172.21.15.60:0/2395677224 >> v2:172.21.15.100:3300/0 conn(0x7fe50815dbe0 msgr2=0x7fe5080d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 --2- 172.21.15.60:0/2395677224 >> v2:172.21.15.100:3300/0 conn(0x7fe50815dbe0 0x7fe5080d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 -- 172.21.15.60:0/2395677224 >> v1:172.21.15.100:6789/0 conn(0x7fe5080b2cb0 legacy=0x7fe5080b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:27.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.079+0000 7fe50e772640 1 -- 172.21.15.60:0/2395677224 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080d1c50 con 0x7fe5080b92a0 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.080+0000 7fe50e772640 1 --2- 172.21.15.60:0/2395677224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5080d0e50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe50806c140 tx=0x7fe4f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680e0e0cc5422817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.080+0000 7fe50cf6f640 1 -- 172.21.15.60:0/2395677224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8037070 con 0x7fe5080b92a0 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.080+0000 7fe50cf6f640 1 -- 172.21.15.60:0/2395677224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f8032440 con 0x7fe5080b92a0 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.080+0000 7fe50cf6f640 1 -- 172.21.15.60:0/2395677224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8032810 con 0x7fe5080b92a0 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.080+0000 7fe5101d5640 1 -- 172.21.15.60:0/2395677224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 msgr2=0x7fe5080d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.080+0000 7fe5101d5640 1 --2- 172.21.15.60:0/2395677224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5080d0e50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe50806c140 tx=0x7fe4f802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.081+0000 7fe5101d5640 1 -- 172.21.15.60:0/2395677224 shutdown_connections 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.081+0000 7fe5101d5640 1 --2- 172.21.15.60:0/2395677224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5080d0e50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.081+0000 7fe5101d5640 1 --2- 172.21.15.60:0/2395677224 >> v2:172.21.15.100:3300/0 conn(0x7fe50815dbe0 0x7fe5080d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.081+0000 7fe5101d5640 1 -- 172.21.15.60:0/2395677224 >> 172.21.15.60:0/2395677224 conn(0x7fe508061a20 msgr2=0x7fe508061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.081+0000 7fe5101d5640 1 -- 172.21.15.60:0/2395677224 shutdown_connections 2024-04-25T08:33:27.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.081+0000 7fe5101d5640 1 -- 172.21.15.60:0/2395677224 wait complete. 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.082+0000 7fe5101d5640 1 Processor -- start 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.082+0000 7fe5101d5640 1 -- start start 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe5101d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5080b2cb0 0x7fe5080c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe5101d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5081598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe5101d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50815dbe0 0x7fe508159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe5101d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080b6ab0 con 0x7fe5080b92a0 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe5101d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080d2210 con 0x7fe50815dbe0 2024-04-25T08:33:27.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe5101d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080b6c30 con 0x7fe5080b2cb0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5081598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5081598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42704/0 (socket says 172.21.15.60:42704) 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 -- 172.21.15.60:0/1440919039 learned_addr learned my addr 172.21.15.60:0/1440919039 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50df71640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5080b2cb0 0x7fe5080c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50e772640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50815dbe0 0x7fe508159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 -- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5080b2cb0 msgr2=0x7fe5080c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5080b2cb0 0x7fe5080c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 -- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50815dbe0 msgr2=0x7fe508159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50815dbe0 0x7fe508159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.083+0000 7fe50d770640 1 -- 172.21.15.60:0/1440919039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080b33f0 con 0x7fe5080b92a0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.084+0000 7fe50d770640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5081598b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c400 tx=0x7fe4fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.084+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc011790 con 0x7fe5080b92a0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.084+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc017070 con 0x7fe5080b92a0 2024-04-25T08:33:27.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.084+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc011bb0 con 0x7fe5080b92a0 2024-04-25T08:33:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.084+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5080b36d0 con 0x7fe5080b92a0 2024-04-25T08:33:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.084+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5080b3c90 con 0x7fe5080b92a0 2024-04-25T08:33:27.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.085+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5080b3e90 con 0x7fe5080b92a0 2024-04-25T08:33:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.085+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4fc01c460 con 0x7fe5080b92a0 2024-04-25T08:33:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.086+0000 7fe5067fc640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe4f406b4d0 0x7fe4f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.086+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe4fc08a1a0 con 0x7fe5080b92a0 2024-04-25T08:33:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.086+0000 7fe50df71640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe4f406b4d0 0x7fe4f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:27.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.087+0000 7fe50df71640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe4f406b4d0 0x7fe4f406d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe5000099a0 tx=0x7fe500003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:27.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.090+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5080b3e90 con 0x7fe5080b92a0 2024-04-25T08:33:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.307+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe508073990 con 0x7fe5080b92a0 2024-04-25T08:33:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.308+0000 7fe5067fc640 1 -- 172.21.15.60:0/1440919039 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe4fc058ed0 con 0x7fe5080b92a0 2024-04-25T08:33:27.304 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:27.304 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:26.157240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:27.304 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 41 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe4f406b4d0 msgr2=0x7fe4f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe4f406b4d0 0x7fe4f406d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe5000099a0 tx=0x7fe500003990 comp rx=0 tx=0).stop 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 msgr2=0x7fe5081598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5081598b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c400 tx=0x7fe4fc00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 shutdown_connections 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe4f406b4d0 0x7fe4f406d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe50815dbe0 0x7fe508159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5080b92a0 0x7fe5081598b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 --2- 172.21.15.60:0/1440919039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5080b2cb0 0x7fe5080c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:27.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 >> 172.21.15.60:0/1440919039 conn(0x7fe508061a20 msgr2=0x7fe5080b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:27.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 shutdown_connections 2024-04-25T08:33:27.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:27.310+0000 7fe5101d5640 1 -- 172.21.15.60:0/1440919039 wait complete. 2024-04-25T08:33:27.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:33:07.899265+0000', 'modified': '2024-04-25T08:33:26.157240+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6936}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6936': {'gid': 6936, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.60:6835/1247115122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1247115122}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1247115122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:33:27.719 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6936:up:rejoin 2024-04-25T08:33:28.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:28.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.839+0000 7f460a407640 1 Processor -- start 2024-04-25T08:33:28.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.839+0000 7f460a407640 1 -- start start 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f460a407640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f460413d190 0x7f46040cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f460a407640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f460415b320 0x7f46040cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f460a407640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4604071ed0 con 0x7f46040cf620 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f460a407640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460413d570 con 0x7f460413d190 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f460a407640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460415a840 con 0x7f460415b320 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f460413d190 0x7f46040cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f460413d190 0x7f46040cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42716/0 (socket says 172.21.15.60:42716) 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 -- 172.21.15.60:0/3849009391 learned_addr learned my addr 172.21.15.60:0/3849009391 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:28.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 -- 172.21.15.60:0/3849009391 >> v2:172.21.15.100:3300/0 conn(0x7f460415b320 msgr2=0x7f46040cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f46037fe640 1 --2- 172.21.15.60:0/3849009391 >> v2:172.21.15.100:3300/0 conn(0x7f460415b320 0x7f46040cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 --2- 172.21.15.60:0/3849009391 >> v2:172.21.15.100:3300/0 conn(0x7f460415b320 0x7f46040cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 -- 172.21.15.60:0/3849009391 >> v1:172.21.15.100:6789/0 conn(0x7f46040cf620 legacy=0x7f4604071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f4603fff640 1 -- 172.21.15.60:0/3849009391 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460415ab20 con 0x7f460413d190 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.840+0000 7f46037fe640 1 --2- 172.21.15.60:0/3849009391 >> v2:172.21.15.100:3300/0 conn(0x7f460415b320 0x7f46040cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.841+0000 7f4603fff640 1 --2- 172.21.15.60:0/3849009391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f46040cbfb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f460406f6f0 tx=0x7f45f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54751ed3d8a2c283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.841+0000 7f4602ffd640 1 -- 172.21.15.60:0/3849009391 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4037070 con 0x7f460413d190 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.841+0000 7f4602ffd640 1 -- 172.21.15.60:0/3849009391 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f4032440 con 0x7f460413d190 2024-04-25T08:33:28.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.841+0000 7f4602ffd640 1 -- 172.21.15.60:0/3849009391 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4032810 con 0x7f460413d190 2024-04-25T08:33:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 -- 172.21.15.60:0/3849009391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 msgr2=0x7f46040cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 --2- 172.21.15.60:0/3849009391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f46040cbfb0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f460406f6f0 tx=0x7f45f402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 -- 172.21.15.60:0/3849009391 shutdown_connections 2024-04-25T08:33:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 --2- 172.21.15.60:0/3849009391 >> v2:172.21.15.100:3300/0 conn(0x7f460415b320 0x7f46040cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 --2- 172.21.15.60:0/3849009391 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f46040cbfb0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:28.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 -- 172.21.15.60:0/3849009391 >> 172.21.15.60:0/3849009391 conn(0x7f460406abe0 msgr2=0x7f460406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:28.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 -- 172.21.15.60:0/3849009391 shutdown_connections 2024-04-25T08:33:28.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.842+0000 7f460a407640 1 -- 172.21.15.60:0/3849009391 wait complete. 2024-04-25T08:33:28.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.843+0000 7f460a407640 1 Processor -- start 2024-04-25T08:33:28.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.843+0000 7f460a407640 1 -- start start 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f460a407640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46040cf620 0x7f46041efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f460a407640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f4604154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f460a407640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f460415b320 0x7f4604156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f460a407640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040cfbb0 con 0x7f460413d190 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f460a407640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040cfa30 con 0x7f46040cf620 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f460a407640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46040cfd30 con 0x7f460415b320 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f46037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f4604154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f46037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f4604154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42730/0 (socket says 172.21.15.60:42730) 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f46037fe640 1 -- 172.21.15.60:0/3664906510 learned_addr learned my addr 172.21.15.60:0/3664906510 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.844+0000 7f4603fff640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46040cf620 0x7f46041efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:28.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46089a4640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f460415b320 0x7f4604156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46037fe640 1 -- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f460415b320 msgr2=0x7f4604156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46037fe640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f460415b320 0x7f4604156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46037fe640 1 -- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46040cf620 msgr2=0x7f46041efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46037fe640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46040cf620 0x7f46041efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46037fe640 1 -- 172.21.15.60:0/3664906510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46041575d0 con 0x7f460413d190 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f46037fe640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f4604154020 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c7f0 tx=0x7f45f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8015070 con 0x7f460413d190 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8007820 con 0x7f460413d190 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8007bd0 con 0x7f460413d190 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46041578b0 con 0x7f460413d190 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.845+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46040af2c0 con 0x7f460413d190 2024-04-25T08:33:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.846+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604155be0 con 0x7f460413d190 2024-04-25T08:33:28.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.847+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45f8018070 con 0x7f460413d190 2024-04-25T08:33:28.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.847+0000 7f4600ff9640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45f006b4d0 0x7f45f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:28.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.847+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f45f808a4a0 con 0x7f460413d190 2024-04-25T08:33:28.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.847+0000 7f4603fff640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45f006b4d0 0x7f45f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:28.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.848+0000 7f4603fff640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45f006b4d0 0x7f45f006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:28.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:28.852+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4604155be0 con 0x7f460413d190 2024-04-25T08:33:29.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.071+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46040cfeb0 con 0x7f460413d190 2024-04-25T08:33:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.071+0000 7f4600ff9640 1 -- 172.21.15.60:0/3664906510 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f45f8058af0 con 0x7f460413d190 2024-04-25T08:33:29.067 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:29.067 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:27.161423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-04-25T08:33:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45f006b4d0 msgr2=0x7f45f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45f006b4d0 0x7f45f006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).stop 2024-04-25T08:33:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 msgr2=0x7f4604154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f4604154020 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c7f0 tx=0x7f45f800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 shutdown_connections 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f460415b320 0x7f4604156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f460413d190 0x7f4604154020 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45f006b4d0 0x7f45f006d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 --2- 172.21.15.60:0/3664906510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46040cf620 0x7f46041efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 >> 172.21.15.60:0/3664906510 conn(0x7f460406abe0 msgr2=0x7f460413bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 shutdown_connections 2024-04-25T08:33:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.073+0000 7f460a407640 1 -- 172.21.15.60:0/3664906510 wait complete. 2024-04-25T08:33:29.483 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:33:07.899265+0000', 'modified': '2024-04-25T08:33:27.161423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6936}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6936': {'gid': 6936, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.60:6835/1247115122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 1247115122}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 1247115122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:33:29.483 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:33:29.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:33:29.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.592+0000 7f12d4041640 1 Processor -- start 2024-04-25T08:33:29.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.592+0000 7f12d4041640 1 -- start start 2024-04-25T08:33:29.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d4041640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12cc15dc20 0x7f12cc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d4041640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc161710 0x7f12cc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d4041640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc15e000 con 0x7f12cc0b9330 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d4041640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0d1de0 con 0x7f12cc15dc20 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d4041640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc161c60 con 0x7f12cc161710 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12cc15dc20 0x7f12cc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f12cc15dc20 0x7f12cc0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42744/0 (socket says 172.21.15.60:42744) 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1363716313 learned_addr learned my addr 172.21.15.60:0/1363716313 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1363716313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc161710 msgr2=0x7f12cc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d15dc640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc161710 0x7f12cc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc161710 0x7f12cc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1363716313 >> v1:172.21.15.60:6789/0 conn(0x7f12cc0b9330 legacy=0x7f12cc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1363716313 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc161f40 con 0x7f12cc15dc20 2024-04-25T08:33:29.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d0ddb640 1 -- 172.21.15.60:0/1363716313 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998653094 0 0) 0x7f12cc15e000 con 0x7f12cc0b9330 2024-04-25T08:33:29.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.593+0000 7f12d15dc640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc161710 0x7f12cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:29.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.594+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0ce670 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009870 tx=0x7f12c802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef4d14fda1da9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.594+0000 7f12d0ddb640 1 -- 172.21.15.60:0/1363716313 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8037070 con 0x7f12cc15dc20 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.594+0000 7f12d0ddb640 1 -- 172.21.15.60:0/1363716313 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8032440 con 0x7f12cc15dc20 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.594+0000 7f12d0ddb640 1 -- 172.21.15.60:0/1363716313 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8032810 con 0x7f12cc15dc20 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 -- 172.21.15.60:0/1363716313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 msgr2=0x7f12cc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0ce670 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009870 tx=0x7f12c802a250 comp rx=0 tx=0).stop 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 -- 172.21.15.60:0/1363716313 shutdown_connections 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc161710 0x7f12cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 --2- 172.21.15.60:0/1363716313 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0ce670 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 -- 172.21.15.60:0/1363716313 >> 172.21.15.60:0/1363716313 conn(0x7f12cc0d0420 msgr2=0x7f12cc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 -- 172.21.15.60:0/1363716313 shutdown_connections 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.595+0000 7f12d4041640 1 -- 172.21.15.60:0/1363716313 wait complete. 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.596+0000 7f12d4041640 1 Processor -- start 2024-04-25T08:33:29.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.596+0000 7f12d4041640 1 -- start start 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d4041640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc0b9330 0x7f12cc15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d4041640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d4041640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12cc161710 0x7f12cc0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d4041640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0b5e50 con 0x7f12cc15dc20 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d4041640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc162500 con 0x7f12cc161710 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d4041640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0b5fd0 con 0x7f12cc0b9330 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42758/0 (socket says 172.21.15.60:42758) 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 -- 172.21.15.60:0/3196557914 learned_addr learned my addr 172.21.15.60:0/3196557914 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc0b9330 0x7f12cc15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d25de640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12cc161710 0x7f12cc0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 -- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc0b9330 msgr2=0x7f12cc15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc0b9330 0x7f12cc15cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 -- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12cc161710 msgr2=0x7f12cc0cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12cc161710 0x7f12cc0cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 -- 172.21.15.60:0/3196557914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc15d480 con 0x7f12cc15dc20 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc0b9330 0x7f12cc15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.597+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0cbac0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c7f0 tx=0x7f12bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.598+0000 7f12c27fc640 1 -- 172.21.15.60:0/3196557914 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc015070 con 0x7f12cc15dc20 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.598+0000 7f12c27fc640 1 -- 172.21.15.60:0/3196557914 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc007820 con 0x7f12cc15dc20 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.598+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc1f3af0 con 0x7f12cc15dc20 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.598+0000 7f12c27fc640 1 -- 172.21.15.60:0/3196557914 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc007bd0 con 0x7f12cc15dc20 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.598+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc0cc910 con 0x7f12cc15dc20 2024-04-25T08:33:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.599+0000 7f12c27fc640 1 -- 172.21.15.60:0/3196557914 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12bc018070 con 0x7f12cc15dc20 2024-04-25T08:33:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.600+0000 7f12c27fc640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.600+0000 7f12c27fc640 1 -- 172.21.15.60:0/3196557914 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f12bc08a550 con 0x7f12cc15dc20 2024-04-25T08:33:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.600+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.600+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f12c8034000 tx=0x7f12c802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 Processor -- start 2024-04-25T08:33:29.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 -- start start 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 0x55e74eda4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edc1ad0 0x55e74edc2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74ed79f10 0x55e74edc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74ecdca10 con 0x55e74eda4230 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74ec85b30 con 0x55e74ed79f10 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.601+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e74ecd8c50 con 0x55e74edc1ad0 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 0x55e74eda4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 0x55e74eda4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42766/0 (socket says 172.21.15.60:42766) 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1604678114 learned_addr learned my addr 172.21.15.60:0/1604678114 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d15dc640 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edc1ad0 0x55e74edc2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d25de640 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74ed79f10 0x55e74edc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edc1ad0 msgr2=0x55e74edc2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edc1ad0 0x55e74edc2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74ed79f10 msgr2=0x55e74edc0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74ed79f10 0x55e74edc0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 -- 172.21.15.60:0/1604678114 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e74ec9ae10 con 0x55e74eda4230 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 0x55e74eda4610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f12c802a290 tx=0x7f12c802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2d96adcd4a61f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12af7fe640 1 -- 172.21.15.60:0/1604678114 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c804e070 con 0x55e74eda4230 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12af7fe640 1 -- 172.21.15.60:0/1604678114 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c803e460 con 0x55e74eda4230 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.602+0000 7f12af7fe640 1 -- 172.21.15.60:0/1604678114 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8037040 con 0x55e74eda4230 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 -- 172.21.15.60:0/1604678114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 msgr2=0x55e74eda4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 0x55e74eda4610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f12c802a290 tx=0x7f12c802c000 comp rx=0 tx=0).stop 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 -- 172.21.15.60:0/1604678114 shutdown_connections 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74ed79f10 0x55e74edc0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edc1ad0 0x55e74edc2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 --2- 172.21.15.60:0/1604678114 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74eda4230 0x55e74eda4610 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 -- 172.21.15.60:0/1604678114 >> 172.21.15.60:0/1604678114 conn(0x55e74ed76840 msgr2=0x55e74edbf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 -- 172.21.15.60:0/1604678114 shutdown_connections 2024-04-25T08:33:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 -- 172.21.15.60:0/1604678114 wait complete. 2024-04-25T08:33:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.603+0000 7f12d6e17740 1 -- start start 2024-04-25T08:33:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 Processor -- start 2024-04-25T08:33:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edc0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74edc05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74ed7acb0 con 0x55e74edc1ad0 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74ec990c0 con 0x55e74eda4230 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e74ed74770 con 0x55e74edb6f00 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d6e17740 10 client.0 _ll_register_callbacks cb 0x55e74ec8cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.604+0000 7f12d15dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edc0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edc0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42770/0 (socket says 172.21.15.60:42770) 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 -- 172.21.15.60:0/2274294714 learned_addr learned my addr 172.21.15.60:0/2274294714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d25de640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74edc05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 -- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 msgr2=0x55e74edc05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74edc05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 -- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 msgr2=0x55e74edb69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 -- 172.21.15.60:0/2274294714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e74e9f12d0 con 0x55e74edc1ad0 2024-04-25T08:33:29.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d25de640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74edc05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d15dc640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edc0090 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0587f0 tx=0x7f12bc058c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b41582f910e8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12acff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12acff9640 1 -- 172.21.15.60:0/2274294714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc01f040 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12acff9640 1 -- 172.21.15.60:0/2274294714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f12bc05dd60 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12c803e460 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12acff9640 1 -- 172.21.15.60:0/2274294714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12bc08a0b0 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d6e17740 10 client.6972 fetch_fsmap learned FSMap version 42 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e74eabb6b0 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.605+0000 7f12acff9640 1 -- 172.21.15.60:0/2274294714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c803e460 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12acff9640 1 -- 172.21.15.60:0/2274294714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f12bc057f40 con 0x55e74edc1ad0 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d6e17740 10 client.6972 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d6e17740 10 client.6972 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:33:29.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d6e17740 20 client.6972 populate_metadata read hostname 'smithi060' 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d6e17740 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda7dd0 0x55e74eda54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d6e17740 4 client.6972 mds_command: new command op to 6936 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 --> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e74ed3b920 con 0x55e74eda7dd0 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda7dd0 0x55e74eda54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda7dd0 0x55e74eda54c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd66122b3ad0f6aa server_cookie=4de71ad0478786a1 in_seq=0 out_seq=0 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.606+0000 7f12acff9640 10 client.6972 ms_handle_connect on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12acff9640 1 -- 172.21.15.60:0/2274294714 <== mds.0 v2:172.21.15.60:6834/1247115122 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e74ed3b920 con 0x55e74eda7dd0 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12acff9640 10 client.6972 handle_command_reply: tid=0 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 client.6972 shutdown 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 msgr2=0x55e74edc0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edc0090 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0587f0 tx=0x7f12bc058c00 comp rx=0 tx=0).stop 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 shutdown_connections 2024-04-25T08:33:29.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda7dd0 0x55e74eda54c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74edc05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edc0090 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 --2- 172.21.15.60:0/2274294714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 >> 172.21.15.60:0/2274294714 conn(0x55e74ed79f10 msgr2=0x55e74edbe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12acff9640 0 client.6972 ms_handle_reset on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 shutdown_connections 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 1 -- 172.21.15.60:0/2274294714 wait complete. 2024-04-25T08:33:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.607+0000 7f12d6e17740 20 client.6972 trim_cache size 0 max 16384 2024-04-25T08:33:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 Processor -- start 2024-04-25T08:33:29.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 -- start start 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74eda4230 0x55e74ede1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 0x55e74edcfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74edc1ad0 0x55e74edcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1ff590 con 0x55e74edb6f00 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1fe990 con 0x55e74edc1ad0 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.623+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1fef90 con 0x55e74eda4230 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 0x55e74edcfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 0x55e74edcfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42774/0 (socket says 172.21.15.60:42774) 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 -- 172.21.15.60:0/3494428989 learned_addr learned my addr 172.21.15.60:0/3494428989 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d25de640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74edc1ad0 0x55e74edcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74eda4230 0x55e74ede1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 -- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74eda4230 msgr2=0x55e74ede1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74eda4230 0x55e74ede1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 -- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74edc1ad0 msgr2=0x55e74edcd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74edc1ad0 0x55e74edcd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 -- 172.21.15.60:0/3494428989 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e74ed70550 con 0x55e74edb6f00 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d25de640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74edc1ad0 0x55e74edcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74eda4230 0x55e74ede1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 0x55e74edcfa90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f12bc08a010 tx=0x7f12bc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ec8546b294a560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12aeffd640 1 -- 172.21.15.60:0/3494428989 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc01f050 con 0x55e74edb6f00 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12aeffd640 1 -- 172.21.15.60:0/3494428989 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc05db60 con 0x55e74edb6f00 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.624+0000 7f12aeffd640 1 -- 172.21.15.60:0/3494428989 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc054840 con 0x55e74edb6f00 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 -- 172.21.15.60:0/3494428989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 msgr2=0x55e74edcfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 0x55e74edcfa90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f12bc08a010 tx=0x7f12bc01b000 comp rx=0 tx=0).stop 2024-04-25T08:33:29.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 -- 172.21.15.60:0/3494428989 shutdown_connections 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74edc1ad0 0x55e74edcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edb6f00 0x55e74edcfa90 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3494428989 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74eda4230 0x55e74ede1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 -- 172.21.15.60:0/3494428989 >> 172.21.15.60:0/3494428989 conn(0x55e74ed79f10 msgr2=0x55e74ed79430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 -- 172.21.15.60:0/3494428989 shutdown_connections 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 -- 172.21.15.60:0/3494428989 wait complete. 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 -- start start 2024-04-25T08:33:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.625+0000 7f12d6e17740 1 Processor -- start 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74ed9e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edbf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74eda7c50 con 0x55e74edc1ad0 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74eda7260 con 0x55e74eda4230 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e74eda8250 con 0x55e74edb6f00 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d6e17740 10 client.0 _ll_register_callbacks cb 0x55e74ed74c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d25de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edbf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d25de640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edbf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42786/0 (socket says 172.21.15.60:42786) 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d25de640 1 -- 172.21.15.60:0/3457016125 learned_addr learned my addr 172.21.15.60:0/3457016125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d15dc640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74ed9e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.626+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d25de640 1 -- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 msgr2=0x55e74ed9e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d25de640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74ed9e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d25de640 1 -- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 msgr2=0x55e74edb6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d25de640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d25de640 1 -- 172.21.15.60:0/3457016125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e74ee14260 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d25de640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edbf7c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f12c40012e0 tx=0x7f12c40057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc057e21a94be62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12acff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12acff9640 1 -- 172.21.15.60:0/3457016125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c401b020 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12acff9640 1 -- 172.21.15.60:0/3457016125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f12c4001fe0 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12bc05db60 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12acff9640 1 -- 172.21.15.60:0/3457016125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12c4025050 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d6e17740 10 client.6978 fetch_fsmap learned FSMap version 42 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e74edd0d70 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12acff9640 1 -- 172.21.15.60:0/3457016125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc05db60 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12acff9640 1 -- 172.21.15.60:0/3457016125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f12c4028020 con 0x55e74edc1ad0 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.627+0000 7f12d6e17740 10 client.6978 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 10 client.6978 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:33:29.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 20 client.6978 populate_metadata read hostname 'smithi060' 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda2350 0x55e74eda2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 4 client.6978 mds_command: new command op to 6936 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 --> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e74ece9f50 con 0x55e74eda2350 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda2350 0x55e74eda2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d1ddd640 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda2350 0x55e74eda2730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717ed1b5b3fa3ba0 server_cookie=35e5f1b93f2a86b5 in_seq=0 out_seq=0 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12acff9640 10 client.6978 ms_handle_connect on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12acff9640 1 -- 172.21.15.60:0/3457016125 <== mds.0 v2:172.21.15.60:6834/1247115122 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e74ece9f50 con 0x55e74eda2350 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12acff9640 10 client.6978 handle_command_reply: tid=0 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 client.6978 shutdown 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 msgr2=0x55e74edbf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edbf7c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f12c40012e0 tx=0x7f12c40057d0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 shutdown_connections 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e74edc1ad0 0x55e74edbf7c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x55e74eda2350 0x55e74eda2730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55e74edb6f00 0x55e74ed9e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 --2- 172.21.15.60:0/3457016125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e74eda4230 0x55e74edb6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12acff9640 0 client.6978 ms_handle_reset on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 >> 172.21.15.60:0/3457016125 conn(0x55e74ed79f10 msgr2=0x55e74ed79430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 shutdown_connections 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.628+0000 7f12d6e17740 1 -- 172.21.15.60:0/3457016125 wait complete. 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d6e17740 20 client.6978 trim_cache size 0 max 16384 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:33:29.630 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 6936, 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 4.4747918569999996, 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 42, 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 71, 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 71, 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 5.3451890779999998 2024-04-25T08:33:29.631 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f12b006b4d0 msgr2=0x7f12b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f12c8034000 tx=0x7f12c802e000 comp rx=0 tx=0).stop 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 msgr2=0x7f12cc0cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0cbac0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00c7f0 tx=0x7f12bc00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 shutdown_connections 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12cc161710 0x7f12cc0cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12cc15dc20 0x7f12cc0cbac0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 --2- 172.21.15.60:0/3196557914 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12cc0b9330 0x7f12cc15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 >> 172.21.15.60:0/3196557914 conn(0x7f12cc0d0420 msgr2=0x7f12cc0c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 shutdown_connections 2024-04-25T08:33:29.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.629+0000 7f12d4041640 1 -- 172.21.15.60:0/3196557914 wait complete. 2024-04-25T08:33:29.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:33:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b0d9d5640 1 Processor -- start 2024-04-25T08:33:29.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b0d9d5640 1 -- start start 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b0d9d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7b080d1020 0x7f7b080d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b0d9d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080b97a0 con 0x7f7b080b2cb0 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b0d9d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b080d1c50 con 0x7f7b0815dbe0 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b0d9d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0815dfc0 con 0x7f7b080d1020 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.741+0000 7f7b067fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7b080b2cb0 0x7f7b080d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35530/0 (socket says 172.21.15.60:35530) 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b067fc640 1 -- 172.21.15.60:0/1444000890 learned_addr learned my addr 172.21.15.60:0/1444000890 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b077fe640 1 --2- 172.21.15.60:0/1444000890 >> v2:172.21.15.100:3300/0 conn(0x7f7b080d1020 0x7f7b080d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b05ffb640 1 -- 172.21.15.60:0/1444000890 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610233706 0 0) 0x7f7b080b97a0 con 0x7f7b080b2cb0 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b05ffb640 1 -- 172.21.15.60:0/1444000890 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af0003520 con 0x7f7b080b2cb0 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b077fe640 1 -- 172.21.15.60:0/1444000890 >> v1:172.21.15.100:6789/0 conn(0x7f7b0815dbe0 legacy=0x7f7b080d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b077fe640 1 -- 172.21.15.60:0/1444000890 >> v1:172.21.15.60:6789/0 conn(0x7f7b080b2cb0 legacy=0x7f7b080d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.742+0000 7f7b077fe640 1 -- 172.21.15.60:0/1444000890 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b080d1f30 con 0x7f7b080d1020 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b077fe640 1 --2- 172.21.15.60:0/1444000890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080d1020 0x7f7b080d2e40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009d90 tx=0x7f7b0002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb18c0665fc9c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b05ffb640 1 -- 172.21.15.60:0/1444000890 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00007740 con 0x7f7b080d1020 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b05ffb640 1 -- 172.21.15.60:0/1444000890 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b000078e0 con 0x7f7b080d1020 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b05ffb640 1 -- 172.21.15.60:0/1444000890 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00007cb0 con 0x7f7b080d1020 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/1444000890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080d1020 msgr2=0x7f7b080d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/1444000890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080d1020 0x7f7b080d2e40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009d90 tx=0x7f7b0002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/1444000890 shutdown_connections 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/1444000890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080d1020 0x7f7b080d2e40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/1444000890 >> 172.21.15.60:0/1444000890 conn(0x7f7b08061a20 msgr2=0x7f7b08061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.743+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/1444000890 shutdown_connections 2024-04-25T08:33:29.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.744+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/1444000890 wait complete. 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.745+0000 7f7b0d9d5640 1 Processor -- start 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.745+0000 7f7b0d9d5640 1 -- start start 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080b2cb0 0x7f7b0814be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b080d1020 0x7f7b08157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 0x7f7b08157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080b92a0 con 0x7f7b0815dbe0 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080d1a00 con 0x7f7b080b2cb0 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080b9420 con 0x7f7b080d1020 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 0x7f7b08157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 0x7f7b08157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42802/0 (socket says 172.21.15.60:42802) 2024-04-25T08:33:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 -- 172.21.15.60:0/2635293346 learned_addr learned my addr 172.21.15.60:0/2635293346 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b067fc640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b080d1020 0x7f7b08157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b06ffd640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080b2cb0 0x7f7b0814be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 -- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b080d1020 msgr2=0x7f7b08157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b080d1020 0x7f7b08157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 -- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080b2cb0 msgr2=0x7f7b0814be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080b2cb0 0x7f7b0814be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 -- 172.21.15.60:0/2635293346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b080b3f60 con 0x7f7b0815dbe0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b077fe640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 0x7f7b08157fc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7b0002ace0 tx=0x7f7b00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00040070 con 0x7f7b0815dbe0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00031bc0 con 0x7f7b0815dbe0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b080b4240 con 0x7f7b0815dbe0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00009450 con 0x7f7b0815dbe0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.746+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b080b4520 con 0x7f7b0815dbe0 2024-04-25T08:33:29.744 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.747+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b08073990 con 0x7f7b0815dbe0 2024-04-25T08:33:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.748+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b00039040 con 0x7f7b0815dbe0 2024-04-25T08:33:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.748+0000 7f7aeffff640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7af406b500 0x7f7af406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.748+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7b000aba70 con 0x7f7b0815dbe0 2024-04-25T08:33:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.749+0000 7f7b06ffd640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7af406b500 0x7f7af406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.749+0000 7f7b06ffd640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7af406b500 0x7f7af406d9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7af80016e0 tx=0x7f7af8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:29.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.753+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b0007a0c0 con 0x7f7b0815dbe0 2024-04-25T08:33:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.971+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b0815d2d0 con 0x7f7b0815dbe0 2024-04-25T08:33:29.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.971+0000 7f7aeffff640 1 -- 172.21.15.60:0/2635293346 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7b0007e160 con 0x7f7b0815dbe0 2024-04-25T08:33:29.967 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7af406b500 msgr2=0x7f7af406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7af406b500 0x7f7af406d9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7af80016e0 tx=0x7f7af8009040 comp rx=0 tx=0).stop 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 msgr2=0x7f7b08157fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 0x7f7b08157fc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7b0002ace0 tx=0x7f7b00004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 shutdown_connections 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7b0815dbe0 0x7f7b08157fc0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7b080d1020 0x7f7b08157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7af406b500 0x7f7af406d9c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 --2- 172.21.15.60:0/2635293346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b080b2cb0 0x7f7b0814be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 >> 172.21.15.60:0/2635293346 conn(0x7f7b08061a20 msgr2=0x7f7b08158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 shutdown_connections 2024-04-25T08:33:29.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:29.973+0000 7f7b0d9d5640 1 -- 172.21.15.60:0/2635293346 wait complete. 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:33:29.980 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:33:29.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:29.980 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:33:29.996 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42500sec preferred_lft 42500sec 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:33:29.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:29.998 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:33:30.059 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:33:30.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.061 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:33:30.117 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:33:30.117 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:33:30.117 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:33:30.118 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:33:30.118 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:33:30.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.118 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:33:30.118 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:33:30.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:33:30.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:33:30.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:33:30.118 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:33:30.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.203 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:33:30.218 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:33:30.218 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:33:30.218 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:33:30.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:33:30.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:33:30.283 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:30.339 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:33:30.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:33:30.339 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:30.394 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:33:30.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:33:30.476 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:33:30.485 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:30.485 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:33:30.485 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:33:30.485 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:33:30.485 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:33:30.490 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:33:30.491 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:33:30.491 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:33:30.491 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:33:30.491 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:33:30.491 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:33:30.492 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:33:30.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:30.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.557 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:33:30.614 INFO:teuthology.orchestra.run.smithi060.stdout:client5819 2024-04-25T08:33:30.614 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:30.614 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:30.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:30.670 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T08:33:30.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:33:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T08:33:30.727 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T08:33:30.727 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:33:30.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T08:33:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.878+0000 7f14db2dd640 1 Processor -- start 2024-04-25T08:33:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.878+0000 7f14db2dd640 1 -- start start 2024-04-25T08:33:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14db2dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 0x7f14d40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14db2dd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d406cf50 con 0x7f14d406c9e0 2024-04-25T08:33:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14db2dd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d40ce530 con 0x7f14d40b5620 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14db2dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40ce810 con 0x7f14d415b7d0 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14d987a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f14d406c9e0 0x7f14d40d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35542/0 (socket says 172.21.15.60:35542) 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14d987a640 1 -- 172.21.15.60:0/2759822251 learned_addr learned my addr 172.21.15.60:0/2759822251 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271305039 0 0) 0x7f14d406cf50 con 0x7f14d406c9e0 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c0003520 con 0x7f14d406c9e0 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1415953511 0 0) 0x7f14c0003520 con 0x7f14d406c9e0 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 msgr2=0x7f14d40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.879+0000 7f14cbfff640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 0x7f14d40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 >> v1:172.21.15.100:6789/0 conn(0x7f14d40b5620 legacy=0x7f14d406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40ceaf0 con 0x7f14d406c9e0 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f14cc0019f0 con 0x7f14d406c9e0 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 >> v1:172.21.15.60:6789/0 conn(0x7f14d406c9e0 legacy=0x7f14d40d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 0x7f14c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14c00040d0 0x7f14c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14c0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40ce810 con 0x7f14c0004db0 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c000bc20 con 0x7f14c00040d0 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c000be20 con 0x7f14d415b7d0 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d987a640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14c0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d9079640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 0x7f14c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d8878640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14c00040d0 0x7f14c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d987a640 1 -- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 msgr2=0x7f14c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d987a640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d415b7d0 0x7f14c00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d987a640 1 -- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14c00040d0 msgr2=0x7f14c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d987a640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14c00040d0 0x7f14c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.880+0000 7f14d987a640 1 -- 172.21.15.60:0/2759822251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0003520 con 0x7f14c0004db0 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14d987a640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14c0005230 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00a860 tx=0x7f14c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a49d68462579db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00a480 con 0x7f14c0004db0 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc02f070 con 0x7f14c0004db0 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14cbfff640 1 -- 172.21.15.60:0/2759822251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc02bc40 con 0x7f14c0004db0 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14db2dd640 1 -- 172.21.15.60:0/2759822251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 msgr2=0x7f14c0005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14db2dd640 1 --2- 172.21.15.60:0/2759822251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14c0005230 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00a860 tx=0x7f14c0005830 comp rx=0 tx=0).stop 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14d987a640 1 -- 172.21.15.60:0/2759822251 reap_dead start 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14db2dd640 1 -- 172.21.15.60:0/2759822251 shutdown_connections 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.881+0000 7f14db2dd640 1 -- 172.21.15.60:0/2759822251 >> 172.21.15.60:0/2759822251 conn(0x7f14d40d0800 msgr2=0x7f14d40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.882+0000 7f14db2dd640 1 -- 172.21.15.60:0/2759822251 shutdown_connections 2024-04-25T08:33:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.882+0000 7f14db2dd640 1 -- 172.21.15.60:0/2759822251 wait complete. 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.883+0000 7f14db2dd640 1 Processor -- start 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.883+0000 7f14db2dd640 1 -- start start 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.883+0000 7f14db2dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14d40c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.883+0000 7f14db2dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d41f7280 0x7f14d41f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.883+0000 7f14db2dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d40b8090 0x7f14d40b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14db2dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d415bd30 con 0x7f14c0004db0 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14db2dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d415bbb0 con 0x7f14d41f7280 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14db2dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40d2650 con 0x7f14d40b8090 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14d40c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14d40c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42818/0 (socket says 172.21.15.60:42818) 2024-04-25T08:33:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 -- 172.21.15.60:0/1188556312 learned_addr learned my addr 172.21.15.60:0/1188556312 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d987a640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d40b8090 0x7f14d40b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d8878640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d41f7280 0x7f14d41f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 -- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d40b8090 msgr2=0x7f14d40b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d40b8090 0x7f14d40b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 -- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d41f7280 msgr2=0x7f14d41f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d41f7280 0x7f14d41f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 -- 172.21.15.60:0/1188556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d41f7700 con 0x7f14c0004db0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d987a640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d40b8090 0x7f14d40b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d8878640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d41f7280 0x7f14d41f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14d9079640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14d40c76d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e8a0 tx=0x7f14d000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14c9ffb640 1 -- 172.21.15.60:0/1188556312 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000ee90 con 0x7f14c0004db0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14c9ffb640 1 -- 172.21.15.60:0/1188556312 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d0004030 con 0x7f14c0004db0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d40c7d40 con 0x7f14c0004db0 2024-04-25T08:33:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.884+0000 7f14c9ffb640 1 -- 172.21.15.60:0/1188556312 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0004440 con 0x7f14c0004db0 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.885+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d40c8300 con 0x7f14c0004db0 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.886+0000 7f14c9ffb640 1 -- 172.21.15.60:0/1188556312 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14d000c940 con 0x7f14c0004db0 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.886+0000 7f14c9ffb640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b670 0x7f14b806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.886+0000 7f14c9ffb640 1 -- 172.21.15.60:0/1188556312 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f14d008a270 con 0x7f14c0004db0 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.886+0000 7f14d8878640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b670 0x7f14b806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.887+0000 7f14d8878640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b670 0x7f14b806db30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f14d415f150 tx=0x7f14c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.887+0000 7f14ddfda740 1 Processor -- start 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.887+0000 7f14ddfda740 1 -- start start 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074700 0x5629da049510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14ddfda740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d9facf40 con 0x5629da074e40 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d9f4c0d0 con 0x5629da074700 2024-04-25T08:33:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629d9fa8fe0 con 0x5629da091d20 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42828/0 (socket says 172.21.15.60:42828) 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 -- 172.21.15.60:0/682716150 learned_addr learned my addr 172.21.15.60:0/682716150 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d9079640 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d987a640 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074700 0x5629da049510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 -- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 msgr2=0x5629da0446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 -- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074700 msgr2=0x5629da049510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074700 0x5629da049510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.888+0000 7f14d8878640 1 -- 172.21.15.60:0/682716150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d9d8daf0 con 0x5629da074e40 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14d8878640 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0741c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f14c4011af0 tx=0x7f14c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836ceb5f39eff8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14beffd640 1 -- 172.21.15.60:0/682716150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4028070 con 0x5629da074e40 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14beffd640 1 -- 172.21.15.60:0/682716150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4015c00 con 0x5629da074e40 2024-04-25T08:33:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14beffd640 1 -- 172.21.15.60:0/682716150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40143d0 con 0x5629da074e40 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 -- 172.21.15.60:0/682716150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 msgr2=0x5629da0741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0741c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f14c4011af0 tx=0x7f14c4014000 comp rx=0 tx=0).stop 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 -- 172.21.15.60:0/682716150 shutdown_connections 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074700 0x5629da049510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0741c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 --2- 172.21.15.60:0/682716150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 -- 172.21.15.60:0/682716150 >> 172.21.15.60:0/682716150 conn(0x5629da046920 msgr2=0x5629da08f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 -- 172.21.15.60:0/682716150 shutdown_connections 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.889+0000 7f14ddfda740 1 -- 172.21.15.60:0/682716150 wait complete. 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.890+0000 7f14ddfda740 1 -- start start 2024-04-25T08:33:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.890+0000 7f14ddfda740 1 Processor -- start 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da091d20 0x5629da06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da077cb0 0x5629da086890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629da048ad0 con 0x5629da074e40 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d9f49ba0 con 0x5629da091d20 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629da047ec0 con 0x5629da077cb0 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14ddfda740 10 client.0 _ll_register_callbacks cb 0x5629d9f49570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42840/0 (socket says 172.21.15.60:42840) 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 -- 172.21.15.60:0/4246202224 learned_addr learned my addr 172.21.15.60:0/4246202224 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d987a640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da077cb0 0x5629da086890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d8878640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da091d20 0x5629da06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 -- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da077cb0 msgr2=0x5629da086890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da077cb0 0x5629da086890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 -- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da091d20 msgr2=0x5629da06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da091d20 0x5629da06fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d9079640 1 -- 172.21.15.60:0/4246202224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629d9c813e0 con 0x5629da074e40 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.891+0000 7f14d8878640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da091d20 0x5629da06fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14d9079640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0a0b40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f14d0054d00 tx=0x7f14d0059060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d224879aeff6ebd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14b3fff640 1 -- 172.21.15.60:0/4246202224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0070020 con 0x5629da074e40 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14b3fff640 1 -- 172.21.15.60:0/4246202224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f14d00703c0 con 0x5629da074e40 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14c4015c00 con 0x5629da074e40 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14b3fff640 1 -- 172.21.15.60:0/4246202224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d0020040 con 0x5629da074e40 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 10 client.6996 fetch_fsmap learned FSMap version 42 2024-04-25T08:33:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x5629d9dee280 con 0x5629da074e40 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14b3fff640 1 -- 172.21.15.60:0/4246202224 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4015c00 con 0x5629da074e40 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14b3fff640 1 -- 172.21.15.60:0/4246202224 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14d0023060 con 0x5629da074e40 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 10 client.6996 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 10 client.6996 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 20 client.6996 populate_metadata read hostname 'smithi060' 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0880c0 0x5629da0a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 4 client.6996 mds_command: new command op to 6936 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 --> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629d9fba480 con 0x5629da0880c0 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.892+0000 7f14d8878640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0880c0 0x5629da0a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14d8878640 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0880c0 0x5629da0a2f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac317c3598e005c8 server_cookie=4911f5384faddbbc in_seq=0 out_seq=0 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14b3fff640 10 client.6996 ms_handle_connect on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14b3fff640 1 -- 172.21.15.60:0/4246202224 <== mds.0 v2:172.21.15.60:6834/1247115122 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629d9fba480 con 0x5629da0880c0 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14b3fff640 10 client.6996 handle_command_reply: tid=0 2024-04-25T08:33:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 client.6996 shutdown 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 msgr2=0x5629da0a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0a0b40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f14d0054d00 tx=0x7f14d0059060 comp rx=0 tx=0).stop 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 shutdown_connections 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0880c0 0x5629da0a2f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da077cb0 0x5629da086890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da091d20 0x5629da06fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 --2- 172.21.15.60:0/4246202224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da074e40 0x5629da0a0b40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14b3fff640 0 client.6996 ms_handle_reset on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 >> 172.21.15.60:0/4246202224 conn(0x5629da074700 msgr2=0x5629da08f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 shutdown_connections 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.893+0000 7f14ddfda740 1 -- 172.21.15.60:0/4246202224 wait complete. 2024-04-25T08:33:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.894+0000 7f14ddfda740 20 client.6996 trim_cache size 0 max 16384 2024-04-25T08:33:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 Processor -- start 2024-04-25T08:33:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 -- start start 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da06fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da08efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da08f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d4200c00 con 0x5629da077cb0 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d41ffdd0 con 0x5629da074e40 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40ccf40 con 0x5629da091d20 2024-04-25T08:33:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.909+0000 7f14d8878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da08efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da08efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42850/0 (socket says 172.21.15.60:42850) 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 -- 172.21.15.60:0/3250107710 learned_addr learned my addr 172.21.15.60:0/3250107710 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 -- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 msgr2=0x5629da08f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d987a640 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da08f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da08f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d9079640 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da06fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 -- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 msgr2=0x5629da06fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da06fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 -- 172.21.15.60:0/3250107710 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d9c5a560 con 0x5629da077cb0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14d8878640 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da08efc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f14c40055e0 tx=0x7f14c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc23fc3f6d5d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14be7fc640 1 -- 172.21.15.60:0/3250107710 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4028040 con 0x5629da077cb0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14be7fc640 1 -- 172.21.15.60:0/3250107710 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4038070 con 0x5629da077cb0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.910+0000 7f14be7fc640 1 -- 172.21.15.60:0/3250107710 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c402f870 con 0x5629da077cb0 2024-04-25T08:33:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 -- 172.21.15.60:0/3250107710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 msgr2=0x5629da08efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da08efc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f14c40055e0 tx=0x7f14c4016000 comp rx=0 tx=0).stop 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 -- 172.21.15.60:0/3250107710 shutdown_connections 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da08f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da08efc0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 --2- 172.21.15.60:0/3250107710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da06fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 -- 172.21.15.60:0/3250107710 >> 172.21.15.60:0/3250107710 conn(0x5629da074700 msgr2=0x5629da044560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 -- 172.21.15.60:0/3250107710 shutdown_connections 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 -- 172.21.15.60:0/3250107710 wait complete. 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 -- start start 2024-04-25T08:33:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.911+0000 7f14ddfda740 1 Processor -- start 2024-04-25T08:33:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da049680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da0a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629da0b2cd0 con 0x5629da077cb0 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d41ff7d0 con 0x5629da074e40 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629d9fb8360 con 0x5629da091d20 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14ddfda740 10 client.0 _ll_register_callbacks cb 0x7f14d00047d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14d8878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da0a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14d8878640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da0a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42864/0 (socket says 172.21.15.60:42864) 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14d8878640 1 -- 172.21.15.60:0/2349077303 learned_addr learned my addr 172.21.15.60:0/2349077303 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14d9079640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da049680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.912+0000 7f14d987a640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d8878640 1 -- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 msgr2=0x5629da0a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d8878640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d8878640 1 -- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 msgr2=0x5629da049680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d8878640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da049680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d8878640 1 -- 172.21.15.60:0/2349077303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629da09f8d0 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d987a640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14d8878640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da0a8f30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f14c4011620 tx=0x7f14c4000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2ad8acfb45e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14b3fff640 1 -- 172.21.15.60:0/2349077303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4041060 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14b3fff640 1 -- 172.21.15.60:0/2349077303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f14c402f630 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14c4038070 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14b3fff640 1 -- 172.21.15.60:0/2349077303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14c4028050 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 10 client.7002 fetch_fsmap learned FSMap version 42 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x5629da047fb0 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14b3fff640 1 -- 172.21.15.60:0/2349077303 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4038070 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14b3fff640 1 -- 172.21.15.60:0/2349077303 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14c402f830 con 0x5629da077cb0 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 10 client.7002 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 10 client.7002 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T08:33:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 20 client.7002 populate_metadata read hostname 'smithi060' 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0dd6d0 0x5629da048470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 4 client.7002 mds_command: new command op to 6936 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.913+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 --> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x5629d9fb47d0 con 0x5629da0dd6d0 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14d9079640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0dd6d0 0x5629da048470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14d9079640 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0dd6d0 0x5629da048470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29cbc1a158fc4b5b server_cookie=4a8286e654c4039 in_seq=0 out_seq=0 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14b3fff640 10 client.7002 ms_handle_connect on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14b3fff640 1 -- 172.21.15.60:0/2349077303 <== mds.0 v2:172.21.15.60:6834/1247115122 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5629d9fb47d0 con 0x5629da0dd6d0 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14b3fff640 10 client.7002 handle_command_reply: tid=0 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 client.7002 shutdown 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 msgr2=0x5629da0a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da0a8f30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f14c4011620 tx=0x7f14c4000fb0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 shutdown_connections 2024-04-25T08:33:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:6834/1247115122,v1:172.21.15.60:6835/1247115122] conn(0x5629da0dd6d0 0x5629da048470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5629da091d20 0x5629da0a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629da077cb0 0x5629da0a8f30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14b3fff640 0 client.7002 ms_handle_reset on v2:172.21.15.60:6834/1247115122 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 --2- 172.21.15.60:0/2349077303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5629da074e40 0x5629da049680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.914+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 >> 172.21.15.60:0/2349077303 conn(0x5629da074700 msgr2=0x5629da0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 shutdown_connections 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14ddfda740 1 -- 172.21.15.60:0/2349077303 wait complete. 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14ddfda740 20 client.7002 trim_cache size 0 max 16384 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:33:30.917 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:33:30.918 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "79f953c5-1d19-4647-8804-6b33de2973ec", 2024-04-25T08:33:30.918 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:33:30.918 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b670 msgr2=0x7f14b806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b670 0x7f14b806db30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f14d415f150 tx=0x7f14c4009040 comp rx=0 tx=0).stop 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 msgr2=0x7f14d40c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14d40c76d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e8a0 tx=0x7f14d000ed60 comp rx=0 tx=0).stop 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 shutdown_connections 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d40b8090 0x7f14d40b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b670 0x7f14b806db30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d41f7280 0x7f14d41f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 --2- 172.21.15.60:0/1188556312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14c0004db0 0x7f14d40c76d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 >> 172.21.15.60:0/1188556312 conn(0x7f14d40d0800 msgr2=0x7f14d40d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 shutdown_connections 2024-04-25T08:33:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:30.915+0000 7f14db2dd640 1 -- 172.21.15.60:0/1188556312 wait complete. 2024-04-25T08:33:30.921 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '79f953c5-1d19-4647-8804-6b33de2973ec', 'mode': 'asynchronous'}' 2024-04-25T08:33:31.149 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:33:31.154+0000 7f1703d8f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:33:40.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:40.923 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T08:33:40.944 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:33:40.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:33:41.026 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:33:41.065 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:33:41.066 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:41.066 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:41.084 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:41.084 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:41.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:41.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.293+0000 7fb9dc4fc640 1 Processor -- start 2024-04-25T08:33:41.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.293+0000 7fb9dc4fc640 1 -- start start 2024-04-25T08:33:41.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.294+0000 7fb9dc4fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9d40d1120 0x7fb9d409a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.294+0000 7fb9dc4fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d406c220 con 0x7fb9d406ce10 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.294+0000 7fb9dc4fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40d0080 con 0x7fb9d415dc70 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.294+0000 7fb9dc4fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d40d0c90 con 0x7fb9d40d1120 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.294+0000 7fb9daa99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9d40d1120 0x7fb9d409a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9daa99640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb9d40d1120 0x7fb9d409a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51008/0 (socket says 172.21.15.60:51008) 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9daa99640 1 -- 172.21.15.60:0/1157430596 learned_addr learned my addr 172.21.15.60:0/1157430596 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9d9a97640 1 --1- 172.21.15.60:0/1157430596 >> v1:172.21.15.60:6789/0 conn(0x7fb9d406ce10 0x7fb9d40cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33372/0 (socket says 172.21.15.60:33372) 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9daa99640 1 -- 172.21.15.60:0/1157430596 >> v1:172.21.15.100:6789/0 conn(0x7fb9d415dc70 legacy=0x7fb9d406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9daa99640 1 -- 172.21.15.60:0/1157430596 >> v1:172.21.15.60:6789/0 conn(0x7fb9d406ce10 legacy=0x7fb9d40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9daa99640 1 -- 172.21.15.60:0/1157430596 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40d0360 con 0x7fb9d40d1120 2024-04-25T08:33:41.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.295+0000 7fb9daa99640 1 --2- 172.21.15.60:0/1157430596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d40d1120 0x7fb9d409a450 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009870 tx=0x7fb9cc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285bdd0aacaaf13b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9d9296640 1 -- 172.21.15.60:0/1157430596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037070 con 0x7fb9d40d1120 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9d9296640 1 -- 172.21.15.60:0/1157430596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc032440 con 0x7fb9d40d1120 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9d9296640 1 -- 172.21.15.60:0/1157430596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc032810 con 0x7fb9d40d1120 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/1157430596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d40d1120 msgr2=0x7fb9d409a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/1157430596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d40d1120 0x7fb9d409a450 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009870 tx=0x7fb9cc02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/1157430596 shutdown_connections 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/1157430596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d40d1120 0x7fb9d409a450 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/1157430596 >> 172.21.15.60:0/1157430596 conn(0x7fb9d40b3270 msgr2=0x7fb9d40b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/1157430596 shutdown_connections 2024-04-25T08:33:41.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.296+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/1157430596 wait complete. 2024-04-25T08:33:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.297+0000 7fb9dc4fc640 1 Processor -- start 2024-04-25T08:33:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 -- start start 2024-04-25T08:33:41.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9d406ce10 0x7fb9d41446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d40d1120 0x7fb9d4145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 0x7fb9d40c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d415e050 con 0x7fb9d415dc70 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40d0920 con 0x7fb9d406ce10 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9dc4fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40cf9c0 con 0x7fb9d40d1120 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9daa99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 0x7fb9d40c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9daa99640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 0x7fb9d40c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51024/0 (socket says 172.21.15.60:51024) 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.298+0000 7fb9daa99640 1 -- 172.21.15.60:0/132512725 learned_addr learned my addr 172.21.15.60:0/132512725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9da298640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9d406ce10 0x7fb9d41446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9daa99640 1 -- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d40d1120 msgr2=0x7fb9d4145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9daa99640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d40d1120 0x7fb9d4145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9daa99640 1 -- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9d406ce10 msgr2=0x7fb9d41446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9daa99640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9d406ce10 0x7fb9d41446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9daa99640 1 -- 172.21.15.60:0/132512725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4144c90 con 0x7fb9d415dc70 2024-04-25T08:33:41.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9daa99640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 0x7fb9d40c5290 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009870 tx=0x7fb9cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc037040 con 0x7fb9d415dc70 2024-04-25T08:33:41.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc042070 con 0x7fb9d415dc70 2024-04-25T08:33:41.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d4144f70 con 0x7fb9d415dc70 2024-04-25T08:33:41.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc030c90 con 0x7fb9d415dc70 2024-04-25T08:33:41.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.299+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d4145530 con 0x7fb9d415dc70 2024-04-25T08:33:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.300+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d4073990 con 0x7fb9d415dc70 2024-04-25T08:33:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.301+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9cc047460 con 0x7fb9d415dc70 2024-04-25T08:33:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.301+0000 7fb9d2ffd640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb9c006b4d0 0x7fb9c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.301+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb9cc0ab170 con 0x7fb9d415dc70 2024-04-25T08:33:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.301+0000 7fb9da298640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb9c006b4d0 0x7fb9c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.302+0000 7fb9da298640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb9c006b4d0 0x7fb9c006d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb9d41f7750 tx=0x7fb9c40072f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.305+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9cc0797c0 con 0x7fb9d415dc70 2024-04-25T08:33:41.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.486+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9d40b40a0 con 0x7fb9d415dc70 2024-04-25T08:33:41.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.487+0000 7fb9d2ffd640 1 -- 172.21.15.60:0/132512725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17001 (secure 0 0 0) 0x7fb9cc0a2e20 con 0x7fb9d415dc70 2024-04-25T08:33:41.483 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:41.483 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:24.149268+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/1590331682":"2024-04-26T08:33:24.140229+0000","172.21.15.60:6832/1590331682":"2024-04-26T08:33:24.140229+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb9c006b4d0 msgr2=0x7fb9c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb9c006b4d0 0x7fb9c006d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb9d41f7750 tx=0x7fb9c40072f0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 msgr2=0x7fb9d40c5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 0x7fb9d40c5290 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009870 tx=0x7fb9cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 shutdown_connections 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb9d415dc70 0x7fb9d40c5290 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb9d40d1120 0x7fb9d4145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb9c006b4d0 0x7fb9c006d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 --2- 172.21.15.60:0/132512725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb9d406ce10 0x7fb9d41446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 >> 172.21.15.60:0/132512725 conn(0x7fb9d40b3270 msgr2=0x7fb9d40b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 shutdown_connections 2024-04-25T08:33:41.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.489+0000 7fb9dc4fc640 1 -- 172.21.15.60:0/132512725 wait complete. 2024-04-25T08:33:41.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:33:06.174848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:33:06.884806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:33:41.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:33:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.605+0000 7f0d186e1640 1 Processor -- start 2024-04-25T08:33:41.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.605+0000 7f0d186e1640 1 -- start start 2024-04-25T08:33:41.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d186e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 0x7f0d10168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d186e1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100d2420 con 0x7f0d100d34c0 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d186e1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100d2700 con 0x7f0d1015dac0 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d186e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100d3030 con 0x7f0d100d0ab0 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d15c7c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0d100d34c0 0x7f0d100d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:33384/0 (socket says 172.21.15.60:33384) 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d15c7c640 1 -- 172.21.15.60:0/152403777 learned_addr learned my addr 172.21.15.60:0/152403777 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026373029 0 0) 0x7f0d100d2420 con 0x7f0d100d34c0 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc003520 con 0x7f0d100d34c0 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d16c7e640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 0x7f0d10168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445630472 0 0) 0x7f0cfc003520 con 0x7f0d100d34c0 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.606+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 msgr2=0x7f0d10168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 0x7f0d10168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 >> v1:172.21.15.100:6789/0 conn(0x7f0d1015dac0 legacy=0x7f0d100d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100d29e0 con 0x7f0d100d34c0 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f0d000017f0 con 0x7f0d100d34c0 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 >> v1:172.21.15.60:6789/0 conn(0x7f0d100d34c0 legacy=0x7f0d100d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 0x7f0cfc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cfc003fd0 0x7f0cfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc004d40 0x7f0cfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100d3030 con 0x7f0cfc003fd0 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bbb0 con 0x7f0cfc004d40 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bdb0 con 0x7f0d100d0ab0 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cfc003fd0 0x7f0cfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d1647d640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 0x7f0cfc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d16c7e640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc004d40 0x7f0cfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d15c7c640 1 -- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 msgr2=0x7f0cfc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d100d0ab0 0x7f0cfc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d15c7c640 1 -- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc004d40 msgr2=0x7f0cfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc004d40 0x7f0cfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.607+0000 7f0d15c7c640 1 -- 172.21.15.60:0/152403777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc003520 con 0x7f0cfc003fd0 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cfc003fd0 0x7f0cfc004800 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c160 tx=0x7f0cfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e50c1e896c8835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00007c20 con 0x7f0cfc003fd0 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00007290 con 0x7f0cfc003fd0 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d1547b640 1 -- 172.21.15.60:0/152403777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000057a0 con 0x7f0cfc003fd0 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d186e1640 1 -- 172.21.15.60:0/152403777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cfc003fd0 msgr2=0x7f0cfc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d186e1640 1 --2- 172.21.15.60:0/152403777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cfc003fd0 0x7f0cfc004800 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c160 tx=0x7f0cfc005830 comp rx=0 tx=0).stop 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.608+0000 7f0d16c7e640 1 -- 172.21.15.60:0/152403777 reap_dead start 2024-04-25T08:33:41.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.609+0000 7f0d186e1640 1 -- 172.21.15.60:0/152403777 shutdown_connections 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.609+0000 7f0d186e1640 1 -- 172.21.15.60:0/152403777 >> 172.21.15.60:0/152403777 conn(0x7f0d10061c30 msgr2=0x7f0d1006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.609+0000 7f0d186e1640 1 -- 172.21.15.60:0/152403777 shutdown_connections 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.609+0000 7f0d186e1640 1 -- 172.21.15.60:0/152403777 wait complete. 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.610+0000 7f0d186e1640 1 Processor -- start 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.610+0000 7f0d186e1640 1 -- start start 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.610+0000 7f0d186e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc003fd0 0x7f0d101501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 0x7f0d10160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d10160a90 0x7f0d100cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015dea0 con 0x7f0d1014d850 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100d2100 con 0x7f0cfc003fd0 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100b97d0 con 0x7f0d10160a90 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 0x7f0d10160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 0x7f0d10160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51030/0 (socket says 172.21.15.60:51030) 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 -- 172.21.15.60:0/1004532235 learned_addr learned my addr 172.21.15.60:0/1004532235 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d16c7e640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d10160a90 0x7f0d100cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d1647d640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc003fd0 0x7f0d101501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 -- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d10160a90 msgr2=0x7f0d100cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d10160a90 0x7f0d100cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 -- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc003fd0 msgr2=0x7f0d101501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc003fd0 0x7f0d101501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 -- 172.21.15.60:0/1004532235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d101f3440 con 0x7f0d1014d850 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d15c7c640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 0x7f0d10160550 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d00000ef0 tx=0x7f0d100d28d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00040020 con 0x7f0d1014d850 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d000015b0 con 0x7f0d1014d850 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d100cc2c0 con 0x7f0d1014d850 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00037040 con 0x7f0d1014d850 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.611+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1015eb90 con 0x7f0d1014d850 2024-04-25T08:33:41.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.612+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d10073a00 con 0x7f0d1014d850 2024-04-25T08:33:41.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.613+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d00043020 con 0x7f0d1014d850 2024-04-25T08:33:41.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.613+0000 7f0d06ffd640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.613+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0d000a9ef0 con 0x7f0d1014d850 2024-04-25T08:33:41.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.613+0000 7f0d1647d640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.614+0000 7f0d1647d640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf406b4d0 0x7f0cf406d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.617+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d0004a070 con 0x7f0d1014d850 2024-04-25T08:33:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.799+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d10160e70 con 0x7f0d1014d850 2024-04-25T08:33:41.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.800+0000 7f0d06ffd640 1 -- 172.21.15.60:0/1004532235 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d101f3440 con 0x7f0d1014d850 2024-04-25T08:33:41.795 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf406b4d0 msgr2=0x7f0cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf406b4d0 0x7f0cf406d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c009040 comp rx=0 tx=0).stop 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 msgr2=0x7f0d10160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 0x7f0d10160550 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d00000ef0 tx=0x7f0d100d28d0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 shutdown_connections 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d10160a90 0x7f0d100cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d1014d850 0x7f0d10160550 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf406b4d0 0x7f0cf406d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 --2- 172.21.15.60:0/1004532235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cfc003fd0 0x7f0d101501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 >> 172.21.15.60:0/1004532235 conn(0x7f0d10061c30 msgr2=0x7f0d100c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 shutdown_connections 2024-04-25T08:33:41.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.802+0000 7f0d186e1640 1 -- 172.21.15.60:0/1004532235 wait complete. 2024-04-25T08:33:41.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:33:41.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.917+0000 7f1571438640 1 Processor -- start 2024-04-25T08:33:41.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.917+0000 7f1571438640 1 -- start start 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f1571438640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f156c0ced80 0x7f156c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f1571438640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f156c161710 0x7f156c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f1571438640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c15e000 con 0x7f156c15dc20 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f1571438640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0d1de0 con 0x7f156c0ced80 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f1571438640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c161c60 con 0x7f156c161710 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f156c0ced80 0x7f156c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f156c0ced80 0x7f156c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43580/0 (socket says 172.21.15.60:43580) 2024-04-25T08:33:41.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 -- 172.21.15.60:0/1722245213 learned_addr learned my addr 172.21.15.60:0/1722245213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:41.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 -- 172.21.15.60:0/1722245213 >> v2:172.21.15.100:3300/0 conn(0x7f156c161710 msgr2=0x7f156c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:41.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156b7fe640 1 --2- 172.21.15.60:0/1722245213 >> v2:172.21.15.100:3300/0 conn(0x7f156c161710 0x7f156c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 --2- 172.21.15.60:0/1722245213 >> v2:172.21.15.100:3300/0 conn(0x7f156c161710 0x7f156c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 -- 172.21.15.60:0/1722245213 >> v1:172.21.15.100:6789/0 conn(0x7f156c15dc20 legacy=0x7f156c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:41.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156a7fc640 1 -- 172.21.15.60:0/1722245213 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c161f40 con 0x7f156c0ced80 2024-04-25T08:33:41.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.918+0000 7f156b7fe640 1 --2- 172.21.15.60:0/1722245213 >> v2:172.21.15.100:3300/0 conn(0x7f156c161710 0x7f156c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.919+0000 7f156a7fc640 1 --2- 172.21.15.60:0/1722245213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c0ced80 0x7f156c0d22e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b6570 tx=0x7f156002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580207887a7267d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.919+0000 7f1569ffb640 1 -- 172.21.15.60:0/1722245213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560037070 con 0x7f156c0ced80 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.919+0000 7f1569ffb640 1 -- 172.21.15.60:0/1722245213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1560032440 con 0x7f156c0ced80 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.919+0000 7f1569ffb640 1 -- 172.21.15.60:0/1722245213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560032810 con 0x7f156c0ced80 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 -- 172.21.15.60:0/1722245213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c0ced80 msgr2=0x7f156c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 --2- 172.21.15.60:0/1722245213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c0ced80 0x7f156c0d22e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b6570 tx=0x7f156002a230 comp rx=0 tx=0).stop 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 -- 172.21.15.60:0/1722245213 shutdown_connections 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 --2- 172.21.15.60:0/1722245213 >> v2:172.21.15.100:3300/0 conn(0x7f156c161710 0x7f156c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 --2- 172.21.15.60:0/1722245213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c0ced80 0x7f156c0d22e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 -- 172.21.15.60:0/1722245213 >> 172.21.15.60:0/1722245213 conn(0x7f156c0d0420 msgr2=0x7f156c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 -- 172.21.15.60:0/1722245213 shutdown_connections 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.920+0000 7f1571438640 1 -- 172.21.15.60:0/1722245213 wait complete. 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.921+0000 7f1571438640 1 Processor -- start 2024-04-25T08:33:41.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.921+0000 7f1571438640 1 -- start start 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f1571438640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f156c0ced80 0x7f156c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f1571438640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c15dc20 0x7f156c156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f1571438640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 0x7f156c15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f1571438640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0b5e50 con 0x7f156c161710 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f1571438640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c162500 con 0x7f156c0ced80 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f1571438640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0b5fd0 con 0x7f156c15dc20 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 0x7f156c15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 0x7f156c15a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43584/0 (socket says 172.21.15.60:43584) 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 -- 172.21.15.60:0/3867409021 learned_addr learned my addr 172.21.15.60:0/3867409021 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156a7fc640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c15dc20 0x7f156c156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156affd640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f156c0ced80 0x7f156c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 -- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c15dc20 msgr2=0x7f156c156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c15dc20 0x7f156c156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 -- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f156c0ced80 msgr2=0x7f156c156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f156c0ced80 0x7f156c156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.922+0000 7f156b7fe640 1 -- 172.21.15.60:0/3867409021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c1f3b60 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.923+0000 7f156b7fe640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 0x7f156c15a1c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f156400cca0 tx=0x7f156400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.923+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400f680 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.923+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400f840 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.923+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c1f3e00 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.923+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400fc30 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.923+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c1f43c0 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.924+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c073a00 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.924+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f156400e3e0 con 0x7f156c161710 2024-04-25T08:33:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.924+0000 7f15537fe640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f155806b4d0 0x7f155806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:41.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.925+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f156408af10 con 0x7f156c161710 2024-04-25T08:33:41.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.925+0000 7f156affd640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f155806b4d0 0x7f155806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:41.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.925+0000 7f156affd640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f155806b4d0 0x7f155806d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b2540 tx=0x7f155c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:41.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:41.929+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1564014030 con 0x7f156c161710 2024-04-25T08:33:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.107+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f156c0b2fc0 con 0x7f156c161710 2024-04-25T08:33:42.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.108+0000 7f15537fe640 1 -- 172.21.15.60:0/3867409021 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1564059570 con 0x7f156c161710 2024-04-25T08:33:42.104 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f155806b4d0 msgr2=0x7f155806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f155806b4d0 0x7f155806d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b2540 tx=0x7f155c009060 comp rx=0 tx=0).stop 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 msgr2=0x7f156c15a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 0x7f156c15a1c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f156400cca0 tx=0x7f156400a650 comp rx=0 tx=0).stop 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 shutdown_connections 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f156c161710 0x7f156c15a1c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f156c15dc20 0x7f156c156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f155806b4d0 0x7f155806d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 --2- 172.21.15.60:0/3867409021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f156c0ced80 0x7f156c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 >> 172.21.15.60:0/3867409021 conn(0x7f156c0d0420 msgr2=0x7f156c0b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 shutdown_connections 2024-04-25T08:33:42.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.110+0000 7f1571438640 1 -- 172.21.15.60:0/3867409021 wait complete. 2024-04-25T08:33:42.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:33:42.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.225+0000 7f383e1f5640 1 Processor -- start 2024-04-25T08:33:42.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.225+0000 7f383e1f5640 1 -- start start 2024-04-25T08:33:42.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f383e1f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38380b5620 0x7f38380b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f383e1f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f383806c2d0 0x7f383806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f383e1f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383806cc80 con 0x7f383815b7d0 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f383e1f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383806cf50 con 0x7f38380b5620 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f383e1f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38380ce530 con 0x7f383806c2d0 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38380b5620 0x7f38380b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38380b5620 0x7f38380b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43588/0 (socket says 172.21.15.60:43588) 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 -- 172.21.15.60:0/901363501 learned_addr learned my addr 172.21.15.60:0/901363501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3837fff640 1 --2- 172.21.15.60:0/901363501 >> v2:172.21.15.100:3300/0 conn(0x7f383806c2d0 0x7f383806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 -- 172.21.15.60:0/901363501 >> v2:172.21.15.100:3300/0 conn(0x7f383806c2d0 msgr2=0x7f383806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 --2- 172.21.15.60:0/901363501 >> v2:172.21.15.100:3300/0 conn(0x7f383806c2d0 0x7f383806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 -- 172.21.15.60:0/901363501 >> v1:172.21.15.60:6789/0 conn(0x7f383815b7d0 legacy=0x7f38380cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.226+0000 7f3836ffd640 1 -- 172.21.15.60:0/901363501 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380ce810 con 0x7f38380b5620 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.227+0000 7f3836ffd640 1 --2- 172.21.15.60:0/901363501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b5ab0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3838161890 tx=0x7f382c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8204787fdef66443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.227+0000 7f38367fc640 1 -- 172.21.15.60:0/901363501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c037070 con 0x7f38380b5620 2024-04-25T08:33:42.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.227+0000 7f38367fc640 1 -- 172.21.15.60:0/901363501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c032440 con 0x7f38380b5620 2024-04-25T08:33:42.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.227+0000 7f38367fc640 1 -- 172.21.15.60:0/901363501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c032810 con 0x7f38380b5620 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 -- 172.21.15.60:0/901363501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 msgr2=0x7f38380b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 --2- 172.21.15.60:0/901363501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b5ab0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3838161890 tx=0x7f382c02a270 comp rx=0 tx=0).stop 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 -- 172.21.15.60:0/901363501 shutdown_connections 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 --2- 172.21.15.60:0/901363501 >> v2:172.21.15.100:3300/0 conn(0x7f383806c2d0 0x7f383806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 --2- 172.21.15.60:0/901363501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b5ab0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 -- 172.21.15.60:0/901363501 >> 172.21.15.60:0/901363501 conn(0x7f38380d0800 msgr2=0x7f38380d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 -- 172.21.15.60:0/901363501 shutdown_connections 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.228+0000 7f383e1f5640 1 -- 172.21.15.60:0/901363501 wait complete. 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.229+0000 7f383e1f5640 1 Processor -- start 2024-04-25T08:33:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.229+0000 7f383e1f5640 1 -- start start 2024-04-25T08:33:42.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f383e1f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383806c2d0 0x7f3838153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f383e1f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f383e1f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383815b7d0 0x7f3838150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f383e1f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383815bbb0 con 0x7f38380b5620 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f383e1f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383806d230 con 0x7f383806c2d0 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f383e1f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383815bd30 con 0x7f383815b7d0 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43590/0 (socket says 172.21.15.60:43590) 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 -- 172.21.15.60:0/2215977504 learned_addr learned my addr 172.21.15.60:0/2215977504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f38377fe640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383806c2d0 0x7f3838153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3837fff640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383815b7d0 0x7f3838150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 -- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383815b7d0 msgr2=0x7f3838150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383815b7d0 0x7f3838150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 -- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383806c2d0 msgr2=0x7f3838153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383806c2d0 0x7f3838153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 -- 172.21.15.60:0/2215977504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838154280 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.230+0000 7f3836ffd640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b8a60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f382c02a2b0 tx=0x7f382c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.231+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c041020 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.231+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c0306c0 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.231+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3838154560 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.231+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c037040 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.231+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38380c76d0 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.232+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3838073a00 con 0x7f38380b5620 2024-04-25T08:33:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.232+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f382c044020 con 0x7f38380b5620 2024-04-25T08:33:42.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.233+0000 7f381ffff640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382406b4d0 0x7f382406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.233+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f382c0aaba0 con 0x7f38380b5620 2024-04-25T08:33:42.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.233+0000 7f38377fe640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382406b4d0 0x7f382406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.233+0000 7f38377fe640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382406b4d0 0x7f382406d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f38280016e0 tx=0x7f3828009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:42.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.237+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f382c04b030 con 0x7f38380b5620 2024-04-25T08:33:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.416+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f383815c1b0 con 0x7f38380b5620 2024-04-25T08:33:42.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.416+0000 7f381ffff640 1 -- 172.21.15.60:0/2215977504 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f382c079150 con 0x7f38380b5620 2024-04-25T08:33:42.412 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:33:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382406b4d0 msgr2=0x7f382406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382406b4d0 0x7f382406d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f38280016e0 tx=0x7f3828009040 comp rx=0 tx=0).stop 2024-04-25T08:33:42.414 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 msgr2=0x7f38380b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b8a60 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f382c02a2b0 tx=0x7f382c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 shutdown_connections 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f383815b7d0 0x7f3838150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38380b5620 0x7f38380b8a60 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f382406b4d0 0x7f382406d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 --2- 172.21.15.60:0/2215977504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f383806c2d0 0x7f3838153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 >> 172.21.15.60:0/2215977504 conn(0x7f38380d0800 msgr2=0x7f38380b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 shutdown_connections 2024-04-25T08:33:42.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.418+0000 7f383e1f5640 1 -- 172.21.15.60:0/2215977504 wait complete. 2024-04-25T08:33:42.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:33:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.531+0000 7f79ad9b0640 1 Processor -- start 2024-04-25T08:33:42.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.531+0000 7f79ad9b0640 1 -- start start 2024-04-25T08:33:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79ad9b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a8161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79ad9b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79a80b9750 0x7f79a80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79ad9b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a8162040 con 0x7f79a8161ba0 2024-04-25T08:33:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79ad9b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a8162600 con 0x7f79a80b9750 2024-04-25T08:33:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79ad9b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8162320 con 0x7f79a815da90 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79a80b9750 0x7f79a80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79a80b9750 0x7f79a80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43592/0 (socket says 172.21.15.60:43592) 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 -- 172.21.15.60:0/512546889 learned_addr learned my addr 172.21.15.60:0/512546889 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a6ffd640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a8161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 -- 172.21.15.60:0/512546889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 msgr2=0x7f79a8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a8161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 -- 172.21.15.60:0/512546889 >> v1:172.21.15.100:6789/0 conn(0x7f79a8161ba0 legacy=0x7f79a806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a67fc640 1 -- 172.21.15.60:0/512546889 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80d1b10 con 0x7f79a80b9750 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.532+0000 7f79a6ffd640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.533+0000 7f79a67fc640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80b9750 0x7f79a80ce670 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f799c009870 tx=0x7f799c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab41b9982324329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.533+0000 7f79a5ffb640 1 -- 172.21.15.60:0/512546889 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c037070 con 0x7f79a80b9750 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.533+0000 7f79a5ffb640 1 -- 172.21.15.60:0/512546889 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c032440 con 0x7f79a80b9750 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.533+0000 7f79a5ffb640 1 -- 172.21.15.60:0/512546889 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c032810 con 0x7f79a80b9750 2024-04-25T08:33:42.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 -- 172.21.15.60:0/512546889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80b9750 msgr2=0x7f79a80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80b9750 0x7f79a80ce670 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f799c009870 tx=0x7f799c02a180 comp rx=0 tx=0).stop 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 -- 172.21.15.60:0/512546889 shutdown_connections 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80b9750 0x7f79a80ce670 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/512546889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 -- 172.21.15.60:0/512546889 >> 172.21.15.60:0/512546889 conn(0x7f79a8061a20 msgr2=0x7f79a8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 -- 172.21.15.60:0/512546889 shutdown_connections 2024-04-25T08:33:42.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.534+0000 7f79ad9b0640 1 -- 172.21.15.60:0/512546889 wait complete. 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.535+0000 7f79ad9b0640 1 Processor -- start 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.535+0000 7f79ad9b0640 1 -- start start 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79ad9b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79ad9b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a8161ba0 0x7f79a814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79ad9b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 0x7f79a8158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79ad9b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80d3320 con 0x7f79a80c9120 2024-04-25T08:33:42.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79ad9b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a815de70 con 0x7f79a8161ba0 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79ad9b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80d34a0 con 0x7f79a815da90 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 0x7f79a8158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 0x7f79a8158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43604/0 (socket says 172.21.15.60:43604) 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 -- 172.21.15.60:0/816124848 learned_addr learned my addr 172.21.15.60:0/816124848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a67fc640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a8161ba0 0x7f79a814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a6ffd640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 -- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 msgr2=0x7f79a80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a80b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 -- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a8161ba0 msgr2=0x7f79a814f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a8161ba0 0x7f79a814f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 -- 172.21.15.60:0/816124848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a814f810 con 0x7f79a80c9120 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.536+0000 7f79a77fe640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 0x7f79a8158770 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c700 tx=0x7f79a000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.537+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0016020 con 0x7f79a80c9120 2024-04-25T08:33:42.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.537+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a000aa50 con 0x7f79a80c9120 2024-04-25T08:33:42.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.537+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a000ae40 con 0x7f79a80c9120 2024-04-25T08:33:42.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.537+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a80c95e0 con 0x7f79a80c9120 2024-04-25T08:33:42.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.537+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a815e240 con 0x7f79a80c9120 2024-04-25T08:33:42.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.538+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8073a00 con 0x7f79a80c9120 2024-04-25T08:33:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.538+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79a0019020 con 0x7f79a80c9120 2024-04-25T08:33:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.539+0000 7f798ffff640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f799406b4d0 0x7f799406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.539+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f79a008a520 con 0x7f79a80c9120 2024-04-25T08:33:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.539+0000 7f79a6ffd640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f799406b4d0 0x7f799406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.539+0000 7f79a6ffd640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f799406b4d0 0x7f799406d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f79a80b4480 tx=0x7f7998008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:42.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.543+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79a0020020 con 0x7f79a80c9120 2024-04-25T08:33:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.734+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79a80d1590 con 0x7f79a80c9120 2024-04-25T08:33:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.735+0000 7f798ffff640 1 -- 172.21.15.60:0/816124848 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+122 (secure 0 0 0) 0x7f79a0058b70 con 0x7f79a80c9120 2024-04-25T08:33:42.731 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6834/1590331682 2024-04-26T08:33:24.140229+0000 2024-04-25T08:33:42.731 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.60:6832/1590331682 2024-04-26T08:33:24.140229+0000 2024-04-25T08:33:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.736+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f799406b4d0 msgr2=0x7f799406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.736+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f799406b4d0 0x7f799406d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f79a80b4480 tx=0x7f7998008040 comp rx=0 tx=0).stop 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 msgr2=0x7f79a8158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 0x7f79a8158770 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c700 tx=0x7f79a000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 shutdown_connections 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79a80c9120 0x7f79a8158770 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f799406b4d0 0x7f799406d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79a8161ba0 0x7f79a814f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 --2- 172.21.15.60:0/816124848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79a815da90 0x7f79a80b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:42.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 >> 172.21.15.60:0/816124848 conn(0x7f79a8061a20 msgr2=0x7f79a80d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 shutdown_connections 2024-04-25T08:33:42.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:42.737+0000 7f79ad9b0640 1 -- 172.21.15.60:0/816124848 wait complete. 2024-04-25T08:33:42.743 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:33:42.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:42.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:33:42.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:33:42.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:42.776 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:33:42.831 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:33:42.831 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:42.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:33:42.851 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:42.851 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:42.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:33:42.888 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:33:42.888 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:42.908 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:33:42.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.033+0000 7fd91ee38640 1 Processor -- start 2024-04-25T08:33:43.029 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.033+0000 7fd91ee38640 1 -- start start 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd91ee38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 0x7fd9180d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd91ee38640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91815dfe0 con 0x7fd91815dc00 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd91ee38640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9180d00c0 con 0x7fd9180d1c00 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd91ee38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180d2060 con 0x7fd9180c6df0 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd91cbd4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd91815dc00 0x7fd9180ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43264/0 (socket says 172.21.15.60:43264) 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd91cbd4640 1 -- 172.21.15.60:0/3301768797 learned_addr learned my addr 172.21.15.60:0/3301768797 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd917fff640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 0x7fd9180d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200484350 0 0) 0x7fd90c001110 con 0x7fd91815dc00 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd908003520 con 0x7fd91815dc00 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4131845802 0 0) 0x7fd908003520 con 0x7fd91815dc00 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 msgr2=0x7fd9180d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 0x7fd9180d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 >> v1:172.21.15.100:6789/0 conn(0x7fd9180d1c00 legacy=0x7fd9180ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180d03a0 con 0x7fd91815dc00 2024-04-25T08:33:43.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fd90c001840 con 0x7fd91815dc00 2024-04-25T08:33:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.034+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 >> v1:172.21.15.60:6789/0 conn(0x7fd91815dc00 legacy=0x7fd9180ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd9177fe640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 0x7fd9080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd9177fe640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908003f60 0x7fd908004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd9177fe640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd908004d40 0x7fd9080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180d2060 con 0x7fd908004d40 2024-04-25T08:33:43.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90800baa0 con 0x7fd908003f60 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90800bca0 con 0x7fd9180c6df0 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91d3d5640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd908004d40 0x7fd9080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd917fff640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908003f60 0x7fd908004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91cbd4640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 0x7fd9080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91d3d5640 1 -- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 msgr2=0x7fd9080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91d3d5640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180c6df0 0x7fd9080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91d3d5640 1 -- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908003f60 msgr2=0x7fd908004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91d3d5640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908003f60 0x7fd908004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.035+0000 7fd91d3d5640 1 -- 172.21.15.60:0/3301768797 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908003520 con 0x7fd908004d40 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd91d3d5640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd908004d40 0x7fd9080051c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd910001e50 tx=0x7fd908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a85d1d6053c987 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910001a70 con 0x7fd908004d40 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910003970 con 0x7fd908004d40 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd9177fe640 1 -- 172.21.15.60:0/3301768797 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910003d60 con 0x7fd908004d40 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd91ee38640 1 -- 172.21.15.60:0/3301768797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd908004d40 msgr2=0x7fd9080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd91ee38640 1 --2- 172.21.15.60:0/3301768797 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd908004d40 0x7fd9080051c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd910001e50 tx=0x7fd908005830 comp rx=0 tx=0).stop 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd91d3d5640 1 -- 172.21.15.60:0/3301768797 reap_dead start 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd91ee38640 1 -- 172.21.15.60:0/3301768797 shutdown_connections 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.036+0000 7fd91ee38640 1 -- 172.21.15.60:0/3301768797 >> 172.21.15.60:0/3301768797 conn(0x7fd91806f5d0 msgr2=0x7fd91806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.037+0000 7fd91ee38640 1 -- 172.21.15.60:0/3301768797 shutdown_connections 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.037+0000 7fd91ee38640 1 -- 172.21.15.60:0/3301768797 wait complete. 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.038+0000 7fd91ee38640 1 Processor -- start 2024-04-25T08:33:43.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.038+0000 7fd91ee38640 1 -- start start 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.038+0000 7fd91ee38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908004d40 0x7fd9181e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 0x7fd9180cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180b70a0 0x7fd9181567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cc130 con 0x7fd9180b51b0 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cc720 con 0x7fd908004d40 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cfd90 con 0x7fd9180b70a0 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 0x7fd9180cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 0x7fd9180cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43632/0 (socket says 172.21.15.60:43632) 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 -- 172.21.15.60:0/2441818464 learned_addr learned my addr 172.21.15.60:0/2441818464 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91d3d5640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180b70a0 0x7fd9181567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91cbd4640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908004d40 0x7fd9181e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 -- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180b70a0 msgr2=0x7fd9181567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180b70a0 0x7fd9181567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 -- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908004d40 msgr2=0x7fd9181e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908004d40 0x7fd9181e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 -- 172.21.15.60:0/2441818464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9181e97e0 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91cbd4640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908004d40 0x7fd9181e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91d3d5640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180b70a0 0x7fd9181567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd917fff640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 0x7fd9180cac80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd9040119a0 tx=0x7fd904011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90400faf0 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90400fc90 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9181e8200 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904014440 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.039+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9181e87c0 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.041+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91815d460 con 0x7fd9180b51b0 2024-04-25T08:33:43.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.041+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd904012040 con 0x7fd9180b51b0 2024-04-25T08:33:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.041+0000 7fd9157fa640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8fc06b500 0x7fd8fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.041+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd90408aa90 con 0x7fd9180b51b0 2024-04-25T08:33:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.042+0000 7fd91cbd4640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8fc06b500 0x7fd8fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.042+0000 7fd91cbd4640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8fc06b500 0x7fd8fc06d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd90c001eb0 tx=0x7fd90c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.046+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd91815d460 con 0x7fd9180b51b0 2024-04-25T08:33:43.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.266+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9180c66c0 con 0x7fd9180b51b0 2024-04-25T08:33:43.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.266+0000 7fd9157fa640 1 -- 172.21.15.60:0/2441818464 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd904019030 con 0x7fd9180b51b0 2024-04-25T08:33:43.262 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:43.262 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:27.161423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:43.263 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-04-25T08:33:43.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.268+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8fc06b500 msgr2=0x7fd8fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.268+0000 7fd91ee38640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8fc06b500 0x7fd8fc06d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd90c001eb0 tx=0x7fd90c009040 comp rx=0 tx=0).stop 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.268+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 msgr2=0x7fd9180cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.268+0000 7fd91ee38640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 0x7fd9180cac80 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd9040119a0 tx=0x7fd904011e60 comp rx=0 tx=0).stop 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 shutdown_connections 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9180b70a0 0x7fd9181567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9180b51b0 0x7fd9180cac80 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8fc06b500 0x7fd8fc06d9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 --2- 172.21.15.60:0/2441818464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd908004d40 0x7fd9181e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 >> 172.21.15.60:0/2441818464 conn(0x7fd91806f5d0 msgr2=0x7fd9180ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 shutdown_connections 2024-04-25T08:33:43.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.269+0000 7fd91ee38640 1 -- 172.21.15.60:0/2441818464 wait complete. 2024-04-25T08:33:43.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.382+0000 7f63f4b89640 1 Processor -- start 2024-04-25T08:33:43.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.382+0000 7f63f4b89640 1 -- start start 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.382+0000 7f63f4b89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 0x7f63f00d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.382+0000 7f63f4b89640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f006c220 con 0x7f63f006ce10 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.382+0000 7f63f4b89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f00d0080 con 0x7f63f009a450 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.382+0000 7f63f4b89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f009a8b0 con 0x7f63f015dc70 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63eed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63f006ce10 0x7f63f00cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43276/0 (socket says 172.21.15.60:43276) 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63eed9d640 1 -- 172.21.15.60:0/1131334968 learned_addr learned my addr 172.21.15.60:0/1131334968 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63ee59c640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 0x7f63f00d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609789318 0 0) 0x7f63f006c220 con 0x7f63f006ce10 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63dc003520 con 0x7f63f006ce10 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922499176 0 0) 0x7f63dc003520 con 0x7f63f006ce10 2024-04-25T08:33:43.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 msgr2=0x7f63f00d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 0x7f63f00d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 >> v1:172.21.15.100:6789/0 conn(0x7f63f009a450 legacy=0x7f63f006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00d0360 con 0x7f63f006ce10 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f63e80019f0 con 0x7f63f006ce10 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 >> v1:172.21.15.60:6789/0 conn(0x7f63f006ce10 legacy=0x7f63f00cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 0x7f63dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc003f20 0x7f63dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f009a8b0 con 0x7f63dc004d40 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00ba20 con 0x7f63dc003f20 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc00bc20 con 0x7f63f015dc70 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63eed9d640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63ee59c640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 0x7f63dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.383+0000 7f63e7fff640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc003f20 0x7f63dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.384+0000 7f63eed9d640 1 -- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 msgr2=0x7f63dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.384+0000 7f63eed9d640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f015dc70 0x7f63dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.384+0000 7f63eed9d640 1 -- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc003f20 msgr2=0x7f63dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.384+0000 7f63eed9d640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc003f20 0x7f63dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.384+0000 7f63eed9d640 1 -- 172.21.15.60:0/1131334968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc003520 con 0x7f63dc004d40 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.384+0000 7f63eed9d640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f63e800a800 tx=0x7f63dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309091ba045d02e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800a450 con 0x7f63dc004d40 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e802f070 con 0x7f63dc004d40 2024-04-25T08:33:43.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63edd9b640 1 -- 172.21.15.60:0/1131334968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e802bc90 con 0x7f63dc004d40 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63f4b89640 1 -- 172.21.15.60:0/1131334968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc004d40 msgr2=0x7f63dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1131334968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63dc004d40 0x7f63dc0051c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f63e800a800 tx=0x7f63dc005830 comp rx=0 tx=0).stop 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63eed9d640 1 -- 172.21.15.60:0/1131334968 reap_dead start 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63f4b89640 1 -- 172.21.15.60:0/1131334968 shutdown_connections 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63f4b89640 1 -- 172.21.15.60:0/1131334968 >> 172.21.15.60:0/1131334968 conn(0x7f63f00b3270 msgr2=0x7f63f00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63f4b89640 1 -- 172.21.15.60:0/1131334968 shutdown_connections 2024-04-25T08:33:43.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.385+0000 7f63f4b89640 1 -- 172.21.15.60:0/1131334968 wait complete. 2024-04-25T08:33:43.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.386+0000 7f63f4b89640 1 Processor -- start 2024-04-25T08:33:43.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 -- start start 2024-04-25T08:33:43.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d40 0x7f63f0160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 0x7f63f015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00cbdb0 0x7f63f00cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00d0920 con 0x7f63f0159f10 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00d0c90 con 0x7f63dc004d40 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63f4b89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f015e050 con 0x7f63f00cbdb0 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 0x7f63f015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 0x7f63f015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43646/0 (socket says 172.21.15.60:43646) 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.387+0000 7f63e7fff640 1 -- 172.21.15.60:0/1062869038 learned_addr learned my addr 172.21.15.60:0/1062869038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63ee59c640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d40 0x7f63f0160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e7fff640 1 -- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00cbdb0 msgr2=0x7f63f00cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63eed9d640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00cbdb0 0x7f63f00cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e7fff640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00cbdb0 0x7f63f00cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e7fff640 1 -- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d40 msgr2=0x7f63f0160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e7fff640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d40 0x7f63f0160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e7fff640 1 -- 172.21.15.60:0/1062869038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00ccac0 con 0x7f63f0159f10 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e7fff640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 0x7f63f015a390 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f63d800f7c0 tx=0x7f63d800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d80052e0 con 0x7f63f0159f10 2024-04-25T08:33:43.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8018070 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f00ccda0 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800d970 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.388+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f00cd360 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.389+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f0073990 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.389+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63d800db10 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.390+0000 7f63e6ffd640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63d006b4d0 0x7f63d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.390+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f63d808a3a0 con 0x7f63f0159f10 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.390+0000 7f63ee59c640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63d006b4d0 0x7f63d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.391+0000 7f63ee59c640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63d006b4d0 0x7f63d006d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f63f0161090 tx=0x7f63e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.394+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63d801a310 con 0x7f63f0159f10 2024-04-25T08:33:43.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.574+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f00b40a0 con 0x7f63f0159f10 2024-04-25T08:33:43.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.576+0000 7f63e6ffd640 1 -- 172.21.15.60:0/1062869038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17001 (secure 0 0 0) 0x7f63d8059030 con 0x7f63f0159f10 2024-04-25T08:33:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:43.572 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:24.149268+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/1590331682":"2024-04-26T08:33:24.140229+0000","172.21.15.60:6832/1590331682":"2024-04-26T08:33:24.140229+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.577+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63d006b4d0 msgr2=0x7f63d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.577+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63d006b4d0 0x7f63d006d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f63f0161090 tx=0x7f63e0008040 comp rx=0 tx=0).stop 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 msgr2=0x7f63f015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 0x7f63f015a390 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f63d800f7c0 tx=0x7f63d800fc80 comp rx=0 tx=0).stop 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 shutdown_connections 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63f00cbdb0 0x7f63f00cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63f0159f10 0x7f63f015a390 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63d006b4d0 0x7f63d006d990 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 --2- 172.21.15.60:0/1062869038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63dc004d40 0x7f63f0160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 >> 172.21.15.60:0/1062869038 conn(0x7f63f00b3270 msgr2=0x7f63f00b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 shutdown_connections 2024-04-25T08:33:43.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.578+0000 7f63f4b89640 1 -- 172.21.15.60:0/1062869038 wait complete. 2024-04-25T08:33:43.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:33:06.174848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:33:06.884806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:33:43.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:33:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.694+0000 7f6b63acb640 1 Processor -- start 2024-04-25T08:33:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.694+0000 7f6b63acb640 1 -- start start 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b63acb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6b5c15b7d0 0x7f6b5c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b63acb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6b5c06c9e0 0x7f6b5c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b63acb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c06cf50 con 0x7f6b5c0b5620 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b63acb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5c0ce530 con 0x7f6b5c06c9e0 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b63acb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5c0ce810 con 0x7f6b5c15b7d0 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6b5c06c9e0 0x7f6b5c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6b5c06c9e0 0x7f6b5c0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43650/0 (socket says 172.21.15.60:43650) 2024-04-25T08:33:43.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 -- 172.21.15.60:0/1719425314 learned_addr learned my addr 172.21.15.60:0/1719425314 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b61867640 1 --2- 172.21.15.60:0/1719425314 >> v2:172.21.15.100:3300/0 conn(0x7f6b5c15b7d0 0x7f6b5c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 -- 172.21.15.60:0/1719425314 >> v2:172.21.15.100:3300/0 conn(0x7f6b5c15b7d0 msgr2=0x7f6b5c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 --2- 172.21.15.60:0/1719425314 >> v2:172.21.15.100:3300/0 conn(0x7f6b5c15b7d0 0x7f6b5c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 -- 172.21.15.60:0/1719425314 >> v1:172.21.15.100:6789/0 conn(0x7f6b5c0b5620 legacy=0x7f6b5c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.695+0000 7f6b62068640 1 -- 172.21.15.60:0/1719425314 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0ceaf0 con 0x7f6b5c06c9e0 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.696+0000 7f6b62068640 1 --2- 172.21.15.60:0/1719425314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c06c9e0 0x7f6b5c0b5ab0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c08c0d0 tx=0x7f6b5402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f7632be742e859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.696+0000 7f6b60865640 1 -- 172.21.15.60:0/1719425314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54036070 con 0x7f6b5c06c9e0 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.696+0000 7f6b60865640 1 -- 172.21.15.60:0/1719425314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54031600 con 0x7f6b5c06c9e0 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.696+0000 7f6b60865640 1 -- 172.21.15.60:0/1719425314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540319d0 con 0x7f6b5c06c9e0 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 -- 172.21.15.60:0/1719425314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c06c9e0 msgr2=0x7f6b5c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1719425314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c06c9e0 0x7f6b5c0b5ab0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c08c0d0 tx=0x7f6b5402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 -- 172.21.15.60:0/1719425314 shutdown_connections 2024-04-25T08:33:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1719425314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c06c9e0 0x7f6b5c0b5ab0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1719425314 >> v2:172.21.15.100:3300/0 conn(0x7f6b5c15b7d0 0x7f6b5c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 -- 172.21.15.60:0/1719425314 >> 172.21.15.60:0/1719425314 conn(0x7f6b5c0d0800 msgr2=0x7f6b5c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 -- 172.21.15.60:0/1719425314 shutdown_connections 2024-04-25T08:33:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.697+0000 7f6b63acb640 1 -- 172.21.15.60:0/1719425314 wait complete. 2024-04-25T08:33:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.698+0000 7f6b63acb640 1 Processor -- start 2024-04-25T08:33:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.698+0000 7f6b63acb640 1 -- start start 2024-04-25T08:33:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b63acb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b5c06c9e0 0x7f6b5c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b63acb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 0x7f6b5c0b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b63acb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b5c15b7d0 0x7f6b5c0b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b63acb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0cf6d0 con 0x7f6b5c0b5620 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b63acb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0cf550 con 0x7f6b5c15b7d0 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b63acb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c15bbb0 con 0x7f6b5c06c9e0 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 0x7f6b5c0b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 0x7f6b5c0b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43652/0 (socket says 172.21.15.60:43652) 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 -- 172.21.15.60:0/1604086426 learned_addr learned my addr 172.21.15.60:0/1604086426 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61867640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b5c06c9e0 0x7f6b5c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b62068640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b5c15b7d0 0x7f6b5c0b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 -- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b5c06c9e0 msgr2=0x7f6b5c0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b5c06c9e0 0x7f6b5c0c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 -- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b5c15b7d0 msgr2=0x7f6b5c0b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b5c15b7d0 0x7f6b5c0b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 -- 172.21.15.60:0/1604086426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0b76e0 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b62068640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b5c15b7d0 0x7f6b5c0b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.699+0000 7f6b61066640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 0x7f6b5c0b8780 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c430 tx=0x7f6b4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.700+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c011790 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.700+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c017070 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.700+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c0b79c0 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.700+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c011bb0 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.700+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c0b7f80 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.700+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c073a00 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.701+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b4c01c460 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.701+0000 7f6b527fc640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b4006b4d0 0x7f6b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.702+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6b4c08a110 con 0x7f6b5c0b5620 2024-04-25T08:33:43.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.702+0000 7f6b61867640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b4006b4d0 0x7f6b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.702+0000 7f6b61867640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b4006b4d0 0x7f6b4006d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6b580079a0 tx=0x7f6b58008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:43.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.706+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b4c058e40 con 0x7f6b5c0b5620 2024-04-25T08:33:43.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.889+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b5c15b0f0 con 0x7f6b5c0b5620 2024-04-25T08:33:43.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.890+0000 7f6b527fc640 1 -- 172.21.15.60:0/1604086426 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b4c058740 con 0x7f6b5c0b5620 2024-04-25T08:33:43.885 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b4006b4d0 msgr2=0x7f6b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b4006b4d0 0x7f6b4006d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6b580079a0 tx=0x7f6b58008040 comp rx=0 tx=0).stop 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 msgr2=0x7f6b5c0b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 0x7f6b5c0b8780 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00c430 tx=0x7f6b4c00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 shutdown_connections 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6b4006b4d0 0x7f6b4006d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b5c15b7d0 0x7f6b5c0b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b5c0b5620 0x7f6b5c0b8780 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 --2- 172.21.15.60:0/1604086426 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b5c06c9e0 0x7f6b5c0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:43.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 >> 172.21.15.60:0/1604086426 conn(0x7f6b5c0d0800 msgr2=0x7f6b5c15e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:43.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 shutdown_connections 2024-04-25T08:33:43.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:43.892+0000 7f6b63acb640 1 -- 172.21.15.60:0/1604086426 wait complete. 2024-04-25T08:33:43.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:33:44.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.007+0000 7f2505b29640 1 Processor -- start 2024-04-25T08:33:44.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.007+0000 7f2505b29640 1 -- start start 2024-04-25T08:33:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.007+0000 7f2505b29640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f250006c9e0 0x7f25000b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f2505b29640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250006cf50 con 0x7f250015b7d0 2024-04-25T08:33:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f2505b29640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25000ce530 con 0x7f25000b5620 2024-04-25T08:33:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f2505b29640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25000ce810 con 0x7f250006c9e0 2024-04-25T08:33:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24ff7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f250015b7d0 0x7f25000cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43278/0 (socket says 172.21.15.60:43278) 2024-04-25T08:33:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24ff7fe640 1 -- 172.21.15.60:0/1540386501 learned_addr learned my addr 172.21.15.60:0/1540386501 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136680155 0 0) 0x7f250006cf50 con 0x7f250015b7d0 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e8003520 con 0x7f250015b7d0 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fffff640 1 --2- 172.21.15.60:0/1540386501 >> v2:172.21.15.100:3300/0 conn(0x7f250006c9e0 0x7f25000b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309015247 0 0) 0x7f24e8003520 con 0x7f250015b7d0 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 >> v2:172.21.15.100:3300/0 conn(0x7f250006c9e0 msgr2=0x7f25000b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/1540386501 >> v2:172.21.15.100:3300/0 conn(0x7f250006c9e0 0x7f25000b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 >> v1:172.21.15.100:6789/0 conn(0x7f25000b5620 legacy=0x7f250006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000ceaf0 con 0x7f250015b7d0 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f24f4001670 con 0x7f250015b7d0 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 >> v1:172.21.15.60:6789/0 conn(0x7f250015b7d0 legacy=0x7f25000cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e80039b0 0x7f24e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e8004800 0x7f24e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e80051c0 0x7f24e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25000ce810 con 0x7f24e8004800 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e800c060 con 0x7f24e80039b0 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e800c260 con 0x7f24e80051c0 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24feffd640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e8004800 0x7f24e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.008+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e80039b0 0x7f24e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24fffff640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e80051c0 0x7f24e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24feffd640 1 -- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e80051c0 msgr2=0x7f24e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24feffd640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e80051c0 0x7f24e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24feffd640 1 -- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e80039b0 msgr2=0x7f24e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24feffd640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24e80039b0 0x7f24e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24feffd640 1 -- 172.21.15.60:0/1540386501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e8003520 con 0x7f24e8004800 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24fffff640 1 -- 172.21.15.60:0/1540386501 reap_dead start 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.009+0000 7f24feffd640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e8004800 0x7f24e8004c80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f24f0001750 tx=0x7f24e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b56bdc26a6d270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f0001370 con 0x7f24e8004800 2024-04-25T08:33:44.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f00052d0 con 0x7f24e8004800 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f24fe7fc640 1 -- 172.21.15.60:0/1540386501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f002abc0 con 0x7f24e8004800 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 -- 172.21.15.60:0/1540386501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e8004800 msgr2=0x7f24e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e8004800 0x7f24e8004c80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f24f0001750 tx=0x7f24e8005830 comp rx=0 tx=0).stop 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 -- 172.21.15.60:0/1540386501 shutdown_connections 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 --2- 172.21.15.60:0/1540386501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24e8004800 0x7f24e8004c80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 -- 172.21.15.60:0/1540386501 >> 172.21.15.60:0/1540386501 conn(0x7f25000d0800 msgr2=0x7f25000d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 -- 172.21.15.60:0/1540386501 shutdown_connections 2024-04-25T08:33:44.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.010+0000 7f2505b29640 1 -- 172.21.15.60:0/1540386501 wait complete. 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.011+0000 7f2505b29640 1 Processor -- start 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 -- start start 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e8004800 0x7f2500148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25000c7620 0x7f25000cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 0x7f250014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250015bd30 con 0x7f25000b8460 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250015bbb0 con 0x7f25000c7620 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f2505b29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25000b5ab0 con 0x7f24e8004800 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f24fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 0x7f250014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f24fffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 0x7f250014b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43668/0 (socket says 172.21.15.60:43668) 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.012+0000 7f24fffff640 1 -- 172.21.15.60:0/3477276747 learned_addr learned my addr 172.21.15.60:0/3477276747 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24feffd640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25000c7620 0x7f25000cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e8004800 0x7f2500148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24fffff640 1 -- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e8004800 msgr2=0x7f2500148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24fffff640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e8004800 0x7f2500148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24fffff640 1 -- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25000c7620 msgr2=0x7f25000cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24fffff640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25000c7620 0x7f25000cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24fffff640 1 -- 172.21.15.60:0/3477276747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25000cdd00 con 0x7f25000b8460 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24feffd640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25000c7620 0x7f25000cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24fffff640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 0x7f250014b380 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f250006b4d0 tx=0x7f24f8011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800fca0 con 0x7f25000b8460 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f800fe40 con 0x7f25000b8460 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8015670 con 0x7f25000b8460 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25000ca140 con 0x7f25000b8460 2024-04-25T08:33:44.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.013+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25000b7c60 con 0x7f25000b8460 2024-04-25T08:33:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.014+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2500073a00 con 0x7f25000b8460 2024-04-25T08:33:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.015+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24f8015810 con 0x7f25000b8460 2024-04-25T08:33:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.015+0000 7f24e7fff640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ec06b500 0x7f24ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.015+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f24f808b140 con 0x7f25000b8460 2024-04-25T08:33:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.015+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ec06b500 0x7f24ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.016+0000 7f24ff7fe640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ec06b500 0x7f24ec06d9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f24f4007c10 tx=0x7f24f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.019+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24f8019030 con 0x7f25000b8460 2024-04-25T08:33:44.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.199+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25000b4260 con 0x7f25000b8460 2024-04-25T08:33:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.200+0000 7f24e7fff640 1 -- 172.21.15.60:0/3477276747 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24f8010040 con 0x7f25000b8460 2024-04-25T08:33:44.195 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ec06b500 msgr2=0x7f24ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ec06b500 0x7f24ec06d9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f24f4007c10 tx=0x7f24f4009040 comp rx=0 tx=0).stop 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 msgr2=0x7f250014b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 0x7f250014b380 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f250006b4d0 tx=0x7f24f8011f20 comp rx=0 tx=0).stop 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 shutdown_connections 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25000b8460 0x7f250014b380 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f24ec06b500 0x7f24ec06d9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25000c7620 0x7f25000cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 --2- 172.21.15.60:0/3477276747 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f24e8004800 0x7f2500148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 >> 172.21.15.60:0/3477276747 conn(0x7f25000d0800 msgr2=0x7f25000b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 shutdown_connections 2024-04-25T08:33:44.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.202+0000 7f2505b29640 1 -- 172.21.15.60:0/3477276747 wait complete. 2024-04-25T08:33:44.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:33:44.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.317+0000 7f3b81f0a640 1 Processor -- start 2024-04-25T08:33:44.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.317+0000 7f3b81f0a640 1 -- start start 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b81f0a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3b7c1610f0 0x7f3b7c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b81f0a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b7c1616f0 0x7f3b7c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b81f0a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0c6960 con 0x7f3b7c0ce0a0 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b81f0a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c1614d0 con 0x7f3b7c1616f0 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b81f0a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c162260 con 0x7f3b7c1610f0 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b7affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b7c1616f0 0x7f3b7c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b7affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3b7c1616f0 0x7f3b7c161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43680/0 (socket says 172.21.15.60:43680) 2024-04-25T08:33:44.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.318+0000 7f3b7affd640 1 -- 172.21.15.60:0/3686505554 learned_addr learned my addr 172.21.15.60:0/3686505554 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.319+0000 7f3b7affd640 1 -- 172.21.15.60:0/3686505554 >> v2:172.21.15.100:3300/0 conn(0x7f3b7c1610f0 msgr2=0x7f3b7c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.319+0000 7f3b7affd640 1 --2- 172.21.15.60:0/3686505554 >> v2:172.21.15.100:3300/0 conn(0x7f3b7c1610f0 0x7f3b7c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.319+0000 7f3b7affd640 1 -- 172.21.15.60:0/3686505554 >> v1:172.21.15.60:6789/0 conn(0x7f3b7c0ce0a0 legacy=0x7f3b7c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.319+0000 7f3b7affd640 1 -- 172.21.15.60:0/3686505554 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c162540 con 0x7f3b7c1616f0 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.319+0000 7f3b7affd640 1 --2- 172.21.15.60:0/3686505554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1616f0 0x7f3b7c161b50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c071fb0 tx=0x7f3b7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a39196cec84cf1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.319+0000 7f3b7a7fc640 1 -- 172.21.15.60:0/3686505554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70037070 con 0x7f3b7c1616f0 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b7a7fc640 1 -- 172.21.15.60:0/3686505554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b70032440 con 0x7f3b7c1616f0 2024-04-25T08:33:44.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b7a7fc640 1 -- 172.21.15.60:0/3686505554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70032810 con 0x7f3b7c1616f0 2024-04-25T08:33:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3686505554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1616f0 msgr2=0x7f3b7c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3686505554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1616f0 0x7f3b7c161b50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c071fb0 tx=0x7f3b7002a270 comp rx=0 tx=0).stop 2024-04-25T08:33:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3686505554 shutdown_connections 2024-04-25T08:33:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3686505554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1616f0 0x7f3b7c161b50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3686505554 >> v2:172.21.15.100:3300/0 conn(0x7f3b7c1610f0 0x7f3b7c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3686505554 >> 172.21.15.60:0/3686505554 conn(0x7f3b7c06f500 msgr2=0x7f3b7c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3686505554 shutdown_connections 2024-04-25T08:33:44.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.320+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3686505554 wait complete. 2024-04-25T08:33:44.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.321+0000 7f3b81f0a640 1 Processor -- start 2024-04-25T08:33:44.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 -- start start 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c0ce0a0 0x7f3b7c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 0x7f3b7c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c1616f0 0x7f3b7c1e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0cf690 con 0x7f3b7c1610f0 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0cfce0 con 0x7f3b7c0ce0a0 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b81f0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c06c700 con 0x7f3b7c1616f0 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b7affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 0x7f3b7c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b7affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 0x7f3b7c158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43686/0 (socket says 172.21.15.60:43686) 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.322+0000 7f3b7affd640 1 -- 172.21.15.60:0/3467568072 learned_addr learned my addr 172.21.15.60:0/3467568072 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7affd640 1 -- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c1616f0 msgr2=0x7f3b7c1e7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7affd640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c1616f0 0x7f3b7c1e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7affd640 1 -- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c0ce0a0 msgr2=0x7f3b7c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c0ce0a0 0x7f3b7c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7affd640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c0ce0a0 0x7f3b7c1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7affd640 1 -- 172.21.15.60:0/3467568072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c1e8080 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b7affd640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 0x7f3b7c158580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3b70000c00 tx=0x7f3b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70037040 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b70042070 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c1e8360 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70030bb0 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.323+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c153580 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.324+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c1e8530 con 0x7f3b7c1610f0 2024-04-25T08:33:44.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.324+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b70047460 con 0x7f3b7c1610f0 2024-04-25T08:33:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.325+0000 7f3b63fff640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b6806b4d0 0x7f3b6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.325+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3b700aafd0 con 0x7f3b7c1610f0 2024-04-25T08:33:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.325+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b6806b4d0 0x7f3b6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.326+0000 7f3b7b7fe640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b6806b4d0 0x7f3b6806d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c1e8bf0 tx=0x7f3b6c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.329+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b70045360 con 0x7f3b7c1610f0 2024-04-25T08:33:44.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.512+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b7c0cf150 con 0x7f3b7c1610f0 2024-04-25T08:33:44.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.513+0000 7f3b63fff640 1 -- 172.21.15.60:0/3467568072 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b70079d00 con 0x7f3b7c1610f0 2024-04-25T08:33:44.509 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b6806b4d0 msgr2=0x7f3b6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b6806b4d0 0x7f3b6806d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c1e8bf0 tx=0x7f3b6c008040 comp rx=0 tx=0).stop 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 msgr2=0x7f3b7c158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 0x7f3b7c158580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3b70000c00 tx=0x7f3b700046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 shutdown_connections 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b7c1616f0 0x7f3b7c1e7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b7c1610f0 0x7f3b7c158580 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b6806b4d0 0x7f3b6806d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 --2- 172.21.15.60:0/3467568072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c0ce0a0 0x7f3b7c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 >> 172.21.15.60:0/3467568072 conn(0x7f3b7c06f500 msgr2=0x7f3b7c156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 shutdown_connections 2024-04-25T08:33:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.515+0000 7f3b81f0a640 1 -- 172.21.15.60:0/3467568072 wait complete. 2024-04-25T08:33:44.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.626+0000 7f53ef142640 1 Processor -- start 2024-04-25T08:33:44.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.626+0000 7f53ef142640 1 -- start start 2024-04-25T08:33:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53ef142640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f53e815ba00 0x7f53e809a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53ef142640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53e81602e0 0x7f53e80ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53ef142640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e80cd710 con 0x7f53e815f5f0 2024-04-25T08:33:44.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53ef142640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e8099b70 con 0x7f53e81602e0 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53ef142640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e8099e50 con 0x7f53e815ba00 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53e81602e0 0x7f53e80ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53e7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53e81602e0 0x7f53e80ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43702/0 (socket says 172.21.15.60:43702) 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.627+0000 7f53e7fff640 1 -- 172.21.15.60:0/508819655 learned_addr learned my addr 172.21.15.60:0/508819655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.628+0000 7f53ecede640 1 --2- 172.21.15.60:0/508819655 >> v2:172.21.15.100:3300/0 conn(0x7f53e815ba00 0x7f53e809a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.628+0000 7f53e7fff640 1 -- 172.21.15.60:0/508819655 >> v2:172.21.15.100:3300/0 conn(0x7f53e815ba00 msgr2=0x7f53e809a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.628+0000 7f53e7fff640 1 --2- 172.21.15.60:0/508819655 >> v2:172.21.15.100:3300/0 conn(0x7f53e815ba00 0x7f53e809a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.628+0000 7f53e7fff640 1 -- 172.21.15.60:0/508819655 >> v1:172.21.15.100:6789/0 conn(0x7f53e815f5f0 legacy=0x7f53e80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.628+0000 7f53e7fff640 1 -- 172.21.15.60:0/508819655 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e809a130 con 0x7f53e81602e0 2024-04-25T08:33:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.628+0000 7f53e7fff640 1 --2- 172.21.15.60:0/508819655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81602e0 0x7f53e80ccda0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f53e80b4370 tx=0x7f53d802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfec6beb75bfe3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53e77fe640 1 -- 172.21.15.60:0/508819655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8037070 con 0x7f53e81602e0 2024-04-25T08:33:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53e77fe640 1 -- 172.21.15.60:0/508819655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8032440 con 0x7f53e81602e0 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53e77fe640 1 -- 172.21.15.60:0/508819655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8032810 con 0x7f53e81602e0 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 -- 172.21.15.60:0/508819655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81602e0 msgr2=0x7f53e80ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 --2- 172.21.15.60:0/508819655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81602e0 0x7f53e80ccda0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f53e80b4370 tx=0x7f53d802a150 comp rx=0 tx=0).stop 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 -- 172.21.15.60:0/508819655 shutdown_connections 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 --2- 172.21.15.60:0/508819655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e81602e0 0x7f53e80ccda0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 --2- 172.21.15.60:0/508819655 >> v2:172.21.15.100:3300/0 conn(0x7f53e815ba00 0x7f53e809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 -- 172.21.15.60:0/508819655 >> 172.21.15.60:0/508819655 conn(0x7f53e80b3270 msgr2=0x7f53e80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 -- 172.21.15.60:0/508819655 shutdown_connections 2024-04-25T08:33:44.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.629+0000 7f53ef142640 1 -- 172.21.15.60:0/508819655 wait complete. 2024-04-25T08:33:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.630+0000 7f53ef142640 1 Processor -- start 2024-04-25T08:33:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 -- start start 2024-04-25T08:33:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 0x7f53e8158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e815f5f0 0x7f53e80caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81602e0 0x7f53e80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e815bde0 con 0x7f53e815ba00 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80cd2e0 con 0x7f53e81602e0 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ef142640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b71a0 con 0x7f53e815f5f0 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ecede640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 0x7f53e8158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ecede640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 0x7f53e8158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43714/0 (socket says 172.21.15.60:43714) 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.631+0000 7f53ecede640 1 -- 172.21.15.60:0/20162628 learned_addr learned my addr 172.21.15.60:0/20162628 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53e7fff640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e815f5f0 0x7f53e80caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ed6df640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81602e0 0x7f53e80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ecede640 1 -- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e815f5f0 msgr2=0x7f53e80caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ecede640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e815f5f0 0x7f53e80caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ecede640 1 -- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81602e0 msgr2=0x7f53e80caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ecede640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81602e0 0x7f53e80caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ecede640 1 -- 172.21.15.60:0/20162628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e81e68d0 con 0x7f53e815ba00 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ed6df640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81602e0 0x7f53e80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ecede640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 0x7f53e8158e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007e40 tx=0x7f53e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0014620 con 0x7f53e815ba00 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0019070 con 0x7f53e815ba00 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e815d4b0 con 0x7f53e815ba00 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0014a00 con 0x7f53e815ba00 2024-04-25T08:33:44.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.632+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e8159340 con 0x7f53e815ba00 2024-04-25T08:33:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.633+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e815dbc0 con 0x7f53e815ba00 2024-04-25T08:33:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.634+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53e0014ba0 con 0x7f53e815ba00 2024-04-25T08:33:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.634+0000 7f53e57fa640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f53c806b4d0 0x7f53c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.634+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f53e008aca0 con 0x7f53e815ba00 2024-04-25T08:33:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.634+0000 7f53e7fff640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f53c806b4d0 0x7f53c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.635+0000 7f53e7fff640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f53c806b4d0 0x7f53c806d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f53d8034000 tx=0x7f53d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.638+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53e0059940 con 0x7f53e815ba00 2024-04-25T08:33:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.854+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53e80b40a0 con 0x7f53e815ba00 2024-04-25T08:33:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.855+0000 7f53e57fa640 1 -- 172.21.15.60:0/20162628 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f53e0059240 con 0x7f53e815ba00 2024-04-25T08:33:44.851 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:44.851 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:27.161423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:44.851 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f53c806b4d0 msgr2=0x7f53c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f53c806b4d0 0x7f53c806d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f53d8034000 tx=0x7f53d802e000 comp rx=0 tx=0).stop 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 msgr2=0x7f53e8158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 0x7f53e8158e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f53e0007e40 tx=0x7f53e000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 shutdown_connections 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f53c806b4d0 0x7f53c806d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81602e0 0x7f53e80caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e815f5f0 0x7f53e80caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 --2- 172.21.15.60:0/20162628 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53e815ba00 0x7f53e8158e00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 >> 172.21.15.60:0/20162628 conn(0x7f53e80b3270 msgr2=0x7f53e80cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 shutdown_connections 2024-04-25T08:33:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.857+0000 7f53ef142640 1 -- 172.21.15.60:0/20162628 wait complete. 2024-04-25T08:33:44.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.969+0000 7f93a5742640 1 Processor -- start 2024-04-25T08:33:44.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 -- start start 2024-04-25T08:33:44.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93a0154ee0 0x7f93a00cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93a00cfcf0 0x7f93a00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a00ce830 con 0x7f93a00cfcf0 2024-04-25T08:33:44.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a0072090 con 0x7f93a0154ee0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.970+0000 7f93a5742640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a0071230 con 0x7f93a00ce3c0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93a00cfcf0 0x7f93a00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f93a00cfcf0 0x7f93a00cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43718/0 (socket says 172.21.15.60:43718) 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 -- 172.21.15.60:0/765183225 learned_addr learned my addr 172.21.15.60:0/765183225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939f7fe640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939effd640 1 --2- 172.21.15.60:0/765183225 >> v2:172.21.15.100:3300/0 conn(0x7f93a0154ee0 0x7f93a00cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 -- 172.21.15.60:0/765183225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 msgr2=0x7f93a015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 -- 172.21.15.60:0/765183225 >> v2:172.21.15.100:3300/0 conn(0x7f93a0154ee0 msgr2=0x7f93a00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 --2- 172.21.15.60:0/765183225 >> v2:172.21.15.100:3300/0 conn(0x7f93a0154ee0 0x7f93a00cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939e7fc640 1 -- 172.21.15.60:0/765183225 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0071480 con 0x7f93a00cfcf0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.971+0000 7f939f7fe640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f939e7fc640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a00cfcf0 0x7f93a00cde80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f93a00d02e0 tx=0x7f939402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea190f1ffdff8183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f939dffb640 1 -- 172.21.15.60:0/765183225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394032790 con 0x7f93a00cfcf0 2024-04-25T08:33:44.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f939dffb640 1 -- 172.21.15.60:0/765183225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394038070 con 0x7f93a00cfcf0 2024-04-25T08:33:44.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f939dffb640 1 -- 172.21.15.60:0/765183225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394032b50 con 0x7f93a00cfcf0 2024-04-25T08:33:44.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 -- 172.21.15.60:0/765183225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a00cfcf0 msgr2=0x7f93a00cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a00cfcf0 0x7f93a00cde80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f93a00d02e0 tx=0x7f939402c890 comp rx=0 tx=0).stop 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 -- 172.21.15.60:0/765183225 shutdown_connections 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 --2- 172.21.15.60:0/765183225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a00cfcf0 0x7f93a00cde80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 --2- 172.21.15.60:0/765183225 >> v2:172.21.15.100:3300/0 conn(0x7f93a0154ee0 0x7f93a00cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 -- 172.21.15.60:0/765183225 >> 172.21.15.60:0/765183225 conn(0x7f93a006abe0 msgr2=0x7f93a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 -- 172.21.15.60:0/765183225 shutdown_connections 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.972+0000 7f93a5742640 1 -- 172.21.15.60:0/765183225 wait complete. 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 Processor -- start 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 -- start start 2024-04-25T08:33:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a0144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a00cfcf0 0x7f93a0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 0x7f93a0159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a0071d20 con 0x7f93a0154ee0 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a0071a40 con 0x7f93a00cfcf0 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.974+0000 7f93a5742640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a01552c0 con 0x7f93a00ce3c0 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 0x7f93a0159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 0x7f93a0159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43734/0 (socket says 172.21.15.60:43734) 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 -- 172.21.15.60:0/1569225737 learned_addr learned my addr 172.21.15.60:0/1569225737 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939e7fc640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a00cfcf0 0x7f93a0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939effd640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a0144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 -- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 msgr2=0x7f93a0144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a0144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 -- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a00cfcf0 msgr2=0x7f93a0159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a00cfcf0 0x7f93a0159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 -- 172.21.15.60:0/1569225737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a01458b0 con 0x7f93a0154ee0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939e7fc640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a00cfcf0 0x7f93a0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f939f7fe640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 0x7f93a0159f90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9398007e10 tx=0x7f939800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939801a070 con 0x7f93a0154ee0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398003b90 con 0x7f93a0154ee0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398004030 con 0x7f93a0154ee0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a0145b90 con 0x7f93a0154ee0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.975+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a0146150 con 0x7f93a0154ee0 2024-04-25T08:33:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.976+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0146350 con 0x7f93a0154ee0 2024-04-25T08:33:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.977+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f939801d070 con 0x7f93a0154ee0 2024-04-25T08:33:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.977+0000 7f93877fe640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f938c06b500 0x7f938c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.977+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f939808a620 con 0x7f93a0154ee0 2024-04-25T08:33:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.977+0000 7f939effd640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f938c06b500 0x7f938c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.978+0000 7f939effd640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f938c06b500 0x7f938c06d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9390009a30 tx=0x7f9390008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:44.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:44.981+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93a0146350 con 0x7f93a0154ee0 2024-04-25T08:33:45.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.162+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93a0073990 con 0x7f93a0154ee0 2024-04-25T08:33:45.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.163+0000 7f93877fe640 1 -- 172.21.15.60:0/1569225737 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17001 (secure 0 0 0) 0x7f9398010070 con 0x7f93a0154ee0 2024-04-25T08:33:45.159 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:45.159 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:24.149268+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/1590331682":"2024-04-26T08:33:24.140229+0000","172.21.15.60:6832/1590331682":"2024-04-26T08:33:24.140229+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f938c06b500 msgr2=0x7f938c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f938c06b500 0x7f938c06d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9390009a30 tx=0x7f9390008040 comp rx=0 tx=0).stop 2024-04-25T08:33:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 msgr2=0x7f93a0159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 0x7f93a0159f90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9398007e10 tx=0x7f939800d4a0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 shutdown_connections 2024-04-25T08:33:45.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93a0154ee0 0x7f93a0159f90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f938c06b500 0x7f938c06d9c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93a00cfcf0 0x7f93a0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 --2- 172.21.15.60:0/1569225737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f93a00ce3c0 0x7f93a0144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 >> 172.21.15.60:0/1569225737 conn(0x7f93a006abe0 msgr2=0x7f93a015d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 shutdown_connections 2024-04-25T08:33:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.165+0000 7f93a5742640 1 -- 172.21.15.60:0/1569225737 wait complete. 2024-04-25T08:33:45.172 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:33:45.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:45.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.278+0000 7ffa175e7640 1 Processor -- start 2024-04-25T08:33:45.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.279+0000 7ffa175e7640 1 -- start start 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.279+0000 7ffa175e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffa1015dbe0 0x7ffa100d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.279+0000 7ffa175e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa100b92a0 0x7ffa100d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.279+0000 7ffa175e7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa100b97a0 con 0x7ffa100b2cb0 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.279+0000 7ffa175e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1015dfc0 con 0x7ffa1015dbe0 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.279+0000 7ffa175e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100d1a00 con 0x7ffa100b92a0 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15383640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ffa1015dbe0 0x7ffa100d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15b84640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa100b92a0 0x7ffa100d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa14b82640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ffa100b2cb0 0x7ffa100b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38774/0 (socket says 172.21.15.60:38774) 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa14b82640 1 -- 172.21.15.60:0/3845164830 learned_addr learned my addr 172.21.15.60:0/3845164830 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15383640 1 -- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa100b92a0 msgr2=0x7ffa100d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15383640 1 --2- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa100b92a0 0x7ffa100d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15383640 1 -- 172.21.15.60:0/3845164830 >> v1:172.21.15.100:6789/0 conn(0x7ffa100b2cb0 legacy=0x7ffa100b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa07fff640 1 -- 172.21.15.60:0/3845164830 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203352073 0 0) 0x7ffa100b97a0 con 0x7ffa100b2cb0 2024-04-25T08:33:45.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15383640 1 -- 172.21.15.60:0/3845164830 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa100d1c50 con 0x7ffa1015dbe0 2024-04-25T08:33:45.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.280+0000 7ffa15b84640 1 --2- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa100b92a0 0x7ffa100d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.281+0000 7ffa15383640 1 --2- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa1015dbe0 0x7ffa100d0910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0097a0 tx=0x7ffa0c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf93475fde1c2729 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.281+0000 7ffa07fff640 1 -- 172.21.15.60:0/3845164830 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c037070 con 0x7ffa1015dbe0 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.281+0000 7ffa07fff640 1 -- 172.21.15.60:0/3845164830 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c032440 con 0x7ffa1015dbe0 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.281+0000 7ffa07fff640 1 -- 172.21.15.60:0/3845164830 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c032810 con 0x7ffa1015dbe0 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 -- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa1015dbe0 msgr2=0x7ffa100d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 --2- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa1015dbe0 0x7ffa100d0910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0097a0 tx=0x7ffa0c02a180 comp rx=0 tx=0).stop 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 -- 172.21.15.60:0/3845164830 shutdown_connections 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 --2- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa100b92a0 0x7ffa100d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 --2- 172.21.15.60:0/3845164830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa1015dbe0 0x7ffa100d0910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 -- 172.21.15.60:0/3845164830 >> 172.21.15.60:0/3845164830 conn(0x7ffa10061a20 msgr2=0x7ffa10061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:45.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 -- 172.21.15.60:0/3845164830 shutdown_connections 2024-04-25T08:33:45.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.282+0000 7ffa175e7640 1 -- 172.21.15.60:0/3845164830 wait complete. 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.283+0000 7ffa175e7640 1 Processor -- start 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.283+0000 7ffa175e7640 1 -- start start 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa175e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa100b2cb0 0x7ffa100b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa175e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 0x7ffa100b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa175e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa1015dbe0 0x7ffa100c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa175e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100b6ab0 con 0x7ffa100b92a0 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa175e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100d2210 con 0x7ffa100b2cb0 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa175e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100b6c30 con 0x7ffa1015dbe0 2024-04-25T08:33:45.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 0x7ffa100b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 0x7ffa100b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43742/0 (socket says 172.21.15.60:43742) 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 -- 172.21.15.60:0/4243262142 learned_addr learned my addr 172.21.15.60:0/4243262142 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa15383640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa100b2cb0 0x7ffa100b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa15b84640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa1015dbe0 0x7ffa100c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 -- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa1015dbe0 msgr2=0x7ffa100c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa1015dbe0 0x7ffa100c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 -- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa100b2cb0 msgr2=0x7ffa100b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa100b2cb0 0x7ffa100b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa14b82640 1 -- 172.21.15.60:0/4243262142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa100b5330 con 0x7ffa100b92a0 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa15383640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa100b2cb0 0x7ffa100b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.284+0000 7ffa15b84640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa1015dbe0 0x7ffa100c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.285+0000 7ffa14b82640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 0x7ffa100b4df0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000c410 tx=0x7ffa0000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.285+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa000074e0 con 0x7ffa100b92a0 2024-04-25T08:33:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.285+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa000047e0 con 0x7ffa100b92a0 2024-04-25T08:33:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.285+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa100b5600 con 0x7ffa100b92a0 2024-04-25T08:33:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.285+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00004bd0 con 0x7ffa100b92a0 2024-04-25T08:33:45.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.285+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa100b5ea0 con 0x7ffa100b92a0 2024-04-25T08:33:45.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.286+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa10073990 con 0x7ffa100b92a0 2024-04-25T08:33:45.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.286+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa00011070 con 0x7ffa100b92a0 2024-04-25T08:33:45.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.287+0000 7ffa05ffb640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9f406b4d0 0x7ff9f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.287+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ffa0008a0d0 con 0x7ffa100b92a0 2024-04-25T08:33:45.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.287+0000 7ffa15383640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9f406b4d0 0x7ff9f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.287+0000 7ffa15383640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9f406b4d0 0x7ff9f406d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c034000 tx=0x7ffa0c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.291+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa0000e280 con 0x7ffa100b92a0 2024-04-25T08:33:45.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.510+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa100b26b0 con 0x7ffa100b92a0 2024-04-25T08:33:45.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.511+0000 7ffa05ffb640 1 -- 172.21.15.60:0/4243262142 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ffa00058c40 con 0x7ffa100b92a0 2024-04-25T08:33:45.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:45.507 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:27.161423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:45.507 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9f406b4d0 msgr2=0x7ff9f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9f406b4d0 0x7ff9f406d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c034000 tx=0x7ffa0c02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 msgr2=0x7ffa100b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 0x7ffa100b4df0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000c410 tx=0x7ffa0000c8d0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 shutdown_connections 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffa1015dbe0 0x7ffa100c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffa100b92a0 0x7ffa100b4df0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9f406b4d0 0x7ff9f406d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 --2- 172.21.15.60:0/4243262142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffa100b2cb0 0x7ffa100b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 >> 172.21.15.60:0/4243262142 conn(0x7ffa10061a20 msgr2=0x7ffa100d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 shutdown_connections 2024-04-25T08:33:45.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.513+0000 7ffa175e7640 1 -- 172.21.15.60:0/4243262142 wait complete. 2024-04-25T08:33:45.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 Processor -- start 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 -- start start 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f7815dbe0 0x7f8f780d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f780b2cb0 0x7f8f780d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f780b92a0 0x7f8f780b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7815dfc0 con 0x7f8f7815dbe0 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f780d1a00 con 0x7f8f780b92a0 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f7d0e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780b97a0 con 0x7f8f780b2cb0 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.625+0000 7f8f76d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f7815dbe0 0x7f8f780d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8f7815dbe0 0x7f8f780d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43744/0 (socket says 172.21.15.60:43744) 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 -- 172.21.15.60:0/1697276981 learned_addr learned my addr 172.21.15.60:0/1697276981 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f7759e640 1 --2- 172.21.15.60:0/1697276981 >> v2:172.21.15.100:3300/0 conn(0x7f8f780b92a0 0x7f8f780b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 -- 172.21.15.60:0/1697276981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f780b2cb0 msgr2=0x7f8f780d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f6ffff640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f780b2cb0 0x7f8f780d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f780b2cb0 0x7f8f780d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 -- 172.21.15.60:0/1697276981 >> v2:172.21.15.100:3300/0 conn(0x7f8f780b92a0 msgr2=0x7f8f780b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 --2- 172.21.15.60:0/1697276981 >> v2:172.21.15.100:3300/0 conn(0x7f8f780b92a0 0x7f8f780b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f76d9d640 1 -- 172.21.15.60:0/1697276981 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780d1c50 con 0x7f8f7815dbe0 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f6ffff640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f780b2cb0 0x7f8f780d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.626+0000 7f8f7759e640 1 --2- 172.21.15.60:0/1697276981 >> v2:172.21.15.100:3300/0 conn(0x7f8f780b92a0 0x7f8f780b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f76d9d640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7815dbe0 0x7f8f780d0910 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806c640 tx=0x7f8f6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bdd2eee1d10614 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7659c640 1 -- 172.21.15.60:0/1697276981 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68037070 con 0x7f8f7815dbe0 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7659c640 1 -- 172.21.15.60:0/1697276981 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f68032440 con 0x7f8f7815dbe0 2024-04-25T08:33:45.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7659c640 1 -- 172.21.15.60:0/1697276981 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68032810 con 0x7f8f7815dbe0 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/1697276981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7815dbe0 msgr2=0x7f8f780d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7815dbe0 0x7f8f780d0910 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806c640 tx=0x7f8f6802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/1697276981 shutdown_connections 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/1697276981 >> v2:172.21.15.100:3300/0 conn(0x7f8f780b92a0 0x7f8f780b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f780b2cb0 0x7f8f780d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/1697276981 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f7815dbe0 0x7f8f780d0910 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/1697276981 >> 172.21.15.60:0/1697276981 conn(0x7f8f78061a20 msgr2=0x7f8f78061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/1697276981 shutdown_connections 2024-04-25T08:33:45.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.627+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/1697276981 wait complete. 2024-04-25T08:33:45.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 Processor -- start 2024-04-25T08:33:45.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 -- start start 2024-04-25T08:33:45.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 0x7f8f780b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f780b92a0 0x7f8f780b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f7815dbe0 0x7f8f781f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780b6ab0 con 0x7f8f780b2cb0 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780d2210 con 0x7f8f780b92a0 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f7d0e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780b6c30 con 0x7f8f7815dbe0 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f76d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 0x7f8f780b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f76d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 0x7f8f780b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43754/0 (socket says 172.21.15.60:43754) 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.629+0000 7f8f76d9d640 1 -- 172.21.15.60:0/3230338918 learned_addr learned my addr 172.21.15.60:0/3230338918 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f6ffff640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f780b92a0 0x7f8f780b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f7759e640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f7815dbe0 0x7f8f781f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f76d9d640 1 -- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f7815dbe0 msgr2=0x7f8f781f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f76d9d640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f7815dbe0 0x7f8f781f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f76d9d640 1 -- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f780b92a0 msgr2=0x7f8f780b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f76d9d640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f780b92a0 0x7f8f780b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f76d9d640 1 -- 172.21.15.60:0/3230338918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780b4430 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f6ffff640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f780b92a0 0x7f8f780b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f76d9d640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 0x7f8f780b3da0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8f68000c00 tx=0x7f8f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68041020 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f680306c0 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f780b4710 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68037040 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.630+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f780b4cd0 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.631+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f78073990 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.632+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f68044020 con 0x7f8f780b2cb0 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.632+0000 7f8f6f7fe640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f5806b500 0x7f8f5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.632+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8f680aabe0 con 0x7f8f780b2cb0 2024-04-25T08:33:45.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.632+0000 7f8f6ffff640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f5806b500 0x7f8f5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.633+0000 7f8f6ffff640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f5806b500 0x7f8f5806d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f781610a0 tx=0x7f8f60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.636+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f6804b030 con 0x7f8f780b2cb0 2024-04-25T08:33:45.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.855+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f780cda80 con 0x7f8f780b2cb0 2024-04-25T08:33:45.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.856+0000 7f8f6f7fe640 1 -- 172.21.15.60:0/3230338918 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8f68079880 con 0x7f8f780b2cb0 2024-04-25T08:33:45.851 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:45.851 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:07.899265+0000","modified":"2024-04-25T08:33:27.161423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6936":{"gid":6936,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.60:6835/1247115122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1247115122},{"type":"v1","addr":"172.21.15.60:6835","nonce":1247115122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:33:45.852 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 42 2024-04-25T08:33:45.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f5806b500 msgr2=0x7f8f5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f5806b500 0x7f8f5806d9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f781610a0 tx=0x7f8f60008040 comp rx=0 tx=0).stop 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 msgr2=0x7f8f780b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 0x7f8f780b3da0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8f68000c00 tx=0x7f8f680046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 shutdown_connections 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f7815dbe0 0x7f8f781f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f5806b500 0x7f8f5806d9c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f780b92a0 0x7f8f780b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 --2- 172.21.15.60:0/3230338918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f780b2cb0 0x7f8f780b3da0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 >> 172.21.15.60:0/3230338918 conn(0x7f8f78061a20 msgr2=0x7f8f7815b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 shutdown_connections 2024-04-25T08:33:45.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.858+0000 7f8f7d0e1640 1 -- 172.21.15.60:0/3230338918 wait complete. 2024-04-25T08:33:45.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:45.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.970+0000 7f8e7b6ee640 1 Processor -- start 2024-04-25T08:33:45.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.970+0000 7f8e7b6ee640 1 -- start start 2024-04-25T08:33:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e7b6ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 0x7f8e740ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e7b6ee640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7415dfe0 con 0x7f8e7415dc00 2024-04-25T08:33:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e7b6ee640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e740d00c0 con 0x7f8e740c6df0 2024-04-25T08:33:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e7b6ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740d03a0 con 0x7f8e740d1dd0 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e7948a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e7415dc00 0x7f8e740ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43308/0 (socket says 172.21.15.60:43308) 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e7948a640 1 -- 172.21.15.60:0/1650513208 learned_addr learned my addr 172.21.15.60:0/1650513208 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 0x7f8e740ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930085527 0 0) 0x7f8e70001110 con 0x7f8e7415dc00 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e60003520 con 0x7f8e7415dc00 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112781496 0 0) 0x7f8e60003520 con 0x7f8e7415dc00 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 msgr2=0x7f8e740ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 0x7f8e740ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 >> v1:172.21.15.100:6789/0 conn(0x7f8e740c6df0 legacy=0x7f8e740d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740cc440 con 0x7f8e7415dc00 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f8e70001840 con 0x7f8e7415dc00 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 >> v1:172.21.15.60:6789/0 conn(0x7f8e7415dc00 legacy=0x7f8e740ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 0x7f8e600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60003fb0 0x7f8e60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e60004d40 0x7f8e600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740d03a0 con 0x7f8e60004d40 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6000bb00 con 0x7f8e60003fb0 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.971+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6000bd00 con 0x7f8e740d1dd0 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e60004d40 0x7f8e600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e78c89640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60003fb0 0x7f8e60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e7948a640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 0x7f8e600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e79c8b640 1 -- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 msgr2=0x7f8e600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740d1dd0 0x7f8e600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e79c8b640 1 -- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60003fb0 msgr2=0x7f8e60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60003fb0 0x7f8e60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.972+0000 7f8e79c8b640 1 -- 172.21.15.60:0/1650513208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60003520 con 0x7f8e60004d40 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e60004d40 0x7f8e600051c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8e740b1060 tx=0x7f8e60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcd15c81b70613a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c037070 con 0x7f8e60004d40 2024-04-25T08:33:45.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c02e050 con 0x7f8e60004d40 2024-04-25T08:33:45.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e6bfff640 1 -- 172.21.15.60:0/1650513208 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c009c10 con 0x7f8e60004d40 2024-04-25T08:33:45.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/1650513208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e60004d40 msgr2=0x7f8e600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/1650513208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e60004d40 0x7f8e600051c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8e740b1060 tx=0x7f8e60005830 comp rx=0 tx=0).stop 2024-04-25T08:33:45.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e79c8b640 1 -- 172.21.15.60:0/1650513208 reap_dead start 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/1650513208 shutdown_connections 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/1650513208 >> 172.21.15.60:0/1650513208 conn(0x7f8e7406f5d0 msgr2=0x7f8e7406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/1650513208 shutdown_connections 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.973+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/1650513208 wait complete. 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 Processor -- start 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 -- start start 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60004d40 0x7f8e7414f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740b63d0 0x7f8e740b7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 0x7f8e7415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cfd90 con 0x7f8e7415e2a0 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cc130 con 0x7f8e60004d40 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.975+0000 7f8e7b6ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cdd70 con 0x7f8e740b63d0 2024-04-25T08:33:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 0x7f8e7415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 0x7f8e7415fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43770/0 (socket says 172.21.15.60:43770) 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 -- 172.21.15.60:0/3880046082 learned_addr learned my addr 172.21.15.60:0/3880046082 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e7948a640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60004d40 0x7f8e7414f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e78c89640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740b63d0 0x7f8e740b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 -- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740b63d0 msgr2=0x7f8e740b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740b63d0 0x7f8e740b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 -- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60004d40 msgr2=0x7f8e7414f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60004d40 0x7f8e7414f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 -- 172.21.15.60:0/3880046082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740b5370 con 0x7f8e7415e2a0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e79c8b640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 0x7f8e7415fab0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c002230 tx=0x7f8e6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c037040 con 0x7f8e7415e2a0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c044070 con 0x7f8e7415e2a0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7414f730 con 0x7f8e7415e2a0 2024-04-25T08:33:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c009c10 con 0x7f8e7415e2a0 2024-04-25T08:33:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.976+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e740b58f0 con 0x7f8e7415e2a0 2024-04-25T08:33:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.978+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e740cbaa0 con 0x7f8e7415e2a0 2024-04-25T08:33:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.978+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e6c03a070 con 0x7f8e7415e2a0 2024-04-25T08:33:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.978+0000 7f8e69ffb640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e5806b500 0x7f8e5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.979+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8e6c0aa980 con 0x7f8e7415e2a0 2024-04-25T08:33:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.979+0000 7f8e7948a640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e5806b500 0x7f8e5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:45.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.979+0000 7f8e7948a640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e5806b500 0x7f8e5806d9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8e70001eb0 tx=0x7f8e70001110 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:45.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:45.983+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e6c078fd0 con 0x7f8e7415e2a0 2024-04-25T08:33:46.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.164+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e740c66c0 con 0x7f8e7415e2a0 2024-04-25T08:33:46.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.165+0000 7f8e69ffb640 1 -- 172.21.15.60:0/3880046082 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17001 (secure 0 0 0) 0x7f8e6c07d070 con 0x7f8e7415e2a0 2024-04-25T08:33:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:46.161 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:24.149268+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:06.174848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:06.884806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/1590331682":"2024-04-26T08:33:24.140229+0000","172.21.15.60:6832/1590331682":"2024-04-26T08:33:24.140229+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e5806b500 msgr2=0x7f8e5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e5806b500 0x7f8e5806d9c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8e70001eb0 tx=0x7f8e70001110 comp rx=0 tx=0).stop 2024-04-25T08:33:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 msgr2=0x7f8e7415fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 0x7f8e7415fab0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c002230 tx=0x7f8e6c009750 comp rx=0 tx=0).stop 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 shutdown_connections 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e7415e2a0 0x7f8e7415fab0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e740b63d0 0x7f8e740b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e5806b500 0x7f8e5806d9c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.167+0000 7f8e7b6ee640 1 --2- 172.21.15.60:0/3880046082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e60004d40 0x7f8e7414f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.168+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 >> 172.21.15.60:0/3880046082 conn(0x7f8e7406f5d0 msgr2=0x7f8e7406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.168+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 shutdown_connections 2024-04-25T08:33:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.168+0000 7f8e7b6ee640 1 -- 172.21.15.60:0/3880046082 wait complete. 2024-04-25T08:33:46.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:33:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 Processor -- start 2024-04-25T08:33:46.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 -- start start 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff480b5620 0x7eff480b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7eff4806c2d0 0x7eff4806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4806cc80 con 0x7eff4815b7d0 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4806cf50 con 0x7eff480b5620 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.281+0000 7eff4e7b7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff480ce530 con 0x7eff4806c2d0 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff480b5620 0x7eff480b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7eff480b5620 0x7eff480b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43776/0 (socket says 172.21.15.60:43776) 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 -- 172.21.15.60:0/54931247 learned_addr learned my addr 172.21.15.60:0/54931247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff4cd54640 1 --2- 172.21.15.60:0/54931247 >> v2:172.21.15.100:3300/0 conn(0x7eff4806c2d0 0x7eff4806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 -- 172.21.15.60:0/54931247 >> v2:172.21.15.100:3300/0 conn(0x7eff4806c2d0 msgr2=0x7eff4806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:46.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 --2- 172.21.15.60:0/54931247 >> v2:172.21.15.100:3300/0 conn(0x7eff4806c2d0 0x7eff4806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 -- 172.21.15.60:0/54931247 >> v1:172.21.15.100:6789/0 conn(0x7eff4815b7d0 legacy=0x7eff480cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff477fe640 1 -- 172.21.15.60:0/54931247 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480ce810 con 0x7eff480b5620 2024-04-25T08:33:46.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.282+0000 7eff4cd54640 1 --2- 172.21.15.60:0/54931247 >> v2:172.21.15.100:3300/0 conn(0x7eff4806c2d0 0x7eff4806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff477fe640 1 --2- 172.21.15.60:0/54931247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff480b5620 0x7eff480b5ab0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff48161890 tx=0x7eff3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bb84bba5225775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff46ffd640 1 -- 172.21.15.60:0/54931247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c037070 con 0x7eff480b5620 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff46ffd640 1 -- 172.21.15.60:0/54931247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c032440 con 0x7eff480b5620 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff46ffd640 1 -- 172.21.15.60:0/54931247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c032810 con 0x7eff480b5620 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 -- 172.21.15.60:0/54931247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff480b5620 msgr2=0x7eff480b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/54931247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff480b5620 0x7eff480b5ab0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff48161890 tx=0x7eff3c02a270 comp rx=0 tx=0).stop 2024-04-25T08:33:46.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 -- 172.21.15.60:0/54931247 shutdown_connections 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/54931247 >> v2:172.21.15.100:3300/0 conn(0x7eff4806c2d0 0x7eff4806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/54931247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff480b5620 0x7eff480b5ab0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 -- 172.21.15.60:0/54931247 >> 172.21.15.60:0/54931247 conn(0x7eff480d0800 msgr2=0x7eff480d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 -- 172.21.15.60:0/54931247 shutdown_connections 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.283+0000 7eff4e7b7640 1 -- 172.21.15.60:0/54931247 wait complete. 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 Processor -- start 2024-04-25T08:33:46.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 -- start start 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 0x7eff480c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff480b5620 0x7eff480cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4815b7d0 0x7eff480cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815bbb0 con 0x7eff4806c2d0 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4806d230 con 0x7eff4815b7d0 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.285+0000 7eff4e7b7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815bd30 con 0x7eff480b5620 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 0x7eff480c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 0x7eff480c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43780/0 (socket says 172.21.15.60:43780) 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 -- 172.21.15.60:0/2312408911 learned_addr learned my addr 172.21.15.60:0/2312408911 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:46.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff477fe640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff480b5620 0x7eff480cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff4cd54640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4815b7d0 0x7eff480cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 -- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff480b5620 msgr2=0x7eff480cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff480b5620 0x7eff480cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 -- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4815b7d0 msgr2=0x7eff480cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4815b7d0 0x7eff480cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 -- 172.21.15.60:0/2312408911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480c7af0 con 0x7eff4806c2d0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff47fff640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 0x7eff480c8b90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c3c0 tx=0x7eff3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380052d0 con 0x7eff4806c2d0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff38005490 con 0x7eff4806c2d0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38007260 con 0x7eff4806c2d0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff480c7dd0 con 0x7eff4806c2d0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.286+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff480c8390 con 0x7eff4806c2d0 2024-04-25T08:33:46.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.287+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff48073a00 con 0x7eff4806c2d0 2024-04-25T08:33:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.288+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff38007400 con 0x7eff4806c2d0 2024-04-25T08:33:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.288+0000 7eff44ff9640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7eff3406b4d0 0x7eff3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.289+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7eff380898a0 con 0x7eff4806c2d0 2024-04-25T08:33:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.289+0000 7eff477fe640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7eff3406b4d0 0x7eff3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.289+0000 7eff477fe640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7eff3406b4d0 0x7eff3406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff3c034000 tx=0x7eff3c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:46.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.293+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff38057e60 con 0x7eff4806c2d0 2024-04-25T08:33:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.513+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7eff4806c7d0 con 0x7eff4806c2d0 2024-04-25T08:33:46.509 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:33:46.513+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:33:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.515+0000 7eff44ff9640 1 -- 172.21.15.60:0/2312408911 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7eff3805bf00 con 0x7eff4806c2d0 2024-04-25T08:33:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:33:46.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7eff3406b4d0 msgr2=0x7eff3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7eff3406b4d0 0x7eff3406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7eff3c034000 tx=0x7eff3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 msgr2=0x7eff480c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 0x7eff480c8b90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c3c0 tx=0x7eff3800c880 comp rx=0 tx=0).stop 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 shutdown_connections 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7eff3406b4d0 0x7eff3406d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff4815b7d0 0x7eff480cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7eff480b5620 0x7eff480cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.517+0000 7eff4e7b7640 1 --2- 172.21.15.60:0/2312408911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7eff4806c2d0 0x7eff480c8b90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.518+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 >> 172.21.15.60:0/2312408911 conn(0x7eff480d0800 msgr2=0x7eff4815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.518+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 shutdown_connections 2024-04-25T08:33:46.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.518+0000 7eff4e7b7640 1 -- 172.21.15.60:0/2312408911 wait complete. 2024-04-25T08:33:46.516 INFO:tasks.ceph.mds.a.smithi060.stderr: -1684> 2024-04-25T08:33:31.154+0000 7f1703d8f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:33:46.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:33:46.543 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:33:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.632+0000 7f69d42db640 1 Processor -- start 2024-04-25T08:33:46.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.632+0000 7f69d42db640 1 -- start start 2024-04-25T08:33:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d42db640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69cc15dc20 0x7f69cc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d42db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc161710 0x7f69cc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d42db640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc15e000 con 0x7f69cc0b9330 2024-04-25T08:33:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d42db640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc0d1de0 con 0x7f69cc15dc20 2024-04-25T08:33:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d42db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc161c60 con 0x7f69cc161710 2024-04-25T08:33:46.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d2077640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69cc15dc20 0x7f69cc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d2077640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69cc15dc20 0x7f69cc0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43802/0 (socket says 172.21.15.60:43802) 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.633+0000 7f69d2077640 1 -- 172.21.15.60:0/354392601 learned_addr learned my addr 172.21.15.60:0/354392601 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d1876640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc161710 0x7f69cc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d2077640 1 -- 172.21.15.60:0/354392601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc161710 msgr2=0x7f69cc0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d2077640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc161710 0x7f69cc0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d2077640 1 -- 172.21.15.60:0/354392601 >> v1:172.21.15.100:6789/0 conn(0x7f69cc0b9330 legacy=0x7f69cc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d2077640 1 -- 172.21.15.60:0/354392601 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc161f40 con 0x7f69cc15dc20 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d1876640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc161710 0x7f69cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.634+0000 7f69d2077640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc0ce670 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009e50 tx=0x7f69c402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca3457a13c18d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d1075640 1 -- 172.21.15.60:0/354392601 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4036070 con 0x7f69cc15dc20 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d1075640 1 -- 172.21.15.60:0/354392601 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c4031600 con 0x7f69cc15dc20 2024-04-25T08:33:46.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d1075640 1 -- 172.21.15.60:0/354392601 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c40319d0 con 0x7f69cc15dc20 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 -- 172.21.15.60:0/354392601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 msgr2=0x7f69cc0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc0ce670 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009e50 tx=0x7f69c402a6c0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 -- 172.21.15.60:0/354392601 shutdown_connections 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc161710 0x7f69cc0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 --2- 172.21.15.60:0/354392601 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc0ce670 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 -- 172.21.15.60:0/354392601 >> 172.21.15.60:0/354392601 conn(0x7f69cc0d0420 msgr2=0x7f69cc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 -- 172.21.15.60:0/354392601 shutdown_connections 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.635+0000 7f69d42db640 1 -- 172.21.15.60:0/354392601 wait complete. 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.636+0000 7f69d42db640 1 Processor -- start 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.636+0000 7f69d42db640 1 -- start start 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d42db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0b9330 0x7f69cc158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d42db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d42db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc161710 0x7f69cc0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d42db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b5e50 con 0x7f69cc15dc20 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d42db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc162500 con 0x7f69cc161710 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d42db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b5fd0 con 0x7f69cc0b9330 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d1876640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d1876640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43816/0 (socket says 172.21.15.60:43816) 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d1876640 1 -- 172.21.15.60:0/3520309099 learned_addr learned my addr 172.21.15.60:0/3520309099 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d2077640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0b9330 0x7f69cc158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.637+0000 7f69d2878640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc161710 0x7f69cc0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d1876640 1 -- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0b9330 msgr2=0x7f69cc158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d1876640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0b9330 0x7f69cc158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d1876640 1 -- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc161710 msgr2=0x7f69cc0cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d1876640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc161710 0x7f69cc0cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d1876640 1 -- 172.21.15.60:0/3520309099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0cab80 con 0x7f69cc15dc20 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d1876640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc160510 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f69c000c7f0 tx=0x7f69c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0015070 con 0x7f69cc15dc20 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c0007820 con 0x7f69cc15dc20 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0007bd0 con 0x7f69cc15dc20 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc0cae60 con 0x7f69cc15dc20 2024-04-25T08:33:46.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.638+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc0cb420 con 0x7f69cc15dc20 2024-04-25T08:33:46.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.639+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc073a00 con 0x7f69cc15dc20 2024-04-25T08:33:46.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.639+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69c0018070 con 0x7f69cc15dc20 2024-04-25T08:33:46.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.640+0000 7f69caffd640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f69b806b500 0x7f69b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:46.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.640+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f69c008a4f0 con 0x7f69cc15dc20 2024-04-25T08:33:46.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.640+0000 7f69d2077640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f69b806b500 0x7f69b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:46.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.641+0000 7f69d2077640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f69b806b500 0x7f69b806d9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f69c4033000 tx=0x7f69c4009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:46.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.644+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69c0059130 con 0x7f69cc15dc20 2024-04-25T08:33:46.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:46.863+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f69cc0d16b0 con 0x7f69cc15dc20 2024-04-25T08:33:47.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.516+0000 7f69caffd640 1 -- 172.21.15.60:0/3520309099 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f69c0058a30 con 0x7f69cc15dc20 2024-04-25T08:33:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f69b806b500 msgr2=0x7f69b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f69b806b500 0x7f69b806d9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f69c4033000 tx=0x7f69c4009000 comp rx=0 tx=0).stop 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 msgr2=0x7f69cc160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc160510 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f69c000c7f0 tx=0x7f69c000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 shutdown_connections 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f69b806b500 0x7f69b806d9c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69cc161710 0x7f69cc0cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69cc15dc20 0x7f69cc160510 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 --2- 172.21.15.60:0/3520309099 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69cc0b9330 0x7f69cc158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 >> 172.21.15.60:0/3520309099 conn(0x7f69cc0d0420 msgr2=0x7f69cc15bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 shutdown_connections 2024-04-25T08:33:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.519+0000 7f69d42db640 1 -- 172.21.15.60:0/3520309099 wait complete. 2024-04-25T08:33:47.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:33:47.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.684+0000 7f5d7f06b640 1 Processor -- start 2024-04-25T08:33:47.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.684+0000 7f5d7f06b640 1 -- start start 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.685+0000 7f5d7f06b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5d781616f0 0x7f5d78161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.685+0000 7f5d7f06b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7806c8d0 con 0x7f5d780ce0a0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.685+0000 7f5d7f06b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780c6960 con 0x7f5d781610f0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.685+0000 7f5d7f06b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d781614d0 con 0x7f5d781616f0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.685+0000 7f5d7d608640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5d780ce0a0 0x7f5d780d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43318/0 (socket says 172.21.15.60:43318) 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.685+0000 7f5d7d608640 1 -- 172.21.15.60:0/241458073 learned_addr learned my addr 172.21.15.60:0/241458073 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d77fff640 1 --2- 172.21.15.60:0/241458073 >> v2:172.21.15.100:3300/0 conn(0x7f5d781616f0 0x7f5d78161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238405544 0 0) 0x7f5d7806c8d0 con 0x7f5d780ce0a0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d60003520 con 0x7f5d780ce0a0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131323232 0 0) 0x7f5d60003520 con 0x7f5d780ce0a0 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 >> v2:172.21.15.100:3300/0 conn(0x7f5d781616f0 msgr2=0x7f5d78161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:47.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 --2- 172.21.15.60:0/241458073 >> v2:172.21.15.100:3300/0 conn(0x7f5d781616f0 0x7f5d78161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 >> v1:172.21.15.100:6789/0 conn(0x7f5d781610f0 legacy=0x7f5d7806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78162260 con 0x7f5d780ce0a0 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f5d700019f0 con 0x7f5d780ce0a0 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 >> v1:172.21.15.60:6789/0 conn(0x7f5d780ce0a0 legacy=0x7f5d780d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d60004800 0x7f5d60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d600051c0 0x7f5d6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781614d0 con 0x7f5d600039b0 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6000c060 con 0x7f5d600051c0 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6000c260 con 0x7f5d60004800 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d77fff640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d60004800 0x7f5d60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d7d608640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d600051c0 0x7f5d6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d7ce07640 1 -- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d60004800 msgr2=0x7f5d60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d60004800 0x7f5d60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.686+0000 7f5d7ce07640 1 -- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d600051c0 msgr2=0x7f5d6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d600051c0 0x7f5d6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d7ce07640 1 -- 172.21.15.60:0/241458073 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d60003520 con 0x7f5d600039b0 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d7d608640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d600051c0 0x7f5d6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d7d608640 1 -- 172.21.15.60:0/241458073 reap_dead start 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d60003dd0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001750 tx=0x7f5d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2873312ca51553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68001370 con 0x7f5d600039b0 2024-04-25T08:33:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d680052d0 con 0x7f5d600039b0 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.687+0000 7f5d777fe640 1 -- 172.21.15.60:0/241458073 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6802abc0 con 0x7f5d600039b0 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 -- 172.21.15.60:0/241458073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 msgr2=0x7f5d60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d60003dd0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001750 tx=0x7f5d60005830 comp rx=0 tx=0).stop 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 -- 172.21.15.60:0/241458073 shutdown_connections 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/241458073 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d60003dd0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 -- 172.21.15.60:0/241458073 >> 172.21.15.60:0/241458073 conn(0x7f5d7806f500 msgr2=0x7f5d7806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 -- 172.21.15.60:0/241458073 shutdown_connections 2024-04-25T08:33:47.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.688+0000 7f5d7f06b640 1 -- 172.21.15.60:0/241458073 wait complete. 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.689+0000 7f5d7f06b640 1 Processor -- start 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.689+0000 7f5d7f06b640 1 -- start start 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7f06b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d78156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7f06b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d781554b0 0x7f5d78147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7f06b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d7815fb40 0x7f5d7815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7f06b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780cfce0 con 0x7f5d600039b0 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7f06b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780cfa00 con 0x7f5d781554b0 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7f06b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780cf690 con 0x7f5d7815fb40 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7ce07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d78156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7ce07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d78156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43842/0 (socket says 172.21.15.60:43842) 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7ce07640 1 -- 172.21.15.60:0/3199114195 learned_addr learned my addr 172.21.15.60:0/3199114195 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7d608640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d7815fb40 0x7f5d7815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d7ce07640 1 -- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d7815fb40 msgr2=0x7f5d7815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.690+0000 7f5d77fff640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d781554b0 0x7f5d78147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d7815fb40 0x7f5d7815cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7ce07640 1 -- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d781554b0 msgr2=0x7f5d78147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d781554b0 0x7f5d78147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7ce07640 1 -- 172.21.15.60:0/3199114195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78160500 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7ce07640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d78156ba0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001870 tx=0x7f5d68001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68040030 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6802be50 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780b5840 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68001370 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.691+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7815ffc0 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.692+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78073a00 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.692+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d68043070 con 0x7f5d600039b0 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.693+0000 7f5d757fa640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d6406b500 0x7f5d6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:47.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.693+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f5d680a9c30 con 0x7f5d600039b0 2024-04-25T08:33:47.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.693+0000 7f5d77fff640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d6406b500 0x7f5d6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:47.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.694+0000 7f5d77fff640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d6406b500 0x7f5d6406d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5d780713b0 tx=0x7f5d6c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:47.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.697+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d6804b020 con 0x7f5d600039b0 2024-04-25T08:33:47.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:47.880+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d780d0b20 con 0x7f5d600039b0 2024-04-25T08:33:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.518+0000 7f5d757fa640 1 -- 172.21.15.60:0/3199114195 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5d680787e0 con 0x7f5d600039b0 2024-04-25T08:33:48.514 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d6406b500 msgr2=0x7f5d6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d6406b500 0x7f5d6406d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5d780713b0 tx=0x7f5d6c008040 comp rx=0 tx=0).stop 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 msgr2=0x7f5d78156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d78156ba0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5d68001870 tx=0x7f5d68001120 comp rx=0 tx=0).stop 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 shutdown_connections 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d7815fb40 0x7f5d7815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d6406b500 0x7f5d6406d9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d781554b0 0x7f5d78147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 --2- 172.21.15.60:0/3199114195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d600039b0 0x7f5d78156ba0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 >> 172.21.15.60:0/3199114195 conn(0x7f5d7806f500 msgr2=0x7f5d780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 shutdown_connections 2024-04-25T08:33:48.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.523+0000 7f5d7f06b640 1 -- 172.21.15.60:0/3199114195 wait complete. 2024-04-25T08:33:48.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:33:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.649+0000 7f66c25e0640 1 Processor -- start 2024-04-25T08:33:48.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.649+0000 7f66c25e0640 1 -- start start 2024-04-25T08:33:48.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c25e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66bc15dac0 0x7f66bc0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c25e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66bc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c25e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66bc0d2420 con 0x7f66bc0d34c0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c25e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc0d3030 con 0x7f66bc15dac0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c25e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0d2700 con 0x7f66bc0d1f30 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c0b7d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f66bc0d34c0 0x7f66bc0b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43324/0 (socket says 172.21.15.60:43324) 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66c0b7d640 1 -- 172.21.15.60:0/1754654387 learned_addr learned my addr 172.21.15.60:0/1754654387 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66bb7fe640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66bc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66bbfff640 1 --2- 172.21.15.60:0/1754654387 >> v2:172.21.15.100:3300/0 conn(0x7f66bc15dac0 0x7f66bc0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210185537 0 0) 0x7f66bc0d2420 con 0x7f66bc0d34c0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a8003520 con 0x7f66bc0d34c0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059807493 0 0) 0x7f66a8003520 con 0x7f66bc0d34c0 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 msgr2=0x7f66bc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66bc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 >> v2:172.21.15.100:3300/0 conn(0x7f66bc15dac0 msgr2=0x7f66bc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.650+0000 7f66baffd640 1 --2- 172.21.15.60:0/1754654387 >> v2:172.21.15.100:3300/0 conn(0x7f66bc15dac0 0x7f66bc0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0d29e0 con 0x7f66bc0d34c0 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f66b40019f0 con 0x7f66bc0d34c0 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 >> v1:172.21.15.60:6789/0 conn(0x7f66bc0d34c0 legacy=0x7f66bc0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8003f30 0x7f66a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0d3030 con 0x7f66a8004d40 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0d2700 con 0x7f66a8003f30 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a800b9f0 con 0x7f66bc0d1f30 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66c0b7d640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66bb7fe640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8003f30 0x7f66a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66bbfff640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66c0b7d640 1 -- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 msgr2=0x7f66a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66c0b7d640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66c0b7d640 1 -- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8003f30 msgr2=0x7f66a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66c0b7d640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8003f30 0x7f66a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66c0b7d640 1 -- 172.21.15.60:0/1754654387 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8003520 con 0x7f66a8004d40 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66bbfff640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0d1f30 0x7f66a80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.651+0000 7f66bb7fe640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8003f30 0x7f66a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66c0b7d640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f66b400a850 tx=0x7f66a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6230b1e2cd1bffef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b400a4a0 con 0x7f66a8004d40 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b402f070 con 0x7f66a8004d40 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66baffd640 1 -- 172.21.15.60:0/1754654387 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b402bd00 con 0x7f66a8004d40 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66c25e0640 1 -- 172.21.15.60:0/1754654387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66a8004d40 msgr2=0x7f66a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:48.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1754654387 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66a8004d40 0x7f66a80051c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f66b400a850 tx=0x7f66a8005830 comp rx=0 tx=0).stop 2024-04-25T08:33:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.652+0000 7f66c0b7d640 1 -- 172.21.15.60:0/1754654387 reap_dead start 2024-04-25T08:33:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.653+0000 7f66c25e0640 1 -- 172.21.15.60:0/1754654387 shutdown_connections 2024-04-25T08:33:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.653+0000 7f66c25e0640 1 -- 172.21.15.60:0/1754654387 >> 172.21.15.60:0/1754654387 conn(0x7f66bc061c30 msgr2=0x7f66bc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.653+0000 7f66c25e0640 1 -- 172.21.15.60:0/1754654387 shutdown_connections 2024-04-25T08:33:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.653+0000 7f66c25e0640 1 -- 172.21.15.60:0/1754654387 wait complete. 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.654+0000 7f66c25e0640 1 Processor -- start 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.654+0000 7f66c25e0640 1 -- start start 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8004d40 0x7f66bc0b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 0x7f66bc1f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0b82a0 0x7f66bc0b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0b97d0 con 0x7f66bc14ce70 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc15dea0 con 0x7f66a8004d40 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0b9950 con 0x7f66bc0b82a0 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 0x7f66bc1f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 0x7f66bc1f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43846/0 (socket says 172.21.15.60:43846) 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 -- 172.21.15.60:0/1925072187 learned_addr learned my addr 172.21.15.60:0/1925072187 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:48.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bbfff640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8004d40 0x7f66bc0b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c0b7d640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0b82a0 0x7f66bc0b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 -- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0b82a0 msgr2=0x7f66bc0b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0b82a0 0x7f66bc0b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 -- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8004d40 msgr2=0x7f66bc0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8004d40 0x7f66bc0b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 -- 172.21.15.60:0/1925072187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc14d2f0 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66bb7fe640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 0x7f66bc1f6940 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f66a4006ba0 tx=0x7f66a4011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400fa60 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a400fc00 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc1f70b0 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.655+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4004020 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.656+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc1f7670 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.656+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc0b8c60 con 0x7f66bc14ce70 2024-04-25T08:33:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.657+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66a4012040 con 0x7f66bc14ce70 2024-04-25T08:33:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.657+0000 7f66b8ff9640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f669c06b500 0x7f669c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.658+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f66a408a480 con 0x7f66bc14ce70 2024-04-25T08:33:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.658+0000 7f66bbfff640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f669c06b500 0x7f669c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.658+0000 7f66bbfff640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f669c06b500 0x7f669c06d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cdb30 tx=0x7f66ac009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.662+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66a4019030 con 0x7f66bc14ce70 2024-04-25T08:33:48.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:48.844+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66bc073a00 con 0x7f66bc14ce70 2024-04-25T08:33:49.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.520+0000 7f66b8ff9640 1 -- 172.21.15.60:0/1925072187 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f66a4058cc0 con 0x7f66bc14ce70 2024-04-25T08:33:49.516 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:33:49.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.523+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f669c06b500 msgr2=0x7f669c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.523+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f669c06b500 0x7f669c06d9c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0cdb30 tx=0x7f66ac009190 comp rx=0 tx=0).stop 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.523+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 msgr2=0x7f66bc1f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.523+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 0x7f66bc1f6940 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f66a4006ba0 tx=0x7f66a4011e40 comp rx=0 tx=0).stop 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 shutdown_connections 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66bc0b82a0 0x7f66bc0b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66bc14ce70 0x7f66bc1f6940 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f669c06b500 0x7f669c06d9c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 --2- 172.21.15.60:0/1925072187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66a8004d40 0x7f66bc0b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 >> 172.21.15.60:0/1925072187 conn(0x7f66bc061c30 msgr2=0x7f66bc0b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 shutdown_connections 2024-04-25T08:33:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.524+0000 7f66c25e0640 1 -- 172.21.15.60:0/1925072187 wait complete. 2024-04-25T08:33:49.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:33:49.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.648+0000 7f4083d29640 1 Processor -- start 2024-04-25T08:33:49.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.648+0000 7f4083d29640 1 -- start start 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f4083d29640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f407c15b980 0x7f407c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f4083d29640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c072280 con 0x7f407c0ce230 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f4083d29640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0714c0 con 0x7f407c0cc4f0 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f4083d29640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407c071270 con 0x7f407c15b980 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f40822c6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f407c15b980 0x7f407c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f40812c4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f407c0ce230 0x7f407c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43336/0 (socket says 172.21.15.60:43336) 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f40812c4640 1 -- 172.21.15.60:0/1071728563 learned_addr learned my addr 172.21.15.60:0/1071728563 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f40822c6640 1 -- 172.21.15.60:0/1071728563 >> v1:172.21.15.100:6789/0 conn(0x7f407c0cc4f0 legacy=0x7f407c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f40822c6640 1 -- 172.21.15.60:0/1071728563 >> v1:172.21.15.60:6789/0 conn(0x7f407c0ce230 legacy=0x7f407c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f40822c6640 1 -- 172.21.15.60:0/1071728563 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0717a0 con 0x7f407c15b980 2024-04-25T08:33:49.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.649+0000 7f4080ac3640 1 -- 172.21.15.60:0/1071728563 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465301122 0 0) 0x7f407c072280 con 0x7f407c0ce230 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.650+0000 7f40822c6640 1 --2- 172.21.15.60:0/1071728563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c0cc8d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4074009d90 tx=0x7f407402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f4fc7e4a0718e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.650+0000 7f4080ac3640 1 -- 172.21.15.60:0/1071728563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074007740 con 0x7f407c15b980 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.650+0000 7f4080ac3640 1 -- 172.21.15.60:0/1071728563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40740078e0 con 0x7f407c15b980 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.650+0000 7f4080ac3640 1 -- 172.21.15.60:0/1071728563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074007cb0 con 0x7f407c15b980 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.650+0000 7f4083d29640 1 -- 172.21.15.60:0/1071728563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 msgr2=0x7f407c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.650+0000 7f4083d29640 1 --2- 172.21.15.60:0/1071728563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c0cc8d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4074009d90 tx=0x7f407402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.651+0000 7f4083d29640 1 -- 172.21.15.60:0/1071728563 shutdown_connections 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.651+0000 7f4083d29640 1 --2- 172.21.15.60:0/1071728563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c0cc8d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.651+0000 7f4083d29640 1 -- 172.21.15.60:0/1071728563 >> 172.21.15.60:0/1071728563 conn(0x7f407c06abe0 msgr2=0x7f407c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:49.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.651+0000 7f4083d29640 1 -- 172.21.15.60:0/1071728563 shutdown_connections 2024-04-25T08:33:49.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.651+0000 7f4083d29640 1 -- 172.21.15.60:0/1071728563 wait complete. 2024-04-25T08:33:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.652+0000 7f4083d29640 1 Processor -- start 2024-04-25T08:33:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.652+0000 7f4083d29640 1 -- start start 2024-04-25T08:33:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f407c0cc4f0 0x7f407c0b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407c0ce230 0x7f407c1e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c1e6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c06e8a0 con 0x7f407c15b980 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c071d60 con 0x7f407c0ce230 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0ae500 con 0x7f407c0cc4f0 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c1e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c1e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43866/0 (socket says 172.21.15.60:43866) 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 -- 172.21.15.60:0/3425626725 learned_addr learned my addr 172.21.15.60:0/3425626725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40812c4640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407c0ce230 0x7f407c1e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4081ac5640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f407c0cc4f0 0x7f407c0b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 -- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f407c0cc4f0 msgr2=0x7f407c0b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f407c0cc4f0 0x7f407c0b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 -- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407c0ce230 msgr2=0x7f407c1e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407c0ce230 0x7f407c1e5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 -- 172.21.15.60:0/3425626725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c1e4050 con 0x7f407c15b980 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40812c4640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407c0ce230 0x7f407c1e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:49.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40822c6640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c1e6040 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4074009eb0 tx=0x7f4074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074040070 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074031c90 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c1e4330 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074045680 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.653+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c1e48f0 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.654+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c073a00 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.655+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4074039040 con 0x7f407c15b980 2024-04-25T08:33:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.655+0000 7f40727fc640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f406006b4d0 0x7f406006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.655+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f40740aba40 con 0x7f407c15b980 2024-04-25T08:33:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.656+0000 7f4081ac5640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f406006b4d0 0x7f406006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.656+0000 7f4081ac5640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f406006b4d0 0x7f406006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4078006fd0 tx=0x7f4078008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:49.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.660+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f407407ab30 con 0x7f407c15b980 2024-04-25T08:33:49.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.848+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f407c06efb0 con 0x7f407c15b980 2024-04-25T08:33:49.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.851+0000 7f40727fc640 1 -- 172.21.15.60:0/3425626725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f407407a430 con 0x7f407c15b980 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f406006b4d0 msgr2=0x7f406006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f406006b4d0 0x7f406006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4078006fd0 tx=0x7f4078008040 comp rx=0 tx=0).stop 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 msgr2=0x7f407c1e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c1e6040 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4074009eb0 tx=0x7f4074004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 shutdown_connections 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f407c15b980 0x7f407c1e6040 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f406006b4d0 0x7f406006d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f407c0ce230 0x7f407c1e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 --2- 172.21.15.60:0/3425626725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f407c0cc4f0 0x7f407c0b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 >> 172.21.15.60:0/3425626725 conn(0x7f407c06abe0 msgr2=0x7f407c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:49.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 shutdown_connections 2024-04-25T08:33:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.853+0000 7f4083d29640 1 -- 172.21.15.60:0/3425626725 wait complete. 2024-04-25T08:33:49.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T08:33:49.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.966+0000 7f6439800640 1 Processor -- start 2024-04-25T08:33:49.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.966+0000 7f6439800640 1 -- start start 2024-04-25T08:33:49.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6439800640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f643415dbd0 0x7f64340d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6439800640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f64340b5970 0x7f64340b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6439800640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643415dfb0 con 0x7f64340d2250 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6439800640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64340d3370 con 0x7f643415dbd0 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6439800640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64340d2760 con 0x7f64340b5970 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f643415dbd0 0x7f64340d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f643415dbd0 0x7f64340d1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43882/0 (socket says 172.21.15.60:43882) 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 -- 172.21.15.60:0/1426309366 learned_addr learned my addr 172.21.15.60:0/1426309366 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 -- 172.21.15.60:0/1426309366 >> v2:172.21.15.100:3300/0 conn(0x7f64340b5970 msgr2=0x7f64340b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f64337fe640 1 --2- 172.21.15.60:0/1426309366 >> v2:172.21.15.100:3300/0 conn(0x7f64340b5970 0x7f64340b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 --2- 172.21.15.60:0/1426309366 >> v2:172.21.15.100:3300/0 conn(0x7f64340b5970 0x7f64340b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 -- 172.21.15.60:0/1426309366 >> v1:172.21.15.100:6789/0 conn(0x7f64340d2250 legacy=0x7f64340b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f6432ffd640 1 -- 172.21.15.60:0/1426309366 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340d2a40 con 0x7f643415dbd0 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.967+0000 7f64337fe640 1 --2- 172.21.15.60:0/1426309366 >> v2:172.21.15.100:3300/0 conn(0x7f64340b5970 0x7f64340b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.968+0000 7f6432ffd640 1 --2- 172.21.15.60:0/1426309366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f643415dbd0 0x7f64340d1d10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f64340d01f0 tx=0x7f642402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301c8233fedef3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.968+0000 7f64327fc640 1 -- 172.21.15.60:0/1426309366 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424037070 con 0x7f643415dbd0 2024-04-25T08:33:49.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.968+0000 7f64327fc640 1 -- 172.21.15.60:0/1426309366 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424032440 con 0x7f643415dbd0 2024-04-25T08:33:49.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.968+0000 7f64327fc640 1 -- 172.21.15.60:0/1426309366 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424032810 con 0x7f643415dbd0 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 -- 172.21.15.60:0/1426309366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f643415dbd0 msgr2=0x7f64340d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 --2- 172.21.15.60:0/1426309366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f643415dbd0 0x7f64340d1d10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f64340d01f0 tx=0x7f642402a230 comp rx=0 tx=0).stop 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 -- 172.21.15.60:0/1426309366 shutdown_connections 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 --2- 172.21.15.60:0/1426309366 >> v2:172.21.15.100:3300/0 conn(0x7f64340b5970 0x7f64340b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 --2- 172.21.15.60:0/1426309366 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f643415dbd0 0x7f64340d1d10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 -- 172.21.15.60:0/1426309366 >> 172.21.15.60:0/1426309366 conn(0x7f64340d0440 msgr2=0x7f64340d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 -- 172.21.15.60:0/1426309366 shutdown_connections 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.969+0000 7f6439800640 1 -- 172.21.15.60:0/1426309366 wait complete. 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.970+0000 7f6439800640 1 Processor -- start 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.970+0000 7f6439800640 1 -- start start 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6439800640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 0x7f64341e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6439800640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64340d2250 0x7f64340c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6439800640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643415dbd0 0x7f64340cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6439800640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340ce200 con 0x7f64340b5970 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6439800640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340d3000 con 0x7f643415dbd0 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6439800640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340ce380 con 0x7f64340d2250 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 0x7f64341e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 0x7f64341e6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43898/0 (socket says 172.21.15.60:43898) 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 -- 172.21.15.60:0/3296415171 learned_addr learned my addr 172.21.15.60:0/3296415171 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f642bfff640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64340d2250 0x7f64340c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f64337fe640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643415dbd0 0x7f64340cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 -- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64340d2250 msgr2=0x7f64340c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64340d2250 0x7f64340c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 -- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643415dbd0 msgr2=0x7f64340cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643415dbd0 0x7f64340cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 -- 172.21.15.60:0/3296415171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64341471b0 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f64337fe640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643415dbd0 0x7f64340cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.971+0000 7f6432ffd640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 0x7f64341e6d30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6424000c00 tx=0x7f64240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.972+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424041070 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.972+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642403c870 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.972+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424037040 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.972+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434147490 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.972+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6434147a50 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.973+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434073990 con 0x7f64340b5970 2024-04-25T08:33:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.973+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6424044070 con 0x7f64340b5970 2024-04-25T08:33:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.974+0000 7f642b7fe640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641406b500 0x7f641406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.974+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f64240aab20 con 0x7f64340b5970 2024-04-25T08:33:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.974+0000 7f642bfff640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641406b500 0x7f641406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:49.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.975+0000 7f642bfff640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641406b500 0x7f641406d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f64340ab190 tx=0x7f641c007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:49.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:49.978+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6424079500 con 0x7f64340b5970 2024-04-25T08:33:50.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.227+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f64340d0b00 con 0x7f64340b5970 2024-04-25T08:33:50.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.521+0000 7f642b7fe640 1 -- 172.21.15.60:0/3296415171 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v239) v1 ==== 143+0+0 (secure 0 0 0) 0x7f642407d5a0 con 0x7f64340b5970 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641406b500 msgr2=0x7f641406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641406b500 0x7f641406d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f64340ab190 tx=0x7f641c007e30 comp rx=0 tx=0).stop 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 msgr2=0x7f64341e6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 0x7f64341e6d30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6424000c00 tx=0x7f64240046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 shutdown_connections 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641406b500 0x7f641406d9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f643415dbd0 0x7f64340cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64340d2250 0x7f64340c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 --2- 172.21.15.60:0/3296415171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64340b5970 0x7f64341e6d30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 >> 172.21.15.60:0/3296415171 conn(0x7f64340d0440 msgr2=0x7f64340cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 shutdown_connections 2024-04-25T08:33:50.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.524+0000 7f6439800640 1 -- 172.21.15.60:0/3296415171 wait complete. 2024-04-25T08:33:50.532 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T08:33:50.532 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T08:33:50.532 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:33:50.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.641+0000 7fd0d04ac640 1 Processor -- start 2024-04-25T08:33:50.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.641+0000 7fd0d04ac640 1 -- start start 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.641+0000 7fd0d04ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0c806b1a0 0x7fd0c806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0d04ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b580 0x7fd0c80cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0d04ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c8161960 con 0x7fd0c80cffa0 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0d04ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c80d0430 con 0x7fd0c806b1a0 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0d04ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80cbd50 con 0x7fd0c806b580 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0c806b1a0 0x7fd0c806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0c806b1a0 0x7fd0c806f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43912/0 (socket says 172.21.15.60:43912) 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 -- 172.21.15.60:0/1248627654 learned_addr learned my addr 172.21.15.60:0/1248627654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 -- 172.21.15.60:0/1248627654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b580 msgr2=0x7fd0c80cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0cda47640 1 --2- 172.21.15.60:0/1248627654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b580 0x7fd0c80cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 --2- 172.21.15.60:0/1248627654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b580 0x7fd0c80cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 -- 172.21.15.60:0/1248627654 >> v1:172.21.15.100:6789/0 conn(0x7fd0c80cffa0 legacy=0x7fd0c806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:50.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.642+0000 7fd0ce248640 1 -- 172.21.15.60:0/1248627654 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8161c40 con 0x7fd0c806b1a0 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.643+0000 7fd0ce248640 1 --2- 172.21.15.60:0/1248627654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c806b1a0 0x7fd0c806f9c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009e50 tx=0x7fd0c002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd167f2fabd2d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.643+0000 7fd0cd246640 1 -- 172.21.15.60:0/1248627654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0036070 con 0x7fd0c806b1a0 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.643+0000 7fd0cd246640 1 -- 172.21.15.60:0/1248627654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c0031600 con 0x7fd0c806b1a0 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.643+0000 7fd0cd246640 1 -- 172.21.15.60:0/1248627654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00319d0 con 0x7fd0c806b1a0 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.643+0000 7fd0d04ac640 1 -- 172.21.15.60:0/1248627654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c806b1a0 msgr2=0x7fd0c806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.643+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/1248627654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c806b1a0 0x7fd0c806f9c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009e50 tx=0x7fd0c002a630 comp rx=0 tx=0).stop 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.644+0000 7fd0d04ac640 1 -- 172.21.15.60:0/1248627654 shutdown_connections 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.644+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/1248627654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b580 0x7fd0c80cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.644+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/1248627654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c806b1a0 0x7fd0c806f9c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.644+0000 7fd0d04ac640 1 -- 172.21.15.60:0/1248627654 >> 172.21.15.60:0/1248627654 conn(0x7fd0c80b2220 msgr2=0x7fd0c80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.644+0000 7fd0d04ac640 1 -- 172.21.15.60:0/1248627654 shutdown_connections 2024-04-25T08:33:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.644+0000 7fd0d04ac640 1 -- 172.21.15.60:0/1248627654 wait complete. 2024-04-25T08:33:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 Processor -- start 2024-04-25T08:33:50.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 -- start start 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b1a0 0x7fd0c8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c806b580 0x7fd0c81597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 0x7fd0c81e75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8162720 con 0x7fd0c80cffa0 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8162200 con 0x7fd0c806b580 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.645+0000 7fd0d04ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8070330 con 0x7fd0c806b1a0 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 0x7fd0c81e75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 0x7fd0c81e75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43928/0 (socket says 172.21.15.60:43928) 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 -- 172.21.15.60:0/2199607077 learned_addr learned my addr 172.21.15.60:0/2199607077 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 -- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b1a0 msgr2=0x7fd0c8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b1a0 0x7fd0c8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 -- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c806b580 msgr2=0x7fd0c81597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c806b580 0x7fd0c81597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 -- 172.21.15.60:0/2199607077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c81e7af0 con 0x7fd0c80cffa0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0cea49640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 0x7fd0c81e75b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c410 tx=0x7fd0b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8011790 con 0x7fd0c80cffa0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b8017070 con 0x7fd0c80cffa0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c81e7d90 con 0x7fd0c80cffa0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8011bb0 con 0x7fd0c80cffa0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.646+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c81e8350 con 0x7fd0c80cffa0 2024-04-25T08:33:50.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.647+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c8073990 con 0x7fd0c80cffa0 2024-04-25T08:33:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.648+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0b801c460 con 0x7fd0c80cffa0 2024-04-25T08:33:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.648+0000 7fd0c6ffd640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd0b406b500 0x7fd0b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.648+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd0b8089c90 con 0x7fd0c80cffa0 2024-04-25T08:33:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.648+0000 7fd0ce248640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd0b406b500 0x7fd0b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.649+0000 7fd0ce248640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd0b406b500 0x7fd0b406d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0033000 tx=0x7fd0c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:50.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.652+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0b8021020 con 0x7fd0c80cffa0 2024-04-25T08:33:50.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.832+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0c8160b90 con 0x7fd0c80cffa0 2024-04-25T08:33:50.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.833+0000 7fd0c6ffd640 1 -- 172.21.15.60:0/2199607077 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13412 (secure 0 0 0) 0x7fd0b8058c30 con 0x7fd0c80cffa0 2024-04-25T08:33:50.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:50.829 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":74,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:49.519559+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/1247115122":"2024-04-26T08:33:46.514630+0000","172.21.15.60:6834/1247115122":"2024-04-26T08:33:46.514630+0000","172.21.15.60:6834/1590331682":"2024-04-26T08:33:24.140229+0000","172.21.15.60:6832/1590331682":"2024-04-26T08:33:24.140229+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:50.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd0b406b500 msgr2=0x7fd0b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd0b406b500 0x7fd0b406d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0033000 tx=0x7fd0c002e000 comp rx=0 tx=0).stop 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 msgr2=0x7fd0c81e75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 0x7fd0c81e75b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c410 tx=0x7fd0b800c8d0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 shutdown_connections 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0c80cffa0 0x7fd0c81e75b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd0b406b500 0x7fd0b406d9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0c806b580 0x7fd0c81597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 --2- 172.21.15.60:0/2199607077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0c806b1a0 0x7fd0c8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 >> 172.21.15.60:0/2199607077 conn(0x7fd0c80b2220 msgr2=0x7fd0c80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 shutdown_connections 2024-04-25T08:33:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.836+0000 7fd0d04ac640 1 -- 172.21.15.60:0/2199607077 wait complete. 2024-04-25T08:33:50.842 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:33:50.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:33:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.951+0000 7fbd71322640 1 Processor -- start 2024-04-25T08:33:50.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.951+0000 7fbd71322640 1 -- start start 2024-04-25T08:33:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd71322640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd6c1616f0 0x7fbd6c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd71322640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbd6c0ce0a0 0x7fbd6c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd71322640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c06c8d0 con 0x7fbd6c1610f0 2024-04-25T08:33:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd71322640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0cea50 con 0x7fbd6c1616f0 2024-04-25T08:33:50.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd71322640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c1614d0 con 0x7fbd6c0ce0a0 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd6a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbd6c1616f0 0x7fbd6c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd6affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbd6c1610f0 0x7fbd6c06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43352/0 (socket says 172.21.15.60:43352) 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.952+0000 7fbd6affd640 1 -- 172.21.15.60:0/1494173942 learned_addr learned my addr 172.21.15.60:0/1494173942 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd6b7fe640 1 --2- 172.21.15.60:0/1494173942 >> v2:172.21.15.100:3300/0 conn(0x7fbd6c0ce0a0 0x7fbd6c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1494173942 >> v2:172.21.15.100:3300/0 conn(0x7fbd6c0ce0a0 msgr2=0x7fbd6c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd6a7fc640 1 --2- 172.21.15.60:0/1494173942 >> v2:172.21.15.100:3300/0 conn(0x7fbd6c0ce0a0 0x7fbd6c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1494173942 >> v1:172.21.15.60:6789/0 conn(0x7fbd6c1610f0 legacy=0x7fbd6c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1494173942 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0c6960 con 0x7fbd6c1616f0 2024-04-25T08:33:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd69ffb640 1 -- 172.21.15.60:0/1494173942 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262524959 0 0) 0x7fbd6c06c8d0 con 0x7fbd6c1610f0 2024-04-25T08:33:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd6a7fc640 1 --2- 172.21.15.60:0/1494173942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1616f0 0x7fbd6c161b50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0b33e0 tx=0x7fbd5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8199ea1eb94d98cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.953+0000 7fbd69ffb640 1 -- 172.21.15.60:0/1494173942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54037070 con 0x7fbd6c1616f0 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd69ffb640 1 -- 172.21.15.60:0/1494173942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54032440 con 0x7fbd6c1616f0 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd69ffb640 1 -- 172.21.15.60:0/1494173942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54032810 con 0x7fbd6c1616f0 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 -- 172.21.15.60:0/1494173942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1616f0 msgr2=0x7fbd6c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 --2- 172.21.15.60:0/1494173942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1616f0 0x7fbd6c161b50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0b33e0 tx=0x7fbd5402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 -- 172.21.15.60:0/1494173942 shutdown_connections 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 --2- 172.21.15.60:0/1494173942 >> v2:172.21.15.100:3300/0 conn(0x7fbd6c0ce0a0 0x7fbd6c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 --2- 172.21.15.60:0/1494173942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1616f0 0x7fbd6c161b50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 -- 172.21.15.60:0/1494173942 >> 172.21.15.60:0/1494173942 conn(0x7fbd6c06f500 msgr2=0x7fbd6c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 -- 172.21.15.60:0/1494173942 shutdown_connections 2024-04-25T08:33:50.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.954+0000 7fbd71322640 1 -- 172.21.15.60:0/1494173942 wait complete. 2024-04-25T08:33:50.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.955+0000 7fbd71322640 1 Processor -- start 2024-04-25T08:33:50.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 -- start start 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0ce0a0 0x7fbd6c1f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 0x7fbd6c160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c1616f0 0x7fbd6c0b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0cfa00 con 0x7fbd6c1610f0 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c162540 con 0x7fbd6c0ce0a0 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd71322640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0cfce0 con 0x7fbd6c1616f0 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd6a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 0x7fbd6c160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd6a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 0x7fbd6c160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43934/0 (socket says 172.21.15.60:43934) 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.956+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1274567893 learned_addr learned my addr 172.21.15.60:0/1274567893 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6b7fe640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c1616f0 0x7fbd6c0b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6affd640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0ce0a0 0x7fbd6c1f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c1616f0 msgr2=0x7fbd6c0b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6a7fc640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c1616f0 0x7fbd6c0b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0ce0a0 msgr2=0x7fbd6c1f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6a7fc640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0ce0a0 0x7fbd6c1f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6a7fc640 1 -- 172.21.15.60:0/1274567893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0b4a00 con 0x7fbd6c1610f0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd6a7fc640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 0x7fbd6c160410 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54037050 con 0x7fbd6c1610f0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54038040 con 0x7fbd6c1610f0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c0b4ce0 con 0x7fbd6c1610f0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54004450 con 0x7fbd6c1610f0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.957+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c0b52a0 con 0x7fbd6c1610f0 2024-04-25T08:33:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.958+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c073a00 con 0x7fbd6c1610f0 2024-04-25T08:33:50.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.959+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd54030980 con 0x7fbd6c1610f0 2024-04-25T08:33:50.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.959+0000 7fbd637fe640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:50.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.959+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbd540aabf0 con 0x7fbd6c1610f0 2024-04-25T08:33:50.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.959+0000 7fbd6affd640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:50.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.960+0000 7fbd6affd640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbd64007fd0 tx=0x7fbd64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:50.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:50.963+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd54079d40 con 0x7fbd6c1610f0 2024-04-25T08:33:51.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.142+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd6c15d2f0 con 0x7fbd6c1610f0 2024-04-25T08:33:51.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.142+0000 7fbd637fe640 1 -- 172.21.15.60:0/1274567893 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd54079640 con 0x7fbd6c1610f0 2024-04-25T08:33:51.138 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:33:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbd4c06b4d0 msgr2=0x7fbd4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbd64007fd0 tx=0x7fbd64009040 comp rx=0 tx=0).stop 2024-04-25T08:33:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 msgr2=0x7fbd6c160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 0x7fbd6c160410 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd540046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 shutdown_connections 2024-04-25T08:33:51.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbd6c1616f0 0x7fbd6c0b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbd6c1610f0 0x7fbd6c160410 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbd4c06b4d0 0x7fbd4c06d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 --2- 172.21.15.60:0/1274567893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd6c0ce0a0 0x7fbd6c1f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 >> 172.21.15.60:0/1274567893 conn(0x7fbd6c06f500 msgr2=0x7fbd6c0c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:51.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 shutdown_connections 2024-04-25T08:33:51.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.144+0000 7fbd71322640 1 -- 172.21.15.60:0/1274567893 wait complete. 2024-04-25T08:33:51.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T08:33:51.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.256+0000 7f5ecf6ea640 1 Processor -- start 2024-04-25T08:33:51.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.256+0000 7f5ecf6ea640 1 -- start start 2024-04-25T08:33:51.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecf6ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815dbe0 0x7f5ec80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecf6ea640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ec80b2cb0 0x7f5ec80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecf6ea640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80b97a0 con 0x7f5ec80b92a0 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecf6ea640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec815dfc0 con 0x7f5ec80b2cb0 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecf6ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80d1a00 con 0x7f5ec815dbe0 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5eccc85640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ec80b2cb0 0x7f5ec80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecdc87640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5ec80b92a0 0x7f5ec80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43356/0 (socket says 172.21.15.60:43356) 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5eccc85640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ec80b2cb0 0x7f5ec80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43936/0 (socket says 172.21.15.60:43936) 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.257+0000 7f5ecdc87640 1 -- 172.21.15.60:0/1659360757 learned_addr learned my addr 172.21.15.60:0/1659360757 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5ecd486640 1 --2- 172.21.15.60:0/1659360757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815dbe0 0x7f5ec80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5eccc85640 1 -- 172.21.15.60:0/1659360757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815dbe0 msgr2=0x7f5ec80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5eccc85640 1 --2- 172.21.15.60:0/1659360757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815dbe0 0x7f5ec80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5eccc85640 1 -- 172.21.15.60:0/1659360757 >> v1:172.21.15.60:6789/0 conn(0x7f5ec80b92a0 legacy=0x7f5ec80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5ec7fff640 1 -- 172.21.15.60:0/1659360757 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498073220 0 0) 0x7f5ec80b97a0 con 0x7f5ec80b92a0 2024-04-25T08:33:51.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5eccc85640 1 -- 172.21.15.60:0/1659360757 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80d1c50 con 0x7f5ec80b2cb0 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.258+0000 7f5eccc85640 1 --2- 172.21.15.60:0/1659360757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b2cb0 0x7f5ec80d0e50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0097a0 tx=0x7f5ebc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a861ca60b8f915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ec7fff640 1 -- 172.21.15.60:0/1659360757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc037070 con 0x7f5ec80b2cb0 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ec7fff640 1 -- 172.21.15.60:0/1659360757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc032440 con 0x7f5ec80b2cb0 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ec7fff640 1 -- 172.21.15.60:0/1659360757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc032810 con 0x7f5ec80b2cb0 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/1659360757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b2cb0 msgr2=0x7f5ec80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/1659360757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b2cb0 0x7f5ec80d0e50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0097a0 tx=0x7f5ebc02a180 comp rx=0 tx=0).stop 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/1659360757 shutdown_connections 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/1659360757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b2cb0 0x7f5ec80d0e50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/1659360757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815dbe0 0x7f5ec80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/1659360757 >> 172.21.15.60:0/1659360757 conn(0x7f5ec8061a20 msgr2=0x7f5ec8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/1659360757 shutdown_connections 2024-04-25T08:33:51.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.259+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/1659360757 wait complete. 2024-04-25T08:33:51.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.260+0000 7f5ecf6ea640 1 Processor -- start 2024-04-25T08:33:51.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.260+0000 7f5ecf6ea640 1 -- start start 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecf6ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec80b2cb0 0x7f5ec815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecf6ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80b92a0 0x7f5ec80cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecf6ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 0x7f5ec80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecf6ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80b6ab0 con 0x7f5ec815dbe0 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecf6ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80d2210 con 0x7f5ec80b92a0 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecf6ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80b6c30 con 0x7f5ec80b2cb0 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecdc87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 0x7f5ec80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecdc87640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 0x7f5ec80b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43938/0 (socket says 172.21.15.60:43938) 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecdc87640 1 -- 172.21.15.60:0/4074092827 learned_addr learned my addr 172.21.15.60:0/4074092827 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.261+0000 7f5ecd486640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec80b2cb0 0x7f5ec815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5eccc85640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80b92a0 0x7f5ec80cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecdc87640 1 -- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec80b2cb0 msgr2=0x7f5ec815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecdc87640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec80b2cb0 0x7f5ec815f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecdc87640 1 -- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80b92a0 msgr2=0x7f5ec80cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecdc87640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80b92a0 0x7f5ec80cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecdc87640 1 -- 172.21.15.60:0/4074092827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80cd110 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecdc87640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 0x7f5ec80b4cc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c730 tx=0x7f5eb800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8018070 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb800ce40 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec80b41e0 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800a860 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.262+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec80b47a0 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.263+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8073990 con 0x7f5ec815dbe0 2024-04-25T08:33:51.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.263+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5eb801b070 con 0x7f5ec815dbe0 2024-04-25T08:33:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.264+0000 7f5ec5ffb640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5eb406b4d0 0x7f5eb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.264+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5eb8089650 con 0x7f5ec815dbe0 2024-04-25T08:33:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.264+0000 7f5ecd486640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5eb406b4d0 0x7f5eb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.264+0000 7f5ecd486640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5eb406b4d0 0x7f5eb406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006fd0 tx=0x7f5ec0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:51.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.268+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5eb80585f0 con 0x7f5ec815dbe0 2024-04-25T08:33:51.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.518+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f5ec80b7fc0 con 0x7f5ec815dbe0 2024-04-25T08:33:51.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.525+0000 7f5ec5ffb640 1 -- 172.21.15.60:0/4074092827 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v240) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5eb8057ef0 con 0x7f5ec815dbe0 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5eb406b4d0 msgr2=0x7f5eb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5eb406b4d0 0x7f5eb406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006fd0 tx=0x7f5ec0008040 comp rx=0 tx=0).stop 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 msgr2=0x7f5ec80b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 0x7f5ec80b4cc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c730 tx=0x7f5eb800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 shutdown_connections 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec815dbe0 0x7f5ec80b4cc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5eb406b4d0 0x7f5eb406d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec80b92a0 0x7f5ec80cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 --2- 172.21.15.60:0/4074092827 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec80b2cb0 0x7f5ec815f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 >> 172.21.15.60:0/4074092827 conn(0x7f5ec8061a20 msgr2=0x7f5ec815b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.527+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 shutdown_connections 2024-04-25T08:33:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.528+0000 7f5ecf6ea640 1 -- 172.21.15.60:0/4074092827 wait complete. 2024-04-25T08:33:51.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:33:51.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.641+0000 7f5e1ca5d640 1 Processor -- start 2024-04-25T08:33:51.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.641+0000 7f5e1ca5d640 1 -- start start 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.642+0000 7f5e1ca5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e180ced80 0x7f5e180d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.642+0000 7f5e1ca5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 0x7f5e180d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.642+0000 7f5e1ca5d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1815e000 con 0x7f5e1815dc20 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.642+0000 7f5e1ca5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180d1de0 con 0x7f5e180ced80 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.642+0000 7f5e1ca5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18161c60 con 0x7f5e18161710 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.642+0000 7f5e1659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e1815dc20 0x7f5e180ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43358/0 (socket says 172.21.15.60:43358) 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1659c640 1 -- 172.21.15.60:0/205104820 learned_addr learned my addr 172.21.15.60:0/205104820 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529993390 0 0) 0x7f5e1815e000 con 0x7f5e1815dc20 2024-04-25T08:33:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e15d9b640 1 --2- 172.21.15.60:0/205104820 >> v2:172.21.15.100:3300/0 conn(0x7f5e180ced80 0x7f5e180d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e16d9d640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 0x7f5e180d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e04003520 con 0x7f5e1815dc20 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1847766593 0 0) 0x7f5e04003520 con 0x7f5e1815dc20 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 msgr2=0x7f5e180d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 0x7f5e180d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 >> v2:172.21.15.100:3300/0 conn(0x7f5e180ced80 msgr2=0x7f5e180d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 --2- 172.21.15.60:0/205104820 >> v2:172.21.15.100:3300/0 conn(0x7f5e180ced80 0x7f5e180d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18161f40 con 0x7f5e1815dc20 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f5e080019f0 con 0x7f5e1815dc20 2024-04-25T08:33:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 >> v1:172.21.15.60:6789/0 conn(0x7f5e1815dc20 legacy=0x7f5e180ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e040039b0 0x7f5e04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 0x7f5e04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e04004d40 0x7f5e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180d1de0 con 0x7f5e040039b0 2024-04-25T08:33:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18161c60 con 0x7f5e04004d40 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0400b9c0 con 0x7f5e18161710 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e1659c640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e040039b0 0x7f5e04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e15d9b640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 0x7f5e04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.643+0000 7f5e16d9d640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e04004d40 0x7f5e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1659c640 1 -- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 msgr2=0x7f5e04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1659c640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18161710 0x7f5e04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1659c640 1 -- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e04004d40 msgr2=0x7f5e040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1659c640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e04004d40 0x7f5e040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1659c640 1 -- 172.21.15.60:0/205104820 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e04003520 con 0x7f5e040039b0 2024-04-25T08:33:51.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1659c640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e040039b0 0x7f5e04003dd0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800a920 tx=0x7f5e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dd646d06648044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800a4c0 con 0x7f5e040039b0 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0802f070 con 0x7f5e040039b0 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.644+0000 7f5e1559a640 1 -- 172.21.15.60:0/205104820 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0802bd20 con 0x7f5e040039b0 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/205104820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e040039b0 msgr2=0x7f5e04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/205104820 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e040039b0 0x7f5e04003dd0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800a920 tx=0x7f5e04005830 comp rx=0 tx=0).stop 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e16d9d640 1 -- 172.21.15.60:0/205104820 reap_dead start 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/205104820 shutdown_connections 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/205104820 >> 172.21.15.60:0/205104820 conn(0x7f5e180d0420 msgr2=0x7f5e180d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/205104820 shutdown_connections 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.645+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/205104820 wait complete. 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.646+0000 7f5e1ca5d640 1 Processor -- start 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.646+0000 7f5e1ca5d640 1 -- start start 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1ca5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e040039b0 0x7f5e18157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1ca5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 0x7f5e18159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1ca5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180d2730 0x7f5e180d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1ca5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b5e50 con 0x7f5e180c9db0 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1ca5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18162500 con 0x7f5e040039b0 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1ca5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b5fd0 con 0x7f5e180d2730 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e15d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 0x7f5e18159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e15d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 0x7f5e18159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:43958/0 (socket says 172.21.15.60:43958) 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e15d9b640 1 -- 172.21.15.60:0/2436785794 learned_addr learned my addr 172.21.15.60:0/2436785794 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e16d9d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180d2730 0x7f5e180d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.647+0000 7f5e1659c640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e040039b0 0x7f5e18157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e15d9b640 1 -- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180d2730 msgr2=0x7f5e180d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e15d9b640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180d2730 0x7f5e180d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e15d9b640 1 -- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e040039b0 msgr2=0x7f5e18157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e15d9b640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e040039b0 0x7f5e18157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e15d9b640 1 -- 172.21.15.60:0/2436785794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180ca210 con 0x7f5e180c9db0 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e1659c640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e040039b0 0x7f5e18157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:51.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e15d9b640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 0x7f5e18159420 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5e00011aa0 tx=0x7f5e00011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000fb80 con 0x7f5e180c9db0 2024-04-25T08:33:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0000fd20 con 0x7f5e180c9db0 2024-04-25T08:33:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e18159cc0 con 0x7f5e180c9db0 2024-04-25T08:33:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00014660 con 0x7f5e180c9db0 2024-04-25T08:33:51.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.648+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1815a280 con 0x7f5e180c9db0 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.649+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18073a00 con 0x7f5e180c9db0 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.650+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e00012040 con 0x7f5e180c9db0 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.650+0000 7f5e0effd640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5df806b500 0x7f5df806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.650+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5e0008a6a0 con 0x7f5e180c9db0 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.650+0000 7f5e1659c640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5df806b500 0x7f5df806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.651+0000 7f5e1659c640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5df806b500 0x7f5df806d9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5e08033000 tx=0x7f5e08007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:51.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.655+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e00019030 con 0x7f5e180c9db0 2024-04-25T08:33:51.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.833+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5e180d16b0 con 0x7f5e180c9db0 2024-04-25T08:33:51.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.836+0000 7f5e0effd640 1 -- 172.21.15.60:0/2436785794 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e00059640 con 0x7f5e180c9db0 2024-04-25T08:33:51.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5df806b500 msgr2=0x7f5df806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5df806b500 0x7f5df806d9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5e08033000 tx=0x7f5e08007000 comp rx=0 tx=0).stop 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 msgr2=0x7f5e18159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 0x7f5e18159420 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5e00011aa0 tx=0x7f5e00011f60 comp rx=0 tx=0).stop 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 shutdown_connections 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180d2730 0x7f5e180d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e180c9db0 0x7f5e18159420 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5df806b500 0x7f5df806d9c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 --2- 172.21.15.60:0/2436785794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e040039b0 0x7f5e18157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 >> 172.21.15.60:0/2436785794 conn(0x7f5e180d0420 msgr2=0x7f5e180b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 shutdown_connections 2024-04-25T08:33:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:51.838+0000 7f5e1ca5d640 1 -- 172.21.15.60:0/2436785794 wait complete. 2024-04-25T08:33:51.845 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:33:51.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:33:51.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:33:51.924 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:33:51.924 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:51.924 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:51.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:33:51.940 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:33:51.940 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:51.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:33:51.983 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:33:51.983 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:51.998 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:33:51.998 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:33:51.998 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:33:52.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:33:52.040 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:33:52.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:52.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.145+0000 7f3e53f0a640 1 Processor -- start 2024-04-25T08:33:52.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.145+0000 7f3e53f0a640 1 -- start start 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e53f0a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3e4c0b2cb0 0x7f3e4c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e53f0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 0x7f3e4c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e53f0a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4c0b97a0 con 0x7f3e4c15dbe0 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e53f0a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4c15dfc0 con 0x7f3e4c0b2cb0 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e53f0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0d1a00 con 0x7f3e4c0d1020 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e51ca6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3e4c15dbe0 0x7f3e4c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35358/0 (socket says 172.21.15.60:35358) 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1940083124 learned_addr learned my addr 172.21.15.60:0/1940083124 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940546717 0 0) 0x7f3e4c0b97a0 con 0x7f3e4c15dbe0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e38003520 con 0x7f3e4c15dbe0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e524a7640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 0x7f3e4c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e514a5640 1 --2- 172.21.15.60:0/1940083124 >> v2:172.21.15.100:3300/0 conn(0x7f3e4c0b2cb0 0x7f3e4c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1652868838 0 0) 0x7f3e38003520 con 0x7f3e4c15dbe0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 msgr2=0x7f3e4c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 0x7f3e4c0d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 >> v2:172.21.15.100:3300/0 conn(0x7f3e4c0b2cb0 msgr2=0x7f3e4c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 --2- 172.21.15.60:0/1940083124 >> v2:172.21.15.100:3300/0 conn(0x7f3e4c0b2cb0 0x7f3e4c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c0d1c50 con 0x7f3e4c15dbe0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3e48001670 con 0x7f3e4c15dbe0 2024-04-25T08:33:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 >> v1:172.21.15.60:6789/0 conn(0x7f3e4c15dbe0 legacy=0x7f3e4c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38004800 0x7f3e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 0x7f3e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0d1a00 con 0x7f3e380039b0 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3800ba20 con 0x7f3e38004800 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.146+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3800bc20 con 0x7f3e4c0d1020 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e514a5640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38004800 0x7f3e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e524a7640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 0x7f3e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 msgr2=0x7f3e380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c0d1020 0x7f3e380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38004800 msgr2=0x7f3e38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e38004800 0x7f3e38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.147+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1940083124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38003520 con 0x7f3e380039b0 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e38003dd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3e48007fd0 tx=0x7f3e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d701d80a2e88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48007d50 con 0x7f3e380039b0 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e480058d0 con 0x7f3e380039b0 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e50ca4640 1 -- 172.21.15.60:0/1940083124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48005ca0 con 0x7f3e380039b0 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1940083124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 msgr2=0x7f3e38003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1940083124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e38003dd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3e48007fd0 tx=0x7f3e38005830 comp rx=0 tx=0).stop 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e524a7640 1 -- 172.21.15.60:0/1940083124 reap_dead start 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1940083124 shutdown_connections 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1940083124 >> 172.21.15.60:0/1940083124 conn(0x7f3e4c061a20 msgr2=0x7f3e4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1940083124 shutdown_connections 2024-04-25T08:33:52.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.148+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1940083124 wait complete. 2024-04-25T08:33:52.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 Processor -- start 2024-04-25T08:33:52.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 -- start start 2024-04-25T08:33:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e4c154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e4c0cc280 0x7f3e4c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c156450 0x7f3e4c156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0b92a0 con 0x7f3e380039b0 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0d2210 con 0x7f3e4c0cc280 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e53f0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c0b9420 con 0x7f3e4c156450 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e51ca6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e4c154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e51ca6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e4c154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40298/0 (socket says 172.21.15.60:40298) 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.150+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1557246112 learned_addr learned my addr 172.21.15.60:0/1557246112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e514a5640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e4c0cc280 0x7f3e4c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e524a7640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c156450 0x7f3e4c156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c156450 msgr2=0x7f3e4c156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c156450 0x7f3e4c156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e4c0cc280 msgr2=0x7f3e4c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e4c0cc280 0x7f3e4c0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e51ca6640 1 -- 172.21.15.60:0/1557246112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4c15e060 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e51ca6640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e4c154ce0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3e4802adf0 tx=0x7f3e4c0d1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48041020 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48007070 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48038040 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4c15e340 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.151+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c15e900 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.152+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4c073990 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.153+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e48044020 con 0x7f3e380039b0 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.153+0000 7f3e427fc640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3e3006b500 0x7f3e3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.153+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3e480aa040 con 0x7f3e380039b0 2024-04-25T08:33:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.153+0000 7f3e514a5640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3e3006b500 0x7f3e3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.154+0000 7f3e514a5640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3e3006b500 0x7f3e3006d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0b29f0 tx=0x7f3e3c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.157+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e4804b030 con 0x7f3e380039b0 2024-04-25T08:33:52.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.374+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e4c062040 con 0x7f3e380039b0 2024-04-25T08:33:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.374+0000 7f3e427fc640 1 -- 172.21.15.60:0/1557246112 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3e48078fe0 con 0x7f3e380039b0 2024-04-25T08:33:52.370 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:52.370 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2179605441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2179605441},{"type":"v1","addr":"172.21.15.100:6835","nonce":2179605441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5792,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3422598251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3422598251},{"type":"v1","addr":"172.21.15.100:6837","nonce":3422598251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6924,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/2695184719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2695184719},{"type":"v1","addr":"172.21.15.60:6833","nonce":2695184719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7107,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/3414283743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":3414283743},{"type":"v1","addr":"172.21.15.60:6835","nonce":3414283743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[]} 2024-04-25T08:33:52.370 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 44 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3e3006b500 msgr2=0x7f3e3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3e3006b500 0x7f3e3006d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0b29f0 tx=0x7f3e3c008040 comp rx=0 tx=0).stop 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 msgr2=0x7f3e4c154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e4c154ce0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3e4802adf0 tx=0x7f3e4c0d1e20 comp rx=0 tx=0).stop 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 shutdown_connections 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3e4c156450 0x7f3e4c156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3e3006b500 0x7f3e3006d9c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e4c0cc280 0x7f3e4c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 --2- 172.21.15.60:0/1557246112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3e380039b0 0x7f3e4c154ce0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 >> 172.21.15.60:0/1557246112 conn(0x7f3e4c061a20 msgr2=0x7f3e4c0b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 shutdown_connections 2024-04-25T08:33:52.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.376+0000 7f3e53f0a640 1 -- 172.21.15.60:0/1557246112 wait complete. 2024-04-25T08:33:52.383 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:33:52.383 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:33:52.383 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:33:52.383 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:52.383 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:33:52.383 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:33:52.384 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:33:52.384 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:52.384 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:33:52.384 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:33:52.384 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:33:52.384 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:52.384 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:33:52.385 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:33:52.385 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:33:52.385 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:33:52.393 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:33:52.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:33:52.393 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:33:52.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:33:52.396 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:33:52.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:33:52.397 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:33:52.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:33:52.436 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:33:52.436 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:52.437 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:33:52.437 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:52.439 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:33:52.439 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:52.440 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:33:52.440 DEBUG:teuthology.parallel:result is None 2024-04-25T08:33:52.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:33:52.511 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:33:52.511 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:33:52.517 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:33:52.518 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:33:52.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.552+0000 7fb68d8ea640 1 Processor -- start 2024-04-25T08:33:52.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.552+0000 7fb68d8ea640 1 -- start start 2024-04-25T08:33:52.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.553+0000 7fb68d8ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.553+0000 7fb68d8ea640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6880b54a0 0x7fb6880b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.553+0000 7fb68d8ea640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6880cf1d0 con 0x7fb688161240 2024-04-25T08:33:52.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.553+0000 7fb68d8ea640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6880b7120 con 0x7fb6880b54a0 2024-04-25T08:33:52.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.553+0000 7fb68d8ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb688161620 con 0x7fb688161890 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6867fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb686ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb688161240 0x7fb6880ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44040/0 (socket says 172.21.15.60:44040) 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb686ffd640 1 -- 172.21.15.60:0/2741271973 learned_addr learned my addr 172.21.15.60:0/2741271973 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6877fe640 1 --2- 172.21.15.60:0/2741271973 >> v2:172.21.15.100:3300/0 conn(0x7fb6880b54a0 0x7fb6880b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb685ffb640 1 -- 172.21.15.60:0/2741271973 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823494637 0 0) 0x7fb6880cf1d0 con 0x7fb688161240 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb685ffb640 1 -- 172.21.15.60:0/2741271973 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb674003520 con 0x7fb688161240 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6877fe640 1 -- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 msgr2=0x7fb6880b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6877fe640 1 --2- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6877fe640 1 -- 172.21.15.60:0/2741271973 >> v1:172.21.15.100:6789/0 conn(0x7fb688161240 legacy=0x7fb6880ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6877fe640 1 -- 172.21.15.60:0/2741271973 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688161e00 con 0x7fb6880b54a0 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.554+0000 7fb6867fc640 1 --2- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.555+0000 7fb6877fe640 1 --2- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6880b54a0 0x7fb6880b5930 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d90 tx=0x7fb68002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8332ee5afceaebe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.555+0000 7fb685ffb640 1 -- 172.21.15.60:0/2741271973 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680007740 con 0x7fb6880b54a0 2024-04-25T08:33:52.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.555+0000 7fb685ffb640 1 -- 172.21.15.60:0/2741271973 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6800078e0 con 0x7fb6880b54a0 2024-04-25T08:33:52.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.555+0000 7fb685ffb640 1 -- 172.21.15.60:0/2741271973 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680007cb0 con 0x7fb6880b54a0 2024-04-25T08:33:52.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.555+0000 7fb68d8ea640 1 -- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6880b54a0 msgr2=0x7fb6880b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.555+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6880b54a0 0x7fb6880b5930 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d90 tx=0x7fb68002a660 comp rx=0 tx=0).stop 2024-04-25T08:33:52.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.556+0000 7fb68d8ea640 1 -- 172.21.15.60:0/2741271973 shutdown_connections 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.556+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6880b54a0 0x7fb6880b5930 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.556+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/2741271973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.556+0000 7fb68d8ea640 1 -- 172.21.15.60:0/2741271973 >> 172.21.15.60:0/2741271973 conn(0x7fb6880d04a0 msgr2=0x7fb6880d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.556+0000 7fb68d8ea640 1 -- 172.21.15.60:0/2741271973 shutdown_connections 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.556+0000 7fb68d8ea640 1 -- 172.21.15.60:0/2741271973 wait complete. 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.557+0000 7fb68d8ea640 1 Processor -- start 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.557+0000 7fb68d8ea640 1 -- start start 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb68d8ea640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 0x7fb6880ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb68d8ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb688161240 0x7fb688151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 0x7fb6880ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb68d8ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb68d8ea640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881626a0 con 0x7fb6880b54a0 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb68d8ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881623c0 con 0x7fb688161240 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb68d8ea640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880d15f0 con 0x7fb688161890 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 0x7fb6880ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40334/0 (socket says 172.21.15.60:40334) 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 -- 172.21.15.60:0/152945683 learned_addr learned my addr 172.21.15.60:0/152945683 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb6867fc640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb688161240 0x7fb688151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb6877fe640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 -- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 msgr2=0x7fb6880c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 -- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb688161240 msgr2=0x7fb688151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb688161240 0x7fb688151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 -- 172.21.15.60:0/152945683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880c7ac0 con 0x7fb6880b54a0 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb6867fc640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb688161240 0x7fb688151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb686ffd640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 0x7fb6880ccf00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c530 tx=0x7fb67800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:52.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678017070 con 0x7fb6880b54a0 2024-04-25T08:33:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.558+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678005ce0 con 0x7fb6880b54a0 2024-04-25T08:33:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.559+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780072b0 con 0x7fb6880b54a0 2024-04-25T08:33:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.559+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6880c7da0 con 0x7fb6880b54a0 2024-04-25T08:33:52.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.559+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6880c8360 con 0x7fb6880b54a0 2024-04-25T08:33:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.559+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6880c87a0 con 0x7fb6880b54a0 2024-04-25T08:33:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.560+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb67801a070 con 0x7fb6880b54a0 2024-04-25T08:33:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.560+0000 7fb67ffff640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb66806b4d0 0x7fb66806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.561+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb678089470 con 0x7fb6880b54a0 2024-04-25T08:33:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.561+0000 7fb6867fc640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb66806b4d0 0x7fb66806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:52.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.561+0000 7fb6867fc640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb66806b4d0 0x7fb66806d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb6880cbcd0 tx=0x7fb670008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:52.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.565+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb678058560 con 0x7fb6880b54a0 2024-04-25T08:33:52.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:52.758+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb688073a00 con 0x7fb6880b54a0 2024-04-25T08:33:53.545 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.550+0000 7fb67ffff640 1 -- 172.21.15.60:0/152945683 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb678057e60 con 0x7fb6880b54a0 2024-04-25T08:33:53.545 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:33:53.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb66806b4d0 msgr2=0x7fb66806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb66806b4d0 0x7fb66806d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb6880cbcd0 tx=0x7fb670008040 comp rx=0 tx=0).stop 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 msgr2=0x7fb6880ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 0x7fb6880ccf00 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c530 tx=0x7fb67800c9f0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 shutdown_connections 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb688161890 0x7fb6880c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb66806b4d0 0x7fb66806d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb688161240 0x7fb688151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 --2- 172.21.15.60:0/152945683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6880b54a0 0x7fb6880ccf00 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 >> 172.21.15.60:0/152945683 conn(0x7fb6880d04a0 msgr2=0x7fb68815b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 shutdown_connections 2024-04-25T08:33:53.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.553+0000 7fb68d8ea640 1 -- 172.21.15.60:0/152945683 wait complete. 2024-04-25T08:33:53.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:33:53.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.674+0000 7f2c29c4d640 1 Processor -- start 2024-04-25T08:33:53.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.674+0000 7f2c29c4d640 1 -- start start 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c29c4d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c240c6df0 0x7f2c240d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c29c4d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c240d1c00 0x7f2c240ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c29c4d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2415dfe0 con 0x7f2c2415dc00 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c29c4d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c240d2070 con 0x7f2c240c6df0 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c29c4d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c240d00c0 con 0x7f2c240d1c00 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c240c6df0 0x7f2c240d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2c240c6df0 0x7f2c240d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40344/0 (socket says 172.21.15.60:40344) 2024-04-25T08:33:53.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 -- 172.21.15.60:0/2071895848 learned_addr learned my addr 172.21.15.60:0/2071895848 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c23fff640 1 --2- 172.21.15.60:0/2071895848 >> v2:172.21.15.100:3300/0 conn(0x7f2c240d1c00 0x7f2c240ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 -- 172.21.15.60:0/2071895848 >> v2:172.21.15.100:3300/0 conn(0x7f2c240d1c00 msgr2=0x7f2c240ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 --2- 172.21.15.60:0/2071895848 >> v2:172.21.15.100:3300/0 conn(0x7f2c240d1c00 0x7f2c240ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 -- 172.21.15.60:0/2071895848 >> v1:172.21.15.100:6789/0 conn(0x7f2c2415dc00 legacy=0x7f2c240ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:53.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.675+0000 7f2c22ffd640 1 -- 172.21.15.60:0/2071895848 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c240d03a0 con 0x7f2c240c6df0 2024-04-25T08:33:53.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.676+0000 7f2c22ffd640 1 --2- 172.21.15.60:0/2071895848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c240d16c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2c240b1040 tx=0x7f2c1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be28db3f6003af1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.676+0000 7f2c227fc640 1 -- 172.21.15.60:0/2071895848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18037070 con 0x7f2c240c6df0 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.676+0000 7f2c227fc640 1 -- 172.21.15.60:0/2071895848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c18032440 con 0x7f2c240c6df0 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.676+0000 7f2c227fc640 1 -- 172.21.15.60:0/2071895848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18032810 con 0x7f2c240c6df0 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 -- 172.21.15.60:0/2071895848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 msgr2=0x7f2c240d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/2071895848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c240d16c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2c240b1040 tx=0x7f2c1802a270 comp rx=0 tx=0).stop 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 -- 172.21.15.60:0/2071895848 shutdown_connections 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/2071895848 >> v2:172.21.15.100:3300/0 conn(0x7f2c240d1c00 0x7f2c240ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/2071895848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c240d16c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 -- 172.21.15.60:0/2071895848 >> 172.21.15.60:0/2071895848 conn(0x7f2c2406f5d0 msgr2=0x7f2c2406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 -- 172.21.15.60:0/2071895848 shutdown_connections 2024-04-25T08:33:53.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.677+0000 7f2c29c4d640 1 -- 172.21.15.60:0/2071895848 wait complete. 2024-04-25T08:33:53.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.678+0000 7f2c29c4d640 1 Processor -- start 2024-04-25T08:33:53.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.678+0000 7f2c29c4d640 1 -- start start 2024-04-25T08:33:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c29c4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c241533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c29c4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c240d1c00 0x7f2c240c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c29c4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c2415dc00 0x7f2c241599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c29c4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240cc130 con 0x7f2c240c6df0 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c29c4d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240cc720 con 0x7f2c240d1c00 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c29c4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240cfd90 con 0x7f2c2415dc00 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c241533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c241533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40356/0 (socket says 172.21.15.60:40356) 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 -- 172.21.15.60:0/3902045016 learned_addr learned my addr 172.21.15.60:0/3902045016 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c22ffd640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c240d1c00 0x7f2c240c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c23fff640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c2415dc00 0x7f2c241599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 -- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c2415dc00 msgr2=0x7f2c241599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c2415dc00 0x7f2c241599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 -- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c240d1c00 msgr2=0x7f2c240c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c240d1c00 0x7f2c240c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c237fe640 1 -- 172.21.15.60:0/3902045016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c240b4d40 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.679+0000 7f2c22ffd640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c240d1c00 0x7f2c240c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.680+0000 7f2c237fe640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c241533b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c3c0 tx=0x7f2c1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.680+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140052d0 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.680+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c14005490 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.680+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c241538f0 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.680+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14007260 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.680+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c240c9fc0 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.681+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c240b5130 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.681+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c14007400 con 0x7f2c240c6df0 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.682+0000 7f2c0bfff640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2c1006b4d0 0x7f2c1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:53.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.682+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2c14089570 con 0x7f2c240c6df0 2024-04-25T08:33:53.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.682+0000 7f2c22ffd640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2c1006b4d0 0x7f2c1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:53.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.682+0000 7f2c22ffd640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2c1006b4d0 0x7f2c1006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c18034000 tx=0x7f2c1802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:53.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.686+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c2415dfe0 con 0x7f2c240c6df0 2024-04-25T08:33:53.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.935+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2c240c66c0 con 0x7f2c240c6df0 2024-04-25T08:33:53.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.935+0000 7f2c0bfff640 1 -- 172.21.15.60:0/3902045016 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2c14058520 con 0x7f2c240c6df0 2024-04-25T08:33:53.931 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:53.931 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:33:53.931 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:33:53.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:33:53.932 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:53.933 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.935 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.936 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:33:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:33:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:33:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:33:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:33:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:33:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:33:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:33:53.945 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:33:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2c1006b4d0 msgr2=0x7f2c1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2c1006b4d0 0x7f2c1006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c18034000 tx=0x7f2c1802e000 comp rx=0 tx=0).stop 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 msgr2=0x7f2c241533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c241533b0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c3c0 tx=0x7f2c1400c880 comp rx=0 tx=0).stop 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 shutdown_connections 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2c2415dc00 0x7f2c241599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2c1006b4d0 0x7f2c1006d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c240d1c00 0x7f2c240c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 --2- 172.21.15.60:0/3902045016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2c240c6df0 0x7f2c241533b0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 >> 172.21.15.60:0/3902045016 conn(0x7f2c2406f5d0 msgr2=0x7f2c24156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 shutdown_connections 2024-04-25T08:33:53.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:53.937+0000 7f2c29c4d640 1 -- 172.21.15.60:0/3902045016 wait complete. 2024-04-25T08:33:53.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.053+0000 7fbbe38d2640 1 Processor -- start 2024-04-25T08:33:54.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.053+0000 7fbbe38d2640 1 -- start start 2024-04-25T08:33:54.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe38d2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbbdc15db70 0x7fbbdc162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe38d2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbdc162690 con 0x7fbbdc0b5c30 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe38d2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbdc0ceee0 con 0x7fbbdc0b6cb0 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe38d2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0d1d10 con 0x7fbbdc15db70 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe1e6f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbbdc0b5c30 0x7fbbdc0d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35394/0 (socket says 172.21.15.60:35394) 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe1e6f640 1 -- 172.21.15.60:0/1255177897 learned_addr learned my addr 172.21.15.60:0/1255177897 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100298478 0 0) 0x7fbbdc162690 con 0x7fbbdc0b5c30 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbe166e640 1 --2- 172.21.15.60:0/1255177897 >> v2:172.21.15.100:3300/0 conn(0x7fbbdc15db70 0x7fbbdc162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.054+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc4003520 con 0x7fbbdc0b5c30 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271639604 0 0) 0x7fbbc4003520 con 0x7fbbdc0b5c30 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 >> v2:172.21.15.100:3300/0 conn(0x7fbbdc15db70 msgr2=0x7fbbdc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 --2- 172.21.15.60:0/1255177897 >> v2:172.21.15.100:3300/0 conn(0x7fbbdc15db70 0x7fbbdc162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 >> v1:172.21.15.100:6789/0 conn(0x7fbbdc0b6cb0 legacy=0x7fbbdc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0d1ff0 con 0x7fbbdc0b5c30 2024-04-25T08:33:54.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fbbcc0019f0 con 0x7fbbdc0b5c30 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 >> v1:172.21.15.60:6789/0 conn(0x7fbbdc0b5c30 legacy=0x7fbbdc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc4004800 0x7fbbc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc40051c0 0x7fbbc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0d1d10 con 0x7fbbc40039b0 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc400c060 con 0x7fbbc40051c0 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc400c260 con 0x7fbbc4004800 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe166e640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe1e6f640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc40051c0 0x7fbbc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe0e6d640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc4004800 0x7fbbc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe166e640 1 -- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc4004800 msgr2=0x7fbbc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe166e640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbc4004800 0x7fbbc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe166e640 1 -- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc40051c0 msgr2=0x7fbbc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe166e640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc40051c0 0x7fbbc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe166e640 1 -- 172.21.15.60:0/1255177897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4003520 con 0x7fbbc40039b0 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe1e6f640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbc40051c0 0x7fbbc400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.055+0000 7fbbe1e6f640 1 -- 172.21.15.60:0/1255177897 reap_dead start 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.056+0000 7fbbe166e640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbc4003dd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009ac0 tx=0x7fbbc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7361caa164f6f5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.056+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4036070 con 0x7fbbc40039b0 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.056+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4031400 con 0x7fbbc40039b0 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.056+0000 7fbbdbfff640 1 -- 172.21.15.60:0/1255177897 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4009690 con 0x7fbbc40039b0 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 -- 172.21.15.60:0/1255177897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 msgr2=0x7fbbc4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbc4003dd0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009ac0 tx=0x7fbbc4005830 comp rx=0 tx=0).stop 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 -- 172.21.15.60:0/1255177897 shutdown_connections 2024-04-25T08:33:54.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/1255177897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbc4003dd0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 -- 172.21.15.60:0/1255177897 >> 172.21.15.60:0/1255177897 conn(0x7fbbdc0d0520 msgr2=0x7fbbdc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 -- 172.21.15.60:0/1255177897 shutdown_connections 2024-04-25T08:33:54.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.057+0000 7fbbe38d2640 1 -- 172.21.15.60:0/1255177897 wait complete. 2024-04-25T08:33:54.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.058+0000 7fbbe38d2640 1 Processor -- start 2024-04-25T08:33:54.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.058+0000 7fbbe38d2640 1 -- start start 2024-04-25T08:33:54.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbdc153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbdc155900 0x7fbbdc155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbdc0c7100 0x7fbbdc0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0ce530 con 0x7fbbc40039b0 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc15df50 con 0x7fbbdc155900 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0cf1c0 con 0x7fbbdc0c7100 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbdc153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbdc153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40366/0 (socket says 172.21.15.60:40366) 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 -- 172.21.15.60:0/3117931610 learned_addr learned my addr 172.21.15.60:0/3117931610 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe0e6d640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbdc155900 0x7fbbdc155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe1e6f640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbdc0c7100 0x7fbbdc0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 -- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbdc0c7100 msgr2=0x7fbbdc0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbdc0c7100 0x7fbbdc0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 -- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbdc155900 msgr2=0x7fbbdc155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbdc155900 0x7fbbdc155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 -- 172.21.15.60:0/3117931610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc156a60 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe1e6f640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbdc0c7100 0x7fbbdc0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe166e640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbdc153950 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4034040 tx=0x7fbbd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd403f020 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd4037040 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc1564a0 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.059+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4036070 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.060+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc15f250 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.060+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbdc073990 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.061+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbd4042020 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.061+0000 7fbbd9ffb640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbbc806b4d0 0x7fbbc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.062+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbbd40aa040 con 0x7fbbc40039b0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.062+0000 7fbbe0e6d640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbbc806b4d0 0x7fbbc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.062+0000 7fbbe0e6d640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbbc806b4d0 0x7fbbc806d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0007fd0 tx=0x7fbbd0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.066+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbd4049030 con 0x7fbbc40039b0 2024-04-25T08:33:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.247+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbdc1f3870 con 0x7fbbc40039b0 2024-04-25T08:33:54.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.248+0000 7fbbd9ffb640 1 -- 172.21.15.60:0/3117931610 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13153 (secure 0 0 0) 0x7fbbdc156a60 con 0x7fbbc40039b0 2024-04-25T08:33:54.243 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:54.244 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":79,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:53.549562+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbbc806b4d0 msgr2=0x7fbbc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbbc806b4d0 0x7fbbc806d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0007fd0 tx=0x7fbbd0009040 comp rx=0 tx=0).stop 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 msgr2=0x7fbbdc153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbdc153950 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4034040 tx=0x7fbbd4004730 comp rx=0 tx=0).stop 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 shutdown_connections 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbdc0c7100 0x7fbbdc0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbbc806b4d0 0x7fbbc806d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbdc155900 0x7fbbdc155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 --2- 172.21.15.60:0/3117931610 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbbc40039b0 0x7fbbdc153950 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 >> 172.21.15.60:0/3117931610 conn(0x7fbbdc0d0520 msgr2=0x7fbbdc0b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 shutdown_connections 2024-04-25T08:33:54.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.250+0000 7fbbe38d2640 1 -- 172.21.15.60:0/3117931610 wait complete. 2024-04-25T08:33:54.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:33:54.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:33:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.365+0000 7efeab553640 1 Processor -- start 2024-04-25T08:33:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.365+0000 7efeab553640 1 -- start start 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efeab553640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 0x7efea40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efeab553640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efea40c70d0 0x7efea40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efeab553640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40c75e0 con 0x7efea40cfb40 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efeab553640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea40c8680 con 0x7efea40c70d0 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efeab553640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c78c0 con 0x7efea415bb50 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efea9af0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efea40cfb40 0x7efea4070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35402/0 (socket says 172.21.15.60:35402) 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.366+0000 7efea9af0640 1 -- 172.21.15.60:0/1147651852 learned_addr learned my addr 172.21.15.60:0/1147651852 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea8aee640 1 --2- 172.21.15.60:0/1147651852 >> v2:172.21.15.100:3300/0 conn(0x7efea40c70d0 0x7efea40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 0x7efea40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422491532 0 0) 0x7efea40c75e0 con 0x7efea40cfb40 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c003520 con 0x7efea40cfb40 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775992494 0 0) 0x7efe8c003520 con 0x7efea40cfb40 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 msgr2=0x7efea40c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 0x7efea40c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 >> v2:172.21.15.100:3300/0 conn(0x7efea40c70d0 msgr2=0x7efea40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 --2- 172.21.15.60:0/1147651852 >> v2:172.21.15.100:3300/0 conn(0x7efea40c70d0 0x7efea40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea40c7ba0 con 0x7efea40cfb40 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7efe94001670 con 0x7efea40cfb40 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 >> v1:172.21.15.60:6789/0 conn(0x7efea40cfb40 legacy=0x7efea4070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efe8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 0x7efe8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe8c004d40 0x7efe8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c8680 con 0x7efe8c0039b0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c78c0 con 0x7efe8c004d40 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00b9c0 con 0x7efea415bb50 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efe8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea9af0640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe8c004d40 0x7efe8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea8aee640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 0x7efe8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 -- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 msgr2=0x7efe8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea415bb50 0x7efe8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 -- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe8c004d40 msgr2=0x7efe8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe8c004d40 0x7efe8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.367+0000 7efea92ef640 1 -- 172.21.15.60:0/1147651852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c003520 con 0x7efe8c0039b0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.368+0000 7efea92ef640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efe8c003dd0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009fc0 tx=0x7efe8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43865d2bc6857cef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.368+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c009c10 con 0x7efe8c0039b0 2024-04-25T08:33:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.368+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c036070 con 0x7efe8c0039b0 2024-04-25T08:33:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.368+0000 7efea3fff640 1 -- 172.21.15.60:0/1147651852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c031880 con 0x7efe8c0039b0 2024-04-25T08:33:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efeab553640 1 -- 172.21.15.60:0/1147651852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 msgr2=0x7efe8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efeab553640 1 --2- 172.21.15.60:0/1147651852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efe8c003dd0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009fc0 tx=0x7efe8c005830 comp rx=0 tx=0).stop 2024-04-25T08:33:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efea9af0640 1 -- 172.21.15.60:0/1147651852 reap_dead start 2024-04-25T08:33:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efeab553640 1 -- 172.21.15.60:0/1147651852 shutdown_connections 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efeab553640 1 -- 172.21.15.60:0/1147651852 >> 172.21.15.60:0/1147651852 conn(0x7efea406f500 msgr2=0x7efea406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efeab553640 1 -- 172.21.15.60:0/1147651852 shutdown_connections 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.369+0000 7efeab553640 1 -- 172.21.15.60:0/1147651852 wait complete. 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.370+0000 7efeab553640 1 Processor -- start 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.370+0000 7efeab553640 1 -- start start 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efeab553640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efea41f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efeab553640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea414c5b0 0x7efea40cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efeab553640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea41f2ae0 0x7efea41f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efeab553640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea415f6d0 con 0x7efe8c0039b0 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efeab553640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c8160 con 0x7efea41f2ae0 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efeab553640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea415f850 con 0x7efea414c5b0 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efea41f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efea41f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40388/0 (socket says 172.21.15.60:40388) 2024-04-25T08:33:54.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 -- 172.21.15.60:0/3955951283 learned_addr learned my addr 172.21.15.60:0/3955951283 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea9af0640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea41f2ae0 0x7efea41f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea8aee640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea414c5b0 0x7efea40cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 -- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea414c5b0 msgr2=0x7efea40cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea414c5b0 0x7efea40cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 -- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea41f2ae0 msgr2=0x7efea41f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea41f2ae0 0x7efea41f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.371+0000 7efea92ef640 1 -- 172.21.15.60:0/3955951283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea41f54f0 con 0x7efe8c0039b0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.372+0000 7efea92ef640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efea41f4f10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efe9c02aee0 tx=0x7efe9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.372+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c009c10 con 0x7efe8c0039b0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.372+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c031e50 con 0x7efe8c0039b0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.372+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea41f57d0 con 0x7efe8c0039b0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.372+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c003bd0 con 0x7efe8c0039b0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.372+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea41f5d90 con 0x7efe8c0039b0 2024-04-25T08:33:54.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.373+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea41f1b20 con 0x7efe8c0039b0 2024-04-25T08:33:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.373+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe9c005d10 con 0x7efe8c0039b0 2024-04-25T08:33:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.374+0000 7efea1ffb640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe9006b500 0x7efe9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.374+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efe9c0ab460 con 0x7efe8c0039b0 2024-04-25T08:33:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.374+0000 7efea8aee640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe9006b500 0x7efe9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.375+0000 7efea8aee640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe9006b500 0x7efe9006d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efea41f4e70 tx=0x7efe98009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.378+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe9c07a5c0 con 0x7efe8c0039b0 2024-04-25T08:33:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.558+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efea40cf150 con 0x7efe8c0039b0 2024-04-25T08:33:54.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.559+0000 7efea1ffb640 1 -- 172.21.15.60:0/3955951283 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe9c079ec0 con 0x7efe8c0039b0 2024-04-25T08:33:54.554 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:33:54.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.560+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe9006b500 msgr2=0x7efe9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.560+0000 7efeab553640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe9006b500 0x7efe9006d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efea41f4e70 tx=0x7efe98009190 comp rx=0 tx=0).stop 2024-04-25T08:33:54.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.560+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 msgr2=0x7efea41f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.560+0000 7efeab553640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efea41f4f10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7efe9c02aee0 tx=0x7efe9c009750 comp rx=0 tx=0).stop 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 shutdown_connections 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe9006b500 0x7efe9006d9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efea41f2ae0 0x7efea41f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efea414c5b0 0x7efea40cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 --2- 172.21.15.60:0/3955951283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe8c0039b0 0x7efea41f4f10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 >> 172.21.15.60:0/3955951283 conn(0x7efea406f500 msgr2=0x7efea4070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 shutdown_connections 2024-04-25T08:33:54.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.561+0000 7efeab553640 1 -- 172.21.15.60:0/3955951283 wait complete. 2024-04-25T08:33:54.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:33:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.673+0000 7f514213b640 1 Processor -- start 2024-04-25T08:33:54.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.673+0000 7f514213b640 1 -- start start 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f514213b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c0b2cb0 0x7f513c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f514213b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f513c0d1020 0x7f513c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f514213b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0b97a0 con 0x7f513c15dbe0 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f514213b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513c15dfc0 con 0x7f513c0d1020 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f514213b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0d1a00 con 0x7f513c0b2cb0 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f513c0d1020 0x7f513c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f513c0d1020 0x7f513c0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40392/0 (socket says 172.21.15.60:40392) 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 -- 172.21.15.60:0/1796763332 learned_addr learned my addr 172.21.15.60:0/1796763332 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513affd640 1 --2- 172.21.15.60:0/1796763332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c0b2cb0 0x7f513c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 -- 172.21.15.60:0/1796763332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c0b2cb0 msgr2=0x7f513c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 --2- 172.21.15.60:0/1796763332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c0b2cb0 0x7f513c0d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 -- 172.21.15.60:0/1796763332 >> v1:172.21.15.100:6789/0 conn(0x7f513c15dbe0 legacy=0x7f513c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.674+0000 7f513bfff640 1 -- 172.21.15.60:0/1796763332 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0d1c50 con 0x7f513c0d1020 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.675+0000 7f513bfff640 1 --2- 172.21.15.60:0/1796763332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c0d2c70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5134009e50 tx=0x7f513402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5effa981c9a5ed9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.675+0000 7f513a7fc640 1 -- 172.21.15.60:0/1796763332 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134036070 con 0x7f513c0d1020 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.675+0000 7f513a7fc640 1 -- 172.21.15.60:0/1796763332 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134031600 con 0x7f513c0d1020 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.675+0000 7f513a7fc640 1 -- 172.21.15.60:0/1796763332 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51340319d0 con 0x7f513c0d1020 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 -- 172.21.15.60:0/1796763332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 msgr2=0x7f513c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 --2- 172.21.15.60:0/1796763332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c0d2c70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5134009e50 tx=0x7f513402a6c0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 -- 172.21.15.60:0/1796763332 shutdown_connections 2024-04-25T08:33:54.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 --2- 172.21.15.60:0/1796763332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c0d2c70 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 --2- 172.21.15.60:0/1796763332 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c0b2cb0 0x7f513c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 -- 172.21.15.60:0/1796763332 >> 172.21.15.60:0/1796763332 conn(0x7f513c061a20 msgr2=0x7f513c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 -- 172.21.15.60:0/1796763332 shutdown_connections 2024-04-25T08:33:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.676+0000 7f514213b640 1 -- 172.21.15.60:0/1796763332 wait complete. 2024-04-25T08:33:54.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.677+0000 7f514213b640 1 Processor -- start 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.677+0000 7f514213b640 1 -- start start 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f514213b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f513c0b2cb0 0x7f513c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f514213b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f514213b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c15dbe0 0x7f513c1f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f514213b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b92a0 con 0x7f513c0d1020 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f514213b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0d2210 con 0x7f513c0b2cb0 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f514213b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0b9420 con 0x7f513c15dbe0 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40404/0 (socket says 172.21.15.60:40404) 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 -- 172.21.15.60:0/2351160030 learned_addr learned my addr 172.21.15.60:0/2351160030 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513b7fe640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f513c0b2cb0 0x7f513c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 -- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c15dbe0 msgr2=0x7f513c1f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513bfff640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c15dbe0 0x7f513c1f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c15dbe0 0x7f513c1f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 -- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f513c0b2cb0 msgr2=0x7f513c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f513c0b2cb0 0x7f513c0b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513affd640 1 -- 172.21.15.60:0/2351160030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0cbf90 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.678+0000 7f513b7fe640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f513c0b2cb0 0x7f513c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.679+0000 7f513affd640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c153bf0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f513000c7f0 tx=0x7f513000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.679+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130015070 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.679+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130004240 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.679+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130007620 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.679+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c0cb9d0 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.679+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c0b31c0 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.680+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c073990 con 0x7f513c0d1020 2024-04-25T08:33:54.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.680+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51300079d0 con 0x7f513c0d1020 2024-04-25T08:33:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.681+0000 7f5123fff640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f512806b500 0x7f512806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.681+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f513008a010 con 0x7f513c0d1020 2024-04-25T08:33:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.681+0000 7f513b7fe640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f512806b500 0x7f512806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.682+0000 7f513b7fe640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f512806b500 0x7f512806d9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f512c00a830 tx=0x7f512c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.685+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51300590e0 con 0x7f513c0d1020 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.870+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f513c15b790 con 0x7f513c0d1020 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.870+0000 7f5123fff640 1 -- 172.21.15.60:0/2351160030 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f51300589e0 con 0x7f513c0d1020 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:33:54.866 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:33:54.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:33:54.867 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:33:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f512806b500 msgr2=0x7f512806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f512806b500 0x7f512806d9c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f512c00a830 tx=0x7f512c008040 comp rx=0 tx=0).stop 2024-04-25T08:33:54.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 msgr2=0x7f513c153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c153bf0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f513000c7f0 tx=0x7f513000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 shutdown_connections 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f513c15dbe0 0x7f513c1f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f513c0d1020 0x7f513c153bf0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f512806b500 0x7f512806d9c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 --2- 172.21.15.60:0/2351160030 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f513c0b2cb0 0x7f513c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 >> 172.21.15.60:0/2351160030 conn(0x7f513c061a20 msgr2=0x7f513c0b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 shutdown_connections 2024-04-25T08:33:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.873+0000 7f514213b640 1 -- 172.21.15.60:0/2351160030 wait complete. 2024-04-25T08:33:54.879 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:33:54.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:33:54.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe392989640 1 Processor -- start 2024-04-25T08:33:54.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe392989640 1 -- start start 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe392989640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe38c15b5a0 0x7fe38c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe392989640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c0cf320 con 0x7fe38c15b980 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe392989640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c0cf040 con 0x7fe38c162450 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe392989640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38c161970 con 0x7fe38c15b5a0 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.989+0000 7fe38b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe38c15b980 0x7fe38c0ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35414/0 (socket says 172.21.15.60:35414) 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38b7fe640 1 -- 172.21.15.60:0/3672972019 learned_addr learned my addr 172.21.15.60:0/3672972019 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38bfff640 1 --2- 172.21.15.60:0/3672972019 >> v2:172.21.15.100:3300/0 conn(0x7fe38c15b5a0 0x7fe38c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721904231 0 0) 0x7fe380001290 con 0x7fe38c15b980 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe374003520 con 0x7fe38c15b980 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380753255 0 0) 0x7fe374003520 con 0x7fe38c15b980 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 >> v2:172.21.15.100:3300/0 conn(0x7fe38c15b5a0 msgr2=0x7fe38c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 --2- 172.21.15.60:0/3672972019 >> v2:172.21.15.100:3300/0 conn(0x7fe38c15b5a0 0x7fe38c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 >> v1:172.21.15.100:6789/0 conn(0x7fe38c162450 legacy=0x7fe38c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c161c50 con 0x7fe38c15b980 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe3800019c0 con 0x7fe38c15b980 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38bfff640 1 --2- 172.21.15.60:0/3672972019 >> v2:172.21.15.100:3300/0 conn(0x7fe38c15b5a0 0x7fe38c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 >> v1:172.21.15.60:6789/0 conn(0x7fe38c15b980 legacy=0x7fe38c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3740039b0 0x7fe374003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe374004800 0x7fe374004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3740051c0 0x7fe37400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c161970 con 0x7fe3740051c0 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37400c060 con 0x7fe3740039b0 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37400c260 con 0x7fe374004800 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe390f26640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3740051c0 0x7fe37400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38bfff640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3740039b0 0x7fe374003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.990+0000 7fe38b7fe640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe374004800 0x7fe374004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 -- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe374004800 msgr2=0x7fe374004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe374004800 0x7fe374004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 -- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3740039b0 msgr2=0x7fe374003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3740039b0 0x7fe374003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 -- 172.21.15.60:0/3672972019 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374003520 con 0x7fe3740051c0 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe38b7fe640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe374004800 0x7fe374004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 -- 172.21.15.60:0/3672972019 reap_dead start 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe390f26640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3740051c0 0x7fe37400b9c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe384001e50 tx=0x7fe374005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db4a367e5d6ca5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384001a70 con 0x7fe3740051c0 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe384003970 con 0x7fe3740051c0 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.991+0000 7fe38affd640 1 -- 172.21.15.60:0/3672972019 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384003d60 con 0x7fe3740051c0 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 -- 172.21.15.60:0/3672972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3740051c0 msgr2=0x7fe37400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3740051c0 0x7fe37400b9c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe384001e50 tx=0x7fe374005830 comp rx=0 tx=0).stop 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 -- 172.21.15.60:0/3672972019 shutdown_connections 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 --2- 172.21.15.60:0/3672972019 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3740051c0 0x7fe37400b9c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 -- 172.21.15.60:0/3672972019 >> 172.21.15.60:0/3672972019 conn(0x7fe38c0d0530 msgr2=0x7fe38c0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 -- 172.21.15.60:0/3672972019 shutdown_connections 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.992+0000 7fe392989640 1 -- 172.21.15.60:0/3672972019 wait complete. 2024-04-25T08:33:54.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.993+0000 7fe392989640 1 Processor -- start 2024-04-25T08:33:54.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.993+0000 7fe392989640 1 -- start start 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe392989640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3740051c0 0x7fe38c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe392989640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38c0c6cf0 0x7fe38c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe392989640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 0x7fe38c15f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe392989640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0b56b0 con 0x7fe38c0c7170 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe392989640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0b5530 con 0x7fe38c0c6cf0 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe392989640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c0b5830 con 0x7fe3740051c0 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe390f26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 0x7fe38c15f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe390f26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 0x7fe38c15f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40418/0 (socket says 172.21.15.60:40418) 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe390f26640 1 -- 172.21.15.60:0/3833245713 learned_addr learned my addr 172.21.15.60:0/3833245713 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe38b7fe640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38c0c6cf0 0x7fe38c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.994+0000 7fe38bfff640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3740051c0 0x7fe38c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe390f26640 1 -- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3740051c0 msgr2=0x7fe38c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe390f26640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3740051c0 0x7fe38c157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe390f26640 1 -- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38c0c6cf0 msgr2=0x7fe38c159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe390f26640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38c0c6cf0 0x7fe38c159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe390f26640 1 -- 172.21.15.60:0/3833245713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0cad20 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe390f26640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 0x7fe38c15f260 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe384001f70 tx=0x7fe3840014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384001a70 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe384006b50 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c157950 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384007030 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.995+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c0cb2a0 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.996+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c073a00 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.996+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3840071d0 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.997+0000 7fe388ff9640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe37806b4d0 0x7fe37806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.997+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe3840a9b20 con 0x7fe38c0c7170 2024-04-25T08:33:54.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.997+0000 7fe38bfff640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe37806b4d0 0x7fe37806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:54.997+0000 7fe38bfff640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe37806b4d0 0x7fe37806d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c10 tx=0x7fe37c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:54.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.001+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe384043060 con 0x7fe38c0c7170 2024-04-25T08:33:55.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.184+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe38c0d17c0 con 0x7fe38c0c7170 2024-04-25T08:33:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.556+0000 7fe388ff9640 1 -- 172.21.15.60:0/3833245713 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe384078ac0 con 0x7fe38c0c7170 2024-04-25T08:33:55.552 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.558+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe37806b4d0 msgr2=0x7fe37806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.558+0000 7fe392989640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe37806b4d0 0x7fe37806d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c10 tx=0x7fe37c008040 comp rx=0 tx=0).stop 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 msgr2=0x7fe38c15f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 0x7fe38c15f260 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe384001f70 tx=0x7fe3840014a0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 shutdown_connections 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe38c0c7170 0x7fe38c15f260 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe37806b4d0 0x7fe37806d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe38c0c6cf0 0x7fe38c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 --2- 172.21.15.60:0/3833245713 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe3740051c0 0x7fe38c157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 >> 172.21.15.60:0/3833245713 conn(0x7fe38c0d0530 msgr2=0x7fe38c15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:55.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 shutdown_connections 2024-04-25T08:33:55.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.559+0000 7fe392989640 1 -- 172.21.15.60:0/3833245713 wait complete. 2024-04-25T08:33:55.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:33:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 Processor -- start 2024-04-25T08:33:55.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 -- start start 2024-04-25T08:33:55.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040cadb0 0x7ff2040cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2040ccfe0 0x7ff2040cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2040afc80 con 0x7ff20415de80 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2040cb210 con 0x7ff2040ccfe0 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.683+0000 7ff20a26b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040ce4d0 con 0x7ff2040cadb0 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2040ccfe0 0x7ff2040cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2040ccfe0 0x7ff2040cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40424/0 (socket says 172.21.15.60:40424) 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 -- 172.21.15.60:0/2268292557 learned_addr learned my addr 172.21.15.60:0/2268292557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff2037fe640 1 --2- 172.21.15.60:0/2268292557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040cadb0 0x7ff2040cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 -- 172.21.15.60:0/2268292557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040cadb0 msgr2=0x7ff2040cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 --2- 172.21.15.60:0/2268292557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040cadb0 0x7ff2040cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 -- 172.21.15.60:0/2268292557 >> v1:172.21.15.60:6789/0 conn(0x7ff20415de80 legacy=0x7ff2040cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.684+0000 7ff208808640 1 -- 172.21.15.60:0/2268292557 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040ce7b0 con 0x7ff2040ccfe0 2024-04-25T08:33:55.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff208808640 1 --2- 172.21.15.60:0/2268292557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040ccfe0 0x7ff2040cd470 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e50 tx=0x7ff1fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fd86da654c86ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff202ffd640 1 -- 172.21.15.60:0/2268292557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc036070 con 0x7ff2040ccfe0 2024-04-25T08:33:55.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff202ffd640 1 -- 172.21.15.60:0/2268292557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc031600 con 0x7ff2040ccfe0 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff202ffd640 1 -- 172.21.15.60:0/2268292557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0319d0 con 0x7ff2040ccfe0 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 -- 172.21.15.60:0/2268292557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040ccfe0 msgr2=0x7ff2040cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 --2- 172.21.15.60:0/2268292557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040ccfe0 0x7ff2040cd470 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e50 tx=0x7ff1fc02a710 comp rx=0 tx=0).stop 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 -- 172.21.15.60:0/2268292557 shutdown_connections 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 --2- 172.21.15.60:0/2268292557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040ccfe0 0x7ff2040cd470 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 --2- 172.21.15.60:0/2268292557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040cadb0 0x7ff2040cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 -- 172.21.15.60:0/2268292557 >> 172.21.15.60:0/2268292557 conn(0x7ff2040b2390 msgr2=0x7ff2040b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 -- 172.21.15.60:0/2268292557 shutdown_connections 2024-04-25T08:33:55.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.685+0000 7ff20a26b640 1 -- 172.21.15.60:0/2268292557 wait complete. 2024-04-25T08:33:55.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.686+0000 7ff20a26b640 1 Processor -- start 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 -- start start 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 0x7ff2041577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040ccfe0 0x7ff2040b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20415de80 0x7ff204158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20415e260 con 0x7ff2040cadb0 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040cfc00 con 0x7ff20415de80 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.687+0000 7ff20a26b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040b2eb0 con 0x7ff2040ccfe0 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 0x7ff2041577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 0x7ff2041577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40432/0 (socket says 172.21.15.60:40432) 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 -- 172.21.15.60:0/3980082044 learned_addr learned my addr 172.21.15.60:0/3980082044 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff2037fe640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040ccfe0 0x7ff2040b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff208808640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20415de80 0x7ff204158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 -- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040ccfe0 msgr2=0x7ff2040b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040ccfe0 0x7ff2040b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 -- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20415de80 msgr2=0x7ff204158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20415de80 0x7ff204158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 -- 172.21.15.60:0/3980082044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040b3ed0 con 0x7ff2040cadb0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff2037fe640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040ccfe0 0x7ff2040b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff208808640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20415de80 0x7ff204158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff203fff640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 0x7ff2041577c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400c3c0 tx=0x7ff1f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40052d0 con 0x7ff2040cadb0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f4005490 con 0x7ff2040cadb0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2040b41b0 con 0x7ff2040cadb0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4007260 con 0x7ff2040cadb0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.688+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2040c94b0 con 0x7ff2040cadb0 2024-04-25T08:33:55.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.689+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff204073a00 con 0x7ff2040cadb0 2024-04-25T08:33:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.690+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1f4007400 con 0x7ff2040cadb0 2024-04-25T08:33:55.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.690+0000 7ff200ff9640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff1f006b4d0 0x7ff1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.691+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff1f4089630 con 0x7ff2040cadb0 2024-04-25T08:33:55.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.691+0000 7ff2037fe640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff1f006b4d0 0x7ff1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.691+0000 7ff2037fe640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff1f006b4d0 0x7ff1f006d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b45e0 tx=0x7ff1f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.695+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1f4058590 con 0x7ff2040cadb0 2024-04-25T08:33:55.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:55.878+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff2040b2c10 con 0x7ff2040cadb0 2024-04-25T08:33:56.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:56.522 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:56.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.558+0000 7ff200ff9640 1 -- 172.21.15.60:0/3980082044 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff1f4057e90 con 0x7ff2040cadb0 2024-04-25T08:33:56.554 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff1f006b4d0 msgr2=0x7ff1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff1f006b4d0 0x7ff1f006d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b45e0 tx=0x7ff1f8008040 comp rx=0 tx=0).stop 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 msgr2=0x7ff2041577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 0x7ff2041577c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400c3c0 tx=0x7ff1f400c880 comp rx=0 tx=0).stop 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 shutdown_connections 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff1f006b4d0 0x7ff1f006d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff20415de80 0x7ff204158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2040ccfe0 0x7ff2040b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 --2- 172.21.15.60:0/3980082044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2040cadb0 0x7ff2041577c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 >> 172.21.15.60:0/3980082044 conn(0x7ff2040b2390 msgr2=0x7ff2040cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 shutdown_connections 2024-04-25T08:33:56.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.561+0000 7ff20a26b640 1 -- 172.21.15.60:0/3980082044 wait complete. 2024-04-25T08:33:56.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:33:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.731+0000 7f4c395e0640 1 Processor -- start 2024-04-25T08:33:56.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.731+0000 7f4c395e0640 1 -- start start 2024-04-25T08:33:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.732+0000 7f4c395e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4c341610f0 0x7f4c3406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.732+0000 7f4c395e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341616f0 0x7f4c34161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.732+0000 7f4c395e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c340c6960 con 0x7f4c340ce0a0 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.732+0000 7f4c395e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c341614d0 con 0x7f4c341610f0 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.732+0000 7f4c395e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34162260 con 0x7f4c341616f0 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c327fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341616f0 0x7f4c34161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c32ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4c341610f0 0x7f4c3406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c337fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4c340ce0a0 0x7f4c340d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:44102/0 (socket says 172.21.15.60:44102) 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c337fe640 1 -- 172.21.15.60:0/4135816527 learned_addr learned my addr 172.21.15.60:0/4135816527 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c327fc640 1 -- 172.21.15.60:0/4135816527 >> v2:172.21.15.100:3300/0 conn(0x7f4c341610f0 msgr2=0x7f4c3406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c327fc640 1 --2- 172.21.15.60:0/4135816527 >> v2:172.21.15.100:3300/0 conn(0x7f4c341610f0 0x7f4c3406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c327fc640 1 -- 172.21.15.60:0/4135816527 >> v1:172.21.15.100:6789/0 conn(0x7f4c340ce0a0 legacy=0x7f4c340d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c327fc640 1 -- 172.21.15.60:0/4135816527 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34162540 con 0x7f4c341616f0 2024-04-25T08:33:56.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.733+0000 7f4c32ffd640 1 --2- 172.21.15.60:0/4135816527 >> v2:172.21.15.100:3300/0 conn(0x7f4c341610f0 0x7f4c3406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.734+0000 7f4c327fc640 1 --2- 172.21.15.60:0/4135816527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341616f0 0x7f4c34161b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0097a0 tx=0x7f4c1c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=446b5126b18d7568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.734+0000 7f4c31ffb640 1 -- 172.21.15.60:0/4135816527 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c037070 con 0x7f4c341616f0 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.734+0000 7f4c31ffb640 1 -- 172.21.15.60:0/4135816527 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c02ad10 con 0x7f4c341616f0 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.734+0000 7f4c31ffb640 1 -- 172.21.15.60:0/4135816527 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c032660 con 0x7f4c341616f0 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 -- 172.21.15.60:0/4135816527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341616f0 msgr2=0x7f4c34161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 --2- 172.21.15.60:0/4135816527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341616f0 0x7f4c34161b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c0097a0 tx=0x7f4c1c02a080 comp rx=0 tx=0).stop 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 -- 172.21.15.60:0/4135816527 shutdown_connections 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 --2- 172.21.15.60:0/4135816527 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341616f0 0x7f4c34161b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 --2- 172.21.15.60:0/4135816527 >> v2:172.21.15.100:3300/0 conn(0x7f4c341610f0 0x7f4c3406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 -- 172.21.15.60:0/4135816527 >> 172.21.15.60:0/4135816527 conn(0x7f4c3406f500 msgr2=0x7f4c3406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 -- 172.21.15.60:0/4135816527 shutdown_connections 2024-04-25T08:33:56.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.735+0000 7f4c395e0640 1 -- 172.21.15.60:0/4135816527 wait complete. 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.736+0000 7f4c395e0640 1 Processor -- start 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.736+0000 7f4c395e0640 1 -- start start 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c395e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 0x7f4c341533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c395e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341610f0 0x7f4c340b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c395e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c341616f0 0x7f4c340b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c395e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340cf690 con 0x7f4c340ce0a0 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c395e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340cfce0 con 0x7f4c341616f0 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c395e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3406c700 con 0x7f4c341610f0 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c32ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 0x7f4c341533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c32ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 0x7f4c341533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40442/0 (socket says 172.21.15.60:40442) 2024-04-25T08:33:56.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c32ffd640 1 -- 172.21.15.60:0/3874085224 learned_addr learned my addr 172.21.15.60:0/3874085224 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c337fe640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c341616f0 0x7f4c340b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c327fc640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341610f0 0x7f4c340b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c32ffd640 1 -- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341610f0 msgr2=0x7f4c340b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.737+0000 7f4c32ffd640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341610f0 0x7f4c340b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c32ffd640 1 -- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c341616f0 msgr2=0x7f4c340b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c32ffd640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c341616f0 0x7f4c340b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c32ffd640 1 -- 172.21.15.60:0/3874085224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3415f570 con 0x7f4c340ce0a0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c32ffd640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 0x7f4c341533d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00ec10 tx=0x7f4c2c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c017070 con 0x7f4c340ce0a0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c00cbd0 con 0x7f4c340ce0a0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3415f850 con 0x7f4c340ce0a0 2024-04-25T08:33:56.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c013470 con 0x7f4c340ce0a0 2024-04-25T08:33:56.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.738+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3415fe10 con 0x7f4c340ce0a0 2024-04-25T08:33:56.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.739+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34073a00 con 0x7f4c340ce0a0 2024-04-25T08:33:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.740+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c2c01a070 con 0x7f4c340ce0a0 2024-04-25T08:33:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.740+0000 7f4c2b7fe640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1406b4d0 0x7f4c1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.740+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f4c2c08a5f0 con 0x7f4c340ce0a0 2024-04-25T08:33:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.740+0000 7f4c327fc640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1406b4d0 0x7f4c1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.741+0000 7f4c327fc640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1406b4d0 0x7f4c1406d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c034000 tx=0x7f4c1c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.745+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c2c058c70 con 0x7f4c340ce0a0 2024-04-25T08:33:56.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:56.965+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c340cf150 con 0x7f4c340ce0a0 2024-04-25T08:33:57.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.570+0000 7f4c2b7fe640 1 -- 172.21.15.60:0/3874085224 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4c2c05cd10 con 0x7f4c340ce0a0 2024-04-25T08:33:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T08:33:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1406b4d0 msgr2=0x7f4c1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1406b4d0 0x7f4c1406d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c034000 tx=0x7f4c1c02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 msgr2=0x7f4c341533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 0x7f4c341533d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00ec10 tx=0x7f4c2c00c520 comp rx=0 tx=0).stop 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 shutdown_connections 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c1406b4d0 0x7f4c1406d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c341616f0 0x7f4c340b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c341610f0 0x7f4c340b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 --2- 172.21.15.60:0/3874085224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c340ce0a0 0x7f4c341533d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.575+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 >> 172.21.15.60:0/3874085224 conn(0x7f4c3406f500 msgr2=0x7f4c34071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:57.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.576+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 shutdown_connections 2024-04-25T08:33:57.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.577+0000 7f4c395e0640 1 -- 172.21.15.60:0/3874085224 wait complete. 2024-04-25T08:33:57.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:57.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.717+0000 7f7121a4a640 1 Processor -- start 2024-04-25T08:33:57.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.717+0000 7f7121a4a640 1 -- start start 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f7121a4a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f711c15dd20 0x7f711c0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f7121a4a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f711c15e100 0x7f711c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f7121a4a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c15eac0 con 0x7f711c15e590 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f7121a4a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c0d1c00 con 0x7f711c15dd20 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f711c15dd20 0x7f711c0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f7121a4a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c15eda0 con 0x7f711c15e100 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f711c15dd20 0x7f711c0b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40454/0 (socket says 172.21.15.60:40454) 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 -- 172.21.15.60:0/263106029 learned_addr learned my addr 172.21.15.60:0/263106029 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:57.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f7113fff640 1 --2- 172.21.15.60:0/263106029 >> v2:172.21.15.100:3300/0 conn(0x7f711c15e100 0x7f711c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:57.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 -- 172.21.15.60:0/263106029 >> v2:172.21.15.100:3300/0 conn(0x7f711c15e100 msgr2=0x7f711c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:57.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 --2- 172.21.15.60:0/263106029 >> v2:172.21.15.100:3300/0 conn(0x7f711c15e100 0x7f711c0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 -- 172.21.15.60:0/263106029 >> v1:172.21.15.100:6789/0 conn(0x7f711c15e590 legacy=0x7f711c0d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.718+0000 7f711affd640 1 -- 172.21.15.60:0/263106029 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c15f080 con 0x7f711c15dd20 2024-04-25T08:33:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.719+0000 7f711affd640 1 --2- 172.21.15.60:0/263106029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c0b96d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b5d20 tx=0x7f710c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bcdc80ef6ec9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.719+0000 7f711a7fc640 1 -- 172.21.15.60:0/263106029 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c037070 con 0x7f711c15dd20 2024-04-25T08:33:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.719+0000 7f711a7fc640 1 -- 172.21.15.60:0/263106029 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c032440 con 0x7f711c15dd20 2024-04-25T08:33:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.719+0000 7f711a7fc640 1 -- 172.21.15.60:0/263106029 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c032810 con 0x7f711c15dd20 2024-04-25T08:33:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 -- 172.21.15.60:0/263106029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 msgr2=0x7f711c0b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 --2- 172.21.15.60:0/263106029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c0b96d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b5d20 tx=0x7f710c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 -- 172.21.15.60:0/263106029 shutdown_connections 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 --2- 172.21.15.60:0/263106029 >> v2:172.21.15.100:3300/0 conn(0x7f711c15e100 0x7f711c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 --2- 172.21.15.60:0/263106029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c0b96d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 -- 172.21.15.60:0/263106029 >> 172.21.15.60:0/263106029 conn(0x7f711c061a20 msgr2=0x7f711c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 -- 172.21.15.60:0/263106029 shutdown_connections 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.720+0000 7f7121a4a640 1 -- 172.21.15.60:0/263106029 wait complete. 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.721+0000 7f7121a4a640 1 Processor -- start 2024-04-25T08:33:57.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.721+0000 7f7121a4a640 1 -- start start 2024-04-25T08:33:57.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7121a4a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:57.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7121a4a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f711c15e100 0x7f711c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:57.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7121a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711c15e590 0x7f711c0d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:57.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7121a4a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c0d3500 con 0x7f711c15dd20 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7121a4a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c0d3380 con 0x7f711c15e590 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7121a4a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c0d3680 con 0x7f711c15e100 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c1f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40458/0 (socket says 172.21.15.60:40458) 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 -- 172.21.15.60:0/29104550 learned_addr learned my addr 172.21.15.60:0/29104550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f7113fff640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f711c15e100 0x7f711c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711b7fe640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711c15e590 0x7f711c0d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 -- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f711c15e100 msgr2=0x7f711c14de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f711c15e100 0x7f711c14de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 -- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711c15e590 msgr2=0x7f711c0d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711c15e590 0x7f711c0d3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 -- 172.21.15.60:0/29104550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0cbbd0 con 0x7f711c15dd20 2024-04-25T08:33:57.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.722+0000 7f711affd640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c1f4840 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f710c000c00 tx=0x7f710c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.723+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c041020 con 0x7f711c15dd20 2024-04-25T08:33:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.723+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c0306c0 con 0x7f711c15dd20 2024-04-25T08:33:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.723+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c0cc190 con 0x7f711c15dd20 2024-04-25T08:33:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.723+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c037040 con 0x7f711c15dd20 2024-04-25T08:33:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.723+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c0cc750 con 0x7f711c15dd20 2024-04-25T08:33:57.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.724+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c073990 con 0x7f711c15dd20 2024-04-25T08:33:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.724+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f710c044020 con 0x7f711c15dd20 2024-04-25T08:33:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.725+0000 7f7118ff9640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.725+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f710c0aaa40 con 0x7f711c15dd20 2024-04-25T08:33:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.725+0000 7f7113fff640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.725+0000 7f7113fff640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f711c1f1270 tx=0x7f7104008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:57.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.729+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f710c04b030 con 0x7f711c15dd20 2024-04-25T08:33:57.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.914+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f711c15d220 con 0x7f711c15dd20 2024-04-25T08:33:57.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.916+0000 7f7118ff9640 1 -- 172.21.15.60:0/29104550 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16946 (secure 0 0 0) 0x7f710c0aae20 con 0x7f711c15dd20 2024-04-25T08:33:57.911 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:57.911 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:56.967038+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70fc06b4d0 msgr2=0x7f70fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f711c1f1270 tx=0x7f7104008040 comp rx=0 tx=0).stop 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 msgr2=0x7f711c1f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c1f4840 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f710c000c00 tx=0x7f710c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 shutdown_connections 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f711c15e590 0x7f711c0d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f711c15e100 0x7f711c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 --2- 172.21.15.60:0/29104550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f711c15dd20 0x7f711c1f4840 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 >> 172.21.15.60:0/29104550 conn(0x7f711c061a20 msgr2=0x7f711c15fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 shutdown_connections 2024-04-25T08:33:57.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:57.918+0000 7f7121a4a640 1 -- 172.21.15.60:0/29104550 wait complete. 2024-04-25T08:33:57.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:33:58.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.029+0000 7f5bee838640 1 Processor -- start 2024-04-25T08:33:58.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.030+0000 7f5bee838640 1 -- start start 2024-04-25T08:33:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.030+0000 7f5bee838640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5be80d0b10 0x7f5be80ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.030+0000 7f5bee838640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 0x7f5be80d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.030+0000 7f5bee838640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be80cec00 con 0x7f5be815db60 2024-04-25T08:33:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.030+0000 7f5bee838640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be80ce930 con 0x7f5be80d0b10 2024-04-25T08:33:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.030+0000 7f5bee838640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80b6450 con 0x7f5be80b5f00 2024-04-25T08:33:58.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 0x7f5be80d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5be77fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5be80d0b10 0x7f5be80ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 0x7f5be80d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:40388/0 (socket says 172.21.15.60:40388) 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 -- 172.21.15.60:0/3688275344 learned_addr learned my addr 172.21.15.60:0/3688275344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 -- 172.21.15.60:0/3688275344 >> v2:172.21.15.100:3300/0 conn(0x7f5be80d0b10 msgr2=0x7f5be80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 --2- 172.21.15.60:0/3688275344 >> v2:172.21.15.100:3300/0 conn(0x7f5be80d0b10 0x7f5be80ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 -- 172.21.15.60:0/3688275344 >> v1:172.21.15.100:6789/0 conn(0x7f5be815db60 legacy=0x7f5be80d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5becdd5640 1 -- 172.21.15.60:0/3688275344 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80ceee0 con 0x7f5be80b5f00 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.031+0000 7f5be77fe640 1 --2- 172.21.15.60:0/3688275344 >> v2:172.21.15.100:3300/0 conn(0x7f5be80d0b10 0x7f5be80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.032+0000 7f5becdd5640 1 --2- 172.21.15.60:0/3688275344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 0x7f5be80d2270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009e80 tx=0x7f5be002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb715cbbd157924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.032+0000 7f5be6ffd640 1 -- 172.21.15.60:0/3688275344 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0036070 con 0x7f5be80b5f00 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.032+0000 7f5be6ffd640 1 -- 172.21.15.60:0/3688275344 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be0031600 con 0x7f5be80b5f00 2024-04-25T08:33:58.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.032+0000 7f5be6ffd640 1 -- 172.21.15.60:0/3688275344 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be00319d0 con 0x7f5be80b5f00 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 -- 172.21.15.60:0/3688275344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 msgr2=0x7f5be80d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 --2- 172.21.15.60:0/3688275344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 0x7f5be80d2270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009e80 tx=0x7f5be002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 -- 172.21.15.60:0/3688275344 shutdown_connections 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 --2- 172.21.15.60:0/3688275344 >> v2:172.21.15.100:3300/0 conn(0x7f5be80d0b10 0x7f5be80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 --2- 172.21.15.60:0/3688275344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80b5f00 0x7f5be80d2270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 -- 172.21.15.60:0/3688275344 >> 172.21.15.60:0/3688275344 conn(0x7f5be8061600 msgr2=0x7f5be8061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 -- 172.21.15.60:0/3688275344 shutdown_connections 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.033+0000 7f5bee838640 1 -- 172.21.15.60:0/3688275344 wait complete. 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.034+0000 7f5bee838640 1 Processor -- start 2024-04-25T08:33:58.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.034+0000 7f5bee838640 1 -- start start 2024-04-25T08:33:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5bee838640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80b5f00 0x7f5be80cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5bee838640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80d0b10 0x7f5be80cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5bee838640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 0x7f5be80c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5bee838640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815df40 con 0x7f5be815db60 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5bee838640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80d1d50 con 0x7f5be80b5f00 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5bee838640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be815e0c0 con 0x7f5be80d0b10 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 0x7f5be80c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 0x7f5be80c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40466/0 (socket says 172.21.15.60:40466) 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 -- 172.21.15.60:0/195558008 learned_addr learned my addr 172.21.15.60:0/195558008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5be7fff640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80b5f00 0x7f5be80cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5be77fe640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80d0b10 0x7f5be80cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 -- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80d0b10 msgr2=0x7f5be80cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80d0b10 0x7f5be80cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 -- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80b5f00 msgr2=0x7f5be80cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80b5f00 0x7f5be80cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.035+0000 7f5becdd5640 1 -- 172.21.15.60:0/195558008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80c94e0 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.036+0000 7f5becdd5640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 0x7f5be80c8fa0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009fa0 tx=0x7f5be0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.036+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0036050 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.036+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be0042070 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.036+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be80c9a60 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.036+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0009510 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.036+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be815f670 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.037+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be8073270 con 0x7f5be815db60 2024-04-25T08:33:58.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.037+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5be0007c50 con 0x7f5be815db60 2024-04-25T08:33:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.038+0000 7f5be4ff9640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5bd406b4d0 0x7f5bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.038+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5be00ab060 con 0x7f5be815db60 2024-04-25T08:33:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.038+0000 7f5be7fff640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5bd406b4d0 0x7f5bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.038+0000 7f5be7fff640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5bd406b4d0 0x7f5bd406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8001780 tx=0x7f5bd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.042+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5be0079710 con 0x7f5be815db60 2024-04-25T08:33:58.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.228+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5be80d1860 con 0x7f5be815db60 2024-04-25T08:33:58.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.229+0000 7f5be4ff9640 1 -- 172.21.15.60:0/195558008 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16946 (secure 0 0 0) 0x7f5be007d7b0 con 0x7f5be815db60 2024-04-25T08:33:58.225 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:58.225 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:56.967038+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5bd406b4d0 msgr2=0x7f5bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5bd406b4d0 0x7f5bd406d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8001780 tx=0x7f5bd8009040 comp rx=0 tx=0).stop 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 msgr2=0x7f5be80c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 0x7f5be80c8fa0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009fa0 tx=0x7f5be0004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 shutdown_connections 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5be815db60 0x7f5be80c8fa0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5be80d0b10 0x7f5be80cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5bd406b4d0 0x7f5bd406d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 --2- 172.21.15.60:0/195558008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5be80b5f00 0x7f5be80cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 >> 172.21.15.60:0/195558008 conn(0x7f5be8061600 msgr2=0x7f5be80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:58.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 shutdown_connections 2024-04-25T08:33:58.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.231+0000 7f5bee838640 1 -- 172.21.15.60:0/195558008 wait complete. 2024-04-25T08:33:58.238 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:33:58.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.346+0000 7f09293a9640 1 Processor -- start 2024-04-25T08:33:58.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.346+0000 7f09293a9640 1 -- start start 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f09293a9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09241610f0 0x7f092406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f09293a9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f09241616f0 0x7f0924161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f09293a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09240ce0a0 0x7f09240ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f09293a9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09241614d0 con 0x7f09241610f0 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f09293a9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09240c6960 con 0x7f09241616f0 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f09293a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0924162260 con 0x7f09240ce0a0 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f0922ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09241610f0 0x7f092406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f0922ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f09241610f0 0x7f092406c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40470/0 (socket says 172.21.15.60:40470) 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.347+0000 7f0922ffd640 1 -- 172.21.15.60:0/1785606373 learned_addr learned my addr 172.21.15.60:0/1785606373 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:58.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f09227fc640 1 --2- 172.21.15.60:0/1785606373 >> v2:172.21.15.100:3300/0 conn(0x7f09241616f0 0x7f0924161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0922ffd640 1 -- 172.21.15.60:0/1785606373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09240ce0a0 msgr2=0x7f09240ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f09237fe640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09240ce0a0 0x7f09240ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0922ffd640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09240ce0a0 0x7f09240ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0922ffd640 1 -- 172.21.15.60:0/1785606373 >> v2:172.21.15.100:3300/0 conn(0x7f09241616f0 msgr2=0x7f0924161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0922ffd640 1 --2- 172.21.15.60:0/1785606373 >> v2:172.21.15.100:3300/0 conn(0x7f09241616f0 0x7f0924161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0922ffd640 1 -- 172.21.15.60:0/1785606373 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0924162540 con 0x7f09241610f0 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f09227fc640 1 --2- 172.21.15.60:0/1785606373 >> v2:172.21.15.100:3300/0 conn(0x7f09241616f0 0x7f0924161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f09237fe640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09240ce0a0 0x7f09240ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0922ffd640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f092406c1c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f092406f220 tx=0x7f091c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e598ce93928b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0921ffb640 1 -- 172.21.15.60:0/1785606373 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c037070 con 0x7f09241610f0 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0921ffb640 1 -- 172.21.15.60:0/1785606373 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c032440 con 0x7f09241610f0 2024-04-25T08:33:58.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.348+0000 7f0921ffb640 1 -- 172.21.15.60:0/1785606373 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c032810 con 0x7f09241610f0 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 -- 172.21.15.60:0/1785606373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 msgr2=0x7f092406c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f092406c1c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f092406f220 tx=0x7f091c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 -- 172.21.15.60:0/1785606373 shutdown_connections 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09240ce0a0 0x7f09240ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 --2- 172.21.15.60:0/1785606373 >> v2:172.21.15.100:3300/0 conn(0x7f09241616f0 0x7f0924161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 --2- 172.21.15.60:0/1785606373 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f092406c1c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 -- 172.21.15.60:0/1785606373 >> 172.21.15.60:0/1785606373 conn(0x7f092406f500 msgr2=0x7f092406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 -- 172.21.15.60:0/1785606373 shutdown_connections 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.349+0000 7f09293a9640 1 -- 172.21.15.60:0/1785606373 wait complete. 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.350+0000 7f09293a9640 1 Processor -- start 2024-04-25T08:33:58.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.350+0000 7f09293a9640 1 -- start start 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09293a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09240ce0a0 0x7f09240ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09293a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f0924155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09293a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09241616f0 0x7f09240b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09293a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cf690 con 0x7f09241610f0 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09293a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cfce0 con 0x7f09240ce0a0 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09293a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092406c700 con 0x7f09241616f0 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09227fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f0924155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09227fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f0924155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40478/0 (socket says 172.21.15.60:40478) 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09227fc640 1 -- 172.21.15.60:0/3973857666 learned_addr learned my addr 172.21.15.60:0/3973857666 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f0922ffd640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09240ce0a0 0x7f09240ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.351+0000 7f09237fe640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09241616f0 0x7f09240b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09227fc640 1 -- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09241616f0 msgr2=0x7f09240b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09227fc640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09241616f0 0x7f09240b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09227fc640 1 -- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09240ce0a0 msgr2=0x7f09240ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09227fc640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09240ce0a0 0x7f09240ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09227fc640 1 -- 172.21.15.60:0/3973857666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092415ff80 con 0x7f09241610f0 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09227fc640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f0924155480 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f091800c820 tx=0x7f091800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918015070 con 0x7f09241610f0 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918007a30 con 0x7f09241610f0 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092415f370 con 0x7f09241610f0 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918007de0 con 0x7f09241610f0 2024-04-25T08:33:58.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.352+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092415f930 con 0x7f09241610f0 2024-04-25T08:33:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.353+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09240cf150 con 0x7f09241610f0 2024-04-25T08:33:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.354+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0918018070 con 0x7f09241610f0 2024-04-25T08:33:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.354+0000 7f090b7fe640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f091006b4d0 0x7f091006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.354+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f091808a100 con 0x7f09241610f0 2024-04-25T08:33:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.354+0000 7f0922ffd640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f091006b4d0 0x7f091006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.355+0000 7f0922ffd640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f091006b4d0 0x7f091006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f091c034000 tx=0x7f091c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.359+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0918058830 con 0x7f09241610f0 2024-04-25T08:33:58.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.575+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f09240739e0 con 0x7f09241610f0 2024-04-25T08:33:58.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.584+0000 7f090b7fe640 1 -- 172.21.15.60:0/3973857666 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f091805c8d0 con 0x7f09241610f0 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f091006b4d0 msgr2=0x7f091006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f091006b4d0 0x7f091006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f091c034000 tx=0x7f091c02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 msgr2=0x7f0924155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f0924155480 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f091800c820 tx=0x7f091800cce0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 shutdown_connections 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09241616f0 0x7f09240b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09241610f0 0x7f0924155480 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f091006b4d0 0x7f091006d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 --2- 172.21.15.60:0/3973857666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09240ce0a0 0x7f09240ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 >> 172.21.15.60:0/3973857666 conn(0x7f092406f500 msgr2=0x7f09241603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 shutdown_connections 2024-04-25T08:33:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.586+0000 7f09293a9640 1 -- 172.21.15.60:0/3973857666 wait complete. 2024-04-25T08:33:58.593 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:33:58.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:33:58.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.701+0000 7f6544db0640 1 Processor -- start 2024-04-25T08:33:58.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.701+0000 7f6544db0640 1 -- start start 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f6544db0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f654015dc20 0x7f65400ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f6544db0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6540161710 0x7f65400cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f6544db0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f65400b9330 0x7f654006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f6544db0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654015e000 con 0x7f6540161710 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f6544db0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65400d1de0 con 0x7f65400b9330 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f6544db0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6540161c60 con 0x7f654015dc20 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f653dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6540161710 0x7f65400cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f653dd9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6540161710 0x7f65400cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40482/0 (socket says 172.21.15.60:40482) 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.702+0000 7f653dd9b640 1 -- 172.21.15.60:0/1672700805 learned_addr learned my addr 172.21.15.60:0/1672700805 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:58.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653e59c640 1 --2- 172.21.15.60:0/1672700805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f654015dc20 0x7f65400ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653ed9d640 1 --2- 172.21.15.60:0/1672700805 >> v2:172.21.15.100:3300/0 conn(0x7f65400b9330 0x7f654006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653dd9b640 1 -- 172.21.15.60:0/1672700805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f654015dc20 msgr2=0x7f65400ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653dd9b640 1 --2- 172.21.15.60:0/1672700805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f654015dc20 0x7f65400ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653dd9b640 1 -- 172.21.15.60:0/1672700805 >> v2:172.21.15.100:3300/0 conn(0x7f65400b9330 msgr2=0x7f654006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653dd9b640 1 --2- 172.21.15.60:0/1672700805 >> v2:172.21.15.100:3300/0 conn(0x7f65400b9330 0x7f654006bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653dd9b640 1 -- 172.21.15.60:0/1672700805 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540161f40 con 0x7f6540161710 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653ed9d640 1 --2- 172.21.15.60:0/1672700805 >> v2:172.21.15.100:3300/0 conn(0x7f65400b9330 0x7f654006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.703+0000 7f653dd9b640 1 --2- 172.21.15.60:0/1672700805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400cebb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f65400b6570 tx=0x7f653402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9405e474cd9e4c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f653d59a640 1 -- 172.21.15.60:0/1672700805 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534037070 con 0x7f6540161710 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f653d59a640 1 -- 172.21.15.60:0/1672700805 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6534032440 con 0x7f6540161710 2024-04-25T08:33:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f653d59a640 1 -- 172.21.15.60:0/1672700805 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6534032810 con 0x7f6540161710 2024-04-25T08:33:58.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 -- 172.21.15.60:0/1672700805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 msgr2=0x7f65400cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 --2- 172.21.15.60:0/1672700805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400cebb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f65400b6570 tx=0x7f653402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 -- 172.21.15.60:0/1672700805 shutdown_connections 2024-04-25T08:33:58.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 --2- 172.21.15.60:0/1672700805 >> v2:172.21.15.100:3300/0 conn(0x7f65400b9330 0x7f654006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 --2- 172.21.15.60:0/1672700805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400cebb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 --2- 172.21.15.60:0/1672700805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f654015dc20 0x7f65400ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 -- 172.21.15.60:0/1672700805 >> 172.21.15.60:0/1672700805 conn(0x7f65400d0420 msgr2=0x7f65400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 -- 172.21.15.60:0/1672700805 shutdown_connections 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.704+0000 7f6544db0640 1 -- 172.21.15.60:0/1672700805 wait complete. 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 Processor -- start 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 -- start start 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65400b9330 0x7f65400aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f654015dc20 0x7f65401f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65400b5e50 con 0x7f6540161710 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540162500 con 0x7f654015dc20 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f6544db0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65400b5fd0 con 0x7f65400b9330 2024-04-25T08:33:58.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f653ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f653ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40496/0 (socket says 172.21.15.60:40496) 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.706+0000 7f653ed9d640 1 -- 172.21.15.60:0/2928816839 learned_addr learned my addr 172.21.15.60:0/2928816839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653e59c640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65400b9330 0x7f65400aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653dd9b640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f654015dc20 0x7f65401f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653ed9d640 1 -- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65400b9330 msgr2=0x7f65400aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653ed9d640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65400b9330 0x7f65400aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653ed9d640 1 -- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f654015dc20 msgr2=0x7f65401f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653ed9d640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f654015dc20 0x7f65401f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653ed9d640 1 -- 172.21.15.60:0/2928816839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400aaab0 con 0x7f6540161710 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653dd9b640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f654015dc20 0x7f65401f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f653ed9d640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400abe90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f653000cd10 tx=0x7f653000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653000f680 con 0x7f6540161710 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653000f840 con 0x7f6540161710 2024-04-25T08:33:58.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653000fc30 con 0x7f6540161710 2024-04-25T08:33:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65400aad90 con 0x7f6540161710 2024-04-25T08:33:58.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.707+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65400ab350 con 0x7f6540161710 2024-04-25T08:33:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.708+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6540073a00 con 0x7f6540161710 2024-04-25T08:33:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.709+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65300213f0 con 0x7f6540161710 2024-04-25T08:33:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.709+0000 7f652effd640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652006b4d0 0x7f652006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.709+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f653008aff0 con 0x7f6540161710 2024-04-25T08:33:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.710+0000 7f653e59c640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652006b4d0 0x7f652006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:58.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.710+0000 7f653e59c640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652006b4d0 0x7f652006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f65380079a0 tx=0x7f6538008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:58.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.714+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6530014030 con 0x7f6540161710 2024-04-25T08:33:58.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:58.930+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f65400d16b0 con 0x7f6540161710 2024-04-25T08:33:59.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.595+0000 7f652effd640 1 -- 172.21.15.60:0/2928816839 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6530059ca0 con 0x7f6540161710 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652006b4d0 msgr2=0x7f652006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652006b4d0 0x7f652006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f65380079a0 tx=0x7f6538008040 comp rx=0 tx=0).stop 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 msgr2=0x7f65400abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400abe90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f653000cd10 tx=0x7f653000a650 comp rx=0 tx=0).stop 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 shutdown_connections 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6540161710 0x7f65400abe90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f652006b4d0 0x7f652006d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f654015dc20 0x7f65401f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 --2- 172.21.15.60:0/2928816839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f65400b9330 0x7f65400aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 >> 172.21.15.60:0/2928816839 conn(0x7f65400d0420 msgr2=0x7f65400c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:59.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 shutdown_connections 2024-04-25T08:33:59.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.598+0000 7f6544db0640 1 -- 172.21.15.60:0/2928816839 wait complete. 2024-04-25T08:33:59.607 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:33:59.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:33:59.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.719+0000 7f01ecb10640 1 Processor -- start 2024-04-25T08:33:59.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.719+0000 7f01ecb10640 1 -- start start 2024-04-25T08:33:59.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01ecb10640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e815dbe0 0x7f01e80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:59.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01ecb10640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01e80b92a0 0x7f01e80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:59.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01ecb10640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80b97a0 con 0x7f01e80b2cb0 2024-04-25T08:33:59.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01ecb10640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e815dfc0 con 0x7f01e80b92a0 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01ecb10640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80d1a00 con 0x7f01e815dbe0 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01e6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01e80b92a0 0x7f01e80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01e6d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01e80b92a0 0x7f01e80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40510/0 (socket says 172.21.15.60:40510) 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01e6d9d640 1 -- 172.21.15.60:0/1825800822 learned_addr learned my addr 172.21.15.60:0/1825800822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.720+0000 7f01e659c640 1 --2- 172.21.15.60:0/1825800822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e815dbe0 0x7f01e80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.721+0000 7f01e6d9d640 1 -- 172.21.15.60:0/1825800822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e815dbe0 msgr2=0x7f01e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.721+0000 7f01e6d9d640 1 --2- 172.21.15.60:0/1825800822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e815dbe0 0x7f01e80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.721+0000 7f01e6d9d640 1 -- 172.21.15.60:0/1825800822 >> v1:172.21.15.100:6789/0 conn(0x7f01e80b2cb0 legacy=0x7f01e80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.721+0000 7f01e6d9d640 1 -- 172.21.15.60:0/1825800822 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80d1c50 con 0x7f01e80b92a0 2024-04-25T08:33:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.721+0000 7f01e6d9d640 1 --2- 172.21.15.60:0/1825800822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e80b92a0 0x7f01e80d0e50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009e50 tx=0x7f01d802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b397433e513e943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01e559a640 1 -- 172.21.15.60:0/1825800822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8036070 con 0x7f01e80b92a0 2024-04-25T08:33:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01e559a640 1 -- 172.21.15.60:0/1825800822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8031600 con 0x7f01e80b92a0 2024-04-25T08:33:59.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01e559a640 1 -- 172.21.15.60:0/1825800822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80319d0 con 0x7f01e80b92a0 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 -- 172.21.15.60:0/1825800822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e80b92a0 msgr2=0x7f01e80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 --2- 172.21.15.60:0/1825800822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e80b92a0 0x7f01e80d0e50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009e50 tx=0x7f01d802a710 comp rx=0 tx=0).stop 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 -- 172.21.15.60:0/1825800822 shutdown_connections 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 --2- 172.21.15.60:0/1825800822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e80b92a0 0x7f01e80d0e50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 --2- 172.21.15.60:0/1825800822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e815dbe0 0x7f01e80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 -- 172.21.15.60:0/1825800822 >> 172.21.15.60:0/1825800822 conn(0x7f01e8061a20 msgr2=0x7f01e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 -- 172.21.15.60:0/1825800822 shutdown_connections 2024-04-25T08:33:59.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.722+0000 7f01ecb10640 1 -- 172.21.15.60:0/1825800822 wait complete. 2024-04-25T08:33:59.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 Processor -- start 2024-04-25T08:33:59.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 -- start start 2024-04-25T08:33:59.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e80b2cb0 0x7f01e80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:59.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e80b92a0 0x7f01e8149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:59.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 0x7f01e80cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:59.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80b6ab0 con 0x7f01e815dbe0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80d2210 con 0x7f01e80b2cb0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01ecb10640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80b6c30 con 0x7f01e80b92a0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.724+0000 7f01e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 0x7f01e80cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 0x7f01e80cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40524/0 (socket says 172.21.15.60:40524) 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 -- 172.21.15.60:0/2451069350 learned_addr learned my addr 172.21.15.60:0/2451069350 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e5d9b640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e80b92a0 0x7f01e8149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e659c640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e80b2cb0 0x7f01e80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 -- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e80b92a0 msgr2=0x7f01e8149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e80b92a0 0x7f01e8149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 -- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e80b2cb0 msgr2=0x7f01e80cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e80b2cb0 0x7f01e80cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 -- 172.21.15.60:0/2451069350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80cd0b0 con 0x7f01e815dbe0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01e6d9d640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 0x7f01e80cacc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f01d8034040 tx=0x7f01d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:59.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8036050 con 0x7f01e815dbe0 2024-04-25T08:33:59.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8042070 con 0x7f01e815dbe0 2024-04-25T08:33:59.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e80cd390 con 0x7f01e815dbe0 2024-04-25T08:33:59.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8009510 con 0x7f01e815dbe0 2024-04-25T08:33:59.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.725+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e80cd950 con 0x7f01e815dbe0 2024-04-25T08:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.726+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e80d3640 con 0x7f01e815dbe0 2024-04-25T08:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.727+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01d8007bf0 con 0x7f01e815dbe0 2024-04-25T08:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.727+0000 7f01deffd640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f01c806b4d0 0x7f01c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.728+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f01d80ab030 con 0x7f01e815dbe0 2024-04-25T08:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.728+0000 7f01e659c640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f01c806b4d0 0x7f01c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:59.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.728+0000 7f01e659c640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f01c806b4d0 0x7f01c806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009aa0 tx=0x7f01e0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:59.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.732+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01e80d3640 con 0x7f01e815dbe0 2024-04-25T08:33:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.949+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01e8073990 con 0x7f01e815dbe0 2024-04-25T08:33:59.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.950+0000 7f01deffd640 1 -- 172.21.15.60:0/2451069350 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f01d80796d0 con 0x7f01e815dbe0 2024-04-25T08:33:59.946 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:33:59.946 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:56.967441+0000","modified":"2024-04-25T08:33:58.932396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7203":{"gid":7203,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1892944367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1892944367},{"type":"v1","addr":"172.21.15.100:6837","nonce":1892944367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:33:59.946 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f01c806b4d0 msgr2=0x7f01c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f01c806b4d0 0x7f01c806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009aa0 tx=0x7f01e0009190 comp rx=0 tx=0).stop 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 msgr2=0x7f01e80cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 0x7f01e80cacc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f01d8034040 tx=0x7f01d8004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 shutdown_connections 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01e815dbe0 0x7f01e80cacc0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f01e80b92a0 0x7f01e8149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f01c806b4d0 0x7f01c806d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 --2- 172.21.15.60:0/2451069350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01e80b2cb0 0x7f01e80cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 >> 172.21.15.60:0/2451069350 conn(0x7f01e8061a20 msgr2=0x7f01e80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 shutdown_connections 2024-04-25T08:33:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:33:59.952+0000 7f01ecb10640 1 -- 172.21.15.60:0/2451069350 wait complete. 2024-04-25T08:33:59.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:00.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.064+0000 7fd387b1d640 1 Processor -- start 2024-04-25T08:34:00.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd387b1d640 1 -- start start 2024-04-25T08:34:00.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd387b1d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd38015dbe0 0x7fd3800d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd387b1d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3800b6220 con 0x7fd3800b2cb0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd387b1d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3800b97a0 con 0x7fd3800b92a0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd387b1d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38015dfc0 con 0x7fd38015dbe0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd3850b8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd3800b2cb0 0x7fd3800b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35426/0 (socket says 172.21.15.60:35426) 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.065+0000 7fd3850b8640 1 -- 172.21.15.60:0/3045063350 learned_addr learned my addr 172.21.15.60:0/3045063350 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3858b9640 1 --2- 172.21.15.60:0/3045063350 >> v2:172.21.15.100:3300/0 conn(0x7fd38015dbe0 0x7fd3800d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98415075 0 0) 0x7fd3800b6220 con 0x7fd3800b2cb0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd368003520 con 0x7fd3800b2cb0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552622024 0 0) 0x7fd3800b97a0 con 0x7fd3800b92a0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3800b6220 con 0x7fd3800b92a0 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3858b9640 1 -- 172.21.15.60:0/3045063350 >> v1:172.21.15.100:6789/0 conn(0x7fd3800b92a0 legacy=0x7fd3800b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3858b9640 1 -- 172.21.15.60:0/3045063350 >> v1:172.21.15.60:6789/0 conn(0x7fd3800b2cb0 legacy=0x7fd3800b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.066+0000 7fd3858b9640 1 -- 172.21.15.60:0/3045063350 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800d1c50 con 0x7fd38015dbe0 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd3858b9640 1 --2- 172.21.15.60:0/3045063350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38015dbe0 0x7fd3800d0910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd370009d90 tx=0x7fd37002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=142e561f0b771850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370007740 con 0x7fd38015dbe0 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3700078e0 con 0x7fd38015dbe0 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd3848b7640 1 -- 172.21.15.60:0/3045063350 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370007cb0 con 0x7fd38015dbe0 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd387b1d640 1 -- 172.21.15.60:0/3045063350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38015dbe0 msgr2=0x7fd3800d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd387b1d640 1 --2- 172.21.15.60:0/3045063350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38015dbe0 0x7fd3800d0910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd370009d90 tx=0x7fd37002a660 comp rx=0 tx=0).stop 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd387b1d640 1 -- 172.21.15.60:0/3045063350 shutdown_connections 2024-04-25T08:34:00.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd387b1d640 1 --2- 172.21.15.60:0/3045063350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38015dbe0 0x7fd3800d0910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.067+0000 7fd387b1d640 1 -- 172.21.15.60:0/3045063350 >> 172.21.15.60:0/3045063350 conn(0x7fd380061a20 msgr2=0x7fd380061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.068+0000 7fd387b1d640 1 -- 172.21.15.60:0/3045063350 shutdown_connections 2024-04-25T08:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.068+0000 7fd387b1d640 1 -- 172.21.15.60:0/3045063350 wait complete. 2024-04-25T08:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 Processor -- start 2024-04-25T08:34:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 -- start start 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b2cb0 0x7fd3801f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3800b92a0 0x7fd380151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 0x7fd38015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800d2210 con 0x7fd38015dbe0 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800d1a00 con 0x7fd3800b2cb0 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.069+0000 7fd387b1d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800b6ab0 con 0x7fd3800b92a0 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 0x7fd38015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 0x7fd38015a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40532/0 (socket says 172.21.15.60:40532) 2024-04-25T08:34:00.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 -- 172.21.15.60:0/788727250 learned_addr learned my addr 172.21.15.60:0/788727250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3858b9640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b2cb0 0x7fd3801f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3850b8640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3800b92a0 0x7fd380151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 -- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3800b92a0 msgr2=0x7fd380151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3800b92a0 0x7fd380151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 -- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b2cb0 msgr2=0x7fd3801f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b2cb0 0x7fd3801f3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 -- 172.21.15.60:0/788727250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800caaf0 con 0x7fd38015dbe0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3858b9640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b2cb0 0x7fd3801f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd3860ba640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 0x7fd38015a940 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c430 tx=0x7fd37800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378017070 con 0x7fd38015dbe0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780052d0 con 0x7fd38015dbe0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3800cc320 con 0x7fd38015dbe0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378003830 con 0x7fd38015dbe0 2024-04-25T08:34:00.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.070+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801f3870 con 0x7fd38015dbe0 2024-04-25T08:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.071+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd380073990 con 0x7fd38015dbe0 2024-04-25T08:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.072+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd37801a070 con 0x7fd38015dbe0 2024-04-25T08:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.072+0000 7fd37e7fc640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd36c06b4d0 0x7fd36c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.072+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd378089710 con 0x7fd38015dbe0 2024-04-25T08:34:00.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.073+0000 7fd3858b9640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd36c06b4d0 0x7fd36c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.073+0000 7fd3858b9640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd36c06b4d0 0x7fd36c06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd370034000 tx=0x7fd370033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.077+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd378058490 con 0x7fd38015dbe0 2024-04-25T08:34:00.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.257+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd38006b080 con 0x7fd38015dbe0 2024-04-25T08:34:00.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.258+0000 7fd37e7fc640 1 -- 172.21.15.60:0/788727250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fd378057d90 con 0x7fd38015dbe0 2024-04-25T08:34:00.254 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:00.254 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":83,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:33:58.584463+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd36c06b4d0 msgr2=0x7fd36c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd36c06b4d0 0x7fd36c06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd370034000 tx=0x7fd370033040 comp rx=0 tx=0).stop 2024-04-25T08:34:00.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 msgr2=0x7fd38015a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 0x7fd38015a940 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c430 tx=0x7fd37800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 shutdown_connections 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd38015dbe0 0x7fd38015a940 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd3800b92a0 0x7fd380151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.260+0000 7fd387b1d640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd36c06b4d0 0x7fd36c06d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.261+0000 7fd387b1d640 1 --2- 172.21.15.60:0/788727250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3800b2cb0 0x7fd3801f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.261+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 >> 172.21.15.60:0/788727250 conn(0x7fd380061a20 msgr2=0x7fd3800b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.261+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 shutdown_connections 2024-04-25T08:34:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.261+0000 7fd387b1d640 1 -- 172.21.15.60:0/788727250 wait complete. 2024-04-25T08:34:00.267 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:34:00.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:00.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.373+0000 7f5a89f22640 1 Processor -- start 2024-04-25T08:34:00.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.373+0000 7f5a89f22640 1 -- start start 2024-04-25T08:34:00.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a89f22640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5a840b6cb0 0x7f5a840b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a89f22640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840ceee0 con 0x7f5a840b5a60 2024-04-25T08:34:00.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a89f22640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840d1d10 con 0x7f5a8415db70 2024-04-25T08:34:00.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a89f22640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a840d1ff0 con 0x7f5a840b6cb0 2024-04-25T08:34:00.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a83fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a840b5a60 0x7f5a840d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35438/0 (socket says 172.21.15.60:35438) 2024-04-25T08:34:00.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a83fff640 1 -- 172.21.15.60:0/230947508 learned_addr learned my addr 172.21.15.60:0/230947508 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a82ffd640 1 --2- 172.21.15.60:0/230947508 >> v2:172.21.15.100:3300/0 conn(0x7f5a840b6cb0 0x7f5a840b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319949645 0 0) 0x7f5a840ceee0 con 0x7f5a840b5a60 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a6c003520 con 0x7f5a840b5a60 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172060281 0 0) 0x7f5a6c003520 con 0x7f5a840b5a60 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 >> v2:172.21.15.100:3300/0 conn(0x7f5a840b6cb0 msgr2=0x7f5a840b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 --2- 172.21.15.60:0/230947508 >> v2:172.21.15.100:3300/0 conn(0x7f5a840b6cb0 0x7f5a840b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 >> v1:172.21.15.100:6789/0 conn(0x7f5a8415db70 legacy=0x7f5a84162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840d22d0 con 0x7f5a840b5a60 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f5a7c0019f0 con 0x7f5a840b5a60 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 >> v1:172.21.15.60:6789/0 conn(0x7f5a840b5a60 legacy=0x7f5a840d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c0039b0 0x7f5a6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c004800 0x7f5a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.374+0000 7f5a827fc640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0051c0 0x7f5a6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840d1ff0 con 0x7f5a6c0039b0 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c00c160 con 0x7f5a6c0051c0 2024-04-25T08:34:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c00c360 con 0x7f5a6c004800 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a837fe640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c0039b0 0x7f5a6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a82ffd640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c004800 0x7f5a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a83fff640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0051c0 0x7f5a6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a837fe640 1 -- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c004800 msgr2=0x7f5a6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a837fe640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c004800 0x7f5a6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a837fe640 1 -- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0051c0 msgr2=0x7f5a6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a837fe640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0051c0 0x7f5a6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a837fe640 1 -- 172.21.15.60:0/230947508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c003520 con 0x7f5a6c0039b0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a83fff640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a6c0051c0 0x7f5a6c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.375+0000 7f5a83fff640 1 -- 172.21.15.60:0/230947508 reap_dead start 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a837fe640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c0039b0 0x7f5a6c003dd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5a74001750 tx=0x7f5a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508e97c940e555fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74001370 con 0x7f5a6c0039b0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a740052d0 con 0x7f5a6c0039b0 2024-04-25T08:34:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a827fc640 1 -- 172.21.15.60:0/230947508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7402abc0 con 0x7f5a6c0039b0 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 -- 172.21.15.60:0/230947508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c0039b0 msgr2=0x7f5a6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c0039b0 0x7f5a6c003dd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5a74001750 tx=0x7f5a6c005830 comp rx=0 tx=0).stop 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 -- 172.21.15.60:0/230947508 shutdown_connections 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 --2- 172.21.15.60:0/230947508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6c0039b0 0x7f5a6c003dd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 -- 172.21.15.60:0/230947508 >> 172.21.15.60:0/230947508 conn(0x7f5a840d0520 msgr2=0x7f5a840d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 -- 172.21.15.60:0/230947508 shutdown_connections 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.376+0000 7f5a89f22640 1 -- 172.21.15.60:0/230947508 wait complete. 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.377+0000 7f5a89f22640 1 Processor -- start 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 -- start start 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0039b0 0x7f5a841f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84159a70 0x7f5a8415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 0x7f5a8415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840cf1c0 con 0x7f5a841f43b0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840ce530 con 0x7f5a84159a70 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a89f22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840cf340 con 0x7f5a6c0039b0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 0x7f5a8415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 0x7f5a8415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40546/0 (socket says 172.21.15.60:40546) 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.378+0000 7f5a83fff640 1 -- 172.21.15.60:0/3610498492 learned_addr learned my addr 172.21.15.60:0/3610498492 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a837fe640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0039b0 0x7f5a841f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a82ffd640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84159a70 0x7f5a8415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a83fff640 1 -- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0039b0 msgr2=0x7f5a841f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a83fff640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0039b0 0x7f5a841f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a83fff640 1 -- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84159a70 msgr2=0x7f5a8415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a83fff640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84159a70 0x7f5a8415b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a83fff640 1 -- 172.21.15.60:0/3610498492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a84159f30 con 0x7f5a841f43b0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a83fff640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 0x7f5a8415beb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5a840b6690 tx=0x7f5a7c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00ea80 con 0x7f5a841f43b0 2024-04-25T08:34:00.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c018070 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8415a210 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c00ee80 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.379+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8415a7d0 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.380+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a84073990 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.381+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a7c01d460 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.381+0000 7f5a6bfff640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a7006b500 0x7f5a7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.381+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a7c08a560 con 0x7f5a841f43b0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.381+0000 7f5a837fe640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a7006b500 0x7f5a7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.382+0000 7f5a837fe640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a7006b500 0x7f5a7006d9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5a74032000 tx=0x7f5a7402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.385+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a7c0592e0 con 0x7f5a841f43b0 2024-04-25T08:34:00.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.562+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a8415ab40 con 0x7f5a7006b500 2024-04-25T08:34:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.569+0000 7f5a6bfff640 1 -- 172.21.15.60:0/3610498492 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238428 (secure 0 0 0) 0x7f5a8415ab40 con 0x7f5a7006b500 2024-04-25T08:34:00.567 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:00.569 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:34:00.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a7006b500 msgr2=0x7f5a7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a7006b500 0x7f5a7006d9c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5a74032000 tx=0x7f5a7402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:00.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 msgr2=0x7f5a8415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 0x7f5a8415beb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5a840b6690 tx=0x7f5a7c007bc0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 shutdown_connections 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a841f43b0 0x7f5a8415beb0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a7006b500 0x7f5a7006d9c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a84159a70 0x7f5a8415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 --2- 172.21.15.60:0/3610498492 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a6c0039b0 0x7f5a841f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 >> 172.21.15.60:0/3610498492 conn(0x7f5a840d0520 msgr2=0x7f5a840c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 shutdown_connections 2024-04-25T08:34:00.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.572+0000 7f5a89f22640 1 -- 172.21.15.60:0/3610498492 wait complete. 2024-04-25T08:34:00.578 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-25T08:33:59.462800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":316500,"kb_used_data":8716,"kb_used_omap":12,"kb_used_meta":307635,"kb_avail":754658220,"statfs":{"total":773094113280,"available":772770017280,"internally_reserved":0,"allocated":8925184,"data_stored":4710323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":315018800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000821"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565131+0000","last_change":"2024-04-25T08:33:55.565131+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565131+0000","last_undegraded":"2024-04-25T08:33:55.565131+0000","last_fullsized":"2024-04-25T08:33:55.565131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565882+0000","last_change":"2024-04-25T08:33:55.565882+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565882+0000","last_undegraded":"2024-04-25T08:33:55.565882+0000","last_fullsized":"2024-04-25T08:33:55.565882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561547+0000","last_change":"2024-04-25T08:33:55.561547+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561547+0000","last_undegraded":"2024-04-25T08:33:55.561547+0000","last_fullsized":"2024-04-25T08:33:55.561547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567205+0000","last_change":"2024-04-25T08:33:55.567205+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567205+0000","last_undegraded":"2024-04-25T08:33:55.567205+0000","last_fullsized":"2024-04-25T08:33:55.567205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565678+0000","last_change":"2024-04-25T08:33:55.565678+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565678+0000","last_undegraded":"2024-04-25T08:33:55.565678+0000","last_fullsized":"2024-04-25T08:33:55.565678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567445+0000","last_change":"2024-04-25T08:33:55.567445+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567445+0000","last_undegraded":"2024-04-25T08:33:55.567445+0000","last_fullsized":"2024-04-25T08:33:55.567445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561049+0000","last_change":"2024-04-25T08:33:55.561049+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561049+0000","last_undegraded":"2024-04-25T08:33:55.561049+0000","last_fullsized":"2024-04-25T08:33:55.561049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567281+0000","last_change":"2024-04-25T08:33:55.567281+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567281+0000","last_undegraded":"2024-04-25T08:33:55.567281+0000","last_fullsized":"2024-04-25T08:33:55.567281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565063+0000","last_change":"2024-04-25T08:33:55.565063+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565063+0000","last_undegraded":"2024-04-25T08:33:55.565063+0000","last_fullsized":"2024-04-25T08:33:55.565063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565190+0000","last_change":"2024-04-25T08:33:55.565190+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565190+0000","last_undegraded":"2024-04-25T08:33:55.565190+0000","last_fullsized":"2024-04-25T08:33:55.565190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567299+0000","last_change":"2024-04-25T08:33:55.567299+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567299+0000","last_undegraded":"2024-04-25T08:33:55.567299+0000","last_fullsized":"2024-04-25T08:33:55.567299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565172+0000","last_change":"2024-04-25T08:33:55.565172+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565172+0000","last_undegraded":"2024-04-25T08:33:55.565172+0000","last_fullsized":"2024-04-25T08:33:55.565172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565137+0000","last_change":"2024-04-25T08:33:55.565137+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565137+0000","last_undegraded":"2024-04-25T08:33:55.565137+0000","last_fullsized":"2024-04-25T08:33:55.565137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561006+0000","last_change":"2024-04-25T08:33:55.561006+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561006+0000","last_undegraded":"2024-04-25T08:33:55.561006+0000","last_fullsized":"2024-04-25T08:33:55.561006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.560966+0000","last_change":"2024-04-25T08:33:55.560966+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.560966+0000","last_undegraded":"2024-04-25T08:33:55.560966+0000","last_fullsized":"2024-04-25T08:33:55.560966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:33:55.566173+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:55.566173+0000","last_peered":"2024-04-25T08:33:55.566173+0000","last_clean":"2024-04-25T08:33:55.566173+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:55.566173+0000","last_undegraded":"2024-04-25T08:33:55.566173+0000","last_fullsized":"2024-04-25T08:33:55.566173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1568768,"data_stored":1035447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":974848,"data_stored":445079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":974848,"data_stored":445079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40440,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331400,"statfs":{"total":96636764160,"available":96595353600,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40444,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40448,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49416,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":47870,"kb_avail":94322424,"statfs":{"total":96636764160,"available":96586162176,"internally_reserved":0,"allocated":1556480,"data_stored":1029238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":49019333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:00.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:00.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.724+0000 7f5290d30640 1 Processor -- start 2024-04-25T08:34:00.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.724+0000 7f5290d30640 1 -- start start 2024-04-25T08:34:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f5290d30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f528c154ee0 0x7f528c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f5290d30640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c071480 con 0x7f528c0cfcf0 2024-04-25T08:34:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f5290d30640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c071760 con 0x7f528c0ce590 2024-04-25T08:34:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f5290d30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c072090 con 0x7f528c154ee0 2024-04-25T08:34:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f528a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f528c154ee0 0x7f528c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f528a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f528c154ee0 0x7f528c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40552/0 (socket says 172.21.15.60:40552) 2024-04-25T08:34:00.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.725+0000 7f528a59c640 1 -- 172.21.15.60:0/3827446714 learned_addr learned my addr 172.21.15.60:0/3827446714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:00.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.726+0000 7f528a59c640 1 -- 172.21.15.60:0/3827446714 >> v1:172.21.15.100:6789/0 conn(0x7f528c0ce590 legacy=0x7f528c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:00.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.726+0000 7f528a59c640 1 -- 172.21.15.60:0/3827446714 >> v1:172.21.15.60:6789/0 conn(0x7f528c0cfcf0 legacy=0x7f528c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.726+0000 7f528a59c640 1 -- 172.21.15.60:0/3827446714 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c071a40 con 0x7f528c154ee0 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.726+0000 7f528a59c640 1 --2- 172.21.15.60:0/3827446714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c0cf7b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5284009870 tx=0x7f528402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d45c065679e4ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f528959a640 1 -- 172.21.15.60:0/3827446714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284037070 con 0x7f528c154ee0 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f528959a640 1 -- 172.21.15.60:0/3827446714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284032440 con 0x7f528c154ee0 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f528959a640 1 -- 172.21.15.60:0/3827446714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284032810 con 0x7f528c154ee0 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 -- 172.21.15.60:0/3827446714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 msgr2=0x7f528c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 --2- 172.21.15.60:0/3827446714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c0cf7b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5284009870 tx=0x7f528402a2a0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 -- 172.21.15.60:0/3827446714 shutdown_connections 2024-04-25T08:34:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 --2- 172.21.15.60:0/3827446714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c0cf7b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 -- 172.21.15.60:0/3827446714 >> 172.21.15.60:0/3827446714 conn(0x7f528c06abe0 msgr2=0x7f528c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 -- 172.21.15.60:0/3827446714 shutdown_connections 2024-04-25T08:34:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.727+0000 7f5290d30640 1 -- 172.21.15.60:0/3827446714 wait complete. 2024-04-25T08:34:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 Processor -- start 2024-04-25T08:34:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 -- start start 2024-04-25T08:34:00.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f528c0ce590 0x7f528c1440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f528c0cfcf0 0x7f528c1e5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1552c0 con 0x7f528c154ee0 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c071230 con 0x7f528c0ce590 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.729+0000 7f5290d30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c155440 con 0x7f528c0cfcf0 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40560/0 (socket says 172.21.15.60:40560) 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 -- 172.21.15.60:0/1506081021 learned_addr learned my addr 172.21.15.60:0/1506081021 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528a59c640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f528c0ce590 0x7f528c1440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f5289d9b640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f528c0cfcf0 0x7f528c1e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 -- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f528c0cfcf0 msgr2=0x7f528c1e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f528c0cfcf0 0x7f528c1e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 -- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f528c0ce590 msgr2=0x7f528c1440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f528c0ce590 0x7f528c1440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 -- 172.21.15.60:0/1506081021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0b0ab0 con 0x7f528c154ee0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f5289d9b640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f528c0cfcf0 0x7f528c1e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f528ad9d640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c1599d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c780 tx=0x7f527c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c004b60 con 0x7f528c154ee0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c004d00 con 0x7f528c154ee0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c00aa90 con 0x7f528c154ee0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c0b0d90 con 0x7f528c154ee0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.730+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c1e5750 con 0x7f528c154ee0 2024-04-25T08:34:00.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.731+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c073990 con 0x7f528c154ee0 2024-04-25T08:34:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.732+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f527c00ac30 con 0x7f528c154ee0 2024-04-25T08:34:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.732+0000 7f527affd640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f526c06b4d0 0x7f526c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.732+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f527c08a5c0 con 0x7f528c154ee0 2024-04-25T08:34:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.732+0000 7f528a59c640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f526c06b4d0 0x7f526c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:00.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.733+0000 7f528a59c640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f526c06b4d0 0x7f526c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5284034000 tx=0x7f528402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.736+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f527c020070 con 0x7f528c154ee0 2024-04-25T08:34:00.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.912+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f528c1e4f70 con 0x7f526c06b4d0 2024-04-25T08:34:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.918+0000 7f527affd640 1 -- 172.21.15.60:0/1506081021 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238428 (secure 0 0 0) 0x7f5284030340 con 0x7f526c06b4d0 2024-04-25T08:34:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:00.917 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f526c06b4d0 msgr2=0x7f526c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f526c06b4d0 0x7f526c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5284034000 tx=0x7f528402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 msgr2=0x7f528c1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c1599d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c780 tx=0x7f527c00cc40 comp rx=0 tx=0).stop 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 shutdown_connections 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f528c154ee0 0x7f528c1599d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f528c0cfcf0 0x7f528c1e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f526c06b4d0 0x7f526c06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 --2- 172.21.15.60:0/1506081021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f528c0ce590 0x7f528c1440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 >> 172.21.15.60:0/1506081021 conn(0x7f528c06abe0 msgr2=0x7f528c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 shutdown_connections 2024-04-25T08:34:00.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:00.921+0000 7f5290d30640 1 -- 172.21.15.60:0/1506081021 wait complete. 2024-04-25T08:34:00.928 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-25T08:33:59.462800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":316500,"kb_used_data":8716,"kb_used_omap":12,"kb_used_meta":307635,"kb_avail":754658220,"statfs":{"total":773094113280,"available":772770017280,"internally_reserved":0,"allocated":8925184,"data_stored":4710323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":315018800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000821"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565131+0000","last_change":"2024-04-25T08:33:55.565131+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565131+0000","last_undegraded":"2024-04-25T08:33:55.565131+0000","last_fullsized":"2024-04-25T08:33:55.565131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565882+0000","last_change":"2024-04-25T08:33:55.565882+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565882+0000","last_undegraded":"2024-04-25T08:33:55.565882+0000","last_fullsized":"2024-04-25T08:33:55.565882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561547+0000","last_change":"2024-04-25T08:33:55.561547+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561547+0000","last_undegraded":"2024-04-25T08:33:55.561547+0000","last_fullsized":"2024-04-25T08:33:55.561547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567205+0000","last_change":"2024-04-25T08:33:55.567205+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567205+0000","last_undegraded":"2024-04-25T08:33:55.567205+0000","last_fullsized":"2024-04-25T08:33:55.567205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565678+0000","last_change":"2024-04-25T08:33:55.565678+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565678+0000","last_undegraded":"2024-04-25T08:33:55.565678+0000","last_fullsized":"2024-04-25T08:33:55.565678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567445+0000","last_change":"2024-04-25T08:33:55.567445+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567445+0000","last_undegraded":"2024-04-25T08:33:55.567445+0000","last_fullsized":"2024-04-25T08:33:55.567445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561049+0000","last_change":"2024-04-25T08:33:55.561049+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561049+0000","last_undegraded":"2024-04-25T08:33:55.561049+0000","last_fullsized":"2024-04-25T08:33:55.561049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567281+0000","last_change":"2024-04-25T08:33:55.567281+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567281+0000","last_undegraded":"2024-04-25T08:33:55.567281+0000","last_fullsized":"2024-04-25T08:33:55.567281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565063+0000","last_change":"2024-04-25T08:33:55.565063+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565063+0000","last_undegraded":"2024-04-25T08:33:55.565063+0000","last_fullsized":"2024-04-25T08:33:55.565063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565190+0000","last_change":"2024-04-25T08:33:55.565190+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565190+0000","last_undegraded":"2024-04-25T08:33:55.565190+0000","last_fullsized":"2024-04-25T08:33:55.565190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567299+0000","last_change":"2024-04-25T08:33:55.567299+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567299+0000","last_undegraded":"2024-04-25T08:33:55.567299+0000","last_fullsized":"2024-04-25T08:33:55.567299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565172+0000","last_change":"2024-04-25T08:33:55.565172+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565172+0000","last_undegraded":"2024-04-25T08:33:55.565172+0000","last_fullsized":"2024-04-25T08:33:55.565172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565137+0000","last_change":"2024-04-25T08:33:55.565137+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565137+0000","last_undegraded":"2024-04-25T08:33:55.565137+0000","last_fullsized":"2024-04-25T08:33:55.565137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561006+0000","last_change":"2024-04-25T08:33:55.561006+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561006+0000","last_undegraded":"2024-04-25T08:33:55.561006+0000","last_fullsized":"2024-04-25T08:33:55.561006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.560966+0000","last_change":"2024-04-25T08:33:55.560966+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.560966+0000","last_undegraded":"2024-04-25T08:33:55.560966+0000","last_fullsized":"2024-04-25T08:33:55.560966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:33:55.566173+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:55.566173+0000","last_peered":"2024-04-25T08:33:55.566173+0000","last_clean":"2024-04-25T08:33:55.566173+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:55.566173+0000","last_undegraded":"2024-04-25T08:33:55.566173+0000","last_fullsized":"2024-04-25T08:33:55.566173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1568768,"data_stored":1035447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":974848,"data_stored":445079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":974848,"data_stored":445079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40440,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331400,"statfs":{"total":96636764160,"available":96595353600,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40444,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40448,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49416,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":47870,"kb_avail":94322424,"statfs":{"total":96636764160,"available":96586162176,"internally_reserved":0,"allocated":1556480,"data_stored":1029238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":49019333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:00.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.074+0000 7f1601aac640 1 Processor -- start 2024-04-25T08:34:01.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.074+0000 7f1601aac640 1 -- start start 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f1601aac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15fc0cfec0 0x7f15fc0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f1601aac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc071480 con 0x7f15fc154ee0 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f1601aac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc071760 con 0x7f15fc0ce3c0 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f1601aac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0ce830 con 0x7f15fc0cfec0 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15faffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15fc0cfec0 0x7f15fc0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15faffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f15fc0cfec0 0x7f15fc0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40568/0 (socket says 172.21.15.60:40568) 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15faffd640 1 -- 172.21.15.60:0/1322637274 learned_addr learned my addr 172.21.15.60:0/1322637274 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15fb7fe640 1 --1- 172.21.15.60:0/1322637274 >> v1:172.21.15.60:6789/0 conn(0x7f15fc154ee0 0x7f15fc0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35454/0 (socket says 172.21.15.60:35454) 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15faffd640 1 -- 172.21.15.60:0/1322637274 >> v1:172.21.15.100:6789/0 conn(0x7f15fc0ce3c0 legacy=0x7f15fc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15faffd640 1 -- 172.21.15.60:0/1322637274 >> v1:172.21.15.60:6789/0 conn(0x7f15fc154ee0 legacy=0x7f15fc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.075+0000 7f15faffd640 1 -- 172.21.15.60:0/1322637274 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc071a40 con 0x7f15fc0cfec0 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.076+0000 7f15faffd640 1 --2- 172.21.15.60:0/1322637274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc0cde80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009870 tx=0x7f15f002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdbb7c3268f49fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.076+0000 7f15fa7fc640 1 -- 172.21.15.60:0/1322637274 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0037070 con 0x7f15fc0cfec0 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.076+0000 7f15fa7fc640 1 -- 172.21.15.60:0/1322637274 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f002ae50 con 0x7f15fc0cfec0 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.076+0000 7f15fa7fc640 1 -- 172.21.15.60:0/1322637274 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0032860 con 0x7f15fc0cfec0 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 -- 172.21.15.60:0/1322637274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 msgr2=0x7f15fc0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 --2- 172.21.15.60:0/1322637274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc0cde80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009870 tx=0x7f15f002a300 comp rx=0 tx=0).stop 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 -- 172.21.15.60:0/1322637274 shutdown_connections 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 --2- 172.21.15.60:0/1322637274 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc0cde80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 -- 172.21.15.60:0/1322637274 >> 172.21.15.60:0/1322637274 conn(0x7f15fc06abe0 msgr2=0x7f15fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 -- 172.21.15.60:0/1322637274 shutdown_connections 2024-04-25T08:34:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.077+0000 7f1601aac640 1 -- 172.21.15.60:0/1322637274 wait complete. 2024-04-25T08:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.078+0000 7f1601aac640 1 Processor -- start 2024-04-25T08:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.078+0000 7f1601aac640 1 -- start start 2024-04-25T08:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f1601aac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15fc0ce3c0 0x7f15fc14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f1601aac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc14e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f1601aac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc154ee0 0x7f15fc1e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f1601aac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc071230 con 0x7f15fc0cfec0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f1601aac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc1552c0 con 0x7f15fc154ee0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f1601aac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc155440 con 0x7f15fc0ce3c0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc14e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc14e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40572/0 (socket says 172.21.15.60:40572) 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 -- 172.21.15.60:0/1236593578 learned_addr learned my addr 172.21.15.60:0/1236593578 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15fb7fe640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15fc0ce3c0 0x7f15fc14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15fbfff640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc154ee0 0x7f15fc1e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 -- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15fc0ce3c0 msgr2=0x7f15fc14fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15fc0ce3c0 0x7f15fc14fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 -- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc154ee0 msgr2=0x7f15fc1e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc154ee0 0x7f15fc1e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 -- 172.21.15.60:0/1236593578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc1f8170 con 0x7f15fc0cfec0 2024-04-25T08:34:01.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15fb7fe640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15fc0ce3c0 0x7f15fc14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15fbfff640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc154ee0 0x7f15fc1e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.079+0000 7f15faffd640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc14e670 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.080+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0041020 con 0x7f15fc0cfec0 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.080+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f003c890 con 0x7f15fc0cfec0 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.080+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc1e40d0 con 0x7f15fc0cfec0 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.080+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0037040 con 0x7f15fc0cfec0 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.080+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc1e4650 con 0x7f15fc0cfec0 2024-04-25T08:34:01.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.081+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc073990 con 0x7f15fc0cfec0 2024-04-25T08:34:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.081+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15f0044020 con 0x7f15fc0cfec0 2024-04-25T08:34:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.082+0000 7f15e3fff640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f15e806b4d0 0x7f15e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.082+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15f00aab90 con 0x7f15fc0cfec0 2024-04-25T08:34:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.082+0000 7f15fb7fe640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f15e806b4d0 0x7f15e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.083+0000 7f15fb7fe640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f15e806b4d0 0x7f15e806d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f15ec007fd0 tx=0x7f15ec007d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.086+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15f004b070 con 0x7f15fc0cfec0 2024-04-25T08:34:01.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.265+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15fc06dbe0 con 0x7f15e806b4d0 2024-04-25T08:34:01.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.271+0000 7f15e3fff640 1 -- 172.21.15.60:0/1236593578 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238428 (secure 0 0 0) 0x7f15fc06dbe0 con 0x7f15e806b4d0 2024-04-25T08:34:01.269 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f15e806b4d0 msgr2=0x7f15e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f15e806b4d0 0x7f15e806d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f15ec007fd0 tx=0x7f15ec007d40 comp rx=0 tx=0).stop 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 msgr2=0x7f15fc14e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc14e670 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f00046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 shutdown_connections 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f15e806b4d0 0x7f15e806d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15fc154ee0 0x7f15fc1e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f15fc0cfec0 0x7f15fc14e670 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 --2- 172.21.15.60:0/1236593578 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15fc0ce3c0 0x7f15fc14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 >> 172.21.15.60:0/1236593578 conn(0x7f15fc06abe0 msgr2=0x7f15fc0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:01.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 shutdown_connections 2024-04-25T08:34:01.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.274+0000 7f1601aac640 1 -- 172.21.15.60:0/1236593578 wait complete. 2024-04-25T08:34:01.281 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-25T08:33:59.462800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":32,"acting":32,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":316500,"kb_used_data":8716,"kb_used_omap":12,"kb_used_meta":307635,"kb_avail":754658220,"statfs":{"total":773094113280,"available":772770017280,"internally_reserved":0,"allocated":8925184,"data_stored":4710323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":315018800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000821"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565131+0000","last_change":"2024-04-25T08:33:55.565131+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565131+0000","last_undegraded":"2024-04-25T08:33:55.565131+0000","last_fullsized":"2024-04-25T08:33:55.565131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565882+0000","last_change":"2024-04-25T08:33:55.565882+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565882+0000","last_undegraded":"2024-04-25T08:33:55.565882+0000","last_fullsized":"2024-04-25T08:33:55.565882+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561547+0000","last_change":"2024-04-25T08:33:55.561547+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561547+0000","last_undegraded":"2024-04-25T08:33:55.561547+0000","last_fullsized":"2024-04-25T08:33:55.561547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567205+0000","last_change":"2024-04-25T08:33:55.567205+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567205+0000","last_undegraded":"2024-04-25T08:33:55.567205+0000","last_fullsized":"2024-04-25T08:33:55.567205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565678+0000","last_change":"2024-04-25T08:33:55.565678+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565678+0000","last_undegraded":"2024-04-25T08:33:55.565678+0000","last_fullsized":"2024-04-25T08:33:55.565678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567445+0000","last_change":"2024-04-25T08:33:55.567445+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567445+0000","last_undegraded":"2024-04-25T08:33:55.567445+0000","last_fullsized":"2024-04-25T08:33:55.567445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561049+0000","last_change":"2024-04-25T08:33:55.561049+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561049+0000","last_undegraded":"2024-04-25T08:33:55.561049+0000","last_fullsized":"2024-04-25T08:33:55.561049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567281+0000","last_change":"2024-04-25T08:33:55.567281+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567281+0000","last_undegraded":"2024-04-25T08:33:55.567281+0000","last_fullsized":"2024-04-25T08:33:55.567281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565063+0000","last_change":"2024-04-25T08:33:55.565063+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565063+0000","last_undegraded":"2024-04-25T08:33:55.565063+0000","last_fullsized":"2024-04-25T08:33:55.565063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565190+0000","last_change":"2024-04-25T08:33:55.565190+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565190+0000","last_undegraded":"2024-04-25T08:33:55.565190+0000","last_fullsized":"2024-04-25T08:33:55.565190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.567299+0000","last_change":"2024-04-25T08:33:55.567299+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.567299+0000","last_undegraded":"2024-04-25T08:33:55.567299+0000","last_fullsized":"2024-04-25T08:33:55.567299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565172+0000","last_change":"2024-04-25T08:33:55.565172+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565172+0000","last_undegraded":"2024-04-25T08:33:55.565172+0000","last_fullsized":"2024-04-25T08:33:55.565172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.565137+0000","last_change":"2024-04-25T08:33:55.565137+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.565137+0000","last_undegraded":"2024-04-25T08:33:55.565137+0000","last_fullsized":"2024-04-25T08:33:55.565137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.561006+0000","last_change":"2024-04-25T08:33:55.561006+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.561006+0000","last_undegraded":"2024-04-25T08:33:55.561006+0000","last_fullsized":"2024-04-25T08:33:55.561006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:33:55.560966+0000","last_change":"2024-04-25T08:33:55.560966+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.560966+0000","last_undegraded":"2024-04-25T08:33:55.560966+0000","last_fullsized":"2024-04-25T08:33:55.560966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:56.557627+0000","last_change":"2024-04-25T08:33:56.557627+0000","last_active":"2024-04-25T08:33:56.557627+0000","last_peered":"2024-04-25T08:33:56.557627+0000","last_clean":"2024-04-25T08:33:56.557627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:56.557627+0000","last_undegraded":"2024-04-25T08:33:56.557627+0000","last_fullsized":"2024-04-25T08:33:56.557627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:33:55.552611+0000","last_change":"2024-04-25T08:33:55.552611+0000","last_active":"2024-04-25T08:33:55.552611+0000","last_peered":"2024-04-25T08:33:55.552611+0000","last_clean":"2024-04-25T08:33:55.552611+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:33:55.552611+0000","last_undegraded":"2024-04-25T08:33:55.552611+0000","last_fullsized":"2024-04-25T08:33:55.552611+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":282,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:33:55.566173+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:55.566173+0000","last_peered":"2024-04-25T08:33:55.566173+0000","last_clean":"2024-04-25T08:33:55.566173+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:55.566173+0000","last_undegraded":"2024-04-25T08:33:55.566173+0000","last_fullsized":"2024-04-25T08:33:55.566173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1568768,"data_stored":1035447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335544,"statfs":{"total":96636764160,"available":96599597056,"internally_reserved":0,"allocated":974848,"data_stored":445079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":974848,"data_stored":445079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335548,"statfs":{"total":96636764160,"available":96599601152,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40440,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331400,"statfs":{"total":96636764160,"available":96595353600,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40444,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40448,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331392,"statfs":{"total":96636764160,"available":96595345408,"internally_reserved":0,"allocated":962560,"data_stored":438870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49416,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":47870,"kb_avail":94322424,"statfs":{"total":96636764160,"available":96586162176,"internally_reserved":0,"allocated":1556480,"data_stored":1029238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":49019333},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2024-04-25T08:34:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2024-04-25T08:34:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2024-04-25T08:34:01.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2024-04-25T08:34:01.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2024-04-25T08:34:01.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2024-04-25T08:34:01.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565131+0000', 'last_change': '2024-04-25T08:33:55.565131+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565131+0000', 'last_undegraded': '2024-04-25T08:33:55.565131+0000', 'last_fullsized': '2024-04-25T08:33:55.565131+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565882+0000', 'last_change': '2024-04-25T08:33:55.565882+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565882+0000', 'last_undegraded': '2024-04-25T08:33:55.565882+0000', 'last_fullsized': '2024-04-25T08:33:55.565882+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.561547+0000', 'last_change': '2024-04-25T08:33:55.561547+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.561547+0000', 'last_undegraded': '2024-04-25T08:33:55.561547+0000', 'last_fullsized': '2024-04-25T08:33:55.561547+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.567205+0000', 'last_change': '2024-04-25T08:33:55.567205+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.567205+0000', 'last_undegraded': '2024-04-25T08:33:55.567205+0000', 'last_fullsized': '2024-04-25T08:33:55.567205+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-25T08:34:01.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565678+0000', 'last_change': '2024-04-25T08:33:55.565678+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565678+0000', 'last_undegraded': '2024-04-25T08:33:55.565678+0000', 'last_fullsized': '2024-04-25T08:33:55.565678+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2024-04-25T08:34:01.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.567445+0000', 'last_change': '2024-04-25T08:33:55.567445+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.567445+0000', 'last_undegraded': '2024-04-25T08:33:55.567445+0000', 'last_fullsized': '2024-04-25T08:33:55.567445+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.561049+0000', 'last_change': '2024-04-25T08:33:55.561049+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.561049+0000', 'last_undegraded': '2024-04-25T08:33:55.561049+0000', 'last_fullsized': '2024-04-25T08:33:55.561049+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.567281+0000', 'last_change': '2024-04-25T08:33:55.567281+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.567281+0000', 'last_undegraded': '2024-04-25T08:33:55.567281+0000', 'last_fullsized': '2024-04-25T08:33:55.567281+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-25T08:34:01.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565063+0000', 'last_change': '2024-04-25T08:33:55.565063+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565063+0000', 'last_undegraded': '2024-04-25T08:33:55.565063+0000', 'last_fullsized': '2024-04-25T08:33:55.565063+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2024-04-25T08:34:01.297 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565190+0000', 'last_change': '2024-04-25T08:33:55.565190+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565190+0000', 'last_undegraded': '2024-04-25T08:33:55.565190+0000', 'last_fullsized': '2024-04-25T08:33:55.565190+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2024-04-25T08:34:01.298 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2024-04-25T08:34:01.299 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-04-25T08:34:01.300 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.567299+0000', 'last_change': '2024-04-25T08:33:55.567299+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.567299+0000', 'last_undegraded': '2024-04-25T08:33:55.567299+0000', 'last_fullsized': '2024-04-25T08:33:55.567299+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565172+0000', 'last_change': '2024-04-25T08:33:55.565172+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565172+0000', 'last_undegraded': '2024-04-25T08:33:55.565172+0000', 'last_fullsized': '2024-04-25T08:33:55.565172+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2024-04-25T08:34:01.301 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.565137+0000', 'last_change': '2024-04-25T08:33:55.565137+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.565137+0000', 'last_undegraded': '2024-04-25T08:33:55.565137+0000', 'last_fullsized': '2024-04-25T08:33:55.565137+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.561006+0000', 'last_change': '2024-04-25T08:33:55.561006+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.561006+0000', 'last_undegraded': '2024-04-25T08:33:55.561006+0000', 'last_fullsized': '2024-04-25T08:33:55.561006+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.302 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:33:55.560966+0000', 'last_change': '2024-04-25T08:33:55.560966+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.560966+0000', 'last_undegraded': '2024-04-25T08:33:55.560966+0000', 'last_fullsized': '2024-04-25T08:33:55.560966+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-04-25T08:34:01.303 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-04-25T08:34:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:56.557627+0000', 'last_change': '2024-04-25T08:33:56.557627+0000', 'last_active': '2024-04-25T08:33:56.557627+0000', 'last_peered': '2024-04-25T08:33:56.557627+0000', 'last_clean': '2024-04-25T08:33:56.557627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:56.557627+0000', 'last_undegraded': '2024-04-25T08:33:56.557627+0000', 'last_fullsized': '2024-04-25T08:33:56.557627+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:56.557627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-04-25T08:34:01.304 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:33:55.552611+0000', 'last_change': '2024-04-25T08:33:55.552611+0000', 'last_active': '2024-04-25T08:33:55.552611+0000', 'last_peered': '2024-04-25T08:33:55.552611+0000', 'last_clean': '2024-04-25T08:33:55.552611+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:33:55.552611+0000', 'last_undegraded': '2024-04-25T08:33:55.552611+0000', 'last_fullsized': '2024-04-25T08:33:55.552611+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'last_clean_scrub_stamp': '2024-04-25T08:33:55.552611+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:01.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:34:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.425+0000 7f3cd6c15640 1 Processor -- start 2024-04-25T08:34:01.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.426+0000 7f3cd6c15640 1 -- start start 2024-04-25T08:34:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.426+0000 7f3cd6c15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3cd015dbe0 0x7f3cd00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.426+0000 7f3cd6c15640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd00b6220 con 0x7f3cd00b2cb0 2024-04-25T08:34:01.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.426+0000 7f3cd6c15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd00b97a0 con 0x7f3cd00b92a0 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.426+0000 7f3cd6c15640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd015dfc0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.426+0000 7f3cd49b1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3cd015dbe0 0x7f3cd00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.427+0000 7f3cd49b1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3cd015dbe0 0x7f3cd00d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40574/0 (socket says 172.21.15.60:40574) 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.427+0000 7f3cd49b1640 1 -- 172.21.15.60:0/1291530935 learned_addr learned my addr 172.21.15.60:0/1291530935 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.427+0000 7f3cd49b1640 1 -- 172.21.15.60:0/1291530935 >> v1:172.21.15.100:6789/0 conn(0x7f3cd00b92a0 legacy=0x7f3cd00b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.427+0000 7f3cd49b1640 1 -- 172.21.15.60:0/1291530935 >> v1:172.21.15.60:6789/0 conn(0x7f3cd00b2cb0 legacy=0x7f3cd00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.427+0000 7f3cd49b1640 1 -- 172.21.15.60:0/1291530935 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd00d1c50 con 0x7f3cd015dbe0 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.427+0000 7f3cd49b1640 1 --2- 172.21.15.60:0/1291530935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd00d0910 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009870 tx=0x7f3cc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856c246be7e7ddf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3ccf7fe640 1 -- 172.21.15.60:0/1291530935 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4037070 con 0x7f3cd015dbe0 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3ccf7fe640 1 -- 172.21.15.60:0/1291530935 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4032440 con 0x7f3cd015dbe0 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3ccf7fe640 1 -- 172.21.15.60:0/1291530935 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4032810 con 0x7f3cd015dbe0 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 -- 172.21.15.60:0/1291530935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 msgr2=0x7f3cd00d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/1291530935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd00d0910 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009870 tx=0x7f3cc402a210 comp rx=0 tx=0).stop 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 -- 172.21.15.60:0/1291530935 shutdown_connections 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/1291530935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd00d0910 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 -- 172.21.15.60:0/1291530935 >> 172.21.15.60:0/1291530935 conn(0x7f3cd0061a20 msgr2=0x7f3cd0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 -- 172.21.15.60:0/1291530935 shutdown_connections 2024-04-25T08:34:01.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.428+0000 7f3cd6c15640 1 -- 172.21.15.60:0/1291530935 wait complete. 2024-04-25T08:34:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.429+0000 7f3cd6c15640 1 Processor -- start 2024-04-25T08:34:01.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.429+0000 7f3cd6c15640 1 -- start start 2024-04-25T08:34:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd6c15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cd00b2cb0 0x7f3cd01f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd6c15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cd00b92a0 0x7f3cd00cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd6c15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd01f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd6c15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd00d2210 con 0x7f3cd015dbe0 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd6c15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd00d1a00 con 0x7f3cd00b2cb0 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd6c15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd00b6ab0 con 0x7f3cd00b92a0 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd51b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd01f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd51b2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd01f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40582/0 (socket says 172.21.15.60:40582) 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.430+0000 7f3cd51b2640 1 -- 172.21.15.60:0/3194383881 learned_addr learned my addr 172.21.15.60:0/3194383881 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd49b1640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cd00b2cb0 0x7f3cd01f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3ccffff640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cd00b92a0 0x7f3cd00cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd51b2640 1 -- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cd00b92a0 msgr2=0x7f3cd00cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd51b2640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cd00b92a0 0x7f3cd00cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd51b2640 1 -- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cd00b2cb0 msgr2=0x7f3cd01f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd51b2640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cd00b2cb0 0x7f3cd01f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd51b2640 1 -- 172.21.15.60:0/3194383881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd01f5340 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd51b2640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd01f3c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c780 tx=0x7f3cc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8017070 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8003da0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd01f4770 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80052b0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.431+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd00cadf0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.432+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd0073990 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.432+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cc801a070 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.433+0000 7f3ccd7fa640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cb406b4d0 0x7f3cb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.433+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cc8089960 con 0x7f3cd015dbe0 2024-04-25T08:34:01.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.433+0000 7f3cd49b1640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cb406b4d0 0x7f3cb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.434+0000 7f3cd49b1640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cb406b4d0 0x7f3cb406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4034000 tx=0x7f3cc402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.437+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cc80586e0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.686+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3cd015c5d0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.687+0000 7f3ccd7fa640 1 -- 172.21.15.60:0/3194383881 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1433 (secure 0 0 0) 0x7f3cc8057fe0 con 0x7f3cd015dbe0 2024-04-25T08:34:01.682 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:01.683 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":302,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":83,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":303140864,"bytes_avail":772790972416,"bytes_total":773094113280,"write_bytes_sec":2428,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":48,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":7203}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T08:33:47.460111+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:34:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.688+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cb406b4d0 msgr2=0x7f3cb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.688+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cb406b4d0 0x7f3cb406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4034000 tx=0x7f3cc402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:01.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.688+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 msgr2=0x7f3cd01f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.688+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd01f3c80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c780 tx=0x7f3cc800cc40 comp rx=0 tx=0).stop 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 shutdown_connections 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cd015dbe0 0x7f3cd01f3c80 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3cd00b92a0 0x7f3cd00cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3cb406b4d0 0x7f3cb406d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 --2- 172.21.15.60:0/3194383881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3cd00b2cb0 0x7f3cd01f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 >> 172.21.15.60:0/3194383881 conn(0x7f3cd0061a20 msgr2=0x7f3cd015caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 shutdown_connections 2024-04-25T08:34:01.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.689+0000 7f3cd6c15640 1 -- 172.21.15.60:0/3194383881 wait complete. 2024-04-25T08:34:01.695 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:34:01.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:01.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.800+0000 7f3c9dc9e640 1 Processor -- start 2024-04-25T08:34:01.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.800+0000 7f3c9dc9e640 1 -- start start 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c9dc9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3c9815db70 0x7f3c98162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c9dc9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 0x7f3c980d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c9dc9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c98162690 con 0x7f3c980b6cb0 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c9dc9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c980ceee0 con 0x7f3c9815db70 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c9dc9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980d1d10 con 0x7f3c980b5c30 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c97fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 0x7f3c980d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c977fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3c9815db70 0x7f3c98162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c97fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 0x7f3c980d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:50912/0 (socket says 172.21.15.60:50912) 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c96ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3c980b6cb0 0x7f3c980b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:53586/0 (socket says 172.21.15.60:53586) 2024-04-25T08:34:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.801+0000 7f3c97fff640 1 -- 172.21.15.60:0/3056154070 learned_addr learned my addr 172.21.15.60:0/3056154070 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.802+0000 7f3c97fff640 1 -- 172.21.15.60:0/3056154070 >> v2:172.21.15.100:3300/0 conn(0x7f3c9815db70 msgr2=0x7f3c98162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.802+0000 7f3c97fff640 1 --2- 172.21.15.60:0/3056154070 >> v2:172.21.15.100:3300/0 conn(0x7f3c9815db70 0x7f3c98162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.802+0000 7f3c97fff640 1 -- 172.21.15.60:0/3056154070 >> v1:172.21.15.100:6789/0 conn(0x7f3c980b6cb0 legacy=0x7f3c980b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.802+0000 7f3c97fff640 1 -- 172.21.15.60:0/3056154070 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980d1ff0 con 0x7f3c980b5c30 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.802+0000 7f3c977fe640 1 --2- 172.21.15.60:0/3056154070 >> v2:172.21.15.100:3300/0 conn(0x7f3c9815db70 0x7f3c98162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c97fff640 1 --2- 172.21.15.60:0/3056154070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 0x7f3c980d27f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009e50 tx=0x7f3c9002a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7dfb7b001ace243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c967fc640 1 -- 172.21.15.60:0/3056154070 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90036070 con 0x7f3c980b5c30 2024-04-25T08:34:01.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c967fc640 1 -- 172.21.15.60:0/3056154070 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90031600 con 0x7f3c980b5c30 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c967fc640 1 -- 172.21.15.60:0/3056154070 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c900319d0 con 0x7f3c980b5c30 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/3056154070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 msgr2=0x7f3c980d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/3056154070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 0x7f3c980d27f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009e50 tx=0x7f3c9002a730 comp rx=0 tx=0).stop 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/3056154070 shutdown_connections 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/3056154070 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c980b5c30 0x7f3c980d27f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/3056154070 >> v2:172.21.15.100:3300/0 conn(0x7f3c9815db70 0x7f3c98162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/3056154070 >> 172.21.15.60:0/3056154070 conn(0x7f3c980d0520 msgr2=0x7f3c980d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/3056154070 shutdown_connections 2024-04-25T08:34:01.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.803+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/3056154070 wait complete. 2024-04-25T08:34:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 Processor -- start 2024-04-25T08:34:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 -- start start 2024-04-25T08:34:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 0x7f3c980c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c980b6cb0 0x7f3c9815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c9815db70 0x7f3c9815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980ce530 con 0x7f3c980b5c30 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9815df50 con 0x7f3c980b6cb0 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.805+0000 7f3c9dc9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980cf1c0 con 0x7f3c9815db70 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 0x7f3c980c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 0x7f3c980c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56652/0 (socket says 172.21.15.60:56652) 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 -- 172.21.15.60:0/2494383052 learned_addr learned my addr 172.21.15.60:0/2494383052 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c96ffd640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c980b6cb0 0x7f3c9815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c97fff640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c9815db70 0x7f3c9815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 -- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c9815db70 msgr2=0x7f3c9815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c9815db70 0x7f3c9815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 -- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c980b6cb0 msgr2=0x7f3c9815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c980b6cb0 0x7f3c9815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 -- 172.21.15.60:0/2494383052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9815ea60 con 0x7f3c980b5c30 2024-04-25T08:34:01.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c977fe640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 0x7f3c980c96c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ebe0 tx=0x7f3c8800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88017070 con 0x7f3c980b5c30 2024-04-25T08:34:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8800cc30 con 0x7f3c980b5c30 2024-04-25T08:34:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9815ecd0 con 0x7f3c980b5c30 2024-04-25T08:34:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88013470 con 0x7f3c980b5c30 2024-04-25T08:34:01.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.806+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9815f290 con 0x7f3c980b5c30 2024-04-25T08:34:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.807+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c980c9c60 con 0x7f3c980b5c30 2024-04-25T08:34:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.808+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c8801a070 con 0x7f3c980b5c30 2024-04-25T08:34:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.808+0000 7f3c7ffff640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c8406b4d0 0x7f3c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.808+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3c8808a300 con 0x7f3c980b5c30 2024-04-25T08:34:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.808+0000 7f3c96ffd640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c8406b4d0 0x7f3c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.809+0000 7f3c96ffd640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c8406b4d0 0x7f3c8406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3c9806cf40 tx=0x7f3c8c007c80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:01.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.812+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c88058ff0 con 0x7f3c980b5c30 2024-04-25T08:34:01.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.989+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c98073990 con 0x7f3c8406b4d0 2024-04-25T08:34:01.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.996+0000 7f3c7ffff640 1 -- 172.21.15.60:0/2494383052 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252082 (secure 0 0 0) 0x7f3c98073990 con 0x7f3c8406b4d0 2024-04-25T08:34:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:01.995 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c8406b4d0 msgr2=0x7f3c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c8406b4d0 0x7f3c8406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3c9806cf40 tx=0x7f3c8c007c80 comp rx=0 tx=0).stop 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 msgr2=0x7f3c980c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 0x7f3c980c96c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800ebe0 tx=0x7f3c8800c520 comp rx=0 tx=0).stop 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 shutdown_connections 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c9815db70 0x7f3c9815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c8406b4d0 0x7f3c8406d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c980b6cb0 0x7f3c9815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 --2- 172.21.15.60:0/2494383052 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c980b5c30 0x7f3c980c96c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 >> 172.21.15.60:0/2494383052 conn(0x7f3c980d0520 msgr2=0x7f3c981599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 shutdown_connections 2024-04-25T08:34:01.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:01.999+0000 7f3c9dc9e640 1 -- 172.21.15.60:0/2494383052 wait complete. 2024-04-25T08:34:02.005 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-04-25T08:34:01.463246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296036,"kb_used_data":9128,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754678684,"statfs":{"total":773094113280,"available":772790972416,"internally_reserved":0,"allocated":9347072,"data_stored":4910396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.904136"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.594955+0000","last_change":"2024-04-25T08:33:57.598905+0000","last_active":"2024-04-25T08:33:58.594955+0000","last_peered":"2024-04-25T08:33:58.594955+0000","last_clean":"2024-04-25T08:33:58.594955+0000","last_became_active":"2024-04-25T08:33:57.598604+0000","last_became_peered":"2024-04-25T08:33:57.598604+0000","last_unstale":"2024-04-25T08:33:58.594955+0000","last_undegraded":"2024-04-25T08:33:58.594955+0000","last_fullsized":"2024-04-25T08:33:58.594955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:06.644729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579347+0000","last_change":"2024-04-25T08:33:57.588813+0000","last_active":"2024-04-25T08:33:59.579347+0000","last_peered":"2024-04-25T08:33:59.579347+0000","last_clean":"2024-04-25T08:33:59.579347+0000","last_became_active":"2024-04-25T08:33:57.588517+0000","last_became_peered":"2024-04-25T08:33:57.588517+0000","last_unstale":"2024-04-25T08:33:59.579347+0000","last_undegraded":"2024-04-25T08:33:59.579347+0000","last_fullsized":"2024-04-25T08:33:59.579347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:03.303562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976285+0000","last_change":"2024-04-25T08:33:57.595973+0000","last_active":"2024-04-25T08:33:59.976285+0000","last_peered":"2024-04-25T08:33:59.976285+0000","last_clean":"2024-04-25T08:33:59.976285+0000","last_became_active":"2024-04-25T08:33:57.595703+0000","last_became_peered":"2024-04-25T08:33:57.595703+0000","last_unstale":"2024-04-25T08:33:59.976285+0000","last_undegraded":"2024-04-25T08:33:59.976285+0000","last_fullsized":"2024-04-25T08:33:59.976285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:35.167824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911696+0000","last_change":"2024-04-25T08:33:57.592529+0000","last_active":"2024-04-25T08:34:00.911696+0000","last_peered":"2024-04-25T08:34:00.911696+0000","last_clean":"2024-04-25T08:34:00.911696+0000","last_became_active":"2024-04-25T08:33:57.592251+0000","last_became_peered":"2024-04-25T08:33:57.592251+0000","last_unstale":"2024-04-25T08:34:00.911696+0000","last_undegraded":"2024-04-25T08:34:00.911696+0000","last_fullsized":"2024-04-25T08:34:00.911696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581515+0000","last_change":"2024-04-25T08:33:57.599457+0000","last_active":"2024-04-25T08:33:59.581515+0000","last_peered":"2024-04-25T08:33:59.581515+0000","last_clean":"2024-04-25T08:33:59.581515+0000","last_became_active":"2024-04-25T08:33:57.598733+0000","last_became_peered":"2024-04-25T08:33:57.598733+0000","last_unstale":"2024-04-25T08:33:59.581515+0000","last_undegraded":"2024-04-25T08:33:59.581515+0000","last_fullsized":"2024-04-25T08:33:59.581515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:31.378434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590285+0000","last_change":"2024-04-25T08:33:57.602197+0000","last_active":"2024-04-25T08:33:58.590285+0000","last_peered":"2024-04-25T08:33:58.590285+0000","last_clean":"2024-04-25T08:33:58.590285+0000","last_became_active":"2024-04-25T08:33:57.601872+0000","last_became_peered":"2024-04-25T08:33:57.601872+0000","last_unstale":"2024-04-25T08:33:58.590285+0000","last_undegraded":"2024-04-25T08:33:58.590285+0000","last_fullsized":"2024-04-25T08:33:58.590285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595931+0000","last_change":"2024-04-25T08:33:57.585236+0000","last_active":"2024-04-25T08:33:58.595931+0000","last_peered":"2024-04-25T08:33:58.595931+0000","last_clean":"2024-04-25T08:33:58.595931+0000","last_became_active":"2024-04-25T08:33:57.584945+0000","last_became_peered":"2024-04-25T08:33:57.584945+0000","last_unstale":"2024-04-25T08:33:58.595931+0000","last_undegraded":"2024-04-25T08:33:58.595931+0000","last_fullsized":"2024-04-25T08:33:58.595931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:16.228013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581425+0000","last_change":"2024-04-25T08:33:57.598182+0000","last_active":"2024-04-25T08:33:59.581425+0000","last_peered":"2024-04-25T08:33:59.581425+0000","last_clean":"2024-04-25T08:33:59.581425+0000","last_became_active":"2024-04-25T08:33:57.597859+0000","last_became_peered":"2024-04-25T08:33:57.597859+0000","last_unstale":"2024-04-25T08:33:59.581425+0000","last_undegraded":"2024-04-25T08:33:59.581425+0000","last_fullsized":"2024-04-25T08:33:59.581425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:00.789203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170760+0000","last_change":"2024-04-25T08:33:57.595471+0000","last_active":"2024-04-25T08:34:00.170760+0000","last_peered":"2024-04-25T08:34:00.170760+0000","last_clean":"2024-04-25T08:34:00.170760+0000","last_became_active":"2024-04-25T08:33:57.595197+0000","last_became_peered":"2024-04-25T08:33:57.595197+0000","last_unstale":"2024-04-25T08:34:00.170760+0000","last_undegraded":"2024-04-25T08:34:00.170760+0000","last_fullsized":"2024-04-25T08:34:00.170760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:53.820463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911729+0000","last_change":"2024-04-25T08:33:57.597464+0000","last_active":"2024-04-25T08:34:00.911729+0000","last_peered":"2024-04-25T08:34:00.911729+0000","last_clean":"2024-04-25T08:34:00.911729+0000","last_became_active":"2024-04-25T08:33:57.597183+0000","last_became_peered":"2024-04-25T08:33:57.597183+0000","last_unstale":"2024-04-25T08:34:00.911729+0000","last_undegraded":"2024-04-25T08:34:00.911729+0000","last_fullsized":"2024-04-25T08:34:00.911729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:32.031804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579266+0000","last_change":"2024-04-25T08:33:57.588125+0000","last_active":"2024-04-25T08:33:59.579266+0000","last_peered":"2024-04-25T08:33:59.579266+0000","last_clean":"2024-04-25T08:33:59.579266+0000","last_became_active":"2024-04-25T08:33:57.587846+0000","last_became_peered":"2024-04-25T08:33:57.587846+0000","last_unstale":"2024-04-25T08:33:59.579266+0000","last_undegraded":"2024-04-25T08:33:59.579266+0000","last_fullsized":"2024-04-25T08:33:59.579266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:27.134575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911733+0000","last_change":"2024-04-25T08:33:57.595123+0000","last_active":"2024-04-25T08:34:00.911733+0000","last_peered":"2024-04-25T08:34:00.911733+0000","last_clean":"2024-04-25T08:34:00.911733+0000","last_became_active":"2024-04-25T08:33:57.594793+0000","last_became_peered":"2024-04-25T08:33:57.594793+0000","last_unstale":"2024-04-25T08:34:00.911733+0000","last_undegraded":"2024-04-25T08:34:00.911733+0000","last_fullsized":"2024-04-25T08:34:00.911733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.699554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581411+0000","last_change":"2024-04-25T08:33:57.598427+0000","last_active":"2024-04-25T08:33:59.581411+0000","last_peered":"2024-04-25T08:33:59.581411+0000","last_clean":"2024-04-25T08:33:59.581411+0000","last_became_active":"2024-04-25T08:33:57.598025+0000","last_became_peered":"2024-04-25T08:33:57.598025+0000","last_unstale":"2024-04-25T08:33:59.581411+0000","last_undegraded":"2024-04-25T08:33:59.581411+0000","last_fullsized":"2024-04-25T08:33:59.581411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:52.044453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590364+0000","last_change":"2024-04-25T08:33:57.602830+0000","last_active":"2024-04-25T08:33:58.590364+0000","last_peered":"2024-04-25T08:33:58.590364+0000","last_clean":"2024-04-25T08:33:58.590364+0000","last_became_active":"2024-04-25T08:33:57.602446+0000","last_became_peered":"2024-04-25T08:33:57.602446+0000","last_unstale":"2024-04-25T08:33:58.590364+0000","last_undegraded":"2024-04-25T08:33:58.590364+0000","last_fullsized":"2024-04-25T08:33:58.590364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.699554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578856+0000","last_change":"2024-04-25T08:33:57.584756+0000","last_active":"2024-04-25T08:33:59.578856+0000","last_peered":"2024-04-25T08:33:59.578856+0000","last_clean":"2024-04-25T08:33:59.578856+0000","last_became_active":"2024-04-25T08:33:57.584454+0000","last_became_peered":"2024-04-25T08:33:57.584454+0000","last_unstale":"2024-04-25T08:33:59.578856+0000","last_undegraded":"2024-04-25T08:33:59.578856+0000","last_fullsized":"2024-04-25T08:33:59.578856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:32.031804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595012+0000","last_change":"2024-04-25T08:33:57.593526+0000","last_active":"2024-04-25T08:33:58.595012+0000","last_peered":"2024-04-25T08:33:58.595012+0000","last_clean":"2024-04-25T08:33:58.595012+0000","last_became_active":"2024-04-25T08:33:57.593270+0000","last_became_peered":"2024-04-25T08:33:57.593270+0000","last_unstale":"2024-04-25T08:33:58.595012+0000","last_undegraded":"2024-04-25T08:33:58.595012+0000","last_fullsized":"2024-04-25T08:33:58.595012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:14.276030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581662+0000","last_change":"2024-04-25T08:33:57.592230+0000","last_active":"2024-04-25T08:33:59.581662+0000","last_peered":"2024-04-25T08:33:59.581662+0000","last_clean":"2024-04-25T08:33:59.581662+0000","last_became_active":"2024-04-25T08:33:57.591926+0000","last_became_peered":"2024-04-25T08:33:57.591926+0000","last_unstale":"2024-04-25T08:33:59.581662+0000","last_undegraded":"2024-04-25T08:33:59.581662+0000","last_fullsized":"2024-04-25T08:33:59.581662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:21.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976191+0000","last_change":"2024-04-25T08:33:57.585568+0000","last_active":"2024-04-25T08:33:59.976191+0000","last_peered":"2024-04-25T08:33:59.976191+0000","last_clean":"2024-04-25T08:33:59.976191+0000","last_became_active":"2024-04-25T08:33:57.585295+0000","last_became_peered":"2024-04-25T08:33:57.585295+0000","last_unstale":"2024-04-25T08:33:59.976191+0000","last_undegraded":"2024-04-25T08:33:59.976191+0000","last_fullsized":"2024-04-25T08:33:59.976191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:16.228013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590219+0000","last_change":"2024-04-25T08:33:57.605299+0000","last_active":"2024-04-25T08:33:58.590219+0000","last_peered":"2024-04-25T08:33:58.590219+0000","last_clean":"2024-04-25T08:33:58.590219+0000","last_became_active":"2024-04-25T08:33:57.604951+0000","last_became_peered":"2024-04-25T08:33:57.604951+0000","last_unstale":"2024-04-25T08:33:58.590219+0000","last_undegraded":"2024-04-25T08:33:58.590219+0000","last_fullsized":"2024-04-25T08:33:58.590219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.617051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581494+0000","last_change":"2024-04-25T08:33:57.587022+0000","last_active":"2024-04-25T08:33:59.581494+0000","last_peered":"2024-04-25T08:33:59.581494+0000","last_clean":"2024-04-25T08:33:59.581494+0000","last_became_active":"2024-04-25T08:33:57.586731+0000","last_became_peered":"2024-04-25T08:33:57.586731+0000","last_unstale":"2024-04-25T08:33:59.581494+0000","last_undegraded":"2024-04-25T08:33:59.581494+0000","last_fullsized":"2024-04-25T08:33:59.581494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:06.305916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170652+0000","last_change":"2024-04-25T08:33:57.586019+0000","last_active":"2024-04-25T08:34:00.170652+0000","last_peered":"2024-04-25T08:34:00.170652+0000","last_clean":"2024-04-25T08:34:00.170652+0000","last_became_active":"2024-04-25T08:33:57.585762+0000","last_became_peered":"2024-04-25T08:33:57.585762+0000","last_unstale":"2024-04-25T08:34:00.170652+0000","last_undegraded":"2024-04-25T08:34:00.170652+0000","last_fullsized":"2024-04-25T08:34:00.170652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:14.819343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581544+0000","last_change":"2024-04-25T08:33:57.598204+0000","last_active":"2024-04-25T08:33:59.581544+0000","last_peered":"2024-04-25T08:33:59.581544+0000","last_clean":"2024-04-25T08:33:59.581544+0000","last_became_active":"2024-04-25T08:33:57.597874+0000","last_became_peered":"2024-04-25T08:33:57.597874+0000","last_unstale":"2024-04-25T08:33:59.581544+0000","last_undegraded":"2024-04-25T08:33:59.581544+0000","last_fullsized":"2024-04-25T08:33:59.581544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:23.873785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170631+0000","last_change":"2024-04-25T08:33:57.585867+0000","last_active":"2024-04-25T08:34:00.170631+0000","last_peered":"2024-04-25T08:34:00.170631+0000","last_clean":"2024-04-25T08:34:00.170631+0000","last_became_active":"2024-04-25T08:33:57.585567+0000","last_became_peered":"2024-04-25T08:33:57.585567+0000","last_unstale":"2024-04-25T08:34:00.170631+0000","last_undegraded":"2024-04-25T08:34:00.170631+0000","last_fullsized":"2024-04-25T08:34:00.170631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:53.092263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578849+0000","last_change":"2024-04-25T08:33:57.594139+0000","last_active":"2024-04-25T08:33:59.578849+0000","last_peered":"2024-04-25T08:33:59.578849+0000","last_clean":"2024-04-25T08:33:59.578849+0000","last_became_active":"2024-04-25T08:33:57.593814+0000","last_became_peered":"2024-04-25T08:33:57.593814+0000","last_unstale":"2024-04-25T08:33:59.578849+0000","last_undegraded":"2024-04-25T08:33:59.578849+0000","last_fullsized":"2024-04-25T08:33:59.578849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579246+0000","last_change":"2024-04-25T08:33:57.595568+0000","last_active":"2024-04-25T08:33:59.579246+0000","last_peered":"2024-04-25T08:33:59.579246+0000","last_clean":"2024-04-25T08:33:59.579246+0000","last_became_active":"2024-04-25T08:33:57.595295+0000","last_became_peered":"2024-04-25T08:33:57.595295+0000","last_unstale":"2024-04-25T08:33:59.579246+0000","last_undegraded":"2024-04-25T08:33:59.579246+0000","last_fullsized":"2024-04-25T08:33:59.579246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.930380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911853+0000","last_change":"2024-04-25T08:33:57.595454+0000","last_active":"2024-04-25T08:34:00.911853+0000","last_peered":"2024-04-25T08:34:00.911853+0000","last_clean":"2024-04-25T08:34:00.911853+0000","last_became_active":"2024-04-25T08:33:57.595172+0000","last_became_peered":"2024-04-25T08:33:57.595172+0000","last_unstale":"2024-04-25T08:34:00.911853+0000","last_undegraded":"2024-04-25T08:34:00.911853+0000","last_fullsized":"2024-04-25T08:34:00.911853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:06.644729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595305+0000","last_change":"2024-04-25T08:33:57.595885+0000","last_active":"2024-04-25T08:33:58.595305+0000","last_peered":"2024-04-25T08:33:58.595305+0000","last_clean":"2024-04-25T08:33:58.595305+0000","last_became_active":"2024-04-25T08:33:57.595610+0000","last_became_peered":"2024-04-25T08:33:57.595610+0000","last_unstale":"2024-04-25T08:33:58.595305+0000","last_undegraded":"2024-04-25T08:33:58.595305+0000","last_fullsized":"2024-04-25T08:33:58.595305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:27.134575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581686+0000","last_change":"2024-04-25T08:33:57.586894+0000","last_active":"2024-04-25T08:33:59.581686+0000","last_peered":"2024-04-25T08:33:59.581686+0000","last_clean":"2024-04-25T08:33:59.581686+0000","last_became_active":"2024-04-25T08:33:57.586596+0000","last_became_peered":"2024-04-25T08:33:57.586596+0000","last_unstale":"2024-04-25T08:33:59.581686+0000","last_undegraded":"2024-04-25T08:33:59.581686+0000","last_fullsized":"2024-04-25T08:33:59.581686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:24.887648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579075+0000","last_change":"2024-04-25T08:33:57.593189+0000","last_active":"2024-04-25T08:33:59.579075+0000","last_peered":"2024-04-25T08:33:59.579075+0000","last_clean":"2024-04-25T08:33:59.579075+0000","last_became_active":"2024-04-25T08:33:57.592900+0000","last_became_peered":"2024-04-25T08:33:57.592900+0000","last_unstale":"2024-04-25T08:33:59.579075+0000","last_undegraded":"2024-04-25T08:33:59.579075+0000","last_fullsized":"2024-04-25T08:33:59.579075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595185+0000","last_change":"2024-04-25T08:33:57.605041+0000","last_active":"2024-04-25T08:33:58.595185+0000","last_peered":"2024-04-25T08:33:58.595185+0000","last_clean":"2024-04-25T08:33:58.595185+0000","last_became_active":"2024-04-25T08:33:57.604785+0000","last_became_peered":"2024-04-25T08:33:57.604785+0000","last_unstale":"2024-04-25T08:33:58.595185+0000","last_undegraded":"2024-04-25T08:33:58.595185+0000","last_fullsized":"2024-04-25T08:33:58.595185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.930380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976125+0000","last_change":"2024-04-25T08:33:57.594798+0000","last_active":"2024-04-25T08:33:59.976125+0000","last_peered":"2024-04-25T08:33:59.976125+0000","last_clean":"2024-04-25T08:33:59.976125+0000","last_became_active":"2024-04-25T08:33:57.594508+0000","last_became_peered":"2024-04-25T08:33:57.594508+0000","last_unstale":"2024-04-25T08:33:59.976125+0000","last_undegraded":"2024-04-25T08:33:59.976125+0000","last_fullsized":"2024-04-25T08:33:59.976125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:10.270627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.594846+0000","last_change":"2024-04-25T08:33:57.598962+0000","last_active":"2024-04-25T08:33:58.594846+0000","last_peered":"2024-04-25T08:33:58.594846+0000","last_clean":"2024-04-25T08:33:58.594846+0000","last_became_active":"2024-04-25T08:33:57.598505+0000","last_became_peered":"2024-04-25T08:33:57.598505+0000","last_unstale":"2024-04-25T08:33:58.594846+0000","last_undegraded":"2024-04-25T08:33:58.594846+0000","last_fullsized":"2024-04-25T08:33:58.594846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976124+0000","last_change":"2024-04-25T08:33:57.584864+0000","last_active":"2024-04-25T08:33:59.976124+0000","last_peered":"2024-04-25T08:33:59.976124+0000","last_clean":"2024-04-25T08:33:59.976124+0000","last_became_active":"2024-04-25T08:33:57.584585+0000","last_became_peered":"2024-04-25T08:33:57.584585+0000","last_unstale":"2024-04-25T08:33:59.976124+0000","last_undegraded":"2024-04-25T08:33:59.976124+0000","last_fullsized":"2024-04-25T08:33:59.976124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:41.938128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581692+0000","last_change":"2024-04-25T08:33:56.578809+0000","last_active":"2024-04-25T08:33:59.581692+0000","last_peered":"2024-04-25T08:33:59.581692+0000","last_clean":"2024-04-25T08:33:59.581692+0000","last_became_active":"2024-04-25T08:33:56.578466+0000","last_became_peered":"2024-04-25T08:33:56.578466+0000","last_unstale":"2024-04-25T08:33:59.581692+0000","last_undegraded":"2024-04-25T08:33:59.581692+0000","last_fullsized":"2024-04-25T08:33:59.581692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:24.033056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578927+0000","last_change":"2024-04-25T08:33:57.593366+0000","last_active":"2024-04-25T08:33:59.578927+0000","last_peered":"2024-04-25T08:33:59.578927+0000","last_clean":"2024-04-25T08:33:59.578927+0000","last_became_active":"2024-04-25T08:33:57.593069+0000","last_became_peered":"2024-04-25T08:33:57.593069+0000","last_unstale":"2024-04-25T08:33:59.578927+0000","last_undegraded":"2024-04-25T08:33:59.578927+0000","last_fullsized":"2024-04-25T08:33:59.578927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.699554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595096+0000","last_change":"2024-04-25T08:33:56.577451+0000","last_active":"2024-04-25T08:33:58.595096+0000","last_peered":"2024-04-25T08:33:58.595096+0000","last_clean":"2024-04-25T08:33:58.595096+0000","last_became_active":"2024-04-25T08:33:56.577182+0000","last_became_peered":"2024-04-25T08:33:56.577182+0000","last_unstale":"2024-04-25T08:33:58.595096+0000","last_undegraded":"2024-04-25T08:33:58.595096+0000","last_fullsized":"2024-04-25T08:33:58.595096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:23.662726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170780+0000","last_change":"2024-04-25T08:33:57.600541+0000","last_active":"2024-04-25T08:34:00.170780+0000","last_peered":"2024-04-25T08:34:00.170780+0000","last_clean":"2024-04-25T08:34:00.170780+0000","last_became_active":"2024-04-25T08:33:57.600239+0000","last_became_peered":"2024-04-25T08:33:57.600239+0000","last_unstale":"2024-04-25T08:34:00.170780+0000","last_undegraded":"2024-04-25T08:34:00.170780+0000","last_fullsized":"2024-04-25T08:34:00.170780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:02.840551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595369+0000","last_change":"2024-04-25T08:33:56.581227+0000","last_active":"2024-04-25T08:33:58.595369+0000","last_peered":"2024-04-25T08:33:58.595369+0000","last_clean":"2024-04-25T08:33:58.595369+0000","last_became_active":"2024-04-25T08:33:56.580846+0000","last_became_peered":"2024-04-25T08:33:56.580846+0000","last_unstale":"2024-04-25T08:33:58.595369+0000","last_undegraded":"2024-04-25T08:33:58.595369+0000","last_fullsized":"2024-04-25T08:33:58.595369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:50.350319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170842+0000","last_change":"2024-04-25T08:33:57.585922+0000","last_active":"2024-04-25T08:34:00.170842+0000","last_peered":"2024-04-25T08:34:00.170842+0000","last_clean":"2024-04-25T08:34:00.170842+0000","last_became_active":"2024-04-25T08:33:57.585623+0000","last_became_peered":"2024-04-25T08:33:57.585623+0000","last_unstale":"2024-04-25T08:34:00.170842+0000","last_undegraded":"2024-04-25T08:34:00.170842+0000","last_fullsized":"2024-04-25T08:34:00.170842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:31.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596125+0000","last_change":"2024-04-25T08:33:56.575045+0000","last_active":"2024-04-25T08:33:58.596125+0000","last_peered":"2024-04-25T08:33:58.596125+0000","last_clean":"2024-04-25T08:33:58.596125+0000","last_became_active":"2024-04-25T08:33:56.574472+0000","last_became_peered":"2024-04-25T08:33:56.574472+0000","last_unstale":"2024-04-25T08:33:58.596125+0000","last_undegraded":"2024-04-25T08:33:58.596125+0000","last_fullsized":"2024-04-25T08:33:58.596125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:48.635135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579315+0000","last_change":"2024-04-25T08:33:57.586041+0000","last_active":"2024-04-25T08:33:59.579315+0000","last_peered":"2024-04-25T08:33:59.579315+0000","last_clean":"2024-04-25T08:33:59.579315+0000","last_became_active":"2024-04-25T08:33:57.585183+0000","last_became_peered":"2024-04-25T08:33:57.585183+0000","last_unstale":"2024-04-25T08:33:59.579315+0000","last_undegraded":"2024-04-25T08:33:59.579315+0000","last_fullsized":"2024-04-25T08:33:59.579315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:47.869752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170813+0000","last_change":"2024-04-25T08:33:56.573057+0000","last_active":"2024-04-25T08:34:00.170813+0000","last_peered":"2024-04-25T08:34:00.170813+0000","last_clean":"2024-04-25T08:34:00.170813+0000","last_became_active":"2024-04-25T08:33:56.572791+0000","last_became_peered":"2024-04-25T08:33:56.572791+0000","last_unstale":"2024-04-25T08:34:00.170813+0000","last_undegraded":"2024-04-25T08:34:00.170813+0000","last_fullsized":"2024-04-25T08:34:00.170813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.350599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579357+0000","last_change":"2024-04-25T08:33:57.585071+0000","last_active":"2024-04-25T08:33:59.579357+0000","last_peered":"2024-04-25T08:33:59.579357+0000","last_clean":"2024-04-25T08:33:59.579357+0000","last_became_active":"2024-04-25T08:33:57.584718+0000","last_became_peered":"2024-04-25T08:33:57.584718+0000","last_unstale":"2024-04-25T08:33:59.579357+0000","last_undegraded":"2024-04-25T08:33:59.579357+0000","last_fullsized":"2024-04-25T08:33:59.579357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.539872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911424+0000","last_change":"2024-04-25T08:33:56.576140+0000","last_active":"2024-04-25T08:34:00.911424+0000","last_peered":"2024-04-25T08:34:00.911424+0000","last_clean":"2024-04-25T08:34:00.911424+0000","last_became_active":"2024-04-25T08:33:56.575855+0000","last_became_peered":"2024-04-25T08:33:56.575855+0000","last_unstale":"2024-04-25T08:34:00.911424+0000","last_undegraded":"2024-04-25T08:34:00.911424+0000","last_fullsized":"2024-04-25T08:34:00.911424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:32.116736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170844+0000","last_change":"2024-04-25T08:33:57.597881+0000","last_active":"2024-04-25T08:34:00.170844+0000","last_peered":"2024-04-25T08:34:00.170844+0000","last_clean":"2024-04-25T08:34:00.170844+0000","last_became_active":"2024-04-25T08:33:57.597510+0000","last_became_peered":"2024-04-25T08:33:57.597510+0000","last_unstale":"2024-04-25T08:34:00.170844+0000","last_undegraded":"2024-04-25T08:34:00.170844+0000","last_fullsized":"2024-04-25T08:34:00.170844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:18.760694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579060+0000","last_change":"2024-04-25T08:33:56.579617+0000","last_active":"2024-04-25T08:33:59.579060+0000","last_peered":"2024-04-25T08:33:59.579060+0000","last_clean":"2024-04-25T08:33:59.579060+0000","last_became_active":"2024-04-25T08:33:56.579378+0000","last_became_peered":"2024-04-25T08:33:56.579378+0000","last_unstale":"2024-04-25T08:33:59.579060+0000","last_undegraded":"2024-04-25T08:33:59.579060+0000","last_fullsized":"2024-04-25T08:33:59.579060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:57.861141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596614+0000","last_change":"2024-04-25T08:33:57.590229+0000","last_active":"2024-04-25T08:33:58.596614+0000","last_peered":"2024-04-25T08:33:58.596614+0000","last_clean":"2024-04-25T08:33:58.596614+0000","last_became_active":"2024-04-25T08:33:57.589126+0000","last_became_peered":"2024-04-25T08:33:57.589126+0000","last_unstale":"2024-04-25T08:33:58.596614+0000","last_undegraded":"2024-04-25T08:33:58.596614+0000","last_fullsized":"2024-04-25T08:33:58.596614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:33.121751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581442+0000","last_change":"2024-04-25T08:33:56.577305+0000","last_active":"2024-04-25T08:33:59.581442+0000","last_peered":"2024-04-25T08:33:59.581442+0000","last_clean":"2024-04-25T08:33:59.581442+0000","last_became_active":"2024-04-25T08:33:56.577000+0000","last_became_peered":"2024-04-25T08:33:56.577000+0000","last_unstale":"2024-04-25T08:33:59.581442+0000","last_undegraded":"2024-04-25T08:33:59.581442+0000","last_fullsized":"2024-04-25T08:33:59.581442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:04.612385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578798+0000","last_change":"2024-04-25T08:33:57.595504+0000","last_active":"2024-04-25T08:33:59.578798+0000","last_peered":"2024-04-25T08:33:59.578798+0000","last_clean":"2024-04-25T08:33:59.578798+0000","last_became_active":"2024-04-25T08:33:57.595218+0000","last_became_peered":"2024-04-25T08:33:57.595218+0000","last_unstale":"2024-04-25T08:33:59.578798+0000","last_undegraded":"2024-04-25T08:33:59.578798+0000","last_fullsized":"2024-04-25T08:33:59.578798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.617051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595009+0000","last_change":"2024-04-25T08:33:56.579508+0000","last_active":"2024-04-25T08:33:58.595009+0000","last_peered":"2024-04-25T08:33:58.595009+0000","last_clean":"2024-04-25T08:33:58.595009+0000","last_became_active":"2024-04-25T08:33:56.579225+0000","last_became_peered":"2024-04-25T08:33:56.579225+0000","last_unstale":"2024-04-25T08:33:58.595009+0000","last_undegraded":"2024-04-25T08:33:58.595009+0000","last_fullsized":"2024-04-25T08:33:58.595009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:27.575448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911617+0000","last_change":"2024-04-25T08:33:57.586353+0000","last_active":"2024-04-25T08:34:00.911617+0000","last_peered":"2024-04-25T08:34:00.911617+0000","last_clean":"2024-04-25T08:34:00.911617+0000","last_became_active":"2024-04-25T08:33:57.586085+0000","last_became_peered":"2024-04-25T08:33:57.586085+0000","last_unstale":"2024-04-25T08:34:00.911617+0000","last_undegraded":"2024-04-25T08:34:00.911617+0000","last_fullsized":"2024-04-25T08:34:00.911617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.617051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911783+0000","last_change":"2024-04-25T08:33:56.571751+0000","last_active":"2024-04-25T08:34:00.911783+0000","last_peered":"2024-04-25T08:34:00.911783+0000","last_clean":"2024-04-25T08:34:00.911783+0000","last_became_active":"2024-04-25T08:33:56.571479+0000","last_became_peered":"2024-04-25T08:33:56.571479+0000","last_unstale":"2024-04-25T08:34:00.911783+0000","last_undegraded":"2024-04-25T08:34:00.911783+0000","last_fullsized":"2024-04-25T08:34:00.911783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590507+0000","last_change":"2024-04-25T08:33:57.600747+0000","last_active":"2024-04-25T08:33:58.590507+0000","last_peered":"2024-04-25T08:33:58.590507+0000","last_clean":"2024-04-25T08:33:58.590507+0000","last_became_active":"2024-04-25T08:33:57.600422+0000","last_became_peered":"2024-04-25T08:33:57.600422+0000","last_unstale":"2024-04-25T08:33:58.590507+0000","last_undegraded":"2024-04-25T08:33:58.590507+0000","last_fullsized":"2024-04-25T08:33:58.590507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581871+0000","last_change":"2024-04-25T08:33:56.573128+0000","last_active":"2024-04-25T08:33:59.581871+0000","last_peered":"2024-04-25T08:33:59.581871+0000","last_clean":"2024-04-25T08:33:59.581871+0000","last_became_active":"2024-04-25T08:33:56.572850+0000","last_became_peered":"2024-04-25T08:33:56.572850+0000","last_unstale":"2024-04-25T08:33:59.581871+0000","last_undegraded":"2024-04-25T08:33:59.581871+0000","last_fullsized":"2024-04-25T08:33:59.581871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:24.509654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976163+0000","last_change":"2024-04-25T08:33:57.597452+0000","last_active":"2024-04-25T08:33:59.976163+0000","last_peered":"2024-04-25T08:33:59.976163+0000","last_clean":"2024-04-25T08:33:59.976163+0000","last_became_active":"2024-04-25T08:33:57.597168+0000","last_became_peered":"2024-04-25T08:33:57.597168+0000","last_unstale":"2024-04-25T08:33:59.976163+0000","last_undegraded":"2024-04-25T08:33:59.976163+0000","last_fullsized":"2024-04-25T08:33:59.976163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:58.866157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596190+0000","last_change":"2024-04-25T08:33:56.577795+0000","last_active":"2024-04-25T08:33:58.596190+0000","last_peered":"2024-04-25T08:33:58.596190+0000","last_clean":"2024-04-25T08:33:58.596190+0000","last_became_active":"2024-04-25T08:33:56.577512+0000","last_became_peered":"2024-04-25T08:33:56.577512+0000","last_unstale":"2024-04-25T08:33:58.596190+0000","last_undegraded":"2024-04-25T08:33:58.596190+0000","last_fullsized":"2024-04-25T08:33:58.596190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:17.755679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911840+0000","last_change":"2024-04-25T08:33:57.594892+0000","last_active":"2024-04-25T08:34:00.911840+0000","last_peered":"2024-04-25T08:34:00.911840+0000","last_clean":"2024-04-25T08:34:00.911840+0000","last_became_active":"2024-04-25T08:33:57.594600+0000","last_became_peered":"2024-04-25T08:33:57.594600+0000","last_unstale":"2024-04-25T08:34:00.911840+0000","last_undegraded":"2024-04-25T08:34:00.911840+0000","last_fullsized":"2024-04-25T08:34:00.911840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911457+0000","last_change":"2024-04-25T08:33:56.577566+0000","last_active":"2024-04-25T08:34:00.911457+0000","last_peered":"2024-04-25T08:34:00.911457+0000","last_clean":"2024-04-25T08:34:00.911457+0000","last_became_active":"2024-04-25T08:33:56.577291+0000","last_became_peered":"2024-04-25T08:33:56.577291+0000","last_unstale":"2024-04-25T08:34:00.911457+0000","last_undegraded":"2024-04-25T08:34:00.911457+0000","last_fullsized":"2024-04-25T08:34:00.911457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:08.663368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596172+0000","last_change":"2024-04-25T08:33:57.583448+0000","last_active":"2024-04-25T08:33:58.596172+0000","last_peered":"2024-04-25T08:33:58.596172+0000","last_clean":"2024-04-25T08:33:58.596172+0000","last_became_active":"2024-04-25T08:33:57.583081+0000","last_became_peered":"2024-04-25T08:33:57.583081+0000","last_unstale":"2024-04-25T08:33:58.596172+0000","last_undegraded":"2024-04-25T08:33:58.596172+0000","last_fullsized":"2024-04-25T08:33:58.596172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:35.167824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595261+0000","last_change":"2024-04-25T08:33:56.580851+0000","last_active":"2024-04-25T08:33:58.595261+0000","last_peered":"2024-04-25T08:33:58.595261+0000","last_clean":"2024-04-25T08:33:58.595261+0000","last_became_active":"2024-04-25T08:33:56.580557+0000","last_became_peered":"2024-04-25T08:33:56.580557+0000","last_unstale":"2024-04-25T08:33:58.595261+0000","last_undegraded":"2024-04-25T08:33:58.595261+0000","last_fullsized":"2024-04-25T08:33:58.595261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:32.116736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596498+0000","last_change":"2024-04-25T08:33:57.592966+0000","last_active":"2024-04-25T08:33:58.596498+0000","last_peered":"2024-04-25T08:33:58.596498+0000","last_clean":"2024-04-25T08:33:58.596498+0000","last_became_active":"2024-04-25T08:33:57.592647+0000","last_became_peered":"2024-04-25T08:33:57.592647+0000","last_unstale":"2024-04-25T08:33:58.596498+0000","last_undegraded":"2024-04-25T08:33:58.596498+0000","last_fullsized":"2024-04-25T08:33:58.596498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:13.093212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581345+0000","last_change":"2024-04-25T08:33:56.575918+0000","last_active":"2024-04-25T08:33:59.581345+0000","last_peered":"2024-04-25T08:33:59.581345+0000","last_clean":"2024-04-25T08:33:59.581345+0000","last_became_active":"2024-04-25T08:33:56.575649+0000","last_became_peered":"2024-04-25T08:33:56.575649+0000","last_unstale":"2024-04-25T08:33:59.581345+0000","last_undegraded":"2024-04-25T08:33:59.581345+0000","last_fullsized":"2024-04-25T08:33:59.581345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.360965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596616+0000","last_change":"2024-04-25T08:33:57.592922+0000","last_active":"2024-04-25T08:33:58.596616+0000","last_peered":"2024-04-25T08:33:58.596616+0000","last_clean":"2024-04-25T08:33:58.596616+0000","last_became_active":"2024-04-25T08:33:57.592604+0000","last_became_peered":"2024-04-25T08:33:57.592604+0000","last_unstale":"2024-04-25T08:33:58.596616+0000","last_undegraded":"2024-04-25T08:33:58.596616+0000","last_fullsized":"2024-04-25T08:33:58.596616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:32.191378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578991+0000","last_change":"2024-04-25T08:33:56.571020+0000","last_active":"2024-04-25T08:33:59.578991+0000","last_peered":"2024-04-25T08:33:59.578991+0000","last_clean":"2024-04-25T08:33:59.578991+0000","last_became_active":"2024-04-25T08:33:56.570742+0000","last_became_peered":"2024-04-25T08:33:56.570742+0000","last_unstale":"2024-04-25T08:33:59.578991+0000","last_undegraded":"2024-04-25T08:33:59.578991+0000","last_fullsized":"2024-04-25T08:33:59.578991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:56.865380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590032+0000","last_change":"2024-04-25T08:33:57.605305+0000","last_active":"2024-04-25T08:33:58.590032+0000","last_peered":"2024-04-25T08:33:58.590032+0000","last_clean":"2024-04-25T08:33:58.590032+0000","last_became_active":"2024-04-25T08:33:57.604956+0000","last_became_peered":"2024-04-25T08:33:57.604956+0000","last_unstale":"2024-04-25T08:33:58.590032+0000","last_undegraded":"2024-04-25T08:33:58.590032+0000","last_fullsized":"2024-04-25T08:33:58.590032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:25.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595276+0000","last_change":"2024-04-25T08:33:56.578816+0000","last_active":"2024-04-25T08:33:58.595276+0000","last_peered":"2024-04-25T08:33:58.595276+0000","last_clean":"2024-04-25T08:33:58.595276+0000","last_became_active":"2024-04-25T08:33:56.578552+0000","last_became_peered":"2024-04-25T08:33:56.578552+0000","last_unstale":"2024-04-25T08:33:58.595276+0000","last_undegraded":"2024-04-25T08:33:58.595276+0000","last_fullsized":"2024-04-25T08:33:58.595276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:21.914391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581289+0000","last_change":"2024-04-25T08:33:57.591560+0000","last_active":"2024-04-25T08:33:59.581289+0000","last_peered":"2024-04-25T08:33:59.581289+0000","last_clean":"2024-04-25T08:33:59.581289+0000","last_became_active":"2024-04-25T08:33:57.591295+0000","last_became_peered":"2024-04-25T08:33:57.591295+0000","last_unstale":"2024-04-25T08:33:59.581289+0000","last_undegraded":"2024-04-25T08:33:59.581289+0000","last_fullsized":"2024-04-25T08:33:59.581289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:58.860879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170900+0000","last_change":"2024-04-25T08:33:56.577328+0000","last_active":"2024-04-25T08:34:00.170900+0000","last_peered":"2024-04-25T08:34:00.170900+0000","last_clean":"2024-04-25T08:34:00.170900+0000","last_became_active":"2024-04-25T08:33:56.577061+0000","last_became_peered":"2024-04-25T08:33:56.577061+0000","last_unstale":"2024-04-25T08:34:00.170900+0000","last_undegraded":"2024-04-25T08:34:00.170900+0000","last_fullsized":"2024-04-25T08:34:00.170900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:14.492442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581379+0000","last_change":"2024-04-25T08:33:57.592930+0000","last_active":"2024-04-25T08:33:59.581379+0000","last_peered":"2024-04-25T08:33:59.581379+0000","last_clean":"2024-04-25T08:33:59.581379+0000","last_became_active":"2024-04-25T08:33:57.592665+0000","last_became_peered":"2024-04-25T08:33:57.592665+0000","last_unstale":"2024-04-25T08:33:59.581379+0000","last_undegraded":"2024-04-25T08:33:59.581379+0000","last_fullsized":"2024-04-25T08:33:59.581379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.545740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581873+0000","last_change":"2024-04-25T08:33:56.577330+0000","last_active":"2024-04-25T08:33:59.581873+0000","last_peered":"2024-04-25T08:33:59.581873+0000","last_clean":"2024-04-25T08:33:59.581873+0000","last_became_active":"2024-04-25T08:33:56.577049+0000","last_became_peered":"2024-04-25T08:33:56.577049+0000","last_unstale":"2024-04-25T08:33:59.581873+0000","last_undegraded":"2024-04-25T08:33:59.581873+0000","last_fullsized":"2024-04-25T08:33:59.581873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.628769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590298+0000","last_change":"2024-04-25T08:33:57.606098+0000","last_active":"2024-04-25T08:33:58.590298+0000","last_peered":"2024-04-25T08:33:58.590298+0000","last_clean":"2024-04-25T08:33:58.590298+0000","last_became_active":"2024-04-25T08:33:57.604969+0000","last_became_peered":"2024-04-25T08:33:57.604969+0000","last_unstale":"2024-04-25T08:33:58.590298+0000","last_undegraded":"2024-04-25T08:33:58.590298+0000","last_fullsized":"2024-04-25T08:33:58.590298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:32.031804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976005+0000","last_change":"2024-04-25T08:33:56.569880+0000","last_active":"2024-04-25T08:33:59.976005+0000","last_peered":"2024-04-25T08:33:59.976005+0000","last_clean":"2024-04-25T08:33:59.976005+0000","last_became_active":"2024-04-25T08:33:56.569586+0000","last_became_peered":"2024-04-25T08:33:56.569586+0000","last_unstale":"2024-04-25T08:33:59.976005+0000","last_undegraded":"2024-04-25T08:33:59.976005+0000","last_fullsized":"2024-04-25T08:33:59.976005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:48.635135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579164+0000","last_change":"2024-04-25T08:33:57.594160+0000","last_active":"2024-04-25T08:33:59.579164+0000","last_peered":"2024-04-25T08:33:59.579164+0000","last_clean":"2024-04-25T08:33:59.579164+0000","last_became_active":"2024-04-25T08:33:57.593858+0000","last_became_peered":"2024-04-25T08:33:57.593858+0000","last_unstale":"2024-04-25T08:33:59.579164+0000","last_undegraded":"2024-04-25T08:33:59.579164+0000","last_fullsized":"2024-04-25T08:33:59.579164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:06.644729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590361+0000","last_change":"2024-04-25T08:33:56.575983+0000","last_active":"2024-04-25T08:33:58.590361+0000","last_peered":"2024-04-25T08:33:58.590361+0000","last_clean":"2024-04-25T08:33:58.590361+0000","last_became_active":"2024-04-25T08:33:56.575727+0000","last_became_peered":"2024-04-25T08:33:56.575727+0000","last_unstale":"2024-04-25T08:33:58.590361+0000","last_undegraded":"2024-04-25T08:33:58.590361+0000","last_fullsized":"2024-04-25T08:33:58.590361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:12.088197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579406+0000","last_change":"2024-04-25T08:33:57.587533+0000","last_active":"2024-04-25T08:33:59.579406+0000","last_peered":"2024-04-25T08:33:59.579406+0000","last_clean":"2024-04-25T08:33:59.579406+0000","last_became_active":"2024-04-25T08:33:57.587244+0000","last_became_peered":"2024-04-25T08:33:57.587244+0000","last_unstale":"2024-04-25T08:33:59.579406+0000","last_undegraded":"2024-04-25T08:33:59.579406+0000","last_fullsized":"2024-04-25T08:33:59.579406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:11.916701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590071+0000","last_change":"2024-04-25T08:33:56.576695+0000","last_active":"2024-04-25T08:33:58.590071+0000","last_peered":"2024-04-25T08:33:58.590071+0000","last_clean":"2024-04-25T08:33:58.590071+0000","last_became_active":"2024-04-25T08:33:56.576181+0000","last_became_peered":"2024-04-25T08:33:56.576181+0000","last_unstale":"2024-04-25T08:33:58.590071+0000","last_undegraded":"2024-04-25T08:33:58.590071+0000","last_fullsized":"2024-04-25T08:33:58.590071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:08.663368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596725+0000","last_change":"2024-04-25T08:33:57.590013+0000","last_active":"2024-04-25T08:33:58.596725+0000","last_peered":"2024-04-25T08:33:58.596725+0000","last_clean":"2024-04-25T08:33:58.596725+0000","last_became_active":"2024-04-25T08:33:57.588434+0000","last_became_peered":"2024-04-25T08:33:57.588434+0000","last_unstale":"2024-04-25T08:33:58.596725+0000","last_undegraded":"2024-04-25T08:33:58.596725+0000","last_fullsized":"2024-04-25T08:33:58.596725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.667742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590105+0000","last_change":"2024-04-25T08:33:56.576592+0000","last_active":"2024-04-25T08:33:58.590105+0000","last_peered":"2024-04-25T08:33:58.590105+0000","last_clean":"2024-04-25T08:33:58.590105+0000","last_became_active":"2024-04-25T08:33:56.576332+0000","last_became_peered":"2024-04-25T08:33:56.576332+0000","last_unstale":"2024-04-25T08:33:58.590105+0000","last_undegraded":"2024-04-25T08:33:58.590105+0000","last_fullsized":"2024-04-25T08:33:58.590105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:32.116736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170780+0000","last_change":"2024-04-25T08:33:57.591109+0000","last_active":"2024-04-25T08:34:00.170780+0000","last_peered":"2024-04-25T08:34:00.170780+0000","last_clean":"2024-04-25T08:34:00.170780+0000","last_became_active":"2024-04-25T08:33:57.590816+0000","last_became_peered":"2024-04-25T08:33:57.590816+0000","last_unstale":"2024-04-25T08:34:00.170780+0000","last_undegraded":"2024-04-25T08:34:00.170780+0000","last_fullsized":"2024-04-25T08:34:00.170780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:49.640150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911704+0000","last_change":"2024-04-25T08:33:56.576983+0000","last_active":"2024-04-25T08:34:00.911704+0000","last_peered":"2024-04-25T08:34:00.911704+0000","last_clean":"2024-04-25T08:34:00.911704+0000","last_became_active":"2024-04-25T08:33:56.576721+0000","last_became_peered":"2024-04-25T08:33:56.576721+0000","last_unstale":"2024-04-25T08:34:00.911704+0000","last_undegraded":"2024-04-25T08:34:00.911704+0000","last_fullsized":"2024-04-25T08:34:00.911704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:12.088197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596050+0000","last_change":"2024-04-25T08:33:57.588606+0000","last_active":"2024-04-25T08:33:58.596050+0000","last_peered":"2024-04-25T08:33:58.596050+0000","last_clean":"2024-04-25T08:33:58.596050+0000","last_became_active":"2024-04-25T08:33:57.588320+0000","last_became_peered":"2024-04-25T08:33:57.588320+0000","last_unstale":"2024-04-25T08:33:58.596050+0000","last_undegraded":"2024-04-25T08:33:58.596050+0000","last_fullsized":"2024-04-25T08:33:58.596050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.870396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.580699+0000","last_change":"2024-04-25T08:33:56.583729+0000","last_active":"2024-04-25T08:33:59.580699+0000","last_peered":"2024-04-25T08:33:59.580699+0000","last_clean":"2024-04-25T08:33:59.580699+0000","last_became_active":"2024-04-25T08:33:56.583455+0000","last_became_peered":"2024-04-25T08:33:56.583455+0000","last_unstale":"2024-04-25T08:33:59.580699+0000","last_undegraded":"2024-04-25T08:33:59.580699+0000","last_fullsized":"2024-04-25T08:33:59.580699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:15.222998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170861+0000","last_change":"2024-04-25T08:33:57.599741+0000","last_active":"2024-04-25T08:34:00.170861+0000","last_peered":"2024-04-25T08:34:00.170861+0000","last_clean":"2024-04-25T08:34:00.170861+0000","last_became_active":"2024-04-25T08:33:57.599416+0000","last_became_peered":"2024-04-25T08:33:57.599416+0000","last_unstale":"2024-04-25T08:34:00.170861+0000","last_undegraded":"2024-04-25T08:34:00.170861+0000","last_fullsized":"2024-04-25T08:34:00.170861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:08.134103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595485+0000","last_change":"2024-04-25T08:33:56.574282+0000","last_active":"2024-04-25T08:33:58.595485+0000","last_peered":"2024-04-25T08:33:58.595485+0000","last_clean":"2024-04-25T08:33:58.595485+0000","last_became_active":"2024-04-25T08:33:56.573840+0000","last_became_peered":"2024-04-25T08:33:56.573840+0000","last_unstale":"2024-04-25T08:33:58.595485+0000","last_undegraded":"2024-04-25T08:33:58.595485+0000","last_fullsized":"2024-04-25T08:33:58.595485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:28.209229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579177+0000","last_change":"2024-04-25T08:33:57.604584+0000","last_active":"2024-04-25T08:33:59.579177+0000","last_peered":"2024-04-25T08:33:59.579177+0000","last_clean":"2024-04-25T08:33:59.579177+0000","last_became_active":"2024-04-25T08:33:57.604055+0000","last_became_peered":"2024-04-25T08:33:57.604055+0000","last_unstale":"2024-04-25T08:33:59.579177+0000","last_undegraded":"2024-04-25T08:33:59.579177+0000","last_fullsized":"2024-04-25T08:33:59.579177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:14.276030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.594880+0000","last_change":"2024-04-25T08:33:56.576675+0000","last_active":"2024-04-25T08:33:58.594880+0000","last_peered":"2024-04-25T08:33:58.594880+0000","last_clean":"2024-04-25T08:33:58.594880+0000","last_became_active":"2024-04-25T08:33:56.576403+0000","last_became_peered":"2024-04-25T08:33:56.576403+0000","last_unstale":"2024-04-25T08:33:58.594880+0000","last_undegraded":"2024-04-25T08:33:58.594880+0000","last_fullsized":"2024-04-25T08:33:58.594880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596129+0000","last_change":"2024-04-25T08:33:57.592225+0000","last_active":"2024-04-25T08:33:58.596129+0000","last_peered":"2024-04-25T08:33:58.596129+0000","last_clean":"2024-04-25T08:33:58.596129+0000","last_became_active":"2024-04-25T08:33:57.591964+0000","last_became_peered":"2024-04-25T08:33:57.591964+0000","last_unstale":"2024-04-25T08:33:58.596129+0000","last_undegraded":"2024-04-25T08:33:58.596129+0000","last_fullsized":"2024-04-25T08:33:58.596129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.203855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595875+0000","last_change":"2024-04-25T08:33:56.573583+0000","last_active":"2024-04-25T08:33:58.595875+0000","last_peered":"2024-04-25T08:33:58.595875+0000","last_clean":"2024-04-25T08:33:58.595875+0000","last_became_active":"2024-04-25T08:33:56.573325+0000","last_became_peered":"2024-04-25T08:33:56.573325+0000","last_unstale":"2024-04-25T08:33:58.595875+0000","last_undegraded":"2024-04-25T08:33:58.595875+0000","last_fullsized":"2024-04-25T08:33:58.595875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.165536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596276+0000","last_change":"2024-04-25T08:33:57.585403+0000","last_active":"2024-04-25T08:33:58.596276+0000","last_peered":"2024-04-25T08:33:58.596276+0000","last_clean":"2024-04-25T08:33:58.596276+0000","last_became_active":"2024-04-25T08:33:57.584657+0000","last_became_peered":"2024-04-25T08:33:57.584657+0000","last_unstale":"2024-04-25T08:33:58.596276+0000","last_undegraded":"2024-04-25T08:33:58.596276+0000","last_fullsized":"2024-04-25T08:33:58.596276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:09.668384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581507+0000","last_change":"2024-04-25T08:33:56.577808+0000","last_active":"2024-04-25T08:33:59.581507+0000","last_peered":"2024-04-25T08:33:59.581507+0000","last_clean":"2024-04-25T08:33:59.581507+0000","last_became_active":"2024-04-25T08:33:56.577539+0000","last_became_peered":"2024-04-25T08:33:56.577539+0000","last_unstale":"2024-04-25T08:33:59.581507+0000","last_undegraded":"2024-04-25T08:33:59.581507+0000","last_fullsized":"2024-04-25T08:33:59.581507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:32.170181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976204+0000","last_change":"2024-04-25T08:33:57.596692+0000","last_active":"2024-04-25T08:33:59.976204+0000","last_peered":"2024-04-25T08:33:59.976204+0000","last_clean":"2024-04-25T08:33:59.976204+0000","last_became_active":"2024-04-25T08:33:57.596423+0000","last_became_peered":"2024-04-25T08:33:57.596423+0000","last_unstale":"2024-04-25T08:33:59.976204+0000","last_undegraded":"2024-04-25T08:33:59.976204+0000","last_fullsized":"2024-04-25T08:33:59.976204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.870396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595977+0000","last_change":"2024-04-25T08:33:56.571995+0000","last_active":"2024-04-25T08:33:58.595977+0000","last_peered":"2024-04-25T08:33:58.595977+0000","last_clean":"2024-04-25T08:33:58.595977+0000","last_became_active":"2024-04-25T08:33:56.571692+0000","last_became_peered":"2024-04-25T08:33:56.571692+0000","last_unstale":"2024-04-25T08:33:58.595977+0000","last_undegraded":"2024-04-25T08:33:58.595977+0000","last_fullsized":"2024-04-25T08:33:58.595977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:01.835536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976214+0000","last_change":"2024-04-25T08:33:57.593871+0000","last_active":"2024-04-25T08:33:59.976214+0000","last_peered":"2024-04-25T08:33:59.976214+0000","last_clean":"2024-04-25T08:33:59.976214+0000","last_became_active":"2024-04-25T08:33:57.593310+0000","last_became_peered":"2024-04-25T08:33:57.593310+0000","last_unstale":"2024-04-25T08:33:59.976214+0000","last_undegraded":"2024-04-25T08:33:59.976214+0000","last_fullsized":"2024-04-25T08:33:59.976214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.203855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.975880+0000","last_change":"2024-04-25T08:33:56.572227+0000","last_active":"2024-04-25T08:33:59.975880+0000","last_peered":"2024-04-25T08:33:59.975880+0000","last_clean":"2024-04-25T08:33:59.975880+0000","last_became_active":"2024-04-25T08:33:56.571968+0000","last_became_peered":"2024-04-25T08:33:56.571968+0000","last_unstale":"2024-04-25T08:33:59.975880+0000","last_undegraded":"2024-04-25T08:33:59.975880+0000","last_fullsized":"2024-04-25T08:33:59.975880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:01.835536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595552+0000","last_change":"2024-04-25T08:33:57.600355+0000","last_active":"2024-04-25T08:33:58.595552+0000","last_peered":"2024-04-25T08:33:58.595552+0000","last_clean":"2024-04-25T08:33:58.595552+0000","last_became_active":"2024-04-25T08:33:57.600053+0000","last_became_peered":"2024-04-25T08:33:57.600053+0000","last_unstale":"2024-04-25T08:33:58.595552+0000","last_undegraded":"2024-04-25T08:33:58.595552+0000","last_fullsized":"2024-04-25T08:33:58.595552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:47.869752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590534+0000","last_change":"2024-04-25T08:33:56.577402+0000","last_active":"2024-04-25T08:33:58.590534+0000","last_peered":"2024-04-25T08:33:58.590534+0000","last_clean":"2024-04-25T08:33:58.590534+0000","last_became_active":"2024-04-25T08:33:56.576932+0000","last_became_peered":"2024-04-25T08:33:56.576932+0000","last_unstale":"2024-04-25T08:33:58.590534+0000","last_undegraded":"2024-04-25T08:33:58.590534+0000","last_fullsized":"2024-04-25T08:33:58.590534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":288,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595075+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:58.595075+0000","last_peered":"2024-04-25T08:33:58.595075+0000","last_clean":"2024-04-25T08:33:58.595075+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:58.595075+0000","last_undegraded":"2024-04-25T08:33:58.595075+0000","last_fullsized":"2024-04-25T08:33:58.595075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1609728,"data_stored":1055995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335492,"statfs":{"total":96636764160,"available":96599543808,"internally_reserved":0,"allocated":1024000,"data_stored":467217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1015808,"data_stored":465639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1032192,"data_stored":467321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40500,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331340,"statfs":{"total":96636764160,"available":96595292160,"internally_reserved":0,"allocated":1015808,"data_stored":466187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331336,"statfs":{"total":96636764160,"available":96595288064,"internally_reserved":0,"allocated":1024000,"data_stored":466347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1019904,"data_stored":465717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343248,"statfs":{"total":96636764160,"available":96607485952,"internally_reserved":0,"allocated":1605632,"data_stored":1055973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:02.838 INFO:tasks.ceph.mds.b.smithi100.stderr:2024-04-25T08:34:02.843+0000 7fe5165ce640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:34:05.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:05.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.129+0000 7ff5856f5640 1 Processor -- start 2024-04-25T08:34:05.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.129+0000 7ff5856f5640 1 -- start start 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff5856f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff580144750 0x7ff58006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff5856f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58006eb10 0x7ff5800ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff5856f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff580144b30 con 0x7ff5800cee60 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff5856f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5800712d0 con 0x7ff580144750 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff5856f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800715b0 con 0x7ff58006eb10 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff57effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff580144750 0x7ff58006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff57effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff580144750 0x7ff58006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56666/0 (socket says 172.21.15.60:56666) 2024-04-25T08:34:05.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff57effd640 1 -- 172.21.15.60:0/3501935789 learned_addr learned my addr 172.21.15.60:0/3501935789 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff57e7fc640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58006eb10 0x7ff5800ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.130+0000 7ff57effd640 1 -- 172.21.15.60:0/3501935789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58006eb10 msgr2=0x7ff5800ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57effd640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58006eb10 0x7ff5800ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57effd640 1 -- 172.21.15.60:0/3501935789 >> v1:172.21.15.60:6789/0 conn(0x7ff5800cee60 legacy=0x7ff58015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57effd640 1 -- 172.21.15.60:0/3501935789 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580071890 con 0x7ff580144750 2024-04-25T08:34:05.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57e7fc640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58006eb10 0x7ff5800ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57effd640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff580144750 0x7ff58006e5d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff570009e50 tx=0x7ff57002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c0d4578e66de20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57dffb640 1 -- 172.21.15.60:0/3501935789 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570036070 con 0x7ff580144750 2024-04-25T08:34:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57dffb640 1 -- 172.21.15.60:0/3501935789 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570031600 con 0x7ff580144750 2024-04-25T08:34:05.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.131+0000 7ff57dffb640 1 -- 172.21.15.60:0/3501935789 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700319d0 con 0x7ff580144750 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 -- 172.21.15.60:0/3501935789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff580144750 msgr2=0x7ff58006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff580144750 0x7ff58006e5d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7ff570009e50 tx=0x7ff57002a710 comp rx=0 tx=0).stop 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 -- 172.21.15.60:0/3501935789 shutdown_connections 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff58006eb10 0x7ff5800ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 --2- 172.21.15.60:0/3501935789 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff580144750 0x7ff58006e5d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 -- 172.21.15.60:0/3501935789 >> 172.21.15.60:0/3501935789 conn(0x7ff58006abe0 msgr2=0x7ff58006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 -- 172.21.15.60:0/3501935789 shutdown_connections 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.132+0000 7ff5856f5640 1 -- 172.21.15.60:0/3501935789 wait complete. 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.133+0000 7ff5856f5640 1 Processor -- start 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 -- start start 2024-04-25T08:34:05.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 0x7ff5800ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800cee60 0x7ff5800b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff580144750 0x7ff58013ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff580145450 con 0x7ff58006eb10 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58006f020 con 0x7ff580144750 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff5856f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5801455d0 con 0x7ff5800cee60 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff57effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 0x7ff5800ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff57effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 0x7ff5800ad7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56676/0 (socket says 172.21.15.60:56676) 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.134+0000 7ff57effd640 1 -- 172.21.15.60:0/1760041228 learned_addr learned my addr 172.21.15.60:0/1760041228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57e7fc640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800cee60 0x7ff5800b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57f7fe640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff580144750 0x7ff58013ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57effd640 1 -- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800cee60 msgr2=0x7ff5800b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57effd640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800cee60 0x7ff5800b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57effd640 1 -- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff580144750 msgr2=0x7ff58013ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57effd640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff580144750 0x7ff58013ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57effd640 1 -- 172.21.15.60:0/1760041228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5801e0f20 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57e7fc640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800cee60 0x7ff5800b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57f7fe640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff580144750 0x7ff58013ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff57effd640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 0x7ff5800ad7a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff570034040 tx=0x7ff570004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570036050 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570042070 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5800ae130 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570009510 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.135+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5801e1780 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.136+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff580073990 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.137+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff570007c50 con 0x7ff58006eb10 2024-04-25T08:34:05.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.137+0000 7ff5777fe640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff56006b4d0 0x7ff56006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.137+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5700ab250 con 0x7ff58006eb10 2024-04-25T08:34:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.138+0000 7ff57e7fc640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff56006b4d0 0x7ff56006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.138+0000 7ff57e7fc640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff56006b4d0 0x7ff56006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff568006fd0 tx=0x7ff568008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.142+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5800ae130 con 0x7ff58006eb10 2024-04-25T08:34:05.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.324+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5801e1120 con 0x7ff56006b4d0 2024-04-25T08:34:05.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.331+0000 7ff5777fe640 1 -- 172.21.15.60:0/1760041228 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252082 (secure 0 0 0) 0x7ff5801e1120 con 0x7ff56006b4d0 2024-04-25T08:34:05.329 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:05.330 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff56006b4d0 msgr2=0x7ff56006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff56006b4d0 0x7ff56006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff568006fd0 tx=0x7ff568008040 comp rx=0 tx=0).stop 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 msgr2=0x7ff5800ad7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 0x7ff5800ad7a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff570034040 tx=0x7ff570004a40 comp rx=0 tx=0).stop 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 shutdown_connections 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff56006b4d0 0x7ff56006d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff580144750 0x7ff58013ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff5800cee60 0x7ff5800b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 --2- 172.21.15.60:0/1760041228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff58006eb10 0x7ff5800ad7a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 >> 172.21.15.60:0/1760041228 conn(0x7ff58006abe0 msgr2=0x7ff580142ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 shutdown_connections 2024-04-25T08:34:05.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.334+0000 7ff5856f5640 1 -- 172.21.15.60:0/1760041228 wait complete. 2024-04-25T08:34:05.340 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-04-25T08:34:03.463758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296036,"kb_used_data":9128,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754678684,"statfs":{"total":773094113280,"available":772790972416,"internally_reserved":0,"allocated":9347072,"data_stored":4910396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.902993"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.594955+0000","last_change":"2024-04-25T08:33:57.598905+0000","last_active":"2024-04-25T08:33:58.594955+0000","last_peered":"2024-04-25T08:33:58.594955+0000","last_clean":"2024-04-25T08:33:58.594955+0000","last_became_active":"2024-04-25T08:33:57.598604+0000","last_became_peered":"2024-04-25T08:33:57.598604+0000","last_unstale":"2024-04-25T08:33:58.594955+0000","last_undegraded":"2024-04-25T08:33:58.594955+0000","last_fullsized":"2024-04-25T08:33:58.594955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:06.644729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579347+0000","last_change":"2024-04-25T08:33:57.588813+0000","last_active":"2024-04-25T08:33:59.579347+0000","last_peered":"2024-04-25T08:33:59.579347+0000","last_clean":"2024-04-25T08:33:59.579347+0000","last_became_active":"2024-04-25T08:33:57.588517+0000","last_became_peered":"2024-04-25T08:33:57.588517+0000","last_unstale":"2024-04-25T08:33:59.579347+0000","last_undegraded":"2024-04-25T08:33:59.579347+0000","last_fullsized":"2024-04-25T08:33:59.579347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:03.303562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976285+0000","last_change":"2024-04-25T08:33:57.595973+0000","last_active":"2024-04-25T08:33:59.976285+0000","last_peered":"2024-04-25T08:33:59.976285+0000","last_clean":"2024-04-25T08:33:59.976285+0000","last_became_active":"2024-04-25T08:33:57.595703+0000","last_became_peered":"2024-04-25T08:33:57.595703+0000","last_unstale":"2024-04-25T08:33:59.976285+0000","last_undegraded":"2024-04-25T08:33:59.976285+0000","last_fullsized":"2024-04-25T08:33:59.976285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:35.167824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911696+0000","last_change":"2024-04-25T08:33:57.592529+0000","last_active":"2024-04-25T08:34:00.911696+0000","last_peered":"2024-04-25T08:34:00.911696+0000","last_clean":"2024-04-25T08:34:00.911696+0000","last_became_active":"2024-04-25T08:33:57.592251+0000","last_became_peered":"2024-04-25T08:33:57.592251+0000","last_unstale":"2024-04-25T08:34:00.911696+0000","last_undegraded":"2024-04-25T08:34:00.911696+0000","last_fullsized":"2024-04-25T08:34:00.911696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581515+0000","last_change":"2024-04-25T08:33:57.599457+0000","last_active":"2024-04-25T08:33:59.581515+0000","last_peered":"2024-04-25T08:33:59.581515+0000","last_clean":"2024-04-25T08:33:59.581515+0000","last_became_active":"2024-04-25T08:33:57.598733+0000","last_became_peered":"2024-04-25T08:33:57.598733+0000","last_unstale":"2024-04-25T08:33:59.581515+0000","last_undegraded":"2024-04-25T08:33:59.581515+0000","last_fullsized":"2024-04-25T08:33:59.581515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:31.378434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590285+0000","last_change":"2024-04-25T08:33:57.602197+0000","last_active":"2024-04-25T08:33:58.590285+0000","last_peered":"2024-04-25T08:33:58.590285+0000","last_clean":"2024-04-25T08:33:58.590285+0000","last_became_active":"2024-04-25T08:33:57.601872+0000","last_became_peered":"2024-04-25T08:33:57.601872+0000","last_unstale":"2024-04-25T08:33:58.590285+0000","last_undegraded":"2024-04-25T08:33:58.590285+0000","last_fullsized":"2024-04-25T08:33:58.590285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595931+0000","last_change":"2024-04-25T08:33:57.585236+0000","last_active":"2024-04-25T08:33:58.595931+0000","last_peered":"2024-04-25T08:33:58.595931+0000","last_clean":"2024-04-25T08:33:58.595931+0000","last_became_active":"2024-04-25T08:33:57.584945+0000","last_became_peered":"2024-04-25T08:33:57.584945+0000","last_unstale":"2024-04-25T08:33:58.595931+0000","last_undegraded":"2024-04-25T08:33:58.595931+0000","last_fullsized":"2024-04-25T08:33:58.595931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:16.228013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581425+0000","last_change":"2024-04-25T08:33:57.598182+0000","last_active":"2024-04-25T08:33:59.581425+0000","last_peered":"2024-04-25T08:33:59.581425+0000","last_clean":"2024-04-25T08:33:59.581425+0000","last_became_active":"2024-04-25T08:33:57.597859+0000","last_became_peered":"2024-04-25T08:33:57.597859+0000","last_unstale":"2024-04-25T08:33:59.581425+0000","last_undegraded":"2024-04-25T08:33:59.581425+0000","last_fullsized":"2024-04-25T08:33:59.581425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:00.789203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170760+0000","last_change":"2024-04-25T08:33:57.595471+0000","last_active":"2024-04-25T08:34:00.170760+0000","last_peered":"2024-04-25T08:34:00.170760+0000","last_clean":"2024-04-25T08:34:00.170760+0000","last_became_active":"2024-04-25T08:33:57.595197+0000","last_became_peered":"2024-04-25T08:33:57.595197+0000","last_unstale":"2024-04-25T08:34:00.170760+0000","last_undegraded":"2024-04-25T08:34:00.170760+0000","last_fullsized":"2024-04-25T08:34:00.170760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:53.820463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911729+0000","last_change":"2024-04-25T08:33:57.597464+0000","last_active":"2024-04-25T08:34:00.911729+0000","last_peered":"2024-04-25T08:34:00.911729+0000","last_clean":"2024-04-25T08:34:00.911729+0000","last_became_active":"2024-04-25T08:33:57.597183+0000","last_became_peered":"2024-04-25T08:33:57.597183+0000","last_unstale":"2024-04-25T08:34:00.911729+0000","last_undegraded":"2024-04-25T08:34:00.911729+0000","last_fullsized":"2024-04-25T08:34:00.911729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:32.031804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579266+0000","last_change":"2024-04-25T08:33:57.588125+0000","last_active":"2024-04-25T08:33:59.579266+0000","last_peered":"2024-04-25T08:33:59.579266+0000","last_clean":"2024-04-25T08:33:59.579266+0000","last_became_active":"2024-04-25T08:33:57.587846+0000","last_became_peered":"2024-04-25T08:33:57.587846+0000","last_unstale":"2024-04-25T08:33:59.579266+0000","last_undegraded":"2024-04-25T08:33:59.579266+0000","last_fullsized":"2024-04-25T08:33:59.579266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:27.134575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911733+0000","last_change":"2024-04-25T08:33:57.595123+0000","last_active":"2024-04-25T08:34:00.911733+0000","last_peered":"2024-04-25T08:34:00.911733+0000","last_clean":"2024-04-25T08:34:00.911733+0000","last_became_active":"2024-04-25T08:33:57.594793+0000","last_became_peered":"2024-04-25T08:33:57.594793+0000","last_unstale":"2024-04-25T08:34:00.911733+0000","last_undegraded":"2024-04-25T08:34:00.911733+0000","last_fullsized":"2024-04-25T08:34:00.911733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.699554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581411+0000","last_change":"2024-04-25T08:33:57.598427+0000","last_active":"2024-04-25T08:33:59.581411+0000","last_peered":"2024-04-25T08:33:59.581411+0000","last_clean":"2024-04-25T08:33:59.581411+0000","last_became_active":"2024-04-25T08:33:57.598025+0000","last_became_peered":"2024-04-25T08:33:57.598025+0000","last_unstale":"2024-04-25T08:33:59.581411+0000","last_undegraded":"2024-04-25T08:33:59.581411+0000","last_fullsized":"2024-04-25T08:33:59.581411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:52.044453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590364+0000","last_change":"2024-04-25T08:33:57.602830+0000","last_active":"2024-04-25T08:33:58.590364+0000","last_peered":"2024-04-25T08:33:58.590364+0000","last_clean":"2024-04-25T08:33:58.590364+0000","last_became_active":"2024-04-25T08:33:57.602446+0000","last_became_peered":"2024-04-25T08:33:57.602446+0000","last_unstale":"2024-04-25T08:33:58.590364+0000","last_undegraded":"2024-04-25T08:33:58.590364+0000","last_fullsized":"2024-04-25T08:33:58.590364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.699554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578856+0000","last_change":"2024-04-25T08:33:57.584756+0000","last_active":"2024-04-25T08:33:59.578856+0000","last_peered":"2024-04-25T08:33:59.578856+0000","last_clean":"2024-04-25T08:33:59.578856+0000","last_became_active":"2024-04-25T08:33:57.584454+0000","last_became_peered":"2024-04-25T08:33:57.584454+0000","last_unstale":"2024-04-25T08:33:59.578856+0000","last_undegraded":"2024-04-25T08:33:59.578856+0000","last_fullsized":"2024-04-25T08:33:59.578856+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:32.031804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595012+0000","last_change":"2024-04-25T08:33:57.593526+0000","last_active":"2024-04-25T08:33:58.595012+0000","last_peered":"2024-04-25T08:33:58.595012+0000","last_clean":"2024-04-25T08:33:58.595012+0000","last_became_active":"2024-04-25T08:33:57.593270+0000","last_became_peered":"2024-04-25T08:33:57.593270+0000","last_unstale":"2024-04-25T08:33:58.595012+0000","last_undegraded":"2024-04-25T08:33:58.595012+0000","last_fullsized":"2024-04-25T08:33:58.595012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:14.276030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581662+0000","last_change":"2024-04-25T08:33:57.592230+0000","last_active":"2024-04-25T08:33:59.581662+0000","last_peered":"2024-04-25T08:33:59.581662+0000","last_clean":"2024-04-25T08:33:59.581662+0000","last_became_active":"2024-04-25T08:33:57.591926+0000","last_became_peered":"2024-04-25T08:33:57.591926+0000","last_unstale":"2024-04-25T08:33:59.581662+0000","last_undegraded":"2024-04-25T08:33:59.581662+0000","last_fullsized":"2024-04-25T08:33:59.581662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:21.497838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976191+0000","last_change":"2024-04-25T08:33:57.585568+0000","last_active":"2024-04-25T08:33:59.976191+0000","last_peered":"2024-04-25T08:33:59.976191+0000","last_clean":"2024-04-25T08:33:59.976191+0000","last_became_active":"2024-04-25T08:33:57.585295+0000","last_became_peered":"2024-04-25T08:33:57.585295+0000","last_unstale":"2024-04-25T08:33:59.976191+0000","last_undegraded":"2024-04-25T08:33:59.976191+0000","last_fullsized":"2024-04-25T08:33:59.976191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:16.228013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590219+0000","last_change":"2024-04-25T08:33:57.605299+0000","last_active":"2024-04-25T08:33:58.590219+0000","last_peered":"2024-04-25T08:33:58.590219+0000","last_clean":"2024-04-25T08:33:58.590219+0000","last_became_active":"2024-04-25T08:33:57.604951+0000","last_became_peered":"2024-04-25T08:33:57.604951+0000","last_unstale":"2024-04-25T08:33:58.590219+0000","last_undegraded":"2024-04-25T08:33:58.590219+0000","last_fullsized":"2024-04-25T08:33:58.590219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.617051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581494+0000","last_change":"2024-04-25T08:33:57.587022+0000","last_active":"2024-04-25T08:33:59.581494+0000","last_peered":"2024-04-25T08:33:59.581494+0000","last_clean":"2024-04-25T08:33:59.581494+0000","last_became_active":"2024-04-25T08:33:57.586731+0000","last_became_peered":"2024-04-25T08:33:57.586731+0000","last_unstale":"2024-04-25T08:33:59.581494+0000","last_undegraded":"2024-04-25T08:33:59.581494+0000","last_fullsized":"2024-04-25T08:33:59.581494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:06.305916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170652+0000","last_change":"2024-04-25T08:33:57.586019+0000","last_active":"2024-04-25T08:34:00.170652+0000","last_peered":"2024-04-25T08:34:00.170652+0000","last_clean":"2024-04-25T08:34:00.170652+0000","last_became_active":"2024-04-25T08:33:57.585762+0000","last_became_peered":"2024-04-25T08:33:57.585762+0000","last_unstale":"2024-04-25T08:34:00.170652+0000","last_undegraded":"2024-04-25T08:34:00.170652+0000","last_fullsized":"2024-04-25T08:34:00.170652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:14.819343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581544+0000","last_change":"2024-04-25T08:33:57.598204+0000","last_active":"2024-04-25T08:33:59.581544+0000","last_peered":"2024-04-25T08:33:59.581544+0000","last_clean":"2024-04-25T08:33:59.581544+0000","last_became_active":"2024-04-25T08:33:57.597874+0000","last_became_peered":"2024-04-25T08:33:57.597874+0000","last_unstale":"2024-04-25T08:33:59.581544+0000","last_undegraded":"2024-04-25T08:33:59.581544+0000","last_fullsized":"2024-04-25T08:33:59.581544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:23.873785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170631+0000","last_change":"2024-04-25T08:33:57.585867+0000","last_active":"2024-04-25T08:34:00.170631+0000","last_peered":"2024-04-25T08:34:00.170631+0000","last_clean":"2024-04-25T08:34:00.170631+0000","last_became_active":"2024-04-25T08:33:57.585567+0000","last_became_peered":"2024-04-25T08:33:57.585567+0000","last_unstale":"2024-04-25T08:34:00.170631+0000","last_undegraded":"2024-04-25T08:34:00.170631+0000","last_fullsized":"2024-04-25T08:34:00.170631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:53.092263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578849+0000","last_change":"2024-04-25T08:33:57.594139+0000","last_active":"2024-04-25T08:33:59.578849+0000","last_peered":"2024-04-25T08:33:59.578849+0000","last_clean":"2024-04-25T08:33:59.578849+0000","last_became_active":"2024-04-25T08:33:57.593814+0000","last_became_peered":"2024-04-25T08:33:57.593814+0000","last_unstale":"2024-04-25T08:33:59.578849+0000","last_undegraded":"2024-04-25T08:33:59.578849+0000","last_fullsized":"2024-04-25T08:33:59.578849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.213482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579246+0000","last_change":"2024-04-25T08:33:57.595568+0000","last_active":"2024-04-25T08:33:59.579246+0000","last_peered":"2024-04-25T08:33:59.579246+0000","last_clean":"2024-04-25T08:33:59.579246+0000","last_became_active":"2024-04-25T08:33:57.595295+0000","last_became_peered":"2024-04-25T08:33:57.595295+0000","last_unstale":"2024-04-25T08:33:59.579246+0000","last_undegraded":"2024-04-25T08:33:59.579246+0000","last_fullsized":"2024-04-25T08:33:59.579246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.930380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911853+0000","last_change":"2024-04-25T08:33:57.595454+0000","last_active":"2024-04-25T08:34:00.911853+0000","last_peered":"2024-04-25T08:34:00.911853+0000","last_clean":"2024-04-25T08:34:00.911853+0000","last_became_active":"2024-04-25T08:33:57.595172+0000","last_became_peered":"2024-04-25T08:33:57.595172+0000","last_unstale":"2024-04-25T08:34:00.911853+0000","last_undegraded":"2024-04-25T08:34:00.911853+0000","last_fullsized":"2024-04-25T08:34:00.911853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:06.644729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595305+0000","last_change":"2024-04-25T08:33:57.595885+0000","last_active":"2024-04-25T08:33:58.595305+0000","last_peered":"2024-04-25T08:33:58.595305+0000","last_clean":"2024-04-25T08:33:58.595305+0000","last_became_active":"2024-04-25T08:33:57.595610+0000","last_became_peered":"2024-04-25T08:33:57.595610+0000","last_unstale":"2024-04-25T08:33:58.595305+0000","last_undegraded":"2024-04-25T08:33:58.595305+0000","last_fullsized":"2024-04-25T08:33:58.595305+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:27.134575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581686+0000","last_change":"2024-04-25T08:33:57.586894+0000","last_active":"2024-04-25T08:33:59.581686+0000","last_peered":"2024-04-25T08:33:59.581686+0000","last_clean":"2024-04-25T08:33:59.581686+0000","last_became_active":"2024-04-25T08:33:57.586596+0000","last_became_peered":"2024-04-25T08:33:57.586596+0000","last_unstale":"2024-04-25T08:33:59.581686+0000","last_undegraded":"2024-04-25T08:33:59.581686+0000","last_fullsized":"2024-04-25T08:33:59.581686+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:24.887648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579075+0000","last_change":"2024-04-25T08:33:57.593189+0000","last_active":"2024-04-25T08:33:59.579075+0000","last_peered":"2024-04-25T08:33:59.579075+0000","last_clean":"2024-04-25T08:33:59.579075+0000","last_became_active":"2024-04-25T08:33:57.592900+0000","last_became_peered":"2024-04-25T08:33:57.592900+0000","last_unstale":"2024-04-25T08:33:59.579075+0000","last_undegraded":"2024-04-25T08:33:59.579075+0000","last_fullsized":"2024-04-25T08:33:59.579075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595185+0000","last_change":"2024-04-25T08:33:57.605041+0000","last_active":"2024-04-25T08:33:58.595185+0000","last_peered":"2024-04-25T08:33:58.595185+0000","last_clean":"2024-04-25T08:33:58.595185+0000","last_became_active":"2024-04-25T08:33:57.604785+0000","last_became_peered":"2024-04-25T08:33:57.604785+0000","last_unstale":"2024-04-25T08:33:58.595185+0000","last_undegraded":"2024-04-25T08:33:58.595185+0000","last_fullsized":"2024-04-25T08:33:58.595185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.930380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976125+0000","last_change":"2024-04-25T08:33:57.594798+0000","last_active":"2024-04-25T08:33:59.976125+0000","last_peered":"2024-04-25T08:33:59.976125+0000","last_clean":"2024-04-25T08:33:59.976125+0000","last_became_active":"2024-04-25T08:33:57.594508+0000","last_became_peered":"2024-04-25T08:33:57.594508+0000","last_unstale":"2024-04-25T08:33:59.976125+0000","last_undegraded":"2024-04-25T08:33:59.976125+0000","last_fullsized":"2024-04-25T08:33:59.976125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:10.270627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.594846+0000","last_change":"2024-04-25T08:33:57.598962+0000","last_active":"2024-04-25T08:33:58.594846+0000","last_peered":"2024-04-25T08:33:58.594846+0000","last_clean":"2024-04-25T08:33:58.594846+0000","last_became_active":"2024-04-25T08:33:57.598505+0000","last_became_peered":"2024-04-25T08:33:57.598505+0000","last_unstale":"2024-04-25T08:33:58.594846+0000","last_undegraded":"2024-04-25T08:33:58.594846+0000","last_fullsized":"2024-04-25T08:33:58.594846+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976124+0000","last_change":"2024-04-25T08:33:57.584864+0000","last_active":"2024-04-25T08:33:59.976124+0000","last_peered":"2024-04-25T08:33:59.976124+0000","last_clean":"2024-04-25T08:33:59.976124+0000","last_became_active":"2024-04-25T08:33:57.584585+0000","last_became_peered":"2024-04-25T08:33:57.584585+0000","last_unstale":"2024-04-25T08:33:59.976124+0000","last_undegraded":"2024-04-25T08:33:59.976124+0000","last_fullsized":"2024-04-25T08:33:59.976124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:41.938128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581692+0000","last_change":"2024-04-25T08:33:56.578809+0000","last_active":"2024-04-25T08:33:59.581692+0000","last_peered":"2024-04-25T08:33:59.581692+0000","last_clean":"2024-04-25T08:33:59.581692+0000","last_became_active":"2024-04-25T08:33:56.578466+0000","last_became_peered":"2024-04-25T08:33:56.578466+0000","last_unstale":"2024-04-25T08:33:59.581692+0000","last_undegraded":"2024-04-25T08:33:59.581692+0000","last_fullsized":"2024-04-25T08:33:59.581692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:24.033056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578927+0000","last_change":"2024-04-25T08:33:57.593366+0000","last_active":"2024-04-25T08:33:59.578927+0000","last_peered":"2024-04-25T08:33:59.578927+0000","last_clean":"2024-04-25T08:33:59.578927+0000","last_became_active":"2024-04-25T08:33:57.593069+0000","last_became_peered":"2024-04-25T08:33:57.593069+0000","last_unstale":"2024-04-25T08:33:59.578927+0000","last_undegraded":"2024-04-25T08:33:59.578927+0000","last_fullsized":"2024-04-25T08:33:59.578927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:02.699554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595096+0000","last_change":"2024-04-25T08:33:56.577451+0000","last_active":"2024-04-25T08:33:58.595096+0000","last_peered":"2024-04-25T08:33:58.595096+0000","last_clean":"2024-04-25T08:33:58.595096+0000","last_became_active":"2024-04-25T08:33:56.577182+0000","last_became_peered":"2024-04-25T08:33:56.577182+0000","last_unstale":"2024-04-25T08:33:58.595096+0000","last_undegraded":"2024-04-25T08:33:58.595096+0000","last_fullsized":"2024-04-25T08:33:58.595096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:23.662726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170780+0000","last_change":"2024-04-25T08:33:57.600541+0000","last_active":"2024-04-25T08:34:00.170780+0000","last_peered":"2024-04-25T08:34:00.170780+0000","last_clean":"2024-04-25T08:34:00.170780+0000","last_became_active":"2024-04-25T08:33:57.600239+0000","last_became_peered":"2024-04-25T08:33:57.600239+0000","last_unstale":"2024-04-25T08:34:00.170780+0000","last_undegraded":"2024-04-25T08:34:00.170780+0000","last_fullsized":"2024-04-25T08:34:00.170780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:02.840551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595369+0000","last_change":"2024-04-25T08:33:56.581227+0000","last_active":"2024-04-25T08:33:58.595369+0000","last_peered":"2024-04-25T08:33:58.595369+0000","last_clean":"2024-04-25T08:33:58.595369+0000","last_became_active":"2024-04-25T08:33:56.580846+0000","last_became_peered":"2024-04-25T08:33:56.580846+0000","last_unstale":"2024-04-25T08:33:58.595369+0000","last_undegraded":"2024-04-25T08:33:58.595369+0000","last_fullsized":"2024-04-25T08:33:58.595369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:50.350319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170842+0000","last_change":"2024-04-25T08:33:57.585922+0000","last_active":"2024-04-25T08:34:00.170842+0000","last_peered":"2024-04-25T08:34:00.170842+0000","last_clean":"2024-04-25T08:34:00.170842+0000","last_became_active":"2024-04-25T08:33:57.585623+0000","last_became_peered":"2024-04-25T08:33:57.585623+0000","last_unstale":"2024-04-25T08:34:00.170842+0000","last_undegraded":"2024-04-25T08:34:00.170842+0000","last_fullsized":"2024-04-25T08:34:00.170842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:31.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596125+0000","last_change":"2024-04-25T08:33:56.575045+0000","last_active":"2024-04-25T08:33:58.596125+0000","last_peered":"2024-04-25T08:33:58.596125+0000","last_clean":"2024-04-25T08:33:58.596125+0000","last_became_active":"2024-04-25T08:33:56.574472+0000","last_became_peered":"2024-04-25T08:33:56.574472+0000","last_unstale":"2024-04-25T08:33:58.596125+0000","last_undegraded":"2024-04-25T08:33:58.596125+0000","last_fullsized":"2024-04-25T08:33:58.596125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:48.635135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579315+0000","last_change":"2024-04-25T08:33:57.586041+0000","last_active":"2024-04-25T08:33:59.579315+0000","last_peered":"2024-04-25T08:33:59.579315+0000","last_clean":"2024-04-25T08:33:59.579315+0000","last_became_active":"2024-04-25T08:33:57.585183+0000","last_became_peered":"2024-04-25T08:33:57.585183+0000","last_unstale":"2024-04-25T08:33:59.579315+0000","last_undegraded":"2024-04-25T08:33:59.579315+0000","last_fullsized":"2024-04-25T08:33:59.579315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:47.869752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170813+0000","last_change":"2024-04-25T08:33:56.573057+0000","last_active":"2024-04-25T08:34:00.170813+0000","last_peered":"2024-04-25T08:34:00.170813+0000","last_clean":"2024-04-25T08:34:00.170813+0000","last_became_active":"2024-04-25T08:33:56.572791+0000","last_became_peered":"2024-04-25T08:33:56.572791+0000","last_unstale":"2024-04-25T08:34:00.170813+0000","last_undegraded":"2024-04-25T08:34:00.170813+0000","last_fullsized":"2024-04-25T08:34:00.170813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:07.350599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579357+0000","last_change":"2024-04-25T08:33:57.585071+0000","last_active":"2024-04-25T08:33:59.579357+0000","last_peered":"2024-04-25T08:33:59.579357+0000","last_clean":"2024-04-25T08:33:59.579357+0000","last_became_active":"2024-04-25T08:33:57.584718+0000","last_became_peered":"2024-04-25T08:33:57.584718+0000","last_unstale":"2024-04-25T08:33:59.579357+0000","last_undegraded":"2024-04-25T08:33:59.579357+0000","last_fullsized":"2024-04-25T08:33:59.579357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.539872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911424+0000","last_change":"2024-04-25T08:33:56.576140+0000","last_active":"2024-04-25T08:34:00.911424+0000","last_peered":"2024-04-25T08:34:00.911424+0000","last_clean":"2024-04-25T08:34:00.911424+0000","last_became_active":"2024-04-25T08:33:56.575855+0000","last_became_peered":"2024-04-25T08:33:56.575855+0000","last_unstale":"2024-04-25T08:34:00.911424+0000","last_undegraded":"2024-04-25T08:34:00.911424+0000","last_fullsized":"2024-04-25T08:34:00.911424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:32.116736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170844+0000","last_change":"2024-04-25T08:33:57.597881+0000","last_active":"2024-04-25T08:34:00.170844+0000","last_peered":"2024-04-25T08:34:00.170844+0000","last_clean":"2024-04-25T08:34:00.170844+0000","last_became_active":"2024-04-25T08:33:57.597510+0000","last_became_peered":"2024-04-25T08:33:57.597510+0000","last_unstale":"2024-04-25T08:34:00.170844+0000","last_undegraded":"2024-04-25T08:34:00.170844+0000","last_fullsized":"2024-04-25T08:34:00.170844+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:18.760694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579060+0000","last_change":"2024-04-25T08:33:56.579617+0000","last_active":"2024-04-25T08:33:59.579060+0000","last_peered":"2024-04-25T08:33:59.579060+0000","last_clean":"2024-04-25T08:33:59.579060+0000","last_became_active":"2024-04-25T08:33:56.579378+0000","last_became_peered":"2024-04-25T08:33:56.579378+0000","last_unstale":"2024-04-25T08:33:59.579060+0000","last_undegraded":"2024-04-25T08:33:59.579060+0000","last_fullsized":"2024-04-25T08:33:59.579060+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:57.861141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596614+0000","last_change":"2024-04-25T08:33:57.590229+0000","last_active":"2024-04-25T08:33:58.596614+0000","last_peered":"2024-04-25T08:33:58.596614+0000","last_clean":"2024-04-25T08:33:58.596614+0000","last_became_active":"2024-04-25T08:33:57.589126+0000","last_became_peered":"2024-04-25T08:33:57.589126+0000","last_unstale":"2024-04-25T08:33:58.596614+0000","last_undegraded":"2024-04-25T08:33:58.596614+0000","last_fullsized":"2024-04-25T08:33:58.596614+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:33.121751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581442+0000","last_change":"2024-04-25T08:33:56.577305+0000","last_active":"2024-04-25T08:33:59.581442+0000","last_peered":"2024-04-25T08:33:59.581442+0000","last_clean":"2024-04-25T08:33:59.581442+0000","last_became_active":"2024-04-25T08:33:56.577000+0000","last_became_peered":"2024-04-25T08:33:56.577000+0000","last_unstale":"2024-04-25T08:33:59.581442+0000","last_undegraded":"2024-04-25T08:33:59.581442+0000","last_fullsized":"2024-04-25T08:33:59.581442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:04.612385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578798+0000","last_change":"2024-04-25T08:33:57.595504+0000","last_active":"2024-04-25T08:33:59.578798+0000","last_peered":"2024-04-25T08:33:59.578798+0000","last_clean":"2024-04-25T08:33:59.578798+0000","last_became_active":"2024-04-25T08:33:57.595218+0000","last_became_peered":"2024-04-25T08:33:57.595218+0000","last_unstale":"2024-04-25T08:33:59.578798+0000","last_undegraded":"2024-04-25T08:33:59.578798+0000","last_fullsized":"2024-04-25T08:33:59.578798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.617051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595009+0000","last_change":"2024-04-25T08:33:56.579508+0000","last_active":"2024-04-25T08:33:58.595009+0000","last_peered":"2024-04-25T08:33:58.595009+0000","last_clean":"2024-04-25T08:33:58.595009+0000","last_became_active":"2024-04-25T08:33:56.579225+0000","last_became_peered":"2024-04-25T08:33:56.579225+0000","last_unstale":"2024-04-25T08:33:58.595009+0000","last_undegraded":"2024-04-25T08:33:58.595009+0000","last_fullsized":"2024-04-25T08:33:58.595009+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:27.575448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911617+0000","last_change":"2024-04-25T08:33:57.586353+0000","last_active":"2024-04-25T08:34:00.911617+0000","last_peered":"2024-04-25T08:34:00.911617+0000","last_clean":"2024-04-25T08:34:00.911617+0000","last_became_active":"2024-04-25T08:33:57.586085+0000","last_became_peered":"2024-04-25T08:33:57.586085+0000","last_unstale":"2024-04-25T08:34:00.911617+0000","last_undegraded":"2024-04-25T08:34:00.911617+0000","last_fullsized":"2024-04-25T08:34:00.911617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.617051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911783+0000","last_change":"2024-04-25T08:33:56.571751+0000","last_active":"2024-04-25T08:34:00.911783+0000","last_peered":"2024-04-25T08:34:00.911783+0000","last_clean":"2024-04-25T08:34:00.911783+0000","last_became_active":"2024-04-25T08:33:56.571479+0000","last_became_peered":"2024-04-25T08:33:56.571479+0000","last_unstale":"2024-04-25T08:34:00.911783+0000","last_undegraded":"2024-04-25T08:34:00.911783+0000","last_fullsized":"2024-04-25T08:34:00.911783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590507+0000","last_change":"2024-04-25T08:33:57.600747+0000","last_active":"2024-04-25T08:33:58.590507+0000","last_peered":"2024-04-25T08:33:58.590507+0000","last_clean":"2024-04-25T08:33:58.590507+0000","last_became_active":"2024-04-25T08:33:57.600422+0000","last_became_peered":"2024-04-25T08:33:57.600422+0000","last_unstale":"2024-04-25T08:33:58.590507+0000","last_undegraded":"2024-04-25T08:33:58.590507+0000","last_fullsized":"2024-04-25T08:33:58.590507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581871+0000","last_change":"2024-04-25T08:33:56.573128+0000","last_active":"2024-04-25T08:33:59.581871+0000","last_peered":"2024-04-25T08:33:59.581871+0000","last_clean":"2024-04-25T08:33:59.581871+0000","last_became_active":"2024-04-25T08:33:56.572850+0000","last_became_peered":"2024-04-25T08:33:56.572850+0000","last_unstale":"2024-04-25T08:33:59.581871+0000","last_undegraded":"2024-04-25T08:33:59.581871+0000","last_fullsized":"2024-04-25T08:33:59.581871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:24.509654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976163+0000","last_change":"2024-04-25T08:33:57.597452+0000","last_active":"2024-04-25T08:33:59.976163+0000","last_peered":"2024-04-25T08:33:59.976163+0000","last_clean":"2024-04-25T08:33:59.976163+0000","last_became_active":"2024-04-25T08:33:57.597168+0000","last_became_peered":"2024-04-25T08:33:57.597168+0000","last_unstale":"2024-04-25T08:33:59.976163+0000","last_undegraded":"2024-04-25T08:33:59.976163+0000","last_fullsized":"2024-04-25T08:33:59.976163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:58.866157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596190+0000","last_change":"2024-04-25T08:33:56.577795+0000","last_active":"2024-04-25T08:33:58.596190+0000","last_peered":"2024-04-25T08:33:58.596190+0000","last_clean":"2024-04-25T08:33:58.596190+0000","last_became_active":"2024-04-25T08:33:56.577512+0000","last_became_peered":"2024-04-25T08:33:56.577512+0000","last_unstale":"2024-04-25T08:33:58.596190+0000","last_undegraded":"2024-04-25T08:33:58.596190+0000","last_fullsized":"2024-04-25T08:33:58.596190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:17.755679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911840+0000","last_change":"2024-04-25T08:33:57.594892+0000","last_active":"2024-04-25T08:34:00.911840+0000","last_peered":"2024-04-25T08:34:00.911840+0000","last_clean":"2024-04-25T08:34:00.911840+0000","last_became_active":"2024-04-25T08:33:57.594600+0000","last_became_peered":"2024-04-25T08:33:57.594600+0000","last_unstale":"2024-04-25T08:34:00.911840+0000","last_undegraded":"2024-04-25T08:34:00.911840+0000","last_fullsized":"2024-04-25T08:34:00.911840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:35.416530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911457+0000","last_change":"2024-04-25T08:33:56.577566+0000","last_active":"2024-04-25T08:34:00.911457+0000","last_peered":"2024-04-25T08:34:00.911457+0000","last_clean":"2024-04-25T08:34:00.911457+0000","last_became_active":"2024-04-25T08:33:56.577291+0000","last_became_peered":"2024-04-25T08:33:56.577291+0000","last_unstale":"2024-04-25T08:34:00.911457+0000","last_undegraded":"2024-04-25T08:34:00.911457+0000","last_fullsized":"2024-04-25T08:34:00.911457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:08.663368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596172+0000","last_change":"2024-04-25T08:33:57.583448+0000","last_active":"2024-04-25T08:33:58.596172+0000","last_peered":"2024-04-25T08:33:58.596172+0000","last_clean":"2024-04-25T08:33:58.596172+0000","last_became_active":"2024-04-25T08:33:57.583081+0000","last_became_peered":"2024-04-25T08:33:57.583081+0000","last_unstale":"2024-04-25T08:33:58.596172+0000","last_undegraded":"2024-04-25T08:33:58.596172+0000","last_fullsized":"2024-04-25T08:33:58.596172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:35.167824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595261+0000","last_change":"2024-04-25T08:33:56.580851+0000","last_active":"2024-04-25T08:33:58.595261+0000","last_peered":"2024-04-25T08:33:58.595261+0000","last_clean":"2024-04-25T08:33:58.595261+0000","last_became_active":"2024-04-25T08:33:56.580557+0000","last_became_peered":"2024-04-25T08:33:56.580557+0000","last_unstale":"2024-04-25T08:33:58.595261+0000","last_undegraded":"2024-04-25T08:33:58.595261+0000","last_fullsized":"2024-04-25T08:33:58.595261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:32.116736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596498+0000","last_change":"2024-04-25T08:33:57.592966+0000","last_active":"2024-04-25T08:33:58.596498+0000","last_peered":"2024-04-25T08:33:58.596498+0000","last_clean":"2024-04-25T08:33:58.596498+0000","last_became_active":"2024-04-25T08:33:57.592647+0000","last_became_peered":"2024-04-25T08:33:57.592647+0000","last_unstale":"2024-04-25T08:33:58.596498+0000","last_undegraded":"2024-04-25T08:33:58.596498+0000","last_fullsized":"2024-04-25T08:33:58.596498+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:13.093212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581345+0000","last_change":"2024-04-25T08:33:56.575918+0000","last_active":"2024-04-25T08:33:59.581345+0000","last_peered":"2024-04-25T08:33:59.581345+0000","last_clean":"2024-04-25T08:33:59.581345+0000","last_became_active":"2024-04-25T08:33:56.575649+0000","last_became_peered":"2024-04-25T08:33:56.575649+0000","last_unstale":"2024-04-25T08:33:59.581345+0000","last_undegraded":"2024-04-25T08:33:59.581345+0000","last_fullsized":"2024-04-25T08:33:59.581345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:43.360965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596616+0000","last_change":"2024-04-25T08:33:57.592922+0000","last_active":"2024-04-25T08:33:58.596616+0000","last_peered":"2024-04-25T08:33:58.596616+0000","last_clean":"2024-04-25T08:33:58.596616+0000","last_became_active":"2024-04-25T08:33:57.592604+0000","last_became_peered":"2024-04-25T08:33:57.592604+0000","last_unstale":"2024-04-25T08:33:58.596616+0000","last_undegraded":"2024-04-25T08:33:58.596616+0000","last_fullsized":"2024-04-25T08:33:58.596616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:32.191378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.578991+0000","last_change":"2024-04-25T08:33:56.571020+0000","last_active":"2024-04-25T08:33:59.578991+0000","last_peered":"2024-04-25T08:33:59.578991+0000","last_clean":"2024-04-25T08:33:59.578991+0000","last_became_active":"2024-04-25T08:33:56.570742+0000","last_became_peered":"2024-04-25T08:33:56.570742+0000","last_unstale":"2024-04-25T08:33:59.578991+0000","last_undegraded":"2024-04-25T08:33:59.578991+0000","last_fullsized":"2024-04-25T08:33:59.578991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:56.865380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590032+0000","last_change":"2024-04-25T08:33:57.605305+0000","last_active":"2024-04-25T08:33:58.590032+0000","last_peered":"2024-04-25T08:33:58.590032+0000","last_clean":"2024-04-25T08:33:58.590032+0000","last_became_active":"2024-04-25T08:33:57.604956+0000","last_became_peered":"2024-04-25T08:33:57.604956+0000","last_unstale":"2024-04-25T08:33:58.590032+0000","last_undegraded":"2024-04-25T08:33:58.590032+0000","last_fullsized":"2024-04-25T08:33:58.590032+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:25.748900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595276+0000","last_change":"2024-04-25T08:33:56.578816+0000","last_active":"2024-04-25T08:33:58.595276+0000","last_peered":"2024-04-25T08:33:58.595276+0000","last_clean":"2024-04-25T08:33:58.595276+0000","last_became_active":"2024-04-25T08:33:56.578552+0000","last_became_peered":"2024-04-25T08:33:56.578552+0000","last_unstale":"2024-04-25T08:33:58.595276+0000","last_undegraded":"2024-04-25T08:33:58.595276+0000","last_fullsized":"2024-04-25T08:33:58.595276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:21.914391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581289+0000","last_change":"2024-04-25T08:33:57.591560+0000","last_active":"2024-04-25T08:33:59.581289+0000","last_peered":"2024-04-25T08:33:59.581289+0000","last_clean":"2024-04-25T08:33:59.581289+0000","last_became_active":"2024-04-25T08:33:57.591295+0000","last_became_peered":"2024-04-25T08:33:57.591295+0000","last_unstale":"2024-04-25T08:33:59.581289+0000","last_undegraded":"2024-04-25T08:33:59.581289+0000","last_fullsized":"2024-04-25T08:33:59.581289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:58.860879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170900+0000","last_change":"2024-04-25T08:33:56.577328+0000","last_active":"2024-04-25T08:34:00.170900+0000","last_peered":"2024-04-25T08:34:00.170900+0000","last_clean":"2024-04-25T08:34:00.170900+0000","last_became_active":"2024-04-25T08:33:56.577061+0000","last_became_peered":"2024-04-25T08:33:56.577061+0000","last_unstale":"2024-04-25T08:34:00.170900+0000","last_undegraded":"2024-04-25T08:34:00.170900+0000","last_fullsized":"2024-04-25T08:34:00.170900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:14.492442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581379+0000","last_change":"2024-04-25T08:33:57.592930+0000","last_active":"2024-04-25T08:33:59.581379+0000","last_peered":"2024-04-25T08:33:59.581379+0000","last_clean":"2024-04-25T08:33:59.581379+0000","last_became_active":"2024-04-25T08:33:57.592665+0000","last_became_peered":"2024-04-25T08:33:57.592665+0000","last_unstale":"2024-04-25T08:33:59.581379+0000","last_undegraded":"2024-04-25T08:33:59.581379+0000","last_fullsized":"2024-04-25T08:33:59.581379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.545740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581873+0000","last_change":"2024-04-25T08:33:56.577330+0000","last_active":"2024-04-25T08:33:59.581873+0000","last_peered":"2024-04-25T08:33:59.581873+0000","last_clean":"2024-04-25T08:33:59.581873+0000","last_became_active":"2024-04-25T08:33:56.577049+0000","last_became_peered":"2024-04-25T08:33:56.577049+0000","last_unstale":"2024-04-25T08:33:59.581873+0000","last_undegraded":"2024-04-25T08:33:59.581873+0000","last_fullsized":"2024-04-25T08:33:59.581873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:41.628769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590298+0000","last_change":"2024-04-25T08:33:57.606098+0000","last_active":"2024-04-25T08:33:58.590298+0000","last_peered":"2024-04-25T08:33:58.590298+0000","last_clean":"2024-04-25T08:33:58.590298+0000","last_became_active":"2024-04-25T08:33:57.604969+0000","last_became_peered":"2024-04-25T08:33:57.604969+0000","last_unstale":"2024-04-25T08:33:58.590298+0000","last_undegraded":"2024-04-25T08:33:58.590298+0000","last_fullsized":"2024-04-25T08:33:58.590298+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:32.031804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976005+0000","last_change":"2024-04-25T08:33:56.569880+0000","last_active":"2024-04-25T08:33:59.976005+0000","last_peered":"2024-04-25T08:33:59.976005+0000","last_clean":"2024-04-25T08:33:59.976005+0000","last_became_active":"2024-04-25T08:33:56.569586+0000","last_became_peered":"2024-04-25T08:33:56.569586+0000","last_unstale":"2024-04-25T08:33:59.976005+0000","last_undegraded":"2024-04-25T08:33:59.976005+0000","last_fullsized":"2024-04-25T08:33:59.976005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:48.635135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579164+0000","last_change":"2024-04-25T08:33:57.594160+0000","last_active":"2024-04-25T08:33:59.579164+0000","last_peered":"2024-04-25T08:33:59.579164+0000","last_clean":"2024-04-25T08:33:59.579164+0000","last_became_active":"2024-04-25T08:33:57.593858+0000","last_became_peered":"2024-04-25T08:33:57.593858+0000","last_unstale":"2024-04-25T08:33:59.579164+0000","last_undegraded":"2024-04-25T08:33:59.579164+0000","last_fullsized":"2024-04-25T08:33:59.579164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:06.644729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590361+0000","last_change":"2024-04-25T08:33:56.575983+0000","last_active":"2024-04-25T08:33:58.590361+0000","last_peered":"2024-04-25T08:33:58.590361+0000","last_clean":"2024-04-25T08:33:58.590361+0000","last_became_active":"2024-04-25T08:33:56.575727+0000","last_became_peered":"2024-04-25T08:33:56.575727+0000","last_unstale":"2024-04-25T08:33:58.590361+0000","last_undegraded":"2024-04-25T08:33:58.590361+0000","last_fullsized":"2024-04-25T08:33:58.590361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:12.088197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579406+0000","last_change":"2024-04-25T08:33:57.587533+0000","last_active":"2024-04-25T08:33:59.579406+0000","last_peered":"2024-04-25T08:33:59.579406+0000","last_clean":"2024-04-25T08:33:59.579406+0000","last_became_active":"2024-04-25T08:33:57.587244+0000","last_became_peered":"2024-04-25T08:33:57.587244+0000","last_unstale":"2024-04-25T08:33:59.579406+0000","last_undegraded":"2024-04-25T08:33:59.579406+0000","last_fullsized":"2024-04-25T08:33:59.579406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:11.916701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590071+0000","last_change":"2024-04-25T08:33:56.576695+0000","last_active":"2024-04-25T08:33:58.590071+0000","last_peered":"2024-04-25T08:33:58.590071+0000","last_clean":"2024-04-25T08:33:58.590071+0000","last_became_active":"2024-04-25T08:33:56.576181+0000","last_became_peered":"2024-04-25T08:33:56.576181+0000","last_unstale":"2024-04-25T08:33:58.590071+0000","last_undegraded":"2024-04-25T08:33:58.590071+0000","last_fullsized":"2024-04-25T08:33:58.590071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:08.663368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596725+0000","last_change":"2024-04-25T08:33:57.590013+0000","last_active":"2024-04-25T08:33:58.596725+0000","last_peered":"2024-04-25T08:33:58.596725+0000","last_clean":"2024-04-25T08:33:58.596725+0000","last_became_active":"2024-04-25T08:33:57.588434+0000","last_became_peered":"2024-04-25T08:33:57.588434+0000","last_unstale":"2024-04-25T08:33:58.596725+0000","last_undegraded":"2024-04-25T08:33:58.596725+0000","last_fullsized":"2024-04-25T08:33:58.596725+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.667742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590105+0000","last_change":"2024-04-25T08:33:56.576592+0000","last_active":"2024-04-25T08:33:58.590105+0000","last_peered":"2024-04-25T08:33:58.590105+0000","last_clean":"2024-04-25T08:33:58.590105+0000","last_became_active":"2024-04-25T08:33:56.576332+0000","last_became_peered":"2024-04-25T08:33:56.576332+0000","last_unstale":"2024-04-25T08:33:58.590105+0000","last_undegraded":"2024-04-25T08:33:58.590105+0000","last_fullsized":"2024-04-25T08:33:58.590105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:32.116736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170780+0000","last_change":"2024-04-25T08:33:57.591109+0000","last_active":"2024-04-25T08:34:00.170780+0000","last_peered":"2024-04-25T08:34:00.170780+0000","last_clean":"2024-04-25T08:34:00.170780+0000","last_became_active":"2024-04-25T08:33:57.590816+0000","last_became_peered":"2024-04-25T08:33:57.590816+0000","last_unstale":"2024-04-25T08:34:00.170780+0000","last_undegraded":"2024-04-25T08:34:00.170780+0000","last_fullsized":"2024-04-25T08:34:00.170780+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:49.640150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.911704+0000","last_change":"2024-04-25T08:33:56.576983+0000","last_active":"2024-04-25T08:34:00.911704+0000","last_peered":"2024-04-25T08:34:00.911704+0000","last_clean":"2024-04-25T08:34:00.911704+0000","last_became_active":"2024-04-25T08:33:56.576721+0000","last_became_peered":"2024-04-25T08:33:56.576721+0000","last_unstale":"2024-04-25T08:34:00.911704+0000","last_undegraded":"2024-04-25T08:34:00.911704+0000","last_fullsized":"2024-04-25T08:34:00.911704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:12.088197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596050+0000","last_change":"2024-04-25T08:33:57.588606+0000","last_active":"2024-04-25T08:33:58.596050+0000","last_peered":"2024-04-25T08:33:58.596050+0000","last_clean":"2024-04-25T08:33:58.596050+0000","last_became_active":"2024-04-25T08:33:57.588320+0000","last_became_peered":"2024-04-25T08:33:57.588320+0000","last_unstale":"2024-04-25T08:33:58.596050+0000","last_undegraded":"2024-04-25T08:33:58.596050+0000","last_fullsized":"2024-04-25T08:33:58.596050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.870396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.580699+0000","last_change":"2024-04-25T08:33:56.583729+0000","last_active":"2024-04-25T08:33:59.580699+0000","last_peered":"2024-04-25T08:33:59.580699+0000","last_clean":"2024-04-25T08:33:59.580699+0000","last_became_active":"2024-04-25T08:33:56.583455+0000","last_became_peered":"2024-04-25T08:33:56.583455+0000","last_unstale":"2024-04-25T08:33:59.580699+0000","last_undegraded":"2024-04-25T08:33:59.580699+0000","last_fullsized":"2024-04-25T08:33:59.580699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:15.222998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:34:00.170861+0000","last_change":"2024-04-25T08:33:57.599741+0000","last_active":"2024-04-25T08:34:00.170861+0000","last_peered":"2024-04-25T08:34:00.170861+0000","last_clean":"2024-04-25T08:34:00.170861+0000","last_became_active":"2024-04-25T08:33:57.599416+0000","last_became_peered":"2024-04-25T08:33:57.599416+0000","last_unstale":"2024-04-25T08:34:00.170861+0000","last_undegraded":"2024-04-25T08:34:00.170861+0000","last_fullsized":"2024-04-25T08:34:00.170861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:08.134103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595485+0000","last_change":"2024-04-25T08:33:56.574282+0000","last_active":"2024-04-25T08:33:58.595485+0000","last_peered":"2024-04-25T08:33:58.595485+0000","last_clean":"2024-04-25T08:33:58.595485+0000","last_became_active":"2024-04-25T08:33:56.573840+0000","last_became_peered":"2024-04-25T08:33:56.573840+0000","last_unstale":"2024-04-25T08:33:58.595485+0000","last_undegraded":"2024-04-25T08:33:58.595485+0000","last_fullsized":"2024-04-25T08:33:58.595485+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:28.209229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.579177+0000","last_change":"2024-04-25T08:33:57.604584+0000","last_active":"2024-04-25T08:33:59.579177+0000","last_peered":"2024-04-25T08:33:59.579177+0000","last_clean":"2024-04-25T08:33:59.579177+0000","last_became_active":"2024-04-25T08:33:57.604055+0000","last_became_peered":"2024-04-25T08:33:57.604055+0000","last_unstale":"2024-04-25T08:33:59.579177+0000","last_undegraded":"2024-04-25T08:33:59.579177+0000","last_fullsized":"2024-04-25T08:33:59.579177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:14.276030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.594880+0000","last_change":"2024-04-25T08:33:56.576675+0000","last_active":"2024-04-25T08:33:58.594880+0000","last_peered":"2024-04-25T08:33:58.594880+0000","last_clean":"2024-04-25T08:33:58.594880+0000","last_became_active":"2024-04-25T08:33:56.576403+0000","last_became_peered":"2024-04-25T08:33:56.576403+0000","last_unstale":"2024-04-25T08:33:58.594880+0000","last_undegraded":"2024-04-25T08:33:58.594880+0000","last_fullsized":"2024-04-25T08:33:58.594880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596129+0000","last_change":"2024-04-25T08:33:57.592225+0000","last_active":"2024-04-25T08:33:58.596129+0000","last_peered":"2024-04-25T08:33:58.596129+0000","last_clean":"2024-04-25T08:33:58.596129+0000","last_became_active":"2024-04-25T08:33:57.591964+0000","last_became_peered":"2024-04-25T08:33:57.591964+0000","last_unstale":"2024-04-25T08:33:58.596129+0000","last_undegraded":"2024-04-25T08:33:58.596129+0000","last_fullsized":"2024-04-25T08:33:58.596129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.203855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595875+0000","last_change":"2024-04-25T08:33:56.573583+0000","last_active":"2024-04-25T08:33:58.595875+0000","last_peered":"2024-04-25T08:33:58.595875+0000","last_clean":"2024-04-25T08:33:58.595875+0000","last_became_active":"2024-04-25T08:33:56.573325+0000","last_became_peered":"2024-04-25T08:33:56.573325+0000","last_unstale":"2024-04-25T08:33:58.595875+0000","last_undegraded":"2024-04-25T08:33:58.595875+0000","last_fullsized":"2024-04-25T08:33:58.595875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.165536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.596276+0000","last_change":"2024-04-25T08:33:57.585403+0000","last_active":"2024-04-25T08:33:58.596276+0000","last_peered":"2024-04-25T08:33:58.596276+0000","last_clean":"2024-04-25T08:33:58.596276+0000","last_became_active":"2024-04-25T08:33:57.584657+0000","last_became_peered":"2024-04-25T08:33:57.584657+0000","last_unstale":"2024-04-25T08:33:58.596276+0000","last_undegraded":"2024-04-25T08:33:58.596276+0000","last_fullsized":"2024-04-25T08:33:58.596276+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:09.668384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.581507+0000","last_change":"2024-04-25T08:33:56.577808+0000","last_active":"2024-04-25T08:33:59.581507+0000","last_peered":"2024-04-25T08:33:59.581507+0000","last_clean":"2024-04-25T08:33:59.581507+0000","last_became_active":"2024-04-25T08:33:56.577539+0000","last_became_peered":"2024-04-25T08:33:56.577539+0000","last_unstale":"2024-04-25T08:33:59.581507+0000","last_undegraded":"2024-04-25T08:33:59.581507+0000","last_fullsized":"2024-04-25T08:33:59.581507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:32.170181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976204+0000","last_change":"2024-04-25T08:33:57.596692+0000","last_active":"2024-04-25T08:33:59.976204+0000","last_peered":"2024-04-25T08:33:59.976204+0000","last_clean":"2024-04-25T08:33:59.976204+0000","last_became_active":"2024-04-25T08:33:57.596423+0000","last_became_peered":"2024-04-25T08:33:57.596423+0000","last_unstale":"2024-04-25T08:33:59.976204+0000","last_undegraded":"2024-04-25T08:33:59.976204+0000","last_fullsized":"2024-04-25T08:33:59.976204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.870396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595977+0000","last_change":"2024-04-25T08:33:56.571995+0000","last_active":"2024-04-25T08:33:58.595977+0000","last_peered":"2024-04-25T08:33:58.595977+0000","last_clean":"2024-04-25T08:33:58.595977+0000","last_became_active":"2024-04-25T08:33:56.571692+0000","last_became_peered":"2024-04-25T08:33:56.571692+0000","last_unstale":"2024-04-25T08:33:58.595977+0000","last_undegraded":"2024-04-25T08:33:58.595977+0000","last_fullsized":"2024-04-25T08:33:58.595977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:01.835536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.976214+0000","last_change":"2024-04-25T08:33:57.593871+0000","last_active":"2024-04-25T08:33:59.976214+0000","last_peered":"2024-04-25T08:33:59.976214+0000","last_clean":"2024-04-25T08:33:59.976214+0000","last_became_active":"2024-04-25T08:33:57.593310+0000","last_became_peered":"2024-04-25T08:33:57.593310+0000","last_unstale":"2024-04-25T08:33:59.976214+0000","last_undegraded":"2024-04-25T08:33:59.976214+0000","last_fullsized":"2024-04-25T08:33:59.976214+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.203855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:59.975880+0000","last_change":"2024-04-25T08:33:56.572227+0000","last_active":"2024-04-25T08:33:59.975880+0000","last_peered":"2024-04-25T08:33:59.975880+0000","last_clean":"2024-04-25T08:33:59.975880+0000","last_became_active":"2024-04-25T08:33:56.571968+0000","last_became_peered":"2024-04-25T08:33:56.571968+0000","last_unstale":"2024-04-25T08:33:59.975880+0000","last_undegraded":"2024-04-25T08:33:59.975880+0000","last_fullsized":"2024-04-25T08:33:59.975880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:01.835536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595552+0000","last_change":"2024-04-25T08:33:57.600355+0000","last_active":"2024-04-25T08:33:58.595552+0000","last_peered":"2024-04-25T08:33:58.595552+0000","last_clean":"2024-04-25T08:33:58.595552+0000","last_became_active":"2024-04-25T08:33:57.600053+0000","last_became_peered":"2024-04-25T08:33:57.600053+0000","last_unstale":"2024-04-25T08:33:58.595552+0000","last_undegraded":"2024-04-25T08:33:58.595552+0000","last_fullsized":"2024-04-25T08:33:58.595552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:56.557627+0000","last_clean_scrub_stamp":"2024-04-25T08:33:56.557627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:47.869752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.590534+0000","last_change":"2024-04-25T08:33:56.577402+0000","last_active":"2024-04-25T08:33:58.590534+0000","last_peered":"2024-04-25T08:33:58.590534+0000","last_clean":"2024-04-25T08:33:58.590534+0000","last_became_active":"2024-04-25T08:33:56.576932+0000","last_became_peered":"2024-04-25T08:33:56.576932+0000","last_unstale":"2024-04-25T08:33:58.590534+0000","last_undegraded":"2024-04-25T08:33:58.590534+0000","last_fullsized":"2024-04-25T08:33:58.590534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:33:55.552611+0000","last_clean_scrub_stamp":"2024-04-25T08:33:55.552611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:31.186363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":288,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-04-25T08:33:58.595075+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:33:58.595075+0000","last_peered":"2024-04-25T08:33:58.595075+0000","last_clean":"2024-04-25T08:33:58.595075+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:33:58.595075+0000","last_undegraded":"2024-04-25T08:33:58.595075+0000","last_fullsized":"2024-04-25T08:33:58.595075+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1609728,"data_stored":1055995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335492,"statfs":{"total":96636764160,"available":96599543808,"internally_reserved":0,"allocated":1024000,"data_stored":467217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335508,"statfs":{"total":96636764160,"available":96599560192,"internally_reserved":0,"allocated":1015808,"data_stored":465639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335488,"statfs":{"total":96636764160,"available":96599539712,"internally_reserved":0,"allocated":1032192,"data_stored":467321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40500,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331340,"statfs":{"total":96636764160,"available":96595292160,"internally_reserved":0,"allocated":1015808,"data_stored":466187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40504,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331336,"statfs":{"total":96636764160,"available":96595288064,"internally_reserved":0,"allocated":1024000,"data_stored":466347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1019904,"data_stored":465717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343248,"statfs":{"total":96636764160,"available":96607485952,"internally_reserved":0,"allocated":1605632,"data_stored":1055973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:05.344 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:34:05.344 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:34:05.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.492+0000 7f148f99e640 1 Processor -- start 2024-04-25T08:34:05.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.493+0000 7f148f99e640 1 -- start start 2024-04-25T08:34:05.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.493+0000 7f148f99e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14881616f0 0x7f1488161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.493+0000 7f148f99e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f14880ce0a0 0x7f14880ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.493+0000 7f148f99e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148806c8d0 con 0x7f14881610f0 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.493+0000 7f148f99e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14880cea50 con 0x7f14881616f0 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.493+0000 7f148f99e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14881614d0 con 0x7f14880ce0a0 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14881616f0 0x7f1488161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14881616f0 0x7f1488161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56684/0 (socket says 172.21.15.60:56684) 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 -- 172.21.15.60:0/1671345944 learned_addr learned my addr 172.21.15.60:0/1671345944 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148d73a640 1 --1- 172.21.15.60:0/1671345944 >> v1:172.21.15.60:6789/0 conn(0x7f14881610f0 0x7f148806c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59562/0 (socket says 172.21.15.60:59562) 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148df3b640 1 --2- 172.21.15.60:0/1671345944 >> v2:172.21.15.100:3300/0 conn(0x7f14880ce0a0 0x7f14880ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 -- 172.21.15.60:0/1671345944 >> v2:172.21.15.100:3300/0 conn(0x7f14880ce0a0 msgr2=0x7f14880ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 --2- 172.21.15.60:0/1671345944 >> v2:172.21.15.100:3300/0 conn(0x7f14880ce0a0 0x7f14880ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 -- 172.21.15.60:0/1671345944 >> v1:172.21.15.60:6789/0 conn(0x7f14881610f0 legacy=0x7f148806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.494+0000 7f148cf39640 1 -- 172.21.15.60:0/1671345944 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880c6960 con 0x7f14881616f0 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148cf39640 1 --2- 172.21.15.60:0/1671345944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881616f0 0x7f1488161b50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f14880b33e0 tx=0x7f147802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f2fb554ef0e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f147ffff640 1 -- 172.21.15.60:0/1671345944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478037070 con 0x7f14881616f0 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f147ffff640 1 -- 172.21.15.60:0/1671345944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478032440 con 0x7f14881616f0 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f147ffff640 1 -- 172.21.15.60:0/1671345944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478032810 con 0x7f14881616f0 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148f99e640 1 -- 172.21.15.60:0/1671345944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881616f0 msgr2=0x7f1488161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148f99e640 1 --2- 172.21.15.60:0/1671345944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881616f0 0x7f1488161b50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f14880b33e0 tx=0x7f147802a270 comp rx=0 tx=0).stop 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148f99e640 1 -- 172.21.15.60:0/1671345944 shutdown_connections 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148f99e640 1 --2- 172.21.15.60:0/1671345944 >> v2:172.21.15.100:3300/0 conn(0x7f14880ce0a0 0x7f14880ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148f99e640 1 --2- 172.21.15.60:0/1671345944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881616f0 0x7f1488161b50 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.495+0000 7f148f99e640 1 -- 172.21.15.60:0/1671345944 >> 172.21.15.60:0/1671345944 conn(0x7f148806f500 msgr2=0x7f148806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.496+0000 7f148f99e640 1 -- 172.21.15.60:0/1671345944 shutdown_connections 2024-04-25T08:34:05.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.496+0000 7f148f99e640 1 -- 172.21.15.60:0/1671345944 wait complete. 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 Processor -- start 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 -- start start 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14880ce0a0 0x7f14881f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 0x7f148815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14881616f0 0x7f148815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880cfa00 con 0x7f14881610f0 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1488162540 con 0x7f14880ce0a0 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.497+0000 7f148f99e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14880cfce0 con 0x7f14881616f0 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 0x7f148815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 0x7f148815a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56688/0 (socket says 172.21.15.60:56688) 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 -- 172.21.15.60:0/4159691831 learned_addr learned my addr 172.21.15.60:0/4159691831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148d73a640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14880ce0a0 0x7f14881f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148df3b640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14881616f0 0x7f148815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 -- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14881616f0 msgr2=0x7f148815c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14881616f0 0x7f148815c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 -- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14880ce0a0 msgr2=0x7f14881f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14880ce0a0 0x7f14881f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 -- 172.21.15.60:0/4159691831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1488157f50 con 0x7f14881610f0 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148df3b640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14881616f0 0x7f148815c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148cf39640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 0x7f148815a140 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f147802a2b0 tx=0x7f14780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478037040 con 0x7f14881610f0 2024-04-25T08:34:05.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1478042070 con 0x7f14881610f0 2024-04-25T08:34:05.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478030880 con 0x7f14881610f0 2024-04-25T08:34:05.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.498+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1488158230 con 0x7f14881610f0 2024-04-25T08:34:05.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.499+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14881587f0 con 0x7f14881610f0 2024-04-25T08:34:05.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.499+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14880cf150 con 0x7f14881610f0 2024-04-25T08:34:05.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.500+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1478030a20 con 0x7f14881610f0 2024-04-25T08:34:05.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.500+0000 7f147dffb640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f146c06b4d0 0x7f146c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.500+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14780aac30 con 0x7f14881610f0 2024-04-25T08:34:05.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.501+0000 7f148d73a640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f146c06b4d0 0x7f146c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.501+0000 7f148d73a640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f146c06b4d0 0x7f146c06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1484007fd0 tx=0x7f1484007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.505+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1478079890 con 0x7f14881610f0 2024-04-25T08:34:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.759+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f14880739e0 con 0x7f14881610f0 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.761+0000 7f147dffb640 1 -- 172.21.15.60:0/4159691831 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1478079190 con 0x7f14881610f0 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:34:05.757 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:34:05.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f146c06b4d0 msgr2=0x7f146c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f146c06b4d0 0x7f146c06d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1484007fd0 tx=0x7f1484007e80 comp rx=0 tx=0).stop 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 msgr2=0x7f148815a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 0x7f148815a140 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f147802a2b0 tx=0x7f14780046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 shutdown_connections 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14881616f0 0x7f148815c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14881610f0 0x7f148815a140 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f146c06b4d0 0x7f146c06d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 --2- 172.21.15.60:0/4159691831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14880ce0a0 0x7f14881f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 >> 172.21.15.60:0/4159691831 conn(0x7f148806f500 msgr2=0x7f14880b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 shutdown_connections 2024-04-25T08:34:05.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.764+0000 7f148f99e640 1 -- 172.21.15.60:0/4159691831 wait complete. 2024-04-25T08:34:05.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:05.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.875+0000 7fc091177640 1 Processor -- start 2024-04-25T08:34:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.875+0000 7fc091177640 1 -- start start 2024-04-25T08:34:05.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc091177640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 0x7fc08c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc091177640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c0b9940 con 0x7fc08c15dd30 2024-04-25T08:34:05.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc091177640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c0d1df0 con 0x7fc08c0d0900 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc091177640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0d0d70 con 0x7fc08c0d2e90 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc08ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc08c15dd30 0x7fc08c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59564/0 (socket says 172.21.15.60:59564) 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc08ad9d640 1 -- 172.21.15.60:0/1212195260 learned_addr learned my addr 172.21.15.60:0/1212195260 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc08a59c640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 0x7fc08c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184797704 0 0) 0x7fc08c0b9940 con 0x7fc08c15dd30 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.876+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003520 con 0x7fc08c15dd30 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3340892427 0 0) 0x7fc078003520 con 0x7fc08c15dd30 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 msgr2=0x7fc08c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 0x7fc08c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 >> v1:172.21.15.100:6789/0 conn(0x7fc08c0d0900 legacy=0x7fc08c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c0d20d0 con 0x7fc08c15dd30 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fc084001670 con 0x7fc08c15dd30 2024-04-25T08:34:05.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 >> v1:172.21.15.60:6789/0 conn(0x7fc08c15dd30 legacy=0x7fc08c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc078004800 0x7fc078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 0x7fc078005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0d0d70 con 0x7fc078004800 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800baf0 con 0x7fc0780039e0 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800bcf0 con 0x7fc08c0d2e90 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08a59c640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc078004800 0x7fc078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08ad9d640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08b59e640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 0x7fc078005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08a59c640 1 -- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 msgr2=0x7fc078005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08a59c640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc08c0d2e90 0x7fc078005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08a59c640 1 -- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0780039e0 msgr2=0x7fc078003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08a59c640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.877+0000 7fc08a59c640 1 -- 172.21.15.60:0/1212195260 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078003520 con 0x7fc078004800 2024-04-25T08:34:05.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.878+0000 7fc08a59c640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc078004800 0x7fc078004c80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc074009ab0 tx=0x7fc078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac58a1992c52bf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.878+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074036070 con 0x7fc078004800 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.878+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc074005990 con 0x7fc078004800 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.878+0000 7fc089d9b640 1 -- 172.21.15.60:0/1212195260 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074009680 con 0x7fc078004800 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc091177640 1 -- 172.21.15.60:0/1212195260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc078004800 msgr2=0x7fc078004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc091177640 1 --2- 172.21.15.60:0/1212195260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc078004800 0x7fc078004c80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc074009ab0 tx=0x7fc078005830 comp rx=0 tx=0).stop 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc08b59e640 1 -- 172.21.15.60:0/1212195260 reap_dead start 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc091177640 1 -- 172.21.15.60:0/1212195260 shutdown_connections 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc091177640 1 -- 172.21.15.60:0/1212195260 >> 172.21.15.60:0/1212195260 conn(0x7fc08c061c30 msgr2=0x7fc08c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc091177640 1 -- 172.21.15.60:0/1212195260 shutdown_connections 2024-04-25T08:34:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.879+0000 7fc091177640 1 -- 172.21.15.60:0/1212195260 wait complete. 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.880+0000 7fc091177640 1 Processor -- start 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.880+0000 7fc091177640 1 -- start start 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc091177640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc078004800 0x7fc08c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc091177640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08c1f72b0 0x7fc08c1f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc091177640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 0x7fc08c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc091177640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0d2970 con 0x7fc08c1f7c50 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc091177640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0d2690 con 0x7fc08c1f72b0 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc091177640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0d1a10 con 0x7fc078004800 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 0x7fc08c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 0x7fc08c157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56702/0 (socket says 172.21.15.60:56702) 2024-04-25T08:34:05.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 -- 172.21.15.60:0/2518403229 learned_addr learned my addr 172.21.15.60:0/2518403229 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08ad9d640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc078004800 0x7fc08c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08a59c640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08c1f72b0 0x7fc08c1f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 -- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc078004800 msgr2=0x7fc08c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc078004800 0x7fc08c159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 -- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08c1f72b0 msgr2=0x7fc08c1f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08c1f72b0 0x7fc08c1f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.881+0000 7fc08b59e640 1 -- 172.21.15.60:0/2518403229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c0b4620 con 0x7fc08c1f7c50 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.882+0000 7fc08b59e640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 0x7fc08c157420 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0ce200 tx=0x7fc07c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.882+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c003b80 con 0x7fc08c1f7c50 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.882+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c018070 con 0x7fc08c1f7c50 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.882+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0050d0 con 0x7fc08c1f7c50 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.882+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c0b4900 con 0x7fc08c1f7c50 2024-04-25T08:34:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.882+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c0b51a0 con 0x7fc08c1f7c50 2024-04-25T08:34:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.883+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c0b4ad0 con 0x7fc08c1f7c50 2024-04-25T08:34:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.883+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc07c005270 con 0x7fc08c1f7c50 2024-04-25T08:34:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.884+0000 7fc0837fe640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc06c06b500 0x7fc06c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.884+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc07c08a250 con 0x7fc08c1f7c50 2024-04-25T08:34:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.884+0000 7fc08ad9d640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc06c06b500 0x7fc06c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:05.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.884+0000 7fc08ad9d640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc06c06b500 0x7fc06c06d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc0840078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:05.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:05.888+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc07c0587d0 con 0x7fc08c1f7c50 2024-04-25T08:34:06.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.104+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc08c0b53a0 con 0x7fc08c1f7c50 2024-04-25T08:34:06.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.104+0000 7fc0837fe640 1 -- 172.21.15.60:0/2518403229 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc08c0b53a0 con 0x7fc08c1f7c50 2024-04-25T08:34:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:34:06.100 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:33:56.967441+0000","modified":"2024-04-25T08:33:58.932396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7203":{"gid":7203,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1892944367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1892944367},{"type":"v1","addr":"172.21.15.100:6837","nonce":1892944367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:34:06.100 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 48 2024-04-25T08:34:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc06c06b500 msgr2=0x7fc06c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc06c06b500 0x7fc06c06d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc0840078f0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 msgr2=0x7fc08c157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 0x7fc08c157420 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0ce200 tx=0x7fc07c00fd40 comp rx=0 tx=0).stop 2024-04-25T08:34:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 shutdown_connections 2024-04-25T08:34:06.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc08c1f7c50 0x7fc08c157420 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc06c06b500 0x7fc06c06d9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc08c1f72b0 0x7fc08c1f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 --2- 172.21.15.60:0/2518403229 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc078004800 0x7fc08c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 >> 172.21.15.60:0/2518403229 conn(0x7fc08c061c30 msgr2=0x7fc08c0b9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 shutdown_connections 2024-04-25T08:34:06.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.106+0000 7fc091177640 1 -- 172.21.15.60:0/2518403229 wait complete. 2024-04-25T08:34:06.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:33:56.967441+0000', 'modified': '2024-04-25T08:33:58.932396+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7203': {'gid': 7203, 'name': 'b', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/1892944367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 1892944367}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 1892944367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:34:06.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:34:06.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:34:06.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.626+0000 7f57a52e1640 1 Processor -- start 2024-04-25T08:34:06.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f57a52e1640 1 -- start start 2024-04-25T08:34:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f57a52e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a015b7d0 0x7f57a00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f57a52e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f57a52e1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a006cf50 con 0x7f57a00b5620 2024-04-25T08:34:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f57a52e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00ce530 con 0x7f57a015b7d0 2024-04-25T08:34:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f57a52e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00ce810 con 0x7f57a006c9e0 2024-04-25T08:34:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f579effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a015b7d0 0x7f57a00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f579effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57a015b7d0 0x7f57a00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56718/0 (socket says 172.21.15.60:56718) 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f579effd640 1 -- 172.21.15.60:0/3285435962 learned_addr learned my addr 172.21.15.60:0/3285435962 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.627+0000 7f579e7fc640 1 --1- 172.21.15.60:0/3285435962 >> v1:172.21.15.60:6789/0 conn(0x7f57a00b5620 0x7f57a006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59576/0 (socket says 172.21.15.60:59576) 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.628+0000 7f579effd640 1 -- 172.21.15.60:0/3285435962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 msgr2=0x7f57a00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.628+0000 7f579effd640 1 --2- 172.21.15.60:0/3285435962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.628+0000 7f579effd640 1 -- 172.21.15.60:0/3285435962 >> v1:172.21.15.60:6789/0 conn(0x7f57a00b5620 legacy=0x7f57a006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.628+0000 7f579effd640 1 -- 172.21.15.60:0/3285435962 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00ceaf0 con 0x7f57a015b7d0 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.628+0000 7f579effd640 1 --2- 172.21.15.60:0/3285435962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00cf010 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5798009870 tx=0x7f579802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef37c0d4ae09c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f579dffb640 1 -- 172.21.15.60:0/3285435962 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798037070 con 0x7f57a015b7d0 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f579dffb640 1 -- 172.21.15.60:0/3285435962 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798032440 con 0x7f57a015b7d0 2024-04-25T08:34:06.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f579dffb640 1 -- 172.21.15.60:0/3285435962 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798032810 con 0x7f57a015b7d0 2024-04-25T08:34:06.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 -- 172.21.15.60:0/3285435962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 msgr2=0x7f57a00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 --2- 172.21.15.60:0/3285435962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00cf010 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5798009870 tx=0x7f579802a270 comp rx=0 tx=0).stop 2024-04-25T08:34:06.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 -- 172.21.15.60:0/3285435962 shutdown_connections 2024-04-25T08:34:06.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 --2- 172.21.15.60:0/3285435962 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 --2- 172.21.15.60:0/3285435962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00cf010 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 -- 172.21.15.60:0/3285435962 >> 172.21.15.60:0/3285435962 conn(0x7f57a00d0800 msgr2=0x7f57a00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 -- 172.21.15.60:0/3285435962 shutdown_connections 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.629+0000 7f57a52e1640 1 -- 172.21.15.60:0/3285435962 wait complete. 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 Processor -- start 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 -- start start 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00b5620 0x7f57a014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cf6d0 con 0x7f57a015b7d0 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cf550 con 0x7f57a00b5620 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f57a52e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015bbb0 con 0x7f57a006c9e0 2024-04-25T08:34:06.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.631+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56730/0 (socket says 172.21.15.60:56730) 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 -- 172.21.15.60:0/2901986957 learned_addr learned my addr 172.21.15.60:0/2901986957 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579e7fc640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00b5620 0x7f57a014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579effd640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 -- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 msgr2=0x7f57a00c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 -- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00b5620 msgr2=0x7f57a014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00b5620 0x7f57a014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 -- 172.21.15.60:0/2901986957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a014eeb0 con 0x7f57a015b7d0 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579e7fc640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00b5620 0x7f57a014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f579f7fe640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00c76d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f579000b9e0 tx=0x7f579000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f57977fe640 1 -- 172.21.15.60:0/2901986957 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790019070 con 0x7f57a015b7d0 2024-04-25T08:34:06.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f57977fe640 1 -- 172.21.15.60:0/2901986957 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000cac0 con 0x7f57a015b7d0 2024-04-25T08:34:06.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00c7f20 con 0x7f57a015b7d0 2024-04-25T08:34:06.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f57977fe640 1 -- 172.21.15.60:0/2901986957 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790014440 con 0x7f57a015b7d0 2024-04-25T08:34:06.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.632+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00c84e0 con 0x7f57a015b7d0 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.634+0000 7f57977fe640 1 -- 172.21.15.60:0/2901986957 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f579001c070 con 0x7f57a015b7d0 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.634+0000 7f57977fe640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f578006b4d0 0x7f578006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.634+0000 7f57977fe640 1 -- 172.21.15.60:0/2901986957 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f579008a0b0 con 0x7f57a015b7d0 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.634+0000 7f579effd640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f578006b4d0 0x7f578006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.635+0000 7f579effd640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f578006b4d0 0x7f578006d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5798034000 tx=0x7f579802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.635+0000 7f57a7fe3740 1 Processor -- start 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.635+0000 7f57a7fe3740 1 -- start start 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9ab4c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab6a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab21f10 0x562a9ab68190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9aa84a10 con 0x562a9ab69ad0 2024-04-25T08:34:06.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9aa2db30 con 0x562a9ab4c230 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9aa80c50 con 0x562a9ab21f10 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab6a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab6a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56744/0 (socket says 172.21.15.60:56744) 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 -- 172.21.15.60:0/4201671406 learned_addr learned my addr 172.21.15.60:0/4201671406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579effd640 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9ab4c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579f7fe640 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab21f10 0x562a9ab68190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 -- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab21f10 msgr2=0x562a9ab68190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab21f10 0x562a9ab68190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 -- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 msgr2=0x562a9ab4c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9ab4c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 -- 172.21.15.60:0/4201671406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9aa42e10 con 0x562a9ab69ad0 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579effd640 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9ab4c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:06.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.636+0000 7f579e7fc640 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab6a020 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5788001710 tx=0x7f5788001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e65ae3ed1b010c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f5794ff9640 1 -- 172.21.15.60:0/4201671406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788018070 con 0x562a9ab69ad0 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f5794ff9640 1 -- 172.21.15.60:0/4201671406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57880074e0 con 0x562a9ab69ad0 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f5794ff9640 1 -- 172.21.15.60:0/4201671406 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788005a50 con 0x562a9ab69ad0 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 -- 172.21.15.60:0/4201671406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 msgr2=0x562a9ab6a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab6a020 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5788001710 tx=0x7f5788001bd0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 -- 172.21.15.60:0/4201671406 shutdown_connections 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab21f10 0x562a9ab68190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab6a020 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/4201671406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9ab4c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 -- 172.21.15.60:0/4201671406 >> 172.21.15.60:0/4201671406 conn(0x562a9ab1e840 msgr2=0x562a9ab67040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.637+0000 7f57a7fe3740 1 -- 172.21.15.60:0/4201671406 shutdown_connections 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.638+0000 7f57a7fe3740 1 -- 172.21.15.60:0/4201671406 wait complete. 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.638+0000 7f57a7fe3740 1 -- start start 2024-04-25T08:34:06.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.638+0000 7f57a7fe3740 1 Processor -- start 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 0x562a9ab48d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9ab77500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab4fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ab22cb0 con 0x562a9ab4c230 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9aa410c0 con 0x562a9ab5e1a0 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9ab1c770 con 0x562a9ab69ad0 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f57a7fe3740 10 client.0 _ll_register_callbacks cb 0x562a9aa34ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 0x562a9ab48d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 0x562a9ab48d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56752/0 (socket says 172.21.15.60:56752) 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 -- 172.21.15.60:0/3003477316 learned_addr learned my addr 172.21.15.60:0/3003477316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579f7fe640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab4fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579e7fc640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9ab77500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 -- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 msgr2=0x562a9ab77500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9ab77500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 -- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 msgr2=0x562a9ab4fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab4fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 -- 172.21.15.60:0/3003477316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9a7992d0 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f579effd640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 0x562a9ab48d20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5798009990 tx=0x7f579802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8da64fb30fd0b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.639+0000 7f577dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f577dffb640 1 -- 172.21.15.60:0/3003477316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798037040 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f577dffb640 1 -- 172.21.15.60:0/3003477316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f579803cdc0 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57880074e0 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f577dffb640 1 -- 172.21.15.60:0/3003477316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5798055050 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 10 client.7347 fetch_fsmap learned FSMap version 48 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x562a9a8636b0 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f577dffb640 1 -- 172.21.15.60:0/3003477316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57880074e0 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f577dffb640 1 -- 172.21.15.60:0/3003477316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5798058020 con 0x562a9ab4c230 2024-04-25T08:34:06.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 10 client.7347 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 10 client.7347 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 20 client.7347 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab4a6f0 0x562a9ab4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 4 client.7347 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a9aae3920 con 0x562a9ab4a6f0 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.640+0000 7f579e7fc640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab4a6f0 0x562a9ab4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.641+0000 7f579e7fc640 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab4a6f0 0x562a9ab4d800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9611efc28f1aa5e server_cookie=c2ac0f9dd1bd7433 in_seq=0 out_seq=0 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.641+0000 7f577dffb640 10 client.7347 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.641+0000 7f577dffb640 1 -- 172.21.15.60:0/3003477316 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a9aae3920 con 0x562a9ab4a6f0 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.641+0000 7f577dffb640 10 client.7347 handle_command_reply: tid=0 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.641+0000 7f57a7fe3740 1 client.7347 shutdown 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 msgr2=0x562a9ab48d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 0x562a9ab48d20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5798009990 tx=0x7f579802c070 comp rx=0 tx=0).stop 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 shutdown_connections 2024-04-25T08:34:06.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab4a6f0 0x562a9ab4d800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab4fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9ab77500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f577dffb640 0 client.7347 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/3003477316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab4c230 0x562a9ab48d20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 >> 172.21.15.60:0/3003477316 conn(0x562a9ab21f10 msgr2=0x562a9ab1b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 shutdown_connections 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 1 -- 172.21.15.60:0/3003477316 wait complete. 2024-04-25T08:34:06.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.642+0000 7f57a7fe3740 20 client.7347 trim_cache size 0 max 16384 2024-04-25T08:34:06.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 Processor -- start 2024-04-25T08:34:06.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 -- start start 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab4c230 0x562a9ab6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab6af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab66010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01ff680 con 0x562a9ab69ad0 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cb8d0 con 0x562a9ab5e1a0 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01ff080 con 0x562a9ab4c230 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab66010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f579f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab66010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56760/0 (socket says 172.21.15.60:56760) 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.658+0000 7f579f7fe640 1 -- 172.21.15.60:0/1658751535 learned_addr learned my addr 172.21.15.60:0/1658751535 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579effd640 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab4c230 0x562a9ab6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579e7fc640 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab6af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579f7fe640 1 -- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab4c230 msgr2=0x562a9ab6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab4c230 0x562a9ab6aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579f7fe640 1 -- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 msgr2=0x562a9ab6af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab6af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579f7fe640 1 -- 172.21.15.60:0/1658751535 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9ab81910 con 0x562a9ab69ad0 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab66010 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f579000cfd0 tx=0x7f579008a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade3b7714ab4dade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f577ffff640 1 -- 172.21.15.60:0/1658751535 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790070170 con 0x562a9ab69ad0 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f577ffff640 1 -- 172.21.15.60:0/1658751535 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790056910 con 0x562a9ab69ad0 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f577ffff640 1 -- 172.21.15.60:0/1658751535 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790060920 con 0x562a9ab69ad0 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1658751535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 msgr2=0x562a9ab66010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.659+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab66010 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f579000cfd0 tx=0x7f579008a040 comp rx=0 tx=0).stop 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1658751535 shutdown_connections 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab69ad0 0x562a9ab66010 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab5e1a0 0x562a9ab6af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1658751535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab4c230 0x562a9ab6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1658751535 >> 172.21.15.60:0/1658751535 conn(0x562a9ab21f10 msgr2=0x562a9ab89e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1658751535 shutdown_connections 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1658751535 wait complete. 2024-04-25T08:34:06.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 -- start start 2024-04-25T08:34:06.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.660+0000 7f57a7fe3740 1 Processor -- start 2024-04-25T08:34:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9abd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 0x562a9ab66410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9abd2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ab4a570 con 0x562a9ab5e1a0 2024-04-25T08:34:06.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ab4f5f0 con 0x562a9ab4c230 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9ab4ab70 con 0x562a9ab69ad0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f57a7fe3740 10 client.0 _ll_register_callbacks cb 0x562a9ab1cc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 0x562a9ab66410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 0x562a9ab66410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56772/0 (socket says 172.21.15.60:56772) 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 -- 172.21.15.60:0/1575227344 learned_addr learned my addr 172.21.15.60:0/1575227344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579f7fe640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9abd2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579effd640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9abd5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 -- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 msgr2=0x562a9abd2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9abd2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 -- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 msgr2=0x562a9abd5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9abd5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.661+0000 7f579e7fc640 1 -- 172.21.15.60:0/1575227344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9abc90d0 con 0x562a9ab5e1a0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f579e7fc640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 0x562a9ab66410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f57880016e0 tx=0x7f5788000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6a3e6772a5624b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f577dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f577dffb640 1 -- 172.21.15.60:0/1575227344 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788018070 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f577dffb640 1 -- 172.21.15.60:0/1575227344 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57880049e0 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5790056910 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f577dffb640 1 -- 172.21.15.60:0/1575227344 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5788033050 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 10 client.7353 fetch_fsmap learned FSMap version 48 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x562a9ab49630 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f577dffb640 1 -- 172.21.15.60:0/1575227344 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790056910 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f577dffb640 1 -- 172.21.15.60:0/1575227344 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5788036020 con 0x562a9ab5e1a0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 10 client.7353 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 10 client.7353 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 20 client.7353 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab5cd80 0x562a9ab5d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 4 client.7353 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562a9aa91f50 con 0x562a9ab5cd80 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.662+0000 7f579effd640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab5cd80 0x562a9ab5d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f579effd640 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab5cd80 0x562a9ab5d160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=255681378f216a24 server_cookie=4e9c5356b7d6b1bc in_seq=0 out_seq=0 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f577dffb640 10 client.7353 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f577dffb640 1 -- 172.21.15.60:0/1575227344 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562a9aa91f50 con 0x562a9ab5cd80 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f577dffb640 10 client.7353 handle_command_reply: tid=0 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 client.7353 shutdown 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 msgr2=0x562a9ab66410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 0x562a9ab66410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f57880016e0 tx=0x7f5788000bc0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 shutdown_connections 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x562a9ab5cd80 0x562a9ab5d160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562a9ab69ad0 0x562a9abd2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562a9ab5e1a0 0x562a9ab66410 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f577dffb640 0 client.7353 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 --2- 172.21.15.60:0/1575227344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562a9ab4c230 0x562a9abd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 >> 172.21.15.60:0/1575227344 conn(0x562a9ab21f10 msgr2=0x562a9ab66f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 shutdown_connections 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 1 -- 172.21.15.60:0/1575227344 wait complete. 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.663+0000 7f57a7fe3740 20 client.7353 trim_cache size 0 max 16384 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7203, 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:34:06.665 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.0923121170000005, 2024-04-25T08:34:06.666 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 48, 2024-04-25T08:34:06.666 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 83, 2024-04-25T08:34:06.666 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 83, 2024-04-25T08:34:06.666 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.139436505999999 2024-04-25T08:34:06.666 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f578006b4d0 msgr2=0x7f578006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f578006b4d0 0x7f578006d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5798034000 tx=0x7f579802e000 comp rx=0 tx=0).stop 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 msgr2=0x7f57a00c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00c76d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f579000b9e0 tx=0x7f579000bea0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 shutdown_connections 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a015b7d0 0x7f57a00c76d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f578006b4d0 0x7f578006d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a00b5620 0x7f57a014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 --2- 172.21.15.60:0/2901986957 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57a006c9e0 0x7f57a00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 >> 172.21.15.60:0/2901986957 conn(0x7f57a00d0800 msgr2=0x7f57a00d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 shutdown_connections 2024-04-25T08:34:06.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.664+0000 7f57a52e1640 1 -- 172.21.15.60:0/2901986957 wait complete. 2024-04-25T08:34:06.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:34:06.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.775+0000 7fa4c31a5640 1 Processor -- start 2024-04-25T08:34:06.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.775+0000 7fa4c31a5640 1 -- start start 2024-04-25T08:34:06.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.776+0000 7fa4c31a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.776+0000 7fa4c31a5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa4bc0b2cb0 0x7fa4bc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.776+0000 7fa4c31a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4bc0b97a0 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.776+0000 7fa4c31a5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4bc15dfc0 con 0x7fa4bc0b2cb0 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.776+0000 7fa4c31a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0d1a00 con 0x7fa4bc15dbe0 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4bbfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa4bc0b2cb0 0x7fa4bc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:51068/0 (socket says 172.21.15.60:51068) 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 -- 172.21.15.60:0/2798591540 learned_addr learned my addr 172.21.15.60:0/2798591540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 -- 172.21.15.60:0/2798591540 >> v2:172.21.15.100:3300/0 conn(0x7fa4bc0b2cb0 msgr2=0x7fa4bc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 --2- 172.21.15.60:0/2798591540 >> v2:172.21.15.100:3300/0 conn(0x7fa4bc0b2cb0 0x7fa4bc0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 -- 172.21.15.60:0/2798591540 >> v1:172.21.15.100:6789/0 conn(0x7fa4bc0b92a0 legacy=0x7fa4bc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4c0f41640 1 -- 172.21.15.60:0/2798591540 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc0d1c50 con 0x7fa4bc15dbe0 2024-04-25T08:34:06.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.777+0000 7fa4bbfff640 1 --2- 172.21.15.60:0/2798591540 >> v2:172.21.15.100:3300/0 conn(0x7fa4bc0b2cb0 0x7fa4bc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c0f41640 1 --2- 172.21.15.60:0/2798591540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc0d0910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009870 tx=0x7fa4b402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e964f47222bcc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4bb7fe640 1 -- 172.21.15.60:0/2798591540 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4037070 con 0x7fa4bc15dbe0 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4bb7fe640 1 -- 172.21.15.60:0/2798591540 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b4032440 con 0x7fa4bc15dbe0 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4bb7fe640 1 -- 172.21.15.60:0/2798591540 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4032810 con 0x7fa4bc15dbe0 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c31a5640 1 -- 172.21.15.60:0/2798591540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 msgr2=0x7fa4bc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/2798591540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc0d0910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009870 tx=0x7fa4b402a270 comp rx=0 tx=0).stop 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c31a5640 1 -- 172.21.15.60:0/2798591540 shutdown_connections 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/2798591540 >> v2:172.21.15.100:3300/0 conn(0x7fa4bc0b2cb0 0x7fa4bc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/2798591540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc0d0910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.778+0000 7fa4c31a5640 1 -- 172.21.15.60:0/2798591540 >> 172.21.15.60:0/2798591540 conn(0x7fa4bc061a20 msgr2=0x7fa4bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.779+0000 7fa4c31a5640 1 -- 172.21.15.60:0/2798591540 shutdown_connections 2024-04-25T08:34:06.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.779+0000 7fa4c31a5640 1 -- 172.21.15.60:0/2798591540 wait complete. 2024-04-25T08:34:06.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.780+0000 7fa4c31a5640 1 Processor -- start 2024-04-25T08:34:06.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.780+0000 7fa4c31a5640 1 -- start start 2024-04-25T08:34:06.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4bc0b2cb0 0x7fa4bc1f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 0x7fa4bc0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0b6ab0 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0d2210 con 0x7fa4bc0b2cb0 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc0b6c30 con 0x7fa4bc15dbe0 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 0x7fa4bc0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 0x7fa4bc0b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56784/0 (socket says 172.21.15.60:56784) 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 -- 172.21.15.60:0/3785548904 learned_addr learned my addr 172.21.15.60:0/3785548904 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c1742640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c0f41640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4bc0b2cb0 0x7fa4bc1f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 -- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 msgr2=0x7fa4bc15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc15b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 -- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4bc0b2cb0 msgr2=0x7fa4bc1f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:06.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4bc0b2cb0 0x7fa4bc1f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 -- 172.21.15.60:0/3785548904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc0cba40 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c1742640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c0f41640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4bc0b2cb0 0x7fa4bc1f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4bbfff640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 0x7fa4bc0b5b90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00efc0 tx=0x7fa4ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac018020 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ac014680 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc0cbd00 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.781+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac014a50 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.782+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc0cc2c0 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.782+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc073990 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.783+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4ac005d10 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.783+0000 7fa4b97fa640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa49c06b4d0 0x7fa49c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:06.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.783+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4ac08b1d0 con 0x7fa4bc0b92a0 2024-04-25T08:34:06.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.784+0000 7fa4c0f41640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa49c06b4d0 0x7fa49c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:06.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.784+0000 7fa4c0f41640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa49c06b4d0 0x7fa49c06d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4034000 tx=0x7fa4b402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:06.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:06.788+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4ac022030 con 0x7fa4bc0b92a0 2024-04-25T08:34:07.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.006+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa4bc06cae0 con 0x7fa4bc0b92a0 2024-04-25T08:34:07.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.006+0000 7fa4b97fa640 1 -- 172.21.15.60:0/3785548904 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa4ac010040 con 0x7fa4bc0b92a0 2024-04-25T08:34:07.001 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:34:07.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa49c06b4d0 msgr2=0x7fa49c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:07.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa49c06b4d0 0x7fa49c06d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4034000 tx=0x7fa4b402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:07.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 msgr2=0x7fa4bc0b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 0x7fa4bc0b5b90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00efc0 tx=0x7fa4ac00c490 comp rx=0 tx=0).stop 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 shutdown_connections 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4bc15dbe0 0x7fa4bc15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4bc0b92a0 0x7fa4bc0b5b90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa49c06b4d0 0x7fa49c06d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 --2- 172.21.15.60:0/3785548904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4bc0b2cb0 0x7fa4bc1f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 >> 172.21.15.60:0/3785548904 conn(0x7fa4bc061a20 msgr2=0x7fa4bc0d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 shutdown_connections 2024-04-25T08:34:07.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.008+0000 7fa4c31a5640 1 -- 172.21.15.60:0/3785548904 wait complete. 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:34:07.014 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:34:07.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:34:07.015 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:34:07.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.015 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:34:07.030 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:34:07.030 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:34:07.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42463sec preferred_lft 42463sec 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:34:07.032 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.033 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:34:07.033 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:34:07.033 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:34:07.033 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:07.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.033 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:34:07.093 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:34:07.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.095 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:34:07.152 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:34:07.152 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:34:07.153 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:34:07.153 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:34:07.153 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:34:07.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.153 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:34:07.153 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:34:07.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:34:07.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:34:07.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:34:07.153 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:34:07.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.239 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:34:07.253 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:34:07.253 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:34:07.253 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:34:07.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:34:07.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:34:07.318 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:07.373 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:34:07.373 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:34:07.374 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:07.427 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:34:07.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:34:07.510 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:34:07.510 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:34:07.510 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:34:07.519 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:34:07.519 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:34:07.519 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:34:07.519 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:34:07.519 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:34:07.524 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:34:07.525 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:34:07.525 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:34:07.525 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:34:07.525 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:34:07.526 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:34:07.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:07.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:07.592 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:34:07.655 INFO:teuthology.orchestra.run.smithi060.stdout:client7359 2024-04-25T08:34:07.655 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:34:07.655 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:07.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T08:34:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.861+0000 7f7bca01f640 1 Processor -- start 2024-04-25T08:34:07.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.861+0000 7f7bca01f640 1 -- start start 2024-04-25T08:34:07.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bca01f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7bc406c330 0x7f7bc40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bca01f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bca01f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc40d1d50 con 0x7f7bc4158230 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bca01f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc4158610 con 0x7f7bc406c330 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bca01f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc406bea0 con 0x7f7bc406c7e0 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7bc406c330 0x7f7bc40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7bc406c330 0x7f7bc40b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56798/0 (socket says 172.21.15.60:56798) 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 -- 172.21.15.60:0/2508826799 learned_addr learned my addr 172.21.15.60:0/2508826799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bc3fff640 1 --2- 172.21.15.60:0/2508826799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 -- 172.21.15.60:0/2508826799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 msgr2=0x7f7bc40b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 --2- 172.21.15.60:0/2508826799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 -- 172.21.15.60:0/2508826799 >> v1:172.21.15.100:6789/0 conn(0x7f7bc4158230 legacy=0x7f7bc40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:07.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.862+0000 7f7bbbfff640 1 -- 172.21.15.60:0/2508826799 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc40d2020 con 0x7f7bc406c330 2024-04-25T08:34:07.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.863+0000 7f7bbbfff640 1 --2- 172.21.15.60:0/2508826799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc40b4e60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009850 tx=0x7f7bac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fedb54b1ea48ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.863+0000 7f7bc2ffd640 1 -- 172.21.15.60:0/2508826799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac004030 con 0x7f7bc406c330 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.863+0000 7f7bc2ffd640 1 -- 172.21.15.60:0/2508826799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac0041d0 con 0x7f7bc406c330 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.863+0000 7f7bc2ffd640 1 -- 172.21.15.60:0/2508826799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac003610 con 0x7f7bc406c330 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 -- 172.21.15.60:0/2508826799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 msgr2=0x7f7bc40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 --2- 172.21.15.60:0/2508826799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc40b4e60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009850 tx=0x7f7bac02a230 comp rx=0 tx=0).stop 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 -- 172.21.15.60:0/2508826799 shutdown_connections 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 --2- 172.21.15.60:0/2508826799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc40b4e60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 --2- 172.21.15.60:0/2508826799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 -- 172.21.15.60:0/2508826799 >> 172.21.15.60:0/2508826799 conn(0x7f7bc4061600 msgr2=0x7f7bc40619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 -- 172.21.15.60:0/2508826799 shutdown_connections 2024-04-25T08:34:07.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.864+0000 7f7bca01f640 1 -- 172.21.15.60:0/2508826799 wait complete. 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.865+0000 7f7bca01f640 1 Processor -- start 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.865+0000 7f7bca01f640 1 -- start start 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc41587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bc4158230 0x7f7bc41565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40d25e0 con 0x7f7bc406c330 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc406bb30 con 0x7f7bc4158230 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc415e320 con 0x7f7bc406c7e0 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc41587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc41587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56812/0 (socket says 172.21.15.60:56812) 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 -- 172.21.15.60:0/3222710701 learned_addr learned my addr 172.21.15.60:0/3222710701 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bbbfff640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc3fff640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bc4158230 0x7f7bc41565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:07.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 -- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 msgr2=0x7f7bc40caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 -- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bc4158230 msgr2=0x7f7bc41565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bc4158230 0x7f7bc41565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 -- 172.21.15.60:0/3222710701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc40cb360 con 0x7f7bc406c330 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bbbfff640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc3fff640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bc4158230 0x7f7bc41565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc37fe640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc41587b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400c400 tx=0x7f7bb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb40052d0 con 0x7f7bc406c330 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb4005490 con 0x7f7bc406c330 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4007260 con 0x7f7bc406c330 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.866+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc40cb640 con 0x7f7bc406c330 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.867+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc40cafa0 con 0x7f7bc406c330 2024-04-25T08:34:07.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.868+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc406de50 con 0x7f7bc406c330 2024-04-25T08:34:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.868+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bb4007400 con 0x7f7bc406c330 2024-04-25T08:34:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.868+0000 7f7bc0ff9640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ba406b500 0x7f7ba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.868+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7bb40897e0 con 0x7f7bc406c330 2024-04-25T08:34:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.869+0000 7f7bbbfff640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ba406b500 0x7f7ba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:07.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.869+0000 7f7bbbfff640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ba406b500 0x7f7ba406d9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bac033000 tx=0x7f7bac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:07.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:07.873+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bb4058560 con 0x7f7bc406c330 2024-04-25T08:34:08.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.093+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7bc40d0640 con 0x7f7bc406c330 2024-04-25T08:34:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.620+0000 7f7bc0ff9640 1 -- 172.21.15.60:0/3222710701 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v49) v1 ==== 167+0+0 (secure 0 0 0) 0x7f7bb4057e60 con 0x7f7bc406c330 2024-04-25T08:34:08.616 INFO:teuthology.orchestra.run.smithi060.stderr:enabled new snapshots 2024-04-25T08:34:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ba406b500 msgr2=0x7f7ba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ba406b500 0x7f7ba406d9c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bac033000 tx=0x7f7bac02e000 comp rx=0 tx=0).stop 2024-04-25T08:34:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 msgr2=0x7f7bc41587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:08.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc41587b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400c400 tx=0x7f7bb400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 shutdown_connections 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7ba406b500 0x7f7ba406d9c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7bc4158230 0x7f7bc41565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7bc406c7e0 0x7f7bc40caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 --2- 172.21.15.60:0/3222710701 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc406c330 0x7f7bc41587b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 >> 172.21.15.60:0/3222710701 conn(0x7f7bc4061600 msgr2=0x7f7bc415d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 shutdown_connections 2024-04-25T08:34:08.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:08.623+0000 7f7bca01f640 1 -- 172.21.15.60:0/3222710701 wait complete. 2024-04-25T08:34:08.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:08.630 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T08:34:08.651 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:34:08.652 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> stdin-killer -- python3 -c ' 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> import os 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> import uuid 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> n = 2000 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = False 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> if 3 > 0: 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> create_hard_links = True 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> dpath = os.path.dirname(path) 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> fnameprefix = os.path.basename(path) 2024-04-25T08:34:08.708 DEBUG:teuthology.orchestra.run.smithi060:> os.makedirs(dpath, exist_ok=True) 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> try: 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> for i in range(n): 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> f.write(f"{i}") 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> f.flush() 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(f.fileno()) 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> os.unlink(fpath) 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> if create_hard_links: 2024-04-25T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi060:> for j in range(3): 2024-04-25T08:34:08.710 DEBUG:teuthology.orchestra.run.smithi060:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T08:34:08.710 DEBUG:teuthology.orchestra.run.smithi060:> if False: 2024-04-25T08:34:08.710 DEBUG:teuthology.orchestra.run.smithi060:> os.fsync(dirfd) 2024-04-25T08:34:08.710 DEBUG:teuthology.orchestra.run.smithi060:> finally: 2024-04-25T08:34:08.710 DEBUG:teuthology.orchestra.run.smithi060:> os.close(dirfd) 2024-04-25T08:34:08.710 DEBUG:teuthology.orchestra.run.smithi060:> ' 2024-04-25T08:34:26.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:26.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:27.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:27.328 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:34:27.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:27.683 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T08:34:54.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.160 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:34:54.180 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:34:54.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T08:34:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.331+0000 7f2a072ee640 1 Processor -- start 2024-04-25T08:34:54.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.331+0000 7f2a072ee640 1 -- start start 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a072ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 0x7f2a000d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a072ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2a000d08e0 0x7f2a000b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a072ee640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a000d2420 con 0x7f2a0015dac0 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a072ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a000d3030 con 0x7f2a000d08e0 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a072ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000d2700 con 0x7f2a000d34c0 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a0508a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2a0015dac0 0x7f2a000d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53166/0 (socket says 172.21.15.60:53166) 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a0508a640 1 -- 172.21.15.60:0/3662269037 learned_addr learned my addr 172.21.15.60:0/3662269037 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a04889640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 0x7f2a000d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076280862 0 0) 0x7f2a000d2420 con 0x7f2a0015dac0 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f2a0588b640 1 --2- 172.21.15.60:0/3662269037 >> v2:172.21.15.100:3300/0 conn(0x7f2a000d08e0 0x7f2a000b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29ec003520 con 0x7f2a0015dac0 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4097846458 0 0) 0x7f29ec003520 con 0x7f2a0015dac0 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 msgr2=0x7f2a000d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 0x7f2a000d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 >> v2:172.21.15.100:3300/0 conn(0x7f2a000d08e0 msgr2=0x7f2a000b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 --2- 172.21.15.60:0/3662269037 >> v2:172.21.15.100:3300/0 conn(0x7f2a000d08e0 0x7f2a000b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.332+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a000d29e0 con 0x7f2a0015dac0 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f29fc0019f0 con 0x7f2a0015dac0 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 >> v1:172.21.15.60:6789/0 conn(0x7f2a0015dac0 legacy=0x7f2a000d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29ec004800 0x7f29ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 0x7f29ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000d3030 con 0x7f29ec004800 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000d2700 con 0x7f29ec0039b0 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec00b9c0 con 0x7f2a000d34c0 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a04889640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29ec004800 0x7f29ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a0588b640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 0x7f29ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a04889640 1 -- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 msgr2=0x7f29ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a04889640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a000d34c0 0x7f29ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a04889640 1 -- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec0039b0 msgr2=0x7f29ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a04889640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec0039b0 0x7f29ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.333+0000 7f2a04889640 1 -- 172.21.15.60:0/3662269037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec003520 con 0x7f29ec004800 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a04889640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29ec004800 0x7f29ec004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009a80 tx=0x7f29ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a5136d32a90a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0036070 con 0x7f29ec004800 2024-04-25T08:34:54.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f0005990 con 0x7f29ec004800 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f29f7fff640 1 -- 172.21.15.60:0/3662269037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0009680 con 0x7f29ec004800 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a072ee640 1 -- 172.21.15.60:0/3662269037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29ec004800 msgr2=0x7f29ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a072ee640 1 --2- 172.21.15.60:0/3662269037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f29ec004800 0x7f29ec004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009a80 tx=0x7f29ec005830 comp rx=0 tx=0).stop 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a0588b640 1 -- 172.21.15.60:0/3662269037 reap_dead start 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a072ee640 1 -- 172.21.15.60:0/3662269037 shutdown_connections 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a072ee640 1 -- 172.21.15.60:0/3662269037 >> 172.21.15.60:0/3662269037 conn(0x7f2a00061c30 msgr2=0x7f2a0006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a072ee640 1 -- 172.21.15.60:0/3662269037 shutdown_connections 2024-04-25T08:34:54.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.334+0000 7f2a072ee640 1 -- 172.21.15.60:0/3662269037 wait complete. 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 Processor -- start 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 -- start start 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec004800 0x7f2a000c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 0x7f2a001607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a00160d20 0x7f2a000c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0015dea0 con 0x7f2a0015f220 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000d2100 con 0x7f29ec004800 2024-04-25T08:34:54.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a072ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000b97d0 con 0x7f2a00160d20 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a04889640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 0x7f2a001607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a04889640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 0x7f2a001607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34262/0 (socket says 172.21.15.60:34262) 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.336+0000 7f2a04889640 1 -- 172.21.15.60:0/1097593463 learned_addr learned my addr 172.21.15.60:0/1097593463 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a0508a640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec004800 0x7f2a000c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a0588b640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a00160d20 0x7f2a000c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a04889640 1 -- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a00160d20 msgr2=0x7f2a000c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a04889640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a00160d20 0x7f2a000c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a04889640 1 -- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec004800 msgr2=0x7f2a000c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a04889640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec004800 0x7f2a000c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a04889640 1 -- 172.21.15.60:0/1097593463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0015e1c0 con 0x7f2a0015f220 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a04889640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 0x7f2a001607e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009ba0 tx=0x7f29f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f29f5ffb640 1 -- 172.21.15.60:0/1097593463 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0040020 con 0x7f2a0015f220 2024-04-25T08:34:54.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f29f5ffb640 1 -- 172.21.15.60:0/1097593463 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f0037040 con 0x7f2a0015f220 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0015e460 con 0x7f2a0015f220 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f29f5ffb640 1 -- 172.21.15.60:0/1097593463 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0036030 con 0x7f2a0015f220 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.337+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0015ea20 con 0x7f2a0015f220 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.339+0000 7f29f5ffb640 1 -- 172.21.15.60:0/1097593463 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29f0043020 con 0x7f2a0015f220 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.339+0000 7f29f5ffb640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29e406b4d0 0x7f29e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.340+0000 7f29f5ffb640 1 -- 172.21.15.60:0/1097593463 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29f00aa600 con 0x7f2a0015f220 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.340+0000 7f2a0508a640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29e406b4d0 0x7f29e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.340+0000 7f2a0508a640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29e406b4d0 0x7f29e406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f29fc004c20 tx=0x7f29fc006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.340+0000 7f2a09fed740 1 Processor -- start 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.340+0000 7f2a09fed740 1 -- start start 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e34de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e388370 0x56172e34c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e376be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a09fed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172e2aeb90 con 0x56172e376760 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172e257e40 con 0x56172e388370 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56172e2aac30 con 0x56172e34bca0 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a0588b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e376be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a0508a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e34de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a04889640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e388370 0x56172e34c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a0508a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e34de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:42996/0 (socket says 172.21.15.60:42996) 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a0508a640 1 -- 172.21.15.60:0/3775650124 learned_addr learned my addr 172.21.15.60:0/3775650124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.341+0000 7f2a0588b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e376be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34266/0 (socket says 172.21.15.60:34266) 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a04889640 1 -- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 msgr2=0x56172e34de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a04889640 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e34de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a04889640 1 -- 172.21.15.60:0/3775650124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 msgr2=0x56172e376be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a04889640 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e376be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a04889640 1 -- 172.21.15.60:0/3775650124 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56172e08d7d0 con 0x56172e388370 2024-04-25T08:34:54.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a0588b640 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e376be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a04889640 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e388370 0x56172e34c080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f29f00a8970 tx=0x7f29f0009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88f8707cb99f169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f29eaffd640 1 -- 172.21.15.60:0/3775650124 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0078bf0 con 0x56172e388370 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f29eaffd640 1 -- 172.21.15.60:0/3775650124 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f007cd40 con 0x56172e388370 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f29eaffd640 1 -- 172.21.15.60:0/3775650124 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0081300 con 0x56172e388370 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 -- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e388370 msgr2=0x56172e34c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e388370 0x56172e34c080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f29f00a8970 tx=0x7f29f0009740 comp rx=0 tx=0).stop 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 -- 172.21.15.60:0/3775650124 shutdown_connections 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e376be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e388370 0x56172e34c080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3775650124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e34de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.342+0000 7f2a09fed740 1 -- 172.21.15.60:0/3775650124 >> 172.21.15.60:0/3775650124 conn(0x56172e313870 msgr2=0x56172e391760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.343+0000 7f2a09fed740 1 -- 172.21.15.60:0/3775650124 shutdown_connections 2024-04-25T08:34:54.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.343+0000 7f2a09fed740 1 -- 172.21.15.60:0/3775650124 wait complete. 2024-04-25T08:34:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.343+0000 7f2a09fed740 1 -- start start 2024-04-25T08:34:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.343+0000 7f2a09fed740 1 Processor -- start 2024-04-25T08:34:54.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e34bca0 0x56172e377620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e392c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e3887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172e2aac30 con 0x56172e376760 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172e26b3e0 con 0x56172e34bca0 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56172e257e40 con 0x56172e388370 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a09fed740 10 client.0 _ll_register_callbacks cb 0x56172e26adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e392c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e392c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34280/0 (socket says 172.21.15.60:34280) 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 -- 172.21.15.60:0/4047327936 learned_addr learned my addr 172.21.15.60:0/4047327936 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a0508a640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e34bca0 0x56172e377620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a0588b640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e3887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 -- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 msgr2=0x56172e3887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e3887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 -- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e34bca0 msgr2=0x56172e377620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.344+0000 7f2a04889640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e34bca0 0x56172e377620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a04889640 1 -- 172.21.15.60:0/4047327936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56172df830f0 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a04889640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e392c40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f29f00a89f0 tx=0x7f29f0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0b6aa1c6e10c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f29dffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f29dffff640 1 -- 172.21.15.60:0/4047327936 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f0081300 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f29dffff640 1 -- 172.21.15.60:0/4047327936 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29f0080e70 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29f007cd40 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f29dffff640 1 -- 172.21.15.60:0/4047327936 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f0078bf0 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 10 client.7377 fetch_fsmap learned FSMap version 49 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x56172e0edf60 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f29dffff640 1 -- 172.21.15.60:0/4047327936 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f007cd40 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f29dffff640 1 -- 172.21.15.60:0/4047327936 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29f008cbb0 con 0x56172e376760 2024-04-25T08:34:54.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 10 client.7377 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 10 client.7377 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 20 client.7377 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e360260 0x56172e35ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 4 client.7377 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56172e2bc0d0 con 0x56172e360260 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.345+0000 7f2a0508a640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e360260 0x56172e35ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.346+0000 7f2a0508a640 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e360260 0x56172e35ef40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c222f6f0027b8900 server_cookie=6c9466ded38b9115 in_seq=0 out_seq=0 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.346+0000 7f29dffff640 10 client.7377 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.346+0000 7f29dffff640 1 -- 172.21.15.60:0/4047327936 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56172e2bc0d0 con 0x56172e360260 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.346+0000 7f29dffff640 10 client.7377 handle_command_reply: tid=0 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.346+0000 7f2a09fed740 1 client.7377 shutdown 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 msgr2=0x56172e392c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e392c40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f29f00a89f0 tx=0x7f29f0046000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 shutdown_connections 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e360260 0x56172e35ef40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e3887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e376760 0x56172e392c40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 --2- 172.21.15.60:0/4047327936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e34bca0 0x56172e377620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f29dffff640 0 client.7377 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 >> 172.21.15.60:0/4047327936 conn(0x56172e313870 msgr2=0x56172e394cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 shutdown_connections 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 1 -- 172.21.15.60:0/4047327936 wait complete. 2024-04-25T08:34:54.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.347+0000 7f2a09fed740 20 client.7377 trim_cache size 0 max 16384 2024-04-25T08:34:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.361+0000 7f2a09fed740 1 Processor -- start 2024-04-25T08:34:54.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.361+0000 7f2a09fed740 1 -- start start 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 0x56172e392c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a09fed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00200d60 con 0x56172e34bca0 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a001fff00 con 0x56172e376760 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a001f76d0 con 0x56172e388370 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 0x56172e392c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 0x56172e392c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34284/0 (socket says 172.21.15.60:34284) 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 -- 172.21.15.60:0/3509946435 learned_addr learned my addr 172.21.15.60:0/3509946435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 -- 172.21.15.60:0/3509946435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 msgr2=0x56172e37a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 -- 172.21.15.60:0/3509946435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 msgr2=0x56172e379d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.362+0000 7f2a0508a640 1 -- 172.21.15.60:0/3509946435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56172df5c1b0 con 0x56172e34bca0 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 0x56172e392c40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007920 tx=0x7f29fc00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dd9ce0d1583915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f29ea7fc640 1 -- 172.21.15.60:0/3509946435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc02e070 con 0x56172e34bca0 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f29ea7fc640 1 -- 172.21.15.60:0/3509946435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc029b00 con 0x56172e34bca0 2024-04-25T08:34:54.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f29ea7fc640 1 -- 172.21.15.60:0/3509946435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc029380 con 0x56172e34bca0 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 -- 172.21.15.60:0/3509946435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 msgr2=0x56172e392c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 0x56172e392c40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007920 tx=0x7f29fc00c000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 -- 172.21.15.60:0/3509946435 shutdown_connections 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3509946435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e34bca0 0x56172e392c40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 -- 172.21.15.60:0/3509946435 >> 172.21.15.60:0/3509946435 conn(0x56172e313870 msgr2=0x56172e341f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 -- 172.21.15.60:0/3509946435 shutdown_connections 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.363+0000 7f2a09fed740 1 -- 172.21.15.60:0/3509946435 wait complete. 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.364+0000 7f2a09fed740 1 -- start start 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.364+0000 7f2a09fed740 1 Processor -- start 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e3887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56172e2aeb90 con 0x56172e388370 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a001ff930 con 0x56172e376760 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56172e2b9a20 con 0x56172e34bca0 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a09fed740 10 client.0 _ll_register_callbacks cb 0x7f29f80095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34290/0 (socket says 172.21.15.60:34290) 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 -- 172.21.15.60:0/3820808002 learned_addr learned my addr 172.21.15.60:0/3820808002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a04889640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e3887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 -- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 msgr2=0x56172e3887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e3887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 -- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 msgr2=0x56172e379d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 -- 172.21.15.60:0/3820808002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56172e377780 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f2a0588b640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 0x56172e37a240 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f29f80179e0 tx=0x7f29f8017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12600111704c980f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.365+0000 7f29dffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f29dffff640 1 -- 172.21.15.60:0/3820808002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8015ac0 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f29dffff640 1 -- 172.21.15.60:0/3820808002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29f800f040 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29fc029b00 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f29dffff640 1 -- 172.21.15.60:0/3820808002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f802a050 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 10 client.7383 fetch_fsmap learned FSMap version 49 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x56172e35fa00 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f29dffff640 1 -- 172.21.15.60:0/3820808002 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc029b00 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f29dffff640 1 -- 172.21.15.60:0/3820808002 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29f802d020 con 0x56172e388370 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 10 client.7383 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 10 client.7383 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:34:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 20 client.7383 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e36cef0 0x56172e36d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 4 client.7383 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x56172e2b6420 con 0x56172e36cef0 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.366+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e36cef0 0x56172e36d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a0508a640 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e36cef0 0x56172e36d300 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ecf1189fdefcd0 server_cookie=9e9f424958f8bdf in_seq=0 out_seq=0 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f29dffff640 10 client.7383 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f29dffff640 1 -- 172.21.15.60:0/3820808002 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17923 (crc 0 0 0) 0x56172e2b6420 con 0x56172e36cef0 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f29dffff640 10 client.7383 handle_command_reply: tid=0 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 client.7383 shutdown 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 msgr2=0x56172e37a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 0x56172e37a240 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f29f80179e0 tx=0x7f29f8017ea0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 shutdown_connections 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x56172e36cef0 0x56172e36d300 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56172e388370 0x56172e37a240 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56172e376760 0x56172e379d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f29dffff640 0 client.7383 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 --2- 172.21.15.60:0/3820808002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56172e34bca0 0x56172e3887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 >> 172.21.15.60:0/3820808002 conn(0x56172e313870 msgr2=0x56172e392c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 shutdown_connections 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 1 -- 172.21.15.60:0/3820808002 wait complete. 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.367+0000 7f2a09fed740 20 client.7383 trim_cache size 0 max 16384 2024-04-25T08:34:54.369 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1282, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 1282, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 510462, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 14256567, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 9, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.148369259, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.074700590, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.025080431, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.038500029, 2024-04-25T08:34:54.370 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1281, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.248981813, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000194365 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000115, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000057 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 27803, 2024-04-25T08:34:54.371 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 49934, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 8878087, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 54796166, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 1.683726115, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.962178570, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.809628727, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.096969067, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 49934, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.195919366, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000023950 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.372 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 46525, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007481353, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000160 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 117120, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 18928 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 2307, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 2321, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 572323, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 43540523, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:34:54.373 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.259229389, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.143349072, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.040064394, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.064780638, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2320, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.252501274, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000108836 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 108368 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1319, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 14.199264936, 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.010765174 2024-04-25T08:34:54.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1913, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.621562818, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000324915 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 25637, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 25637, 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2024-04-25T08:34:54.376 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 25637, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 310.760888177, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.012121577 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 12, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 19885, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 19885, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 0, 2024-04-25T08:34:54.377 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 19885, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 0, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 132, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 132, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 2, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 47616, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 47117, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2024-04-25T08:34:54.378 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 147, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2024-04-25T08:34:54.379 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 1210, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 4038, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:34:54.380 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 3261, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 2242, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 51, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 14014 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 2000, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 166, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 4141, 2024-04-25T08:34:54.381 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 1707, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 2242, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2024-04-25T08:34:54.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "evlrg": 0, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 24476, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 24476, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 34, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2024-04-25T08:34:54.383 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 34, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "segmjr": 3, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 94152569, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 0, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2576, 2024-04-25T08:34:54.384 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 85.911531701, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.033350749 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 0 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 2014, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 2014, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 0, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 13, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 13, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 0, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 19885, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 19886, 2024-04-25T08:34:54.385 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 1, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 132, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 7354, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 7222, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 112520, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 215396 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 29050, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 25637, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 3, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2024-04-25T08:34:54.386 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2000, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.476472163, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000738236 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 29, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.004831644, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000166608 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.387 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 6000, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 7.540886480, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001256814 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 7353, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.329535245, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000860809 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.388 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2024-04-25T08:34:54.389 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000138271, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000138271 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002625617, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002625617 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.335501912, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.335501912 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.209484873, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.026185609 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2242, 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 279.191802296, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.124528011 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.393 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003188177, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003188177 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.394 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000274225, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000274225 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2024-04-25T08:34:54.395 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2024-04-25T08:34:54.396 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8000, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 15.666147274, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001958268 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.397 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.398 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2024-04-25T08:34:54.398 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2024-04-25T08:34:54.398 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 1, 2024-04-25T08:34:54.399 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 0, 2024-04-25T08:34:54.399 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2024-04-25T08:34:54.399 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2024-04-25T08:34:54.399 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 18569, 2024-04-25T08:34:54.399 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 18569, 2024-04-25T08:34:54.399 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 45, 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mdthresh_evicted": 0 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:34:54.400 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:34:54.401 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:34:54.402 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-04-25T08:34:54.402 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:34:54.402 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-04-25T08:34:54.402 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:34:54.402 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:34:54.403 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:34:54.403 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:34:54.403 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:34:54.403 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:34:54.403 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:34:54.403 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:34:54.404 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 17178667, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 6330, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 352, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 4, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 5424, 2024-04-25T08:34:54.405 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 76, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 21200616, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 881990, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.406 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 4513, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 90425935, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 37, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 4513, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4513, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 17.142546805, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003798481 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2024-04-25T08:34:54.407 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4513, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 4603 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 4513, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 49, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 4464, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 0, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 11, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 49, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 2625, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 30, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2024-04-25T08:34:54.408 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 1707, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 33, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 22, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2024-04-25T08:34:54.409 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2024-04-25T08:34:54.410 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2024-04-25T08:34:54.411 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2024-04-25T08:34:54.411 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2024-04-25T08:34:54.411 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2024-04-25T08:34:54.411 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2024-04-25T08:34:54.411 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2024-04-25T08:34:54.413 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 83, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 1, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 8, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 8, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 101, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 0, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 25 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 1, 2024-04-25T08:34:54.414 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 32, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed_ops": 1707, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 1707, 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2024-04-25T08:34:54.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 31386, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 7559928, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 31386, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 31386, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 7559928, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.416 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 4513, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 90611998, 2024-04-25T08:34:54.417 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4513, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 90611998, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:34:54.418 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 4513, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 4513, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 4513, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 4513, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:34:54.419 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 1707, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 186063, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 1707, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 49, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 186063, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x55ad9b57e6e0": { 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 84, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 24476, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 89958265, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 24476, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 2576, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 89958181, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:34:54.421 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:34:54.422 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:34:54.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.422 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:34:54.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29e406b4d0 msgr2=0x7f29e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29e406b4d0 0x7f29e406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f29fc004c20 tx=0x7f29fc006040 comp rx=0 tx=0).stop 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 msgr2=0x7f2a001607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 0x7f2a001607e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f29f0009ba0 tx=0x7f29f0004730 comp rx=0 tx=0).stop 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 shutdown_connections 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a00160d20 0x7f2a000c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0015f220 0x7f2a001607e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29e406b4d0 0x7f29e406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 --2- 172.21.15.60:0/1097593463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29ec004800 0x7f2a000c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 >> 172.21.15.60:0/1097593463 conn(0x7f2a00061c30 msgr2=0x7f2a000b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 shutdown_connections 2024-04-25T08:34:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.368+0000 7f2a072ee640 1 -- 172.21.15.60:0/1097593463 wait complete. 2024-04-25T08:34:54.425 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T08:34:54.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T08:34:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 Processor -- start 2024-04-25T08:34:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 -- start start 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c15dac0 0x7fb39c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb39c0d0ab0 0x7fb39c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c0d2420 con 0x7fb39c0d1f30 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c0d3030 con 0x7fb39c0d0ab0 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.535+0000 7fb3a2bc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0d2700 con 0x7fb39c15dac0 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb39bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb39c0d1f30 0x7fb39c0d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:53170/0 (socket says 172.21.15.60:53170) 2024-04-25T08:34:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb39bfff640 1 -- 172.21.15.60:0/2492797150 learned_addr learned my addr 172.21.15.60:0/2492797150 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a0964640 1 --2- 172.21.15.60:0/2492797150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c15dac0 0x7fb39c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a1165640 1 --2- 172.21.15.60:0/2492797150 >> v2:172.21.15.100:3300/0 conn(0x7fb39c0d0ab0 0x7fb39c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb39b7fe640 1 -- 172.21.15.60:0/2492797150 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461130195 0 0) 0x7fb39c0d2420 con 0x7fb39c0d1f30 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb39b7fe640 1 -- 172.21.15.60:0/2492797150 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb388003520 con 0x7fb39c0d1f30 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb39b7fe640 1 -- 172.21.15.60:0/2492797150 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1844806864 0 0) 0x7fb388003520 con 0x7fb39c0d1f30 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a0964640 1 -- 172.21.15.60:0/2492797150 >> v2:172.21.15.100:3300/0 conn(0x7fb39c0d0ab0 msgr2=0x7fb39c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a0964640 1 --2- 172.21.15.60:0/2492797150 >> v2:172.21.15.100:3300/0 conn(0x7fb39c0d0ab0 0x7fb39c0b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a0964640 1 -- 172.21.15.60:0/2492797150 >> v1:172.21.15.60:6789/0 conn(0x7fb39c0d1f30 legacy=0x7fb39c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a0964640 1 -- 172.21.15.60:0/2492797150 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0d29e0 con 0x7fb39c15dac0 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.536+0000 7fb3a1165640 1 --2- 172.21.15.60:0/2492797150 >> v2:172.21.15.100:3300/0 conn(0x7fb39c0d0ab0 0x7fb39c0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.537+0000 7fb3a0964640 1 --2- 172.21.15.60:0/2492797150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c15dac0 0x7fb39c0d19f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb394009d90 tx=0x7fb39402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cae503fdd764e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.537+0000 7fb39b7fe640 1 -- 172.21.15.60:0/2492797150 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840017f0 con 0x7fb39c15dac0 2024-04-25T08:34:54.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.537+0000 7fb39b7fe640 1 -- 172.21.15.60:0/2492797150 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394007540 con 0x7fb39c15dac0 2024-04-25T08:34:54.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.537+0000 7fb39b7fe640 1 -- 172.21.15.60:0/2492797150 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940078f0 con 0x7fb39c15dac0 2024-04-25T08:34:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/2492797150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c15dac0 msgr2=0x7fb39c0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/2492797150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c15dac0 0x7fb39c0d19f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb394009d90 tx=0x7fb39402a680 comp rx=0 tx=0).stop 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/2492797150 shutdown_connections 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/2492797150 >> v2:172.21.15.100:3300/0 conn(0x7fb39c0d0ab0 0x7fb39c0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/2492797150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c15dac0 0x7fb39c0d19f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/2492797150 >> 172.21.15.60:0/2492797150 conn(0x7fb39c061c30 msgr2=0x7fb39c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/2492797150 shutdown_connections 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.538+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/2492797150 wait complete. 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.539+0000 7fb3a2bc8640 1 Processor -- start 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.539+0000 7fb3a2bc8640 1 -- start start 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a2bc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c0d0ab0 0x7fb39c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a2bc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 0x7fb39c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a2bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39c15dac0 0x7fb39c145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a2bc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0b97d0 con 0x7fb39c0d1f30 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a2bc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c15dea0 con 0x7fb39c15dac0 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a2bc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0b9950 con 0x7fb39c0d0ab0 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 0x7fb39c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 0x7fb39c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34298/0 (socket says 172.21.15.60:34298) 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 -- 172.21.15.60:0/3964164349 learned_addr learned my addr 172.21.15.60:0/3964164349 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c0d0ab0 0x7fb39c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a1165640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39c15dac0 0x7fb39c145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 -- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c0d0ab0 msgr2=0x7fb39c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c0d0ab0 0x7fb39c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 -- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39c15dac0 msgr2=0x7fb39c145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39c15dac0 0x7fb39c145e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 -- 172.21.15.60:0/3964164349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0cc470 con 0x7fb39c0d1f30 2024-04-25T08:34:54.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c0d0ab0 0x7fb39c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.540+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 0x7fb39c0b3640 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c780 tx=0x7fb38400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.541+0000 7fb3997fa640 1 -- 172.21.15.60:0/3964164349 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384017070 con 0x7fb39c0d1f30 2024-04-25T08:34:54.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.541+0000 7fb3997fa640 1 -- 172.21.15.60:0/3964164349 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384007270 con 0x7fb39c0d1f30 2024-04-25T08:34:54.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.541+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39c0cc750 con 0x7fb39c0d1f30 2024-04-25T08:34:54.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.541+0000 7fb3997fa640 1 -- 172.21.15.60:0/3964164349 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840057a0 con 0x7fb39c0d1f30 2024-04-25T08:34:54.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.541+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c0ccd10 con 0x7fb39c0d1f30 2024-04-25T08:34:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.542+0000 7fb3997fa640 1 -- 172.21.15.60:0/3964164349 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb38401a070 con 0x7fb39c0d1f30 2024-04-25T08:34:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.543+0000 7fb3997fa640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb37c06b500 0x7fb37c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.543+0000 7fb3997fa640 1 -- 172.21.15.60:0/3964164349 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb384089770 con 0x7fb39c0d1f30 2024-04-25T08:34:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.543+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb37c06b500 0x7fb37c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.543+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb37c06b500 0x7fb37c06d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb394034000 tx=0x7fb39402f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.543+0000 7fb3a59a6740 1 Processor -- start 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.543+0000 7fb3a59a6740 1 -- start start 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1aa0 0x560e56391200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e56391740 0x560e563b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e562fa0a0 con 0x560e563c1080 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e562a2d40 con 0x560e563c1aa0 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e562f6140 con 0x560e56391740 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563c1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34312/0 (socket says 172.21.15.60:34312) 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a0964640 1 -- 172.21.15.60:0/3543414943 learned_addr learned my addr 172.21.15.60:0/3543414943 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1aa0 0x560e56391200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a1165640 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e56391740 0x560e563b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.544+0000 7fb3a0964640 1 -- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e56391740 msgr2=0x560e563b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e56391740 0x560e563b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a0964640 1 -- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1aa0 msgr2=0x560e56391200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1aa0 0x560e56391200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a0964640 1 -- 172.21.15.60:0/3543414943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e560daa20 con 0x560e563c1080 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1aa0 0x560e56391200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563c1460 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb394009d90 tx=0x7fb39402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efe4af3f5ddde28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3927fc640 1 -- 172.21.15.60:0/3543414943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940046f0 con 0x560e563c1080 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3927fc640 1 -- 172.21.15.60:0/3543414943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39404b070 con 0x560e563c1080 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3927fc640 1 -- 172.21.15.60:0/3543414943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394007780 con 0x560e563c1080 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3543414943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 msgr2=0x560e563c1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563c1460 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb394009d90 tx=0x7fb39402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3543414943 shutdown_connections 2024-04-25T08:34:54.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e56391740 0x560e563b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1aa0 0x560e56391200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3543414943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563c1460 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.545+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3543414943 >> 172.21.15.60:0/3543414943 conn(0x560e56393470 msgr2=0x560e563dbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.546+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3543414943 shutdown_connections 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.546+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3543414943 wait complete. 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.546+0000 7fb3a59a6740 1 -- start start 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.546+0000 7fb3a59a6740 1 Processor -- start 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e563afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e563bd7d0 con 0x560e563c1080 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e562983d0 con 0x560e563b1e20 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e563bda70 con 0x560e563c1aa0 2024-04-25T08:34:54.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a59a6740 10 client.0 _ll_register_callbacks cb 0x560e5600de00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34322/0 (socket says 172.21.15.60:34322) 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 -- 172.21.15.60:0/2357369661 learned_addr learned my addr 172.21.15.60:0/2357369661 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a1165640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e563afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb39bfff640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 -- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 msgr2=0x560e563b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 -- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 msgr2=0x560e563afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e563afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a0964640 1 -- 172.21.15.60:0/2357369661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e55fce310 con 0x560e563c1080 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.547+0000 7fb3a1165640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e563afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a0964640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563af840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb394009bd0 tx=0x7fb394004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89f3986d87a872f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb37b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb37b7fe640 1 -- 172.21.15.60:0/2357369661 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394003fd0 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb37b7fe640 1 -- 172.21.15.60:0/2357369661 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb394007b00 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb39404b070 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb37b7fe640 1 -- 172.21.15.60:0/2357369661 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb394063050 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 10 client.7395 fetch_fsmap learned FSMap version 49 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x560e5613b1b0 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb37b7fe640 1 -- 172.21.15.60:0/2357369661 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39404b070 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb37b7fe640 1 -- 172.21.15.60:0/2357369661 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb394066020 con 0x560e563c1080 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 10 client.7395 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 10 client.7395 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 20 client.7395 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563b4bd0 0x560e563b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 4 client.7395 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:54.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.548+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e563075e0 con 0x560e563b4bd0 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.549+0000 7fb39bfff640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563b4bd0 0x560e563b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.549+0000 7fb39bfff640 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563b4bd0 0x560e563b4fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37aa09f61a77605 server_cookie=9ab1913f14a9a36e in_seq=0 out_seq=0 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.549+0000 7fb37b7fe640 10 client.7395 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.549+0000 7fb37b7fe640 1 -- 172.21.15.60:0/2357369661 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e563075e0 con 0x560e563b4bd0 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.549+0000 7fb37b7fe640 10 client.7395 handle_command_reply: tid=0 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.549+0000 7fb3a59a6740 1 client.7395 shutdown 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 msgr2=0x560e563af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563af840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb394009bd0 tx=0x7fb394004ab0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 shutdown_connections 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563b4bd0 0x560e563b4fb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e563afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/2357369661 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1080 0x560e563af840 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb37b7fe640 0 client.7395 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 >> 172.21.15.60:0/2357369661 conn(0x560e56391740 msgr2=0x560e563b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 shutdown_connections 2024-04-25T08:34:54.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 1 -- 172.21.15.60:0/2357369661 wait complete. 2024-04-25T08:34:54.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.550+0000 7fb3a59a6740 20 client.7395 trim_cache size 0 max 16384 2024-04-25T08:34:54.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.565+0000 7fb3a59a6740 1 Processor -- start 2024-04-25T08:34:54.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.565+0000 7fb3a59a6740 1 -- start start 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 0x560e5643d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1080 0x560e56390480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c200210 con 0x560e563b1e20 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1ff3b0 con 0x560e563c1080 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1fe880 con 0x560e563c1aa0 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 0x560e5643d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 0x560e5643d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34336/0 (socket says 172.21.15.60:34336) 2024-04-25T08:34:54.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 -- 172.21.15.60:0/3600574239 learned_addr learned my addr 172.21.15.60:0/3600574239 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a1165640 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1080 0x560e56390480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 -- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 msgr2=0x560e563909c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563909c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 -- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1080 msgr2=0x560e56390480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1080 0x560e56390480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 -- 172.21.15.60:0/3600574239 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e55fa7490 con 0x560e563b1e20 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.566+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 0x560e5643d790 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb394009d90 tx=0x7fb394056000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c121bfbf18e380ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb391ffb640 1 -- 172.21.15.60:0/3600574239 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39406c070 con 0x560e563b1e20 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb391ffb640 1 -- 172.21.15.60:0/3600574239 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39404b070 con 0x560e563b1e20 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb391ffb640 1 -- 172.21.15.60:0/3600574239 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394054040 con 0x560e563b1e20 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3600574239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 msgr2=0x560e5643d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 0x560e5643d790 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb394009d90 tx=0x7fb394056000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3600574239 shutdown_connections 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1aa0 0x560e563909c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563c1080 0x560e56390480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3600574239 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563b1e20 0x560e5643d790 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3600574239 >> 172.21.15.60:0/3600574239 conn(0x560e56391740 msgr2=0x560e5638c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3600574239 shutdown_connections 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.567+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3600574239 wait complete. 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.568+0000 7fb3a59a6740 1 -- start start 2024-04-25T08:34:54.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.568+0000 7fb3a59a6740 1 Processor -- start 2024-04-25T08:34:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e5642dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1080 0x560e5643bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1aa0 0x560e5643e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e563c5140 con 0x560e563c1aa0 2024-04-25T08:34:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1fedb0 con 0x560e563b1e20 2024-04-25T08:34:54.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e56395530 con 0x560e563c1080 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a59a6740 10 client.0 _ll_register_callbacks cb 0x7fb38c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e5642dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a1165640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1aa0 0x560e5643e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb39bfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1080 0x560e5643bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e5642dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37116/0 (socket says 172.21.15.60:37116) 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 -- 172.21.15.60:0/3559441950 learned_addr learned my addr 172.21.15.60:0/3559441950 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 -- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1080 msgr2=0x560e5643bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1080 0x560e5643bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 -- 172.21.15.60:0/3559441950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1aa0 msgr2=0x560e5643e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1aa0 0x560e5643e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 -- 172.21.15.60:0/3559441950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e56397ca0 con 0x560e563b1e20 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a1165640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1aa0 0x560e5643e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1080 0x560e5643bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb3a0964640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e5642dc80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb394009860 tx=0x7fb394030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88c72bdb7532f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb37b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.569+0000 7fb37b7fe640 1 -- 172.21.15.60:0/3559441950 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39406c070 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb37b7fe640 1 -- 172.21.15.60:0/3559441950 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb39403c420 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb39404b070 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb37b7fe640 1 -- 172.21.15.60:0/3559441950 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb394077050 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 10 client.6745 fetch_fsmap learned FSMap version 49 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x560e5643db10 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb37b7fe640 1 -- 172.21.15.60:0/3559441950 <== mon.1 v2:172.21.15.100:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39404b070 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb37b7fe640 1 -- 172.21.15.60:0/3559441950 <== mon.1 v2:172.21.15.100:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb39407a020 con 0x560e563b1e20 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 10 client.6745 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 10 client.6745 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 20 client.6745 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563974b0 0x560e56395f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 4 client.6745 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x560e563952b0 con 0x560e563974b0 2024-04-25T08:34:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563974b0 0x560e56395f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb37b7fe640 10 client.6745 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.570+0000 7fb39bfff640 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563974b0 0x560e56395f70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c2bbd8a66d80cf server_cookie=29897712d9ab90c0 in_seq=0 out_seq=0 2024-04-25T08:34:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb37b7fe640 1 -- 172.21.15.60:0/3559441950 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560e563952b0 con 0x560e563974b0 2024-04-25T08:34:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb37b7fe640 10 client.6745 handle_command_reply: tid=0 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 client.6745 shutdown 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 msgr2=0x560e5642dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e5642dc80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb394009860 tx=0x7fb394030000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 shutdown_connections 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x560e563974b0 0x560e56395f70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560e563c1aa0 0x560e5643e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x560e563c1080 0x560e5643bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 --2- 172.21.15.60:0/3559441950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e563b1e20 0x560e5642dc80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb37b7fe640 0 client.6745 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 >> 172.21.15.60:0/3559441950 conn(0x560e56391740 msgr2=0x560e563bce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 shutdown_connections 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 1 -- 172.21.15.60:0/3559441950 wait complete. 2024-04-25T08:34:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.578+0000 7fb3a59a6740 20 client.6745 trim_cache size 0 max 16384 2024-04-25T08:34:54.575 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:34:54.575 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:34:54.575 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "40340834-4750-43ee-b445-c3e7a8182afd", 2024-04-25T08:34:54.575 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:34:54.575 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:34:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb37c06b500 msgr2=0x7fb37c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb37c06b500 0x7fb37c06d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb394034000 tx=0x7fb39402f000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 msgr2=0x7fb39c0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 0x7fb39c0b3640 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c780 tx=0x7fb38400cc40 comp rx=0 tx=0).stop 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 shutdown_connections 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb37c06b500 0x7fb37c06d9c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb39c15dac0 0x7fb39c145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb39c0d1f30 0x7fb39c0b3640 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 --2- 172.21.15.60:0/3964164349 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb39c0d0ab0 0x7fb39c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 >> 172.21.15.60:0/3964164349 conn(0x7fb39c061c30 msgr2=0x7fb39c0b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 shutdown_connections 2024-04-25T08:34:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.579+0000 7fb3a2bc8640 1 -- 172.21.15.60:0/3964164349 wait complete. 2024-04-25T08:34:54.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:34:54.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.718+0000 7f1e0e28f640 1 Processor -- start 2024-04-25T08:34:54.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.718+0000 7f1e0e28f640 1 -- start start 2024-04-25T08:34:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.719+0000 7f1e0e28f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e0815a9f0 0x7f1e080c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.719+0000 7f1e0e28f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e080cdbf0 0x7f1e080ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.719+0000 7f1e0e28f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e080ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.719+0000 7f1e0e28f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e080b0020 con 0x7f1e0815a9f0 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.719+0000 7f1e0e28f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e080cb5a0 con 0x7f1e080cdbf0 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.719+0000 7f1e0e28f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080c6940 con 0x7f1e080cd520 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e0815a9f0 0x7f1e080c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e077fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e080cdbf0 0x7f1e080ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e080ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e0815a9f0 0x7f1e080c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34354/0 (socket says 172.21.15.60:34354) 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 -- 172.21.15.60:0/1352773361 learned_addr learned my addr 172.21.15.60:0/1352773361 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 -- 172.21.15.60:0/1352773361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 msgr2=0x7f1e080ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1352773361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e080ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 -- 172.21.15.60:0/1352773361 >> v2:172.21.15.100:3300/0 conn(0x7f1e080cdbf0 msgr2=0x7f1e080ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1352773361 >> v2:172.21.15.100:3300/0 conn(0x7f1e080cdbf0 0x7f1e080ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.720+0000 7f1e07fff640 1 -- 172.21.15.60:0/1352773361 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0815add0 con 0x7f1e0815a9f0 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.721+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1352773361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e0815a9f0 0x7f1e080c6010 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f1e0815e440 tx=0x7f1df802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdddb38bd51f901 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.721+0000 7f1e06ffd640 1 -- 172.21.15.60:0/1352773361 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8037070 con 0x7f1e0815a9f0 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.721+0000 7f1e06ffd640 1 -- 172.21.15.60:0/1352773361 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df8032440 con 0x7f1e0815a9f0 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.721+0000 7f1e06ffd640 1 -- 172.21.15.60:0/1352773361 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8032810 con 0x7f1e0815a9f0 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1352773361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e0815a9f0 msgr2=0x7f1e080c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1352773361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e0815a9f0 0x7f1e080c6010 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f1e0815e440 tx=0x7f1df802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1352773361 shutdown_connections 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1352773361 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e080ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1352773361 >> v2:172.21.15.100:3300/0 conn(0x7f1e080cdbf0 0x7f1e080ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1352773361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e0815a9f0 0x7f1e080c6010 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1352773361 >> 172.21.15.60:0/1352773361 conn(0x7f1e080b2390 msgr2=0x7f1e080b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1352773361 shutdown_connections 2024-04-25T08:34:54.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.722+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1352773361 wait complete. 2024-04-25T08:34:54.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.723+0000 7f1e0e28f640 1 Processor -- start 2024-04-25T08:34:54.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.723+0000 7f1e0e28f640 1 -- start start 2024-04-25T08:34:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.724+0000 7f1e0e28f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e0814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.724+0000 7f1e0e28f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e080cdbf0 0x7f1e080c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.724+0000 7f1e0e28f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 0x7f1e080ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.724+0000 7f1e0e28f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080cb230 con 0x7f1e080cdbf0 2024-04-25T08:34:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.724+0000 7f1e0e28f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080caf50 con 0x7f1e0815a9f0 2024-04-25T08:34:54.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.724+0000 7f1e0e28f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080ce070 con 0x7f1e080cd520 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e080cdbf0 0x7f1e080c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 0x7f1e080ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 0x7f1e080ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:37138/0 (socket says 172.21.15.60:37138) 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 -- 172.21.15.60:0/1062607343 learned_addr learned my addr 172.21.15.60:0/1062607343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e0814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 -- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 msgr2=0x7f1e0814c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e0814c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 -- 172.21.15.60:0/1062607343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e080cdbf0 msgr2=0x7f1e080c9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e080cdbf0 0x7f1e080c9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 -- 172.21.15.60:0/1062607343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080b0d60 con 0x7f1e0815a9f0 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e0814c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.725+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 0x7f1e080ca340 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000eed0 tx=0x7f1e0000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.726+0000 7f1e04ff9640 1 -- 172.21.15.60:0/1062607343 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00018020 con 0x7f1e0815a9f0 2024-04-25T08:34:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.726+0000 7f1e04ff9640 1 -- 172.21.15.60:0/1062607343 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00014400 con 0x7f1e0815a9f0 2024-04-25T08:34:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.726+0000 7f1e04ff9640 1 -- 172.21.15.60:0/1062607343 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e000147f0 con 0x7f1e0815a9f0 2024-04-25T08:34:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.726+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e080b1040 con 0x7f1e0815a9f0 2024-04-25T08:34:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.726+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e080b1600 con 0x7f1e0815a9f0 2024-04-25T08:34:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.728+0000 7f1e04ff9640 1 -- 172.21.15.60:0/1062607343 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e00005d10 con 0x7f1e0815a9f0 2024-04-25T08:34:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.729+0000 7f1e04ff9640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.729+0000 7f1e04ff9640 1 -- 172.21.15.60:0/1062607343 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e0008a9c0 con 0x7f1e0815a9f0 2024-04-25T08:34:54.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.729+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.729+0000 7f1e10f8e740 1 Processor -- start 2024-04-25T08:34:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e10f8e740 1 -- start start 2024-04-25T08:34:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e07fff640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1df8034000 tx=0x7f1df802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba8145b10 0x559ba8148320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 0x559ba8148860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8118770 0x559ba811bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba8081da0 con 0x559ba8145ef0 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.730+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba802ab80 con 0x559ba8145b10 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 0x559ba8148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ba807de40 con 0x559ba8118770 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 0x559ba8148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34370/0 (socket says 172.21.15.60:34370) 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 -- 172.21.15.60:0/2868604486 learned_addr learned my addr 172.21.15.60:0/2868604486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e07fff640 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba8145b10 0x559ba8148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8118770 0x559ba811bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 -- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8118770 msgr2=0x559ba811bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8118770 0x559ba811bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 -- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba8145b10 msgr2=0x559ba8148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba8145b10 0x559ba8148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 -- 172.21.15.60:0/2868604486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba8034f10 con 0x559ba8145ef0 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e07fff640 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba8145b10 0x559ba8148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1e077fe640 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 0x559ba8148860 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009b90 tx=0x7f1dfc011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2de294cc367d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1dedffb640 1 -- 172.21.15.60:0/2868604486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc019070 con 0x559ba8145ef0 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1dedffb640 1 -- 172.21.15.60:0/2868604486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc004030 con 0x559ba8145ef0 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.731+0000 7f1dedffb640 1 -- 172.21.15.60:0/2868604486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0043e0 con 0x559ba8145ef0 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2868604486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 msgr2=0x559ba8148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 0x559ba8148860 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009b90 tx=0x7f1dfc011690 comp rx=0 tx=0).stop 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2868604486 shutdown_connections 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8118770 0x559ba811bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145ef0 0x559ba8148860 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2868604486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba8145b10 0x559ba8148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2868604486 >> 172.21.15.60:0/2868604486 conn(0x559ba811a9e0 msgr2=0x559ba8163320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2868604486 shutdown_connections 2024-04-25T08:34:54.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.732+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2868604486 wait complete. 2024-04-25T08:34:54.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.733+0000 7f1e10f8e740 1 -- start start 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.733+0000 7f1e10f8e740 1 Processor -- start 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba81601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba8144190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 0x559ba817edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba811db90 con 0x559ba81446d0 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba80204b0 con 0x559ba817b480 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ba811de70 con 0x559ba8145b10 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 0x559ba817edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 0x559ba817edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34372/0 (socket says 172.21.15.60:34372) 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba8144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.734+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2228598909 learned_addr learned my addr 172.21.15.60:0/2228598909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e07fff640 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba81601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e10f8e740 10 client.0 _ll_register_callbacks cb 0x559ba8060180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:54.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 msgr2=0x559ba81601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba81601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 msgr2=0x559ba8144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba8144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2228598909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ba7e607d0 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 0x559ba817edb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1e00088d00 tx=0x7f1e000590a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab8ab995931d5fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1de6ffd640 1 -- 172.21.15.60:0/2228598909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00070020 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1de6ffd640 1 -- 172.21.15.60:0/2228598909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e0005f2b0 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dfc004030 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.735+0000 7f1de6ffd640 1 -- 172.21.15.60:0/2228598909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e000762b0 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 10 client.7413 fetch_fsmap learned FSMap version 49 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x559ba7d963f0 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1de6ffd640 1 -- 172.21.15.60:0/2228598909 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0008a040 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1de6ffd640 1 -- 172.21.15.60:0/2228598909 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e00058430 con 0x559ba81446d0 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 10 client.7413 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:34:54.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 10 client.7413 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 20 client.7413 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba8164db0 0x559ba81651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 4 client.7413 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1dfc004030 con 0x559ba8164db0 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.736+0000 7f1e07fff640 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba8164db0 0x559ba81651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.737+0000 7f1e07fff640 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba8164db0 0x559ba81651c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c56c5d73852e5ed server_cookie=1fda3d74e2890525 in_seq=0 out_seq=0 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.737+0000 7f1de6ffd640 10 client.7413 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.737+0000 7f1de6ffd640 1 -- 172.21.15.60:0/2228598909 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f1dfc004030 con 0x559ba8164db0 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.737+0000 7f1de6ffd640 10 client.7413 handle_command_reply: tid=0 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 client.7413 shutdown 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 msgr2=0x559ba817edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 0x559ba817edb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1e00088d00 tx=0x7f1e000590a0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 shutdown_connections 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba8164db0 0x559ba81651c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba81446d0 0x559ba817edb0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba8144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1de6ffd640 0 client.7413 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2228598909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba81601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 >> 172.21.15.60:0/2228598909 conn(0x559ba8118770 msgr2=0x559ba8119400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 shutdown_connections 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2228598909 wait complete. 2024-04-25T08:34:54.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.738+0000 7f1e10f8e740 20 client.7413 trim_cache size 0 max 16384 2024-04-25T08:34:54.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.758+0000 7f1e10f8e740 1 Processor -- start 2024-04-25T08:34:54.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 -- start start 2024-04-25T08:34:54.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba81446d0 0x559ba818e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba8165d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba817b480 0x559ba8166280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e081fc0c0 con 0x559ba817b480 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08166920 con 0x559ba81446d0 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.759+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080ae1b0 con 0x559ba8145b10 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba8165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba81446d0 0x559ba818e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba817b480 0x559ba8166280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba8165d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:43114/0 (socket says 172.21.15.60:43114) 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e077fe640 1 -- 172.21.15.60:0/2410317002 learned_addr learned my addr 172.21.15.60:0/2410317002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 msgr2=0x559ba8165d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba8165d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba81446d0 msgr2=0x559ba818e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba81446d0 0x559ba818e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 -- 172.21.15.60:0/2410317002 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba7d560f0 con 0x559ba817b480 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e07fff640 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba81446d0 0x559ba818e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e077fe640 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba8165d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba817b480 0x559ba8166280 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000b980 tx=0x7f1e0001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76150e48e89bcf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1ded7fa640 1 -- 172.21.15.60:0/2410317002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00070070 con 0x559ba817b480 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1ded7fa640 1 -- 172.21.15.60:0/2410317002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0008a040 con 0x559ba817b480 2024-04-25T08:34:54.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.760+0000 7f1ded7fa640 1 -- 172.21.15.60:0/2410317002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0005f0b0 con 0x559ba817b480 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2410317002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba817b480 msgr2=0x559ba8166280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba817b480 0x559ba8166280 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000b980 tx=0x7f1e0001f000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2410317002 shutdown_connections 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba817b480 0x559ba8166280 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba8145b10 0x559ba8165d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/2410317002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba81446d0 0x559ba818e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2410317002 >> 172.21.15.60:0/2410317002 conn(0x559ba8118770 msgr2=0x559ba815eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2410317002 shutdown_connections 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.761+0000 7f1e10f8e740 1 -- 172.21.15.60:0/2410317002 wait complete. 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.762+0000 7f1e10f8e740 1 -- start start 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.762+0000 7f1e10f8e740 1 Processor -- start 2024-04-25T08:34:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.763+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba81446d0 0x559ba8165d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.763+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 0x559ba8166280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.763+0000 7f1e10f8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.763+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e080ae7b0 con 0x559ba8145b10 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.763+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba816b650 con 0x559ba817b480 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.763+0000 7f1e10f8e740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ba8118230 con 0x559ba81446d0 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e10f8e740 10 client.0 _ll_register_callbacks cb 0x7f1dfc009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 0x559ba8166280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e0c82c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 0x559ba8166280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34398/0 (socket says 172.21.15.60:34398) 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 -- 172.21.15.60:0/3858706028 learned_addr learned my addr 172.21.15.60:0/3858706028 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e07fff640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba81446d0 0x559ba8165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 -- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba81446d0 msgr2=0x559ba8165d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba81446d0 0x559ba8165d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 -- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 msgr2=0x559ba815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 -- 172.21.15.60:0/3858706028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ba8163a30 con 0x559ba8145b10 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e0c82c640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e07fff640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba81446d0 0x559ba8165d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1e077fe640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 0x559ba8166280 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc011b90 tx=0x7f1dfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11575cd137c01ada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.764+0000 7f1de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1de6ffd640 1 -- 172.21.15.60:0/3858706028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc019040 con 0x559ba8145b10 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1de6ffd640 1 -- 172.21.15.60:0/3858706028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1dfc003b70 con 0x559ba8145b10 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e0008a040 con 0x559ba8145b10 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1de6ffd640 1 -- 172.21.15.60:0/3858706028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dfc009e90 con 0x559ba8145b10 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 10 client.7419 fetch_fsmap learned FSMap version 49 2024-04-25T08:34:54.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x559ba8160310 con 0x559ba8145b10 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1de6ffd640 1 -- 172.21.15.60:0/3858706028 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0008a040 con 0x559ba8145b10 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1de6ffd640 1 -- 172.21.15.60:0/3858706028 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1dfc039080 con 0x559ba8145b10 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 10 client.7419 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 10 client.7419 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 20 client.7419 populate_metadata read hostname 'smithi060' 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.765+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba81d60a0 0x559ba815c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.766+0000 7f1e10f8e740 4 client.7419 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.766+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ba808f2e0 con 0x559ba81d60a0 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.766+0000 7f1e07fff640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba81d60a0 0x559ba815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e07fff640 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba81d60a0 0x559ba815c500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9509dbdd205e094d server_cookie=f858edb14b5a5ab0 in_seq=0 out_seq=0 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1de6ffd640 10 client.7419 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1de6ffd640 1 -- 172.21.15.60:0/3858706028 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+264 (crc 0 0 0) 0x559ba808f2e0 con 0x559ba81d60a0 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1de6ffd640 10 client.7419 handle_command_reply: tid=0 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 client.7419 shutdown 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 msgr2=0x559ba8166280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 0x559ba8166280 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc011b90 tx=0x7f1dfc004730 comp rx=0 tx=0).stop 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 shutdown_connections 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x559ba81d60a0 0x559ba815c500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559ba817b480 0x559ba815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba8145b10 0x559ba8166280 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1de6ffd640 0 client.7419 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 --2- 172.21.15.60:0/3858706028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559ba81446d0 0x559ba8165d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 >> 172.21.15.60:0/3858706028 conn(0x559ba8118770 msgr2=0x559ba8159480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 shutdown_connections 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.767+0000 7f1e10f8e740 1 -- 172.21.15.60:0/3858706028 wait complete. 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e10f8e740 20 client.7419 trim_cache size 0 max 16384 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "scrub active (14 inodes in the stack)", 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: "40340834-4750-43ee-b445-c3e7a8182afd": { 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "~mds0", 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "40340834-4750-43ee-b445-c3e7a8182afd", 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T08:34:54.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:34:54.768 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:34:54.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 msgr2=0x7f1df406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1df8034000 tx=0x7f1df802e000 comp rx=0 tx=0).stop 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 msgr2=0x7f1e080ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 0x7f1e080ca340 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000eed0 tx=0x7f1e0000b480 comp rx=0 tx=0).stop 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 shutdown_connections 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e0815a9f0 0x7f1e080ca340 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e080cdbf0 0x7f1e080c9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 --2- 172.21.15.60:0/1062607343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e080cd520 0x7f1e0814c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 >> 172.21.15.60:0/1062607343 conn(0x7f1e080b2390 msgr2=0x7f1e0814ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.768+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 shutdown_connections 2024-04-25T08:34:54.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:34:54.769+0000 7f1e0e28f640 1 -- 172.21.15.60:0/1062607343 wait complete. 2024-04-25T08:34:54.774 INFO:tasks.cephfs.filesystem:scrub status for tag:40340834-4750-43ee-b445-c3e7a8182afd - {'path': '~mds0', 'tag': '40340834-4750-43ee-b445-c3e7a8182afd', 'options': 'recursive'} 2024-04-25T08:34:56.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:56.707 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:24.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:24.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:35:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.896+0000 7f57c34ac640 1 Processor -- start 2024-04-25T08:35:24.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.896+0000 7f57c34ac640 1 -- start start 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c34ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57bc15b7d0 0x7f57bc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c34ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 0x7f57bc0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c34ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57bc06cf50 con 0x7f57bc06c2d0 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c34ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57bc0b5fc0 con 0x7f57bc15b7d0 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c34ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0ce530 con 0x7f57bc0b5620 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c1a49640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f57bc06c2d0 0x7f57bc06c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54364/0 (socket says 172.21.15.60:54364) 2024-04-25T08:35:24.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c1a49640 1 -- 172.21.15.60:0/3253480619 learned_addr learned my addr 172.21.15.60:0/3253480619 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c1248640 1 --2- 172.21.15.60:0/3253480619 >> v2:172.21.15.100:3300/0 conn(0x7f57bc15b7d0 0x7f57bc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.897+0000 7f57c0a47640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 0x7f57bc0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178589967 0 0) 0x7f57bc06cf50 con 0x7f57bc06c2d0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a8003520 con 0x7f57bc06c2d0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 893874242 0 0) 0x7f57a8003520 con 0x7f57bc06c2d0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 msgr2=0x7f57bc0b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 0x7f57bc0b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 >> v2:172.21.15.100:3300/0 conn(0x7f57bc15b7d0 msgr2=0x7f57bc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 --2- 172.21.15.60:0/3253480619 >> v2:172.21.15.100:3300/0 conn(0x7f57bc15b7d0 0x7f57bc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0ce810 con 0x7f57bc06c2d0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f57b40019f0 con 0x7f57bc06c2d0 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 >> v1:172.21.15.60:6789/0 conn(0x7f57bc06c2d0 legacy=0x7f57bc06c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 0x7f57a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a8003f30 0x7f57a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a8004d40 0x7f57a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0ce530 con 0x7f57a8004d40 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a800b9c0 con 0x7f57a8003f30 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a800bb40 con 0x7f57bc0b5620 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1248640 1 --2- 172.21.15.60:0/3253480619 >> v2:172.21.15.100:3300/0 conn(0x7f57bc15b7d0 0x7f57bc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1a49640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a8004d40 0x7f57a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c0a47640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a8003f30 0x7f57a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1248640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 0x7f57a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1a49640 1 -- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 msgr2=0x7f57a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1a49640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc0b5620 0x7f57a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1a49640 1 -- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a8003f30 msgr2=0x7f57a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1a49640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57a8003f30 0x7f57a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.898+0000 7f57c1a49640 1 -- 172.21.15.60:0/3253480619 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a8003520 con 0x7f57a8004d40 2024-04-25T08:35:24.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.899+0000 7f57c1a49640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a8004d40 0x7f57a80051c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f57b400a880 tx=0x7f57a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1721ae556bf27e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.899+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b400a4a0 con 0x7f57a8004d40 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.899+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b402f070 con 0x7f57a8004d40 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.899+0000 7f57b3fff640 1 -- 172.21.15.60:0/3253480619 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b402bce0 con 0x7f57a8004d40 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c34ac640 1 -- 172.21.15.60:0/3253480619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a8004d40 msgr2=0x7f57a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c34ac640 1 --2- 172.21.15.60:0/3253480619 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57a8004d40 0x7f57a80051c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f57b400a880 tx=0x7f57a8005830 comp rx=0 tx=0).stop 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c1a49640 1 -- 172.21.15.60:0/3253480619 reap_dead start 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c34ac640 1 -- 172.21.15.60:0/3253480619 shutdown_connections 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c34ac640 1 -- 172.21.15.60:0/3253480619 >> 172.21.15.60:0/3253480619 conn(0x7f57bc0d0800 msgr2=0x7f57bc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c34ac640 1 -- 172.21.15.60:0/3253480619 shutdown_connections 2024-04-25T08:35:24.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.900+0000 7f57c34ac640 1 -- 172.21.15.60:0/3253480619 wait complete. 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.901+0000 7f57c34ac640 1 Processor -- start 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.901+0000 7f57c34ac640 1 -- start start 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c34ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57bc1593b0 0x7f57bc0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c34ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 0x7f57bc0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c34ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc1f7780 0x7f57bc0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c34ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0cf550 con 0x7f57bc159790 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c34ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc06d230 con 0x7f57bc1593b0 2024-04-25T08:35:24.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c34ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0cf6d0 con 0x7f57bc1f7780 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 0x7f57bc0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 0x7f57bc0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40302/0 (socket says 172.21.15.60:40302) 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 -- 172.21.15.60:0/1932569830 learned_addr learned my addr 172.21.15.60:0/1932569830 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c1248640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57bc1593b0 0x7f57bc0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 -- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc1f7780 msgr2=0x7f57bc0c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc1f7780 0x7f57bc0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 -- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57bc1593b0 msgr2=0x7f57bc0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57bc1593b0 0x7f57bc0cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.902+0000 7f57c0a47640 1 -- 172.21.15.60:0/1932569830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0cb3e0 con 0x7f57bc159790 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.903+0000 7f57c0a47640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 0x7f57bc0cac70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0118e0 tx=0x7f57ac011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.903+0000 7f57b1ffb640 1 -- 172.21.15.60:0/1932569830 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00fa70 con 0x7f57bc159790 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.903+0000 7f57b1ffb640 1 -- 172.21.15.60:0/1932569830 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac00fc10 con 0x7f57bc159790 2024-04-25T08:35:24.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.903+0000 7f57b1ffb640 1 -- 172.21.15.60:0/1932569830 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac014440 con 0x7f57bc159790 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.903+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57bc0cb6c0 con 0x7f57bc159790 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.903+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc1f7260 con 0x7f57bc159790 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.905+0000 7f57b1ffb640 1 -- 172.21.15.60:0/1932569830 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57ac012040 con 0x7f57bc159790 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.905+0000 7f57b1ffb640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57a006b4d0 0x7f57a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.905+0000 7f57b1ffb640 1 -- 172.21.15.60:0/1932569830 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f57ac08a9d0 con 0x7f57bc159790 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.905+0000 7f57c1248640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57a006b4d0 0x7f57a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.906+0000 7f57c1248640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57a006b4d0 0x7f57a006d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009bf0 tx=0x7f57b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.906+0000 7f57c61ae740 1 Processor -- start 2024-04-25T08:35:24.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.906+0000 7f57c61ae740 1 -- start start 2024-04-25T08:35:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cb10 0x561d4e52f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 0x561d4e52f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e4ff770 0x561d4e502f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c61ae740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e468da0 con 0x561d4e52cef0 2024-04-25T08:35:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e411b80 con 0x561d4e4ff770 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e464e40 con 0x561d4e52cb10 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 0x561d4e52f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 0x561d4e52f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40318/0 (socket says 172.21.15.60:40318) 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 -- 172.21.15.60:0/1020659667 learned_addr learned my addr 172.21.15.60:0/1020659667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c1248640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cb10 0x561d4e52f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c1a49640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e4ff770 0x561d4e502f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 -- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cb10 msgr2=0x561d4e52f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cb10 0x561d4e52f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 -- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e4ff770 msgr2=0x561d4e502f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e4ff770 0x561d4e502f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 -- 172.21.15.60:0/1020659667 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4e41bf10 con 0x561d4e52cef0 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c1248640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cb10 0x561d4e52f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c1a49640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e4ff770 0x561d4e502f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:24.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.907+0000 7f57c0a47640 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 0x561d4e52f860 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f57ac061210 tx=0x7f57ac0590f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bfffe181720832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f579effd640 1 -- 172.21.15.60:0/1020659667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac08a040 con 0x561d4e52cef0 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f579effd640 1 -- 172.21.15.60:0/1020659667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac0570d0 con 0x561d4e52cef0 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f579effd640 1 -- 172.21.15.60:0/1020659667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0556a0 con 0x561d4e52cef0 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 -- 172.21.15.60:0/1020659667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 msgr2=0x561d4e52f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 0x561d4e52f860 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f57ac061210 tx=0x7f57ac0590f0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 -- 172.21.15.60:0/1020659667 shutdown_connections 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e4ff770 0x561d4e502f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cef0 0x561d4e52f860 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 --2- 172.21.15.60:0/1020659667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cb10 0x561d4e52f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 -- 172.21.15.60:0/1020659667 >> 172.21.15.60:0/1020659667 conn(0x561d4e5019e0 msgr2=0x561d4e54a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 -- 172.21.15.60:0/1020659667 shutdown_connections 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.908+0000 7f57c61ae740 1 -- 172.21.15.60:0/1020659667 wait complete. 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.909+0000 7f57c61ae740 1 -- start start 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.909+0000 7f57c61ae740 1 Processor -- start 2024-04-25T08:35:24.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e547400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e52aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e518a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e504b90 con 0x561d4e519620 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e4074b0 con 0x561d4e52cb10 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e504e70 con 0x561d4e52cef0 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c61ae740 10 client.0 _ll_register_callbacks cb 0x561d4e447180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e518a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e518a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40322/0 (socket says 172.21.15.60:40322) 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 -- 172.21.15.60:0/2578978595 learned_addr learned my addr 172.21.15.60:0/2578978595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c0a47640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e52aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1248640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e547400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 -- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 msgr2=0x561d4e52aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e52aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 -- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 msgr2=0x561d4e547400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e547400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.910+0000 7f57c1a49640 1 -- 172.21.15.60:0/2578978595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d4e2477d0 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c1a49640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e518a60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f57b402b530 tx=0x7f57bc0ce9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43da0a6e305e99fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f5797fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f5797fff640 1 -- 172.21.15.60:0/2578978595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4044060 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f5797fff640 1 -- 172.21.15.60:0/2578978595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f57b402bc90 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57ac0570d0 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f5797fff640 1 -- 172.21.15.60:0/2578978595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b404a050 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 10 client.7431 fetch_fsmap learned FSMap version 49 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x561d4e17d3f0 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f5797fff640 1 -- 172.21.15.60:0/2578978595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac0570d0 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f5797fff640 1 -- 172.21.15.60:0/2578978595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57b404d020 con 0x561d4e519620 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 10 client.7431 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:35:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 10 client.7431 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 20 client.7431 populate_metadata read hostname 'smithi060' 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e5321e0 0x561d4e5325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 4 client.7431 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d4e4762e0 con 0x561d4e5321e0 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.911+0000 7f57c1248640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e5321e0 0x561d4e5325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.912+0000 7f57c1248640 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e5321e0 0x561d4e5325c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed603ea6dde8291 server_cookie=8f6d3c49db6faa0e in_seq=0 out_seq=0 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.912+0000 7f5797fff640 10 client.7431 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f5797fff640 1 -- 172.21.15.60:0/2578978595 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d4e4762e0 con 0x561d4e5321e0 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f5797fff640 10 client.7431 handle_command_reply: tid=0 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 client.7431 shutdown 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 msgr2=0x561d4e518a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e518a60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f57b402b530 tx=0x7f57bc0ce9e0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 shutdown_connections 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e5321e0 0x561d4e5325c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e518a60 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e52aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 --2- 172.21.15.60:0/2578978595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e547400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f5797fff640 0 client.7431 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 >> 172.21.15.60:0/2578978595 conn(0x561d4e4ff770 msgr2=0x561d4e52f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 shutdown_connections 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 1 -- 172.21.15.60:0/2578978595 wait complete. 2024-04-25T08:35:24.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.913+0000 7f57c61ae740 20 client.7431 trim_cache size 0 max 16384 2024-04-25T08:35:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.928+0000 7f57c61ae740 1 Processor -- start 2024-04-25T08:35:24.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.928+0000 7f57c61ae740 1 -- start start 2024-04-25T08:35:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e54a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e54a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e515160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c61ae740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc200fe0 con 0x561d4e519620 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc2001c0 con 0x561d4e52cb10 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc1ff5f0 con 0x561d4e52cef0 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e54a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e54a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40334/0 (socket says 172.21.15.60:40334) 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 -- 172.21.15.60:0/4122807525 learned_addr learned my addr 172.21.15.60:0/4122807525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 -- 172.21.15.60:0/4122807525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 msgr2=0x561d4e515160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e515160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 -- 172.21.15.60:0/4122807525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 msgr2=0x561d4e54a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e54a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 -- 172.21.15.60:0/4122807525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4e13d0f0 con 0x561d4e519620 2024-04-25T08:35:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.929+0000 7f57c1248640 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e54a320 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x561d4e52b8b0 tx=0x7f57b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a11e141cf89f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f579e7fc640 1 -- 172.21.15.60:0/4122807525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b802d3f0 con 0x561d4e519620 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f579e7fc640 1 -- 172.21.15.60:0/4122807525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b801c040 con 0x561d4e519620 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f579e7fc640 1 -- 172.21.15.60:0/4122807525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b802d7b0 con 0x561d4e519620 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 -- 172.21.15.60:0/4122807525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 msgr2=0x561d4e54a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e54a320 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x561d4e52b8b0 tx=0x7f57b8016000 comp rx=0 tx=0).stop 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 -- 172.21.15.60:0/4122807525 shutdown_connections 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e515160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e52cb10 0x561d4e54a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 --2- 172.21.15.60:0/4122807525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e519620 0x561d4e54a320 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 -- 172.21.15.60:0/4122807525 >> 172.21.15.60:0/4122807525 conn(0x561d4e4ff770 msgr2=0x561d4e532fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 -- 172.21.15.60:0/4122807525 shutdown_connections 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.930+0000 7f57c61ae740 1 -- 172.21.15.60:0/4122807525 wait complete. 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.931+0000 7f57c61ae740 1 -- start start 2024-04-25T08:35:24.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.931+0000 7f57c61ae740 1 Processor -- start 2024-04-25T08:35:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e519620 0x561d4e5b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 0x561d4e52fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e530370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e52f460 con 0x561d4e52cb10 2024-04-25T08:35:24.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc1ffbf0 con 0x561d4e519620 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e5336d0 con 0x561d4e52cef0 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 10 client.0 _ll_register_callbacks cb 0x7f57ac058e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 0x561d4e52fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 0x561d4e52fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40344/0 (socket says 172.21.15.60:40344) 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 -- 172.21.15.60:0/399533658 learned_addr learned my addr 172.21.15.60:0/399533658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c1248640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e519620 0x561d4e5b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c1a49640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e530370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 -- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 msgr2=0x561d4e530370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e530370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 -- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e519620 msgr2=0x561d4e5b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e519620 0x561d4e5b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 -- 172.21.15.60:0/399533658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d4e5048e0 con 0x561d4e52cb10 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c0a47640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 0x561d4e52fe30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f57ac058fc0 tx=0x7f57ac088cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10410d6ffce4a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f5797fff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f5797fff640 1 -- 172.21.15.60:0/399533658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac08a040 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f5797fff640 1 -- 172.21.15.60:0/399533658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f57ac0612b0 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.932+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57b801c040 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f5797fff640 1 -- 172.21.15.60:0/399533658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ac059730 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 10 client.7437 fetch_fsmap learned FSMap version 49 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x561d4e500870 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f5797fff640 1 -- 172.21.15.60:0/399533658 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b801c040 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f5797fff640 1 -- 172.21.15.60:0/399533658 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57ac025030 con 0x561d4e52cb10 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 10 client.7437 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 10 client.7437 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 20 client.7437 populate_metadata read hostname 'smithi060' 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e549d70 0x561d4e4ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 4 client.7437 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:35:24.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d4e470630 con 0x561d4e549d70 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.933+0000 7f57c1248640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e549d70 0x561d4e4ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f5797fff640 10 client.7437 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c1248640 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e549d70 0x561d4e4ffbf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb70949b6ab5590 server_cookie=4fe26cc3f51d1532 in_seq=0 out_seq=0 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f5797fff640 1 -- 172.21.15.60:0/399533658 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561d4e470630 con 0x561d4e549d70 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f5797fff640 10 client.7437 handle_command_reply: tid=0 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 client.7437 shutdown 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 msgr2=0x561d4e52fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 0x561d4e52fe30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f57ac058fc0 tx=0x7f57ac088cb0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 shutdown_connections 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x561d4e549d70 0x561d4e4ffbf0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x561d4e52cef0 0x561d4e530370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561d4e52cb10 0x561d4e52fe30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 --2- 172.21.15.60:0/399533658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561d4e519620 0x561d4e5b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f5797fff640 0 client.7437 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 >> 172.21.15.60:0/399533658 conn(0x561d4e4ff770 msgr2=0x561d4e532f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 shutdown_connections 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 1 -- 172.21.15.60:0/399533658 wait complete. 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.934+0000 7f57c61ae740 20 client.7437 trim_cache size 0 max 16384 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:35:24.935 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:35:24.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57a006b4d0 msgr2=0x7f57a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57a006b4d0 0x7f57a006d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009bf0 tx=0x7f57b8008040 comp rx=0 tx=0).stop 2024-04-25T08:35:24.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 msgr2=0x7f57bc0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 0x7f57bc0cac70 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0118e0 tx=0x7f57ac011da0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 shutdown_connections 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57bc1f7780 0x7f57bc0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57bc159790 0x7f57bc0cac70 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57a006b4d0 0x7f57a006d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 --2- 172.21.15.60:0/1932569830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57bc1593b0 0x7f57bc0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 >> 172.21.15.60:0/1932569830 conn(0x7f57bc0d0800 msgr2=0x7f57bc160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 shutdown_connections 2024-04-25T08:35:24.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:24.935+0000 7f57c34ac640 1 -- 172.21.15.60:0/1932569830 wait complete. 2024-04-25T08:35:24.940 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:35:24.940 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:24.941 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T08:35:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.049+0000 7fb8b53e2640 1 Processor -- start 2024-04-25T08:35:25.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.049+0000 7fb8b53e2640 1 -- start start 2024-04-25T08:35:25.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8b53e2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb8b015da90 0x7fb8b0161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8b53e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b00b9750 0x7fb8b00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8b53e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b0162040 con 0x7fb8b0161ba0 2024-04-25T08:35:25.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8b53e2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b0162600 con 0x7fb8b015da90 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8b53e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0162320 con 0x7fb8b00b9750 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8af7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb8b0161ba0 0x7fb8b006c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54378/0 (socket says 172.21.15.60:54378) 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8af7fe640 1 -- 172.21.15.60:0/1193626635 learned_addr learned my addr 172.21.15.60:0/1193626635 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/1193626635 >> v2:172.21.15.100:3300/0 conn(0x7fb8b015da90 0x7fb8b0161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1193626635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b00b9750 0x7fb8b00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8adffb640 1 -- 172.21.15.60:0/1193626635 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247568970 0 0) 0x7fb8b0162040 con 0x7fb8b0161ba0 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8adffb640 1 -- 172.21.15.60:0/1193626635 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb898003520 con 0x7fb8b0161ba0 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8ae7fc640 1 -- 172.21.15.60:0/1193626635 >> v2:172.21.15.100:3300/0 conn(0x7fb8b015da90 msgr2=0x7fb8b0161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1193626635 >> v2:172.21.15.100:3300/0 conn(0x7fb8b015da90 0x7fb8b0161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8ae7fc640 1 -- 172.21.15.60:0/1193626635 >> v1:172.21.15.60:6789/0 conn(0x7fb8b0161ba0 legacy=0x7fb8b006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8adffb640 1 -- 172.21.15.60:0/1193626635 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899527970 0 0) 0x7fb898003520 con 0x7fb8b0161ba0 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8ae7fc640 1 -- 172.21.15.60:0/1193626635 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b00d1b10 con 0x7fb8b00b9750 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.050+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/1193626635 >> v2:172.21.15.100:3300/0 conn(0x7fb8b015da90 0x7fb8b0161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.051+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1193626635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b00b9750 0x7fb8b00ce670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097a0 tx=0x7fb8a402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e3f09c1a06c9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.051+0000 7fb8adffb640 1 -- 172.21.15.60:0/1193626635 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4037070 con 0x7fb8b00b9750 2024-04-25T08:35:25.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8adffb640 1 -- 172.21.15.60:0/1193626635 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a402ab10 con 0x7fb8b00b9750 2024-04-25T08:35:25.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8adffb640 1 -- 172.21.15.60:0/1193626635 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80019f0 con 0x7fb8b00b9750 2024-04-25T08:35:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 -- 172.21.15.60:0/1193626635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b00b9750 msgr2=0x7fb8b00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/1193626635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b00b9750 0x7fb8b00ce670 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097a0 tx=0x7fb8a402a080 comp rx=0 tx=0).stop 2024-04-25T08:35:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 -- 172.21.15.60:0/1193626635 shutdown_connections 2024-04-25T08:35:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/1193626635 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b00b9750 0x7fb8b00ce670 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/1193626635 >> v2:172.21.15.100:3300/0 conn(0x7fb8b015da90 0x7fb8b0161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 -- 172.21.15.60:0/1193626635 >> 172.21.15.60:0/1193626635 conn(0x7fb8b0061a20 msgr2=0x7fb8b0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 -- 172.21.15.60:0/1193626635 shutdown_connections 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.052+0000 7fb8b53e2640 1 -- 172.21.15.60:0/1193626635 wait complete. 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.053+0000 7fb8b53e2640 1 Processor -- start 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.053+0000 7fb8b53e2640 1 -- start start 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8b53e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 0x7fb8b015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8b53e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0161ba0 0x7fb8b0151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8b53e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b0152280 0x7fb8b015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8b53e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00d3320 con 0x7fb8b015da90 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8b53e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b015de70 con 0x7fb8b0161ba0 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8b53e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00d34a0 con 0x7fb8b0152280 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 0x7fb8b015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 0x7fb8b015b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40352/0 (socket says 172.21.15.60:40352) 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8aeffd640 1 -- 172.21.15.60:0/2833176103 learned_addr learned my addr 172.21.15.60:0/2833176103 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b0152280 0x7fb8b015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.054+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0161ba0 0x7fb8b0151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8aeffd640 1 -- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b0152280 msgr2=0x7fb8b015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b0152280 0x7fb8b015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8aeffd640 1 -- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0161ba0 msgr2=0x7fb8b0151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0161ba0 0x7fb8b0151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8aeffd640 1 -- 172.21.15.60:0/2833176103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b015f5e0 con 0x7fb8b015da90 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0161ba0 0x7fb8b0151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 0x7fb8b015b050 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000ebe0 tx=0x7fb8a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8977fe640 1 -- 172.21.15.60:0/2833176103 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0017070 con 0x7fb8b015da90 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8977fe640 1 -- 172.21.15.60:0/2833176103 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a000cc30 con 0x7fb8b015da90 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b015f8c0 con 0x7fb8b015da90 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8977fe640 1 -- 172.21.15.60:0/2833176103 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0013470 con 0x7fb8b015da90 2024-04-25T08:35:25.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.055+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b015fe80 con 0x7fb8b015da90 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.057+0000 7fb8977fe640 1 -- 172.21.15.60:0/2833176103 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8a001a070 con 0x7fb8b015da90 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.057+0000 7fb8977fe640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89c06b4d0 0x7fb89c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.057+0000 7fb8977fe640 1 -- 172.21.15.60:0/2833176103 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb8a008a280 con 0x7fb8b015da90 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.058+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89c06b4d0 0x7fb89c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.058+0000 7fb8b81bd740 1 Processor -- start 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.058+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89c06b4d0 0x7fb89c06d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4034000 tx=0x7fb8a402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.058+0000 7fb8b81bd740 1 -- start start 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea93e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea93e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 0x555ea940bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea9343b90 con 0x555ea940b760 2024-04-25T08:35:25.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea92ece40 con 0x555ea941d370 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ea933fc30 con 0x555ea93e0ca0 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 0x555ea940bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 0x555ea940bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40354/0 (socket says 172.21.15.60:40354) 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2956210668 learned_addr learned my addr 172.21.15.60:0/2956210668 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea93e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea93e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 msgr2=0x555ea93e2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea93e2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 msgr2=0x555ea93e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea93e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.059+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2956210668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea91227d0 con 0x555ea940b760 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 0x555ea940bbe0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8001a40 tx=0x7fb8a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f455712259a14fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb894ff9640 1 -- 172.21.15.60:0/2956210668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8019070 con 0x555ea940b760 2024-04-25T08:35:25.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb894ff9640 1 -- 172.21.15.60:0/2956210668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8014480 con 0x555ea940b760 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb894ff9640 1 -- 172.21.15.60:0/2956210668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8014850 con 0x555ea940b760 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2956210668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 msgr2=0x555ea940bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 0x555ea940bbe0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8001a40 tx=0x7fb8a800e530 comp rx=0 tx=0).stop 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2956210668 shutdown_connections 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea940b760 0x555ea940bbe0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea93e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2956210668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea93e2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.060+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2956210668 >> 172.21.15.60:0/2956210668 conn(0x555ea93a8870 msgr2=0x555ea9426760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.061+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2956210668 shutdown_connections 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.061+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2956210668 wait complete. 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.061+0000 7fb8b81bd740 1 -- start start 2024-04-25T08:35:25.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.061+0000 7fb8b81bd740 1 Processor -- start 2024-04-25T08:35:25.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea941d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea941e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea941d370 0x555ea93fab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea93003e0 con 0x555ea93e0ca0 2024-04-25T08:35:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea94092b0 con 0x555ea940b760 2024-04-25T08:35:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ea940ea00 con 0x555ea941d370 2024-04-25T08:35:25.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8b81bd740 10 client.0 _ll_register_callbacks cb 0x555ea92ffdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea941d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea941d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40368/0 (socket says 172.21.15.60:40368) 2024-04-25T08:35:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 -- 172.21.15.60:0/1582486009 learned_addr learned my addr 172.21.15.60:0/1582486009 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea941e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea941d370 0x555ea93fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 -- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea941d370 msgr2=0x555ea93fab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea941d370 0x555ea93fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 -- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 msgr2=0x555ea941e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea941e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 -- 172.21.15.60:0/1582486009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea90180f0 con 0x555ea93e0ca0 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea941e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea941d7d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000ca20 tx=0x7fb8a008a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579ddcc71b9d06fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.062+0000 7fb88dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb88dffb640 1 -- 172.21.15.60:0/1582486009 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0070020 con 0x555ea93e0ca0 2024-04-25T08:35:25.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb88dffb640 1 -- 172.21.15.60:0/1582486009 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb8a00703c0 con 0x555ea93e0ca0 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8a8014480 con 0x555ea93e0ca0 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb88dffb640 1 -- 172.21.15.60:0/1582486009 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a0075050 con 0x555ea93e0ca0 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 10 client.7449 fetch_fsmap learned FSMap version 49 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x555ea9182f60 con 0x555ea93e0ca0 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb88dffb640 1 -- 172.21.15.60:0/1582486009 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8014480 con 0x555ea93e0ca0 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb88dffb640 1 -- 172.21.15.60:0/1582486009 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8a0024060 con 0x555ea93e0ca0 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 10 client.7449 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 10 client.7449 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 20 client.7449 populate_metadata read hostname 'smithi060' 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9439920 0x555ea9439d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 4 client.7449 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ea93510d0 con 0x555ea9439920 2024-04-25T08:35:25.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.063+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9439920 0x555ea9439d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9439920 0x555ea9439d20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6cdeaa9169d0824 server_cookie=55e9aeff6a1ab314 in_seq=0 out_seq=0 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb88dffb640 10 client.7449 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb88dffb640 1 -- 172.21.15.60:0/1582486009 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ea93510d0 con 0x555ea9439920 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb88dffb640 10 client.7449 handle_command_reply: tid=0 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8b81bd740 1 client.7449 shutdown 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 msgr2=0x555ea941d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea941d7d0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000ca20 tx=0x7fb8a008a0b0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 shutdown_connections 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9439920 0x555ea9439d20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.064+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea941d370 0x555ea93fab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea941e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/1582486009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea941d7d0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb88dffb640 0 client.7449 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 >> 172.21.15.60:0/1582486009 conn(0x555ea93a8870 msgr2=0x555ea94096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 shutdown_connections 2024-04-25T08:35:25.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb8b81bd740 1 -- 172.21.15.60:0/1582486009 wait complete. 2024-04-25T08:35:25.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.065+0000 7fb8b81bd740 20 client.7449 trim_cache size 0 max 16384 2024-04-25T08:35:25.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.079+0000 7fb8b81bd740 1 Processor -- start 2024-04-25T08:35:25.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.079+0000 7fb8b81bd740 1 -- start start 2024-04-25T08:35:25.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea9409420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea940b760 0x555ea948ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea948b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b02004a0 con 0x555ea93e0ca0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b01ff660 con 0x555ea941d370 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00cc2a0 con 0x555ea940b760 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea9409420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea9409420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40376/0 (socket says 172.21.15.60:40376) 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 -- 172.21.15.60:0/132576491 learned_addr learned my addr 172.21.15.60:0/132576491 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 -- 172.21.15.60:0/132576491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea940b760 msgr2=0x555ea948ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea940b760 0x555ea948ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 -- 172.21.15.60:0/132576491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 msgr2=0x555ea948b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea948b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 -- 172.21.15.60:0/132576491 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ea8ff11b0 con 0x555ea93e0ca0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea9409420 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0060ac0 tx=0x7fb8a005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3ce0751a7875e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb88ffff640 1 -- 172.21.15.60:0/132576491 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0070040 con 0x555ea93e0ca0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb88ffff640 1 -- 172.21.15.60:0/132576491 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0088280 con 0x555ea93e0ca0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.080+0000 7fb88ffff640 1 -- 172.21.15.60:0/132576491 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0021070 con 0x555ea93e0ca0 2024-04-25T08:35:25.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 -- 172.21.15.60:0/132576491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 msgr2=0x555ea9409420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea9409420 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0060ac0 tx=0x7fb8a005a000 comp rx=0 tx=0).stop 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 -- 172.21.15.60:0/132576491 shutdown_connections 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea941d370 0x555ea948b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea940b760 0x555ea948ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/132576491 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea93e0ca0 0x555ea9409420 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 -- 172.21.15.60:0/132576491 >> 172.21.15.60:0/132576491 conn(0x555ea93a8870 msgr2=0x555ea94490e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 -- 172.21.15.60:0/132576491 shutdown_connections 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 -- 172.21.15.60:0/132576491 wait complete. 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 -- start start 2024-04-25T08:35:25.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.081+0000 7fb8b81bd740 1 Processor -- start 2024-04-25T08:35:25.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea9479f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea9409420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 0x555ea93f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ea94272e0 con 0x555ea941d370 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b01ff210 con 0x555ea940b760 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ea9440350 con 0x555ea93e0ca0 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8b81bd740 10 client.0 _ll_register_callbacks cb 0x7fb8a00586b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 0x555ea93f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 0x555ea93f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40386/0 (socket says 172.21.15.60:40386) 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2952087231 learned_addr learned my addr 172.21.15.60:0/2952087231 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea9479f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea9409420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 msgr2=0x555ea9479f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea9479f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 msgr2=0x555ea9409420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.082+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea9409420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8af7fe640 1 -- 172.21.15.60:0/2952087231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ea940bee0 con 0x555ea941d370 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea9479f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8ae7fc640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea9409420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8af7fe640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 0x555ea93f75d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8001930 tx=0x7fb8a8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692861880f9a563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb88dffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb88dffb640 1 -- 172.21.15.60:0/2952087231 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8019070 con 0x555ea941d370 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb88dffb640 1 -- 172.21.15.60:0/2952087231 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb8a8014890 con 0x555ea941d370 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8a0088280 con 0x555ea941d370 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb88dffb640 1 -- 172.21.15.60:0/2952087231 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a802c050 con 0x555ea941d370 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 10 client.7455 fetch_fsmap learned FSMap version 49 2024-04-25T08:35:25.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x555ea940c9f0 con 0x555ea941d370 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb88dffb640 1 -- 172.21.15.60:0/2952087231 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0088280 con 0x555ea941d370 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb88dffb640 1 -- 172.21.15.60:0/2952087231 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8a802f020 con 0x555ea941d370 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 10 client.7455 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 10 client.7455 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 20 client.7455 populate_metadata read hostname 'smithi060' 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9408d00 0x555ea93f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 4 client.7455 mds_command: new command op to 7203 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.083+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 --> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x555ea934b420 con 0x555ea9408d00 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.084+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9408d00 0x555ea93f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.084+0000 7fb8aeffd640 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9408d00 0x555ea93f7b10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b25571bd465fd6 server_cookie=3fa765a10f65c9c7 in_seq=0 out_seq=0 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.084+0000 7fb88dffb640 10 client.7455 ms_handle_connect on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb88dffb640 1 -- 172.21.15.60:0/2952087231 <== mds.0 v2:172.21.15.100:6835/1892944367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17932 (crc 0 0 0) 0x555ea934b420 con 0x555ea9408d00 2024-04-25T08:35:25.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb88dffb640 10 client.7455 handle_command_reply: tid=0 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 client.7455 shutdown 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 msgr2=0x555ea93f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 0x555ea93f75d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8001930 tx=0x7fb8a8001250 comp rx=0 tx=0).stop 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 shutdown_connections 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:6835/1892944367,v1:172.21.15.100:6837/1892944367] conn(0x555ea9408d00 0x555ea93f7b10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555ea941d370 0x555ea93f75d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ea940b760 0x555ea9409420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb88dffb640 0 client.7455 ms_handle_reset on v2:172.21.15.100:6835/1892944367 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 --2- 172.21.15.60:0/2952087231 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555ea93e0ca0 0x555ea9479f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 >> 172.21.15.60:0/2952087231 conn(0x555ea93a8870 msgr2=0x555ea94056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 shutdown_connections 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 1 -- 172.21.15.60:0/2952087231 wait complete. 2024-04-25T08:35:25.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b81bd740 20 client.7455 trim_cache size 0 max 16384 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 3012, 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 3012, 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 1151394, 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 15120855, 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 17, 2024-04-25T08:35:25.087 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 6, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.237888667, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.104904605, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.047447533, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.069961027, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3011, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.335682889, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000111485 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 10, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000803, 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000080 2024-04-25T08:35:25.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 28768, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 50867, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 9131147, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 56321372, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 1.753981772, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.993601455, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.825013021, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.118278304, 2024-04-25T08:35:25.089 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 50867, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.256126077, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000024694 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 46530, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007486540, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000160 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 120992, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 25168 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.090 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 3704, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 3724, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 927801, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 44011178, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.335646308, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.168838336, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.058926057, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.091174534, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3723, 2024-04-25T08:35:25.091 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.338066601, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000090804 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 119120 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.092 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2863, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 14.487146954, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.005060128 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 4119, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.808337261, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000196245 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.093 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 25637, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 25637, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 25637, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 310.760888177, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.012121577 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 12, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2024-04-25T08:35:25.094 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 17885, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 17885, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 17885, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 2, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 2, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 2, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 47617, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 47118, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 147, 2024-04-25T08:35:25.095 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 2011, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 2011, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:35:25.096 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 11, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 2000, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 1210, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 4038, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 3261, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 2242, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 52, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 14014 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.097 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 4141, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 3707, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 2242, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:35:25.098 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "evlrg": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 26484, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 26484, 2024-04-25T08:35:25.099 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 37, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 37, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "segmjr": 4, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 95369277, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 0, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2580, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 85.941949457, 2024-04-25T08:35:25.100 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.033310833 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 0 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 14, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 2015, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 2001, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 13, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 13, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 0, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 17885, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 19886, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 2001, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 2, 2024-04-25T08:35:25.101 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 7354, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 7352, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 100252, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 215396 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 29050, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 25637, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 5, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2000, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1.476472163, 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000738236 2024-04-25T08:35:25.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 29, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.004831644, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000166608 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 6000, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 7.540886480, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001256814 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.103 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 7353, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 6.329535245, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000860809 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.104 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000138271, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000138271 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.105 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.002625617, 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002625617 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.106 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.335501912, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.335501912 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2024-04-25T08:35:25.107 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.209484873, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.026185609 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 2242, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 279.191802296, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.124528011 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2024-04-25T08:35:25.108 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.003188177, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003188177 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.109 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000274225, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000274225 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.110 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2024-04-25T08:35:25.111 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8000, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 15.666147274, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001958268 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 1, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 0, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 18569, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 18569, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 75, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "mdthresh_evicted": 0 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:35:25.113 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2024-04-25T08:35:25.114 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:35:25.115 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 250212, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 91, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 352, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 4, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 5528, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 78, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:35:25.116 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 13467733, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 661400, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 8550, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 91857552, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 37, 2024-04-25T08:35:25.117 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 8550, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8550, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 27.513272669, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.003217926 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 8550, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 8643 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 8550, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 2068, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 6482, 2024-04-25T08:35:25.118 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 0, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 11, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 56, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 2635, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 35, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 1707, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 2033, 2024-04-25T08:35:25.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 2012, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 22, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2024-04-25T08:35:25.120 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2024-04-25T08:35:25.121 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 85, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 3, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 8, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 8, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 105, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 0, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 27 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 1, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:35:25.122 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 32, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed_ops": 3707, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 3707, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.123 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 35478, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 8441041, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 35478, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 35478, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 8441041, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.124 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:35:25.125 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 8550, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 92245615, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 8550, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 92245615, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:35:25.126 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 8550, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 8550, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 8550, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 8550, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.127 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 3707, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 388063, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 3707, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 55, 2024-04-25T08:35:25.128 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 388063, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x55ad9b57e6e0": { 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 26484, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 91174973, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2024-04-25T08:35:25.129 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 26484, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 2580, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 91174973, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:25.130 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89c06b4d0 msgr2=0x7fb89c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89c06b4d0 0x7fb89c06d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4034000 tx=0x7fb8a402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 msgr2=0x7fb8b015b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 0x7fb8b015b050 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000ebe0 tx=0x7fb8a000c520 comp rx=0 tx=0).stop 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 shutdown_connections 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8b0152280 0x7fb8b015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb89c06b4d0 0x7fb89c06d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0161ba0 0x7fb8b0151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 --2- 172.21.15.60:0/2833176103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8b015da90 0x7fb8b015b050 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 >> 172.21.15.60:0/2833176103 conn(0x7fb8b0061a20 msgr2=0x7fb8b00b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 shutdown_connections 2024-04-25T08:35:25.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.085+0000 7fb8b53e2640 1 -- 172.21.15.60:0/2833176103 wait complete. 2024-04-25T08:35:25.133 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:35:25.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:25.187 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:25.224 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:35:25.225 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:35:25.225 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:25.241 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:35:25.241 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:25.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.441+0000 7f544141f640 1 Processor -- start 2024-04-25T08:35:25.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.441+0000 7f544141f640 1 -- start start 2024-04-25T08:35:25.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f544141f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f544141f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f543c0d0fe0 0x7f543c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f544141f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c15e810 con 0x7f543c15dd20 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f544141f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c0b9b30 con 0x7f543c0d0fe0 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f544141f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0d1c00 con 0x7f543c0b96d0 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f543affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f543c15dd20 0x7f543c15e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54390/0 (socket says 172.21.15.60:54390) 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f543affd640 1 -- 172.21.15.60:0/1119148800 learned_addr learned my addr 172.21.15.60:0/1119148800 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f543a7fc640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f543b7fe640 1 --2- 172.21.15.60:0/1119148800 >> v2:172.21.15.100:3300/0 conn(0x7f543c0d0fe0 0x7f543c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f5439ffb640 1 -- 172.21.15.60:0/1119148800 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189801450 0 0) 0x7f543c15e810 con 0x7f543c15dd20 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f5439ffb640 1 -- 172.21.15.60:0/1119148800 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5428003520 con 0x7f543c15dd20 2024-04-25T08:35:25.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f5439ffb640 1 -- 172.21.15.60:0/1119148800 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16260799 0 0) 0x7f5428003520 con 0x7f543c15dd20 2024-04-25T08:35:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.442+0000 7f543b7fe640 1 -- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 msgr2=0x7f543c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.443+0000 7f543b7fe640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.443+0000 7f543b7fe640 1 -- 172.21.15.60:0/1119148800 >> v1:172.21.15.60:6789/0 conn(0x7f543c15dd20 legacy=0x7f543c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.443+0000 7f543b7fe640 1 -- 172.21.15.60:0/1119148800 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c15eac0 con 0x7f543c0d0fe0 2024-04-25T08:35:25.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.443+0000 7f543a7fc640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:25.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.443+0000 7f543b7fe640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0d2370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f542c009d90 tx=0x7f542c02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3dd102f5d3431a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.443+0000 7f5439ffb640 1 -- 172.21.15.60:0/1119148800 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434001670 con 0x7f543c0d0fe0 2024-04-25T08:35:25.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f5439ffb640 1 -- 172.21.15.60:0/1119148800 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c007540 con 0x7f543c0d0fe0 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f5439ffb640 1 -- 172.21.15.60:0/1119148800 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0078f0 con 0x7f543c0d0fe0 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 -- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 msgr2=0x7f543c0d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0d2370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f542c009d90 tx=0x7f542c02a560 comp rx=0 tx=0).stop 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 -- 172.21.15.60:0/1119148800 shutdown_connections 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0d2370 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 --2- 172.21.15.60:0/1119148800 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 -- 172.21.15.60:0/1119148800 >> 172.21.15.60:0/1119148800 conn(0x7f543c061a20 msgr2=0x7f543c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 -- 172.21.15.60:0/1119148800 shutdown_connections 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.444+0000 7f544141f640 1 -- 172.21.15.60:0/1119148800 wait complete. 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.445+0000 7f544141f640 1 Processor -- start 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.445+0000 7f544141f640 1 -- start start 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f544141f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f544141f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f544141f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 0x7f543c0aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f544141f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0d1e50 con 0x7f543c15dd20 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f544141f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c15f080 con 0x7f543c0d0fe0 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f544141f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0d28b0 con 0x7f543c0b96d0 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 0x7f543c0aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f543b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 0x7f543c0aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40400/0 (socket says 172.21.15.60:40400) 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.446+0000 7f543b7fe640 1 -- 172.21.15.60:0/217909290 learned_addr learned my addr 172.21.15.60:0/217909290 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543a7fc640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543affd640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543b7fe640 1 -- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 msgr2=0x7f543c0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543b7fe640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543b7fe640 1 -- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 msgr2=0x7f543c0ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543b7fe640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543b7fe640 1 -- 172.21.15.60:0/217909290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0af6e0 con 0x7f543c15dd20 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543affd640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f543b7fe640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 0x7f543c0aa790 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f542c02ab50 tx=0x7f542c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c040070 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c031c40 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c0ab250 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c009450 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.447+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c0ab810 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.448+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c073990 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.448+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f542c039040 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.449+0000 7f54337fe640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f541c06b4d0 0x7f541c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.449+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f542c0ab8a0 con 0x7f543c15dd20 2024-04-25T08:35:25.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.449+0000 7f543affd640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f541c06b4d0 0x7f541c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.450+0000 7f543affd640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f541c06b4d0 0x7f541c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5434001eb0 tx=0x7f5434009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.453+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f542c0303a0 con 0x7f543c15dd20 2024-04-25T08:35:25.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.641+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f543c0abbb0 con 0x7f543c15dd20 2024-04-25T08:35:25.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.642+0000 7f54337fe640 1 -- 172.21.15.60:0/217909290 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f542c07a690 con 0x7f543c15dd20 2024-04-25T08:35:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:25.637 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:04.110291+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f541c06b4d0 msgr2=0x7f541c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f541c06b4d0 0x7f541c06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5434001eb0 tx=0x7f5434009040 comp rx=0 tx=0).stop 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 msgr2=0x7f543c0aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 0x7f543c0aa790 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f542c02ab50 tx=0x7f542c004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 shutdown_connections 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f543c15dd20 0x7f543c0aa790 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f541c06b4d0 0x7f541c06d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f543c0d0fe0 0x7f543c0ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 --2- 172.21.15.60:0/217909290 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f543c0b96d0 0x7f543c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 >> 172.21.15.60:0/217909290 conn(0x7f543c061a20 msgr2=0x7f543c061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 shutdown_connections 2024-04-25T08:35:25.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.644+0000 7f544141f640 1 -- 172.21.15.60:0/217909290 wait complete. 2024-04-25T08:35:25.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:33:55.215572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:33:55.910093+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:25.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:25.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.759+0000 7f68199d5640 1 Processor -- start 2024-04-25T08:35:25.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.759+0000 7f68199d5640 1 -- start start 2024-04-25T08:35:25.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68199d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 0x7f68140b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68199d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681406cf50 con 0x7f681415b7d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68199d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140ce530 con 0x7f681406c2d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68199d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68140b5fc0 con 0x7f68140b5620 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f6812ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f681415b7d0 0x7f68140cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54398/0 (socket says 172.21.15.60:54398) 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f6812ffd640 1 -- 172.21.15.60:0/4170749039 learned_addr learned my addr 172.21.15.60:0/4170749039 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f680bfff640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 0x7f68140b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382848217 0 0) 0x7f681406cf50 con 0x7f681415b7d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6800003520 con 0x7f681415b7d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884404488 0 0) 0x7f68140ce530 con 0x7f681406c2d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681406cf50 con 0x7f681406c2d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 283360640 0 0) 0x7f6800003520 con 0x7f681415b7d0 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 msgr2=0x7f68140b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 0x7f68140b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 >> v1:172.21.15.100:6789/0 conn(0x7f681406c2d0 legacy=0x7f681406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140ce810 con 0x7f681415b7d0 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6804001670 con 0x7f681415b7d0 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.760+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 >> v1:172.21.15.60:6789/0 conn(0x7f681415b7d0 legacy=0x7f68140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68127fc640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6800003b70 0x7f6800003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68127fc640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 0x7f6800004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68127fc640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6800004d40 0x7f68000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680000bad0 con 0x7f6800003b70 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680000bc80 con 0x7f6800004d40 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680000be80 con 0x7f68140b5620 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f6812ffd640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6800003b70 0x7f6800003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f680bfff640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 0x7f6800004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f68137fe640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6800004d40 0x7f68000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f6812ffd640 1 -- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 msgr2=0x7f6800004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f6812ffd640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b5620 0x7f6800004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f6812ffd640 1 -- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6800004d40 msgr2=0x7f68000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f6812ffd640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6800004d40 0x7f68000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.761+0000 7f6812ffd640 1 -- 172.21.15.60:0/4170749039 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800003520 con 0x7f6800003b70 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f6812ffd640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6800003b70 0x7f6800003f70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6804007fc0 tx=0x7f6800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4df9f36174344c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804007be0 con 0x7f6800003b70 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804005af0 con 0x7f6800003b70 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68127fc640 1 -- 172.21.15.60:0/4170749039 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804007070 con 0x7f6800003b70 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68199d5640 1 -- 172.21.15.60:0/4170749039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6800003b70 msgr2=0x7f6800003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68199d5640 1 --2- 172.21.15.60:0/4170749039 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6800003b70 0x7f6800003f70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6804007fc0 tx=0x7f6800005830 comp rx=0 tx=0).stop 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68137fe640 1 -- 172.21.15.60:0/4170749039 reap_dead start 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68199d5640 1 -- 172.21.15.60:0/4170749039 shutdown_connections 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68199d5640 1 -- 172.21.15.60:0/4170749039 >> 172.21.15.60:0/4170749039 conn(0x7f68140d0800 msgr2=0x7f68140d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68199d5640 1 -- 172.21.15.60:0/4170749039 shutdown_connections 2024-04-25T08:35:25.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.762+0000 7f68199d5640 1 -- 172.21.15.60:0/4170749039 wait complete. 2024-04-25T08:35:25.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 Processor -- start 2024-04-25T08:35:25.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 -- start start 2024-04-25T08:35:25.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 0x7f68140b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b8680 0x7f68141f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68140cbde0 0x7f68141f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681415bbb0 con 0x7f68140cba00 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681406d230 con 0x7f68140cbde0 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f68199d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681415bd30 con 0x7f68140b8680 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.764+0000 7f6812ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 0x7f68140b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 0x7f68140b7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40410/0 (socket says 172.21.15.60:40410) 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 -- 172.21.15.60:0/303663299 learned_addr learned my addr 172.21.15.60:0/303663299 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f68137fe640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68140cbde0 0x7f68141f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 -- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b8680 msgr2=0x7f68141f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f680bfff640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b8680 0x7f68141f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b8680 0x7f68141f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 -- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68140cbde0 msgr2=0x7f68141f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68140cbde0 0x7f68141f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 -- 172.21.15.60:0/303663299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140b75e0 con 0x7f68140cba00 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f68137fe640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68140cbde0 0x7f68141f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f680bfff640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b8680 0x7f68141f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6812ffd640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 0x7f68140b7000 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f680402ac10 tx=0x7f68140ce9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804041020 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804038040 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68140b78c0 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804001670 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.765+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68140b7e80 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.766+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6814073a00 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.767+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6804044020 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.767+0000 7f6810ff9640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67f406b500 0x7f67f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.767+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f68040aa730 con 0x7f68140cba00 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.767+0000 7f680bfff640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67f406b500 0x7f67f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.768+0000 7f680bfff640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67f406b500 0x7f67f406d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f68140d3490 tx=0x7f67fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.771+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6804079360 con 0x7f68140cba00 2024-04-25T08:35:25.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.950+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68140ca620 con 0x7f68140cba00 2024-04-25T08:35:25.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.950+0000 7f6810ff9640 1 -- 172.21.15.60:0/303663299 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6804078c60 con 0x7f68140cba00 2024-04-25T08:35:25.945 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:35:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67f406b500 msgr2=0x7f67f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67f406b500 0x7f67f406d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f68140d3490 tx=0x7f67fc008040 comp rx=0 tx=0).stop 2024-04-25T08:35:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 msgr2=0x7f68140b7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 0x7f68140b7000 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f680402ac10 tx=0x7f68140ce9e0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 shutdown_connections 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67f406b500 0x7f67f406d9c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68140cbde0 0x7f68141f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f68140b8680 0x7f68141f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 --2- 172.21.15.60:0/303663299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f68140cba00 0x7f68140b7000 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 >> 172.21.15.60:0/303663299 conn(0x7f68140d0800 msgr2=0x7f68140d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 shutdown_connections 2024-04-25T08:35:25.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:25.952+0000 7f68199d5640 1 -- 172.21.15.60:0/303663299 wait complete. 2024-04-25T08:35:25.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:35:26.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.066+0000 7f3f4034f640 1 Processor -- start 2024-04-25T08:35:26.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.066+0000 7f3f4034f640 1 -- start start 2024-04-25T08:35:26.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f4034f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f380ce230 0x7f3f380cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f4034f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f3815b270 0x7f3f3815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f4034f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f38072280 con 0x7f3f380cc4f0 2024-04-25T08:35:26.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f4034f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f38071270 con 0x7f3f3815b270 2024-04-25T08:35:26.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f4034f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380714c0 con 0x7f3f380ce230 2024-04-25T08:35:26.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f3e8ec640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f3815b270 0x7f3f3815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f3d8ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f380ce230 0x7f3f380cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f3e0eb640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3f380cc4f0 0x7f3f380cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:55138/0 (socket says 172.21.15.60:55138) 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.067+0000 7f3f3e0eb640 1 -- 172.21.15.60:0/3364158017 learned_addr learned my addr 172.21.15.60:0/3364158017 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.068+0000 7f3f3e8ec640 1 -- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f380ce230 msgr2=0x7f3f380cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.068+0000 7f3f3e8ec640 1 --2- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f380ce230 0x7f3f380cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.068+0000 7f3f3e8ec640 1 -- 172.21.15.60:0/3364158017 >> v1:172.21.15.100:6789/0 conn(0x7f3f380cc4f0 legacy=0x7f3f380cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.068+0000 7f3f3e8ec640 1 -- 172.21.15.60:0/3364158017 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380717a0 con 0x7f3f3815b270 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.068+0000 7f3f3e8ec640 1 --2- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3815b270 0x7f3f3815b6c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097a0 tx=0x7f3f3002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0615e30af787c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f3d0e9640 1 -- 172.21.15.60:0/3364158017 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30037070 con 0x7f3f3815b270 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f3d0e9640 1 -- 172.21.15.60:0/3364158017 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30032440 con 0x7f3f3815b270 2024-04-25T08:35:26.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f3d0e9640 1 -- 172.21.15.60:0/3364158017 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30032810 con 0x7f3f3815b270 2024-04-25T08:35:26.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 -- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3815b270 msgr2=0x7f3f3815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 --2- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3815b270 0x7f3f3815b6c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3f300097a0 tx=0x7f3f3002a180 comp rx=0 tx=0).stop 2024-04-25T08:35:26.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 -- 172.21.15.60:0/3364158017 shutdown_connections 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 --2- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f3815b270 0x7f3f3815b6c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 --2- 172.21.15.60:0/3364158017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f380ce230 0x7f3f380cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 -- 172.21.15.60:0/3364158017 >> 172.21.15.60:0/3364158017 conn(0x7f3f3806abe0 msgr2=0x7f3f3806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 -- 172.21.15.60:0/3364158017 shutdown_connections 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.069+0000 7f3f4034f640 1 -- 172.21.15.60:0/3364158017 wait complete. 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.070+0000 7f3f4034f640 1 Processor -- start 2024-04-25T08:35:26.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.070+0000 7f3f4034f640 1 -- start start 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f4034f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 0x7f3f38148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f4034f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f380ce230 0x7f3f381f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f4034f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f3815b270 0x7f3f381f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f4034f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3815bc00 con 0x7f3f380cc4f0 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f4034f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f38071d60 con 0x7f3f380ce230 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f4034f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3815bd80 con 0x7f3f3815b270 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f3e0eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 0x7f3f38148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f3e0eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 0x7f3f38148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40420/0 (socket says 172.21.15.60:40420) 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f3e0eb640 1 -- 172.21.15.60:0/160313532 learned_addr learned my addr 172.21.15.60:0/160313532 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f3e8ec640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f3815b270 0x7f3f381f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.071+0000 7f3f3d8ea640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f380ce230 0x7f3f381f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f3e0eb640 1 -- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f3815b270 msgr2=0x7f3f381f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f3e0eb640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f3815b270 0x7f3f381f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f3e0eb640 1 -- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f380ce230 msgr2=0x7f3f381f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f3e0eb640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f380ce230 0x7f3f381f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f3e0eb640 1 -- 172.21.15.60:0/160313532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f381546e0 con 0x7f3f380cc4f0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f3e0eb640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 0x7f3f38148c20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800caa0 tx=0x7f3f2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28017070 con 0x7f3f380cc4f0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2800ac00 con 0x7f3f380cc4f0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f38142dc0 con 0x7f3f380cc4f0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28012400 con 0x7f3f380cc4f0 2024-04-25T08:35:26.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.072+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f38154120 con 0x7f3f380cc4f0 2024-04-25T08:35:26.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.073+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38073a00 con 0x7f3f380cc4f0 2024-04-25T08:35:26.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.073+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f2801a070 con 0x7f3f380cc4f0 2024-04-25T08:35:26.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.074+0000 7f3f36ffd640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f2406b4d0 0x7f3f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.074+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f3f28089fb0 con 0x7f3f380cc4f0 2024-04-25T08:35:26.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.074+0000 7f3f3d8ea640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f2406b4d0 0x7f3f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.075+0000 7f3f3d8ea640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f2406b4d0 0x7f3f2406d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3f381f4e20 tx=0x7f3f2c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.078+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f280586c0 con 0x7f3f380cc4f0 2024-04-25T08:35:26.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.257+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f38146df0 con 0x7f3f380cc4f0 2024-04-25T08:35:26.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.258+0000 7f3f36ffd640 1 -- 172.21.15.60:0/160313532 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f2805c760 con 0x7f3f380cc4f0 2024-04-25T08:35:26.253 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f2406b4d0 msgr2=0x7f3f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f2406b4d0 0x7f3f2406d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3f381f4e20 tx=0x7f3f2c009190 comp rx=0 tx=0).stop 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 msgr2=0x7f3f38148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 0x7f3f38148c20 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800caa0 tx=0x7f3f2800cf60 comp rx=0 tx=0).stop 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 shutdown_connections 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f3815b270 0x7f3f381f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f2406b4d0 0x7f3f2406d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f380ce230 0x7f3f381f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.259+0000 7f3f4034f640 1 --2- 172.21.15.60:0/160313532 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f380cc4f0 0x7f3f38148c20 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.260+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 >> 172.21.15.60:0/160313532 conn(0x7f3f3806abe0 msgr2=0x7f3f3806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.260+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 shutdown_connections 2024-04-25T08:35:26.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.260+0000 7f3f4034f640 1 -- 172.21.15.60:0/160313532 wait complete. 2024-04-25T08:35:26.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:35:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.374+0000 7f1d3c529640 1 Processor -- start 2024-04-25T08:35:26.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.374+0000 7f1d3c529640 1 -- start start 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3c529640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1d3415b7d0 0x7f1d340cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3c529640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d340b5620 0x7f1d340b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3c529640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d3406c2d0 0x7f1d3406c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3c529640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d340b5fc0 con 0x7f1d3406c2d0 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3c529640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3406cca0 con 0x7f1d3415b7d0 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3c529640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3406cf50 con 0x7f1d340b5620 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d3406c2d0 0x7f1d3406c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1d3406c2d0 0x7f1d3406c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40426/0 (socket says 172.21.15.60:40426) 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 -- 172.21.15.60:0/3739736785 learned_addr learned my addr 172.21.15.60:0/3739736785 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3a2c5640 1 --2- 172.21.15.60:0/3739736785 >> v2:172.21.15.100:3300/0 conn(0x7f1d3415b7d0 0x7f1d340cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d39ac4640 1 --2- 172.21.15.60:0/3739736785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d340b5620 0x7f1d340b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 -- 172.21.15.60:0/3739736785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d340b5620 msgr2=0x7f1d340b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 --2- 172.21.15.60:0/3739736785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d340b5620 0x7f1d340b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 -- 172.21.15.60:0/3739736785 >> v2:172.21.15.100:3300/0 conn(0x7f1d3415b7d0 msgr2=0x7f1d340cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 --2- 172.21.15.60:0/3739736785 >> v2:172.21.15.100:3300/0 conn(0x7f1d3415b7d0 0x7f1d340cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.375+0000 7f1d3aac6640 1 -- 172.21.15.60:0/3739736785 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340ce530 con 0x7f1d3406c2d0 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.376+0000 7f1d3aac6640 1 --2- 172.21.15.60:0/3739736785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d3406c2d0 0x7f1d3406c760 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1d3408c0d0 tx=0x7f1d2c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6369bcd5583342c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.376+0000 7f1d392c3640 1 -- 172.21.15.60:0/3739736785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c036070 con 0x7f1d3406c2d0 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.376+0000 7f1d392c3640 1 -- 172.21.15.60:0/3739736785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c031600 con 0x7f1d3406c2d0 2024-04-25T08:35:26.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.376+0000 7f1d392c3640 1 -- 172.21.15.60:0/3739736785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0319d0 con 0x7f1d3406c2d0 2024-04-25T08:35:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 -- 172.21.15.60:0/3739736785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d3406c2d0 msgr2=0x7f1d3406c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 --2- 172.21.15.60:0/3739736785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d3406c2d0 0x7f1d3406c760 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1d3408c0d0 tx=0x7f1d2c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 -- 172.21.15.60:0/3739736785 shutdown_connections 2024-04-25T08:35:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 --2- 172.21.15.60:0/3739736785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d3406c2d0 0x7f1d3406c760 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 --2- 172.21.15.60:0/3739736785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d340b5620 0x7f1d340b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 --2- 172.21.15.60:0/3739736785 >> v2:172.21.15.100:3300/0 conn(0x7f1d3415b7d0 0x7f1d340cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 -- 172.21.15.60:0/3739736785 >> 172.21.15.60:0/3739736785 conn(0x7f1d340d0800 msgr2=0x7f1d340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 -- 172.21.15.60:0/3739736785 shutdown_connections 2024-04-25T08:35:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.377+0000 7f1d3c529640 1 -- 172.21.15.60:0/3739736785 wait complete. 2024-04-25T08:35:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.378+0000 7f1d3c529640 1 Processor -- start 2024-04-25T08:35:26.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.378+0000 7f1d3c529640 1 -- start start 2024-04-25T08:35:26.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3c529640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d3406c2d0 0x7f1d340cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3c529640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 0x7f1d340cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3c529640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d3415b7d0 0x7f1d340ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3c529640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3406d230 con 0x7f1d340b5620 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3c529640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340ceaf0 con 0x7f1d3415b7d0 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3c529640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340cf550 con 0x7f1d3406c2d0 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 0x7f1d340cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 0x7f1d340cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40430/0 (socket says 172.21.15.60:40430) 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 -- 172.21.15.60:0/665075971 learned_addr learned my addr 172.21.15.60:0/665075971 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3a2c5640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d3406c2d0 0x7f1d340cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3aac6640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d3415b7d0 0x7f1d340ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 -- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d3406c2d0 msgr2=0x7f1d340cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d3406c2d0 0x7f1d340cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 -- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d3415b7d0 msgr2=0x7f1d340ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d3415b7d0 0x7f1d340ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d39ac4640 1 -- 172.21.15.60:0/665075971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d341529b0 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.379+0000 7f1d3aac6640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d3415b7d0 0x7f1d340ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.380+0000 7f1d39ac4640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 0x7f1d340cc130 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400e890 tx=0x7f1d2400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.380+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24017070 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.380+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2400c940 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.380+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d340c72c0 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.380+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400cd10 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.380+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d340c7880 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.381+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d340ceef0 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.381+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d2401a070 con 0x7f1d340b5620 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.382+0000 7f1d2affd640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.382+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f1d2408a6d0 con 0x7f1d340b5620 2024-04-25T08:35:26.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.382+0000 7f1d3a2c5640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.383+0000 7f1d3a2c5640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009a30 tx=0x7f1d30008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.386+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d240593a0 con 0x7f1d340b5620 2024-04-25T08:35:26.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.566+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d3415f680 con 0x7f1d340b5620 2024-04-25T08:35:26.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.566+0000 7f1d2affd640 1 -- 172.21.15.60:0/665075971 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d24058ca0 con 0x7f1d340b5620 2024-04-25T08:35:26.561 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:35:26.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 msgr2=0x7f1d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009a30 tx=0x7f1d30008040 comp rx=0 tx=0).stop 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 msgr2=0x7f1d340cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 0x7f1d340cc130 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400e890 tx=0x7f1d2400ed50 comp rx=0 tx=0).stop 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 shutdown_connections 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1d1806b4d0 0x7f1d1806d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d3415b7d0 0x7f1d340ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d340b5620 0x7f1d340cc130 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 --2- 172.21.15.60:0/665075971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1d3406c2d0 0x7f1d340cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.568+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 >> 172.21.15.60:0/665075971 conn(0x7f1d340d0800 msgr2=0x7f1d3415d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.569+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 shutdown_connections 2024-04-25T08:35:26.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.569+0000 7f1d3c529640 1 -- 172.21.15.60:0/665075971 wait complete. 2024-04-25T08:35:26.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:35:26.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.681+0000 7f2b091d5640 1 Processor -- start 2024-04-25T08:35:26.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.681+0000 7f2b091d5640 1 -- start start 2024-04-25T08:35:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b091d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2b040ce230 0x7f2b040cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b091d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 0x7f2b0415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b091d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b04072280 con 0x7f2b040cc4f0 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b091d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b04071270 con 0x7f2b040ce230 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b091d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040714c0 con 0x7f2b0415b270 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b02d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2b040cc4f0 0x7f2b040cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54414/0 (socket says 172.21.15.60:54414) 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b02d9d640 1 -- 172.21.15.60:0/2888454113 learned_addr learned my addr 172.21.15.60:0/2888454113 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395256200 0 0) 0x7f2b04072280 con 0x7f2b040cc4f0 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af0003520 con 0x7f2b040cc4f0 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b0359e640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 0x7f2b0415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b0259c640 1 --2- 172.21.15.60:0/2888454113 >> v2:172.21.15.100:3300/0 conn(0x7f2b040ce230 0x7f2b040cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737248270 0 0) 0x7f2af0003520 con 0x7f2b040cc4f0 2024-04-25T08:35:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 msgr2=0x7f2b0415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 0x7f2b0415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 >> v2:172.21.15.100:3300/0 conn(0x7f2b040ce230 msgr2=0x7f2b040cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 --2- 172.21.15.60:0/2888454113 >> v2:172.21.15.100:3300/0 conn(0x7f2b040ce230 0x7f2b040cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040717a0 con 0x7f2b040cc4f0 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.682+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f2afc001670 con 0x7f2b040cc4f0 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 >> v1:172.21.15.60:6789/0 conn(0x7f2b040cc4f0 legacy=0x7f2b040cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2af00039b0 0x7f2af0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0004800 0x7f2af0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 0x7f2af00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04071270 con 0x7f2af00039b0 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040714c0 con 0x7f2af0004800 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af000ba70 con 0x7f2b0415b270 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2af00039b0 0x7f2af0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b0259c640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0004800 0x7f2af0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b0359e640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 0x7f2af00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b02d9d640 1 -- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 msgr2=0x7f2af00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2b0415b270 0x7f2af00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b02d9d640 1 -- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0004800 msgr2=0x7f2af0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2af0004800 0x7f2af0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.683+0000 7f2b02d9d640 1 -- 172.21.15.60:0/2888454113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0003520 con 0x7f2af00039b0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2af00039b0 0x7f2af0003dd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2afc001050 tx=0x7f2af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71eee2742d2dd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc007d50 con 0x7f2af00039b0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc0058d0 con 0x7f2af00039b0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b01d9b640 1 -- 172.21.15.60:0/2888454113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc005cc0 con 0x7f2af00039b0 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b091d5640 1 -- 172.21.15.60:0/2888454113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2af00039b0 msgr2=0x7f2af0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2888454113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2af00039b0 0x7f2af0003dd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2afc001050 tx=0x7f2af0005830 comp rx=0 tx=0).stop 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b0359e640 1 -- 172.21.15.60:0/2888454113 reap_dead start 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b091d5640 1 -- 172.21.15.60:0/2888454113 shutdown_connections 2024-04-25T08:35:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b091d5640 1 -- 172.21.15.60:0/2888454113 >> 172.21.15.60:0/2888454113 conn(0x7f2b0406abe0 msgr2=0x7f2b0406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b091d5640 1 -- 172.21.15.60:0/2888454113 shutdown_connections 2024-04-25T08:35:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.684+0000 7f2b091d5640 1 -- 172.21.15.60:0/2888454113 wait complete. 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.686+0000 7f2b091d5640 1 Processor -- start 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.686+0000 7f2b091d5640 1 -- start start 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b091d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af00039b0 0x7f2b041eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b091d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b04156d40 0x7f2b041571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b091d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 0x7f2b041524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b091d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0415b6b0 con 0x7f2b04152040 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b091d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04071d60 con 0x7f2b04156d40 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b091d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0415b830 con 0x7f2af00039b0 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 0x7f2b041524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 0x7f2b041524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40456/0 (socket says 172.21.15.60:40456) 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 -- 172.21.15.60:0/2979151181 learned_addr learned my addr 172.21.15.60:0/2979151181 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af00039b0 0x7f2b041eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0259c640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b04156d40 0x7f2b041571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 -- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af00039b0 msgr2=0x7f2b041eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af00039b0 0x7f2b041eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 -- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b04156d40 msgr2=0x7f2b041571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b04156d40 0x7f2b041571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 -- 172.21.15.60:0/2979151181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b041ef040 con 0x7f2b04152040 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2b0359e640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 0x7f2b041524c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2b0406b930 tx=0x7f2af4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.687+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40046a0 con 0x7f2b04152040 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.688+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af401a070 con 0x7f2b04152040 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.688+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b04152d70 con 0x7f2b04152040 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.688+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4015640 con 0x7f2b04152040 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.688+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b04153330 con 0x7f2b04152040 2024-04-25T08:35:26.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.689+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b04073a00 con 0x7f2b04152040 2024-04-25T08:35:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.689+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2af40157e0 con 0x7f2b04152040 2024-04-25T08:35:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.690+0000 7f2afb7fe640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ae406b470 0x7f2ae406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.690+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f2af408aa50 con 0x7f2b04152040 2024-04-25T08:35:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.690+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ae406b470 0x7f2ae406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.690+0000 7f2b02d9d640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ae406b470 0x7f2ae406d930 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2afc033000 tx=0x7f2afc009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.694+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2af4024020 con 0x7f2b04152040 2024-04-25T08:35:26.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:26.861 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:26.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.886+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b0406d3a0 con 0x7f2b04152040 2024-04-25T08:35:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.886+0000 7f2afb7fe640 1 -- 172.21.15.60:0/2979151181 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2af40590b0 con 0x7f2b04152040 2024-04-25T08:35:26.881 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:35:26.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ae406b470 msgr2=0x7f2ae406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ae406b470 0x7f2ae406d930 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2afc033000 tx=0x7f2afc009030 comp rx=0 tx=0).stop 2024-04-25T08:35:26.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 msgr2=0x7f2b041524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 0x7f2b041524c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2b0406b930 tx=0x7f2af4007700 comp rx=0 tx=0).stop 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 shutdown_connections 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b04152040 0x7f2b041524c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ae406b470 0x7f2ae406d930 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b04156d40 0x7f2b041571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 --2- 172.21.15.60:0/2979151181 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2af00039b0 0x7f2b041eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 >> 172.21.15.60:0/2979151181 conn(0x7f2b0406abe0 msgr2=0x7f2b04158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 shutdown_connections 2024-04-25T08:35:26.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:26.888+0000 7f2b091d5640 1 -- 172.21.15.60:0/2979151181 wait complete. 2024-04-25T08:35:26.895 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:35:26.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:26.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:26.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:35:26.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:26.991 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:35:27.046 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:35:27.046 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:27.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:27.064 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:35:27.064 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:27.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:35:27.102 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:35:27.102 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:27.119 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:27.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:27.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.247+0000 7f5ff365d640 1 Processor -- start 2024-04-25T08:35:27.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff365d640 1 -- start start 2024-04-25T08:35:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff365d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fec0d0b10 0x7f5fec0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff365d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0cec00 con 0x7f5fec0b5f00 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff365d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0ceee0 con 0x7f5fec15db60 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff365d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec0ce930 con 0x7f5fec0d0b10 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff0bf8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fec0d0b10 0x7f5fec0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.248+0000 7f5ff0bf8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5fec0d0b10 0x7f5fec0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40470/0 (socket says 172.21.15.60:40470) 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.249+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/202114510 learned_addr learned my addr 172.21.15.60:0/202114510 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.249+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/202114510 >> v1:172.21.15.100:6789/0 conn(0x7f5fec15db60 legacy=0x7f5fec0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.249+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/202114510 >> v1:172.21.15.60:6789/0 conn(0x7f5fec0b5f00 legacy=0x7f5fec0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.249+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/202114510 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0cf1c0 con 0x7f5fec0d0b10 2024-04-25T08:35:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.249+0000 7f5ff0bf8640 1 --2- 172.21.15.60:0/202114510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0ce3f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0098a0 tx=0x7f5fdc02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb9200886eb63fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5fe3fff640 1 -- 172.21.15.60:0/202114510 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc037070 con 0x7f5fec0d0b10 2024-04-25T08:35:27.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5fe3fff640 1 -- 172.21.15.60:0/202114510 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc032440 con 0x7f5fec0d0b10 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5fe3fff640 1 -- 172.21.15.60:0/202114510 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc032810 con 0x7f5fec0d0b10 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 -- 172.21.15.60:0/202114510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 msgr2=0x7f5fec0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 --2- 172.21.15.60:0/202114510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0ce3f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0098a0 tx=0x7f5fdc02a290 comp rx=0 tx=0).stop 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 -- 172.21.15.60:0/202114510 shutdown_connections 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 --2- 172.21.15.60:0/202114510 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0ce3f0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 -- 172.21.15.60:0/202114510 >> 172.21.15.60:0/202114510 conn(0x7f5fec061600 msgr2=0x7f5fec061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 -- 172.21.15.60:0/202114510 shutdown_connections 2024-04-25T08:35:27.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.250+0000 7f5ff365d640 1 -- 172.21.15.60:0/202114510 wait complete. 2024-04-25T08:35:27.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.251+0000 7f5ff365d640 1 Processor -- start 2024-04-25T08:35:27.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.251+0000 7f5ff365d640 1 -- start start 2024-04-25T08:35:27.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff365d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec0b5f00 0x7f5fec14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff365d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff365d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec15db60 0x7f5fec0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff365d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec15df40 con 0x7f5fec0d0b10 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff365d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0b6450 con 0x7f5fec0b5f00 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff365d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec15e0c0 con 0x7f5fec15db60 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff0bf8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff0bf8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40476/0 (socket says 172.21.15.60:40476) 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.252+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/2013546435 learned_addr learned my addr 172.21.15.60:0/2013546435 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec15db60 msgr2=0x7f5fec0c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff1bfa640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec15db60 0x7f5fec0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff13f9640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec0b5f00 0x7f5fec14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff0bf8640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec15db60 0x7f5fec0c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec0b5f00 msgr2=0x7f5fec14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff0bf8640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec0b5f00 0x7f5fec14bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff0bf8640 1 -- 172.21.15.60:0/2013546435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec14c2a0 con 0x7f5fec0d0b10 2024-04-25T08:35:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff0bf8640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0c75a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0098a0 tx=0x7f5fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc037040 con 0x7f5fec0d0b10 2024-04-25T08:35:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc042070 con 0x7f5fec0d0b10 2024-04-25T08:35:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec0c7ef0 con 0x7f5fec0d0b10 2024-04-25T08:35:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc030bb0 con 0x7f5fec0d0b10 2024-04-25T08:35:27.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.253+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec0c8790 con 0x7f5fec0d0b10 2024-04-25T08:35:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.254+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec0c82e0 con 0x7f5fec0d0b10 2024-04-25T08:35:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.255+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fdc047460 con 0x7f5fec0d0b10 2024-04-25T08:35:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.255+0000 7f5fe1ffb640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fd006b4d0 0x7f5fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.255+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5fdc0aaf70 con 0x7f5fec0d0b10 2024-04-25T08:35:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.256+0000 7f5ff13f9640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fd006b4d0 0x7f5fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.256+0000 7f5ff13f9640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fd006b4d0 0x7f5fd006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8001500 tx=0x7f5fe8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.260+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fdc045360 con 0x7f5fec0d0b10 2024-04-25T08:35:27.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.476+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fec073270 con 0x7f5fec0d0b10 2024-04-25T08:35:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.477+0000 7f5fe1ffb640 1 -- 172.21.15.60:0/2013546435 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5fdc079c40 con 0x7f5fec0d0b10 2024-04-25T08:35:27.472 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:27.472 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:33:56.967441+0000","modified":"2024-04-25T08:34:08.095428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7203":{"gid":7203,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1892944367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1892944367},{"type":"v1","addr":"172.21.15.100:6837","nonce":1892944367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:35:27.472 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 49 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fd006b4d0 msgr2=0x7f5fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fd006b4d0 0x7f5fd006d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8001500 tx=0x7f5fe8009040 comp rx=0 tx=0).stop 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 msgr2=0x7f5fec0c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0c75a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0098a0 tx=0x7f5fdc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 shutdown_connections 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5fec15db60 0x7f5fec0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fec0d0b10 0x7f5fec0c75a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fd006b4d0 0x7f5fd006d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 --2- 172.21.15.60:0/2013546435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fec0b5f00 0x7f5fec14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 >> 172.21.15.60:0/2013546435 conn(0x7f5fec061600 msgr2=0x7f5fec0b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 shutdown_connections 2024-04-25T08:35:27.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.479+0000 7f5ff365d640 1 -- 172.21.15.60:0/2013546435 wait complete. 2024-04-25T08:35:27.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:27.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.590+0000 7fa9b7422640 1 Processor -- start 2024-04-25T08:35:27.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.591+0000 7fa9b7422640 1 -- start start 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.591+0000 7fa9b7422640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.591+0000 7fa9b7422640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9b00ce330 0x7fa9b006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.591+0000 7fa9b7422640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b01626e0 con 0x7fa9b00700a0 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.591+0000 7fa9b7422640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b006b500 con 0x7fa9b00ce330 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.591+0000 7fa9b7422640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00cd270 con 0x7fa9b006abe0 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9b00ce330 0x7fa9b006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9b00ce330 0x7fa9b006fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40480/0 (socket says 172.21.15.60:40480) 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 -- 172.21.15.60:0/2037407311 learned_addr learned my addr 172.21.15.60:0/2037407311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:27.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b51be640 1 --2- 172.21.15.60:0/2037407311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 -- 172.21.15.60:0/2037407311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 msgr2=0x7fa9b006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 --2- 172.21.15.60:0/2037407311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 -- 172.21.15.60:0/2037407311 >> v1:172.21.15.100:6789/0 conn(0x7fa9b00700a0 legacy=0x7fa9b006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.592+0000 7fa9b49bd640 1 -- 172.21.15.60:0/2037407311 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b00cd550 con 0x7fa9b00ce330 2024-04-25T08:35:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b49bd640 1 --2- 172.21.15.60:0/2037407311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b006fb60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009870 tx=0x7fa9a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1129782a2e7a8d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9a7fff640 1 -- 172.21.15.60:0/2037407311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0037070 con 0x7fa9b00ce330 2024-04-25T08:35:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9a7fff640 1 -- 172.21.15.60:0/2037407311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a0032440 con 0x7fa9b00ce330 2024-04-25T08:35:27.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9a7fff640 1 -- 172.21.15.60:0/2037407311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0032810 con 0x7fa9b00ce330 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 -- 172.21.15.60:0/2037407311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 msgr2=0x7fa9b006fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2037407311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b006fb60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009870 tx=0x7fa9a002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 -- 172.21.15.60:0/2037407311 shutdown_connections 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2037407311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b006fb60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2037407311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 -- 172.21.15.60:0/2037407311 >> 172.21.15.60:0/2037407311 conn(0x7fa9b00b2220 msgr2=0x7fa9b00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 -- 172.21.15.60:0/2037407311 shutdown_connections 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.593+0000 7fa9b7422640 1 -- 172.21.15.60:0/2037407311 wait complete. 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 Processor -- start 2024-04-25T08:35:27.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 -- start start 2024-04-25T08:35:27.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b00700a0 0x7fa9b01f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b015ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b0161780 con 0x7fa9b00ce330 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b00cdb10 con 0x7fa9b00700a0 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b7422640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b006eb00 con 0x7fa9b006abe0 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.595+0000 7fa9b59bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b015ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b015ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40496/0 (socket says 172.21.15.60:40496) 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 -- 172.21.15.60:0/2870308250 learned_addr learned my addr 172.21.15.60:0/2870308250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b49bd640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b00700a0 0x7fa9b01f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b51be640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 -- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 msgr2=0x7fa9b015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 -- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b00700a0 msgr2=0x7fa9b01f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b00700a0 0x7fa9b01f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 -- 172.21.15.60:0/2870308250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b01f8170 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b59bf640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b015ffa0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800c760 tx=0x7fa9a800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8003b70 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8003d10 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b01f8450 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800aa90 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.596+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b015b960 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.597+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b00685b0 con 0x7fa9b00ce330 2024-04-25T08:35:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.598+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9a800ac30 con 0x7fa9b00ce330 2024-04-25T08:35:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.598+0000 7fa9a5ffb640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99406b4d0 0x7fa99406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.598+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fa9a808a1d0 con 0x7fa9b00ce330 2024-04-25T08:35:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.599+0000 7fa9b51be640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99406b4d0 0x7fa99406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.599+0000 7fa9b51be640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99406b4d0 0x7fa99406d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.603+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9b00685b0 con 0x7fa9b00ce330 2024-04-25T08:35:27.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.783+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9b00afe50 con 0x7fa9b00ce330 2024-04-25T08:35:27.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.784+0000 7fa9a5ffb640 1 -- 172.21.15.60:0/2870308250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fa9a8058850 con 0x7fa9b00ce330 2024-04-25T08:35:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:27.779 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:04.110291+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99406b4d0 msgr2=0x7fa99406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99406b4d0 0x7fa99406d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac008040 comp rx=0 tx=0).stop 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 msgr2=0x7fa9b015ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b015ffa0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800c760 tx=0x7fa9a800cc20 comp rx=0 tx=0).stop 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 shutdown_connections 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9b00ce330 0x7fa9b015ffa0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa99406b4d0 0x7fa99406d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9b00700a0 0x7fa9b01f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 --2- 172.21.15.60:0/2870308250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9b006abe0 0x7fa9b015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 >> 172.21.15.60:0/2870308250 conn(0x7fa9b00b2220 msgr2=0x7fa9b00ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 shutdown_connections 2024-04-25T08:35:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.786+0000 7fa9b7422640 1 -- 172.21.15.60:0/2870308250 wait complete. 2024-04-25T08:35:27.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:33:55.215572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:33:55.910093+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:27.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:27.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.901+0000 7f0d477b0640 1 Processor -- start 2024-04-25T08:35:27.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.901+0000 7f0d477b0640 1 -- start start 2024-04-25T08:35:27.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d477b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d4015dc20 0x7f0d400ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d477b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d40161710 0x7f0d400cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d477b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d400b9330 0x7f0d4006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d477b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4015e000 con 0x7f0d4015dc20 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d477b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d400d1de0 con 0x7f0d400b9330 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d477b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40161c60 con 0x7f0d40161710 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d4554c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d4015dc20 0x7f0d400ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d4554c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d4015dc20 0x7f0d400ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40504/0 (socket says 172.21.15.60:40504) 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d4554c640 1 -- 172.21.15.60:0/2861848822 learned_addr learned my addr 172.21.15.60:0/2861848822 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d45d4d640 1 --2- 172.21.15.60:0/2861848822 >> v2:172.21.15.100:3300/0 conn(0x7f0d400b9330 0x7f0d4006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.902+0000 7f0d44d4b640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d40161710 0x7f0d400cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d4554c640 1 -- 172.21.15.60:0/2861848822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d40161710 msgr2=0x7f0d400cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d4554c640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d40161710 0x7f0d400cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d4554c640 1 -- 172.21.15.60:0/2861848822 >> v2:172.21.15.100:3300/0 conn(0x7f0d400b9330 msgr2=0x7f0d4006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d4554c640 1 --2- 172.21.15.60:0/2861848822 >> v2:172.21.15.100:3300/0 conn(0x7f0d400b9330 0x7f0d4006bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d4554c640 1 -- 172.21.15.60:0/2861848822 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40161f40 con 0x7f0d4015dc20 2024-04-25T08:35:27.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d44d4b640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d40161710 0x7f0d400cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d4554c640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4015dc20 0x7f0d400ce670 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0d400b6570 tx=0x7f0d3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e55eef69e46401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d3ffff640 1 -- 172.21.15.60:0/2861848822 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38037070 con 0x7f0d4015dc20 2024-04-25T08:35:27.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d3ffff640 1 -- 172.21.15.60:0/2861848822 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38032440 con 0x7f0d4015dc20 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.903+0000 7f0d3ffff640 1 -- 172.21.15.60:0/2861848822 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38032810 con 0x7f0d4015dc20 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 -- 172.21.15.60:0/2861848822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4015dc20 msgr2=0x7f0d400ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4015dc20 0x7f0d400ce670 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0d400b6570 tx=0x7f0d3802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 -- 172.21.15.60:0/2861848822 shutdown_connections 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 --2- 172.21.15.60:0/2861848822 >> v2:172.21.15.100:3300/0 conn(0x7f0d400b9330 0x7f0d4006bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d40161710 0x7f0d400cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 --2- 172.21.15.60:0/2861848822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d4015dc20 0x7f0d400ce670 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 -- 172.21.15.60:0/2861848822 >> 172.21.15.60:0/2861848822 conn(0x7f0d400d0420 msgr2=0x7f0d400d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 -- 172.21.15.60:0/2861848822 shutdown_connections 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.904+0000 7f0d477b0640 1 -- 172.21.15.60:0/2861848822 wait complete. 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.905+0000 7f0d477b0640 1 Processor -- start 2024-04-25T08:35:27.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.905+0000 7f0d477b0640 1 -- start start 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d477b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 0x7f0d4015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d477b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d4015dc20 0x7f0d400ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d477b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d40161710 0x7f0d400cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d477b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400b5e50 con 0x7f0d400b9330 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d477b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40162500 con 0x7f0d40161710 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d477b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400b5fd0 con 0x7f0d4015dc20 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 0x7f0d4015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 0x7f0d4015c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40506/0 (socket says 172.21.15.60:40506) 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 -- 172.21.15.60:0/3671285769 learned_addr learned my addr 172.21.15.60:0/3671285769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 -- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d4015dc20 msgr2=0x7f0d400ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d4015dc20 0x7f0d400ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 -- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d40161710 msgr2=0x7f0d400cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d40161710 0x7f0d400cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.906+0000 7f0d4554c640 1 -- 172.21.15.60:0/3671285769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400ac3f0 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.907+0000 7f0d4554c640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 0x7f0d4015c730 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.907+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38041070 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.907+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3803c870 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.907+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d400ac6d0 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.907+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38037040 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.907+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4015cc70 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.908+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d40073a00 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.908+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d38044070 con 0x7f0d400b9330 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.909+0000 7f0d3dffb640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.909+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0d380aace0 con 0x7f0d400b9330 2024-04-25T08:35:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.909+0000 7f0d44d4b640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.910+0000 7f0d44d4b640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d34006fd0 tx=0x7f0d34008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:27.913+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d38079a40 con 0x7f0d400b9330 2024-04-25T08:35:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.092+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d400d16b0 con 0x7f0d400b9330 2024-04-25T08:35:28.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.092+0000 7f0d3dffb640 1 -- 172.21.15.60:0/3671285769 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d38079340 con 0x7f0d400b9330 2024-04-25T08:35:28.087 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:35:28.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d2c06b4d0 msgr2=0x7f0d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0d34006fd0 tx=0x7f0d34008040 comp rx=0 tx=0).stop 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 msgr2=0x7f0d4015c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 0x7f0d4015c730 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d380046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 shutdown_connections 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d40161710 0x7f0d400cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d4015dc20 0x7f0d400ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 --2- 172.21.15.60:0/3671285769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d400b9330 0x7f0d4015c730 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 >> 172.21.15.60:0/3671285769 conn(0x7f0d400d0420 msgr2=0x7f0d400d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 shutdown_connections 2024-04-25T08:35:28.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.094+0000 7f0d477b0640 1 -- 172.21.15.60:0/3671285769 wait complete. 2024-04-25T08:35:28.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:35:28.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.208+0000 7f6cd2de0640 1 Processor -- start 2024-04-25T08:35:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.208+0000 7f6cd2de0640 1 -- start start 2024-04-25T08:35:28.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6cd2de0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ccc1616f0 0x7f6ccc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6cd2de0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc06c8d0 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6cd2de0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc0c6960 con 0x7f6ccc1610f0 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6cd2de0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc1614d0 con 0x7f6ccc1616f0 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6ccbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ccc1616f0 0x7f6ccc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6ccbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ccc1616f0 0x7f6ccc161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40510/0 (socket says 172.21.15.60:40510) 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.209+0000 7f6ccbfff640 1 -- 172.21.15.60:0/2003111059 learned_addr learned my addr 172.21.15.60:0/2003111059 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.210+0000 7f6ccbfff640 1 -- 172.21.15.60:0/2003111059 >> v1:172.21.15.100:6789/0 conn(0x7f6ccc1610f0 legacy=0x7f6ccc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.210+0000 7f6ccbfff640 1 -- 172.21.15.60:0/2003111059 >> v1:172.21.15.60:6789/0 conn(0x7f6ccc0ce0a0 legacy=0x7f6ccc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.210+0000 7f6ccbfff640 1 -- 172.21.15.60:0/2003111059 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc162260 con 0x7f6ccc1616f0 2024-04-25T08:35:28.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.210+0000 7f6ccbfff640 1 --2- 172.21.15.60:0/2003111059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc161b50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009870 tx=0x7f6cc002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94de5c8fd1a6d5df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6ccb7fe640 1 -- 172.21.15.60:0/2003111059 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0037070 con 0x7f6ccc1616f0 2024-04-25T08:35:28.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6ccb7fe640 1 -- 172.21.15.60:0/2003111059 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc002ae50 con 0x7f6ccc1616f0 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6ccb7fe640 1 -- 172.21.15.60:0/2003111059 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0032860 con 0x7f6ccc1616f0 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 -- 172.21.15.60:0/2003111059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc1616f0 msgr2=0x7f6ccc161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/2003111059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc161b50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009870 tx=0x7f6cc002a300 comp rx=0 tx=0).stop 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 -- 172.21.15.60:0/2003111059 shutdown_connections 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/2003111059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc161b50 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 -- 172.21.15.60:0/2003111059 >> 172.21.15.60:0/2003111059 conn(0x7f6ccc06f500 msgr2=0x7f6ccc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 -- 172.21.15.60:0/2003111059 shutdown_connections 2024-04-25T08:35:28.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.211+0000 7f6cd2de0640 1 -- 172.21.15.60:0/2003111059 wait complete. 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.212+0000 7f6cd2de0640 1 Processor -- start 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.212+0000 7f6cd2de0640 1 -- start start 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd2de0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 0x7f6ccc153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd2de0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ccc1610f0 0x7f6ccc1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd2de0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd2de0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cfce0 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd2de0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cfa00 con 0x7f6ccc1616f0 2024-04-25T08:35:28.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd2de0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cf690 con 0x7f6ccc1610f0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd0b7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 0x7f6ccc153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd0b7c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 0x7f6ccc153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40516/0 (socket says 172.21.15.60:40516) 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd0b7c640 1 -- 172.21.15.60:0/1276673190 learned_addr learned my addr 172.21.15.60:0/1276673190 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.213+0000 7f6cd137d640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6ccbfff640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ccc1610f0 0x7f6ccc1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd0b7c640 1 -- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ccc1610f0 msgr2=0x7f6ccc1584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd0b7c640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ccc1610f0 0x7f6ccc1584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd0b7c640 1 -- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ccc1616f0 msgr2=0x7f6ccc158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd0b7c640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd0b7c640 1 -- 172.21.15.60:0/1276673190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc157be0 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd0b7c640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 0x7f6ccc153e90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c790 tx=0x7f6cc400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4016020 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc4006d50 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc158160 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4003bb0 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.214+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc15a3f0 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.215+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc073a00 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.215+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cc400a850 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.216+0000 7f6cc97fa640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6cb806b4d0 0x7f6cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.216+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6cc4089fb0 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.216+0000 7f6ccbfff640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6cb806b4d0 0x7f6cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.216+0000 7f6ccbfff640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6cb806b4d0 0x7f6cb806d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0034000 tx=0x7f6cc002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.220+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cc4020020 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.400+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ccc0cf150 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.400+0000 7f6cc97fa640 1 -- 172.21.15.60:0/1276673190 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6cc4058d10 con 0x7f6ccc0ce0a0 2024-04-25T08:35:28.395 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:35:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.402+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6cb806b4d0 msgr2=0x7f6cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.402+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6cb806b4d0 0x7f6cb806d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0034000 tx=0x7f6cc002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.402+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 msgr2=0x7f6ccc153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.402+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 0x7f6ccc153e90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c790 tx=0x7f6cc400cc50 comp rx=0 tx=0).stop 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 shutdown_connections 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6cb806b4d0 0x7f6cb806d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ccc1616f0 0x7f6ccc158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ccc1610f0 0x7f6ccc1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 --2- 172.21.15.60:0/1276673190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ccc0ce0a0 0x7f6ccc153e90 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 >> 172.21.15.60:0/1276673190 conn(0x7f6ccc06f500 msgr2=0x7f6ccc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 shutdown_connections 2024-04-25T08:35:28.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.403+0000 7f6cd2de0640 1 -- 172.21.15.60:0/1276673190 wait complete. 2024-04-25T08:35:28.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:35:28.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.517+0000 7fb3364ac640 1 Processor -- start 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.517+0000 7fb3364ac640 1 -- start start 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.517+0000 7fb3364ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb33015dac0 0x7fb3300d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.517+0000 7fb3364ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300d2420 con 0x7fb3300d0ab0 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.517+0000 7fb3364ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300d2700 con 0x7fb3300d1f30 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.517+0000 7fb3364ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300d3030 con 0x7fb33015dac0 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb334a49640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb3300d0ab0 0x7fb3300b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54438/0 (socket says 172.21.15.60:54438) 2024-04-25T08:35:28.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb334a49640 1 -- 172.21.15.60:0/3305621629 learned_addr learned my addr 172.21.15.60:0/3305621629 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32ffff640 1 --2- 172.21.15.60:0/3305621629 >> v2:172.21.15.100:3300/0 conn(0x7fb33015dac0 0x7fb3300d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135972554 0 0) 0x7fb3300d2420 con 0x7fb3300d0ab0 2024-04-25T08:35:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31c003520 con 0x7fb3300d0ab0 2024-04-25T08:35:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1722450326 0 0) 0x7fb31c003520 con 0x7fb3300d0ab0 2024-04-25T08:35:28.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 >> v2:172.21.15.100:3300/0 conn(0x7fb33015dac0 msgr2=0x7fb3300d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 --2- 172.21.15.60:0/3305621629 >> v2:172.21.15.100:3300/0 conn(0x7fb33015dac0 0x7fb3300d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 >> v1:172.21.15.100:6789/0 conn(0x7fb3300d1f30 legacy=0x7fb3300d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300d29e0 con 0x7fb3300d0ab0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb3280019f0 con 0x7fb3300d0ab0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 >> v1:172.21.15.60:6789/0 conn(0x7fb3300d0ab0 legacy=0x7fb3300b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0039b0 0x7fb31c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb31c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c0051c0 0x7fb31c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300d3030 con 0x7fb31c004800 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c00c070 con 0x7fb31c0051c0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c00c270 con 0x7fb31c0039b0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb31c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb334a49640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c0051c0 0x7fb31c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.518+0000 7fb32ffff640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0039b0 0x7fb31c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb32f7fe640 1 -- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0039b0 msgr2=0x7fb31c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb31c0039b0 0x7fb31c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb32f7fe640 1 -- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c0051c0 msgr2=0x7fb31c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb31c0051c0 0x7fb31c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb334a49640 1 -- 172.21.15.60:0/3305621629 reap_dead start 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb32f7fe640 1 -- 172.21.15.60:0/3305621629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c003520 con 0x7fb31c004800 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.519+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb31c004c80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb320001770 tx=0x7fb31c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a778fbb951e4490 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200013c0 con 0x7fb31c004800 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3200052d0 con 0x7fb31c004800 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb32effd640 1 -- 172.21.15.60:0/3305621629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32002a8a0 con 0x7fb31c004800 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 -- 172.21.15.60:0/3305621629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 msgr2=0x7fb31c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb31c004c80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb320001770 tx=0x7fb31c005830 comp rx=0 tx=0).stop 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 -- 172.21.15.60:0/3305621629 shutdown_connections 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3305621629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb31c004c80 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 -- 172.21.15.60:0/3305621629 >> 172.21.15.60:0/3305621629 conn(0x7fb330061c30 msgr2=0x7fb33006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 -- 172.21.15.60:0/3305621629 shutdown_connections 2024-04-25T08:35:28.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.520+0000 7fb3364ac640 1 -- 172.21.15.60:0/3305621629 wait complete. 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.521+0000 7fb3364ac640 1 Processor -- start 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.521+0000 7fb3364ac640 1 -- start start 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb3364ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb330151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb3364ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33015f100 0x7fb33015f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb3364ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3300cb100 0x7fb3300cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb3364ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300b97d0 con 0x7fb31c004800 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb3364ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015dea0 con 0x7fb33015f100 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb3364ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300b9950 con 0x7fb3300cb100 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb32ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb330151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb32ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb330151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40526/0 (socket says 172.21.15.60:40526) 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb32ffff640 1 -- 172.21.15.60:0/3701740087 learned_addr learned my addr 172.21.15.60:0/3701740087 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:28.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33015f100 0x7fb33015f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.522+0000 7fb334a49640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3300cb100 0x7fb3300cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32ffff640 1 -- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3300cb100 msgr2=0x7fb3300cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32ffff640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3300cb100 0x7fb3300cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32ffff640 1 -- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33015f100 msgr2=0x7fb33015f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32ffff640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33015f100 0x7fb33015f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32ffff640 1 -- 172.21.15.60:0/3701740087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33015fcc0 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32ffff640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb330151560 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb318011aa0 tx=0x7fb318011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31800fb80 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31800fd20 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33015ffa0 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318014660 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.523+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330160560 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.524+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330073a00 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.524+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb318012040 con 0x7fb31c004800 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.525+0000 7fb32cff9640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb31006b4d0 0x7fb31006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.525+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb31808ab40 con 0x7fb31c004800 2024-04-25T08:35:28.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.525+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb31006b4d0 0x7fb31006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.526+0000 7fb32f7fe640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb31006b4d0 0x7fb31006d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb320032000 tx=0x7fb32002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.529+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb318019030 con 0x7fb31c004800 2024-04-25T08:35:28.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.707+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3300b36b0 con 0x7fb31c004800 2024-04-25T08:35:28.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.708+0000 7fb32cff9640 1 -- 172.21.15.60:0/3701740087 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb318059810 con 0x7fb31c004800 2024-04-25T08:35:28.703 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.709+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb31006b4d0 msgr2=0x7fb31006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.709+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb31006b4d0 0x7fb31006d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb320032000 tx=0x7fb32002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 msgr2=0x7fb330151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb330151560 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb318011aa0 tx=0x7fb318011f60 comp rx=0 tx=0).stop 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 shutdown_connections 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3300cb100 0x7fb3300cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb31006b4d0 0x7fb31006d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb33015f100 0x7fb33015f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 --2- 172.21.15.60:0/3701740087 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb31c004800 0x7fb330151560 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 >> 172.21.15.60:0/3701740087 conn(0x7fb330061c30 msgr2=0x7fb33006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 shutdown_connections 2024-04-25T08:35:28.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.710+0000 7fb3364ac640 1 -- 172.21.15.60:0/3701740087 wait complete. 2024-04-25T08:35:28.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:28.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.820+0000 7fc9b15e0640 1 Processor -- start 2024-04-25T08:35:28.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.820+0000 7fc9b15e0640 1 -- start start 2024-04-25T08:35:28.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9b15e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9b15e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc9ac15c5b0 0x7fc9ac15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9b15e0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac06f9c0 con 0x7fc9ac0ad070 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9b15e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0af8a0 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9b15e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac160370 con 0x7fc9ac15c1d0 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9aaffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9aa7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc9ac15c5b0 0x7fc9ac15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.821+0000 7fc9aaffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:58044/0 (socket says 172.21.15.60:58044) 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.822+0000 7fc9aaffd640 1 -- 172.21.15.60:0/271306402 learned_addr learned my addr 172.21.15.60:0/271306402 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.822+0000 7fc9aa7fc640 1 -- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 msgr2=0x7fc9ac0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.822+0000 7fc9aa7fc640 1 --2- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.822+0000 7fc9aa7fc640 1 -- 172.21.15.60:0/271306402 >> v1:172.21.15.100:6789/0 conn(0x7fc9ac0ad070 legacy=0x7fc9ac152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.822+0000 7fc9aa7fc640 1 -- 172.21.15.60:0/271306402 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac15f5b0 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9aa7fc640 1 --2- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ca10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc9940097a0 tx=0x7fc99402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09c196fce272142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9a9ffb640 1 -- 172.21.15.60:0/271306402 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994037070 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9a9ffb640 1 -- 172.21.15.60:0/271306402 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994032440 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9a9ffb640 1 -- 172.21.15.60:0/271306402 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994032810 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 -- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac15c5b0 msgr2=0x7fc9ac15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ca10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc9940097a0 tx=0x7fc99402a180 comp rx=0 tx=0).stop 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 -- 172.21.15.60:0/271306402 shutdown_connections 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ca10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/271306402 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 -- 172.21.15.60:0/271306402 >> 172.21.15.60:0/271306402 conn(0x7fc9ac0cfe10 msgr2=0x7fc9ac0d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 -- 172.21.15.60:0/271306402 shutdown_connections 2024-04-25T08:35:28.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.823+0000 7fc9b15e0640 1 -- 172.21.15.60:0/271306402 wait complete. 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 Processor -- start 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 -- start start 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac0ad070 0x7fc9ac09acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac15fe50 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac15fb70 con 0x7fc9ac0ad070 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.825+0000 7fc9b15e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0ad4c0 con 0x7fc9ac15c1d0 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40530/0 (socket says 172.21.15.60:40530) 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/4204994286 learned_addr learned my addr 172.21.15.60:0/4204994286 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9aa7fc640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9aaffd640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac0ad070 0x7fc9ac09acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 msgr2=0x7fc9ac0b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac0ad070 msgr2=0x7fc9ac09acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac0ad070 0x7fc9ac09acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 -- 172.21.15.60:0/4204994286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac09b390 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9ab7fe640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ae30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00c7a0 tx=0x7fc99c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c003b70 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c003d30 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac09b670 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00aa90 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.826+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac09bc30 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.827+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac073990 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.828+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc99c00ac30 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.828+0000 7fc9a37fe640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc98c06b4d0 0x7fc98c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.828+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fc99c08a250 con 0x7fc9ac15c5b0 2024-04-25T08:35:28.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.829+0000 7fc9aaffd640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc98c06b4d0 0x7fc98c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.829+0000 7fc9aaffd640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc98c06b4d0 0x7fc98c06d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009bd0 tx=0x7fc9a4009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:28.833+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9ac06f9c0 con 0x7fc9ac15c5b0 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.049+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9ac061b70 con 0x7fc9ac15c5b0 2024-04-25T08:35:29.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.049+0000 7fc9a37fe640 1 -- 172.21.15.60:0/4204994286 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc99c058830 con 0x7fc9ac15c5b0 2024-04-25T08:35:29.044 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:29.044 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:33:56.967441+0000","modified":"2024-04-25T08:34:08.095428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7203":{"gid":7203,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1892944367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1892944367},{"type":"v1","addr":"172.21.15.100:6837","nonce":1892944367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:35:29.045 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 49 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc98c06b4d0 msgr2=0x7fc98c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc98c06b4d0 0x7fc98c06d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009bd0 tx=0x7fc9a4009240 comp rx=0 tx=0).stop 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 msgr2=0x7fc9ac15ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ae30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00c7a0 tx=0x7fc99c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 shutdown_connections 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc9ac15c5b0 0x7fc9ac15ae30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc9ac15c1d0 0x7fc9ac0b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc98c06b4d0 0x7fc98c06d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 --2- 172.21.15.60:0/4204994286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9ac0ad070 0x7fc9ac09acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 >> 172.21.15.60:0/4204994286 conn(0x7fc9ac0cfe10 msgr2=0x7fc9ac15c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 shutdown_connections 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.051+0000 7fc9b15e0640 1 -- 172.21.15.60:0/4204994286 wait complete. 2024-04-25T08:35:29.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:29.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.163+0000 7f634096a640 1 Processor -- start 2024-04-25T08:35:29.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.163+0000 7f634096a640 1 -- start start 2024-04-25T08:35:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f634096a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f633c0b6cb0 0x7f633c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f634096a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f633c0b5a60 0x7f633c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f634096a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0ceee0 con 0x7f633c15db70 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f634096a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c0d1d10 con 0x7f633c0b6cb0 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f634096a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c0d1ff0 con 0x7f633c0b5a60 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f633c0b6cb0 0x7f633c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f633c0b6cb0 0x7f633c0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40546/0 (socket says 172.21.15.60:40546) 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 -- 172.21.15.60:0/2112259564 learned_addr learned my addr 172.21.15.60:0/2112259564 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f633ad9d640 1 --2- 172.21.15.60:0/2112259564 >> v2:172.21.15.100:3300/0 conn(0x7f633c0b5a60 0x7f633c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 -- 172.21.15.60:0/2112259564 >> v2:172.21.15.100:3300/0 conn(0x7f633c0b5a60 msgr2=0x7f633c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 --2- 172.21.15.60:0/2112259564 >> v2:172.21.15.100:3300/0 conn(0x7f633c0b5a60 0x7f633c0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 -- 172.21.15.60:0/2112259564 >> v1:172.21.15.60:6789/0 conn(0x7f633c15db70 legacy=0x7f633c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.164+0000 7f6339d9b640 1 -- 172.21.15.60:0/2112259564 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0d22d0 con 0x7f633c0b6cb0 2024-04-25T08:35:29.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.165+0000 7f6339d9b640 1 --2- 172.21.15.60:0/2112259564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c0b5520 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cfa10 tx=0x7f633002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac9739e74673d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.165+0000 7f633959a640 1 -- 172.21.15.60:0/2112259564 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330037070 con 0x7f633c0b6cb0 2024-04-25T08:35:29.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.165+0000 7f633959a640 1 -- 172.21.15.60:0/2112259564 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330032440 con 0x7f633c0b6cb0 2024-04-25T08:35:29.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.165+0000 7f633959a640 1 -- 172.21.15.60:0/2112259564 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330032810 con 0x7f633c0b6cb0 2024-04-25T08:35:29.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 -- 172.21.15.60:0/2112259564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 msgr2=0x7f633c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 --2- 172.21.15.60:0/2112259564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c0b5520 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cfa10 tx=0x7f633002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 -- 172.21.15.60:0/2112259564 shutdown_connections 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 --2- 172.21.15.60:0/2112259564 >> v2:172.21.15.100:3300/0 conn(0x7f633c0b5a60 0x7f633c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 --2- 172.21.15.60:0/2112259564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c0b5520 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 -- 172.21.15.60:0/2112259564 >> 172.21.15.60:0/2112259564 conn(0x7f633c0d0520 msgr2=0x7f633c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 -- 172.21.15.60:0/2112259564 shutdown_connections 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.166+0000 7f634096a640 1 -- 172.21.15.60:0/2112259564 wait complete. 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.167+0000 7f634096a640 1 Processor -- start 2024-04-25T08:35:29.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.167+0000 7f634096a640 1 -- start start 2024-04-25T08:35:29.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f634096a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f633c0b5a60 0x7f633c160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f634096a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f634096a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633c15db70 0x7f633c1f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f634096a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0cf1c0 con 0x7f633c0b6cb0 2024-04-25T08:35:29.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f634096a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0ce530 con 0x7f633c15db70 2024-04-25T08:35:29.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f634096a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0cf340 con 0x7f633c0b5a60 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c1f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40552/0 (socket says 172.21.15.60:40552) 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 -- 172.21.15.60:0/1894356475 learned_addr learned my addr 172.21.15.60:0/1894356475 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f633ad9d640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633c15db70 0x7f633c1f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f633a59c640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f633c0b5a60 0x7f633c160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 -- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f633c0b5a60 msgr2=0x7f633c160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f633c0b5a60 0x7f633c160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 -- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633c15db70 msgr2=0x7f633c1f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633c15db70 0x7f633c1f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f6339d9b640 1 -- 172.21.15.60:0/1894356475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0ce310 con 0x7f633c0b6cb0 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.168+0000 7f633ad9d640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633c15db70 0x7f633c1f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.169+0000 7f6339d9b640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c1f6460 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6330000c00 tx=0x7f63300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.169+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330041020 con 0x7f633c0b6cb0 2024-04-25T08:35:29.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.169+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300306c0 con 0x7f633c0b6cb0 2024-04-25T08:35:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.169+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c0cc630 con 0x7f633c0b6cb0 2024-04-25T08:35:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.169+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330037040 con 0x7f633c0b6cb0 2024-04-25T08:35:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.169+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c0ccbf0 con 0x7f633c0b6cb0 2024-04-25T08:35:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.170+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c072680 con 0x7f633c0b6cb0 2024-04-25T08:35:29.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.170+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6330044020 con 0x7f633c0b6cb0 2024-04-25T08:35:29.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.171+0000 7f632affd640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f631c06b4d0 0x7f631c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.171+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f63300aaaa0 con 0x7f633c0b6cb0 2024-04-25T08:35:29.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.171+0000 7f633a59c640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f631c06b4d0 0x7f631c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.172+0000 7f633a59c640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f631c06b4d0 0x7f631c06d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f632c007fd0 tx=0x7f632c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.175+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f633c072680 con 0x7f633c0b6cb0 2024-04-25T08:35:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.356+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633c0b7db0 con 0x7f633c0b6cb0 2024-04-25T08:35:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.358+0000 7f632affd640 1 -- 172.21.15.60:0/1894356475 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f633004b030 con 0x7f633c0b6cb0 2024-04-25T08:35:29.353 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:29.353 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:04.110291+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f631c06b4d0 msgr2=0x7f631c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f631c06b4d0 0x7f631c06d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f632c007fd0 tx=0x7f632c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 msgr2=0x7f633c1f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c1f6460 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6330000c00 tx=0x7f63300046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 shutdown_connections 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f631c06b4d0 0x7f631c06d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f633c15db70 0x7f633c1f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f633c0b6cb0 0x7f633c1f6460 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 --2- 172.21.15.60:0/1894356475 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f633c0b5a60 0x7f633c160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 >> 172.21.15.60:0/1894356475 conn(0x7f633c0d0520 msgr2=0x7f633c159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 shutdown_connections 2024-04-25T08:35:29.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.360+0000 7f634096a640 1 -- 172.21.15.60:0/1894356475 wait complete. 2024-04-25T08:35:29.365 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:35:29.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:29.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.471+0000 7f5e7f73b640 1 Processor -- start 2024-04-25T08:35:29.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.472+0000 7f5e7f73b640 1 -- start start 2024-04-25T08:35:29.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.472+0000 7f5e7f73b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e7815dd20 0x7f5e780b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.472+0000 7f5e7f73b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e100 0x7f5e780d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.472+0000 7f5e7f73b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7815eac0 con 0x7f5e7815e560 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.472+0000 7f5e7f73b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e780d1c00 con 0x7f5e7815dd20 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.472+0000 7f5e7f73b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7815eda0 con 0x7f5e7815e100 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e7815dd20 0x7f5e780b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e7815dd20 0x7f5e780b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40566/0 (socket says 172.21.15.60:40566) 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/2847505146 learned_addr learned my addr 172.21.15.60:0/2847505146 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7ccd6640 1 --2- 172.21.15.60:0/2847505146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e100 0x7f5e780d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/2847505146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e100 msgr2=0x7f5e780d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 --2- 172.21.15.60:0/2847505146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e100 0x7f5e780d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/2847505146 >> v1:172.21.15.60:6789/0 conn(0x7f5e7815e560 legacy=0x7f5e780d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.473+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/2847505146 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7815f080 con 0x7f5e7815dd20 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7d4d7640 1 --2- 172.21.15.60:0/2847505146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780b96d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009870 tx=0x7f5e7002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8ada2ca3c9cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e77fff640 1 -- 172.21.15.60:0/2847505146 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70037070 con 0x7f5e7815dd20 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e77fff640 1 -- 172.21.15.60:0/2847505146 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70032440 con 0x7f5e7815dd20 2024-04-25T08:35:29.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e77fff640 1 -- 172.21.15.60:0/2847505146 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70032810 con 0x7f5e7815dd20 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 -- 172.21.15.60:0/2847505146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 msgr2=0x7f5e780b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/2847505146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780b96d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009870 tx=0x7f5e7002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 -- 172.21.15.60:0/2847505146 shutdown_connections 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/2847505146 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e100 0x7f5e780d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/2847505146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780b96d0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 -- 172.21.15.60:0/2847505146 >> 172.21.15.60:0/2847505146 conn(0x7f5e78061a20 msgr2=0x7f5e78061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 -- 172.21.15.60:0/2847505146 shutdown_connections 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.474+0000 7f5e7f73b640 1 -- 172.21.15.60:0/2847505146 wait complete. 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.475+0000 7f5e7f73b640 1 Processor -- start 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 -- start start 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7815e100 0x7f5e780cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e560 0x7f5e780d2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780d3500 con 0x7f5e7815dd20 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780d3380 con 0x7f5e7815e100 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7f73b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e780d3680 con 0x7f5e7815e560 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7d4d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7d4d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40576/0 (socket says 172.21.15.60:40576) 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.476+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/3335157784 learned_addr learned my addr 172.21.15.60:0/3335157784 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e560 msgr2=0x7f5e780d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7dcd8640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e560 0x7f5e780d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7ccd6640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7815e100 0x7f5e780cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7d4d7640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e560 0x7f5e780d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7815e100 msgr2=0x7f5e780cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7d4d7640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7815e100 0x7f5e780cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7d4d7640 1 -- 172.21.15.60:0/3335157784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e781e9600 con 0x7f5e7815dd20 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7d4d7640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780c9fe0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5e70000c00 tx=0x7f5e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70041020 con 0x7f5e7815dd20 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e700306c0 con 0x7f5e7815dd20 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70037040 con 0x7f5e7815dd20 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e780cc6b0 con 0x7f5e7815dd20 2024-04-25T08:35:29.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.477+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e78148b90 con 0x7f5e7815dd20 2024-04-25T08:35:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.478+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e78073990 con 0x7f5e7815dd20 2024-04-25T08:35:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.479+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e70044020 con 0x7f5e7815dd20 2024-04-25T08:35:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.479+0000 7f5e75ffb640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6406b4d0 0x7f5e6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.479+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5e700aaab0 con 0x7f5e7815dd20 2024-04-25T08:35:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.479+0000 7f5e7ccd6640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6406b4d0 0x7f5e6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.480+0000 7f5e7ccd6640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6406b4d0 0x7f5e6406d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5e781f6e40 tx=0x7f5e6c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.483+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e7004b030 con 0x7f5e7815dd20 2024-04-25T08:35:29.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.703+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e78062050 con 0x7f5e7815dd20 2024-04-25T08:35:29.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.704+0000 7f5e75ffb640 1 -- 172.21.15.60:0/3335157784 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5e70079810 con 0x7f5e7815dd20 2024-04-25T08:35:29.699 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:29.699 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:33:56.967441+0000","modified":"2024-04-25T08:34:08.095428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7203":{"gid":7203,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1892944367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1892944367},{"type":"v1","addr":"172.21.15.100:6837","nonce":1892944367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:35:29.699 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 49 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6406b4d0 msgr2=0x7f5e6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6406b4d0 0x7f5e6406d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5e781f6e40 tx=0x7f5e6c008040 comp rx=0 tx=0).stop 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 msgr2=0x7f5e780c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780c9fe0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5e70000c00 tx=0x7f5e700046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 shutdown_connections 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e7815e560 0x7f5e780d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6406b4d0 0x7f5e6406d990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e7815e100 0x7f5e780cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 --2- 172.21.15.60:0/3335157784 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e7815dd20 0x7f5e780c9fe0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 >> 172.21.15.60:0/3335157784 conn(0x7f5e78061a20 msgr2=0x7f5e78160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 shutdown_connections 2024-04-25T08:35:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.706+0000 7f5e7f73b640 1 -- 172.21.15.60:0/3335157784 wait complete. 2024-04-25T08:35:29.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.817+0000 7f72d482a640 1 Processor -- start 2024-04-25T08:35:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.817+0000 7f72d482a640 1 -- start start 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.818+0000 7f72d482a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f72d015f5f0 0x7f72d015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.818+0000 7f72d482a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d00cd710 con 0x7f72d01602e0 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.818+0000 7f72d482a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d0099b70 con 0x7f72d015ba00 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.818+0000 7f72d482a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d015ff80 con 0x7f72d015f5f0 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.818+0000 7f72cdd9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f72d01602e0 0x7f72d00ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54460/0 (socket says 172.21.15.60:54460) 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.818+0000 7f72cdd9b640 1 -- 172.21.15.60:0/1214258796 learned_addr learned my addr 172.21.15.60:0/1214258796 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ced9d640 1 --2- 172.21.15.60:0/1214258796 >> v2:172.21.15.100:3300/0 conn(0x7f72d015f5f0 0x7f72d015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416393234 0 0) 0x7f72c4001110 con 0x7f72d01602e0 2024-04-25T08:35:29.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b8003520 con 0x7f72d01602e0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2643804580 0 0) 0x7f72b8003520 con 0x7f72d01602e0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 >> v2:172.21.15.100:3300/0 conn(0x7f72d015f5f0 msgr2=0x7f72d015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 --2- 172.21.15.60:0/1214258796 >> v2:172.21.15.100:3300/0 conn(0x7f72d015f5f0 0x7f72d015fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 >> v1:172.21.15.100:6789/0 conn(0x7f72d015ba00 legacy=0x7f72d009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d0099e50 con 0x7f72d01602e0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f72c4001840 con 0x7f72d01602e0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 >> v1:172.21.15.60:6789/0 conn(0x7f72d01602e0 legacy=0x7f72d00ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b8004800 0x7f72b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80052e0 0x7f72b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015ff80 con 0x7f72b80039b0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b800c1b0 con 0x7f72b80052e0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b800c3b0 con 0x7f72b8004800 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ce59c640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ced9d640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80052e0 0x7f72b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72cdd9b640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b8004800 0x7f72b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ce59c640 1 -- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b8004800 msgr2=0x7f72b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ce59c640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72b8004800 0x7f72b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ce59c640 1 -- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80052e0 msgr2=0x7f72b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ce59c640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80052e0 0x7f72b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ce59c640 1 -- 172.21.15.60:0/1214258796 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8003520 con 0x7f72b80039b0 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ced9d640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72b80052e0 0x7f72b800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.819+0000 7f72ced9d640 1 -- 172.21.15.60:0/1214258796 reap_dead start 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.820+0000 7f72ce59c640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72b8003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f72c0001e20 tx=0x7f72b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1957c1753b8a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.820+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0001a70 con 0x7f72b80039b0 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.820+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c002ae10 con 0x7f72b80039b0 2024-04-25T08:35:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.820+0000 7f72cd59a640 1 -- 172.21.15.60:0/1214258796 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0003bb0 con 0x7f72b80039b0 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 -- 172.21.15.60:0/1214258796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 msgr2=0x7f72b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72b8003dd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f72c0001e20 tx=0x7f72b8005830 comp rx=0 tx=0).stop 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 -- 172.21.15.60:0/1214258796 shutdown_connections 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 --2- 172.21.15.60:0/1214258796 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72b8003dd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 -- 172.21.15.60:0/1214258796 >> 172.21.15.60:0/1214258796 conn(0x7f72d00b3270 msgr2=0x7f72d00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 -- 172.21.15.60:0/1214258796 shutdown_connections 2024-04-25T08:35:29.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.821+0000 7f72d482a640 1 -- 172.21.15.60:0/1214258796 wait complete. 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.822+0000 7f72d482a640 1 Processor -- start 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.822+0000 7f72d482a640 1 -- start start 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72d482a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72d015e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72d482a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72d01e7a50 0x7f72d015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72d482a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72d015dae0 0x7f72d015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72d482a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015bde0 con 0x7f72b80039b0 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72d482a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d015fa40 con 0x7f72d01e7a50 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72d482a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d01e7fa0 con 0x7f72d015dae0 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72ce59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72d015e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72ce59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72d015e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40596/0 (socket says 172.21.15.60:40596) 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72ce59c640 1 -- 172.21.15.60:0/2675981932 learned_addr learned my addr 172.21.15.60:0/2675981932 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72ce59c640 1 -- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72d015dae0 msgr2=0x7f72d015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72ce59c640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72d015dae0 0x7f72d015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72ce59c640 1 -- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72d01e7a50 msgr2=0x7f72d015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.823+0000 7f72cdd9b640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72d01e7a50 0x7f72d015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72ce59c640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72d01e7a50 0x7f72d015e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72ce59c640 1 -- 172.21.15.60:0/2675981932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d01e60e0 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72cdd9b640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72d01e7a50 0x7f72d015e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72ce59c640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72d015e0d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f72c0000f80 tx=0x7f72c00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0037040 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c002ac10 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0001a70 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d01e63c0 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.824+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d01e5db0 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.825+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d0073990 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.825+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72c0044020 con 0x7f72b80039b0 2024-04-25T08:35:29.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.826+0000 7f72beffd640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.826+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f72c00aa160 con 0x7f72b80039b0 2024-04-25T08:35:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.826+0000 7f72cdd9b640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.826+0000 7f72cdd9b640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72b006b4d0 0x7f72b006d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f72c4000ff0 tx=0x7f72c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:29.830+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72c004b020 con 0x7f72b80039b0 2024-04-25T08:35:30.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.046+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72d00b40a0 con 0x7f72b80039b0 2024-04-25T08:35:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.046+0000 7f72beffd640 1 -- 172.21.15.60:0/2675981932 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f72c0078d90 con 0x7f72b80039b0 2024-04-25T08:35:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:30.041 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:33:56.967441+0000","modified":"2024-04-25T08:34:08.095428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7203":{"gid":7203,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/1892944367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1892944367},{"type":"v1","addr":"172.21.15.100:6837","nonce":1892944367}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:35:30.042 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 49 2024-04-25T08:35:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72b006b4d0 msgr2=0x7f72b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72b006b4d0 0x7f72b006d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f72c4000ff0 tx=0x7f72c4009040 comp rx=0 tx=0).stop 2024-04-25T08:35:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 msgr2=0x7f72d015e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72d015e0d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f72c0000f80 tx=0x7f72c00014a0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 shutdown_connections 2024-04-25T08:35:30.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72d015dae0 0x7f72d015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f72b006b4d0 0x7f72b006d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72d01e7a50 0x7f72d015e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 --2- 172.21.15.60:0/2675981932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72b80039b0 0x7f72d015e0d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 >> 172.21.15.60:0/2675981932 conn(0x7f72d00b3270 msgr2=0x7f72d0159870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 shutdown_connections 2024-04-25T08:35:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.048+0000 7f72d482a640 1 -- 172.21.15.60:0/2675981932 wait complete. 2024-04-25T08:35:30.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.160+0000 7f9cd2934640 1 Processor -- start 2024-04-25T08:35:30.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.160+0000 7f9cd2934640 1 -- start start 2024-04-25T08:35:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd2934640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd2934640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc1625d0 con 0x7f9ccc15db50 2024-04-25T08:35:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd2934640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0cc2b0 con 0x7f9ccc0cd070 2024-04-25T08:35:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd2934640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc15df30 con 0x7f9ccc0d0240 2024-04-25T08:35:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9ccbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ccc15db50 0x7f9ccc0c6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:54466/0 (socket says 172.21.15.60:54466) 2024-04-25T08:35:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9ccbfff640 1 -- 172.21.15.60:0/3639358376 learned_addr learned my addr 172.21.15.60:0/3639358376 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd0ed1640 1 --2- 172.21.15.60:0/3639358376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9ccaffd640 1 -- 172.21.15.60:0/3639358376 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848027862 0 0) 0x7f9ccc1625d0 con 0x7f9ccc15db50 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9ccaffd640 1 -- 172.21.15.60:0/3639358376 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb4003520 con 0x7f9ccc15db50 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/3639358376 >> v1:172.21.15.100:6789/0 conn(0x7f9ccc0cd070 legacy=0x7f9ccc0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/3639358376 >> v1:172.21.15.60:6789/0 conn(0x7f9ccc15db50 legacy=0x7f9ccc0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.161+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/3639358376 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0cc590 con 0x7f9ccc0d0240 2024-04-25T08:35:30.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.162+0000 7f9cd0ed1640 1 --2- 172.21.15.60:0/3639358376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc0cd4d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009dc0 tx=0x7f9cc402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3086006a98f5a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.163+0000 7f9ccaffd640 1 -- 172.21.15.60:0/3639358376 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc402ae60 con 0x7f9ccc0d0240 2024-04-25T08:35:30.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.163+0000 7f9ccaffd640 1 -- 172.21.15.60:0/3639358376 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc4007740 con 0x7f9ccc0d0240 2024-04-25T08:35:30.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.163+0000 7f9ccaffd640 1 -- 172.21.15.60:0/3639358376 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4007b10 con 0x7f9ccc0d0240 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 -- 172.21.15.60:0/3639358376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 msgr2=0x7f9ccc0cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 --2- 172.21.15.60:0/3639358376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc0cd4d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009dc0 tx=0x7f9cc402a640 comp rx=0 tx=0).stop 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 -- 172.21.15.60:0/3639358376 shutdown_connections 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 --2- 172.21.15.60:0/3639358376 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc0cd4d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 -- 172.21.15.60:0/3639358376 >> 172.21.15.60:0/3639358376 conn(0x7f9ccc0d1800 msgr2=0x7f9ccc0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 -- 172.21.15.60:0/3639358376 shutdown_connections 2024-04-25T08:35:30.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.164+0000 7f9cd2934640 1 -- 172.21.15.60:0/3639358376 wait complete. 2024-04-25T08:35:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.165+0000 7f9cd2934640 1 Processor -- start 2024-04-25T08:35:30.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.165+0000 7f9cd2934640 1 -- start start 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd2934640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ccc0cd070 0x7f9ccc1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd2934640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd2934640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 0x7f9ccc155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd2934640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0c7220 con 0x7f9ccc15db50 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd2934640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0ccb50 con 0x7f9ccc0cd070 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd2934640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0c73a0 con 0x7f9ccc0d0240 2024-04-25T08:35:30.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 0x7f9ccc155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 0x7f9ccc155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40600/0 (socket says 172.21.15.60:40600) 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/4180276749 learned_addr learned my addr 172.21.15.60:0/4180276749 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9ccbfff640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ccc0cd070 0x7f9ccc1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9ccb7fe640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 msgr2=0x7f9ccc160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ccc0cd070 msgr2=0x7f9ccc1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ccc0cd070 0x7f9ccc1506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.166+0000 7f9cd0ed1640 1 -- 172.21.15.60:0/4180276749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc15e3c0 con 0x7f9ccc15db50 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.167+0000 7f9cd0ed1640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 0x7f9ccc155e60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9cc402ac30 tx=0x7f9cc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.167+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4040070 con 0x7f9ccc15db50 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.167+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc402ae60 con 0x7f9ccc15db50 2024-04-25T08:35:30.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.167+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4031be0 con 0x7f9ccc15db50 2024-04-25T08:35:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.167+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc15e6a0 con 0x7f9ccc15db50 2024-04-25T08:35:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.167+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc15ec60 con 0x7f9ccc15db50 2024-04-25T08:35:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.168+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc073990 con 0x7f9ccc15db50 2024-04-25T08:35:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.168+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cc4039040 con 0x7f9ccc15db50 2024-04-25T08:35:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.169+0000 7f9cc8ff9640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9cb806b4d0 0x7f9cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.169+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f9cc40aba60 con 0x7f9ccc15db50 2024-04-25T08:35:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.169+0000 7f9ccbfff640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9cb806b4d0 0x7f9cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.170+0000 7f9ccbfff640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9cb806b4d0 0x7f9cb806d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0011e0 tx=0x7f9cbc007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.173+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9cc407a850 con 0x7f9ccc15db50 2024-04-25T08:35:30.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.354+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ccc0c8260 con 0x7f9ccc15db50 2024-04-25T08:35:30.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.356+0000 7f9cc8ff9640 1 -- 172.21.15.60:0/4180276749 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f9cc407a150 con 0x7f9ccc15db50 2024-04-25T08:35:30.351 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:30.351 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":85,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:04.110291+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:33:55.215572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:33:55.910093+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9cb806b4d0 msgr2=0x7f9cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9cb806b4d0 0x7f9cb806d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0011e0 tx=0x7f9cbc007d30 comp rx=0 tx=0).stop 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 msgr2=0x7f9ccc155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 0x7f9ccc155e60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9cc402ac30 tx=0x7f9cc4004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 shutdown_connections 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ccc15db50 0x7f9ccc155e60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ccc0d0240 0x7f9ccc160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9cb806b4d0 0x7f9cb806d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 --2- 172.21.15.60:0/4180276749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ccc0cd070 0x7f9ccc1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.357+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 >> 172.21.15.60:0/4180276749 conn(0x7f9ccc0d1800 msgr2=0x7f9ccc0ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.358+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 shutdown_connections 2024-04-25T08:35:30.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.358+0000 7f9cd2934640 1 -- 172.21.15.60:0/4180276749 wait complete. 2024-04-25T08:35:30.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:35:30.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.470+0000 7fc52f822640 1 Processor -- start 2024-04-25T08:35:30.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.470+0000 7fc52f822640 1 -- start start 2024-04-25T08:35:30.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52f822640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc52815bb50 0x7fc5280c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52f822640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5280c70d0 0x7fc5280cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52f822640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280c75e0 con 0x7fc5280cfb40 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52f822640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280c8680 con 0x7fc52815bb50 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52f822640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280c78c0 con 0x7fc5280c70d0 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52d5be640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc52815bb50 0x7fc5280c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52d5be640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc52815bb50 0x7fc5280c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40610/0 (socket says 172.21.15.60:40610) 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.471+0000 7fc52d5be640 1 -- 172.21.15.60:0/1982132894 learned_addr learned my addr 172.21.15.60:0/1982132894 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.472+0000 7fc52cdbd640 1 --2- 172.21.15.60:0/1982132894 >> v2:172.21.15.100:3300/0 conn(0x7fc5280c70d0 0x7fc5280cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.472+0000 7fc52d5be640 1 -- 172.21.15.60:0/1982132894 >> v2:172.21.15.100:3300/0 conn(0x7fc5280c70d0 msgr2=0x7fc5280cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.472+0000 7fc52d5be640 1 --2- 172.21.15.60:0/1982132894 >> v2:172.21.15.100:3300/0 conn(0x7fc5280c70d0 0x7fc5280cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.472+0000 7fc52d5be640 1 -- 172.21.15.60:0/1982132894 >> v1:172.21.15.60:6789/0 conn(0x7fc5280cfb40 legacy=0x7fc528070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.472+0000 7fc52d5be640 1 -- 172.21.15.60:0/1982132894 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280c7ba0 con 0x7fc52815bb50 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.472+0000 7fc52d5be640 1 --2- 172.21.15.60:0/1982132894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc5280c6b90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc52806f220 tx=0x7fc52002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9edfaa5e6112813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc527fff640 1 -- 172.21.15.60:0/1982132894 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520037070 con 0x7fc52815bb50 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc527fff640 1 -- 172.21.15.60:0/1982132894 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52002ae50 con 0x7fc52815bb50 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc527fff640 1 -- 172.21.15.60:0/1982132894 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520032860 con 0x7fc52815bb50 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 -- 172.21.15.60:0/1982132894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 msgr2=0x7fc5280c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 --2- 172.21.15.60:0/1982132894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc5280c6b90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc52806f220 tx=0x7fc52002a300 comp rx=0 tx=0).stop 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 -- 172.21.15.60:0/1982132894 shutdown_connections 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 --2- 172.21.15.60:0/1982132894 >> v2:172.21.15.100:3300/0 conn(0x7fc5280c70d0 0x7fc5280cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 --2- 172.21.15.60:0/1982132894 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc5280c6b90 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 -- 172.21.15.60:0/1982132894 >> 172.21.15.60:0/1982132894 conn(0x7fc52806f500 msgr2=0x7fc52806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 -- 172.21.15.60:0/1982132894 shutdown_connections 2024-04-25T08:35:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.473+0000 7fc52f822640 1 -- 172.21.15.60:0/1982132894 wait complete. 2024-04-25T08:35:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.474+0000 7fc52f822640 1 Processor -- start 2024-04-25T08:35:30.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.474+0000 7fc52f822640 1 -- start start 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52f822640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5280c70d0 0x7fc5280c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52f822640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5280cfb40 0x7fc5280b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52f822640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc528146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52f822640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52815f6d0 con 0x7fc52815bb50 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52f822640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280c8160 con 0x7fc5280c70d0 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52f822640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52815f850 con 0x7fc5280cfb40 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52ddbf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc528146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52ddbf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc528146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40626/0 (socket says 172.21.15.60:40626) 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52ddbf640 1 -- 172.21.15.60:0/1420403088 learned_addr learned my addr 172.21.15.60:0/1420403088 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52d5be640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5280c70d0 0x7fc5280c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.475+0000 7fc52cdbd640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5280cfb40 0x7fc5280b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52ddbf640 1 -- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5280cfb40 msgr2=0x7fc5280b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52ddbf640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5280cfb40 0x7fc5280b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52ddbf640 1 -- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5280c70d0 msgr2=0x7fc5280c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52ddbf640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5280c70d0 0x7fc5280c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52ddbf640 1 -- 172.21.15.60:0/1420403088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280c8eb0 con 0x7fc52815bb50 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52d5be640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5280c70d0 0x7fc5280c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52ddbf640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc528146fe0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c710 tx=0x7fc51800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518016020 con 0x7fc52815bb50 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518006d50 con 0x7fc52815bb50 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5280c9190 con 0x7fc52815bb50 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518003bb0 con 0x7fc52815bb50 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.476+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5280c9750 con 0x7fc52815bb50 2024-04-25T08:35:30.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.477+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528073a00 con 0x7fc52815bb50 2024-04-25T08:35:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.478+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51800a850 con 0x7fc52815bb50 2024-04-25T08:35:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.478+0000 7fc525ffb640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc51406b4d0 0x7fc51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.478+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fc518089fb0 con 0x7fc52815bb50 2024-04-25T08:35:30.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.478+0000 7fc52d5be640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc51406b4d0 0x7fc51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.479+0000 7fc52d5be640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc51406b4d0 0x7fc51406d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc520034000 tx=0x7fc52002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.482+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc518020020 con 0x7fc52815bb50 2024-04-25T08:35:30.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.700+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc528070030 con 0x7fc52815bb50 2024-04-25T08:35:30.696 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:35:30.701+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:35:30.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.702+0000 7fc525ffb640 1 -- 172.21.15.60:0/1420403088 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc518019310 con 0x7fc52815bb50 2024-04-25T08:35:30.697 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:35:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc51406b4d0 msgr2=0x7fc51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc51406b4d0 0x7fc51406d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc520034000 tx=0x7fc52002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 msgr2=0x7fc528146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc528146fe0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c710 tx=0x7fc51800cbd0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 shutdown_connections 2024-04-25T08:35:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc52815bb50 0x7fc528146fe0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc5280cfb40 0x7fc5280b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc51406b4d0 0x7fc51406d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 --2- 172.21.15.60:0/1420403088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5280c70d0 0x7fc5280c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 >> 172.21.15.60:0/1420403088 conn(0x7fc52806f500 msgr2=0x7fc5280cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 shutdown_connections 2024-04-25T08:35:30.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.705+0000 7fc52f822640 1 -- 172.21.15.60:0/1420403088 wait complete. 2024-04-25T08:35:30.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:35:30.742 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:35:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.821+0000 7fb577f89640 1 Processor -- start 2024-04-25T08:35:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.822+0000 7fb577f89640 1 -- start start 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.822+0000 7fb577f89640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb57015b5a0 0x7fb5700b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.822+0000 7fb577f89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb57015b980 0x7fb5700b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.822+0000 7fb577f89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5700cf320 con 0x7fb570162450 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.822+0000 7fb577f89640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5700cf040 con 0x7fb57015b980 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.822+0000 7fb577f89640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb570161970 con 0x7fb57015b5a0 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb57015b980 0x7fb5700b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb57015b980 0x7fb5700b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40636/0 (socket says 172.21.15.60:40636) 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 -- 172.21.15.60:0/1550455248 learned_addr learned my addr 172.21.15.60:0/1550455248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575d25640 1 --2- 172.21.15.60:0/1550455248 >> v2:172.21.15.100:3300/0 conn(0x7fb57015b5a0 0x7fb5700b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 -- 172.21.15.60:0/1550455248 >> v2:172.21.15.100:3300/0 conn(0x7fb57015b5a0 msgr2=0x7fb5700b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 --2- 172.21.15.60:0/1550455248 >> v2:172.21.15.100:3300/0 conn(0x7fb57015b5a0 0x7fb5700b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 -- 172.21.15.60:0/1550455248 >> v1:172.21.15.100:6789/0 conn(0x7fb570162450 legacy=0x7fb5700ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.823+0000 7fb575524640 1 -- 172.21.15.60:0/1550455248 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570161c50 con 0x7fb57015b980 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb575524640 1 --2- 172.21.15.60:0/1550455248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b5a70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb560009850 tx=0x7fb56002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c26cb053d536a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb574d23640 1 -- 172.21.15.60:0/1550455248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560004030 con 0x7fb57015b980 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb574d23640 1 -- 172.21.15.60:0/1550455248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5600041d0 con 0x7fb57015b980 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb574d23640 1 -- 172.21.15.60:0/1550455248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560003610 con 0x7fb57015b980 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 -- 172.21.15.60:0/1550455248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 msgr2=0x7fb5700b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 --2- 172.21.15.60:0/1550455248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b5a70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb560009850 tx=0x7fb56002a290 comp rx=0 tx=0).stop 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 -- 172.21.15.60:0/1550455248 shutdown_connections 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 --2- 172.21.15.60:0/1550455248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b5a70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 --2- 172.21.15.60:0/1550455248 >> v2:172.21.15.100:3300/0 conn(0x7fb57015b5a0 0x7fb5700b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 -- 172.21.15.60:0/1550455248 >> 172.21.15.60:0/1550455248 conn(0x7fb5700d0530 msgr2=0x7fb5700d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 -- 172.21.15.60:0/1550455248 shutdown_connections 2024-04-25T08:35:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.824+0000 7fb577f89640 1 -- 172.21.15.60:0/1550455248 wait complete. 2024-04-25T08:35:30.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 Processor -- start 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 -- start start 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb57015b5a0 0x7fb5700c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb570162450 0x7fb570158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700ceba0 con 0x7fb57015b980 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700cea20 con 0x7fb57015b5a0 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb577f89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700ced20 con 0x7fb570162450 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb575524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb575524640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40652/0 (socket says 172.21.15.60:40652) 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.826+0000 7fb575524640 1 -- 172.21.15.60:0/1412973213 learned_addr learned my addr 172.21.15.60:0/1412973213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575d25640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb57015b5a0 0x7fb5700c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb576526640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb570162450 0x7fb570158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575524640 1 -- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb570162450 msgr2=0x7fb570158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575524640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb570162450 0x7fb570158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575524640 1 -- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb57015b5a0 msgr2=0x7fb5700c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575524640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb57015b5a0 0x7fb5700c8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575524640 1 -- 172.21.15.60:0/1412973213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5700c9630 con 0x7fb57015b980 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb575524640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b8010 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb560000c00 tx=0x7fb5600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560041020 con 0x7fb57015b980 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb560003c50 con 0x7fb57015b980 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5700c7680 con 0x7fb57015b980 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560004030 con 0x7fb57015b980 2024-04-25T08:35:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.827+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5700c9910 con 0x7fb57015b980 2024-04-25T08:35:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.828+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570073a00 con 0x7fb57015b980 2024-04-25T08:35:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.829+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb560044020 con 0x7fb57015b980 2024-04-25T08:35:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.829+0000 7fb5667fc640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.829+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb5600aad30 con 0x7fb57015b980 2024-04-25T08:35:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.829+0000 7fb575d25640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.830+0000 7fb575d25640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb55406b4d0 0x7fb55406d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009aa0 tx=0x7fb56c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:30.833+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb56004b030 con 0x7fb57015b980 2024-04-25T08:35:31.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.050+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb5700d2940 con 0x7fb57015b980 2024-04-25T08:35:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.704+0000 7fb5667fc640 1 -- 172.21.15.60:0/1412973213 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5600794a0 con 0x7fb57015b980 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb55406b4d0 msgr2=0x7fb55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb55406b4d0 0x7fb55406d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009aa0 tx=0x7fb56c009190 comp rx=0 tx=0).stop 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 msgr2=0x7fb5700b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b8010 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb560000c00 tx=0x7fb5600046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 shutdown_connections 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb570162450 0x7fb570158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb57015b980 0x7fb5700b8010 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 --2- 172.21.15.60:0/1412973213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb57015b5a0 0x7fb5700c8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 >> 172.21.15.60:0/1412973213 conn(0x7fb5700d0530 msgr2=0x7fb570157580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 shutdown_connections 2024-04-25T08:35:31.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.707+0000 7fb577f89640 1 -- 172.21.15.60:0/1412973213 wait complete. 2024-04-25T08:35:31.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:35:31.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.832+0000 7fb4a80ac640 1 Processor -- start 2024-04-25T08:35:31.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.832+0000 7fb4a80ac640 1 -- start start 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a80ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb4a015eec0 0x7fb4a01689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a80ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a00d1770 con 0x7fb4a006ccd0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a80ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a0162740 con 0x7fb4a0070dc0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a80ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a00b0eb0 con 0x7fb4a015eec0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a5647640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb4a006ccd0 0x7fb4a00b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58382/0 (socket says 172.21.15.60:58382) 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a5647640 1 -- 172.21.15.60:0/141002593 learned_addr learned my addr 172.21.15.60:0/141002593 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a6649640 1 --2- 172.21.15.60:0/141002593 >> v2:172.21.15.100:3300/0 conn(0x7fb4a015eec0 0x7fb4a01689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318705273 0 0) 0x7fb4a00d1770 con 0x7fb4a006ccd0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb48c003520 con 0x7fb4a006ccd0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453266384 0 0) 0x7fb4a0162740 con 0x7fb4a0070dc0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a00d1770 con 0x7fb4a0070dc0 2024-04-25T08:35:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1664845931 0 0) 0x7fb48c003520 con 0x7fb4a006ccd0 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 >> v2:172.21.15.100:3300/0 conn(0x7fb4a015eec0 msgr2=0x7fb4a01689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 --2- 172.21.15.60:0/141002593 >> v2:172.21.15.100:3300/0 conn(0x7fb4a015eec0 0x7fb4a01689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 >> v1:172.21.15.100:6789/0 conn(0x7fb4a0070dc0 legacy=0x7fb4a00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0161980 con 0x7fb4a006ccd0 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb490001670 con 0x7fb4a006ccd0 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.833+0000 7fb4a6649640 1 --2- 172.21.15.60:0/141002593 >> v2:172.21.15.100:3300/0 conn(0x7fb4a015eec0 0x7fb4a01689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 >> v1:172.21.15.60:6789/0 conn(0x7fb4a006ccd0 legacy=0x7fb4a00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c003b90 0x7fb48c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb48c004800 0x7fb48c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48c005280 0x7fb48c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b0eb0 con 0x7fb48c004800 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c00c1c0 con 0x7fb48c005280 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c00c3c0 con 0x7fb48c003b90 2024-04-25T08:35:31.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5647640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb48c004800 0x7fb48c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a6649640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48c005280 0x7fb48c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5e48640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c003b90 0x7fb48c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5647640 1 -- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c003b90 msgr2=0x7fb48c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5647640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c003b90 0x7fb48c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5647640 1 -- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48c005280 msgr2=0x7fb48c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5647640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb48c005280 0x7fb48c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a6649640 1 -- 172.21.15.60:0/141002593 reap_dead start 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.834+0000 7fb4a5647640 1 -- 172.21.15.60:0/141002593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c003520 con 0x7fb48c004800 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a5647640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb48c004800 0x7fb48c004c80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb490007fc0 tx=0x7fb48c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7461a6f89cad2cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490007be0 con 0x7fb48c004800 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb490005af0 con 0x7fb48c004800 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a4e46640 1 -- 172.21.15.60:0/141002593 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490007070 con 0x7fb48c004800 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a80ac640 1 -- 172.21.15.60:0/141002593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb48c004800 msgr2=0x7fb48c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb48c004800 0x7fb48c004c80 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb490007fc0 tx=0x7fb48c005830 comp rx=0 tx=0).stop 2024-04-25T08:35:31.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a80ac640 1 -- 172.21.15.60:0/141002593 shutdown_connections 2024-04-25T08:35:31.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/141002593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb48c004800 0x7fb48c004c80 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.835+0000 7fb4a80ac640 1 -- 172.21.15.60:0/141002593 >> 172.21.15.60:0/141002593 conn(0x7fb4a00b3530 msgr2=0x7fb4a00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:31.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.836+0000 7fb4a80ac640 1 -- 172.21.15.60:0/141002593 shutdown_connections 2024-04-25T08:35:31.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.836+0000 7fb4a80ac640 1 -- 172.21.15.60:0/141002593 wait complete. 2024-04-25T08:35:31.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.837+0000 7fb4a80ac640 1 Processor -- start 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.837+0000 7fb4a80ac640 1 -- start start 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.837+0000 7fb4a80ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c004800 0x7fb4a0152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.837+0000 7fb4a80ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 0x7fb4a00b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a80ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a00b2d40 0x7fb4a0156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a80ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b10f0 con 0x7fb4a0158f80 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a80ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01622b0 con 0x7fb4a00b2d40 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a80ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a0161f40 con 0x7fb48c004800 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 0x7fb4a00b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 0x7fb4a00b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33082/0 (socket says 172.21.15.60:33082) 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 -- 172.21.15.60:0/1034703042 learned_addr learned my addr 172.21.15.60:0/1034703042 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a6649640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a00b2d40 0x7fb4a0156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5e48640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c004800 0x7fb4a0152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 -- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c004800 msgr2=0x7fb4a0152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c004800 0x7fb4a0152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 -- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a00b2d40 msgr2=0x7fb4a0156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a00b2d40 0x7fb4a0156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 -- 172.21.15.60:0/1034703042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a01581c0 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a5647640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 0x7fb4a00b2800 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb49000c1a0 tx=0x7fb4a0161b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490037040 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb490041070 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a01584a0 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.838+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490001b20 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.839+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a01593e0 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.840+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a0073a00 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.840+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb490004a20 con 0x7fb4a0158f80 2024-04-25T08:35:31.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.840+0000 7fb4967fc640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb48406b4d0 0x7fb48406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.841+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb4900aa030 con 0x7fb4a0158f80 2024-04-25T08:35:31.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.841+0000 7fb4a5e48640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb48406b4d0 0x7fb48406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.841+0000 7fb4a5e48640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb48406b4d0 0x7fb48406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000c00 tx=0x7fb49c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:31.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:31.845+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb490078680 con 0x7fb4a0158f80 2024-04-25T08:35:32.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.024+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb4a00b23b0 con 0x7fb4a0158f80 2024-04-25T08:35:32.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.705+0000 7fb4967fc640 1 -- 172.21.15.60:0/1034703042 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb49007c720 con 0x7fb4a0158f80 2024-04-25T08:35:32.701 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:35:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb48406b4d0 msgr2=0x7fb48406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb48406b4d0 0x7fb48406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb49c000c00 tx=0x7fb49c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 msgr2=0x7fb4a00b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 0x7fb4a00b2800 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb49000c1a0 tx=0x7fb4a0161b50 comp rx=0 tx=0).stop 2024-04-25T08:35:32.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 shutdown_connections 2024-04-25T08:35:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb48406b4d0 0x7fb48406d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4a00b2d40 0x7fb4a0156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb4a0158f80 0x7fb4a00b2800 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 --2- 172.21.15.60:0/1034703042 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb48c004800 0x7fb4a0152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.709+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 >> 172.21.15.60:0/1034703042 conn(0x7fb4a00b3530 msgr2=0x7fb4a015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.710+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 shutdown_connections 2024-04-25T08:35:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.710+0000 7fb4a80ac640 1 -- 172.21.15.60:0/1034703042 wait complete. 2024-04-25T08:35:32.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:35:32.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.838+0000 7f0c5bd09640 1 Processor -- start 2024-04-25T08:35:32.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c5bd09640 1 -- start start 2024-04-25T08:35:32.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c5bd09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c540ced80 0x7f0c540d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c5bd09640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5415e000 con 0x7f0c5415dc20 2024-04-25T08:35:32.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c5bd09640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c540d1de0 con 0x7f0c54161710 2024-04-25T08:35:32.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c5bd09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54161c60 con 0x7f0c540ced80 2024-04-25T08:35:32.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c59aa5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0c5415dc20 0x7f0c540ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58398/0 (socket says 172.21.15.60:58398) 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.839+0000 7f0c59aa5640 1 -- 172.21.15.60:0/4022154873 learned_addr learned my addr 172.21.15.60:0/4022154873 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c592a4640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c540ced80 0x7f0c540d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215396818 0 0) 0x7f0c5415e000 con 0x7f0c5415dc20 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c3c003520 con 0x7f0c5415dc20 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844437582 0 0) 0x7f0c540d1de0 con 0x7f0c54161710 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5415e000 con 0x7f0c54161710 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066704147 0 0) 0x7f0c3c003520 con 0x7f0c5415dc20 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c540ced80 msgr2=0x7f0c540d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c540ced80 0x7f0c540d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 >> v1:172.21.15.100:6789/0 conn(0x7f0c54161710 legacy=0x7f0c540d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54161f40 con 0x7f0c5415dc20 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f0c4c001670 con 0x7f0c5415dc20 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 >> v1:172.21.15.60:6789/0 conn(0x7f0c5415dc20 legacy=0x7f0c540ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c003b70 0x7f0c3c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3c004800 0x7f0c3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c3c0051c0 0x7f0c3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54161c60 con 0x7f0c3c004800 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00bfe0 con 0x7f0c3c0051c0 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00c1e0 con 0x7f0c3c003b70 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c592a4640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c540ced80 0x7f0c540d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c592a4640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3c004800 0x7f0c3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c59aa5640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c003b70 0x7f0c3c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.840+0000 7f0c5a2a6640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c3c0051c0 0x7f0c3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c592a4640 1 -- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c003b70 msgr2=0x7f0c3c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c592a4640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c003b70 0x7f0c3c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c592a4640 1 -- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c3c0051c0 msgr2=0x7f0c3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:32.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c592a4640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c3c0051c0 0x7f0c3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c592a4640 1 -- 172.21.15.60:0/4022154873 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c003520 con 0x7f0c3c004800 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c592a4640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3c004800 0x7f0c3c004c80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0c540d3950 tx=0x7f0c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1341c5783b7f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48036070 con 0x7f0c3c004800 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c48004030 con 0x7f0c3c004800 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.841+0000 7f0c58aa3640 1 -- 172.21.15.60:0/4022154873 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c480097d0 con 0x7f0c3c004800 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5bd09640 1 -- 172.21.15.60:0/4022154873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3c004800 msgr2=0x7f0c3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/4022154873 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c3c004800 0x7f0c3c004c80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0c540d3950 tx=0x7f0c3c005830 comp rx=0 tx=0).stop 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5a2a6640 1 -- 172.21.15.60:0/4022154873 reap_dead start 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5bd09640 1 -- 172.21.15.60:0/4022154873 shutdown_connections 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5bd09640 1 -- 172.21.15.60:0/4022154873 >> 172.21.15.60:0/4022154873 conn(0x7f0c540d0420 msgr2=0x7f0c540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5bd09640 1 -- 172.21.15.60:0/4022154873 shutdown_connections 2024-04-25T08:35:32.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.842+0000 7f0c5bd09640 1 -- 172.21.15.60:0/4022154873 wait complete. 2024-04-25T08:35:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.843+0000 7f0c5bd09640 1 Processor -- start 2024-04-25T08:35:32.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.843+0000 7f0c5bd09640 1 -- start start 2024-04-25T08:35:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5bd09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c004800 0x7f0c5415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5bd09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 0x7f0c5415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5bd09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c541603f0 0x7f0c54160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5bd09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540b5e50 con 0x7f0c5415bbb0 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5bd09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54162500 con 0x7f0c541603f0 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5bd09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540b5fd0 con 0x7f0c3c004800 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c592a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 0x7f0c5415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c592a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 0x7f0c5415c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33098/0 (socket says 172.21.15.60:33098) 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c592a4640 1 -- 172.21.15.60:0/901576795 learned_addr learned my addr 172.21.15.60:0/901576795 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c59aa5640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c004800 0x7f0c5415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.844+0000 7f0c5a2a6640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c541603f0 0x7f0c54160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c592a4640 1 -- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c004800 msgr2=0x7f0c5415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c592a4640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c004800 0x7f0c5415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c592a4640 1 -- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c541603f0 msgr2=0x7f0c54160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c592a4640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c541603f0 0x7f0c54160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c592a4640 1 -- 172.21.15.60:0/901576795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c541f36a0 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c592a4640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 0x7f0c5415c030 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009bc0 tx=0x7f0c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48040020 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c48003c70 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c541f3960 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48036040 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.845+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c541f3f20 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.846+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c54073a00 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.846+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c48043020 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.847+0000 7f0c527fc640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c4006b4d0 0x7f0c4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.847+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f0c480aa830 con 0x7f0c5415bbb0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.847+0000 7f0c59aa5640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c4006b4d0 0x7f0c4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.848+0000 7f0c59aa5640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c4006b4d0 0x7f0c4006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0c54156da0 tx=0x7f0c4c000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:32.853+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c4804a020 con 0x7f0c5415bbb0 2024-04-25T08:35:33.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.033+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c540d16b0 con 0x7f0c5415bbb0 2024-04-25T08:35:33.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.707+0000 7f0c527fc640 1 -- 172.21.15.60:0/901576795 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0c48003e20 con 0x7f0c5415bbb0 2024-04-25T08:35:33.702 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:35:33.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.711+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c4006b4d0 msgr2=0x7f0c4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c4006b4d0 0x7f0c4006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0c54156da0 tx=0x7f0c4c000ed0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 msgr2=0x7f0c5415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 0x7f0c5415c030 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009bc0 tx=0x7f0c48004730 comp rx=0 tx=0).stop 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 shutdown_connections 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c4006b4d0 0x7f0c4006d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c541603f0 0x7f0c54160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c5415bbb0 0x7f0c5415c030 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 --2- 172.21.15.60:0/901576795 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c3c004800 0x7f0c5415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.712+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 >> 172.21.15.60:0/901576795 conn(0x7f0c540d0420 msgr2=0x7f0c540d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.713+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 shutdown_connections 2024-04-25T08:35:33.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.714+0000 7f0c5bd09640 1 -- 172.21.15.60:0/901576795 wait complete. 2024-04-25T08:35:33.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:35:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.840+0000 7f75f065d640 1 Processor -- start 2024-04-25T08:35:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.840+0000 7f75f065d640 1 -- start start 2024-04-25T08:35:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75f065d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f75e806c9e0 0x7f75e80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75f065d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e806cf50 con 0x7f75e80b5620 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75f065d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e80ce530 con 0x7f75e815b7d0 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75f065d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e80ce810 con 0x7f75e806c9e0 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75edbf8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f75e80b5620 0x7f75e806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58402/0 (socket says 172.21.15.60:58402) 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75edbf8640 1 -- 172.21.15.60:0/3241738638 learned_addr learned my addr 172.21.15.60:0/3241738638 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75eebfa640 1 --2- 172.21.15.60:0/3241738638 >> v2:172.21.15.100:3300/0 conn(0x7f75e806c9e0 0x7f75e80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052433368 0 0) 0x7f75e806cf50 con 0x7f75e80b5620 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.841+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d4003520 con 0x7f75e80b5620 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223804846 0 0) 0x7f75d4003520 con 0x7f75e80b5620 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 >> v2:172.21.15.100:3300/0 conn(0x7f75e806c9e0 msgr2=0x7f75e80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 --2- 172.21.15.60:0/3241738638 >> v2:172.21.15.100:3300/0 conn(0x7f75e806c9e0 0x7f75e80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 >> v1:172.21.15.100:6789/0 conn(0x7f75e815b7d0 legacy=0x7f75e80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80ceaf0 con 0x7f75e80b5620 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f75d8001670 con 0x7f75e80b5620 2024-04-25T08:35:33.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 >> v1:172.21.15.60:6789/0 conn(0x7f75e80b5620 legacy=0x7f75e806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d40039b0 0x7f75d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75d4004800 0x7f75d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d40051c0 0x7f75d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80ce810 con 0x7f75d4004800 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d400c060 con 0x7f75d40039b0 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d400c260 con 0x7f75d40051c0 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75edbf8640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75d4004800 0x7f75d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75ee3f9640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d40039b0 0x7f75d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75eebfa640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d40051c0 0x7f75d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75edbf8640 1 -- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d40051c0 msgr2=0x7f75d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75edbf8640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d40051c0 0x7f75d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75edbf8640 1 -- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d40039b0 msgr2=0x7f75d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75edbf8640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75d40039b0 0x7f75d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75edbf8640 1 -- 172.21.15.60:0/3241738638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4003520 con 0x7f75d4004800 2024-04-25T08:35:33.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.842+0000 7f75eebfa640 1 -- 172.21.15.60:0/3241738638 reap_dead start 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.843+0000 7f75edbf8640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75d4004800 0x7f75d4004c80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c240 tx=0x7f75d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e8b86035a21a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.843+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8007cf0 con 0x7f75d4004800 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.843+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d8005af0 con 0x7f75d4004800 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.843+0000 7f75ed3f7640 1 -- 172.21.15.60:0/3241738638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d8007070 con 0x7f75d4004800 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.843+0000 7f75f065d640 1 -- 172.21.15.60:0/3241738638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75d4004800 msgr2=0x7f75d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.843+0000 7f75f065d640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75d4004800 0x7f75d4004c80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c240 tx=0x7f75d4005830 comp rx=0 tx=0).stop 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.844+0000 7f75f065d640 1 -- 172.21.15.60:0/3241738638 shutdown_connections 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.844+0000 7f75f065d640 1 --2- 172.21.15.60:0/3241738638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75d4004800 0x7f75d4004c80 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.844+0000 7f75f065d640 1 -- 172.21.15.60:0/3241738638 >> 172.21.15.60:0/3241738638 conn(0x7f75e80d0800 msgr2=0x7f75e80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.844+0000 7f75f065d640 1 -- 172.21.15.60:0/3241738638 shutdown_connections 2024-04-25T08:35:33.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.844+0000 7f75f065d640 1 -- 172.21.15.60:0/3241738638 wait complete. 2024-04-25T08:35:33.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.845+0000 7f75f065d640 1 Processor -- start 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.845+0000 7f75f065d640 1 -- start start 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d4004800 0x7f75e81f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e815a030 0x7f75e81f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 0x7f75e80b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e815bd30 con 0x7f75e80b82b0 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e815bbb0 con 0x7f75e815a030 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80b5ab0 con 0x7f75d4004800 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 0x7f75e80b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 0x7f75e80b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33118/0 (socket says 172.21.15.60:33118) 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 -- 172.21.15.60:0/1559974304 learned_addr learned my addr 172.21.15.60:0/1559974304 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75edbf8640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e815a030 0x7f75e81f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75ee3f9640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d4004800 0x7f75e81f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 -- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d4004800 msgr2=0x7f75e81f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d4004800 0x7f75e81f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 -- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e815a030 msgr2=0x7f75e81f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e815a030 0x7f75e81f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 -- 172.21.15.60:0/1559974304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e81f46d0 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75eebfa640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 0x7f75e80b8730 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f75e806b4d0 tx=0x7f75e00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0007e80 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e001b070 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e80caf60 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011e40 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.846+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e80cb520 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.848+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e8073a00 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.848+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75e0005d10 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.848+0000 7f75deffd640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75cc06b500 0x7f75cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.848+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f75e008a1e0 con 0x7f75e80b82b0 2024-04-25T08:35:33.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.849+0000 7f75ee3f9640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75cc06b500 0x7f75cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.849+0000 7f75ee3f9640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75cc06b500 0x7f75cc06d9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75e4000c00 tx=0x7f75e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:33.853+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75e81f46d0 con 0x7f75e80b82b0 2024-04-25T08:35:34.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.041+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f75e815ed90 con 0x7f75e80b82b0 2024-04-25T08:35:34.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.044+0000 7f75deffd640 1 -- 172.21.15.60:0/1559974304 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75e0058c10 con 0x7f75e80b82b0 2024-04-25T08:35:34.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75cc06b500 msgr2=0x7f75cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75cc06b500 0x7f75cc06d9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f75e4000c00 tx=0x7f75e4009040 comp rx=0 tx=0).stop 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 msgr2=0x7f75e80b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 0x7f75e80b8730 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f75e806b4d0 tx=0x7f75e00115a0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 shutdown_connections 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75e80b82b0 0x7f75e80b8730 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75cc06b500 0x7f75cc06d9c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75e815a030 0x7f75e81f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 --2- 172.21.15.60:0/1559974304 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75d4004800 0x7f75e81f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 >> 172.21.15.60:0/1559974304 conn(0x7f75e80d0800 msgr2=0x7f75e80b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 shutdown_connections 2024-04-25T08:35:34.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.046+0000 7f75f065d640 1 -- 172.21.15.60:0/1559974304 wait complete. 2024-04-25T08:35:34.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T08:35:34.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.161+0000 7f17c1363640 1 Processor -- start 2024-04-25T08:35:34.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.161+0000 7f17c1363640 1 -- start start 2024-04-25T08:35:34.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17c1363640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17bc0b6cb0 0x7f17bc0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17c1363640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17bc0b5a60 0x7f17bc0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17c1363640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc0ceee0 con 0x7f17bc15db70 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17c1363640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc0d1d10 con 0x7f17bc0b6cb0 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17c1363640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc0d1ff0 con 0x7f17bc0b5a60 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17bc0b6cb0 0x7f17bc0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17bc0b6cb0 0x7f17bc0b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33130/0 (socket says 172.21.15.60:33130) 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 -- 172.21.15.60:0/2062207842 learned_addr learned my addr 172.21.15.60:0/2062207842 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17bb7fe640 1 --2- 172.21.15.60:0/2062207842 >> v2:172.21.15.100:3300/0 conn(0x7f17bc0b5a60 0x7f17bc0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 -- 172.21.15.60:0/2062207842 >> v2:172.21.15.100:3300/0 conn(0x7f17bc0b5a60 msgr2=0x7f17bc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 --2- 172.21.15.60:0/2062207842 >> v2:172.21.15.100:3300/0 conn(0x7f17bc0b5a60 0x7f17bc0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 -- 172.21.15.60:0/2062207842 >> v1:172.21.15.100:6789/0 conn(0x7f17bc15db70 legacy=0x7f17bc162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17ba7fc640 1 -- 172.21.15.60:0/2062207842 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0d22d0 con 0x7f17bc0b6cb0 2024-04-25T08:35:34.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.162+0000 7f17bb7fe640 1 --2- 172.21.15.60:0/2062207842 >> v2:172.21.15.100:3300/0 conn(0x7f17bc0b5a60 0x7f17bc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:34.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.163+0000 7f17ba7fc640 1 --2- 172.21.15.60:0/2062207842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc0b6cb0 0x7f17bc0b5520 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0cfa10 tx=0x7f17a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb2c8c7066d36df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.163+0000 7f17b9ffb640 1 -- 172.21.15.60:0/2062207842 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4037070 con 0x7f17bc0b6cb0 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.163+0000 7f17b9ffb640 1 -- 172.21.15.60:0/2062207842 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4032440 con 0x7f17bc0b6cb0 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.163+0000 7f17b9ffb640 1 -- 172.21.15.60:0/2062207842 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4032810 con 0x7f17bc0b6cb0 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 -- 172.21.15.60:0/2062207842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc0b6cb0 msgr2=0x7f17bc0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 --2- 172.21.15.60:0/2062207842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc0b6cb0 0x7f17bc0b5520 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0cfa10 tx=0x7f17a402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 -- 172.21.15.60:0/2062207842 shutdown_connections 2024-04-25T08:35:34.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 --2- 172.21.15.60:0/2062207842 >> v2:172.21.15.100:3300/0 conn(0x7f17bc0b5a60 0x7f17bc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 --2- 172.21.15.60:0/2062207842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc0b6cb0 0x7f17bc0b5520 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 -- 172.21.15.60:0/2062207842 >> 172.21.15.60:0/2062207842 conn(0x7f17bc0d0520 msgr2=0x7f17bc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 -- 172.21.15.60:0/2062207842 shutdown_connections 2024-04-25T08:35:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.164+0000 7f17c1363640 1 -- 172.21.15.60:0/2062207842 wait complete. 2024-04-25T08:35:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.165+0000 7f17c1363640 1 Processor -- start 2024-04-25T08:35:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.165+0000 7f17c1363640 1 -- start start 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17c1363640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17bc0b5a60 0x7f17bc0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17c1363640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17bc0b6cb0 0x7f17bc149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17c1363640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 0x7f17bc0cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17c1363640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0cf1c0 con 0x7f17bc15db70 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17c1363640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0ce530 con 0x7f17bc0b5a60 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17c1363640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0cf340 con 0x7f17bc0b6cb0 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 0x7f17bc0cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 0x7f17bc0cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33144/0 (socket says 172.21.15.60:33144) 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 -- 172.21.15.60:0/2907482939 learned_addr learned my addr 172.21.15.60:0/2907482939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17baffd640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17bc0b5a60 0x7f17bc0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17ba7fc640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17bc0b6cb0 0x7f17bc149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 -- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17bc0b6cb0 msgr2=0x7f17bc149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17bc0b6cb0 0x7f17bc149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 -- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17bc0b5a60 msgr2=0x7f17bc0c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17bc0b5a60 0x7f17bc0c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.166+0000 7f17bb7fe640 1 -- 172.21.15.60:0/2907482939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc149cd0 con 0x7f17bc15db70 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.167+0000 7f17bb7fe640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 0x7f17bc0cb8c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f17ac007f20 tx=0x7f17ac00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.167+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac01a070 con 0x7f17bc15db70 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.167+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac003b90 con 0x7f17bc15db70 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.167+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc146f70 con 0x7f17bc15db70 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.167+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac004030 con 0x7f17bc15db70 2024-04-25T08:35:34.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.167+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc147530 con 0x7f17bc15db70 2024-04-25T08:35:34.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.168+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc073990 con 0x7f17bc15db70 2024-04-25T08:35:34.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.168+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17ac01d070 con 0x7f17bc15db70 2024-04-25T08:35:34.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.169+0000 7f17b37fe640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179c06b4d0 0x7f179c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.169+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f17ac08a2a0 con 0x7f17bc15db70 2024-04-25T08:35:34.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.169+0000 7f17baffd640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179c06b4d0 0x7f179c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.169+0000 7f17baffd640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179c06b4d0 0x7f179c06d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f17b40016e0 tx=0x7f17b4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:34.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.173+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17ac010070 con 0x7f17bc15db70 2024-04-25T08:35:34.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.426+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f17bc068070 con 0x7f17bc15db70 2024-04-25T08:35:34.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.708+0000 7f17b37fe640 1 -- 172.21.15.60:0/2907482939 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v305) v1 ==== 158+0+0 (secure 0 0 0) 0x7f17ac08a0c0 con 0x7f17bc15db70 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179c06b4d0 msgr2=0x7f179c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179c06b4d0 0x7f179c06d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f17b40016e0 tx=0x7f17b4009040 comp rx=0 tx=0).stop 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 msgr2=0x7f17bc0cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 0x7f17bc0cb8c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f17ac007f20 tx=0x7f17ac00d560 comp rx=0 tx=0).stop 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 shutdown_connections 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17bc15db70 0x7f17bc0cb8c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17bc0b6cb0 0x7f17bc149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179c06b4d0 0x7f179c06d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 --2- 172.21.15.60:0/2907482939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17bc0b5a60 0x7f17bc0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 >> 172.21.15.60:0/2907482939 conn(0x7f17bc0d0520 msgr2=0x7f17bc0b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 shutdown_connections 2024-04-25T08:35:34.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.711+0000 7f17c1363640 1 -- 172.21.15.60:0/2907482939 wait complete. 2024-04-25T08:35:34.717 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T08:35:34.717 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:35:34.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:34.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.824+0000 7f852890a640 1 Processor -- start 2024-04-25T08:35:34.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.824+0000 7f852890a640 1 -- start start 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852890a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f852415db70 0x7f8524162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852890a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f85240b6cb0 0x7f85240b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852890a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b5a60 0x7f852406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852890a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8524162690 con 0x7f852415db70 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852890a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85240ceee0 con 0x7f85240b6cb0 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852890a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85240d1d10 con 0x7f85240b5a60 2024-04-25T08:35:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f852415db70 0x7f8524162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f852415db70 0x7f8524162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33150/0 (socket says 172.21.15.60:33150) 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 -- 172.21.15.60:0/4138632094 learned_addr learned my addr 172.21.15.60:0/4138632094 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f8521d9b640 1 --2- 172.21.15.60:0/4138632094 >> v2:172.21.15.100:3300/0 conn(0x7f85240b6cb0 0x7f85240b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f8522d9d640 1 --2- 172.21.15.60:0/4138632094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b5a60 0x7f852406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 -- 172.21.15.60:0/4138632094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b5a60 msgr2=0x7f852406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 --2- 172.21.15.60:0/4138632094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b5a60 0x7f852406c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 -- 172.21.15.60:0/4138632094 >> v2:172.21.15.100:3300/0 conn(0x7f85240b6cb0 msgr2=0x7f85240b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 --2- 172.21.15.60:0/4138632094 >> v2:172.21.15.100:3300/0 conn(0x7f85240b6cb0 0x7f85240b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f852259c640 1 -- 172.21.15.60:0/4138632094 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240d1ff0 con 0x7f852415db70 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.825+0000 7f8521d9b640 1 --2- 172.21.15.60:0/4138632094 >> v2:172.21.15.100:3300/0 conn(0x7f85240b6cb0 0x7f85240b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.826+0000 7f852259c640 1 --2- 172.21.15.60:0/4138632094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524162150 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f852408c600 tx=0x7f851402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef99e93d04f4cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.826+0000 7f852159a640 1 -- 172.21.15.60:0/4138632094 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514037070 con 0x7f852415db70 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.826+0000 7f852159a640 1 -- 172.21.15.60:0/4138632094 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8514032440 con 0x7f852415db70 2024-04-25T08:35:34.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.826+0000 7f852159a640 1 -- 172.21.15.60:0/4138632094 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514032810 con 0x7f852415db70 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 -- 172.21.15.60:0/4138632094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 msgr2=0x7f8524162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 --2- 172.21.15.60:0/4138632094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524162150 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f852408c600 tx=0x7f851402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 -- 172.21.15.60:0/4138632094 shutdown_connections 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 --2- 172.21.15.60:0/4138632094 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b5a60 0x7f852406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 --2- 172.21.15.60:0/4138632094 >> v2:172.21.15.100:3300/0 conn(0x7f85240b6cb0 0x7f85240b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 --2- 172.21.15.60:0/4138632094 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524162150 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 -- 172.21.15.60:0/4138632094 >> 172.21.15.60:0/4138632094 conn(0x7f85240d0520 msgr2=0x7f85240d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 -- 172.21.15.60:0/4138632094 shutdown_connections 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.827+0000 7f852890a640 1 -- 172.21.15.60:0/4138632094 wait complete. 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.828+0000 7f852890a640 1 Processor -- start 2024-04-25T08:35:34.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.828+0000 7f852890a640 1 -- start start 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852890a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85240b5a60 0x7f85241f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852890a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b6cb0 0x7f85240c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852890a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852890a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85240ce530 con 0x7f852415db70 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852890a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852415df50 con 0x7f85240b5a60 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852890a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85240cf1c0 con 0x7f85240b6cb0 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33158/0 (socket says 172.21.15.60:33158) 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 -- 172.21.15.60:0/1741939429 learned_addr learned my addr 172.21.15.60:0/1741939429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:34.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f852259c640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85240b5a60 0x7f85241f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8521d9b640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b6cb0 0x7f85240c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 -- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b6cb0 msgr2=0x7f85240c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b6cb0 0x7f85240c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 -- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85240b5a60 msgr2=0x7f85241f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85240b5a60 0x7f85241f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.829+0000 7f8522d9d640 1 -- 172.21.15.60:0/1741939429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85240cb680 con 0x7f852415db70 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.830+0000 7f8522d9d640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524143ba0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f851c00cd20 tx=0x7f851c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.830+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c003b90 con 0x7f852415db70 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.830+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c003d30 con 0x7f852415db70 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.830+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c004240 con 0x7f852415db70 2024-04-25T08:35:34.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.830+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8524144510 con 0x7f852415db70 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.830+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85241f6e60 con 0x7f852415db70 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.831+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8524073990 con 0x7f852415db70 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.831+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f851c0043e0 con 0x7f852415db70 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.832+0000 7f8512ffd640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f850406b4d0 0x7f850406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.832+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f851c08a960 con 0x7f852415db70 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.832+0000 7f852259c640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f850406b4d0 0x7f850406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.832+0000 7f852259c640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f850406b4d0 0x7f850406d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8514034000 tx=0x7f851402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:34.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:34.836+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f851c059390 con 0x7f852415db70 2024-04-25T08:35:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.017+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8524158d50 con 0x7f852415db70 2024-04-25T08:35:35.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.018+0000 7f8512ffd640 1 -- 172.21.15.60:0/1741939429 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13284 (secure 0 0 0) 0x7f851c05d430 con 0x7f852415db70 2024-04-25T08:35:35.013 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:35.013 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":88,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:33.706920+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6837/1892944367":"2024-04-26T08:35:30.702237+0000","172.21.15.100:6835/1892944367":"2024-04-26T08:35:30.702237+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:35.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f850406b4d0 msgr2=0x7f850406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f850406b4d0 0x7f850406d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8514034000 tx=0x7f851402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 msgr2=0x7f8524143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524143ba0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f851c00cd20 tx=0x7f851c00a660 comp rx=0 tx=0).stop 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 shutdown_connections 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f852415db70 0x7f8524143ba0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85240b6cb0 0x7f85240c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f850406b4d0 0x7f850406d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 --2- 172.21.15.60:0/1741939429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85240b5a60 0x7f85241f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 >> 172.21.15.60:0/1741939429 conn(0x7f85240d0520 msgr2=0x7f85240b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 shutdown_connections 2024-04-25T08:35:35.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.020+0000 7f852890a640 1 -- 172.21.15.60:0/1741939429 wait complete. 2024-04-25T08:35:35.026 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:35.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.135+0000 7f78e4ecb640 1 Processor -- start 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.135+0000 7f78e4ecb640 1 -- start start 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78e4ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c330 0x7f78e00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78e4ecb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78e006c7e0 0x7f78e00b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78e4ecb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e00d1d50 con 0x7f78e0158230 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78e4ecb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e0158610 con 0x7f78e006c7e0 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78e4ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006bea0 con 0x7f78e006c330 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78ded9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78e006c7e0 0x7f78e00b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78ded9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78e006c7e0 0x7f78e00b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33172/0 (socket says 172.21.15.60:33172) 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78ded9d640 1 -- 172.21.15.60:0/1873051960 learned_addr learned my addr 172.21.15.60:0/1873051960 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:35.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.136+0000 7f78ddd9b640 1 --2- 172.21.15.60:0/1873051960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c330 0x7f78e00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78ded9d640 1 -- 172.21.15.60:0/1873051960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c330 msgr2=0x7f78e00b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78ded9d640 1 --2- 172.21.15.60:0/1873051960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c330 0x7f78e00b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78ded9d640 1 -- 172.21.15.60:0/1873051960 >> v1:172.21.15.100:6789/0 conn(0x7f78e0158230 legacy=0x7f78e00d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78ded9d640 1 -- 172.21.15.60:0/1873051960 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e00d2020 con 0x7f78e006c7e0 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78ded9d640 1 --2- 172.21.15.60:0/1873051960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c7e0 0x7f78e00b53a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009e50 tx=0x7f78d002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242999bdbdb311ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78dd59a640 1 -- 172.21.15.60:0/1873051960 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0036070 con 0x7f78e006c7e0 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78dd59a640 1 -- 172.21.15.60:0/1873051960 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d0031600 con 0x7f78e006c7e0 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.137+0000 7f78dd59a640 1 -- 172.21.15.60:0/1873051960 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00319d0 con 0x7f78e006c7e0 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 -- 172.21.15.60:0/1873051960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c7e0 msgr2=0x7f78e00b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/1873051960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c7e0 0x7f78e00b53a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009e50 tx=0x7f78d002a710 comp rx=0 tx=0).stop 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 -- 172.21.15.60:0/1873051960 shutdown_connections 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/1873051960 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c7e0 0x7f78e00b53a0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/1873051960 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c330 0x7f78e00b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 -- 172.21.15.60:0/1873051960 >> 172.21.15.60:0/1873051960 conn(0x7f78e0061600 msgr2=0x7f78e00619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 -- 172.21.15.60:0/1873051960 shutdown_connections 2024-04-25T08:35:35.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.138+0000 7f78e4ecb640 1 -- 172.21.15.60:0/1873051960 wait complete. 2024-04-25T08:35:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.139+0000 7f78e4ecb640 1 Processor -- start 2024-04-25T08:35:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.139+0000 7f78e4ecb640 1 -- start start 2024-04-25T08:35:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78e4ecb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 0x7f78e00cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78e4ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c7e0 0x7f78e00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78e4ecb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78e0158230 0x7f78e014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78e4ecb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00d25e0 con 0x7f78e006c330 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78e4ecb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006bb30 con 0x7f78e0158230 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78e4ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e015e320 con 0x7f78e006c7e0 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78de59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 0x7f78e00cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78de59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 0x7f78e00cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33182/0 (socket says 172.21.15.60:33182) 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78de59c640 1 -- 172.21.15.60:0/981448394 learned_addr learned my addr 172.21.15.60:0/981448394 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78ded9d640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78e0158230 0x7f78e014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.140+0000 7f78ddd9b640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c7e0 0x7f78e00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78de59c640 1 -- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c7e0 msgr2=0x7f78e00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78de59c640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c7e0 0x7f78e00d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78de59c640 1 -- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78e0158230 msgr2=0x7f78e014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78de59c640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78e0158230 0x7f78e014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78de59c640 1 -- 172.21.15.60:0/981448394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e01e4510 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78de59c640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 0x7f78e00cd3b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f78d800c3f0 tx=0x7f78d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80052d0 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8005490 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e01e47f0 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8007260 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.141+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e00d0750 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.142+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e006de50 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.142+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78d8007400 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.143+0000 7f78d6ffd640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78c006b4d0 0x7f78c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.143+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f78d80894c0 con 0x7f78e006c330 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.143+0000 7f78ddd9b640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78c006b4d0 0x7f78c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.144+0000 7f78ddd9b640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78c006b4d0 0x7f78c006d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f78e01e5570 tx=0x7f78c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.147+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78d8057e60 con 0x7f78e006c330 2024-04-25T08:35:35.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.329+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78e0073a00 con 0x7f78e006c330 2024-04-25T08:35:35.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.329+0000 7f78d6ffd640 1 -- 172.21.15.60:0/981448394 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78d805ce20 con 0x7f78e006c330 2024-04-25T08:35:35.324 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78c006b4d0 msgr2=0x7f78c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78c006b4d0 0x7f78c006d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f78e01e5570 tx=0x7f78c8008040 comp rx=0 tx=0).stop 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 msgr2=0x7f78e00cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 0x7f78e00cd3b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f78d800c3f0 tx=0x7f78d800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 shutdown_connections 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78c006b4d0 0x7f78c006d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78e0158230 0x7f78e014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78e006c7e0 0x7f78e00d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 --2- 172.21.15.60:0/981448394 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78e006c330 0x7f78e00cd3b0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 >> 172.21.15.60:0/981448394 conn(0x7f78e0061600 msgr2=0x7f78e00ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 shutdown_connections 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.331+0000 7f78e4ecb640 1 -- 172.21.15.60:0/981448394 wait complete. 2024-04-25T08:35:35.337 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T08:35:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.443+0000 7ff09deac640 1 Processor -- start 2024-04-25T08:35:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.443+0000 7ff09deac640 1 -- start start 2024-04-25T08:35:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff09deac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff09815dbe0 0x7ff0980d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff09deac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0980b2cb0 0x7ff0980d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff09deac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0980b97a0 con 0x7ff0980b92a0 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff09deac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09815dfc0 con 0x7ff0980b2cb0 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff09deac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980d1a00 con 0x7ff09815dbe0 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0980b2cb0 0x7ff0980d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0980b2cb0 0x7ff0980d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33188/0 (socket says 172.21.15.60:33188) 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 -- 172.21.15.60:0/1606256947 learned_addr learned my addr 172.21.15.60:0/1606256947 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff0977fe640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff09815dbe0 0x7ff0980d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 -- 172.21.15.60:0/1606256947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff09815dbe0 msgr2=0x7ff0980d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff09815dbe0 0x7ff0980d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 -- 172.21.15.60:0/1606256947 >> v1:172.21.15.100:6789/0 conn(0x7ff0980b92a0 legacy=0x7ff0980b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff096ffd640 1 -- 172.21.15.60:0/1606256947 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980d1c50 con 0x7ff0980b2cb0 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.444+0000 7ff0977fe640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff09815dbe0 0x7ff0980d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.445+0000 7ff096ffd640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0980b2cb0 0x7ff0980d0e50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009870 tx=0x7ff08c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda42858d21d0b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.445+0000 7ff0967fc640 1 -- 172.21.15.60:0/1606256947 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c037070 con 0x7ff0980b2cb0 2024-04-25T08:35:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.445+0000 7ff0967fc640 1 -- 172.21.15.60:0/1606256947 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c032440 con 0x7ff0980b2cb0 2024-04-25T08:35:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.445+0000 7ff0967fc640 1 -- 172.21.15.60:0/1606256947 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c032810 con 0x7ff0980b2cb0 2024-04-25T08:35:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 -- 172.21.15.60:0/1606256947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0980b2cb0 msgr2=0x7ff0980d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0980b2cb0 0x7ff0980d0e50 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009870 tx=0x7ff08c02a270 comp rx=0 tx=0).stop 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 -- 172.21.15.60:0/1606256947 shutdown_connections 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0980b2cb0 0x7ff0980d0e50 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 --2- 172.21.15.60:0/1606256947 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff09815dbe0 0x7ff0980d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 -- 172.21.15.60:0/1606256947 >> 172.21.15.60:0/1606256947 conn(0x7ff098061a20 msgr2=0x7ff098061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 -- 172.21.15.60:0/1606256947 shutdown_connections 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.446+0000 7ff09deac640 1 -- 172.21.15.60:0/1606256947 wait complete. 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.447+0000 7ff09deac640 1 Processor -- start 2024-04-25T08:35:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.447+0000 7ff09deac640 1 -- start start 2024-04-25T08:35:35.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff09deac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0980b2cb0 0x7ff0981459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff09deac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0980b92a0 0x7ff098150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff09deac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 0x7ff098159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff09deac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980b6ab0 con 0x7ff09815dbe0 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff09deac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980d2210 con 0x7ff0980b2cb0 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff09deac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980b6c30 con 0x7ff0980b92a0 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 0x7ff098159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 0x7ff098159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33200/0 (socket says 172.21.15.60:33200) 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 -- 172.21.15.60:0/928550408 learned_addr learned my addr 172.21.15.60:0/928550408 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff0977fe640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0980b2cb0 0x7ff0981459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff096ffd640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0980b92a0 0x7ff098150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 -- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0980b92a0 msgr2=0x7ff098150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0980b92a0 0x7ff098150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 -- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0980b2cb0 msgr2=0x7ff0981459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.448+0000 7ff097fff640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0980b2cb0 0x7ff0981459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff097fff640 1 -- 172.21.15.60:0/928550408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098152370 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff097fff640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 0x7ff098159de0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff09000c760 tx=0x7ff09000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090003b70 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090003d10 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0981512d0 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09000aa90 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.449+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff098151890 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.450+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0980b6320 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.451+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff09000ac30 con 0x7ff09815dbe0 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.451+0000 7ff07ffff640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08406b4d0 0x7ff08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.451+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff090089e10 con 0x7ff09815dbe0 2024-04-25T08:35:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.451+0000 7ff0977fe640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08406b4d0 0x7ff08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.452+0000 7ff0977fe640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08406b4d0 0x7ff08406d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff08800aac0 tx=0x7ff088008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.455+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0980b6320 con 0x7ff09815dbe0 2024-04-25T08:35:35.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:35.709+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7ff098073990 con 0x7ff09815dbe0 2024-04-25T08:35:36.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.714+0000 7ff07ffff640 1 -- 172.21.15.60:0/928550408 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v307) v1 ==== 147+0+0 (secure 0 0 0) 0x7ff0900587b0 con 0x7ff09815dbe0 2024-04-25T08:35:36.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08406b4d0 msgr2=0x7ff08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08406b4d0 0x7ff08406d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff08800aac0 tx=0x7ff088008040 comp rx=0 tx=0).stop 2024-04-25T08:35:36.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 msgr2=0x7ff098159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 0x7ff098159de0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff09000c760 tx=0x7ff09000cc20 comp rx=0 tx=0).stop 2024-04-25T08:35:36.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 shutdown_connections 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff09815dbe0 0x7ff098159de0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0980b92a0 0x7ff098150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff08406b4d0 0x7ff08406d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 --2- 172.21.15.60:0/928550408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0980b2cb0 0x7ff0981459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 >> 172.21.15.60:0/928550408 conn(0x7ff098061a20 msgr2=0x7ff0980b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 shutdown_connections 2024-04-25T08:35:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.717+0000 7ff09deac640 1 -- 172.21.15.60:0/928550408 wait complete. 2024-04-25T08:35:36.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:35:36.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.842+0000 7fdde029e640 1 Processor -- start 2024-04-25T08:35:36.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.842+0000 7fdde029e640 1 -- start start 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.843+0000 7fdde029e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fddd815a9f0 0x7fddd80c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.843+0000 7fdde029e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd80c6940 con 0x7fddd80cdbf0 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.843+0000 7fdde029e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd815add0 con 0x7fddd80cd6f0 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.843+0000 7fdde029e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd80b0020 con 0x7fddd815a9f0 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.843+0000 7fdddd839640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fddd80cdbf0 0x7fddd80ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58410/0 (socket says 172.21.15.60:58410) 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.843+0000 7fdddd839640 1 -- 172.21.15.60:0/1568594637 learned_addr learned my addr 172.21.15.60:0/1568594637 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061006410 0 0) 0x7fddc8001110 con 0x7fddd80cdbf0 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fddde03a640 1 --2- 172.21.15.60:0/1568594637 >> v2:172.21.15.100:3300/0 conn(0x7fddd815a9f0 0x7fddd80c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc4005730 con 0x7fddd80cdbf0 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2238825244 0 0) 0x7fddc4005730 con 0x7fddd80cdbf0 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 >> v2:172.21.15.100:3300/0 conn(0x7fddd815a9f0 msgr2=0x7fddd80c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 --2- 172.21.15.60:0/1568594637 >> v2:172.21.15.100:3300/0 conn(0x7fddd815a9f0 0x7fddd80c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 >> v1:172.21.15.100:6789/0 conn(0x7fddd80cd6f0 legacy=0x7fddd80cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd80c6660 con 0x7fddd80cdbf0 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fddc8001840 con 0x7fddd80cdbf0 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 >> v1:172.21.15.60:6789/0 conn(0x7fddd80cdbf0 legacy=0x7fddd80ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc4005bc0 0x7fddc4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc4006a10 0x7fddc4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc40073d0 0x7fddc400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc400e240 con 0x7fddc4005bc0 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc400e3f0 con 0x7fddc40073d0 2024-04-25T08:35:36.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc400e5f0 con 0x7fddc4006a10 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fddde03a640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc4005bc0 0x7fddc4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fddde83b640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc40073d0 0x7fddc400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.844+0000 7fdddd839640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc4006a10 0x7fddc4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde03a640 1 -- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc4006a10 msgr2=0x7fddc4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde03a640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc4006a10 0x7fddc4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde03a640 1 -- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc40073d0 msgr2=0x7fddc400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde03a640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc40073d0 0x7fddc400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde03a640 1 -- 172.21.15.60:0/1568594637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc4005730 con 0x7fddc4005bc0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde83b640 1 -- 172.21.15.60:0/1568594637 reap_dead start 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fdddd839640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddc4006a10 0x7fddc4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fddde03a640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc4005bc0 0x7fddc4005fe0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009ac0 tx=0x7fddc4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e118fa6ed96be3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0036070 con 0x7fddc4005bc0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0031400 con 0x7fddc4005bc0 2024-04-25T08:35:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.845+0000 7fdddd038640 1 -- 172.21.15.60:0/1568594637 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0009690 con 0x7fddc4005bc0 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 -- 172.21.15.60:0/1568594637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc4005bc0 msgr2=0x7fddc4005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc4005bc0 0x7fddc4005fe0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009ac0 tx=0x7fddc4007a40 comp rx=0 tx=0).stop 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 -- 172.21.15.60:0/1568594637 shutdown_connections 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 --2- 172.21.15.60:0/1568594637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddc4005bc0 0x7fddc4005fe0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 -- 172.21.15.60:0/1568594637 >> 172.21.15.60:0/1568594637 conn(0x7fddd80b2390 msgr2=0x7fddd80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 -- 172.21.15.60:0/1568594637 shutdown_connections 2024-04-25T08:35:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.846+0000 7fdde029e640 1 -- 172.21.15.60:0/1568594637 wait complete. 2024-04-25T08:35:36.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.847+0000 7fdde029e640 1 Processor -- start 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.847+0000 7fdde029e640 1 -- start start 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdde029e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4005bc0 0x7fddd80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdde029e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddd80af5a0 0x7fddd80cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdde029e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 0x7fddd80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdde029e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80cb230 con 0x7fddd8153160 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdde029e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80cb5a0 con 0x7fddc4005bc0 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdde029e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80ce070 con 0x7fddd80af5a0 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 0x7fddd80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 0x7fddd80cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33220/0 (socket says 172.21.15.60:33220) 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 -- 172.21.15.60:0/608464344 learned_addr learned my addr 172.21.15.60:0/608464344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde03a640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4005bc0 0x7fddd80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fdddd839640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddd80af5a0 0x7fddd80cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 -- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddd80af5a0 msgr2=0x7fddd80cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddd80af5a0 0x7fddd80cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 -- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4005bc0 msgr2=0x7fddd80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4005bc0 0x7fddd80af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.848+0000 7fddde83b640 1 -- 172.21.15.60:0/608464344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd80c9430 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fdddd839640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddd80af5a0 0x7fddd80cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fddde83b640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 0x7fddd80cfc10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fddd80b3450 tx=0x7fddd400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd401a070 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4006d60 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd80c96b0 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4003bb0 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.849+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd80c9c70 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.850+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd8070bb0 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.850+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddd401d070 con 0x7fddd8153160 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.851+0000 7fddceffd640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddbc06b4d0 0x7fddbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.851+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fddd4089580 con 0x7fddd8153160 2024-04-25T08:35:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.851+0000 7fddde03a640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddbc06b4d0 0x7fddbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.852+0000 7fddde03a640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddbc06b4d0 0x7fddbc06d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fddd0033000 tx=0x7fddd0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:36.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:36.855+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddd8070bb0 con 0x7fddd8153160 2024-04-25T08:35:37.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.037+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fddd80ceed0 con 0x7fddd8153160 2024-04-25T08:35:37.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.040+0000 7fddceffd640 1 -- 172.21.15.60:0/608464344 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fddd80c9430 con 0x7fddd8153160 2024-04-25T08:35:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddbc06b4d0 msgr2=0x7fddbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddbc06b4d0 0x7fddbc06d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fddd0033000 tx=0x7fddd0008030 comp rx=0 tx=0).stop 2024-04-25T08:35:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 msgr2=0x7fddd80cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 0x7fddd80cfc10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fddd80b3450 tx=0x7fddd400fc00 comp rx=0 tx=0).stop 2024-04-25T08:35:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 shutdown_connections 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fddd8153160 0x7fddd80cfc10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fddd80af5a0 0x7fddd80cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddbc06b4d0 0x7fddbc06d990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 --2- 172.21.15.60:0/608464344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fddc4005bc0 0x7fddd80af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 >> 172.21.15.60:0/608464344 conn(0x7fddd80b2390 msgr2=0x7fddd806e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 shutdown_connections 2024-04-25T08:35:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.042+0000 7fdde029e640 1 -- 172.21.15.60:0/608464344 wait complete. 2024-04-25T08:35:37.048 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:35:37.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:37.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:37.129 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:35:37.129 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:35:37.129 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:37.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:35:37.145 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:35:37.145 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:37.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:37.187 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:35:37.187 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:37.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:35:37.203 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:35:37.203 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:37.244 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:37.245 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:35:37.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.350+0000 7efcf992b640 1 Processor -- start 2024-04-25T08:35:37.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf992b640 1 -- start start 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf992b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efcf40ce920 0x7efcf40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf992b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf4144b30 con 0x7efcf4144750 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf992b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf40712d0 con 0x7efcf406d290 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf992b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf40715b0 con 0x7efcf40ce920 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf2ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efcf4144750 0x7efcf406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58424/0 (socket says 172.21.15.60:58424) 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.351+0000 7efcf2ffd640 1 -- 172.21.15.60:0/3525354022 learned_addr learned my addr 172.21.15.60:0/3525354022 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf27fc640 1 --2- 172.21.15.60:0/3525354022 >> v2:172.21.15.100:3300/0 conn(0x7efcf40ce920 0x7efcf40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8641957 0 0) 0x7efcf4144b30 con 0x7efcf4144750 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcdc003520 con 0x7efcf4144750 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1778383585 0 0) 0x7efcdc003520 con 0x7efcf4144750 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 >> v2:172.21.15.100:3300/0 conn(0x7efcf40ce920 msgr2=0x7efcf40cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 --2- 172.21.15.60:0/3525354022 >> v2:172.21.15.100:3300/0 conn(0x7efcf40ce920 0x7efcf40cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 >> v1:172.21.15.100:6789/0 conn(0x7efcf406d290 legacy=0x7efcf406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4071890 con 0x7efcf4144750 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7efce8001670 con 0x7efcf4144750 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 >> v1:172.21.15.60:6789/0 conn(0x7efcf4144750 legacy=0x7efcf406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc004800 0x7efcdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc0051c0 0x7efcdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40715b0 con 0x7efcdc004800 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc00c060 con 0x7efcdc0039b0 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc00c260 con 0x7efcdc0051c0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf27fc640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc004800 0x7efcdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.352+0000 7efcf37fe640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc0051c0 0x7efcdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf2ffd640 1 -- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc0051c0 msgr2=0x7efcdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcdc0051c0 0x7efcdc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf2ffd640 1 -- 172.21.15.60:0/3525354022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc004800 msgr2=0x7efcdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcdc004800 0x7efcdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf2ffd640 1 -- 172.21.15.60:0/3525354022 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc003520 con 0x7efcdc0039b0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf37fe640 1 -- 172.21.15.60:0/3525354022 reap_dead start 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.353+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcdc003dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efce800c240 tx=0x7efcdc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c8316d352953fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8007d10 con 0x7efcdc0039b0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce80058d0 con 0x7efcdc0039b0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf1ffb640 1 -- 172.21.15.60:0/3525354022 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8005d40 con 0x7efcdc0039b0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 -- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 msgr2=0x7efcdc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcdc003dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efce800c240 tx=0x7efcdc005830 comp rx=0 tx=0).stop 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 -- 172.21.15.60:0/3525354022 shutdown_connections 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 --2- 172.21.15.60:0/3525354022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcdc003dd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 -- 172.21.15.60:0/3525354022 >> 172.21.15.60:0/3525354022 conn(0x7efcf406abe0 msgr2=0x7efcf406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 -- 172.21.15.60:0/3525354022 shutdown_connections 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.354+0000 7efcf992b640 1 -- 172.21.15.60:0/3525354022 wait complete. 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.355+0000 7efcf992b640 1 Processor -- start 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 -- start start 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcf40b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 0x7efcf415a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf415ab30 0x7efcf41583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41455d0 con 0x7efcf40b3970 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4145450 con 0x7efcdc0039b0 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.356+0000 7efcf992b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf415c000 con 0x7efcf415ab30 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 0x7efcf415a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 0x7efcf415a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33242/0 (socket says 172.21.15.60:33242) 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 -- 172.21.15.60:0/3159912606 learned_addr learned my addr 172.21.15.60:0/3159912606 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcf40b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf37fe640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf415ab30 0x7efcf41583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 -- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf415ab30 msgr2=0x7efcf41583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf415ab30 0x7efcf41583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 -- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 msgr2=0x7efcf40b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcf40b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 -- 172.21.15.60:0/3159912606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf40b3160 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf27fc640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 0x7efcf415a5f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efce40118c0 tx=0x7efce4011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4019070 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce4004240 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf415e870 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400f820 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.357+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf415ee30 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.358+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf4073990 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.359+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efce401c070 con 0x7efcf40b3970 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.359+0000 7efcdbfff640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efce006b500 0x7efce006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.359+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efce408a250 con 0x7efcf40b3970 2024-04-25T08:35:37.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.359+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efce006b500 0x7efce006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.360+0000 7efcf2ffd640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efce006b500 0x7efce006d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efce8033000 tx=0x7efce802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.363+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efce4058c80 con 0x7efcf40b3970 2024-04-25T08:35:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.583+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcf40cd7a0 con 0x7efcf40b3970 2024-04-25T08:35:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.584+0000 7efcdbfff640 1 -- 172.21.15.60:0/3159912606 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3702 (secure 0 0 0) 0x7efce405cd20 con 0x7efcf40b3970 2024-04-25T08:35:37.579 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:37.579 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/4273371258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":4273371258},{"type":"v1","addr":"172.21.15.100:6836","nonce":4273371258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6170,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1779653508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1779653508},{"type":"v1","addr":"172.21.15.100:6837","nonce":1779653508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6544,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/661792740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":661792740},{"type":"v1","addr":"172.21.15.60:6834","nonce":661792740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7191,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2216279733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2216279733},{"type":"v1","addr":"172.21.15.60:6835","nonce":2216279733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[]} 2024-04-25T08:35:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 51 2024-04-25T08:35:37.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efce006b500 msgr2=0x7efce006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efce006b500 0x7efce006d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efce8033000 tx=0x7efce802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 msgr2=0x7efcf415a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 0x7efcf415a5f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efce40118c0 tx=0x7efce4011d80 comp rx=0 tx=0).stop 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 shutdown_connections 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efcf415ab30 0x7efcf41583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efcf40b3970 0x7efcf415a5f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efce006b500 0x7efce006d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.585+0000 7efcf992b640 1 --2- 172.21.15.60:0/3159912606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efcdc0039b0 0x7efcf40b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.586+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 >> 172.21.15.60:0/3159912606 conn(0x7efcf406abe0 msgr2=0x7efcf40b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.586+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 shutdown_connections 2024-04-25T08:35:37.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.586+0000 7efcf992b640 1 -- 172.21.15.60:0/3159912606 wait complete. 2024-04-25T08:35:37.594 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:35:37.594 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:35:37.595 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:35:37.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:37.595 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:35:37.595 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:35:37.595 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:35:37.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:37.596 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:35:37.596 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:35:37.596 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:35:37.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:37.596 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:35:37.596 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:35:37.596 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:35:37.597 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:37.604 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:35:37.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:35:37.605 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:35:37.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:35:37.608 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:35:37.608 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:35:37.608 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:35:37.609 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:35:37.647 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:35:37.648 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:37.648 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:35:37.648 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:37.650 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:35:37.650 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:37.651 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:35:37.651 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:37.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:35:37.723 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:35:37.723 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:35:37.730 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:35:37.730 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:35:37.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.763+0000 7f173efe7640 1 Processor -- start 2024-04-25T08:35:37.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.763+0000 7f173efe7640 1 -- start start 2024-04-25T08:35:37.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f173efe7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17380ccdd0 0x7f17380cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f173efe7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17380cb190 con 0x7f17380cdcd0 2024-04-25T08:35:37.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f173efe7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17380cb470 con 0x7f173815bb20 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f173efe7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17380cb750 con 0x7f17380ccdd0 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f173d584640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f17380cdcd0 0x7f17380ad600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58448/0 (socket says 172.21.15.60:58448) 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f173d584640 1 -- 172.21.15.60:0/3326763432 learned_addr learned my addr 172.21.15.60:0/3326763432 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f1737fff640 1 --2- 172.21.15.60:0/3326763432 >> v2:172.21.15.100:3300/0 conn(0x7f17380ccdd0 0x7f17380cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568320159 0 0) 0x7f17380cb190 con 0x7f17380cdcd0 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1728003520 con 0x7f17380cdcd0 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879463654 0 0) 0x7f17380cb470 con 0x7f173815bb20 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17380cb190 con 0x7f173815bb20 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.764+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379317700 0 0) 0x7f1728003520 con 0x7f17380cdcd0 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f1737fff640 1 -- 172.21.15.60:0/3326763432 >> v1:172.21.15.100:6789/0 conn(0x7f173815bb20 legacy=0x7f17380cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f1737fff640 1 -- 172.21.15.60:0/3326763432 >> v1:172.21.15.60:6789/0 conn(0x7f17380cdcd0 legacy=0x7f17380ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f1737fff640 1 -- 172.21.15.60:0/3326763432 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17380cba30 con 0x7f17380ccdd0 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f1737fff640 1 --2- 172.21.15.60:0/3326763432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380ccdd0 0x7f17380cf9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f17240097a0 tx=0x7f172402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9545f209711537e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724037070 con 0x7f17380ccdd0 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172402abc0 con 0x7f17380ccdd0 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.765+0000 7f17377fe640 1 -- 172.21.15.60:0/3326763432 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300019f0 con 0x7f17380ccdd0 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 -- 172.21.15.60:0/3326763432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380ccdd0 msgr2=0x7f17380cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 --2- 172.21.15.60:0/3326763432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380ccdd0 0x7f17380cf9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f17240097a0 tx=0x7f172402a130 comp rx=0 tx=0).stop 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 -- 172.21.15.60:0/3326763432 shutdown_connections 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 --2- 172.21.15.60:0/3326763432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380ccdd0 0x7f17380cf9d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 -- 172.21.15.60:0/3326763432 >> 172.21.15.60:0/3326763432 conn(0x7f17380b2390 msgr2=0x7f17380b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 -- 172.21.15.60:0/3326763432 shutdown_connections 2024-04-25T08:35:37.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.766+0000 7f173efe7640 1 -- 172.21.15.60:0/3326763432 wait complete. 2024-04-25T08:35:37.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.767+0000 7f173efe7640 1 Processor -- start 2024-04-25T08:35:37.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.767+0000 7f173efe7640 1 -- start start 2024-04-25T08:35:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173efe7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 0x7f1738143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173efe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380cdcd0 0x7f1738146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173efe7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f173815bb20 0x7f1738157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173efe7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173815c080 con 0x7f17380ccdd0 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173efe7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173815bf00 con 0x7f17380cdcd0 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173efe7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17380cadb0 con 0x7f173815bb20 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173cd83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 0x7f1738143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173cd83640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 0x7f1738143980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33278/0 (socket says 172.21.15.60:33278) 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173cd83640 1 -- 172.21.15.60:0/2654723770 learned_addr learned my addr 172.21.15.60:0/2654723770 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f1737fff640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380cdcd0 0x7f1738146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.768+0000 7f173d584640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f173815bb20 0x7f1738157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173cd83640 1 -- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f173815bb20 msgr2=0x7f1738157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173cd83640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f173815bb20 0x7f1738157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173cd83640 1 -- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380cdcd0 msgr2=0x7f1738146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173cd83640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380cdcd0 0x7f1738146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173cd83640 1 -- 172.21.15.60:0/2654723770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738143ec0 con 0x7f17380ccdd0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173cd83640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 0x7f1738143980 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c490 tx=0x7f172c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0052d0 con 0x7f17380ccdd0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c005470 con 0x7f17380ccdd0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738146bd0 con 0x7f17380ccdd0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c012850 con 0x7f17380ccdd0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.769+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1738144300 con 0x7f17380ccdd0 2024-04-25T08:35:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.770+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1738073a00 con 0x7f17380ccdd0 2024-04-25T08:35:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.770+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f172c0129f0 con 0x7f17380ccdd0 2024-04-25T08:35:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.771+0000 7f17357fa640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f171c06b4d0 0x7f171c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.771+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f172c089bb0 con 0x7f17380ccdd0 2024-04-25T08:35:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.771+0000 7f1737fff640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f171c06b4d0 0x7f171c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.772+0000 7f1737fff640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f171c06b4d0 0x7f171c06d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1724034000 tx=0x7f172402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.775+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f172c0585e0 con 0x7f17380ccdd0 2024-04-25T08:35:37.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T08:35:37.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T08:35:37.810 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T08:35:37.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:37.967+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f173815a1c0 con 0x7f17380ccdd0 2024-04-25T08:35:39.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.054+0000 7f17357fa640 1 -- 172.21.15.60:0/2654723770 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f172c05c680 con 0x7f17380ccdd0 2024-04-25T08:35:39.049 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f171c06b4d0 msgr2=0x7f171c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f171c06b4d0 0x7f171c06d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1724034000 tx=0x7f172402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 msgr2=0x7f1738143980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 0x7f1738143980 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c490 tx=0x7f172c00c950 comp rx=0 tx=0).stop 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 shutdown_connections 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f173815bb20 0x7f1738157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f171c06b4d0 0x7f171c06d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17380cdcd0 0x7f1738146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 --2- 172.21.15.60:0/2654723770 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17380ccdd0 0x7f1738143980 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 >> 172.21.15.60:0/2654723770 conn(0x7f17380b2390 msgr2=0x7f173806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 shutdown_connections 2024-04-25T08:35:39.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.057+0000 7f173efe7640 1 -- 172.21.15.60:0/2654723770 wait complete. 2024-04-25T08:35:39.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:35:39.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.178+0000 7fdb43eb5640 1 Processor -- start 2024-04-25T08:35:39.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.178+0000 7fdb43eb5640 1 -- start start 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb43eb5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb3c15db70 0x7fdb3c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb43eb5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb43eb5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3c162690 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb43eb5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c0d3800 con 0x7fdb3c15db70 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb43eb5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ceee0 con 0x7fdb3c0b6cb0 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41c51640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdb3c15db70 0x7fdb3c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb42452640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdb3c0b5a60 0x7fdb3c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58458/0 (socket says 172.21.15.60:58458) 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb42452640 1 -- 172.21.15.60:0/2830407715 learned_addr learned my addr 172.21.15.60:0/2830407715 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41450640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41c51640 1 -- 172.21.15.60:0/2830407715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c0b6cb0 msgr2=0x7fdb3c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41c51640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41c51640 1 -- 172.21.15.60:0/2830407715 >> v1:172.21.15.60:6789/0 conn(0x7fdb3c0b5a60 legacy=0x7fdb3c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41c51640 1 -- 172.21.15.60:0/2830407715 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c0d1d10 con 0x7fdb3c15db70 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb40c4f640 1 -- 172.21.15.60:0/2830407715 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973076841 0 0) 0x7fdb3c162690 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.179+0000 7fdb41450640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.180+0000 7fdb41c51640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c15db70 0x7fdb3c162150 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097a0 tx=0x7fdb3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f957456b3bc3e579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.180+0000 7fdb40c4f640 1 -- 172.21.15.60:0/2830407715 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38037070 con 0x7fdb3c15db70 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.180+0000 7fdb40c4f640 1 -- 172.21.15.60:0/2830407715 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38032440 con 0x7fdb3c15db70 2024-04-25T08:35:39.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.180+0000 7fdb40c4f640 1 -- 172.21.15.60:0/2830407715 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38032810 con 0x7fdb3c15db70 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 -- 172.21.15.60:0/2830407715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c15db70 msgr2=0x7fdb3c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c15db70 0x7fdb3c162150 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097a0 tx=0x7fdb3802a180 comp rx=0 tx=0).stop 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 -- 172.21.15.60:0/2830407715 shutdown_connections 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/2830407715 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c15db70 0x7fdb3c162150 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 -- 172.21.15.60:0/2830407715 >> 172.21.15.60:0/2830407715 conn(0x7fdb3c0d0520 msgr2=0x7fdb3c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 -- 172.21.15.60:0/2830407715 shutdown_connections 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.181+0000 7fdb43eb5640 1 -- 172.21.15.60:0/2830407715 wait complete. 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.182+0000 7fdb43eb5640 1 Processor -- start 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.182+0000 7fdb43eb5640 1 -- start start 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb43eb5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 0x7fdb3c0cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb43eb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb43eb5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c15db70 0x7fdb3c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb43eb5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c15df50 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb43eb5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0d22d0 con 0x7fdb3c0b6cb0 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb43eb5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ce530 con 0x7fdb3c15db70 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 0x7fdb3c0cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 0x7fdb3c0cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33310/0 (socket says 172.21.15.60:33310) 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 -- 172.21.15.60:0/1841270245 learned_addr learned my addr 172.21.15.60:0/1841270245 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb42452640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c15db70 0x7fdb3c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41450640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 -- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c15db70 msgr2=0x7fdb3c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c15db70 0x7fdb3c0cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 -- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3c0b6cb0 msgr2=0x7fdb3c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 -- 172.21.15.60:0/1841270245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c0cb0b0 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb42452640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c15db70 0x7fdb3c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.183+0000 7fdb41c51640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 0x7fdb3c0cc540 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdb380098c0 tx=0x7fdb380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.184+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38037040 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.184+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38030770 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.184+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c0ccbe0 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.184+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38030b40 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.184+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c0cd480 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.185+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c0ccfd0 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.185+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb38044020 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.185+0000 7fdb327fc640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.186+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdb380aa610 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.186+0000 7fdb41450640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.186+0000 7fdb41450640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0680d0 tx=0x7fdb2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.190+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb38032e20 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.443+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdb3c0d1820 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.443+0000 7fdb327fc640 1 -- 172.21.15.60:0/1841270245 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fdb380313a0 con 0x7fdb3c0b5a60 2024-04-25T08:35:39.438 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:39.438 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:35:39.439 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:35:39.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.441 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:35:39.443 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.445 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.446 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:35:39.447 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.449 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:35:39.451 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 msgr2=0x7fdb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0680d0 tx=0x7fdb2c008040 comp rx=0 tx=0).stop 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 msgr2=0x7fdb3c0cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 0x7fdb3c0cc540 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fdb380098c0 tx=0x7fdb380046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 shutdown_connections 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdb3c15db70 0x7fdb3c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdb2006b4d0 0x7fdb2006d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdb3c0b6cb0 0x7fdb3c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 --2- 172.21.15.60:0/1841270245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdb3c0b5a60 0x7fdb3c0cc540 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 >> 172.21.15.60:0/1841270245 conn(0x7fdb3c0d0520 msgr2=0x7fdb3c15a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 shutdown_connections 2024-04-25T08:35:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.445+0000 7fdb43eb5640 1 -- 172.21.15.60:0/1841270245 wait complete. 2024-04-25T08:35:39.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:39.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.561+0000 7fa8a039e640 1 Processor -- start 2024-04-25T08:35:39.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.561+0000 7fa8a039e640 1 -- start start 2024-04-25T08:35:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa8a039e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa8980b6cb0 0x7fa8980b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa8a039e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980ceee0 con 0x7fa89815db70 2024-04-25T08:35:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa8a039e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980d1d10 con 0x7fa8980b5a60 2024-04-25T08:35:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa8a039e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980d1ff0 con 0x7fa8980b6cb0 2024-04-25T08:35:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89e13a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa89815db70 0x7fa898162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58472/0 (socket says 172.21.15.60:58472) 2024-04-25T08:35:39.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89e13a640 1 -- 172.21.15.60:0/679847422 learned_addr learned my addr 172.21.15.60:0/679847422 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732334919 0 0) 0x7fa8980ceee0 con 0x7fa89815db70 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d939640 1 --2- 172.21.15.60:0/679847422 >> v2:172.21.15.100:3300/0 conn(0x7fa8980b6cb0 0x7fa8980b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa884003520 con 0x7fa89815db70 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3176327326 0 0) 0x7fa884003520 con 0x7fa89815db70 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 >> v2:172.21.15.100:3300/0 conn(0x7fa8980b6cb0 msgr2=0x7fa8980b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 --2- 172.21.15.60:0/679847422 >> v2:172.21.15.100:3300/0 conn(0x7fa8980b6cb0 0x7fa8980b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 >> v1:172.21.15.100:6789/0 conn(0x7fa8980b5a60 legacy=0x7fa8980d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980d22d0 con 0x7fa89815db70 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fa894001670 con 0x7fa89815db70 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 >> v1:172.21.15.60:6789/0 conn(0x7fa89815db70 legacy=0x7fa898162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8840039b0 0x7fa884003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa884004800 0x7fa884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8840051c0 0x7fa88400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980d1ff0 con 0x7fa8840051c0 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88400c160 con 0x7fa8840039b0 2024-04-25T08:35:39.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88400c360 con 0x7fa884004800 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.562+0000 7fa89e93b640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8840051c0 0x7fa88400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e13a640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8840039b0 0x7fa884003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89d939640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa884004800 0x7fa884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 -- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa884004800 msgr2=0x7fa884004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa884004800 0x7fa884004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 -- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8840039b0 msgr2=0x7fa884003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa8840039b0 0x7fa884003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 -- 172.21.15.60:0/679847422 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa884003520 con 0x7fa8840051c0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 -- 172.21.15.60:0/679847422 reap_dead start 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.563+0000 7fa89e93b640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8840051c0 0x7fa88400b9c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa890001e50 tx=0x7fa884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbdf38e4c8bb972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890001a70 con 0x7fa8840051c0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa890003970 con 0x7fa8840051c0 2024-04-25T08:35:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa89d138640 1 -- 172.21.15.60:0/679847422 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890003d60 con 0x7fa8840051c0 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 -- 172.21.15.60:0/679847422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8840051c0 msgr2=0x7fa88400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8840051c0 0x7fa88400b9c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa890001e50 tx=0x7fa884005830 comp rx=0 tx=0).stop 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 -- 172.21.15.60:0/679847422 shutdown_connections 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 --2- 172.21.15.60:0/679847422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8840051c0 0x7fa88400b9c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 -- 172.21.15.60:0/679847422 >> 172.21.15.60:0/679847422 conn(0x7fa8980d0520 msgr2=0x7fa8980d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 -- 172.21.15.60:0/679847422 shutdown_connections 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.564+0000 7fa8a039e640 1 -- 172.21.15.60:0/679847422 wait complete. 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.565+0000 7fa8a039e640 1 Processor -- start 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.565+0000 7fa8a039e640 1 -- start start 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa8a039e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8840051c0 0x7fa898152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa8a039e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 0x7fa8980cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa8a039e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa898153ae0 0x7fa898160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa8a039e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980cf1c0 con 0x7fa8980cd710 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa8a039e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980ce530 con 0x7fa898153ae0 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa8a039e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980cf340 con 0x7fa8840051c0 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 0x7fa8980cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 0x7fa8980cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33326/0 (socket says 172.21.15.60:33326) 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 -- 172.21.15.60:0/2050312398 learned_addr learned my addr 172.21.15.60:0/2050312398 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89e13a640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8840051c0 0x7fa898152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89e93b640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa898153ae0 0x7fa898160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 -- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8840051c0 msgr2=0x7fa898152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8840051c0 0x7fa898152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 -- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa898153ae0 msgr2=0x7fa898160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.566+0000 7fa89d939640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa898153ae0 0x7fa898160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa89d939640 1 -- 172.21.15.60:0/2050312398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980cc670 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa89e13a640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8840051c0 0x7fa898152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa89e93b640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa898153ae0 0x7fa898160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa89d939640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 0x7fa8980cbf70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa888011990 tx=0x7fa888011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888019070 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888004240 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8980cc950 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800f860 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.567+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980ccf10 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.568+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa898073990 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.568+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa88801c070 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.569+0000 7fa88effd640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa87c06b4d0 0x7fa87c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.569+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa88808a570 con 0x7fa8980cd710 2024-04-25T08:35:39.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.569+0000 7fa89e13a640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa87c06b4d0 0x7fa87c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.570+0000 7fa89e13a640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa87c06b4d0 0x7fa87c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa894007c70 tx=0x7fa894009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.573+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa888059520 con 0x7fa8980cd710 2024-04-25T08:35:39.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.755+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa898158f90 con 0x7fa8980cd710 2024-04-25T08:35:39.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.756+0000 7fa88effd640 1 -- 172.21.15.60:0/2050312398 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13153 (secure 0 0 0) 0x7fa888058e20 con 0x7fa8980cd710 2024-04-25T08:35:39.751 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:39.751 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":93,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:39.053532+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa87c06b4d0 msgr2=0x7fa87c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa87c06b4d0 0x7fa87c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa894007c70 tx=0x7fa894009040 comp rx=0 tx=0).stop 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 msgr2=0x7fa8980cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 0x7fa8980cbf70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa888011990 tx=0x7fa888011e50 comp rx=0 tx=0).stop 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 shutdown_connections 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa87c06b4d0 0x7fa87c06d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa898153ae0 0x7fa898160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa8980cd710 0x7fa8980cbf70 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 --2- 172.21.15.60:0/2050312398 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa8840051c0 0x7fa898152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 >> 172.21.15.60:0/2050312398 conn(0x7fa8980d0520 msgr2=0x7fa89815c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 shutdown_connections 2024-04-25T08:35:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.758+0000 7fa8a039e640 1 -- 172.21.15.60:0/2050312398 wait complete. 2024-04-25T08:35:39.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:39.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.874+0000 7fc4433f4640 1 Processor -- start 2024-04-25T08:35:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.874+0000 7fc4433f4640 1 -- start start 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc4433f4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc43c15dc20 0x7fc43c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc4433f4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc43c161710 0x7fc43c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc4433f4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43c15e000 con 0x7fc43c0b9330 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc4433f4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c0d1de0 con 0x7fc43c15dc20 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc4433f4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43c161c60 con 0x7fc43c161710 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc43c15dc20 0x7fc43c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc43c15dc20 0x7fc43c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33328/0 (socket says 172.21.15.60:33328) 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 -- 172.21.15.60:0/358938225 learned_addr learned my addr 172.21.15.60:0/358938225 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:39.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc44098f640 1 --2- 172.21.15.60:0/358938225 >> v2:172.21.15.100:3300/0 conn(0x7fc43c161710 0x7fc43c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 -- 172.21.15.60:0/358938225 >> v2:172.21.15.100:3300/0 conn(0x7fc43c161710 msgr2=0x7fc43c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 --2- 172.21.15.60:0/358938225 >> v2:172.21.15.100:3300/0 conn(0x7fc43c161710 0x7fc43c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 -- 172.21.15.60:0/358938225 >> v1:172.21.15.100:6789/0 conn(0x7fc43c0b9330 legacy=0x7fc43c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.875+0000 7fc441190640 1 -- 172.21.15.60:0/358938225 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c161f40 con 0x7fc43c15dc20 2024-04-25T08:35:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.876+0000 7fc441190640 1 --2- 172.21.15.60:0/358938225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0ce670 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc43c06da90 tx=0x7fc43802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3885388084d591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.876+0000 7fc433fff640 1 -- 172.21.15.60:0/358938225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438037070 con 0x7fc43c15dc20 2024-04-25T08:35:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.876+0000 7fc433fff640 1 -- 172.21.15.60:0/358938225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438032440 con 0x7fc43c15dc20 2024-04-25T08:35:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.876+0000 7fc433fff640 1 -- 172.21.15.60:0/358938225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438032810 con 0x7fc43c15dc20 2024-04-25T08:35:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 -- 172.21.15.60:0/358938225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 msgr2=0x7fc43c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 --2- 172.21.15.60:0/358938225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0ce670 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fc43c06da90 tx=0x7fc43802a270 comp rx=0 tx=0).stop 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 -- 172.21.15.60:0/358938225 shutdown_connections 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 --2- 172.21.15.60:0/358938225 >> v2:172.21.15.100:3300/0 conn(0x7fc43c161710 0x7fc43c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 --2- 172.21.15.60:0/358938225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0ce670 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 -- 172.21.15.60:0/358938225 >> 172.21.15.60:0/358938225 conn(0x7fc43c0d0420 msgr2=0x7fc43c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 -- 172.21.15.60:0/358938225 shutdown_connections 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.877+0000 7fc4433f4640 1 -- 172.21.15.60:0/358938225 wait complete. 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.878+0000 7fc4433f4640 1 Processor -- start 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.878+0000 7fc4433f4640 1 -- start start 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43c0b9330 0x7fc43c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc43c161710 0x7fc43c0d3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c0b5e50 con 0x7fc43c15dc20 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c162500 con 0x7fc43c0b9330 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43c0b5fd0 con 0x7fc43c161710 2024-04-25T08:35:39.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33342/0 (socket says 172.21.15.60:33342) 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 -- 172.21.15.60:0/2944654750 learned_addr learned my addr 172.21.15.60:0/2944654750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc441190640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43c0b9330 0x7fc43c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc441991640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc43c161710 0x7fc43c0d3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 -- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc43c161710 msgr2=0x7fc43c0d3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc43c161710 0x7fc43c0d3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 -- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43c0b9330 msgr2=0x7fc43c15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43c0b9330 0x7fc43c15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 -- 172.21.15.60:0/2944654750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c0aae40 con 0x7fc43c15dc20 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc441190640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43c0b9330 0x7fc43c15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc44098f640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0aa380 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00d5f0 tx=0x7fc42c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0125f0 con 0x7fc43c15dc20 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c019070 con 0x7fc43c15dc20 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43c0ab120 con 0x7fc43c15dc20 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.879+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0129d0 con 0x7fc43c15dc20 2024-04-25T08:35:39.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.880+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43c0ab6e0 con 0x7fc43c15dc20 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.881+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c073a00 con 0x7fc43c15dc20 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.881+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc42c012b70 con 0x7fc43c15dc20 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.881+0000 7fc431ffb640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc42006b4d0 0x7fc42006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.881+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc42c08a430 con 0x7fc43c15dc20 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.882+0000 7fc441190640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc42006b4d0 0x7fc42006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.882+0000 7fc441190640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc42006b4d0 0x7fc42006d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc438034000 tx=0x7fc43802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:39.886+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc42c059470 con 0x7fc43c15dc20 2024-04-25T08:35:40.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.067+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc43c0d16b0 con 0x7fc43c15dc20 2024-04-25T08:35:40.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.068+0000 7fc431ffb640 1 -- 172.21.15.60:0/2944654750 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc42c058d70 con 0x7fc43c15dc20 2024-04-25T08:35:40.063 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc42006b4d0 msgr2=0x7fc42006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc42006b4d0 0x7fc42006d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc438034000 tx=0x7fc43802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 msgr2=0x7fc43c0aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0aa380 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00d5f0 tx=0x7fc42c00dab0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 shutdown_connections 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc43c161710 0x7fc43c0d3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc43c15dc20 0x7fc43c0aa380 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc42006b4d0 0x7fc42006d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 --2- 172.21.15.60:0/2944654750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc43c0b9330 0x7fc43c15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 >> 172.21.15.60:0/2944654750 conn(0x7fc43c0d0420 msgr2=0x7fc43c0c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 shutdown_connections 2024-04-25T08:35:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.070+0000 7fc4433f4640 1 -- 172.21.15.60:0/2944654750 wait complete. 2024-04-25T08:35:40.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:35:40.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.181+0000 7f640f44d640 1 Processor -- start 2024-04-25T08:35:40.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.181+0000 7f640f44d640 1 -- start start 2024-04-25T08:35:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640f44d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f640815dbe0 0x7f64080d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640f44d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64080b92a0 0x7f64080d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640f44d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080b97a0 con 0x7f64080b2cb0 2024-04-25T08:35:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640f44d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640815dfc0 con 0x7f64080b92a0 2024-04-25T08:35:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640f44d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080d1a00 con 0x7f640815dbe0 2024-04-25T08:35:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640d9ea640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f64080b92a0 0x7f64080d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640c9e8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f64080b2cb0 0x7f64080b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58474/0 (socket says 172.21.15.60:58474) 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.182+0000 7f640c9e8640 1 -- 172.21.15.60:0/755331324 learned_addr learned my addr 172.21.15.60:0/755331324 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d1e9640 1 --2- 172.21.15.60:0/755331324 >> v2:172.21.15.100:3300/0 conn(0x7f640815dbe0 0x7f64080d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d9ea640 1 -- 172.21.15.60:0/755331324 >> v2:172.21.15.100:3300/0 conn(0x7f640815dbe0 msgr2=0x7f64080d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d9ea640 1 --2- 172.21.15.60:0/755331324 >> v2:172.21.15.100:3300/0 conn(0x7f640815dbe0 0x7f64080d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d9ea640 1 -- 172.21.15.60:0/755331324 >> v1:172.21.15.60:6789/0 conn(0x7f64080b2cb0 legacy=0x7f64080b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d9ea640 1 -- 172.21.15.60:0/755331324 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080d1c50 con 0x7f64080b92a0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f6407fff640 1 -- 172.21.15.60:0/755331324 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278219585 0 0) 0x7f64080b97a0 con 0x7f64080b2cb0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d1e9640 1 --2- 172.21.15.60:0/755331324 >> v2:172.21.15.100:3300/0 conn(0x7f640815dbe0 0x7f64080d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.183+0000 7f640d9ea640 1 --2- 172.21.15.60:0/755331324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080d0e50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f640806c140 tx=0x7f63f802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b4700e69829223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f6407fff640 1 -- 172.21.15.60:0/755331324 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8007740 con 0x7f64080b92a0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f6407fff640 1 -- 172.21.15.60:0/755331324 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f80078e0 con 0x7f64080b92a0 2024-04-25T08:35:40.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f6407fff640 1 -- 172.21.15.60:0/755331324 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8007cb0 con 0x7f64080b92a0 2024-04-25T08:35:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 -- 172.21.15.60:0/755331324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 msgr2=0x7f64080d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 --2- 172.21.15.60:0/755331324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080d0e50 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f640806c140 tx=0x7f63f802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 -- 172.21.15.60:0/755331324 shutdown_connections 2024-04-25T08:35:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 --2- 172.21.15.60:0/755331324 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080d0e50 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 --2- 172.21.15.60:0/755331324 >> v2:172.21.15.100:3300/0 conn(0x7f640815dbe0 0x7f64080d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 -- 172.21.15.60:0/755331324 >> 172.21.15.60:0/755331324 conn(0x7f6408061a20 msgr2=0x7f6408061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 -- 172.21.15.60:0/755331324 shutdown_connections 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.184+0000 7f640f44d640 1 -- 172.21.15.60:0/755331324 wait complete. 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.185+0000 7f640f44d640 1 Processor -- start 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.185+0000 7f640f44d640 1 -- start start 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640f44d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64080b2cb0 0x7f640814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640f44d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640f44d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640815dbe0 0x7f6408153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640f44d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b6ab0 con 0x7f64080b92a0 2024-04-25T08:35:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640f44d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080d2210 con 0x7f640815dbe0 2024-04-25T08:35:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640f44d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b6c30 con 0x7f64080b2cb0 2024-04-25T08:35:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640c9e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640c9e8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33366/0 (socket says 172.21.15.60:33366) 2024-04-25T08:35:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640c9e8640 1 -- 172.21.15.60:0/1935718364 learned_addr learned my addr 172.21.15.60:0/1935718364 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:40.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640d9ea640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640815dbe0 0x7f6408153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.186+0000 7f640d1e9640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64080b2cb0 0x7f640814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640c9e8640 1 -- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64080b2cb0 msgr2=0x7f640814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640c9e8640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64080b2cb0 0x7f640814cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640c9e8640 1 -- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640815dbe0 msgr2=0x7f6408153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640c9e8640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640815dbe0 0x7f6408153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640c9e8640 1 -- 172.21.15.60:0/1935718364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080b4e90 con 0x7f64080b92a0 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640d1e9640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64080b2cb0 0x7f640814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640c9e8640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080cd260 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c490 tx=0x7f63fc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc018070 con 0x7f64080b92a0 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0074e0 con 0x7f64080b92a0 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080b5150 con 0x7f64080b92a0 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004a20 con 0x7f64080b92a0 2024-04-25T08:35:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.187+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64080b5710 con 0x7f64080b92a0 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.188+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408073990 con 0x7f64080b92a0 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.188+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63fc01b070 con 0x7f64080b92a0 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.189+0000 7f6405ffb640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63f406b500 0x7f63f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.189+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f63fc089270 con 0x7f64080b92a0 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.189+0000 7f640d1e9640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63f406b500 0x7f63f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.190+0000 7f640d1e9640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63f406b500 0x7f63f406d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6400006fd0 tx=0x7f6400008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.193+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63fc057bd0 con 0x7f64080b92a0 2024-04-25T08:35:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.374+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f640814d2f0 con 0x7f64080b92a0 2024-04-25T08:35:40.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.374+0000 7f6405ffb640 1 -- 172.21.15.60:0/1935718364 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+65 (secure 0 0 0) 0x7f63fc05bc70 con 0x7f64080b92a0 2024-04-25T08:35:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:40.369 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:35:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:35:40.369 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:35:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:35:40.370 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:35:40.370 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63f406b500 msgr2=0x7f63f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63f406b500 0x7f63f406d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6400006fd0 tx=0x7f6400008040 comp rx=0 tx=0).stop 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 msgr2=0x7f64080cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080cd260 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c490 tx=0x7f63fc00c950 comp rx=0 tx=0).stop 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 shutdown_connections 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63f406b500 0x7f63f406d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640815dbe0 0x7f6408153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64080b92a0 0x7f64080cd260 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 --2- 172.21.15.60:0/1935718364 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64080b2cb0 0x7f640814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.376+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 >> 172.21.15.60:0/1935718364 conn(0x7f6408061a20 msgr2=0x7f64080b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.377+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 shutdown_connections 2024-04-25T08:35:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.377+0000 7f640f44d640 1 -- 172.21.15.60:0/1935718364 wait complete. 2024-04-25T08:35:40.382 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:35:40.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:35:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 Processor -- start 2024-04-25T08:35:40.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 -- start start 2024-04-25T08:35:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d815b7d0 0x7f32d80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f32d80b5620 0x7f32d80b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d806cf50 con 0x7f32d806c2d0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d80ce530 con 0x7f32d815b7d0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.493+0000 7f32ddcb3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d80ce810 con 0x7f32d80b5620 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d815b7d0 0x7f32d80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d815b7d0 0x7f32d80cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33382/0 (socket says 172.21.15.60:33382) 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 -- 172.21.15.60:0/2835734092 learned_addr learned my addr 172.21.15.60:0/2835734092 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d6ffd640 1 --2- 172.21.15.60:0/2835734092 >> v2:172.21.15.100:3300/0 conn(0x7f32d80b5620 0x7f32d80b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 -- 172.21.15.60:0/2835734092 >> v2:172.21.15.100:3300/0 conn(0x7f32d80b5620 msgr2=0x7f32d80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 --2- 172.21.15.60:0/2835734092 >> v2:172.21.15.100:3300/0 conn(0x7f32d80b5620 0x7f32d80b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 -- 172.21.15.60:0/2835734092 >> v1:172.21.15.100:6789/0 conn(0x7f32d806c2d0 legacy=0x7f32d806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.494+0000 7f32d77fe640 1 -- 172.21.15.60:0/2835734092 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d80ceaf0 con 0x7f32d815b7d0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32d77fe640 1 --2- 172.21.15.60:0/2835734092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d815b7d0 0x7f32d80cf010 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f32d80ce180 tx=0x7f32c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b565a89cf4addba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32d67fc640 1 -- 172.21.15.60:0/2835734092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8036070 con 0x7f32d815b7d0 2024-04-25T08:35:40.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32d67fc640 1 -- 172.21.15.60:0/2835734092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8031600 con 0x7f32d815b7d0 2024-04-25T08:35:40.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32d67fc640 1 -- 172.21.15.60:0/2835734092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80319d0 con 0x7f32d815b7d0 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 -- 172.21.15.60:0/2835734092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d815b7d0 msgr2=0x7f32d80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/2835734092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d815b7d0 0x7f32d80cf010 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f32d80ce180 tx=0x7f32c802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 -- 172.21.15.60:0/2835734092 shutdown_connections 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/2835734092 >> v2:172.21.15.100:3300/0 conn(0x7f32d80b5620 0x7f32d80b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/2835734092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d815b7d0 0x7f32d80cf010 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 -- 172.21.15.60:0/2835734092 >> 172.21.15.60:0/2835734092 conn(0x7f32d80d0800 msgr2=0x7f32d80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.495+0000 7f32ddcb3640 1 -- 172.21.15.60:0/2835734092 shutdown_connections 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.496+0000 7f32ddcb3640 1 -- 172.21.15.60:0/2835734092 wait complete. 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.497+0000 7f32ddcb3640 1 Processor -- start 2024-04-25T08:35:40.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.497+0000 7f32ddcb3640 1 -- start start 2024-04-25T08:35:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 0x7f32d80cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d80b5620 0x7f32d815d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d815b7d0 0x7f32d815d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80cf6d0 con 0x7f32d806c2d0 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80cf550 con 0x7f32d815b7d0 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d815bbb0 con 0x7f32d80b5620 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 0x7f32d80cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 0x7f32d80cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33394/0 (socket says 172.21.15.60:33394) 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 -- 172.21.15.60:0/4085960750 learned_addr learned my addr 172.21.15.60:0/4085960750 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d6ffd640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d80b5620 0x7f32d815d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d7fff640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d815b7d0 0x7f32d815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 -- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d80b5620 msgr2=0x7f32d815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d80b5620 0x7f32d815d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 -- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d815b7d0 msgr2=0x7f32d815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d815b7d0 0x7f32d815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 -- 172.21.15.60:0/4085960750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d80cb870 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32d77fe640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 0x7f32d80cb330 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009f40 tx=0x7f32c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8036050 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8042070 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d80cdc60 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8009510 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.498+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d80c8340 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.499+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d8073a00 con 0x7f32d806c2d0 2024-04-25T08:35:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.500+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32c8007c50 con 0x7f32d806c2d0 2024-04-25T08:35:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.500+0000 7f32cffff640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.500+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f32c80aafe0 con 0x7f32d806c2d0 2024-04-25T08:35:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.501+0000 7f32d6ffd640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.501+0000 7f32d6ffd640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f32d80c7520 tx=0x7f32c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.505+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32c807a030 con 0x7f32d806c2d0 2024-04-25T08:35:40.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:40.686+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f32d80b9470 con 0x7f32d806c2d0 2024-04-25T08:35:41.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.057+0000 7f32cffff640 1 -- 172.21.15.60:0/4085960750 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) v1 ==== 138+0+0 (secure 0 0 0) 0x7f32c8079930 con 0x7f32d806c2d0 2024-04-25T08:35:41.052 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32b806b4d0 msgr2=0x7f32b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f32d80c7520 tx=0x7f32c0008040 comp rx=0 tx=0).stop 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 msgr2=0x7f32d80cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 0x7f32d80cb330 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009f40 tx=0x7f32c8004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 shutdown_connections 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d815b7d0 0x7f32d815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d80b5620 0x7f32d815d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 --2- 172.21.15.60:0/4085960750 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d806c2d0 0x7f32d80cb330 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 >> 172.21.15.60:0/4085960750 conn(0x7f32d80d0800 msgr2=0x7f32d80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 shutdown_connections 2024-04-25T08:35:41.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.060+0000 7f32ddcb3640 1 -- 172.21.15.60:0/4085960750 wait complete. 2024-04-25T08:35:41.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:35:41.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 Processor -- start 2024-04-25T08:35:41.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 -- start start 2024-04-25T08:35:41.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2241610f0 0x7ff22406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 0x7ff224161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2240c6960 con 0x7ff2240ce0a0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2241614d0 con 0x7ff2241610f0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.185+0000 7ff22b03a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff224162260 con 0x7ff2241616f0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2295d7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff2240ce0a0 0x7ff2240d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:58476/0 (socket says 172.21.15.60:58476) 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2295d7640 1 -- 172.21.15.60:0/2736707223 learned_addr learned my addr 172.21.15.60:0/2736707223 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071968208 0 0) 0x7ff2240c6960 con 0x7ff2240ce0a0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff214003520 con 0x7ff2240ce0a0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff228dd6640 1 --2- 172.21.15.60:0/2736707223 >> v2:172.21.15.100:3300/0 conn(0x7ff2241610f0 0x7ff22406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff223fff640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 0x7ff224161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621862400 0 0) 0x7ff214003520 con 0x7ff2240ce0a0 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 msgr2=0x7ff224161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 0x7ff224161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 >> v2:172.21.15.100:3300/0 conn(0x7ff2241610f0 msgr2=0x7ff22406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 --2- 172.21.15.60:0/2736707223 >> v2:172.21.15.100:3300/0 conn(0x7ff2241610f0 0x7ff22406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224162540 con 0x7ff2240ce0a0 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ff21c0019f0 con 0x7ff2240ce0a0 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 >> v1:172.21.15.60:6789/0 conn(0x7ff2240ce0a0 legacy=0x7ff2240d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 0x7ff2140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff214003f30 0x7ff214004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2241614d0 con 0x7ff214003f30 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff224162260 con 0x7ff214004d40 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21400b9c0 con 0x7ff2241616f0 2024-04-25T08:35:41.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff223fff640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff214003f30 0x7ff214004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.186+0000 7ff228dd6640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 0x7ff2140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff2295d7640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff223fff640 1 -- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 msgr2=0x7ff2140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff223fff640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2241616f0 0x7ff2140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff223fff640 1 -- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff214004d40 msgr2=0x7ff2140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff223fff640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff223fff640 1 -- 172.21.15.60:0/2736707223 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214003520 con 0x7ff214003f30 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff2295d7640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff214004d40 0x7ff2140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.187+0000 7ff223fff640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff214003f30 0x7ff214004800 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ac0 tx=0x7ff214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90486791c1f005bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210036070 con 0x7ff214003f30 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210031400 con 0x7ff214003f30 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff2237fe640 1 -- 172.21.15.60:0/2736707223 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210009690 con 0x7ff214003f30 2024-04-25T08:35:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff22b03a640 1 -- 172.21.15.60:0/2736707223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff214003f30 msgr2=0x7ff214004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff22b03a640 1 --2- 172.21.15.60:0/2736707223 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff214003f30 0x7ff214004800 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ac0 tx=0x7ff214005830 comp rx=0 tx=0).stop 2024-04-25T08:35:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff2295d7640 1 -- 172.21.15.60:0/2736707223 reap_dead start 2024-04-25T08:35:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff22b03a640 1 -- 172.21.15.60:0/2736707223 shutdown_connections 2024-04-25T08:35:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff22b03a640 1 -- 172.21.15.60:0/2736707223 >> 172.21.15.60:0/2736707223 conn(0x7ff22406f500 msgr2=0x7ff22406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff22b03a640 1 -- 172.21.15.60:0/2736707223 shutdown_connections 2024-04-25T08:35:41.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.188+0000 7ff22b03a640 1 -- 172.21.15.60:0/2736707223 wait complete. 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.189+0000 7ff22b03a640 1 Processor -- start 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.189+0000 7ff22b03a640 1 -- start start 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff22b03a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff214003f30 0x7ff224158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff22b03a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff224155f60 0x7ff2241563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff22b03a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 0x7ff22415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff22b03a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240cf690 con 0x7ff22415dd40 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff22b03a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240cfce0 con 0x7ff224155f60 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff22b03a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22406c700 con 0x7ff214003f30 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff2295d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 0x7ff22415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff2295d7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 0x7ff22415e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33406/0 (socket says 172.21.15.60:33406) 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.190+0000 7ff2295d7640 1 -- 172.21.15.60:0/3124068405 learned_addr learned my addr 172.21.15.60:0/3124068405 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff228dd6640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff214003f30 0x7ff224158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff223fff640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff224155f60 0x7ff2241563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2295d7640 1 -- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff214003f30 msgr2=0x7ff224158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2295d7640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff214003f30 0x7ff224158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2295d7640 1 -- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff224155f60 msgr2=0x7ff2241563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2295d7640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff224155f60 0x7ff2241563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2295d7640 1 -- 172.21.15.60:0/3124068405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22415ea30 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2295d7640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 0x7ff22415e1c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff21c000f80 tx=0x7ff21c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c007a90 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c018070 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22415ed10 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c013420 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.191+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22415f2d0 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.192+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22406afe0 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.193+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff21c0135c0 con 0x7ff22415dd40 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.193+0000 7ff2217fa640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff20806b500 0x7ff20806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.193+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff21c08a590 con 0x7ff22415dd40 2024-04-25T08:35:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.193+0000 7ff228dd6640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff20806b500 0x7ff20806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.194+0000 7ff228dd6640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff20806b500 0x7ff20806d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff218009b80 tx=0x7ff218008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.198+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff22406afe0 con 0x7ff22415dd40 2024-04-25T08:35:41.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:41.377+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff224073a00 con 0x7ff22415dd40 2024-04-25T08:35:42.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.062+0000 7ff2217fa640 1 -- 172.21.15.60:0/3124068405 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff21c059460 con 0x7ff22415dd40 2024-04-25T08:35:42.057 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:35:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.065+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff20806b500 msgr2=0x7ff20806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.065+0000 7ff22b03a640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff20806b500 0x7ff20806d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff218009b80 tx=0x7ff218008040 comp rx=0 tx=0).stop 2024-04-25T08:35:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.065+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 msgr2=0x7ff22415e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.065+0000 7ff22b03a640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 0x7ff22415e1c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff21c000f80 tx=0x7ff21c00e530 comp rx=0 tx=0).stop 2024-04-25T08:35:42.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 shutdown_connections 2024-04-25T08:35:42.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff22415dd40 0x7ff22415e1c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff20806b500 0x7ff20806d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff224155f60 0x7ff2241563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 --2- 172.21.15.60:0/3124068405 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff214003f30 0x7ff224158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 >> 172.21.15.60:0/3124068405 conn(0x7ff22406f500 msgr2=0x7ff224071570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:42.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.066+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 shutdown_connections 2024-04-25T08:35:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.068+0000 7ff22b03a640 1 -- 172.21.15.60:0/3124068405 wait complete. 2024-04-25T08:35:42.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:35:42.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.204+0000 7fa0a6109640 1 Processor -- start 2024-04-25T08:35:42.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.204+0000 7fa0a6109640 1 -- start start 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa0a6109640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0a00b5620 0x7fa0a00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa0a6109640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa0a006c2d0 0x7fa0a006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa0a6109640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a006cc80 con 0x7fa0a015b7d0 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa0a6109640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a006cf50 con 0x7fa0a00b5620 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa0a6109640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a00ce530 con 0x7fa0a006c2d0 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0a00b5620 0x7fa0a00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0a00b5620 0x7fa0a00b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35640/0 (socket says 172.21.15.60:35640) 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 -- 172.21.15.60:0/1093510053 learned_addr learned my addr 172.21.15.60:0/1093510053 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:42.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09ffff640 1 --2- 172.21.15.60:0/1093510053 >> v2:172.21.15.100:3300/0 conn(0x7fa0a006c2d0 0x7fa0a006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 -- 172.21.15.60:0/1093510053 >> v2:172.21.15.100:3300/0 conn(0x7fa0a006c2d0 msgr2=0x7fa0a006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 --2- 172.21.15.60:0/1093510053 >> v2:172.21.15.100:3300/0 conn(0x7fa0a006c2d0 0x7fa0a006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 -- 172.21.15.60:0/1093510053 >> v1:172.21.15.100:6789/0 conn(0x7fa0a015b7d0 legacy=0x7fa0a00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.205+0000 7fa09effd640 1 -- 172.21.15.60:0/1093510053 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00ce810 con 0x7fa0a00b5620 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.206+0000 7fa09effd640 1 --2- 172.21.15.60:0/1093510053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a00b5ab0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0161890 tx=0x7fa09402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cd6809579df6ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.206+0000 7fa09e7fc640 1 -- 172.21.15.60:0/1093510053 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094037070 con 0x7fa0a00b5620 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.206+0000 7fa09e7fc640 1 -- 172.21.15.60:0/1093510053 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa094032440 con 0x7fa0a00b5620 2024-04-25T08:35:42.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.206+0000 7fa09e7fc640 1 -- 172.21.15.60:0/1093510053 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094032810 con 0x7fa0a00b5620 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 -- 172.21.15.60:0/1093510053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 msgr2=0x7fa0a00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 --2- 172.21.15.60:0/1093510053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a00b5ab0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0161890 tx=0x7fa09402a230 comp rx=0 tx=0).stop 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 -- 172.21.15.60:0/1093510053 shutdown_connections 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 --2- 172.21.15.60:0/1093510053 >> v2:172.21.15.100:3300/0 conn(0x7fa0a006c2d0 0x7fa0a006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 --2- 172.21.15.60:0/1093510053 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a00b5ab0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 -- 172.21.15.60:0/1093510053 >> 172.21.15.60:0/1093510053 conn(0x7fa0a00d0800 msgr2=0x7fa0a00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:42.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 -- 172.21.15.60:0/1093510053 shutdown_connections 2024-04-25T08:35:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.207+0000 7fa0a6109640 1 -- 172.21.15.60:0/1093510053 wait complete. 2024-04-25T08:35:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.208+0000 7fa0a6109640 1 Processor -- start 2024-04-25T08:35:42.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.208+0000 7fa0a6109640 1 -- start start 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa0a6109640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a006c2d0 0x7fa0a00cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa0a6109640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a01f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa0a6109640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a015b7d0 0x7fa0a00c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa0a6109640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a015bbb0 con 0x7fa0a00b5620 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa0a6109640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a006d230 con 0x7fa0a015b7d0 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa0a6109640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a015bd30 con 0x7fa0a006c2d0 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a01f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a01f0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35656/0 (socket says 172.21.15.60:35656) 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 -- 172.21.15.60:0/2930283613 learned_addr learned my addr 172.21.15.60:0/2930283613 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09f7fe640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a006c2d0 0x7fa0a00cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09ffff640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a015b7d0 0x7fa0a00c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 -- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a006c2d0 msgr2=0x7fa0a00cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a006c2d0 0x7fa0a00cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 -- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a015b7d0 msgr2=0x7fa0a00c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a015b7d0 0x7fa0a00c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09effd640 1 -- 172.21.15.60:0/2930283613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00ca360 con 0x7fa0a00b5620 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.209+0000 7fa09f7fe640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a006c2d0 0x7fa0a00cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.210+0000 7fa09effd640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a01f0a00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa09402a270 tx=0x7fa0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.210+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094041020 con 0x7fa0a00b5620 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.210+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0940306c0 con 0x7fa0a00b5620 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.210+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a00b7130 con 0x7fa0a00b5620 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.210+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094037040 con 0x7fa0a00b5620 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.210+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a01f0f40 con 0x7fa0a00b5620 2024-04-25T08:35:42.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.211+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0073a00 con 0x7fa0a00b5620 2024-04-25T08:35:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.212+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa094044020 con 0x7fa0a00b5620 2024-04-25T08:35:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.212+0000 7fa087fff640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa08c06b500 0x7fa08c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.212+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fa0940aa9c0 con 0x7fa0a00b5620 2024-04-25T08:35:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.212+0000 7fa09f7fe640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa08c06b500 0x7fa08c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.213+0000 7fa09f7fe640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa08c06b500 0x7fa08c06d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa090006fd0 tx=0x7fa090008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.217+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0940ace20 con 0x7fa0a00b5620 2024-04-25T08:35:42.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:42.435+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa0a015b490 con 0x7fa0a00b5620 2024-04-25T08:35:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.078+0000 7fa087fff640 1 -- 172.21.15.60:0/2930283613 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa09404b030 con 0x7fa0a00b5620 2024-04-25T08:35:43.073 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T08:35:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.088+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa08c06b500 msgr2=0x7fa08c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.088+0000 7fa0a6109640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa08c06b500 0x7fa08c06d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa090006fd0 tx=0x7fa090008040 comp rx=0 tx=0).stop 2024-04-25T08:35:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.088+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 msgr2=0x7fa0a01f0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.088+0000 7fa0a6109640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a01f0a00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa09402a270 tx=0x7fa0940046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.089+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 shutdown_connections 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.089+0000 7fa0a6109640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa08c06b500 0x7fa08c06d9c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.089+0000 7fa0a6109640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a015b7d0 0x7fa0a00c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.089+0000 7fa0a6109640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a00b5620 0x7fa0a01f0a00 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.089+0000 7fa0a6109640 1 --2- 172.21.15.60:0/2930283613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a006c2d0 0x7fa0a00cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.089+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 >> 172.21.15.60:0/2930283613 conn(0x7fa0a00d0800 msgr2=0x7fa0a00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.090+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 shutdown_connections 2024-04-25T08:35:43.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.090+0000 7fa0a6109640 1 -- 172.21.15.60:0/2930283613 wait complete. 2024-04-25T08:35:43.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:43.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.212+0000 7f78e1914640 1 Processor -- start 2024-04-25T08:35:43.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.212+0000 7f78e1914640 1 -- start start 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78e1914640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78dc15dbe0 0x7f78dc0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78e1914640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f78dc0b2cb0 0x7f78dc0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78e1914640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0b97a0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78e1914640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc15dfc0 con 0x7f78dc15dbe0 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78e1914640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0d1a00 con 0x7f78dc0b2cb0 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78daffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f78dc15dbe0 0x7f78dc0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78db7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f78dc0b92a0 0x7f78dc0b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56292/0 (socket says 172.21.15.60:56292) 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78db7fe640 1 -- 172.21.15.60:0/3729408539 learned_addr learned my addr 172.21.15.60:0/3729408539 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78da7fc640 1 --2- 172.21.15.60:0/3729408539 >> v2:172.21.15.100:3300/0 conn(0x7f78dc0b2cb0 0x7f78dc0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78daffd640 1 -- 172.21.15.60:0/3729408539 >> v2:172.21.15.100:3300/0 conn(0x7f78dc0b2cb0 msgr2=0x7f78dc0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78daffd640 1 --2- 172.21.15.60:0/3729408539 >> v2:172.21.15.100:3300/0 conn(0x7f78dc0b2cb0 0x7f78dc0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78daffd640 1 -- 172.21.15.60:0/3729408539 >> v1:172.21.15.60:6789/0 conn(0x7f78dc0b92a0 legacy=0x7f78dc0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78daffd640 1 -- 172.21.15.60:0/3729408539 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0d1c50 con 0x7f78dc15dbe0 2024-04-25T08:35:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.213+0000 7f78d9ffb640 1 -- 172.21.15.60:0/3729408539 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223036951 0 0) 0x7f78dc0b97a0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.214+0000 7f78daffd640 1 --2- 172.21.15.60:0/3729408539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0d0910 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f78dc06cff0 tx=0x7f78d402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d8a08bd4c71a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.214+0000 7f78d9ffb640 1 -- 172.21.15.60:0/3729408539 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4037070 con 0x7f78dc15dbe0 2024-04-25T08:35:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.214+0000 7f78d9ffb640 1 -- 172.21.15.60:0/3729408539 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d4032440 con 0x7f78dc15dbe0 2024-04-25T08:35:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.214+0000 7f78d9ffb640 1 -- 172.21.15.60:0/3729408539 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d402ae60 con 0x7f78dc15dbe0 2024-04-25T08:35:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 -- 172.21.15.60:0/3729408539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc15dbe0 msgr2=0x7f78dc0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 --2- 172.21.15.60:0/3729408539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0d0910 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f78dc06cff0 tx=0x7f78d402a110 comp rx=0 tx=0).stop 2024-04-25T08:35:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 -- 172.21.15.60:0/3729408539 shutdown_connections 2024-04-25T08:35:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 --2- 172.21.15.60:0/3729408539 >> v2:172.21.15.100:3300/0 conn(0x7f78dc0b2cb0 0x7f78dc0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 --2- 172.21.15.60:0/3729408539 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0d0910 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 -- 172.21.15.60:0/3729408539 >> 172.21.15.60:0/3729408539 conn(0x7f78dc061a20 msgr2=0x7f78dc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 -- 172.21.15.60:0/3729408539 shutdown_connections 2024-04-25T08:35:43.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.215+0000 7f78e1914640 1 -- 172.21.15.60:0/3729408539 wait complete. 2024-04-25T08:35:43.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.216+0000 7f78e1914640 1 Processor -- start 2024-04-25T08:35:43.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.216+0000 7f78e1914640 1 -- start start 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78e1914640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78dc0b2cb0 0x7f78dc153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78e1914640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 0x7f78dc0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78e1914640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78e1914640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0b6ab0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78e1914640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0d2210 con 0x7f78dc15dbe0 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78e1914640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0b6c30 con 0x7f78dc0b2cb0 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78da7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 0x7f78dc0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78da7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 0x7f78dc0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35662/0 (socket says 172.21.15.60:35662) 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78da7fc640 1 -- 172.21.15.60:0/3046776548 learned_addr learned my addr 172.21.15.60:0/3046776548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78daffd640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78dc0b2cb0 0x7f78dc153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.217+0000 7f78db7fe640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78da7fc640 1 -- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78dc0b2cb0 msgr2=0x7f78dc153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78da7fc640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78dc0b2cb0 0x7f78dc153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78da7fc640 1 -- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78dc15dbe0 msgr2=0x7f78dc0cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78da7fc640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78da7fc640 1 -- 172.21.15.60:0/3046776548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0cc2f0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78db7fe640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78da7fc640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 0x7f78dc0cb7a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f78c400d7d0 tx=0x7f78c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400b660 con 0x7f78dc0b92a0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4019070 con 0x7f78dc0b92a0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400ba60 con 0x7f78dc0b92a0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc0cc5d0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.218+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc0ccb90 con 0x7f78dc0b92a0 2024-04-25T08:35:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.219+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc073990 con 0x7f78dc0b92a0 2024-04-25T08:35:43.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.219+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78c400bc00 con 0x7f78dc0b92a0 2024-04-25T08:35:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.220+0000 7f78d3fff640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78bc06b500 0x7f78bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.220+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78c408a630 con 0x7f78dc0b92a0 2024-04-25T08:35:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.220+0000 7f78daffd640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78bc06b500 0x7f78bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.221+0000 7f78daffd640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78bc06b500 0x7f78bc06d9c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f78d4034000 tx=0x7f78d402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.224+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78c40593b0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.400 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.405+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78dc15b040 con 0x7f78dc0b92a0 2024-04-25T08:35:43.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.406+0000 7f78d3fff640 1 -- 172.21.15.60:0/3046776548 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f78c4058cb0 con 0x7f78dc0b92a0 2024-04-25T08:35:43.401 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:43.401 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":96,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:42.436510+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78bc06b500 msgr2=0x7f78bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78bc06b500 0x7f78bc06d9c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f78d4034000 tx=0x7f78d402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 msgr2=0x7f78dc0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 0x7f78dc0cb7a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f78c400d7d0 tx=0x7f78c400dc90 comp rx=0 tx=0).stop 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 shutdown_connections 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f78bc06b500 0x7f78bc06d9c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f78dc15dbe0 0x7f78dc0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78dc0b92a0 0x7f78dc0cb7a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 --2- 172.21.15.60:0/3046776548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f78dc0b2cb0 0x7f78dc153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 >> 172.21.15.60:0/3046776548 conn(0x7f78dc061a20 msgr2=0x7f78dc0d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 shutdown_connections 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.408+0000 7f78e1914640 1 -- 172.21.15.60:0/3046776548 wait complete. 2024-04-25T08:35:43.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.520+0000 7f1f1226f640 1 Processor -- start 2024-04-25T08:35:43.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.520+0000 7f1f1226f640 1 -- start start 2024-04-25T08:35:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1226f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1226f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f0c0d0ad0 0x7f1f0c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1226f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0d1df0 con 0x7f1f0c0b9770 2024-04-25T08:35:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1226f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0d20d0 con 0x7f1f0c0d0ad0 2024-04-25T08:35:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1226f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0d23b0 con 0x7f1f0c15dd30 2024-04-25T08:35:43.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1080c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1f0c0d0ad0 0x7f1f0c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f0b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1f0c0b9770 0x7f1f0c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56294/0 (socket says 172.21.15.60:56294) 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f0b7fe640 1 -- 172.21.15.60:0/1714797884 learned_addr learned my addr 172.21.15.60:0/1714797884 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f0bfff640 1 --2- 172.21.15.60:0/1714797884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1080c640 1 -- 172.21.15.60:0/1714797884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 msgr2=0x7f1f0c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1080c640 1 --2- 172.21.15.60:0/1714797884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1080c640 1 -- 172.21.15.60:0/1714797884 >> v1:172.21.15.60:6789/0 conn(0x7f1f0c0b9770 legacy=0x7f1f0c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.521+0000 7f1f1080c640 1 -- 172.21.15.60:0/1714797884 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0d2690 con 0x7f1f0c0d0ad0 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.522+0000 7f1f1080c640 1 --2- 172.21.15.60:0/1714797884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0ad0 0x7f1f0c0b6a40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009d90 tx=0x7f1f0402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f44a7bc4baea5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.522+0000 7f1f0affd640 1 -- 172.21.15.60:0/1714797884 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04031600 con 0x7f1f0c0d0ad0 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.522+0000 7f1f0affd640 1 -- 172.21.15.60:0/1714797884 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f040317a0 con 0x7f1f0c0d0ad0 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.522+0000 7f1f0affd640 1 -- 172.21.15.60:0/1714797884 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04031b70 con 0x7f1f0c0d0ad0 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 -- 172.21.15.60:0/1714797884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0ad0 msgr2=0x7f1f0c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 --2- 172.21.15.60:0/1714797884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0ad0 0x7f1f0c0b6a40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009d90 tx=0x7f1f0402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 -- 172.21.15.60:0/1714797884 shutdown_connections 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 --2- 172.21.15.60:0/1714797884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0d0ad0 0x7f1f0c0b6a40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 --2- 172.21.15.60:0/1714797884 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 -- 172.21.15.60:0/1714797884 >> 172.21.15.60:0/1714797884 conn(0x7f1f0c061c30 msgr2=0x7f1f0c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 -- 172.21.15.60:0/1714797884 shutdown_connections 2024-04-25T08:35:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.523+0000 7f1f1226f640 1 -- 172.21.15.60:0/1714797884 wait complete. 2024-04-25T08:35:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.524+0000 7f1f1226f640 1 Processor -- start 2024-04-25T08:35:43.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.524+0000 7f1f1226f640 1 -- start start 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f1226f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 0x7f1f0c1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f1226f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f0c0d0ad0 0x7f1f0c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f1226f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f1226f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0d1b90 con 0x7f1f0c0b9770 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f1226f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0d1a10 con 0x7f1f0c0d0ad0 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f1226f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0d2e90 con 0x7f1f0c15dd30 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 0x7f1f0c1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 0x7f1f0c1f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35676/0 (socket says 172.21.15.60:35676) 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 -- 172.21.15.60:0/2325999659 learned_addr learned my addr 172.21.15.60:0/2325999659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 -- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 msgr2=0x7f1f0c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 -- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f0c0d0ad0 msgr2=0x7f1f0c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f0c0d0ad0 0x7f1f0c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 -- 172.21.15.60:0/2325999659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0b38c0 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.525+0000 7f1f0bfff640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 0x7f1f0c1f4060 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400d5f0 tx=0x7f1ef400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.526+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40125f0 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.526+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4019070 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.526+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40129d0 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.526+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c0b3ba0 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.526+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c0b4160 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.527+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c073990 con 0x7f1f0c0b9770 2024-04-25T08:35:43.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.527+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ef4012b70 con 0x7f1f0c0b9770 2024-04-25T08:35:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.528+0000 7f1f08ff9640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1eec06b500 0x7f1eec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.528+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ef408a6e0 con 0x7f1f0c0b9770 2024-04-25T08:35:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.528+0000 7f1f0b7fe640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1eec06b500 0x7f1eec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.529+0000 7f1f0b7fe640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1eec06b500 0x7f1eec06d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1efc001000 tx=0x7f1efc0078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.532+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ef4059460 con 0x7f1f0c0b9770 2024-04-25T08:35:43.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.714+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f0c160970 con 0x7f1f0c0b9770 2024-04-25T08:35:43.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.715+0000 7f1f08ff9640 1 -- 172.21.15.60:0/2325999659 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f1ef4058d60 con 0x7f1f0c0b9770 2024-04-25T08:35:43.710 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:43.710 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":96,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:42.436510+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1eec06b500 msgr2=0x7f1eec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1eec06b500 0x7f1eec06d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1efc001000 tx=0x7f1efc0078f0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 msgr2=0x7f1f0c1f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 0x7f1f0c1f4060 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400d5f0 tx=0x7f1ef400dab0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 shutdown_connections 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1f0c15dd30 0x7f1f0c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1eec06b500 0x7f1eec06d9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f0c0d0ad0 0x7f1f0c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 --2- 172.21.15.60:0/2325999659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1f0c0b9770 0x7f1f0c1f4060 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 >> 172.21.15.60:0/2325999659 conn(0x7f1f0c061c30 msgr2=0x7f1f0c0c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 shutdown_connections 2024-04-25T08:35:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.717+0000 7f1f1226f640 1 -- 172.21.15.60:0/2325999659 wait complete. 2024-04-25T08:35:43.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:35:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.831+0000 7f1b29b63640 1 Processor -- start 2024-04-25T08:35:43.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.831+0000 7f1b29b63640 1 -- start start 2024-04-25T08:35:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b29b63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1b24161240 0x7f1b240b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b29b63640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b240cf1d0 con 0x7f1b24161890 2024-04-25T08:35:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b29b63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b24161620 con 0x7f1b240b54a0 2024-04-25T08:35:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b29b63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240b7120 con 0x7f1b24161240 2024-04-25T08:35:43.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b237fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1b24161240 0x7f1b240b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b22ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1b24161890 0x7f1b240ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56300/0 (socket says 172.21.15.60:56300) 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.832+0000 7f1b22ffd640 1 -- 172.21.15.60:0/1493017587 learned_addr learned my addr 172.21.15.60:0/1493017587 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.833+0000 7f1b237fe640 1 -- 172.21.15.60:0/1493017587 >> v1:172.21.15.100:6789/0 conn(0x7f1b240b54a0 legacy=0x7f1b240d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.833+0000 7f1b237fe640 1 -- 172.21.15.60:0/1493017587 >> v1:172.21.15.60:6789/0 conn(0x7f1b24161890 legacy=0x7f1b240ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.833+0000 7f1b227fc640 1 -- 172.21.15.60:0/1493017587 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256849127 0 0) 0x7f1b240cf1d0 con 0x7f1b24161890 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.833+0000 7f1b237fe640 1 -- 172.21.15.60:0/1493017587 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24161e00 con 0x7f1b24161240 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.833+0000 7f1b237fe640 1 --2- 172.21.15.60:0/1493017587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240b6be0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f1b140097a0 tx=0x7f1b1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31596c1f062dd3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b227fc640 1 -- 172.21.15.60:0/1493017587 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14037070 con 0x7f1b24161240 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b227fc640 1 -- 172.21.15.60:0/1493017587 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14032440 con 0x7f1b24161240 2024-04-25T08:35:43.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b227fc640 1 -- 172.21.15.60:0/1493017587 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14032810 con 0x7f1b24161240 2024-04-25T08:35:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 -- 172.21.15.60:0/1493017587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 msgr2=0x7f1b240b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1493017587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240b6be0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f1b140097a0 tx=0x7f1b1402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 -- 172.21.15.60:0/1493017587 shutdown_connections 2024-04-25T08:35:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1493017587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240b6be0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 -- 172.21.15.60:0/1493017587 >> 172.21.15.60:0/1493017587 conn(0x7f1b240d04a0 msgr2=0x7f1b240d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 -- 172.21.15.60:0/1493017587 shutdown_connections 2024-04-25T08:35:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.834+0000 7f1b29b63640 1 -- 172.21.15.60:0/1493017587 wait complete. 2024-04-25T08:35:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.835+0000 7f1b29b63640 1 Processor -- start 2024-04-25T08:35:43.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 -- start start 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b240b54a0 0x7f1b240ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b24161890 0x7f1b2415fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b241626a0 con 0x7f1b24161240 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b241623c0 con 0x7f1b24161890 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b29b63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240d15f0 con 0x7f1b240b54a0 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35696/0 (socket says 172.21.15.60:35696) 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b22ffd640 1 -- 172.21.15.60:0/1242853579 learned_addr learned my addr 172.21.15.60:0/1242853579 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b237fe640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b240b54a0 0x7f1b240ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.836+0000 7f1b23fff640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b24161890 0x7f1b2415fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b22ffd640 1 -- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b240b54a0 msgr2=0x7f1b240ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b22ffd640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b240b54a0 0x7f1b240ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b22ffd640 1 -- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b24161890 msgr2=0x7f1b2415fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b22ffd640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b24161890 0x7f1b2415fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b22ffd640 1 -- 172.21.15.60:0/1242853579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240c7ca0 con 0x7f1b24161240 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b22ffd640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240c7550 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c460 tx=0x7f1b1800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18018070 con 0x7f1b24161240 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b18007500 con 0x7f1b24161240 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18004a20 con 0x7f1b24161240 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b240c7f80 con 0x7f1b24161240 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.837+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b240c8540 con 0x7f1b24161240 2024-04-25T08:35:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.838+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b24073a00 con 0x7f1b24161240 2024-04-25T08:35:43.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.839+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b18004bc0 con 0x7f1b24161240 2024-04-25T08:35:43.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.839+0000 7f1b0bfff640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1b1006b4d0 0x7f1b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.839+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b18089690 con 0x7f1b24161240 2024-04-25T08:35:43.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.839+0000 7f1b237fe640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1b1006b4d0 0x7f1b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.840+0000 7f1b237fe640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1b1006b4d0 0x7f1b1006d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1b14034000 tx=0x7f1b1402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:43.843+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b18057c90 con 0x7f1b24161240 2024-04-25T08:35:44.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.063+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b2406d240 con 0x7f1b24161240 2024-04-25T08:35:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.080+0000 7f1b0bfff640 1 -- 172.21.15.60:0/1242853579 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1b1805bd30 con 0x7f1b24161240 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1b1006b4d0 msgr2=0x7f1b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1b1006b4d0 0x7f1b1006d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1b14034000 tx=0x7f1b1402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 msgr2=0x7f1b240c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240c7550 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c460 tx=0x7f1b1800c920 comp rx=0 tx=0).stop 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 shutdown_connections 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1b1006b4d0 0x7f1b1006d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b24161890 0x7f1b2415fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b24161240 0x7f1b240c7550 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 --2- 172.21.15.60:0/1242853579 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1b240b54a0 0x7f1b240ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 >> 172.21.15.60:0/1242853579 conn(0x7f1b240d04a0 msgr2=0x7f1b2406c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 shutdown_connections 2024-04-25T08:35:44.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.083+0000 7f1b29b63640 1 -- 172.21.15.60:0/1242853579 wait complete. 2024-04-25T08:35:44.089 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:35:44.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:35:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.199+0000 7f9183610640 1 Processor -- start 2024-04-25T08:35:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.199+0000 7f9183610640 1 -- start start 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9183610640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cf600 0x7f917c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9183610640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f917c0cbf20 0x7f917c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9183610640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0c75e0 con 0x7f917c15bb50 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9183610640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0c8680 con 0x7f917c0cbf20 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9183610640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0cff80 con 0x7f917c0cf600 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9181bad640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f917c0cbf20 0x7f917c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f91813ac640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f917c15bb50 0x7f917c0c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56316/0 (socket says 172.21.15.60:56316) 2024-04-25T08:35:44.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f91813ac640 1 -- 172.21.15.60:0/939485359 learned_addr learned my addr 172.21.15.60:0/939485359 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9180bab640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cf600 0x7f917c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9181bad640 1 -- 172.21.15.60:0/939485359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cf600 msgr2=0x7f917c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9181bad640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cf600 0x7f917c0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9181bad640 1 -- 172.21.15.60:0/939485359 >> v1:172.21.15.60:6789/0 conn(0x7f917c15bb50 legacy=0x7f917c0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f917bfff640 1 -- 172.21.15.60:0/939485359 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926339371 0 0) 0x7f917c0c75e0 con 0x7f917c15bb50 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9181bad640 1 -- 172.21.15.60:0/939485359 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0c78c0 con 0x7f917c0cbf20 2024-04-25T08:35:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.200+0000 7f9180bab640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cf600 0x7f917c0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f9181bad640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cbf20 0x7f917c0cc3b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f916c009d90 tx=0x7f916c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645b37812fc6bd79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f917bfff640 1 -- 172.21.15.60:0/939485359 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c007740 con 0x7f917c0cbf20 2024-04-25T08:35:44.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f917bfff640 1 -- 172.21.15.60:0/939485359 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c0078e0 con 0x7f917c0cbf20 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f917bfff640 1 -- 172.21.15.60:0/939485359 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c007cb0 con 0x7f917c0cbf20 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f9183610640 1 -- 172.21.15.60:0/939485359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cbf20 msgr2=0x7f917c0cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f9183610640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cbf20 0x7f917c0cc3b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f916c009d90 tx=0x7f916c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f9183610640 1 -- 172.21.15.60:0/939485359 shutdown_connections 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f9183610640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cbf20 0x7f917c0cc3b0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.201+0000 7f9183610640 1 --2- 172.21.15.60:0/939485359 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cf600 0x7f917c0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.202+0000 7f9183610640 1 -- 172.21.15.60:0/939485359 >> 172.21.15.60:0/939485359 conn(0x7f917c06f500 msgr2=0x7f917c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.202+0000 7f9183610640 1 -- 172.21.15.60:0/939485359 shutdown_connections 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.202+0000 7f9183610640 1 -- 172.21.15.60:0/939485359 wait complete. 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 Processor -- start 2024-04-25T08:35:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 -- start start 2024-04-25T08:35:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cbf20 0x7f917c147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 0x7f917c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f917c15bb50 0x7f917c156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0c8160 con 0x7f917c0cf600 2024-04-25T08:35:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0c7e80 con 0x7f917c15bb50 2024-04-25T08:35:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.203+0000 7f9183610640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0c72a0 con 0x7f917c0cbf20 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 0x7f917c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 0x7f917c0cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35714/0 (socket says 172.21.15.60:35714) 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 -- 172.21.15.60:0/3498478804 learned_addr learned my addr 172.21.15.60:0/3498478804 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 -- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cbf20 msgr2=0x7f917c147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cbf20 0x7f917c147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 -- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f917c15bb50 msgr2=0x7f917c156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f917c15bb50 0x7f917c156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 -- 172.21.15.60:0/3498478804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0ca360 con 0x7f917c0cf600 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9180bab640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 0x7f917c0cb120 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f917000c7f0 tx=0x7f917000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170015070 con 0x7f917c0cf600 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9170007a30 con 0x7f917c0cf600 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c0ca640 con 0x7f917c0cf600 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.204+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170007e00 con 0x7f917c0cf600 2024-04-25T08:35:44.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.205+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c0cb660 con 0x7f917c0cf600 2024-04-25T08:35:44.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.205+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c0b2790 con 0x7f917c0cf600 2024-04-25T08:35:44.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.206+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9170018070 con 0x7f917c0cf600 2024-04-25T08:35:44.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.206+0000 7f9179ffb640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f916806b4d0 0x7f916806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.206+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f917008a2b0 con 0x7f917c0cf600 2024-04-25T08:35:44.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.206+0000 7f91813ac640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f916806b4d0 0x7f916806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.207+0000 7f91813ac640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f916806b4d0 0x7f916806d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9174007fd0 tx=0x7f9174007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.211+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f917c0b2790 con 0x7f917c0cf600 2024-04-25T08:35:44.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:44.433+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f917c15f6d0 con 0x7f917c0cf600 2024-04-25T08:35:44.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T08:35:44.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T08:35:44.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T08:35:45.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.081+0000 7f9179ffb640 1 -- 172.21.15.60:0/3498478804 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7f917c15f6d0 con 0x7f917c0cf600 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f916806b4d0 msgr2=0x7f916806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f916806b4d0 0x7f916806d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9174007fd0 tx=0x7f9174007960 comp rx=0 tx=0).stop 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 msgr2=0x7f917c0cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 0x7f917c0cb120 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f917000c7f0 tx=0x7f917000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 shutdown_connections 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f916806b4d0 0x7f916806d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f917c15bb50 0x7f917c156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917c0cf600 0x7f917c0cb120 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 --2- 172.21.15.60:0/3498478804 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f917c0cbf20 0x7f917c147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 >> 172.21.15.60:0/3498478804 conn(0x7f917c06f500 msgr2=0x7f917c06fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 shutdown_connections 2024-04-25T08:35:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.084+0000 7f9183610640 1 -- 172.21.15.60:0/3498478804 wait complete. 2024-04-25T08:35:45.092 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:35:45.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:45.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 Processor -- start 2024-04-25T08:35:45.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 -- start start 2024-04-25T08:35:45.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f539815dd30 0x7f53980b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53980d0ad0 0x7f53980b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53980d1df0 con 0x7f53980b9770 2024-04-25T08:35:45.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53980d20d0 con 0x7f53980d0ad0 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.205+0000 7f539ce15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53980d23b0 con 0x7f539815dd30 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f5396d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53980d0ad0 0x7f53980b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f538dd9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f53980b9770 0x7f53980d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56320/0 (socket says 172.21.15.60:56320) 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f538dd9b640 1 -- 172.21.15.60:0/2956512944 learned_addr learned my addr 172.21.15.60:0/2956512944 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f539659c640 1 --2- 172.21.15.60:0/2956512944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f539815dd30 0x7f53980b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f5396d9d640 1 -- 172.21.15.60:0/2956512944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f539815dd30 msgr2=0x7f53980b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f5396d9d640 1 --2- 172.21.15.60:0/2956512944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f539815dd30 0x7f53980b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f5396d9d640 1 -- 172.21.15.60:0/2956512944 >> v1:172.21.15.60:6789/0 conn(0x7f53980b9770 legacy=0x7f53980d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f5395d9b640 1 -- 172.21.15.60:0/2956512944 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633673282 0 0) 0x7f53980d1df0 con 0x7f53980b9770 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.206+0000 7f5396d9d640 1 -- 172.21.15.60:0/2956512944 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53980d2690 con 0x7f53980d0ad0 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f5396d9d640 1 --2- 172.21.15.60:0/2956512944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53980d0ad0 0x7f53980b6a40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5390009d90 tx=0x7f539002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e1fe40406eecc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f5395d9b640 1 -- 172.21.15.60:0/2956512944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390031600 con 0x7f53980d0ad0 2024-04-25T08:35:45.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f5395d9b640 1 -- 172.21.15.60:0/2956512944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53900317a0 con 0x7f53980d0ad0 2024-04-25T08:35:45.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f5395d9b640 1 -- 172.21.15.60:0/2956512944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390031b70 con 0x7f53980d0ad0 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f539ce15640 1 -- 172.21.15.60:0/2956512944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53980d0ad0 msgr2=0x7f53980b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f539ce15640 1 --2- 172.21.15.60:0/2956512944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53980d0ad0 0x7f53980b6a40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5390009d90 tx=0x7f539002a5d0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f539ce15640 1 -- 172.21.15.60:0/2956512944 shutdown_connections 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f539ce15640 1 --2- 172.21.15.60:0/2956512944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53980d0ad0 0x7f53980b6a40 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f539ce15640 1 --2- 172.21.15.60:0/2956512944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f539815dd30 0x7f53980b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.207+0000 7f539ce15640 1 -- 172.21.15.60:0/2956512944 >> 172.21.15.60:0/2956512944 conn(0x7f5398061c30 msgr2=0x7f539806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.208+0000 7f539ce15640 1 -- 172.21.15.60:0/2956512944 shutdown_connections 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.208+0000 7f539ce15640 1 -- 172.21.15.60:0/2956512944 wait complete. 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.209+0000 7f539ce15640 1 Processor -- start 2024-04-25T08:35:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.209+0000 7f539ce15640 1 -- start start 2024-04-25T08:35:45.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539ce15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53980b9770 0x7f53981582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539ce15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53980d0ad0 0x7f5398159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539ce15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 0x7f53980b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539ce15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53980d1b90 con 0x7f539815dd30 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539ce15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53980d1a10 con 0x7f53980b9770 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539ce15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53980d2e90 con 0x7f53980d0ad0 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 0x7f53980b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 0x7f53980b3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35728/0 (socket says 172.21.15.60:35728) 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 -- 172.21.15.60:0/470524662 learned_addr learned my addr 172.21.15.60:0/470524662 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f539659c640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53980b9770 0x7f53981582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f538dd9b640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53980d0ad0 0x7f5398159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 -- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53980d0ad0 msgr2=0x7f5398159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53980d0ad0 0x7f5398159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 -- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53980b9770 msgr2=0x7f53981582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53980b9770 0x7f53981582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 -- 172.21.15.60:0/470524662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398158d40 con 0x7f539815dd30 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.210+0000 7f5396d9d640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 0x7f53980b3150 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5390009eb0 tx=0x7f5390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.211+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390040070 con 0x7f539815dd30 2024-04-25T08:35:45.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.211+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5390030b60 con 0x7f539815dd30 2024-04-25T08:35:45.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.211+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5398159020 con 0x7f539815dd30 2024-04-25T08:35:45.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.211+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390009450 con 0x7f539815dd30 2024-04-25T08:35:45.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.211+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53981595e0 con 0x7f539815dd30 2024-04-25T08:35:45.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.212+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5398073990 con 0x7f539815dd30 2024-04-25T08:35:45.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.212+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5390039040 con 0x7f539815dd30 2024-04-25T08:35:45.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.213+0000 7f538f7fe640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f537806b4d0 0x7f537806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.213+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53900ab9e0 con 0x7f539815dd30 2024-04-25T08:35:45.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.213+0000 7f539659c640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f537806b4d0 0x7f537806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.213+0000 7f539659c640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f537806b4d0 0x7f537806d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5388006fd0 tx=0x7f5388008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.217+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f539007a760 con 0x7f539815dd30 2024-04-25T08:35:45.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.438+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5398160790 con 0x7f539815dd30 2024-04-25T08:35:45.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.439+0000 7f538f7fe640 1 -- 172.21.15.60:0/470524662 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f539007a060 con 0x7f539815dd30 2024-04-25T08:35:45.434 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:45.434 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:35:45.434 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f537806b4d0 msgr2=0x7f537806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f537806b4d0 0x7f537806d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5388006fd0 tx=0x7f5388008040 comp rx=0 tx=0).stop 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 msgr2=0x7f53980b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 0x7f53980b3150 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5390009eb0 tx=0x7f5390004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 shutdown_connections 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f539815dd30 0x7f53980b3150 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53980d0ad0 0x7f5398159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f537806b4d0 0x7f537806d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 --2- 172.21.15.60:0/470524662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53980b9770 0x7f53981582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 >> 172.21.15.60:0/470524662 conn(0x7f5398061c30 msgr2=0x7f539815cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 shutdown_connections 2024-04-25T08:35:45.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.441+0000 7f539ce15640 1 -- 172.21.15.60:0/470524662 wait complete. 2024-04-25T08:35:45.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.553+0000 7f438202a640 1 Processor -- start 2024-04-25T08:35:45.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.553+0000 7f438202a640 1 -- start start 2024-04-25T08:35:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f438202a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 0x7f437c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f438202a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437c0b9940 con 0x7f437c15dd30 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f438202a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437c0d1df0 con 0x7f437c0d0900 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f438202a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0d0d70 con 0x7f437c0d2e90 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f437c15dd30 0x7f437c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56322/0 (socket says 172.21.15.60:56322) 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437b7fe640 1 -- 172.21.15.60:0/2573041768 learned_addr learned my addr 172.21.15.60:0/2573041768 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647405778 0 0) 0x7f437c0b9940 con 0x7f437c15dd30 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437affd640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 0x7f437c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4364003520 con 0x7f437c15dd30 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3164013303 0 0) 0x7f4364003520 con 0x7f437c15dd30 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 msgr2=0x7f437c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 0x7f437c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 >> v1:172.21.15.100:6789/0 conn(0x7f437c0d0900 legacy=0x7f437c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c0d20d0 con 0x7f437c15dd30 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f436c001670 con 0x7f437c15dd30 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 >> v1:172.21.15.60:6789/0 conn(0x7f437c15dd30 legacy=0x7f437c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43640039e0 0x7f4364003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.554+0000 7f437a7fc640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 0x7f4364005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0d0d70 con 0x7f43640039e0 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436400baf0 con 0x7f4364004800 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436400bcf0 con 0x7f437c0d2e90 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437b7fe640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43640039e0 0x7f4364003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437affd640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437bfff640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 0x7f4364005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437b7fe640 1 -- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 msgr2=0x7f4364005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437b7fe640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c0d2e90 0x7f4364005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437b7fe640 1 -- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4364004800 msgr2=0x7f4364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437b7fe640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437b7fe640 1 -- 172.21.15.60:0/2573041768 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364003520 con 0x7f43640039e0 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.555+0000 7f437affd640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4364004800 0x7f4364004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f437b7fe640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43640039e0 0x7f4364003e00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c240 tx=0x7f4364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f982dc1a1678bf96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c007d10 con 0x7f43640039e0 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c005ad0 con 0x7f43640039e0 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f437a7fc640 1 -- 172.21.15.60:0/2573041768 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c007070 con 0x7f43640039e0 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f438202a640 1 -- 172.21.15.60:0/2573041768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43640039e0 msgr2=0x7f4364003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f438202a640 1 --2- 172.21.15.60:0/2573041768 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43640039e0 0x7f4364003e00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c240 tx=0x7f4364005830 comp rx=0 tx=0).stop 2024-04-25T08:35:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f437bfff640 1 -- 172.21.15.60:0/2573041768 reap_dead start 2024-04-25T08:35:45.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f438202a640 1 -- 172.21.15.60:0/2573041768 shutdown_connections 2024-04-25T08:35:45.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f438202a640 1 -- 172.21.15.60:0/2573041768 >> 172.21.15.60:0/2573041768 conn(0x7f437c061c30 msgr2=0x7f437c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f438202a640 1 -- 172.21.15.60:0/2573041768 shutdown_connections 2024-04-25T08:35:45.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.556+0000 7f438202a640 1 -- 172.21.15.60:0/2573041768 wait complete. 2024-04-25T08:35:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.557+0000 7f438202a640 1 Processor -- start 2024-04-25T08:35:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 -- start start 2024-04-25T08:35:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43640039e0 0x7f437c149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 0x7f437c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c157030 0x7f437c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0d2970 con 0x7f437c152fe0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0d2690 con 0x7f43640039e0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f438202a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437c0d1a10 con 0x7f437c157030 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f437affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 0x7f437c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f437affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 0x7f437c0b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35758/0 (socket says 172.21.15.60:35758) 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.558+0000 7f437affd640 1 -- 172.21.15.60:0/1855644854 learned_addr learned my addr 172.21.15.60:0/1855644854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437bfff640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c157030 0x7f437c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437b7fe640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43640039e0 0x7f437c149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437affd640 1 -- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c157030 msgr2=0x7f437c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437affd640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c157030 0x7f437c1574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437affd640 1 -- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43640039e0 msgr2=0x7f437c149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437affd640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43640039e0 0x7f437c149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437affd640 1 -- 172.21.15.60:0/1855644854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c0b5770 con 0x7f437c152fe0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f437affd640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 0x7f437c0b4e40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f43700116a0 tx=0x7f4370011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370019070 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4370004220 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437c0b5a50 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4370005990 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.559+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437c154c30 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.560+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437c073990 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.560+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4370005b30 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.561+0000 7f4363fff640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f436806b4d0 0x7f436806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.561+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f437008a7e0 con 0x7f437c152fe0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.561+0000 7f437b7fe640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f436806b4d0 0x7f436806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.562+0000 7f437b7fe640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f436806b4d0 0x7f436806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f436c033000 tx=0x7f436c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.565+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4370059440 con 0x7f437c152fe0 2024-04-25T08:35:45.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.747+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f437c160fe0 con 0x7f437c152fe0 2024-04-25T08:35:45.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.748+0000 7f4363fff640 1 -- 172.21.15.60:0/1855644854 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f4370058d40 con 0x7f437c152fe0 2024-04-25T08:35:45.743 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:45.743 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":97,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:35:44.078138+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f436806b4d0 msgr2=0x7f436806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f436806b4d0 0x7f436806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f436c033000 tx=0x7f436c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 msgr2=0x7f437c0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 0x7f437c0b4e40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f43700116a0 tx=0x7f4370011b60 comp rx=0 tx=0).stop 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 shutdown_connections 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f437c157030 0x7f437c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f437c152fe0 0x7f437c0b4e40 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f436806b4d0 0x7f436806d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 --2- 172.21.15.60:0/1855644854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43640039e0 0x7f437c149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 >> 172.21.15.60:0/1855644854 conn(0x7f437c061c30 msgr2=0x7f437c15ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 shutdown_connections 2024-04-25T08:35:45.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.750+0000 7f438202a640 1 -- 172.21.15.60:0/1855644854 wait complete. 2024-04-25T08:35:45.756 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:35:45.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.862+0000 7f44b6607640 1 Processor -- start 2024-04-25T08:35:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.862+0000 7f44b6607640 1 -- start start 2024-04-25T08:35:45.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44b6607640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44b6607640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f44b00b9770 0x7f44b00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44b6607640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b00d1df0 con 0x7f44b00d2e90 2024-04-25T08:35:45.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44b6607640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b00b7a50 con 0x7f44b00b9770 2024-04-25T08:35:45.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44b6607640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00d20d0 con 0x7f44b015dd30 2024-04-25T08:35:45.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f44b00b9770 0x7f44b00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f44b00b9770 0x7f44b00d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35768/0 (socket says 172.21.15.60:35768) 2024-04-25T08:35:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 -- 172.21.15.60:0/2623798716 learned_addr learned my addr 172.21.15.60:0/2623798716 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 -- 172.21.15.60:0/2623798716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 msgr2=0x7f44b00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 --2- 172.21.15.60:0/2623798716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 -- 172.21.15.60:0/2623798716 >> v1:172.21.15.100:6789/0 conn(0x7f44b00d2e90 legacy=0x7f44b00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.863+0000 7f44af7fe640 1 -- 172.21.15.60:0/2623798716 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b00d23b0 con 0x7f44b00b9770 2024-04-25T08:35:45.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44af7fe640 1 --2- 172.21.15.60:0/2623798716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b00d03c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009850 tx=0x7f44a002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766250bd07bdcf4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44aeffd640 1 -- 172.21.15.60:0/2623798716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a002ae60 con 0x7f44b00b9770 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44aeffd640 1 -- 172.21.15.60:0/2623798716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a0004030 con 0x7f44b00b9770 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44aeffd640 1 -- 172.21.15.60:0/2623798716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0004460 con 0x7f44b00b9770 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44b6607640 1 -- 172.21.15.60:0/2623798716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 msgr2=0x7f44b00d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44b6607640 1 --2- 172.21.15.60:0/2623798716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b00d03c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009850 tx=0x7f44a002a150 comp rx=0 tx=0).stop 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44b6607640 1 -- 172.21.15.60:0/2623798716 shutdown_connections 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44b6607640 1 --2- 172.21.15.60:0/2623798716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b00d03c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.864+0000 7f44b6607640 1 --2- 172.21.15.60:0/2623798716 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b00b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.865+0000 7f44b6607640 1 -- 172.21.15.60:0/2623798716 >> 172.21.15.60:0/2623798716 conn(0x7f44b0061c30 msgr2=0x7f44b006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.865+0000 7f44b6607640 1 -- 172.21.15.60:0/2623798716 shutdown_connections 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.865+0000 7f44b6607640 1 -- 172.21.15.60:0/2623798716 wait complete. 2024-04-25T08:35:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.866+0000 7f44b6607640 1 Processor -- start 2024-04-25T08:35:45.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.866+0000 7f44b6607640 1 -- start start 2024-04-25T08:35:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.866+0000 7f44b6607640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b0153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b00d2e90 0x7f44b0158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b01f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00d1a10 con 0x7f44b00b9770 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00d2970 con 0x7f44b00d2e90 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00d1b90 con 0x7f44b015dd30 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b0153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b0153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35770/0 (socket says 172.21.15.60:35770) 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 -- 172.21.15.60:0/4008140673 learned_addr learned my addr 172.21.15.60:0/4008140673 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44af7fe640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b00d2e90 0x7f44b0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b4ba4640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b01f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 -- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 msgr2=0x7f44b01f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b01f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 -- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b00d2e90 msgr2=0x7f44b0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b00d2e90 0x7f44b0158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 -- 172.21.15.60:0/4008140673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0158fe0 con 0x7f44b00b9770 2024-04-25T08:35:45.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44affff640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b0153da0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f449800b410 tx=0x7f449800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498015020 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498011850 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b01592c0 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498011c40 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.867+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b01542e0 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.868+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b01f4e10 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.869+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4498018020 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.869+0000 7f44acff9640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f449006b4d0 0x7f449006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.869+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f449808a340 con 0x7f44b00b9770 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.870+0000 7f44af7fe640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f449006b4d0 0x7f449006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.870+0000 7f44af7fe640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f449006b4d0 0x7f449006d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f44a0033000 tx=0x7f44a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:45.874+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44b01f4e10 con 0x7f44b00b9770 2024-04-25T08:35:46.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.053+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44b00b7f00 con 0x7f449006b4d0 2024-04-25T08:35:46.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.059+0000 7f44acff9640 1 -- 172.21.15.60:0/4008140673 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+245857 (secure 0 0 0) 0x7f44b00b7f00 con 0x7f449006b4d0 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:46.059 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f449006b4d0 msgr2=0x7f449006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f449006b4d0 0x7f449006d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f44a0033000 tx=0x7f44a002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 msgr2=0x7f44b0153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b0153da0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f449800b410 tx=0x7f449800b8d0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 shutdown_connections 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f44b015dd30 0x7f44b01f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f449006b4d0 0x7f449006d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f44b00d2e90 0x7f44b0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 --2- 172.21.15.60:0/4008140673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f44b00b9770 0x7f44b0153da0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 >> 172.21.15.60:0/4008140673 conn(0x7f44b0061c30 msgr2=0x7f44b015e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 shutdown_connections 2024-04-25T08:35:46.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.062+0000 7f44b6607640 1 -- 172.21.15.60:0/4008140673 wait complete. 2024-04-25T08:35:46.068 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:35:45.485489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":128,"acting":128,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356932,"kb_used_data":10316,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754617788,"statfs":{"total":773094113280,"available":772728614912,"internally_reserved":0,"allocated":10563584,"data_stored":5502100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":354799152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.426505"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353876+0000","last_change":"2024-04-25T08:35:43.090294+0000","last_active":"2024-04-25T08:35:44.353876+0000","last_peered":"2024-04-25T08:35:44.353876+0000","last_clean":"2024-04-25T08:35:44.353876+0000","last_became_active":"2024-04-25T08:35:43.089694+0000","last_became_peered":"2024-04-25T08:35:43.089694+0000","last_unstale":"2024-04-25T08:35:44.353876+0000","last_undegraded":"2024-04-25T08:35:44.353876+0000","last_fullsized":"2024-04-25T08:35:44.353876+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353866+0000","last_change":"2024-04-25T08:35:43.081017+0000","last_active":"2024-04-25T08:35:44.353866+0000","last_peered":"2024-04-25T08:35:44.353866+0000","last_clean":"2024-04-25T08:35:44.353866+0000","last_became_active":"2024-04-25T08:35:43.080694+0000","last_became_peered":"2024-04-25T08:35:43.080694+0000","last_unstale":"2024-04-25T08:35:44.353866+0000","last_undegraded":"2024-04-25T08:35:44.353866+0000","last_fullsized":"2024-04-25T08:35:44.353866+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354842+0000","last_change":"2024-04-25T08:35:43.095431+0000","last_active":"2024-04-25T08:35:44.354842+0000","last_peered":"2024-04-25T08:35:44.354842+0000","last_clean":"2024-04-25T08:35:44.354842+0000","last_became_active":"2024-04-25T08:35:43.095148+0000","last_became_peered":"2024-04-25T08:35:43.095148+0000","last_unstale":"2024-04-25T08:35:44.354842+0000","last_undegraded":"2024-04-25T08:35:44.354842+0000","last_fullsized":"2024-04-25T08:35:44.354842+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083569+0000","last_change":"2024-04-25T08:35:43.080094+0000","last_active":"2024-04-25T08:35:44.083569+0000","last_peered":"2024-04-25T08:35:44.083569+0000","last_clean":"2024-04-25T08:35:44.083569+0000","last_became_active":"2024-04-25T08:35:43.079466+0000","last_became_peered":"2024-04-25T08:35:43.079466+0000","last_unstale":"2024-04-25T08:35:44.083569+0000","last_undegraded":"2024-04-25T08:35:44.083569+0000","last_fullsized":"2024-04-25T08:35:44.083569+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354780+0000","last_change":"2024-04-25T08:35:43.095408+0000","last_active":"2024-04-25T08:35:44.354780+0000","last_peered":"2024-04-25T08:35:44.354780+0000","last_clean":"2024-04-25T08:35:44.354780+0000","last_became_active":"2024-04-25T08:35:43.095119+0000","last_became_peered":"2024-04-25T08:35:43.095119+0000","last_unstale":"2024-04-25T08:35:44.354780+0000","last_undegraded":"2024-04-25T08:35:44.354780+0000","last_fullsized":"2024-04-25T08:35:44.354780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083849+0000","last_change":"2024-04-25T08:35:43.087871+0000","last_active":"2024-04-25T08:35:44.083849+0000","last_peered":"2024-04-25T08:35:44.083849+0000","last_clean":"2024-04-25T08:35:44.083849+0000","last_became_active":"2024-04-25T08:35:43.087575+0000","last_became_peered":"2024-04-25T08:35:43.087575+0000","last_unstale":"2024-04-25T08:35:44.083849+0000","last_undegraded":"2024-04-25T08:35:44.083849+0000","last_fullsized":"2024-04-25T08:35:44.083849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:32.414798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354147+0000","last_change":"2024-04-25T08:35:43.101307+0000","last_active":"2024-04-25T08:35:44.354147+0000","last_peered":"2024-04-25T08:35:44.354147+0000","last_clean":"2024-04-25T08:35:44.354147+0000","last_became_active":"2024-04-25T08:35:43.101003+0000","last_became_peered":"2024-04-25T08:35:43.101003+0000","last_unstale":"2024-04-25T08:35:44.354147+0000","last_undegraded":"2024-04-25T08:35:44.354147+0000","last_fullsized":"2024-04-25T08:35:44.354147+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:14.081591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084050+0000","last_change":"2024-04-25T08:35:43.079921+0000","last_active":"2024-04-25T08:35:44.084050+0000","last_peered":"2024-04-25T08:35:44.084050+0000","last_clean":"2024-04-25T08:35:44.084050+0000","last_became_active":"2024-04-25T08:35:43.079607+0000","last_became_peered":"2024-04-25T08:35:43.079607+0000","last_unstale":"2024-04-25T08:35:44.084050+0000","last_undegraded":"2024-04-25T08:35:44.084050+0000","last_fullsized":"2024-04-25T08:35:44.084050+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:57.986549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353925+0000","last_change":"2024-04-25T08:35:43.087944+0000","last_active":"2024-04-25T08:35:44.353925+0000","last_peered":"2024-04-25T08:35:44.353925+0000","last_clean":"2024-04-25T08:35:44.353925+0000","last_became_active":"2024-04-25T08:35:43.087638+0000","last_became_peered":"2024-04-25T08:35:43.087638+0000","last_unstale":"2024-04-25T08:35:44.353925+0000","last_undegraded":"2024-04-25T08:35:44.353925+0000","last_fullsized":"2024-04-25T08:35:44.353925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.692505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354540+0000","last_change":"2024-04-25T08:35:43.095160+0000","last_active":"2024-04-25T08:35:44.354540+0000","last_peered":"2024-04-25T08:35:44.354540+0000","last_clean":"2024-04-25T08:35:44.354540+0000","last_became_active":"2024-04-25T08:35:43.094865+0000","last_became_peered":"2024-04-25T08:35:43.094865+0000","last_unstale":"2024-04-25T08:35:44.354540+0000","last_undegraded":"2024-04-25T08:35:44.354540+0000","last_fullsized":"2024-04-25T08:35:44.354540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083498+0000","last_change":"2024-04-25T08:35:43.096325+0000","last_active":"2024-04-25T08:35:44.083498+0000","last_peered":"2024-04-25T08:35:44.083498+0000","last_clean":"2024-04-25T08:35:44.083498+0000","last_became_active":"2024-04-25T08:35:43.096042+0000","last_became_peered":"2024-04-25T08:35:43.096042+0000","last_unstale":"2024-04-25T08:35:44.083498+0000","last_undegraded":"2024-04-25T08:35:44.083498+0000","last_fullsized":"2024-04-25T08:35:44.083498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:27.748314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354752+0000","last_change":"2024-04-25T08:35:43.080738+0000","last_active":"2024-04-25T08:35:44.354752+0000","last_peered":"2024-04-25T08:35:44.354752+0000","last_clean":"2024-04-25T08:35:44.354752+0000","last_became_active":"2024-04-25T08:35:43.080444+0000","last_became_peered":"2024-04-25T08:35:43.080444+0000","last_unstale":"2024-04-25T08:35:44.354752+0000","last_undegraded":"2024-04-25T08:35:44.354752+0000","last_fullsized":"2024-04-25T08:35:44.354752+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354121+0000","last_change":"2024-04-25T08:35:43.089177+0000","last_active":"2024-04-25T08:35:44.354121+0000","last_peered":"2024-04-25T08:35:44.354121+0000","last_clean":"2024-04-25T08:35:44.354121+0000","last_became_active":"2024-04-25T08:35:43.088869+0000","last_became_peered":"2024-04-25T08:35:43.088869+0000","last_unstale":"2024-04-25T08:35:44.354121+0000","last_undegraded":"2024-04-25T08:35:44.354121+0000","last_fullsized":"2024-04-25T08:35:44.354121+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084198+0000","last_change":"2024-04-25T08:35:43.094595+0000","last_active":"2024-04-25T08:35:44.084198+0000","last_peered":"2024-04-25T08:35:44.084198+0000","last_clean":"2024-04-25T08:35:44.084198+0000","last_became_active":"2024-04-25T08:35:43.093864+0000","last_became_peered":"2024-04-25T08:35:43.093864+0000","last_unstale":"2024-04-25T08:35:44.084198+0000","last_undegraded":"2024-04-25T08:35:44.084198+0000","last_fullsized":"2024-04-25T08:35:44.084198+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354674+0000","last_change":"2024-04-25T08:35:43.086251+0000","last_active":"2024-04-25T08:35:44.354674+0000","last_peered":"2024-04-25T08:35:44.354674+0000","last_clean":"2024-04-25T08:35:44.354674+0000","last_became_active":"2024-04-25T08:35:43.085938+0000","last_became_peered":"2024-04-25T08:35:43.085938+0000","last_unstale":"2024-04-25T08:35:44.354674+0000","last_undegraded":"2024-04-25T08:35:44.354674+0000","last_fullsized":"2024-04-25T08:35:44.354674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084122+0000","last_change":"2024-04-25T08:35:43.084909+0000","last_active":"2024-04-25T08:35:44.084122+0000","last_peered":"2024-04-25T08:35:44.084122+0000","last_clean":"2024-04-25T08:35:44.084122+0000","last_became_active":"2024-04-25T08:35:43.084519+0000","last_became_peered":"2024-04-25T08:35:43.084519+0000","last_unstale":"2024-04-25T08:35:44.084122+0000","last_undegraded":"2024-04-25T08:35:44.084122+0000","last_fullsized":"2024-04-25T08:35:44.084122+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083572+0000","last_change":"2024-04-25T08:35:43.083871+0000","last_active":"2024-04-25T08:35:44.083572+0000","last_peered":"2024-04-25T08:35:44.083572+0000","last_clean":"2024-04-25T08:35:44.083572+0000","last_became_active":"2024-04-25T08:35:43.083574+0000","last_became_peered":"2024-04-25T08:35:43.083574+0000","last_unstale":"2024-04-25T08:35:44.083572+0000","last_undegraded":"2024-04-25T08:35:44.083572+0000","last_fullsized":"2024-04-25T08:35:44.083572+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.102694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354442+0000","last_change":"2024-04-25T08:35:43.080351+0000","last_active":"2024-04-25T08:35:44.354442+0000","last_peered":"2024-04-25T08:35:44.354442+0000","last_clean":"2024-04-25T08:35:44.354442+0000","last_became_active":"2024-04-25T08:35:43.080020+0000","last_became_peered":"2024-04-25T08:35:43.080020+0000","last_unstale":"2024-04-25T08:35:44.354442+0000","last_undegraded":"2024-04-25T08:35:44.354442+0000","last_fullsized":"2024-04-25T08:35:44.354442+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083931+0000","last_change":"2024-04-25T08:35:43.087490+0000","last_active":"2024-04-25T08:35:44.083931+0000","last_peered":"2024-04-25T08:35:44.083931+0000","last_clean":"2024-04-25T08:35:44.083931+0000","last_became_active":"2024-04-25T08:35:43.087039+0000","last_became_peered":"2024-04-25T08:35:43.087039+0000","last_unstale":"2024-04-25T08:35:44.083931+0000","last_undegraded":"2024-04-25T08:35:44.083931+0000","last_fullsized":"2024-04-25T08:35:44.083931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.866982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354143+0000","last_change":"2024-04-25T08:35:43.090435+0000","last_active":"2024-04-25T08:35:44.354143+0000","last_peered":"2024-04-25T08:35:44.354143+0000","last_clean":"2024-04-25T08:35:44.354143+0000","last_became_active":"2024-04-25T08:35:43.090166+0000","last_became_peered":"2024-04-25T08:35:43.090166+0000","last_unstale":"2024-04-25T08:35:44.354143+0000","last_undegraded":"2024-04-25T08:35:44.354143+0000","last_fullsized":"2024-04-25T08:35:44.354143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:10.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354827+0000","last_change":"2024-04-25T08:35:43.084313+0000","last_active":"2024-04-25T08:35:44.354827+0000","last_peered":"2024-04-25T08:35:44.354827+0000","last_clean":"2024-04-25T08:35:44.354827+0000","last_became_active":"2024-04-25T08:35:43.084027+0000","last_became_peered":"2024-04-25T08:35:43.084027+0000","last_unstale":"2024-04-25T08:35:44.354827+0000","last_undegraded":"2024-04-25T08:35:44.354827+0000","last_fullsized":"2024-04-25T08:35:44.354827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083847+0000","last_change":"2024-04-25T08:35:43.088502+0000","last_active":"2024-04-25T08:35:44.083847+0000","last_peered":"2024-04-25T08:35:44.083847+0000","last_clean":"2024-04-25T08:35:44.083847+0000","last_became_active":"2024-04-25T08:35:43.088156+0000","last_became_peered":"2024-04-25T08:35:43.088156+0000","last_unstale":"2024-04-25T08:35:44.083847+0000","last_undegraded":"2024-04-25T08:35:44.083847+0000","last_fullsized":"2024-04-25T08:35:44.083847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:29.004324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061987+0000","last_change":"2024-04-25T08:35:41.061987+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061987+0000","last_undegraded":"2024-04-25T08:35:41.061987+0000","last_fullsized":"2024-04-25T08:35:41.061987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084482+0000","last_change":"2024-04-25T08:35:42.077498+0000","last_active":"2024-04-25T08:35:44.084482+0000","last_peered":"2024-04-25T08:35:44.084482+0000","last_clean":"2024-04-25T08:35:44.084482+0000","last_became_active":"2024-04-25T08:35:42.077203+0000","last_became_peered":"2024-04-25T08:35:42.077203+0000","last_unstale":"2024-04-25T08:35:44.084482+0000","last_undegraded":"2024-04-25T08:35:44.084482+0000","last_fullsized":"2024-04-25T08:35:44.084482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063607+0000","last_change":"2024-04-25T08:35:41.063607+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063607+0000","last_undegraded":"2024-04-25T08:35:41.063607+0000","last_fullsized":"2024-04-25T08:35:41.063607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061876+0000","last_change":"2024-04-25T08:35:41.061876+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061876+0000","last_undegraded":"2024-04-25T08:35:41.061876+0000","last_fullsized":"2024-04-25T08:35:41.061876+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354942+0000","last_change":"2024-04-25T08:35:43.085489+0000","last_active":"2024-04-25T08:35:44.354942+0000","last_peered":"2024-04-25T08:35:44.354942+0000","last_clean":"2024-04-25T08:35:44.354942+0000","last_became_active":"2024-04-25T08:35:43.085149+0000","last_became_peered":"2024-04-25T08:35:43.085149+0000","last_unstale":"2024-04-25T08:35:44.354942+0000","last_undegraded":"2024-04-25T08:35:44.354942+0000","last_fullsized":"2024-04-25T08:35:44.354942+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083722+0000","last_change":"2024-04-25T08:35:42.083359+0000","last_active":"2024-04-25T08:35:44.083722+0000","last_peered":"2024-04-25T08:35:44.083722+0000","last_clean":"2024-04-25T08:35:44.083722+0000","last_became_active":"2024-04-25T08:35:42.083075+0000","last_became_peered":"2024-04-25T08:35:42.083075+0000","last_unstale":"2024-04-25T08:35:44.083722+0000","last_undegraded":"2024-04-25T08:35:44.083722+0000","last_fullsized":"2024-04-25T08:35:44.083722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:59.795889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354789+0000","last_change":"2024-04-25T08:35:42.087255+0000","last_active":"2024-04-25T08:35:44.354789+0000","last_peered":"2024-04-25T08:35:44.354789+0000","last_clean":"2024-04-25T08:35:44.354789+0000","last_became_active":"2024-04-25T08:35:42.086862+0000","last_became_peered":"2024-04-25T08:35:42.086862+0000","last_unstale":"2024-04-25T08:35:44.354789+0000","last_undegraded":"2024-04-25T08:35:44.354789+0000","last_fullsized":"2024-04-25T08:35:44.354789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.231173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354848+0000","last_change":"2024-04-25T08:35:42.078496+0000","last_active":"2024-04-25T08:35:44.354848+0000","last_peered":"2024-04-25T08:35:44.354848+0000","last_clean":"2024-04-25T08:35:44.354848+0000","last_became_active":"2024-04-25T08:35:42.078199+0000","last_became_peered":"2024-04-25T08:35:42.078199+0000","last_unstale":"2024-04-25T08:35:44.354848+0000","last_undegraded":"2024-04-25T08:35:44.354848+0000","last_fullsized":"2024-04-25T08:35:44.354848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354685+0000","last_change":"2024-04-25T08:35:43.096815+0000","last_active":"2024-04-25T08:35:44.354685+0000","last_peered":"2024-04-25T08:35:44.354685+0000","last_clean":"2024-04-25T08:35:44.354685+0000","last_became_active":"2024-04-25T08:35:43.096237+0000","last_became_peered":"2024-04-25T08:35:43.096237+0000","last_unstale":"2024-04-25T08:35:44.354685+0000","last_undegraded":"2024-04-25T08:35:44.354685+0000","last_fullsized":"2024-04-25T08:35:44.354685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063996+0000","last_change":"2024-04-25T08:35:41.063996+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063996+0000","last_undegraded":"2024-04-25T08:35:41.063996+0000","last_fullsized":"2024-04-25T08:35:41.063996+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083757+0000","last_change":"2024-04-25T08:35:43.084977+0000","last_active":"2024-04-25T08:35:44.083757+0000","last_peered":"2024-04-25T08:35:44.083757+0000","last_clean":"2024-04-25T08:35:44.083757+0000","last_became_active":"2024-04-25T08:35:43.084694+0000","last_became_peered":"2024-04-25T08:35:43.084694+0000","last_unstale":"2024-04-25T08:35:44.083757+0000","last_undegraded":"2024-04-25T08:35:44.083757+0000","last_fullsized":"2024-04-25T08:35:44.083757+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.121067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068599+0000","last_change":"2024-04-25T08:35:41.068599+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068599+0000","last_undegraded":"2024-04-25T08:35:41.068599+0000","last_fullsized":"2024-04-25T08:35:41.068599+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354368+0000","last_change":"2024-04-25T08:35:42.079015+0000","last_active":"2024-04-25T08:35:44.354368+0000","last_peered":"2024-04-25T08:35:44.354368+0000","last_clean":"2024-04-25T08:35:44.354368+0000","last_became_active":"2024-04-25T08:35:42.078722+0000","last_became_peered":"2024-04-25T08:35:42.078722+0000","last_unstale":"2024-04-25T08:35:44.354368+0000","last_undegraded":"2024-04-25T08:35:44.354368+0000","last_fullsized":"2024-04-25T08:35:44.354368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354270+0000","last_change":"2024-04-25T08:35:43.085992+0000","last_active":"2024-04-25T08:35:44.354270+0000","last_peered":"2024-04-25T08:35:44.354270+0000","last_clean":"2024-04-25T08:35:44.354270+0000","last_became_active":"2024-04-25T08:35:43.082970+0000","last_became_peered":"2024-04-25T08:35:43.082970+0000","last_unstale":"2024-04-25T08:35:44.354270+0000","last_undegraded":"2024-04-25T08:35:44.354270+0000","last_fullsized":"2024-04-25T08:35:44.354270+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084306+0000","last_change":"2024-04-25T08:35:42.085258+0000","last_active":"2024-04-25T08:35:44.084306+0000","last_peered":"2024-04-25T08:35:44.084306+0000","last_clean":"2024-04-25T08:35:44.084306+0000","last_became_active":"2024-04-25T08:35:42.084942+0000","last_became_peered":"2024-04-25T08:35:42.084942+0000","last_unstale":"2024-04-25T08:35:44.084306+0000","last_undegraded":"2024-04-25T08:35:44.084306+0000","last_fullsized":"2024-04-25T08:35:44.084306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:58.723955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354667+0000","last_change":"2024-04-25T08:35:43.094836+0000","last_active":"2024-04-25T08:35:44.354667+0000","last_peered":"2024-04-25T08:35:44.354667+0000","last_clean":"2024-04-25T08:35:44.354667+0000","last_became_active":"2024-04-25T08:35:43.093301+0000","last_became_peered":"2024-04-25T08:35:43.093301+0000","last_unstale":"2024-04-25T08:35:44.354667+0000","last_undegraded":"2024-04-25T08:35:44.354667+0000","last_fullsized":"2024-04-25T08:35:44.354667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355036+0000","last_change":"2024-04-25T08:35:43.095386+0000","last_active":"2024-04-25T08:35:44.355036+0000","last_peered":"2024-04-25T08:35:44.355036+0000","last_clean":"2024-04-25T08:35:44.355036+0000","last_became_active":"2024-04-25T08:35:43.093364+0000","last_became_peered":"2024-04-25T08:35:43.093364+0000","last_unstale":"2024-04-25T08:35:44.355036+0000","last_undegraded":"2024-04-25T08:35:44.355036+0000","last_fullsized":"2024-04-25T08:35:44.355036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063458+0000","last_change":"2024-04-25T08:35:41.063458+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063458+0000","last_undegraded":"2024-04-25T08:35:41.063458+0000","last_fullsized":"2024-04-25T08:35:41.063458+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:43.087418+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:43.087100+0000","last_became_peered":"2024-04-25T08:35:43.087100+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:23.384291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354284+0000","last_change":"2024-04-25T08:35:43.100537+0000","last_active":"2024-04-25T08:35:44.354284+0000","last_peered":"2024-04-25T08:35:44.354284+0000","last_clean":"2024-04-25T08:35:44.354284+0000","last_became_active":"2024-04-25T08:35:43.100246+0000","last_became_peered":"2024-04-25T08:35:43.100246+0000","last_unstale":"2024-04-25T08:35:44.354284+0000","last_undegraded":"2024-04-25T08:35:44.354284+0000","last_fullsized":"2024-04-25T08:35:44.354284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354726+0000","last_change":"2024-04-25T08:35:42.079064+0000","last_active":"2024-04-25T08:35:44.354726+0000","last_peered":"2024-04-25T08:35:44.354726+0000","last_clean":"2024-04-25T08:35:44.354726+0000","last_became_active":"2024-04-25T08:35:42.078692+0000","last_became_peered":"2024-04-25T08:35:42.078692+0000","last_unstale":"2024-04-25T08:35:44.354726+0000","last_undegraded":"2024-04-25T08:35:44.354726+0000","last_fullsized":"2024-04-25T08:35:44.354726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:03.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353823+0000","last_change":"2024-04-25T08:35:43.088607+0000","last_active":"2024-04-25T08:35:44.353823+0000","last_peered":"2024-04-25T08:35:44.353823+0000","last_clean":"2024-04-25T08:35:44.353823+0000","last_became_active":"2024-04-25T08:35:43.088317+0000","last_became_peered":"2024-04-25T08:35:43.088317+0000","last_unstale":"2024-04-25T08:35:44.353823+0000","last_undegraded":"2024-04-25T08:35:44.353823+0000","last_fullsized":"2024-04-25T08:35:44.353823+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:18.622878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355062+0000","last_change":"2024-04-25T08:35:42.086642+0000","last_active":"2024-04-25T08:35:44.355062+0000","last_peered":"2024-04-25T08:35:44.355062+0000","last_clean":"2024-04-25T08:35:44.355062+0000","last_became_active":"2024-04-25T08:35:42.086219+0000","last_became_peered":"2024-04-25T08:35:42.086219+0000","last_unstale":"2024-04-25T08:35:44.355062+0000","last_undegraded":"2024-04-25T08:35:44.355062+0000","last_fullsized":"2024-04-25T08:35:44.355062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:16.239961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.060959+0000","last_change":"2024-04-25T08:35:41.060959+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.060959+0000","last_undegraded":"2024-04-25T08:35:41.060959+0000","last_fullsized":"2024-04-25T08:35:41.060959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063514+0000","last_change":"2024-04-25T08:35:41.063514+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063514+0000","last_undegraded":"2024-04-25T08:35:41.063514+0000","last_fullsized":"2024-04-25T08:35:41.063514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084175+0000","last_change":"2024-04-25T08:35:43.092287+0000","last_active":"2024-04-25T08:35:44.084175+0000","last_peered":"2024-04-25T08:35:44.084175+0000","last_clean":"2024-04-25T08:35:44.084175+0000","last_became_active":"2024-04-25T08:35:43.091768+0000","last_became_peered":"2024-04-25T08:35:43.091768+0000","last_unstale":"2024-04-25T08:35:44.084175+0000","last_undegraded":"2024-04-25T08:35:44.084175+0000","last_fullsized":"2024-04-25T08:35:44.084175+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084367+0000","last_change":"2024-04-25T08:35:42.075127+0000","last_active":"2024-04-25T08:35:44.084367+0000","last_peered":"2024-04-25T08:35:44.084367+0000","last_clean":"2024-04-25T08:35:44.084367+0000","last_became_active":"2024-04-25T08:35:42.074842+0000","last_became_peered":"2024-04-25T08:35:42.074842+0000","last_unstale":"2024-04-25T08:35:44.084367+0000","last_undegraded":"2024-04-25T08:35:44.084367+0000","last_fullsized":"2024-04-25T08:35:44.084367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.370203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068005+0000","last_change":"2024-04-25T08:35:41.068005+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068005+0000","last_undegraded":"2024-04-25T08:35:41.068005+0000","last_fullsized":"2024-04-25T08:35:41.068005+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354254+0000","last_change":"2024-04-25T08:35:42.077041+0000","last_active":"2024-04-25T08:35:44.354254+0000","last_peered":"2024-04-25T08:35:44.354254+0000","last_clean":"2024-04-25T08:35:44.354254+0000","last_became_active":"2024-04-25T08:35:42.076737+0000","last_became_peered":"2024-04-25T08:35:42.076737+0000","last_unstale":"2024-04-25T08:35:44.354254+0000","last_undegraded":"2024-04-25T08:35:44.354254+0000","last_fullsized":"2024-04-25T08:35:44.354254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:54.769371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.067973+0000","last_change":"2024-04-25T08:35:41.067973+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.067973+0000","last_undegraded":"2024-04-25T08:35:41.067973+0000","last_fullsized":"2024-04-25T08:35:41.067973+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353713+0000","last_change":"2024-04-25T08:35:42.079469+0000","last_active":"2024-04-25T08:35:44.353713+0000","last_peered":"2024-04-25T08:35:44.353713+0000","last_clean":"2024-04-25T08:35:44.353713+0000","last_became_active":"2024-04-25T08:35:42.079187+0000","last_became_peered":"2024-04-25T08:35:42.079187+0000","last_unstale":"2024-04-25T08:35:44.353713+0000","last_undegraded":"2024-04-25T08:35:44.353713+0000","last_fullsized":"2024-04-25T08:35:44.353713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.436872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063728+0000","last_change":"2024-04-25T08:35:41.063728+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063728+0000","last_undegraded":"2024-04-25T08:35:41.063728+0000","last_fullsized":"2024-04-25T08:35:41.063728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088238+0000","last_change":"2024-04-25T08:35:42.080608+0000","last_active":"2024-04-25T08:35:44.088238+0000","last_peered":"2024-04-25T08:35:44.088238+0000","last_clean":"2024-04-25T08:35:44.088238+0000","last_became_active":"2024-04-25T08:35:42.080267+0000","last_became_peered":"2024-04-25T08:35:42.080267+0000","last_unstale":"2024-04-25T08:35:44.088238+0000","last_undegraded":"2024-04-25T08:35:44.088238+0000","last_fullsized":"2024-04-25T08:35:44.088238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:09.842643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084421+0000","last_change":"2024-04-25T08:35:42.084974+0000","last_active":"2024-04-25T08:35:44.084421+0000","last_peered":"2024-04-25T08:35:44.084421+0000","last_clean":"2024-04-25T08:35:44.084421+0000","last_became_active":"2024-04-25T08:35:42.084703+0000","last_became_peered":"2024-04-25T08:35:42.084703+0000","last_unstale":"2024-04-25T08:35:44.084421+0000","last_undegraded":"2024-04-25T08:35:44.084421+0000","last_fullsized":"2024-04-25T08:35:44.084421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:56.731886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061055+0000","last_change":"2024-04-25T08:35:41.061055+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061055+0000","last_undegraded":"2024-04-25T08:35:41.061055+0000","last_fullsized":"2024-04-25T08:35:41.061055+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353624+0000","last_change":"2024-04-25T08:35:43.089060+0000","last_active":"2024-04-25T08:35:44.353624+0000","last_peered":"2024-04-25T08:35:44.353624+0000","last_clean":"2024-04-25T08:35:44.353624+0000","last_became_active":"2024-04-25T08:35:43.088777+0000","last_became_peered":"2024-04-25T08:35:43.088777+0000","last_unstale":"2024-04-25T08:35:44.353624+0000","last_undegraded":"2024-04-25T08:35:44.353624+0000","last_fullsized":"2024-04-25T08:35:44.353624+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:58.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354681+0000","last_change":"2024-04-25T08:35:43.080706+0000","last_active":"2024-04-25T08:35:44.354681+0000","last_peered":"2024-04-25T08:35:44.354681+0000","last_clean":"2024-04-25T08:35:44.354681+0000","last_became_active":"2024-04-25T08:35:43.080411+0000","last_became_peered":"2024-04-25T08:35:43.080411+0000","last_unstale":"2024-04-25T08:35:44.354681+0000","last_undegraded":"2024-04-25T08:35:44.354681+0000","last_fullsized":"2024-04-25T08:35:44.354681+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068003+0000","last_change":"2024-04-25T08:35:41.068003+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068003+0000","last_undegraded":"2024-04-25T08:35:41.068003+0000","last_fullsized":"2024-04-25T08:35:41.068003+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.060868+0000","last_change":"2024-04-25T08:35:41.060868+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.060868+0000","last_undegraded":"2024-04-25T08:35:41.060868+0000","last_fullsized":"2024-04-25T08:35:41.060868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063500+0000","last_change":"2024-04-25T08:35:41.063500+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063500+0000","last_undegraded":"2024-04-25T08:35:41.063500+0000","last_fullsized":"2024-04-25T08:35:41.063500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354943+0000","last_change":"2024-04-25T08:35:42.080661+0000","last_active":"2024-04-25T08:35:44.354943+0000","last_peered":"2024-04-25T08:35:44.354943+0000","last_clean":"2024-04-25T08:35:44.354943+0000","last_became_active":"2024-04-25T08:35:42.080378+0000","last_became_peered":"2024-04-25T08:35:42.080378+0000","last_unstale":"2024-04-25T08:35:44.354943+0000","last_undegraded":"2024-04-25T08:35:44.354943+0000","last_fullsized":"2024-04-25T08:35:44.354943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:11.863733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":309,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.064192+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:35:41.064192+0000","last_peered":"2024-04-25T08:35:41.064192+0000","last_clean":"2024-04-25T08:35:41.064192+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:35:41.064192+0000","last_undegraded":"2024-04-25T08:35:41.064192+0000","last_fullsized":"2024-04-25T08:35:41.064192+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":58,"acting":58,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41196,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330644,"statfs":{"total":96636764160,"available":96594579456,"internally_reserved":0,"allocated":1736704,"data_stored":1115590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1155072,"data_stored":531431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1155072,"data_stored":531431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":552117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50392,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321448,"statfs":{"total":96636764160,"available":96585162752,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54556,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317284,"statfs":{"total":96636764160,"available":96580898816,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":552027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55084,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316756,"statfs":{"total":96636764160,"available":96580358144,"internally_reserved":0,"allocated":1736704,"data_stored":1115590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:46.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.215+0000 7f3ac5e58640 1 Processor -- start 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.215+0000 7f3ac5e58640 1 -- start start 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3ac5e58640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ac015dd30 0x7f3ac00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3ac5e58640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3ac00b9770 0x7f3ac00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3ac5e58640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac00d1df0 con 0x7f3ac00d2e90 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3ac5e58640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00b7a50 con 0x7f3ac015dd30 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3ac5e58640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00d20d0 con 0x7f3ac00b9770 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ac015dd30 0x7f3ac00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3ac00d2e90 0x7f3ac00b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56328/0 (socket says 172.21.15.60:56328) 2024-04-25T08:35:46.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abffff640 1 -- 172.21.15.60:0/343484267 learned_addr learned my addr 172.21.15.60:0/343484267 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abeffd640 1 --2- 172.21.15.60:0/343484267 >> v2:172.21.15.100:3300/0 conn(0x7f3ac00b9770 0x7f3ac00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abf7fe640 1 -- 172.21.15.60:0/343484267 >> v2:172.21.15.100:3300/0 conn(0x7f3ac00b9770 msgr2=0x7f3ac00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abf7fe640 1 --2- 172.21.15.60:0/343484267 >> v2:172.21.15.100:3300/0 conn(0x7f3ac00b9770 0x7f3ac00d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abf7fe640 1 -- 172.21.15.60:0/343484267 >> v1:172.21.15.60:6789/0 conn(0x7f3ac00d2e90 legacy=0x7f3ac00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abe7fc640 1 -- 172.21.15.60:0/343484267 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042762777 0 0) 0x7f3ac00d1df0 con 0x7f3ac00d2e90 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.216+0000 7f3abf7fe640 1 -- 172.21.15.60:0/343484267 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac00d23b0 con 0x7f3ac015dd30 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.217+0000 7f3abf7fe640 1 --2- 172.21.15.60:0/343484267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00b9230 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00b6540 tx=0x7f3ab002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3066676c64bec59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.217+0000 7f3abe7fc640 1 -- 172.21.15.60:0/343484267 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0037070 con 0x7f3ac015dd30 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.217+0000 7f3abe7fc640 1 -- 172.21.15.60:0/343484267 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab002add0 con 0x7f3ac015dd30 2024-04-25T08:35:46.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.217+0000 7f3abe7fc640 1 -- 172.21.15.60:0/343484267 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0032660 con 0x7f3ac015dd30 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 -- 172.21.15.60:0/343484267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 msgr2=0x7f3ac00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/343484267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00b9230 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00b6540 tx=0x7f3ab002a080 comp rx=0 tx=0).stop 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 -- 172.21.15.60:0/343484267 shutdown_connections 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/343484267 >> v2:172.21.15.100:3300/0 conn(0x7f3ac00b9770 0x7f3ac00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/343484267 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00b9230 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 -- 172.21.15.60:0/343484267 >> 172.21.15.60:0/343484267 conn(0x7f3ac0061c30 msgr2=0x7f3ac006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 -- 172.21.15.60:0/343484267 shutdown_connections 2024-04-25T08:35:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.218+0000 7f3ac5e58640 1 -- 172.21.15.60:0/343484267 wait complete. 2024-04-25T08:35:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.219+0000 7f3ac5e58640 1 Processor -- start 2024-04-25T08:35:46.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.219+0000 7f3ac5e58640 1 -- start start 2024-04-25T08:35:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3ac5e58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac00b9770 0x7f3ac0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3ac5e58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ac00d2e90 0x7f3ac00cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3ac5e58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3ac5e58640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00d1a10 con 0x7f3ac015dd30 2024-04-25T08:35:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3ac5e58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00d2970 con 0x7f3ac00b9770 2024-04-25T08:35:46.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3ac5e58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00d1b90 con 0x7f3ac00d2e90 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35782/0 (socket says 172.21.15.60:35782) 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 -- 172.21.15.60:0/1777108213 learned_addr learned my addr 172.21.15.60:0/1777108213 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abf7fe640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac00b9770 0x7f3ac0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abeffd640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ac00d2e90 0x7f3ac00cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 -- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ac00d2e90 msgr2=0x7f3ac00cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ac00d2e90 0x7f3ac00cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 -- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac00b9770 msgr2=0x7f3ac0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac00b9770 0x7f3ac0151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abffff640 1 -- 172.21.15.60:0/1777108213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0151840 con 0x7f3ac015dd30 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.220+0000 7f3abf7fe640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac00b9770 0x7f3ac0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.221+0000 7f3abffff640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00cd3d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c7d0 tx=0x7f3ab800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.221+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8018070 con 0x7f3ac015dd30 2024-04-25T08:35:46.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.221+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab800a640 con 0x7f3ac015dd30 2024-04-25T08:35:46.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.221+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab800aa70 con 0x7f3ac015dd30 2024-04-25T08:35:46.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.221+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac01f4d50 con 0x7f3ac015dd30 2024-04-25T08:35:46.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.221+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac015ff60 con 0x7f3ac015dd30 2024-04-25T08:35:46.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.222+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac0073990 con 0x7f3ac015dd30 2024-04-25T08:35:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.222+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ab801b070 con 0x7f3ac015dd30 2024-04-25T08:35:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.223+0000 7f3aa7fff640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.223+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ab80898b0 con 0x7f3ac015dd30 2024-04-25T08:35:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.223+0000 7f3abf7fe640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.224+0000 7f3abf7fe640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0034000 tx=0x7f3ab002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.227+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ab8058630 con 0x7f3ac015dd30 2024-04-25T08:35:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.409+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ac00b8750 con 0x7f3aac06b4d0 2024-04-25T08:35:46.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.415+0000 7f3aa7fff640 1 -- 172.21.15.60:0/1777108213 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+245857 (secure 0 0 0) 0x7f3ac00b8750 con 0x7f3aac06b4d0 2024-04-25T08:35:46.413 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:46.414 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:35:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 msgr2=0x7f3aac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0034000 tx=0x7f3ab002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 msgr2=0x7f3ac00cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00cd3d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c7d0 tx=0x7f3ab800cc90 comp rx=0 tx=0).stop 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 shutdown_connections 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac015dd30 0x7f3ac00cd3d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ac00d2e90 0x7f3ac00cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3aac06b4d0 0x7f3aac06d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 --2- 172.21.15.60:0/1777108213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ac00b9770 0x7f3ac0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.418+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 >> 172.21.15.60:0/1777108213 conn(0x7f3ac0061c30 msgr2=0x7f3ac00d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.419+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 shutdown_connections 2024-04-25T08:35:46.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.419+0000 7f3ac5e58640 1 -- 172.21.15.60:0/1777108213 wait complete. 2024-04-25T08:35:46.424 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:35:45.485489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":128,"acting":128,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356932,"kb_used_data":10316,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754617788,"statfs":{"total":773094113280,"available":772728614912,"internally_reserved":0,"allocated":10563584,"data_stored":5502100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":354799152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.426505"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353876+0000","last_change":"2024-04-25T08:35:43.090294+0000","last_active":"2024-04-25T08:35:44.353876+0000","last_peered":"2024-04-25T08:35:44.353876+0000","last_clean":"2024-04-25T08:35:44.353876+0000","last_became_active":"2024-04-25T08:35:43.089694+0000","last_became_peered":"2024-04-25T08:35:43.089694+0000","last_unstale":"2024-04-25T08:35:44.353876+0000","last_undegraded":"2024-04-25T08:35:44.353876+0000","last_fullsized":"2024-04-25T08:35:44.353876+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353866+0000","last_change":"2024-04-25T08:35:43.081017+0000","last_active":"2024-04-25T08:35:44.353866+0000","last_peered":"2024-04-25T08:35:44.353866+0000","last_clean":"2024-04-25T08:35:44.353866+0000","last_became_active":"2024-04-25T08:35:43.080694+0000","last_became_peered":"2024-04-25T08:35:43.080694+0000","last_unstale":"2024-04-25T08:35:44.353866+0000","last_undegraded":"2024-04-25T08:35:44.353866+0000","last_fullsized":"2024-04-25T08:35:44.353866+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354842+0000","last_change":"2024-04-25T08:35:43.095431+0000","last_active":"2024-04-25T08:35:44.354842+0000","last_peered":"2024-04-25T08:35:44.354842+0000","last_clean":"2024-04-25T08:35:44.354842+0000","last_became_active":"2024-04-25T08:35:43.095148+0000","last_became_peered":"2024-04-25T08:35:43.095148+0000","last_unstale":"2024-04-25T08:35:44.354842+0000","last_undegraded":"2024-04-25T08:35:44.354842+0000","last_fullsized":"2024-04-25T08:35:44.354842+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083569+0000","last_change":"2024-04-25T08:35:43.080094+0000","last_active":"2024-04-25T08:35:44.083569+0000","last_peered":"2024-04-25T08:35:44.083569+0000","last_clean":"2024-04-25T08:35:44.083569+0000","last_became_active":"2024-04-25T08:35:43.079466+0000","last_became_peered":"2024-04-25T08:35:43.079466+0000","last_unstale":"2024-04-25T08:35:44.083569+0000","last_undegraded":"2024-04-25T08:35:44.083569+0000","last_fullsized":"2024-04-25T08:35:44.083569+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354780+0000","last_change":"2024-04-25T08:35:43.095408+0000","last_active":"2024-04-25T08:35:44.354780+0000","last_peered":"2024-04-25T08:35:44.354780+0000","last_clean":"2024-04-25T08:35:44.354780+0000","last_became_active":"2024-04-25T08:35:43.095119+0000","last_became_peered":"2024-04-25T08:35:43.095119+0000","last_unstale":"2024-04-25T08:35:44.354780+0000","last_undegraded":"2024-04-25T08:35:44.354780+0000","last_fullsized":"2024-04-25T08:35:44.354780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083849+0000","last_change":"2024-04-25T08:35:43.087871+0000","last_active":"2024-04-25T08:35:44.083849+0000","last_peered":"2024-04-25T08:35:44.083849+0000","last_clean":"2024-04-25T08:35:44.083849+0000","last_became_active":"2024-04-25T08:35:43.087575+0000","last_became_peered":"2024-04-25T08:35:43.087575+0000","last_unstale":"2024-04-25T08:35:44.083849+0000","last_undegraded":"2024-04-25T08:35:44.083849+0000","last_fullsized":"2024-04-25T08:35:44.083849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:32.414798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354147+0000","last_change":"2024-04-25T08:35:43.101307+0000","last_active":"2024-04-25T08:35:44.354147+0000","last_peered":"2024-04-25T08:35:44.354147+0000","last_clean":"2024-04-25T08:35:44.354147+0000","last_became_active":"2024-04-25T08:35:43.101003+0000","last_became_peered":"2024-04-25T08:35:43.101003+0000","last_unstale":"2024-04-25T08:35:44.354147+0000","last_undegraded":"2024-04-25T08:35:44.354147+0000","last_fullsized":"2024-04-25T08:35:44.354147+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:14.081591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084050+0000","last_change":"2024-04-25T08:35:43.079921+0000","last_active":"2024-04-25T08:35:44.084050+0000","last_peered":"2024-04-25T08:35:44.084050+0000","last_clean":"2024-04-25T08:35:44.084050+0000","last_became_active":"2024-04-25T08:35:43.079607+0000","last_became_peered":"2024-04-25T08:35:43.079607+0000","last_unstale":"2024-04-25T08:35:44.084050+0000","last_undegraded":"2024-04-25T08:35:44.084050+0000","last_fullsized":"2024-04-25T08:35:44.084050+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:57.986549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353925+0000","last_change":"2024-04-25T08:35:43.087944+0000","last_active":"2024-04-25T08:35:44.353925+0000","last_peered":"2024-04-25T08:35:44.353925+0000","last_clean":"2024-04-25T08:35:44.353925+0000","last_became_active":"2024-04-25T08:35:43.087638+0000","last_became_peered":"2024-04-25T08:35:43.087638+0000","last_unstale":"2024-04-25T08:35:44.353925+0000","last_undegraded":"2024-04-25T08:35:44.353925+0000","last_fullsized":"2024-04-25T08:35:44.353925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.692505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354540+0000","last_change":"2024-04-25T08:35:43.095160+0000","last_active":"2024-04-25T08:35:44.354540+0000","last_peered":"2024-04-25T08:35:44.354540+0000","last_clean":"2024-04-25T08:35:44.354540+0000","last_became_active":"2024-04-25T08:35:43.094865+0000","last_became_peered":"2024-04-25T08:35:43.094865+0000","last_unstale":"2024-04-25T08:35:44.354540+0000","last_undegraded":"2024-04-25T08:35:44.354540+0000","last_fullsized":"2024-04-25T08:35:44.354540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083498+0000","last_change":"2024-04-25T08:35:43.096325+0000","last_active":"2024-04-25T08:35:44.083498+0000","last_peered":"2024-04-25T08:35:44.083498+0000","last_clean":"2024-04-25T08:35:44.083498+0000","last_became_active":"2024-04-25T08:35:43.096042+0000","last_became_peered":"2024-04-25T08:35:43.096042+0000","last_unstale":"2024-04-25T08:35:44.083498+0000","last_undegraded":"2024-04-25T08:35:44.083498+0000","last_fullsized":"2024-04-25T08:35:44.083498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:27.748314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354752+0000","last_change":"2024-04-25T08:35:43.080738+0000","last_active":"2024-04-25T08:35:44.354752+0000","last_peered":"2024-04-25T08:35:44.354752+0000","last_clean":"2024-04-25T08:35:44.354752+0000","last_became_active":"2024-04-25T08:35:43.080444+0000","last_became_peered":"2024-04-25T08:35:43.080444+0000","last_unstale":"2024-04-25T08:35:44.354752+0000","last_undegraded":"2024-04-25T08:35:44.354752+0000","last_fullsized":"2024-04-25T08:35:44.354752+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354121+0000","last_change":"2024-04-25T08:35:43.089177+0000","last_active":"2024-04-25T08:35:44.354121+0000","last_peered":"2024-04-25T08:35:44.354121+0000","last_clean":"2024-04-25T08:35:44.354121+0000","last_became_active":"2024-04-25T08:35:43.088869+0000","last_became_peered":"2024-04-25T08:35:43.088869+0000","last_unstale":"2024-04-25T08:35:44.354121+0000","last_undegraded":"2024-04-25T08:35:44.354121+0000","last_fullsized":"2024-04-25T08:35:44.354121+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084198+0000","last_change":"2024-04-25T08:35:43.094595+0000","last_active":"2024-04-25T08:35:44.084198+0000","last_peered":"2024-04-25T08:35:44.084198+0000","last_clean":"2024-04-25T08:35:44.084198+0000","last_became_active":"2024-04-25T08:35:43.093864+0000","last_became_peered":"2024-04-25T08:35:43.093864+0000","last_unstale":"2024-04-25T08:35:44.084198+0000","last_undegraded":"2024-04-25T08:35:44.084198+0000","last_fullsized":"2024-04-25T08:35:44.084198+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354674+0000","last_change":"2024-04-25T08:35:43.086251+0000","last_active":"2024-04-25T08:35:44.354674+0000","last_peered":"2024-04-25T08:35:44.354674+0000","last_clean":"2024-04-25T08:35:44.354674+0000","last_became_active":"2024-04-25T08:35:43.085938+0000","last_became_peered":"2024-04-25T08:35:43.085938+0000","last_unstale":"2024-04-25T08:35:44.354674+0000","last_undegraded":"2024-04-25T08:35:44.354674+0000","last_fullsized":"2024-04-25T08:35:44.354674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084122+0000","last_change":"2024-04-25T08:35:43.084909+0000","last_active":"2024-04-25T08:35:44.084122+0000","last_peered":"2024-04-25T08:35:44.084122+0000","last_clean":"2024-04-25T08:35:44.084122+0000","last_became_active":"2024-04-25T08:35:43.084519+0000","last_became_peered":"2024-04-25T08:35:43.084519+0000","last_unstale":"2024-04-25T08:35:44.084122+0000","last_undegraded":"2024-04-25T08:35:44.084122+0000","last_fullsized":"2024-04-25T08:35:44.084122+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083572+0000","last_change":"2024-04-25T08:35:43.083871+0000","last_active":"2024-04-25T08:35:44.083572+0000","last_peered":"2024-04-25T08:35:44.083572+0000","last_clean":"2024-04-25T08:35:44.083572+0000","last_became_active":"2024-04-25T08:35:43.083574+0000","last_became_peered":"2024-04-25T08:35:43.083574+0000","last_unstale":"2024-04-25T08:35:44.083572+0000","last_undegraded":"2024-04-25T08:35:44.083572+0000","last_fullsized":"2024-04-25T08:35:44.083572+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.102694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354442+0000","last_change":"2024-04-25T08:35:43.080351+0000","last_active":"2024-04-25T08:35:44.354442+0000","last_peered":"2024-04-25T08:35:44.354442+0000","last_clean":"2024-04-25T08:35:44.354442+0000","last_became_active":"2024-04-25T08:35:43.080020+0000","last_became_peered":"2024-04-25T08:35:43.080020+0000","last_unstale":"2024-04-25T08:35:44.354442+0000","last_undegraded":"2024-04-25T08:35:44.354442+0000","last_fullsized":"2024-04-25T08:35:44.354442+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083931+0000","last_change":"2024-04-25T08:35:43.087490+0000","last_active":"2024-04-25T08:35:44.083931+0000","last_peered":"2024-04-25T08:35:44.083931+0000","last_clean":"2024-04-25T08:35:44.083931+0000","last_became_active":"2024-04-25T08:35:43.087039+0000","last_became_peered":"2024-04-25T08:35:43.087039+0000","last_unstale":"2024-04-25T08:35:44.083931+0000","last_undegraded":"2024-04-25T08:35:44.083931+0000","last_fullsized":"2024-04-25T08:35:44.083931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.866982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354143+0000","last_change":"2024-04-25T08:35:43.090435+0000","last_active":"2024-04-25T08:35:44.354143+0000","last_peered":"2024-04-25T08:35:44.354143+0000","last_clean":"2024-04-25T08:35:44.354143+0000","last_became_active":"2024-04-25T08:35:43.090166+0000","last_became_peered":"2024-04-25T08:35:43.090166+0000","last_unstale":"2024-04-25T08:35:44.354143+0000","last_undegraded":"2024-04-25T08:35:44.354143+0000","last_fullsized":"2024-04-25T08:35:44.354143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:10.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354827+0000","last_change":"2024-04-25T08:35:43.084313+0000","last_active":"2024-04-25T08:35:44.354827+0000","last_peered":"2024-04-25T08:35:44.354827+0000","last_clean":"2024-04-25T08:35:44.354827+0000","last_became_active":"2024-04-25T08:35:43.084027+0000","last_became_peered":"2024-04-25T08:35:43.084027+0000","last_unstale":"2024-04-25T08:35:44.354827+0000","last_undegraded":"2024-04-25T08:35:44.354827+0000","last_fullsized":"2024-04-25T08:35:44.354827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083847+0000","last_change":"2024-04-25T08:35:43.088502+0000","last_active":"2024-04-25T08:35:44.083847+0000","last_peered":"2024-04-25T08:35:44.083847+0000","last_clean":"2024-04-25T08:35:44.083847+0000","last_became_active":"2024-04-25T08:35:43.088156+0000","last_became_peered":"2024-04-25T08:35:43.088156+0000","last_unstale":"2024-04-25T08:35:44.083847+0000","last_undegraded":"2024-04-25T08:35:44.083847+0000","last_fullsized":"2024-04-25T08:35:44.083847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:29.004324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061987+0000","last_change":"2024-04-25T08:35:41.061987+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061987+0000","last_undegraded":"2024-04-25T08:35:41.061987+0000","last_fullsized":"2024-04-25T08:35:41.061987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084482+0000","last_change":"2024-04-25T08:35:42.077498+0000","last_active":"2024-04-25T08:35:44.084482+0000","last_peered":"2024-04-25T08:35:44.084482+0000","last_clean":"2024-04-25T08:35:44.084482+0000","last_became_active":"2024-04-25T08:35:42.077203+0000","last_became_peered":"2024-04-25T08:35:42.077203+0000","last_unstale":"2024-04-25T08:35:44.084482+0000","last_undegraded":"2024-04-25T08:35:44.084482+0000","last_fullsized":"2024-04-25T08:35:44.084482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063607+0000","last_change":"2024-04-25T08:35:41.063607+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063607+0000","last_undegraded":"2024-04-25T08:35:41.063607+0000","last_fullsized":"2024-04-25T08:35:41.063607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061876+0000","last_change":"2024-04-25T08:35:41.061876+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061876+0000","last_undegraded":"2024-04-25T08:35:41.061876+0000","last_fullsized":"2024-04-25T08:35:41.061876+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354942+0000","last_change":"2024-04-25T08:35:43.085489+0000","last_active":"2024-04-25T08:35:44.354942+0000","last_peered":"2024-04-25T08:35:44.354942+0000","last_clean":"2024-04-25T08:35:44.354942+0000","last_became_active":"2024-04-25T08:35:43.085149+0000","last_became_peered":"2024-04-25T08:35:43.085149+0000","last_unstale":"2024-04-25T08:35:44.354942+0000","last_undegraded":"2024-04-25T08:35:44.354942+0000","last_fullsized":"2024-04-25T08:35:44.354942+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083722+0000","last_change":"2024-04-25T08:35:42.083359+0000","last_active":"2024-04-25T08:35:44.083722+0000","last_peered":"2024-04-25T08:35:44.083722+0000","last_clean":"2024-04-25T08:35:44.083722+0000","last_became_active":"2024-04-25T08:35:42.083075+0000","last_became_peered":"2024-04-25T08:35:42.083075+0000","last_unstale":"2024-04-25T08:35:44.083722+0000","last_undegraded":"2024-04-25T08:35:44.083722+0000","last_fullsized":"2024-04-25T08:35:44.083722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:59.795889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354789+0000","last_change":"2024-04-25T08:35:42.087255+0000","last_active":"2024-04-25T08:35:44.354789+0000","last_peered":"2024-04-25T08:35:44.354789+0000","last_clean":"2024-04-25T08:35:44.354789+0000","last_became_active":"2024-04-25T08:35:42.086862+0000","last_became_peered":"2024-04-25T08:35:42.086862+0000","last_unstale":"2024-04-25T08:35:44.354789+0000","last_undegraded":"2024-04-25T08:35:44.354789+0000","last_fullsized":"2024-04-25T08:35:44.354789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.231173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354848+0000","last_change":"2024-04-25T08:35:42.078496+0000","last_active":"2024-04-25T08:35:44.354848+0000","last_peered":"2024-04-25T08:35:44.354848+0000","last_clean":"2024-04-25T08:35:44.354848+0000","last_became_active":"2024-04-25T08:35:42.078199+0000","last_became_peered":"2024-04-25T08:35:42.078199+0000","last_unstale":"2024-04-25T08:35:44.354848+0000","last_undegraded":"2024-04-25T08:35:44.354848+0000","last_fullsized":"2024-04-25T08:35:44.354848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354685+0000","last_change":"2024-04-25T08:35:43.096815+0000","last_active":"2024-04-25T08:35:44.354685+0000","last_peered":"2024-04-25T08:35:44.354685+0000","last_clean":"2024-04-25T08:35:44.354685+0000","last_became_active":"2024-04-25T08:35:43.096237+0000","last_became_peered":"2024-04-25T08:35:43.096237+0000","last_unstale":"2024-04-25T08:35:44.354685+0000","last_undegraded":"2024-04-25T08:35:44.354685+0000","last_fullsized":"2024-04-25T08:35:44.354685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063996+0000","last_change":"2024-04-25T08:35:41.063996+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063996+0000","last_undegraded":"2024-04-25T08:35:41.063996+0000","last_fullsized":"2024-04-25T08:35:41.063996+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083757+0000","last_change":"2024-04-25T08:35:43.084977+0000","last_active":"2024-04-25T08:35:44.083757+0000","last_peered":"2024-04-25T08:35:44.083757+0000","last_clean":"2024-04-25T08:35:44.083757+0000","last_became_active":"2024-04-25T08:35:43.084694+0000","last_became_peered":"2024-04-25T08:35:43.084694+0000","last_unstale":"2024-04-25T08:35:44.083757+0000","last_undegraded":"2024-04-25T08:35:44.083757+0000","last_fullsized":"2024-04-25T08:35:44.083757+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.121067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068599+0000","last_change":"2024-04-25T08:35:41.068599+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068599+0000","last_undegraded":"2024-04-25T08:35:41.068599+0000","last_fullsized":"2024-04-25T08:35:41.068599+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354368+0000","last_change":"2024-04-25T08:35:42.079015+0000","last_active":"2024-04-25T08:35:44.354368+0000","last_peered":"2024-04-25T08:35:44.354368+0000","last_clean":"2024-04-25T08:35:44.354368+0000","last_became_active":"2024-04-25T08:35:42.078722+0000","last_became_peered":"2024-04-25T08:35:42.078722+0000","last_unstale":"2024-04-25T08:35:44.354368+0000","last_undegraded":"2024-04-25T08:35:44.354368+0000","last_fullsized":"2024-04-25T08:35:44.354368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354270+0000","last_change":"2024-04-25T08:35:43.085992+0000","last_active":"2024-04-25T08:35:44.354270+0000","last_peered":"2024-04-25T08:35:44.354270+0000","last_clean":"2024-04-25T08:35:44.354270+0000","last_became_active":"2024-04-25T08:35:43.082970+0000","last_became_peered":"2024-04-25T08:35:43.082970+0000","last_unstale":"2024-04-25T08:35:44.354270+0000","last_undegraded":"2024-04-25T08:35:44.354270+0000","last_fullsized":"2024-04-25T08:35:44.354270+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084306+0000","last_change":"2024-04-25T08:35:42.085258+0000","last_active":"2024-04-25T08:35:44.084306+0000","last_peered":"2024-04-25T08:35:44.084306+0000","last_clean":"2024-04-25T08:35:44.084306+0000","last_became_active":"2024-04-25T08:35:42.084942+0000","last_became_peered":"2024-04-25T08:35:42.084942+0000","last_unstale":"2024-04-25T08:35:44.084306+0000","last_undegraded":"2024-04-25T08:35:44.084306+0000","last_fullsized":"2024-04-25T08:35:44.084306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:58.723955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354667+0000","last_change":"2024-04-25T08:35:43.094836+0000","last_active":"2024-04-25T08:35:44.354667+0000","last_peered":"2024-04-25T08:35:44.354667+0000","last_clean":"2024-04-25T08:35:44.354667+0000","last_became_active":"2024-04-25T08:35:43.093301+0000","last_became_peered":"2024-04-25T08:35:43.093301+0000","last_unstale":"2024-04-25T08:35:44.354667+0000","last_undegraded":"2024-04-25T08:35:44.354667+0000","last_fullsized":"2024-04-25T08:35:44.354667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355036+0000","last_change":"2024-04-25T08:35:43.095386+0000","last_active":"2024-04-25T08:35:44.355036+0000","last_peered":"2024-04-25T08:35:44.355036+0000","last_clean":"2024-04-25T08:35:44.355036+0000","last_became_active":"2024-04-25T08:35:43.093364+0000","last_became_peered":"2024-04-25T08:35:43.093364+0000","last_unstale":"2024-04-25T08:35:44.355036+0000","last_undegraded":"2024-04-25T08:35:44.355036+0000","last_fullsized":"2024-04-25T08:35:44.355036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063458+0000","last_change":"2024-04-25T08:35:41.063458+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063458+0000","last_undegraded":"2024-04-25T08:35:41.063458+0000","last_fullsized":"2024-04-25T08:35:41.063458+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:43.087418+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:43.087100+0000","last_became_peered":"2024-04-25T08:35:43.087100+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:23.384291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354284+0000","last_change":"2024-04-25T08:35:43.100537+0000","last_active":"2024-04-25T08:35:44.354284+0000","last_peered":"2024-04-25T08:35:44.354284+0000","last_clean":"2024-04-25T08:35:44.354284+0000","last_became_active":"2024-04-25T08:35:43.100246+0000","last_became_peered":"2024-04-25T08:35:43.100246+0000","last_unstale":"2024-04-25T08:35:44.354284+0000","last_undegraded":"2024-04-25T08:35:44.354284+0000","last_fullsized":"2024-04-25T08:35:44.354284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354726+0000","last_change":"2024-04-25T08:35:42.079064+0000","last_active":"2024-04-25T08:35:44.354726+0000","last_peered":"2024-04-25T08:35:44.354726+0000","last_clean":"2024-04-25T08:35:44.354726+0000","last_became_active":"2024-04-25T08:35:42.078692+0000","last_became_peered":"2024-04-25T08:35:42.078692+0000","last_unstale":"2024-04-25T08:35:44.354726+0000","last_undegraded":"2024-04-25T08:35:44.354726+0000","last_fullsized":"2024-04-25T08:35:44.354726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:03.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353823+0000","last_change":"2024-04-25T08:35:43.088607+0000","last_active":"2024-04-25T08:35:44.353823+0000","last_peered":"2024-04-25T08:35:44.353823+0000","last_clean":"2024-04-25T08:35:44.353823+0000","last_became_active":"2024-04-25T08:35:43.088317+0000","last_became_peered":"2024-04-25T08:35:43.088317+0000","last_unstale":"2024-04-25T08:35:44.353823+0000","last_undegraded":"2024-04-25T08:35:44.353823+0000","last_fullsized":"2024-04-25T08:35:44.353823+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:18.622878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355062+0000","last_change":"2024-04-25T08:35:42.086642+0000","last_active":"2024-04-25T08:35:44.355062+0000","last_peered":"2024-04-25T08:35:44.355062+0000","last_clean":"2024-04-25T08:35:44.355062+0000","last_became_active":"2024-04-25T08:35:42.086219+0000","last_became_peered":"2024-04-25T08:35:42.086219+0000","last_unstale":"2024-04-25T08:35:44.355062+0000","last_undegraded":"2024-04-25T08:35:44.355062+0000","last_fullsized":"2024-04-25T08:35:44.355062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:16.239961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.060959+0000","last_change":"2024-04-25T08:35:41.060959+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.060959+0000","last_undegraded":"2024-04-25T08:35:41.060959+0000","last_fullsized":"2024-04-25T08:35:41.060959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063514+0000","last_change":"2024-04-25T08:35:41.063514+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063514+0000","last_undegraded":"2024-04-25T08:35:41.063514+0000","last_fullsized":"2024-04-25T08:35:41.063514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084175+0000","last_change":"2024-04-25T08:35:43.092287+0000","last_active":"2024-04-25T08:35:44.084175+0000","last_peered":"2024-04-25T08:35:44.084175+0000","last_clean":"2024-04-25T08:35:44.084175+0000","last_became_active":"2024-04-25T08:35:43.091768+0000","last_became_peered":"2024-04-25T08:35:43.091768+0000","last_unstale":"2024-04-25T08:35:44.084175+0000","last_undegraded":"2024-04-25T08:35:44.084175+0000","last_fullsized":"2024-04-25T08:35:44.084175+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084367+0000","last_change":"2024-04-25T08:35:42.075127+0000","last_active":"2024-04-25T08:35:44.084367+0000","last_peered":"2024-04-25T08:35:44.084367+0000","last_clean":"2024-04-25T08:35:44.084367+0000","last_became_active":"2024-04-25T08:35:42.074842+0000","last_became_peered":"2024-04-25T08:35:42.074842+0000","last_unstale":"2024-04-25T08:35:44.084367+0000","last_undegraded":"2024-04-25T08:35:44.084367+0000","last_fullsized":"2024-04-25T08:35:44.084367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.370203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068005+0000","last_change":"2024-04-25T08:35:41.068005+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068005+0000","last_undegraded":"2024-04-25T08:35:41.068005+0000","last_fullsized":"2024-04-25T08:35:41.068005+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354254+0000","last_change":"2024-04-25T08:35:42.077041+0000","last_active":"2024-04-25T08:35:44.354254+0000","last_peered":"2024-04-25T08:35:44.354254+0000","last_clean":"2024-04-25T08:35:44.354254+0000","last_became_active":"2024-04-25T08:35:42.076737+0000","last_became_peered":"2024-04-25T08:35:42.076737+0000","last_unstale":"2024-04-25T08:35:44.354254+0000","last_undegraded":"2024-04-25T08:35:44.354254+0000","last_fullsized":"2024-04-25T08:35:44.354254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:54.769371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.067973+0000","last_change":"2024-04-25T08:35:41.067973+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.067973+0000","last_undegraded":"2024-04-25T08:35:41.067973+0000","last_fullsized":"2024-04-25T08:35:41.067973+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353713+0000","last_change":"2024-04-25T08:35:42.079469+0000","last_active":"2024-04-25T08:35:44.353713+0000","last_peered":"2024-04-25T08:35:44.353713+0000","last_clean":"2024-04-25T08:35:44.353713+0000","last_became_active":"2024-04-25T08:35:42.079187+0000","last_became_peered":"2024-04-25T08:35:42.079187+0000","last_unstale":"2024-04-25T08:35:44.353713+0000","last_undegraded":"2024-04-25T08:35:44.353713+0000","last_fullsized":"2024-04-25T08:35:44.353713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.436872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063728+0000","last_change":"2024-04-25T08:35:41.063728+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063728+0000","last_undegraded":"2024-04-25T08:35:41.063728+0000","last_fullsized":"2024-04-25T08:35:41.063728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088238+0000","last_change":"2024-04-25T08:35:42.080608+0000","last_active":"2024-04-25T08:35:44.088238+0000","last_peered":"2024-04-25T08:35:44.088238+0000","last_clean":"2024-04-25T08:35:44.088238+0000","last_became_active":"2024-04-25T08:35:42.080267+0000","last_became_peered":"2024-04-25T08:35:42.080267+0000","last_unstale":"2024-04-25T08:35:44.088238+0000","last_undegraded":"2024-04-25T08:35:44.088238+0000","last_fullsized":"2024-04-25T08:35:44.088238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:09.842643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084421+0000","last_change":"2024-04-25T08:35:42.084974+0000","last_active":"2024-04-25T08:35:44.084421+0000","last_peered":"2024-04-25T08:35:44.084421+0000","last_clean":"2024-04-25T08:35:44.084421+0000","last_became_active":"2024-04-25T08:35:42.084703+0000","last_became_peered":"2024-04-25T08:35:42.084703+0000","last_unstale":"2024-04-25T08:35:44.084421+0000","last_undegraded":"2024-04-25T08:35:44.084421+0000","last_fullsized":"2024-04-25T08:35:44.084421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:56.731886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061055+0000","last_change":"2024-04-25T08:35:41.061055+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061055+0000","last_undegraded":"2024-04-25T08:35:41.061055+0000","last_fullsized":"2024-04-25T08:35:41.061055+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353624+0000","last_change":"2024-04-25T08:35:43.089060+0000","last_active":"2024-04-25T08:35:44.353624+0000","last_peered":"2024-04-25T08:35:44.353624+0000","last_clean":"2024-04-25T08:35:44.353624+0000","last_became_active":"2024-04-25T08:35:43.088777+0000","last_became_peered":"2024-04-25T08:35:43.088777+0000","last_unstale":"2024-04-25T08:35:44.353624+0000","last_undegraded":"2024-04-25T08:35:44.353624+0000","last_fullsized":"2024-04-25T08:35:44.353624+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:58.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354681+0000","last_change":"2024-04-25T08:35:43.080706+0000","last_active":"2024-04-25T08:35:44.354681+0000","last_peered":"2024-04-25T08:35:44.354681+0000","last_clean":"2024-04-25T08:35:44.354681+0000","last_became_active":"2024-04-25T08:35:43.080411+0000","last_became_peered":"2024-04-25T08:35:43.080411+0000","last_unstale":"2024-04-25T08:35:44.354681+0000","last_undegraded":"2024-04-25T08:35:44.354681+0000","last_fullsized":"2024-04-25T08:35:44.354681+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068003+0000","last_change":"2024-04-25T08:35:41.068003+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068003+0000","last_undegraded":"2024-04-25T08:35:41.068003+0000","last_fullsized":"2024-04-25T08:35:41.068003+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.060868+0000","last_change":"2024-04-25T08:35:41.060868+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.060868+0000","last_undegraded":"2024-04-25T08:35:41.060868+0000","last_fullsized":"2024-04-25T08:35:41.060868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063500+0000","last_change":"2024-04-25T08:35:41.063500+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063500+0000","last_undegraded":"2024-04-25T08:35:41.063500+0000","last_fullsized":"2024-04-25T08:35:41.063500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354943+0000","last_change":"2024-04-25T08:35:42.080661+0000","last_active":"2024-04-25T08:35:44.354943+0000","last_peered":"2024-04-25T08:35:44.354943+0000","last_clean":"2024-04-25T08:35:44.354943+0000","last_became_active":"2024-04-25T08:35:42.080378+0000","last_became_peered":"2024-04-25T08:35:42.080378+0000","last_unstale":"2024-04-25T08:35:44.354943+0000","last_undegraded":"2024-04-25T08:35:44.354943+0000","last_fullsized":"2024-04-25T08:35:44.354943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:11.863733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":309,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.064192+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:35:41.064192+0000","last_peered":"2024-04-25T08:35:41.064192+0000","last_clean":"2024-04-25T08:35:41.064192+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:35:41.064192+0000","last_undegraded":"2024-04-25T08:35:41.064192+0000","last_fullsized":"2024-04-25T08:35:41.064192+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":58,"acting":58,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41196,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330644,"statfs":{"total":96636764160,"available":96594579456,"internally_reserved":0,"allocated":1736704,"data_stored":1115590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1155072,"data_stored":531431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1155072,"data_stored":531431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":552117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50392,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321448,"statfs":{"total":96636764160,"available":96585162752,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54556,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317284,"statfs":{"total":96636764160,"available":96580898816,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":552027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55084,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316756,"statfs":{"total":96636764160,"available":96580358144,"internally_reserved":0,"allocated":1736704,"data_stored":1115590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:46.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:46.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.571+0000 7f51833ce640 1 Processor -- start 2024-04-25T08:35:46.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.571+0000 7f51833ce640 1 -- start start 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f51833ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f517c15db70 0x7f517c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f51833ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f517c0b6cb0 0x7f517c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f51833ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c162690 con 0x7f517c0b5a60 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f51833ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c0d3800 con 0x7f517c0b6cb0 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f51833ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c0ceee0 con 0x7f517c15db70 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f5180969640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f517c0b6cb0 0x7f517c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f518196b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f517c0b5a60 0x7f517c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56334/0 (socket says 172.21.15.60:56334) 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f518196b640 1 -- 172.21.15.60:0/204880189 learned_addr learned my addr 172.21.15.60:0/204880189 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f518116a640 1 --2- 172.21.15.60:0/204880189 >> v2:172.21.15.100:3300/0 conn(0x7f517c15db70 0x7f517c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f5180969640 1 -- 172.21.15.60:0/204880189 >> v2:172.21.15.100:3300/0 conn(0x7f517c15db70 msgr2=0x7f517c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:46.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f5180969640 1 --2- 172.21.15.60:0/204880189 >> v2:172.21.15.100:3300/0 conn(0x7f517c15db70 0x7f517c162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f5180969640 1 -- 172.21.15.60:0/204880189 >> v1:172.21.15.60:6789/0 conn(0x7f517c0b5a60 legacy=0x7f517c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f5173fff640 1 -- 172.21.15.60:0/204880189 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230675361 0 0) 0x7f517c162690 con 0x7f517c0b5a60 2024-04-25T08:35:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f5180969640 1 -- 172.21.15.60:0/204880189 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0d1d10 con 0x7f517c0b6cb0 2024-04-25T08:35:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.572+0000 7f518116a640 1 --2- 172.21.15.60:0/204880189 >> v2:172.21.15.100:3300/0 conn(0x7f517c15db70 0x7f517c162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f5180969640 1 --2- 172.21.15.60:0/204880189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c0b6cb0 0x7f517c0b5520 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f517c0d3980 tx=0x7f516c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929ecb07a3cde89b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f5173fff640 1 -- 172.21.15.60:0/204880189 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c037070 con 0x7f517c0b6cb0 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f5173fff640 1 -- 172.21.15.60:0/204880189 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c032440 con 0x7f517c0b6cb0 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f5173fff640 1 -- 172.21.15.60:0/204880189 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c02ae60 con 0x7f517c0b6cb0 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f51833ce640 1 -- 172.21.15.60:0/204880189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c0b6cb0 msgr2=0x7f517c0b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f51833ce640 1 --2- 172.21.15.60:0/204880189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c0b6cb0 0x7f517c0b5520 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f517c0d3980 tx=0x7f516c02a110 comp rx=0 tx=0).stop 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f51833ce640 1 -- 172.21.15.60:0/204880189 shutdown_connections 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f51833ce640 1 --2- 172.21.15.60:0/204880189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c0b6cb0 0x7f517c0b5520 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f51833ce640 1 --2- 172.21.15.60:0/204880189 >> v2:172.21.15.100:3300/0 conn(0x7f517c15db70 0x7f517c162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.573+0000 7f51833ce640 1 -- 172.21.15.60:0/204880189 >> 172.21.15.60:0/204880189 conn(0x7f517c0d0520 msgr2=0x7f517c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.574+0000 7f51833ce640 1 -- 172.21.15.60:0/204880189 shutdown_connections 2024-04-25T08:35:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.574+0000 7f51833ce640 1 -- 172.21.15.60:0/204880189 wait complete. 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.575+0000 7f51833ce640 1 Processor -- start 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.575+0000 7f51833ce640 1 -- start start 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f517c0b5a60 0x7f517c0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f517c0b6cb0 0x7f517c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 0x7f517c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c15df50 con 0x7f517c15db70 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0d22d0 con 0x7f517c0b6cb0 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0ce530 con 0x7f517c0b5a60 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 0x7f517c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 0x7f517c15f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35794/0 (socket says 172.21.15.60:35794) 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 -- 172.21.15.60:0/1353294624 learned_addr learned my addr 172.21.15.60:0/1353294624 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518116a640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f517c0b5a60 0x7f517c0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f5180969640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f517c0b6cb0 0x7f517c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 -- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f517c0b5a60 msgr2=0x7f517c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f517c0b5a60 0x7f517c0c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 -- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f517c0b6cb0 msgr2=0x7f517c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f517c0b6cb0 0x7f517c159160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 -- 172.21.15.60:0/1353294624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0c9b90 con 0x7f517c15db70 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f518196b640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 0x7f517c15f2b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f517400c7a0 tx=0x7f517400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174018070 con 0x7f517c15db70 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400a640 con 0x7f517c15db70 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c1580c0 con 0x7f517c15db70 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.576+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517400aab0 con 0x7f517c15db70 2024-04-25T08:35:46.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.577+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c158680 con 0x7f517c15db70 2024-04-25T08:35:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.578+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c0ce830 con 0x7f517c15db70 2024-04-25T08:35:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.578+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f517401b070 con 0x7f517c15db70 2024-04-25T08:35:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.578+0000 7f5171ffb640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f516006b4d0 0x7f516006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.579+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51740899c0 con 0x7f517c15db70 2024-04-25T08:35:46.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.579+0000 7f518116a640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f516006b4d0 0x7f516006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.579+0000 7f518116a640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f516006b4d0 0x7f516006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f517c15af10 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.583+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f517c0ce830 con 0x7f517c15db70 2024-04-25T08:35:46.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.759+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f517c0ce180 con 0x7f516006b4d0 2024-04-25T08:35:46.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.765+0000 7f5171ffb640 1 -- 172.21.15.60:0/1353294624 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+245857 (secure 0 0 0) 0x7f517c0ce180 con 0x7f516006b4d0 2024-04-25T08:35:46.763 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:46.764 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f516006b4d0 msgr2=0x7f516006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f516006b4d0 0x7f516006d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f517c15af10 tx=0x7f5178008040 comp rx=0 tx=0).stop 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 msgr2=0x7f517c15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 0x7f517c15f2b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f517400c7a0 tx=0x7f517400cc60 comp rx=0 tx=0).stop 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 shutdown_connections 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f517c15db70 0x7f517c15f2b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f516006b4d0 0x7f516006d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f517c0b6cb0 0x7f517c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 --2- 172.21.15.60:0/1353294624 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f517c0b5a60 0x7f517c0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 >> 172.21.15.60:0/1353294624 conn(0x7f517c0d0520 msgr2=0x7f517c15ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 shutdown_connections 2024-04-25T08:35:46.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.768+0000 7f51833ce640 1 -- 172.21.15.60:0/1353294624 wait complete. 2024-04-25T08:35:46.774 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:35:45.485489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590528,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":128,"acting":128,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356932,"kb_used_data":10316,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754617788,"statfs":{"total":773094113280,"available":772728614912,"internally_reserved":0,"allocated":10563584,"data_stored":5502100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":354799152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.426505"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353876+0000","last_change":"2024-04-25T08:35:43.090294+0000","last_active":"2024-04-25T08:35:44.353876+0000","last_peered":"2024-04-25T08:35:44.353876+0000","last_clean":"2024-04-25T08:35:44.353876+0000","last_became_active":"2024-04-25T08:35:43.089694+0000","last_became_peered":"2024-04-25T08:35:43.089694+0000","last_unstale":"2024-04-25T08:35:44.353876+0000","last_undegraded":"2024-04-25T08:35:44.353876+0000","last_fullsized":"2024-04-25T08:35:44.353876+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353866+0000","last_change":"2024-04-25T08:35:43.081017+0000","last_active":"2024-04-25T08:35:44.353866+0000","last_peered":"2024-04-25T08:35:44.353866+0000","last_clean":"2024-04-25T08:35:44.353866+0000","last_became_active":"2024-04-25T08:35:43.080694+0000","last_became_peered":"2024-04-25T08:35:43.080694+0000","last_unstale":"2024-04-25T08:35:44.353866+0000","last_undegraded":"2024-04-25T08:35:44.353866+0000","last_fullsized":"2024-04-25T08:35:44.353866+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354842+0000","last_change":"2024-04-25T08:35:43.095431+0000","last_active":"2024-04-25T08:35:44.354842+0000","last_peered":"2024-04-25T08:35:44.354842+0000","last_clean":"2024-04-25T08:35:44.354842+0000","last_became_active":"2024-04-25T08:35:43.095148+0000","last_became_peered":"2024-04-25T08:35:43.095148+0000","last_unstale":"2024-04-25T08:35:44.354842+0000","last_undegraded":"2024-04-25T08:35:44.354842+0000","last_fullsized":"2024-04-25T08:35:44.354842+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083569+0000","last_change":"2024-04-25T08:35:43.080094+0000","last_active":"2024-04-25T08:35:44.083569+0000","last_peered":"2024-04-25T08:35:44.083569+0000","last_clean":"2024-04-25T08:35:44.083569+0000","last_became_active":"2024-04-25T08:35:43.079466+0000","last_became_peered":"2024-04-25T08:35:43.079466+0000","last_unstale":"2024-04-25T08:35:44.083569+0000","last_undegraded":"2024-04-25T08:35:44.083569+0000","last_fullsized":"2024-04-25T08:35:44.083569+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354780+0000","last_change":"2024-04-25T08:35:43.095408+0000","last_active":"2024-04-25T08:35:44.354780+0000","last_peered":"2024-04-25T08:35:44.354780+0000","last_clean":"2024-04-25T08:35:44.354780+0000","last_became_active":"2024-04-25T08:35:43.095119+0000","last_became_peered":"2024-04-25T08:35:43.095119+0000","last_unstale":"2024-04-25T08:35:44.354780+0000","last_undegraded":"2024-04-25T08:35:44.354780+0000","last_fullsized":"2024-04-25T08:35:44.354780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083849+0000","last_change":"2024-04-25T08:35:43.087871+0000","last_active":"2024-04-25T08:35:44.083849+0000","last_peered":"2024-04-25T08:35:44.083849+0000","last_clean":"2024-04-25T08:35:44.083849+0000","last_became_active":"2024-04-25T08:35:43.087575+0000","last_became_peered":"2024-04-25T08:35:43.087575+0000","last_unstale":"2024-04-25T08:35:44.083849+0000","last_undegraded":"2024-04-25T08:35:44.083849+0000","last_fullsized":"2024-04-25T08:35:44.083849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:32.414798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354147+0000","last_change":"2024-04-25T08:35:43.101307+0000","last_active":"2024-04-25T08:35:44.354147+0000","last_peered":"2024-04-25T08:35:44.354147+0000","last_clean":"2024-04-25T08:35:44.354147+0000","last_became_active":"2024-04-25T08:35:43.101003+0000","last_became_peered":"2024-04-25T08:35:43.101003+0000","last_unstale":"2024-04-25T08:35:44.354147+0000","last_undegraded":"2024-04-25T08:35:44.354147+0000","last_fullsized":"2024-04-25T08:35:44.354147+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:14.081591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084050+0000","last_change":"2024-04-25T08:35:43.079921+0000","last_active":"2024-04-25T08:35:44.084050+0000","last_peered":"2024-04-25T08:35:44.084050+0000","last_clean":"2024-04-25T08:35:44.084050+0000","last_became_active":"2024-04-25T08:35:43.079607+0000","last_became_peered":"2024-04-25T08:35:43.079607+0000","last_unstale":"2024-04-25T08:35:44.084050+0000","last_undegraded":"2024-04-25T08:35:44.084050+0000","last_fullsized":"2024-04-25T08:35:44.084050+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:57.986549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353925+0000","last_change":"2024-04-25T08:35:43.087944+0000","last_active":"2024-04-25T08:35:44.353925+0000","last_peered":"2024-04-25T08:35:44.353925+0000","last_clean":"2024-04-25T08:35:44.353925+0000","last_became_active":"2024-04-25T08:35:43.087638+0000","last_became_peered":"2024-04-25T08:35:43.087638+0000","last_unstale":"2024-04-25T08:35:44.353925+0000","last_undegraded":"2024-04-25T08:35:44.353925+0000","last_fullsized":"2024-04-25T08:35:44.353925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.692505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354540+0000","last_change":"2024-04-25T08:35:43.095160+0000","last_active":"2024-04-25T08:35:44.354540+0000","last_peered":"2024-04-25T08:35:44.354540+0000","last_clean":"2024-04-25T08:35:44.354540+0000","last_became_active":"2024-04-25T08:35:43.094865+0000","last_became_peered":"2024-04-25T08:35:43.094865+0000","last_unstale":"2024-04-25T08:35:44.354540+0000","last_undegraded":"2024-04-25T08:35:44.354540+0000","last_fullsized":"2024-04-25T08:35:44.354540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083498+0000","last_change":"2024-04-25T08:35:43.096325+0000","last_active":"2024-04-25T08:35:44.083498+0000","last_peered":"2024-04-25T08:35:44.083498+0000","last_clean":"2024-04-25T08:35:44.083498+0000","last_became_active":"2024-04-25T08:35:43.096042+0000","last_became_peered":"2024-04-25T08:35:43.096042+0000","last_unstale":"2024-04-25T08:35:44.083498+0000","last_undegraded":"2024-04-25T08:35:44.083498+0000","last_fullsized":"2024-04-25T08:35:44.083498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:27.748314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354752+0000","last_change":"2024-04-25T08:35:43.080738+0000","last_active":"2024-04-25T08:35:44.354752+0000","last_peered":"2024-04-25T08:35:44.354752+0000","last_clean":"2024-04-25T08:35:44.354752+0000","last_became_active":"2024-04-25T08:35:43.080444+0000","last_became_peered":"2024-04-25T08:35:43.080444+0000","last_unstale":"2024-04-25T08:35:44.354752+0000","last_undegraded":"2024-04-25T08:35:44.354752+0000","last_fullsized":"2024-04-25T08:35:44.354752+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354121+0000","last_change":"2024-04-25T08:35:43.089177+0000","last_active":"2024-04-25T08:35:44.354121+0000","last_peered":"2024-04-25T08:35:44.354121+0000","last_clean":"2024-04-25T08:35:44.354121+0000","last_became_active":"2024-04-25T08:35:43.088869+0000","last_became_peered":"2024-04-25T08:35:43.088869+0000","last_unstale":"2024-04-25T08:35:44.354121+0000","last_undegraded":"2024-04-25T08:35:44.354121+0000","last_fullsized":"2024-04-25T08:35:44.354121+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084198+0000","last_change":"2024-04-25T08:35:43.094595+0000","last_active":"2024-04-25T08:35:44.084198+0000","last_peered":"2024-04-25T08:35:44.084198+0000","last_clean":"2024-04-25T08:35:44.084198+0000","last_became_active":"2024-04-25T08:35:43.093864+0000","last_became_peered":"2024-04-25T08:35:43.093864+0000","last_unstale":"2024-04-25T08:35:44.084198+0000","last_undegraded":"2024-04-25T08:35:44.084198+0000","last_fullsized":"2024-04-25T08:35:44.084198+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354674+0000","last_change":"2024-04-25T08:35:43.086251+0000","last_active":"2024-04-25T08:35:44.354674+0000","last_peered":"2024-04-25T08:35:44.354674+0000","last_clean":"2024-04-25T08:35:44.354674+0000","last_became_active":"2024-04-25T08:35:43.085938+0000","last_became_peered":"2024-04-25T08:35:43.085938+0000","last_unstale":"2024-04-25T08:35:44.354674+0000","last_undegraded":"2024-04-25T08:35:44.354674+0000","last_fullsized":"2024-04-25T08:35:44.354674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084122+0000","last_change":"2024-04-25T08:35:43.084909+0000","last_active":"2024-04-25T08:35:44.084122+0000","last_peered":"2024-04-25T08:35:44.084122+0000","last_clean":"2024-04-25T08:35:44.084122+0000","last_became_active":"2024-04-25T08:35:43.084519+0000","last_became_peered":"2024-04-25T08:35:43.084519+0000","last_unstale":"2024-04-25T08:35:44.084122+0000","last_undegraded":"2024-04-25T08:35:44.084122+0000","last_fullsized":"2024-04-25T08:35:44.084122+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083572+0000","last_change":"2024-04-25T08:35:43.083871+0000","last_active":"2024-04-25T08:35:44.083572+0000","last_peered":"2024-04-25T08:35:44.083572+0000","last_clean":"2024-04-25T08:35:44.083572+0000","last_became_active":"2024-04-25T08:35:43.083574+0000","last_became_peered":"2024-04-25T08:35:43.083574+0000","last_unstale":"2024-04-25T08:35:44.083572+0000","last_undegraded":"2024-04-25T08:35:44.083572+0000","last_fullsized":"2024-04-25T08:35:44.083572+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.102694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354442+0000","last_change":"2024-04-25T08:35:43.080351+0000","last_active":"2024-04-25T08:35:44.354442+0000","last_peered":"2024-04-25T08:35:44.354442+0000","last_clean":"2024-04-25T08:35:44.354442+0000","last_became_active":"2024-04-25T08:35:43.080020+0000","last_became_peered":"2024-04-25T08:35:43.080020+0000","last_unstale":"2024-04-25T08:35:44.354442+0000","last_undegraded":"2024-04-25T08:35:44.354442+0000","last_fullsized":"2024-04-25T08:35:44.354442+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083931+0000","last_change":"2024-04-25T08:35:43.087490+0000","last_active":"2024-04-25T08:35:44.083931+0000","last_peered":"2024-04-25T08:35:44.083931+0000","last_clean":"2024-04-25T08:35:44.083931+0000","last_became_active":"2024-04-25T08:35:43.087039+0000","last_became_peered":"2024-04-25T08:35:43.087039+0000","last_unstale":"2024-04-25T08:35:44.083931+0000","last_undegraded":"2024-04-25T08:35:44.083931+0000","last_fullsized":"2024-04-25T08:35:44.083931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.866982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354143+0000","last_change":"2024-04-25T08:35:43.090435+0000","last_active":"2024-04-25T08:35:44.354143+0000","last_peered":"2024-04-25T08:35:44.354143+0000","last_clean":"2024-04-25T08:35:44.354143+0000","last_became_active":"2024-04-25T08:35:43.090166+0000","last_became_peered":"2024-04-25T08:35:43.090166+0000","last_unstale":"2024-04-25T08:35:44.354143+0000","last_undegraded":"2024-04-25T08:35:44.354143+0000","last_fullsized":"2024-04-25T08:35:44.354143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:10.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354827+0000","last_change":"2024-04-25T08:35:43.084313+0000","last_active":"2024-04-25T08:35:44.354827+0000","last_peered":"2024-04-25T08:35:44.354827+0000","last_clean":"2024-04-25T08:35:44.354827+0000","last_became_active":"2024-04-25T08:35:43.084027+0000","last_became_peered":"2024-04-25T08:35:43.084027+0000","last_unstale":"2024-04-25T08:35:44.354827+0000","last_undegraded":"2024-04-25T08:35:44.354827+0000","last_fullsized":"2024-04-25T08:35:44.354827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083847+0000","last_change":"2024-04-25T08:35:43.088502+0000","last_active":"2024-04-25T08:35:44.083847+0000","last_peered":"2024-04-25T08:35:44.083847+0000","last_clean":"2024-04-25T08:35:44.083847+0000","last_became_active":"2024-04-25T08:35:43.088156+0000","last_became_peered":"2024-04-25T08:35:43.088156+0000","last_unstale":"2024-04-25T08:35:44.083847+0000","last_undegraded":"2024-04-25T08:35:44.083847+0000","last_fullsized":"2024-04-25T08:35:44.083847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:29.004324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061987+0000","last_change":"2024-04-25T08:35:41.061987+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061987+0000","last_undegraded":"2024-04-25T08:35:41.061987+0000","last_fullsized":"2024-04-25T08:35:41.061987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084482+0000","last_change":"2024-04-25T08:35:42.077498+0000","last_active":"2024-04-25T08:35:44.084482+0000","last_peered":"2024-04-25T08:35:44.084482+0000","last_clean":"2024-04-25T08:35:44.084482+0000","last_became_active":"2024-04-25T08:35:42.077203+0000","last_became_peered":"2024-04-25T08:35:42.077203+0000","last_unstale":"2024-04-25T08:35:44.084482+0000","last_undegraded":"2024-04-25T08:35:44.084482+0000","last_fullsized":"2024-04-25T08:35:44.084482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063607+0000","last_change":"2024-04-25T08:35:41.063607+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063607+0000","last_undegraded":"2024-04-25T08:35:41.063607+0000","last_fullsized":"2024-04-25T08:35:41.063607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061876+0000","last_change":"2024-04-25T08:35:41.061876+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061876+0000","last_undegraded":"2024-04-25T08:35:41.061876+0000","last_fullsized":"2024-04-25T08:35:41.061876+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354942+0000","last_change":"2024-04-25T08:35:43.085489+0000","last_active":"2024-04-25T08:35:44.354942+0000","last_peered":"2024-04-25T08:35:44.354942+0000","last_clean":"2024-04-25T08:35:44.354942+0000","last_became_active":"2024-04-25T08:35:43.085149+0000","last_became_peered":"2024-04-25T08:35:43.085149+0000","last_unstale":"2024-04-25T08:35:44.354942+0000","last_undegraded":"2024-04-25T08:35:44.354942+0000","last_fullsized":"2024-04-25T08:35:44.354942+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083722+0000","last_change":"2024-04-25T08:35:42.083359+0000","last_active":"2024-04-25T08:35:44.083722+0000","last_peered":"2024-04-25T08:35:44.083722+0000","last_clean":"2024-04-25T08:35:44.083722+0000","last_became_active":"2024-04-25T08:35:42.083075+0000","last_became_peered":"2024-04-25T08:35:42.083075+0000","last_unstale":"2024-04-25T08:35:44.083722+0000","last_undegraded":"2024-04-25T08:35:44.083722+0000","last_fullsized":"2024-04-25T08:35:44.083722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:59.795889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354789+0000","last_change":"2024-04-25T08:35:42.087255+0000","last_active":"2024-04-25T08:35:44.354789+0000","last_peered":"2024-04-25T08:35:44.354789+0000","last_clean":"2024-04-25T08:35:44.354789+0000","last_became_active":"2024-04-25T08:35:42.086862+0000","last_became_peered":"2024-04-25T08:35:42.086862+0000","last_unstale":"2024-04-25T08:35:44.354789+0000","last_undegraded":"2024-04-25T08:35:44.354789+0000","last_fullsized":"2024-04-25T08:35:44.354789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.231173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354848+0000","last_change":"2024-04-25T08:35:42.078496+0000","last_active":"2024-04-25T08:35:44.354848+0000","last_peered":"2024-04-25T08:35:44.354848+0000","last_clean":"2024-04-25T08:35:44.354848+0000","last_became_active":"2024-04-25T08:35:42.078199+0000","last_became_peered":"2024-04-25T08:35:42.078199+0000","last_unstale":"2024-04-25T08:35:44.354848+0000","last_undegraded":"2024-04-25T08:35:44.354848+0000","last_fullsized":"2024-04-25T08:35:44.354848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354685+0000","last_change":"2024-04-25T08:35:43.096815+0000","last_active":"2024-04-25T08:35:44.354685+0000","last_peered":"2024-04-25T08:35:44.354685+0000","last_clean":"2024-04-25T08:35:44.354685+0000","last_became_active":"2024-04-25T08:35:43.096237+0000","last_became_peered":"2024-04-25T08:35:43.096237+0000","last_unstale":"2024-04-25T08:35:44.354685+0000","last_undegraded":"2024-04-25T08:35:44.354685+0000","last_fullsized":"2024-04-25T08:35:44.354685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063996+0000","last_change":"2024-04-25T08:35:41.063996+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063996+0000","last_undegraded":"2024-04-25T08:35:41.063996+0000","last_fullsized":"2024-04-25T08:35:41.063996+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083757+0000","last_change":"2024-04-25T08:35:43.084977+0000","last_active":"2024-04-25T08:35:44.083757+0000","last_peered":"2024-04-25T08:35:44.083757+0000","last_clean":"2024-04-25T08:35:44.083757+0000","last_became_active":"2024-04-25T08:35:43.084694+0000","last_became_peered":"2024-04-25T08:35:43.084694+0000","last_unstale":"2024-04-25T08:35:44.083757+0000","last_undegraded":"2024-04-25T08:35:44.083757+0000","last_fullsized":"2024-04-25T08:35:44.083757+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.121067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068599+0000","last_change":"2024-04-25T08:35:41.068599+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068599+0000","last_undegraded":"2024-04-25T08:35:41.068599+0000","last_fullsized":"2024-04-25T08:35:41.068599+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354368+0000","last_change":"2024-04-25T08:35:42.079015+0000","last_active":"2024-04-25T08:35:44.354368+0000","last_peered":"2024-04-25T08:35:44.354368+0000","last_clean":"2024-04-25T08:35:44.354368+0000","last_became_active":"2024-04-25T08:35:42.078722+0000","last_became_peered":"2024-04-25T08:35:42.078722+0000","last_unstale":"2024-04-25T08:35:44.354368+0000","last_undegraded":"2024-04-25T08:35:44.354368+0000","last_fullsized":"2024-04-25T08:35:44.354368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354270+0000","last_change":"2024-04-25T08:35:43.085992+0000","last_active":"2024-04-25T08:35:44.354270+0000","last_peered":"2024-04-25T08:35:44.354270+0000","last_clean":"2024-04-25T08:35:44.354270+0000","last_became_active":"2024-04-25T08:35:43.082970+0000","last_became_peered":"2024-04-25T08:35:43.082970+0000","last_unstale":"2024-04-25T08:35:44.354270+0000","last_undegraded":"2024-04-25T08:35:44.354270+0000","last_fullsized":"2024-04-25T08:35:44.354270+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084306+0000","last_change":"2024-04-25T08:35:42.085258+0000","last_active":"2024-04-25T08:35:44.084306+0000","last_peered":"2024-04-25T08:35:44.084306+0000","last_clean":"2024-04-25T08:35:44.084306+0000","last_became_active":"2024-04-25T08:35:42.084942+0000","last_became_peered":"2024-04-25T08:35:42.084942+0000","last_unstale":"2024-04-25T08:35:44.084306+0000","last_undegraded":"2024-04-25T08:35:44.084306+0000","last_fullsized":"2024-04-25T08:35:44.084306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:58.723955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354667+0000","last_change":"2024-04-25T08:35:43.094836+0000","last_active":"2024-04-25T08:35:44.354667+0000","last_peered":"2024-04-25T08:35:44.354667+0000","last_clean":"2024-04-25T08:35:44.354667+0000","last_became_active":"2024-04-25T08:35:43.093301+0000","last_became_peered":"2024-04-25T08:35:43.093301+0000","last_unstale":"2024-04-25T08:35:44.354667+0000","last_undegraded":"2024-04-25T08:35:44.354667+0000","last_fullsized":"2024-04-25T08:35:44.354667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355036+0000","last_change":"2024-04-25T08:35:43.095386+0000","last_active":"2024-04-25T08:35:44.355036+0000","last_peered":"2024-04-25T08:35:44.355036+0000","last_clean":"2024-04-25T08:35:44.355036+0000","last_became_active":"2024-04-25T08:35:43.093364+0000","last_became_peered":"2024-04-25T08:35:43.093364+0000","last_unstale":"2024-04-25T08:35:44.355036+0000","last_undegraded":"2024-04-25T08:35:44.355036+0000","last_fullsized":"2024-04-25T08:35:44.355036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063458+0000","last_change":"2024-04-25T08:35:41.063458+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063458+0000","last_undegraded":"2024-04-25T08:35:41.063458+0000","last_fullsized":"2024-04-25T08:35:41.063458+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:43.087418+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:43.087100+0000","last_became_peered":"2024-04-25T08:35:43.087100+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:23.384291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354284+0000","last_change":"2024-04-25T08:35:43.100537+0000","last_active":"2024-04-25T08:35:44.354284+0000","last_peered":"2024-04-25T08:35:44.354284+0000","last_clean":"2024-04-25T08:35:44.354284+0000","last_became_active":"2024-04-25T08:35:43.100246+0000","last_became_peered":"2024-04-25T08:35:43.100246+0000","last_unstale":"2024-04-25T08:35:44.354284+0000","last_undegraded":"2024-04-25T08:35:44.354284+0000","last_fullsized":"2024-04-25T08:35:44.354284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354726+0000","last_change":"2024-04-25T08:35:42.079064+0000","last_active":"2024-04-25T08:35:44.354726+0000","last_peered":"2024-04-25T08:35:44.354726+0000","last_clean":"2024-04-25T08:35:44.354726+0000","last_became_active":"2024-04-25T08:35:42.078692+0000","last_became_peered":"2024-04-25T08:35:42.078692+0000","last_unstale":"2024-04-25T08:35:44.354726+0000","last_undegraded":"2024-04-25T08:35:44.354726+0000","last_fullsized":"2024-04-25T08:35:44.354726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:03.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353823+0000","last_change":"2024-04-25T08:35:43.088607+0000","last_active":"2024-04-25T08:35:44.353823+0000","last_peered":"2024-04-25T08:35:44.353823+0000","last_clean":"2024-04-25T08:35:44.353823+0000","last_became_active":"2024-04-25T08:35:43.088317+0000","last_became_peered":"2024-04-25T08:35:43.088317+0000","last_unstale":"2024-04-25T08:35:44.353823+0000","last_undegraded":"2024-04-25T08:35:44.353823+0000","last_fullsized":"2024-04-25T08:35:44.353823+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:18.622878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355062+0000","last_change":"2024-04-25T08:35:42.086642+0000","last_active":"2024-04-25T08:35:44.355062+0000","last_peered":"2024-04-25T08:35:44.355062+0000","last_clean":"2024-04-25T08:35:44.355062+0000","last_became_active":"2024-04-25T08:35:42.086219+0000","last_became_peered":"2024-04-25T08:35:42.086219+0000","last_unstale":"2024-04-25T08:35:44.355062+0000","last_undegraded":"2024-04-25T08:35:44.355062+0000","last_fullsized":"2024-04-25T08:35:44.355062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:16.239961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.060959+0000","last_change":"2024-04-25T08:35:41.060959+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.060959+0000","last_undegraded":"2024-04-25T08:35:41.060959+0000","last_fullsized":"2024-04-25T08:35:41.060959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063514+0000","last_change":"2024-04-25T08:35:41.063514+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063514+0000","last_undegraded":"2024-04-25T08:35:41.063514+0000","last_fullsized":"2024-04-25T08:35:41.063514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084175+0000","last_change":"2024-04-25T08:35:43.092287+0000","last_active":"2024-04-25T08:35:44.084175+0000","last_peered":"2024-04-25T08:35:44.084175+0000","last_clean":"2024-04-25T08:35:44.084175+0000","last_became_active":"2024-04-25T08:35:43.091768+0000","last_became_peered":"2024-04-25T08:35:43.091768+0000","last_unstale":"2024-04-25T08:35:44.084175+0000","last_undegraded":"2024-04-25T08:35:44.084175+0000","last_fullsized":"2024-04-25T08:35:44.084175+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084367+0000","last_change":"2024-04-25T08:35:42.075127+0000","last_active":"2024-04-25T08:35:44.084367+0000","last_peered":"2024-04-25T08:35:44.084367+0000","last_clean":"2024-04-25T08:35:44.084367+0000","last_became_active":"2024-04-25T08:35:42.074842+0000","last_became_peered":"2024-04-25T08:35:42.074842+0000","last_unstale":"2024-04-25T08:35:44.084367+0000","last_undegraded":"2024-04-25T08:35:44.084367+0000","last_fullsized":"2024-04-25T08:35:44.084367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.370203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068005+0000","last_change":"2024-04-25T08:35:41.068005+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068005+0000","last_undegraded":"2024-04-25T08:35:41.068005+0000","last_fullsized":"2024-04-25T08:35:41.068005+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354254+0000","last_change":"2024-04-25T08:35:42.077041+0000","last_active":"2024-04-25T08:35:44.354254+0000","last_peered":"2024-04-25T08:35:44.354254+0000","last_clean":"2024-04-25T08:35:44.354254+0000","last_became_active":"2024-04-25T08:35:42.076737+0000","last_became_peered":"2024-04-25T08:35:42.076737+0000","last_unstale":"2024-04-25T08:35:44.354254+0000","last_undegraded":"2024-04-25T08:35:44.354254+0000","last_fullsized":"2024-04-25T08:35:44.354254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:54.769371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.067973+0000","last_change":"2024-04-25T08:35:41.067973+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.067973+0000","last_undegraded":"2024-04-25T08:35:41.067973+0000","last_fullsized":"2024-04-25T08:35:41.067973+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353713+0000","last_change":"2024-04-25T08:35:42.079469+0000","last_active":"2024-04-25T08:35:44.353713+0000","last_peered":"2024-04-25T08:35:44.353713+0000","last_clean":"2024-04-25T08:35:44.353713+0000","last_became_active":"2024-04-25T08:35:42.079187+0000","last_became_peered":"2024-04-25T08:35:42.079187+0000","last_unstale":"2024-04-25T08:35:44.353713+0000","last_undegraded":"2024-04-25T08:35:44.353713+0000","last_fullsized":"2024-04-25T08:35:44.353713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.436872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063728+0000","last_change":"2024-04-25T08:35:41.063728+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063728+0000","last_undegraded":"2024-04-25T08:35:41.063728+0000","last_fullsized":"2024-04-25T08:35:41.063728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088238+0000","last_change":"2024-04-25T08:35:42.080608+0000","last_active":"2024-04-25T08:35:44.088238+0000","last_peered":"2024-04-25T08:35:44.088238+0000","last_clean":"2024-04-25T08:35:44.088238+0000","last_became_active":"2024-04-25T08:35:42.080267+0000","last_became_peered":"2024-04-25T08:35:42.080267+0000","last_unstale":"2024-04-25T08:35:44.088238+0000","last_undegraded":"2024-04-25T08:35:44.088238+0000","last_fullsized":"2024-04-25T08:35:44.088238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:09.842643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084421+0000","last_change":"2024-04-25T08:35:42.084974+0000","last_active":"2024-04-25T08:35:44.084421+0000","last_peered":"2024-04-25T08:35:44.084421+0000","last_clean":"2024-04-25T08:35:44.084421+0000","last_became_active":"2024-04-25T08:35:42.084703+0000","last_became_peered":"2024-04-25T08:35:42.084703+0000","last_unstale":"2024-04-25T08:35:44.084421+0000","last_undegraded":"2024-04-25T08:35:44.084421+0000","last_fullsized":"2024-04-25T08:35:44.084421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:56.731886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.061055+0000","last_change":"2024-04-25T08:35:41.061055+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.061055+0000","last_undegraded":"2024-04-25T08:35:41.061055+0000","last_fullsized":"2024-04-25T08:35:41.061055+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353624+0000","last_change":"2024-04-25T08:35:43.089060+0000","last_active":"2024-04-25T08:35:44.353624+0000","last_peered":"2024-04-25T08:35:44.353624+0000","last_clean":"2024-04-25T08:35:44.353624+0000","last_became_active":"2024-04-25T08:35:43.088777+0000","last_became_peered":"2024-04-25T08:35:43.088777+0000","last_unstale":"2024-04-25T08:35:44.353624+0000","last_undegraded":"2024-04-25T08:35:44.353624+0000","last_fullsized":"2024-04-25T08:35:44.353624+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:58.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:41.056371+0000","last_change":"2024-04-25T08:35:41.056371+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.056371+0000","last_undegraded":"2024-04-25T08:35:41.056371+0000","last_fullsized":"2024-04-25T08:35:41.056371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354681+0000","last_change":"2024-04-25T08:35:43.080706+0000","last_active":"2024-04-25T08:35:44.354681+0000","last_peered":"2024-04-25T08:35:44.354681+0000","last_clean":"2024-04-25T08:35:44.354681+0000","last_became_active":"2024-04-25T08:35:43.080411+0000","last_became_peered":"2024-04-25T08:35:43.080411+0000","last_unstale":"2024-04-25T08:35:44.354681+0000","last_undegraded":"2024-04-25T08:35:44.354681+0000","last_fullsized":"2024-04-25T08:35:44.354681+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.068003+0000","last_change":"2024-04-25T08:35:41.068003+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.068003+0000","last_undegraded":"2024-04-25T08:35:41.068003+0000","last_fullsized":"2024-04-25T08:35:41.068003+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.060868+0000","last_change":"2024-04-25T08:35:41.060868+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.060868+0000","last_undegraded":"2024-04-25T08:35:41.060868+0000","last_fullsized":"2024-04-25T08:35:41.060868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":94,"state":"creating+peering","last_fresh":"2024-04-25T08:35:41.063500+0000","last_change":"2024-04-25T08:35:41.063500+0000","last_active":"2024-04-25T08:35:41.056371+0000","last_peered":"2024-04-25T08:35:41.056371+0000","last_clean":"2024-04-25T08:35:41.056371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:41.063500+0000","last_undegraded":"2024-04-25T08:35:41.063500+0000","last_fullsized":"2024-04-25T08:35:41.063500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:35:42.058754+0000","last_change":"2024-04-25T08:35:42.058754+0000","last_active":"2024-04-25T08:35:42.058754+0000","last_peered":"2024-04-25T08:35:42.058754+0000","last_clean":"2024-04-25T08:35:42.058754+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:35:42.058754+0000","last_undegraded":"2024-04-25T08:35:42.058754+0000","last_fullsized":"2024-04-25T08:35:42.058754+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354943+0000","last_change":"2024-04-25T08:35:42.080661+0000","last_active":"2024-04-25T08:35:44.354943+0000","last_peered":"2024-04-25T08:35:44.354943+0000","last_clean":"2024-04-25T08:35:44.354943+0000","last_became_active":"2024-04-25T08:35:42.080378+0000","last_became_peered":"2024-04-25T08:35:42.080378+0000","last_unstale":"2024-04-25T08:35:44.354943+0000","last_undegraded":"2024-04-25T08:35:44.354943+0000","last_fullsized":"2024-04-25T08:35:44.354943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:11.863733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":309,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.064192+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:35:41.064192+0000","last_peered":"2024-04-25T08:35:41.064192+0000","last_clean":"2024-04-25T08:35:41.064192+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:35:41.064192+0000","last_undegraded":"2024-04-25T08:35:41.064192+0000","last_fullsized":"2024-04-25T08:35:41.064192+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":160,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":58,"acting":58,"num_store_stats":7},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":68,"acting":68,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41196,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330644,"statfs":{"total":96636764160,"available":96594579456,"internally_reserved":0,"allocated":1736704,"data_stored":1115590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1155072,"data_stored":531431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40632,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331208,"statfs":{"total":96636764160,"available":96595156992,"internally_reserved":0,"allocated":1155072,"data_stored":531431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":552117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50392,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321448,"statfs":{"total":96636764160,"available":96585162752,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54556,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317284,"statfs":{"total":96636764160,"available":96580898816,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":552027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55084,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316756,"statfs":{"total":96636764160,"available":96580358144,"internally_reserved":0,"allocated":1736704,"data_stored":1115590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.778 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.779 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.061987+0000', 'last_change': '2024-04-25T08:35:41.061987+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.061987+0000', 'last_undegraded': '2024-04-25T08:35:41.061987+0000', 'last_fullsized': '2024-04-25T08:35:41.061987+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.780 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.063607+0000', 'last_change': '2024-04-25T08:35:41.063607+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.063607+0000', 'last_undegraded': '2024-04-25T08:35:41.063607+0000', 'last_fullsized': '2024-04-25T08:35:41.063607+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.061876+0000', 'last_change': '2024-04-25T08:35:41.061876+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.061876+0000', 'last_undegraded': '2024-04-25T08:35:41.061876+0000', 'last_fullsized': '2024-04-25T08:35:41.061876+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2024-04-25T08:35:46.781 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.063996+0000', 'last_change': '2024-04-25T08:35:41.063996+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.063996+0000', 'last_undegraded': '2024-04-25T08:35:41.063996+0000', 'last_fullsized': '2024-04-25T08:35:41.063996+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.068599+0000', 'last_change': '2024-04-25T08:35:41.068599+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.068599+0000', 'last_undegraded': '2024-04-25T08:35:41.068599+0000', 'last_fullsized': '2024-04-25T08:35:41.068599+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:41.056371+0000', 'last_change': '2024-04-25T08:35:41.056371+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.056371+0000', 'last_undegraded': '2024-04-25T08:35:41.056371+0000', 'last_fullsized': '2024-04-25T08:35:41.056371+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.063458+0000', 'last_change': '2024-04-25T08:35:41.063458+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.063458+0000', 'last_undegraded': '2024-04-25T08:35:41.063458+0000', 'last_fullsized': '2024-04-25T08:35:41.063458+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-04-25T08:35:46.782 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:41.056371+0000', 'last_change': '2024-04-25T08:35:41.056371+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.056371+0000', 'last_undegraded': '2024-04-25T08:35:41.056371+0000', 'last_fullsized': '2024-04-25T08:35:41.056371+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.060959+0000', 'last_change': '2024-04-25T08:35:41.060959+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.060959+0000', 'last_undegraded': '2024-04-25T08:35:41.060959+0000', 'last_fullsized': '2024-04-25T08:35:41.060959+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.063514+0000', 'last_change': '2024-04-25T08:35:41.063514+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.063514+0000', 'last_undegraded': '2024-04-25T08:35:41.063514+0000', 'last_fullsized': '2024-04-25T08:35:41.063514+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:35:46.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.068005+0000', 'last_change': '2024-04-25T08:35:41.068005+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.068005+0000', 'last_undegraded': '2024-04-25T08:35:41.068005+0000', 'last_fullsized': '2024-04-25T08:35:41.068005+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.067973+0000', 'last_change': '2024-04-25T08:35:41.067973+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.067973+0000', 'last_undegraded': '2024-04-25T08:35:41.067973+0000', 'last_fullsized': '2024-04-25T08:35:41.067973+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T08:35:46.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.063728+0000', 'last_change': '2024-04-25T08:35:41.063728+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.063728+0000', 'last_undegraded': '2024-04-25T08:35:41.063728+0000', 'last_fullsized': '2024-04-25T08:35:41.063728+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2024-04-25T08:35:46.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.061055+0000', 'last_change': '2024-04-25T08:35:41.061055+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.061055+0000', 'last_undegraded': '2024-04-25T08:35:41.061055+0000', 'last_fullsized': '2024-04-25T08:35:41.061055+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:41.056371+0000', 'last_change': '2024-04-25T08:35:41.056371+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.056371+0000', 'last_undegraded': '2024-04-25T08:35:41.056371+0000', 'last_fullsized': '2024-04-25T08:35:41.056371+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.068003+0000', 'last_change': '2024-04-25T08:35:41.068003+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.068003+0000', 'last_undegraded': '2024-04-25T08:35:41.068003+0000', 'last_fullsized': '2024-04-25T08:35:41.068003+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.060868+0000', 'last_change': '2024-04-25T08:35:41.060868+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.060868+0000', 'last_undegraded': '2024-04-25T08:35:41.060868+0000', 'last_fullsized': '2024-04-25T08:35:41.060868+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2024-04-25T08:35:46.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 94, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:35:41.063500+0000', 'last_change': '2024-04-25T08:35:41.063500+0000', 'last_active': '2024-04-25T08:35:41.056371+0000', 'last_peered': '2024-04-25T08:35:41.056371+0000', 'last_clean': '2024-04-25T08:35:41.056371+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:41.063500+0000', 'last_undegraded': '2024-04-25T08:35:41.063500+0000', 'last_fullsized': '2024-04-25T08:35:41.063500+0000', 'mapping_epoch': 94, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 94, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:41.056371+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:35:46.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T08:35:46.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:35:42.058754+0000', 'last_change': '2024-04-25T08:35:42.058754+0000', 'last_active': '2024-04-25T08:35:42.058754+0000', 'last_peered': '2024-04-25T08:35:42.058754+0000', 'last_clean': '2024-04-25T08:35:42.058754+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:35:42.058754+0000', 'last_undegraded': '2024-04-25T08:35:42.058754+0000', 'last_fullsized': '2024-04-25T08:35:42.058754+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'last_clean_scrub_stamp': '2024-04-25T08:35:42.058754+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:35:46.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:35:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.919+0000 7f17b7207640 1 Processor -- start 2024-04-25T08:35:46.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.919+0000 7f17b7207640 1 -- start start 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17b7207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b00cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17b7207640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17b00cfcf0 0x7f17b00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17b7207640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b0071480 con 0x7f17b00ce3c0 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17b7207640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b00ce830 con 0x7f17b00cfcf0 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17b7207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0072090 con 0x7f17b0154ee0 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17b00cfcf0 0x7f17b00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17b00cfcf0 0x7f17b00cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35802/0 (socket says 172.21.15.60:35802) 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 -- 172.21.15.60:0/1577436772 learned_addr learned my addr 172.21.15.60:0/1577436772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 -- 172.21.15.60:0/1577436772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 msgr2=0x7f17b00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 --2- 172.21.15.60:0/1577436772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b00cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 -- 172.21.15.60:0/1577436772 >> v1:172.21.15.100:6789/0 conn(0x7f17b00ce3c0 legacy=0x7f17b015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.920+0000 7f17affff640 1 -- 172.21.15.60:0/1577436772 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0071760 con 0x7f17b00cfcf0 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.921+0000 7f17affff640 1 --2- 172.21.15.60:0/1577436772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00cfcf0 0x7f17b00cde80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009e50 tx=0x7f17a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8e998dafcedaa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.921+0000 7f17af7fe640 1 -- 172.21.15.60:0/1577436772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0036070 con 0x7f17b00cfcf0 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.921+0000 7f17af7fe640 1 -- 172.21.15.60:0/1577436772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0031600 con 0x7f17b00cfcf0 2024-04-25T08:35:46.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.921+0000 7f17af7fe640 1 -- 172.21.15.60:0/1577436772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00319d0 con 0x7f17b00cfcf0 2024-04-25T08:35:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 -- 172.21.15.60:0/1577436772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00cfcf0 msgr2=0x7f17b00cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 --2- 172.21.15.60:0/1577436772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00cfcf0 0x7f17b00cde80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009e50 tx=0x7f17a002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 -- 172.21.15.60:0/1577436772 shutdown_connections 2024-04-25T08:35:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 --2- 172.21.15.60:0/1577436772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00cfcf0 0x7f17b00cde80 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 --2- 172.21.15.60:0/1577436772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b00cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 -- 172.21.15.60:0/1577436772 >> 172.21.15.60:0/1577436772 conn(0x7f17b006abe0 msgr2=0x7f17b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 -- 172.21.15.60:0/1577436772 shutdown_connections 2024-04-25T08:35:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.922+0000 7f17b7207640 1 -- 172.21.15.60:0/1577436772 wait complete. 2024-04-25T08:35:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.923+0000 7f17b7207640 1 Processor -- start 2024-04-25T08:35:46.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.923+0000 7f17b7207640 1 -- start start 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b7207640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 0x7f17b00afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b7207640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b00cfcf0 0x7f17b0144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b7207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b0144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b7207640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0071230 con 0x7f17b00ce3c0 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b7207640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0071d20 con 0x7f17b00cfcf0 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b7207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b01552c0 con 0x7f17b0154ee0 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 0x7f17b00afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 0x7f17b00afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35818/0 (socket says 172.21.15.60:35818) 2024-04-25T08:35:46.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 -- 172.21.15.60:0/3947381023 learned_addr learned my addr 172.21.15.60:0/3947381023 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b57a4640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b0144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17affff640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b00cfcf0 0x7f17b0144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 -- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 msgr2=0x7f17b0144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b0144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 -- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b00cfcf0 msgr2=0x7f17b0144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b00cfcf0 0x7f17b0144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b4fa3640 1 -- 172.21.15.60:0/3947381023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00b01e0 con 0x7f17b00ce3c0 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17affff640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b00cfcf0 0x7f17b0144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.924+0000 7f17b57a4640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b0144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.925+0000 7f17b4fa3640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 0x7f17b00afca0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b410 tx=0x7f17a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.925+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8015020 con 0x7f17b00ce3c0 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.925+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a8011850 con 0x7f17b00ce3c0 2024-04-25T08:35:46.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.925+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b00b08f0 con 0x7f17b00ce3c0 2024-04-25T08:35:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.925+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8011c40 con 0x7f17b00ce3c0 2024-04-25T08:35:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.925+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b01e6750 con 0x7f17b00ce3c0 2024-04-25T08:35:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.926+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b0073990 con 0x7f17b00ce3c0 2024-04-25T08:35:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.926+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17a8018020 con 0x7f17b00ce3c0 2024-04-25T08:35:46.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.927+0000 7f17ad7fa640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179006b4d0 0x7f179006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.927+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17a808a240 con 0x7f17b00ce3c0 2024-04-25T08:35:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.927+0000 7f17affff640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179006b4d0 0x7f179006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.928+0000 7f17affff640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179006b4d0 0x7f179006d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f17a0033000 tx=0x7f17a0009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:46.931+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17a801f020 con 0x7f17b00ce3c0 2024-04-25T08:35:47.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.181+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f17b00aeb60 con 0x7f17b00ce3c0 2024-04-25T08:35:47.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.181+0000 7f17ad7fa640 1 -- 172.21.15.60:0/3947381023 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1175 (secure 0 0 0) 0x7f17a80172c0 con 0x7f17b00ce3c0 2024-04-25T08:35:47.176 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:47.176 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":407,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":97,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":49},{"state_name":"unknown","count":33},{"state_name":"creating+peering","count":15}],"num_pgs":97,"num_pools":3,"num_objects":10,"data_bytes":590528,"bytes_used":365498368,"bytes_avail":772728614912,"bytes_total":773094113280,"unknown_pgs_ratio":0.34020617604255676,"inactive_pgs_ratio":0.15463916957378387,"write_bytes_sec":462,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T08:35:31.482633+0000","services":{}},"progress_events":{}} 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179006b4d0 msgr2=0x7f179006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179006b4d0 0x7f179006d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f17a0033000 tx=0x7f17a0009000 comp rx=0 tx=0).stop 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 msgr2=0x7f17b00afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 0x7f17b00afca0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f17a800b410 tx=0x7f17a800b8d0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 shutdown_connections 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17b0154ee0 0x7f17b0144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f179006b4d0 0x7f179006d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b00cfcf0 0x7f17b0144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 --2- 172.21.15.60:0/3947381023 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17b00ce3c0 0x7f17b00afca0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 >> 172.21.15.60:0/3947381023 conn(0x7f17b006abe0 msgr2=0x7f17b015d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 shutdown_connections 2024-04-25T08:35:47.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.183+0000 7f17b7207640 1 -- 172.21.15.60:0/3947381023 wait complete. 2024-04-25T08:35:47.189 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:35:47.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.296+0000 7f4949909640 1 Processor -- start 2024-04-25T08:35:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.296+0000 7f4949909640 1 -- start start 2024-04-25T08:35:47.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4949909640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49440d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4949909640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440b6220 con 0x7f49440b2cb0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4949909640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440b97a0 con 0x7f49440b92a0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4949909640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494415dfc0 con 0x7f494415dbe0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f493bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f49440b2cb0 0x7f49440b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56336/0 (socket says 172.21.15.60:56336) 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f493bfff640 1 -- 172.21.15.60:0/2310955446 learned_addr learned my addr 172.21.15.60:0/2310955446 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385705550 0 0) 0x7f49440b6220 con 0x7f49440b2cb0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4930003520 con 0x7f49440b2cb0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4942ffd640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49440d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1452017364 0 0) 0x7f4930003520 con 0x7f49440b2cb0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 msgr2=0x7f49440d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49440d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 >> v1:172.21.15.100:6789/0 conn(0x7f49440b92a0 legacy=0x7f49440b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440d1c50 con 0x7f49440b2cb0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f492c001670 con 0x7f49440b2cb0 2024-04-25T08:35:47.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4942ffd640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49440d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 >> v1:172.21.15.60:6789/0 conn(0x7f49440b2cb0 legacy=0x7f49440b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49300039e0 0x7f4930003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930004800 0x7f4930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493000ba90 con 0x7f49300039e0 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493000bc40 con 0x7f4930004800 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493000be40 con 0x7f494415dbe0 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.297+0000 7f4942ffd640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49300039e0 0x7f4930003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f493bfff640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930004800 0x7f4930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f49437fe640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f4942ffd640 1 -- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 msgr2=0x7f49300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f4942ffd640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f494415dbe0 0x7f49300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f4942ffd640 1 -- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930004800 msgr2=0x7f4930004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f4942ffd640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930004800 0x7f4930004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f4942ffd640 1 -- 172.21.15.60:0/2310955446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930003520 con 0x7f49300039e0 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f493bfff640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930004800 0x7f4930004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.298+0000 7f4942ffd640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49300039e0 0x7f4930003e00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f49340098a0 tx=0x7f4930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5191a30b1b8803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400f040 con 0x7f49300039e0 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493402ace0 con 0x7f49300039e0 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f49427fc640 1 -- 172.21.15.60:0/2310955446 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934004250 con 0x7f49300039e0 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f4949909640 1 -- 172.21.15.60:0/2310955446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49300039e0 msgr2=0x7f4930003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f4949909640 1 --2- 172.21.15.60:0/2310955446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49300039e0 0x7f4930003e00 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f49340098a0 tx=0x7f4930005830 comp rx=0 tx=0).stop 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f49437fe640 1 -- 172.21.15.60:0/2310955446 reap_dead start 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f4949909640 1 -- 172.21.15.60:0/2310955446 shutdown_connections 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f4949909640 1 -- 172.21.15.60:0/2310955446 >> 172.21.15.60:0/2310955446 conn(0x7f4944061a20 msgr2=0x7f4944061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f4949909640 1 -- 172.21.15.60:0/2310955446 shutdown_connections 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.299+0000 7f4949909640 1 -- 172.21.15.60:0/2310955446 wait complete. 2024-04-25T08:35:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.300+0000 7f4949909640 1 Processor -- start 2024-04-25T08:35:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.300+0000 7f4949909640 1 -- start start 2024-04-25T08:35:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4949909640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49300039e0 0x7f49440cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4949909640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49441609c0 0x7f49440cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4949909640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 0x7f49440aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4949909640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440d2210 con 0x7f49440b1bc0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4949909640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440d1a00 con 0x7f49300039e0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4949909640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440b6ab0 con 0x7f49441609c0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f49437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 0x7f49440aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f49437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 0x7f49440aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35824/0 (socket says 172.21.15.60:35824) 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f49437fe640 1 -- 172.21.15.60:0/3811186910 learned_addr learned my addr 172.21.15.60:0/3811186910 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f493bfff640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49441609c0 0x7f49440cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.301+0000 7f4942ffd640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49300039e0 0x7f49440cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f49437fe640 1 -- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49441609c0 msgr2=0x7f49440cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f49437fe640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49441609c0 0x7f49440cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f49437fe640 1 -- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49300039e0 msgr2=0x7f49440cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f49437fe640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49300039e0 0x7f49440cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f49437fe640 1 -- 172.21.15.60:0/3811186910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440cc710 con 0x7f49440b1bc0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f493bfff640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49441609c0 0x7f49440cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f49437fe640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 0x7f49440aa370 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f49440d3930 tx=0x7f493c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:47.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c003970 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c018070 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c003d50 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440cc9e0 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.302+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440ccfa0 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.303+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944073990 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.303+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f493c0050d0 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.304+0000 7f4940ff9640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f492406b4d0 0x7f492406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.304+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f493c08a090 con 0x7f49440b1bc0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.304+0000 7f4942ffd640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f492406b4d0 0x7f492406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.304+0000 7f4942ffd640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f492406b4d0 0x7f492406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4934033000 tx=0x7f493402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:47.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.308+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f493c058cf0 con 0x7f49440b1bc0 2024-04-25T08:35:47.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.489+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49440b6670 con 0x7f492406b4d0 2024-04-25T08:35:47.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.496+0000 7f4940ff9640 1 -- 172.21.15.60:0/3811186910 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7f49440b6670 con 0x7f492406b4d0 2024-04-25T08:35:47.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:47.494 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.498+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f492406b4d0 msgr2=0x7f492406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.498+0000 7f4949909640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f492406b4d0 0x7f492406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4934033000 tx=0x7f493402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.498+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 msgr2=0x7f49440aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 0x7f49440aa370 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f49440d3930 tx=0x7f493c00eb90 comp rx=0 tx=0).stop 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 shutdown_connections 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f49440b1bc0 0x7f49440aa370 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f49441609c0 0x7f49440cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f492406b4d0 0x7f492406d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 --2- 172.21.15.60:0/3811186910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49300039e0 0x7f49440cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 >> 172.21.15.60:0/3811186910 conn(0x7f4944061a20 msgr2=0x7f494415cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 shutdown_connections 2024-04-25T08:35:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:47.499+0000 7f4949909640 1 -- 172.21.15.60:0/3811186910 wait complete. 2024-04-25T08:35:47.504 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-04-25T08:35:47.486010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":375220,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599500,"statfs":{"total":773094113280,"available":772709888000,"internally_reserved":0,"allocated":10788864,"data_stored":5601212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001356"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353876+0000","last_change":"2024-04-25T08:35:43.090294+0000","last_active":"2024-04-25T08:35:44.353876+0000","last_peered":"2024-04-25T08:35:44.353876+0000","last_clean":"2024-04-25T08:35:44.353876+0000","last_became_active":"2024-04-25T08:35:43.089694+0000","last_became_peered":"2024-04-25T08:35:43.089694+0000","last_unstale":"2024-04-25T08:35:44.353876+0000","last_undegraded":"2024-04-25T08:35:44.353876+0000","last_fullsized":"2024-04-25T08:35:44.353876+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088889+0000","last_change":"2024-04-25T08:35:43.090499+0000","last_active":"2024-04-25T08:35:44.088889+0000","last_peered":"2024-04-25T08:35:44.088889+0000","last_clean":"2024-04-25T08:35:44.088889+0000","last_became_active":"2024-04-25T08:35:43.090183+0000","last_became_peered":"2024-04-25T08:35:43.090183+0000","last_unstale":"2024-04-25T08:35:44.088889+0000","last_undegraded":"2024-04-25T08:35:44.088889+0000","last_fullsized":"2024-04-25T08:35:44.088889+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:01.231065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353866+0000","last_change":"2024-04-25T08:35:43.081017+0000","last_active":"2024-04-25T08:35:44.353866+0000","last_peered":"2024-04-25T08:35:44.353866+0000","last_clean":"2024-04-25T08:35:44.353866+0000","last_became_active":"2024-04-25T08:35:43.080694+0000","last_became_peered":"2024-04-25T08:35:43.080694+0000","last_unstale":"2024-04-25T08:35:44.353866+0000","last_undegraded":"2024-04-25T08:35:44.353866+0000","last_fullsized":"2024-04-25T08:35:44.353866+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354842+0000","last_change":"2024-04-25T08:35:43.095431+0000","last_active":"2024-04-25T08:35:44.354842+0000","last_peered":"2024-04-25T08:35:44.354842+0000","last_clean":"2024-04-25T08:35:44.354842+0000","last_became_active":"2024-04-25T08:35:43.095148+0000","last_became_peered":"2024-04-25T08:35:43.095148+0000","last_unstale":"2024-04-25T08:35:44.354842+0000","last_undegraded":"2024-04-25T08:35:44.354842+0000","last_fullsized":"2024-04-25T08:35:44.354842+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083569+0000","last_change":"2024-04-25T08:35:43.080094+0000","last_active":"2024-04-25T08:35:44.083569+0000","last_peered":"2024-04-25T08:35:44.083569+0000","last_clean":"2024-04-25T08:35:44.083569+0000","last_became_active":"2024-04-25T08:35:43.079466+0000","last_became_peered":"2024-04-25T08:35:43.079466+0000","last_unstale":"2024-04-25T08:35:44.083569+0000","last_undegraded":"2024-04-25T08:35:44.083569+0000","last_fullsized":"2024-04-25T08:35:44.083569+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083148+0000","last_change":"2024-04-25T08:35:43.086363+0000","last_active":"2024-04-25T08:35:44.083148+0000","last_peered":"2024-04-25T08:35:44.083148+0000","last_clean":"2024-04-25T08:35:44.083148+0000","last_became_active":"2024-04-25T08:35:43.085889+0000","last_became_peered":"2024-04-25T08:35:43.085889+0000","last_unstale":"2024-04-25T08:35:44.083148+0000","last_undegraded":"2024-04-25T08:35:44.083148+0000","last_fullsized":"2024-04-25T08:35:44.083148+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:02.377998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347483+0000","last_change":"2024-04-25T08:35:43.089185+0000","last_active":"2024-04-25T08:35:44.347483+0000","last_peered":"2024-04-25T08:35:44.347483+0000","last_clean":"2024-04-25T08:35:44.347483+0000","last_became_active":"2024-04-25T08:35:43.088871+0000","last_became_peered":"2024-04-25T08:35:43.088871+0000","last_unstale":"2024-04-25T08:35:44.347483+0000","last_undegraded":"2024-04-25T08:35:44.347483+0000","last_fullsized":"2024-04-25T08:35:44.347483+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:29.018247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354780+0000","last_change":"2024-04-25T08:35:43.095408+0000","last_active":"2024-04-25T08:35:44.354780+0000","last_peered":"2024-04-25T08:35:44.354780+0000","last_clean":"2024-04-25T08:35:44.354780+0000","last_became_active":"2024-04-25T08:35:43.095119+0000","last_became_peered":"2024-04-25T08:35:43.095119+0000","last_unstale":"2024-04-25T08:35:44.354780+0000","last_undegraded":"2024-04-25T08:35:44.354780+0000","last_fullsized":"2024-04-25T08:35:44.354780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083849+0000","last_change":"2024-04-25T08:35:43.087871+0000","last_active":"2024-04-25T08:35:44.083849+0000","last_peered":"2024-04-25T08:35:44.083849+0000","last_clean":"2024-04-25T08:35:44.083849+0000","last_became_active":"2024-04-25T08:35:43.087575+0000","last_became_peered":"2024-04-25T08:35:43.087575+0000","last_unstale":"2024-04-25T08:35:44.083849+0000","last_undegraded":"2024-04-25T08:35:44.083849+0000","last_fullsized":"2024-04-25T08:35:44.083849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:32.414798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354147+0000","last_change":"2024-04-25T08:35:43.101307+0000","last_active":"2024-04-25T08:35:44.354147+0000","last_peered":"2024-04-25T08:35:44.354147+0000","last_clean":"2024-04-25T08:35:44.354147+0000","last_became_active":"2024-04-25T08:35:43.101003+0000","last_became_peered":"2024-04-25T08:35:43.101003+0000","last_unstale":"2024-04-25T08:35:44.354147+0000","last_undegraded":"2024-04-25T08:35:44.354147+0000","last_fullsized":"2024-04-25T08:35:44.354147+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:14.081591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084050+0000","last_change":"2024-04-25T08:35:43.079921+0000","last_active":"2024-04-25T08:35:44.084050+0000","last_peered":"2024-04-25T08:35:44.084050+0000","last_clean":"2024-04-25T08:35:44.084050+0000","last_became_active":"2024-04-25T08:35:43.079607+0000","last_became_peered":"2024-04-25T08:35:43.079607+0000","last_unstale":"2024-04-25T08:35:44.084050+0000","last_undegraded":"2024-04-25T08:35:44.084050+0000","last_fullsized":"2024-04-25T08:35:44.084050+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:57.986549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353925+0000","last_change":"2024-04-25T08:35:43.087944+0000","last_active":"2024-04-25T08:35:44.353925+0000","last_peered":"2024-04-25T08:35:44.353925+0000","last_clean":"2024-04-25T08:35:44.353925+0000","last_became_active":"2024-04-25T08:35:43.087638+0000","last_became_peered":"2024-04-25T08:35:43.087638+0000","last_unstale":"2024-04-25T08:35:44.353925+0000","last_undegraded":"2024-04-25T08:35:44.353925+0000","last_fullsized":"2024-04-25T08:35:44.353925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.692505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354540+0000","last_change":"2024-04-25T08:35:43.095160+0000","last_active":"2024-04-25T08:35:44.354540+0000","last_peered":"2024-04-25T08:35:44.354540+0000","last_clean":"2024-04-25T08:35:44.354540+0000","last_became_active":"2024-04-25T08:35:43.094865+0000","last_became_peered":"2024-04-25T08:35:43.094865+0000","last_unstale":"2024-04-25T08:35:44.354540+0000","last_undegraded":"2024-04-25T08:35:44.354540+0000","last_fullsized":"2024-04-25T08:35:44.354540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083498+0000","last_change":"2024-04-25T08:35:43.096325+0000","last_active":"2024-04-25T08:35:44.083498+0000","last_peered":"2024-04-25T08:35:44.083498+0000","last_clean":"2024-04-25T08:35:44.083498+0000","last_became_active":"2024-04-25T08:35:43.096042+0000","last_became_peered":"2024-04-25T08:35:43.096042+0000","last_unstale":"2024-04-25T08:35:44.083498+0000","last_undegraded":"2024-04-25T08:35:44.083498+0000","last_fullsized":"2024-04-25T08:35:44.083498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:27.748314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354752+0000","last_change":"2024-04-25T08:35:43.080738+0000","last_active":"2024-04-25T08:35:44.354752+0000","last_peered":"2024-04-25T08:35:44.354752+0000","last_clean":"2024-04-25T08:35:44.354752+0000","last_became_active":"2024-04-25T08:35:43.080444+0000","last_became_peered":"2024-04-25T08:35:43.080444+0000","last_unstale":"2024-04-25T08:35:44.354752+0000","last_undegraded":"2024-04-25T08:35:44.354752+0000","last_fullsized":"2024-04-25T08:35:44.354752+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083259+0000","last_change":"2024-04-25T08:35:43.079286+0000","last_active":"2024-04-25T08:35:44.083259+0000","last_peered":"2024-04-25T08:35:44.083259+0000","last_clean":"2024-04-25T08:35:44.083259+0000","last_became_active":"2024-04-25T08:35:43.078966+0000","last_became_peered":"2024-04-25T08:35:43.078966+0000","last_unstale":"2024-04-25T08:35:44.083259+0000","last_undegraded":"2024-04-25T08:35:44.083259+0000","last_fullsized":"2024-04-25T08:35:44.083259+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:48.519958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083467+0000","last_change":"2024-04-25T08:35:43.079723+0000","last_active":"2024-04-25T08:35:44.083467+0000","last_peered":"2024-04-25T08:35:44.083467+0000","last_clean":"2024-04-25T08:35:44.083467+0000","last_became_active":"2024-04-25T08:35:43.079074+0000","last_became_peered":"2024-04-25T08:35:43.079074+0000","last_unstale":"2024-04-25T08:35:44.083467+0000","last_undegraded":"2024-04-25T08:35:44.083467+0000","last_fullsized":"2024-04-25T08:35:44.083467+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:07.692269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354121+0000","last_change":"2024-04-25T08:35:43.089177+0000","last_active":"2024-04-25T08:35:44.354121+0000","last_peered":"2024-04-25T08:35:44.354121+0000","last_clean":"2024-04-25T08:35:44.354121+0000","last_became_active":"2024-04-25T08:35:43.088869+0000","last_became_peered":"2024-04-25T08:35:43.088869+0000","last_unstale":"2024-04-25T08:35:44.354121+0000","last_undegraded":"2024-04-25T08:35:44.354121+0000","last_fullsized":"2024-04-25T08:35:44.354121+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084198+0000","last_change":"2024-04-25T08:35:43.094595+0000","last_active":"2024-04-25T08:35:44.084198+0000","last_peered":"2024-04-25T08:35:44.084198+0000","last_clean":"2024-04-25T08:35:44.084198+0000","last_became_active":"2024-04-25T08:35:43.093864+0000","last_became_peered":"2024-04-25T08:35:43.093864+0000","last_unstale":"2024-04-25T08:35:44.084198+0000","last_undegraded":"2024-04-25T08:35:44.084198+0000","last_fullsized":"2024-04-25T08:35:44.084198+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354674+0000","last_change":"2024-04-25T08:35:43.086251+0000","last_active":"2024-04-25T08:35:44.354674+0000","last_peered":"2024-04-25T08:35:44.354674+0000","last_clean":"2024-04-25T08:35:44.354674+0000","last_became_active":"2024-04-25T08:35:43.085938+0000","last_became_peered":"2024-04-25T08:35:43.085938+0000","last_unstale":"2024-04-25T08:35:44.354674+0000","last_undegraded":"2024-04-25T08:35:44.354674+0000","last_fullsized":"2024-04-25T08:35:44.354674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084122+0000","last_change":"2024-04-25T08:35:43.084909+0000","last_active":"2024-04-25T08:35:44.084122+0000","last_peered":"2024-04-25T08:35:44.084122+0000","last_clean":"2024-04-25T08:35:44.084122+0000","last_became_active":"2024-04-25T08:35:43.084519+0000","last_became_peered":"2024-04-25T08:35:43.084519+0000","last_unstale":"2024-04-25T08:35:44.084122+0000","last_undegraded":"2024-04-25T08:35:44.084122+0000","last_fullsized":"2024-04-25T08:35:44.084122+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083572+0000","last_change":"2024-04-25T08:35:43.083871+0000","last_active":"2024-04-25T08:35:44.083572+0000","last_peered":"2024-04-25T08:35:44.083572+0000","last_clean":"2024-04-25T08:35:44.083572+0000","last_became_active":"2024-04-25T08:35:43.083574+0000","last_became_peered":"2024-04-25T08:35:43.083574+0000","last_unstale":"2024-04-25T08:35:44.083572+0000","last_undegraded":"2024-04-25T08:35:44.083572+0000","last_fullsized":"2024-04-25T08:35:44.083572+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.102694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084409+0000","last_change":"2024-04-25T08:35:43.100281+0000","last_active":"2024-04-25T08:35:44.084409+0000","last_peered":"2024-04-25T08:35:44.084409+0000","last_clean":"2024-04-25T08:35:44.084409+0000","last_became_active":"2024-04-25T08:35:43.100008+0000","last_became_peered":"2024-04-25T08:35:43.100008+0000","last_unstale":"2024-04-25T08:35:44.084409+0000","last_undegraded":"2024-04-25T08:35:44.084409+0000","last_fullsized":"2024-04-25T08:35:44.084409+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354442+0000","last_change":"2024-04-25T08:35:43.080351+0000","last_active":"2024-04-25T08:35:44.354442+0000","last_peered":"2024-04-25T08:35:44.354442+0000","last_clean":"2024-04-25T08:35:44.354442+0000","last_became_active":"2024-04-25T08:35:43.080020+0000","last_became_peered":"2024-04-25T08:35:43.080020+0000","last_unstale":"2024-04-25T08:35:44.354442+0000","last_undegraded":"2024-04-25T08:35:44.354442+0000","last_fullsized":"2024-04-25T08:35:44.354442+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.089034+0000","last_change":"2024-04-25T08:35:43.090646+0000","last_active":"2024-04-25T08:35:44.089034+0000","last_peered":"2024-04-25T08:35:44.089034+0000","last_clean":"2024-04-25T08:35:44.089034+0000","last_became_active":"2024-04-25T08:35:43.090325+0000","last_became_peered":"2024-04-25T08:35:43.090325+0000","last_unstale":"2024-04-25T08:35:44.089034+0000","last_undegraded":"2024-04-25T08:35:44.089034+0000","last_fullsized":"2024-04-25T08:35:44.089034+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:38.824878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083931+0000","last_change":"2024-04-25T08:35:43.087490+0000","last_active":"2024-04-25T08:35:44.083931+0000","last_peered":"2024-04-25T08:35:44.083931+0000","last_clean":"2024-04-25T08:35:44.083931+0000","last_became_active":"2024-04-25T08:35:43.087039+0000","last_became_peered":"2024-04-25T08:35:43.087039+0000","last_unstale":"2024-04-25T08:35:44.083931+0000","last_undegraded":"2024-04-25T08:35:44.083931+0000","last_fullsized":"2024-04-25T08:35:44.083931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.866982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354143+0000","last_change":"2024-04-25T08:35:43.090435+0000","last_active":"2024-04-25T08:35:44.354143+0000","last_peered":"2024-04-25T08:35:44.354143+0000","last_clean":"2024-04-25T08:35:44.354143+0000","last_became_active":"2024-04-25T08:35:43.090166+0000","last_became_peered":"2024-04-25T08:35:43.090166+0000","last_unstale":"2024-04-25T08:35:44.354143+0000","last_undegraded":"2024-04-25T08:35:44.354143+0000","last_fullsized":"2024-04-25T08:35:44.354143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:10.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354827+0000","last_change":"2024-04-25T08:35:43.084313+0000","last_active":"2024-04-25T08:35:44.354827+0000","last_peered":"2024-04-25T08:35:44.354827+0000","last_clean":"2024-04-25T08:35:44.354827+0000","last_became_active":"2024-04-25T08:35:43.084027+0000","last_became_peered":"2024-04-25T08:35:43.084027+0000","last_unstale":"2024-04-25T08:35:44.354827+0000","last_undegraded":"2024-04-25T08:35:44.354827+0000","last_fullsized":"2024-04-25T08:35:44.354827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347659+0000","last_change":"2024-04-25T08:35:43.091542+0000","last_active":"2024-04-25T08:35:44.347659+0000","last_peered":"2024-04-25T08:35:44.347659+0000","last_clean":"2024-04-25T08:35:44.347659+0000","last_became_active":"2024-04-25T08:35:43.091251+0000","last_became_peered":"2024-04-25T08:35:43.091251+0000","last_unstale":"2024-04-25T08:35:44.347659+0000","last_undegraded":"2024-04-25T08:35:44.347659+0000","last_fullsized":"2024-04-25T08:35:44.347659+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:17.458170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083847+0000","last_change":"2024-04-25T08:35:43.088502+0000","last_active":"2024-04-25T08:35:44.083847+0000","last_peered":"2024-04-25T08:35:44.083847+0000","last_clean":"2024-04-25T08:35:44.083847+0000","last_became_active":"2024-04-25T08:35:43.088156+0000","last_became_peered":"2024-04-25T08:35:43.088156+0000","last_unstale":"2024-04-25T08:35:44.083847+0000","last_undegraded":"2024-04-25T08:35:44.083847+0000","last_fullsized":"2024-04-25T08:35:44.083847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:29.004324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088676+0000","last_change":"2024-04-25T08:35:43.099672+0000","last_active":"2024-04-25T08:35:44.088676+0000","last_peered":"2024-04-25T08:35:44.088676+0000","last_clean":"2024-04-25T08:35:44.088676+0000","last_became_active":"2024-04-25T08:35:43.099376+0000","last_became_peered":"2024-04-25T08:35:43.099376+0000","last_unstale":"2024-04-25T08:35:44.088676+0000","last_undegraded":"2024-04-25T08:35:44.088676+0000","last_fullsized":"2024-04-25T08:35:44.088676+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083093+0000","last_change":"2024-04-25T08:35:43.080033+0000","last_active":"2024-04-25T08:35:44.083093+0000","last_peered":"2024-04-25T08:35:44.083093+0000","last_clean":"2024-04-25T08:35:44.083093+0000","last_became_active":"2024-04-25T08:35:43.079712+0000","last_became_peered":"2024-04-25T08:35:43.079712+0000","last_unstale":"2024-04-25T08:35:44.083093+0000","last_undegraded":"2024-04-25T08:35:44.083093+0000","last_fullsized":"2024-04-25T08:35:44.083093+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.767670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083669+0000","last_change":"2024-04-25T08:35:43.084040+0000","last_active":"2024-04-25T08:35:44.083669+0000","last_peered":"2024-04-25T08:35:44.083669+0000","last_clean":"2024-04-25T08:35:44.083669+0000","last_became_active":"2024-04-25T08:35:43.082074+0000","last_became_peered":"2024-04-25T08:35:43.082074+0000","last_unstale":"2024-04-25T08:35:44.083669+0000","last_undegraded":"2024-04-25T08:35:44.083669+0000","last_fullsized":"2024-04-25T08:35:44.083669+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:57.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084379+0000","last_change":"2024-04-25T08:35:42.083008+0000","last_active":"2024-04-25T08:35:44.084379+0000","last_peered":"2024-04-25T08:35:44.084379+0000","last_clean":"2024-04-25T08:35:44.084379+0000","last_became_active":"2024-04-25T08:35:42.082722+0000","last_became_peered":"2024-04-25T08:35:42.082722+0000","last_unstale":"2024-04-25T08:35:44.084379+0000","last_undegraded":"2024-04-25T08:35:44.084379+0000","last_fullsized":"2024-04-25T08:35:44.084379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083922+0000","last_change":"2024-04-25T08:35:43.092634+0000","last_active":"2024-04-25T08:35:44.083922+0000","last_peered":"2024-04-25T08:35:44.083922+0000","last_clean":"2024-04-25T08:35:44.083922+0000","last_became_active":"2024-04-25T08:35:43.092347+0000","last_became_peered":"2024-04-25T08:35:43.092347+0000","last_unstale":"2024-04-25T08:35:44.083922+0000","last_undegraded":"2024-04-25T08:35:44.083922+0000","last_fullsized":"2024-04-25T08:35:44.083922+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:37.775158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084482+0000","last_change":"2024-04-25T08:35:42.077498+0000","last_active":"2024-04-25T08:35:44.084482+0000","last_peered":"2024-04-25T08:35:44.084482+0000","last_clean":"2024-04-25T08:35:44.084482+0000","last_became_active":"2024-04-25T08:35:42.077203+0000","last_became_peered":"2024-04-25T08:35:42.077203+0000","last_unstale":"2024-04-25T08:35:44.084482+0000","last_undegraded":"2024-04-25T08:35:44.084482+0000","last_fullsized":"2024-04-25T08:35:44.084482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347666+0000","last_change":"2024-04-25T08:35:43.099723+0000","last_active":"2024-04-25T08:35:44.347666+0000","last_peered":"2024-04-25T08:35:44.347666+0000","last_clean":"2024-04-25T08:35:44.347666+0000","last_became_active":"2024-04-25T08:35:43.099412+0000","last_became_peered":"2024-04-25T08:35:43.099412+0000","last_unstale":"2024-04-25T08:35:44.347666+0000","last_undegraded":"2024-04-25T08:35:44.347666+0000","last_fullsized":"2024-04-25T08:35:44.347666+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:14.707807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347703+0000","last_change":"2024-04-25T08:35:42.084090+0000","last_active":"2024-04-25T08:35:44.347703+0000","last_peered":"2024-04-25T08:35:44.347703+0000","last_clean":"2024-04-25T08:35:44.347703+0000","last_became_active":"2024-04-25T08:35:42.083831+0000","last_became_peered":"2024-04-25T08:35:42.083831+0000","last_unstale":"2024-04-25T08:35:44.347703+0000","last_undegraded":"2024-04-25T08:35:44.347703+0000","last_fullsized":"2024-04-25T08:35:44.347703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:10.496016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083517+0000","last_change":"2024-04-25T08:35:43.079045+0000","last_active":"2024-04-25T08:35:44.083517+0000","last_peered":"2024-04-25T08:35:44.083517+0000","last_clean":"2024-04-25T08:35:44.083517+0000","last_became_active":"2024-04-25T08:35:43.078753+0000","last_became_peered":"2024-04-25T08:35:43.078753+0000","last_unstale":"2024-04-25T08:35:44.083517+0000","last_undegraded":"2024-04-25T08:35:44.083517+0000","last_fullsized":"2024-04-25T08:35:44.083517+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:29.703738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084081+0000","last_change":"2024-04-25T08:35:42.076044+0000","last_active":"2024-04-25T08:35:44.084081+0000","last_peered":"2024-04-25T08:35:44.084081+0000","last_clean":"2024-04-25T08:35:44.084081+0000","last_became_active":"2024-04-25T08:35:42.075775+0000","last_became_peered":"2024-04-25T08:35:42.075775+0000","last_unstale":"2024-04-25T08:35:44.084081+0000","last_undegraded":"2024-04-25T08:35:44.084081+0000","last_fullsized":"2024-04-25T08:35:44.084081+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:57.424993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354942+0000","last_change":"2024-04-25T08:35:43.085489+0000","last_active":"2024-04-25T08:35:44.354942+0000","last_peered":"2024-04-25T08:35:44.354942+0000","last_clean":"2024-04-25T08:35:44.354942+0000","last_became_active":"2024-04-25T08:35:43.085149+0000","last_became_peered":"2024-04-25T08:35:43.085149+0000","last_unstale":"2024-04-25T08:35:44.354942+0000","last_undegraded":"2024-04-25T08:35:44.354942+0000","last_fullsized":"2024-04-25T08:35:44.354942+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083722+0000","last_change":"2024-04-25T08:35:42.083359+0000","last_active":"2024-04-25T08:35:44.083722+0000","last_peered":"2024-04-25T08:35:44.083722+0000","last_clean":"2024-04-25T08:35:44.083722+0000","last_became_active":"2024-04-25T08:35:42.083075+0000","last_became_peered":"2024-04-25T08:35:42.083075+0000","last_unstale":"2024-04-25T08:35:44.083722+0000","last_undegraded":"2024-04-25T08:35:44.083722+0000","last_fullsized":"2024-04-25T08:35:44.083722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:59.795889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084226+0000","last_change":"2024-04-25T08:35:43.085701+0000","last_active":"2024-04-25T08:35:44.084226+0000","last_peered":"2024-04-25T08:35:44.084226+0000","last_clean":"2024-04-25T08:35:44.084226+0000","last_became_active":"2024-04-25T08:35:43.085405+0000","last_became_peered":"2024-04-25T08:35:43.085405+0000","last_unstale":"2024-04-25T08:35:44.084226+0000","last_undegraded":"2024-04-25T08:35:44.084226+0000","last_fullsized":"2024-04-25T08:35:44.084226+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354789+0000","last_change":"2024-04-25T08:35:42.087255+0000","last_active":"2024-04-25T08:35:44.354789+0000","last_peered":"2024-04-25T08:35:44.354789+0000","last_clean":"2024-04-25T08:35:44.354789+0000","last_became_active":"2024-04-25T08:35:42.086862+0000","last_became_peered":"2024-04-25T08:35:42.086862+0000","last_unstale":"2024-04-25T08:35:44.354789+0000","last_undegraded":"2024-04-25T08:35:44.354789+0000","last_fullsized":"2024-04-25T08:35:44.354789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.231173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083302+0000","last_change":"2024-04-25T08:35:43.085631+0000","last_active":"2024-04-25T08:35:44.083302+0000","last_peered":"2024-04-25T08:35:44.083302+0000","last_clean":"2024-04-25T08:35:44.083302+0000","last_became_active":"2024-04-25T08:35:43.085345+0000","last_became_peered":"2024-04-25T08:35:43.085345+0000","last_unstale":"2024-04-25T08:35:44.083302+0000","last_undegraded":"2024-04-25T08:35:44.083302+0000","last_fullsized":"2024-04-25T08:35:44.083302+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:52.198356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354848+0000","last_change":"2024-04-25T08:35:42.078496+0000","last_active":"2024-04-25T08:35:44.354848+0000","last_peered":"2024-04-25T08:35:44.354848+0000","last_clean":"2024-04-25T08:35:44.354848+0000","last_became_active":"2024-04-25T08:35:42.078199+0000","last_became_peered":"2024-04-25T08:35:42.078199+0000","last_unstale":"2024-04-25T08:35:44.354848+0000","last_undegraded":"2024-04-25T08:35:44.354848+0000","last_fullsized":"2024-04-25T08:35:44.354848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354685+0000","last_change":"2024-04-25T08:35:43.096815+0000","last_active":"2024-04-25T08:35:44.354685+0000","last_peered":"2024-04-25T08:35:44.354685+0000","last_clean":"2024-04-25T08:35:44.354685+0000","last_became_active":"2024-04-25T08:35:43.096237+0000","last_became_peered":"2024-04-25T08:35:43.096237+0000","last_unstale":"2024-04-25T08:35:44.354685+0000","last_undegraded":"2024-04-25T08:35:44.354685+0000","last_fullsized":"2024-04-25T08:35:44.354685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347360+0000","last_change":"2024-04-25T08:35:42.084779+0000","last_active":"2024-04-25T08:35:44.347360+0000","last_peered":"2024-04-25T08:35:44.347360+0000","last_clean":"2024-04-25T08:35:44.347360+0000","last_became_active":"2024-04-25T08:35:42.084495+0000","last_became_peered":"2024-04-25T08:35:42.084495+0000","last_unstale":"2024-04-25T08:35:44.347360+0000","last_undegraded":"2024-04-25T08:35:44.347360+0000","last_fullsized":"2024-04-25T08:35:44.347360+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:37.822495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083757+0000","last_change":"2024-04-25T08:35:43.084977+0000","last_active":"2024-04-25T08:35:44.083757+0000","last_peered":"2024-04-25T08:35:44.083757+0000","last_clean":"2024-04-25T08:35:44.083757+0000","last_became_active":"2024-04-25T08:35:43.084694+0000","last_became_peered":"2024-04-25T08:35:43.084694+0000","last_unstale":"2024-04-25T08:35:44.083757+0000","last_undegraded":"2024-04-25T08:35:44.083757+0000","last_fullsized":"2024-04-25T08:35:44.083757+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.121067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.089065+0000","last_change":"2024-04-25T08:35:42.084063+0000","last_active":"2024-04-25T08:35:44.089065+0000","last_peered":"2024-04-25T08:35:44.089065+0000","last_clean":"2024-04-25T08:35:44.089065+0000","last_became_active":"2024-04-25T08:35:42.083798+0000","last_became_peered":"2024-04-25T08:35:42.083798+0000","last_unstale":"2024-04-25T08:35:44.089065+0000","last_undegraded":"2024-04-25T08:35:44.089065+0000","last_fullsized":"2024-04-25T08:35:44.089065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:16.323534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347787+0000","last_change":"2024-04-25T08:35:43.087459+0000","last_active":"2024-04-25T08:35:44.347787+0000","last_peered":"2024-04-25T08:35:44.347787+0000","last_clean":"2024-04-25T08:35:44.347787+0000","last_became_active":"2024-04-25T08:35:43.087040+0000","last_became_peered":"2024-04-25T08:35:43.087040+0000","last_unstale":"2024-04-25T08:35:44.347787+0000","last_undegraded":"2024-04-25T08:35:44.347787+0000","last_fullsized":"2024-04-25T08:35:44.347787+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:14.637748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354368+0000","last_change":"2024-04-25T08:35:42.079015+0000","last_active":"2024-04-25T08:35:44.354368+0000","last_peered":"2024-04-25T08:35:44.354368+0000","last_clean":"2024-04-25T08:35:44.354368+0000","last_became_active":"2024-04-25T08:35:42.078722+0000","last_became_peered":"2024-04-25T08:35:42.078722+0000","last_unstale":"2024-04-25T08:35:44.354368+0000","last_undegraded":"2024-04-25T08:35:44.354368+0000","last_fullsized":"2024-04-25T08:35:44.354368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354270+0000","last_change":"2024-04-25T08:35:43.085992+0000","last_active":"2024-04-25T08:35:44.354270+0000","last_peered":"2024-04-25T08:35:44.354270+0000","last_clean":"2024-04-25T08:35:44.354270+0000","last_became_active":"2024-04-25T08:35:43.082970+0000","last_became_peered":"2024-04-25T08:35:43.082970+0000","last_unstale":"2024-04-25T08:35:44.354270+0000","last_undegraded":"2024-04-25T08:35:44.354270+0000","last_fullsized":"2024-04-25T08:35:44.354270+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084306+0000","last_change":"2024-04-25T08:35:42.085258+0000","last_active":"2024-04-25T08:35:44.084306+0000","last_peered":"2024-04-25T08:35:44.084306+0000","last_clean":"2024-04-25T08:35:44.084306+0000","last_became_active":"2024-04-25T08:35:42.084942+0000","last_became_peered":"2024-04-25T08:35:42.084942+0000","last_unstale":"2024-04-25T08:35:44.084306+0000","last_undegraded":"2024-04-25T08:35:44.084306+0000","last_fullsized":"2024-04-25T08:35:44.084306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:58.723955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354667+0000","last_change":"2024-04-25T08:35:43.094836+0000","last_active":"2024-04-25T08:35:44.354667+0000","last_peered":"2024-04-25T08:35:44.354667+0000","last_clean":"2024-04-25T08:35:44.354667+0000","last_became_active":"2024-04-25T08:35:43.093301+0000","last_became_peered":"2024-04-25T08:35:43.093301+0000","last_unstale":"2024-04-25T08:35:44.354667+0000","last_undegraded":"2024-04-25T08:35:44.354667+0000","last_fullsized":"2024-04-25T08:35:44.354667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083767+0000","last_change":"2024-04-25T08:35:42.083983+0000","last_active":"2024-04-25T08:35:44.083767+0000","last_peered":"2024-04-25T08:35:44.083767+0000","last_clean":"2024-04-25T08:35:44.083767+0000","last_became_active":"2024-04-25T08:35:42.083670+0000","last_became_peered":"2024-04-25T08:35:42.083670+0000","last_unstale":"2024-04-25T08:35:44.083767+0000","last_undegraded":"2024-04-25T08:35:44.083767+0000","last_fullsized":"2024-04-25T08:35:44.083767+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:57.266743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355036+0000","last_change":"2024-04-25T08:35:43.095386+0000","last_active":"2024-04-25T08:35:44.355036+0000","last_peered":"2024-04-25T08:35:44.355036+0000","last_clean":"2024-04-25T08:35:44.355036+0000","last_became_active":"2024-04-25T08:35:43.093364+0000","last_became_peered":"2024-04-25T08:35:43.093364+0000","last_unstale":"2024-04-25T08:35:44.355036+0000","last_undegraded":"2024-04-25T08:35:44.355036+0000","last_fullsized":"2024-04-25T08:35:44.355036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347540+0000","last_change":"2024-04-25T08:35:42.086794+0000","last_active":"2024-04-25T08:35:44.347540+0000","last_peered":"2024-04-25T08:35:44.347540+0000","last_clean":"2024-04-25T08:35:44.347540+0000","last_became_active":"2024-04-25T08:35:42.086433+0000","last_became_peered":"2024-04-25T08:35:42.086433+0000","last_unstale":"2024-04-25T08:35:44.347540+0000","last_undegraded":"2024-04-25T08:35:44.347540+0000","last_fullsized":"2024-04-25T08:35:44.347540+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:00.228682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:43.087418+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:43.087100+0000","last_became_peered":"2024-04-25T08:35:43.087100+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:23.384291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:42.084034+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:42.083714+0000","last_became_peered":"2024-04-25T08:35:42.083714+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:35.130250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354284+0000","last_change":"2024-04-25T08:35:43.100537+0000","last_active":"2024-04-25T08:35:44.354284+0000","last_peered":"2024-04-25T08:35:44.354284+0000","last_clean":"2024-04-25T08:35:44.354284+0000","last_became_active":"2024-04-25T08:35:43.100246+0000","last_became_peered":"2024-04-25T08:35:43.100246+0000","last_unstale":"2024-04-25T08:35:44.354284+0000","last_undegraded":"2024-04-25T08:35:44.354284+0000","last_fullsized":"2024-04-25T08:35:44.354284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354726+0000","last_change":"2024-04-25T08:35:42.079064+0000","last_active":"2024-04-25T08:35:44.354726+0000","last_peered":"2024-04-25T08:35:44.354726+0000","last_clean":"2024-04-25T08:35:44.354726+0000","last_became_active":"2024-04-25T08:35:42.078692+0000","last_became_peered":"2024-04-25T08:35:42.078692+0000","last_unstale":"2024-04-25T08:35:44.354726+0000","last_undegraded":"2024-04-25T08:35:44.354726+0000","last_fullsized":"2024-04-25T08:35:44.354726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:03.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353823+0000","last_change":"2024-04-25T08:35:43.088607+0000","last_active":"2024-04-25T08:35:44.353823+0000","last_peered":"2024-04-25T08:35:44.353823+0000","last_clean":"2024-04-25T08:35:44.353823+0000","last_became_active":"2024-04-25T08:35:43.088317+0000","last_became_peered":"2024-04-25T08:35:43.088317+0000","last_unstale":"2024-04-25T08:35:44.353823+0000","last_undegraded":"2024-04-25T08:35:44.353823+0000","last_fullsized":"2024-04-25T08:35:44.353823+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:18.622878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355062+0000","last_change":"2024-04-25T08:35:42.086642+0000","last_active":"2024-04-25T08:35:44.355062+0000","last_peered":"2024-04-25T08:35:44.355062+0000","last_clean":"2024-04-25T08:35:44.355062+0000","last_became_active":"2024-04-25T08:35:42.086219+0000","last_became_peered":"2024-04-25T08:35:42.086219+0000","last_unstale":"2024-04-25T08:35:44.355062+0000","last_undegraded":"2024-04-25T08:35:44.355062+0000","last_fullsized":"2024-04-25T08:35:44.355062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:16.239961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083745+0000","last_change":"2024-04-25T08:35:43.084349+0000","last_active":"2024-04-25T08:35:44.083745+0000","last_peered":"2024-04-25T08:35:44.083745+0000","last_clean":"2024-04-25T08:35:44.083745+0000","last_became_active":"2024-04-25T08:35:43.081684+0000","last_became_peered":"2024-04-25T08:35:43.081684+0000","last_unstale":"2024-04-25T08:35:44.083745+0000","last_undegraded":"2024-04-25T08:35:44.083745+0000","last_fullsized":"2024-04-25T08:35:44.083745+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:47.474056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084497+0000","last_change":"2024-04-25T08:35:42.074237+0000","last_active":"2024-04-25T08:35:44.084497+0000","last_peered":"2024-04-25T08:35:44.084497+0000","last_clean":"2024-04-25T08:35:44.084497+0000","last_became_active":"2024-04-25T08:35:42.073719+0000","last_became_peered":"2024-04-25T08:35:42.073719+0000","last_unstale":"2024-04-25T08:35:44.084497+0000","last_undegraded":"2024-04-25T08:35:44.084497+0000","last_fullsized":"2024-04-25T08:35:44.084497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083912+0000","last_change":"2024-04-25T08:35:43.088892+0000","last_active":"2024-04-25T08:35:44.083912+0000","last_peered":"2024-04-25T08:35:44.083912+0000","last_clean":"2024-04-25T08:35:44.083912+0000","last_became_active":"2024-04-25T08:35:43.088572+0000","last_became_peered":"2024-04-25T08:35:43.088572+0000","last_unstale":"2024-04-25T08:35:44.083912+0000","last_undegraded":"2024-04-25T08:35:44.083912+0000","last_fullsized":"2024-04-25T08:35:44.083912+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347530+0000","last_change":"2024-04-25T08:35:42.079831+0000","last_active":"2024-04-25T08:35:44.347530+0000","last_peered":"2024-04-25T08:35:44.347530+0000","last_clean":"2024-04-25T08:35:44.347530+0000","last_became_active":"2024-04-25T08:35:42.079514+0000","last_became_peered":"2024-04-25T08:35:42.079514+0000","last_unstale":"2024-04-25T08:35:44.347530+0000","last_undegraded":"2024-04-25T08:35:44.347530+0000","last_fullsized":"2024-04-25T08:35:44.347530+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:55.808679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084175+0000","last_change":"2024-04-25T08:35:43.092287+0000","last_active":"2024-04-25T08:35:44.084175+0000","last_peered":"2024-04-25T08:35:44.084175+0000","last_clean":"2024-04-25T08:35:44.084175+0000","last_became_active":"2024-04-25T08:35:43.091768+0000","last_became_peered":"2024-04-25T08:35:43.091768+0000","last_unstale":"2024-04-25T08:35:44.084175+0000","last_undegraded":"2024-04-25T08:35:44.084175+0000","last_fullsized":"2024-04-25T08:35:44.084175+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084367+0000","last_change":"2024-04-25T08:35:42.075127+0000","last_active":"2024-04-25T08:35:44.084367+0000","last_peered":"2024-04-25T08:35:44.084367+0000","last_clean":"2024-04-25T08:35:44.084367+0000","last_became_active":"2024-04-25T08:35:42.074842+0000","last_became_peered":"2024-04-25T08:35:42.074842+0000","last_unstale":"2024-04-25T08:35:44.084367+0000","last_undegraded":"2024-04-25T08:35:44.084367+0000","last_fullsized":"2024-04-25T08:35:44.084367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.370203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083108+0000","last_change":"2024-04-25T08:35:43.083888+0000","last_active":"2024-04-25T08:35:44.083108+0000","last_peered":"2024-04-25T08:35:44.083108+0000","last_clean":"2024-04-25T08:35:44.083108+0000","last_became_active":"2024-04-25T08:35:43.083585+0000","last_became_peered":"2024-04-25T08:35:43.083585+0000","last_unstale":"2024-04-25T08:35:44.083108+0000","last_undegraded":"2024-04-25T08:35:44.083108+0000","last_fullsized":"2024-04-25T08:35:44.083108+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:43.652486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088676+0000","last_change":"2024-04-25T08:35:42.082861+0000","last_active":"2024-04-25T08:35:44.088676+0000","last_peered":"2024-04-25T08:35:44.088676+0000","last_clean":"2024-04-25T08:35:44.088676+0000","last_became_active":"2024-04-25T08:35:42.082582+0000","last_became_peered":"2024-04-25T08:35:42.082582+0000","last_unstale":"2024-04-25T08:35:44.088676+0000","last_undegraded":"2024-04-25T08:35:44.088676+0000","last_fullsized":"2024-04-25T08:35:44.088676+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:55.864599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088780+0000","last_change":"2024-04-25T08:35:43.090667+0000","last_active":"2024-04-25T08:35:44.088780+0000","last_peered":"2024-04-25T08:35:44.088780+0000","last_clean":"2024-04-25T08:35:44.088780+0000","last_became_active":"2024-04-25T08:35:43.090389+0000","last_became_peered":"2024-04-25T08:35:43.090389+0000","last_unstale":"2024-04-25T08:35:44.088780+0000","last_undegraded":"2024-04-25T08:35:44.088780+0000","last_fullsized":"2024-04-25T08:35:44.088780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:11.498399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354254+0000","last_change":"2024-04-25T08:35:42.077041+0000","last_active":"2024-04-25T08:35:44.354254+0000","last_peered":"2024-04-25T08:35:44.354254+0000","last_clean":"2024-04-25T08:35:44.354254+0000","last_became_active":"2024-04-25T08:35:42.076737+0000","last_became_peered":"2024-04-25T08:35:42.076737+0000","last_unstale":"2024-04-25T08:35:44.354254+0000","last_undegraded":"2024-04-25T08:35:44.354254+0000","last_fullsized":"2024-04-25T08:35:44.354254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:54.769371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088891+0000","last_change":"2024-04-25T08:35:43.099702+0000","last_active":"2024-04-25T08:35:44.088891+0000","last_peered":"2024-04-25T08:35:44.088891+0000","last_clean":"2024-04-25T08:35:44.088891+0000","last_became_active":"2024-04-25T08:35:43.099426+0000","last_became_peered":"2024-04-25T08:35:43.099426+0000","last_unstale":"2024-04-25T08:35:44.088891+0000","last_undegraded":"2024-04-25T08:35:44.088891+0000","last_fullsized":"2024-04-25T08:35:44.088891+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:19.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088746+0000","last_change":"2024-04-25T08:35:42.086887+0000","last_active":"2024-04-25T08:35:44.088746+0000","last_peered":"2024-04-25T08:35:44.088746+0000","last_clean":"2024-04-25T08:35:44.088746+0000","last_became_active":"2024-04-25T08:35:42.086576+0000","last_became_peered":"2024-04-25T08:35:42.086576+0000","last_unstale":"2024-04-25T08:35:44.088746+0000","last_undegraded":"2024-04-25T08:35:44.088746+0000","last_fullsized":"2024-04-25T08:35:44.088746+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:46.771509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083581+0000","last_change":"2024-04-25T08:35:43.091743+0000","last_active":"2024-04-25T08:35:44.083581+0000","last_peered":"2024-04-25T08:35:44.083581+0000","last_clean":"2024-04-25T08:35:44.083581+0000","last_became_active":"2024-04-25T08:35:43.091470+0000","last_became_peered":"2024-04-25T08:35:43.091470+0000","last_unstale":"2024-04-25T08:35:44.083581+0000","last_undegraded":"2024-04-25T08:35:44.083581+0000","last_fullsized":"2024-04-25T08:35:44.083581+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.583746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353713+0000","last_change":"2024-04-25T08:35:42.079469+0000","last_active":"2024-04-25T08:35:44.353713+0000","last_peered":"2024-04-25T08:35:44.353713+0000","last_clean":"2024-04-25T08:35:44.353713+0000","last_became_active":"2024-04-25T08:35:42.079187+0000","last_became_peered":"2024-04-25T08:35:42.079187+0000","last_unstale":"2024-04-25T08:35:44.353713+0000","last_undegraded":"2024-04-25T08:35:44.353713+0000","last_fullsized":"2024-04-25T08:35:44.353713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.436872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083965+0000","last_change":"2024-04-25T08:35:43.085978+0000","last_active":"2024-04-25T08:35:44.083965+0000","last_peered":"2024-04-25T08:35:44.083965+0000","last_clean":"2024-04-25T08:35:44.083965+0000","last_became_active":"2024-04-25T08:35:43.085695+0000","last_became_peered":"2024-04-25T08:35:43.085695+0000","last_unstale":"2024-04-25T08:35:44.083965+0000","last_undegraded":"2024-04-25T08:35:44.083965+0000","last_fullsized":"2024-04-25T08:35:44.083965+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347642+0000","last_change":"2024-04-25T08:35:42.079458+0000","last_active":"2024-04-25T08:35:44.347642+0000","last_peered":"2024-04-25T08:35:44.347642+0000","last_clean":"2024-04-25T08:35:44.347642+0000","last_became_active":"2024-04-25T08:35:42.079154+0000","last_became_peered":"2024-04-25T08:35:42.079154+0000","last_unstale":"2024-04-25T08:35:44.347642+0000","last_undegraded":"2024-04-25T08:35:44.347642+0000","last_fullsized":"2024-04-25T08:35:44.347642+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:49.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083967+0000","last_change":"2024-04-25T08:35:43.086850+0000","last_active":"2024-04-25T08:35:44.083967+0000","last_peered":"2024-04-25T08:35:44.083967+0000","last_clean":"2024-04-25T08:35:44.083967+0000","last_became_active":"2024-04-25T08:35:43.082016+0000","last_became_peered":"2024-04-25T08:35:43.082016+0000","last_unstale":"2024-04-25T08:35:44.083967+0000","last_undegraded":"2024-04-25T08:35:44.083967+0000","last_fullsized":"2024-04-25T08:35:44.083967+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.754062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088238+0000","last_change":"2024-04-25T08:35:42.080608+0000","last_active":"2024-04-25T08:35:44.088238+0000","last_peered":"2024-04-25T08:35:44.088238+0000","last_clean":"2024-04-25T08:35:44.088238+0000","last_became_active":"2024-04-25T08:35:42.080267+0000","last_became_peered":"2024-04-25T08:35:42.080267+0000","last_unstale":"2024-04-25T08:35:44.088238+0000","last_undegraded":"2024-04-25T08:35:44.088238+0000","last_fullsized":"2024-04-25T08:35:44.088238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:09.842643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083371+0000","last_change":"2024-04-25T08:35:43.084195+0000","last_active":"2024-04-25T08:35:44.083371+0000","last_peered":"2024-04-25T08:35:44.083371+0000","last_clean":"2024-04-25T08:35:44.083371+0000","last_became_active":"2024-04-25T08:35:43.081209+0000","last_became_peered":"2024-04-25T08:35:43.081209+0000","last_unstale":"2024-04-25T08:35:44.083371+0000","last_undegraded":"2024-04-25T08:35:44.083371+0000","last_fullsized":"2024-04-25T08:35:44.083371+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:31.817643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084421+0000","last_change":"2024-04-25T08:35:42.084974+0000","last_active":"2024-04-25T08:35:44.084421+0000","last_peered":"2024-04-25T08:35:44.084421+0000","last_clean":"2024-04-25T08:35:44.084421+0000","last_became_active":"2024-04-25T08:35:42.084703+0000","last_became_peered":"2024-04-25T08:35:42.084703+0000","last_unstale":"2024-04-25T08:35:44.084421+0000","last_undegraded":"2024-04-25T08:35:44.084421+0000","last_fullsized":"2024-04-25T08:35:44.084421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:56.731886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.089053+0000","last_change":"2024-04-25T08:35:43.091302+0000","last_active":"2024-04-25T08:35:44.089053+0000","last_peered":"2024-04-25T08:35:44.089053+0000","last_clean":"2024-04-25T08:35:44.089053+0000","last_became_active":"2024-04-25T08:35:43.091001+0000","last_became_peered":"2024-04-25T08:35:43.091001+0000","last_unstale":"2024-04-25T08:35:44.089053+0000","last_undegraded":"2024-04-25T08:35:44.089053+0000","last_fullsized":"2024-04-25T08:35:44.089053+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:32.012827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084172+0000","last_change":"2024-04-25T08:35:42.072420+0000","last_active":"2024-04-25T08:35:44.084172+0000","last_peered":"2024-04-25T08:35:44.084172+0000","last_clean":"2024-04-25T08:35:44.084172+0000","last_became_active":"2024-04-25T08:35:42.072142+0000","last_became_peered":"2024-04-25T08:35:42.072142+0000","last_unstale":"2024-04-25T08:35:44.084172+0000","last_undegraded":"2024-04-25T08:35:44.084172+0000","last_fullsized":"2024-04-25T08:35:44.084172+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:49.118684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353624+0000","last_change":"2024-04-25T08:35:43.089060+0000","last_active":"2024-04-25T08:35:44.353624+0000","last_peered":"2024-04-25T08:35:44.353624+0000","last_clean":"2024-04-25T08:35:44.353624+0000","last_became_active":"2024-04-25T08:35:43.088777+0000","last_became_peered":"2024-04-25T08:35:43.088777+0000","last_unstale":"2024-04-25T08:35:44.353624+0000","last_undegraded":"2024-04-25T08:35:44.353624+0000","last_fullsized":"2024-04-25T08:35:44.353624+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:58.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083672+0000","last_change":"2024-04-25T08:35:42.085881+0000","last_active":"2024-04-25T08:35:44.083672+0000","last_peered":"2024-04-25T08:35:44.083672+0000","last_clean":"2024-04-25T08:35:44.083672+0000","last_became_active":"2024-04-25T08:35:42.085574+0000","last_became_peered":"2024-04-25T08:35:42.085574+0000","last_unstale":"2024-04-25T08:35:44.083672+0000","last_undegraded":"2024-04-25T08:35:44.083672+0000","last_fullsized":"2024-04-25T08:35:44.083672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:28.015864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354681+0000","last_change":"2024-04-25T08:35:43.080706+0000","last_active":"2024-04-25T08:35:44.354681+0000","last_peered":"2024-04-25T08:35:44.354681+0000","last_clean":"2024-04-25T08:35:44.354681+0000","last_became_active":"2024-04-25T08:35:43.080411+0000","last_became_peered":"2024-04-25T08:35:43.080411+0000","last_unstale":"2024-04-25T08:35:44.354681+0000","last_undegraded":"2024-04-25T08:35:44.354681+0000","last_fullsized":"2024-04-25T08:35:44.354681+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088748+0000","last_change":"2024-04-25T08:35:42.082825+0000","last_active":"2024-04-25T08:35:44.088748+0000","last_peered":"2024-04-25T08:35:44.088748+0000","last_clean":"2024-04-25T08:35:44.088748+0000","last_became_active":"2024-04-25T08:35:42.082542+0000","last_became_peered":"2024-04-25T08:35:42.082542+0000","last_unstale":"2024-04-25T08:35:44.088748+0000","last_undegraded":"2024-04-25T08:35:44.088748+0000","last_fullsized":"2024-04-25T08:35:44.088748+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.984166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083712+0000","last_change":"2024-04-25T08:35:43.101917+0000","last_active":"2024-04-25T08:35:44.083712+0000","last_peered":"2024-04-25T08:35:44.083712+0000","last_clean":"2024-04-25T08:35:44.083712+0000","last_became_active":"2024-04-25T08:35:43.101697+0000","last_became_peered":"2024-04-25T08:35:43.101697+0000","last_unstale":"2024-04-25T08:35:44.083712+0000","last_undegraded":"2024-04-25T08:35:44.083712+0000","last_fullsized":"2024-04-25T08:35:44.083712+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084467+0000","last_change":"2024-04-25T08:35:42.073922+0000","last_active":"2024-04-25T08:35:44.084467+0000","last_peered":"2024-04-25T08:35:44.084467+0000","last_clean":"2024-04-25T08:35:44.084467+0000","last_became_active":"2024-04-25T08:35:42.072926+0000","last_became_peered":"2024-04-25T08:35:42.072926+0000","last_unstale":"2024-04-25T08:35:44.084467+0000","last_undegraded":"2024-04-25T08:35:44.084467+0000","last_fullsized":"2024-04-25T08:35:44.084467+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:40.100311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083687+0000","last_change":"2024-04-25T08:35:43.079481+0000","last_active":"2024-04-25T08:35:44.083687+0000","last_peered":"2024-04-25T08:35:44.083687+0000","last_clean":"2024-04-25T08:35:44.083687+0000","last_became_active":"2024-04-25T08:35:43.078668+0000","last_became_peered":"2024-04-25T08:35:43.078668+0000","last_unstale":"2024-04-25T08:35:44.083687+0000","last_undegraded":"2024-04-25T08:35:44.083687+0000","last_fullsized":"2024-04-25T08:35:44.083687+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:08.825229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347391+0000","last_change":"2024-04-25T08:35:42.082145+0000","last_active":"2024-04-25T08:35:44.347391+0000","last_peered":"2024-04-25T08:35:44.347391+0000","last_clean":"2024-04-25T08:35:44.347391+0000","last_became_active":"2024-04-25T08:35:42.081772+0000","last_became_peered":"2024-04-25T08:35:42.081772+0000","last_unstale":"2024-04-25T08:35:44.347391+0000","last_undegraded":"2024-04-25T08:35:44.347391+0000","last_fullsized":"2024-04-25T08:35:44.347391+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:18.561753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347652+0000","last_change":"2024-04-25T08:35:43.088971+0000","last_active":"2024-04-25T08:35:44.347652+0000","last_peered":"2024-04-25T08:35:44.347652+0000","last_clean":"2024-04-25T08:35:44.347652+0000","last_became_active":"2024-04-25T08:35:43.088662+0000","last_became_peered":"2024-04-25T08:35:43.088662+0000","last_unstale":"2024-04-25T08:35:44.347652+0000","last_undegraded":"2024-04-25T08:35:44.347652+0000","last_fullsized":"2024-04-25T08:35:44.347652+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:58.269126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354943+0000","last_change":"2024-04-25T08:35:42.080661+0000","last_active":"2024-04-25T08:35:44.354943+0000","last_peered":"2024-04-25T08:35:44.354943+0000","last_clean":"2024-04-25T08:35:44.354943+0000","last_became_active":"2024-04-25T08:35:42.080378+0000","last_became_peered":"2024-04-25T08:35:42.080378+0000","last_unstale":"2024-04-25T08:35:44.354943+0000","last_undegraded":"2024-04-25T08:35:44.354943+0000","last_fullsized":"2024-04-25T08:35:44.354943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:11.863733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":315,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347529+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:35:44.347529+0000","last_peered":"2024-04-25T08:35:44.347529+0000","last_clean":"2024-04-25T08:35:44.347529+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:35:44.347529+0000","last_undegraded":"2024-04-25T08:35:44.347529+0000","last_fullsized":"2024-04-25T08:35:44.347529+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542227,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41256,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330584,"statfs":{"total":96636764160,"available":96594518016,"internally_reserved":0,"allocated":1794048,"data_stored":1142381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":1212416,"data_stored":554173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40672,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331168,"statfs":{"total":96636764160,"available":96595116032,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":552117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50392,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321448,"statfs":{"total":96636764160,"available":96585162752,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54556,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317284,"statfs":{"total":96636764160,"available":96580898816,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":552027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1810432,"data_stored":1144643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:48.722 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:35:48.727+0000 7fb645171640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:35:50.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:50.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.628+0000 7f9c9b71d640 1 Processor -- start 2024-04-25T08:35:50.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.628+0000 7f9c9b71d640 1 -- start start 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c9b71d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c94154ed0 0x7f9c940aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c9b71d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c9406c1d0 0x7f9c9406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c9b71d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9415ec60 con 0x7f9c94071ac0 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c9b71d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9415ef40 con 0x7f9c94154ed0 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c9b71d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9406db90 con 0x7f9c9406c1d0 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c994b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c94154ed0 0x7f9c940aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c994b9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c94154ed0 0x7f9c940aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35832/0 (socket says 172.21.15.60:35832) 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c994b9640 1 -- 172.21.15.60:0/3472677475 learned_addr learned my addr 172.21.15.60:0/3472677475 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.629+0000 7f9c994b9640 1 -- 172.21.15.60:0/3472677475 >> v2:172.21.15.100:3300/0 conn(0x7f9c9406c1d0 msgr2=0x7f9c9406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:50.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c99cba640 1 --2- 172.21.15.60:0/3472677475 >> v2:172.21.15.100:3300/0 conn(0x7f9c9406c1d0 0x7f9c9406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c994b9640 1 --2- 172.21.15.60:0/3472677475 >> v2:172.21.15.100:3300/0 conn(0x7f9c9406c1d0 0x7f9c9406afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c994b9640 1 -- 172.21.15.60:0/3472677475 >> v1:172.21.15.100:6789/0 conn(0x7f9c94071ac0 legacy=0x7f9c940ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c994b9640 1 -- 172.21.15.60:0/3472677475 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9406de70 con 0x7f9c94154ed0 2024-04-25T08:35:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c99cba640 1 --2- 172.21.15.60:0/3472677475 >> v2:172.21.15.100:3300/0 conn(0x7f9c9406c1d0 0x7f9c9406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:50.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c994b9640 1 --2- 172.21.15.60:0/3472677475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94154ed0 0x7f9c940aced0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f9c940cc3d0 tx=0x7f9c9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e5e3003f9e52de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.630+0000 7f9c8bfff640 1 -- 172.21.15.60:0/3472677475 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90037070 con 0x7f9c94154ed0 2024-04-25T08:35:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c8bfff640 1 -- 172.21.15.60:0/3472677475 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c90032440 con 0x7f9c94154ed0 2024-04-25T08:35:50.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c8bfff640 1 -- 172.21.15.60:0/3472677475 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90032810 con 0x7f9c94154ed0 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 -- 172.21.15.60:0/3472677475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94154ed0 msgr2=0x7f9c940aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/3472677475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94154ed0 0x7f9c940aced0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f9c940cc3d0 tx=0x7f9c9002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 -- 172.21.15.60:0/3472677475 shutdown_connections 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/3472677475 >> v2:172.21.15.100:3300/0 conn(0x7f9c9406c1d0 0x7f9c9406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/3472677475 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94154ed0 0x7f9c940aced0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 -- 172.21.15.60:0/3472677475 >> 172.21.15.60:0/3472677475 conn(0x7f9c940af670 msgr2=0x7f9c940b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 -- 172.21.15.60:0/3472677475 shutdown_connections 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.631+0000 7f9c9b71d640 1 -- 172.21.15.60:0/3472677475 wait complete. 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.632+0000 7f9c9b71d640 1 Processor -- start 2024-04-25T08:35:50.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 -- start start 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c94071ac0 0x7f9c941405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c94154ed0 0x7f9c940cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 0x7f9c94143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9406b660 con 0x7f9c94141e00 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9406b4e0 con 0x7f9c94071ac0 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c9b71d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9406b7e0 con 0x7f9c94154ed0 2024-04-25T08:35:50.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c99cba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 0x7f9c94143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c99cba640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 0x7f9c94143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35840/0 (socket says 172.21.15.60:35840) 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.633+0000 7f9c99cba640 1 -- 172.21.15.60:0/236209209 learned_addr learned my addr 172.21.15.60:0/236209209 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c99cba640 1 -- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c94154ed0 msgr2=0x7f9c940cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c99cba640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c94154ed0 0x7f9c940cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c99cba640 1 -- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c94071ac0 msgr2=0x7f9c941405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c99cba640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c94071ac0 0x7f9c941405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c99cba640 1 -- 172.21.15.60:0/236209209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940cef60 con 0x7f9c94141e00 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c99cba640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 0x7f9c94143ab0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00ce70 tx=0x7f9c8c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c01a070 con 0x7f9c94141e00 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c00d840 con 0x7f9c94141e00 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c94140d60 con 0x7f9c94141e00 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00dc30 con 0x7f9c94141e00 2024-04-25T08:35:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.634+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c94141320 con 0x7f9c94141e00 2024-04-25T08:35:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.635+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c940b2b10 con 0x7f9c94141e00 2024-04-25T08:35:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.636+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c8c01d070 con 0x7f9c94141e00 2024-04-25T08:35:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.636+0000 7f9c89ffb640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c7806b500 0x7f9c7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.636+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c8c08ab40 con 0x7f9c94141e00 2024-04-25T08:35:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.636+0000 7f9c994b9640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c7806b500 0x7f9c7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.637+0000 7f9c994b9640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c7806b500 0x7f9c7806d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c9002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.641+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c940cef60 con 0x7f9c94141e00 2024-04-25T08:35:50.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.817+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c94140310 con 0x7f9c7806b500 2024-04-25T08:35:50.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.824+0000 7f9c89ffb640 1 -- 172.21.15.60:0/236209209 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7f9c94140310 con 0x7f9c7806b500 2024-04-25T08:35:50.821 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:50.822 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:35:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c7806b500 msgr2=0x7f9c7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c7806b500 0x7f9c7806d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9c90034000 tx=0x7f9c9002c000 comp rx=0 tx=0).stop 2024-04-25T08:35:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 msgr2=0x7f9c94143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 0x7f9c94143ab0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00ce70 tx=0x7f9c8c00a710 comp rx=0 tx=0).stop 2024-04-25T08:35:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 shutdown_connections 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c94141e00 0x7f9c94143ab0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c94154ed0 0x7f9c940cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c7806b500 0x7f9c7806d9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 --2- 172.21.15.60:0/236209209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c94071ac0 0x7f9c941405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 >> 172.21.15.60:0/236209209 conn(0x7f9c9406c1d0 msgr2=0x7f9c940b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 shutdown_connections 2024-04-25T08:35:50.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.827+0000 7f9c9b71d640 1 -- 172.21.15.60:0/236209209 wait complete. 2024-04-25T08:35:50.832 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-25T08:35:49.486536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":375220,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":364531,"kb_avail":754599500,"statfs":{"total":773094113280,"available":772709888000,"internally_reserved":0,"allocated":10788864,"data_stored":5601212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":373280304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.421513"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353876+0000","last_change":"2024-04-25T08:35:43.090294+0000","last_active":"2024-04-25T08:35:44.353876+0000","last_peered":"2024-04-25T08:35:44.353876+0000","last_clean":"2024-04-25T08:35:44.353876+0000","last_became_active":"2024-04-25T08:35:43.089694+0000","last_became_peered":"2024-04-25T08:35:43.089694+0000","last_unstale":"2024-04-25T08:35:44.353876+0000","last_undegraded":"2024-04-25T08:35:44.353876+0000","last_fullsized":"2024-04-25T08:35:44.353876+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088889+0000","last_change":"2024-04-25T08:35:43.090499+0000","last_active":"2024-04-25T08:35:44.088889+0000","last_peered":"2024-04-25T08:35:44.088889+0000","last_clean":"2024-04-25T08:35:44.088889+0000","last_became_active":"2024-04-25T08:35:43.090183+0000","last_became_peered":"2024-04-25T08:35:43.090183+0000","last_unstale":"2024-04-25T08:35:44.088889+0000","last_undegraded":"2024-04-25T08:35:44.088889+0000","last_fullsized":"2024-04-25T08:35:44.088889+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:01.231065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353866+0000","last_change":"2024-04-25T08:35:43.081017+0000","last_active":"2024-04-25T08:35:44.353866+0000","last_peered":"2024-04-25T08:35:44.353866+0000","last_clean":"2024-04-25T08:35:44.353866+0000","last_became_active":"2024-04-25T08:35:43.080694+0000","last_became_peered":"2024-04-25T08:35:43.080694+0000","last_unstale":"2024-04-25T08:35:44.353866+0000","last_undegraded":"2024-04-25T08:35:44.353866+0000","last_fullsized":"2024-04-25T08:35:44.353866+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354842+0000","last_change":"2024-04-25T08:35:43.095431+0000","last_active":"2024-04-25T08:35:44.354842+0000","last_peered":"2024-04-25T08:35:44.354842+0000","last_clean":"2024-04-25T08:35:44.354842+0000","last_became_active":"2024-04-25T08:35:43.095148+0000","last_became_peered":"2024-04-25T08:35:43.095148+0000","last_unstale":"2024-04-25T08:35:44.354842+0000","last_undegraded":"2024-04-25T08:35:44.354842+0000","last_fullsized":"2024-04-25T08:35:44.354842+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083569+0000","last_change":"2024-04-25T08:35:43.080094+0000","last_active":"2024-04-25T08:35:44.083569+0000","last_peered":"2024-04-25T08:35:44.083569+0000","last_clean":"2024-04-25T08:35:44.083569+0000","last_became_active":"2024-04-25T08:35:43.079466+0000","last_became_peered":"2024-04-25T08:35:43.079466+0000","last_unstale":"2024-04-25T08:35:44.083569+0000","last_undegraded":"2024-04-25T08:35:44.083569+0000","last_fullsized":"2024-04-25T08:35:44.083569+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:58.427376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083148+0000","last_change":"2024-04-25T08:35:43.086363+0000","last_active":"2024-04-25T08:35:44.083148+0000","last_peered":"2024-04-25T08:35:44.083148+0000","last_clean":"2024-04-25T08:35:44.083148+0000","last_became_active":"2024-04-25T08:35:43.085889+0000","last_became_peered":"2024-04-25T08:35:43.085889+0000","last_unstale":"2024-04-25T08:35:44.083148+0000","last_undegraded":"2024-04-25T08:35:44.083148+0000","last_fullsized":"2024-04-25T08:35:44.083148+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:02.377998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347483+0000","last_change":"2024-04-25T08:35:43.089185+0000","last_active":"2024-04-25T08:35:44.347483+0000","last_peered":"2024-04-25T08:35:44.347483+0000","last_clean":"2024-04-25T08:35:44.347483+0000","last_became_active":"2024-04-25T08:35:43.088871+0000","last_became_peered":"2024-04-25T08:35:43.088871+0000","last_unstale":"2024-04-25T08:35:44.347483+0000","last_undegraded":"2024-04-25T08:35:44.347483+0000","last_fullsized":"2024-04-25T08:35:44.347483+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:29.018247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354780+0000","last_change":"2024-04-25T08:35:43.095408+0000","last_active":"2024-04-25T08:35:44.354780+0000","last_peered":"2024-04-25T08:35:44.354780+0000","last_clean":"2024-04-25T08:35:44.354780+0000","last_became_active":"2024-04-25T08:35:43.095119+0000","last_became_peered":"2024-04-25T08:35:43.095119+0000","last_unstale":"2024-04-25T08:35:44.354780+0000","last_undegraded":"2024-04-25T08:35:44.354780+0000","last_fullsized":"2024-04-25T08:35:44.354780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083849+0000","last_change":"2024-04-25T08:35:43.087871+0000","last_active":"2024-04-25T08:35:44.083849+0000","last_peered":"2024-04-25T08:35:44.083849+0000","last_clean":"2024-04-25T08:35:44.083849+0000","last_became_active":"2024-04-25T08:35:43.087575+0000","last_became_peered":"2024-04-25T08:35:43.087575+0000","last_unstale":"2024-04-25T08:35:44.083849+0000","last_undegraded":"2024-04-25T08:35:44.083849+0000","last_fullsized":"2024-04-25T08:35:44.083849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:32.414798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354147+0000","last_change":"2024-04-25T08:35:43.101307+0000","last_active":"2024-04-25T08:35:44.354147+0000","last_peered":"2024-04-25T08:35:44.354147+0000","last_clean":"2024-04-25T08:35:44.354147+0000","last_became_active":"2024-04-25T08:35:43.101003+0000","last_became_peered":"2024-04-25T08:35:43.101003+0000","last_unstale":"2024-04-25T08:35:44.354147+0000","last_undegraded":"2024-04-25T08:35:44.354147+0000","last_fullsized":"2024-04-25T08:35:44.354147+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:14.081591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084050+0000","last_change":"2024-04-25T08:35:43.079921+0000","last_active":"2024-04-25T08:35:44.084050+0000","last_peered":"2024-04-25T08:35:44.084050+0000","last_clean":"2024-04-25T08:35:44.084050+0000","last_became_active":"2024-04-25T08:35:43.079607+0000","last_became_peered":"2024-04-25T08:35:43.079607+0000","last_unstale":"2024-04-25T08:35:44.084050+0000","last_undegraded":"2024-04-25T08:35:44.084050+0000","last_fullsized":"2024-04-25T08:35:44.084050+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:57.986549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353925+0000","last_change":"2024-04-25T08:35:43.087944+0000","last_active":"2024-04-25T08:35:44.353925+0000","last_peered":"2024-04-25T08:35:44.353925+0000","last_clean":"2024-04-25T08:35:44.353925+0000","last_became_active":"2024-04-25T08:35:43.087638+0000","last_became_peered":"2024-04-25T08:35:43.087638+0000","last_unstale":"2024-04-25T08:35:44.353925+0000","last_undegraded":"2024-04-25T08:35:44.353925+0000","last_fullsized":"2024-04-25T08:35:44.353925+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.692505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354540+0000","last_change":"2024-04-25T08:35:43.095160+0000","last_active":"2024-04-25T08:35:44.354540+0000","last_peered":"2024-04-25T08:35:44.354540+0000","last_clean":"2024-04-25T08:35:44.354540+0000","last_became_active":"2024-04-25T08:35:43.094865+0000","last_became_peered":"2024-04-25T08:35:43.094865+0000","last_unstale":"2024-04-25T08:35:44.354540+0000","last_undegraded":"2024-04-25T08:35:44.354540+0000","last_fullsized":"2024-04-25T08:35:44.354540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083498+0000","last_change":"2024-04-25T08:35:43.096325+0000","last_active":"2024-04-25T08:35:44.083498+0000","last_peered":"2024-04-25T08:35:44.083498+0000","last_clean":"2024-04-25T08:35:44.083498+0000","last_became_active":"2024-04-25T08:35:43.096042+0000","last_became_peered":"2024-04-25T08:35:43.096042+0000","last_unstale":"2024-04-25T08:35:44.083498+0000","last_undegraded":"2024-04-25T08:35:44.083498+0000","last_fullsized":"2024-04-25T08:35:44.083498+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:27.748314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354752+0000","last_change":"2024-04-25T08:35:43.080738+0000","last_active":"2024-04-25T08:35:44.354752+0000","last_peered":"2024-04-25T08:35:44.354752+0000","last_clean":"2024-04-25T08:35:44.354752+0000","last_became_active":"2024-04-25T08:35:43.080444+0000","last_became_peered":"2024-04-25T08:35:43.080444+0000","last_unstale":"2024-04-25T08:35:44.354752+0000","last_undegraded":"2024-04-25T08:35:44.354752+0000","last_fullsized":"2024-04-25T08:35:44.354752+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083259+0000","last_change":"2024-04-25T08:35:43.079286+0000","last_active":"2024-04-25T08:35:44.083259+0000","last_peered":"2024-04-25T08:35:44.083259+0000","last_clean":"2024-04-25T08:35:44.083259+0000","last_became_active":"2024-04-25T08:35:43.078966+0000","last_became_peered":"2024-04-25T08:35:43.078966+0000","last_unstale":"2024-04-25T08:35:44.083259+0000","last_undegraded":"2024-04-25T08:35:44.083259+0000","last_fullsized":"2024-04-25T08:35:44.083259+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:48.519958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083467+0000","last_change":"2024-04-25T08:35:43.079723+0000","last_active":"2024-04-25T08:35:44.083467+0000","last_peered":"2024-04-25T08:35:44.083467+0000","last_clean":"2024-04-25T08:35:44.083467+0000","last_became_active":"2024-04-25T08:35:43.079074+0000","last_became_peered":"2024-04-25T08:35:43.079074+0000","last_unstale":"2024-04-25T08:35:44.083467+0000","last_undegraded":"2024-04-25T08:35:44.083467+0000","last_fullsized":"2024-04-25T08:35:44.083467+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:07.692269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354121+0000","last_change":"2024-04-25T08:35:43.089177+0000","last_active":"2024-04-25T08:35:44.354121+0000","last_peered":"2024-04-25T08:35:44.354121+0000","last_clean":"2024-04-25T08:35:44.354121+0000","last_became_active":"2024-04-25T08:35:43.088869+0000","last_became_peered":"2024-04-25T08:35:43.088869+0000","last_unstale":"2024-04-25T08:35:44.354121+0000","last_undegraded":"2024-04-25T08:35:44.354121+0000","last_fullsized":"2024-04-25T08:35:44.354121+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084198+0000","last_change":"2024-04-25T08:35:43.094595+0000","last_active":"2024-04-25T08:35:44.084198+0000","last_peered":"2024-04-25T08:35:44.084198+0000","last_clean":"2024-04-25T08:35:44.084198+0000","last_became_active":"2024-04-25T08:35:43.093864+0000","last_became_peered":"2024-04-25T08:35:43.093864+0000","last_unstale":"2024-04-25T08:35:44.084198+0000","last_undegraded":"2024-04-25T08:35:44.084198+0000","last_fullsized":"2024-04-25T08:35:44.084198+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354674+0000","last_change":"2024-04-25T08:35:43.086251+0000","last_active":"2024-04-25T08:35:44.354674+0000","last_peered":"2024-04-25T08:35:44.354674+0000","last_clean":"2024-04-25T08:35:44.354674+0000","last_became_active":"2024-04-25T08:35:43.085938+0000","last_became_peered":"2024-04-25T08:35:43.085938+0000","last_unstale":"2024-04-25T08:35:44.354674+0000","last_undegraded":"2024-04-25T08:35:44.354674+0000","last_fullsized":"2024-04-25T08:35:44.354674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:12.349295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084122+0000","last_change":"2024-04-25T08:35:43.084909+0000","last_active":"2024-04-25T08:35:44.084122+0000","last_peered":"2024-04-25T08:35:44.084122+0000","last_clean":"2024-04-25T08:35:44.084122+0000","last_became_active":"2024-04-25T08:35:43.084519+0000","last_became_peered":"2024-04-25T08:35:43.084519+0000","last_unstale":"2024-04-25T08:35:44.084122+0000","last_undegraded":"2024-04-25T08:35:44.084122+0000","last_fullsized":"2024-04-25T08:35:44.084122+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083572+0000","last_change":"2024-04-25T08:35:43.083871+0000","last_active":"2024-04-25T08:35:44.083572+0000","last_peered":"2024-04-25T08:35:44.083572+0000","last_clean":"2024-04-25T08:35:44.083572+0000","last_became_active":"2024-04-25T08:35:43.083574+0000","last_became_peered":"2024-04-25T08:35:43.083574+0000","last_unstale":"2024-04-25T08:35:44.083572+0000","last_undegraded":"2024-04-25T08:35:44.083572+0000","last_fullsized":"2024-04-25T08:35:44.083572+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.102694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084409+0000","last_change":"2024-04-25T08:35:43.100281+0000","last_active":"2024-04-25T08:35:44.084409+0000","last_peered":"2024-04-25T08:35:44.084409+0000","last_clean":"2024-04-25T08:35:44.084409+0000","last_became_active":"2024-04-25T08:35:43.100008+0000","last_became_peered":"2024-04-25T08:35:43.100008+0000","last_unstale":"2024-04-25T08:35:44.084409+0000","last_undegraded":"2024-04-25T08:35:44.084409+0000","last_fullsized":"2024-04-25T08:35:44.084409+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354442+0000","last_change":"2024-04-25T08:35:43.080351+0000","last_active":"2024-04-25T08:35:44.354442+0000","last_peered":"2024-04-25T08:35:44.354442+0000","last_clean":"2024-04-25T08:35:44.354442+0000","last_became_active":"2024-04-25T08:35:43.080020+0000","last_became_peered":"2024-04-25T08:35:43.080020+0000","last_unstale":"2024-04-25T08:35:44.354442+0000","last_undegraded":"2024-04-25T08:35:44.354442+0000","last_fullsized":"2024-04-25T08:35:44.354442+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.089034+0000","last_change":"2024-04-25T08:35:43.090646+0000","last_active":"2024-04-25T08:35:44.089034+0000","last_peered":"2024-04-25T08:35:44.089034+0000","last_clean":"2024-04-25T08:35:44.089034+0000","last_became_active":"2024-04-25T08:35:43.090325+0000","last_became_peered":"2024-04-25T08:35:43.090325+0000","last_unstale":"2024-04-25T08:35:44.089034+0000","last_undegraded":"2024-04-25T08:35:44.089034+0000","last_fullsized":"2024-04-25T08:35:44.089034+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:38.824878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083931+0000","last_change":"2024-04-25T08:35:43.087490+0000","last_active":"2024-04-25T08:35:44.083931+0000","last_peered":"2024-04-25T08:35:44.083931+0000","last_clean":"2024-04-25T08:35:44.083931+0000","last_became_active":"2024-04-25T08:35:43.087039+0000","last_became_peered":"2024-04-25T08:35:43.087039+0000","last_unstale":"2024-04-25T08:35:44.083931+0000","last_undegraded":"2024-04-25T08:35:44.083931+0000","last_fullsized":"2024-04-25T08:35:44.083931+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.866982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354143+0000","last_change":"2024-04-25T08:35:43.090435+0000","last_active":"2024-04-25T08:35:44.354143+0000","last_peered":"2024-04-25T08:35:44.354143+0000","last_clean":"2024-04-25T08:35:44.354143+0000","last_became_active":"2024-04-25T08:35:43.090166+0000","last_became_peered":"2024-04-25T08:35:43.090166+0000","last_unstale":"2024-04-25T08:35:44.354143+0000","last_undegraded":"2024-04-25T08:35:44.354143+0000","last_fullsized":"2024-04-25T08:35:44.354143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:10.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354827+0000","last_change":"2024-04-25T08:35:43.084313+0000","last_active":"2024-04-25T08:35:44.354827+0000","last_peered":"2024-04-25T08:35:44.354827+0000","last_clean":"2024-04-25T08:35:44.354827+0000","last_became_active":"2024-04-25T08:35:43.084027+0000","last_became_peered":"2024-04-25T08:35:43.084027+0000","last_unstale":"2024-04-25T08:35:44.354827+0000","last_undegraded":"2024-04-25T08:35:44.354827+0000","last_fullsized":"2024-04-25T08:35:44.354827+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347659+0000","last_change":"2024-04-25T08:35:43.091542+0000","last_active":"2024-04-25T08:35:44.347659+0000","last_peered":"2024-04-25T08:35:44.347659+0000","last_clean":"2024-04-25T08:35:44.347659+0000","last_became_active":"2024-04-25T08:35:43.091251+0000","last_became_peered":"2024-04-25T08:35:43.091251+0000","last_unstale":"2024-04-25T08:35:44.347659+0000","last_undegraded":"2024-04-25T08:35:44.347659+0000","last_fullsized":"2024-04-25T08:35:44.347659+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:17.458170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083847+0000","last_change":"2024-04-25T08:35:43.088502+0000","last_active":"2024-04-25T08:35:44.083847+0000","last_peered":"2024-04-25T08:35:44.083847+0000","last_clean":"2024-04-25T08:35:44.083847+0000","last_became_active":"2024-04-25T08:35:43.088156+0000","last_became_peered":"2024-04-25T08:35:43.088156+0000","last_unstale":"2024-04-25T08:35:44.083847+0000","last_undegraded":"2024-04-25T08:35:44.083847+0000","last_fullsized":"2024-04-25T08:35:44.083847+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:29.004324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088676+0000","last_change":"2024-04-25T08:35:43.099672+0000","last_active":"2024-04-25T08:35:44.088676+0000","last_peered":"2024-04-25T08:35:44.088676+0000","last_clean":"2024-04-25T08:35:44.088676+0000","last_became_active":"2024-04-25T08:35:43.099376+0000","last_became_peered":"2024-04-25T08:35:43.099376+0000","last_unstale":"2024-04-25T08:35:44.088676+0000","last_undegraded":"2024-04-25T08:35:44.088676+0000","last_fullsized":"2024-04-25T08:35:44.088676+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083093+0000","last_change":"2024-04-25T08:35:43.080033+0000","last_active":"2024-04-25T08:35:44.083093+0000","last_peered":"2024-04-25T08:35:44.083093+0000","last_clean":"2024-04-25T08:35:44.083093+0000","last_became_active":"2024-04-25T08:35:43.079712+0000","last_became_peered":"2024-04-25T08:35:43.079712+0000","last_unstale":"2024-04-25T08:35:44.083093+0000","last_undegraded":"2024-04-25T08:35:44.083093+0000","last_fullsized":"2024-04-25T08:35:44.083093+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.767670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083669+0000","last_change":"2024-04-25T08:35:43.084040+0000","last_active":"2024-04-25T08:35:44.083669+0000","last_peered":"2024-04-25T08:35:44.083669+0000","last_clean":"2024-04-25T08:35:44.083669+0000","last_became_active":"2024-04-25T08:35:43.082074+0000","last_became_peered":"2024-04-25T08:35:43.082074+0000","last_unstale":"2024-04-25T08:35:44.083669+0000","last_undegraded":"2024-04-25T08:35:44.083669+0000","last_fullsized":"2024-04-25T08:35:44.083669+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:57.646322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"96'4","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084379+0000","last_change":"2024-04-25T08:35:42.083008+0000","last_active":"2024-04-25T08:35:44.084379+0000","last_peered":"2024-04-25T08:35:44.084379+0000","last_clean":"2024-04-25T08:35:44.084379+0000","last_became_active":"2024-04-25T08:35:42.082722+0000","last_became_peered":"2024-04-25T08:35:42.082722+0000","last_unstale":"2024-04-25T08:35:44.084379+0000","last_undegraded":"2024-04-25T08:35:44.084379+0000","last_fullsized":"2024-04-25T08:35:44.084379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083922+0000","last_change":"2024-04-25T08:35:43.092634+0000","last_active":"2024-04-25T08:35:44.083922+0000","last_peered":"2024-04-25T08:35:44.083922+0000","last_clean":"2024-04-25T08:35:44.083922+0000","last_became_active":"2024-04-25T08:35:43.092347+0000","last_became_peered":"2024-04-25T08:35:43.092347+0000","last_unstale":"2024-04-25T08:35:44.083922+0000","last_undegraded":"2024-04-25T08:35:44.083922+0000","last_fullsized":"2024-04-25T08:35:44.083922+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:37.775158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084482+0000","last_change":"2024-04-25T08:35:42.077498+0000","last_active":"2024-04-25T08:35:44.084482+0000","last_peered":"2024-04-25T08:35:44.084482+0000","last_clean":"2024-04-25T08:35:44.084482+0000","last_became_active":"2024-04-25T08:35:42.077203+0000","last_became_peered":"2024-04-25T08:35:42.077203+0000","last_unstale":"2024-04-25T08:35:44.084482+0000","last_undegraded":"2024-04-25T08:35:44.084482+0000","last_fullsized":"2024-04-25T08:35:44.084482+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347666+0000","last_change":"2024-04-25T08:35:43.099723+0000","last_active":"2024-04-25T08:35:44.347666+0000","last_peered":"2024-04-25T08:35:44.347666+0000","last_clean":"2024-04-25T08:35:44.347666+0000","last_became_active":"2024-04-25T08:35:43.099412+0000","last_became_peered":"2024-04-25T08:35:43.099412+0000","last_unstale":"2024-04-25T08:35:44.347666+0000","last_undegraded":"2024-04-25T08:35:44.347666+0000","last_fullsized":"2024-04-25T08:35:44.347666+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:14.707807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347703+0000","last_change":"2024-04-25T08:35:42.084090+0000","last_active":"2024-04-25T08:35:44.347703+0000","last_peered":"2024-04-25T08:35:44.347703+0000","last_clean":"2024-04-25T08:35:44.347703+0000","last_became_active":"2024-04-25T08:35:42.083831+0000","last_became_peered":"2024-04-25T08:35:42.083831+0000","last_unstale":"2024-04-25T08:35:44.347703+0000","last_undegraded":"2024-04-25T08:35:44.347703+0000","last_fullsized":"2024-04-25T08:35:44.347703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:10.496016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083517+0000","last_change":"2024-04-25T08:35:43.079045+0000","last_active":"2024-04-25T08:35:44.083517+0000","last_peered":"2024-04-25T08:35:44.083517+0000","last_clean":"2024-04-25T08:35:44.083517+0000","last_became_active":"2024-04-25T08:35:43.078753+0000","last_became_peered":"2024-04-25T08:35:43.078753+0000","last_unstale":"2024-04-25T08:35:44.083517+0000","last_undegraded":"2024-04-25T08:35:44.083517+0000","last_fullsized":"2024-04-25T08:35:44.083517+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:29.703738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084081+0000","last_change":"2024-04-25T08:35:42.076044+0000","last_active":"2024-04-25T08:35:44.084081+0000","last_peered":"2024-04-25T08:35:44.084081+0000","last_clean":"2024-04-25T08:35:44.084081+0000","last_became_active":"2024-04-25T08:35:42.075775+0000","last_became_peered":"2024-04-25T08:35:42.075775+0000","last_unstale":"2024-04-25T08:35:44.084081+0000","last_undegraded":"2024-04-25T08:35:44.084081+0000","last_fullsized":"2024-04-25T08:35:44.084081+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:43:57.424993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354942+0000","last_change":"2024-04-25T08:35:43.085489+0000","last_active":"2024-04-25T08:35:44.354942+0000","last_peered":"2024-04-25T08:35:44.354942+0000","last_clean":"2024-04-25T08:35:44.354942+0000","last_became_active":"2024-04-25T08:35:43.085149+0000","last_became_peered":"2024-04-25T08:35:43.085149+0000","last_unstale":"2024-04-25T08:35:44.354942+0000","last_undegraded":"2024-04-25T08:35:44.354942+0000","last_fullsized":"2024-04-25T08:35:44.354942+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:56.811062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083722+0000","last_change":"2024-04-25T08:35:42.083359+0000","last_active":"2024-04-25T08:35:44.083722+0000","last_peered":"2024-04-25T08:35:44.083722+0000","last_clean":"2024-04-25T08:35:44.083722+0000","last_became_active":"2024-04-25T08:35:42.083075+0000","last_became_peered":"2024-04-25T08:35:42.083075+0000","last_unstale":"2024-04-25T08:35:44.083722+0000","last_undegraded":"2024-04-25T08:35:44.083722+0000","last_fullsized":"2024-04-25T08:35:44.083722+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:59.795889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084226+0000","last_change":"2024-04-25T08:35:43.085701+0000","last_active":"2024-04-25T08:35:44.084226+0000","last_peered":"2024-04-25T08:35:44.084226+0000","last_clean":"2024-04-25T08:35:44.084226+0000","last_became_active":"2024-04-25T08:35:43.085405+0000","last_became_peered":"2024-04-25T08:35:43.085405+0000","last_unstale":"2024-04-25T08:35:44.084226+0000","last_undegraded":"2024-04-25T08:35:44.084226+0000","last_fullsized":"2024-04-25T08:35:44.084226+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:49.755267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354789+0000","last_change":"2024-04-25T08:35:42.087255+0000","last_active":"2024-04-25T08:35:44.354789+0000","last_peered":"2024-04-25T08:35:44.354789+0000","last_clean":"2024-04-25T08:35:44.354789+0000","last_became_active":"2024-04-25T08:35:42.086862+0000","last_became_peered":"2024-04-25T08:35:42.086862+0000","last_unstale":"2024-04-25T08:35:44.354789+0000","last_undegraded":"2024-04-25T08:35:44.354789+0000","last_fullsized":"2024-04-25T08:35:44.354789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:31.231173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083302+0000","last_change":"2024-04-25T08:35:43.085631+0000","last_active":"2024-04-25T08:35:44.083302+0000","last_peered":"2024-04-25T08:35:44.083302+0000","last_clean":"2024-04-25T08:35:44.083302+0000","last_became_active":"2024-04-25T08:35:43.085345+0000","last_became_peered":"2024-04-25T08:35:43.085345+0000","last_unstale":"2024-04-25T08:35:44.083302+0000","last_undegraded":"2024-04-25T08:35:44.083302+0000","last_fullsized":"2024-04-25T08:35:44.083302+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:52.198356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354848+0000","last_change":"2024-04-25T08:35:42.078496+0000","last_active":"2024-04-25T08:35:44.354848+0000","last_peered":"2024-04-25T08:35:44.354848+0000","last_clean":"2024-04-25T08:35:44.354848+0000","last_became_active":"2024-04-25T08:35:42.078199+0000","last_became_peered":"2024-04-25T08:35:42.078199+0000","last_unstale":"2024-04-25T08:35:44.354848+0000","last_undegraded":"2024-04-25T08:35:44.354848+0000","last_fullsized":"2024-04-25T08:35:44.354848+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:28.001941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354685+0000","last_change":"2024-04-25T08:35:43.096815+0000","last_active":"2024-04-25T08:35:44.354685+0000","last_peered":"2024-04-25T08:35:44.354685+0000","last_clean":"2024-04-25T08:35:44.354685+0000","last_became_active":"2024-04-25T08:35:43.096237+0000","last_became_peered":"2024-04-25T08:35:43.096237+0000","last_unstale":"2024-04-25T08:35:44.354685+0000","last_undegraded":"2024-04-25T08:35:44.354685+0000","last_fullsized":"2024-04-25T08:35:44.354685+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347360+0000","last_change":"2024-04-25T08:35:42.084779+0000","last_active":"2024-04-25T08:35:44.347360+0000","last_peered":"2024-04-25T08:35:44.347360+0000","last_clean":"2024-04-25T08:35:44.347360+0000","last_became_active":"2024-04-25T08:35:42.084495+0000","last_became_peered":"2024-04-25T08:35:42.084495+0000","last_unstale":"2024-04-25T08:35:44.347360+0000","last_undegraded":"2024-04-25T08:35:44.347360+0000","last_fullsized":"2024-04-25T08:35:44.347360+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:37.822495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083757+0000","last_change":"2024-04-25T08:35:43.084977+0000","last_active":"2024-04-25T08:35:44.083757+0000","last_peered":"2024-04-25T08:35:44.083757+0000","last_clean":"2024-04-25T08:35:44.083757+0000","last_became_active":"2024-04-25T08:35:43.084694+0000","last_became_peered":"2024-04-25T08:35:43.084694+0000","last_unstale":"2024-04-25T08:35:44.083757+0000","last_undegraded":"2024-04-25T08:35:44.083757+0000","last_fullsized":"2024-04-25T08:35:44.083757+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:50.121067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.089065+0000","last_change":"2024-04-25T08:35:42.084063+0000","last_active":"2024-04-25T08:35:44.089065+0000","last_peered":"2024-04-25T08:35:44.089065+0000","last_clean":"2024-04-25T08:35:44.089065+0000","last_became_active":"2024-04-25T08:35:42.083798+0000","last_became_peered":"2024-04-25T08:35:42.083798+0000","last_unstale":"2024-04-25T08:35:44.089065+0000","last_undegraded":"2024-04-25T08:35:44.089065+0000","last_fullsized":"2024-04-25T08:35:44.089065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:16.323534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347787+0000","last_change":"2024-04-25T08:35:43.087459+0000","last_active":"2024-04-25T08:35:44.347787+0000","last_peered":"2024-04-25T08:35:44.347787+0000","last_clean":"2024-04-25T08:35:44.347787+0000","last_became_active":"2024-04-25T08:35:43.087040+0000","last_became_peered":"2024-04-25T08:35:43.087040+0000","last_unstale":"2024-04-25T08:35:44.347787+0000","last_undegraded":"2024-04-25T08:35:44.347787+0000","last_fullsized":"2024-04-25T08:35:44.347787+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:14.637748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354368+0000","last_change":"2024-04-25T08:35:42.079015+0000","last_active":"2024-04-25T08:35:44.354368+0000","last_peered":"2024-04-25T08:35:44.354368+0000","last_clean":"2024-04-25T08:35:44.354368+0000","last_became_active":"2024-04-25T08:35:42.078722+0000","last_became_peered":"2024-04-25T08:35:42.078722+0000","last_unstale":"2024-04-25T08:35:44.354368+0000","last_undegraded":"2024-04-25T08:35:44.354368+0000","last_fullsized":"2024-04-25T08:35:44.354368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354270+0000","last_change":"2024-04-25T08:35:43.085992+0000","last_active":"2024-04-25T08:35:44.354270+0000","last_peered":"2024-04-25T08:35:44.354270+0000","last_clean":"2024-04-25T08:35:44.354270+0000","last_became_active":"2024-04-25T08:35:43.082970+0000","last_became_peered":"2024-04-25T08:35:43.082970+0000","last_unstale":"2024-04-25T08:35:44.354270+0000","last_undegraded":"2024-04-25T08:35:44.354270+0000","last_fullsized":"2024-04-25T08:35:44.354270+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:16.194776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"96'3","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084306+0000","last_change":"2024-04-25T08:35:42.085258+0000","last_active":"2024-04-25T08:35:44.084306+0000","last_peered":"2024-04-25T08:35:44.084306+0000","last_clean":"2024-04-25T08:35:44.084306+0000","last_became_active":"2024-04-25T08:35:42.084942+0000","last_became_peered":"2024-04-25T08:35:42.084942+0000","last_unstale":"2024-04-25T08:35:44.084306+0000","last_undegraded":"2024-04-25T08:35:44.084306+0000","last_fullsized":"2024-04-25T08:35:44.084306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:58.723955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354667+0000","last_change":"2024-04-25T08:35:43.094836+0000","last_active":"2024-04-25T08:35:44.354667+0000","last_peered":"2024-04-25T08:35:44.354667+0000","last_clean":"2024-04-25T08:35:44.354667+0000","last_became_active":"2024-04-25T08:35:43.093301+0000","last_became_peered":"2024-04-25T08:35:43.093301+0000","last_unstale":"2024-04-25T08:35:44.354667+0000","last_undegraded":"2024-04-25T08:35:44.354667+0000","last_fullsized":"2024-04-25T08:35:44.354667+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:17.325917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"96'6","reported_seq":20,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083767+0000","last_change":"2024-04-25T08:35:42.083983+0000","last_active":"2024-04-25T08:35:44.083767+0000","last_peered":"2024-04-25T08:35:44.083767+0000","last_clean":"2024-04-25T08:35:44.083767+0000","last_became_active":"2024-04-25T08:35:42.083670+0000","last_became_peered":"2024-04-25T08:35:42.083670+0000","last_unstale":"2024-04-25T08:35:44.083767+0000","last_undegraded":"2024-04-25T08:35:44.083767+0000","last_fullsized":"2024-04-25T08:35:44.083767+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:57.266743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355036+0000","last_change":"2024-04-25T08:35:43.095386+0000","last_active":"2024-04-25T08:35:44.355036+0000","last_peered":"2024-04-25T08:35:44.355036+0000","last_clean":"2024-04-25T08:35:44.355036+0000","last_became_active":"2024-04-25T08:35:43.093364+0000","last_became_peered":"2024-04-25T08:35:43.093364+0000","last_unstale":"2024-04-25T08:35:44.355036+0000","last_undegraded":"2024-04-25T08:35:44.355036+0000","last_fullsized":"2024-04-25T08:35:44.355036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347540+0000","last_change":"2024-04-25T08:35:42.086794+0000","last_active":"2024-04-25T08:35:44.347540+0000","last_peered":"2024-04-25T08:35:44.347540+0000","last_clean":"2024-04-25T08:35:44.347540+0000","last_became_active":"2024-04-25T08:35:42.086433+0000","last_became_peered":"2024-04-25T08:35:42.086433+0000","last_unstale":"2024-04-25T08:35:44.347540+0000","last_undegraded":"2024-04-25T08:35:44.347540+0000","last_fullsized":"2024-04-25T08:35:44.347540+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:00.228682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:43.087418+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:43.087100+0000","last_became_peered":"2024-04-25T08:35:43.087100+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:23.384291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083399+0000","last_change":"2024-04-25T08:35:42.084034+0000","last_active":"2024-04-25T08:35:44.083399+0000","last_peered":"2024-04-25T08:35:44.083399+0000","last_clean":"2024-04-25T08:35:44.083399+0000","last_became_active":"2024-04-25T08:35:42.083714+0000","last_became_peered":"2024-04-25T08:35:42.083714+0000","last_unstale":"2024-04-25T08:35:44.083399+0000","last_undegraded":"2024-04-25T08:35:44.083399+0000","last_fullsized":"2024-04-25T08:35:44.083399+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:41:35.130250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354284+0000","last_change":"2024-04-25T08:35:43.100537+0000","last_active":"2024-04-25T08:35:44.354284+0000","last_peered":"2024-04-25T08:35:44.354284+0000","last_clean":"2024-04-25T08:35:44.354284+0000","last_became_active":"2024-04-25T08:35:43.100246+0000","last_became_peered":"2024-04-25T08:35:43.100246+0000","last_unstale":"2024-04-25T08:35:44.354284+0000","last_undegraded":"2024-04-25T08:35:44.354284+0000","last_fullsized":"2024-04-25T08:35:44.354284+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.420534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354726+0000","last_change":"2024-04-25T08:35:42.079064+0000","last_active":"2024-04-25T08:35:44.354726+0000","last_peered":"2024-04-25T08:35:44.354726+0000","last_clean":"2024-04-25T08:35:44.354726+0000","last_became_active":"2024-04-25T08:35:42.078692+0000","last_became_peered":"2024-04-25T08:35:42.078692+0000","last_unstale":"2024-04-25T08:35:44.354726+0000","last_undegraded":"2024-04-25T08:35:44.354726+0000","last_fullsized":"2024-04-25T08:35:44.354726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:03.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353823+0000","last_change":"2024-04-25T08:35:43.088607+0000","last_active":"2024-04-25T08:35:44.353823+0000","last_peered":"2024-04-25T08:35:44.353823+0000","last_clean":"2024-04-25T08:35:44.353823+0000","last_became_active":"2024-04-25T08:35:43.088317+0000","last_became_peered":"2024-04-25T08:35:43.088317+0000","last_unstale":"2024-04-25T08:35:44.353823+0000","last_undegraded":"2024-04-25T08:35:44.353823+0000","last_fullsized":"2024-04-25T08:35:44.353823+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:18.622878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"96'8","reported_seq":22,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.355062+0000","last_change":"2024-04-25T08:35:42.086642+0000","last_active":"2024-04-25T08:35:44.355062+0000","last_peered":"2024-04-25T08:35:44.355062+0000","last_clean":"2024-04-25T08:35:44.355062+0000","last_became_active":"2024-04-25T08:35:42.086219+0000","last_became_peered":"2024-04-25T08:35:42.086219+0000","last_unstale":"2024-04-25T08:35:44.355062+0000","last_undegraded":"2024-04-25T08:35:44.355062+0000","last_fullsized":"2024-04-25T08:35:44.355062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:16.239961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083745+0000","last_change":"2024-04-25T08:35:43.084349+0000","last_active":"2024-04-25T08:35:44.083745+0000","last_peered":"2024-04-25T08:35:44.083745+0000","last_clean":"2024-04-25T08:35:44.083745+0000","last_became_active":"2024-04-25T08:35:43.081684+0000","last_became_peered":"2024-04-25T08:35:43.081684+0000","last_unstale":"2024-04-25T08:35:44.083745+0000","last_undegraded":"2024-04-25T08:35:44.083745+0000","last_fullsized":"2024-04-25T08:35:44.083745+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:47.474056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084497+0000","last_change":"2024-04-25T08:35:42.074237+0000","last_active":"2024-04-25T08:35:44.084497+0000","last_peered":"2024-04-25T08:35:44.084497+0000","last_clean":"2024-04-25T08:35:44.084497+0000","last_became_active":"2024-04-25T08:35:42.073719+0000","last_became_peered":"2024-04-25T08:35:42.073719+0000","last_unstale":"2024-04-25T08:35:44.084497+0000","last_undegraded":"2024-04-25T08:35:44.084497+0000","last_fullsized":"2024-04-25T08:35:44.084497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:31.412415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083912+0000","last_change":"2024-04-25T08:35:43.088892+0000","last_active":"2024-04-25T08:35:44.083912+0000","last_peered":"2024-04-25T08:35:44.083912+0000","last_clean":"2024-04-25T08:35:44.083912+0000","last_became_active":"2024-04-25T08:35:43.088572+0000","last_became_peered":"2024-04-25T08:35:43.088572+0000","last_unstale":"2024-04-25T08:35:44.083912+0000","last_undegraded":"2024-04-25T08:35:44.083912+0000","last_fullsized":"2024-04-25T08:35:44.083912+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.455238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347530+0000","last_change":"2024-04-25T08:35:42.079831+0000","last_active":"2024-04-25T08:35:44.347530+0000","last_peered":"2024-04-25T08:35:44.347530+0000","last_clean":"2024-04-25T08:35:44.347530+0000","last_became_active":"2024-04-25T08:35:42.079514+0000","last_became_peered":"2024-04-25T08:35:42.079514+0000","last_unstale":"2024-04-25T08:35:44.347530+0000","last_undegraded":"2024-04-25T08:35:44.347530+0000","last_fullsized":"2024-04-25T08:35:44.347530+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:55.808679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084175+0000","last_change":"2024-04-25T08:35:43.092287+0000","last_active":"2024-04-25T08:35:44.084175+0000","last_peered":"2024-04-25T08:35:44.084175+0000","last_clean":"2024-04-25T08:35:44.084175+0000","last_became_active":"2024-04-25T08:35:43.091768+0000","last_became_peered":"2024-04-25T08:35:43.091768+0000","last_unstale":"2024-04-25T08:35:44.084175+0000","last_undegraded":"2024-04-25T08:35:44.084175+0000","last_fullsized":"2024-04-25T08:35:44.084175+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:47.773892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"96'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084367+0000","last_change":"2024-04-25T08:35:42.075127+0000","last_active":"2024-04-25T08:35:44.084367+0000","last_peered":"2024-04-25T08:35:44.084367+0000","last_clean":"2024-04-25T08:35:44.084367+0000","last_became_active":"2024-04-25T08:35:42.074842+0000","last_became_peered":"2024-04-25T08:35:42.074842+0000","last_unstale":"2024-04-25T08:35:44.084367+0000","last_undegraded":"2024-04-25T08:35:44.084367+0000","last_fullsized":"2024-04-25T08:35:44.084367+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:25.370203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083108+0000","last_change":"2024-04-25T08:35:43.083888+0000","last_active":"2024-04-25T08:35:44.083108+0000","last_peered":"2024-04-25T08:35:44.083108+0000","last_clean":"2024-04-25T08:35:44.083108+0000","last_became_active":"2024-04-25T08:35:43.083585+0000","last_became_peered":"2024-04-25T08:35:43.083585+0000","last_unstale":"2024-04-25T08:35:44.083108+0000","last_undegraded":"2024-04-25T08:35:44.083108+0000","last_fullsized":"2024-04-25T08:35:44.083108+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:43.652486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088676+0000","last_change":"2024-04-25T08:35:42.082861+0000","last_active":"2024-04-25T08:35:44.088676+0000","last_peered":"2024-04-25T08:35:44.088676+0000","last_clean":"2024-04-25T08:35:44.088676+0000","last_became_active":"2024-04-25T08:35:42.082582+0000","last_became_peered":"2024-04-25T08:35:42.082582+0000","last_unstale":"2024-04-25T08:35:44.088676+0000","last_undegraded":"2024-04-25T08:35:44.088676+0000","last_fullsized":"2024-04-25T08:35:44.088676+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:55.864599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088780+0000","last_change":"2024-04-25T08:35:43.090667+0000","last_active":"2024-04-25T08:35:44.088780+0000","last_peered":"2024-04-25T08:35:44.088780+0000","last_clean":"2024-04-25T08:35:44.088780+0000","last_became_active":"2024-04-25T08:35:43.090389+0000","last_became_peered":"2024-04-25T08:35:43.090389+0000","last_unstale":"2024-04-25T08:35:44.088780+0000","last_undegraded":"2024-04-25T08:35:44.088780+0000","last_fullsized":"2024-04-25T08:35:44.088780+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:11.498399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354254+0000","last_change":"2024-04-25T08:35:42.077041+0000","last_active":"2024-04-25T08:35:44.354254+0000","last_peered":"2024-04-25T08:35:44.354254+0000","last_clean":"2024-04-25T08:35:44.354254+0000","last_became_active":"2024-04-25T08:35:42.076737+0000","last_became_peered":"2024-04-25T08:35:42.076737+0000","last_unstale":"2024-04-25T08:35:44.354254+0000","last_undegraded":"2024-04-25T08:35:44.354254+0000","last_fullsized":"2024-04-25T08:35:44.354254+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:54.769371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088891+0000","last_change":"2024-04-25T08:35:43.099702+0000","last_active":"2024-04-25T08:35:44.088891+0000","last_peered":"2024-04-25T08:35:44.088891+0000","last_clean":"2024-04-25T08:35:44.088891+0000","last_became_active":"2024-04-25T08:35:43.099426+0000","last_became_peered":"2024-04-25T08:35:43.099426+0000","last_unstale":"2024-04-25T08:35:44.088891+0000","last_undegraded":"2024-04-25T08:35:44.088891+0000","last_fullsized":"2024-04-25T08:35:44.088891+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:19.564136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088746+0000","last_change":"2024-04-25T08:35:42.086887+0000","last_active":"2024-04-25T08:35:44.088746+0000","last_peered":"2024-04-25T08:35:44.088746+0000","last_clean":"2024-04-25T08:35:44.088746+0000","last_became_active":"2024-04-25T08:35:42.086576+0000","last_became_peered":"2024-04-25T08:35:42.086576+0000","last_unstale":"2024-04-25T08:35:44.088746+0000","last_undegraded":"2024-04-25T08:35:44.088746+0000","last_fullsized":"2024-04-25T08:35:44.088746+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:46.771509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083581+0000","last_change":"2024-04-25T08:35:43.091743+0000","last_active":"2024-04-25T08:35:44.083581+0000","last_peered":"2024-04-25T08:35:44.083581+0000","last_clean":"2024-04-25T08:35:44.083581+0000","last_became_active":"2024-04-25T08:35:43.091470+0000","last_became_peered":"2024-04-25T08:35:43.091470+0000","last_unstale":"2024-04-25T08:35:44.083581+0000","last_undegraded":"2024-04-25T08:35:44.083581+0000","last_fullsized":"2024-04-25T08:35:44.083581+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.583746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353713+0000","last_change":"2024-04-25T08:35:42.079469+0000","last_active":"2024-04-25T08:35:44.353713+0000","last_peered":"2024-04-25T08:35:44.353713+0000","last_clean":"2024-04-25T08:35:44.353713+0000","last_became_active":"2024-04-25T08:35:42.079187+0000","last_became_peered":"2024-04-25T08:35:42.079187+0000","last_unstale":"2024-04-25T08:35:44.353713+0000","last_undegraded":"2024-04-25T08:35:44.353713+0000","last_fullsized":"2024-04-25T08:35:44.353713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:26.436872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083965+0000","last_change":"2024-04-25T08:35:43.085978+0000","last_active":"2024-04-25T08:35:44.083965+0000","last_peered":"2024-04-25T08:35:44.083965+0000","last_clean":"2024-04-25T08:35:44.083965+0000","last_became_active":"2024-04-25T08:35:43.085695+0000","last_became_peered":"2024-04-25T08:35:43.085695+0000","last_unstale":"2024-04-25T08:35:44.083965+0000","last_undegraded":"2024-04-25T08:35:44.083965+0000","last_fullsized":"2024-04-25T08:35:44.083965+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:28.024790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"96'2","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347642+0000","last_change":"2024-04-25T08:35:42.079458+0000","last_active":"2024-04-25T08:35:44.347642+0000","last_peered":"2024-04-25T08:35:44.347642+0000","last_clean":"2024-04-25T08:35:44.347642+0000","last_became_active":"2024-04-25T08:35:42.079154+0000","last_became_peered":"2024-04-25T08:35:42.079154+0000","last_unstale":"2024-04-25T08:35:44.347642+0000","last_undegraded":"2024-04-25T08:35:44.347642+0000","last_fullsized":"2024-04-25T08:35:44.347642+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:49.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083967+0000","last_change":"2024-04-25T08:35:43.086850+0000","last_active":"2024-04-25T08:35:44.083967+0000","last_peered":"2024-04-25T08:35:44.083967+0000","last_clean":"2024-04-25T08:35:44.083967+0000","last_became_active":"2024-04-25T08:35:43.082016+0000","last_became_peered":"2024-04-25T08:35:43.082016+0000","last_unstale":"2024-04-25T08:35:44.083967+0000","last_undegraded":"2024-04-25T08:35:44.083967+0000","last_fullsized":"2024-04-25T08:35:44.083967+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:18.754062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"97'1","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088238+0000","last_change":"2024-04-25T08:35:42.080608+0000","last_active":"2024-04-25T08:35:44.088238+0000","last_peered":"2024-04-25T08:35:44.088238+0000","last_clean":"2024-04-25T08:35:44.088238+0000","last_became_active":"2024-04-25T08:35:42.080267+0000","last_became_peered":"2024-04-25T08:35:42.080267+0000","last_unstale":"2024-04-25T08:35:44.088238+0000","last_undegraded":"2024-04-25T08:35:44.088238+0000","last_fullsized":"2024-04-25T08:35:44.088238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:09.842643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083371+0000","last_change":"2024-04-25T08:35:43.084195+0000","last_active":"2024-04-25T08:35:44.083371+0000","last_peered":"2024-04-25T08:35:44.083371+0000","last_clean":"2024-04-25T08:35:44.083371+0000","last_became_active":"2024-04-25T08:35:43.081209+0000","last_became_peered":"2024-04-25T08:35:43.081209+0000","last_unstale":"2024-04-25T08:35:44.083371+0000","last_undegraded":"2024-04-25T08:35:44.083371+0000","last_fullsized":"2024-04-25T08:35:44.083371+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:31.817643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084421+0000","last_change":"2024-04-25T08:35:42.084974+0000","last_active":"2024-04-25T08:35:44.084421+0000","last_peered":"2024-04-25T08:35:44.084421+0000","last_clean":"2024-04-25T08:35:44.084421+0000","last_became_active":"2024-04-25T08:35:42.084703+0000","last_became_peered":"2024-04-25T08:35:42.084703+0000","last_unstale":"2024-04-25T08:35:44.084421+0000","last_undegraded":"2024-04-25T08:35:44.084421+0000","last_fullsized":"2024-04-25T08:35:44.084421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:56.731886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.089053+0000","last_change":"2024-04-25T08:35:43.091302+0000","last_active":"2024-04-25T08:35:44.089053+0000","last_peered":"2024-04-25T08:35:44.089053+0000","last_clean":"2024-04-25T08:35:44.089053+0000","last_became_active":"2024-04-25T08:35:43.091001+0000","last_became_peered":"2024-04-25T08:35:43.091001+0000","last_unstale":"2024-04-25T08:35:44.089053+0000","last_undegraded":"2024-04-25T08:35:44.089053+0000","last_fullsized":"2024-04-25T08:35:44.089053+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:32.012827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084172+0000","last_change":"2024-04-25T08:35:42.072420+0000","last_active":"2024-04-25T08:35:44.084172+0000","last_peered":"2024-04-25T08:35:44.084172+0000","last_clean":"2024-04-25T08:35:44.084172+0000","last_became_active":"2024-04-25T08:35:42.072142+0000","last_became_peered":"2024-04-25T08:35:42.072142+0000","last_unstale":"2024-04-25T08:35:44.084172+0000","last_undegraded":"2024-04-25T08:35:44.084172+0000","last_fullsized":"2024-04-25T08:35:44.084172+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:49.118684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.353624+0000","last_change":"2024-04-25T08:35:43.089060+0000","last_active":"2024-04-25T08:35:44.353624+0000","last_peered":"2024-04-25T08:35:44.353624+0000","last_clean":"2024-04-25T08:35:44.353624+0000","last_became_active":"2024-04-25T08:35:43.088777+0000","last_became_peered":"2024-04-25T08:35:43.088777+0000","last_unstale":"2024-04-25T08:35:44.353624+0000","last_undegraded":"2024-04-25T08:35:44.353624+0000","last_fullsized":"2024-04-25T08:35:44.353624+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:58.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083672+0000","last_change":"2024-04-25T08:35:42.085881+0000","last_active":"2024-04-25T08:35:44.083672+0000","last_peered":"2024-04-25T08:35:44.083672+0000","last_clean":"2024-04-25T08:35:44.083672+0000","last_became_active":"2024-04-25T08:35:42.085574+0000","last_became_peered":"2024-04-25T08:35:42.085574+0000","last_unstale":"2024-04-25T08:35:44.083672+0000","last_undegraded":"2024-04-25T08:35:44.083672+0000","last_fullsized":"2024-04-25T08:35:44.083672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:28.015864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354681+0000","last_change":"2024-04-25T08:35:43.080706+0000","last_active":"2024-04-25T08:35:44.354681+0000","last_peered":"2024-04-25T08:35:44.354681+0000","last_clean":"2024-04-25T08:35:44.354681+0000","last_became_active":"2024-04-25T08:35:43.080411+0000","last_became_peered":"2024-04-25T08:35:43.080411+0000","last_unstale":"2024-04-25T08:35:44.354681+0000","last_undegraded":"2024-04-25T08:35:44.354681+0000","last_fullsized":"2024-04-25T08:35:44.354681+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:50.604644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.088748+0000","last_change":"2024-04-25T08:35:42.082825+0000","last_active":"2024-04-25T08:35:44.088748+0000","last_peered":"2024-04-25T08:35:44.088748+0000","last_clean":"2024-04-25T08:35:44.088748+0000","last_became_active":"2024-04-25T08:35:42.082542+0000","last_became_peered":"2024-04-25T08:35:42.082542+0000","last_unstale":"2024-04-25T08:35:44.088748+0000","last_undegraded":"2024-04-25T08:35:44.088748+0000","last_fullsized":"2024-04-25T08:35:44.088748+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:56.984166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083712+0000","last_change":"2024-04-25T08:35:43.101917+0000","last_active":"2024-04-25T08:35:44.083712+0000","last_peered":"2024-04-25T08:35:44.083712+0000","last_clean":"2024-04-25T08:35:44.083712+0000","last_became_active":"2024-04-25T08:35:43.101697+0000","last_became_peered":"2024-04-25T08:35:43.101697+0000","last_unstale":"2024-04-25T08:35:44.083712+0000","last_undegraded":"2024-04-25T08:35:44.083712+0000","last_fullsized":"2024-04-25T08:35:44.083712+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:05.441456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.084467+0000","last_change":"2024-04-25T08:35:42.073922+0000","last_active":"2024-04-25T08:35:44.084467+0000","last_peered":"2024-04-25T08:35:44.084467+0000","last_clean":"2024-04-25T08:35:44.084467+0000","last_became_active":"2024-04-25T08:35:42.072926+0000","last_became_peered":"2024-04-25T08:35:42.072926+0000","last_unstale":"2024-04-25T08:35:44.084467+0000","last_undegraded":"2024-04-25T08:35:44.084467+0000","last_fullsized":"2024-04-25T08:35:44.084467+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:40.100311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.083687+0000","last_change":"2024-04-25T08:35:43.079481+0000","last_active":"2024-04-25T08:35:44.083687+0000","last_peered":"2024-04-25T08:35:44.083687+0000","last_clean":"2024-04-25T08:35:44.083687+0000","last_became_active":"2024-04-25T08:35:43.078668+0000","last_became_peered":"2024-04-25T08:35:43.078668+0000","last_unstale":"2024-04-25T08:35:44.083687+0000","last_undegraded":"2024-04-25T08:35:44.083687+0000","last_fullsized":"2024-04-25T08:35:44.083687+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:08.825229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347391+0000","last_change":"2024-04-25T08:35:42.082145+0000","last_active":"2024-04-25T08:35:44.347391+0000","last_peered":"2024-04-25T08:35:44.347391+0000","last_clean":"2024-04-25T08:35:44.347391+0000","last_became_active":"2024-04-25T08:35:42.081772+0000","last_became_peered":"2024-04-25T08:35:42.081772+0000","last_unstale":"2024-04-25T08:35:44.347391+0000","last_undegraded":"2024-04-25T08:35:44.347391+0000","last_fullsized":"2024-04-25T08:35:44.347391+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:18.561753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347652+0000","last_change":"2024-04-25T08:35:43.088971+0000","last_active":"2024-04-25T08:35:44.347652+0000","last_peered":"2024-04-25T08:35:44.347652+0000","last_clean":"2024-04-25T08:35:44.347652+0000","last_became_active":"2024-04-25T08:35:43.088662+0000","last_became_peered":"2024-04-25T08:35:43.088662+0000","last_unstale":"2024-04-25T08:35:44.347652+0000","last_undegraded":"2024-04-25T08:35:44.347652+0000","last_fullsized":"2024-04-25T08:35:44.347652+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:42.058754+0000","last_clean_scrub_stamp":"2024-04-25T08:35:42.058754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:58.269126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.354943+0000","last_change":"2024-04-25T08:35:42.080661+0000","last_active":"2024-04-25T08:35:44.354943+0000","last_peered":"2024-04-25T08:35:44.354943+0000","last_clean":"2024-04-25T08:35:44.354943+0000","last_became_active":"2024-04-25T08:35:42.080378+0000","last_became_peered":"2024-04-25T08:35:42.080378+0000","last_unstale":"2024-04-25T08:35:44.354943+0000","last_undegraded":"2024-04-25T08:35:44.354943+0000","last_fullsized":"2024-04-25T08:35:44.354943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:41.056371+0000","last_clean_scrub_stamp":"2024-04-25T08:35:41.056371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:11.863733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":315,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T08:35:44.347529+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:35:44.347529+0000","last_peered":"2024-04-25T08:35:44.347529+0000","last_clean":"2024-04-25T08:35:44.347529+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:35:44.347529+0000","last_undegraded":"2024-04-25T08:35:44.347529+0000","last_fullsized":"2024-04-25T08:35:44.347529+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542227,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41256,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330584,"statfs":{"total":96636764160,"available":96594518016,"internally_reserved":0,"allocated":1794048,"data_stored":1142381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58740,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313100,"statfs":{"total":96636764160,"available":96576614400,"internally_reserved":0,"allocated":1212416,"data_stored":554173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40672,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331168,"statfs":{"total":96636764160,"available":96595116032,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1200128,"data_stored":552117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50392,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321448,"statfs":{"total":96636764160,"available":96585162752,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54556,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317284,"statfs":{"total":96636764160,"available":96580898816,"internally_reserved":0,"allocated":1191936,"data_stored":551957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":552027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1810432,"data_stored":1144643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:50.837 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:35:50.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:35:50.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.986+0000 7f265e3c8640 1 Processor -- start 2024-04-25T08:35:50.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.986+0000 7f265e3c8640 1 -- start start 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.986+0000 7f265e3c8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f26581610f0 0x7f265806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f265e3c8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26580c6960 con 0x7f26581616f0 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f265e3c8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2658162260 con 0x7f26580ce7b0 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f265e3c8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26581614d0 con 0x7f26581610f0 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f26577fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f26581616f0 0x7f26580ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56342/0 (socket says 172.21.15.60:56342) 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f26577fe640 1 -- 172.21.15.60:0/742018314 learned_addr learned my addr 172.21.15.60:0/742018314 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2657fff640 1 --2- 172.21.15.60:0/742018314 >> v2:172.21.15.100:3300/0 conn(0x7f26581610f0 0x7f265806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2656ffd640 1 -- 172.21.15.60:0/742018314 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752223249 0 0) 0x7f26580c6960 con 0x7f26581616f0 2024-04-25T08:35:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2656ffd640 1 -- 172.21.15.60:0/742018314 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2640003520 con 0x7f26581616f0 2024-04-25T08:35:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2657fff640 1 -- 172.21.15.60:0/742018314 >> v1:172.21.15.100:6789/0 conn(0x7f26580ce7b0 legacy=0x7f26580d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2657fff640 1 -- 172.21.15.60:0/742018314 >> v1:172.21.15.60:6789/0 conn(0x7f26581616f0 legacy=0x7f26580ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2656ffd640 1 -- 172.21.15.60:0/742018314 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379068795 0 0) 0x7f2640003520 con 0x7f26581616f0 2024-04-25T08:35:50.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.987+0000 7f2657fff640 1 -- 172.21.15.60:0/742018314 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658162540 con 0x7f26581610f0 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.988+0000 7f2657fff640 1 --2- 172.21.15.60:0/742018314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f265806c1c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=993d9929d1312cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.988+0000 7f2656ffd640 1 -- 172.21.15.60:0/742018314 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648037070 con 0x7f26581610f0 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.988+0000 7f2656ffd640 1 -- 172.21.15.60:0/742018314 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264802ac50 con 0x7f26581610f0 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.988+0000 7f2656ffd640 1 -- 172.21.15.60:0/742018314 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c001670 con 0x7f26581610f0 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 -- 172.21.15.60:0/742018314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 msgr2=0x7f265806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 --2- 172.21.15.60:0/742018314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f265806c1c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f26480097a0 tx=0x7f264802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 -- 172.21.15.60:0/742018314 shutdown_connections 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 --2- 172.21.15.60:0/742018314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f265806c1c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 -- 172.21.15.60:0/742018314 >> 172.21.15.60:0/742018314 conn(0x7f265806f500 msgr2=0x7f265806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 -- 172.21.15.60:0/742018314 shutdown_connections 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.989+0000 7f265e3c8640 1 -- 172.21.15.60:0/742018314 wait complete. 2024-04-25T08:35:50.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.990+0000 7f265e3c8640 1 Processor -- start 2024-04-25T08:35:50.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.990+0000 7f265e3c8640 1 -- start start 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265e3c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26580ce7b0 0x7f265815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265e3c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f26580b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265e3c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 0x7f2658144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265e3c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26580cf690 con 0x7f26581616f0 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265e3c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26580cfce0 con 0x7f26581610f0 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265e3c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265806c700 con 0x7f26580ce7b0 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 0x7f2658144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 0x7f2658144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35846/0 (socket says 172.21.15.60:35846) 2024-04-25T08:35:50.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 -- 172.21.15.60:0/3534996077 learned_addr learned my addr 172.21.15.60:0/3534996077 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f26577fe640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f26580b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f2657fff640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26580ce7b0 0x7f265815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 -- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26580ce7b0 msgr2=0x7f265815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26580ce7b0 0x7f265815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 -- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 msgr2=0x7f26580b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.991+0000 7f265c965640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f26580b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f265c965640 1 -- 172.21.15.60:0/3534996077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265815e590 con 0x7f26581616f0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f265c965640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 0x7f2658144620 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f265000c730 tx=0x7f265000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650003b70 con 0x7f26581616f0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650003d10 con 0x7f26581616f0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650005310 con 0x7f26581616f0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265815e870 con 0x7f26581616f0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.992+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265815ee30 con 0x7f26581616f0 2024-04-25T08:35:50.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.993+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2658073a00 con 0x7f26581616f0 2024-04-25T08:35:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.993+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26500056c0 con 0x7f26581616f0 2024-04-25T08:35:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.994+0000 7f2654ff9640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f264406b4d0 0x7f264406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.994+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f265008a340 con 0x7f26581616f0 2024-04-25T08:35:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.994+0000 7f2657fff640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f264406b4d0 0x7f264406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.995+0000 7f2657fff640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f264406b4d0 0x7f264406d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2648034000 tx=0x7f264802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:50.998+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2650058940 con 0x7f26581616f0 2024-04-25T08:35:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.248+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f26580cf150 con 0x7f26581616f0 2024-04-25T08:35:51.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.250+0000 7f2654ff9640 1 -- 172.21.15.60:0/3534996077 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f265005c9e0 con 0x7f26581616f0 2024-04-25T08:35:51.245 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:35:51.245 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:35:51.245 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:35:51.245 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:35:51.245 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:35:51.245 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f264406b4d0 msgr2=0x7f264406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f264406b4d0 0x7f264406d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2648034000 tx=0x7f264802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 msgr2=0x7f2658144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 0x7f2658144620 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f265000c730 tx=0x7f265000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 shutdown_connections 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26581616f0 0x7f2658144620 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f264406b4d0 0x7f264406d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26581610f0 0x7f26580b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 --2- 172.21.15.60:0/3534996077 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f26580ce7b0 0x7f265815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 >> 172.21.15.60:0/3534996077 conn(0x7f265806f500 msgr2=0x7f265815c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 shutdown_connections 2024-04-25T08:35:51.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.252+0000 7f265e3c8640 1 -- 172.21.15.60:0/3534996077 wait complete. 2024-04-25T08:35:51.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.363+0000 7ff4d3ecb640 1 Processor -- start 2024-04-25T08:35:51.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.364+0000 7ff4d3ecb640 1 -- start start 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.364+0000 7ff4d3ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc15a790 0x7ff4cc0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.364+0000 7ff4d3ecb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4cc06fea0 0x7ff4cc0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.364+0000 7ff4d3ecb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc0b3710 con 0x7ff4cc0af9c0 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.364+0000 7ff4d3ecb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc0cf0e0 con 0x7ff4cc06fea0 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.364+0000 7ff4d3ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0cf3c0 con 0x7ff4cc15a790 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d1466640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4cc06fea0 0x7ff4cc0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d2468640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff4cc0af9c0 0x7ff4cc072300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56346/0 (socket says 172.21.15.60:56346) 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d2468640 1 -- 172.21.15.60:0/4180272646 learned_addr learned my addr 172.21.15.60:0/4180272646 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d1c67640 1 --2- 172.21.15.60:0/4180272646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc15a790 0x7ff4cc0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d1466640 1 -- 172.21.15.60:0/4180272646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc15a790 msgr2=0x7ff4cc0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d1466640 1 --2- 172.21.15.60:0/4180272646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc15a790 0x7ff4cc0d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d1466640 1 -- 172.21.15.60:0/4180272646 >> v1:172.21.15.60:6789/0 conn(0x7ff4cc0af9c0 legacy=0x7ff4cc072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d1466640 1 -- 172.21.15.60:0/4180272646 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0cf6a0 con 0x7ff4cc06fea0 2024-04-25T08:35:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.365+0000 7ff4d0c65640 1 -- 172.21.15.60:0/4180272646 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092061381 0 0) 0x7ff4cc0b3710 con 0x7ff4cc0af9c0 2024-04-25T08:35:51.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d1466640 1 --2- 172.21.15.60:0/4180272646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc0d08d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097a0 tx=0x7ff4bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab76b1874d5df02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d0c65640 1 -- 172.21.15.60:0/4180272646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc037070 con 0x7ff4cc06fea0 2024-04-25T08:35:51.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d0c65640 1 -- 172.21.15.60:0/4180272646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc032440 con 0x7ff4cc06fea0 2024-04-25T08:35:51.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d0c65640 1 -- 172.21.15.60:0/4180272646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc032810 con 0x7ff4cc06fea0 2024-04-25T08:35:51.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4180272646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 msgr2=0x7ff4cc0d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4180272646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc0d08d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097a0 tx=0x7ff4bc02a180 comp rx=0 tx=0).stop 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4180272646 shutdown_connections 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4180272646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc0d08d0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4180272646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc15a790 0x7ff4cc0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4180272646 >> 172.21.15.60:0/4180272646 conn(0x7ff4cc0b2370 msgr2=0x7ff4cc0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4180272646 shutdown_connections 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.366+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4180272646 wait complete. 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 Processor -- start 2024-04-25T08:35:51.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 -- start start 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc13e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc0af9c0 0x7ff4cc13ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4cc15a790 0x7ff4cc1413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc15ab70 con 0x7ff4cc06fea0 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc0cf980 con 0x7ff4cc15a790 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d3ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc15acf0 con 0x7ff4cc0af9c0 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d1c67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc13e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d1c67640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc13e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35872/0 (socket says 172.21.15.60:35872) 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.368+0000 7ff4d1c67640 1 -- 172.21.15.60:0/4268898686 learned_addr learned my addr 172.21.15.60:0/4268898686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d2468640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4cc15a790 0x7ff4cc1413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1466640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc0af9c0 0x7ff4cc13ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1c67640 1 -- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc0af9c0 msgr2=0x7ff4cc13ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1c67640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc0af9c0 0x7ff4cc13ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1c67640 1 -- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4cc15a790 msgr2=0x7ff4cc1413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1c67640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4cc15a790 0x7ff4cc1413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1c67640 1 -- 172.21.15.60:0/4268898686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0ae1f0 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1466640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc0af9c0 0x7ff4cc13ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d1c67640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc13e7a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800d800 tx=0x7ff4c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800b6c0 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c8019070 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc0adf10 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800ba80 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.369+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc13f770 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.370+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc152790 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.371+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4c800bc20 con 0x7ff4cc06fea0 2024-04-25T08:35:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.371+0000 7ff4c27fc640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4b006b500 0x7ff4b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.371+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4c808a760 con 0x7ff4cc06fea0 2024-04-25T08:35:51.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.371+0000 7ff4d1466640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4b006b500 0x7ff4b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.372+0000 7ff4d1466640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4b006b500 0x7ff4b006d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc034000 tx=0x7ff4bc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.375+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4cc152790 con 0x7ff4cc06fea0 2024-04-25T08:35:51.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.592+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4cc073990 con 0x7ff4cc06fea0 2024-04-25T08:35:51.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.593+0000 7ff4c27fc640 1 -- 172.21.15.60:0/4268898686 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff4c801c360 con 0x7ff4cc06fea0 2024-04-25T08:35:51.588 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:35:51.588 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:35:51.588 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4b006b500 msgr2=0x7ff4b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4b006b500 0x7ff4b006d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc034000 tx=0x7ff4bc02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 msgr2=0x7ff4cc13e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc13e7a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800d800 tx=0x7ff4c800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 shutdown_connections 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4b006b500 0x7ff4b006d9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4cc15a790 0x7ff4cc1413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4cc0af9c0 0x7ff4cc13ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 --2- 172.21.15.60:0/4268898686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4cc06fea0 0x7ff4cc13e7a0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 >> 172.21.15.60:0/4268898686 conn(0x7ff4cc0b2370 msgr2=0x7ff4cc159170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 shutdown_connections 2024-04-25T08:35:51.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:51.595+0000 7ff4d3ecb640 1 -- 172.21.15.60:0/4268898686 wait complete. 2024-04-25T08:35:52.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:35:42.436804+0000', 'modified': '2024-04-25T08:35:44.434970+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7626}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7626': {'gid': 7626, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/3135665870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 3135665870}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 3135665870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:35:52.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:35:52.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:35:52.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 Processor -- start 2024-04-25T08:35:52.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 -- start start 2024-04-25T08:35:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd8c815dc20 0x7fd8c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8c8161710 0x7fd8c80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c815e000 con 0x7fd8c8161710 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c80d1de0 con 0x7fd8c815dc20 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.117+0000 7fd8cf022640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8161c60 con 0x7fd8c80b9330 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8c8161710 0x7fd8c80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd8c8161710 0x7fd8c80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45596/0 (socket says 172.21.15.60:45596) 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 -- 172.21.15.60:0/821089502 learned_addr learned my addr 172.21.15.60:0/821089502 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/821089502 >> v2:172.21.15.100:3300/0 conn(0x7fd8c815dc20 0x7fd8c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 -- 172.21.15.60:0/821089502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 msgr2=0x7fd8c806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c806bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 -- 172.21.15.60:0/821089502 >> v2:172.21.15.100:3300/0 conn(0x7fd8c815dc20 msgr2=0x7fd8c80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/821089502 >> v2:172.21.15.100:3300/0 conn(0x7fd8c815dc20 0x7fd8c80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8c7fff640 1 -- 172.21.15.60:0/821089502 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8161f40 con 0x7fd8c8161710 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/821089502 >> v2:172.21.15.100:3300/0 conn(0x7fd8c815dc20 0x7fd8c80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.118+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c8161710 0x7fd8c80cebb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80d1520 tx=0x7fd8b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90acd52b430d4105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8c77fe640 1 -- 172.21.15.60:0/821089502 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4037070 con 0x7fd8c8161710 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8c77fe640 1 -- 172.21.15.60:0/821089502 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4032440 con 0x7fd8c8161710 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8c77fe640 1 -- 172.21.15.60:0/821089502 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4032810 con 0x7fd8c8161710 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 -- 172.21.15.60:0/821089502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c8161710 msgr2=0x7fd8c80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c8161710 0x7fd8c80cebb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80d1520 tx=0x7fd8b402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 -- 172.21.15.60:0/821089502 shutdown_connections 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 --2- 172.21.15.60:0/821089502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c8161710 0x7fd8c80cebb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 --2- 172.21.15.60:0/821089502 >> v2:172.21.15.100:3300/0 conn(0x7fd8c815dc20 0x7fd8c80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 -- 172.21.15.60:0/821089502 >> 172.21.15.60:0/821089502 conn(0x7fd8c80d0420 msgr2=0x7fd8c80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 -- 172.21.15.60:0/821089502 shutdown_connections 2024-04-25T08:35:52.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.119+0000 7fd8cf022640 1 -- 172.21.15.60:0/821089502 wait complete. 2024-04-25T08:35:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.120+0000 7fd8cf022640 1 Processor -- start 2024-04-25T08:35:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 -- start start 2024-04-25T08:35:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c81519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 0x7fd8c80cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161710 0x7fd8c80b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80b5e50 con 0x7fd8c815dc20 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8162500 con 0x7fd8c8161710 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8cf022640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80b5fd0 con 0x7fd8c80b9330 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.121+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 0x7fd8c80cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 0x7fd8c80cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45600/0 (socket says 172.21.15.60:45600) 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 -- 172.21.15.60:0/4051872566 learned_addr learned my addr 172.21.15.60:0/4051872566 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c81519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161710 0x7fd8c80b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 -- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 msgr2=0x7fd8c81519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c81519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 -- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161710 msgr2=0x7fd8c80b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161710 0x7fd8c80b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 -- 172.21.15.60:0/4051872566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8152490 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161710 0x7fd8c80b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 0x7fd8c80cb4a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd8b402a220 tx=0x7fd8b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c57fa640 1 -- 172.21.15.60:0/4051872566 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4037040 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c57fa640 1 -- 172.21.15.60:0/4051872566 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4042070 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c8151f10 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8c57fa640 1 -- 172.21.15.60:0/4051872566 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4030bb0 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.122+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c8160e70 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.124+0000 7fd8c57fa640 1 -- 172.21.15.60:0/4051872566 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8b4047460 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.124+0000 7fd8c57fa640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8ac06b4d0 0x7fd8ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.124+0000 7fd8c57fa640 1 -- 172.21.15.60:0/4051872566 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8b40aaeb0 con 0x7fd8c815dc20 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.125+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8ac06b4d0 0x7fd8ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.125+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8ac06b4d0 0x7fd8ac06d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00aac0 tx=0x7fd8bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.125+0000 7fd8d1dfd740 1 Processor -- start 2024-04-25T08:35:52.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.125+0000 7fd8d1dfd740 1 -- start start 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f381d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37d4f10 0x5558f381b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558f3737a10 con 0x5558f381cad0 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558f36e0b30 con 0x5558f37ff230 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558f3733c50 con 0x5558f37d4f10 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f381d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f381d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45616/0 (socket says 172.21.15.60:45616) 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 -- 172.21.15.60:0/3043859037 learned_addr learned my addr 172.21.15.60:0/3043859037 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37d4f10 0x5558f381b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 -- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37d4f10 msgr2=0x5558f381b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37d4f10 0x5558f381b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 -- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 msgr2=0x5558f37ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 -- 172.21.15.60:0/3043859037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558f36f5e10 con 0x5558f381cad0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.126+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f381d020 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40a9280 tx=0x7fd8b40791a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb82fabb7a5ed94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8b27fc640 1 -- 172.21.15.60:0/3043859037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b407d680 con 0x5558f381cad0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8b27fc640 1 -- 172.21.15.60:0/3043859037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4081c40 con 0x5558f381cad0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8b27fc640 1 -- 172.21.15.60:0/3043859037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4030d50 con 0x5558f381cad0 2024-04-25T08:35:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/3043859037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 msgr2=0x5558f381d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f381d020 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40a9280 tx=0x7fd8b40791a0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/3043859037 shutdown_connections 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37d4f10 0x5558f381b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f381d020 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/3043859037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/3043859037 >> 172.21.15.60:0/3043859037 conn(0x5558f37d1840 msgr2=0x5558f381a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/3043859037 shutdown_connections 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.127+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/3043859037 wait complete. 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.128+0000 7fd8d1dfd740 1 -- start start 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.128+0000 7fd8d1dfd740 1 Processor -- start 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f38110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3811630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558f36e0b30 con 0x5558f381cad0 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558f36f40c0 con 0x5558f37ff230 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558f3737a10 con 0x5558f37ec5d0 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8d1dfd740 10 client.0 _ll_register_callbacks cb 0x5558f36e7ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f38110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f38110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45618/0 (socket says 172.21.15.60:45618) 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 -- 172.21.15.60:0/218683845 learned_addr learned my addr 172.21.15.60:0/218683845 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3811630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 -- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 msgr2=0x5558f3811630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3811630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 -- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 msgr2=0x5558f37eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 -- 172.21.15.60:0/218683845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558f344c2d0 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f38110f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40791e0 tx=0x7fd8b4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69afdd910b38f391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.129+0000 7fd8a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8a77fe640 1 -- 172.21.15.60:0/218683845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b407d680 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8b4081c40 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8a77fe640 1 -- 172.21.15.60:0/218683845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8b4095c50 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8a77fe640 1 -- 172.21.15.60:0/218683845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b4095e50 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 10 client.7812 fetch_fsmap learned FSMap version 55 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5558f35166b0 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8a77fe640 1 -- 172.21.15.60:0/218683845 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4081c40 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8a77fe640 1 -- 172.21.15.60:0/218683845 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8b403a030 con 0x5558f381cad0 2024-04-25T08:35:52.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 10 client.7812 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 10 client.7812 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 20 client.7812 populate_metadata read hostname 'smithi060' 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f3801690 0x5558f3801a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 4 client.7812 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558f3796920 con 0x5558f3801690 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.130+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f3801690 0x5558f3801a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8a77fe640 10 client.7812 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f3801690 0x5558f3801a70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39ecfa7702ff21 server_cookie=26513e5ea6670d6b in_seq=0 out_seq=0 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8a77fe640 1 -- 172.21.15.60:0/218683845 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558f3796920 con 0x5558f3801690 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8a77fe640 10 client.7812 handle_command_reply: tid=0 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 client.7812 shutdown 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 msgr2=0x5558f38110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f38110f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40791e0 tx=0x7fd8b4009240 comp rx=0 tx=0).stop 2024-04-25T08:35:52.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 shutdown_connections 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f3801690 0x5558f3801a70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3811630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f38110f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8a77fe640 0 client.7812 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/218683845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f37eaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.131+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 >> 172.21.15.60:0/218683845 conn(0x5558f37d4f10 msgr2=0x5558f37fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.132+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 shutdown_connections 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.132+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/218683845 wait complete. 2024-04-25T08:35:52.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.132+0000 7fd8d1dfd740 20 client.7812 trim_cache size 0 max 16384 2024-04-25T08:35:52.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.147+0000 7fd8d1dfd740 1 Processor -- start 2024-04-25T08:35:52.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.147+0000 7fd8d1dfd740 1 -- start start 2024-04-25T08:35:52.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 0x5558f381a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f38330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f381cad0 0x5558f3833620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81ff630 con 0x5558f37ec5d0 2024-04-25T08:35:52.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c815eb20 con 0x5558f37ff230 2024-04-25T08:35:52.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81ff190 con 0x5558f381cad0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8ccdbe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 0x5558f381a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8ccdbe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 0x5558f381a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45626/0 (socket says 172.21.15.60:45626) 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8ccdbe640 1 -- 172.21.15.60:0/2967923540 learned_addr learned my addr 172.21.15.60:0/2967923540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f381cad0 0x5558f3833620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.148+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f38330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8ccdbe640 1 -- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f381cad0 msgr2=0x5558f3833620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f381cad0 0x5558f3833620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8ccdbe640 1 -- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 msgr2=0x5558f38330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f38330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8ccdbe640 1 -- 172.21.15.60:0/2967923540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558f3801be0 con 0x5558f37ec5d0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 0x5558f381a4f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00a980 tx=0x7fd8bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd64339e459e2dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8b1ffb640 1 -- 172.21.15.60:0/2967923540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc015e30 con 0x5558f37ec5d0 2024-04-25T08:35:52.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8b1ffb640 1 -- 172.21.15.60:0/2967923540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc02e070 con 0x5558f37ec5d0 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8b1ffb640 1 -- 172.21.15.60:0/2967923540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc028530 con 0x5558f37ec5d0 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/2967923540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 msgr2=0x5558f381a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 0x5558f381a4f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00a980 tx=0x7fd8bc016000 comp rx=0 tx=0).stop 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/2967923540 shutdown_connections 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f381cad0 0x5558f3833620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f38330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/2967923540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f37ec5d0 0x5558f381a4f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.149+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/2967923540 >> 172.21.15.60:0/2967923540 conn(0x5558f37d4f10 msgr2=0x5558f37fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.150+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/2967923540 shutdown_connections 2024-04-25T08:35:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.150+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/2967923540 wait complete. 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.150+0000 7fd8d1dfd740 1 -- start start 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.150+0000 7fd8d1dfd740 1 Processor -- start 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3802580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f3802ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f37e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558f36e0b30 con 0x5558f381cad0 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558f36f40c0 con 0x5558f37ff230 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558f37ebaf0 con 0x5558f37ec5d0 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8d1dfd740 10 client.0 _ll_register_callbacks cb 0x5558f37cfc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f37e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f37e8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45628/0 (socket says 172.21.15.60:45628) 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 -- 172.21.15.60:0/1437745245 learned_addr learned my addr 172.21.15.60:0/1437745245 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3802580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8c7fff640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f3802ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 -- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 msgr2=0x5558f3802580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3802580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 -- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 msgr2=0x5558f3802ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f3802ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.151+0000 7fd8cd5bf640 1 -- 172.21.15.60:0/1437745245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558f381bf70 con 0x5558f381cad0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8cd5bf640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f37e8f10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0010f10 tx=0x7fd8c000e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97696af2b635aa19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8a77fe640 1 -- 172.21.15.60:0/1437745245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0003da0 con 0x5558f381cad0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8a77fe640 1 -- 172.21.15.60:0/1437745245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8c0004030 con 0x5558f381cad0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8bc02e070 con 0x5558f381cad0 2024-04-25T08:35:52.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8a77fe640 1 -- 172.21.15.60:0/1437745245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8c0027050 con 0x5558f381cad0 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 10 client.7818 fetch_fsmap learned FSMap version 55 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5558f3833eb0 con 0x5558f381cad0 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8a77fe640 1 -- 172.21.15.60:0/1437745245 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc02e070 con 0x5558f381cad0 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8a77fe640 1 -- 172.21.15.60:0/1437745245 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8c002a020 con 0x5558f381cad0 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 10 client.7818 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 10 client.7818 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 20 client.7818 populate_metadata read hostname 'smithi060' 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f381ad50 0x5558f381b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 4 client.7818 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5558f37eb8a0 con 0x5558f381ad50 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.152+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f381ad50 0x5558f381b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8a77fe640 10 client.7818 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8ccdbe640 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f381ad50 0x5558f381b130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3937da4422cb91 server_cookie=93fe739bfcb8a271 in_seq=0 out_seq=0 2024-04-25T08:35:52.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8a77fe640 1 -- 172.21.15.60:0/1437745245 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5558f37eb8a0 con 0x5558f381ad50 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8a77fe640 10 client.7818 handle_command_reply: tid=0 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 client.7818 shutdown 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 msgr2=0x5558f37e8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f37e8f10 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0010f10 tx=0x7fd8c000e7b0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 shutdown_connections 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5558f381ad50 0x5558f381b130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5558f381cad0 0x5558f37e8f10 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558f37ff230 0x5558f3802ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8a77fe640 0 client.7818 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 --2- 172.21.15.60:0/1437745245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5558f37ec5d0 0x5558f3802580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 >> 172.21.15.60:0/1437745245 conn(0x5558f37d4f10 msgr2=0x5558f37fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 shutdown_connections 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 1 -- 172.21.15.60:0/1437745245 wait complete. 2024-04-25T08:35:52.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.153+0000 7fd8d1dfd740 20 client.7818 trim_cache size 0 max 16384 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 7626, 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.0729758839999999, 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 55, 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 97, 2024-04-25T08:35:52.155 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 97, 2024-04-25T08:35:52.156 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.422664436 2024-04-25T08:35:52.156 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:35:52.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8ac06b4d0 msgr2=0x7fd8ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8ac06b4d0 0x7fd8ac06d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00aac0 tx=0x7fd8bc008040 comp rx=0 tx=0).stop 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 msgr2=0x7fd8c80cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 0x7fd8c80cb4a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd8b402a220 tx=0x7fd8b40046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 shutdown_connections 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd8ac06b4d0 0x7fd8ac06d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c8161710 0x7fd8c80b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8c815dc20 0x7fd8c80cb4a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 --2- 172.21.15.60:0/4051872566 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8c80b9330 0x7fd8c81519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 >> 172.21.15.60:0/4051872566 conn(0x7fd8c80d0420 msgr2=0x7fd8c80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 shutdown_connections 2024-04-25T08:35:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.154+0000 7fd8cf022640 1 -- 172.21.15.60:0/4051872566 wait complete. 2024-04-25T08:35:52.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:35:52.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.266+0000 7f4001fd5640 1 Processor -- start 2024-04-25T08:35:52.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.266+0000 7f4001fd5640 1 -- start start 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.266+0000 7f4001fd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ffc15da70 0x7f3ffc161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.266+0000 7f4001fd5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc162020 con 0x7f3ffc161b80 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.266+0000 7f4001fd5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ffc162300 con 0x7f3ffc0b9730 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.266+0000 7f4001fd5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ffc1625e0 con 0x7f3ffc15da70 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3ffc15da70 0x7f3ffc161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffbfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3ffc161b80 0x7f3ffc0d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49134/0 (socket says 172.21.15.60:49134) 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffbfff640 1 -- 172.21.15.60:0/2240861511 learned_addr learned my addr 172.21.15.60:0/2240861511 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffb7fe640 1 -- 172.21.15.60:0/2240861511 >> v1:172.21.15.100:6789/0 conn(0x7f3ffc0b9730 legacy=0x7f3ffc0ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffb7fe640 1 -- 172.21.15.60:0/2240861511 >> v1:172.21.15.60:6789/0 conn(0x7f3ffc161b80 legacy=0x7f3ffc0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffa7fc640 1 -- 172.21.15.60:0/2240861511 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591708158 0 0) 0x7f3ffc162020 con 0x7f3ffc161b80 2024-04-25T08:35:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.267+0000 7f3ffb7fe640 1 -- 172.21.15.60:0/2240861511 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0d1af0 con 0x7f3ffc15da70 2024-04-25T08:35:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f3ffb7fe640 1 --2- 172.21.15.60:0/2240861511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc15da70 0x7f3ffc161640 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0097a0 tx=0x7f3fec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97a7fb14cc48fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f3ffa7fc640 1 -- 172.21.15.60:0/2240861511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec037070 con 0x7f3ffc15da70 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f3ffa7fc640 1 -- 172.21.15.60:0/2240861511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec02ae50 con 0x7f3ffc15da70 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f3ffa7fc640 1 -- 172.21.15.60:0/2240861511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec032660 con 0x7f3ffc15da70 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 -- 172.21.15.60:0/2240861511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc15da70 msgr2=0x7f3ffc161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 --2- 172.21.15.60:0/2240861511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc15da70 0x7f3ffc161640 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0097a0 tx=0x7f3fec02a140 comp rx=0 tx=0).stop 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 -- 172.21.15.60:0/2240861511 shutdown_connections 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 --2- 172.21.15.60:0/2240861511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc15da70 0x7f3ffc161640 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 -- 172.21.15.60:0/2240861511 >> 172.21.15.60:0/2240861511 conn(0x7f3ffc061a20 msgr2=0x7f3ffc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 -- 172.21.15.60:0/2240861511 shutdown_connections 2024-04-25T08:35:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.268+0000 7f4001fd5640 1 -- 172.21.15.60:0/2240861511 wait complete. 2024-04-25T08:35:52.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.269+0000 7f4001fd5640 1 Processor -- start 2024-04-25T08:35:52.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 -- start start 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ffc15da70 0x7f3ffc160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 0x7f3ffc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ffc0c9cb0 0x7f3ffc1e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0ced60 con 0x7f3ffc161b80 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc15de50 con 0x7f3ffc15da70 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.270+0000 7f4001fd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc0ceee0 con 0x7f3ffc0c9cb0 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 0x7f3ffc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 0x7f3ffc0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45652/0 (socket says 172.21.15.60:45652) 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 -- 172.21.15.60:0/934433460 learned_addr learned my addr 172.21.15.60:0/934433460 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffb7fe640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ffc15da70 0x7f3ffc160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffbfff640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ffc0c9cb0 0x7f3ffc1e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 -- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ffc0c9cb0 msgr2=0x7f3ffc1e8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ffc0c9cb0 0x7f3ffc1e8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 -- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ffc15da70 msgr2=0x7f3ffc160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ffc15da70 0x7f3ffc160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 -- 172.21.15.60:0/934433460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0c8280 con 0x7f3ffc161b80 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffb7fe640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ffc15da70 0x7f3ffc160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ffaffd640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 0x7f3ffc0c9770 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c430 tx=0x7f3fe400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011790 con 0x7f3ffc161b80 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe4017070 con 0x7f3ffc161b80 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4011b70 con 0x7f3ffc161b80 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc0c8560 con 0x7f3ffc161b80 2024-04-25T08:35:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.271+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc0c8b20 con 0x7f3ffc161b80 2024-04-25T08:35:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.272+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc073990 con 0x7f3ffc161b80 2024-04-25T08:35:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.273+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fe4011d10 con 0x7f3ffc161b80 2024-04-25T08:35:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.273+0000 7f3ff3fff640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fdc06b4d0 0x7f3fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.273+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3fe408a1c0 con 0x7f3ffc161b80 2024-04-25T08:35:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.273+0000 7f3ffb7fe640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fdc06b4d0 0x7f3fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.274+0000 7f3ffb7fe640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fdc06b4d0 0x7f3fdc06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3fec034000 tx=0x7f3fec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:52.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.277+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fe4058f40 con 0x7f3ffc161b80 2024-04-25T08:35:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.496+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ffc0b8a80 con 0x7f3ffc161b80 2024-04-25T08:35:52.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.497+0000 7f3ff3fff640 1 -- 172.21.15.60:0/934433460 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3fe4058840 con 0x7f3ffc161b80 2024-04-25T08:35:52.492 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:35:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.498+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fdc06b4d0 msgr2=0x7f3fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.498+0000 7f4001fd5640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fdc06b4d0 0x7f3fdc06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3fec034000 tx=0x7f3fec02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.498+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 msgr2=0x7f3ffc0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 0x7f3ffc0c9770 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c430 tx=0x7f3fe400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 shutdown_connections 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ffc0c9cb0 0x7f3ffc1e8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ffc161b80 0x7f3ffc0c9770 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fdc06b4d0 0x7f3fdc06d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 --2- 172.21.15.60:0/934433460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ffc15da70 0x7f3ffc160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 >> 172.21.15.60:0/934433460 conn(0x7f3ffc061a20 msgr2=0x7f3ffc0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 shutdown_connections 2024-04-25T08:35:52.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:35:52.499+0000 7f4001fd5640 1 -- 172.21.15.60:0/934433460 wait complete. 2024-04-25T08:35:52.504 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:35:52.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:35:52.504 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:35:52.504 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:35:52.504 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:52.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:35:52.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:35:52.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:35:52.505 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:52.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.505 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:35:52.520 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:35:52.521 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42358sec preferred_lft 42358sec 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:35:52.522 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:52.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.523 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:35:52.584 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:35:52.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.586 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:35:52.642 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:35:52.642 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:35:52.642 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:35:52.642 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:35:52.642 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:35:52.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.643 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:35:52.643 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:52.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:35:52.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:35:52.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:35:52.643 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:52.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.728 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:35:52.743 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:35:52.743 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:35:52.744 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:52.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:35:52.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:35:52.808 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:52.862 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:52.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:35:52.863 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:52.917 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:35:52.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:52.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:35:52.999 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:35:52.999 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:35:52.999 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:35:53.008 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:35:53.008 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:35:53.008 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:35:53.008 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:35:53.008 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:35:53.014 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:35:53.015 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:35:53.015 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:35:53.015 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:35:53.015 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:35:53.015 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:35:53.016 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:35:53.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:53.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:53.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:53.081 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:35:53.142 INFO:teuthology.orchestra.run.smithi060.stdout:client6314 2024-04-25T08:35:53.142 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:35:53.142 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:53.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:53.198 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:53.198 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:53.198 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:35:53.198 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:53.198 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:53.198 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:53.199 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.270 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797407 s, 131 MB/s 2024-04-25T08:35:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.279 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849013 s, 124 MB/s 2024-04-25T08:35:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.288 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769031 s, 136 MB/s 2024-04-25T08:35:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862008 s, 122 MB/s 2024-04-25T08:35:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716223 s, 146 MB/s 2024-04-25T08:35:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.313 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578717 s, 181 MB/s 2024-04-25T08:35:53.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712357 s, 147 MB/s 2024-04-25T08:35:53.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580761 s, 181 MB/s 2024-04-25T08:35:53.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.336 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732087 s, 143 MB/s 2024-04-25T08:35:53.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.345 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843187 s, 124 MB/s 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674338 s, 155 MB/s 2024-04-25T08:35:53.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.360 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716155 s, 146 MB/s 2024-04-25T08:35:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.368 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679719 s, 154 MB/s 2024-04-25T08:35:53.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593843 s, 177 MB/s 2024-04-25T08:35:53.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662832 s, 158 MB/s 2024-04-25T08:35:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.391 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074615 s, 141 MB/s 2024-04-25T08:35:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.398 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067096 s, 156 MB/s 2024-04-25T08:35:53.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.406 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652751 s, 161 MB/s 2024-04-25T08:35:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669839 s, 157 MB/s 2024-04-25T08:35:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.421 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637898 s, 164 MB/s 2024-04-25T08:35:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.428 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634516 s, 165 MB/s 2024-04-25T08:35:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.436 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727486 s, 144 MB/s 2024-04-25T08:35:53.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.443 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578781 s, 181 MB/s 2024-04-25T08:35:53.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759767 s, 138 MB/s 2024-04-25T08:35:53.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.458 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055901 s, 188 MB/s 2024-04-25T08:35:53.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.465 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588245 s, 178 MB/s 2024-04-25T08:35:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.472 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663076 s, 158 MB/s 2024-04-25T08:35:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.480 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637748 s, 164 MB/s 2024-04-25T08:35:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785164 s, 134 MB/s 2024-04-25T08:35:53.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565381 s, 185 MB/s 2024-04-25T08:35:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.502 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656675 s, 160 MB/s 2024-04-25T08:35:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.509 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583615 s, 180 MB/s 2024-04-25T08:35:53.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:53.510 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:53.510 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:53.510 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:35:53.510 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:53.511 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:53.511 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:53.511 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:53.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.577 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750552 s, 140 MB/s 2024-04-25T08:35:53.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056963 s, 184 MB/s 2024-04-25T08:35:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656129 s, 160 MB/s 2024-04-25T08:35:53.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595259 s, 176 MB/s 2024-04-25T08:35:53.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734784 s, 143 MB/s 2024-04-25T08:35:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626866 s, 167 MB/s 2024-04-25T08:35:53.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695009 s, 151 MB/s 2024-04-25T08:35:53.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.626 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566592 s, 185 MB/s 2024-04-25T08:35:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.633 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575477 s, 182 MB/s 2024-04-25T08:35:53.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.641 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641013 s, 164 MB/s 2024-04-25T08:35:53.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.649 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721842 s, 145 MB/s 2024-04-25T08:35:53.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628698 s, 167 MB/s 2024-04-25T08:35:53.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.664 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714489 s, 147 MB/s 2024-04-25T08:35:53.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578735 s, 181 MB/s 2024-04-25T08:35:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077175 s, 136 MB/s 2024-04-25T08:35:53.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623521 s, 168 MB/s 2024-04-25T08:35:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645766 s, 162 MB/s 2024-04-25T08:35:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.700 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568329 s, 185 MB/s 2024-04-25T08:35:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.708 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701276 s, 150 MB/s 2024-04-25T08:35:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578413 s, 181 MB/s 2024-04-25T08:35:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.722 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629588 s, 167 MB/s 2024-04-25T08:35:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688269 s, 152 MB/s 2024-04-25T08:35:53.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056881 s, 184 MB/s 2024-04-25T08:35:53.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700169 s, 150 MB/s 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682528 s, 154 MB/s 2024-04-25T08:35:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710592 s, 148 MB/s 2024-04-25T08:35:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615048 s, 170 MB/s 2024-04-25T08:35:53.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.774 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561914 s, 187 MB/s 2024-04-25T08:35:53.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584954 s, 179 MB/s 2024-04-25T08:35:53.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.787 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570111 s, 184 MB/s 2024-04-25T08:35:53.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731604 s, 143 MB/s 2024-04-25T08:35:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.804 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706301 s, 148 MB/s 2024-04-25T08:35:53.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:53.805 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:53.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559264 s, 187 MB/s 2024-04-25T08:35:53.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.877 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655094 s, 160 MB/s 2024-04-25T08:35:53.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618235 s, 170 MB/s 2024-04-25T08:35:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551236 s, 190 MB/s 2024-04-25T08:35:53.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.897 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584688 s, 179 MB/s 2024-04-25T08:35:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562566 s, 186 MB/s 2024-04-25T08:35:53.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737559 s, 142 MB/s 2024-04-25T08:35:53.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565359 s, 185 MB/s 2024-04-25T08:35:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.926 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588864 s, 178 MB/s 2024-04-25T08:35:53.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.933 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060987 s, 172 MB/s 2024-04-25T08:35:53.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553093 s, 190 MB/s 2024-04-25T08:35:53.945 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.946 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575335 s, 182 MB/s 2024-04-25T08:35:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.954 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758974 s, 138 MB/s 2024-04-25T08:35:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.961 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566744 s, 185 MB/s 2024-04-25T08:35:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.969 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702564 s, 149 MB/s 2024-04-25T08:35:53.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684199 s, 153 MB/s 2024-04-25T08:35:53.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.983 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579774 s, 181 MB/s 2024-04-25T08:35:53.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.991 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699458 s, 150 MB/s 2024-04-25T08:35:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:53.998 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561114 s, 187 MB/s 2024-04-25T08:35:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556662 s, 188 MB/s 2024-04-25T08:35:54.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.011 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575869 s, 182 MB/s 2024-04-25T08:35:54.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.019 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716076 s, 146 MB/s 2024-04-25T08:35:54.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596235 s, 176 MB/s 2024-04-25T08:35:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629788 s, 166 MB/s 2024-04-25T08:35:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722729 s, 145 MB/s 2024-04-25T08:35:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.048 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574226 s, 183 MB/s 2024-04-25T08:35:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555138 s, 189 MB/s 2024-04-25T08:35:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.063 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695039 s, 151 MB/s 2024-04-25T08:35:54.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.069 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585938 s, 179 MB/s 2024-04-25T08:35:54.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.077 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701563 s, 149 MB/s 2024-04-25T08:35:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689499 s, 152 MB/s 2024-04-25T08:35:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549981 s, 191 MB/s 2024-04-25T08:35:54.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:54.092 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:54.092 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:54.092 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:35:54.093 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:54.093 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:54.093 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:54.093 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:54.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.157 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624808 s, 168 MB/s 2024-04-25T08:35:54.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.164 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570971 s, 184 MB/s 2024-04-25T08:35:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546281 s, 192 MB/s 2024-04-25T08:35:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.177 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597463 s, 176 MB/s 2024-04-25T08:35:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.185 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698739 s, 150 MB/s 2024-04-25T08:35:54.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.191 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558615 s, 188 MB/s 2024-04-25T08:35:54.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.199 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703064 s, 149 MB/s 2024-04-25T08:35:54.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659677 s, 159 MB/s 2024-04-25T08:35:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.215 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643098 s, 163 MB/s 2024-04-25T08:35:54.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606481 s, 173 MB/s 2024-04-25T08:35:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583363 s, 180 MB/s 2024-04-25T08:35:54.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616701 s, 170 MB/s 2024-04-25T08:35:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.242 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561046 s, 187 MB/s 2024-04-25T08:35:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.250 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666904 s, 157 MB/s 2024-04-25T08:35:54.256 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.256 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.256 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605542 s, 173 MB/s 2024-04-25T08:35:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.263 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063238 s, 166 MB/s 2024-04-25T08:35:54.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.272 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776394 s, 135 MB/s 2024-04-25T08:35:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553353 s, 189 MB/s 2024-04-25T08:35:54.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628842 s, 167 MB/s 2024-04-25T08:35:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622908 s, 168 MB/s 2024-04-25T08:35:54.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.302 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828409 s, 127 MB/s 2024-04-25T08:35:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632705 s, 166 MB/s 2024-04-25T08:35:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618156 s, 170 MB/s 2024-04-25T08:35:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.323 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629154 s, 167 MB/s 2024-04-25T08:35:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572513 s, 183 MB/s 2024-04-25T08:35:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610636 s, 172 MB/s 2024-04-25T08:35:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.345 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689582 s, 152 MB/s 2024-04-25T08:35:54.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.353 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719991 s, 146 MB/s 2024-04-25T08:35:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709438 s, 148 MB/s 2024-04-25T08:35:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569645 s, 184 MB/s 2024-04-25T08:35:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708742 s, 148 MB/s 2024-04-25T08:35:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607712 s, 173 MB/s 2024-04-25T08:35:54.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:54.384 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.449 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608673 s, 172 MB/s 2024-04-25T08:35:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549031 s, 191 MB/s 2024-04-25T08:35:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.462 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571834 s, 183 MB/s 2024-04-25T08:35:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005708 s, 184 MB/s 2024-04-25T08:35:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.476 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697844 s, 150 MB/s 2024-04-25T08:35:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.484 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717595 s, 146 MB/s 2024-04-25T08:35:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.491 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616128 s, 170 MB/s 2024-04-25T08:35:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.498 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598019 s, 175 MB/s 2024-04-25T08:35:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639095 s, 164 MB/s 2024-04-25T08:35:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557723 s, 188 MB/s 2024-04-25T08:35:54.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572354 s, 183 MB/s 2024-04-25T08:35:54.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.525 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576563 s, 182 MB/s 2024-04-25T08:35:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703831 s, 149 MB/s 2024-04-25T08:35:54.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583893 s, 180 MB/s 2024-04-25T08:35:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607059 s, 173 MB/s 2024-04-25T08:35:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706251 s, 148 MB/s 2024-04-25T08:35:54.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.562 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576142 s, 182 MB/s 2024-04-25T08:35:54.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.570 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688362 s, 152 MB/s 2024-04-25T08:35:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.578 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703812 s, 149 MB/s 2024-04-25T08:35:54.586 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.586 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.586 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072206 s, 145 MB/s 2024-04-25T08:35:54.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.593 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607468 s, 173 MB/s 2024-04-25T08:35:54.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620565 s, 169 MB/s 2024-04-25T08:35:54.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.607 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606921 s, 173 MB/s 2024-04-25T08:35:54.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585234 s, 179 MB/s 2024-04-25T08:35:54.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623347 s, 168 MB/s 2024-04-25T08:35:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612328 s, 171 MB/s 2024-04-25T08:35:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597937 s, 175 MB/s 2024-04-25T08:35:54.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.643 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739268 s, 142 MB/s 2024-04-25T08:35:54.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585565 s, 179 MB/s 2024-04-25T08:35:54.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.657 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634293 s, 165 MB/s 2024-04-25T08:35:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.663 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056284 s, 186 MB/s 2024-04-25T08:35:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005766 s, 182 MB/s 2024-04-25T08:35:54.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:54.671 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:54.671 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:54.671 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:35:54.671 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:54.671 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:54.671 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:54.672 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070045 s, 150 MB/s 2024-04-25T08:35:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.745 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732105 s, 143 MB/s 2024-04-25T08:35:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.752 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624815 s, 168 MB/s 2024-04-25T08:35:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.760 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654637 s, 160 MB/s 2024-04-25T08:35:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.767 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656709 s, 160 MB/s 2024-04-25T08:35:54.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705605 s, 149 MB/s 2024-04-25T08:35:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073651 s, 142 MB/s 2024-04-25T08:35:54.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648784 s, 162 MB/s 2024-04-25T08:35:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.798 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637659 s, 164 MB/s 2024-04-25T08:35:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593412 s, 177 MB/s 2024-04-25T08:35:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.812 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687157 s, 153 MB/s 2024-04-25T08:35:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.820 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063615 s, 165 MB/s 2024-04-25T08:35:54.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.826 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562482 s, 186 MB/s 2024-04-25T08:35:54.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.834 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640827 s, 164 MB/s 2024-04-25T08:35:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.841 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604414 s, 173 MB/s 2024-04-25T08:35:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.849 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712785 s, 147 MB/s 2024-04-25T08:35:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.856 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599136 s, 175 MB/s 2024-04-25T08:35:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.862 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597174 s, 176 MB/s 2024-04-25T08:35:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573441 s, 183 MB/s 2024-04-25T08:35:54.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.877 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747763 s, 140 MB/s 2024-04-25T08:35:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658128 s, 159 MB/s 2024-04-25T08:35:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567342 s, 185 MB/s 2024-04-25T08:35:54.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596375 s, 176 MB/s 2024-04-25T08:35:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567002 s, 185 MB/s 2024-04-25T08:35:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.914 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075069 s, 140 MB/s 2024-04-25T08:35:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695368 s, 151 MB/s 2024-04-25T08:35:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625227 s, 168 MB/s 2024-04-25T08:35:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684293 s, 153 MB/s 2024-04-25T08:35:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707072 s, 148 MB/s 2024-04-25T08:35:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006142 s, 171 MB/s 2024-04-25T08:35:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.958 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635186 s, 165 MB/s 2024-04-25T08:35:54.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:54.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:54.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568464 s, 184 MB/s 2024-04-25T08:35:54.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:54.966 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614172 s, 171 MB/s 2024-04-25T08:35:55.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724951 s, 145 MB/s 2024-04-25T08:35:55.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.048 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560841 s, 187 MB/s 2024-04-25T08:35:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059074 s, 178 MB/s 2024-04-25T08:35:55.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702893 s, 149 MB/s 2024-04-25T08:35:55.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.069 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583892 s, 180 MB/s 2024-04-25T08:35:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616834 s, 170 MB/s 2024-04-25T08:35:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.084 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624708 s, 168 MB/s 2024-04-25T08:35:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621611 s, 169 MB/s 2024-04-25T08:35:55.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.097 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571212 s, 184 MB/s 2024-04-25T08:35:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.106 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685204 s, 153 MB/s 2024-04-25T08:35:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562457 s, 186 MB/s 2024-04-25T08:35:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.120 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763603 s, 137 MB/s 2024-04-25T08:35:55.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.128 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636747 s, 165 MB/s 2024-04-25T08:35:55.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.136 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706553 s, 148 MB/s 2024-04-25T08:35:55.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.143 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602445 s, 174 MB/s 2024-04-25T08:35:55.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.151 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756617 s, 139 MB/s 2024-04-25T08:35:55.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.158 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637806 s, 164 MB/s 2024-04-25T08:35:55.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.165 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618323 s, 170 MB/s 2024-04-25T08:35:55.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.173 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697941 s, 150 MB/s 2024-04-25T08:35:55.180 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.180 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.180 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601932 s, 174 MB/s 2024-04-25T08:35:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.187 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634248 s, 165 MB/s 2024-04-25T08:35:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612332 s, 171 MB/s 2024-04-25T08:35:55.201 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.201 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.201 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060463 s, 173 MB/s 2024-04-25T08:35:55.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059058 s, 178 MB/s 2024-04-25T08:35:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.216 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693892 s, 151 MB/s 2024-04-25T08:35:55.223 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.223 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.223 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604104 s, 174 MB/s 2024-04-25T08:35:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.230 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571401 s, 184 MB/s 2024-04-25T08:35:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.238 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704729 s, 149 MB/s 2024-04-25T08:35:55.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.245 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615778 s, 170 MB/s 2024-04-25T08:35:55.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072308 s, 145 MB/s 2024-04-25T08:35:55.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.260 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630488 s, 166 MB/s 2024-04-25T08:35:55.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:55.261 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:55.261 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:55.261 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:35:55.261 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:55.262 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:55.262 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:55.262 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.346 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.346 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637281 s, 165 MB/s 2024-04-25T08:35:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641949 s, 163 MB/s 2024-04-25T08:35:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698035 s, 150 MB/s 2024-04-25T08:35:55.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728087 s, 144 MB/s 2024-04-25T08:35:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.356 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560716 s, 187 MB/s 2024-04-25T08:35:55.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.363 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565339 s, 185 MB/s 2024-04-25T08:35:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.370 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617062 s, 170 MB/s 2024-04-25T08:35:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588476 s, 178 MB/s 2024-04-25T08:35:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.384 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060747 s, 173 MB/s 2024-04-25T08:35:55.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.391 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705612 s, 149 MB/s 2024-04-25T08:35:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.398 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005764 s, 182 MB/s 2024-04-25T08:35:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727747 s, 144 MB/s 2024-04-25T08:35:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.413 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588362 s, 178 MB/s 2024-04-25T08:35:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.420 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607066 s, 173 MB/s 2024-04-25T08:35:55.427 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.427 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.427 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063497 s, 165 MB/s 2024-04-25T08:35:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.436 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738636 s, 142 MB/s 2024-04-25T08:35:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.443 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623737 s, 168 MB/s 2024-04-25T08:35:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672902 s, 156 MB/s 2024-04-25T08:35:55.457 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.457 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.457 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058205 s, 180 MB/s 2024-04-25T08:35:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547822 s, 191 MB/s 2024-04-25T08:35:55.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.471 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605254 s, 173 MB/s 2024-04-25T08:35:55.478 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.478 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.478 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643799 s, 163 MB/s 2024-04-25T08:35:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595808 s, 176 MB/s 2024-04-25T08:35:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.492 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614132 s, 171 MB/s 2024-04-25T08:35:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.499 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608399 s, 172 MB/s 2024-04-25T08:35:55.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.506 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619422 s, 169 MB/s 2024-04-25T08:35:55.513 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.513 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.513 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612591 s, 171 MB/s 2024-04-25T08:35:55.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.520 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575803 s, 182 MB/s 2024-04-25T08:35:55.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611642 s, 171 MB/s 2024-04-25T08:35:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.534 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612044 s, 171 MB/s 2024-04-25T08:35:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.541 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699284 s, 150 MB/s 2024-04-25T08:35:55.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.549 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625399 s, 168 MB/s 2024-04-25T08:35:55.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:55.550 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:55.550 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:55.550 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:35:55.550 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:55.550 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:55.551 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:55.551 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609829 s, 172 MB/s 2024-04-25T08:35:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059829 s, 175 MB/s 2024-04-25T08:35:55.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607171 s, 173 MB/s 2024-04-25T08:35:55.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.636 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070274 s, 149 MB/s 2024-04-25T08:35:55.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639948 s, 164 MB/s 2024-04-25T08:35:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658758 s, 159 MB/s 2024-04-25T08:35:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.658 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565119 s, 186 MB/s 2024-04-25T08:35:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766128 s, 137 MB/s 2024-04-25T08:35:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.673 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614749 s, 171 MB/s 2024-04-25T08:35:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583211 s, 180 MB/s 2024-04-25T08:35:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561046 s, 187 MB/s 2024-04-25T08:35:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061761 s, 170 MB/s 2024-04-25T08:35:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.702 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731052 s, 143 MB/s 2024-04-25T08:35:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634387 s, 165 MB/s 2024-04-25T08:35:55.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599662 s, 175 MB/s 2024-04-25T08:35:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.724 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624111 s, 168 MB/s 2024-04-25T08:35:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.731 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698929 s, 150 MB/s 2024-04-25T08:35:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565003 s, 186 MB/s 2024-04-25T08:35:55.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680603 s, 154 MB/s 2024-04-25T08:35:55.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694332 s, 151 MB/s 2024-04-25T08:35:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699793 s, 150 MB/s 2024-04-25T08:35:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581093 s, 180 MB/s 2024-04-25T08:35:55.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.776 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686121 s, 153 MB/s 2024-04-25T08:35:55.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594645 s, 176 MB/s 2024-04-25T08:35:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576605 s, 182 MB/s 2024-04-25T08:35:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.796 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601529 s, 174 MB/s 2024-04-25T08:35:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058153 s, 180 MB/s 2024-04-25T08:35:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697193 s, 150 MB/s 2024-04-25T08:35:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.818 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650228 s, 161 MB/s 2024-04-25T08:35:55.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.827 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760383 s, 138 MB/s 2024-04-25T08:35:55.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.834 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658491 s, 159 MB/s 2024-04-25T08:35:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.841 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057535 s, 182 MB/s 2024-04-25T08:35:55.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:55.842 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:55.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722026 s, 145 MB/s 2024-04-25T08:35:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071793 s, 146 MB/s 2024-04-25T08:35:55.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.923 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577236 s, 182 MB/s 2024-04-25T08:35:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579048 s, 181 MB/s 2024-04-25T08:35:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569987 s, 184 MB/s 2024-04-25T08:35:55.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.943 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061854 s, 170 MB/s 2024-04-25T08:35:55.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.950 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617104 s, 170 MB/s 2024-04-25T08:35:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062667 s, 167 MB/s 2024-04-25T08:35:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705289 s, 149 MB/s 2024-04-25T08:35:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700377 s, 150 MB/s 2024-04-25T08:35:55.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.980 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.980 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552355 s, 190 MB/s 2024-04-25T08:35:55.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.987 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621286 s, 169 MB/s 2024-04-25T08:35:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:55.995 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703279 s, 149 MB/s 2024-04-25T08:35:56.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.002 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574747 s, 182 MB/s 2024-04-25T08:35:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581754 s, 180 MB/s 2024-04-25T08:35:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736893 s, 142 MB/s 2024-04-25T08:35:56.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565474 s, 185 MB/s 2024-04-25T08:35:56.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607966 s, 172 MB/s 2024-04-25T08:35:56.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704007 s, 149 MB/s 2024-04-25T08:35:56.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.046 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697281 s, 150 MB/s 2024-04-25T08:35:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.053 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617526 s, 170 MB/s 2024-04-25T08:35:56.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.061 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712015 s, 147 MB/s 2024-04-25T08:35:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.068 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642486 s, 163 MB/s 2024-04-25T08:35:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615883 s, 170 MB/s 2024-04-25T08:35:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739454 s, 142 MB/s 2024-04-25T08:35:56.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612435 s, 171 MB/s 2024-04-25T08:35:56.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.098 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602586 s, 174 MB/s 2024-04-25T08:35:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707794 s, 148 MB/s 2024-04-25T08:35:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582476 s, 180 MB/s 2024-04-25T08:35:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.119 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056534 s, 185 MB/s 2024-04-25T08:35:56.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.127 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648798 s, 162 MB/s 2024-04-25T08:35:56.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579345 s, 181 MB/s 2024-04-25T08:35:56.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:56.134 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705959 s, 149 MB/s 2024-04-25T08:35:56.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611419 s, 171 MB/s 2024-04-25T08:35:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562325 s, 186 MB/s 2024-04-25T08:35:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.221 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617831 s, 170 MB/s 2024-04-25T08:35:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589791 s, 178 MB/s 2024-04-25T08:35:56.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619224 s, 169 MB/s 2024-04-25T08:35:56.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702584 s, 149 MB/s 2024-04-25T08:35:56.252 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.252 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.252 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779998 s, 134 MB/s 2024-04-25T08:35:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.259 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629541 s, 167 MB/s 2024-04-25T08:35:56.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.266 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631144 s, 166 MB/s 2024-04-25T08:35:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.273 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618348 s, 170 MB/s 2024-04-25T08:35:56.280 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.280 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646474 s, 162 MB/s 2024-04-25T08:35:56.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735717 s, 143 MB/s 2024-04-25T08:35:56.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.295 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571268 s, 184 MB/s 2024-04-25T08:35:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.302 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586735 s, 179 MB/s 2024-04-25T08:35:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.309 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608022 s, 172 MB/s 2024-04-25T08:35:56.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.317 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704282 s, 149 MB/s 2024-04-25T08:35:56.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.324 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060026 s, 175 MB/s 2024-04-25T08:35:56.331 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.331 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.331 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612484 s, 171 MB/s 2024-04-25T08:35:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.338 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586497 s, 179 MB/s 2024-04-25T08:35:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057129 s, 184 MB/s 2024-04-25T08:35:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.352 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626262 s, 167 MB/s 2024-04-25T08:35:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.359 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635372 s, 165 MB/s 2024-04-25T08:35:56.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.368 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772615 s, 136 MB/s 2024-04-25T08:35:56.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.374 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573746 s, 183 MB/s 2024-04-25T08:35:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574284 s, 183 MB/s 2024-04-25T08:35:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694132 s, 151 MB/s 2024-04-25T08:35:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708235 s, 148 MB/s 2024-04-25T08:35:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057598 s, 182 MB/s 2024-04-25T08:35:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582341 s, 180 MB/s 2024-04-25T08:35:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.418 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715185 s, 147 MB/s 2024-04-25T08:35:56.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006161 s, 170 MB/s 2024-04-25T08:35:56.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:56.427 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.490 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561592 s, 187 MB/s 2024-04-25T08:35:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616105 s, 170 MB/s 2024-04-25T08:35:56.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.505 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745034 s, 141 MB/s 2024-04-25T08:35:56.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.513 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622746 s, 168 MB/s 2024-04-25T08:35:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607118 s, 173 MB/s 2024-04-25T08:35:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.527 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711514 s, 147 MB/s 2024-04-25T08:35:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.535 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723207 s, 145 MB/s 2024-04-25T08:35:56.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.542 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589338 s, 178 MB/s 2024-04-25T08:35:56.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.550 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706054 s, 149 MB/s 2024-04-25T08:35:56.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.559 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770775 s, 136 MB/s 2024-04-25T08:35:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.566 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587461 s, 178 MB/s 2024-04-25T08:35:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.572 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577047 s, 182 MB/s 2024-04-25T08:35:56.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.580 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631491 s, 166 MB/s 2024-04-25T08:35:56.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.588 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694258 s, 151 MB/s 2024-04-25T08:35:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.594 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592868 s, 177 MB/s 2024-04-25T08:35:56.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.601 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567412 s, 185 MB/s 2024-04-25T08:35:56.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.608 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613716 s, 171 MB/s 2024-04-25T08:35:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.615 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626808 s, 167 MB/s 2024-04-25T08:35:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.623 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713199 s, 147 MB/s 2024-04-25T08:35:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.630 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610975 s, 172 MB/s 2024-04-25T08:35:56.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.637 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625748 s, 168 MB/s 2024-04-25T08:35:56.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670295 s, 156 MB/s 2024-04-25T08:35:56.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568214 s, 185 MB/s 2024-04-25T08:35:56.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610947 s, 172 MB/s 2024-04-25T08:35:56.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692707 s, 151 MB/s 2024-04-25T08:35:56.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.675 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071643 s, 146 MB/s 2024-04-25T08:35:56.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.682 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069871 s, 150 MB/s 2024-04-25T08:35:56.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.689 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602812 s, 174 MB/s 2024-04-25T08:35:56.696 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.696 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.696 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562202 s, 187 MB/s 2024-04-25T08:35:56.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.704 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635676 s, 165 MB/s 2024-04-25T08:35:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566133 s, 185 MB/s 2024-04-25T08:35:56.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.718 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707866 s, 148 MB/s 2024-04-25T08:35:56.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:56.719 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:56.719 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:56.719 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:35:56.719 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:56.719 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:56.719 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:56.720 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618244 s, 170 MB/s 2024-04-25T08:35:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.792 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718172 s, 146 MB/s 2024-04-25T08:35:56.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.799 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612117 s, 171 MB/s 2024-04-25T08:35:56.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.807 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718535 s, 146 MB/s 2024-04-25T08:35:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.814 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615758 s, 170 MB/s 2024-04-25T08:35:56.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.822 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692266 s, 151 MB/s 2024-04-25T08:35:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.829 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690913 s, 152 MB/s 2024-04-25T08:35:56.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625619 s, 168 MB/s 2024-04-25T08:35:56.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.844 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064702 s, 162 MB/s 2024-04-25T08:35:56.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.850 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562529 s, 186 MB/s 2024-04-25T08:35:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.857 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564018 s, 186 MB/s 2024-04-25T08:35:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.865 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730996 s, 143 MB/s 2024-04-25T08:35:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.872 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608837 s, 172 MB/s 2024-04-25T08:35:56.879 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.879 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.879 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585301 s, 179 MB/s 2024-04-25T08:35:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.886 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629011 s, 167 MB/s 2024-04-25T08:35:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.894 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662683 s, 158 MB/s 2024-04-25T08:35:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.901 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649221 s, 162 MB/s 2024-04-25T08:35:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582433 s, 180 MB/s 2024-04-25T08:35:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.914 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558523 s, 188 MB/s 2024-04-25T08:35:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.921 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564095 s, 186 MB/s 2024-04-25T08:35:56.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638221 s, 164 MB/s 2024-04-25T08:35:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585718 s, 179 MB/s 2024-04-25T08:35:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056323 s, 186 MB/s 2024-04-25T08:35:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603061 s, 174 MB/s 2024-04-25T08:35:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.956 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617986 s, 170 MB/s 2024-04-25T08:35:56.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.964 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691496 s, 152 MB/s 2024-04-25T08:35:56.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.971 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061191 s, 171 MB/s 2024-04-25T08:35:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.979 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707941 s, 148 MB/s 2024-04-25T08:35:56.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.985 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581421 s, 180 MB/s 2024-04-25T08:35:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:56.992 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571782 s, 183 MB/s 2024-04-25T08:35:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.000 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706571 s, 148 MB/s 2024-04-25T08:35:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.007 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639137 s, 164 MB/s 2024-04-25T08:35:57.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:57.008 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:57.009 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:57.009 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:35:57.009 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:57.009 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:57.009 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:57.009 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:57.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:57.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:57.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648839 s, 162 MB/s 2024-04-25T08:35:57.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702731 s, 149 MB/s 2024-04-25T08:35:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.090 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624528 s, 168 MB/s 2024-04-25T08:35:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.097 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590681 s, 178 MB/s 2024-04-25T08:35:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663758 s, 158 MB/s 2024-04-25T08:35:57.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572724 s, 183 MB/s 2024-04-25T08:35:57.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.118 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629271 s, 167 MB/s 2024-04-25T08:35:57.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.126 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618043 s, 170 MB/s 2024-04-25T08:35:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.132 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565955 s, 185 MB/s 2024-04-25T08:35:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.139 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569094 s, 184 MB/s 2024-04-25T08:35:57.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.146 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639367 s, 164 MB/s 2024-04-25T08:35:57.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.153 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062003 s, 169 MB/s 2024-04-25T08:35:57.160 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.160 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.160 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564929 s, 186 MB/s 2024-04-25T08:35:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.167 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602866 s, 174 MB/s 2024-04-25T08:35:57.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.174 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062214 s, 169 MB/s 2024-04-25T08:35:57.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.181 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060401 s, 174 MB/s 2024-04-25T08:35:57.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.188 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601784 s, 174 MB/s 2024-04-25T08:35:57.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668113 s, 157 MB/s 2024-04-25T08:35:57.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.203 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720653 s, 146 MB/s 2024-04-25T08:35:57.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.211 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751562 s, 140 MB/s 2024-04-25T08:35:57.218 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.218 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.218 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058515 s, 179 MB/s 2024-04-25T08:35:57.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657122 s, 160 MB/s 2024-04-25T08:35:57.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.233 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605011 s, 173 MB/s 2024-04-25T08:35:57.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617931 s, 170 MB/s 2024-04-25T08:35:57.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602674 s, 174 MB/s 2024-04-25T08:35:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.255 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736167 s, 142 MB/s 2024-04-25T08:35:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.263 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697813 s, 150 MB/s 2024-04-25T08:35:57.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.271 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674085 s, 156 MB/s 2024-04-25T08:35:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575302 s, 182 MB/s 2024-04-25T08:35:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605664 s, 173 MB/s 2024-04-25T08:35:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710549 s, 148 MB/s 2024-04-25T08:35:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614055 s, 171 MB/s 2024-04-25T08:35:57.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:57.300 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:57.300 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:57.300 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:35:57.300 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:57.300 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:57.301 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:57.301 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:57.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741322 s, 141 MB/s 2024-04-25T08:35:57.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.373 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636175 s, 165 MB/s 2024-04-25T08:35:57.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.380 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589748 s, 178 MB/s 2024-04-25T08:35:57.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.388 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731573 s, 143 MB/s 2024-04-25T08:35:57.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603622 s, 174 MB/s 2024-04-25T08:35:57.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.403 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070279 s, 149 MB/s 2024-04-25T08:35:57.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602396 s, 174 MB/s 2024-04-25T08:35:57.417 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.417 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.417 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608483 s, 172 MB/s 2024-04-25T08:35:57.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.425 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696103 s, 151 MB/s 2024-04-25T08:35:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.431 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576386 s, 182 MB/s 2024-04-25T08:35:57.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.438 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583983 s, 180 MB/s 2024-04-25T08:35:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.446 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695791 s, 151 MB/s 2024-04-25T08:35:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068291 s, 154 MB/s 2024-04-25T08:35:57.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.460 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561464 s, 187 MB/s 2024-04-25T08:35:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.469 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705912 s, 149 MB/s 2024-04-25T08:35:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.476 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632888 s, 166 MB/s 2024-04-25T08:35:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559365 s, 187 MB/s 2024-04-25T08:35:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622688 s, 168 MB/s 2024-04-25T08:35:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.496 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612077 s, 171 MB/s 2024-04-25T08:35:57.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636121 s, 165 MB/s 2024-04-25T08:35:57.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599475 s, 175 MB/s 2024-04-25T08:35:57.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575548 s, 182 MB/s 2024-04-25T08:35:57.523 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573048 s, 183 MB/s 2024-04-25T08:35:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.530 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593235 s, 177 MB/s 2024-04-25T08:35:57.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.539 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709509 s, 148 MB/s 2024-04-25T08:35:57.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691553 s, 152 MB/s 2024-04-25T08:35:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717107 s, 146 MB/s 2024-04-25T08:35:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750881 s, 140 MB/s 2024-04-25T08:35:57.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.569 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576118 s, 182 MB/s 2024-04-25T08:35:57.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.577 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685597 s, 153 MB/s 2024-04-25T08:35:57.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574275 s, 183 MB/s 2024-04-25T08:35:57.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.592 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072097 s, 145 MB/s 2024-04-25T08:35:57.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:57.593 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:57.593 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:57.593 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:35:57.593 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:57.593 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:57.594 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:57.594 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:57.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.660 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622304 s, 168 MB/s 2024-04-25T08:35:57.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626118 s, 167 MB/s 2024-04-25T08:35:57.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634321 s, 165 MB/s 2024-04-25T08:35:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.681 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.681 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599167 s, 175 MB/s 2024-04-25T08:35:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.688 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613322 s, 171 MB/s 2024-04-25T08:35:57.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.695 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591982 s, 177 MB/s 2024-04-25T08:35:57.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578736 s, 181 MB/s 2024-04-25T08:35:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676194 s, 155 MB/s 2024-04-25T08:35:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605422 s, 173 MB/s 2024-04-25T08:35:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.724 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065866 s, 159 MB/s 2024-04-25T08:35:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645403 s, 162 MB/s 2024-04-25T08:35:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.739 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670203 s, 156 MB/s 2024-04-25T08:35:57.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591667 s, 177 MB/s 2024-04-25T08:35:57.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640135 s, 164 MB/s 2024-04-25T08:35:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.760 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630274 s, 166 MB/s 2024-04-25T08:35:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630327 s, 166 MB/s 2024-04-25T08:35:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600512 s, 175 MB/s 2024-04-25T08:35:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624774 s, 168 MB/s 2024-04-25T08:35:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.790 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679956 s, 154 MB/s 2024-04-25T08:35:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.798 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744925 s, 141 MB/s 2024-04-25T08:35:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.805 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644093 s, 163 MB/s 2024-04-25T08:35:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630443 s, 166 MB/s 2024-04-25T08:35:57.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.819 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608015 s, 172 MB/s 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601612 s, 174 MB/s 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768261 s, 136 MB/s 2024-04-25T08:35:57.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.843 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727059 s, 144 MB/s 2024-04-25T08:35:57.851 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732444 s, 143 MB/s 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557913 s, 188 MB/s 2024-04-25T08:35:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069697 s, 150 MB/s 2024-04-25T08:35:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063379 s, 165 MB/s 2024-04-25T08:35:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.880 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620568 s, 169 MB/s 2024-04-25T08:35:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.888 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555656 s, 189 MB/s 2024-04-25T08:35:57.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:57.888 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613465 s, 171 MB/s 2024-04-25T08:35:57.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.959 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632049 s, 166 MB/s 2024-04-25T08:35:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.967 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698744 s, 150 MB/s 2024-04-25T08:35:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.974 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639366 s, 164 MB/s 2024-04-25T08:35:57.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699507 s, 150 MB/s 2024-04-25T08:35:57.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597463 s, 176 MB/s 2024-04-25T08:35:57.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:57.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:57.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659148 s, 159 MB/s 2024-04-25T08:35:58.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.003 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579002 s, 181 MB/s 2024-04-25T08:35:58.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.011 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706273 s, 148 MB/s 2024-04-25T08:35:58.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562209 s, 187 MB/s 2024-04-25T08:35:58.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071058 s, 148 MB/s 2024-04-25T08:35:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.037 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106147 s, 98.8 MB/s 2024-04-25T08:35:58.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699379 s, 150 MB/s 2024-04-25T08:35:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056736 s, 185 MB/s 2024-04-25T08:35:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619861 s, 169 MB/s 2024-04-25T08:35:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.067 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071914 s, 146 MB/s 2024-04-25T08:35:58.074 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.074 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.074 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619042 s, 169 MB/s 2024-04-25T08:35:58.082 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.082 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.082 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712031 s, 147 MB/s 2024-04-25T08:35:58.089 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.089 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.089 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610082 s, 172 MB/s 2024-04-25T08:35:58.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.096 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633478 s, 166 MB/s 2024-04-25T08:35:58.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572464 s, 183 MB/s 2024-04-25T08:35:58.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.109 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558725 s, 188 MB/s 2024-04-25T08:35:58.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.117 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692816 s, 151 MB/s 2024-04-25T08:35:58.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621405 s, 169 MB/s 2024-04-25T08:35:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.132 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646125 s, 162 MB/s 2024-04-25T08:35:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060059 s, 175 MB/s 2024-04-25T08:35:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.146 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631123 s, 166 MB/s 2024-04-25T08:35:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.152 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546772 s, 192 MB/s 2024-04-25T08:35:58.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618946 s, 169 MB/s 2024-04-25T08:35:58.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.166 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059715 s, 176 MB/s 2024-04-25T08:35:58.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.173 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619059 s, 169 MB/s 2024-04-25T08:35:58.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761272 s, 138 MB/s 2024-04-25T08:35:58.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:58.183 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:58.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.250 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701699 s, 149 MB/s 2024-04-25T08:35:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720819 s, 145 MB/s 2024-04-25T08:35:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575636 s, 182 MB/s 2024-04-25T08:35:58.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.271 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612351 s, 171 MB/s 2024-04-25T08:35:58.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560502 s, 187 MB/s 2024-04-25T08:35:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589532 s, 178 MB/s 2024-04-25T08:35:58.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.291 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562033 s, 187 MB/s 2024-04-25T08:35:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.297 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054891 s, 191 MB/s 2024-04-25T08:35:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706476 s, 148 MB/s 2024-04-25T08:35:58.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.312 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631257 s, 166 MB/s 2024-04-25T08:35:58.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735607 s, 143 MB/s 2024-04-25T08:35:58.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567527 s, 185 MB/s 2024-04-25T08:35:58.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.335 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666188 s, 157 MB/s 2024-04-25T08:35:58.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715789 s, 146 MB/s 2024-04-25T08:35:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716074 s, 146 MB/s 2024-04-25T08:35:58.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.359 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700694 s, 150 MB/s 2024-04-25T08:35:58.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646271 s, 162 MB/s 2024-04-25T08:35:58.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.373 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562199 s, 187 MB/s 2024-04-25T08:35:58.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.380 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059971 s, 175 MB/s 2024-04-25T08:35:58.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.387 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589947 s, 178 MB/s 2024-04-25T08:35:58.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.394 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610412 s, 172 MB/s 2024-04-25T08:35:58.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.401 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610869 s, 172 MB/s 2024-04-25T08:35:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645341 s, 162 MB/s 2024-04-25T08:35:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606123 s, 173 MB/s 2024-04-25T08:35:58.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.422 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609718 s, 172 MB/s 2024-04-25T08:35:58.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.429 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565008 s, 186 MB/s 2024-04-25T08:35:58.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.436 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708071 s, 148 MB/s 2024-04-25T08:35:58.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.443 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612638 s, 171 MB/s 2024-04-25T08:35:58.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061557 s, 170 MB/s 2024-04-25T08:35:58.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.458 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630487 s, 166 MB/s 2024-04-25T08:35:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.465 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692152 s, 151 MB/s 2024-04-25T08:35:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.473 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.474 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718584 s, 146 MB/s 2024-04-25T08:35:58.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:58.475 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:58.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562945 s, 186 MB/s 2024-04-25T08:35:58.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617701 s, 170 MB/s 2024-04-25T08:35:58.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615199 s, 170 MB/s 2024-04-25T08:35:58.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.561 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602599 s, 174 MB/s 2024-04-25T08:35:58.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608214 s, 172 MB/s 2024-04-25T08:35:58.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.576 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700004 s, 150 MB/s 2024-04-25T08:35:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594722 s, 176 MB/s 2024-04-25T08:35:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.591 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742051 s, 141 MB/s 2024-04-25T08:35:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.598 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619897 s, 169 MB/s 2024-04-25T08:35:58.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602315 s, 174 MB/s 2024-04-25T08:35:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686434 s, 153 MB/s 2024-04-25T08:35:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590353 s, 178 MB/s 2024-04-25T08:35:58.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698535 s, 150 MB/s 2024-04-25T08:35:58.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687923 s, 152 MB/s 2024-04-25T08:35:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.643 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708541 s, 148 MB/s 2024-04-25T08:35:58.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060851 s, 172 MB/s 2024-04-25T08:35:58.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578096 s, 181 MB/s 2024-04-25T08:35:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.663 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580382 s, 181 MB/s 2024-04-25T08:35:58.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.671 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710564 s, 148 MB/s 2024-04-25T08:35:58.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.678 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620917 s, 169 MB/s 2024-04-25T08:35:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.685 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612999 s, 171 MB/s 2024-04-25T08:35:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.692 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609726 s, 172 MB/s 2024-04-25T08:35:58.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739749 s, 142 MB/s 2024-04-25T08:35:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.708 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653079 s, 161 MB/s 2024-04-25T08:35:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733902 s, 143 MB/s 2024-04-25T08:35:58.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604361 s, 174 MB/s 2024-04-25T08:35:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061184 s, 171 MB/s 2024-04-25T08:35:58.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593086 s, 177 MB/s 2024-04-25T08:35:58.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614583 s, 171 MB/s 2024-04-25T08:35:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781683 s, 134 MB/s 2024-04-25T08:35:58.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727055 s, 144 MB/s 2024-04-25T08:35:58.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632791 s, 166 MB/s 2024-04-25T08:35:58.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:58.770 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:58.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593877 s, 177 MB/s 2024-04-25T08:35:58.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.841 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645665 s, 162 MB/s 2024-04-25T08:35:58.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006098 s, 172 MB/s 2024-04-25T08:35:58.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.856 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707687 s, 148 MB/s 2024-04-25T08:35:58.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.863 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651816 s, 161 MB/s 2024-04-25T08:35:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629989 s, 166 MB/s 2024-04-25T08:35:58.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.877 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614957 s, 171 MB/s 2024-04-25T08:35:58.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587469 s, 178 MB/s 2024-04-25T08:35:58.892 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.892 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.892 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743326 s, 141 MB/s 2024-04-25T08:35:58.902 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.902 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.903 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710227 s, 148 MB/s 2024-04-25T08:35:58.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639056 s, 164 MB/s 2024-04-25T08:35:58.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.915 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618078 s, 170 MB/s 2024-04-25T08:35:58.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.922 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596239 s, 176 MB/s 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055808 s, 188 MB/s 2024-04-25T08:35:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055042 s, 191 MB/s 2024-04-25T08:35:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.943 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709166 s, 148 MB/s 2024-04-25T08:35:58.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.950 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619288 s, 169 MB/s 2024-04-25T08:35:58.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631376 s, 166 MB/s 2024-04-25T08:35:58.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070504 s, 149 MB/s 2024-04-25T08:35:58.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.972 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575412 s, 182 MB/s 2024-04-25T08:35:58.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.978 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592634 s, 177 MB/s 2024-04-25T08:35:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631269 s, 166 MB/s 2024-04-25T08:35:58.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:58.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:58.994 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730998 s, 143 MB/s 2024-04-25T08:35:59.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.002 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720723 s, 145 MB/s 2024-04-25T08:35:59.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569593 s, 184 MB/s 2024-04-25T08:35:59.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642495 s, 163 MB/s 2024-04-25T08:35:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.024 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711311 s, 147 MB/s 2024-04-25T08:35:59.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.031 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626233 s, 167 MB/s 2024-04-25T08:35:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622533 s, 168 MB/s 2024-04-25T08:35:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.046 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705945 s, 149 MB/s 2024-04-25T08:35:59.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.054 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608585 s, 172 MB/s 2024-04-25T08:35:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.061 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688616 s, 152 MB/s 2024-04-25T08:35:59.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:59.063 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:59.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.132 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714143 s, 147 MB/s 2024-04-25T08:35:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599637 s, 175 MB/s 2024-04-25T08:35:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.146 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722788 s, 145 MB/s 2024-04-25T08:35:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.154 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073023 s, 144 MB/s 2024-04-25T08:35:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613833 s, 171 MB/s 2024-04-25T08:35:59.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577661 s, 182 MB/s 2024-04-25T08:35:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062163 s, 169 MB/s 2024-04-25T08:35:59.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693493 s, 151 MB/s 2024-04-25T08:35:59.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616052 s, 170 MB/s 2024-04-25T08:35:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.196 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567522 s, 185 MB/s 2024-04-25T08:35:59.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.204 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636577 s, 165 MB/s 2024-04-25T08:35:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604912 s, 173 MB/s 2024-04-25T08:35:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580502 s, 181 MB/s 2024-04-25T08:35:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570428 s, 184 MB/s 2024-04-25T08:35:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.230 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558776 s, 188 MB/s 2024-04-25T08:35:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.239 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072742 s, 144 MB/s 2024-04-25T08:35:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.246 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639294 s, 164 MB/s 2024-04-25T08:35:59.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.254 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726353 s, 144 MB/s 2024-04-25T08:35:59.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.262 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726116 s, 144 MB/s 2024-04-25T08:35:59.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.270 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645768 s, 162 MB/s 2024-04-25T08:35:59.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.278 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669989 s, 157 MB/s 2024-04-25T08:35:59.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699578 s, 150 MB/s 2024-04-25T08:35:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597589 s, 175 MB/s 2024-04-25T08:35:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586341 s, 179 MB/s 2024-04-25T08:35:59.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694061 s, 151 MB/s 2024-04-25T08:35:59.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643085 s, 163 MB/s 2024-04-25T08:35:59.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.322 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624405 s, 168 MB/s 2024-04-25T08:35:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077826 s, 135 MB/s 2024-04-25T08:35:59.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624401 s, 168 MB/s 2024-04-25T08:35:59.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588109 s, 178 MB/s 2024-04-25T08:35:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622102 s, 169 MB/s 2024-04-25T08:35:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619583 s, 169 MB/s 2024-04-25T08:35:59.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:59.360 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612648 s, 171 MB/s 2024-04-25T08:35:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.433 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072682 s, 144 MB/s 2024-04-25T08:35:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.440 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614367 s, 171 MB/s 2024-04-25T08:35:59.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.448 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070426 s, 149 MB/s 2024-04-25T08:35:59.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.455 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583182 s, 180 MB/s 2024-04-25T08:35:59.462 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.462 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.462 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637337 s, 165 MB/s 2024-04-25T08:35:59.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055196 s, 190 MB/s 2024-04-25T08:35:59.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.475 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618611 s, 170 MB/s 2024-04-25T08:35:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643102 s, 163 MB/s 2024-04-25T08:35:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.490 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588259 s, 178 MB/s 2024-04-25T08:35:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622231 s, 169 MB/s 2024-04-25T08:35:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631401 s, 166 MB/s 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060286 s, 174 MB/s 2024-04-25T08:35:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056948 s, 184 MB/s 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007415 s, 141 MB/s 2024-04-25T08:35:59.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.534 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687494 s, 153 MB/s 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056637 s, 185 MB/s 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564625 s, 186 MB/s 2024-04-25T08:35:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600711 s, 175 MB/s 2024-04-25T08:35:59.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.561 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620299 s, 169 MB/s 2024-04-25T08:35:59.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645092 s, 163 MB/s 2024-04-25T08:35:59.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.576 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064716 s, 162 MB/s 2024-04-25T08:35:59.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.583 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627158 s, 167 MB/s 2024-04-25T08:35:59.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618608 s, 170 MB/s 2024-04-25T08:35:59.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062317 s, 168 MB/s 2024-04-25T08:35:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635214 s, 165 MB/s 2024-04-25T08:35:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073555 s, 143 MB/s 2024-04-25T08:35:59.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060335 s, 174 MB/s 2024-04-25T08:35:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.626 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593982 s, 177 MB/s 2024-04-25T08:35:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.633 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622992 s, 168 MB/s 2024-04-25T08:35:59.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.640 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005805 s, 181 MB/s 2024-04-25T08:35:59.647 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.647 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.647 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613041 s, 171 MB/s 2024-04-25T08:35:59.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.648 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:59.648 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:59.648 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:35:59.648 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:59.648 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:59.649 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:59.649 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:35:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.713 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606569 s, 173 MB/s 2024-04-25T08:35:59.720 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.720 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.720 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597784 s, 175 MB/s 2024-04-25T08:35:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.726 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566425 s, 185 MB/s 2024-04-25T08:35:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580969 s, 180 MB/s 2024-04-25T08:35:59.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.740 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607095 s, 173 MB/s 2024-04-25T08:35:59.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.748 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639059 s, 164 MB/s 2024-04-25T08:35:59.755 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.756 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724764 s, 145 MB/s 2024-04-25T08:35:59.763 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.763 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.763 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670475 s, 156 MB/s 2024-04-25T08:35:59.770 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.770 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.770 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060397 s, 174 MB/s 2024-04-25T08:35:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.777 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565489 s, 185 MB/s 2024-04-25T08:35:59.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.784 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656777 s, 160 MB/s 2024-04-25T08:35:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.792 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696446 s, 151 MB/s 2024-04-25T08:35:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.799 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643508 s, 163 MB/s 2024-04-25T08:35:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.806 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594755 s, 176 MB/s 2024-04-25T08:35:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.813 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640779 s, 164 MB/s 2024-04-25T08:35:59.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.821 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636333 s, 165 MB/s 2024-04-25T08:35:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070084 s, 150 MB/s 2024-04-25T08:35:59.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.835 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574867 s, 182 MB/s 2024-04-25T08:35:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.842 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616207 s, 170 MB/s 2024-04-25T08:35:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.849 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568893 s, 184 MB/s 2024-04-25T08:35:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.856 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597194 s, 176 MB/s 2024-04-25T08:35:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.862 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589876 s, 178 MB/s 2024-04-25T08:35:59.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601953 s, 174 MB/s 2024-04-25T08:35:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584721 s, 179 MB/s 2024-04-25T08:35:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563544 s, 186 MB/s 2024-04-25T08:35:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605312 s, 173 MB/s 2024-04-25T08:35:59.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713907 s, 147 MB/s 2024-04-25T08:35:59.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613349 s, 171 MB/s 2024-04-25T08:35:59.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061299 s, 171 MB/s 2024-04-25T08:35:59.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622965 s, 168 MB/s 2024-04-25T08:35:59.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.926 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571219 s, 184 MB/s 2024-04-25T08:35:59.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:35:59.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:35:59.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648043 s, 162 MB/s 2024-04-25T08:35:59.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.935 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:35:59.935 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:35:59.935 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:35:59.936 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:35:59.936 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:35:59.936 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:35:59.936 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:00.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694252 s, 151 MB/s 2024-04-25T08:36:00.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057706 s, 182 MB/s 2024-04-25T08:36:00.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753454 s, 139 MB/s 2024-04-25T08:36:00.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614289 s, 171 MB/s 2024-04-25T08:36:00.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585338 s, 179 MB/s 2024-04-25T08:36:00.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.037 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557385 s, 188 MB/s 2024-04-25T08:36:00.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.044 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642919 s, 163 MB/s 2024-04-25T08:36:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619915 s, 169 MB/s 2024-04-25T08:36:00.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698606 s, 150 MB/s 2024-04-25T08:36:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.065 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574516 s, 183 MB/s 2024-04-25T08:36:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571764 s, 183 MB/s 2024-04-25T08:36:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630562 s, 166 MB/s 2024-04-25T08:36:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637024 s, 165 MB/s 2024-04-25T08:36:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.093 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572328 s, 183 MB/s 2024-04-25T08:36:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.101 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704829 s, 149 MB/s 2024-04-25T08:36:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611052 s, 172 MB/s 2024-04-25T08:36:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.116 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705921 s, 149 MB/s 2024-04-25T08:36:00.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.123 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609164 s, 172 MB/s 2024-04-25T08:36:00.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614183 s, 171 MB/s 2024-04-25T08:36:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.137 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572972 s, 183 MB/s 2024-04-25T08:36:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.144 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689986 s, 152 MB/s 2024-04-25T08:36:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.152 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634407 s, 165 MB/s 2024-04-25T08:36:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633692 s, 165 MB/s 2024-04-25T08:36:00.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.166 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608015 s, 172 MB/s 2024-04-25T08:36:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.174 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073175 s, 143 MB/s 2024-04-25T08:36:00.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704631 s, 149 MB/s 2024-04-25T08:36:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070365 s, 149 MB/s 2024-04-25T08:36:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.197 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589377 s, 178 MB/s 2024-04-25T08:36:00.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.204 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635907 s, 165 MB/s 2024-04-25T08:36:00.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.212 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702253 s, 149 MB/s 2024-04-25T08:36:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594395 s, 176 MB/s 2024-04-25T08:36:00.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062134 s, 169 MB/s 2024-04-25T08:36:00.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:00.227 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:00.227 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:00.227 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:36:00.227 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:00.227 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:00.227 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:00.228 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.291 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586276 s, 179 MB/s 2024-04-25T08:36:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577609 s, 182 MB/s 2024-04-25T08:36:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643539 s, 163 MB/s 2024-04-25T08:36:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619564 s, 169 MB/s 2024-04-25T08:36:00.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.320 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700742 s, 150 MB/s 2024-04-25T08:36:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.328 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697898 s, 150 MB/s 2024-04-25T08:36:00.338 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.338 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.338 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715164 s, 147 MB/s 2024-04-25T08:36:00.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608312 s, 172 MB/s 2024-04-25T08:36:00.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.350 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061036 s, 172 MB/s 2024-04-25T08:36:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607126 s, 173 MB/s 2024-04-25T08:36:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.364 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639476 s, 164 MB/s 2024-04-25T08:36:00.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.373 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731461 s, 143 MB/s 2024-04-25T08:36:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597615 s, 175 MB/s 2024-04-25T08:36:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.387 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698437 s, 150 MB/s 2024-04-25T08:36:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703856 s, 149 MB/s 2024-04-25T08:36:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618144 s, 170 MB/s 2024-04-25T08:36:00.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.409 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059325 s, 177 MB/s 2024-04-25T08:36:00.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.417 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695511 s, 151 MB/s 2024-04-25T08:36:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577031 s, 182 MB/s 2024-04-25T08:36:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.430 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561337 s, 187 MB/s 2024-04-25T08:36:00.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582232 s, 180 MB/s 2024-04-25T08:36:00.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611395 s, 172 MB/s 2024-04-25T08:36:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611774 s, 171 MB/s 2024-04-25T08:36:00.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.458 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617493 s, 170 MB/s 2024-04-25T08:36:00.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.465 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056921 s, 184 MB/s 2024-04-25T08:36:00.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.471 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566674 s, 185 MB/s 2024-04-25T08:36:00.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.479 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073047 s, 144 MB/s 2024-04-25T08:36:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.487 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707836 s, 148 MB/s 2024-04-25T08:36:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642033 s, 163 MB/s 2024-04-25T08:36:00.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.502 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607214 s, 173 MB/s 2024-04-25T08:36:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701661 s, 149 MB/s 2024-04-25T08:36:00.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.518 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696625 s, 151 MB/s 2024-04-25T08:36:00.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:00.519 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:00.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716906 s, 146 MB/s 2024-04-25T08:36:00.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005863 s, 179 MB/s 2024-04-25T08:36:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.598 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718623 s, 146 MB/s 2024-04-25T08:36:00.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614321 s, 171 MB/s 2024-04-25T08:36:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582194 s, 180 MB/s 2024-04-25T08:36:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607967 s, 172 MB/s 2024-04-25T08:36:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696176 s, 151 MB/s 2024-04-25T08:36:00.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663613 s, 158 MB/s 2024-04-25T08:36:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.642 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706938 s, 148 MB/s 2024-04-25T08:36:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660074 s, 159 MB/s 2024-04-25T08:36:00.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.658 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708364 s, 148 MB/s 2024-04-25T08:36:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732073 s, 143 MB/s 2024-04-25T08:36:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589172 s, 178 MB/s 2024-04-25T08:36:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607162 s, 173 MB/s 2024-04-25T08:36:00.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575147 s, 182 MB/s 2024-04-25T08:36:00.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627601 s, 167 MB/s 2024-04-25T08:36:00.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590873 s, 177 MB/s 2024-04-25T08:36:00.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690065 s, 152 MB/s 2024-04-25T08:36:00.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.717 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706201 s, 148 MB/s 2024-04-25T08:36:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556575 s, 188 MB/s 2024-04-25T08:36:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654473 s, 160 MB/s 2024-04-25T08:36:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682918 s, 154 MB/s 2024-04-25T08:36:00.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649248 s, 162 MB/s 2024-04-25T08:36:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655819 s, 160 MB/s 2024-04-25T08:36:00.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.760 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578134 s, 181 MB/s 2024-04-25T08:36:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.767 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668414 s, 157 MB/s 2024-04-25T08:36:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699458 s, 150 MB/s 2024-04-25T08:36:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589075 s, 178 MB/s 2024-04-25T08:36:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.789 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629662 s, 167 MB/s 2024-04-25T08:36:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.797 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668874 s, 157 MB/s 2024-04-25T08:36:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069875 s, 150 MB/s 2024-04-25T08:36:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.811 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.812 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610834 s, 172 MB/s 2024-04-25T08:36:00.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:00.813 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:00.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.877 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006503 s, 161 MB/s 2024-04-25T08:36:00.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596804 s, 176 MB/s 2024-04-25T08:36:00.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.891 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.891 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584811 s, 179 MB/s 2024-04-25T08:36:00.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.899 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740594 s, 142 MB/s 2024-04-25T08:36:00.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055566 s, 189 MB/s 2024-04-25T08:36:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.913 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064174 s, 163 MB/s 2024-04-25T08:36:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651311 s, 161 MB/s 2024-04-25T08:36:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585393 s, 179 MB/s 2024-04-25T08:36:00.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570303 s, 184 MB/s 2024-04-25T08:36:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.940 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588166 s, 178 MB/s 2024-04-25T08:36:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.948 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617193 s, 170 MB/s 2024-04-25T08:36:00.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.955 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577014 s, 182 MB/s 2024-04-25T08:36:00.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.961 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552597 s, 190 MB/s 2024-04-25T08:36:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.969 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740274 s, 142 MB/s 2024-04-25T08:36:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635131 s, 165 MB/s 2024-04-25T08:36:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00530561 s, 198 MB/s 2024-04-25T08:36:00.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.990 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646932 s, 162 MB/s 2024-04-25T08:36:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:00.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611926 s, 171 MB/s 2024-04-25T08:36:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600347 s, 175 MB/s 2024-04-25T08:36:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.011 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636723 s, 165 MB/s 2024-04-25T08:36:01.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.019 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706446 s, 148 MB/s 2024-04-25T08:36:01.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619603 s, 169 MB/s 2024-04-25T08:36:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621907 s, 169 MB/s 2024-04-25T08:36:01.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.040 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625211 s, 168 MB/s 2024-04-25T08:36:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.047 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560409 s, 187 MB/s 2024-04-25T08:36:01.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.055 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075443 s, 139 MB/s 2024-04-25T08:36:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577661 s, 182 MB/s 2024-04-25T08:36:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.069 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618605 s, 170 MB/s 2024-04-25T08:36:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.076 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619904 s, 169 MB/s 2024-04-25T08:36:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.084 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690851 s, 152 MB/s 2024-04-25T08:36:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648158 s, 162 MB/s 2024-04-25T08:36:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.098 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598172 s, 175 MB/s 2024-04-25T08:36:01.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:01.100 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:01.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.164 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061781 s, 170 MB/s 2024-04-25T08:36:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056834 s, 184 MB/s 2024-04-25T08:36:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.177 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567356 s, 185 MB/s 2024-04-25T08:36:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.185 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738864 s, 142 MB/s 2024-04-25T08:36:01.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.192 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598923 s, 175 MB/s 2024-04-25T08:36:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.199 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616641 s, 170 MB/s 2024-04-25T08:36:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631887 s, 166 MB/s 2024-04-25T08:36:01.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.213 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057628 s, 182 MB/s 2024-04-25T08:36:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573655 s, 183 MB/s 2024-04-25T08:36:01.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.227 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057722 s, 182 MB/s 2024-04-25T08:36:01.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.234 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620431 s, 169 MB/s 2024-04-25T08:36:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611559 s, 171 MB/s 2024-04-25T08:36:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.248 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634399 s, 165 MB/s 2024-04-25T08:36:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703714 s, 149 MB/s 2024-04-25T08:36:01.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.264 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709106 s, 148 MB/s 2024-04-25T08:36:01.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585407 s, 179 MB/s 2024-04-25T08:36:01.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.278 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600548 s, 175 MB/s 2024-04-25T08:36:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653195 s, 161 MB/s 2024-04-25T08:36:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059485 s, 176 MB/s 2024-04-25T08:36:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619372 s, 169 MB/s 2024-04-25T08:36:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577982 s, 181 MB/s 2024-04-25T08:36:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766093 s, 137 MB/s 2024-04-25T08:36:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619695 s, 169 MB/s 2024-04-25T08:36:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633636 s, 165 MB/s 2024-04-25T08:36:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.336 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061917 s, 169 MB/s 2024-04-25T08:36:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629233 s, 167 MB/s 2024-04-25T08:36:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590403 s, 178 MB/s 2024-04-25T08:36:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.357 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578238 s, 181 MB/s 2024-04-25T08:36:01.363 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.365 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557747 s, 188 MB/s 2024-04-25T08:36:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.370 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598566 s, 175 MB/s 2024-04-25T08:36:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.377 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595571 s, 176 MB/s 2024-04-25T08:36:01.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.384 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583221 s, 180 MB/s 2024-04-25T08:36:01.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:01.385 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608324 s, 172 MB/s 2024-04-25T08:36:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.460 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705216 s, 149 MB/s 2024-04-25T08:36:01.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720019 s, 146 MB/s 2024-04-25T08:36:01.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.474 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587324 s, 179 MB/s 2024-04-25T08:36:01.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.483 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704851 s, 149 MB/s 2024-04-25T08:36:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557193 s, 188 MB/s 2024-04-25T08:36:01.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713658 s, 147 MB/s 2024-04-25T08:36:01.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.505 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713573 s, 147 MB/s 2024-04-25T08:36:01.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627347 s, 167 MB/s 2024-04-25T08:36:01.520 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.521 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748816 s, 140 MB/s 2024-04-25T08:36:01.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.528 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607735 s, 173 MB/s 2024-04-25T08:36:01.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708466 s, 148 MB/s 2024-04-25T08:36:01.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.542 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057141 s, 184 MB/s 2024-04-25T08:36:01.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.549 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627392 s, 167 MB/s 2024-04-25T08:36:01.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.557 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630878 s, 166 MB/s 2024-04-25T08:36:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.565 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072368 s, 145 MB/s 2024-04-25T08:36:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.572 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619429 s, 169 MB/s 2024-04-25T08:36:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600417 s, 175 MB/s 2024-04-25T08:36:01.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.587 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724428 s, 145 MB/s 2024-04-25T08:36:01.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.595 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688345 s, 152 MB/s 2024-04-25T08:36:01.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589042 s, 178 MB/s 2024-04-25T08:36:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062746 s, 167 MB/s 2024-04-25T08:36:01.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.617 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.617 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691497 s, 152 MB/s 2024-04-25T08:36:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.623 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597972 s, 175 MB/s 2024-04-25T08:36:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.630 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594835 s, 176 MB/s 2024-04-25T08:36:01.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645042 s, 163 MB/s 2024-04-25T08:36:01.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617491 s, 170 MB/s 2024-04-25T08:36:01.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632759 s, 166 MB/s 2024-04-25T08:36:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604078 s, 174 MB/s 2024-04-25T08:36:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590194 s, 178 MB/s 2024-04-25T08:36:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723471 s, 145 MB/s 2024-04-25T08:36:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.682 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686085 s, 153 MB/s 2024-04-25T08:36:01.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:01.683 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:01.683 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:01.683 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:36:01.683 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:01.683 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:01.683 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:01.684 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.748 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666276 s, 157 MB/s 2024-04-25T08:36:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.755 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596129 s, 176 MB/s 2024-04-25T08:36:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.763 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629493 s, 167 MB/s 2024-04-25T08:36:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.769 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548747 s, 191 MB/s 2024-04-25T08:36:01.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.777 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628961 s, 167 MB/s 2024-04-25T08:36:01.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590218 s, 178 MB/s 2024-04-25T08:36:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.791 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698469 s, 150 MB/s 2024-04-25T08:36:01.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.798 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057628 s, 182 MB/s 2024-04-25T08:36:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.805 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648119 s, 162 MB/s 2024-04-25T08:36:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.812 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643513 s, 163 MB/s 2024-04-25T08:36:01.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.819 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060428 s, 174 MB/s 2024-04-25T08:36:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.827 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711264 s, 147 MB/s 2024-04-25T08:36:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.834 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561127 s, 187 MB/s 2024-04-25T08:36:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.842 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700645 s, 150 MB/s 2024-04-25T08:36:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580999 s, 180 MB/s 2024-04-25T08:36:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.855 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550184 s, 191 MB/s 2024-04-25T08:36:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.862 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612186 s, 171 MB/s 2024-04-25T08:36:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579033 s, 181 MB/s 2024-04-25T08:36:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646578 s, 162 MB/s 2024-04-25T08:36:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619642 s, 169 MB/s 2024-04-25T08:36:01.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.891 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620837 s, 169 MB/s 2024-04-25T08:36:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698738 s, 150 MB/s 2024-04-25T08:36:01.906 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.907 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745802 s, 141 MB/s 2024-04-25T08:36:01.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.913 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549396 s, 191 MB/s 2024-04-25T08:36:01.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564143 s, 186 MB/s 2024-04-25T08:36:01.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627578 s, 167 MB/s 2024-04-25T08:36:01.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703028 s, 149 MB/s 2024-04-25T08:36:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599441 s, 175 MB/s 2024-04-25T08:36:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057523 s, 182 MB/s 2024-04-25T08:36:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704843 s, 149 MB/s 2024-04-25T08:36:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.964 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597685 s, 175 MB/s 2024-04-25T08:36:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:01.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:01.971 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604299 s, 174 MB/s 2024-04-25T08:36:01.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:01.972 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:02.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.036 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602873 s, 174 MB/s 2024-04-25T08:36:02.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.044 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722694 s, 145 MB/s 2024-04-25T08:36:02.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.051 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575203 s, 182 MB/s 2024-04-25T08:36:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704066 s, 149 MB/s 2024-04-25T08:36:02.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.067 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070124 s, 150 MB/s 2024-04-25T08:36:02.074 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072507 s, 145 MB/s 2024-04-25T08:36:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.081 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561046 s, 187 MB/s 2024-04-25T08:36:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.089 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624412 s, 168 MB/s 2024-04-25T08:36:02.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.096 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613087 s, 171 MB/s 2024-04-25T08:36:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633891 s, 165 MB/s 2024-04-25T08:36:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694133 s, 151 MB/s 2024-04-25T08:36:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.117 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572098 s, 183 MB/s 2024-04-25T08:36:02.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602374 s, 174 MB/s 2024-04-25T08:36:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.131 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561588 s, 187 MB/s 2024-04-25T08:36:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.139 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068782 s, 152 MB/s 2024-04-25T08:36:02.147 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.147 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.147 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736605 s, 142 MB/s 2024-04-25T08:36:02.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.155 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715059 s, 147 MB/s 2024-04-25T08:36:02.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549545 s, 191 MB/s 2024-04-25T08:36:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559803 s, 187 MB/s 2024-04-25T08:36:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058364 s, 180 MB/s 2024-04-25T08:36:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643047 s, 163 MB/s 2024-04-25T08:36:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714388 s, 147 MB/s 2024-04-25T08:36:02.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.197 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585854 s, 179 MB/s 2024-04-25T08:36:02.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.206 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722329 s, 145 MB/s 2024-04-25T08:36:02.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.211 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055383 s, 189 MB/s 2024-04-25T08:36:02.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688493 s, 152 MB/s 2024-04-25T08:36:02.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600373 s, 175 MB/s 2024-04-25T08:36:02.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763022 s, 137 MB/s 2024-04-25T08:36:02.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.242 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675176 s, 155 MB/s 2024-04-25T08:36:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694023 s, 151 MB/s 2024-04-25T08:36:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657077 s, 160 MB/s 2024-04-25T08:36:02.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.266 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068515 s, 153 MB/s 2024-04-25T08:36:02.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:02.267 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724246 s, 145 MB/s 2024-04-25T08:36:02.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626019 s, 167 MB/s 2024-04-25T08:36:02.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623616 s, 168 MB/s 2024-04-25T08:36:02.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.354 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614861 s, 171 MB/s 2024-04-25T08:36:02.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556044 s, 189 MB/s 2024-04-25T08:36:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.368 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647603 s, 162 MB/s 2024-04-25T08:36:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576579 s, 182 MB/s 2024-04-25T08:36:02.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614626 s, 171 MB/s 2024-04-25T08:36:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597553 s, 175 MB/s 2024-04-25T08:36:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565674 s, 185 MB/s 2024-04-25T08:36:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559818 s, 187 MB/s 2024-04-25T08:36:02.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542479 s, 193 MB/s 2024-04-25T08:36:02.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.416 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730919 s, 143 MB/s 2024-04-25T08:36:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.424 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658821 s, 159 MB/s 2024-04-25T08:36:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.431 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058567 s, 179 MB/s 2024-04-25T08:36:02.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.438 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622603 s, 168 MB/s 2024-04-25T08:36:02.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.445 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058623 s, 179 MB/s 2024-04-25T08:36:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637191 s, 165 MB/s 2024-04-25T08:36:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.459 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593282 s, 177 MB/s 2024-04-25T08:36:02.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.465 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579959 s, 181 MB/s 2024-04-25T08:36:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.472 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553283 s, 190 MB/s 2024-04-25T08:36:02.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.479 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603602 s, 174 MB/s 2024-04-25T08:36:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.485 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557909 s, 188 MB/s 2024-04-25T08:36:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.492 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556598 s, 188 MB/s 2024-04-25T08:36:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.499 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602906 s, 174 MB/s 2024-04-25T08:36:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.506 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060584 s, 173 MB/s 2024-04-25T08:36:02.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.513 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.513 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594064 s, 177 MB/s 2024-04-25T08:36:02.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.521 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722724 s, 145 MB/s 2024-04-25T08:36:02.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.528 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599758 s, 175 MB/s 2024-04-25T08:36:02.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.535 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623043 s, 168 MB/s 2024-04-25T08:36:02.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727339 s, 144 MB/s 2024-04-25T08:36:02.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:02.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:02.549 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555349 s, 189 MB/s 2024-04-25T08:36:02.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:02.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:36:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.701+0000 7f32d5638640 1 Processor -- start 2024-04-25T08:36:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.701+0000 7f32d5638640 1 -- start start 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.701+0000 7f32d5638640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d00b4f10 0x7f32d00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.701+0000 7f32d5638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00af5e0 0x7f32d006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32d5638640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00b52f0 con 0x7f32d015f630 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32d5638640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d006bc40 con 0x7f32d00b4f10 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32d5638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00adaa0 con 0x7f32d00af5e0 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ceffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f32d00b4f10 0x7f32d00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32cf7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f32d015f630 0x7f32d00b3910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57674/0 (socket says 172.21.15.60:57674) 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32cf7fe640 1 -- 172.21.15.60:0/328358898 learned_addr learned my addr 172.21.15.60:0/328358898 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00af5e0 0x7f32d006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ceffd640 1 -- 172.21.15.60:0/328358898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00af5e0 msgr2=0x7f32d006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ceffd640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00af5e0 0x7f32d006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ceffd640 1 -- 172.21.15.60:0/328358898 >> v1:172.21.15.60:6789/0 conn(0x7f32d015f630 legacy=0x7f32d00b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ceffd640 1 -- 172.21.15.60:0/328358898 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00add80 con 0x7f32d00b4f10 2024-04-25T08:36:02.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.702+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00af5e0 0x7f32d006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32ceffd640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d00b4f10 0x7f32d00b2e10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f32c00097a0 tx=0x7f32c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe7d621f39a19c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32cdffb640 1 -- 172.21.15.60:0/328358898 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0037070 con 0x7f32d00b4f10 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32cdffb640 1 -- 172.21.15.60:0/328358898 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0032440 con 0x7f32d00b4f10 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32cdffb640 1 -- 172.21.15.60:0/328358898 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0032810 con 0x7f32d00b4f10 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 -- 172.21.15.60:0/328358898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d00b4f10 msgr2=0x7f32d00b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d00b4f10 0x7f32d00b2e10 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f32c00097a0 tx=0x7f32c002a180 comp rx=0 tx=0).stop 2024-04-25T08:36:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 -- 172.21.15.60:0/328358898 shutdown_connections 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00af5e0 0x7f32d006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 --2- 172.21.15.60:0/328358898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d00b4f10 0x7f32d00b2e10 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 -- 172.21.15.60:0/328358898 >> 172.21.15.60:0/328358898 conn(0x7f32d00cfec0 msgr2=0x7f32d00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 -- 172.21.15.60:0/328358898 shutdown_connections 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.703+0000 7f32d5638640 1 -- 172.21.15.60:0/328358898 wait complete. 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 Processor -- start 2024-04-25T08:36:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 -- start start 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d00af5e0 0x7f32d0150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00b4f10 0x7f32d0099c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 0x7f32d013b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00ad730 con 0x7f32d015f630 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d006b8d0 con 0x7f32d00af5e0 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.705+0000 7f32d5638640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015fa80 con 0x7f32d00b4f10 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 0x7f32d013b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 0x7f32d013b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51714/0 (socket says 172.21.15.60:51714) 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 -- 172.21.15.60:0/88455228 learned_addr learned my addr 172.21.15.60:0/88455228 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00b4f10 0x7f32d0099c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32ceffd640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d00af5e0 0x7f32d0150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 -- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00b4f10 msgr2=0x7f32d0099c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00b4f10 0x7f32d0099c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 -- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d00af5e0 msgr2=0x7f32d0150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d00af5e0 0x7f32d0150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 -- 172.21.15.60:0/88455228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d014faf0 con 0x7f32d015f630 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 0x7f32d013b6d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f32c800cab0 tx=0x7f32c800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32b77fe640 1 -- 172.21.15.60:0/88455228 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32d015f630 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32b77fe640 1 -- 172.21.15.60:0/88455228 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800ac30 con 0x7f32d015f630 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d014fdd0 con 0x7f32d015f630 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32b77fe640 1 -- 172.21.15.60:0/88455228 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8012400 con 0x7f32d015f630 2024-04-25T08:36:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.706+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0150390 con 0x7f32d015f630 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.708+0000 7f32b77fe640 1 -- 172.21.15.60:0/88455228 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32c801a070 con 0x7f32d015f630 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.708+0000 7f32b77fe640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32bc06b4d0 0x7f32bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.708+0000 7f32b77fe640 1 -- 172.21.15.60:0/88455228 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32c8089f80 con 0x7f32d015f630 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.708+0000 7f32ceffd640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32bc06b4d0 0x7f32bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.709+0000 7f32ceffd640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32bc06b4d0 0x7f32bc06d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32c0034000 tx=0x7f32c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.709+0000 7f32d8412740 1 Processor -- start 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.709+0000 7f32d8412740 1 -- start start 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8ec73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8ee5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32d8412740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 0x5588e8e9e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32d8412740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e8e03000 con 0x5588e8ee6350 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e8dabf20 con 0x5588e8ea13a0 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e8dff0a0 con 0x5588e8ec6ff0 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 0x5588e8e9e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32cf7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 0x5588e8e9e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51728/0 (socket says 172.21.15.60:51728) 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32cf7fe640 1 -- 172.21.15.60:0/3126692299 learned_addr learned my addr 172.21.15.60:0/3126692299 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8ee5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.710+0000 7f32ceffd640 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8ec73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32cf7fe640 1 -- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 msgr2=0x5588e8ec73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8ec73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32cf7fe640 1 -- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 msgr2=0x5588e8ee5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8ee5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32cf7fe640 1 -- 172.21.15.60:0/3126692299 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e8be3ba0 con 0x5588e8ee6350 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 0x5588e8e9e260 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f32c8060740 tx=0x7f32c805c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c025e39aee6f717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32b4ff9640 1 -- 172.21.15.60:0/3126692299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8054c80 con 0x5588e8ee6350 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32b4ff9640 1 -- 172.21.15.60:0/3126692299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c80607e0 con 0x5588e8ee6350 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32b4ff9640 1 -- 172.21.15.60:0/3126692299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8054e20 con 0x5588e8ee6350 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.711+0000 7f32d8412740 1 -- 172.21.15.60:0/3126692299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 msgr2=0x5588e8e9e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 0x5588e8e9e260 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f32c8060740 tx=0x7f32c805c7f0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 -- 172.21.15.60:0/3126692299 shutdown_connections 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ee6350 0x5588e8e9e260 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8ee5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 --2- 172.21.15.60:0/3126692299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8ec73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 -- 172.21.15.60:0/3126692299 >> 172.21.15.60:0/3126692299 conn(0x5588e8e67050 msgr2=0x5588e8ee4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 -- 172.21.15.60:0/3126692299 shutdown_connections 2024-04-25T08:36:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 -- 172.21.15.60:0/3126692299 wait complete. 2024-04-25T08:36:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 -- start start 2024-04-25T08:36:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.712+0000 7f32d8412740 1 Processor -- start 2024-04-25T08:36:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8ec09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8ecca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8ef9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e8e9b800 con 0x5588e8ea13a0 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e8da1610 con 0x5588e8ec6ff0 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e8ea0330 con 0x5588e8ec1640 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32d8412740 10 client.0 _ll_register_callbacks cb 0x5588e8db9f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8ec09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8ec09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51738/0 (socket says 172.21.15.60:51738) 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 -- 172.21.15.60:0/2673757251 learned_addr learned my addr 172.21.15.60:0/2673757251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8ef9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8ecca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 -- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 msgr2=0x5588e8ef9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8ef9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 -- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 msgr2=0x5588e8ecca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8ecca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.713+0000 7f32ceffd640 1 -- 172.21.15.60:0/2673757251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588e8ad7490 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32ceffd640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8ec09a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f32c002a770 tx=0x7f32c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b4c9503d497e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32adffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32adffb640 1 -- 172.21.15.60:0/2673757251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0037040 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32adffb640 1 -- 172.21.15.60:0/2673757251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32c003ed40 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c80607e0 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32adffb640 1 -- 172.21.15.60:0/2673757251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c0056050 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 10 client.7842 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5588e8c44330 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32adffb640 1 -- 172.21.15.60:0/2673757251 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c80607e0 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32adffb640 1 -- 172.21.15.60:0/2673757251 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32c0059020 con 0x5588e8ea13a0 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 10 client.7842 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 10 client.7842 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 20 client.7842 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ecb160 0x5588e8ee7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 4 client.7842 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.714+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588e8e10540 con 0x5588e8ecb160 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ecb160 0x5588e8ee7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32adffb640 10 client.7842 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ecb160 0x5588e8ee7290 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78baa24466fdb962 server_cookie=f8a53ce1edef1083 in_seq=0 out_seq=0 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32adffb640 1 -- 172.21.15.60:0/2673757251 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588e8e10540 con 0x5588e8ecb160 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32adffb640 10 client.7842 handle_command_reply: tid=0 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32d8412740 1 client.7842 shutdown 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 msgr2=0x5588e8ec09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.715+0000 7f32d8412740 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8ec09a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f32c002a770 tx=0x7f32c0004730 comp rx=0 tx=0).stop 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 shutdown_connections 2024-04-25T08:36:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ecb160 0x5588e8ee7290 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8ef9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8ecca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32adffb640 0 client.7842 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 --2- 172.21.15.60:0/2673757251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8ec09a0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 >> 172.21.15.60:0/2673757251 conn(0x5588e8e67050 msgr2=0x5588e8ee8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 shutdown_connections 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 1 -- 172.21.15.60:0/2673757251 wait complete. 2024-04-25T08:36:02.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.716+0000 7f32d8412740 20 client.7842 trim_cache size 0 max 16384 2024-04-25T08:36:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.731+0000 7f32d8412740 1 Processor -- start 2024-04-25T08:36:02.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.731+0000 7f32d8412740 1 -- start start 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32d8412740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8edac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8edb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8f02380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32d8412740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01fc220 con 0x5588e8ea13a0 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01fb3c0 con 0x5588e8ec6ff0 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d006f750 con 0x5588e8ec1640 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8edac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8edac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51740/0 (socket says 172.21.15.60:51740) 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 -- 172.21.15.60:0/3392013551 learned_addr learned my addr 172.21.15.60:0/3392013551 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8edb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8f02380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 -- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 msgr2=0x5588e8edb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8edb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 -- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 msgr2=0x5588e8f02380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8f02380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 -- 172.21.15.60:0/3392013551 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588e8ab0610 con 0x5588e8ea13a0 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32ceffd640 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8edac30 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006ef0 tx=0x7f32c0047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e867ea3d5aeaad28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32affff640 1 -- 172.21.15.60:0/3392013551 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0037040 con 0x5588e8ea13a0 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32affff640 1 -- 172.21.15.60:0/3392013551 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c004e070 con 0x5588e8ea13a0 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.732+0000 7f32affff640 1 -- 172.21.15.60:0/3392013551 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c003c430 con 0x5588e8ea13a0 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 -- 172.21.15.60:0/3392013551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 msgr2=0x5588e8edac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8edac30 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006ef0 tx=0x7f32c0047000 comp rx=0 tx=0).stop 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 -- 172.21.15.60:0/3392013551 shutdown_connections 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ec6ff0 0x5588e8f02380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec1640 0x5588e8edb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 --2- 172.21.15.60:0/3392013551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ea13a0 0x5588e8edac30 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 -- 172.21.15.60:0/3392013551 >> 172.21.15.60:0/3392013551 conn(0x5588e8e67050 msgr2=0x5588e8ee7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 -- 172.21.15.60:0/3392013551 shutdown_connections 2024-04-25T08:36:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 -- 172.21.15.60:0/3392013551 wait complete. 2024-04-25T08:36:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 -- start start 2024-04-25T08:36:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.733+0000 7f32d8412740 1 Processor -- start 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8f546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ec1640 0x5588e8ecc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8e9eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588e8e10540 con 0x5588e8ec1640 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01fadc0 con 0x5588e8ea13a0 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588e8e9cfc0 con 0x5588e8ec6ff0 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32d8412740 10 client.0 _ll_register_callbacks cb 0x7f32c80548b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ec1640 0x5588e8ecc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8f546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ce7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ec1640 0x5588e8ecc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51754/0 (socket says 172.21.15.60:51754) 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ce7fc640 1 -- 172.21.15.60:0/2365542382 learned_addr learned my addr 172.21.15.60:0/2365542382 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32cf7fe640 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8e9eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 -- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 msgr2=0x5588e8e9eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8e9eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 -- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ec1640 msgr2=0x5588e8ecc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ec1640 0x5588e8ecc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 -- 172.21.15.60:0/2365542382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588e8ee71f0 con 0x5588e8ea13a0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32ceffd640 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8f546c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f32c0035040 tx=0x7f32c0009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51dd24dbd3cb9cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32adffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.735+0000 7f32adffb640 1 -- 172.21.15.60:0/2365542382 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c005e030 con 0x5588e8ea13a0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32adffb640 1 -- 172.21.15.60:0/2365542382 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32c003ed50 con 0x5588e8ea13a0 2024-04-25T08:36:02.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c004e070 con 0x5588e8ea13a0 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32adffb640 1 -- 172.21.15.60:0/2365542382 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c0037040 con 0x5588e8ea13a0 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 10 client.7099 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5588e8ee5df0 con 0x5588e8ea13a0 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32adffb640 1 -- 172.21.15.60:0/2365542382 <== mon.1 v2:172.21.15.100:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c004e070 con 0x5588e8ea13a0 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32adffb640 1 -- 172.21.15.60:0/2365542382 <== mon.1 v2:172.21.15.100:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32c0059040 con 0x5588e8ea13a0 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 10 client.7099 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 10 client.7099 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 20 client.7099 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ec0160 0x5588e8e9f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 4 client.7099 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5588e8e0a890 con 0x5588e8ec0160 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.736+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ec0160 0x5588e8e9f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.737+0000 7f32adffb640 10 client.7099 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.737+0000 7f32ce7fc640 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ec0160 0x5588e8e9f0e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac18880cccb876d4 server_cookie=1e5963ea83f84211 in_seq=0 out_seq=0 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32adffb640 1 -- 172.21.15.60:0/2365542382 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5588e8e0a890 con 0x5588e8ec0160 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32adffb640 10 client.7099 handle_command_reply: tid=0 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 client.7099 shutdown 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 msgr2=0x5588e8f546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8f546c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f32c0035040 tx=0x7f32c0009190 comp rx=0 tx=0).stop 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 shutdown_connections 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5588e8ec0160 0x5588e8e9f0e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5588e8ec6ff0 0x5588e8e9eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5588e8ec1640 0x5588e8ecc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 --2- 172.21.15.60:0/2365542382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5588e8ea13a0 0x5588e8f546c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32adffb640 0 client.7099 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 >> 172.21.15.60:0/2365542382 conn(0x5588e8e67050 msgr2=0x5588e8eccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 shutdown_connections 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 1 -- 172.21.15.60:0/2365542382 wait complete. 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.740+0000 7f32d8412740 20 client.7099 trim_cache size 0 max 16384 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "5846264e-8e58-4815-aec3-b62d634809c2", 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:36:02.738 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32bc06b4d0 msgr2=0x7f32bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32bc06b4d0 0x7f32bc06d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32c0034000 tx=0x7f32c002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 msgr2=0x7f32d013b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 0x7f32d013b6d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f32c800cab0 tx=0x7f32c800cf70 comp rx=0 tx=0).stop 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 shutdown_connections 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32d015f630 0x7f32d013b6d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32d00b4f10 0x7f32d0099c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f32bc06b4d0 0x7f32bc06d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 --2- 172.21.15.60:0/88455228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32d00af5e0 0x7f32d0150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 >> 172.21.15.60:0/88455228 conn(0x7f32d00cfec0 msgr2=0x7f32d015fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 shutdown_connections 2024-04-25T08:36:02.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.741+0000 7f32d5638640 1 -- 172.21.15.60:0/88455228 wait complete. 2024-04-25T08:36:02.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:02.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T08:36:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.856+0000 7fa92a84c640 1 Processor -- start 2024-04-25T08:36:02.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.856+0000 7fa92a84c640 1 -- start start 2024-04-25T08:36:02.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa92a84c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9240ce7b0 0x7fa924161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa92a84c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92406c8d0 con 0x7fa9241616f0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa92a84c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9240c6960 con 0x7fa9241610f0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa92a84c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241614d0 con 0x7fa9240ce7b0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa9237fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa9241616f0 0x7fa9240ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57682/0 (socket says 172.21.15.60:57682) 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa9237fe640 1 -- 172.21.15.60:0/1581914337 learned_addr learned my addr 172.21.15.60:0/1581914337 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.857+0000 7fa928de9640 1 --2- 172.21.15.60:0/1581914337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9240ce7b0 0x7fa924161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943450313 0 0) 0x7fa92406c8d0 con 0x7fa9241616f0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa90c003520 con 0x7fa9241616f0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877882487 0 0) 0x7fa9240c6960 con 0x7fa9241610f0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92406c8d0 con 0x7fa9241610f0 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa928de9640 1 -- 172.21.15.60:0/1581914337 >> v1:172.21.15.100:6789/0 conn(0x7fa9241610f0 legacy=0x7fa92406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa928de9640 1 -- 172.21.15.60:0/1581914337 >> v1:172.21.15.60:6789/0 conn(0x7fa9241616f0 legacy=0x7fa9240ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.858+0000 7fa928de9640 1 -- 172.21.15.60:0/1581914337 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa924162260 con 0x7fa9240ce7b0 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa928de9640 1 --2- 172.21.15.60:0/1581914337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9240ce7b0 0x7fa924161b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009dc0 tx=0x7fa91c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d1224d35055e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c02ae60 con 0x7fa9240ce7b0 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c007740 con 0x7fa9240ce7b0 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa922ffd640 1 -- 172.21.15.60:0/1581914337 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c007b10 con 0x7fa9240ce7b0 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa92a84c640 1 -- 172.21.15.60:0/1581914337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9240ce7b0 msgr2=0x7fa924161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa92a84c640 1 --2- 172.21.15.60:0/1581914337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9240ce7b0 0x7fa924161b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009dc0 tx=0x7fa91c02a680 comp rx=0 tx=0).stop 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa92a84c640 1 -- 172.21.15.60:0/1581914337 shutdown_connections 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa92a84c640 1 --2- 172.21.15.60:0/1581914337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9240ce7b0 0x7fa924161b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.859+0000 7fa92a84c640 1 -- 172.21.15.60:0/1581914337 >> 172.21.15.60:0/1581914337 conn(0x7fa92406f500 msgr2=0x7fa92406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.860+0000 7fa92a84c640 1 -- 172.21.15.60:0/1581914337 shutdown_connections 2024-04-25T08:36:02.855 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.860+0000 7fa92a84c640 1 -- 172.21.15.60:0/1581914337 wait complete. 2024-04-25T08:36:02.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.861+0000 7fa92a84c640 1 Processor -- start 2024-04-25T08:36:02.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.861+0000 7fa92a84c640 1 -- start start 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa92a84c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9240ce7b0 0x7fa9240b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa92a84c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9241610f0 0x7fa924157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa92a84c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 0x7fa924156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa92a84c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240cfce0 con 0x7fa9241616f0 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa92a84c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240cfa00 con 0x7fa9240ce7b0 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa92a84c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240cf690 con 0x7fa9241610f0 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 0x7fa924156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 0x7fa924156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51758/0 (socket says 172.21.15.60:51758) 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 -- 172.21.15.60:0/2061042157 learned_addr learned my addr 172.21.15.60:0/2061042157 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa923fff640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9240ce7b0 0x7fa9240b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa9237fe640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9241610f0 0x7fa924157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 -- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9241610f0 msgr2=0x7fa924157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9241610f0 0x7fa924157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 -- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9240ce7b0 msgr2=0x7fa9240b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9240ce7b0 0x7fa9240b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.862+0000 7fa928de9640 1 -- 172.21.15.60:0/2061042157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9241584c0 con 0x7fa9241616f0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.863+0000 7fa928de9640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 0x7fa924156560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009ee0 tx=0x7fa91c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.863+0000 7fa920ff9640 1 -- 172.21.15.60:0/2061042157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c040070 con 0x7fa9241616f0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.863+0000 7fa920ff9640 1 -- 172.21.15.60:0/2061042157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c02ae60 con 0x7fa9241616f0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.863+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9241587a0 con 0x7fa9241616f0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.863+0000 7fa920ff9640 1 -- 172.21.15.60:0/2061042157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c031be0 con 0x7fa9241616f0 2024-04-25T08:36:02.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.863+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa924158d60 con 0x7fa9241616f0 2024-04-25T08:36:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.865+0000 7fa920ff9640 1 -- 172.21.15.60:0/2061042157 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa91c039040 con 0x7fa9241616f0 2024-04-25T08:36:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.865+0000 7fa920ff9640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.865+0000 7fa920ff9640 1 -- 172.21.15.60:0/2061042157 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa91c0ab910 con 0x7fa9241616f0 2024-04-25T08:36:02.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.866+0000 7fa923fff640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.866+0000 7fa92d61f740 1 Processor -- start 2024-04-25T08:36:02.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.866+0000 7fa92d61f740 1 -- start start 2024-04-25T08:36:02.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.866+0000 7fa923fff640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa91006b4d0 0x7fa91006d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa914001840 tx=0x7fa914009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 0x55bb15e48e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e83370 0x55bb15e47080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e71be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa92d61f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb15da9b90 con 0x55bb15e46ca0 2024-04-25T08:36:02.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb15d52e40 con 0x55bb15e83370 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb15da5c30 con 0x55bb15e71760 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 0x55bb15e48e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 0x55bb15e48e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51772/0 (socket says 172.21.15.60:51772) 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 -- 172.21.15.60:0/837930857 learned_addr learned my addr 172.21.15.60:0/837930857 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa9237fe640 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e83370 0x55bb15e47080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa928de9640 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e71be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 -- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 msgr2=0x55bb15e71be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e71be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 -- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e83370 msgr2=0x55bb15e47080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e83370 0x55bb15e47080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 -- 172.21.15.60:0/837930857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb15b887d0 con 0x55bb15e46ca0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.867+0000 7fa923fff640 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 0x55bb15e48e00 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa91400fb50 tx=0x7fa914012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb5ec93baa89a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa909ffb640 1 -- 172.21.15.60:0/837930857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914025070 con 0x55bb15e46ca0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa909ffb640 1 -- 172.21.15.60:0/837930857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91401b050 con 0x55bb15e46ca0 2024-04-25T08:36:02.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa909ffb640 1 -- 172.21.15.60:0/837930857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914014de0 con 0x55bb15e46ca0 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 -- 172.21.15.60:0/837930857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 msgr2=0x55bb15e48e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 0x55bb15e48e00 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa91400fb50 tx=0x7fa914012000 comp rx=0 tx=0).stop 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 -- 172.21.15.60:0/837930857 shutdown_connections 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e71be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e83370 0x55bb15e47080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 --2- 172.21.15.60:0/837930857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e46ca0 0x55bb15e48e00 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 -- 172.21.15.60:0/837930857 >> 172.21.15.60:0/837930857 conn(0x55bb15e0e870 msgr2=0x55bb15e8c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 -- 172.21.15.60:0/837930857 shutdown_connections 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.868+0000 7fa92d61f740 1 -- 172.21.15.60:0/837930857 wait complete. 2024-04-25T08:36:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.869+0000 7fa92d61f740 1 -- start start 2024-04-25T08:36:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.869+0000 7fa92d61f740 1 Processor -- start 2024-04-25T08:36:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e75040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e89160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e62c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb15d663e0 con 0x55bb15e83370 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb15e6f2b0 con 0x55bb15e46ca0 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb15e9d400 con 0x55bb15e71760 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa92d61f740 10 client.0 _ll_register_callbacks cb 0x55bb15d65db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa928de9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e62c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa923fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e75040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa928de9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e62c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51778/0 (socket says 172.21.15.60:51778) 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa928de9640 1 -- 172.21.15.60:0/1817985528 learned_addr learned my addr 172.21.15.60:0/1817985528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa9237fe640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e89160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa923fff640 1 -- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 msgr2=0x55bb15e89160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.870+0000 7fa923fff640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e89160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa923fff640 1 -- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 msgr2=0x55bb15e62c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa923fff640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e62c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa923fff640 1 -- 172.21.15.60:0/1817985528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb15a7e0f0 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa928de9640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e62c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa923fff640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e75040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa91400fb50 tx=0x7fa914007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94bfd71e2a8aeccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa902ffd640 1 -- 172.21.15.60:0/1817985528 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914031060 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa902ffd640 1 -- 172.21.15.60:0/1817985528 <== mon.1 v2:172.21.15.100:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa914020a40 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa91401b050 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa902ffd640 1 -- 172.21.15.60:0/1817985528 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa914037050 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 10 client.7111 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55bb15be8f60 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa902ffd640 1 -- 172.21.15.60:0/1817985528 <== mon.1 v2:172.21.15.100:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91401b050 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa902ffd640 1 -- 172.21.15.60:0/1817985528 <== mon.1 v2:172.21.15.100:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa91403a020 con 0x55bb15e46ca0 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 10 client.7111 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:02.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 10 client.7111 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 20 client.7111 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e63180 0x55bb15e9c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 4 client.7111 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.871+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb15db70d0 con 0x55bb15e63180 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.872+0000 7fa9237fe640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e63180 0x55bb15e9c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.872+0000 7fa902ffd640 10 client.7111 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.872+0000 7fa9237fe640 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e63180 0x55bb15e9c4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e932e39a7347260 server_cookie=cf36e0877ac5a2b5 in_seq=0 out_seq=0 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa902ffd640 1 -- 172.21.15.60:0/1817985528 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bb15db70d0 con 0x55bb15e63180 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa902ffd640 10 client.7111 handle_command_reply: tid=0 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 client.7111 shutdown 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 msgr2=0x55bb15e75040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e75040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa91400fb50 tx=0x7fa914007d10 comp rx=0 tx=0).stop 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 shutdown_connections 2024-04-25T08:36:02.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e63180 0x55bb15e9c4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e62c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e89160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 --2- 172.21.15.60:0/1817985528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e75040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa902ffd640 0 client.7111 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.873+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 >> 172.21.15.60:0/1817985528 conn(0x55bb15e0e870 msgr2=0x55bb15e6eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.874+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 shutdown_connections 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.874+0000 7fa92d61f740 1 -- 172.21.15.60:0/1817985528 wait complete. 2024-04-25T08:36:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.874+0000 7fa92d61f740 20 client.7111 trim_cache size 0 max 16384 2024-04-25T08:36:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 Processor -- start 2024-04-25T08:36:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 -- start start 2024-04-25T08:36:02.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e6d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa924200440 con 0x55bb15e83370 2024-04-25T08:36:02.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9241ff670 con 0x55bb15e46ca0 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240b5f70 con 0x55bb15e71760 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa928de9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa928de9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51782/0 (socket says 172.21.15.60:51782) 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa928de9640 1 -- 172.21.15.60:0/780678249 learned_addr learned my addr 172.21.15.60:0/780678249 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.889+0000 7fa923fff640 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e6d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa9237fe640 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa928de9640 1 -- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 msgr2=0x55bb15e86c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa928de9640 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa928de9640 1 -- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 msgr2=0x55bb15e6d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa928de9640 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e6d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa928de9640 1 -- 172.21.15.60:0/780678249 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb15a571b0 con 0x55bb15e83370 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa923fff640 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e6d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa928de9640 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa91c079e20 tx=0x7fa91c009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5942781ed0b38f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa9097fa640 1 -- 172.21.15.60:0/780678249 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c096070 con 0x55bb15e83370 2024-04-25T08:36:02.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa9097fa640 1 -- 172.21.15.60:0/780678249 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c091030 con 0x55bb15e83370 2024-04-25T08:36:02.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.890+0000 7fa9097fa640 1 -- 172.21.15.60:0/780678249 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c04a040 con 0x55bb15e83370 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 -- 172.21.15.60:0/780678249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 msgr2=0x55bb15e87160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa91c079e20 tx=0x7fa91c009af0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 -- 172.21.15.60:0/780678249 shutdown_connections 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 --2- 172.21.15.60:0/780678249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e46ca0 0x55bb15e6d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 -- 172.21.15.60:0/780678249 >> 172.21.15.60:0/780678249 conn(0x55bb15e0e870 msgr2=0x55bb15e6eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 -- 172.21.15.60:0/780678249 shutdown_connections 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.891+0000 7fa92d61f740 1 -- 172.21.15.60:0/780678249 wait complete. 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.892+0000 7fa92d61f740 1 -- start start 2024-04-25T08:36:02.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.892+0000 7fa92d61f740 1 Processor -- start 2024-04-25T08:36:02.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 0x55bb15ef3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb15e88b10 con 0x55bb15e83370 2024-04-25T08:36:02.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240b6570 con 0x55bb15e71760 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb15ef9140 con 0x55bb15e46ca0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa92d61f740 10 client.0 _ll_register_callbacks cb 0x7fa914023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 0x55bb15ef3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa9237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 0x55bb15ef3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56182/0 (socket says 172.21.15.60:56182) 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 -- 172.21.15.60:0/2813732202 learned_addr learned my addr 172.21.15.60:0/2813732202 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa928de9640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 -- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e71760 msgr2=0x55bb15e86c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 -- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 msgr2=0x55bb15e87160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa923fff640 1 -- 172.21.15.60:0/2813732202 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb15e3d370 con 0x55bb15e46ca0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.894+0000 7fa928de9640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa923fff640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 0x55bb15ef3db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa91400fb50 tx=0x7fa91400a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18c73e851ec9015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:02.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3301/0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa902ffd640 1 -- 172.21.15.60:0/2813732202 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914025040 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa902ffd640 1 -- 172.21.15.60:0/2813732202 <== mon.2 v2:172.21.15.100:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa914007440 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa91c091030 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa902ffd640 1 -- 172.21.15.60:0/2813732202 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa914048050 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 10 client.6350 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55bb15e74410 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa902ffd640 1 -- 172.21.15.60:0/2813732202 <== mon.2 v2:172.21.15.100:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c091030 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa902ffd640 1 -- 172.21.15.60:0/2813732202 <== mon.2 v2:172.21.15.100:3301/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa91404b020 con 0x55bb15e46ca0 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 10 client.6350 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 10 client.6350 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 20 client.6350 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e6f040 0x55bb15e9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:02.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 4 client.6350 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T08:36:02.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.895+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55bb15db1420 con 0x55bb15e6f040 2024-04-25T08:36:02.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.896+0000 7fa9237fe640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e6f040 0x55bb15e9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:02.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.896+0000 7fa9237fe640 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e6f040 0x55bb15e9b050 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4441d5956b7bc69 server_cookie=9564bdc43c3ccd16 in_seq=0 out_seq=0 2024-04-25T08:36:02.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.896+0000 7fa902ffd640 10 client.6350 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa902ffd640 1 -- 172.21.15.60:0/2813732202 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55bb15db1420 con 0x55bb15e6f040 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa902ffd640 10 client.6350 handle_command_reply: tid=0 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 client.6350 shutdown 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 msgr2=0x55bb15ef3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 0x55bb15ef3db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa91400fb50 tx=0x7fa91400a000 comp rx=0 tx=0).stop 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 shutdown_connections 2024-04-25T08:36:02.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55bb15e6f040 0x55bb15e9b050 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55bb15e83370 0x55bb15e87160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bb15e71760 0x55bb15e86c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 --2- 172.21.15.60:0/2813732202 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55bb15e46ca0 0x55bb15ef3db0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa902ffd640 0 client.6350 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 >> 172.21.15.60:0/2813732202 conn(0x55bb15e0e870 msgr2=0x55bb15e893c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 shutdown_connections 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 1 -- 172.21.15.60:0/2813732202 wait complete. 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.905+0000 7fa92d61f740 20 client.6350 trim_cache size 0 max 16384 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:02.901 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:36:02.902 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:02.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa91006b4d0 msgr2=0x7fa91006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa91006b4d0 0x7fa91006d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa914001840 tx=0x7fa914009040 comp rx=0 tx=0).stop 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 msgr2=0x7fa924156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 0x7fa924156560 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009ee0 tx=0x7fa91c004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 shutdown_connections 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9241616f0 0x7fa924156560 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9241610f0 0x7fa924157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 --2- 172.21.15.60:0/2061042157 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9240ce7b0 0x7fa9240b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 >> 172.21.15.60:0/2061042157 conn(0x7fa92406f500 msgr2=0x7fa9240d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 shutdown_connections 2024-04-25T08:36:02.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:02.906+0000 7fa92a84c640 1 -- 172.21.15.60:0/2061042157 wait complete. 2024-04-25T08:36:02.911 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:36:02.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:36:03.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.019+0000 7fa025cf7640 1 Processor -- start 2024-04-25T08:36:03.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.019+0000 7fa025cf7640 1 -- start start 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa025cf7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa02015da90 0x7fa020161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa025cf7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa0200b9750 0x7fa0200ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa025cf7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa020161ba0 0x7fa0200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa025cf7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa020162600 con 0x7fa020161ba0 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa025cf7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa020162040 con 0x7fa0200b9750 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa025cf7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa020162320 con 0x7fa02015da90 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa020161ba0 0x7fa0200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa020161ba0 0x7fa0200cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51806/0 (socket says 172.21.15.60:51806) 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 -- 172.21.15.60:0/193298883 learned_addr learned my addr 172.21.15.60:0/193298883 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01effd640 1 --2- 172.21.15.60:0/193298883 >> v2:172.21.15.100:3300/0 conn(0x7fa0200b9750 0x7fa0200ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/193298883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa02015da90 0x7fa020161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 -- 172.21.15.60:0/193298883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa02015da90 msgr2=0x7fa020161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 --2- 172.21.15.60:0/193298883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa02015da90 0x7fa020161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 -- 172.21.15.60:0/193298883 >> v2:172.21.15.100:3300/0 conn(0x7fa0200b9750 msgr2=0x7fa0200ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 --2- 172.21.15.60:0/193298883 >> v2:172.21.15.100:3300/0 conn(0x7fa0200b9750 0x7fa0200ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.020+0000 7fa01ffff640 1 -- 172.21.15.60:0/193298883 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200d1b10 con 0x7fa020161ba0 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.021+0000 7fa01ffff640 1 --2- 172.21.15.60:0/193298883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa020161ba0 0x7fa0200cebb0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa02006d130 tx=0x7fa01802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d926fd2fbabee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.021+0000 7fa01e7fc640 1 -- 172.21.15.60:0/193298883 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018036070 con 0x7fa020161ba0 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.021+0000 7fa01e7fc640 1 -- 172.21.15.60:0/193298883 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018031600 con 0x7fa020161ba0 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.021+0000 7fa01e7fc640 1 -- 172.21.15.60:0/193298883 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180319d0 con 0x7fa020161ba0 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 -- 172.21.15.60:0/193298883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa020161ba0 msgr2=0x7fa0200cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 --2- 172.21.15.60:0/193298883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa020161ba0 0x7fa0200cebb0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa02006d130 tx=0x7fa01802a610 comp rx=0 tx=0).stop 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 -- 172.21.15.60:0/193298883 shutdown_connections 2024-04-25T08:36:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 --2- 172.21.15.60:0/193298883 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa020161ba0 0x7fa0200cebb0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 --2- 172.21.15.60:0/193298883 >> v2:172.21.15.100:3300/0 conn(0x7fa0200b9750 0x7fa0200ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 --2- 172.21.15.60:0/193298883 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa02015da90 0x7fa020161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 -- 172.21.15.60:0/193298883 >> 172.21.15.60:0/193298883 conn(0x7fa020061a20 msgr2=0x7fa020061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 -- 172.21.15.60:0/193298883 shutdown_connections 2024-04-25T08:36:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.022+0000 7fa025cf7640 1 -- 172.21.15.60:0/193298883 wait complete. 2024-04-25T08:36:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.023+0000 7fa025cf7640 1 Processor -- start 2024-04-25T08:36:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.023+0000 7fa025cf7640 1 -- start start 2024-04-25T08:36:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa025cf7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 0x7fa0201521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa025cf7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa020161ba0 0x7fa0200c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa025cf7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0200b5460 0x7fa0200c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa025cf7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02006c120 con 0x7fa02015da90 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa025cf7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02015de70 con 0x7fa0200b5460 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa025cf7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02006c2a0 con 0x7fa020161ba0 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 0x7fa0201521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 0x7fa0201521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51810/0 (socket says 172.21.15.60:51810) 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2032133378 learned_addr learned my addr 172.21.15.60:0/2032133378 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa020161ba0 0x7fa0200c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01ffff640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0200b5460 0x7fa0200c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa020161ba0 msgr2=0x7fa0200c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa020161ba0 0x7fa0200c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0200b5460 msgr2=0x7fa0200c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0200b5460 0x7fa0200c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2032133378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200b5910 con 0x7fa02015da90 2024-04-25T08:36:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.024+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 0x7fa0201521d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9b0 tx=0x7fa01000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.025+0000 7fa007fff640 1 -- 172.21.15.60:0/2032133378 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010017070 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.025+0000 7fa007fff640 1 -- 172.21.15.60:0/2032133378 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01000c9a0 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.025+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0200c98e0 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.025+0000 7fa007fff640 1 -- 172.21.15.60:0/2032133378 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cd90 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.025+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0200c76c0 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.026+0000 7fa007fff640 1 -- 172.21.15.60:0/2032133378 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa01001a070 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.027+0000 7fa007fff640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa00c06b500 0x7fa00c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.027+0000 7fa007fff640 1 -- 172.21.15.60:0/2032133378 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa01008a620 con 0x7fa02015da90 2024-04-25T08:36:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.027+0000 7fa01effd640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa00c06b500 0x7fa00c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.027+0000 7fa01effd640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa00c06b500 0x7fa00c06d9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa0200c6ff0 tx=0x7fa014009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.028+0000 7fa0289f0740 1 Processor -- start 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.028+0000 7fa0289f0740 1 -- start start 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774eced320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774eced860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774ecbd770 0x55774ecc0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa0289f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ec26da0 con 0x55774eceaef0 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ebcfb80 con 0x55774ecbd770 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774ec22e40 con 0x55774eceab10 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774eced860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774eced860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51822/0 (socket says 172.21.15.60:51822) 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 -- 172.21.15.60:0/3244201724 learned_addr learned my addr 172.21.15.60:0/3244201724 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:03.030 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774eced320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01ffff640 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774ecbd770 0x55774ecc0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 -- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 msgr2=0x55774eced320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774eced320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 -- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774ecbd770 msgr2=0x55774ecc0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774ecbd770 0x55774ecc0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 -- 172.21.15.60:0/3244201724 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774ebd9f10 con 0x55774eceaef0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.029+0000 7fa01effd640 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774eced860 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa014006a90 tx=0x7fa014014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2638dc3c37ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0057fa640 1 -- 172.21.15.60:0/3244201724 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01401b070 con 0x55774eceaef0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0057fa640 1 -- 172.21.15.60:0/3244201724 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014027070 con 0x55774eceaef0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0057fa640 1 -- 172.21.15.60:0/3244201724 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014003a40 con 0x55774eceaef0 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 -- 172.21.15.60:0/3244201724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 msgr2=0x55774eced860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774eced860 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa014006a90 tx=0x7fa014014000 comp rx=0 tx=0).stop 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 -- 172.21.15.60:0/3244201724 shutdown_connections 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774ecbd770 0x55774ecc0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774eced860 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3244201724 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774eced320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 -- 172.21.15.60:0/3244201724 >> 172.21.15.60:0/3244201724 conn(0x55774ecbf9e0 msgr2=0x55774ed08320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 -- 172.21.15.60:0/3244201724 shutdown_connections 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.030+0000 7fa0289f0740 1 -- 172.21.15.60:0/3244201724 wait complete. 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.031+0000 7fa0289f0740 1 -- start start 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.031+0000 7fa0289f0740 1 Processor -- start 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ed0b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed0b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ecc2b90 con 0x55774ecee6b0 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ebc54b0 con 0x55774eceaef0 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774ecc2e70 con 0x55774eceab10 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa0289f0740 10 client.0 _ll_register_callbacks cb 0x55774ec05180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:03.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed0b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ed0b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:56220/0 (socket says 172.21.15.60:56220) 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2899592014 learned_addr learned my addr 172.21.15.60:0/2899592014 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 msgr2=0x55774ed0b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ed0b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 msgr2=0x55774ed0b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed0b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 -- 172.21.15.60:0/2899592014 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774ea057d0 con 0x55774eceab10 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0e0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0100589b0 tx=0x7fa010058cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2c25d2901fb1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.032+0000 7f9ffe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3301/0 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/2899592014 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010070020 con 0x55774eceab10 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/2899592014 <== mon.2 v2:172.21.15.100:3301/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa01006cdc0 con 0x55774eceab10 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa014027070 con 0x55774eceab10 2024-04-25T08:36:03.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/2899592014 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa010075050 con 0x55774eceab10 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 10 client.6362 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55774e93b3f0 con 0x55774eceab10 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/2899592014 <== mon.2 v2:172.21.15.100:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014027070 con 0x55774eceab10 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/2899592014 <== mon.2 v2:172.21.15.100:3301/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa010058100 con 0x55774eceab10 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 10 client.6362 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 10 client.6362 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 20 client.6362 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ecef510 0x55774ecef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 4 client.6362 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55774ec342e0 con 0x55774ecef510 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa01effd640 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ecef510 0x55774ecef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7f9ffe7fc640 10 client.6362 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.033+0000 7fa01effd640 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ecef510 0x55774ecef960 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b5ca29d900546c server_cookie=ecc12c9bc9ae3c6f in_seq=0 out_seq=0 2024-04-25T08:36:03.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/2899592014 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55774ec342e0 con 0x55774ecef510 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7f9ffe7fc640 10 client.6362 handle_command_reply: tid=0 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 client.6362 shutdown 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 msgr2=0x55774ed0e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0e0b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa0100589b0 tx=0x7fa010058cb0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 shutdown_connections 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ecef510 0x55774ecef960 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed0b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ed0b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7f9ffe7fc640 0 client.6362 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 --2- 172.21.15.60:0/2899592014 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0e0b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 >> 172.21.15.60:0/2899592014 conn(0x55774ecbd770 msgr2=0x55774eced2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 shutdown_connections 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 1 -- 172.21.15.60:0/2899592014 wait complete. 2024-04-25T08:36:03.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.034+0000 7fa0289f0740 20 client.6362 trim_cache size 0 max 16384 2024-04-25T08:36:03.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 Processor -- start 2024-04-25T08:36:03.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 -- start start 2024-04-25T08:36:03.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceab10 0x55774eced500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774ecbc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774ecee6b0 0x55774ecbcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0202004f0 con 0x55774eceaef0 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201ff6f0 con 0x55774eceab10 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.050+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0200b3f40 con 0x55774ecee6b0 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774ecbc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774ecbc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51850/0 (socket says 172.21.15.60:51850) 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 -- 172.21.15.60:0/1515692533 learned_addr learned my addr 172.21.15.60:0/1515692533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceab10 0x55774eced500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01ffff640 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774ecee6b0 0x55774ecbcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 -- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774ecee6b0 msgr2=0x55774ecbcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774ecee6b0 0x55774ecbcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 -- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceab10 msgr2=0x55774eced500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceab10 0x55774eced500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 -- 172.21.15.60:0/1515692533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774e8fb0f0 con 0x55774eceaef0 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceab10 0x55774eced500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa01effd640 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774ecbc600 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x55774ecc5070 tx=0x7fa014016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bc9e7ed5a3eb7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa004ff9640 1 -- 172.21.15.60:0/1515692533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01401b040 con 0x55774eceaef0 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa004ff9640 1 -- 172.21.15.60:0/1515692533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01403d070 con 0x55774eceaef0 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.051+0000 7fa004ff9640 1 -- 172.21.15.60:0/1515692533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014037820 con 0x55774eceaef0 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 -- 172.21.15.60:0/1515692533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 msgr2=0x55774ecbc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774ecbc600 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x55774ecc5070 tx=0x7fa014016000 comp rx=0 tx=0).stop 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 -- 172.21.15.60:0/1515692533 shutdown_connections 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774ecee6b0 0x55774ecbcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774eceaef0 0x55774ecbc600 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 --2- 172.21.15.60:0/1515692533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceab10 0x55774eced500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 -- 172.21.15.60:0/1515692533 >> 172.21.15.60:0/1515692533 conn(0x55774ecbd770 msgr2=0x55774ecc37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 -- 172.21.15.60:0/1515692533 shutdown_connections 2024-04-25T08:36:03.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 -- 172.21.15.60:0/1515692533 wait complete. 2024-04-25T08:36:03.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 -- start start 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.052+0000 7fa0289f0740 1 Processor -- start 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ecf0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed71740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774ec292b0 con 0x55774ecee6b0 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201ff170 con 0x55774eceaef0 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774ecef960 con 0x55774eceab10 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa0289f0740 10 client.0 _ll_register_callbacks cb 0x7fa0140095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed71740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa01ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed71740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51864/0 (socket says 172.21.15.60:51864) 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa01ffff640 1 -- 172.21.15.60:0/3032228017 learned_addr learned my addr 172.21.15.60:0/3032228017 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.053+0000 7fa01effd640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ecf0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa01ffff640 1 -- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 msgr2=0x55774ed0a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa01ffff640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa01ffff640 1 -- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 msgr2=0x55774ecf0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa01ffff640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ecf0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa01ffff640 1 -- 172.21.15.60:0/3032228017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774ed71c80 con 0x55774ecee6b0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa01ffff640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed71740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa018009f40 tx=0x7fa01802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7744cfeeb087e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7f9ffe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/3032228017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01804c060 con 0x55774ecee6b0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/3032228017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa01803ddf0 con 0x55774ecee6b0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa01403d070 con 0x55774ecee6b0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/3032228017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa018052050 con 0x55774ecee6b0 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 10 client.7884 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:03.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55774ece3420 con 0x55774ecee6b0 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/3032228017 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01403d070 con 0x55774ecee6b0 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/3032228017 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa018055020 con 0x55774ecee6b0 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 10 client.7884 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 10 client.7884 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 20 client.7884 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ece2920 0x55774ece2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 4 client.7884 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.054+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55774ec2e630 con 0x55774ece2920 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ece2920 0x55774ece2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7f9ffe7fc640 10 client.7884 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa01f7fe640 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ece2920 0x55774ece2d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4356c7cadf53abb4 server_cookie=b3b69dd0658e154d in_seq=0 out_seq=0 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7f9ffe7fc640 1 -- 172.21.15.60:0/3032228017 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55774ec2e630 con 0x55774ece2920 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7f9ffe7fc640 10 client.7884 handle_command_reply: tid=0 2024-04-25T08:36:03.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 client.7884 shutdown 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 msgr2=0x55774ed71740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed71740 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fa018009f40 tx=0x7fa01802c000 comp rx=0 tx=0).stop 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 shutdown_connections 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x55774ece2920 0x55774ece2d30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55774ecee6b0 0x55774ed71740 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55774eceaef0 0x55774ecf0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7f9ffe7fc640 0 client.7884 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 --2- 172.21.15.60:0/3032228017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55774eceab10 0x55774ed0a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 >> 172.21.15.60:0/3032228017 conn(0x55774ecbd770 msgr2=0x55774ed29910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 shutdown_connections 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 1 -- 172.21.15.60:0/3032228017 wait complete. 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.055+0000 7fa0289f0740 20 client.7884 trim_cache size 0 max 16384 2024-04-25T08:36:03.056 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: "5846264e-8e58-4815-aec3-b62d634809c2": { 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "5846264e-8e58-4815-aec3-b62d634809c2", 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:03.057 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa00c06b500 msgr2=0x7fa00c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa00c06b500 0x7fa00c06d9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa0200c6ff0 tx=0x7fa014009190 comp rx=0 tx=0).stop 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 msgr2=0x7fa0201521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 0x7fa0201521d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9b0 tx=0x7fa01000ee70 comp rx=0 tx=0).stop 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 shutdown_connections 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa00c06b500 0x7fa00c06d9c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0200b5460 0x7fa0200c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa020161ba0 0x7fa0200c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.058 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 --2- 172.21.15.60:0/2032133378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa02015da90 0x7fa0201521d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 >> 172.21.15.60:0/2032133378 conn(0x7fa020061a20 msgr2=0x7fa0200d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 shutdown_connections 2024-04-25T08:36:03.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:03.056+0000 7fa025cf7640 1 -- 172.21.15.60:0/2032133378 wait complete. 2024-04-25T08:36:08.062 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:36:08.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:36:08.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.184+0000 7f6c5dc5a640 1 Processor -- start 2024-04-25T08:36:08.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.184+0000 7f6c5dc5a640 1 -- start start 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c5dc5a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c5815dac0 0x7f6c580d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c5dc5a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c580d2420 con 0x7f6c580d0ab0 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c5dc5a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c580d2700 con 0x7f6c580d1f30 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c5dc5a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580d3030 con 0x7f6c5815dac0 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c577fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c5815dac0 0x7f6c580d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c57fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6c580d0ab0 0x7f6c580b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57696/0 (socket says 172.21.15.60:57696) 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c57fff640 1 -- 172.21.15.60:0/1211060377 learned_addr learned my addr 172.21.15.60:0/1211060377 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c577fe640 1 -- 172.21.15.60:0/1211060377 >> v1:172.21.15.100:6789/0 conn(0x7f6c580d1f30 legacy=0x7f6c580d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c577fe640 1 -- 172.21.15.60:0/1211060377 >> v1:172.21.15.60:6789/0 conn(0x7f6c580d0ab0 legacy=0x7f6c580b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c577fe640 1 -- 172.21.15.60:0/1211060377 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580d29e0 con 0x7f6c5815dac0 2024-04-25T08:36:08.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.185+0000 7f6c567fc640 1 -- 172.21.15.60:0/1211060377 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598587262 0 0) 0x7f6c580d2420 con 0x7f6c580d0ab0 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.186+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1211060377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580d19f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6c480097a0 tx=0x7f6c4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b42ab7bff40166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.186+0000 7f6c567fc640 1 -- 172.21.15.60:0/1211060377 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48037070 con 0x7f6c5815dac0 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.186+0000 7f6c567fc640 1 -- 172.21.15.60:0/1211060377 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48032440 con 0x7f6c5815dac0 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.186+0000 7f6c567fc640 1 -- 172.21.15.60:0/1211060377 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48032810 con 0x7f6c5815dac0 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1211060377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 msgr2=0x7f6c580d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1211060377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580d19f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6c480097a0 tx=0x7f6c4802a270 comp rx=0 tx=0).stop 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1211060377 shutdown_connections 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1211060377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580d19f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1211060377 >> 172.21.15.60:0/1211060377 conn(0x7f6c58061c30 msgr2=0x7f6c5806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1211060377 shutdown_connections 2024-04-25T08:36:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.187+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1211060377 wait complete. 2024-04-25T08:36:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.188+0000 7f6c5dc5a640 1 Processor -- start 2024-04-25T08:36:08.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.188+0000 7f6c5dc5a640 1 -- start start 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c5dc5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c580d0ab0 0x7f6c58152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c5dc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c580d1f30 0x7f6c5815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c5dc5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c5dc5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b97d0 con 0x7f6c5815dac0 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c5dc5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5815dea0 con 0x7f6c580d1f30 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c5dc5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580b9950 con 0x7f6c580d0ab0 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51876/0 (socket says 172.21.15.60:51876) 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c57fff640 1 -- 172.21.15.60:0/1728016076 learned_addr learned my addr 172.21.15.60:0/1728016076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c580d0ab0 0x7f6c58152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.189+0000 7f6c56ffd640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c580d1f30 0x7f6c5815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c57fff640 1 -- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c580d0ab0 msgr2=0x7f6c58152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c57fff640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c580d0ab0 0x7f6c58152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c57fff640 1 -- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c580d1f30 msgr2=0x7f6c5815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c57fff640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c580d1f30 0x7f6c5815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c57fff640 1 -- 172.21.15.60:0/1728016076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580c8610 con 0x7f6c5815dac0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c57fff640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580c9860 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000c7d0 tx=0x7f6c5000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c3ffff640 1 -- 172.21.15.60:0/1728016076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50018070 con 0x7f6c5815dac0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c3ffff640 1 -- 172.21.15.60:0/1728016076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5000a640 con 0x7f6c5815dac0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c580c88f0 con 0x7f6c5815dac0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c3ffff640 1 -- 172.21.15.60:0/1728016076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000aa70 con 0x7f6c5815dac0 2024-04-25T08:36:08.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.190+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c580c8eb0 con 0x7f6c5815dac0 2024-04-25T08:36:08.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.192+0000 7f6c3ffff640 1 -- 172.21.15.60:0/1728016076 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c5001b070 con 0x7f6c5815dac0 2024-04-25T08:36:08.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.192+0000 7f6c3ffff640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c4406b4d0 0x7f6c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.192+0000 7f6c3ffff640 1 -- 172.21.15.60:0/1728016076 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6c50089970 con 0x7f6c5815dac0 2024-04-25T08:36:08.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.192+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c4406b4d0 0x7f6c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.193+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c4406b4d0 0x7f6c4406d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6c48034000 tx=0x7f6c4802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.193+0000 7f6c60a34740 1 Processor -- start 2024-04-25T08:36:08.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.193+0000 7f6c60a34740 1 -- start start 2024-04-25T08:36:08.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 0x5586cb87d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87aef0 0x5586cb87d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb84d770 0x5586cb850f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c60a34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb7b6da0 con 0x5586cb87ab10 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb75fb80 con 0x5586cb84d770 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586cb7b2e40 con 0x5586cb87aef0 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 0x5586cb87d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 0x5586cb87d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51892/0 (socket says 172.21.15.60:51892) 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c577fe640 1 -- 172.21.15.60:0/1316678043 learned_addr learned my addr 172.21.15.60:0/1316678043 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c56ffd640 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87aef0 0x5586cb87d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c57fff640 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb84d770 0x5586cb850f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c577fe640 1 -- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87aef0 msgr2=0x5586cb87d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.194+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87aef0 0x5586cb87d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c577fe640 1 -- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb84d770 msgr2=0x5586cb850f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb84d770 0x5586cb850f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c577fe640 1 -- 172.21.15.60:0/1316678043 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586cb769f10 con 0x5586cb87ab10 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c577fe640 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 0x5586cb87d320 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000c00 tx=0x7f6c4802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d58f1fd7fc8d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c3d7fa640 1 -- 172.21.15.60:0/1316678043 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4804c070 con 0x5586cb87ab10 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c3d7fa640 1 -- 172.21.15.60:0/1316678043 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48032d70 con 0x5586cb87ab10 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c3d7fa640 1 -- 172.21.15.60:0/1316678043 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48037040 con 0x5586cb87ab10 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c60a34740 1 -- 172.21.15.60:0/1316678043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 msgr2=0x5586cb87d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.195+0000 7f6c60a34740 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 0x5586cb87d320 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000c00 tx=0x7f6c4802c000 comp rx=0 tx=0).stop 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 -- 172.21.15.60:0/1316678043 shutdown_connections 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb84d770 0x5586cb850f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87aef0 0x5586cb87d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 --2- 172.21.15.60:0/1316678043 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87ab10 0x5586cb87d320 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 -- 172.21.15.60:0/1316678043 >> 172.21.15.60:0/1316678043 conn(0x5586cb84f9e0 msgr2=0x5586cb898320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 -- 172.21.15.60:0/1316678043 shutdown_connections 2024-04-25T08:36:08.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 -- 172.21.15.60:0/1316678043 wait complete. 2024-04-25T08:36:08.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 -- start start 2024-04-25T08:36:08.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.196+0000 7f6c60a34740 1 Processor -- start 2024-04-25T08:36:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb881150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb870140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb852b90 con 0x5586cb86fcc0 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb7554b0 con 0x5586cb87aef0 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586cb852e70 con 0x5586cb87ab10 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c60a34740 10 client.0 _ll_register_callbacks cb 0x5586cb795180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb870140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb870140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51896/0 (socket says 172.21.15.60:51896) 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c57fff640 1 -- 172.21.15.60:0/707953000 learned_addr learned my addr 172.21.15.60:0/707953000 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c56ffd640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb881150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c577fe640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.197+0000 7f6c57fff640 1 -- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 msgr2=0x5586cb84ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c57fff640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c57fff640 1 -- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 msgr2=0x5586cb881150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c57fff640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb881150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c57fff640 1 -- 172.21.15.60:0/707953000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586cb5957d0 con 0x5586cb86fcc0 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c577fe640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c56ffd640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb881150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c57fff640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb870140 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6c50087c40 tx=0x7f6c50058060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b1e659da4f1778 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c367fc640 1 -- 172.21.15.60:0/707953000 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50060370 con 0x5586cb86fcc0 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c367fc640 1 -- 172.21.15.60:0/707953000 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6c5005e070 con 0x5586cb86fcc0 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c48032d70 con 0x5586cb86fcc0 2024-04-25T08:36:08.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c367fc640 1 -- 172.21.15.60:0/707953000 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c5005e270 con 0x5586cb86fcc0 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 10 client.7899 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5586cb4cb3f0 con 0x5586cb86fcc0 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c367fc640 1 -- 172.21.15.60:0/707953000 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48032d70 con 0x5586cb86fcc0 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c367fc640 1 -- 172.21.15.60:0/707953000 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c500573f0 con 0x5586cb86fcc0 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 10 client.7899 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 10 client.7899 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 20 client.7899 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb899920 0x5586cb899d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 4 client.7899 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.198+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586cb7c42e0 con 0x5586cb899920 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.199+0000 7f6c577fe640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb899920 0x5586cb899d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.199+0000 7f6c577fe640 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb899920 0x5586cb899d30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc2f76a0afa9705 server_cookie=9288e73fe9eafa5a in_seq=0 out_seq=0 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.199+0000 7f6c367fc640 10 client.7899 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.199+0000 7f6c367fc640 1 -- 172.21.15.60:0/707953000 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586cb7c42e0 con 0x5586cb899920 2024-04-25T08:36:08.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.199+0000 7f6c367fc640 10 client.7899 handle_command_reply: tid=0 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.199+0000 7f6c60a34740 1 client.7899 shutdown 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 msgr2=0x5586cb870140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb870140 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6c50087c40 tx=0x7f6c50058060 comp rx=0 tx=0).stop 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 shutdown_connections 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb899920 0x5586cb899d30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb870140 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb881150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 --2- 172.21.15.60:0/707953000 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c367fc640 0 client.7899 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 >> 172.21.15.60:0/707953000 conn(0x5586cb84d770 msgr2=0x5586cb878a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 shutdown_connections 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 1 -- 172.21.15.60:0/707953000 wait complete. 2024-04-25T08:36:08.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.200+0000 7f6c60a34740 20 client.7899 trim_cache size 0 max 16384 2024-04-25T08:36:08.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.215+0000 7f6c60a34740 1 Processor -- start 2024-04-25T08:36:08.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.215+0000 7f6c60a34740 1 -- start start 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb86f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c60a34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c582005b0 con 0x5586cb86fcc0 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581ff7b0 con 0x5586cb87aef0 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580cbe40 con 0x5586cb87ab10 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb86f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb86f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51906/0 (socket says 172.21.15.60:51906) 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 -- 172.21.15.60:0/4049968943 learned_addr learned my addr 172.21.15.60:0/4049968943 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c56ffd640 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c57fff640 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 -- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 msgr2=0x5586cb84dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 -- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 msgr2=0x5586cb84e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c577fe640 1 -- 172.21.15.60:0/4049968943 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586cb48b0f0 con 0x5586cb86fcc0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.216+0000 7f6c57fff640 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c577fe640 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb86f3b0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009560 tx=0x7f6c4802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccb68d4697270b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c3cff9640 1 -- 172.21.15.60:0/4049968943 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48064070 con 0x5586cb86fcc0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c3cff9640 1 -- 172.21.15.60:0/4049968943 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48057040 con 0x5586cb86fcc0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c3cff9640 1 -- 172.21.15.60:0/4049968943 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48037040 con 0x5586cb86fcc0 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 -- 172.21.15.60:0/4049968943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 msgr2=0x5586cb86f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb86f3b0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009560 tx=0x7f6c4802c070 comp rx=0 tx=0).stop 2024-04-25T08:36:08.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 -- 172.21.15.60:0/4049968943 shutdown_connections 2024-04-25T08:36:08.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 --2- 172.21.15.60:0/4049968943 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb86fcc0 0x5586cb86f3b0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 -- 172.21.15.60:0/4049968943 >> 172.21.15.60:0/4049968943 conn(0x5586cb84d770 msgr2=0x5586cb8522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 -- 172.21.15.60:0/4049968943 shutdown_connections 2024-04-25T08:36:08.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.217+0000 7f6c60a34740 1 -- 172.21.15.60:0/4049968943 wait complete. 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.218+0000 7f6c60a34740 1 -- start start 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.218+0000 7f6c60a34740 1 Processor -- start 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb86fcc0 0x5586cb87ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586cb878b30 con 0x5586cb87aef0 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581ff1b0 con 0x5586cb86fcc0 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586cb76dd90 con 0x5586cb87ab10 2024-04-25T08:36:08.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c60a34740 10 client.0 _ll_register_callbacks cb 0x7f6c48040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51918/0 (socket says 172.21.15.60:51918) 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 -- 172.21.15.60:0/3235728337 learned_addr learned my addr 172.21.15.60:0/3235728337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c56ffd640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c577fe640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb86fcc0 0x5586cb87ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 -- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 msgr2=0x5586cb84dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 -- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb86fcc0 msgr2=0x5586cb87ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb86fcc0 0x5586cb87ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.219+0000 7f6c57fff640 1 -- 172.21.15.60:0/3235728337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586cb880560 con 0x5586cb87aef0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c57fff640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6c50057ca0 tx=0x7f6c50014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6530468ef4114260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c367fc640 1 -- 172.21.15.60:0/3235728337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5007b060 con 0x5586cb87aef0 2024-04-25T08:36:08.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c367fc640 1 -- 172.21.15.60:0/3235728337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6c50053f00 con 0x5586cb87aef0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c48057040 con 0x5586cb87aef0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c367fc640 1 -- 172.21.15.60:0/3235728337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c50081050 con 0x5586cb87aef0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 10 client.7905 fetch_fsmap learned FSMap version 55 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5586cb903380 con 0x5586cb87aef0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c367fc640 1 -- 172.21.15.60:0/3235728337 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48057040 con 0x5586cb87aef0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c367fc640 1 -- 172.21.15.60:0/3235728337 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c50076070 con 0x5586cb87aef0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 10 client.7905 fetch_fsmap finished waiting for FSMap version 55 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 10 client.7905 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 20 client.7905 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb8b20f0 0x5586cb905bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 4 client.7905 mds_command: new command op to 7626 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 --> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586cb7be630 con 0x5586cb8b20f0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.220+0000 7f6c577fe640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb8b20f0 0x5586cb905bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c577fe640 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb8b20f0 0x5586cb905bb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358907a61886e232 server_cookie=4166d54386dc79b2 in_seq=0 out_seq=0 2024-04-25T08:36:08.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c367fc640 10 client.7905 ms_handle_connect on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c367fc640 1 -- 172.21.15.60:0/3235728337 <== mds.0 v2:172.21.15.60:6833/3135665870 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5586cb7be630 con 0x5586cb8b20f0 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c367fc640 10 client.7905 handle_command_reply: tid=0 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 client.7905 shutdown 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 msgr2=0x5586cb84e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6c50057ca0 tx=0x7f6c50014040 comp rx=0 tx=0).stop 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 shutdown_connections 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:6833/3135665870,v1:172.21.15.60:6835/3135665870] conn(0x5586cb8b20f0 0x5586cb905bb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5586cb87aef0 0x5586cb84e190 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5586cb87ab10 0x5586cb84dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c367fc640 0 client.7905 ms_handle_reset on v2:172.21.15.60:6833/3135665870 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 --2- 172.21.15.60:0/3235728337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5586cb86fcc0 0x5586cb87ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 >> 172.21.15.60:0/3235728337 conn(0x5586cb84d770 msgr2=0x5586cb86f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 shutdown_connections 2024-04-25T08:36:08.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 1 -- 172.21.15.60:0/3235728337 wait complete. 2024-04-25T08:36:08.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.221+0000 7f6c60a34740 20 client.7905 trim_cache size 0 max 16384 2024-04-25T08:36:08.222 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:36:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:36:08.222 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c4406b4d0 msgr2=0x7f6c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c4406b4d0 0x7f6c4406d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6c48034000 tx=0x7f6c4802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 msgr2=0x7f6c580c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580c9860 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000c7d0 tx=0x7f6c5000cc90 comp rx=0 tx=0).stop 2024-04-25T08:36:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 shutdown_connections 2024-04-25T08:36:08.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c5815dac0 0x7f6c580c9860 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c4406b4d0 0x7f6c4406d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c580d1f30 0x7f6c5815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 --2- 172.21.15.60:0/1728016076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c580d0ab0 0x7f6c58152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 >> 172.21.15.60:0/1728016076 conn(0x7f6c58061c30 msgr2=0x7f6c580b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 shutdown_connections 2024-04-25T08:36:08.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.222+0000 7f6c5dc5a640 1 -- 172.21.15.60:0/1728016076 wait complete. 2024-04-25T08:36:08.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:36:08.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.334+0000 7f0c25e07640 1 Processor -- start 2024-04-25T08:36:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.334+0000 7f0c25e07640 1 -- start start 2024-04-25T08:36:08.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c25e07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 0x7f0c200b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c25e07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0c200b9770 0x7f0c200d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c25e07640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200d1df0 con 0x7f0c200d2e90 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c25e07640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200b7a50 con 0x7f0c200b9770 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c25e07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d20d0 con 0x7f0c2015dd30 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c1ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0c200d2e90 0x7f0c200b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57712/0 (socket says 172.21.15.60:57712) 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c1ffff640 1 -- 172.21.15.60:0/611359936 learned_addr learned my addr 172.21.15.60:0/611359936 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c1effd640 1 --2- 172.21.15.60:0/611359936 >> v2:172.21.15.100:3300/0 conn(0x7f0c200b9770 0x7f0c200d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 0x7f0c200b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416609896 0 0) 0x7f0c200d1df0 con 0x7f0c200d2e90 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.335+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003520 con 0x7f0c200d2e90 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514918078 0 0) 0x7f0c08003520 con 0x7f0c200d2e90 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 msgr2=0x7f0c200b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 0x7f0c200b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 >> v2:172.21.15.100:3300/0 conn(0x7f0c200b9770 msgr2=0x7f0c200d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 --2- 172.21.15.60:0/611359936 >> v2:172.21.15.100:3300/0 conn(0x7f0c200b9770 0x7f0c200d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200d23b0 con 0x7f0c200d2e90 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f0c180019f0 con 0x7f0c200d2e90 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 >> v1:172.21.15.60:6789/0 conn(0x7f0c200d2e90 legacy=0x7f0c200b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c080039b0 0x7f0c08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 0x7f0c080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200b7a50 con 0x7f0c080039b0 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d20d0 con 0x7f0c08004800 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800ba10 con 0x7f0c2015dd30 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c080039b0 0x7f0c08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1effd640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1ffff640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 0x7f0c080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1f7fe640 1 -- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 msgr2=0x7f0c080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c2015dd30 0x7f0c080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1f7fe640 1 -- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c08004800 msgr2=0x7f0c08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.336+0000 7f0c1f7fe640 1 -- 172.21.15.60:0/611359936 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08003520 con 0x7f0c080039b0 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.337+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c080039b0 0x7f0c08003dd0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009af0 tx=0x7f0c08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcace3e582bf89b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.337+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10036070 con 0x7f0c080039b0 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.337+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10031400 con 0x7f0c080039b0 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.337+0000 7f0c1e7fc640 1 -- 172.21.15.60:0/611359936 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10009690 con 0x7f0c080039b0 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c25e07640 1 -- 172.21.15.60:0/611359936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c080039b0 msgr2=0x7f0c08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c25e07640 1 --2- 172.21.15.60:0/611359936 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c080039b0 0x7f0c08003dd0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009af0 tx=0x7f0c08005830 comp rx=0 tx=0).stop 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c1ffff640 1 -- 172.21.15.60:0/611359936 reap_dead start 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c25e07640 1 -- 172.21.15.60:0/611359936 shutdown_connections 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c25e07640 1 -- 172.21.15.60:0/611359936 >> 172.21.15.60:0/611359936 conn(0x7f0c20061c30 msgr2=0x7f0c2006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c25e07640 1 -- 172.21.15.60:0/611359936 shutdown_connections 2024-04-25T08:36:08.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.338+0000 7f0c25e07640 1 -- 172.21.15.60:0/611359936 wait complete. 2024-04-25T08:36:08.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.339+0000 7f0c25e07640 1 Processor -- start 2024-04-25T08:36:08.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.339+0000 7f0c25e07640 1 -- start start 2024-04-25T08:36:08.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c25e07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c080039b0 0x7f0c201f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c25e07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 0x7f0c200acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c25e07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c2015bb10 0x7f0c2015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c25e07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d1a10 con 0x7f0c200aabf0 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c25e07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d2970 con 0x7f0c2015bb10 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c25e07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d1b90 con 0x7f0c080039b0 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 0x7f0c200acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 0x7f0c200acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51940/0 (socket says 172.21.15.60:51940) 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 -- 172.21.15.60:0/3951241895 learned_addr learned my addr 172.21.15.60:0/3951241895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c080039b0 0x7f0c201f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1ffff640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c2015bb10 0x7f0c2015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 -- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c080039b0 msgr2=0x7f0c201f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c080039b0 0x7f0c201f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 -- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c2015bb10 msgr2=0x7f0c2015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c2015bb10 0x7f0c2015bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 -- 172.21.15.60:0/3951241895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200aa680 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c1effd640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 0x7f0c200acca0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0c14011950 tx=0x7f0c14011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.340+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400faf0 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.341+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1400fc90 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.341+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14014440 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.341+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c200ab0d0 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.341+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c200ab690 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.341+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20073990 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.342+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c14012040 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.342+0000 7f0c07fff640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.342+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0c1408a860 con 0x7f0c200aabf0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.343+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.343+0000 7f0c1f7fe640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0c10033000 tx=0x7f0c10008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.347+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c14019030 con 0x7f0c200aabf0 2024-04-25T08:36:08.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.562+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c200aa9d0 con 0x7f0c200aabf0 2024-04-25T08:36:08.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.563+0000 7f0c07fff640 1 -- 172.21.15.60:0/3951241895 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0c14058f20 con 0x7f0c200aabf0 2024-04-25T08:36:08.558 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:08.558 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:08.558 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0c06b4d0 msgr2=0x7f0c0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0c10033000 tx=0x7f0c10008030 comp rx=0 tx=0).stop 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 msgr2=0x7f0c200acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 0x7f0c200acca0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0c14011950 tx=0x7f0c14011e10 comp rx=0 tx=0).stop 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 shutdown_connections 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c2015bb10 0x7f0c2015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c200aabf0 0x7f0c200acca0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 --2- 172.21.15.60:0/3951241895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c080039b0 0x7f0c201f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 >> 172.21.15.60:0/3951241895 conn(0x7f0c20061c30 msgr2=0x7f0c200b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 shutdown_connections 2024-04-25T08:36:08.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.565+0000 7f0c25e07640 1 -- 172.21.15.60:0/3951241895 wait complete. 2024-04-25T08:36:08.570 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:36:08.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:08.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.675+0000 7f60099f5640 1 Processor -- start 2024-04-25T08:36:08.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.675+0000 7f60099f5640 1 -- start start 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60099f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415dd30 0x7f60040b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60099f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60040b9770 0x7f60040d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60099f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60040d1df0 con 0x7f60040d2e90 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60099f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60040b7a50 con 0x7f60040b9770 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60099f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d20d0 con 0x7f600415dd30 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60040b9770 0x7f60040d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60040b9770 0x7f60040d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51950/0 (socket says 172.21.15.60:51950) 2024-04-25T08:36:08.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 -- 172.21.15.60:0/2867638154 learned_addr learned my addr 172.21.15.60:0/2867638154 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f6002ffd640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415dd30 0x7f60040b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 -- 172.21.15.60:0/2867638154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415dd30 msgr2=0x7f60040b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415dd30 0x7f60040b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 -- 172.21.15.60:0/2867638154 >> v1:172.21.15.100:6789/0 conn(0x7f60040d2e90 legacy=0x7f60040b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f60027fc640 1 -- 172.21.15.60:0/2867638154 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040d23b0 con 0x7f60040b9770 2024-04-25T08:36:08.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.676+0000 7f6002ffd640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415dd30 0x7f60040b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:08.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.677+0000 7f60027fc640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040b9770 0x7f60040d03c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009850 tx=0x7f5ff802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4bef04fd4503f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.677+0000 7f6001ffb640 1 -- 172.21.15.60:0/2867638154 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8004030 con 0x7f60040b9770 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.677+0000 7f6001ffb640 1 -- 172.21.15.60:0/2867638154 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff80041d0 con 0x7f60040b9770 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.677+0000 7f6001ffb640 1 -- 172.21.15.60:0/2867638154 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8003610 con 0x7f60040b9770 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 -- 172.21.15.60:0/2867638154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040b9770 msgr2=0x7f60040d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040b9770 0x7f60040d03c0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009850 tx=0x7f5ff802a160 comp rx=0 tx=0).stop 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 -- 172.21.15.60:0/2867638154 shutdown_connections 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60040b9770 0x7f60040d03c0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 --2- 172.21.15.60:0/2867638154 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600415dd30 0x7f60040b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 -- 172.21.15.60:0/2867638154 >> 172.21.15.60:0/2867638154 conn(0x7f6004061c30 msgr2=0x7f600406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 -- 172.21.15.60:0/2867638154 shutdown_connections 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.678+0000 7f60099f5640 1 -- 172.21.15.60:0/2867638154 wait complete. 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.679+0000 7f60099f5640 1 Processor -- start 2024-04-25T08:36:08.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.679+0000 7f60099f5640 1 -- start start 2024-04-25T08:36:08.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60099f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040b9770 0x7f60040caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60099f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60040d2e90 0x7f6004155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60099f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 0x7f60040cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60099f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d1a10 con 0x7f600415dd30 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60099f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d2970 con 0x7f60040b9770 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60099f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d1b90 con 0x7f60040d2e90 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 0x7f60040cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 0x7f60040cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51962/0 (socket says 172.21.15.60:51962) 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 -- 172.21.15.60:0/1532156149 learned_addr learned my addr 172.21.15.60:0/1532156149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f6002ffd640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040b9770 0x7f60040caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60027fc640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60040d2e90 0x7f6004155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 -- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60040d2e90 msgr2=0x7f6004155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60040d2e90 0x7f6004155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 -- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040b9770 msgr2=0x7f60040caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040b9770 0x7f60040caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 -- 172.21.15.60:0/1532156149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004154f40 con 0x7f600415dd30 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.680+0000 7f60037fe640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 0x7f60040cc0e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00c440 tx=0x7f5ffc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.681+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc011790 con 0x7f600415dd30 2024-04-25T08:36:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.681+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc017070 con 0x7f600415dd30 2024-04-25T08:36:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.681+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040cb060 con 0x7f600415dd30 2024-04-25T08:36:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.681+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc011b70 con 0x7f600415dd30 2024-04-25T08:36:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.681+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040cb620 con 0x7f600415dd30 2024-04-25T08:36:08.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.682+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004073990 con 0x7f600415dd30 2024-04-25T08:36:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.682+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ffc011d10 con 0x7f600415dd30 2024-04-25T08:36:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.683+0000 7f5febfff640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ff006b4d0 0x7f5ff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.683+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5ffc08a0b0 con 0x7f600415dd30 2024-04-25T08:36:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.683+0000 7f6002ffd640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ff006b4d0 0x7f5ff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:08.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.683+0000 7f6002ffd640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ff006b4d0 0x7f5ff006d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400f500 tx=0x7f5ff4007770 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:08.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.687+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ffc0586d0 con 0x7f600415dd30 2024-04-25T08:36:08.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.935+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f60040d0ca0 con 0x7f600415dd30 2024-04-25T08:36:08.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.935+0000 7f5febfff640 1 -- 172.21.15.60:0/1532156149 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1888 (secure 0 0 0) 0x7f5ffc05c770 con 0x7f600415dd30 2024-04-25T08:36:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:08.930 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":429,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":94},{"state_name":"peering","count":3}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2554626048,"bytes_avail":770539487232,"bytes_total":773094113280,"inactive_pgs_ratio":0.030927835032343864,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2227764,"recovering_keys_per_sec":0,"num_objects_recovered":17,"num_bytes_recovered":17825792,"num_keys_recovered":0,"write_bytes_sec":99886199,"read_op_per_sec":0,"write_op_per_sec":97},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:36:05.490012+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ff006b4d0 msgr2=0x7f5ff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ff006b4d0 0x7f5ff006d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400f500 tx=0x7f5ff4007770 comp rx=0 tx=0).stop 2024-04-25T08:36:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 msgr2=0x7f60040cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 0x7f60040cc0e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00c440 tx=0x7f5ffc00c900 comp rx=0 tx=0).stop 2024-04-25T08:36:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 shutdown_connections 2024-04-25T08:36:08.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600415dd30 0x7f60040cc0e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60040d2e90 0x7f6004155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ff006b4d0 0x7f5ff006d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 --2- 172.21.15.60:0/1532156149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f60040b9770 0x7f60040caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 >> 172.21.15.60:0/1532156149 conn(0x7f6004061c30 msgr2=0x7f60040b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 shutdown_connections 2024-04-25T08:36:08.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:08.937+0000 7f60099f5640 1 -- 172.21.15.60:0/1532156149 wait complete. 2024-04-25T08:36:08.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T08:36:09.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:10.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 Processor -- start 2024-04-25T08:36:10.057 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 -- start start 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61200b2cb0 0x7f61200d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f61200d1020 0x7f61200d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61200b97a0 con 0x7f612015dbe0 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612015dfc0 con 0x7f61200b2cb0 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.063+0000 7f612583a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61200d1a00 con 0x7f61200d1020 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f61200b2cb0 0x7f61200d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611effd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f612015dbe0 0x7f61200d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57724/0 (socket says 172.21.15.60:57724) 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611effd640 1 -- 172.21.15.60:0/3645815022 learned_addr learned my addr 172.21.15.60:0/3645815022 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611f7fe640 1 --2- 172.21.15.60:0/3645815022 >> v2:172.21.15.100:3300/0 conn(0x7f61200d1020 0x7f61200d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611e7fc640 1 -- 172.21.15.60:0/3645815022 >> v2:172.21.15.100:3300/0 conn(0x7f61200d1020 msgr2=0x7f61200d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611e7fc640 1 --2- 172.21.15.60:0/3645815022 >> v2:172.21.15.100:3300/0 conn(0x7f61200d1020 0x7f61200d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611e7fc640 1 -- 172.21.15.60:0/3645815022 >> v1:172.21.15.60:6789/0 conn(0x7f612015dbe0 legacy=0x7f61200d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.064+0000 7f611e7fc640 1 -- 172.21.15.60:0/3645815022 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200d1c50 con 0x7f61200b2cb0 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f611e7fc640 1 --2- 172.21.15.60:0/3645815022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f61200d2730 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f61200cf570 tx=0x7f611402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421b108375471945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f611dffb640 1 -- 172.21.15.60:0/3645815022 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114037070 con 0x7f61200b2cb0 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f611dffb640 1 -- 172.21.15.60:0/3645815022 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6114032440 con 0x7f61200b2cb0 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f611dffb640 1 -- 172.21.15.60:0/3645815022 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114032810 con 0x7f61200b2cb0 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 -- 172.21.15.60:0/3645815022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 msgr2=0x7f61200d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 --2- 172.21.15.60:0/3645815022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f61200d2730 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f61200cf570 tx=0x7f611402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 -- 172.21.15.60:0/3645815022 shutdown_connections 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 --2- 172.21.15.60:0/3645815022 >> v2:172.21.15.100:3300/0 conn(0x7f61200d1020 0x7f61200d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 --2- 172.21.15.60:0/3645815022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f61200d2730 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 -- 172.21.15.60:0/3645815022 >> 172.21.15.60:0/3645815022 conn(0x7f6120061a20 msgr2=0x7f6120061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 -- 172.21.15.60:0/3645815022 shutdown_connections 2024-04-25T08:36:10.062 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.065+0000 7f612583a640 1 -- 172.21.15.60:0/3645815022 wait complete. 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.067+0000 7f612583a640 1 Processor -- start 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.067+0000 7f612583a640 1 -- start start 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f612583a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f6120152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f612583a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61200d1020 0x7f61200cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f612583a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f612015dbe0 0x7f61200cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f612583a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200b92a0 con 0x7f61200b2cb0 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f612583a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200d2210 con 0x7f61200d1020 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f612583a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61200b9420 con 0x7f612015dbe0 2024-04-25T08:36:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f6120152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f6120152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51984/0 (socket says 172.21.15.60:51984) 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 -- 172.21.15.60:0/3277677112 learned_addr learned my addr 172.21.15.60:0/3277677112 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611e7fc640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61200d1020 0x7f61200cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611f7fe640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f612015dbe0 0x7f61200cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 -- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f612015dbe0 msgr2=0x7f61200cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f612015dbe0 0x7f61200cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 -- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61200d1020 msgr2=0x7f61200cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61200d1020 0x7f61200cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 -- 172.21.15.60:0/3277677112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200cb150 con 0x7f61200b2cb0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611f7fe640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f612015dbe0 0x7f61200cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.068+0000 7f611effd640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f6120152e50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f611800c3c0 tx=0x7f611800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.069+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118017070 con 0x7f61200b2cb0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.069+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6118005ce0 con 0x7f61200b2cb0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.069+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61180072f0 con 0x7f61200b2cb0 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.069+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61201534c0 con 0x7f61200b2cb0 2024-04-25T08:36:10.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.069+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6120153a80 con 0x7f61200b2cb0 2024-04-25T08:36:10.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.070+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6120073990 con 0x7f61200b2cb0 2024-04-25T08:36:10.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.070+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f611801a070 con 0x7f61200b2cb0 2024-04-25T08:36:10.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.071+0000 7f6107fff640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f610c06b4d0 0x7f610c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.071+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6118089750 con 0x7f61200b2cb0 2024-04-25T08:36:10.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.071+0000 7f611e7fc640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f610c06b4d0 0x7f610c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.072+0000 7f611e7fc640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f610c06b4d0 0x7f610c06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6114034000 tx=0x7f611402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.075+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6118058460 con 0x7f61200b2cb0 2024-04-25T08:36:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.292+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61200c9ad0 con 0x7f61200b2cb0 2024-04-25T08:36:10.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.293+0000 7f6107fff640 1 -- 172.21.15.60:0/3277677112 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6118057d60 con 0x7f61200b2cb0 2024-04-25T08:36:10.288 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:10.288 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:10.288 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f610c06b4d0 msgr2=0x7f610c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f610c06b4d0 0x7f610c06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6114034000 tx=0x7f611402e000 comp rx=0 tx=0).stop 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 msgr2=0x7f6120152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f6120152e50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f611800c3c0 tx=0x7f611800c880 comp rx=0 tx=0).stop 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 shutdown_connections 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f612015dbe0 0x7f61200cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f610c06b4d0 0x7f610c06d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f61200d1020 0x7f61200cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 --2- 172.21.15.60:0/3277677112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f61200b2cb0 0x7f6120152e50 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 >> 172.21.15.60:0/3277677112 conn(0x7f6120061a20 msgr2=0x7f6120159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 shutdown_connections 2024-04-25T08:36:10.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.295+0000 7f612583a640 1 -- 172.21.15.60:0/3277677112 wait complete. 2024-04-25T08:36:10.300 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:36:10.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:10.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.407+0000 7f7f2c0f5640 1 Processor -- start 2024-04-25T08:36:10.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.407+0000 7f7f2c0f5640 1 -- start start 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.407+0000 7f7f2c0f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f240cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.407+0000 7f7f2c0f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f240cfcf0 0x7f7f240cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f2c0f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f24071480 con 0x7f7f240ce3c0 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f2c0f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240ce830 con 0x7f7f240cfcf0 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f2c0f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24072090 con 0x7f7f24154ee0 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f240cfcf0 0x7f7f240cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7f240cfcf0 0x7f7f240cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:51996/0 (socket says 172.21.15.60:51996) 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 -- 172.21.15.60:0/2320465299 learned_addr learned my addr 172.21.15.60:0/2320465299 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:10.403 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29e91640 1 --2- 172.21.15.60:0/2320465299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f240cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 -- 172.21.15.60:0/2320465299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 msgr2=0x7f7f240cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 --2- 172.21.15.60:0/2320465299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f240cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 -- 172.21.15.60:0/2320465299 >> v1:172.21.15.100:6789/0 conn(0x7f7f240ce3c0 legacy=0x7f7f2415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.408+0000 7f7f29690640 1 -- 172.21.15.60:0/2320465299 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f24071760 con 0x7f7f240cfcf0 2024-04-25T08:36:10.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f29690640 1 --2- 172.21.15.60:0/2320465299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240cfcf0 0x7f7f240cde80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009870 tx=0x7f7f1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7503261c6fa23ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f28e8f640 1 -- 172.21.15.60:0/2320465299 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18037070 con 0x7f7f240cfcf0 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f28e8f640 1 -- 172.21.15.60:0/2320465299 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18032440 con 0x7f7f240cfcf0 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f28e8f640 1 -- 172.21.15.60:0/2320465299 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18032810 con 0x7f7f240cfcf0 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/2320465299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240cfcf0 msgr2=0x7f7f240cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/2320465299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240cfcf0 0x7f7f240cde80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009870 tx=0x7f7f1802a270 comp rx=0 tx=0).stop 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/2320465299 shutdown_connections 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/2320465299 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240cfcf0 0x7f7f240cde80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.407 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/2320465299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f240cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.409+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/2320465299 >> 172.21.15.60:0/2320465299 conn(0x7f7f2406abe0 msgr2=0x7f7f2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.410+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/2320465299 shutdown_connections 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.410+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/2320465299 wait complete. 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 Processor -- start 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 -- start start 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 0x7f7f241e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f240cfcf0 0x7f7f24143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f24158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24071230 con 0x7f7f240ce3c0 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24071d20 con 0x7f7f240cfcf0 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.411+0000 7f7f2c0f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241552c0 con 0x7f7f24154ee0 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 0x7f7f241e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 0x7f7f241e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52000/0 (socket says 172.21.15.60:52000) 2024-04-25T08:36:10.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 -- 172.21.15.60:0/3492824490 learned_addr learned my addr 172.21.15.60:0/3492824490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29690640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f240cfcf0 0x7f7f24143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f2a692640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f24158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 -- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 msgr2=0x7f7f24158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f24158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 -- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f240cfcf0 msgr2=0x7f7f24143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f240cfcf0 0x7f7f24143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 -- 172.21.15.60:0/3492824490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f241e43c0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29690640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f240cfcf0 0x7f7f24143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f2a692640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f24158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f29e91640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 0x7f7f241e3b30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00d5f0 tx=0x7f7f1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0125f0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c019070 con 0x7f7f240ce3c0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f241e46a0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0129d0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.412+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f241f1c10 con 0x7f7f240ce3c0 2024-04-25T08:36:10.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.413+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2406e9e0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.414+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f1c012b70 con 0x7f7f240ce3c0 2024-04-25T08:36:10.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.414+0000 7f7f227fc640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f1006b4d0 0x7f7f1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.414+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7f1c08a7c0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.414+0000 7f7f29690640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f1006b4d0 0x7f7f1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.411 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.415+0000 7f7f29690640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f1006b4d0 0x7f7f1006d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7f18034000 tx=0x7f7f1802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.418+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f2406e9e0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.668+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7f24073990 con 0x7f7f240ce3c0 2024-04-25T08:36:10.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.669+0000 7f7f227fc640 1 -- 172.21.15.60:0/3492824490 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1888 (secure 0 0 0) 0x7f7f1c0594d0 con 0x7f7f240ce3c0 2024-04-25T08:36:10.663 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:10.663 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":431,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":94},{"state_name":"peering","count":3}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2554626048,"bytes_avail":770539487232,"bytes_total":773094113280,"inactive_pgs_ratio":0.030927835032343864,"recovering_objects_per_sec":2,"recovering_bytes_per_sec":2227709,"recovering_keys_per_sec":0,"num_objects_recovered":17,"num_bytes_recovered":17825792,"num_keys_recovered":0,"write_bytes_sec":28063147,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:36:05.490012+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f1006b4d0 msgr2=0x7f7f1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f1006b4d0 0x7f7f1006d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7f18034000 tx=0x7f7f1802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 msgr2=0x7f7f241e3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 0x7f7f241e3b30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00d5f0 tx=0x7f7f1c00dab0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 shutdown_connections 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7f24154ee0 0x7f7f24158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f1006b4d0 0x7f7f1006d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f240cfcf0 0x7f7f24143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 --2- 172.21.15.60:0/3492824490 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f240ce3c0 0x7f7f241e3b30 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 >> 172.21.15.60:0/3492824490 conn(0x7f7f2406abe0 msgr2=0x7f7f240b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 shutdown_connections 2024-04-25T08:36:10.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:10.671+0000 7f7f2c0f5640 1 -- 172.21.15.60:0/3492824490 wait complete. 2024-04-25T08:36:10.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T08:36:11.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.796+0000 7f3c10ee2640 1 Processor -- start 2024-04-25T08:36:11.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.796+0000 7f3c10ee2640 1 -- start start 2024-04-25T08:36:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c10ee2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3c0c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c10ee2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0c0b97a0 con 0x7f3c0c0d1020 2024-04-25T08:36:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c10ee2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0c0d1c50 con 0x7f3c0c15dbe0 2024-04-25T08:36:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c10ee2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c15dfc0 con 0x7f3c0c0b2cb0 2024-04-25T08:36:11.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c0ad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3c0c0d1020 0x7f3c0c0d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44318/0 (socket says 172.21.15.60:44318) 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c0ad9d640 1 -- 172.21.15.60:0/3214217028 learned_addr learned my addr 172.21.15.60:0/3214217028 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c03fff640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3c0c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355135715 0 0) 0x7f3c0c0b97a0 con 0x7f3c0c0d1020 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bf8003520 con 0x7f3c0c0d1020 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164911289 0 0) 0x7f3c0c0d1c50 con 0x7f3c0c15dbe0 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c0c0b97a0 con 0x7f3c0c15dbe0 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2193032160 0 0) 0x7f3bf8003520 con 0x7f3c0c0d1020 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 msgr2=0x7f3c0c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3c0c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 >> v1:172.21.15.100:6789/0 conn(0x7f3c0c15dbe0 legacy=0x7f3c0c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c0d1f30 con 0x7f3c0c0d1020 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3c040019f0 con 0x7f3c0c0d1020 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 >> v1:172.21.15.60:6789/0 conn(0x7f3c0c0d1020 legacy=0x7f3c0c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bf8003b70 0x7f3bf8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8004800 0x7f3bf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.797+0000 7f3c09d9b640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3bf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf800ba70 con 0x7f3bf8003b70 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf800bbf0 con 0x7f3bf8004800 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf800bdf0 con 0x7f3c0c0b2cb0 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bf8003b70 0x7f3bf8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c03fff640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8004800 0x7f3bf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0ad9d640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3bf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0a59c640 1 -- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 msgr2=0x7f3bf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:11.796 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3bf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0a59c640 1 -- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8004800 msgr2=0x7f3bf8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8004800 0x7f3bf8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0a59c640 1 -- 172.21.15.60:0/3214217028 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8003520 con 0x7f3bf8003b70 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.798+0000 7f3c0ad9d640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c0b2cb0 0x7f3bf80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bf8003b70 0x7f3bf8003f70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0018d0 tx=0x7f3bf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a1210e808152d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0014f0 con 0x7f3bf8003b70 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc0052d0 con 0x7f3bf8003b70 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c09d9b640 1 -- 172.21.15.60:0/3214217028 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc02aca0 con 0x7f3bf8003b70 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3214217028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bf8003b70 msgr2=0x7f3bf8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3214217028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bf8003b70 0x7f3bf8003f70 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0018d0 tx=0x7f3bf8005830 comp rx=0 tx=0).stop 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c0ad9d640 1 -- 172.21.15.60:0/3214217028 reap_dead start 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3214217028 shutdown_connections 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3214217028 >> 172.21.15.60:0/3214217028 conn(0x7f3c0c061a20 msgr2=0x7f3c0c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3214217028 shutdown_connections 2024-04-25T08:36:11.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.799+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3214217028 wait complete. 2024-04-25T08:36:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.801+0000 7f3c10ee2640 1 Processor -- start 2024-04-25T08:36:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.801+0000 7f3c10ee2640 1 -- start start 2024-04-25T08:36:11.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c10ee2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8003b70 0x7f3c0c158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c10ee2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 0x7f3c0c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c10ee2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c157b50 0x7f3c0c0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c10ee2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0b92a0 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c10ee2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0d1a00 con 0x7f3bf8003b70 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c10ee2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c0b9420 con 0x7f3c0c157b50 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 0x7f3c0c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 0x7f3c0c151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39570/0 (socket says 172.21.15.60:39570) 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 -- 172.21.15.60:0/3822276445 learned_addr learned my addr 172.21.15.60:0/3822276445 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8003b70 0x7f3c0c158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c0ad9d640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c157b50 0x7f3c0c0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 -- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c157b50 msgr2=0x7f3c0c0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c157b50 0x7f3c0c0b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 -- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8003b70 msgr2=0x7f3c0c158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8003b70 0x7f3c0c158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 -- 172.21.15.60:0/3822276445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c156df0 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c03fff640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 0x7f3c0c151a20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40119d0 tx=0x7f3bf4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.802+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf400fad0 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.803+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf400fc70 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.803+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0c1f7f50 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.803+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4014440 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.803+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c0c157370 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.804+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0c073990 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.804+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bf4012040 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.804+0000 7f3c02ffd640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bec06b4d0 0x7f3bec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.804+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f3bf408a8b0 con 0x7f3c0c1f7af0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.805+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bec06b4d0 0x7f3bec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.805+0000 7f3c0a59c640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bec06b4d0 0x7f3bec06d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc032000 tx=0x7f3bfc009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:11.809+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bf4019030 con 0x7f3c0c1f7af0 2024-04-25T08:36:12.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.030+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c0c15c120 con 0x7f3c0c1f7af0 2024-04-25T08:36:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.030+0000 7f3c02ffd640 1 -- 172.21.15.60:0/3822276445 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3bf4058f70 con 0x7f3c0c1f7af0 2024-04-25T08:36:12.025 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:12.025 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:12.025 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bec06b4d0 msgr2=0x7f3bec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.027 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bec06b4d0 0x7f3bec06d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc032000 tx=0x7f3bfc009030 comp rx=0 tx=0).stop 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 msgr2=0x7f3c0c151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 0x7f3c0c151a20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40119d0 tx=0x7f3bf4011e90 comp rx=0 tx=0).stop 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 shutdown_connections 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c0c157b50 0x7f3c0c0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c0c1f7af0 0x7f3c0c151a20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3bec06b4d0 0x7f3bec06d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 --2- 172.21.15.60:0/3822276445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bf8003b70 0x7f3c0c158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.032+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 >> 172.21.15.60:0/3822276445 conn(0x7f3c0c061a20 msgr2=0x7f3c0c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.033+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 shutdown_connections 2024-04-25T08:36:12.028 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.033+0000 7f3c10ee2640 1 -- 172.21.15.60:0/3822276445 wait complete. 2024-04-25T08:36:12.038 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:36:12.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.143+0000 7fe68139e640 1 Processor -- start 2024-04-25T08:36:12.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.143+0000 7fe68139e640 1 -- start start 2024-04-25T08:36:12.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe68139e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe67c0b6cb0 0x7fe67c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe68139e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe67c0b5a60 0x7fe67c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe68139e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0ceee0 con 0x7fe67c15db70 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe68139e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c0d1d10 con 0x7fe67c0b5a60 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe68139e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c0d1ff0 con 0x7fe67c0b6cb0 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe67c0b5a60 0x7fe67c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe67c0b5a60 0x7fe67c0d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39580/0 (socket says 172.21.15.60:39580) 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 -- 172.21.15.60:0/2182098570 learned_addr learned my addr 172.21.15.60:0/2182098570 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67a7fc640 1 --2- 172.21.15.60:0/2182098570 >> v2:172.21.15.100:3300/0 conn(0x7fe67c0b6cb0 0x7fe67c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 -- 172.21.15.60:0/2182098570 >> v2:172.21.15.100:3300/0 conn(0x7fe67c0b6cb0 msgr2=0x7fe67c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 --2- 172.21.15.60:0/2182098570 >> v2:172.21.15.100:3300/0 conn(0x7fe67c0b6cb0 0x7fe67c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 -- 172.21.15.60:0/2182098570 >> v1:172.21.15.100:6789/0 conn(0x7fe67c15db70 legacy=0x7fe67c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67b7fe640 1 -- 172.21.15.60:0/2182098570 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0d22d0 con 0x7fe67c0b5a60 2024-04-25T08:36:12.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.144+0000 7fe67a7fc640 1 --2- 172.21.15.60:0/2182098570 >> v2:172.21.15.100:3300/0 conn(0x7fe67c0b6cb0 0x7fe67c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.145+0000 7fe67b7fe640 1 --2- 172.21.15.60:0/2182098570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0d27f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0b6aa0 tx=0x7fe67402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0678f65dc541c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.145+0000 7fe679ffb640 1 -- 172.21.15.60:0/2182098570 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674036070 con 0x7fe67c0b5a60 2024-04-25T08:36:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.145+0000 7fe679ffb640 1 -- 172.21.15.60:0/2182098570 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe674031600 con 0x7fe67c0b5a60 2024-04-25T08:36:12.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.145+0000 7fe679ffb640 1 -- 172.21.15.60:0/2182098570 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6740319d0 con 0x7fe67c0b5a60 2024-04-25T08:36:12.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 -- 172.21.15.60:0/2182098570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c0b5a60 msgr2=0x7fe67c0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 --2- 172.21.15.60:0/2182098570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0d27f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0b6aa0 tx=0x7fe67402a660 comp rx=0 tx=0).stop 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 -- 172.21.15.60:0/2182098570 shutdown_connections 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 --2- 172.21.15.60:0/2182098570 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0d27f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 --2- 172.21.15.60:0/2182098570 >> v2:172.21.15.100:3300/0 conn(0x7fe67c0b6cb0 0x7fe67c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 -- 172.21.15.60:0/2182098570 >> 172.21.15.60:0/2182098570 conn(0x7fe67c0d0520 msgr2=0x7fe67c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 -- 172.21.15.60:0/2182098570 shutdown_connections 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.146+0000 7fe68139e640 1 -- 172.21.15.60:0/2182098570 wait complete. 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.147+0000 7fe68139e640 1 Processor -- start 2024-04-25T08:36:12.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.147+0000 7fe68139e640 1 -- start start 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe68139e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe68139e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67c0b6cb0 0x7fe67c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe68139e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 0x7fe67c0cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe68139e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0cf1c0 con 0x7fe67c15db70 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe68139e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0ce530 con 0x7fe67c0b5a60 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe68139e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0cf340 con 0x7fe67c0b6cb0 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 0x7fe67c0cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 0x7fe67c0cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39590/0 (socket says 172.21.15.60:39590) 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 -- 172.21.15.60:0/3821136423 learned_addr learned my addr 172.21.15.60:0/3821136423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67affd640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67a7fc640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67c0b6cb0 0x7fe67c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 -- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67c0b6cb0 msgr2=0x7fe67c150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67c0b6cb0 0x7fe67c150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 -- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0b5a60 msgr2=0x7fe67c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67b7fe640 1 -- 172.21.15.60:0/3821136423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0cb280 con 0x7fe67c15db70 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.148+0000 7fe67a7fc640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67c0b6cb0 0x7fe67c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.149+0000 7fe67b7fe640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 0x7fe67c0cd030 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe67402ac50 tx=0x7fe674004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.149+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674036050 con 0x7fe67c15db70 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.149+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe674042070 con 0x7fe67c15db70 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.149+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c0c97a0 con 0x7fe67c15db70 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.149+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674009510 con 0x7fe67c15db70 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.149+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c0cb820 con 0x7fe67c15db70 2024-04-25T08:36:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.150+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c0c7f80 con 0x7fe67c15db70 2024-04-25T08:36:12.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.150+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe674007bd0 con 0x7fe67c15db70 2024-04-25T08:36:12.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.151+0000 7fe6637fe640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe66806b500 0x7fe66806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.151+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe6740ab3b0 con 0x7fe67c15db70 2024-04-25T08:36:12.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.151+0000 7fe67affd640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe66806b500 0x7fe66806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.151+0000 7fe67affd640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe66806b500 0x7fe66806d9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006fd0 tx=0x7fe66c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.155+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe67407a020 con 0x7fe67c15db70 2024-04-25T08:36:12.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.403+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe67c073990 con 0x7fe67c15db70 2024-04-25T08:36:12.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.403+0000 7fe6637fe640 1 -- 172.21.15.60:0/3821136423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1811 (secure 0 0 0) 0x7fe674079920 con 0x7fe67c15db70 2024-04-25T08:36:12.398 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:12.398 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":433,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2554806272,"bytes_avail":770539307008,"bytes_total":773094113280,"recovering_objects_per_sec":8,"recovering_bytes_per_sec":8779895,"recovering_keys_per_sec":0,"num_objects_recovered":67,"num_bytes_recovered":70254592,"num_keys_recovered":0,"write_bytes_sec":28063468,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:36:05.490012+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe66806b500 msgr2=0x7fe66806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe66806b500 0x7fe66806d9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006fd0 tx=0x7fe66c008040 comp rx=0 tx=0).stop 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 msgr2=0x7fe67c0cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 0x7fe67c0cd030 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe67402ac50 tx=0x7fe674004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 shutdown_connections 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe67c15db70 0x7fe67c0cd030 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe67c0b6cb0 0x7fe67c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe66806b500 0x7fe66806d9c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 --2- 172.21.15.60:0/3821136423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe67c0b5a60 0x7fe67c0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 >> 172.21.15.60:0/3821136423 conn(0x7fe67c0d0520 msgr2=0x7fe67c06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 shutdown_connections 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:12.405+0000 7fe68139e640 1 -- 172.21.15.60:0/3821136423 wait complete. 2024-04-25T08:36:12.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T08:36:13.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:13.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.531+0000 7f70a6390640 1 Processor -- start 2024-04-25T08:36:13.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.531+0000 7f70a6390640 1 -- start start 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f70a6390640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f70a015dd20 0x7f70a00b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f70a6390640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70a015e100 0x7f70a00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f70a6390640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a00d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f70a6390640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a00d1c00 con 0x7f70a015e100 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f70a6390640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a015eac0 con 0x7f70a015dd20 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f70a6390640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a015eda0 con 0x7f70a015e590 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f709f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70a015e100 0x7f70a00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f709f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70a015e100 0x7f70a00d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39592/0 (socket says 172.21.15.60:39592) 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.532+0000 7f709f7fe640 1 -- 172.21.15.60:0/2920258891 learned_addr learned my addr 172.21.15.60:0/2920258891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709ffff640 1 --2- 172.21.15.60:0/2920258891 >> v2:172.21.15.100:3300/0 conn(0x7f70a015dd20 0x7f70a00b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f70a492d640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a00d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709f7fe640 1 -- 172.21.15.60:0/2920258891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 msgr2=0x7f70a00d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709f7fe640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a00d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709f7fe640 1 -- 172.21.15.60:0/2920258891 >> v2:172.21.15.100:3300/0 conn(0x7f70a015dd20 msgr2=0x7f70a00b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709f7fe640 1 --2- 172.21.15.60:0/2920258891 >> v2:172.21.15.100:3300/0 conn(0x7f70a015dd20 0x7f70a00b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709f7fe640 1 -- 172.21.15.60:0/2920258891 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a015f080 con 0x7f70a015e100 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f70a492d640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a00d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.533+0000 7f709f7fe640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a00d0aa0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f70a00b6520 tx=0x7f709002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8329de485752aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f709effd640 1 -- 172.21.15.60:0/2920258891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090037070 con 0x7f70a015e100 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f709effd640 1 -- 172.21.15.60:0/2920258891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090032440 con 0x7f70a015e100 2024-04-25T08:36:13.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f709effd640 1 -- 172.21.15.60:0/2920258891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090032810 con 0x7f70a015e100 2024-04-25T08:36:13.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 -- 172.21.15.60:0/2920258891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 msgr2=0x7f70a00d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a00d0aa0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f70a00b6520 tx=0x7f709002a140 comp rx=0 tx=0).stop 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 -- 172.21.15.60:0/2920258891 shutdown_connections 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a00d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 --2- 172.21.15.60:0/2920258891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a00d0aa0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 --2- 172.21.15.60:0/2920258891 >> v2:172.21.15.100:3300/0 conn(0x7f70a015dd20 0x7f70a00b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 -- 172.21.15.60:0/2920258891 >> 172.21.15.60:0/2920258891 conn(0x7f70a0061a20 msgr2=0x7f70a0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 -- 172.21.15.60:0/2920258891 shutdown_connections 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.534+0000 7f70a6390640 1 -- 172.21.15.60:0/2920258891 wait complete. 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.535+0000 7f70a6390640 1 Processor -- start 2024-04-25T08:36:13.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 -- start start 2024-04-25T08:36:13.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a015dd20 0x7f70a00cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a01f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a01f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00d3c20 con 0x7f70a015e100 2024-04-25T08:36:13.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00d3aa0 con 0x7f70a015dd20 2024-04-25T08:36:13.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.536+0000 7f70a6390640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00d3da0 con 0x7f70a015e590 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a01f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a01f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39608/0 (socket says 172.21.15.60:39608) 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 -- 172.21.15.60:0/4209388899 learned_addr learned my addr 172.21.15.60:0/4209388899 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f70a492d640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a01f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709ffff640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a015dd20 0x7f70a00cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 -- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 msgr2=0x7f70a01f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a01f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 -- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a015dd20 msgr2=0x7f70a00cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a015dd20 0x7f70a00cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 -- 172.21.15.60:0/4209388899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00c9860 con 0x7f70a015e100 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709f7fe640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a01f3b50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f709002a180 tx=0x7f70900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090041020 con 0x7f70a015e100 2024-04-25T08:36:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70900306c0 con 0x7f70a015e100 2024-04-25T08:36:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a00cc320 con 0x7f70a015e100 2024-04-25T08:36:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090037040 con 0x7f70a015e100 2024-04-25T08:36:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.537+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a00cc8e0 con 0x7f70a015e100 2024-04-25T08:36:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.538+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a0073990 con 0x7f70a015e100 2024-04-25T08:36:13.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.539+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7090044020 con 0x7f70a015e100 2024-04-25T08:36:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.539+0000 7f709cff9640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f708006b500 0x7f708006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.539+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f70900aab30 con 0x7f70a015e100 2024-04-25T08:36:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.539+0000 7f709ffff640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f708006b500 0x7f708006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.540+0000 7f709ffff640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f708006b500 0x7f708006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7088006fd0 tx=0x7f7088008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.544+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f709004b030 con 0x7f70a015e100 2024-04-25T08:36:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.760+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70a00b6150 con 0x7f70a015e100 2024-04-25T08:36:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.760+0000 7f709cff9640 1 -- 172.21.15.60:0/4209388899 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f709007f020 con 0x7f70a015e100 2024-04-25T08:36:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:13.755 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:13.755 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f708006b500 msgr2=0x7f708006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f708006b500 0x7f708006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7088006fd0 tx=0x7f7088008040 comp rx=0 tx=0).stop 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 msgr2=0x7f70a01f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a01f3b50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f709002a180 tx=0x7f70900046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 shutdown_connections 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70a015e590 0x7f70a01f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70a015e100 0x7f70a01f3b50 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f708006b500 0x7f708006d9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 --2- 172.21.15.60:0/4209388899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70a015dd20 0x7f70a00cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 >> 172.21.15.60:0/4209388899 conn(0x7f70a0061a20 msgr2=0x7f70a00b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 shutdown_connections 2024-04-25T08:36:13.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.762+0000 7f70a6390640 1 -- 172.21.15.60:0/4209388899 wait complete. 2024-04-25T08:36:13.767 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:36:13.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.873+0000 7f229fd7e640 1 Processor -- start 2024-04-25T08:36:13.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.873+0000 7f229fd7e640 1 -- start start 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229fd7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 0x7f2298161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229fd7e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f22980b9730 0x7f22980ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229fd7e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2298162020 con 0x7f2298161b80 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229fd7e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22981625e0 con 0x7f22980b9730 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229fd7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2298162300 con 0x7f229815da70 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229e31b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2298161b80 0x7f229806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44328/0 (socket says 172.21.15.60:44328) 2024-04-25T08:36:13.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229e31b640 1 -- 172.21.15.60:0/2851442473 learned_addr learned my addr 172.21.15.60:0/2851442473 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229db1a640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 0x7f2298161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229d319640 1 --2- 172.21.15.60:0/2851442473 >> v2:172.21.15.100:3300/0 conn(0x7f22980b9730 0x7f22980ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813884474 0 0) 0x7f2298162020 con 0x7f2298161b80 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2284003520 con 0x7f2298161b80 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4053203693 0 0) 0x7f2284003520 con 0x7f2298161b80 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 msgr2=0x7f2298161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 0x7f2298161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 >> v2:172.21.15.100:3300/0 conn(0x7f22980b9730 msgr2=0x7f22980ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 --2- 172.21.15.60:0/2851442473 >> v2:172.21.15.100:3300/0 conn(0x7f22980b9730 0x7f22980ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22980d1af0 con 0x7f2298161b80 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f22900019f0 con 0x7f2298161b80 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 >> v1:172.21.15.60:6789/0 conn(0x7f2298161b80 legacy=0x7f229806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22840039b0 0x7f2284003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f2284004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 0x7f22840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22981625e0 con 0x7f2284004800 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2298162300 con 0x7f22840039b0 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.874+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228400b9c0 con 0x7f229815da70 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f2284004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229e31b640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 0x7f22840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229db1a640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22840039b0 0x7f2284003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 -- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 msgr2=0x7f22840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815da70 0x7f22840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 -- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22840039b0 msgr2=0x7f2284003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22840039b0 0x7f2284003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 -- 172.21.15.60:0/2851442473 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2284003520 con 0x7f2284004800 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.875+0000 7f229d319640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f2284004c80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2288009af0 tx=0x7f2284005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cdbeb36a11c41d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288036070 con 0x7f2284004800 2024-04-25T08:36:13.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288031400 con 0x7f2284004800 2024-04-25T08:36:13.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229cb18640 1 -- 172.21.15.60:0/2851442473 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288009690 con 0x7f2284004800 2024-04-25T08:36:13.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229fd7e640 1 -- 172.21.15.60:0/2851442473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 msgr2=0x7f2284004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229fd7e640 1 --2- 172.21.15.60:0/2851442473 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f2284004c80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f2288009af0 tx=0x7f2284005830 comp rx=0 tx=0).stop 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229e31b640 1 -- 172.21.15.60:0/2851442473 reap_dead start 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229fd7e640 1 -- 172.21.15.60:0/2851442473 shutdown_connections 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229fd7e640 1 -- 172.21.15.60:0/2851442473 >> 172.21.15.60:0/2851442473 conn(0x7f2298061a20 msgr2=0x7f2298061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229fd7e640 1 -- 172.21.15.60:0/2851442473 shutdown_connections 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.876+0000 7f229fd7e640 1 -- 172.21.15.60:0/2851442473 wait complete. 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.877+0000 7f229fd7e640 1 Processor -- start 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 -- start start 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f22981521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815f950 0x7f22980b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229815fdd0 0x7f22980b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22980d3300 con 0x7f2284004800 2024-04-25T08:36:13.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229815de50 con 0x7f229815fdd0 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229fd7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22980d3480 con 0x7f229815f950 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229db1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f22981521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229db1a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f22981521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39632/0 (socket says 172.21.15.60:39632) 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.878+0000 7f229db1a640 1 -- 172.21.15.60:0/424530684 learned_addr learned my addr 172.21.15.60:0/424530684 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229e31b640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229815fdd0 0x7f22980b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229d319640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815f950 0x7f22980b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229db1a640 1 -- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815f950 msgr2=0x7f22980b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229db1a640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815f950 0x7f22980b7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229db1a640 1 -- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229815fdd0 msgr2=0x7f22980b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229db1a640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229815fdd0 0x7f22980b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229db1a640 1 -- 172.21.15.60:0/424530684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22981526f0 con 0x7f2284004800 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229db1a640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f22981521b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f22940118e0 tx=0x7f2294011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400fa90 con 0x7f2284004800 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229400fc30 con 0x7f2284004800 2024-04-25T08:36:13.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22980c7d10 con 0x7f2284004800 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294014440 con 0x7f2284004800 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.879+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22980c82d0 con 0x7f2284004800 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.880+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22980c8ee0 con 0x7f2284004800 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.881+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2294012040 con 0x7f2284004800 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.881+0000 7f228e7fc640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f227c06b500 0x7f227c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.881+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f229408a930 con 0x7f2284004800 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.881+0000 7f229d319640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f227c06b500 0x7f227c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.882+0000 7f229d319640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f227c06b500 0x7f227c06d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2288033000 tx=0x7f2288008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:13.885+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2294019030 con 0x7f2284004800 2024-04-25T08:36:14.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.140+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2298073990 con 0x7f2284004800 2024-04-25T08:36:14.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.140+0000 7f228e7fc640 1 -- 172.21.15.60:0/424530684 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1811 (secure 0 0 0) 0x7f22940595b0 con 0x7f2284004800 2024-04-25T08:36:14.135 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:14.135 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":434,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2554806272,"bytes_avail":770539307008,"bytes_total":773094113280,"recovering_objects_per_sec":8,"recovering_bytes_per_sec":8779895,"recovering_keys_per_sec":0,"num_objects_recovered":67,"num_bytes_recovered":70254592,"num_keys_recovered":0,"write_bytes_sec":28063468,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:36:05.490012+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f227c06b500 msgr2=0x7f227c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f227c06b500 0x7f227c06d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2288033000 tx=0x7f2288008030 comp rx=0 tx=0).stop 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 msgr2=0x7f22981521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f22981521b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f22940118e0 tx=0x7f2294011da0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 shutdown_connections 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f227c06b500 0x7f227c06d9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f229815fdd0 0x7f22980b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f229815f950 0x7f22980b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 --2- 172.21.15.60:0/424530684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2284004800 0x7f22981521b0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 >> 172.21.15.60:0/424530684 conn(0x7f2298061a20 msgr2=0x7f229815c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 shutdown_connections 2024-04-25T08:36:14.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:14.142+0000 7f229fd7e640 1 -- 172.21.15.60:0/424530684 wait complete. 2024-04-25T08:36:14.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T08:36:15.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:15.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.268+0000 7f6ffcc6f640 1 Processor -- start 2024-04-25T08:36:15.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.268+0000 7f6ffcc6f640 1 -- start start 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ffcc6f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ff80cc8e0 0x7f6ff81625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ffcc6f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6ff80d1810 0x7f6ff80d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ffcc6f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff80cce50 con 0x7f6ff815dc90 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ffcc6f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff80d21c0 con 0x7f6ff80cc8e0 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ffcc6f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff8161af0 con 0x7f6ff80d1810 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6feffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6ff80cc8e0 0x7f6ff81625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ff659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6ff815dc90 0x7f6ff815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44344/0 (socket says 172.21.15.60:44344) 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ff659c640 1 -- 172.21.15.60:0/286216550 learned_addr learned my addr 172.21.15.60:0/286216550 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6ff6d9d640 1 --2- 172.21.15.60:0/286216550 >> v2:172.21.15.100:3300/0 conn(0x7f6ff80d1810 0x7f6ff80d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6feffff640 1 -- 172.21.15.60:0/286216550 >> v2:172.21.15.100:3300/0 conn(0x7f6ff80d1810 msgr2=0x7f6ff80d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6feffff640 1 --2- 172.21.15.60:0/286216550 >> v2:172.21.15.100:3300/0 conn(0x7f6ff80d1810 0x7f6ff80d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6feffff640 1 -- 172.21.15.60:0/286216550 >> v1:172.21.15.60:6789/0 conn(0x7f6ff815dc90 legacy=0x7f6ff815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.269+0000 7f6feffff640 1 -- 172.21.15.60:0/286216550 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8161dd0 con 0x7f6ff80cc8e0 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.270+0000 7f6feffff640 1 --2- 172.21.15.60:0/286216550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff81625d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80cdbb0 tx=0x7f6fe002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8d487196ec1694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.270+0000 7f6ff5d9b640 1 -- 172.21.15.60:0/286216550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0037070 con 0x7f6ff80cc8e0 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.270+0000 7f6ff5d9b640 1 -- 172.21.15.60:0/286216550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0032440 con 0x7f6ff80cc8e0 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.270+0000 7f6ff5d9b640 1 -- 172.21.15.60:0/286216550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe002ae60 con 0x7f6ff80cc8e0 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/286216550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80cc8e0 msgr2=0x7f6ff81625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/286216550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff81625d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80cdbb0 tx=0x7f6fe002a110 comp rx=0 tx=0).stop 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/286216550 shutdown_connections 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/286216550 >> v2:172.21.15.100:3300/0 conn(0x7f6ff80d1810 0x7f6ff80d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/286216550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff81625d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/286216550 >> 172.21.15.60:0/286216550 conn(0x7f6ff806f590 msgr2=0x7f6ff806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/286216550 shutdown_connections 2024-04-25T08:36:15.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.271+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/286216550 wait complete. 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.272+0000 7f6ffcc6f640 1 Processor -- start 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.272+0000 7f6ffcc6f640 1 -- start start 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ffcc6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff80cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ffcc6f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 0x7f6ff815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ffcc6f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ff815dc90 0x7f6ff815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ffcc6f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff815e070 con 0x7f6ff80d1810 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ffcc6f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80cd130 con 0x7f6ff80cc8e0 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ffcc6f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff815e1f0 con 0x7f6ff815dc90 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 0x7f6ff815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 0x7f6ff815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39658/0 (socket says 172.21.15.60:39658) 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 -- 172.21.15.60:0/1984806651 learned_addr learned my addr 172.21.15.60:0/1984806651 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ff6d9d640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ff815dc90 0x7f6ff815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ff659c640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff80cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 -- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ff815dc90 msgr2=0x7f6ff815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ff815dc90 0x7f6ff815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 -- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff80cc8e0 msgr2=0x7f6ff80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff80cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6feffff640 1 -- 172.21.15.60:0/1984806651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80c70a0 con 0x7f6ff80d1810 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.273+0000 7f6ff659c640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff80cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.274+0000 7f6feffff640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 0x7f6ff815aa10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.274+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0037040 con 0x7f6ff80d1810 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.274+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0042070 con 0x7f6ff80d1810 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.274+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff80c7380 con 0x7f6ff80d1810 2024-04-25T08:36:15.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.274+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0030bd0 con 0x7f6ff80d1810 2024-04-25T08:36:15.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.274+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff80c7940 con 0x7f6ff80d1810 2024-04-25T08:36:15.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.275+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff80ade00 con 0x7f6ff80d1810 2024-04-25T08:36:15.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.275+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fe0047460 con 0x7f6ff80d1810 2024-04-25T08:36:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.276+0000 7f6feeffd640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fd806b4d0 0x7f6fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.276+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6fe00aaf70 con 0x7f6ff80d1810 2024-04-25T08:36:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.276+0000 7f6ff659c640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fd806b4d0 0x7f6fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.277+0000 7f6ff659c640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fd806b4d0 0x7f6fd806d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8000f60 tx=0x7f6fe8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.280+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ff80ade00 con 0x7f6ff80d1810 2024-04-25T08:36:15.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.498+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ff80cf6b0 con 0x7f6ff80d1810 2024-04-25T08:36:15.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.498+0000 7f6feeffd640 1 -- 172.21.15.60:0/1984806651 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6fe0032e20 con 0x7f6ff80d1810 2024-04-25T08:36:15.493 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:15.493 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:15.493 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fd806b4d0 msgr2=0x7f6fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fd806b4d0 0x7f6fd806d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8000f60 tx=0x7f6fe8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 msgr2=0x7f6ff815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 0x7f6ff815aa10 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0000c00 tx=0x7f6fe00046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 shutdown_connections 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6ff815dc90 0x7f6ff815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ff80d1810 0x7f6ff815aa10 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6fd806b4d0 0x7f6fd806d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 --2- 172.21.15.60:0/1984806651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ff80cc8e0 0x7f6ff80cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 >> 172.21.15.60:0/1984806651 conn(0x7f6ff806f590 msgr2=0x7f6ff80b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 shutdown_connections 2024-04-25T08:36:15.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.500+0000 7f6ffcc6f640 1 -- 172.21.15.60:0/1984806651 wait complete. 2024-04-25T08:36:15.506 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:36:15.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:15.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.611+0000 7fd15ca38640 1 Processor -- start 2024-04-25T08:36:15.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.611+0000 7fd15ca38640 1 -- start start 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd15ca38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 0x7fd158162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd15ca38640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd158162690 con 0x7fd1580b5c30 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd15ca38640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580ceee0 con 0x7fd1580b6cb0 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd15ca38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580d1d10 con 0x7fd15815db70 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd156d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd1580b5c30 0x7fd1580d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44354/0 (socket says 172.21.15.60:44354) 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd156d9d640 1 -- 172.21.15.60:0/918478931 learned_addr learned my addr 172.21.15.60:0/918478931 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd15659c640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 0x7fd158162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920292238 0 0) 0x7fd158162690 con 0x7fd1580b5c30 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd144003520 con 0x7fd1580b5c30 2024-04-25T08:36:15.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561261071 0 0) 0x7fd144003520 con 0x7fd1580b5c30 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 msgr2=0x7fd158162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 0x7fd158162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 >> v1:172.21.15.100:6789/0 conn(0x7fd1580b6cb0 legacy=0x7fd1580b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580d1ff0 con 0x7fd1580b5c30 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fd1500019f0 con 0x7fd1580b5c30 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 >> v1:172.21.15.60:6789/0 conn(0x7fd1580b5c30 legacy=0x7fd1580d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 0x7fd144004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.612+0000 7fd155d9b640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd144004d40 0x7fd1440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580d1d10 con 0x7fd1440039e0 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14400bb50 con 0x7fd144004d40 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14400bd50 con 0x7fd15815db70 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd15659c640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd14ffff640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 0x7fd144004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd156d9d640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd144004d40 0x7fd1440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd15659c640 1 -- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 msgr2=0x7fd144004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd15659c640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd15815db70 0x7fd144004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd15659c640 1 -- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd144004d40 msgr2=0x7fd1440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd15659c640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd144004d40 0x7fd1440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.613+0000 7fd15659c640 1 -- 172.21.15.60:0/918478931 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144003520 con 0x7fd1440039e0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15659c640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd148009ac0 tx=0x7fd144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a690773a12d42bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148036070 con 0x7fd1440039e0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148031400 con 0x7fd1440039e0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd155d9b640 1 -- 172.21.15.60:0/918478931 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148009690 con 0x7fd1440039e0 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15ca38640 1 -- 172.21.15.60:0/918478931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1440039e0 msgr2=0x7fd144003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15ca38640 1 --2- 172.21.15.60:0/918478931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd148009ac0 tx=0x7fd144005830 comp rx=0 tx=0).stop 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd156d9d640 1 -- 172.21.15.60:0/918478931 reap_dead start 2024-04-25T08:36:15.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15ca38640 1 -- 172.21.15.60:0/918478931 shutdown_connections 2024-04-25T08:36:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15ca38640 1 -- 172.21.15.60:0/918478931 >> 172.21.15.60:0/918478931 conn(0x7fd1580d0520 msgr2=0x7fd1580d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15ca38640 1 -- 172.21.15.60:0/918478931 shutdown_connections 2024-04-25T08:36:15.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.614+0000 7fd15ca38640 1 -- 172.21.15.60:0/918478931 wait complete. 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.616+0000 7fd15ca38640 1 Processor -- start 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.616+0000 7fd15ca38640 1 -- start start 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1440039e0 0x7fd158156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 0x7fd158152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1581f6ba0 0x7fd1581f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580ce530 con 0x7fd1580cab70 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15815df50 con 0x7fd1581f6ba0 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580cf1c0 con 0x7fd1440039e0 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 0x7fd158152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 0x7fd158152fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39678/0 (socket says 172.21.15.60:39678) 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 -- 172.21.15.60:0/3381778666 learned_addr learned my addr 172.21.15.60:0/3381778666 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd156d9d640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1581f6ba0 0x7fd1581f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 -- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1440039e0 msgr2=0x7fd158156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15659c640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1440039e0 0x7fd158156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1440039e0 0x7fd158156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 -- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1581f6ba0 msgr2=0x7fd1581f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1581f6ba0 0x7fd1581f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 -- 172.21.15.60:0/3381778666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1581534e0 con 0x7fd1580cab70 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd156d9d640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1581f6ba0 0x7fd1581f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15659c640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1440039e0 0x7fd158156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14ffff640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 0x7fd158152fa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd14000f570 tx=0x7fd14000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1400052e0 con 0x7fd1580cab70 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd140018070 con 0x7fd1580cab70 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15815fc60 con 0x7fd1580cab70 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140003830 con 0x7fd1580cab70 2024-04-25T08:36:15.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.617+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1581601e0 con 0x7fd1580cab70 2024-04-25T08:36:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.618+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd158073990 con 0x7fd1580cab70 2024-04-25T08:36:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.619+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1400039d0 con 0x7fd1580cab70 2024-04-25T08:36:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.619+0000 7fd14effd640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd13806b500 0x7fd13806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.619+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd140089de0 con 0x7fd1580cab70 2024-04-25T08:36:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.620+0000 7fd15659c640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd13806b500 0x7fd13806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.620+0000 7fd15659c640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd13806b500 0x7fd13806d9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd148033000 tx=0x7fd148008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.623+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd140058a50 con 0x7fd1580cab70 2024-04-25T08:36:15.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.878+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd1580d1820 con 0x7fd1580cab70 2024-04-25T08:36:15.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.879+0000 7fd14effd640 1 -- 172.21.15.60:0/3381778666 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1811 (secure 0 0 0) 0x7fd140058350 con 0x7fd1580cab70 2024-04-25T08:36:15.873 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:15.873 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":436,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2554806272,"bytes_avail":770539307008,"bytes_total":773094113280,"recovering_objects_per_sec":8,"recovering_bytes_per_sec":8549929,"recovering_keys_per_sec":0,"num_objects_recovered":67,"num_bytes_recovered":70254592,"num_keys_recovered":0,"write_bytes_sec":27328420,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:36:05.490012+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.880+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd13806b500 msgr2=0x7fd13806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.880+0000 7fd15ca38640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd13806b500 0x7fd13806d9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd148033000 tx=0x7fd148008030 comp rx=0 tx=0).stop 2024-04-25T08:36:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.880+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 msgr2=0x7fd158152fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.880+0000 7fd15ca38640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 0x7fd158152fa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd14000f570 tx=0x7fd14000fa30 comp rx=0 tx=0).stop 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 shutdown_connections 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd13806b500 0x7fd13806d9c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1581f6ba0 0x7fd1581f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1580cab70 0x7fd158152fa0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 --2- 172.21.15.60:0/3381778666 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd1440039e0 0x7fd158156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 >> 172.21.15.60:0/3381778666 conn(0x7fd1580d0520 msgr2=0x7fd15815b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 shutdown_connections 2024-04-25T08:36:15.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:15.881+0000 7fd15ca38640 1 -- 172.21.15.60:0/3381778666 wait complete. 2024-04-25T08:36:15.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T08:36:16.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.005+0000 7f43d7929640 1 Processor -- start 2024-04-25T08:36:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.005+0000 7f43d7929640 1 -- start start 2024-04-25T08:36:17.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.006+0000 7f43d7929640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f43d00d1020 0x7f43d00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.006+0000 7f43d7929640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00b97a0 con 0x7f43d00b2cb0 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.006+0000 7f43d7929640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00d1c50 con 0x7f43d015dbe0 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.006+0000 7f43d7929640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d015dfc0 con 0x7f43d00d1020 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.006+0000 7f43d4ec4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f43d00b2cb0 0x7f43d00d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44364/0 (socket says 172.21.15.60:44364) 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.006+0000 7f43d4ec4640 1 -- 172.21.15.60:0/3637435482 learned_addr learned my addr 172.21.15.60:0/3637435482 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43d5ec6640 1 --2- 172.21.15.60:0/3637435482 >> v2:172.21.15.100:3300/0 conn(0x7f43d00d1020 0x7f43d00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43c7fff640 1 -- 172.21.15.60:0/3637435482 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250780843 0 0) 0x7f43d00b97a0 con 0x7f43d00b2cb0 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43c7fff640 1 -- 172.21.15.60:0/3637435482 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43bc003520 con 0x7f43d00b2cb0 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43c7fff640 1 -- 172.21.15.60:0/3637435482 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233724234 0 0) 0x7f43bc003520 con 0x7f43d00b2cb0 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43d5ec6640 1 -- 172.21.15.60:0/3637435482 >> v1:172.21.15.100:6789/0 conn(0x7f43d015dbe0 legacy=0x7f43d00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43d5ec6640 1 -- 172.21.15.60:0/3637435482 >> v1:172.21.15.60:6789/0 conn(0x7f43d00b2cb0 legacy=0x7f43d00d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.007+0000 7f43d5ec6640 1 -- 172.21.15.60:0/3637435482 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00d1f30 con 0x7f43d00d1020 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.008+0000 7f43d5ec6640 1 --2- 172.21.15.60:0/3637435482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d00d2e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009d90 tx=0x7f43cc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa05a3bb886d024a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.008+0000 7f43c7fff640 1 -- 172.21.15.60:0/3637435482 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0001670 con 0x7f43d00d1020 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.008+0000 7f43c7fff640 1 -- 172.21.15.60:0/3637435482 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc007540 con 0x7f43d00d1020 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.008+0000 7f43c7fff640 1 -- 172.21.15.60:0/3637435482 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0078f0 con 0x7f43d00d1020 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.008+0000 7f43d7929640 1 -- 172.21.15.60:0/3637435482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 msgr2=0x7f43d00d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.008+0000 7f43d7929640 1 --2- 172.21.15.60:0/3637435482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d00d2e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009d90 tx=0x7f43cc02a680 comp rx=0 tx=0).stop 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.009+0000 7f43d7929640 1 -- 172.21.15.60:0/3637435482 shutdown_connections 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.009+0000 7f43d7929640 1 --2- 172.21.15.60:0/3637435482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d00d2e40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.009+0000 7f43d7929640 1 -- 172.21.15.60:0/3637435482 >> 172.21.15.60:0/3637435482 conn(0x7f43d0061a20 msgr2=0x7f43d0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.009+0000 7f43d7929640 1 -- 172.21.15.60:0/3637435482 shutdown_connections 2024-04-25T08:36:17.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.009+0000 7f43d7929640 1 -- 172.21.15.60:0/3637435482 wait complete. 2024-04-25T08:36:17.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.010+0000 7f43d7929640 1 Processor -- start 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.010+0000 7f43d7929640 1 -- start start 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d7929640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 0x7f43d00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d7929640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d7929640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43d015dbe0 0x7f43d015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d7929640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00b92a0 con 0x7f43d00b2cb0 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d7929640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00d1a00 con 0x7f43d00d1020 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d7929640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d00b9420 con 0x7f43d015dbe0 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 0x7f43d00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 0x7f43d00c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39682/0 (socket says 172.21.15.60:39682) 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 -- 172.21.15.60:0/1732211971 learned_addr learned my addr 172.21.15.60:0/1732211971 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d4ec4640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d5ec6640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43d015dbe0 0x7f43d015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 -- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43d015dbe0 msgr2=0x7f43d015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43d015dbe0 0x7f43d015fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 -- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 msgr2=0x7f43d015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d015f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 -- 172.21.15.60:0/1732211971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0160530 con 0x7f43d00b2cb0 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d5ec6640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43d015dbe0 0x7f43d015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43d56c5640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 0x7f43d00c97e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c400 tx=0x7f43c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8017070 con 0x7f43d00b2cb0 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.011+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80052d0 con 0x7f43d00b2cb0 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.012+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d0160810 con 0x7f43d00b2cb0 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.012+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8003830 con 0x7f43d00b2cb0 2024-04-25T08:36:17.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.012+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d00b4190 con 0x7f43d00b2cb0 2024-04-25T08:36:17.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.013+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d0073990 con 0x7f43d00b2cb0 2024-04-25T08:36:17.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.013+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43c801a070 con 0x7f43d00b2cb0 2024-04-25T08:36:17.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.014+0000 7f43c5ffb640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f43b406b4d0 0x7f43b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.014+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f43c80896b0 con 0x7f43d00b2cb0 2024-04-25T08:36:17.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.014+0000 7f43d4ec4640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f43b406b4d0 0x7f43b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.014+0000 7f43d4ec4640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f43b406b4d0 0x7f43b406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f43d014a770 tx=0x7f43c0001dc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.018+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43c8057ce0 con 0x7f43d00b2cb0 2024-04-25T08:36:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.237+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43d015c660 con 0x7f43d00b2cb0 2024-04-25T08:36:17.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.237+0000 7f43c5ffb640 1 -- 172.21.15.60:0/1732211971 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f43c805bd80 con 0x7f43d00b2cb0 2024-04-25T08:36:17.232 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:17.232 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:17.232 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f43b406b4d0 msgr2=0x7f43b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f43b406b4d0 0x7f43b406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f43d014a770 tx=0x7f43c0001dc0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 msgr2=0x7f43d00c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 0x7f43d00c97e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c400 tx=0x7f43c800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 shutdown_connections 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43d015dbe0 0x7f43d015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f43b406b4d0 0x7f43b406d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43d00d1020 0x7f43d015f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 --2- 172.21.15.60:0/1732211971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43d00b2cb0 0x7f43d00c97e0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 >> 172.21.15.60:0/1732211971 conn(0x7f43d0061a20 msgr2=0x7f43d00b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 shutdown_connections 2024-04-25T08:36:17.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.239+0000 7f43d7929640 1 -- 172.21.15.60:0/1732211971 wait complete. 2024-04-25T08:36:17.245 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:36:17.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:17.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.350+0000 7f537bcf5640 1 Processor -- start 2024-04-25T08:36:17.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.350+0000 7f537bcf5640 1 -- start start 2024-04-25T08:36:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f537bcf5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f537415dbe0 0x7f53740d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f537bcf5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740b6220 con 0x7f53740b2cb0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f537bcf5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740b97a0 con 0x7f53740b92a0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f537bcf5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537415dfc0 con 0x7f537415dbe0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5379290640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f53740b2cb0 0x7f53740b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44378/0 (socket says 172.21.15.60:44378) 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5379290640 1 -- 172.21.15.60:0/2319181691 learned_addr learned my addr 172.21.15.60:0/2319181691 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5379a91640 1 --2- 172.21.15.60:0/2319181691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f537415dbe0 0x7f53740d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582110812 0 0) 0x7f53740b6220 con 0x7f53740b2cb0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5360003520 con 0x7f53740b2cb0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237538347 0 0) 0x7f53740b97a0 con 0x7f53740b92a0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53740b6220 con 0x7f53740b92a0 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5379a91640 1 -- 172.21.15.60:0/2319181691 >> v1:172.21.15.100:6789/0 conn(0x7f53740b92a0 legacy=0x7f53740b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5379a91640 1 -- 172.21.15.60:0/2319181691 >> v1:172.21.15.60:6789/0 conn(0x7f53740b2cb0 legacy=0x7f53740b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.351+0000 7f5379a91640 1 -- 172.21.15.60:0/2319181691 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740d1c50 con 0x7f537415dbe0 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.352+0000 7f5379a91640 1 --2- 172.21.15.60:0/2319181691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f537415dbe0 0x7f53740d0910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53700097d0 tx=0x7f537002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8440f94dfa5d4dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.352+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370037070 con 0x7f537415dbe0 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.352+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5370032440 con 0x7f537415dbe0 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.352+0000 7f5378a8f640 1 -- 172.21.15.60:0/2319181691 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700327f0 con 0x7f537415dbe0 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 -- 172.21.15.60:0/2319181691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f537415dbe0 msgr2=0x7f53740d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 --2- 172.21.15.60:0/2319181691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f537415dbe0 0x7f53740d0910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53700097d0 tx=0x7f537002a210 comp rx=0 tx=0).stop 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 -- 172.21.15.60:0/2319181691 shutdown_connections 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 --2- 172.21.15.60:0/2319181691 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f537415dbe0 0x7f53740d0910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 -- 172.21.15.60:0/2319181691 >> 172.21.15.60:0/2319181691 conn(0x7f5374061a20 msgr2=0x7f5374061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 -- 172.21.15.60:0/2319181691 shutdown_connections 2024-04-25T08:36:17.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.353+0000 7f537bcf5640 1 -- 172.21.15.60:0/2319181691 wait complete. 2024-04-25T08:36:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.354+0000 7f537bcf5640 1 Processor -- start 2024-04-25T08:36:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.354+0000 7f537bcf5640 1 -- start start 2024-04-25T08:36:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53740b2cb0 0x7f53741f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53740b92a0 0x7f537415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 0x7f53740aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740d2210 con 0x7f537415dbe0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740d1a00 con 0x7f53740b92a0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53740b6ab0 con 0x7f53740b2cb0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 0x7f53740aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 0x7f53740aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39686/0 (socket says 172.21.15.60:39686) 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 -- 172.21.15.60:0/3414630065 learned_addr learned my addr 172.21.15.60:0/3414630065 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f5379290640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53740b92a0 0x7f537415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f5379a91640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53740b2cb0 0x7f53741f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 -- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53740b2cb0 msgr2=0x7f53741f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53740b2cb0 0x7f53741f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 -- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53740b92a0 msgr2=0x7f537415fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53740b92a0 0x7f537415fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 -- 172.21.15.60:0/3414630065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374160730 con 0x7f537415dbe0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537a292640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 0x7f53740aa370 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c730 tx=0x7f536c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c003b70 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c003d10 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.355+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5374160a10 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.356+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00aa30 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.356+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53740cb830 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.356+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5374073990 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.357+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f536c00abd0 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.357+0000 7f536a7fc640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535806b4d0 0x7f535806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.357+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f536c08a1c0 con 0x7f537415dbe0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.358+0000 7f5379a91640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535806b4d0 0x7f535806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.358+0000 7f5379a91640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535806b4d0 0x7f535806d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5370034000 tx=0x7f537002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.362+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f536c0587f0 con 0x7f537415dbe0 2024-04-25T08:36:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.612+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f53740b2990 con 0x7f537415dbe0 2024-04-25T08:36:17.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.612+0000 7f536a7fc640 1 -- 172.21.15.60:0/3414630065 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1747 (secure 0 0 0) 0x7f536c05c890 con 0x7f537415dbe0 2024-04-25T08:36:17.607 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:17.607 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":438,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":99,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2554806272,"bytes_avail":770539307008,"bytes_total":773094113280,"recovering_objects_per_sec":6,"recovering_bytes_per_sec":7023763,"recovering_keys_per_sec":0,"num_objects_recovered":67,"num_bytes_recovered":70254592,"num_keys_recovered":0,"write_bytes_sec":9760664,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":55,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7626}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535806b4d0 msgr2=0x7f535806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535806b4d0 0x7f535806d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5370034000 tx=0x7f537002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 msgr2=0x7f53740aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 0x7f53740aa370 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f536c00c730 tx=0x7f536c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 shutdown_connections 2024-04-25T08:36:17.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f537415dbe0 0x7f53740aa370 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535806b4d0 0x7f535806d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53740b92a0 0x7f537415fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 --2- 172.21.15.60:0/3414630065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53740b2cb0 0x7f53741f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 >> 172.21.15.60:0/3414630065 conn(0x7f5374061a20 msgr2=0x7f53740d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 shutdown_connections 2024-04-25T08:36:17.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.614+0000 7f537bcf5640 1 -- 172.21.15.60:0/3414630065 wait complete. 2024-04-25T08:36:17.620 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:36:17.620 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:36:17.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:17.667 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:17.713 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:36:17.713 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:36:17.714 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:17.729 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:36:17.729 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:17.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.932+0000 7fbe5b517640 1 Processor -- start 2024-04-25T08:36:17.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe5b517640 1 -- start start 2024-04-25T08:36:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe5b517640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe5415db70 0x7fbe54162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe5b517640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe540b6cb0 0x7fbe540b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe5b517640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe54162690 con 0x7fbe540b5a60 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe5b517640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe540d3800 con 0x7fbe5415db70 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe5b517640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe540ceee0 con 0x7fbe540b6cb0 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe592b3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbe5415db70 0x7fbe54162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe59ab4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbe540b5a60 0x7fbe5406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44380/0 (socket says 172.21.15.60:44380) 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.933+0000 7fbe59ab4640 1 -- 172.21.15.60:0/178330200 learned_addr learned my addr 172.21.15.60:0/178330200 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe58ab2640 1 --2- 172.21.15.60:0/178330200 >> v2:172.21.15.100:3300/0 conn(0x7fbe540b6cb0 0x7fbe540b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe592b3640 1 -- 172.21.15.60:0/178330200 >> v2:172.21.15.100:3300/0 conn(0x7fbe540b6cb0 msgr2=0x7fbe540b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe592b3640 1 --2- 172.21.15.60:0/178330200 >> v2:172.21.15.100:3300/0 conn(0x7fbe540b6cb0 0x7fbe540b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe592b3640 1 -- 172.21.15.60:0/178330200 >> v1:172.21.15.60:6789/0 conn(0x7fbe540b5a60 legacy=0x7fbe5406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe592b3640 1 -- 172.21.15.60:0/178330200 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540d1d10 con 0x7fbe5415db70 2024-04-25T08:36:17.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe53fff640 1 -- 172.21.15.60:0/178330200 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238349717 0 0) 0x7fbe54162690 con 0x7fbe540b5a60 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.934+0000 7fbe592b3640 1 --2- 172.21.15.60:0/178330200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe54162150 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbe540d3980 tx=0x7fbe4c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20452ef43b0712ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe53fff640 1 -- 172.21.15.60:0/178330200 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c037070 con 0x7fbe5415db70 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe53fff640 1 -- 172.21.15.60:0/178330200 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c032440 con 0x7fbe5415db70 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe53fff640 1 -- 172.21.15.60:0/178330200 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c02ae60 con 0x7fbe5415db70 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 -- 172.21.15.60:0/178330200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 msgr2=0x7fbe54162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 --2- 172.21.15.60:0/178330200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe54162150 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbe540d3980 tx=0x7fbe4c02a110 comp rx=0 tx=0).stop 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 -- 172.21.15.60:0/178330200 shutdown_connections 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 --2- 172.21.15.60:0/178330200 >> v2:172.21.15.100:3300/0 conn(0x7fbe540b6cb0 0x7fbe540b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 --2- 172.21.15.60:0/178330200 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe54162150 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 -- 172.21.15.60:0/178330200 >> 172.21.15.60:0/178330200 conn(0x7fbe540d0520 msgr2=0x7fbe540d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 -- 172.21.15.60:0/178330200 shutdown_connections 2024-04-25T08:36:17.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.935+0000 7fbe5b517640 1 -- 172.21.15.60:0/178330200 wait complete. 2024-04-25T08:36:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.936+0000 7fbe5b517640 1 Processor -- start 2024-04-25T08:36:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 -- start start 2024-04-25T08:36:17.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe540b5a60 0x7fbe541f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe540b6cb0 0x7fbe5414c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe540aa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5415df50 con 0x7fbe5415db70 2024-04-25T08:36:17.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540d22d0 con 0x7fbe540b5a60 2024-04-25T08:36:17.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe5b517640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540ce530 con 0x7fbe540b6cb0 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe59ab4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe540aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe59ab4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe540aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39696/0 (socket says 172.21.15.60:39696) 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.937+0000 7fbe59ab4640 1 -- 172.21.15.60:0/3638852604 learned_addr learned my addr 172.21.15.60:0/3638852604 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe592b3640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe540b5a60 0x7fbe541f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe58ab2640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe540b6cb0 0x7fbe5414c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe59ab4640 1 -- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe540b6cb0 msgr2=0x7fbe5414c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe59ab4640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe540b6cb0 0x7fbe5414c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe59ab4640 1 -- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe540b5a60 msgr2=0x7fbe541f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe59ab4640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe540b5a60 0x7fbe541f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe59ab4640 1 -- 172.21.15.60:0/3638852604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe541f43b0 con 0x7fbe5415db70 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe592b3640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe540b5a60 0x7fbe541f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe59ab4640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe540aa350 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c810 tx=0x7fbe4400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44018070 con 0x7fbe5415db70 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4400a640 con 0x7fbe5415db70 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe540cbda0 con 0x7fbe5415db70 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400aa30 con 0x7fbe5415db70 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.938+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe540cc360 con 0x7fbe5415db70 2024-04-25T08:36:17.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.939+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe54073990 con 0x7fbe5415db70 2024-04-25T08:36:17.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.940+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe4401b070 con 0x7fbe5415db70 2024-04-25T08:36:17.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.940+0000 7fbe51ffb640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbe4006b4d0 0x7fbe4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.940+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fbe44089a20 con 0x7fbe5415db70 2024-04-25T08:36:17.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.940+0000 7fbe592b3640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbe4006b4d0 0x7fbe4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.941+0000 7fbe592b3640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbe4006b4d0 0x7fbe4006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c034000 tx=0x7fbe4c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:17.945+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe440586a0 con 0x7fbe5415db70 2024-04-25T08:36:18.119 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.124+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe540b7ce0 con 0x7fbe5415db70 2024-04-25T08:36:18.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.126+0000 7fbe51ffb640 1 -- 172.21.15.60:0/3638852604 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17067 (secure 0 0 0) 0x7fbe44057fa0 con 0x7fbe5415db70 2024-04-25T08:36:18.120 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:18.121 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:06.273972+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":2}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":5}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:18.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.127+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbe4006b4d0 msgr2=0x7fbe4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.127+0000 7fbe5b517640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbe4006b4d0 0x7fbe4006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c034000 tx=0x7fbe4c02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:18.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.127+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 msgr2=0x7fbe540aa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.127+0000 7fbe5b517640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe540aa350 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c810 tx=0x7fbe4400ccd0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 shutdown_connections 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbe5415db70 0x7fbe540aa350 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbe540b6cb0 0x7fbe5414c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbe4006b4d0 0x7fbe4006d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 --2- 172.21.15.60:0/3638852604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe540b5a60 0x7fbe541f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 >> 172.21.15.60:0/3638852604 conn(0x7fbe540d0520 msgr2=0x7fbe540b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 shutdown_connections 2024-04-25T08:36:18.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.128+0000 7fbe5b517640 1 -- 172.21.15.60:0/3638852604 wait complete. 2024-04-25T08:36:18.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:35:40.718590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:35:41.409652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:18.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:18.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.242+0000 7f3866de0640 1 Processor -- start 2024-04-25T08:36:18.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.242+0000 7f3866de0640 1 -- start start 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f3866de0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38600cc4f0 0x7f38600cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f3866de0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38600cdb20 0x7f38600cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f3866de0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3860072280 con 0x7f386015b270 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f3866de0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3860071270 con 0x7f38600cdb20 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f3866de0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386015bd80 con 0x7f38600cc4f0 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f385ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38600cdb20 0x7f38600cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f386537d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f386015b270 0x7f386015b670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44390/0 (socket says 172.21.15.60:44390) 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f386537d640 1 -- 172.21.15.60:0/3935051924 learned_addr learned my addr 172.21.15.60:0/3935051924 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.243+0000 7f3864b7c640 1 --2- 172.21.15.60:0/3935051924 >> v2:172.21.15.100:3300/0 conn(0x7f38600cc4f0 0x7f38600cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385ffff640 1 -- 172.21.15.60:0/3935051924 >> v2:172.21.15.100:3300/0 conn(0x7f38600cc4f0 msgr2=0x7f38600cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385ffff640 1 --2- 172.21.15.60:0/3935051924 >> v2:172.21.15.100:3300/0 conn(0x7f38600cc4f0 0x7f38600cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385ffff640 1 -- 172.21.15.60:0/3935051924 >> v1:172.21.15.60:6789/0 conn(0x7f386015b270 legacy=0x7f386015b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385f7fe640 1 -- 172.21.15.60:0/3935051924 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114554143 0 0) 0x7f3860072280 con 0x7f386015b270 2024-04-25T08:36:18.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385ffff640 1 -- 172.21.15.60:0/3935051924 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600714c0 con 0x7f38600cdb20 2024-04-25T08:36:18.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385ffff640 1 --2- 172.21.15.60:0/3935051924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600cdfb0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f38600cd290 tx=0x7f385002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19ae9083c09cb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385f7fe640 1 -- 172.21.15.60:0/3935051924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850007740 con 0x7f38600cdb20 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.244+0000 7f385f7fe640 1 -- 172.21.15.60:0/3935051924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38500078e0 con 0x7f38600cdb20 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f385f7fe640 1 -- 172.21.15.60:0/3935051924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850007cb0 con 0x7f38600cdb20 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 -- 172.21.15.60:0/3935051924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 msgr2=0x7f38600cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 --2- 172.21.15.60:0/3935051924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600cdfb0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f38600cd290 tx=0x7f385002a680 comp rx=0 tx=0).stop 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 -- 172.21.15.60:0/3935051924 shutdown_connections 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 --2- 172.21.15.60:0/3935051924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600cdfb0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 --2- 172.21.15.60:0/3935051924 >> v2:172.21.15.100:3300/0 conn(0x7f38600cc4f0 0x7f38600cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 -- 172.21.15.60:0/3935051924 >> 172.21.15.60:0/3935051924 conn(0x7f386006abe0 msgr2=0x7f386006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 -- 172.21.15.60:0/3935051924 shutdown_connections 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.245+0000 7f3866de0640 1 -- 172.21.15.60:0/3935051924 wait complete. 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.246+0000 7f3866de0640 1 Processor -- start 2024-04-25T08:36:18.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.246+0000 7f3866de0640 1 -- start start 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f3866de0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38600cc4f0 0x7f3860149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f3866de0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f3866de0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386015b270 0x7f38600b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f3866de0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860071d60 con 0x7f38600cdb20 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f3866de0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860071a80 con 0x7f386015b270 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f3866de0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38600ae500 con 0x7f38600cc4f0 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f385ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f385ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39726/0 (socket says 172.21.15.60:39726) 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.247+0000 7f385ffff640 1 -- 172.21.15.60:0/3481907309 learned_addr learned my addr 172.21.15.60:0/3481907309 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f3864b7c640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38600cc4f0 0x7f3860149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f386537d640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386015b270 0x7f38600b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385ffff640 1 -- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38600cc4f0 msgr2=0x7f3860149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385ffff640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38600cc4f0 0x7f3860149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385ffff640 1 -- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386015b270 msgr2=0x7f38600b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385ffff640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386015b270 0x7f38600b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385ffff640 1 -- 172.21.15.60:0/3481907309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860153590 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f386537d640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386015b270 0x7f38600b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385ffff640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600b0e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f385002ac70 tx=0x7f3850004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850040070 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3850031c40 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385003c420 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3860153870 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.248+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3860153e30 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.249+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860073a00 con 0x7f38600cdb20 2024-04-25T08:36:18.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.250+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3850039040 con 0x7f38600cdb20 2024-04-25T08:36:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.250+0000 7f385d7fa640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f384006b500 0x7f384006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.250+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f38500ab920 con 0x7f38600cdb20 2024-04-25T08:36:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.250+0000 7f3864b7c640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f384006b500 0x7f384006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.251+0000 7f3864b7c640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f384006b500 0x7f384006d9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3858006fd0 tx=0x7f3858008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.255+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f385007a5a0 con 0x7f38600cdb20 2024-04-25T08:36:18.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.436+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f386006d3a0 con 0x7f38600cdb20 2024-04-25T08:36:18.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.437+0000 7f385d7fa640 1 -- 172.21.15.60:0/3481907309 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3850079ea0 con 0x7f38600cdb20 2024-04-25T08:36:18.432 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:36:18.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f384006b500 msgr2=0x7f384006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f384006b500 0x7f384006d9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3858006fd0 tx=0x7f3858008040 comp rx=0 tx=0).stop 2024-04-25T08:36:18.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 msgr2=0x7f38600b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600b0e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f385002ac70 tx=0x7f3850004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:18.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 shutdown_connections 2024-04-25T08:36:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f384006b500 0x7f384006d9c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f386015b270 0x7f38600b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38600cdb20 0x7f38600b0e70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 --2- 172.21.15.60:0/3481907309 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38600cc4f0 0x7f3860149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 >> 172.21.15.60:0/3481907309 conn(0x7f386006abe0 msgr2=0x7f38600b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.438+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 shutdown_connections 2024-04-25T08:36:18.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.439+0000 7f3866de0640 1 -- 172.21.15.60:0/3481907309 wait complete. 2024-04-25T08:36:18.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:36:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.552+0000 7f0d8f2c4640 1 Processor -- start 2024-04-25T08:36:18.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 -- start start 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d8815dac0 0x7f0d880d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d880d1f30 0x7f0d880d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d880d34c0 0x7f0d880d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d880d3030 con 0x7f0d880d1f30 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d880d2420 con 0x7f0d8815dac0 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8f2c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880d2700 con 0x7f0d880d34c0 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.553+0000 7f0d8c85f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d880d1f30 0x7f0d880d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d880d1f30 0x7f0d880d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39738/0 (socket says 172.21.15.60:39738) 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 -- 172.21.15.60:0/854224829 learned_addr learned my addr 172.21.15.60:0/854224829 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8d060640 1 --2- 172.21.15.60:0/854224829 >> v2:172.21.15.100:3300/0 conn(0x7f0d8815dac0 0x7f0d880d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8d861640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d880d34c0 0x7f0d880d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 -- 172.21.15.60:0/854224829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d880d34c0 msgr2=0x7f0d880d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d880d34c0 0x7f0d880d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 -- 172.21.15.60:0/854224829 >> v2:172.21.15.100:3300/0 conn(0x7f0d8815dac0 msgr2=0x7f0d880d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/854224829 >> v2:172.21.15.100:3300/0 conn(0x7f0d8815dac0 0x7f0d880d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8c85f640 1 -- 172.21.15.60:0/854224829 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880d29e0 con 0x7f0d880d1f30 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8d060640 1 --2- 172.21.15.60:0/854224829 >> v2:172.21.15.100:3300/0 conn(0x7f0d8815dac0 0x7f0d880d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:18.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.554+0000 7f0d8d861640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d880d34c0 0x7f0d880d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d880d03a0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f0d880d2910 tx=0x7f0d7c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2360653a9153a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d87fff640 1 -- 172.21.15.60:0/854224829 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004030 con 0x7f0d880d1f30 2024-04-25T08:36:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d87fff640 1 -- 172.21.15.60:0/854224829 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c0041d0 con 0x7f0d880d1f30 2024-04-25T08:36:18.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d87fff640 1 -- 172.21.15.60:0/854224829 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c003610 con 0x7f0d880d1f30 2024-04-25T08:36:18.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/854224829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 msgr2=0x7f0d880d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d880d03a0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f0d880d2910 tx=0x7f0d7c02a250 comp rx=0 tx=0).stop 2024-04-25T08:36:18.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/854224829 shutdown_connections 2024-04-25T08:36:18.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d880d34c0 0x7f0d880d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/854224829 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d880d03a0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/854224829 >> v2:172.21.15.100:3300/0 conn(0x7f0d8815dac0 0x7f0d880d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/854224829 >> 172.21.15.60:0/854224829 conn(0x7f0d88061c30 msgr2=0x7f0d8806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/854224829 shutdown_connections 2024-04-25T08:36:18.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.555+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/854224829 wait complete. 2024-04-25T08:36:18.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 Processor -- start 2024-04-25T08:36:18.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 -- start start 2024-04-25T08:36:18.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d88149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d880d34c0 0x7f0d8815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d8815dac0 0x7f0d8815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880b97d0 con 0x7f0d880d1f30 2024-04-25T08:36:18.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8815dea0 con 0x7f0d880d34c0 2024-04-25T08:36:18.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8f2c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880b9950 con 0x7f0d8815dac0 2024-04-25T08:36:18.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8d060640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d88149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8d060640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d88149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39742/0 (socket says 172.21.15.60:39742) 2024-04-25T08:36:18.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.557+0000 7f0d8d060640 1 -- 172.21.15.60:0/3457255906 learned_addr learned my addr 172.21.15.60:0/3457255906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:18.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d880d34c0 0x7f0d8815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d861640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d8815dac0 0x7f0d8815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d060640 1 -- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d8815dac0 msgr2=0x7f0d8815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d060640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d8815dac0 0x7f0d8815fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d060640 1 -- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d880d34c0 msgr2=0x7f0d8815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d060640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d880d34c0 0x7f0d8815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d060640 1 -- 172.21.15.60:0/3457255906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d881601b0 con 0x7f0d880d1f30 2024-04-25T08:36:18.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d880d34c0 0x7f0d8815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:18.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8d060640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d88149d60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c7f0 tx=0x7f0d8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80015070 con 0x7f0d880d1f30 2024-04-25T08:36:18.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d80007a30 con 0x7f0d880d1f30 2024-04-25T08:36:18.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80007e00 con 0x7f0d880d1f30 2024-04-25T08:36:18.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d88160490 con 0x7f0d880d1f30 2024-04-25T08:36:18.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.558+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88160a50 con 0x7f0d880d1f30 2024-04-25T08:36:18.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.559+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d88073a00 con 0x7f0d880d1f30 2024-04-25T08:36:18.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.560+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d80018070 con 0x7f0d880d1f30 2024-04-25T08:36:18.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.560+0000 7f0d85ffb640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d7406b4d0 0x7f0d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.560+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0d8008a390 con 0x7f0d880d1f30 2024-04-25T08:36:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.560+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d7406b4d0 0x7f0d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.561+0000 7f0d8c85f640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d7406b4d0 0x7f0d7406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c033000 tx=0x7f0d7c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.564+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d80059010 con 0x7f0d880d1f30 2024-04-25T08:36:18.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.744+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d880d19e0 con 0x7f0d880d1f30 2024-04-25T08:36:18.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.745+0000 7f0d85ffb640 1 -- 172.21.15.60:0/3457255906 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d80058910 con 0x7f0d880d1f30 2024-04-25T08:36:18.739 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:36:18.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d7406b4d0 msgr2=0x7f0d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d7406b4d0 0x7f0d7406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c033000 tx=0x7f0d7c02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 msgr2=0x7f0d88149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d88149d60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000c7f0 tx=0x7f0d8000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 shutdown_connections 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d8815dac0 0x7f0d8815fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0d7406b4d0 0x7f0d7406d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d880d34c0 0x7f0d8815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 --2- 172.21.15.60:0/3457255906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d880d1f30 0x7f0d88149d60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 >> 172.21.15.60:0/3457255906 conn(0x7f0d88061c30 msgr2=0x7f0d880c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 shutdown_connections 2024-04-25T08:36:18.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.747+0000 7f0d8f2c4640 1 -- 172.21.15.60:0/3457255906 wait complete. 2024-04-25T08:36:18.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:36:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.861+0000 7ff4dc56a640 1 Processor -- start 2024-04-25T08:36:18.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.861+0000 7ff4dc56a640 1 -- start start 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4dc56a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff4d40ced80 0x7ff4d40d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4dc56a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff4d4161710 0x7ff4d40d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4dc56a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d415e000 con 0x7ff4d415dc20 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4dc56a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d40d1de0 con 0x7ff4d4161710 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4dc56a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d4161c60 con 0x7ff4d40ced80 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4da306640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff4d415dc20 0x7ff4d40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44392/0 (socket says 172.21.15.60:44392) 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4da306640 1 -- 172.21.15.60:0/4239938753 learned_addr learned my addr 172.21.15.60:0/4239938753 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:18.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4dab07640 1 --2- 172.21.15.60:0/4239938753 >> v2:172.21.15.60:3300/0 conn(0x7ff4d4161710 0x7ff4d40d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4d9b05640 1 --2- 172.21.15.60:0/4239938753 >> v2:172.21.15.100:3300/0 conn(0x7ff4d40ced80 0x7ff4d40d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.862+0000 7ff4d9304640 1 -- 172.21.15.60:0/4239938753 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534362021 0 0) 0x7ff4d415e000 con 0x7ff4d415dc20 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4d9304640 1 -- 172.21.15.60:0/4239938753 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4bc003520 con 0x7ff4d415dc20 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4dab07640 1 -- 172.21.15.60:0/4239938753 >> v2:172.21.15.100:3300/0 conn(0x7ff4d40ced80 msgr2=0x7ff4d40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4dab07640 1 --2- 172.21.15.60:0/4239938753 >> v2:172.21.15.100:3300/0 conn(0x7ff4d40ced80 0x7ff4d40d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4dab07640 1 -- 172.21.15.60:0/4239938753 >> v1:172.21.15.60:6789/0 conn(0x7ff4d415dc20 legacy=0x7ff4d40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4dab07640 1 -- 172.21.15.60:0/4239938753 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4161f40 con 0x7ff4d4161710 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4d9b05640 1 --2- 172.21.15.60:0/4239938753 >> v2:172.21.15.100:3300/0 conn(0x7ff4d40ced80 0x7ff4d40d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4dab07640 1 --2- 172.21.15.60:0/4239938753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d4161710 0x7ff4d40d2820 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff4d406da90 tx=0x7ff4cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c8a4cc9855f943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4d9304640 1 -- 172.21.15.60:0/4239938753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc037070 con 0x7ff4d4161710 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4d9304640 1 -- 172.21.15.60:0/4239938753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc032440 con 0x7ff4d4161710 2024-04-25T08:36:18.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.863+0000 7ff4d9304640 1 -- 172.21.15.60:0/4239938753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc032810 con 0x7ff4d4161710 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 -- 172.21.15.60:0/4239938753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d4161710 msgr2=0x7ff4d40d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/4239938753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d4161710 0x7ff4d40d2820 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ff4d406da90 tx=0x7ff4cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 -- 172.21.15.60:0/4239938753 shutdown_connections 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/4239938753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d4161710 0x7ff4d40d2820 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/4239938753 >> v2:172.21.15.100:3300/0 conn(0x7ff4d40ced80 0x7ff4d40d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 -- 172.21.15.60:0/4239938753 >> 172.21.15.60:0/4239938753 conn(0x7ff4d40d0420 msgr2=0x7ff4d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 -- 172.21.15.60:0/4239938753 shutdown_connections 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.864+0000 7ff4dc56a640 1 -- 172.21.15.60:0/4239938753 wait complete. 2024-04-25T08:36:18.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.865+0000 7ff4dc56a640 1 Processor -- start 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.865+0000 7ff4dc56a640 1 -- start start 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dc56a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 0x7ff4d4154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dc56a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d415dc20 0x7ff4d41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dc56a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d4161710 0x7ff4d415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dc56a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b5e50 con 0x7ff4d40ced80 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dc56a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d4162500 con 0x7ff4d4161710 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dc56a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d40b5fd0 con 0x7ff4d415dc20 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 0x7ff4d4154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 0x7ff4d4154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39756/0 (socket says 172.21.15.60:39756) 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 -- 172.21.15.60:0/2946708560 learned_addr learned my addr 172.21.15.60:0/2946708560 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4dab07640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d4161710 0x7ff4d415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4d9b05640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d415dc20 0x7ff4d41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 -- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d415dc20 msgr2=0x7ff4d41547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d415dc20 0x7ff4d41547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 -- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d4161710 msgr2=0x7ff4d415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d4161710 0x7ff4d415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.866+0000 7ff4da306640 1 -- 172.21.15.60:0/2946708560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4154d10 con 0x7ff4d40ced80 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.867+0000 7ff4da306640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 0x7ff4d4154f00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400cad0 tx=0x7ff4c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.867+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4017070 con 0x7ff4d40ced80 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.867+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c400ac30 con 0x7ff4d40ced80 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.867+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d4160760 con 0x7ff4d40ced80 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.867+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4012400 con 0x7ff4d40ced80 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.867+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d40c7730 con 0x7ff4d40ced80 2024-04-25T08:36:18.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.868+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d4073a00 con 0x7ff4d40ced80 2024-04-25T08:36:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.868+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4c401a070 con 0x7ff4d40ced80 2024-04-25T08:36:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.868+0000 7ff4d2ffd640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4c006b4d0 0x7ff4c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.869+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff4c408a170 con 0x7ff4d40ced80 2024-04-25T08:36:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.869+0000 7ff4d9b05640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4c006b4d0 0x7ff4c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.869+0000 7ff4d9b05640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4c006b4d0 0x7ff4c006d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40cf960 tx=0x7ff4c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:18.873+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4c4058e80 con 0x7ff4d40ced80 2024-04-25T08:36:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.057+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4d40d16b0 con 0x7ff4d40ced80 2024-04-25T08:36:19.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.057+0000 7ff4d2ffd640 1 -- 172.21.15.60:0/2946708560 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4c4058780 con 0x7ff4d40ced80 2024-04-25T08:36:19.052 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4c006b4d0 msgr2=0x7ff4c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4c006b4d0 0x7ff4c006d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40cf960 tx=0x7ff4c8008040 comp rx=0 tx=0).stop 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 msgr2=0x7ff4d4154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 0x7ff4d4154f00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400cad0 tx=0x7ff4c400cf90 comp rx=0 tx=0).stop 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 shutdown_connections 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff4c006b4d0 0x7ff4c006d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d4161710 0x7ff4d415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff4d415dc20 0x7ff4d41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 --2- 172.21.15.60:0/2946708560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4d40ced80 0x7ff4d4154f00 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 >> 172.21.15.60:0/2946708560 conn(0x7ff4d40d0420 msgr2=0x7ff4d40d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 shutdown_connections 2024-04-25T08:36:19.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.059+0000 7ff4dc56a640 1 -- 172.21.15.60:0/2946708560 wait complete. 2024-04-25T08:36:19.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:36:19.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.172+0000 7fb2a5ff4640 1 Processor -- start 2024-04-25T08:36:19.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.172+0000 7fb2a5ff4640 1 -- start start 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.172+0000 7fb2a5ff4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb2a00cf600 0x7fb2a00cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.172+0000 7fb2a5ff4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00c75e0 con 0x7fb2a015bb50 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.172+0000 7fb2a5ff4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00c78c0 con 0x7fb2a00cbf20 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.172+0000 7fb2a5ff4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a00c8680 con 0x7fb2a00cf600 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb2a015bb50 0x7fb2a00c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44404/0 (socket says 172.21.15.60:44404) 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29f7fe640 1 -- 172.21.15.60:0/1604441337 learned_addr learned my addr 172.21.15.60:0/1604441337 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29effd640 1 --2- 172.21.15.60:0/1604441337 >> v2:172.21.15.100:3300/0 conn(0x7fb2a00cf600 0x7fb2a00cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816127130 0 0) 0x7fb2a00c75e0 con 0x7fb2a015bb50 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb288003520 con 0x7fb2a015bb50 2024-04-25T08:36:19.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2571546843 0 0) 0x7fb288003520 con 0x7fb2a015bb50 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 >> v2:172.21.15.100:3300/0 conn(0x7fb2a00cf600 msgr2=0x7fb2a00cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 --2- 172.21.15.60:0/1604441337 >> v2:172.21.15.100:3300/0 conn(0x7fb2a00cf600 0x7fb2a00cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 >> v1:172.21.15.100:6789/0 conn(0x7fb2a00cbf20 legacy=0x7fb2a00cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00c7ba0 con 0x7fb2a015bb50 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb290001670 con 0x7fb2a015bb50 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 >> v1:172.21.15.60:6789/0 conn(0x7fb2a015bb50 legacy=0x7fb2a00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2880039b0 0x7fb288003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb288004800 0x7fb288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2880051c0 0x7fb28800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00c8680 con 0x7fb288004800 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28800c070 con 0x7fb2880051c0 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28800c270 con 0x7fb2880039b0 2024-04-25T08:36:19.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29effd640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb288004800 0x7fb288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29f7fe640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2880039b0 0x7fb288003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.173+0000 7fb29ffff640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2880051c0 0x7fb28800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29effd640 1 -- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2880039b0 msgr2=0x7fb288003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29effd640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2880039b0 0x7fb288003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29effd640 1 -- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2880051c0 msgr2=0x7fb28800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29effd640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2880051c0 0x7fb28800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29effd640 1 -- 172.21.15.60:0/1604441337 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288003520 con 0x7fb288004800 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29ffff640 1 -- 172.21.15.60:0/1604441337 reap_dead start 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29effd640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb288004800 0x7fb288004c80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb294009ac0 tx=0x7fb288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56d709fb247d9aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294036070 con 0x7fb288004800 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294031400 con 0x7fb288004800 2024-04-25T08:36:19.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.174+0000 7fb29e7fc640 1 -- 172.21.15.60:0/1604441337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294009690 con 0x7fb288004800 2024-04-25T08:36:19.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1604441337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb288004800 msgr2=0x7fb288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb288004800 0x7fb288004c80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb294009ac0 tx=0x7fb288005830 comp rx=0 tx=0).stop 2024-04-25T08:36:19.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1604441337 shutdown_connections 2024-04-25T08:36:19.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1604441337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb288004800 0x7fb288004c80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.174 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1604441337 >> 172.21.15.60:0/1604441337 conn(0x7fb2a006f500 msgr2=0x7fb2a006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1604441337 shutdown_connections 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.175+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1604441337 wait complete. 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.176+0000 7fb2a5ff4640 1 Processor -- start 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.176+0000 7fb2a5ff4640 1 -- start start 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb288004800 0x7fb2a0150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01562c0 0x7fb2a0156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb2a5ff4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 0x7fb2a00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015f6d0 con 0x7fb2a00b4f30 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00c8160 con 0x7fb288004800 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb2a5ff4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015f850 con 0x7fb2a01562c0 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 0x7fb2a00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 0x7fb2a00b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39780/0 (socket says 172.21.15.60:39780) 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 -- 172.21.15.60:0/1736112488 learned_addr learned my addr 172.21.15.60:0/1736112488 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:19.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29effd640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01562c0 0x7fb2a0156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29f7fe640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb288004800 0x7fb2a0150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 -- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01562c0 msgr2=0x7fb2a0156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01562c0 0x7fb2a0156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 -- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb288004800 msgr2=0x7fb2a0150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb288004800 0x7fb2a0150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29ffff640 1 -- 172.21.15.60:0/1736112488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a01f14e0 con 0x7fb2a00b4f30 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.177+0000 7fb29f7fe640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb288004800 0x7fb2a0150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.178+0000 7fb29ffff640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 0x7fb2a00b53b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb2a006f270 tx=0x7fb29800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.178+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29801a070 con 0x7fb2a00b4f30 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.178+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298006d60 con 0x7fb2a00b4f30 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.178+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298003bb0 con 0x7fb2a00b4f30 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.178+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0151200 con 0x7fb2a00b4f30 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.178+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a00b5f40 con 0x7fb2a00b4f30 2024-04-25T08:36:19.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.179+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a0073a00 con 0x7fb2a00b4f30 2024-04-25T08:36:19.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.179+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb298005070 con 0x7fb2a00b4f30 2024-04-25T08:36:19.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.180+0000 7fb287fff640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb28c06b4d0 0x7fb28c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.180+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb2980897d0 con 0x7fb2a00b4f30 2024-04-25T08:36:19.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.180+0000 7fb29f7fe640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb28c06b4d0 0x7fb28c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.180+0000 7fb29f7fe640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb28c06b4d0 0x7fb28c06d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0068c30 tx=0x7fb290009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.184+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb298057e00 con 0x7fb2a00b4f30 2024-04-25T08:36:19.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.377+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb2a00cf110 con 0x7fb2a00b4f30 2024-04-25T08:36:19.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.378+0000 7fb287fff640 1 -- 172.21.15.60:0/1736112488 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb29805bea0 con 0x7fb2a00b4f30 2024-04-25T08:36:19.372 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:36:19.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.379+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb28c06b4d0 msgr2=0x7fb28c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.379+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb28c06b4d0 0x7fb28c06d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0068c30 tx=0x7fb290009040 comp rx=0 tx=0).stop 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 msgr2=0x7fb2a00b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 0x7fb2a00b53b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb2a006f270 tx=0x7fb29800fc90 comp rx=0 tx=0).stop 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 shutdown_connections 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb2a00b4f30 0x7fb2a00b53b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb2a01562c0 0x7fb2a0156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb28c06b4d0 0x7fb28c06d990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 --2- 172.21.15.60:0/1736112488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb288004800 0x7fb2a0150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 >> 172.21.15.60:0/1736112488 conn(0x7fb2a006f500 msgr2=0x7fb2a00b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 shutdown_connections 2024-04-25T08:36:19.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:19.380+0000 7fb2a5ff4640 1 -- 172.21.15.60:0/1736112488 wait complete. 2024-04-25T08:36:19.385 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:36:19.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:19.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:20.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:36:20.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:20.001 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:36:20.019 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:36:20.019 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:20.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:36:20.036 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:36:20.036 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:20.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:36:20.075 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:36:20.076 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:20.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:36:20.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.221+0000 7f600da4a640 1 Processor -- start 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.221+0000 7f600da4a640 1 -- start start 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f600da4a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600815db60 0x7f60080ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f600da4a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60080b5f00 0x7f60080d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f600da4a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f600815df40 0x7f60080d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f600da4a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080ce930 con 0x7f60080b5f00 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f600da4a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080b6450 con 0x7f600815df40 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f600da4a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60080cec00 con 0x7f600815db60 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f60067fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60080b5f00 0x7f60080d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f60067fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f60080b5f00 0x7f60080d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39788/0 (socket says 172.21.15.60:39788) 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.222+0000 7f60067fc640 1 -- 172.21.15.60:0/1599137152 learned_addr learned my addr 172.21.15.60:0/1599137152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f6006ffd640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600815db60 0x7f60080ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60077fe640 1 --2- 172.21.15.60:0/1599137152 >> v2:172.21.15.100:3300/0 conn(0x7f600815df40 0x7f60080d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60067fc640 1 -- 172.21.15.60:0/1599137152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600815db60 msgr2=0x7f60080ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60067fc640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600815db60 0x7f60080ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60067fc640 1 -- 172.21.15.60:0/1599137152 >> v2:172.21.15.100:3300/0 conn(0x7f600815df40 msgr2=0x7f60080d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60067fc640 1 --2- 172.21.15.60:0/1599137152 >> v2:172.21.15.100:3300/0 conn(0x7f600815df40 0x7f60080d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60067fc640 1 -- 172.21.15.60:0/1599137152 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080ceee0 con 0x7f60080b5f00 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f60077fe640 1 --2- 172.21.15.60:0/1599137152 >> v2:172.21.15.100:3300/0 conn(0x7f600815df40 0x7f60080d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.223+0000 7f6006ffd640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600815db60 0x7f60080ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f60067fc640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60080b5f00 0x7f60080d0400 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6008161890 tx=0x7f5ff002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e202745d1644f3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f6005ffb640 1 -- 172.21.15.60:0/1599137152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0037070 con 0x7f60080b5f00 2024-04-25T08:36:20.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f6005ffb640 1 -- 172.21.15.60:0/1599137152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0032440 con 0x7f60080b5f00 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f6005ffb640 1 -- 172.21.15.60:0/1599137152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0032810 con 0x7f60080b5f00 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 -- 172.21.15.60:0/1599137152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60080b5f00 msgr2=0x7f60080d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60080b5f00 0x7f60080d0400 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6008161890 tx=0x7f5ff002a180 comp rx=0 tx=0).stop 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 -- 172.21.15.60:0/1599137152 shutdown_connections 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 --2- 172.21.15.60:0/1599137152 >> v2:172.21.15.100:3300/0 conn(0x7f600815df40 0x7f60080d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f60080b5f00 0x7f60080d0400 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 --2- 172.21.15.60:0/1599137152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f600815db60 0x7f60080ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 -- 172.21.15.60:0/1599137152 >> 172.21.15.60:0/1599137152 conn(0x7f6008061600 msgr2=0x7f6008061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 -- 172.21.15.60:0/1599137152 shutdown_connections 2024-04-25T08:36:20.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.224+0000 7f600da4a640 1 -- 172.21.15.60:0/1599137152 wait complete. 2024-04-25T08:36:20.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.225+0000 7f600da4a640 1 Processor -- start 2024-04-25T08:36:20.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 -- start start 2024-04-25T08:36:20.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60080b5f00 0x7f6008160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 0x7f60080cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f600815df40 0x7f600815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008161950 con 0x7f600815db60 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080d1d50 con 0x7f600815df40 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.226+0000 7f600da4a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008161ad0 con 0x7f60080b5f00 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 0x7f60080cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 0x7f60080cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39794/0 (socket says 172.21.15.60:39794) 2024-04-25T08:36:20.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 -- 172.21.15.60:0/248898572 learned_addr learned my addr 172.21.15.60:0/248898572 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60077fe640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f600815df40 0x7f600815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f6006ffd640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60080b5f00 0x7f6008160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 -- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60080b5f00 msgr2=0x7f6008160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60080b5f00 0x7f6008160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 -- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f600815df40 msgr2=0x7f600815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f600815df40 0x7f600815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 -- 172.21.15.60:0/248898572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600815e8e0 con 0x7f600815db60 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f60067fc640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 0x7f60080cab40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0041020 con 0x7f600815db60 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00306c0 con 0x7f600815db60 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600815ebc0 con 0x7f600815db60 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0037040 con 0x7f600815db60 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.227+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600815f180 con 0x7f600815db60 2024-04-25T08:36:20.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.228+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008073270 con 0x7f600815db60 2024-04-25T08:36:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.229+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ff0044020 con 0x7f600815db60 2024-04-25T08:36:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.229+0000 7f5ffffff640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fe806b4d0 0x7f5fe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.229+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5ff00aaa30 con 0x7f600815db60 2024-04-25T08:36:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.229+0000 7f6006ffd640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fe806b4d0 0x7f5fe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.230+0000 7f6006ffd640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fe806b4d0 0x7f5fe806d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6000009950 tx=0x7f6000008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.233+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff004b030 con 0x7f600815db60 2024-04-25T08:36:20.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.459+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60080d1860 con 0x7f600815db60 2024-04-25T08:36:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.459+0000 7f5ffffff640 1 -- 172.21.15.60:0/248898572 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5ff0079060 con 0x7f600815db60 2024-04-25T08:36:20.454 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:20.454 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:20.454 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:20.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fe806b4d0 msgr2=0x7f5fe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fe806b4d0 0x7f5fe806d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6000009950 tx=0x7f6000008040 comp rx=0 tx=0).stop 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 msgr2=0x7f60080cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 0x7f60080cab40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff00046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 shutdown_connections 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5fe806b4d0 0x7f5fe806d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f600815df40 0x7f600815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f600815db60 0x7f60080cab40 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 --2- 172.21.15.60:0/248898572 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f60080b5f00 0x7f6008160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 >> 172.21.15.60:0/248898572 conn(0x7f6008061600 msgr2=0x7f60080b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 shutdown_connections 2024-04-25T08:36:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.462+0000 7f600da4a640 1 -- 172.21.15.60:0/248898572 wait complete. 2024-04-25T08:36:20.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:20.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.574+0000 7f0ca9ccd640 1 Processor -- start 2024-04-25T08:36:20.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.574+0000 7f0ca9ccd640 1 -- start start 2024-04-25T08:36:20.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca9ccd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ca415dbe0 0x7f0ca40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca9ccd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0ca40b92a0 0x7f0ca40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca9ccd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca40b97a0 con 0x7f0ca40b2cb0 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca9ccd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca415dfc0 con 0x7f0ca415dbe0 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca9ccd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca40d1a00 con 0x7f0ca40b92a0 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0ca415dbe0 0x7f0ca40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca2ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0ca40b2cb0 0x7f0ca40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44412/0 (socket says 172.21.15.60:44412) 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca2ffd640 1 -- 172.21.15.60:0/3166612291 learned_addr learned my addr 172.21.15.60:0/3166612291 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca3fff640 1 --2- 172.21.15.60:0/3166612291 >> v2:172.21.15.100:3300/0 conn(0x7f0ca40b92a0 0x7f0ca40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca37fe640 1 -- 172.21.15.60:0/3166612291 >> v2:172.21.15.100:3300/0 conn(0x7f0ca40b92a0 msgr2=0x7f0ca40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca37fe640 1 --2- 172.21.15.60:0/3166612291 >> v2:172.21.15.100:3300/0 conn(0x7f0ca40b92a0 0x7f0ca40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca37fe640 1 -- 172.21.15.60:0/3166612291 >> v1:172.21.15.60:6789/0 conn(0x7f0ca40b2cb0 legacy=0x7f0ca40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca37fe640 1 -- 172.21.15.60:0/3166612291 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40d1c50 con 0x7f0ca415dbe0 2024-04-25T08:36:20.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca27fc640 1 -- 172.21.15.60:0/3166612291 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273282014 0 0) 0x7f0ca40b97a0 con 0x7f0ca40b2cb0 2024-04-25T08:36:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.575+0000 7f0ca3fff640 1 --2- 172.21.15.60:0/3166612291 >> v2:172.21.15.100:3300/0 conn(0x7f0ca40b92a0 0x7f0ca40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.576+0000 7f0ca37fe640 1 --2- 172.21.15.60:0/3166612291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca415dbe0 0x7f0ca40d0910 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40cfb90 tx=0x7f0c9402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1a75af87df766f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.576+0000 7f0ca27fc640 1 -- 172.21.15.60:0/3166612291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c94037070 con 0x7f0ca415dbe0 2024-04-25T08:36:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.576+0000 7f0ca27fc640 1 -- 172.21.15.60:0/3166612291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c94032440 con 0x7f0ca415dbe0 2024-04-25T08:36:20.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.576+0000 7f0ca27fc640 1 -- 172.21.15.60:0/3166612291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9402ae60 con 0x7f0ca415dbe0 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/3166612291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca415dbe0 msgr2=0x7f0ca40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/3166612291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca415dbe0 0x7f0ca40d0910 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40cfb90 tx=0x7f0c9402a110 comp rx=0 tx=0).stop 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/3166612291 shutdown_connections 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/3166612291 >> v2:172.21.15.100:3300/0 conn(0x7f0ca40b92a0 0x7f0ca40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/3166612291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca415dbe0 0x7f0ca40d0910 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/3166612291 >> 172.21.15.60:0/3166612291 conn(0x7f0ca4061a20 msgr2=0x7f0ca4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/3166612291 shutdown_connections 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.577+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/3166612291 wait complete. 2024-04-25T08:36:20.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.578+0000 7f0ca9ccd640 1 Processor -- start 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.578+0000 7f0ca9ccd640 1 -- start start 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca9ccd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40b2cb0 0x7f0ca41f0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca9ccd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 0x7f0ca40b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca9ccd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca415dbe0 0x7f0ca41f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca9ccd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b6ab0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca9ccd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40d2210 con 0x7f0ca40b2cb0 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca9ccd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b6c30 con 0x7f0ca415dbe0 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 0x7f0ca40b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 0x7f0ca40b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39818/0 (socket says 172.21.15.60:39818) 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 -- 172.21.15.60:0/206477917 learned_addr learned my addr 172.21.15.60:0/206477917 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca37fe640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40b2cb0 0x7f0ca41f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca3fff640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca415dbe0 0x7f0ca41f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 -- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca415dbe0 msgr2=0x7f0ca41f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca415dbe0 0x7f0ca41f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 -- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40b2cb0 msgr2=0x7f0ca41f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40b2cb0 0x7f0ca41f0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 -- 172.21.15.60:0/206477917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca41f38f0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.579+0000 7f0ca2ffd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 0x7f0ca40b5660 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c4e0 tx=0x7f0c9800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.580+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98018070 con 0x7f0ca40b92a0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.580+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98007500 con 0x7f0ca40b92a0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.580+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca41f3bd0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.580+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98004a20 con 0x7f0ca40b92a0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.580+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca41f42f0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.581+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca41f40f0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.581+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c9801b070 con 0x7f0ca40b92a0 2024-04-25T08:36:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.582+0000 7f0c8bfff640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c9006b4d0 0x7f0c9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.582+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0c98089750 con 0x7f0ca40b92a0 2024-04-25T08:36:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.582+0000 7f0ca37fe640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c9006b4d0 0x7f0c9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.582+0000 7f0ca37fe640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c9006b4d0 0x7f0c9006d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0c94034000 tx=0x7f0c9402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.586+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ca41f40f0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.768+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ca4073990 con 0x7f0ca40b92a0 2024-04-25T08:36:20.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.769+0000 7f0c8bfff640 1 -- 172.21.15.60:0/206477917 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17067 (secure 0 0 0) 0x7f0c98057da0 con 0x7f0ca40b92a0 2024-04-25T08:36:20.764 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:20.764 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:06.273972+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":2}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":5}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c9006b4d0 msgr2=0x7f0c9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c9006b4d0 0x7f0c9006d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0c94034000 tx=0x7f0c9402e000 comp rx=0 tx=0).stop 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 msgr2=0x7f0ca40b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 0x7f0ca40b5660 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c4e0 tx=0x7f0c9800c9a0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 shutdown_connections 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0ca415dbe0 0x7f0ca41f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ca40b92a0 0x7f0ca40b5660 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c9006b4d0 0x7f0c9006d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 --2- 172.21.15.60:0/206477917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ca40b2cb0 0x7f0ca41f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 >> 172.21.15.60:0/206477917 conn(0x7f0ca4061a20 msgr2=0x7f0ca406c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 shutdown_connections 2024-04-25T08:36:20.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.771+0000 7f0ca9ccd640 1 -- 172.21.15.60:0/206477917 wait complete. 2024-04-25T08:36:20.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:35:40.718590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:35:41.409652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:20.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.887+0000 7fe8f92fc640 1 Processor -- start 2024-04-25T08:36:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.887+0000 7fe8f92fc640 1 -- start start 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.887+0000 7fe8f92fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 0x7fe8f40b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f92fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f40cf320 con 0x7fe8f415b980 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f92fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f40cf040 con 0x7fe8f4162450 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f92fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4161970 con 0x7fe8f415b5a0 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f27fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe8f415b980 0x7fe8f40ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44428/0 (socket says 172.21.15.60:44428) 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2065520419 learned_addr learned my addr 172.21.15.60:0/2065520419 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175927069 0 0) 0x7fe8e8001290 con 0x7fe8f415b980 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8dc003520 con 0x7fe8f415b980 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f2ffd640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 0x7fe8f40b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 264559770 0 0) 0x7fe8dc003520 con 0x7fe8f415b980 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 msgr2=0x7fe8f40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 0x7fe8f40b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 >> v1:172.21.15.100:6789/0 conn(0x7fe8f4162450 legacy=0x7fe8f40d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f4161c50 con 0x7fe8f415b980 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe8e80019c0 con 0x7fe8f415b980 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 >> v1:172.21.15.60:6789/0 conn(0x7fe8f415b980 legacy=0x7fe8f40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc0039e0 0x7fe8dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8dc004800 0x7fe8dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 0x7fe8dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4161970 con 0x7fe8dc004800 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc00baf0 con 0x7fe8dc0039e0 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc00bcf0 con 0x7fe8f415b5a0 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8dc004800 0x7fe8dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f2ffd640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc0039e0 0x7fe8dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.888+0000 7fe8f37fe640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 0x7fe8dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.889+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 msgr2=0x7fe8dc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.889+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f415b5a0 0x7fe8dc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.889+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc0039e0 msgr2=0x7fe8dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.889+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc0039e0 0x7fe8dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.889+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2065520419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc003520 con 0x7fe8dc004800 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.889+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8dc004800 0x7fe8dc004c80 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c4c0 tx=0x7fe8dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ce440fc876f13d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800c0e0 con 0x7fe8dc004800 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8034070 con 0x7fe8dc004800 2024-04-25T08:36:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f1ffb640 1 -- 172.21.15.60:0/2065520419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80074f0 con 0x7fe8dc004800 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2065520419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8dc004800 msgr2=0x7fe8dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2065520419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8dc004800 0x7fe8dc004c80 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c4c0 tx=0x7fe8dc005830 comp rx=0 tx=0).stop 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f37fe640 1 -- 172.21.15.60:0/2065520419 reap_dead start 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2065520419 shutdown_connections 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2065520419 >> 172.21.15.60:0/2065520419 conn(0x7fe8f40d0530 msgr2=0x7fe8f40d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2065520419 shutdown_connections 2024-04-25T08:36:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.890+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2065520419 wait complete. 2024-04-25T08:36:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.891+0000 7fe8f92fc640 1 Processor -- start 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 -- start start 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc004800 0x7fe8f40c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 0x7fe8f41f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f41f4330 0x7fe8f41f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40ceba0 con 0x7fe8f40c75e0 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40cea20 con 0x7fe8dc004800 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f92fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40ced20 con 0x7fe8f41f4330 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 0x7fe8f41f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 0x7fe8f41f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39846/0 (socket says 172.21.15.60:39846) 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2979110626 learned_addr learned my addr 172.21.15.60:0/2979110626 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.892+0000 7fe8f37fe640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f41f4330 0x7fe8f41f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f2ffd640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc004800 0x7fe8f40c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f41f4330 msgr2=0x7fe8f41f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f41f4330 0x7fe8f41f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc004800 msgr2=0x7fe8f40c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc004800 0x7fe8f40c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f27fc640 1 -- 172.21.15.60:0/2979110626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f41f4cf0 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f27fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 0x7fe8f41f3df0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000ef0 tx=0x7fe8f40cf210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8007580 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e80057a0 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f40ca440 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8005b90 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.893+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f40c7e60 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.894+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f4073a00 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.895+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8e8039480 con 0x7fe8f40c75e0 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.895+0000 7fe8db7fe640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e006b4d0 0x7fe8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.895+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe8e80ab4b0 con 0x7fe8f40c75e0 2024-04-25T08:36:20.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.895+0000 7fe8f2ffd640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e006b4d0 0x7fe8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.896+0000 7fe8f2ffd640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e006b4d0 0x7fe8e006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40cbc90 tx=0x7fe8e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:20.899+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8f40cf320 con 0x7fe8f40c75e0 2024-04-25T08:36:21.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.078+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8f40ce0c0 con 0x7fe8f40c75e0 2024-04-25T08:36:21.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.079+0000 7fe8db7fe640 1 -- 172.21.15.60:0/2979110626 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8e8079a50 con 0x7fe8f40c75e0 2024-04-25T08:36:21.073 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:36:21.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e006b4d0 msgr2=0x7fe8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e006b4d0 0x7fe8e006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40cbc90 tx=0x7fe8e4008040 comp rx=0 tx=0).stop 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 msgr2=0x7fe8f41f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 0x7fe8f41f3df0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8000ef0 tx=0x7fe8f40cf210 comp rx=0 tx=0).stop 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 shutdown_connections 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe8f41f4330 0x7fe8f41f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f40c75e0 0x7fe8f41f3df0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe8e006b4d0 0x7fe8e006d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 --2- 172.21.15.60:0/2979110626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8dc004800 0x7fe8f40c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 >> 172.21.15.60:0/2979110626 conn(0x7fe8f40d0530 msgr2=0x7fe8f40d3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 shutdown_connections 2024-04-25T08:36:21.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.081+0000 7fe8f92fc640 1 -- 172.21.15.60:0/2979110626 wait complete. 2024-04-25T08:36:21.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:36:21.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.196+0000 7f3707e5d640 1 Processor -- start 2024-04-25T08:36:21.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.196+0000 7f3707e5d640 1 -- start start 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3707e5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f37000b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3707e5d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37000cf1d0 con 0x7f3700161890 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3707e5d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3700161620 con 0x7f37000b54a0 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3707e5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37000b7120 con 0x7f3700161240 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f37053f8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3700161890 0x7f37000ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44432/0 (socket says 172.21.15.60:44432) 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f37053f8640 1 -- 172.21.15.60:0/3437337097 learned_addr learned my addr 172.21.15.60:0/3437337097 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3705bf9640 1 --2- 172.21.15.60:0/3437337097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f37000b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397475784 0 0) 0x7f37000cf1d0 con 0x7f3700161890 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ec003520 con 0x7f3700161890 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256994189 0 0) 0x7f3700161620 con 0x7f37000b54a0 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37000cf1d0 con 0x7f37000b54a0 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3705bf9640 1 -- 172.21.15.60:0/3437337097 >> v1:172.21.15.100:6789/0 conn(0x7f37000b54a0 legacy=0x7f37000d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.197+0000 7f3705bf9640 1 -- 172.21.15.60:0/3437337097 >> v1:172.21.15.60:6789/0 conn(0x7f3700161890 legacy=0x7f37000ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.198+0000 7f3705bf9640 1 -- 172.21.15.60:0/3437337097 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700161e00 con 0x7f3700161240 2024-04-25T08:36:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.198+0000 7f3705bf9640 1 --2- 172.21.15.60:0/3437337097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f37000b6be0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097d0 tx=0x7f36fc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383b810eb43eb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.198+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc037070 con 0x7f3700161240 2024-04-25T08:36:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc032440 con 0x7f3700161240 2024-04-25T08:36:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3704bf7640 1 -- 172.21.15.60:0/3437337097 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0327f0 con 0x7f3700161240 2024-04-25T08:36:21.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 -- 172.21.15.60:0/3437337097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 msgr2=0x7f37000b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 --2- 172.21.15.60:0/3437337097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f37000b6be0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36fc0097d0 tx=0x7f36fc02a210 comp rx=0 tx=0).stop 2024-04-25T08:36:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 -- 172.21.15.60:0/3437337097 shutdown_connections 2024-04-25T08:36:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 --2- 172.21.15.60:0/3437337097 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f37000b6be0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 -- 172.21.15.60:0/3437337097 >> 172.21.15.60:0/3437337097 conn(0x7f37000d04a0 msgr2=0x7f37000d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 -- 172.21.15.60:0/3437337097 shutdown_connections 2024-04-25T08:36:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.199+0000 7f3707e5d640 1 -- 172.21.15.60:0/3437337097 wait complete. 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.200+0000 7f3707e5d640 1 Processor -- start 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.200+0000 7f3707e5d640 1 -- start start 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f3707e5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37000b54a0 0x7f37000c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f3707e5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f370015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f3707e5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 0x7f37000b8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f3707e5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001626a0 con 0x7f3700161890 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f3707e5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001623c0 con 0x7f37000b54a0 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f3707e5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37000d15f0 con 0x7f3700161240 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f37063fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 0x7f37000b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f37063fa640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 0x7f37000b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39860/0 (socket says 172.21.15.60:39860) 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f37063fa640 1 -- 172.21.15.60:0/2491971247 learned_addr learned my addr 172.21.15.60:0/2491971247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:21.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.201+0000 7f37053f8640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f370015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f3705bf9640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37000b54a0 0x7f37000c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37063fa640 1 -- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 msgr2=0x7f370015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37063fa640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f370015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37063fa640 1 -- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37000b54a0 msgr2=0x7f37000c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37063fa640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37000b54a0 0x7f37000c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37063fa640 1 -- 172.21.15.60:0/2491971247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000ca3f0 con 0x7f3700161890 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f3705bf9640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37000b54a0 0x7f37000c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37053f8640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f370015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f37063fa640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 0x7f37000b8430 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c7c0 tx=0x7f36f800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8003b70 con 0x7f3700161890 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8003d10 con 0x7f3700161890 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f800aa50 con 0x7f3700161890 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370015e6d0 con 0x7f3700161890 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.202+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370015ec90 con 0x7f3700161890 2024-04-25T08:36:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.203+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37000cf6a0 con 0x7f3700161890 2024-04-25T08:36:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.203+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36f800abf0 con 0x7f3700161890 2024-04-25T08:36:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.204+0000 7f36f67fc640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f36e406b4d0 0x7f36e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.204+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f36f808a3c0 con 0x7f3700161890 2024-04-25T08:36:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.204+0000 7f3705bf9640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f36e406b4d0 0x7f36e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.205+0000 7f3705bf9640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f36e406b4d0 0x7f36e406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f36fc034000 tx=0x7f36fc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.208+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37000cf6a0 con 0x7f3700161890 2024-04-25T08:36:21.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.389+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3700073a00 con 0x7f3700161890 2024-04-25T08:36:21.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.389+0000 7f36f67fc640 1 -- 172.21.15.60:0/2491971247 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36f8059040 con 0x7f3700161890 2024-04-25T08:36:21.384 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:36:21.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f36e406b4d0 msgr2=0x7f36e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f36e406b4d0 0x7f36e406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f36fc034000 tx=0x7f36fc02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 msgr2=0x7f37000b8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 0x7f37000b8430 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f36f800c7c0 tx=0x7f36f800cc80 comp rx=0 tx=0).stop 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 shutdown_connections 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3700161890 0x7f37000b8430 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3700161240 0x7f370015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f36e406b4d0 0x7f36e406d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 --2- 172.21.15.60:0/2491971247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37000b54a0 0x7f37000c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 >> 172.21.15.60:0/2491971247 conn(0x7f37000d04a0 msgr2=0x7f37000d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 shutdown_connections 2024-04-25T08:36:21.386 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.391+0000 7f3707e5d640 1 -- 172.21.15.60:0/2491971247 wait complete. 2024-04-25T08:36:21.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:36:21.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.507+0000 7f537336a640 1 Processor -- start 2024-04-25T08:36:21.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.508+0000 7f537336a640 1 -- start start 2024-04-25T08:36:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.508+0000 7f537336a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f536c15de80 0x7f536c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f537336a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0afc80 con 0x7f536c0cefb0 2024-04-25T08:36:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f537336a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c0ce4d0 con 0x7f536c0ccfe0 2024-04-25T08:36:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f537336a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0ce7b0 con 0x7f536c15de80 2024-04-25T08:36:21.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5371106640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f536c15de80 0x7f536c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5370905640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f536c0cefb0 0x7f536c0cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44446/0 (socket says 172.21.15.60:44446) 2024-04-25T08:36:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5370905640 1 -- 172.21.15.60:0/1039319260 learned_addr learned my addr 172.21.15.60:0/1039319260 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5371106640 1 -- 172.21.15.60:0/1039319260 >> v1:172.21.15.100:6789/0 conn(0x7f536c0ccfe0 legacy=0x7f536c0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5371106640 1 -- 172.21.15.60:0/1039319260 >> v1:172.21.15.60:6789/0 conn(0x7f536c0cefb0 legacy=0x7f536c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5363fff640 1 -- 172.21.15.60:0/1039319260 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3600818395 0 0) 0x7f536c0afc80 con 0x7f536c0cefb0 2024-04-25T08:36:21.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.509+0000 7f5371106640 1 -- 172.21.15.60:0/1039319260 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0cea90 con 0x7f536c15de80 2024-04-25T08:36:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f5371106640 1 --2- 172.21.15.60:0/1039319260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c15de80 0x7f536c0cadb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f53680097a0 tx=0x7f536802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9150105ef987b90f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f5363fff640 1 -- 172.21.15.60:0/1039319260 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368037070 con 0x7f536c15de80 2024-04-25T08:36:21.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f5363fff640 1 -- 172.21.15.60:0/1039319260 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5368032440 con 0x7f536c15de80 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f5363fff640 1 -- 172.21.15.60:0/1039319260 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368032810 con 0x7f536c15de80 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f537336a640 1 -- 172.21.15.60:0/1039319260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c15de80 msgr2=0x7f536c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f537336a640 1 --2- 172.21.15.60:0/1039319260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c15de80 0x7f536c0cadb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f53680097a0 tx=0x7f536802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f537336a640 1 -- 172.21.15.60:0/1039319260 shutdown_connections 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f537336a640 1 --2- 172.21.15.60:0/1039319260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c15de80 0x7f536c0cadb0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.510+0000 7f537336a640 1 -- 172.21.15.60:0/1039319260 >> 172.21.15.60:0/1039319260 conn(0x7f536c0b2390 msgr2=0x7f536c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.511+0000 7f537336a640 1 -- 172.21.15.60:0/1039319260 shutdown_connections 2024-04-25T08:36:21.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.511+0000 7f537336a640 1 -- 172.21.15.60:0/1039319260 wait complete. 2024-04-25T08:36:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.512+0000 7f537336a640 1 Processor -- start 2024-04-25T08:36:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.512+0000 7f537336a640 1 -- start start 2024-04-25T08:36:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536c0ccfe0 0x7f536c152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 0x7f536c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c15de80 0x7f536c1e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0cfb50 con 0x7f536c0cefb0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0b2eb0 con 0x7f536c0ccfe0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c153ae0 con 0x7f536c15de80 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 0x7f536c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 0x7f536c0ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39874/0 (socket says 172.21.15.60:39874) 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 -- 172.21.15.60:0/2563092485 learned_addr learned my addr 172.21.15.60:0/2563092485 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5371106640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536c0ccfe0 0x7f536c152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5371907640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c15de80 0x7f536c1e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 -- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c15de80 msgr2=0x7f536c1e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c15de80 0x7f536c1e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 -- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536c0ccfe0 msgr2=0x7f536c152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536c0ccfe0 0x7f536c152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 -- 172.21.15.60:0/2563092485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0ad600 con 0x7f536c0cefb0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5370905640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 0x7f536c0ae980 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c4d0 tx=0x7f535c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c018070 con 0x7f536c0cefb0 2024-04-25T08:36:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c007500 con 0x7f536c0cefb0 2024-04-25T08:36:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c0ad8e0 con 0x7f536c0cefb0 2024-04-25T08:36:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.513+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c004a20 con 0x7f536c0cefb0 2024-04-25T08:36:21.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.514+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c0adea0 con 0x7f536c0cefb0 2024-04-25T08:36:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.514+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c073a00 con 0x7f536c0cefb0 2024-04-25T08:36:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.515+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f535c01b070 con 0x7f536c0cefb0 2024-04-25T08:36:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.515+0000 7f5361ffb640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535006b4d0 0x7f535006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.515+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f535c0897b0 con 0x7f536c0cefb0 2024-04-25T08:36:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.516+0000 7f5371106640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535006b4d0 0x7f535006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.516+0000 7f5371106640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535006b4d0 0x7f535006d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5368034000 tx=0x7f536802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.520+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f535c058390 con 0x7f536c0cefb0 2024-04-25T08:36:21.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.699+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f536c153530 con 0x7f536c0cefb0 2024-04-25T08:36:21.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.699+0000 7f5361ffb640 1 -- 172.21.15.60:0/2563092485 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f535c057c90 con 0x7f536c0cefb0 2024-04-25T08:36:21.694 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:36:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535006b4d0 msgr2=0x7f535006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535006b4d0 0x7f535006d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5368034000 tx=0x7f536802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 msgr2=0x7f536c0ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 0x7f536c0ae980 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c4d0 tx=0x7f535c00c990 comp rx=0 tx=0).stop 2024-04-25T08:36:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 shutdown_connections 2024-04-25T08:36:21.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f536c15de80 0x7f536c1e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f536c0cefb0 0x7f536c0ae980 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f535006b4d0 0x7f535006d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 --2- 172.21.15.60:0/2563092485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f536c0ccfe0 0x7f536c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 >> 172.21.15.60:0/2563092485 conn(0x7f536c0b2390 msgr2=0x7f536c153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.701+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 shutdown_connections 2024-04-25T08:36:21.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.702+0000 7f537336a640 1 -- 172.21.15.60:0/2563092485 wait complete. 2024-04-25T08:36:21.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:21.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.813+0000 7fed0eea3640 1 Processor -- start 2024-04-25T08:36:21.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.813+0000 7fed0eea3640 1 -- start start 2024-04-25T08:36:21.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed0eea3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fed0815da70 0x7fed08161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed0eea3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed080b9730 0x7fed080ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed0eea3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed08162020 con 0x7fed08161b80 2024-04-25T08:36:21.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed0eea3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed081625e0 con 0x7fed080b9730 2024-04-25T08:36:21.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed0eea3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed08162300 con 0x7fed0815da70 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed07fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed080b9730 0x7fed080ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed07fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed080b9730 0x7fed080ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53032/0 (socket says 172.21.15.60:53032) 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed07fff640 1 -- 172.21.15.60:0/3842871076 learned_addr learned my addr 172.21.15.60:0/3842871076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.814+0000 7fed0cc3f640 1 --2- 172.21.15.60:0/3842871076 >> v2:172.21.15.100:3300/0 conn(0x7fed0815da70 0x7fed08161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.815+0000 7fed07fff640 1 -- 172.21.15.60:0/3842871076 >> v2:172.21.15.100:3300/0 conn(0x7fed0815da70 msgr2=0x7fed08161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.815+0000 7fed07fff640 1 --2- 172.21.15.60:0/3842871076 >> v2:172.21.15.100:3300/0 conn(0x7fed0815da70 0x7fed08161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.815+0000 7fed07fff640 1 -- 172.21.15.60:0/3842871076 >> v1:172.21.15.100:6789/0 conn(0x7fed08161b80 legacy=0x7fed0806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.815+0000 7fed07fff640 1 -- 172.21.15.60:0/3842871076 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed080d1af0 con 0x7fed080b9730 2024-04-25T08:36:21.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.815+0000 7fed07fff640 1 --2- 172.21.15.60:0/3842871076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080b9730 0x7fed080ce650 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fed080b61a0 tx=0x7fecf802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1342047d5961da2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.815+0000 7fed077fe640 1 -- 172.21.15.60:0/3842871076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf8036070 con 0x7fed080b9730 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed077fe640 1 -- 172.21.15.60:0/3842871076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf8031600 con 0x7fed080b9730 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed077fe640 1 -- 172.21.15.60:0/3842871076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80319d0 con 0x7fed080b9730 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 -- 172.21.15.60:0/3842871076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080b9730 msgr2=0x7fed080ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 --2- 172.21.15.60:0/3842871076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080b9730 0x7fed080ce650 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fed080b61a0 tx=0x7fecf802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 -- 172.21.15.60:0/3842871076 shutdown_connections 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 --2- 172.21.15.60:0/3842871076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080b9730 0x7fed080ce650 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 --2- 172.21.15.60:0/3842871076 >> v2:172.21.15.100:3300/0 conn(0x7fed0815da70 0x7fed08161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 -- 172.21.15.60:0/3842871076 >> 172.21.15.60:0/3842871076 conn(0x7fed08061a20 msgr2=0x7fed08061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 -- 172.21.15.60:0/3842871076 shutdown_connections 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.816+0000 7fed0eea3640 1 -- 172.21.15.60:0/3842871076 wait complete. 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.817+0000 7fed0eea3640 1 Processor -- start 2024-04-25T08:36:21.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.817+0000 7fed0eea3640 1 -- start start 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0eea3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed0815da70 0x7fed08151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0eea3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08161b80 0x7fed080cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0eea3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 0x7fed0815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0eea3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed080d3300 con 0x7fed080c9580 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0eea3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0815de50 con 0x7fed08161b80 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0eea3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed080d3480 con 0x7fed0815da70 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0d440640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 0x7fed0815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0d440640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 0x7fed0815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53036/0 (socket says 172.21.15.60:53036) 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.818+0000 7fed0d440640 1 -- 172.21.15.60:0/2578857689 learned_addr learned my addr 172.21.15.60:0/2578857689 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0cc3f640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed0815da70 0x7fed08151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed07fff640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08161b80 0x7fed080cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0d440640 1 -- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed0815da70 msgr2=0x7fed08151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0d440640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed0815da70 0x7fed08151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0d440640 1 -- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08161b80 msgr2=0x7fed080cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0d440640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08161b80 0x7fed080cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0d440640 1 -- 172.21.15.60:0/2578857689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed080c9a10 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0d440640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 0x7fed0815dfd0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fecf000c390 tx=0x7fecf000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0017070 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf00052d0 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed080cbf30 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0003810 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.819+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed080cc4f0 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.820+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed080683b0 con 0x7fed080c9580 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.820+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecf001a070 con 0x7fed080c9580 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.821+0000 7fed057fa640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fece806b4d0 0x7fece806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.821+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fecf00896c0 con 0x7fed080c9580 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.821+0000 7fed0cc3f640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fece806b4d0 0x7fece806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.822+0000 7fed0cc3f640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fece806b4d0 0x7fece806d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fed00009a10 tx=0x7fed00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:21.825+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed080683b0 con 0x7fed080c9580 2024-04-25T08:36:22.035 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.041+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed08073990 con 0x7fed080c9580 2024-04-25T08:36:22.036 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.042+0000 7fed057fa640 1 -- 172.21.15.60:0/2578857689 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fecf00583d0 con 0x7fed080c9580 2024-04-25T08:36:22.036 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:22.036 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:22.037 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:22.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fece806b4d0 msgr2=0x7fece806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fece806b4d0 0x7fece806d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fed00009a10 tx=0x7fed00008040 comp rx=0 tx=0).stop 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 msgr2=0x7fed0815dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 0x7fed0815dfd0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fecf000c390 tx=0x7fecf000c850 comp rx=0 tx=0).stop 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 shutdown_connections 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080c9580 0x7fed0815dfd0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fece806b4d0 0x7fece806d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08161b80 0x7fed080cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 --2- 172.21.15.60:0/2578857689 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed0815da70 0x7fed08151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 >> 172.21.15.60:0/2578857689 conn(0x7fed08061a20 msgr2=0x7fed080ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 shutdown_connections 2024-04-25T08:36:22.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.044+0000 7fed0eea3640 1 -- 172.21.15.60:0/2578857689 wait complete. 2024-04-25T08:36:22.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:22.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.157+0000 7f5353fb6640 1 Processor -- start 2024-04-25T08:36:22.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.157+0000 7f5353fb6640 1 -- start start 2024-04-25T08:36:22.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.158+0000 7f5353fb6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f534c1602e0 0x7f534c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.158+0000 7f5353fb6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f534c15f5f0 0x7f534c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.158+0000 7f5353fb6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534c0cd710 con 0x7f534c15ba00 2024-04-25T08:36:22.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.158+0000 7f5353fb6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534c15ff80 con 0x7f534c15f5f0 2024-04-25T08:36:22.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.158+0000 7f5353fb6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534c099b70 con 0x7f534c1602e0 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.158+0000 7f5352553640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f534c15f5f0 0x7f534c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f534c15f5f0 0x7f534c15fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53052/0 (socket says 172.21.15.60:53052) 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 -- 172.21.15.60:0/258186604 learned_addr learned my addr 172.21.15.60:0/258186604 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5351551640 1 --2- 172.21.15.60:0/258186604 >> v2:172.21.15.100:3300/0 conn(0x7f534c1602e0 0x7f534c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 -- 172.21.15.60:0/258186604 >> v2:172.21.15.100:3300/0 conn(0x7f534c1602e0 msgr2=0x7f534c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 --2- 172.21.15.60:0/258186604 >> v2:172.21.15.100:3300/0 conn(0x7f534c1602e0 0x7f534c0ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 -- 172.21.15.60:0/258186604 >> v1:172.21.15.100:6789/0 conn(0x7f534c15ba00 legacy=0x7f534c09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 -- 172.21.15.60:0/258186604 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c099e50 con 0x7f534c15f5f0 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.159+0000 7f5352553640 1 --2- 172.21.15.60:0/258186604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c15fa40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f534c0b7830 tx=0x7f533c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d343a117acff007c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5350d50640 1 -- 172.21.15.60:0/258186604 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c036070 con 0x7f534c15f5f0 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5350d50640 1 -- 172.21.15.60:0/258186604 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c031600 con 0x7f534c15f5f0 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5350d50640 1 -- 172.21.15.60:0/258186604 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0319d0 con 0x7f534c15f5f0 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 -- 172.21.15.60:0/258186604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 msgr2=0x7f534c15fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 --2- 172.21.15.60:0/258186604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c15fa40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f534c0b7830 tx=0x7f533c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 -- 172.21.15.60:0/258186604 shutdown_connections 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 --2- 172.21.15.60:0/258186604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c15fa40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 --2- 172.21.15.60:0/258186604 >> v2:172.21.15.100:3300/0 conn(0x7f534c1602e0 0x7f534c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 -- 172.21.15.60:0/258186604 >> 172.21.15.60:0/258186604 conn(0x7f534c0b3270 msgr2=0x7f534c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 -- 172.21.15.60:0/258186604 shutdown_connections 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.160+0000 7f5353fb6640 1 -- 172.21.15.60:0/258186604 wait complete. 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.161+0000 7f5353fb6640 1 Processor -- start 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.161+0000 7f5353fb6640 1 -- start start 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5353fb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f534c15ba00 0x7f534c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5353fb6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5353fb6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f534c1602e0 0x7f534c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5353fb6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c15bde0 con 0x7f534c15f5f0 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5353fb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c0cd2e0 con 0x7f534c15ba00 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5353fb6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c0cb3c0 con 0x7f534c1602e0 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5351551640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5351551640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53062/0 (socket says 172.21.15.60:53062) 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.162+0000 7f5351551640 1 -- 172.21.15.60:0/466309939 learned_addr learned my addr 172.21.15.60:0/466309939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5352553640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f534c1602e0 0x7f534c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351d52640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f534c15ba00 0x7f534c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351551640 1 -- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f534c1602e0 msgr2=0x7f534c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351551640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f534c1602e0 0x7f534c0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351551640 1 -- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f534c15ba00 msgr2=0x7f534c0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351551640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f534c15ba00 0x7f534c0b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351551640 1 -- 172.21.15.60:0/466309939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c15c190 con 0x7f534c15f5f0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5351551640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c152f50 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f534000d5f0 tx=0x7f534000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400125f0 con 0x7f534c15f5f0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340019070 con 0x7f534c15f5f0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c15c430 con 0x7f534c15f5f0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53400129d0 con 0x7f534c15f5f0 2024-04-25T08:36:22.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.163+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c15c9f0 con 0x7f534c15f5f0 2024-04-25T08:36:22.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.164+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c067be0 con 0x7f534c15f5f0 2024-04-25T08:36:22.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.165+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5340012b70 con 0x7f534c15f5f0 2024-04-25T08:36:22.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.165+0000 7f534a7fc640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f533806b4d0 0x7f533806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.165+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f534008a750 con 0x7f534c15f5f0 2024-04-25T08:36:22.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.165+0000 7f5351d52640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f533806b4d0 0x7f533806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.166+0000 7f5351d52640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f533806b4d0 0x7f533806d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f534c15f0f0 tx=0x7f5344009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.170+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f534c067be0 con 0x7f534c15f5f0 2024-04-25T08:36:22.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.349+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f534c071290 con 0x7f534c15f5f0 2024-04-25T08:36:22.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.350+0000 7f534a7fc640 1 -- 172.21.15.60:0/466309939 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17067 (secure 0 0 0) 0x7f534c071290 con 0x7f534c15f5f0 2024-04-25T08:36:22.345 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:22.345 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:06.273972+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":2}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":5}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f533806b4d0 msgr2=0x7f533806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f533806b4d0 0x7f533806d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f534c15f0f0 tx=0x7f5344009040 comp rx=0 tx=0).stop 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 msgr2=0x7f534c152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c152f50 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f534000d5f0 tx=0x7f534000dab0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 shutdown_connections 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f534c1602e0 0x7f534c0ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f534c15f5f0 0x7f534c152f50 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f533806b4d0 0x7f533806d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 --2- 172.21.15.60:0/466309939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f534c15ba00 0x7f534c0b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 >> 172.21.15.60:0/466309939 conn(0x7f534c0b3270 msgr2=0x7f534c0b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 shutdown_connections 2024-04-25T08:36:22.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.352+0000 7f5353fb6640 1 -- 172.21.15.60:0/466309939 wait complete. 2024-04-25T08:36:22.358 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:36:22.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:22.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.465+0000 7f94b4563640 1 Processor -- start 2024-04-25T08:36:22.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.465+0000 7f94b4563640 1 -- start start 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.466+0000 7f94b4563640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 0x7f94ac0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.466+0000 7f94b4563640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac06c220 con 0x7f94ac09a450 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.466+0000 7f94b4563640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac0d0080 con 0x7f94ac15dc70 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.466+0000 7f94b4563640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0d0c90 con 0x7f94ac06ce10 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b2b00640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f94ac09a450 0x7f94ac0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56602/0 (socket says 172.21.15.60:56602) 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b2b00640 1 -- 172.21.15.60:0/2070946864 learned_addr learned my addr 172.21.15.60:0/2070946864 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474190533 0 0) 0x7f94ac06c220 con 0x7f94ac09a450 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b1afe640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 0x7f94ac0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9498003520 con 0x7f94ac09a450 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1628193636 0 0) 0x7f9498003520 con 0x7f94ac09a450 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 msgr2=0x7f94ac0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 0x7f94ac0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 >> v1:172.21.15.100:6789/0 conn(0x7f94ac15dc70 legacy=0x7f94ac06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0d0360 con 0x7f94ac09a450 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f94a40019f0 con 0x7f94ac09a450 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 >> v1:172.21.15.60:6789/0 conn(0x7f94ac09a450 legacy=0x7f94ac0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94980039e0 0x7f9498003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9498004800 0x7f9498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 0x7f94980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0d0c90 con 0x7f9498004800 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949800ba70 con 0x7f94980039e0 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949800bc70 con 0x7f94ac06ce10 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b1afe640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9498004800 0x7f9498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b22ff640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94980039e0 0x7f9498003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.467+0000 7f94b2b00640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 0x7f94980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b1afe640 1 -- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 msgr2=0x7f94980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b1afe640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac06ce10 0x7f94980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b1afe640 1 -- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94980039e0 msgr2=0x7f9498003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b1afe640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94980039e0 0x7f9498003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b1afe640 1 -- 172.21.15.60:0/2070946864 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498003520 con 0x7f9498004800 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b1afe640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9498004800 0x7f9498004c80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f949c009a90 tx=0x7f9498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aa3c906237907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c036070 con 0x7f9498004800 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.468+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c031400 con 0x7f9498004800 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b12fd640 1 -- 172.21.15.60:0/2070946864 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c009690 con 0x7f9498004800 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b4563640 1 -- 172.21.15.60:0/2070946864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9498004800 msgr2=0x7f9498004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b4563640 1 --2- 172.21.15.60:0/2070946864 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9498004800 0x7f9498004c80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f949c009a90 tx=0x7f9498005830 comp rx=0 tx=0).stop 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b2b00640 1 -- 172.21.15.60:0/2070946864 reap_dead start 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b4563640 1 -- 172.21.15.60:0/2070946864 shutdown_connections 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b4563640 1 -- 172.21.15.60:0/2070946864 >> 172.21.15.60:0/2070946864 conn(0x7f94ac0b3270 msgr2=0x7f94ac0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b4563640 1 -- 172.21.15.60:0/2070946864 shutdown_connections 2024-04-25T08:36:22.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.469+0000 7f94b4563640 1 -- 172.21.15.60:0/2070946864 wait complete. 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.470+0000 7f94b4563640 1 Processor -- start 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.470+0000 7f94b4563640 1 -- start start 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b4563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498004800 0x7f94ac14e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b4563640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 0x7f94ac14f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b4563640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac150e50 0x7f94ac1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b4563640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac15e050 con 0x7f94ac14edc0 2024-04-25T08:36:22.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b4563640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0d0920 con 0x7f9498004800 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b4563640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0d1120 con 0x7f94ac150e50 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b1afe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 0x7f94ac14f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b1afe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 0x7f94ac14f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53092/0 (socket says 172.21.15.60:53092) 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b1afe640 1 -- 172.21.15.60:0/1503744746 learned_addr learned my addr 172.21.15.60:0/1503744746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b22ff640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498004800 0x7f94ac14e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b2b00640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac150e50 0x7f94ac1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b1afe640 1 -- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac150e50 msgr2=0x7f94ac1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b1afe640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac150e50 0x7f94ac1512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.471+0000 7f94b1afe640 1 -- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498004800 msgr2=0x7f94ac14e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94b1afe640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498004800 0x7f94ac14e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94b1afe640 1 -- 172.21.15.60:0/1503744746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac14eab0 con 0x7f94ac14edc0 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94b1afe640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 0x7f94ac14f240 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f949c009a90 tx=0x7f949c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c040020 con 0x7f94ac14edc0 2024-04-25T08:36:22.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c003bc0 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac15b100 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c036040 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.472+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac14fc10 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.473+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac073990 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.473+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f949c043020 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.474+0000 7f94a2ffd640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f949006b500 0x7f949006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.474+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f949c0aa500 con 0x7f94ac14edc0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.474+0000 7f94b22ff640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f949006b500 0x7f949006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.475+0000 7f94b22ff640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f949006b500 0x7f949006d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f94a8005590 tx=0x7f94a8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.478+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f949c04a070 con 0x7f94ac14edc0 2024-04-25T08:36:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.696+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94ac0b40a0 con 0x7f94ac14edc0 2024-04-25T08:36:22.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.696+0000 7f94a2ffd640 1 -- 172.21.15.60:0/1503744746 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f949c0044e0 con 0x7f94ac14edc0 2024-04-25T08:36:22.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:22.691 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:22.692 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f949006b500 msgr2=0x7f949006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f949006b500 0x7f949006d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f94a8005590 tx=0x7f94a8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 msgr2=0x7f94ac14f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 0x7f94ac14f240 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f949c009a90 tx=0x7f949c004730 comp rx=0 tx=0).stop 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 shutdown_connections 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f94ac150e50 0x7f94ac1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f94ac14edc0 0x7f94ac14f240 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f949006b500 0x7f949006d9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 --2- 172.21.15.60:0/1503744746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9498004800 0x7f94ac14e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 >> 172.21.15.60:0/1503744746 conn(0x7f94ac0b3270 msgr2=0x7f94ac0ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 shutdown_connections 2024-04-25T08:36:22.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.698+0000 7f94b4563640 1 -- 172.21.15.60:0/1503744746 wait complete. 2024-04-25T08:36:22.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:22.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.810+0000 7f32384e1640 1 Processor -- start 2024-04-25T08:36:22.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.810+0000 7f32384e1640 1 -- start start 2024-04-25T08:36:22.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.811+0000 7f32384e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.811+0000 7f32384e1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32300b97a0 con 0x7f323015dbe0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.811+0000 7f32384e1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32300d1c50 con 0x7f32300b2cb0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.811+0000 7f32384e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323015dfc0 con 0x7f32300d1020 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.811+0000 7f323627d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f323015dbe0 0x7f32300d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56608/0 (socket says 172.21.15.60:56608) 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.811+0000 7f323627d640 1 -- 172.21.15.60:0/2653895438 learned_addr learned my addr 172.21.15.60:0/2653895438 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f3236a7e640 1 --2- 172.21.15.60:0/2653895438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144715406 0 0) 0x7f32300b97a0 con 0x7f323015dbe0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3218003520 con 0x7f323015dbe0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456624385 0 0) 0x7f32300d1c50 con 0x7f32300b2cb0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32300b97a0 con 0x7f32300b2cb0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909202205 0 0) 0x7f3218003520 con 0x7f323015dbe0 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f3236a7e640 1 -- 172.21.15.60:0/2653895438 >> v1:172.21.15.100:6789/0 conn(0x7f32300b2cb0 legacy=0x7f32300d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f3236a7e640 1 -- 172.21.15.60:0/2653895438 >> v1:172.21.15.60:6789/0 conn(0x7f323015dbe0 legacy=0x7f32300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.812+0000 7f3236a7e640 1 -- 172.21.15.60:0/2653895438 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300d1f30 con 0x7f32300d1020 2024-04-25T08:36:22.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f3236a7e640 1 --2- 172.21.15.60:0/2653895438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300d2e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3220009dc0 tx=0x7f322002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34625a5c78ac2d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228001670 con 0x7f32300d1020 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220007540 con 0x7f32300d1020 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f323527b640 1 -- 172.21.15.60:0/2653895438 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220007910 con 0x7f32300d1020 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 -- 172.21.15.60:0/2653895438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 msgr2=0x7f32300d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 --2- 172.21.15.60:0/2653895438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300d2e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3220009dc0 tx=0x7f322002a660 comp rx=0 tx=0).stop 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 -- 172.21.15.60:0/2653895438 shutdown_connections 2024-04-25T08:36:22.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 --2- 172.21.15.60:0/2653895438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300d2e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 -- 172.21.15.60:0/2653895438 >> 172.21.15.60:0/2653895438 conn(0x7f3230061a20 msgr2=0x7f3230061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 -- 172.21.15.60:0/2653895438 shutdown_connections 2024-04-25T08:36:22.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.813+0000 7f32384e1640 1 -- 172.21.15.60:0/2653895438 wait complete. 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 Processor -- start 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 -- start start 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 0x7f323015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f323015dbe0 0x7f32300b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32300b9420 con 0x7f32300b2cb0 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32300b92a0 con 0x7f323015dbe0 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f32384e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32300b6ab0 con 0x7f32300d1020 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f323627d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 0x7f323015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f323627d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 0x7f323015eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53104/0 (socket says 172.21.15.60:53104) 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.815+0000 7f323627d640 1 -- 172.21.15.60:0/504879559 learned_addr learned my addr 172.21.15.60:0/504879559 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f3235a7c640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f3236a7e640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f323015dbe0 0x7f32300b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f323627d640 1 -- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 msgr2=0x7f32300b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f323627d640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f323627d640 1 -- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f323015dbe0 msgr2=0x7f32300b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f323627d640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f323015dbe0 0x7f32300b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f323627d640 1 -- 172.21.15.60:0/504879559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32300cabd0 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f323627d640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 0x7f323015eb60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f322800c4f0 tx=0x7f322800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228017070 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3228005ce0 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228007290 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3230152570 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.816+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323015f890 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.817+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230073990 con 0x7f32300b2cb0 2024-04-25T08:36:22.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.817+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f322801a070 con 0x7f32300b2cb0 2024-04-25T08:36:22.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.818+0000 7f322effd640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f321c06b4d0 0x7f321c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.818+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f32280897d0 con 0x7f32300b2cb0 2024-04-25T08:36:22.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.818+0000 7f3235a7c640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f321c06b4d0 0x7f321c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.819+0000 7f3235a7c640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f321c06b4d0 0x7f321c06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32300621b0 tx=0x7f3224009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:22.822+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32280584e0 con 0x7f32300b2cb0 2024-04-25T08:36:23.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.041+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f323015fa60 con 0x7f32300b2cb0 2024-04-25T08:36:23.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.041+0000 7f322effd640 1 -- 172.21.15.60:0/504879559 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3228057de0 con 0x7f32300b2cb0 2024-04-25T08:36:23.038 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:23.038 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:42.436804+0000","modified":"2024-04-25T08:35:44.434970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7626},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7626":{"gid":7626,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/3135665870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":3135665870},{"type":"v1","addr":"172.21.15.60:6835","nonce":3135665870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:36:23.039 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 55 2024-04-25T08:36:23.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f321c06b4d0 msgr2=0x7f321c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f321c06b4d0 0x7f321c06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f32300621b0 tx=0x7f3224009040 comp rx=0 tx=0).stop 2024-04-25T08:36:23.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 msgr2=0x7f323015eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 0x7f323015eb60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f322800c4f0 tx=0x7f322800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 shutdown_connections 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f321c06b4d0 0x7f321c06d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f323015dbe0 0x7f32300b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f32300d1020 0x7f32300b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 --2- 172.21.15.60:0/504879559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f32300b2cb0 0x7f323015eb60 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 >> 172.21.15.60:0/504879559 conn(0x7f3230061a20 msgr2=0x7f3230159430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 shutdown_connections 2024-04-25T08:36:23.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.043+0000 7f32384e1640 1 -- 172.21.15.60:0/504879559 wait complete. 2024-04-25T08:36:23.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.157+0000 7f914909e640 1 Processor -- start 2024-04-25T08:36:23.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.157+0000 7f914909e640 1 -- start start 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914909e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f914415da70 0x7f9144161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914909e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91440b9730 0x7f91440ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914909e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9144162020 con 0x7f9144161b80 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914909e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91441625e0 con 0x7f914415da70 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914909e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9144162300 con 0x7f91440b9730 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f9142d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f914415da70 0x7f9144161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914359e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9144161b80 0x7f914406c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56624/0 (socket says 172.21.15.60:56624) 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.158+0000 7f914359e640 1 -- 172.21.15.60:0/2004299382 learned_addr learned my addr 172.21.15.60:0/2004299382 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f913bfff640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91440b9730 0x7f91440ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f9142d9d640 1 -- 172.21.15.60:0/2004299382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91440b9730 msgr2=0x7f91440ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f9142d9d640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91440b9730 0x7f91440ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f9142d9d640 1 -- 172.21.15.60:0/2004299382 >> v1:172.21.15.60:6789/0 conn(0x7f9144161b80 legacy=0x7f914406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f914259c640 1 -- 172.21.15.60:0/2004299382 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741804325 0 0) 0x7f9144162020 con 0x7f9144161b80 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f9142d9d640 1 -- 172.21.15.60:0/2004299382 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440d1af0 con 0x7f914415da70 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f913bfff640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91440b9730 0x7f91440ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.159+0000 7f9142d9d640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f9144161640 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f91340097a0 tx=0x7f913402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0a1660f05fe077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914259c640 1 -- 172.21.15.60:0/2004299382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134037070 con 0x7f914415da70 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914259c640 1 -- 172.21.15.60:0/2004299382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913402ae50 con 0x7f914415da70 2024-04-25T08:36:23.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914259c640 1 -- 172.21.15.60:0/2004299382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134032660 con 0x7f914415da70 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 -- 172.21.15.60:0/2004299382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 msgr2=0x7f9144161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f9144161640 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f91340097a0 tx=0x7f913402a100 comp rx=0 tx=0).stop 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 -- 172.21.15.60:0/2004299382 shutdown_connections 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91440b9730 0x7f91440ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 --2- 172.21.15.60:0/2004299382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f9144161640 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 -- 172.21.15.60:0/2004299382 >> 172.21.15.60:0/2004299382 conn(0x7f9144061a20 msgr2=0x7f9144061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 -- 172.21.15.60:0/2004299382 shutdown_connections 2024-04-25T08:36:23.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.160+0000 7f914909e640 1 -- 172.21.15.60:0/2004299382 wait complete. 2024-04-25T08:36:23.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.161+0000 7f914909e640 1 Processor -- start 2024-04-25T08:36:23.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 -- start start 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f91440c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9144161b80 0x7f91440b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9144160a00 0x7f9144151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440d3300 con 0x7f914415da70 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914415de50 con 0x7f9144161b80 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f914909e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440d3480 con 0x7f9144160a00 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f9142d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f91440c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f9142d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f91440c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53122/0 (socket says 172.21.15.60:53122) 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.162+0000 7f9142d9d640 1 -- 172.21.15.60:0/4246297046 learned_addr learned my addr 172.21.15.60:0/4246297046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f9142d9d640 1 -- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9144160a00 msgr2=0x7f9144151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f9142d9d640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9144160a00 0x7f9144151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f9142d9d640 1 -- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9144161b80 msgr2=0x7f91440b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f9142d9d640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9144161b80 0x7f91440b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f9142d9d640 1 -- 172.21.15.60:0/4246297046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91441f6f10 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f9142d9d640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f91440c7ec0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9134035040 tx=0x7f91340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134037040 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9134004230 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91441f71f0 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134003610 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.163+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440cab70 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.164+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9144073990 con 0x7f914415da70 2024-04-25T08:36:23.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.164+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91340037b0 con 0x7f914415da70 2024-04-25T08:36:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.165+0000 7f913b7fe640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f912406b4d0 0x7f912406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.165+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f91340ab240 con 0x7f914415da70 2024-04-25T08:36:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.165+0000 7f913bfff640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f912406b4d0 0x7f912406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.166+0000 7f913bfff640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f912406b4d0 0x7f912406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.169+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9134079fe0 con 0x7f914415da70 2024-04-25T08:36:23.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.352+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91440d1570 con 0x7f914415da70 2024-04-25T08:36:23.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.354+0000 7f913b7fe640 1 -- 172.21.15.60:0/4246297046 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17067 (secure 0 0 0) 0x7f91340798e0 con 0x7f914415da70 2024-04-25T08:36:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:23.349 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":99,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:06.273972+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:40.718590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:41.409652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.10","mappings":[{"from":1,"to":2}]},{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.31","mappings":[{"from":1,"to":5}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:23.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f912406b4d0 msgr2=0x7f912406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f912406b4d0 0x7f912406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f912c006fd0 tx=0x7f912c008040 comp rx=0 tx=0).stop 2024-04-25T08:36:23.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 msgr2=0x7f91440c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f91440c7ec0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9134035040 tx=0x7f91340046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 shutdown_connections 2024-04-25T08:36:23.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9144160a00 0x7f9144151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f912406b4d0 0x7f912406d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9144161b80 0x7f91440b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 --2- 172.21.15.60:0/4246297046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f914415da70 0x7f91440c7ec0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 >> 172.21.15.60:0/4246297046 conn(0x7f9144061a20 msgr2=0x7f91440b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 shutdown_connections 2024-04-25T08:36:23.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.356+0000 7f914909e640 1 -- 172.21.15.60:0/4246297046 wait complete. 2024-04-25T08:36:23.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:36:23.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.470+0000 7f59b361f640 1 Processor -- start 2024-04-25T08:36:23.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.470+0000 7f59b361f640 1 -- start start 2024-04-25T08:36:23.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b361f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f59ac15a9f0 0x7f59ac0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b361f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 0x7f59ac0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b361f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ac0c6940 con 0x7f59ac0cdbf0 2024-04-25T08:36:23.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b361f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac0b0020 con 0x7f59ac15a9f0 2024-04-25T08:36:23.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b361f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0cb5a0 con 0x7f59ac0cd6f0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b0bba640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f59ac0cdbf0 0x7f59ac0ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56626/0 (socket says 172.21.15.60:56626) 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.471+0000 7f59b0bba640 1 -- 172.21.15.60:0/1281619397 learned_addr learned my addr 172.21.15.60:0/1281619397 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1281619397 >> v2:172.21.15.100:3300/0 conn(0x7f59ac15a9f0 0x7f59ac0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59b1bbc640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 0x7f59ac0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501325227 0 0) 0x7f59a0001110 con 0x7f59ac0cdbf0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5994003520 con 0x7f59ac0cdbf0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632961450 0 0) 0x7f5994003520 con 0x7f59ac0cdbf0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 msgr2=0x7f59ac0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 0x7f59ac0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 >> v2:172.21.15.100:3300/0 conn(0x7f59ac15a9f0 msgr2=0x7f59ac0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 --2- 172.21.15.60:0/1281619397 >> v2:172.21.15.100:3300/0 conn(0x7f59ac15a9f0 0x7f59ac0c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac15add0 con 0x7f59ac0cdbf0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f59a0001840 con 0x7f59ac0cdbf0 2024-04-25T08:36:23.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 >> v1:172.21.15.60:6789/0 conn(0x7f59ac0cdbf0 legacy=0x7f59ac0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f5994003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 0x7f59940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0cb5a0 con 0x7f59940039b0 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599400b9c0 con 0x7f5994004800 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599400bb40 con 0x7f59ac0cd6f0 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f5994003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59b0bba640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.472+0000 7f59b1bbc640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 0x7f59940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59b13bb640 1 -- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 msgr2=0x7f59940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac0cd6f0 0x7f59940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59b13bb640 1 -- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5994004800 msgr2=0x7f5994004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59b13bb640 1 -- 172.21.15.60:0/1281619397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5994003520 con 0x7f59940039b0 2024-04-25T08:36:23.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f5994003dd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009ac0 tx=0x7f5994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69e9f04553cc1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4036070 con 0x7f59940039b0 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4031400 con 0x7f59940039b0 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.473+0000 7f59abfff640 1 -- 172.21.15.60:0/1281619397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4009690 con 0x7f59940039b0 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b361f640 1 -- 172.21.15.60:0/1281619397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 msgr2=0x7f5994003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b361f640 1 --2- 172.21.15.60:0/1281619397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f5994003dd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009ac0 tx=0x7f5994005830 comp rx=0 tx=0).stop 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b1bbc640 1 -- 172.21.15.60:0/1281619397 reap_dead start 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b361f640 1 -- 172.21.15.60:0/1281619397 shutdown_connections 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b361f640 1 -- 172.21.15.60:0/1281619397 >> 172.21.15.60:0/1281619397 conn(0x7f59ac0b2390 msgr2=0x7f59ac0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b361f640 1 -- 172.21.15.60:0/1281619397 shutdown_connections 2024-04-25T08:36:23.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.474+0000 7f59b361f640 1 -- 172.21.15.60:0/1281619397 wait complete. 2024-04-25T08:36:23.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.475+0000 7f59b361f640 1 Processor -- start 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.475+0000 7f59b361f640 1 -- start start 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b361f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f59ac149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b361f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0c9ff0 0x7f59ac0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b361f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac14c270 0x7f59ac0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b361f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0cb230 con 0x7f59940039b0 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b361f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0caf50 con 0x7f59ac0c9ff0 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b361f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0ce070 con 0x7f59ac14c270 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f59ac149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f59ac149200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53126/0 (socket says 172.21.15.60:53126) 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 -- 172.21.15.60:0/1815774703 learned_addr learned my addr 172.21.15.60:0/1815774703 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b0bba640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0c9ff0 0x7f59ac0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b1bbc640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac14c270 0x7f59ac0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 -- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac14c270 msgr2=0x7f59ac0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac14c270 0x7f59ac0cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 -- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0c9ff0 msgr2=0x7f59ac0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.476+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0c9ff0 0x7f59ac0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59b13bb640 1 -- 172.21.15.60:0/1815774703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac149740 con 0x7f59940039b0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59b1bbc640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac14c270 0x7f59ac0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59b13bb640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f59ac149200 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009ac0 tx=0x7f59a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4041020 con 0x7f59940039b0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a403c470 con 0x7f59940039b0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac14bab0 con 0x7f59940039b0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4036040 con 0x7f59940039b0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.477+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac0b5fa0 con 0x7f59940039b0 2024-04-25T08:36:23.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.478+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac06cc80 con 0x7f59940039b0 2024-04-25T08:36:23.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.479+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59a4044020 con 0x7f59940039b0 2024-04-25T08:36:23.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.479+0000 7f59a9ffb640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f599806b4d0 0x7f599806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.479+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f59a40aab90 con 0x7f59940039b0 2024-04-25T08:36:23.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.479+0000 7f59b0bba640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f599806b4d0 0x7f599806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.480+0000 7f59b0bba640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f599806b4d0 0x7f599806d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f59a00012f0 tx=0x7f59a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.483+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59a404b030 con 0x7f59940039b0 2024-04-25T08:36:23.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.700+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f59ac0cf150 con 0x7f59940039b0 2024-04-25T08:36:23.695 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:36:23.701+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:36:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.702+0000 7f59a9ffb640 1 -- 172.21.15.60:0/1815774703 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7f59ac149740 con 0x7f59940039b0 2024-04-25T08:36:23.697 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f599806b4d0 msgr2=0x7f599806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f599806b4d0 0x7f599806d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f59a00012f0 tx=0x7f59a0009040 comp rx=0 tx=0).stop 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 msgr2=0x7f59ac149200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f59ac149200 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009ac0 tx=0x7f59a4004730 comp rx=0 tx=0).stop 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 shutdown_connections 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f59ac14c270 0x7f59ac0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f599806b4d0 0x7f599806d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59ac0c9ff0 0x7f59ac0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 --2- 172.21.15.60:0/1815774703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59940039b0 0x7f59ac149200 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 >> 172.21.15.60:0/1815774703 conn(0x7f59ac0b2390 msgr2=0x7f59ac0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 shutdown_connections 2024-04-25T08:36:23.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.705+0000 7f59b361f640 1 -- 172.21.15.60:0/1815774703 wait complete. 2024-04-25T08:36:23.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:36:23.743 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:36:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.825+0000 7fd809f53640 1 Processor -- start 2024-04-25T08:36:23.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd809f53640 1 -- start start 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd809f53640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd8040d3320 0x7fd8041689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd809f53640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd804161d70 con 0x7fd80415da90 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd809f53640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd804162040 con 0x7fd8040b9750 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd809f53640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd804162600 con 0x7fd8040d3320 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd8037fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd80415da90 0x7fd804161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56630/0 (socket says 172.21.15.60:56630) 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.826+0000 7fd8037fe640 1 -- 172.21.15.60:0/3179435841 learned_addr learned my addr 172.21.15.60:0/3179435841 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd803fff640 1 --2- 172.21.15.60:0/3179435841 >> v2:172.21.15.100:3300/0 conn(0x7fd8040d3320 0x7fd8041689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd8027fc640 1 -- 172.21.15.60:0/3179435841 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180155748 0 0) 0x7fd804161d70 con 0x7fd80415da90 2024-04-25T08:36:23.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd8027fc640 1 -- 172.21.15.60:0/3179435841 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ec003520 con 0x7fd80415da90 2024-04-25T08:36:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd803fff640 1 -- 172.21.15.60:0/3179435841 >> v1:172.21.15.100:6789/0 conn(0x7fd8040b9750 legacy=0x7fd8040ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd803fff640 1 -- 172.21.15.60:0/3179435841 >> v1:172.21.15.60:6789/0 conn(0x7fd80415da90 legacy=0x7fd804161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd8027fc640 1 -- 172.21.15.60:0/3179435841 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3368190452 0 0) 0x7fd7ec003520 con 0x7fd80415da90 2024-04-25T08:36:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.827+0000 7fd803fff640 1 -- 172.21.15.60:0/3179435841 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd804162320 con 0x7fd8040d3320 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd803fff640 1 --2- 172.21.15.60:0/3179435841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8040d3320 0x7fd8041689f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009d90 tx=0x7fd7fc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcee4e224ee1b095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd8027fc640 1 -- 172.21.15.60:0/3179435841 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4001670 con 0x7fd8040d3320 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd8027fc640 1 -- 172.21.15.60:0/3179435841 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc007540 con 0x7fd8040d3320 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd8027fc640 1 -- 172.21.15.60:0/3179435841 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0078f0 con 0x7fd8040d3320 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd809f53640 1 -- 172.21.15.60:0/3179435841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8040d3320 msgr2=0x7fd8041689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd809f53640 1 --2- 172.21.15.60:0/3179435841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8040d3320 0x7fd8041689f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009d90 tx=0x7fd7fc02a5f0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd809f53640 1 -- 172.21.15.60:0/3179435841 shutdown_connections 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd809f53640 1 --2- 172.21.15.60:0/3179435841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8040d3320 0x7fd8041689f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd809f53640 1 -- 172.21.15.60:0/3179435841 >> 172.21.15.60:0/3179435841 conn(0x7fd804061a20 msgr2=0x7fd804061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.828+0000 7fd809f53640 1 -- 172.21.15.60:0/3179435841 shutdown_connections 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.829+0000 7fd809f53640 1 -- 172.21.15.60:0/3179435841 wait complete. 2024-04-25T08:36:23.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.830+0000 7fd809f53640 1 Processor -- start 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.830+0000 7fd809f53640 1 -- start start 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.830+0000 7fd809f53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 0x7fd804152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.830+0000 7fd809f53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80415da90 0x7fd8040b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd809f53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8040cb820 0x7fd80406c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd809f53640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80415de70 con 0x7fd8040d3320 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd809f53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040d1df0 con 0x7fd80415da90 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd809f53640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040ced80 con 0x7fd8040cb820 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 0x7fd804152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 0x7fd804152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53152/0 (socket says 172.21.15.60:53152) 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 -- 172.21.15.60:0/3277126174 learned_addr learned my addr 172.21.15.60:0/3277126174 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:23.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd803fff640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8040cb820 0x7fd80406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd802ffd640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80415da90 0x7fd8040b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 -- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8040cb820 msgr2=0x7fd80406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8040cb820 0x7fd80406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 -- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80415da90 msgr2=0x7fd8040b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80415da90 0x7fd8040b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 -- 172.21.15.60:0/3277126174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040cbd90 con 0x7fd8040d3320 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd802ffd640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80415da90 0x7fd8040b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd803fff640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8040cb820 0x7fd80406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd8037fe640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 0x7fd804152e30 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c580 tx=0x7fd7f400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4017070 con 0x7fd8040d3320 2024-04-25T08:36:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f4005ce0 con 0x7fd8040d3320 2024-04-25T08:36:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8040cc070 con 0x7fd8040d3320 2024-04-25T08:36:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4007330 con 0x7fd8040d3320 2024-04-25T08:36:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.831+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8040cc630 con 0x7fd8040d3320 2024-04-25T08:36:23.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.832+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd804073a00 con 0x7fd8040d3320 2024-04-25T08:36:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.833+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7f401a070 con 0x7fd8040d3320 2024-04-25T08:36:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.833+0000 7fd7ebfff640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7f006b4d0 0x7fd7f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.833+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd7f4089920 con 0x7fd8040d3320 2024-04-25T08:36:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.833+0000 7fd802ffd640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7f006b4d0 0x7fd7f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.834+0000 7fd802ffd640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7f006b4d0 0x7fd7f006d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8007fd0 tx=0x7fd7f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:23.837+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7f4058560 con 0x7fd8040d3320 2024-04-25T08:36:24.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.059+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd8040d1590 con 0x7fd8040d3320 2024-04-25T08:36:24.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.704+0000 7fd7ebfff640 1 -- 172.21.15.60:0/3277126174 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7f4057e60 con 0x7fd8040d3320 2024-04-25T08:36:24.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7f006b4d0 msgr2=0x7fd7f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7f006b4d0 0x7fd7f006d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8007fd0 tx=0x7fd7f8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 msgr2=0x7fd804152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 0x7fd804152e30 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c580 tx=0x7fd7f400ca40 comp rx=0 tx=0).stop 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 shutdown_connections 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd8040cb820 0x7fd80406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7f006b4d0 0x7fd7f006d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd80415da90 0x7fd8040b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 --2- 172.21.15.60:0/3277126174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd8040d3320 0x7fd804152e30 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 >> 172.21.15.60:0/3277126174 conn(0x7fd804061a20 msgr2=0x7fd80415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 shutdown_connections 2024-04-25T08:36:24.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.707+0000 7fd809f53640 1 -- 172.21.15.60:0/3277126174 wait complete. 2024-04-25T08:36:24.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:36:24.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.876+0000 7fb77de9c640 1 Processor -- start 2024-04-25T08:36:24.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.876+0000 7fb77de9c640 1 -- start start 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb77de9c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb7780cc4f0 0x7fb7780cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb77de9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7780cdb20 0x7fb7780cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb77de9c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb77815b270 0x7fb77815b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb77de9c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780ce4c0 con 0x7fb77815b270 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb77de9c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb778071270 con 0x7fb7780cc4f0 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb77de9c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb778072280 con 0x7fb7780cdb20 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb77815b270 0x7fb77815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb77815b270 0x7fb77815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53154/0 (socket says 172.21.15.60:53154) 2024-04-25T08:36:24.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 -- 172.21.15.60:0/2279193725 learned_addr learned my addr 172.21.15.60:0/2279193725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb7777fe640 1 --2- 172.21.15.60:0/2279193725 >> v2:172.21.15.100:3300/0 conn(0x7fb7780cc4f0 0x7fb7780cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb776ffd640 1 --2- 172.21.15.60:0/2279193725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7780cdb20 0x7fb7780cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 -- 172.21.15.60:0/2279193725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7780cdb20 msgr2=0x7fb7780cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 --2- 172.21.15.60:0/2279193725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7780cdb20 0x7fb7780cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 -- 172.21.15.60:0/2279193725 >> v2:172.21.15.100:3300/0 conn(0x7fb7780cc4f0 msgr2=0x7fb7780cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 --2- 172.21.15.60:0/2279193725 >> v2:172.21.15.100:3300/0 conn(0x7fb7780cc4f0 0x7fb7780cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.877+0000 7fb777fff640 1 -- 172.21.15.60:0/2279193725 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780714c0 con 0x7fb77815b270 2024-04-25T08:36:24.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.878+0000 7fb777fff640 1 --2- 172.21.15.60:0/2279193725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb77815b270 0x7fb77815b680 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb7780cd290 tx=0x7fb77002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b81869cfb5400b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.878+0000 7fb7767fc640 1 -- 172.21.15.60:0/2279193725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770036070 con 0x7fb77815b270 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.878+0000 7fb7767fc640 1 -- 172.21.15.60:0/2279193725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb770031600 con 0x7fb77815b270 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.878+0000 7fb7767fc640 1 -- 172.21.15.60:0/2279193725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700319d0 con 0x7fb77815b270 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 -- 172.21.15.60:0/2279193725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb77815b270 msgr2=0x7fb77815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 --2- 172.21.15.60:0/2279193725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb77815b270 0x7fb77815b680 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb7780cd290 tx=0x7fb77002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 -- 172.21.15.60:0/2279193725 shutdown_connections 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 --2- 172.21.15.60:0/2279193725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb77815b270 0x7fb77815b680 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 --2- 172.21.15.60:0/2279193725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb7780cdb20 0x7fb7780cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 --2- 172.21.15.60:0/2279193725 >> v2:172.21.15.100:3300/0 conn(0x7fb7780cc4f0 0x7fb7780cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 -- 172.21.15.60:0/2279193725 >> 172.21.15.60:0/2279193725 conn(0x7fb77806abe0 msgr2=0x7fb77806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 -- 172.21.15.60:0/2279193725 shutdown_connections 2024-04-25T08:36:24.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.879+0000 7fb77de9c640 1 -- 172.21.15.60:0/2279193725 wait complete. 2024-04-25T08:36:24.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.880+0000 7fb77de9c640 1 Processor -- start 2024-04-25T08:36:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.880+0000 7fb77de9c640 1 -- start start 2024-04-25T08:36:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb77de9c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 0x7fb77813fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb77de9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7780cdb20 0x7fb7781e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb77de9c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb77815b270 0x7fb7781e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb77de9c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778071d60 con 0x7fb7780cc4f0 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb77de9c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778071a80 con 0x7fb7780cdb20 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb77de9c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77815bbc0 con 0x7fb77815b270 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb7777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 0x7fb77813fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb7777fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 0x7fb77813fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53168/0 (socket says 172.21.15.60:53168) 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb7777fe640 1 -- 172.21.15.60:0/1265819481 learned_addr learned my addr 172.21.15.60:0/1265819481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb776ffd640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7780cdb20 0x7fb7781e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.881+0000 7fb777fff640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb77815b270 0x7fb7781e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb7777fe640 1 -- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb77815b270 msgr2=0x7fb7781e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb7777fe640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb77815b270 0x7fb7781e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb7777fe640 1 -- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7780cdb20 msgr2=0x7fb7781e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb7777fe640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7780cdb20 0x7fb7781e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb7777fe640 1 -- 172.21.15.60:0/1265819481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7781404c0 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb777fff640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb77815b270 0x7fb7781e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb7777fe640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 0x7fb77813fdc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e8a0 tx=0x7fb76800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768017070 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76800ee90 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7781407a0 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800cb80 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.882+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780cf6f0 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.883+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778073a00 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.883+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb76801a070 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.884+0000 7fb75ffff640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb76406b4d0 0x7fb76406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.884+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb76808a610 con 0x7fb7780cc4f0 2024-04-25T08:36:24.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.884+0000 7fb776ffd640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb76406b4d0 0x7fb76406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.885+0000 7fb776ffd640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb76406b4d0 0x7fb76406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb77815a640 tx=0x7fb76c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:24.888+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb768059240 con 0x7fb7780cc4f0 2024-04-25T08:36:25.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.073+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb77806d3a0 con 0x7fb7780cc4f0 2024-04-25T08:36:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.706+0000 7fb75ffff640 1 -- 172.21.15.60:0/1265819481 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb768058b40 con 0x7fb7780cc4f0 2024-04-25T08:36:25.701 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:36:25.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb76406b4d0 msgr2=0x7fb76406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb76406b4d0 0x7fb76406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb77815a640 tx=0x7fb76c008040 comp rx=0 tx=0).stop 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 msgr2=0x7fb77813fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 0x7fb77813fdc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e8a0 tx=0x7fb76800ed60 comp rx=0 tx=0).stop 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 shutdown_connections 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb77815b270 0x7fb7781e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb76406b4d0 0x7fb76406d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7780cdb20 0x7fb7781e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 --2- 172.21.15.60:0/1265819481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb7780cc4f0 0x7fb77813fdc0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 >> 172.21.15.60:0/1265819481 conn(0x7fb77806abe0 msgr2=0x7fb77815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 shutdown_connections 2024-04-25T08:36:25.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.710+0000 7fb77de9c640 1 -- 172.21.15.60:0/1265819481 wait complete. 2024-04-25T08:36:25.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:36:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.835+0000 7f53c048f640 1 Processor -- start 2024-04-25T08:36:25.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.835+0000 7f53c048f640 1 -- start start 2024-04-25T08:36:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53c048f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b80acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53c048f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53b806b6d0 0x7f53b806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53c048f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b806db40 con 0x7f53b80b3920 2024-04-25T08:36:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53c048f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b806de20 con 0x7f53b806b6d0 2024-04-25T08:36:25.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53c048f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b806e100 con 0x7f53b806abe0 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53bea2c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f53b806b6d0 0x7f53b806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53bda2a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f53b80b3920 0x7f53b806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56632/0 (socket says 172.21.15.60:56632) 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.836+0000 7f53bda2a640 1 -- 172.21.15.60:0/140124415 learned_addr learned my addr 172.21.15.60:0/140124415 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53be22b640 1 --2- 172.21.15.60:0/140124415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b80acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53bea2c640 1 -- 172.21.15.60:0/140124415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 msgr2=0x7f53b80acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53bea2c640 1 --2- 172.21.15.60:0/140124415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b80acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53bea2c640 1 -- 172.21.15.60:0/140124415 >> v1:172.21.15.60:6789/0 conn(0x7f53b80b3920 legacy=0x7f53b806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53bd229640 1 -- 172.21.15.60:0/140124415 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453200936 0 0) 0x7f53b806db40 con 0x7f53b80b3920 2024-04-25T08:36:25.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53bea2c640 1 -- 172.21.15.60:0/140124415 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b806e3e0 con 0x7f53b806b6d0 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.837+0000 7f53bea2c640 1 --2- 172.21.15.60:0/140124415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ebe0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009d90 tx=0x7f53b402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d575eead0d3cf074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53bd229640 1 -- 172.21.15.60:0/140124415 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4031600 con 0x7f53b806b6d0 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53bd229640 1 -- 172.21.15.60:0/140124415 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b40317a0 con 0x7f53b806b6d0 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53bd229640 1 -- 172.21.15.60:0/140124415 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4031b70 con 0x7f53b806b6d0 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 -- 172.21.15.60:0/140124415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 msgr2=0x7f53b806ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 --2- 172.21.15.60:0/140124415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ebe0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009d90 tx=0x7f53b402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 -- 172.21.15.60:0/140124415 shutdown_connections 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 --2- 172.21.15.60:0/140124415 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ebe0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 --2- 172.21.15.60:0/140124415 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b80acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 -- 172.21.15.60:0/140124415 >> 172.21.15.60:0/140124415 conn(0x7f53b80cfec0 msgr2=0x7f53b80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 -- 172.21.15.60:0/140124415 shutdown_connections 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.838+0000 7f53c048f640 1 -- 172.21.15.60:0/140124415 wait complete. 2024-04-25T08:36:25.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.839+0000 7f53c048f640 1 Processor -- start 2024-04-25T08:36:25.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.839+0000 7f53c048f640 1 -- start start 2024-04-25T08:36:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53c048f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53c048f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53c048f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b80b3920 0x7f53b80c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53c048f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b3f00 con 0x7f53b806b6d0 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53c048f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b3d80 con 0x7f53b80b3920 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53c048f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b80b4080 con 0x7f53b806abe0 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53bda2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53bda2a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53200/0 (socket says 172.21.15.60:53200) 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53bda2a640 1 -- 172.21.15.60:0/4172598631 learned_addr learned my addr 172.21.15.60:0/4172598631 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53bea2c640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b80b3920 0x7f53b80c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.840+0000 7f53be22b640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bda2a640 1 -- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 msgr2=0x7f53b815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bda2a640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bda2a640 1 -- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b80b3920 msgr2=0x7f53b80c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bda2a640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b80b3920 0x7f53b80c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bda2a640 1 -- 172.21.15.60:0/4172598631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b814b060 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bea2c640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b80b3920 0x7f53b80c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53be22b640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53bda2a640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ed10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c420 tx=0x7f53a800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80074e0 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80047e0 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b814ad80 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8004c10 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.841+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b815a570 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.842+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b80b4940 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.842+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53a8011070 con 0x7f53b806b6d0 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.843+0000 7f53aeffd640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.843+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5865+0+0 (secure 0 0 0) 0x7f53a8089e60 con 0x7f53b806b6d0 2024-04-25T08:36:25.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.843+0000 7f53be22b640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.844+0000 7f53be22b640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f539c06b4d0 0x7f539c06d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f53b0006fd0 tx=0x7f53b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:25.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:25.847+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53b80b4940 con 0x7f53b806b6d0 2024-04-25T08:36:26.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.036+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53b8073a00 con 0x7f53b806b6d0 2024-04-25T08:36:26.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.707+0000 7f53aeffd640 1 -- 172.21.15.60:0/4172598631 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7f53a800e280 con 0x7f53b806b6d0 2024-04-25T08:36:26.702 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:36:26.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f539c06b4d0 msgr2=0x7f539c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f539c06b4d0 0x7f539c06d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f53b0006fd0 tx=0x7f53b0008040 comp rx=0 tx=0).stop 2024-04-25T08:36:26.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 msgr2=0x7f53b806ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.706 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ed10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c420 tx=0x7f53a800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 shutdown_connections 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53b80b3920 0x7f53b80c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f53b806b6d0 0x7f53b806ed10 secure :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c420 tx=0x7f53a800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 --2- 172.21.15.60:0/4172598631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53b806abe0 0x7f53b815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 >> 172.21.15.60:0/4172598631 conn(0x7f53b80cfec0 msgr2=0x7f53b80afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 shutdown_connections 2024-04-25T08:36:26.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.712+0000 7f53c048f640 1 -- 172.21.15.60:0/4172598631 wait complete. 2024-04-25T08:36:26.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:36:26.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.853+0000 7ff623c9a640 1 Processor -- start 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.854+0000 7ff623c9a640 1 -- start start 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.854+0000 7ff623c9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c072220 0x7ff61c06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.854+0000 7ff623c9a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff61c06ec30 0x7ff61c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.854+0000 7ff623c9a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c16e5a0 con 0x7ff61c16a720 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.854+0000 7ff623c9a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c16f170 con 0x7ff61c06ec30 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.854+0000 7ff623c9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c16e840 con 0x7ff61c072220 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.855+0000 7ff621a36640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff61c16a720 0x7ff61c071b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56646/0 (socket says 172.21.15.60:56646) 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.855+0000 7ff622237640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff61c06ec30 0x7ff61c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.855+0000 7ff621a36640 1 -- 172.21.15.60:0/1591364593 learned_addr learned my addr 172.21.15.60:0/1591364593 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.855+0000 7ff621235640 1 --2- 172.21.15.60:0/1591364593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c072220 0x7ff61c06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.855+0000 7ff620a34640 1 -- 172.21.15.60:0/1591364593 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057530663 0 0) 0x7ff61c16e5a0 con 0x7ff61c16a720 2024-04-25T08:36:26.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.870+0000 7ff620a34640 1 -- 172.21.15.60:0/1591364593 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff608003500 con 0x7ff61c16a720 2024-04-25T08:36:26.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.870+0000 7ff620a34640 1 -- 172.21.15.60:0/1591364593 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513139094 0 0) 0x7ff608003500 con 0x7ff61c16a720 2024-04-25T08:36:26.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.870+0000 7ff622237640 1 -- 172.21.15.60:0/1591364593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c072220 msgr2=0x7ff61c06e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:26.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.870+0000 7ff622237640 1 --2- 172.21.15.60:0/1591364593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c072220 0x7ff61c06e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.870+0000 7ff622237640 1 -- 172.21.15.60:0/1591364593 >> v1:172.21.15.60:6789/0 conn(0x7ff61c16a720 legacy=0x7ff61c071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.870+0000 7ff622237640 1 -- 172.21.15.60:0/1591364593 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c16eb20 con 0x7ff61c06ec30 2024-04-25T08:36:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.871+0000 7ff622237640 1 --2- 172.21.15.60:0/1591364593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c06ec30 0x7ff61c0cecc0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff61400be30 tx=0x7ff6140092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797a71524563be33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.871+0000 7ff620a34640 1 -- 172.21.15.60:0/1591364593 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614036070 con 0x7ff61c06ec30 2024-04-25T08:36:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.871+0000 7ff620a34640 1 -- 172.21.15.60:0/1591364593 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61400b2e0 con 0x7ff61c06ec30 2024-04-25T08:36:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.871+0000 7ff620a34640 1 -- 172.21.15.60:0/1591364593 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618001670 con 0x7ff61c06ec30 2024-04-25T08:36:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 -- 172.21.15.60:0/1591364593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c06ec30 msgr2=0x7ff61c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 --2- 172.21.15.60:0/1591364593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c06ec30 0x7ff61c0cecc0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff61400be30 tx=0x7ff6140092f0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 -- 172.21.15.60:0/1591364593 shutdown_connections 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 --2- 172.21.15.60:0/1591364593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c06ec30 0x7ff61c0cecc0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 --2- 172.21.15.60:0/1591364593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c072220 0x7ff61c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 -- 172.21.15.60:0/1591364593 >> 172.21.15.60:0/1591364593 conn(0x7ff61c06abe0 msgr2=0x7ff61c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 -- 172.21.15.60:0/1591364593 shutdown_connections 2024-04-25T08:36:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.872+0000 7ff623c9a640 1 -- 172.21.15.60:0/1591364593 wait complete. 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.874+0000 7ff623c9a640 1 Processor -- start 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.874+0000 7ff623c9a640 1 -- start start 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff623c9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c06ec30 0x7ff61c0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff623c9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c072220 0x7ff61c2043e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff623c9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 0x7ff61c089a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff623c9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c16ab00 con 0x7ff61c16a720 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff623c9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c06f080 con 0x7ff61c072220 2024-04-25T08:36:26.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff623c9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c16ac80 con 0x7ff61c06ec30 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 0x7ff61c089a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 0x7ff61c089a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53220/0 (socket says 172.21.15.60:53220) 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 -- 172.21.15.60:0/2574361276 learned_addr learned my addr 172.21.15.60:0/2574361276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff621a36640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c06ec30 0x7ff61c0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 -- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c06ec30 msgr2=0x7ff61c0c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c06ec30 0x7ff61c0c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 -- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c072220 msgr2=0x7ff61c2043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c072220 0x7ff61c2043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 -- 172.21.15.60:0/2574361276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c0c7180 con 0x7ff61c16a720 2024-04-25T08:36:26.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.875+0000 7ff622237640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 0x7ff61c089a10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff61400bf50 tx=0x7ff614004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.876+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614036050 con 0x7ff61c16a720 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.876+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff614042070 con 0x7ff61c16a720 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.876+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c204920 con 0x7ff61c16a720 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.876+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61400b660 con 0x7ff61c16a720 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.876+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c0c4b90 con 0x7ff61c16a720 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.877+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c0c5910 con 0x7ff61c16a720 2024-04-25T08:36:26.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.878+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff61403ba70 con 0x7ff61c16a720 2024-04-25T08:36:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.878+0000 7ff6127fc640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff60006b4d0 0x7ff60006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.878+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff6140aae20 con 0x7ff61c16a720 2024-04-25T08:36:26.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.879+0000 7ff621a36640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff60006b4d0 0x7ff60006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.879+0000 7ff621a36640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff60006b4d0 0x7ff60006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff6180011e0 tx=0x7ff6180016d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:26.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:26.884+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff61c204920 con 0x7ff61c16a720 2024-04-25T08:36:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.070+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff61c0716c0 con 0x7ff61c16a720 2024-04-25T08:36:27.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.073+0000 7ff6127fc640 1 -- 172.21.15.60:0/2574361276 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff61c0c7180 con 0x7ff61c16a720 2024-04-25T08:36:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff60006b4d0 msgr2=0x7ff60006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff60006b4d0 0x7ff60006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff6180011e0 tx=0x7ff6180016d0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 msgr2=0x7ff61c089a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 0x7ff61c089a10 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff61400bf50 tx=0x7ff614004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 shutdown_connections 2024-04-25T08:36:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff61c16a720 0x7ff61c089a10 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff60006b4d0 0x7ff60006d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff61c072220 0x7ff61c2043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 --2- 172.21.15.60:0/2574361276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff61c06ec30 0x7ff61c0c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 >> 172.21.15.60:0/2574361276 conn(0x7ff61c06abe0 msgr2=0x7ff61c16cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 shutdown_connections 2024-04-25T08:36:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.075+0000 7ff623c9a640 1 -- 172.21.15.60:0/2574361276 wait complete. 2024-04-25T08:36:27.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T08:36:27.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:27.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.188+0000 7f63ec276640 1 Processor -- start 2024-04-25T08:36:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.188+0000 7f63ec276640 1 -- start start 2024-04-25T08:36:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63ec276640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f63e40af970 0x7f63e40ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63ec276640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e406abe0 con 0x7f63e40d0010 2024-04-25T08:36:27.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63ec276640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e40ccc50 con 0x7f63e415dce0 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63ec276640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e406b4f0 con 0x7f63e40af970 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63ea813640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f63e40d0010 0x7f63e40cd4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56650/0 (socket says 172.21.15.60:56650) 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63ea813640 1 -- 172.21.15.60:0/3238330438 learned_addr learned my addr 172.21.15.60:0/3238330438 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9811640 1 --2- 172.21.15.60:0/3238330438 >> v2:172.21.15.100:3300/0 conn(0x7f63e40af970 0x7f63e40ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115280557 0 0) 0x7f63e406abe0 con 0x7f63e40d0010 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d0003520 con 0x7f63e40d0010 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438163183 0 0) 0x7f63e40ccc50 con 0x7f63e415dce0 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63e406abe0 con 0x7f63e415dce0 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1690937494 0 0) 0x7f63d0003520 con 0x7f63e40d0010 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 >> v2:172.21.15.100:3300/0 conn(0x7f63e40af970 msgr2=0x7f63e40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 --2- 172.21.15.60:0/3238330438 >> v2:172.21.15.100:3300/0 conn(0x7f63e40af970 0x7f63e40ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 >> v1:172.21.15.100:6789/0 conn(0x7f63e415dce0 legacy=0x7f63e40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.189+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e406aea0 con 0x7f63e40d0010 2024-04-25T08:36:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f63dc0019f0 con 0x7f63e40d0010 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 >> v1:172.21.15.60:6789/0 conn(0x7f63e40d0010 legacy=0x7f63e40cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d0003b40 0x7f63d0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0004800 0x7f63d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63d00051c0 0x7f63d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e406b4f0 con 0x7f63d0003b40 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d000c0f0 con 0x7f63d0004800 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d000c2f0 con 0x7f63d00051c0 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea012640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d0003b40 0x7f63d0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63e9811640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0004800 0x7f63d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea813640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63d00051c0 0x7f63d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea012640 1 -- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63d00051c0 msgr2=0x7f63d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea012640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63d00051c0 0x7f63d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea012640 1 -- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0004800 msgr2=0x7f63d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea012640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0004800 0x7f63d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea012640 1 -- 172.21.15.60:0/3238330438 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0003520 con 0x7f63d0003b40 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea813640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63d00051c0 0x7f63d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.190+0000 7f63ea813640 1 -- 172.21.15.60:0/3238330438 reap_dead start 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.191+0000 7f63ea012640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d0003b40 0x7f63d0003f40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f63e00018d0 tx=0x7f63d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd71ca6f9d7ef5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.191+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e00014f0 con 0x7f63d0003b40 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.191+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e00052d0 con 0x7f63d0003b40 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.191+0000 7f63e9010640 1 -- 172.21.15.60:0/3238330438 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e002ac70 con 0x7f63d0003b40 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 -- 172.21.15.60:0/3238330438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d0003b40 msgr2=0x7f63d0003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d0003b40 0x7f63d0003f40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f63e00018d0 tx=0x7f63d0005830 comp rx=0 tx=0).stop 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 -- 172.21.15.60:0/3238330438 shutdown_connections 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 --2- 172.21.15.60:0/3238330438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63d0003b40 0x7f63d0003f40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 -- 172.21.15.60:0/3238330438 >> 172.21.15.60:0/3238330438 conn(0x7f63e40b2290 msgr2=0x7f63e40b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 -- 172.21.15.60:0/3238330438 shutdown_connections 2024-04-25T08:36:27.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.192+0000 7f63ec276640 1 -- 172.21.15.60:0/3238330438 wait complete. 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.193+0000 7f63ec276640 1 Processor -- start 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.193+0000 7f63ec276640 1 -- start start 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0003b40 0x7f63e41f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 0x7f63e414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63e40b5e60 0x7f63e415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e415e0c0 con 0x7f63e41579f0 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e406b980 con 0x7f63d0003b40 2024-04-25T08:36:27.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e415f610 con 0x7f63e40b5e60 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 0x7f63e414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 0x7f63e414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53242/0 (socket says 172.21.15.60:53242) 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 -- 172.21.15.60:0/3913270584 learned_addr learned my addr 172.21.15.60:0/3913270584 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ea012640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0003b40 0x7f63e41f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ea813640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63e40b5e60 0x7f63e415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 -- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63e40b5e60 msgr2=0x7f63e415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63e40b5e60 0x7f63e415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 -- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0003b40 msgr2=0x7f63e41f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0003b40 0x7f63e41f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 -- 172.21.15.60:0/3913270584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e415f8f0 con 0x7f63e41579f0 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63e9811640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 0x7f63e414e9c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f63d4011a70 tx=0x7f63d4011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400fb50 con 0x7f63e41579f0 2024-04-25T08:36:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d400fcf0 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e415fbd0 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400fea0 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.194+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e41f3fe0 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.195+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e406cc80 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.196+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63d4012040 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.196+0000 7f63daffd640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63c806b500 0x7f63c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.197+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f63d408a580 con 0x7f63e41579f0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.197+0000 7f63ea012640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63c806b500 0x7f63c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.197+0000 7f63ea012640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63c806b500 0x7f63c806d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f63e0032000 tx=0x7f63e0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.201+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63d4019030 con 0x7f63e41579f0 2024-04-25T08:36:27.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.450+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f63e4073990 con 0x7f63e41579f0 2024-04-25T08:36:27.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.715+0000 7f63daffd640 1 -- 172.21.15.60:0/3913270584 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v351) v1 ==== 144+0+0 (secure 0 0 0) 0x7f63d4058f20 con 0x7f63e41579f0 2024-04-25T08:36:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63c806b500 msgr2=0x7f63c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63c806b500 0x7f63c806d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f63e0032000 tx=0x7f63e0009030 comp rx=0 tx=0).stop 2024-04-25T08:36:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 msgr2=0x7f63e414e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 0x7f63e414e9c0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f63d4011a70 tx=0x7f63d4011f30 comp rx=0 tx=0).stop 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 shutdown_connections 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63e40b5e60 0x7f63e415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f63e41579f0 0x7f63e414e9c0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f63c806b500 0x7f63c806d9c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 --2- 172.21.15.60:0/3913270584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63d0003b40 0x7f63e41f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 >> 172.21.15.60:0/3913270584 conn(0x7f63e40b2290 msgr2=0x7f63e406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 shutdown_connections 2024-04-25T08:36:27.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.719+0000 7f63ec276640 1 -- 172.21.15.60:0/3913270584 wait complete. 2024-04-25T08:36:27.725 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:36:27.726 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:36:27.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.834+0000 7fda374ac640 1 Processor -- start 2024-04-25T08:36:27.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.834+0000 7fda374ac640 1 -- start start 2024-04-25T08:36:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda374ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fda3015dbe0 0x7fda300d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda374ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda300b2cb0 0x7fda300d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda374ac640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda300b97a0 con 0x7fda300b92a0 2024-04-25T08:36:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda374ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3015dfc0 con 0x7fda300b2cb0 2024-04-25T08:36:27.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda374ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda300d1a00 con 0x7fda3015dbe0 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda34a47640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fda300b2cb0 0x7fda300d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda35a49640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fda300b92a0 0x7fda300b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56652/0 (socket says 172.21.15.60:56652) 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.835+0000 7fda35a49640 1 -- 172.21.15.60:0/3374330066 learned_addr learned my addr 172.21.15.60:0/3374330066 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda35248640 1 --2- 172.21.15.60:0/3374330066 >> v2:172.21.15.100:3300/0 conn(0x7fda3015dbe0 0x7fda300d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda34a47640 1 -- 172.21.15.60:0/3374330066 >> v2:172.21.15.100:3300/0 conn(0x7fda3015dbe0 msgr2=0x7fda300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda34a47640 1 --2- 172.21.15.60:0/3374330066 >> v2:172.21.15.100:3300/0 conn(0x7fda3015dbe0 0x7fda300d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda34a47640 1 -- 172.21.15.60:0/3374330066 >> v1:172.21.15.60:6789/0 conn(0x7fda300b92a0 legacy=0x7fda300b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda27fff640 1 -- 172.21.15.60:0/3374330066 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385277112 0 0) 0x7fda300b97a0 con 0x7fda300b92a0 2024-04-25T08:36:27.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda34a47640 1 -- 172.21.15.60:0/3374330066 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda300d1c50 con 0x7fda300b2cb0 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.836+0000 7fda34a47640 1 --2- 172.21.15.60:0/3374330066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda300d0e50 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fda3006c140 tx=0x7fda2002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd5991823183700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda27fff640 1 -- 172.21.15.60:0/3374330066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20037070 con 0x7fda300b2cb0 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda27fff640 1 -- 172.21.15.60:0/3374330066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20032440 con 0x7fda300b2cb0 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda27fff640 1 -- 172.21.15.60:0/3374330066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2002ae60 con 0x7fda300b2cb0 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 -- 172.21.15.60:0/3374330066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 msgr2=0x7fda300d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 --2- 172.21.15.60:0/3374330066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda300d0e50 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fda3006c140 tx=0x7fda2002a110 comp rx=0 tx=0).stop 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 -- 172.21.15.60:0/3374330066 shutdown_connections 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 --2- 172.21.15.60:0/3374330066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda300d0e50 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 --2- 172.21.15.60:0/3374330066 >> v2:172.21.15.100:3300/0 conn(0x7fda3015dbe0 0x7fda300d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 -- 172.21.15.60:0/3374330066 >> 172.21.15.60:0/3374330066 conn(0x7fda30061a20 msgr2=0x7fda30061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 -- 172.21.15.60:0/3374330066 shutdown_connections 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.837+0000 7fda374ac640 1 -- 172.21.15.60:0/3374330066 wait complete. 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.838+0000 7fda374ac640 1 Processor -- start 2024-04-25T08:36:27.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.838+0000 7fda374ac640 1 -- start start 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda374ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda301599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda374ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda300b92a0 0x7fda300b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda374ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda3015dbe0 0x7fda30157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda374ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300b6ab0 con 0x7fda300b2cb0 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda374ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300d2210 con 0x7fda3015dbe0 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda374ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda300b6c30 con 0x7fda300b92a0 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda301599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda301599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53252/0 (socket says 172.21.15.60:53252) 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 -- 172.21.15.60:0/3104191423 learned_addr learned my addr 172.21.15.60:0/3104191423 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 -- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda300b92a0 msgr2=0x7fda300b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda300b92a0 0x7fda300b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 -- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda3015dbe0 msgr2=0x7fda30157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35a49640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda3015dbe0 0x7fda30157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.839+0000 7fda35248640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda3015dbe0 0x7fda30157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda35248640 1 -- 172.21.15.60:0/3104191423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda300b5310 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda35a49640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda3015dbe0 0x7fda30157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda35248640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda301599c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d5f0 tx=0x7fda2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0125f0 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c019070 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0129d0 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda300b55f0 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.840+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda300b5bb0 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.841+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda30073990 con 0x7fda300b2cb0 2024-04-25T08:36:27.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.841+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda2c012b70 con 0x7fda300b2cb0 2024-04-25T08:36:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.842+0000 7fda25ffb640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fda1406b4d0 0x7fda1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.842+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fda2c08a400 con 0x7fda300b2cb0 2024-04-25T08:36:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.842+0000 7fda34a47640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fda1406b4d0 0x7fda1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.843+0000 7fda34a47640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fda1406b4d0 0x7fda1406d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fda20034000 tx=0x7fda2002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:27.846+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda2c059480 con 0x7fda300b2cb0 2024-04-25T08:36:28.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.055+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda301588a0 con 0x7fda300b2cb0 2024-04-25T08:36:28.050 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.056+0000 7fda25ffb640 1 -- 172.21.15.60:0/3104191423 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+13283 (secure 0 0 0) 0x7fda2c058d80 con 0x7fda300b2cb0 2024-04-25T08:36:28.050 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:28.050 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":102,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:26.707317+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/3135665870":"2024-04-26T08:36:23.702258+0000","172.21.15.60:6833/3135665870":"2024-04-26T08:36:23.702258+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fda1406b4d0 msgr2=0x7fda1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fda1406b4d0 0x7fda1406d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fda20034000 tx=0x7fda2002c000 comp rx=0 tx=0).stop 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 msgr2=0x7fda301599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda301599c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00d5f0 tx=0x7fda2c00dab0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 shutdown_connections 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fda1406b4d0 0x7fda1406d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fda3015dbe0 0x7fda30157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fda300b92a0 0x7fda300b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 --2- 172.21.15.60:0/3104191423 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fda300b2cb0 0x7fda301599c0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 >> 172.21.15.60:0/3104191423 conn(0x7fda30061a20 msgr2=0x7fda300b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 shutdown_connections 2024-04-25T08:36:28.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.058+0000 7fda374ac640 1 -- 172.21.15.60:0/3104191423 wait complete. 2024-04-25T08:36:28.064 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:28.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:28.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.173+0000 7fe0775a9640 1 Processor -- start 2024-04-25T08:36:28.168 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.173+0000 7fe0775a9640 1 -- start start 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe0775a9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe07015dc20 0x7fe0700ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe0775a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe0700cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe0775a9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0700b9330 0x7fe07006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe0775a9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07015e000 con 0x7fe0700b9330 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe0775a9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0700d1de0 con 0x7fe07015dc20 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe0775a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe070161c60 con 0x7fe070161710 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe075b46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0700b9330 0x7fe07006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe075b46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe0700b9330 0x7fe07006bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53256/0 (socket says 172.21.15.60:53256) 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.174+0000 7fe075b46640 1 -- 172.21.15.60:0/190970541 learned_addr learned my addr 172.21.15.60:0/190970541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:28.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe074b44640 1 --2- 172.21.15.60:0/190970541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe0700cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075345640 1 --2- 172.21.15.60:0/190970541 >> v2:172.21.15.100:3300/0 conn(0x7fe07015dc20 0x7fe0700ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075b46640 1 -- 172.21.15.60:0/190970541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 msgr2=0x7fe0700cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075b46640 1 --2- 172.21.15.60:0/190970541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe0700cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075b46640 1 -- 172.21.15.60:0/190970541 >> v2:172.21.15.100:3300/0 conn(0x7fe07015dc20 msgr2=0x7fe0700ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075b46640 1 --2- 172.21.15.60:0/190970541 >> v2:172.21.15.100:3300/0 conn(0x7fe07015dc20 0x7fe0700ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075b46640 1 -- 172.21.15.60:0/190970541 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070161f40 con 0x7fe0700b9330 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.175+0000 7fe075b46640 1 --2- 172.21.15.60:0/190970541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe07006bf80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe0700d1520 tx=0x7fe06002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ed640766d0d2ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe06ffff640 1 -- 172.21.15.60:0/190970541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060036070 con 0x7fe0700b9330 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe06ffff640 1 -- 172.21.15.60:0/190970541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060031600 con 0x7fe0700b9330 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe06ffff640 1 -- 172.21.15.60:0/190970541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600319d0 con 0x7fe0700b9330 2024-04-25T08:36:28.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 -- 172.21.15.60:0/190970541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 msgr2=0x7fe07006bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 --2- 172.21.15.60:0/190970541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe07006bf80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe0700d1520 tx=0x7fe06002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 -- 172.21.15.60:0/190970541 shutdown_connections 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 --2- 172.21.15.60:0/190970541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe07006bf80 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 --2- 172.21.15.60:0/190970541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe0700cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 --2- 172.21.15.60:0/190970541 >> v2:172.21.15.100:3300/0 conn(0x7fe07015dc20 0x7fe0700ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 -- 172.21.15.60:0/190970541 >> 172.21.15.60:0/190970541 conn(0x7fe0700d0420 msgr2=0x7fe0700d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 -- 172.21.15.60:0/190970541 shutdown_connections 2024-04-25T08:36:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.176+0000 7fe0775a9640 1 -- 172.21.15.60:0/190970541 wait complete. 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.177+0000 7fe0775a9640 1 Processor -- start 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.177+0000 7fe0775a9640 1 -- start start 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe0775a9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe0700cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe0775a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07015dc20 0x7fe070156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe0775a9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe070070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe0775a9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700b5e50 con 0x7fe0700b9330 2024-04-25T08:36:28.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe0775a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070162500 con 0x7fe07015dc20 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe0775a9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700b5fd0 con 0x7fe070161710 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe075345640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe0700cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe075345640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe0700cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53266/0 (socket says 172.21.15.60:53266) 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.178+0000 7fe075345640 1 -- 172.21.15.60:0/3448883622 learned_addr learned my addr 172.21.15.60:0/3448883622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe075345640 1 -- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 msgr2=0x7fe070070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe075345640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe070070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe075345640 1 -- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07015dc20 msgr2=0x7fe070156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe074b44640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07015dc20 0x7fe070156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe075345640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07015dc20 0x7fe070156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe075345640 1 -- 172.21.15.60:0/3448883622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700cc670 con 0x7fe0700b9330 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe074b44640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07015dc20 0x7fe070156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe075345640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe0700cd7b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fe06800c570 tx=0x7fe06800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068015070 con 0x7fe0700b9330 2024-04-25T08:36:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe068011800 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0700cc950 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068011bf0 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.179+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0700cd430 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.180+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe070073a00 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.181+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe068018070 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.181+0000 7fe06dffb640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.181+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe068089e30 con 0x7fe0700b9330 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.181+0000 7fe074b44640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.182+0000 7fe074b44640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe05c06b4d0 0x7fe05c06d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe0700b5270 tx=0x7fe064008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.185+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe068058eb0 con 0x7fe0700b9330 2024-04-25T08:36:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.367+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0700d16b0 con 0x7fe0700b9330 2024-04-25T08:36:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.367+0000 7fe06dffb640 1 -- 172.21.15.60:0/3448883622 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0680587b0 con 0x7fe0700b9330 2024-04-25T08:36:28.362 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe05c06b4d0 msgr2=0x7fe05c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe05c06b4d0 0x7fe05c06d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe0700b5270 tx=0x7fe064008040 comp rx=0 tx=0).stop 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 msgr2=0x7fe0700cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe0700cd7b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fe06800c570 tx=0x7fe06800ca30 comp rx=0 tx=0).stop 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 shutdown_connections 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe070161710 0x7fe070070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe07015dc20 0x7fe070156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 --2- 172.21.15.60:0/3448883622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0700b9330 0x7fe0700cd7b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 >> 172.21.15.60:0/3448883622 conn(0x7fe0700d0420 msgr2=0x7fe0700b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 shutdown_connections 2024-04-25T08:36:28.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.369+0000 7fe0775a9640 1 -- 172.21.15.60:0/3448883622 wait complete. 2024-04-25T08:36:28.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T08:36:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.481+0000 7f6e50eac640 1 Processor -- start 2024-04-25T08:36:28.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.481+0000 7f6e50eac640 1 -- start start 2024-04-25T08:36:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.482+0000 7f6e50eac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e4c15dc70 0x7f6e4c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.482+0000 7f6e50eac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c09a450 0x7f6e4c0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.482+0000 7f6e50eac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c06c220 con 0x7f6e4c06c700 2024-04-25T08:36:28.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.482+0000 7f6e50eac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c09a8b0 con 0x7f6e4c15dc70 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.482+0000 7f6e50eac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0d0c90 con 0x7f6e4c09a450 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e4c15dc70 0x7f6e4c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e49d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c09a450 0x7f6e4c0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e4c15dc70 0x7f6e4c0d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:50896/0 (socket says 172.21.15.60:50896) 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 -- 172.21.15.60:0/516251568 learned_addr learned my addr 172.21.15.60:0/516251568 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4ad9d640 1 --1- 172.21.15.60:0/516251568 >> v1:172.21.15.100:6789/0 conn(0x7f6e4c06c700 0x7f6e4c0cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:56742/0 (socket says 172.21.15.60:56742) 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 -- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c09a450 msgr2=0x7f6e4c0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 --2- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c09a450 0x7f6e4c0cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 -- 172.21.15.60:0/516251568 >> v1:172.21.15.100:6789/0 conn(0x7f6e4c06c700 legacy=0x7f6e4c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e4a59c640 1 -- 172.21.15.60:0/516251568 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c0d0080 con 0x7f6e4c15dc70 2024-04-25T08:36:28.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.483+0000 7f6e49d9b640 1 --2- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c09a450 0x7f6e4c0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:28.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.484+0000 7f6e4a59c640 1 --2- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c0d1120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009e50 tx=0x7f6e4402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e2662f5bbd4235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.484+0000 7f6e4959a640 1 -- 172.21.15.60:0/516251568 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44036070 con 0x7f6e4c15dc70 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.484+0000 7f6e4959a640 1 -- 172.21.15.60:0/516251568 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44031600 con 0x7f6e4c15dc70 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.484+0000 7f6e4959a640 1 -- 172.21.15.60:0/516251568 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e440319d0 con 0x7f6e4c15dc70 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 -- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 msgr2=0x7f6e4c0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 --2- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c0d1120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009e50 tx=0x7f6e4402a730 comp rx=0 tx=0).stop 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 -- 172.21.15.60:0/516251568 shutdown_connections 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 --2- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c09a450 0x7f6e4c0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 --2- 172.21.15.60:0/516251568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c0d1120 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 -- 172.21.15.60:0/516251568 >> 172.21.15.60:0/516251568 conn(0x7f6e4c0b3270 msgr2=0x7f6e4c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 -- 172.21.15.60:0/516251568 shutdown_connections 2024-04-25T08:36:28.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.485+0000 7f6e50eac640 1 -- 172.21.15.60:0/516251568 wait complete. 2024-04-25T08:36:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.486+0000 7f6e50eac640 1 Processor -- start 2024-04-25T08:36:28.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.486+0000 7f6e50eac640 1 -- start start 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e50eac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c06c700 0x7f6e4c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e50eac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 0x7f6e4c1601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e50eac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e50eac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0d0920 con 0x7f6e4c09a450 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e50eac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0d0640 con 0x7f6e4c15dc70 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e50eac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c15e050 con 0x7f6e4c06c700 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 0x7f6e4c1601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 0x7f6e4c1601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53276/0 (socket says 172.21.15.60:53276) 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 -- 172.21.15.60:0/3948576411 learned_addr learned my addr 172.21.15.60:0/3948576411 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e4a59c640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c06c700 0x7f6e4c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e4ad9d640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 -- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c06c700 msgr2=0x7f6e4c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c06c700 0x7f6e4c15e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 -- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 msgr2=0x7f6e4c160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 -- 172.21.15.60:0/3948576411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c15eab0 con 0x7f6e4c09a450 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.487+0000 7f6e49d9b640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 0x7f6e4c1601d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400ebe0 tx=0x7f6e3400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.488+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34017070 con 0x7f6e4c09a450 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.488+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3400cc30 con 0x7f6e4c09a450 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.488+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c15ed90 con 0x7f6e4c09a450 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.488+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34013470 con 0x7f6e4c09a450 2024-04-25T08:36:28.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.488+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c15f350 con 0x7f6e4c09a450 2024-04-25T08:36:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.489+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c0b2dc0 con 0x7f6e4c09a450 2024-04-25T08:36:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.489+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e3401a070 con 0x7f6e4c09a450 2024-04-25T08:36:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.490+0000 7f6e42ffd640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.490+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6e3408a250 con 0x7f6e4c09a450 2024-04-25T08:36:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.490+0000 7f6e4a59c640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.490+0000 7f6e4a59c640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6e44033000 tx=0x7f6e44009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.494+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e4c0b2dc0 con 0x7f6e4c09a450 2024-04-25T08:36:28.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:28.747+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f6e4c0b40a0 con 0x7f6e4c09a450 2024-04-25T08:36:29.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.721+0000 7f6e42ffd640 1 -- 172.21.15.60:0/3948576411 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v353) v1 ==== 158+0+0 (secure 0 0 0) 0x7f6e34058c80 con 0x7f6e4c09a450 2024-04-25T08:36:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e2c06b4d0 msgr2=0x7f6e2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6e44033000 tx=0x7f6e44009000 comp rx=0 tx=0).stop 2024-04-25T08:36:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 msgr2=0x7f6e4c1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 0x7f6e4c1601d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400ebe0 tx=0x7f6e3400c520 comp rx=0 tx=0).stop 2024-04-25T08:36:29.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 shutdown_connections 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e2c06b4d0 0x7f6e2c06d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e4c15dc70 0x7f6e4c160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6e4c09a450 0x7f6e4c1601d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 --2- 172.21.15.60:0/3948576411 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e4c06c700 0x7f6e4c15e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 >> 172.21.15.60:0/3948576411 conn(0x7f6e4c0b3270 msgr2=0x7f6e4c0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 shutdown_connections 2024-04-25T08:36:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.724+0000 7f6e50eac640 1 -- 172.21.15.60:0/3948576411 wait complete. 2024-04-25T08:36:29.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:36:29.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.849+0000 7f453c207640 1 Processor -- start 2024-04-25T08:36:29.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.849+0000 7f453c207640 1 -- start start 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.850+0000 7f453c207640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45340cc4f0 0x7f45340cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.850+0000 7f453c207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cdb20 0x7f45340cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.850+0000 7f453c207640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4534072280 con 0x7f453415b270 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.850+0000 7f453c207640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340ce4c0 con 0x7f45340cc4f0 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.850+0000 7f453c207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4534071270 con 0x7f45340cdb20 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f45397a2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cdb20 0x7f45340cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4539fa3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45340cc4f0 0x7f45340cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f453a7a4640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f453415b270 0x7f453415b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:56750/0 (socket says 172.21.15.60:56750) 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f453a7a4640 1 -- 172.21.15.60:0/1238013808 learned_addr learned my addr 172.21.15.60:0/1238013808 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4538fa1640 1 -- 172.21.15.60:0/1238013808 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677167012 0 0) 0x7f4534072280 con 0x7f453415b270 2024-04-25T08:36:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4538fa1640 1 -- 172.21.15.60:0/1238013808 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4520003520 con 0x7f453415b270 2024-04-25T08:36:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4539fa3640 1 -- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cdb20 msgr2=0x7f45340cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4539fa3640 1 --2- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cdb20 0x7f45340cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4539fa3640 1 -- 172.21.15.60:0/1238013808 >> v1:172.21.15.100:6789/0 conn(0x7f453415b270 legacy=0x7f453415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f4539fa3640 1 -- 172.21.15.60:0/1238013808 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340714c0 con 0x7f45340cc4f0 2024-04-25T08:36:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.851+0000 7f45397a2640 1 --2- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cdb20 0x7f45340cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.852+0000 7f4539fa3640 1 --2- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cc4f0 0x7f45340cc8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f45300097a0 tx=0x7f453002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535fc0a099412c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.852+0000 7f4538fa1640 1 -- 172.21.15.60:0/1238013808 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530037070 con 0x7f45340cc4f0 2024-04-25T08:36:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.852+0000 7f4538fa1640 1 -- 172.21.15.60:0/1238013808 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530032440 con 0x7f45340cc4f0 2024-04-25T08:36:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.852+0000 7f4538fa1640 1 -- 172.21.15.60:0/1238013808 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300327f0 con 0x7f45340cc4f0 2024-04-25T08:36:29.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.852+0000 7f453c207640 1 -- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cc4f0 msgr2=0x7f45340cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.852+0000 7f453c207640 1 --2- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cc4f0 0x7f45340cc8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f45300097a0 tx=0x7f453002a180 comp rx=0 tx=0).stop 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.853+0000 7f453c207640 1 -- 172.21.15.60:0/1238013808 shutdown_connections 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.853+0000 7f453c207640 1 --2- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cdb20 0x7f45340cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.853+0000 7f453c207640 1 --2- 172.21.15.60:0/1238013808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cc4f0 0x7f45340cc8d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.853+0000 7f453c207640 1 -- 172.21.15.60:0/1238013808 >> 172.21.15.60:0/1238013808 conn(0x7f453406abe0 msgr2=0x7f453406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.853+0000 7f453c207640 1 -- 172.21.15.60:0/1238013808 shutdown_connections 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.853+0000 7f453c207640 1 -- 172.21.15.60:0/1238013808 wait complete. 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.854+0000 7f453c207640 1 Processor -- start 2024-04-25T08:36:29.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.854+0000 7f453c207640 1 -- start start 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453c207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cc4f0 0x7f45341582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453c207640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cdb20 0x7f453414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453c207640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 0x7f45341498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453c207640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4534071a80 con 0x7f453415b270 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453c207640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453415bd90 con 0x7f45340cdb20 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453c207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4534071d60 con 0x7f45340cc4f0 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 0x7f45341498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 0x7f45341498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53290/0 (socket says 172.21.15.60:53290) 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 -- 172.21.15.60:0/3120016328 learned_addr learned my addr 172.21.15.60:0/3120016328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f4539fa3640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cc4f0 0x7f45341582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f45397a2640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cdb20 0x7f453414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 -- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cc4f0 msgr2=0x7f45341582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cc4f0 0x7f45341582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 -- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cdb20 msgr2=0x7f453414a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cdb20 0x7f453414a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f453a7a4640 1 -- 172.21.15.60:0/3120016328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45341542f0 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.855+0000 7f45397a2640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cdb20 0x7f453414a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.856+0000 7f453a7a4640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 0x7f45341498d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f452c00cb40 tx=0x7f452c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.856+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c017070 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.856+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c00aca0 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.856+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45341545d0 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.856+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c012620 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.856+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4534155c00 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.857+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4534073a00 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.857+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f452c01a070 con 0x7f453415b270 2024-04-25T08:36:29.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.858+0000 7f452a7fc640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f451806b4d0 0x7f451806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.858+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f452c089ee0 con 0x7f453415b270 2024-04-25T08:36:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.858+0000 7f4539fa3640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f451806b4d0 0x7f451806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.858+0000 7f4539fa3640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f451806b4d0 0x7f451806d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4530034000 tx=0x7f453002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:29.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:29.862+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f452c058800 con 0x7f453415b270 2024-04-25T08:36:30.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.044+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f453406d3a0 con 0x7f453415b270 2024-04-25T08:36:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.047+0000 7f452a7fc640 1 -- 172.21.15.60:0/3120016328 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f452c05c8a0 con 0x7f453415b270 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f451806b4d0 msgr2=0x7f451806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f451806b4d0 0x7f451806d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4530034000 tx=0x7f453002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 msgr2=0x7f45341498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 0x7f45341498d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f452c00cb40 tx=0x7f452c00a060 comp rx=0 tx=0).stop 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 shutdown_connections 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f453415b270 0x7f45341498d0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f451806b4d0 0x7f451806d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45340cdb20 0x7f453414a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 --2- 172.21.15.60:0/3120016328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45340cc4f0 0x7f45341582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 >> 172.21.15.60:0/3120016328 conn(0x7f453406abe0 msgr2=0x7f453415b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 shutdown_connections 2024-04-25T08:36:30.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.049+0000 7f453c207640 1 -- 172.21.15.60:0/3120016328 wait complete. 2024-04-25T08:36:30.055 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:36:30.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:30.096 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:30.142 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:36:30.142 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:36:30.142 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:30.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:36:30.158 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:36:30.158 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:30.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 none /run/credentials/systemd-tmpfiles-clean.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 2024-04-25T08:36:30.175 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:36:30.175 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:30.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:36:30.213 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:36:30.213 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:30.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 none /run/credentials/systemd-tmpfiles-clean.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 2024-04-25T08:36:30.231 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:36:30.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.359+0000 7efc29ba9640 1 Processor -- start 2024-04-25T08:36:30.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.359+0000 7efc29ba9640 1 -- start start 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc29ba9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc24144750 0x7efc2406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc29ba9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efc2406eb10 0x7efc240ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc29ba9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc24144b30 con 0x7efc240cee60 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc29ba9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc240712d0 con 0x7efc24144750 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc29ba9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc240715b0 con 0x7efc2406eb10 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc237fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc24144750 0x7efc2406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc23fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efc240cee60 0x7efc2415f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56664/0 (socket says 172.21.15.60:56664) 2024-04-25T08:36:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.360+0000 7efc23fff640 1 -- 172.21.15.60:0/1670409466 learned_addr learned my addr 172.21.15.60:0/1670409466 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc237fe640 1 -- 172.21.15.60:0/1670409466 >> v2:172.21.15.100:3300/0 conn(0x7efc2406eb10 msgr2=0x7efc240ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc237fe640 1 --2- 172.21.15.60:0/1670409466 >> v2:172.21.15.100:3300/0 conn(0x7efc2406eb10 0x7efc240ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc237fe640 1 -- 172.21.15.60:0/1670409466 >> v1:172.21.15.60:6789/0 conn(0x7efc240cee60 legacy=0x7efc2415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc237fe640 1 -- 172.21.15.60:0/1670409466 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24071890 con 0x7efc24144750 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc237fe640 1 --2- 172.21.15.60:0/1670409466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24144750 0x7efc2406e5d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7efc241606b0 tx=0x7efc1402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982be843748e15a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc227fc640 1 -- 172.21.15.60:0/1670409466 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14037070 con 0x7efc24144750 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc227fc640 1 -- 172.21.15.60:0/1670409466 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14032440 con 0x7efc24144750 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.361+0000 7efc227fc640 1 -- 172.21.15.60:0/1670409466 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14032810 con 0x7efc24144750 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 -- 172.21.15.60:0/1670409466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24144750 msgr2=0x7efc2406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 --2- 172.21.15.60:0/1670409466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24144750 0x7efc2406e5d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7efc241606b0 tx=0x7efc1402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 -- 172.21.15.60:0/1670409466 shutdown_connections 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 --2- 172.21.15.60:0/1670409466 >> v2:172.21.15.100:3300/0 conn(0x7efc2406eb10 0x7efc240ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 --2- 172.21.15.60:0/1670409466 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc24144750 0x7efc2406e5d0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 -- 172.21.15.60:0/1670409466 >> 172.21.15.60:0/1670409466 conn(0x7efc2406abe0 msgr2=0x7efc2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 -- 172.21.15.60:0/1670409466 shutdown_connections 2024-04-25T08:36:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.362+0000 7efc29ba9640 1 -- 172.21.15.60:0/1670409466 wait complete. 2024-04-25T08:36:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.363+0000 7efc29ba9640 1 Processor -- start 2024-04-25T08:36:30.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 -- start start 2024-04-25T08:36:30.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 0x7efc2415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc240cee60 0x7efc241f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc24144750 0x7efc241f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc24145450 con 0x7efc2406eb10 2024-04-25T08:36:30.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2406f020 con 0x7efc240cee60 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc29ba9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc241455d0 con 0x7efc24144750 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.364+0000 7efc237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 0x7efc2415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 0x7efc2415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53318/0 (socket says 172.21.15.60:53318) 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 -- 172.21.15.60:0/3905384017 learned_addr learned my addr 172.21.15.60:0/3905384017 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc23fff640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc24144750 0x7efc241f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc22ffd640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc240cee60 0x7efc241f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 -- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc24144750 msgr2=0x7efc241f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc24144750 0x7efc241f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 -- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc240cee60 msgr2=0x7efc241f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc240cee60 0x7efc241f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 -- 172.21.15.60:0/3905384017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc241f5770 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc23fff640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc24144750 0x7efc241f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc22ffd640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc240cee60 0x7efc241f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc237fe640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 0x7efc2415e740 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7efc14000c00 tx=0x7efc140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14037040 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14042070 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc24140af0 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14030bd0 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.365+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc241410b0 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.366+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc24073990 con 0x7efc2406eb10 2024-04-25T08:36:30.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.367+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc14047460 con 0x7efc2406eb10 2024-04-25T08:36:30.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.367+0000 7efc0bfff640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc1006b4d0 0x7efc1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.367+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efc140aac50 con 0x7efc2406eb10 2024-04-25T08:36:30.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.367+0000 7efc22ffd640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc1006b4d0 0x7efc1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.368+0000 7efc22ffd640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc1006b4d0 0x7efc1006d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efc2415bab0 tx=0x7efc18008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.371+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc140b0020 con 0x7efc2406eb10 2024-04-25T08:36:30.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.589+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc240d0510 con 0x7efc2406eb10 2024-04-25T08:36:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.589+0000 7efc0bfff640 1 -- 172.21.15.60:0/3905384017 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3705 (secure 0 0 0) 0x7efc14032e20 con 0x7efc2406eb10 2024-04-25T08:36:30.584 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:30.584 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3488383778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3488383778},{"type":"v1","addr":"172.21.15.100:6836","nonce":3488383778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3462301419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":3462301419},{"type":"v1","addr":"172.21.15.100:6837","nonce":3462301419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7623,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/1278584896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1278584896},{"type":"v1","addr":"172.21.15.60:6834","nonce":1278584896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":8103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2298751314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2298751314},{"type":"v1","addr":"172.21.15.60:6835","nonce":2298751314}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[]} 2024-04-25T08:36:30.584 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 57 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc1006b4d0 msgr2=0x7efc1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc1006b4d0 0x7efc1006d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efc2415bab0 tx=0x7efc18008040 comp rx=0 tx=0).stop 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 msgr2=0x7efc2415e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 0x7efc2415e740 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7efc14000c00 tx=0x7efc140046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 shutdown_connections 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc24144750 0x7efc241f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc1006b4d0 0x7efc1006d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc240cee60 0x7efc241f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 --2- 172.21.15.60:0/3905384017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc2406eb10 0x7efc2415e740 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 >> 172.21.15.60:0/3905384017 conn(0x7efc2406abe0 msgr2=0x7efc2415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 shutdown_connections 2024-04-25T08:36:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.591+0000 7efc29ba9640 1 -- 172.21.15.60:0/3905384017 wait complete. 2024-04-25T08:36:30.597 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:36:30.597 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:36:30.597 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:36:30.598 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:30.598 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:36:30.598 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:36:30.598 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:36:30.598 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:30.598 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:36:30.598 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:36:30.599 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:36:30.599 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:30.599 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:36:30.599 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:36:30.599 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:36:30.599 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:30.609 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:36:30.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:36:30.610 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:36:30.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:36:30.611 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:36:30.612 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:36:30.612 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:36:30.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:36:30.652 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:36:30.652 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:30.652 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:36:30.653 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:30.655 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:36:30.655 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:30.655 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:36:30.656 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:30.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:36:30.729 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:36:30.729 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:36:30.736 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:36:30.736 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:36:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.769+0000 7f00b2bc8640 1 Processor -- start 2024-04-25T08:36:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.769+0000 7f00b2bc8640 1 -- start start 2024-04-25T08:36:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b2bc8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 0x7f00ac0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b2bc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cef10 0x7f00ac0cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b2bc8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac15b1c0 con 0x7f00ac0cc680 2024-04-25T08:36:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b2bc8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac06f0d0 con 0x7f00ac154150 2024-04-25T08:36:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b2bc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15a2a0 con 0x7f00ac0cef10 2024-04-25T08:36:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b0964640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 0x7f00ac0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b0964640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 0x7f00ac0cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53366/0 (socket says 172.21.15.60:53366) 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00b0964640 1 -- 172.21.15.60:0/3981890272 learned_addr learned my addr 172.21.15.60:0/3981890272 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.770+0000 7f00abfff640 1 --2- 172.21.15.60:0/3981890272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cef10 0x7f00ac0cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.771+0000 7f00ab7fe640 1 -- 172.21.15.60:0/3981890272 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860187406 0 0) 0x7f00ac15b1c0 con 0x7f00ac0cc680 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.771+0000 7f00abfff640 1 -- 172.21.15.60:0/3981890272 >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 msgr2=0x7f00ac0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.771+0000 7f00abfff640 1 --2- 172.21.15.60:0/3981890272 >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 0x7f00ac0cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.771+0000 7f00abfff640 1 -- 172.21.15.60:0/3981890272 >> v1:172.21.15.100:6789/0 conn(0x7f00ac0cc680 legacy=0x7f00ac15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.771+0000 7f00abfff640 1 -- 172.21.15.60:0/3981890272 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac15a540 con 0x7f00ac0cef10 2024-04-25T08:36:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.771+0000 7f00b0964640 1 --2- 172.21.15.60:0/3981890272 >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 0x7f00ac0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00abfff640 1 --2- 172.21.15.60:0/3981890272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cef10 0x7f00ac0cc140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00980087a0 tx=0x7f009802c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ac65fe0c06a4458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00ab7fe640 1 -- 172.21.15.60:0/3981890272 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009803a070 con 0x7f00ac0cef10 2024-04-25T08:36:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00ab7fe640 1 -- 172.21.15.60:0/3981890272 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098032480 con 0x7f00ac0cef10 2024-04-25T08:36:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00ab7fe640 1 -- 172.21.15.60:0/3981890272 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098032850 con 0x7f00ac0cef10 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3981890272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cef10 msgr2=0x7f00ac0cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3981890272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cef10 0x7f00ac0cc140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f00980087a0 tx=0x7f009802c5f0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3981890272 shutdown_connections 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3981890272 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cef10 0x7f00ac0cc140 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3981890272 >> v2:172.21.15.60:3300/0 conn(0x7f00ac154150 0x7f00ac0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3981890272 >> 172.21.15.60:0/3981890272 conn(0x7f00ac06abe0 msgr2=0x7f00ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3981890272 shutdown_connections 2024-04-25T08:36:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.772+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3981890272 wait complete. 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 Processor -- start 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 -- start start 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cc680 0x7f00ac144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 0x7f00ac0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00ac154150 0x7f00ac0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15ade0 con 0x7f00ac0cef10 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac15ab00 con 0x7f00ac154150 2024-04-25T08:36:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.774+0000 7f00b2bc8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac154530 con 0x7f00ac0cc680 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 0x7f00ac0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 0x7f00ac0c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53372/0 (socket says 172.21.15.60:53372) 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 -- 172.21.15.60:0/3405369320 learned_addr learned my addr 172.21.15.60:0/3405369320 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00b0964640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cc680 0x7f00ac144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00b1165640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00ac154150 0x7f00ac0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 -- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cc680 msgr2=0x7f00ac144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cc680 0x7f00ac144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 -- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00ac154150 msgr2=0x7f00ac0c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00ac154150 0x7f00ac0c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 -- 172.21.15.60:0/3405369320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac1583f0 con 0x7f00ac0cef10 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00abfff640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 0x7f00ac0c5060 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0098038040 tx=0x7f00980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098045070 con 0x7f00ac0cef10 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098041670 con 0x7f00ac0cef10 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac1586d0 con 0x7f00ac0cef10 2024-04-25T08:36:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009803a040 con 0x7f00ac0cef10 2024-04-25T08:36:30.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.775+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac144850 con 0x7f00ac0cef10 2024-04-25T08:36:30.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.776+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac073a00 con 0x7f00ac0cef10 2024-04-25T08:36:30.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.777+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f009803d040 con 0x7f00ac0cef10 2024-04-25T08:36:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.777+0000 7f00a97fa640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f009006b520 0x7f009006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.777+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f00980ac5f0 con 0x7f00ac0cef10 2024-04-25T08:36:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.777+0000 7f00b0964640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f009006b520 0x7f009006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.778+0000 7f00b0964640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f009006b520 0x7f009006d9e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009630 tx=0x7f00a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.782+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f009807b750 con 0x7f00ac0cef10 2024-04-25T08:36:30.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:30.973+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f00ac0746d0 con 0x7f00ac0cef10 2024-04-25T08:36:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.736+0000 7f00a97fa640 1 -- 172.21.15.60:0/3405369320 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7f009807b050 con 0x7f00ac0cef10 2024-04-25T08:36:31.731 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f009006b520 msgr2=0x7f009006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f009006b520 0x7f009006d9e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009630 tx=0x7f00a0008040 comp rx=0 tx=0).stop 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 msgr2=0x7f00ac0c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 0x7f00ac0c5060 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0098038040 tx=0x7f00980046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 shutdown_connections 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f009006b520 0x7f009006d9e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00ac154150 0x7f00ac0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00ac0cef10 0x7f00ac0c5060 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 --2- 172.21.15.60:0/3405369320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00ac0cc680 0x7f00ac144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 >> 172.21.15.60:0/3405369320 conn(0x7f00ac06abe0 msgr2=0x7f00ac0b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 shutdown_connections 2024-04-25T08:36:31.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.739+0000 7f00b2bc8640 1 -- 172.21.15.60:0/3405369320 wait complete. 2024-04-25T08:36:31.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:36:31.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.862+0000 7fd270841640 1 Processor -- start 2024-04-25T08:36:31.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.862+0000 7fd270841640 1 -- start start 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd270841640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd26c15dc20 0x7fd26c1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd270841640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd26c0cedf0 0x7fd26c0d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd270841640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26c15e000 con 0x7fd26c0b93a0 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd270841640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26c0b6bb0 con 0x7fd26c15dc20 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd270841640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26c0b6030 con 0x7fd26c0cedf0 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd26a59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd26c15dc20 0x7fd26c1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd269d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd26c0b93a0 0x7fd26c0ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35604/0 (socket says 172.21.15.60:35604) 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.863+0000 7fd269d9b640 1 -- 172.21.15.60:0/3715607400 learned_addr learned my addr 172.21.15.60:0/3715607400 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26ad9d640 1 --2- 172.21.15.60:0/3715607400 >> v2:172.21.15.100:3300/0 conn(0x7fd26c0cedf0 0x7fd26c0d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26a59c640 1 -- 172.21.15.60:0/3715607400 >> v2:172.21.15.100:3300/0 conn(0x7fd26c0cedf0 msgr2=0x7fd26c0d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26a59c640 1 --2- 172.21.15.60:0/3715607400 >> v2:172.21.15.100:3300/0 conn(0x7fd26c0cedf0 0x7fd26c0d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26a59c640 1 -- 172.21.15.60:0/3715607400 >> v1:172.21.15.60:6789/0 conn(0x7fd26c0b93a0 legacy=0x7fd26c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26a59c640 1 -- 172.21.15.60:0/3715607400 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c161de0 con 0x7fd26c15dc20 2024-04-25T08:36:31.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26959a640 1 -- 172.21.15.60:0/3715607400 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399017483 0 0) 0x7fd254001110 con 0x7fd26c0b93a0 2024-04-25T08:36:31.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26a59c640 1 --2- 172.21.15.60:0/3715607400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c15dc20 0x7fd26c1616b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0b6980 tx=0x7fd26402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1584bf1d2664197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.864+0000 7fd26959a640 1 -- 172.21.15.60:0/3715607400 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264007740 con 0x7fd26c15dc20 2024-04-25T08:36:31.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd26959a640 1 -- 172.21.15.60:0/3715607400 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2640078e0 con 0x7fd26c15dc20 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd26959a640 1 -- 172.21.15.60:0/3715607400 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264007cb0 con 0x7fd26c15dc20 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 -- 172.21.15.60:0/3715607400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c15dc20 msgr2=0x7fd26c1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 --2- 172.21.15.60:0/3715607400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c15dc20 0x7fd26c1616b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0b6980 tx=0x7fd26402a680 comp rx=0 tx=0).stop 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 -- 172.21.15.60:0/3715607400 shutdown_connections 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 --2- 172.21.15.60:0/3715607400 >> v2:172.21.15.100:3300/0 conn(0x7fd26c0cedf0 0x7fd26c0d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 --2- 172.21.15.60:0/3715607400 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c15dc20 0x7fd26c1616b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 -- 172.21.15.60:0/3715607400 >> 172.21.15.60:0/3715607400 conn(0x7fd26c0d0490 msgr2=0x7fd26c0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 -- 172.21.15.60:0/3715607400 shutdown_connections 2024-04-25T08:36:31.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.865+0000 7fd270841640 1 -- 172.21.15.60:0/3715607400 wait complete. 2024-04-25T08:36:31.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 Processor -- start 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 -- start start 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 0x7fd26c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd26c0cedf0 0x7fd26c14f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd26c15dc20 0x7fd26c14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0d2a50 con 0x7fd26c0b93a0 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c1623a0 con 0x7fd26c15dc20 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd270841640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c0d2bd0 con 0x7fd26c0cedf0 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd26a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 0x7fd26c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd26a59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 0x7fd26c15f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54784/0 (socket says 172.21.15.60:54784) 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.867+0000 7fd26a59c640 1 -- 172.21.15.60:0/446485675 learned_addr learned my addr 172.21.15.60:0/446485675 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26ad9d640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd26c15dc20 0x7fd26c14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd269d9b640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd26c0cedf0 0x7fd26c14f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26a59c640 1 -- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd26c0cedf0 msgr2=0x7fd26c14f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26a59c640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd26c0cedf0 0x7fd26c14f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26a59c640 1 -- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd26c15dc20 msgr2=0x7fd26c14bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26a59c640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd26c15dc20 0x7fd26c14bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26a59c640 1 -- 172.21.15.60:0/446485675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c14f600 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26ad9d640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd26c15dc20 0x7fd26c14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd26a59c640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 0x7fd26c15f6d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd264009e80 tx=0x7fd264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26403f070 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264031d00 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26c15e630 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264044680 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.868+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26c15ebf0 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.869+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26c073990 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.870+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd26403b410 con 0x7fd26c0b93a0 2024-04-25T08:36:31.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.870+0000 7fd262ffd640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd24c06b500 0x7fd24c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.870+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd2640ab970 con 0x7fd26c0b93a0 2024-04-25T08:36:31.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.870+0000 7fd269d9b640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd24c06b500 0x7fd24c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.871+0000 7fd269d9b640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd24c06b500 0x7fd24c06d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd26c1605e0 tx=0x7fd254007970 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:31.875+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd26407a9b0 con 0x7fd26c0b93a0 2024-04-25T08:36:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.121+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd26c0d1720 con 0x7fd26c0b93a0 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.121+0000 7fd262ffd640 1 -- 172.21.15.60:0/446485675 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd26407a2b0 con 0x7fd26c0b93a0 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:36:32.116 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.117 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.118 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.120 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.121 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.123 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.124 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.125 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:36:32.126 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:36:32.126 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:36:32.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.127 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:36:32.127 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:36:32.127 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:36:32.127 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:36:32.128 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:36:32.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.128 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.129 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.130 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:36:32.131 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:36:32.132 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.132 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:32.132 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:36:32.132 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:32.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd24c06b500 msgr2=0x7fd24c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd24c06b500 0x7fd24c06d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd26c1605e0 tx=0x7fd254007970 comp rx=0 tx=0).stop 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 msgr2=0x7fd26c15f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 0x7fd26c15f6d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd264009e80 tx=0x7fd264004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 shutdown_connections 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd24c06b500 0x7fd24c06d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd26c15dc20 0x7fd26c14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd26c0cedf0 0x7fd26c14f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 --2- 172.21.15.60:0/446485675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd26c0b93a0 0x7fd26c15f6d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 >> 172.21.15.60:0/446485675 conn(0x7fd26c0d0490 msgr2=0x7fd26c0b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 shutdown_connections 2024-04-25T08:36:32.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.123+0000 7fd270841640 1 -- 172.21.15.60:0/446485675 wait complete. 2024-04-25T08:36:32.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:32.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.241+0000 7fa12e76a640 1 Processor -- start 2024-04-25T08:36:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.241+0000 7fa12e76a640 1 -- start start 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa12e76a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1280cfec0 0x7fa1280cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa12e76a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa128071480 con 0x7fa128154ee0 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa12e76a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa128071760 con 0x7fa1280ce3c0 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa12e76a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280ce830 con 0x7fa1280cfec0 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa1277fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa1280cfec0 0x7fa1280cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa127fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa128154ee0 0x7fa1280cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35614/0 (socket says 172.21.15.60:35614) 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa127fff640 1 -- 172.21.15.60:0/1876997708 learned_addr learned my addr 172.21.15.60:0/1876997708 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa1277fe640 1 -- 172.21.15.60:0/1876997708 >> v1:172.21.15.100:6789/0 conn(0x7fa1280ce3c0 legacy=0x7fa12806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa1277fe640 1 -- 172.21.15.60:0/1876997708 >> v1:172.21.15.60:6789/0 conn(0x7fa128154ee0 legacy=0x7fa1280cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.242+0000 7fa1277fe640 1 -- 172.21.15.60:0/1876997708 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128071a40 con 0x7fa1280cfec0 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.243+0000 7fa1277fe640 1 --2- 172.21.15.60:0/1876997708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cfec0 0x7fa1280cde80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa1180097a0 tx=0x7fa11802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805ae5691dad5a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.243+0000 7fa126ffd640 1 -- 172.21.15.60:0/1876997708 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118037070 con 0x7fa1280cfec0 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.243+0000 7fa126ffd640 1 -- 172.21.15.60:0/1876997708 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118032440 con 0x7fa1280cfec0 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.243+0000 7fa126ffd640 1 -- 172.21.15.60:0/1876997708 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118032810 con 0x7fa1280cfec0 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 -- 172.21.15.60:0/1876997708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cfec0 msgr2=0x7fa1280cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 --2- 172.21.15.60:0/1876997708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cfec0 0x7fa1280cde80 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa1180097a0 tx=0x7fa11802a230 comp rx=0 tx=0).stop 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 -- 172.21.15.60:0/1876997708 shutdown_connections 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 --2- 172.21.15.60:0/1876997708 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280cfec0 0x7fa1280cde80 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 -- 172.21.15.60:0/1876997708 >> 172.21.15.60:0/1876997708 conn(0x7fa12806abe0 msgr2=0x7fa12806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 -- 172.21.15.60:0/1876997708 shutdown_connections 2024-04-25T08:36:32.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.244+0000 7fa12e76a640 1 -- 172.21.15.60:0/1876997708 wait complete. 2024-04-25T08:36:32.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.245+0000 7fa12e76a640 1 Processor -- start 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.245+0000 7fa12e76a640 1 -- start start 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12e76a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 0x7fa1281580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12e76a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1280cfec0 0x7fa128158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12e76a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa128154ee0 0x7fa12814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12e76a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128071230 con 0x7fa1280ce3c0 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12e76a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281552c0 con 0x7fa1280cfec0 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12e76a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa128155440 con 0x7fa128154ee0 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 0x7fa1281580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 0x7fa1281580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54804/0 (socket says 172.21.15.60:54804) 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 -- 172.21.15.60:0/3438165033 learned_addr learned my addr 172.21.15.60:0/3438165033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa1277fe640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1280cfec0 0x7fa128158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa12cd07640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa128154ee0 0x7fa12814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 -- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa128154ee0 msgr2=0x7fa12814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa128154ee0 0x7fa12814f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 -- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1280cfec0 msgr2=0x7fa128158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1280cfec0 0x7fa128158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 -- 172.21.15.60:0/3438165033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12813dfe0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa1277fe640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1280cfec0 0x7fa128158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.246+0000 7fa127fff640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 0x7fa1281580e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00c480 tx=0x7fa11c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.247+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c017070 con 0x7fa1280ce3c0 2024-04-25T08:36:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.247+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c005ce0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.247+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12813e2c0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.247+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0072f0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.247+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12813e880 con 0x7fa1280ce3c0 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.248+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa128073990 con 0x7fa1280ce3c0 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.248+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa11c01a070 con 0x7fa1280ce3c0 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.249+0000 7fa124ff9640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa11406b4d0 0x7fa11406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.249+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa11c089230 con 0x7fa1280ce3c0 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.249+0000 7fa1277fe640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa11406b4d0 0x7fa11406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.249+0000 7fa1277fe640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa11406b4d0 0x7fa11406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa118034000 tx=0x7fa11802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.253+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa11c057cb0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.439+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa12813eac0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.439+0000 7fa124ff9640 1 -- 172.21.15.60:0/3438165033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13154 (secure 0 0 0) 0x7fa12813eac0 con 0x7fa1280ce3c0 2024-04-25T08:36:32.434 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:32.434 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":106,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:31.745543+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa11406b4d0 msgr2=0x7fa11406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa11406b4d0 0x7fa11406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa118034000 tx=0x7fa11802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 msgr2=0x7fa1281580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 0x7fa1281580e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00c480 tx=0x7fa11c00c940 comp rx=0 tx=0).stop 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 shutdown_connections 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa128154ee0 0x7fa12814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa11406b4d0 0x7fa11406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1280cfec0 0x7fa128158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 --2- 172.21.15.60:0/3438165033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1280ce3c0 0x7fa1281580e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 >> 172.21.15.60:0/3438165033 conn(0x7fa12806abe0 msgr2=0x7fa1280b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 shutdown_connections 2024-04-25T08:36:32.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.441+0000 7fa12e76a640 1 -- 172.21.15.60:0/3438165033 wait complete. 2024-04-25T08:36:32.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:32.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f5641b8b640 1 Processor -- start 2024-04-25T08:36:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f5641b8b640 1 -- start start 2024-04-25T08:36:32.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f5641b8b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f563c161890 0x7f563c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f5641b8b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0cf1d0 con 0x7f563c161240 2024-04-25T08:36:32.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f5641b8b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c161620 con 0x7f563c0b54a0 2024-04-25T08:36:32.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f5641b8b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0b7120 con 0x7f563c161890 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f563affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f563c161890 0x7f563c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f563b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f563c161240 0x7f563c0ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35630/0 (socket says 172.21.15.60:35630) 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.560+0000 7f563b7fe640 1 -- 172.21.15.60:0/3546063147 learned_addr learned my addr 172.21.15.60:0/3546063147 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.561+0000 7f563affd640 1 -- 172.21.15.60:0/3546063147 >> v1:172.21.15.100:6789/0 conn(0x7f563c0b54a0 legacy=0x7f563c0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.561+0000 7f563affd640 1 -- 172.21.15.60:0/3546063147 >> v1:172.21.15.60:6789/0 conn(0x7f563c161240 legacy=0x7f563c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.561+0000 7f563affd640 1 -- 172.21.15.60:0/3546063147 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c161e00 con 0x7f563c161890 2024-04-25T08:36:32.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.561+0000 7f563a7fc640 1 -- 172.21.15.60:0/3546063147 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760966743 0 0) 0x7f563c0cf1d0 con 0x7f563c161240 2024-04-25T08:36:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f563affd640 1 --2- 172.21.15.60:0/3546063147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c161890 0x7f563c0b6be0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f56300097a0 tx=0x7f563002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7e0fd4088b75c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f563a7fc640 1 -- 172.21.15.60:0/3546063147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630037070 con 0x7f563c161890 2024-04-25T08:36:32.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f563a7fc640 1 -- 172.21.15.60:0/3546063147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5630032440 con 0x7f563c161890 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f563a7fc640 1 -- 172.21.15.60:0/3546063147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630032810 con 0x7f563c161890 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 -- 172.21.15.60:0/3546063147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c161890 msgr2=0x7f563c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 --2- 172.21.15.60:0/3546063147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c161890 0x7f563c0b6be0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f56300097a0 tx=0x7f563002a230 comp rx=0 tx=0).stop 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 -- 172.21.15.60:0/3546063147 shutdown_connections 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 --2- 172.21.15.60:0/3546063147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c161890 0x7f563c0b6be0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 -- 172.21.15.60:0/3546063147 >> 172.21.15.60:0/3546063147 conn(0x7f563c0d04a0 msgr2=0x7f563c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 -- 172.21.15.60:0/3546063147 shutdown_connections 2024-04-25T08:36:32.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.562+0000 7f5641b8b640 1 -- 172.21.15.60:0/3546063147 wait complete. 2024-04-25T08:36:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.563+0000 7f5641b8b640 1 Processor -- start 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 -- start start 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 0x7f563c14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563c161240 0x7f563c15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f563c161890 0x7f563c15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1626a0 con 0x7f563c0b54a0 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1623c0 con 0x7f563c161890 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f5641b8b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0d15f0 con 0x7f563c161240 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.564+0000 7f563b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 0x7f563c14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 0x7f563c14bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54816/0 (socket says 172.21.15.60:54816) 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 -- 172.21.15.60:0/957890469 learned_addr learned my addr 172.21.15.60:0/957890469 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563bfff640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f563c161890 0x7f563c15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563affd640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563c161240 0x7f563c15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 -- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563c161240 msgr2=0x7f563c15daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563c161240 0x7f563c15daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 -- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f563c161890 msgr2=0x7f563c15dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f563c161890 0x7f563c15dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 -- 172.21.15.60:0/957890469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c14c310 con 0x7f563c0b54a0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563bfff640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f563c161890 0x7f563c15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f563b7fe640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 0x7f563c14bdd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c4a0 tx=0x7f562c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c018070 con 0x7f563c0b54a0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0074e0 con 0x7f563c0b54a0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c160a20 con 0x7f563c0b54a0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c004a20 con 0x7f563c0b54a0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.565+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c15f980 con 0x7f563c0b54a0 2024-04-25T08:36:32.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.566+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c073a00 con 0x7f563c0b54a0 2024-04-25T08:36:32.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.567+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f562c01b070 con 0x7f563c0b54a0 2024-04-25T08:36:32.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.567+0000 7f5623fff640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.567+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f562c0892b0 con 0x7f563c0b54a0 2024-04-25T08:36:32.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.567+0000 7f563affd640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.568+0000 7f563affd640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.571+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f562c0582f0 con 0x7f563c0b54a0 2024-04-25T08:36:32.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.751+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f563c0c67c0 con 0x7f563c0b54a0 2024-04-25T08:36:32.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.752+0000 7f5623fff640 1 -- 172.21.15.60:0/957890469 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f562c057bf0 con 0x7f563c0b54a0 2024-04-25T08:36:32.746 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:36:32.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 msgr2=0x7f562806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5630034000 tx=0x7f563002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:32.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 msgr2=0x7f563c14bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 0x7f563c14bdd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c4a0 tx=0x7f562c00c960 comp rx=0 tx=0).stop 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 shutdown_connections 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f563c161890 0x7f563c15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f563c161240 0x7f563c15daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 --2- 172.21.15.60:0/957890469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f563c0b54a0 0x7f563c14bdd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 >> 172.21.15.60:0/957890469 conn(0x7f563c0d04a0 msgr2=0x7f563c0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 shutdown_connections 2024-04-25T08:36:32.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.754+0000 7f5641b8b640 1 -- 172.21.15.60:0/957890469 wait complete. 2024-04-25T08:36:32.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:36:32.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 Processor -- start 2024-04-25T08:36:32.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 -- start start 2024-04-25T08:36:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c7815db70 0x7f0c78162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0c780b5c30 0x7f0c780d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c78162690 con 0x7f0c780b6cb0 2024-04-25T08:36:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780ceee0 con 0x7f0c7815db70 2024-04-25T08:36:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.866+0000 7f0c7d038640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780d1d10 con 0x7f0c780b5c30 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c7815db70 0x7f0c78162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c7815db70 0x7f0c78162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54826/0 (socket says 172.21.15.60:54826) 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 -- 172.21.15.60:0/226825384 learned_addr learned my addr 172.21.15.60:0/226825384 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 -- 172.21.15.60:0/226825384 >> v2:172.21.15.100:3300/0 conn(0x7f0c780b5c30 msgr2=0x7f0c780d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c7759e640 1 --2- 172.21.15.60:0/226825384 >> v2:172.21.15.100:3300/0 conn(0x7f0c780b5c30 0x7f0c780d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 --2- 172.21.15.60:0/226825384 >> v2:172.21.15.100:3300/0 conn(0x7f0c780b5c30 0x7f0c780d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 -- 172.21.15.60:0/226825384 >> v1:172.21.15.100:6789/0 conn(0x7f0c780b6cb0 legacy=0x7f0c780b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c76d9d640 1 -- 172.21.15.60:0/226825384 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c780d1ff0 con 0x7f0c7815db70 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.867+0000 7f0c7759e640 1 --2- 172.21.15.60:0/226825384 >> v2:172.21.15.100:3300/0 conn(0x7f0c780b5c30 0x7f0c780d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c76d9d640 1 --2- 172.21.15.60:0/226825384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7815db70 0x7f0c78162150 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0c7808ccc0 tx=0x7f0c6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823637ffe2563f76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7659c640 1 -- 172.21.15.60:0/226825384 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68037070 con 0x7f0c7815db70 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7659c640 1 -- 172.21.15.60:0/226825384 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68032440 con 0x7f0c7815db70 2024-04-25T08:36:32.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7659c640 1 -- 172.21.15.60:0/226825384 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68032810 con 0x7f0c7815db70 2024-04-25T08:36:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 -- 172.21.15.60:0/226825384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7815db70 msgr2=0x7f0c78162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 --2- 172.21.15.60:0/226825384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7815db70 0x7f0c78162150 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f0c7808ccc0 tx=0x7f0c6802a230 comp rx=0 tx=0).stop 2024-04-25T08:36:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 -- 172.21.15.60:0/226825384 shutdown_connections 2024-04-25T08:36:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 --2- 172.21.15.60:0/226825384 >> v2:172.21.15.100:3300/0 conn(0x7f0c780b5c30 0x7f0c780d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 --2- 172.21.15.60:0/226825384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c7815db70 0x7f0c78162150 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 -- 172.21.15.60:0/226825384 >> 172.21.15.60:0/226825384 conn(0x7f0c780d0520 msgr2=0x7f0c780d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 -- 172.21.15.60:0/226825384 shutdown_connections 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.868+0000 7f0c7d038640 1 -- 172.21.15.60:0/226825384 wait complete. 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 Processor -- start 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 -- start start 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c780b5c30 0x7f0c781f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 0x7f0c780aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c7815db70 0x7f0c780cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780ce530 con 0x7f0c780b6cb0 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815df50 con 0x7f0c7815db70 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.870+0000 7f0c7d038640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c780cf1c0 con 0x7f0c780b5c30 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 0x7f0c780aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 0x7f0c780aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54830/0 (socket says 172.21.15.60:54830) 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 -- 172.21.15.60:0/3872511991 learned_addr learned my addr 172.21.15.60:0/3872511991 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:32.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c7759e640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c7815db70 0x7f0c780cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c76d9d640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c780b5c30 0x7f0c781f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 -- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c780b5c30 msgr2=0x7f0c781f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c780b5c30 0x7f0c781f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 -- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c7815db70 msgr2=0x7f0c780cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c7815db70 0x7f0c780cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 -- 172.21.15.60:0/3872511991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c781f4520 con 0x7f0c780b6cb0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6ffff640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 0x7f0c780aabf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c400 tx=0x7f0c6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60011790 con 0x7f0c780b6cb0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c60017070 con 0x7f0c780b6cb0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60011bb0 con 0x7f0c780b6cb0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c781f4800 con 0x7f0c780b6cb0 2024-04-25T08:36:32.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.871+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c780aa3f0 con 0x7f0c780b6cb0 2024-04-25T08:36:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.872+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78073990 con 0x7f0c780b6cb0 2024-04-25T08:36:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.873+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c6001c460 con 0x7f0c780b6cb0 2024-04-25T08:36:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.873+0000 7f0c6f7fe640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c5806b500 0x7f0c5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.873+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0c60089df0 con 0x7f0c780b6cb0 2024-04-25T08:36:32.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.873+0000 7f0c76d9d640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c5806b500 0x7f0c5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.874+0000 7f0c76d9d640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c5806b500 0x7f0c5806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c68034000 tx=0x7f0c6802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:32.878+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c60058ec0 con 0x7f0c780b6cb0 2024-04-25T08:36:33.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.058+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0c780d1820 con 0x7f0c780b6cb0 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.059+0000 7f0c6f7fe640 1 -- 172.21.15.60:0/3872511991 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0c600587c0 con 0x7f0c780b6cb0 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:36:33.054 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:36:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c5806b500 msgr2=0x7f0c5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c5806b500 0x7f0c5806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c68034000 tx=0x7f0c6802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 msgr2=0x7f0c780aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 0x7f0c780aabf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c400 tx=0x7f0c6000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 shutdown_connections 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c5806b500 0x7f0c5806d9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c7815db70 0x7f0c780cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c780b6cb0 0x7f0c780aabf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 --2- 172.21.15.60:0/3872511991 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c780b5c30 0x7f0c781f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 >> 172.21.15.60:0/3872511991 conn(0x7f0c780d0520 msgr2=0x7f0c780d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 shutdown_connections 2024-04-25T08:36:33.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.061+0000 7f0c7d038640 1 -- 172.21.15.60:0/3872511991 wait complete. 2024-04-25T08:36:33.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:36:33.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:36:33.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.177+0000 7fa362509640 1 Processor -- start 2024-04-25T08:36:33.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.177+0000 7fa362509640 1 -- start start 2024-04-25T08:36:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa362509640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa35c0d34c0 0x7fa35c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa362509640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0d2420 con 0x7fa35c15dac0 2024-04-25T08:36:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa362509640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0d2700 con 0x7fa35c0d08e0 2024-04-25T08:36:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa362509640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c0d0d50 con 0x7fa35c0d34c0 2024-04-25T08:36:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa35bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa35c15dac0 0x7fa35c0d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35634/0 (socket says 172.21.15.60:35634) 2024-04-25T08:36:33.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa35bfff640 1 -- 172.21.15.60:0/692562044 learned_addr learned my addr 172.21.15.60:0/692562044 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363014623 0 0) 0x7fa35c0d2420 con 0x7fa35c15dac0 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa348003520 con 0x7fa35c15dac0 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.178+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/692562044 >> v2:172.21.15.100:3300/0 conn(0x7fa35c0d34c0 0x7fa35c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1988294348 0 0) 0x7fa348003520 con 0x7fa35c15dac0 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 >> v2:172.21.15.100:3300/0 conn(0x7fa35c0d34c0 msgr2=0x7fa35c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 --2- 172.21.15.60:0/692562044 >> v2:172.21.15.100:3300/0 conn(0x7fa35c0d34c0 0x7fa35c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 >> v1:172.21.15.100:6789/0 conn(0x7fa35c0d08e0 legacy=0x7fa35c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0d29e0 con 0x7fa35c15dac0 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fa344001670 con 0x7fa35c15dac0 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 >> v1:172.21.15.60:6789/0 conn(0x7fa35c15dac0 legacy=0x7fa35c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0d0d50 con 0x7fa348004800 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800c070 con 0x7fa3480039b0 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800c270 con 0x7fa3480051c0 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35bfff640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa360aa6640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35b7fe640 1 -- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480051c0 msgr2=0x7fa34800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa3480051c0 0x7fa34800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35b7fe640 1 -- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480039b0 msgr2=0x7fa348003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3480039b0 0x7fa348003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa35b7fe640 1 -- 172.21.15.60:0/692562044 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348003520 con 0x7fa348004800 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.179+0000 7fa360aa6640 1 -- 172.21.15.60:0/692562044 reap_dead start 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.180+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa348004800 0x7fa348004c80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009ab0 tx=0x7fa348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907b61a0e6a1fb49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.180+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c036070 con 0x7fa348004800 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.180+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c005990 con 0x7fa348004800 2024-04-25T08:36:33.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.180+0000 7fa35affd640 1 -- 172.21.15.60:0/692562044 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009680 con 0x7fa348004800 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 -- 172.21.15.60:0/692562044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa348004800 msgr2=0x7fa348004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa348004800 0x7fa348004c80 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009ab0 tx=0x7fa348005830 comp rx=0 tx=0).stop 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 -- 172.21.15.60:0/692562044 shutdown_connections 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 --2- 172.21.15.60:0/692562044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa348004800 0x7fa348004c80 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 -- 172.21.15.60:0/692562044 >> 172.21.15.60:0/692562044 conn(0x7fa35c061c30 msgr2=0x7fa35c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 -- 172.21.15.60:0/692562044 shutdown_connections 2024-04-25T08:36:33.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.181+0000 7fa362509640 1 -- 172.21.15.60:0/692562044 wait complete. 2024-04-25T08:36:33.180 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.182+0000 7fa362509640 1 Processor -- start 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.182+0000 7fa362509640 1 -- start start 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa348004800 0x7fa35c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 0x7fa35c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35c0b4bd0 0x7fa35c15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c15dea0 con 0x7fa35c158030 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0d3030 con 0x7fa35c0b4bd0 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0b97d0 con 0x7fa348004800 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 0x7fa35c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 0x7fa35c15e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54844/0 (socket says 172.21.15.60:54844) 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 -- 172.21.15.60:0/3158112693 learned_addr learned my addr 172.21.15.60:0/3158112693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35bfff640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa348004800 0x7fa35c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa360aa6640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35c0b4bd0 0x7fa35c15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 -- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa348004800 msgr2=0x7fa35c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa348004800 0x7fa35c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 -- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35c0b4bd0 msgr2=0x7fa35c15fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35c0b4bd0 0x7fa35c15fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 -- 172.21.15.60:0/3158112693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c15f2a0 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa35b7fe640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 0x7fa35c15e020 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa34c034040 tx=0x7fa34c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c040020 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c006660 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c15e720 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.183+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c036040 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.184+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c15ece0 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.185+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c073a00 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.185+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa34c043020 con 0x7fa35c158030 2024-04-25T08:36:33.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.185+0000 7fa358ff9640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa33c06b500 0x7fa33c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.185+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa34c0aa4a0 con 0x7fa35c158030 2024-04-25T08:36:33.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.185+0000 7fa35bfff640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa33c06b500 0x7fa33c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.186+0000 7fa35bfff640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa33c06b500 0x7fa33c06d9c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa3440011e0 tx=0x7fa3440078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:33.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.190+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa34c04a030 con 0x7fa35c158030 2024-04-25T08:36:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.369+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa35c0c78d0 con 0x7fa35c158030 2024-04-25T08:36:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.736+0000 7fa358ff9640 1 -- 172.21.15.60:0/3158112693 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa34c078e80 con 0x7fa35c158030 2024-04-25T08:36:33.731 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.738+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa33c06b500 msgr2=0x7fa33c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.738+0000 7fa362509640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa33c06b500 0x7fa33c06d9c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa3440011e0 tx=0x7fa3440078f0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.738+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 msgr2=0x7fa35c15e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.738+0000 7fa362509640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 0x7fa35c15e020 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa34c034040 tx=0x7fa34c004730 comp rx=0 tx=0).stop 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 shutdown_connections 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa33c06b500 0x7fa33c06d9c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa35c0b4bd0 0x7fa35c15fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa35c158030 0x7fa35c15e020 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 --2- 172.21.15.60:0/3158112693 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa348004800 0x7fa35c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 >> 172.21.15.60:0/3158112693 conn(0x7fa35c061c30 msgr2=0x7fa35c0b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:33.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 shutdown_connections 2024-04-25T08:36:33.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.739+0000 7fa362509640 1 -- 172.21.15.60:0/3158112693 wait complete. 2024-04-25T08:36:33.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:36:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.862+0000 7fdfdfd70640 1 Processor -- start 2024-04-25T08:36:33.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.862+0000 7fdfdfd70640 1 -- start start 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdfd70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfd806e360 0x7fdfd806e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdfd70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd806ec80 0x7fdfd80b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdfd70640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd815e020 con 0x7fdfd80b3e10 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdfd70640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd806b890 con 0x7fdfd806e360 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdfd70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd806ac80 con 0x7fdfd806ec80 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfddb0c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdfd806e360 0x7fdfd806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfde30d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdfd80b3e10 0x7fdfd806bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35650/0 (socket says 172.21.15.60:35650) 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfde30d640 1 -- 172.21.15.60:0/2522139051 learned_addr learned my addr 172.21.15.60:0/2522139051 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdd30b640 1 --2- 172.21.15.60:0/2522139051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd806ec80 0x7fdfd80b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfddb0c640 1 -- 172.21.15.60:0/2522139051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd806ec80 msgr2=0x7fdfd80b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfddb0c640 1 --2- 172.21.15.60:0/2522139051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd806ec80 0x7fdfd80b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfddb0c640 1 -- 172.21.15.60:0/2522139051 >> v1:172.21.15.60:6789/0 conn(0x7fdfd80b3e10 legacy=0x7fdfd806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfddb0c640 1 -- 172.21.15.60:0/2522139051 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd806af60 con 0x7fdfd806e360 2024-04-25T08:36:33.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.863+0000 7fdfdcb0a640 1 -- 172.21.15.60:0/2522139051 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068093173 0 0) 0x7fdfd815e020 con 0x7fdfd80b3e10 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.864+0000 7fdfddb0c640 1 --2- 172.21.15.60:0/2522139051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806e360 0x7fdfd806e740 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40097a0 tx=0x7fdfd402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2f22b9ba65f7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.864+0000 7fdfdcb0a640 1 -- 172.21.15.60:0/2522139051 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4037070 con 0x7fdfd806e360 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.864+0000 7fdfdcb0a640 1 -- 172.21.15.60:0/2522139051 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd4032440 con 0x7fdfd806e360 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.864+0000 7fdfdcb0a640 1 -- 172.21.15.60:0/2522139051 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4032810 con 0x7fdfd806e360 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 -- 172.21.15.60:0/2522139051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806e360 msgr2=0x7fdfd806e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/2522139051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806e360 0x7fdfd806e740 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40097a0 tx=0x7fdfd402a210 comp rx=0 tx=0).stop 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 -- 172.21.15.60:0/2522139051 shutdown_connections 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/2522139051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd806ec80 0x7fdfd80b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/2522139051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806e360 0x7fdfd806e740 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 -- 172.21.15.60:0/2522139051 >> 172.21.15.60:0/2522139051 conn(0x7fdfd80cfec0 msgr2=0x7fdfd80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:33.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 -- 172.21.15.60:0/2522139051 shutdown_connections 2024-04-25T08:36:33.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.865+0000 7fdfdfd70640 1 -- 172.21.15.60:0/2522139051 wait complete. 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.866+0000 7fdfdfd70640 1 Processor -- start 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.866+0000 7fdfdfd70640 1 -- start start 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdfd70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfd806e360 0x7fdfd80cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdfd70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 0x7fdfd81f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdfd70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd80b3e10 0x7fdfd81f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdfd70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80b2fd0 con 0x7fdfd806ec80 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdfd70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd806b520 con 0x7fdfd806e360 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdfd70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80b3150 con 0x7fdfd80b3e10 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 0x7fdfd81f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 0x7fdfd81f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54852/0 (socket says 172.21.15.60:54852) 2024-04-25T08:36:33.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 -- 172.21.15.60:0/4165842967 learned_addr learned my addr 172.21.15.60:0/4165842967 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfde30d640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd80b3e10 0x7fdfd81f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfddb0c640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfd806e360 0x7fdfd80cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 -- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd80b3e10 msgr2=0x7fdfd81f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd80b3e10 0x7fdfd81f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 -- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfd806e360 msgr2=0x7fdfd80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfd806e360 0x7fdfd80cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 -- 172.21.15.60:0/4165842967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd81f4630 con 0x7fdfd806ec80 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.867+0000 7fdfdd30b640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 0x7fdfd81f3430 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e9b0 tx=0x7fdfc800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.868+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8017070 con 0x7fdfd806ec80 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.868+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc800c9a0 con 0x7fdfd806ec80 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.868+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd81549c0 con 0x7fdfd806ec80 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.868+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc800cd90 con 0x7fdfd806ec80 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.868+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd8145c50 con 0x7fdfd806ec80 2024-04-25T08:36:33.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.869+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd8073a00 con 0x7fdfd806ec80 2024-04-25T08:36:33.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.869+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfc801a070 con 0x7fdfd806ec80 2024-04-25T08:36:33.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.869+0000 7fdfce7fc640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdfbc06b4d0 0x7fdfbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:33.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.870+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fdfc808a110 con 0x7fdfd806ec80 2024-04-25T08:36:33.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.870+0000 7fdfddb0c640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdfbc06b4d0 0x7fdfbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:33.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.870+0000 7fdfddb0c640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdfbc06b4d0 0x7fdfbc06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4034000 tx=0x7fdfd402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:33.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:33.874+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfd81f4630 con 0x7fdfd806ec80 2024-04-25T08:36:34.049 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.055+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdfd806f240 con 0x7fdfd806ec80 2024-04-25T08:36:34.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.738+0000 7fdfce7fc640 1 -- 172.21.15.60:0/4165842967 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdfc8059070 con 0x7fdfd806ec80 2024-04-25T08:36:34.733 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:36:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.742+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdfbc06b4d0 msgr2=0x7fdfbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.742+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdfbc06b4d0 0x7fdfbc06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4034000 tx=0x7fdfd402e000 comp rx=0 tx=0).stop 2024-04-25T08:36:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.742+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 msgr2=0x7fdfd81f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.742+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 0x7fdfd81f3430 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e9b0 tx=0x7fdfc800ee70 comp rx=0 tx=0).stop 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 shutdown_connections 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdfd80b3e10 0x7fdfd81f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdfd806ec80 0x7fdfd81f3430 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdfbc06b4d0 0x7fdfbc06d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 --2- 172.21.15.60:0/4165842967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfd806e360 0x7fdfd80cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 >> 172.21.15.60:0/4165842967 conn(0x7fdfd80cfec0 msgr2=0x7fdfd815cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 shutdown_connections 2024-04-25T08:36:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.743+0000 7fdfdfd70640 1 -- 172.21.15.60:0/4165842967 wait complete. 2024-04-25T08:36:34.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:36:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.880+0000 7f3ca9bcf640 1 Processor -- start 2024-04-25T08:36:34.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.880+0000 7f3ca9bcf640 1 -- start start 2024-04-25T08:36:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca9bcf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3ca40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca9bcf640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca406cf50 con 0x7f3ca415b7d0 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca9bcf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca40ce530 con 0x7f3ca40b5620 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca9bcf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca40ce810 con 0x7f3ca406c9e0 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3ca415b7d0 0x7f3ca40cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35652/0 (socket says 172.21.15.60:35652) 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca37fe640 1 -- 172.21.15.60:0/2364105755 learned_addr learned my addr 172.21.15.60:0/2364105755 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca3fff640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3ca40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182969454 0 0) 0x7f3ca406cf50 con 0x7f3ca415b7d0 2024-04-25T08:36:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c8c003520 con 0x7f3ca415b7d0 2024-04-25T08:36:34.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.881+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979522418 0 0) 0x7f3ca40ce530 con 0x7f3ca40b5620 2024-04-25T08:36:34.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ca406cf50 con 0x7f3ca40b5620 2024-04-25T08:36:34.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2111002716 0 0) 0x7f3c8c003520 con 0x7f3ca415b7d0 2024-04-25T08:36:34.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 msgr2=0x7f3ca40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:34.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3ca40b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 >> v1:172.21.15.100:6789/0 conn(0x7f3ca40b5620 legacy=0x7f3ca406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca40ceaf0 con 0x7f3ca415b7d0 2024-04-25T08:36:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca3fff640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3ca40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3c94001690 con 0x7f3ca415b7d0 2024-04-25T08:36:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 >> v1:172.21.15.60:6789/0 conn(0x7f3ca415b7d0 legacy=0x7f3ca40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8c003b70 0x7f3c8c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3c8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c004d40 0x7f3c8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca40ce810 con 0x7f3c8c003b70 2024-04-25T08:36:34.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c00baa0 con 0x7f3c8c004d40 2024-04-25T08:36:34.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c00bca0 con 0x7f3ca406c9e0 2024-04-25T08:36:34.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8c003b70 0x7f3c8c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca3fff640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c004d40 0x7f3c8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca2ffd640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3c8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca37fe640 1 -- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 msgr2=0x7f3c8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:34.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3ca406c9e0 0x7f3c8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca37fe640 1 -- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c004d40 msgr2=0x7f3c8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:34.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c8c004d40 0x7f3c8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.882+0000 7f3ca37fe640 1 -- 172.21.15.60:0/2364105755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c003520 con 0x7f3c8c003b70 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8c003b70 0x7f3c8c003f70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3c94007fb0 tx=0x7f3c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eac39b7ff66fde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c94007c00 con 0x7f3c8c003b70 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c94005ad0 con 0x7f3c8c003b70 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca27fc640 1 -- 172.21.15.60:0/2364105755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c94007070 con 0x7f3c8c003b70 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/2364105755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8c003b70 msgr2=0x7f3c8c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/2364105755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3c8c003b70 0x7f3c8c003f70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3c94007fb0 tx=0x7f3c8c005830 comp rx=0 tx=0).stop 2024-04-25T08:36:34.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.883+0000 7f3ca3fff640 1 -- 172.21.15.60:0/2364105755 reap_dead start 2024-04-25T08:36:34.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.884+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/2364105755 shutdown_connections 2024-04-25T08:36:34.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.884+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/2364105755 >> 172.21.15.60:0/2364105755 conn(0x7f3ca40d0800 msgr2=0x7f3ca40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:34.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.884+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/2364105755 shutdown_connections 2024-04-25T08:36:34.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.884+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/2364105755 wait complete. 2024-04-25T08:36:34.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.885+0000 7f3ca9bcf640 1 Processor -- start 2024-04-25T08:36:34.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.885+0000 7f3ca9bcf640 1 -- start start 2024-04-25T08:36:34.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c8c003b70 0x7f3ca40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca41568a0 0x7f3ca40c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 0x7f3ca40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca415bd30 con 0x7f3ca41561d0 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca415bbb0 con 0x7f3ca41568a0 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca40b5ab0 con 0x7f3c8c003b70 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 0x7f3ca40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 0x7f3ca40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54870/0 (socket says 172.21.15.60:54870) 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 -- 172.21.15.60:0/1280300504 learned_addr learned my addr 172.21.15.60:0/1280300504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:34.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c8c003b70 0x7f3ca40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca2ffd640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca41568a0 0x7f3ca40c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 -- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c8c003b70 msgr2=0x7f3ca40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c8c003b70 0x7f3ca40cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 -- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca41568a0 msgr2=0x7f3ca40c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca41568a0 0x7f3ca40c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:34.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 -- 172.21.15.60:0/1280300504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4156650 con 0x7f3ca41561d0 2024-04-25T08:36:34.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca3fff640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 0x7f3ca40cef10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40d4ca0 tx=0x7f3c9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:34.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c00fd10 con 0x7f3ca41561d0 2024-04-25T08:36:34.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c003990 con 0x7f3ca41561d0 2024-04-25T08:36:34.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.886+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca40c86d0 con 0x7f3ca41561d0 2024-04-25T08:36:34.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.887+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c003d80 con 0x7f3ca41561d0 2024-04-25T08:36:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.887+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca40c8fc0 con 0x7f3ca41561d0 2024-04-25T08:36:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.887+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca4073a00 con 0x7f3ca41561d0 2024-04-25T08:36:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.888+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c9c004020 con 0x7f3ca41561d0 2024-04-25T08:36:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.888+0000 7f3c8bfff640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c9006b500 0x7f3c9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.889+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3c9c08ad80 con 0x7f3ca41561d0 2024-04-25T08:36:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.889+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c9006b500 0x7f3c9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:34.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.889+0000 7f3ca37fe640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c9006b500 0x7f3c9006d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c94032000 tx=0x7f3c94009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:34.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:34.893+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c9c019030 con 0x7f3ca41561d0 2024-04-25T08:36:35.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.116+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3ca40b3790 con 0x7f3ca41561d0 2024-04-25T08:36:35.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.762+0000 7f3c8bfff640 1 -- 172.21.15.60:0/1280300504 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3c9c010040 con 0x7f3ca41561d0 2024-04-25T08:36:35.757 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T08:36:35.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.770+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c9006b500 msgr2=0x7f3c9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.770+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c9006b500 0x7f3c9006d9c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c94032000 tx=0x7f3c94009030 comp rx=0 tx=0).stop 2024-04-25T08:36:35.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.770+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 msgr2=0x7f3ca40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.770+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 0x7f3ca40cef10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40d4ca0 tx=0x7f3c9c009f90 comp rx=0 tx=0).stop 2024-04-25T08:36:35.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.772+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 shutdown_connections 2024-04-25T08:36:35.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.772+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ca41561d0 0x7f3ca40cef10 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.772+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3c9006b500 0x7f3c9006d9c0 secure :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c94032000 tx=0x7f3c94009030 comp rx=0 tx=0).stop 2024-04-25T08:36:35.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.772+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca41568a0 0x7f3ca40c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.772+0000 7f3ca9bcf640 1 --2- 172.21.15.60:0/1280300504 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3c8c003b70 0x7f3ca40cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.772+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 >> 172.21.15.60:0/1280300504 conn(0x7f3ca40d0800 msgr2=0x7f3ca415f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:35.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.776+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 shutdown_connections 2024-04-25T08:36:35.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.776+0000 7f3ca9bcf640 1 -- 172.21.15.60:0/1280300504 wait complete. 2024-04-25T08:36:35.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:35.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.901+0000 7f09962e2640 1 Processor -- start 2024-04-25T08:36:35.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.901+0000 7f09962e2640 1 -- start start 2024-04-25T08:36:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f09962e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 0x7f09900d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f09962e2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900b97a0 con 0x7f099015dbe0 2024-04-25T08:36:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f09962e2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900d1c50 con 0x7f09900d1020 2024-04-25T08:36:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f09962e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099015dfc0 con 0x7f09900b2cb0 2024-04-25T08:36:35.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f098ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f099015dbe0 0x7f09900d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35664/0 (socket says 172.21.15.60:35664) 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f098ffff640 1 -- 172.21.15.60:0/747605246 learned_addr learned my addr 172.21.15.60:0/747605246 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.902+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515021663 0 0) 0x7f09900b97a0 con 0x7f099015dbe0 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 0x7f09900d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097c003520 con 0x7f099015dbe0 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1804020742 0 0) 0x7f097c003520 con 0x7f099015dbe0 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 msgr2=0x7f09900d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 0x7f09900d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 >> v1:172.21.15.100:6789/0 conn(0x7f09900d1020 legacy=0x7f09900d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900d1f30 con 0x7f099015dbe0 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f0980001670 con 0x7f099015dbe0 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 >> v1:172.21.15.60:6789/0 conn(0x7f099015dbe0 legacy=0x7f09900d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 0x7f097c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097c003f60 0x7f097c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c00ba70 con 0x7f097c003f60 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c00bc20 con 0x7f097c004d40 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c00be20 con 0x7f09900b2cb0 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097c003f60 0x7f097c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098ffff640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 0x7f097c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f099487f640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 -- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 msgr2=0x7f097c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f09900b2cb0 0x7f097c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 -- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f097c004d40 msgr2=0x7f097c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f097c004d40 0x7f097c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.903+0000 7f098f7fe640 1 -- 172.21.15.60:0/747605246 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c003520 con 0x7f097c003f60 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.904+0000 7f098f7fe640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097c003f60 0x7f097c004800 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f0978009ac0 tx=0x7f097c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b556307a85bb7a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.904+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978036070 con 0x7f097c003f60 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.904+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978031400 con 0x7f097c003f60 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.904+0000 7f098effd640 1 -- 172.21.15.60:0/747605246 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978009690 con 0x7f097c003f60 2024-04-25T08:36:35.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f09962e2640 1 -- 172.21.15.60:0/747605246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097c003f60 msgr2=0x7f097c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f09962e2640 1 --2- 172.21.15.60:0/747605246 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f097c003f60 0x7f097c004800 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f0978009ac0 tx=0x7f097c005830 comp rx=0 tx=0).stop 2024-04-25T08:36:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f099487f640 1 -- 172.21.15.60:0/747605246 reap_dead start 2024-04-25T08:36:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f09962e2640 1 -- 172.21.15.60:0/747605246 shutdown_connections 2024-04-25T08:36:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f09962e2640 1 -- 172.21.15.60:0/747605246 >> 172.21.15.60:0/747605246 conn(0x7f0990061a20 msgr2=0x7f0990061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f09962e2640 1 -- 172.21.15.60:0/747605246 shutdown_connections 2024-04-25T08:36:35.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.905+0000 7f09962e2640 1 -- 172.21.15.60:0/747605246 wait complete. 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.906+0000 7f09962e2640 1 Processor -- start 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.906+0000 7f09962e2640 1 -- start start 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f09962e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f097c003f60 0x7f099014bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f09962e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09900cd9a0 0x7f0990159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f09962e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 0x7f0990157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f09962e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900b92a0 con 0x7f09901565c0 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f09962e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900d1a00 con 0x7f09900cd9a0 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f09962e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09900b9420 con 0x7f097c003f60 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 0x7f0990157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 0x7f0990157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54882/0 (socket says 172.21.15.60:54882) 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 -- 172.21.15.60:0/3123740343 learned_addr learned my addr 172.21.15.60:0/3123740343 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f098ffff640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f097c003f60 0x7f099014bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f098f7fe640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09900cd9a0 0x7f0990159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 -- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f097c003f60 msgr2=0x7f099014bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f097c003f60 0x7f099014bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 -- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09900cd9a0 msgr2=0x7f0990159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09900cd9a0 0x7f0990159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.907+0000 7f099487f640 1 -- 172.21.15.60:0/3123740343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990156da0 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.908+0000 7f099487f640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 0x7f0990157a60 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f09900b64a0 tx=0x7f098800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.908+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098801a070 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.908+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988006d60 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.908+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09900caf40 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.908+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988003b90 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.908+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0990156a50 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.909+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0990073990 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.909+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f098801d070 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.910+0000 7f098cff9640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f097006b500 0x7f097006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.910+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09880899d0 con 0x7f09901565c0 2024-04-25T08:36:35.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.910+0000 7f098ffff640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f097006b500 0x7f097006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:35.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.910+0000 7f098ffff640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f097006b500 0x7f097006d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f09900cd930 tx=0x7f0980009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:35.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:35.914+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0988058750 con 0x7f09901565c0 2024-04-25T08:36:36.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.095+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09900cf660 con 0x7f09901565c0 2024-04-25T08:36:36.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.097+0000 7f098cff9640 1 -- 172.21.15.60:0/3123740343 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f0988058050 con 0x7f09901565c0 2024-04-25T08:36:36.091 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:36.091 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":109,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:35.118366+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.098+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f097006b500 msgr2=0x7f097006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.098+0000 7f09962e2640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f097006b500 0x7f097006d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f09900cd930 tx=0x7f0980009040 comp rx=0 tx=0).stop 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.098+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 msgr2=0x7f0990157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.098+0000 7f09962e2640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 0x7f0990157a60 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f09900b64a0 tx=0x7f098800fd40 comp rx=0 tx=0).stop 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 shutdown_connections 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f09901565c0 0x7f0990157a60 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f097006b500 0x7f097006d9c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09900cd9a0 0x7f0990159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 --2- 172.21.15.60:0/3123740343 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f097c003f60 0x7f099014bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 >> 172.21.15.60:0/3123740343 conn(0x7f0990061a20 msgr2=0x7f09900b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 shutdown_connections 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.099+0000 7f09962e2640 1 -- 172.21.15.60:0/3123740343 wait complete. 2024-04-25T08:36:36.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:36.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.212+0000 7f34be577640 1 Processor -- start 2024-04-25T08:36:36.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.212+0000 7f34be577640 1 -- start start 2024-04-25T08:36:36.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.212+0000 7f34be577640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f34b80d1020 0x7f34b80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.212+0000 7f34be577640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80b97a0 con 0x7f34b815dbe0 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.212+0000 7f34be577640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80d1c50 con 0x7f34b80b2cb0 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.212+0000 7f34be577640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b815dfc0 con 0x7f34b80d1020 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34b7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f34b815dbe0 0x7f34b80d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35674/0 (socket says 172.21.15.60:35674) 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34b7fff640 1 -- 172.21.15.60:0/692935569 learned_addr learned my addr 172.21.15.60:0/692935569 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34bcb14640 1 --2- 172.21.15.60:0/692935569 >> v2:172.21.15.60:3300/0 conn(0x7f34b80d1020 0x7f34b80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34b6ffd640 1 -- 172.21.15.60:0/692935569 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584826017 0 0) 0x7f34b80b97a0 con 0x7f34b815dbe0 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34b6ffd640 1 -- 172.21.15.60:0/692935569 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a0003520 con 0x7f34b815dbe0 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34bcb14640 1 -- 172.21.15.60:0/692935569 >> v1:172.21.15.100:6789/0 conn(0x7f34b80b2cb0 legacy=0x7f34b80d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34bcb14640 1 -- 172.21.15.60:0/692935569 >> v1:172.21.15.60:6789/0 conn(0x7f34b815dbe0 legacy=0x7f34b80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.213+0000 7f34bcb14640 1 -- 172.21.15.60:0/692935569 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80d1f30 con 0x7f34b80d1020 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34bcb14640 1 --2- 172.21.15.60:0/692935569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b80d1020 0x7f34b80d2e40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009d90 tx=0x7f34b002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bab0a8c85ffd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34b6ffd640 1 -- 172.21.15.60:0/692935569 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007740 con 0x7f34b80d1020 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34b6ffd640 1 -- 172.21.15.60:0/692935569 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b00078e0 con 0x7f34b80d1020 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34b6ffd640 1 -- 172.21.15.60:0/692935569 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0007cb0 con 0x7f34b80d1020 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 -- 172.21.15.60:0/692935569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b80d1020 msgr2=0x7f34b80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 --2- 172.21.15.60:0/692935569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b80d1020 0x7f34b80d2e40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009d90 tx=0x7f34b002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 -- 172.21.15.60:0/692935569 shutdown_connections 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 --2- 172.21.15.60:0/692935569 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b80d1020 0x7f34b80d2e40 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 -- 172.21.15.60:0/692935569 >> 172.21.15.60:0/692935569 conn(0x7f34b8061a20 msgr2=0x7f34b8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 -- 172.21.15.60:0/692935569 shutdown_connections 2024-04-25T08:36:36.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.214+0000 7f34be577640 1 -- 172.21.15.60:0/692935569 wait complete. 2024-04-25T08:36:36.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 Processor -- start 2024-04-25T08:36:36.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 -- start start 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80b2cb0 0x7f34b80b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34b80d1020 0x7f34b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 0x7f34b80ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80b92a0 con 0x7f34b815dbe0 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80d1a00 con 0x7f34b80b2cb0 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.216+0000 7f34be577640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80b9420 con 0x7f34b80d1020 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 0x7f34b80ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 0x7f34b80ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54898/0 (socket says 172.21.15.60:54898) 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 -- 172.21.15.60:0/780466498 learned_addr learned my addr 172.21.15.60:0/780466498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34b7fff640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80b2cb0 0x7f34b80b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34b77fe640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34b80d1020 0x7f34b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 -- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34b80d1020 msgr2=0x7f34b80b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34b80d1020 0x7f34b80b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 -- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80b2cb0 msgr2=0x7f34b80b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80b2cb0 0x7f34b80b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 -- 172.21.15.60:0/780466498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80b57c0 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34bcb14640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 0x7f34b80ccb60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009eb0 tx=0x7f34b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0040070 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b0031bc0 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b80b5aa0 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0009450 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.217+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b80b6060 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.219+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b80cab00 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.219+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34b0039040 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.219+0000 7f34b4ff9640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a406b4d0 0x7f34a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.219+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34b00aba70 con 0x7f34b815dbe0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.220+0000 7f34b7fff640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a406b4d0 0x7f34a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.220+0000 7f34b7fff640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a406b4d0 0x7f34a406d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f34a80011e0 tx=0x7f34a8007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.224+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34b007a230 con 0x7f34b815dbe0 2024-04-25T08:36:36.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.406+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34b80b28b0 con 0x7f34b815dbe0 2024-04-25T08:36:36.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.408+0000 7f34b4ff9640 1 -- 172.21.15.60:0/780466498 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f34b007e2d0 con 0x7f34b815dbe0 2024-04-25T08:36:36.402 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:36.402 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":109,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:35.118366+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a406b4d0 msgr2=0x7f34a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a406b4d0 0x7f34a406d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f34a80011e0 tx=0x7f34a8007d30 comp rx=0 tx=0).stop 2024-04-25T08:36:36.404 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 msgr2=0x7f34b80ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 0x7f34b80ccb60 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009eb0 tx=0x7f34b0004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 shutdown_connections 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f34b815dbe0 0x7f34b80ccb60 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f34b80d1020 0x7f34b80b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f34a406b4d0 0x7f34a406d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 --2- 172.21.15.60:0/780466498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34b80b2cb0 0x7f34b80b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 >> 172.21.15.60:0/780466498 conn(0x7f34b8061a20 msgr2=0x7f34b815a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 shutdown_connections 2024-04-25T08:36:36.405 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.410+0000 7f34be577640 1 -- 172.21.15.60:0/780466498 wait complete. 2024-04-25T08:36:36.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:36:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.524+0000 7feb68e46640 1 Processor -- start 2024-04-25T08:36:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.524+0000 7feb68e46640 1 -- start start 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.525+0000 7feb68e46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161240 0x7feb640b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.525+0000 7feb68e46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb640b54a0 0x7feb640b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.525+0000 7feb68e46640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb640cf1d0 con 0x7feb64161890 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.525+0000 7feb68e46640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb640b7120 con 0x7feb640b54a0 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.525+0000 7feb68e46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb64161620 con 0x7feb64161240 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.525+0000 7feb62d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb640b54a0 0x7feb640b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb62d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feb640b54a0 0x7feb640b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54906/0 (socket says 172.21.15.60:54906) 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb62d9d640 1 -- 172.21.15.60:0/1141821563 learned_addr learned my addr 172.21.15.60:0/1141821563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb6259c640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161240 0x7feb640b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb62d9d640 1 -- 172.21.15.60:0/1141821563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161240 msgr2=0x7feb640b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb62d9d640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161240 0x7feb640b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb62d9d640 1 -- 172.21.15.60:0/1141821563 >> v1:172.21.15.100:6789/0 conn(0x7feb64161890 legacy=0x7feb640ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb62d9d640 1 -- 172.21.15.60:0/1141821563 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64161e00 con 0x7feb640b54a0 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.526+0000 7feb6259c640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161240 0x7feb640b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb62d9d640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb640b54a0 0x7feb640b5930 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb5402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269be4fbef3dcfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb6159a640 1 -- 172.21.15.60:0/1141821563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54036070 con 0x7feb640b54a0 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb6159a640 1 -- 172.21.15.60:0/1141821563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb54031600 con 0x7feb640b54a0 2024-04-25T08:36:36.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb6159a640 1 -- 172.21.15.60:0/1141821563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb540319d0 con 0x7feb640b54a0 2024-04-25T08:36:36.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 -- 172.21.15.60:0/1141821563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb640b54a0 msgr2=0x7feb640b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb640b54a0 0x7feb640b5930 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb5402a710 comp rx=0 tx=0).stop 2024-04-25T08:36:36.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 -- 172.21.15.60:0/1141821563 shutdown_connections 2024-04-25T08:36:36.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb640b54a0 0x7feb640b5930 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 --2- 172.21.15.60:0/1141821563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161240 0x7feb640b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 -- 172.21.15.60:0/1141821563 >> 172.21.15.60:0/1141821563 conn(0x7feb640d04a0 msgr2=0x7feb640d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 -- 172.21.15.60:0/1141821563 shutdown_connections 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.527+0000 7feb68e46640 1 -- 172.21.15.60:0/1141821563 wait complete. 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 Processor -- start 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 -- start start 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb640b54a0 0x7feb6415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 0x7feb6415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161890 0x7feb641f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb641626a0 con 0x7feb64161240 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.529+0000 7feb68e46640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb641623c0 con 0x7feb640b54a0 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb68e46640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb640d15f0 con 0x7feb64161890 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 0x7feb6415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 0x7feb6415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54914/0 (socket says 172.21.15.60:54914) 2024-04-25T08:36:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 -- 172.21.15.60:0/1526032326 learned_addr learned my addr 172.21.15.60:0/1526032326 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb6259c640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb640b54a0 0x7feb6415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb62d9d640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161890 0x7feb641f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 -- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161890 msgr2=0x7feb641f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161890 0x7feb641f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 -- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb640b54a0 msgr2=0x7feb6415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb640b54a0 0x7feb6415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 -- 172.21.15.60:0/1526032326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb641f7180 con 0x7feb64161240 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb61d9b640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 0x7feb6415a9f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c3c0 tx=0x7feb5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb580052d0 con 0x7feb64161240 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb58005490 con 0x7feb64161240 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58007260 con 0x7feb64161240 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb641f7460 con 0x7feb64161240 2024-04-25T08:36:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.530+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb641f7a20 con 0x7feb64161240 2024-04-25T08:36:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.531+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb64073a00 con 0x7feb64161240 2024-04-25T08:36:36.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.532+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb58007400 con 0x7feb64161240 2024-04-25T08:36:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.532+0000 7feb52ffd640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb4406b500 0x7feb4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.532+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb58089730 con 0x7feb64161240 2024-04-25T08:36:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.533+0000 7feb6259c640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb4406b500 0x7feb4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.533+0000 7feb6259c640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb4406b500 0x7feb4406d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feb6415b0c0 tx=0x7feb5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.537+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb58057dd0 con 0x7feb64161240 2024-04-25T08:36:36.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.754+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feb6415c6c0 con 0x7feb64161240 2024-04-25T08:36:36.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.762+0000 7feb52ffd640 1 -- 172.21.15.60:0/1526032326 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7feb5805be70 con 0x7feb64161240 2024-04-25T08:36:36.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb4406b500 msgr2=0x7feb4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb4406b500 0x7feb4406d9c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feb6415b0c0 tx=0x7feb5c008040 comp rx=0 tx=0).stop 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 msgr2=0x7feb6415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 0x7feb6415a9f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7feb5800c3c0 tx=0x7feb5800c880 comp rx=0 tx=0).stop 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 shutdown_connections 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feb64161890 0x7feb641f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb64161240 0x7feb6415a9f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feb4406b500 0x7feb4406d9c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 --2- 172.21.15.60:0/1526032326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb640b54a0 0x7feb6415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 >> 172.21.15.60:0/1526032326 conn(0x7feb640d04a0 msgr2=0x7feb640d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 shutdown_connections 2024-04-25T08:36:36.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.764+0000 7feb68e46640 1 -- 172.21.15.60:0/1526032326 wait complete. 2024-04-25T08:36:36.770 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:36:36.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:36:36.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.880+0000 7f5ecc95c640 1 Processor -- start 2024-04-25T08:36:36.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.880+0000 7f5ecc95c640 1 -- start start 2024-04-25T08:36:36.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ecc95c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815b7d0 0x7f5ec80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ecc95c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ec80b5620 0x7f5ec80b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ecc95c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ec806c2d0 0x7f5ec806c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ecc95c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80b5fc0 con 0x7f5ec80b5620 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ecc95c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec806cca0 con 0x7f5ec806c2d0 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ecc95c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec806cf50 con 0x7f5ec815b7d0 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ec80b5620 0x7f5ec80b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ec80b5620 0x7f5ec80b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54926/0 (socket says 172.21.15.60:54926) 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 -- 172.21.15.60:0/4171484164 learned_addr learned my addr 172.21.15.60:0/4171484164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ec659c640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815b7d0 0x7f5ec80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ec6d9d640 1 --2- 172.21.15.60:0/4171484164 >> v2:172.21.15.100:3300/0 conn(0x7f5ec806c2d0 0x7f5ec806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 -- 172.21.15.60:0/4171484164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815b7d0 msgr2=0x7f5ec80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815b7d0 0x7f5ec80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 -- 172.21.15.60:0/4171484164 >> v2:172.21.15.100:3300/0 conn(0x7f5ec806c2d0 msgr2=0x7f5ec806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 --2- 172.21.15.60:0/4171484164 >> v2:172.21.15.100:3300/0 conn(0x7f5ec806c2d0 0x7f5ec806c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ebffff640 1 -- 172.21.15.60:0/4171484164 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80ce530 con 0x7f5ec80b5620 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.881+0000 7f5ec659c640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815b7d0 0x7f5ec80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.882+0000 7f5ebffff640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b5a80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5ec815e340 tx=0x7f5eb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a50953237b73e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.882+0000 7f5ec5d9b640 1 -- 172.21.15.60:0/4171484164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037070 con 0x7f5ec80b5620 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.882+0000 7f5ec5d9b640 1 -- 172.21.15.60:0/4171484164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0032440 con 0x7f5ec80b5620 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.882+0000 7f5ec5d9b640 1 -- 172.21.15.60:0/4171484164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0032810 con 0x7f5ec80b5620 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 -- 172.21.15.60:0/4171484164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 msgr2=0x7f5ec80b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b5a80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f5ec815e340 tx=0x7f5eb002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 -- 172.21.15.60:0/4171484164 shutdown_connections 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/4171484164 >> v2:172.21.15.100:3300/0 conn(0x7f5ec806c2d0 0x7f5ec806c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b5a80 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/4171484164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec815b7d0 0x7f5ec80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 -- 172.21.15.60:0/4171484164 >> 172.21.15.60:0/4171484164 conn(0x7f5ec80d0800 msgr2=0x7f5ec80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 -- 172.21.15.60:0/4171484164 shutdown_connections 2024-04-25T08:36:36.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.883+0000 7f5ecc95c640 1 -- 172.21.15.60:0/4171484164 wait complete. 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.884+0000 7f5ecc95c640 1 Processor -- start 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.884+0000 7f5ecc95c640 1 -- start start 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec806c2d0 0x7f5ec80b8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec815b7d0 0x7f5ec80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec806d230 con 0x7f5ec80b5620 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80ceaf0 con 0x7f5ec815b7d0 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80cf550 con 0x7f5ec806c2d0 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54934/0 (socket says 172.21.15.60:54934) 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 -- 172.21.15.60:0/3085115803 learned_addr learned my addr 172.21.15.60:0/3085115803 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ec659c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec806c2d0 0x7f5ec80b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ec6d9d640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec815b7d0 0x7f5ec80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 -- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec806c2d0 msgr2=0x7f5ec80b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec806c2d0 0x7f5ec80b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 -- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec815b7d0 msgr2=0x7f5ec80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec815b7d0 0x7f5ec80cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 -- 172.21.15.60:0/3085115803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80c7640 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebffff640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b86c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0041020 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb00306c0 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec80c7920 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037040 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.885+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec80c7ee0 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.887+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8073a00 con 0x7f5ec80b5620 2024-04-25T08:36:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.887+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5eb0044020 con 0x7f5ec80b5620 2024-04-25T08:36:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.887+0000 7f5ebeffd640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ea806b4d0 0x7f5ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.887+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5eb00aab90 con 0x7f5ec80b5620 2024-04-25T08:36:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.888+0000 7f5ec659c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ea806b4d0 0x7f5ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.888+0000 7f5ec659c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ea806b4d0 0x7f5ea806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800a950 tx=0x7f5eb8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:36.892+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5eb004b030 con 0x7f5ec80b5620 2024-04-25T08:36:37.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.110+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5ec806b210 con 0x7f5ec80b5620 2024-04-25T08:36:37.730 INFO:tasks.ceph.mds.c.smithi060.stderr:2024-04-25T08:36:37.737+0000 7f9acd768640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:36:37.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.770+0000 7f5ebeffd640 1 -- 172.21.15.60:0/3085115803 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5eb0079880 con 0x7f5ec80b5620 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ea806b4d0 msgr2=0x7f5ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ea806b4d0 0x7f5ea806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800a950 tx=0x7f5eb8008040 comp rx=0 tx=0).stop 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 msgr2=0x7f5ec80b86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b86c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 shutdown_connections 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5ea806b4d0 0x7f5ea806d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ec815b7d0 0x7f5ec80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ec80b5620 0x7f5ec80b86c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 --2- 172.21.15.60:0/3085115803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ec806c2d0 0x7f5ec80b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 >> 172.21.15.60:0/3085115803 conn(0x7f5ec80d0800 msgr2=0x7f5ec80d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 shutdown_connections 2024-04-25T08:36:37.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.773+0000 7f5ecc95c640 1 -- 172.21.15.60:0/3085115803 wait complete. 2024-04-25T08:36:37.781 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:36:37.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:37.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.895+0000 7f03416c4640 1 Processor -- start 2024-04-25T08:36:37.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.895+0000 7f03416c4640 1 -- start start 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f03416c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f033c0d2e90 0x7f033c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f03416c4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0b9940 con 0x7f033c15dd30 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f03416c4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0d1df0 con 0x7f033c0d0900 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f03416c4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0d0d40 con 0x7f033c0d2e90 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f0333fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f033c0d2e90 0x7f033c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f033affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f033c15dd30 0x7f033c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35686/0 (socket says 172.21.15.60:35686) 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f033affd640 1 -- 172.21.15.60:0/3517033124 learned_addr learned my addr 172.21.15.60:0/3517033124 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f0333fff640 1 -- 172.21.15.60:0/3517033124 >> v1:172.21.15.100:6789/0 conn(0x7f033c0d0900 legacy=0x7f033c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f0333fff640 1 -- 172.21.15.60:0/3517033124 >> v1:172.21.15.60:6789/0 conn(0x7f033c15dd30 legacy=0x7f033c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f033a7fc640 1 -- 172.21.15.60:0/3517033124 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734744380 0 0) 0x7f033c0b9940 con 0x7f033c15dd30 2024-04-25T08:36:37.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.896+0000 7f0333fff640 1 -- 172.21.15.60:0/3517033124 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c0d20d0 con 0x7f033c0d2e90 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.897+0000 7f0333fff640 1 --2- 172.21.15.60:0/3517033124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c0d03c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0324009790 tx=0x7f032402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8671ce4719764af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.897+0000 7f033a7fc640 1 -- 172.21.15.60:0/3517033124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324037070 con 0x7f033c0d2e90 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.897+0000 7f033a7fc640 1 -- 172.21.15.60:0/3517033124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032402ae50 con 0x7f033c0d2e90 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.897+0000 7f033a7fc640 1 -- 172.21.15.60:0/3517033124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324032660 con 0x7f033c0d2e90 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 -- 172.21.15.60:0/3517033124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 msgr2=0x7f033c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 --2- 172.21.15.60:0/3517033124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c0d03c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0324009790 tx=0x7f032402a100 comp rx=0 tx=0).stop 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 -- 172.21.15.60:0/3517033124 shutdown_connections 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 --2- 172.21.15.60:0/3517033124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c0d03c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 -- 172.21.15.60:0/3517033124 >> 172.21.15.60:0/3517033124 conn(0x7f033c061c30 msgr2=0x7f033c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 -- 172.21.15.60:0/3517033124 shutdown_connections 2024-04-25T08:36:37.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.898+0000 7f03416c4640 1 -- 172.21.15.60:0/3517033124 wait complete. 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.899+0000 7f03416c4640 1 Processor -- start 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.899+0000 7f03416c4640 1 -- start start 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f03416c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033c0d0900 0x7f033c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f03416c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c1e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f03416c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033c15dd30 0x7f033c1e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f03416c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0d2970 con 0x7f033c0d2e90 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f03416c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0d2690 con 0x7f033c15dd30 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f03416c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0d1a10 con 0x7f033c0d0900 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c1e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c1e7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54952/0 (socket says 172.21.15.60:54952) 2024-04-25T08:36:37.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 -- 172.21.15.60:0/3784444891 learned_addr learned my addr 172.21.15.60:0/3784444891 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f033b7fe640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033c15dd30 0x7f033c1e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f033affd640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033c0d0900 0x7f033c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 -- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033c0d0900 msgr2=0x7f033c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033c0d0900 0x7f033c15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 -- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033c15dd30 msgr2=0x7f033c1e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033c15dd30 0x7f033c1e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f0333fff640 1 -- 172.21.15.60:0/3784444891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c1f35c0 con 0x7f033c0d2e90 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.900+0000 7f033affd640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033c0d0900 0x7f033c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.901+0000 7f0333fff640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c1e7ad0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0324035040 tx=0x7f03240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.901+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324042070 con 0x7f033c0d2e90 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.901+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0324004240 con 0x7f033c0d2e90 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.901+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c15bfc0 con 0x7f033c0d2e90 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.901+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324037040 con 0x7f033c0d2e90 2024-04-25T08:36:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.901+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c1e8b50 con 0x7f033c0d2e90 2024-04-25T08:36:37.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.902+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c0cc860 con 0x7f033c0d2e90 2024-04-25T08:36:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.902+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0324005990 con 0x7f033c0d2e90 2024-04-25T08:36:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.903+0000 7f03337fe640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031c06b500 0x7f031c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.903+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03240aabf0 con 0x7f033c0d2e90 2024-04-25T08:36:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.903+0000 7f033affd640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031c06b500 0x7f031c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.904+0000 7f033affd640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031c06b500 0x7f031c06d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f032c000f60 tx=0x7f032c007c50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:37.907+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f033c0cc860 con 0x7f033c0d2e90 2024-04-25T08:36:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.129+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f033c0ca220 con 0x7f033c0d2e90 2024-04-25T08:36:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.129+0000 7f03337fe640 1 -- 172.21.15.60:0/3784444891 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0324079970 con 0x7f033c0d2e90 2024-04-25T08:36:38.124 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:38.124 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:36:38.124 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031c06b500 msgr2=0x7f031c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031c06b500 0x7f031c06d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f032c000f60 tx=0x7f032c007c50 comp rx=0 tx=0).stop 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 msgr2=0x7f033c1e7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c1e7ad0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0324035040 tx=0x7f03240046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 shutdown_connections 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031c06b500 0x7f031c06d9c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f033c15dd30 0x7f033c1e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f033c0d2e90 0x7f033c1e7ad0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 --2- 172.21.15.60:0/3784444891 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f033c0d0900 0x7f033c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 >> 172.21.15.60:0/3784444891 conn(0x7f033c061c30 msgr2=0x7f033c0c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:38.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 shutdown_connections 2024-04-25T08:36:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.131+0000 7f03416c4640 1 -- 172.21.15.60:0/3784444891 wait complete. 2024-04-25T08:36:38.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:38.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.243+0000 7f9587cf7640 1 Processor -- start 2024-04-25T08:36:38.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.244+0000 7f9587cf7640 1 -- start start 2024-04-25T08:36:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.244+0000 7f9587cf7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95801602e0 0x7f95800ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.244+0000 7f9587cf7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f958015f5f0 0x7f958015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.244+0000 7f9587cf7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95800cd710 con 0x7f958015ba00 2024-04-25T08:36:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.244+0000 7f9587cf7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9580099b70 con 0x7f958015f5f0 2024-04-25T08:36:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.244+0000 7f9587cf7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9580099e50 con 0x7f95801602e0 2024-04-25T08:36:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f958015f5f0 0x7f958015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f958015f5f0 0x7f958015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54960/0 (socket says 172.21.15.60:54960) 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 -- 172.21.15.60:0/3602133548 learned_addr learned my addr 172.21.15.60:0/3602133548 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9585292640 1 --2- 172.21.15.60:0/3602133548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95801602e0 0x7f95800ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 -- 172.21.15.60:0/3602133548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95801602e0 msgr2=0x7f95800ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 --2- 172.21.15.60:0/3602133548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95801602e0 0x7f95800ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 -- 172.21.15.60:0/3602133548 >> v1:172.21.15.100:6789/0 conn(0x7f958015ba00 legacy=0x7f958009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.245+0000 7f9586294640 1 -- 172.21.15.60:0/3602133548 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958009a130 con 0x7f958015f5f0 2024-04-25T08:36:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9586294640 1 --2- 172.21.15.60:0/3602133548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f958015f5f0 0x7f958015faa0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab1fc7b2ba73fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9584a91640 1 -- 172.21.15.60:0/3602133548 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578036070 con 0x7f958015f5f0 2024-04-25T08:36:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9584a91640 1 -- 172.21.15.60:0/3602133548 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9578031600 con 0x7f958015f5f0 2024-04-25T08:36:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9584a91640 1 -- 172.21.15.60:0/3602133548 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780319d0 con 0x7f958015f5f0 2024-04-25T08:36:38.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 -- 172.21.15.60:0/3602133548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f958015f5f0 msgr2=0x7f958015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3602133548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f958015f5f0 0x7f958015faa0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 -- 172.21.15.60:0/3602133548 shutdown_connections 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3602133548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f958015f5f0 0x7f958015faa0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3602133548 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95801602e0 0x7f95800ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 -- 172.21.15.60:0/3602133548 >> 172.21.15.60:0/3602133548 conn(0x7f95800b3270 msgr2=0x7f95800b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 -- 172.21.15.60:0/3602133548 shutdown_connections 2024-04-25T08:36:38.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.246+0000 7f9587cf7640 1 -- 172.21.15.60:0/3602133548 wait complete. 2024-04-25T08:36:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 Processor -- start 2024-04-25T08:36:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 -- start start 2024-04-25T08:36:38.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f958015ba00 0x7f9580159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f958015f5f0 0x7f958015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 0x7f958015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800cd460 con 0x7f95801602e0 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800cd2e0 con 0x7f958015ba00 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9587cf7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95800c52d0 con 0x7f958015f5f0 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9586294640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 0x7f958015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9586294640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 0x7f958015c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54968/0 (socket says 172.21.15.60:54968) 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.248+0000 7f9586294640 1 -- 172.21.15.60:0/3992735224 learned_addr learned my addr 172.21.15.60:0/3992735224 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9585292640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f958015f5f0 0x7f958015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9585a93640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f958015ba00 0x7f9580159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9586294640 1 -- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f958015f5f0 msgr2=0x7f958015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9586294640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f958015f5f0 0x7f958015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9586294640 1 -- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f958015ba00 msgr2=0x7f9580159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9586294640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f958015ba00 0x7f9580159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9586294640 1 -- 172.21.15.60:0/3992735224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958015ca70 con 0x7f95801602e0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9586294640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 0x7f958015c530 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9578034040 tx=0x7f9578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578036050 con 0x7f95801602e0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9578042070 con 0x7f95801602e0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958015cd00 con 0x7f95801602e0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578007bf0 con 0x7f95801602e0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.249+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9580149dc0 con 0x7f95801602e0 2024-04-25T08:36:38.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.250+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9580073990 con 0x7f95801602e0 2024-04-25T08:36:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.251+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9578047460 con 0x7f95801602e0 2024-04-25T08:36:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.251+0000 7f95767fc640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.251+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95780ab1f0 con 0x7f95801602e0 2024-04-25T08:36:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.252+0000 7f9585a93640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.252+0000 7f9585a93640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f956406b4d0 0x7f956406d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f957c007fd0 tx=0x7f957c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.256+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9578045360 con 0x7f95801602e0 2024-04-25T08:36:38.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.438+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95800b7610 con 0x7f95801602e0 2024-04-25T08:36:38.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.439+0000 7f95767fc640 1 -- 172.21.15.60:0/3992735224 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f9578079f70 con 0x7f95801602e0 2024-04-25T08:36:38.434 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:38.434 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":110,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:36:36.750359+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f956406b4d0 msgr2=0x7f956406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f956406b4d0 0x7f956406d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f957c007fd0 tx=0x7f957c009040 comp rx=0 tx=0).stop 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 msgr2=0x7f958015c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 0x7f958015c530 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f9578034040 tx=0x7f9578004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 shutdown_connections 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801602e0 0x7f958015c530 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f958015f5f0 0x7f958015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 --2- 172.21.15.60:0/3992735224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f958015ba00 0x7f9580159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 >> 172.21.15.60:0/3992735224 conn(0x7f95800b3270 msgr2=0x7f958006fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 shutdown_connections 2024-04-25T08:36:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.441+0000 7f9587cf7640 1 -- 172.21.15.60:0/3992735224 wait complete. 2024-04-25T08:36:38.447 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:36:38.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.553+0000 7f5a676b6640 1 Processor -- start 2024-04-25T08:36:38.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.553+0000 7f5a676b6640 1 -- start start 2024-04-25T08:36:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a676b6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6015dbe0 0x7f5a600d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a676b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b92a0 0x7f5a600d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a676b6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a600b97a0 con 0x7f5a600b2cb0 2024-04-25T08:36:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a676b6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6015dfc0 con 0x7f5a6015dbe0 2024-04-25T08:36:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a676b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600d1a00 con 0x7f5a600b92a0 2024-04-25T08:36:38.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a64c51640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5a600b2cb0 0x7f5a600b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35700/0 (socket says 172.21.15.60:35700) 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a64c51640 1 -- 172.21.15.60:0/671260596 learned_addr learned my addr 172.21.15.60:0/671260596 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a65452640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5a6015dbe0 0x7f5a600d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.554+0000 7f5a65c53640 1 --2- 172.21.15.60:0/671260596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b92a0 0x7f5a600d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a5ffff640 1 -- 172.21.15.60:0/671260596 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252832972 0 0) 0x7f5a600b97a0 con 0x7f5a600b2cb0 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a5ffff640 1 -- 172.21.15.60:0/671260596 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a48003520 con 0x7f5a600b2cb0 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a65452640 1 -- 172.21.15.60:0/671260596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b92a0 msgr2=0x7f5a600d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a65452640 1 --2- 172.21.15.60:0/671260596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b92a0 0x7f5a600d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a65452640 1 -- 172.21.15.60:0/671260596 >> v1:172.21.15.60:6789/0 conn(0x7f5a600b2cb0 legacy=0x7f5a600b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a65452640 1 -- 172.21.15.60:0/671260596 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600d1c50 con 0x7f5a6015dbe0 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.555+0000 7f5a65452640 1 --2- 172.21.15.60:0/671260596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600d0910 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5a580097a0 tx=0x7f5a5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93157fb50492ebbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a5ffff640 1 -- 172.21.15.60:0/671260596 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58037070 con 0x7f5a6015dbe0 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a5ffff640 1 -- 172.21.15.60:0/671260596 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58032440 con 0x7f5a6015dbe0 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a5ffff640 1 -- 172.21.15.60:0/671260596 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58032810 con 0x7f5a6015dbe0 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 -- 172.21.15.60:0/671260596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 msgr2=0x7f5a600d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 --2- 172.21.15.60:0/671260596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600d0910 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5a580097a0 tx=0x7f5a5802a180 comp rx=0 tx=0).stop 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 -- 172.21.15.60:0/671260596 shutdown_connections 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 --2- 172.21.15.60:0/671260596 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b92a0 0x7f5a600d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 --2- 172.21.15.60:0/671260596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600d0910 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 -- 172.21.15.60:0/671260596 >> 172.21.15.60:0/671260596 conn(0x7f5a60061a20 msgr2=0x7f5a60061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 -- 172.21.15.60:0/671260596 shutdown_connections 2024-04-25T08:36:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.556+0000 7f5a676b6640 1 -- 172.21.15.60:0/671260596 wait complete. 2024-04-25T08:36:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.557+0000 7f5a676b6640 1 Processor -- start 2024-04-25T08:36:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.557+0000 7f5a676b6640 1 -- start start 2024-04-25T08:36:38.556 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a676b6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b2cb0 0x7f5a600b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a676b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a600b92a0 0x7f5a60151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a676b6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a676b6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600b6ab0 con 0x7f5a6015dbe0 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a676b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600d2210 con 0x7f5a600b92a0 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a676b6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600b6c30 con 0x7f5a600b2cb0 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a65c53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a65c53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54996/0 (socket says 172.21.15.60:54996) 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.558+0000 7f5a65c53640 1 -- 172.21.15.60:0/2971640845 learned_addr learned my addr 172.21.15.60:0/2971640845 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a64c51640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a600b92a0 0x7f5a60151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65c53640 1 -- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b2cb0 msgr2=0x7f5a600b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65452640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b2cb0 0x7f5a600b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65c53640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b2cb0 0x7f5a600b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65c53640 1 -- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a600b92a0 msgr2=0x7f5a60151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65c53640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a600b92a0 0x7f5a60151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65c53640 1 -- 172.21.15.60:0/2971640845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600b42f0 con 0x7f5a6015dbe0 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a64c51640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a600b92a0 0x7f5a60151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a65c53640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600cc6b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000efc0 tx=0x7f5a5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50018020 con 0x7f5a6015dbe0 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a50014480 con 0x7f5a6015dbe0 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50014870 con 0x7f5a6015dbe0 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a600b45d0 con 0x7f5a6015dbe0 2024-04-25T08:36:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.559+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a600b4b90 con 0x7f5a6015dbe0 2024-04-25T08:36:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.560+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a60073990 con 0x7f5a6015dbe0 2024-04-25T08:36:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.560+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a5001b020 con 0x7f5a6015dbe0 2024-04-25T08:36:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.561+0000 7f5a5dffb640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.561+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a5008b3b0 con 0x7f5a6015dbe0 2024-04-25T08:36:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.561+0000 7f5a65452640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.562+0000 7f5a65452640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5a58034000 tx=0x7f5a5802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.565+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a50022020 con 0x7f5a6015dbe0 2024-04-25T08:36:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.742+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a600b2860 con 0x7f5a4c06b4d0 2024-04-25T08:36:38.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.749+0000 7f5a5dffb640 1 -- 172.21.15.60:0/2971640845 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248853 (secure 0 0 0) 0x7f5a600b2860 con 0x7f5a4c06b4d0 2024-04-25T08:36:38.746 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:38.747 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:36:38.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 msgr2=0x7f5a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5a58034000 tx=0x7f5a5802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:38.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 msgr2=0x7f5a600cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600cc6b0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000efc0 tx=0x7f5a5000c490 comp rx=0 tx=0).stop 2024-04-25T08:36:38.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 shutdown_connections 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5a6015dbe0 0x7f5a600cc6b0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5a4c06b4d0 0x7f5a4c06d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a600b92a0 0x7f5a60151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 --2- 172.21.15.60:0/2971640845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5a600b2cb0 0x7f5a600b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 >> 172.21.15.60:0/2971640845 conn(0x7f5a60061a20 msgr2=0x7f5a600b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 shutdown_connections 2024-04-25T08:36:38.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.752+0000 7f5a676b6640 1 -- 172.21.15.60:0/2971640845 wait complete. 2024-04-25T08:36:38.758 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-25T08:36:37.497672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394360,"kb_used_data":11644,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754580360,"statfs":{"total":773094113280,"available":772690288640,"internally_reserved":0,"allocated":11923456,"data_stored":6149018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001159"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750939+0000","last_change":"2024-04-25T08:36:34.750939+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750939+0000","last_undegraded":"2024-04-25T08:36:34.750939+0000","last_fullsized":"2024-04-25T08:36:34.750939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766771+0000","last_change":"2024-04-25T08:36:35.765297+0000","last_active":"2024-04-25T08:36:35.766771+0000","last_peered":"2024-04-25T08:36:35.766771+0000","last_clean":"2024-04-25T08:36:35.766771+0000","last_became_active":"2024-04-25T08:36:35.763995+0000","last_became_peered":"2024-04-25T08:36:35.763995+0000","last_unstale":"2024-04-25T08:36:35.766771+0000","last_undegraded":"2024-04-25T08:36:35.766771+0000","last_fullsized":"2024-04-25T08:36:35.766771+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749594+0000","last_change":"2024-04-25T08:36:34.749594+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749594+0000","last_undegraded":"2024-04-25T08:36:34.749594+0000","last_fullsized":"2024-04-25T08:36:34.749594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752769+0000","last_change":"2024-04-25T08:36:34.752769+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752769+0000","last_undegraded":"2024-04-25T08:36:34.752769+0000","last_fullsized":"2024-04-25T08:36:34.752769+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749401+0000","last_change":"2024-04-25T08:36:34.749401+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749401+0000","last_undegraded":"2024-04-25T08:36:34.749401+0000","last_fullsized":"2024-04-25T08:36:34.749401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783055+0000","last_change":"2024-04-25T08:36:35.782370+0000","last_active":"2024-04-25T08:36:35.783055+0000","last_peered":"2024-04-25T08:36:35.783055+0000","last_clean":"2024-04-25T08:36:35.783055+0000","last_became_active":"2024-04-25T08:36:35.782129+0000","last_became_peered":"2024-04-25T08:36:35.782129+0000","last_unstale":"2024-04-25T08:36:35.783055+0000","last_undegraded":"2024-04-25T08:36:35.783055+0000","last_fullsized":"2024-04-25T08:36:35.783055+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772945+0000","last_change":"2024-04-25T08:36:35.771864+0000","last_active":"2024-04-25T08:36:35.772945+0000","last_peered":"2024-04-25T08:36:35.772945+0000","last_clean":"2024-04-25T08:36:35.772945+0000","last_became_active":"2024-04-25T08:36:35.771591+0000","last_became_peered":"2024-04-25T08:36:35.771591+0000","last_unstale":"2024-04-25T08:36:35.772945+0000","last_undegraded":"2024-04-25T08:36:35.772945+0000","last_fullsized":"2024-04-25T08:36:35.772945+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.756279+0000","last_change":"2024-04-25T08:36:34.756279+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.756279+0000","last_undegraded":"2024-04-25T08:36:34.756279+0000","last_fullsized":"2024-04-25T08:36:34.756279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770474+0000","last_change":"2024-04-25T08:36:35.768883+0000","last_active":"2024-04-25T08:36:35.770474+0000","last_peered":"2024-04-25T08:36:35.770474+0000","last_clean":"2024-04-25T08:36:35.770474+0000","last_became_active":"2024-04-25T08:36:35.768603+0000","last_became_peered":"2024-04-25T08:36:35.768603+0000","last_unstale":"2024-04-25T08:36:35.770474+0000","last_undegraded":"2024-04-25T08:36:35.770474+0000","last_fullsized":"2024-04-25T08:36:35.770474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750944+0000","last_change":"2024-04-25T08:36:34.750944+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750944+0000","last_undegraded":"2024-04-25T08:36:34.750944+0000","last_fullsized":"2024-04-25T08:36:34.750944+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774841+0000","last_change":"2024-04-25T08:36:35.772717+0000","last_active":"2024-04-25T08:36:35.774841+0000","last_peered":"2024-04-25T08:36:35.774841+0000","last_clean":"2024-04-25T08:36:35.774841+0000","last_became_active":"2024-04-25T08:36:35.772232+0000","last_became_peered":"2024-04-25T08:36:35.772232+0000","last_unstale":"2024-04-25T08:36:35.774841+0000","last_undegraded":"2024-04-25T08:36:35.774841+0000","last_fullsized":"2024-04-25T08:36:35.774841+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744825+0000","last_change":"2024-04-25T08:36:34.744825+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744825+0000","last_undegraded":"2024-04-25T08:36:34.744825+0000","last_fullsized":"2024-04-25T08:36:34.744825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749875+0000","last_change":"2024-04-25T08:36:34.749875+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749875+0000","last_undegraded":"2024-04-25T08:36:34.749875+0000","last_fullsized":"2024-04-25T08:36:34.749875+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768324+0000","last_change":"2024-04-25T08:36:35.767092+0000","last_active":"2024-04-25T08:36:35.768324+0000","last_peered":"2024-04-25T08:36:35.768324+0000","last_clean":"2024-04-25T08:36:35.768324+0000","last_became_active":"2024-04-25T08:36:35.766827+0000","last_became_peered":"2024-04-25T08:36:35.766827+0000","last_unstale":"2024-04-25T08:36:35.768324+0000","last_undegraded":"2024-04-25T08:36:35.768324+0000","last_fullsized":"2024-04-25T08:36:35.768324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.776596+0000","last_change":"2024-04-25T08:36:35.775011+0000","last_active":"2024-04-25T08:36:35.776596+0000","last_peered":"2024-04-25T08:36:35.776596+0000","last_clean":"2024-04-25T08:36:35.776596+0000","last_became_active":"2024-04-25T08:36:35.774695+0000","last_became_peered":"2024-04-25T08:36:35.774695+0000","last_unstale":"2024-04-25T08:36:35.776596+0000","last_undegraded":"2024-04-25T08:36:35.776596+0000","last_fullsized":"2024-04-25T08:36:35.776596+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.432682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768288+0000","last_change":"2024-04-25T08:36:35.766209+0000","last_active":"2024-04-25T08:36:35.768288+0000","last_peered":"2024-04-25T08:36:35.768288+0000","last_clean":"2024-04-25T08:36:35.768288+0000","last_became_active":"2024-04-25T08:36:35.765932+0000","last_became_peered":"2024-04-25T08:36:35.765932+0000","last_unstale":"2024-04-25T08:36:35.768288+0000","last_undegraded":"2024-04-25T08:36:35.768288+0000","last_fullsized":"2024-04-25T08:36:35.768288+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749979+0000","last_change":"2024-04-25T08:36:34.749979+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749979+0000","last_undegraded":"2024-04-25T08:36:34.749979+0000","last_fullsized":"2024-04-25T08:36:34.749979+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772836+0000","last_change":"2024-04-25T08:36:35.771768+0000","last_active":"2024-04-25T08:36:35.772836+0000","last_peered":"2024-04-25T08:36:35.772836+0000","last_clean":"2024-04-25T08:36:35.772836+0000","last_became_active":"2024-04-25T08:36:35.771487+0000","last_became_peered":"2024-04-25T08:36:35.771487+0000","last_unstale":"2024-04-25T08:36:35.772836+0000","last_undegraded":"2024-04-25T08:36:35.772836+0000","last_fullsized":"2024-04-25T08:36:35.772836+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748333+0000","last_change":"2024-04-25T08:36:34.748333+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748333+0000","last_undegraded":"2024-04-25T08:36:34.748333+0000","last_fullsized":"2024-04-25T08:36:34.748333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744421+0000","last_change":"2024-04-25T08:36:34.744421+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744421+0000","last_undegraded":"2024-04-25T08:36:34.744421+0000","last_fullsized":"2024-04-25T08:36:34.744421+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754852+0000","last_change":"2024-04-25T08:36:34.754852+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754852+0000","last_undegraded":"2024-04-25T08:36:34.754852+0000","last_fullsized":"2024-04-25T08:36:34.754852+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751587+0000","last_change":"2024-04-25T08:36:34.751587+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751587+0000","last_undegraded":"2024-04-25T08:36:34.751587+0000","last_fullsized":"2024-04-25T08:36:34.751587+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747496+0000","last_change":"2024-04-25T08:36:34.747496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747496+0000","last_undegraded":"2024-04-25T08:36:34.747496+0000","last_fullsized":"2024-04-25T08:36:34.747496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749242+0000","last_change":"2024-04-25T08:36:34.749242+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749242+0000","last_undegraded":"2024-04-25T08:36:34.749242+0000","last_fullsized":"2024-04-25T08:36:34.749242+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.746069+0000","last_change":"2024-04-25T08:36:34.746069+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.746069+0000","last_undegraded":"2024-04-25T08:36:34.746069+0000","last_fullsized":"2024-04-25T08:36:34.746069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.745098+0000","last_change":"2024-04-25T08:36:34.745098+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.745098+0000","last_undegraded":"2024-04-25T08:36:34.745098+0000","last_fullsized":"2024-04-25T08:36:34.745098+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774306+0000","last_change":"2024-04-25T08:36:35.768841+0000","last_active":"2024-04-25T08:36:35.774306+0000","last_peered":"2024-04-25T08:36:35.774306+0000","last_clean":"2024-04-25T08:36:35.774306+0000","last_became_active":"2024-04-25T08:36:35.768565+0000","last_became_peered":"2024-04-25T08:36:35.768565+0000","last_unstale":"2024-04-25T08:36:35.774306+0000","last_undegraded":"2024-04-25T08:36:35.774306+0000","last_fullsized":"2024-04-25T08:36:35.774306+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.769495+0000","last_change":"2024-04-25T08:36:35.764415+0000","last_active":"2024-04-25T08:36:35.769495+0000","last_peered":"2024-04-25T08:36:35.769495+0000","last_clean":"2024-04-25T08:36:35.769495+0000","last_became_active":"2024-04-25T08:36:35.764127+0000","last_became_peered":"2024-04-25T08:36:35.764127+0000","last_unstale":"2024-04-25T08:36:35.769495+0000","last_undegraded":"2024-04-25T08:36:35.769495+0000","last_fullsized":"2024-04-25T08:36:35.769495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754375+0000","last_change":"2024-04-25T08:36:34.754375+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754375+0000","last_undegraded":"2024-04-25T08:36:34.754375+0000","last_fullsized":"2024-04-25T08:36:34.754375+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744739+0000","last_change":"2024-04-25T08:36:34.744739+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744739+0000","last_undegraded":"2024-04-25T08:36:34.744739+0000","last_fullsized":"2024-04-25T08:36:34.744739+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770339+0000","last_change":"2024-04-25T08:36:35.767653+0000","last_active":"2024-04-25T08:36:35.770339+0000","last_peered":"2024-04-25T08:36:35.770339+0000","last_clean":"2024-04-25T08:36:35.770339+0000","last_became_active":"2024-04-25T08:36:35.767382+0000","last_became_peered":"2024-04-25T08:36:35.767382+0000","last_unstale":"2024-04-25T08:36:35.770339+0000","last_undegraded":"2024-04-25T08:36:35.770339+0000","last_fullsized":"2024-04-25T08:36:35.770339+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754189+0000","last_change":"2024-04-25T08:36:34.754189+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754189+0000","last_undegraded":"2024-04-25T08:36:34.754189+0000","last_fullsized":"2024-04-25T08:36:34.754189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748276+0000","last_change":"2024-04-25T08:36:34.748276+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748276+0000","last_undegraded":"2024-04-25T08:36:34.748276+0000","last_fullsized":"2024-04-25T08:36:34.748276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757104+0000","last_change":"2024-04-25T08:36:34.755252+0000","last_active":"2024-04-25T08:36:34.757104+0000","last_peered":"2024-04-25T08:36:34.757104+0000","last_clean":"2024-04-25T08:36:34.757104+0000","last_became_active":"2024-04-25T08:36:34.754950+0000","last_became_peered":"2024-04-25T08:36:34.754950+0000","last_unstale":"2024-04-25T08:36:34.757104+0000","last_undegraded":"2024-04-25T08:36:34.757104+0000","last_fullsized":"2024-04-25T08:36:34.757104+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750505+0000","last_change":"2024-04-25T08:36:34.750505+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750505+0000","last_undegraded":"2024-04-25T08:36:34.750505+0000","last_fullsized":"2024-04-25T08:36:34.750505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762949+0000","last_change":"2024-04-25T08:36:34.759886+0000","last_active":"2024-04-25T08:36:34.762949+0000","last_peered":"2024-04-25T08:36:34.762949+0000","last_clean":"2024-04-25T08:36:34.762949+0000","last_became_active":"2024-04-25T08:36:34.759596+0000","last_became_peered":"2024-04-25T08:36:34.759596+0000","last_unstale":"2024-04-25T08:36:34.762949+0000","last_undegraded":"2024-04-25T08:36:34.762949+0000","last_fullsized":"2024-04-25T08:36:34.762949+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:49.092358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744294+0000","last_change":"2024-04-25T08:36:34.744294+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744294+0000","last_undegraded":"2024-04-25T08:36:34.744294+0000","last_fullsized":"2024-04-25T08:36:34.744294+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760269+0000","last_change":"2024-04-25T08:36:34.759233+0000","last_active":"2024-04-25T08:36:34.760269+0000","last_peered":"2024-04-25T08:36:34.760269+0000","last_clean":"2024-04-25T08:36:34.760269+0000","last_became_active":"2024-04-25T08:36:34.758929+0000","last_became_peered":"2024-04-25T08:36:34.758929+0000","last_unstale":"2024-04-25T08:36:34.760269+0000","last_undegraded":"2024-04-25T08:36:34.760269+0000","last_fullsized":"2024-04-25T08:36:34.760269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.176620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774869+0000","last_change":"2024-04-25T08:36:35.773004+0000","last_active":"2024-04-25T08:36:35.774869+0000","last_peered":"2024-04-25T08:36:35.774869+0000","last_clean":"2024-04-25T08:36:35.774869+0000","last_became_active":"2024-04-25T08:36:35.772491+0000","last_became_peered":"2024-04-25T08:36:35.772491+0000","last_unstale":"2024-04-25T08:36:35.774869+0000","last_undegraded":"2024-04-25T08:36:35.774869+0000","last_fullsized":"2024-04-25T08:36:35.774869+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762413+0000","last_change":"2024-04-25T08:36:34.761740+0000","last_active":"2024-04-25T08:36:34.762413+0000","last_peered":"2024-04-25T08:36:34.762413+0000","last_clean":"2024-04-25T08:36:34.762413+0000","last_became_active":"2024-04-25T08:36:34.761387+0000","last_became_peered":"2024-04-25T08:36:34.761387+0000","last_unstale":"2024-04-25T08:36:34.762413+0000","last_undegraded":"2024-04-25T08:36:34.762413+0000","last_fullsized":"2024-04-25T08:36:34.762413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747266+0000","last_change":"2024-04-25T08:36:34.747266+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747266+0000","last_undegraded":"2024-04-25T08:36:34.747266+0000","last_fullsized":"2024-04-25T08:36:34.747266+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783225+0000","last_change":"2024-04-25T08:36:34.752807+0000","last_active":"2024-04-25T08:36:35.783225+0000","last_peered":"2024-04-25T08:36:35.783225+0000","last_clean":"2024-04-25T08:36:35.783225+0000","last_became_active":"2024-04-25T08:36:34.752501+0000","last_became_peered":"2024-04-25T08:36:34.752501+0000","last_unstale":"2024-04-25T08:36:35.783225+0000","last_undegraded":"2024-04-25T08:36:35.783225+0000","last_fullsized":"2024-04-25T08:36:35.783225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.320853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747253+0000","last_change":"2024-04-25T08:36:34.747253+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747253+0000","last_undegraded":"2024-04-25T08:36:34.747253+0000","last_fullsized":"2024-04-25T08:36:34.747253+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783449+0000","last_change":"2024-04-25T08:36:34.761095+0000","last_active":"2024-04-25T08:36:35.783449+0000","last_peered":"2024-04-25T08:36:35.783449+0000","last_clean":"2024-04-25T08:36:35.783449+0000","last_became_active":"2024-04-25T08:36:34.760735+0000","last_became_peered":"2024-04-25T08:36:34.760735+0000","last_unstale":"2024-04-25T08:36:35.783449+0000","last_undegraded":"2024-04-25T08:36:35.783449+0000","last_fullsized":"2024-04-25T08:36:35.783449+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766752+0000","last_change":"2024-04-25T08:36:35.764305+0000","last_active":"2024-04-25T08:36:35.766752+0000","last_peered":"2024-04-25T08:36:35.766752+0000","last_clean":"2024-04-25T08:36:35.766752+0000","last_became_active":"2024-04-25T08:36:35.764067+0000","last_became_peered":"2024-04-25T08:36:35.764067+0000","last_unstale":"2024-04-25T08:36:35.766752+0000","last_undegraded":"2024-04-25T08:36:35.766752+0000","last_fullsized":"2024-04-25T08:36:35.766752+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:20.920616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759815+0000","last_change":"2024-04-25T08:36:34.758831+0000","last_active":"2024-04-25T08:36:34.759815+0000","last_peered":"2024-04-25T08:36:34.759815+0000","last_clean":"2024-04-25T08:36:34.759815+0000","last_became_active":"2024-04-25T08:36:34.758382+0000","last_became_peered":"2024-04-25T08:36:34.758382+0000","last_unstale":"2024-04-25T08:36:34.759815+0000","last_undegraded":"2024-04-25T08:36:34.759815+0000","last_fullsized":"2024-04-25T08:36:34.759815+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:06.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747326+0000","last_change":"2024-04-25T08:36:34.747326+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747326+0000","last_undegraded":"2024-04-25T08:36:34.747326+0000","last_fullsized":"2024-04-25T08:36:34.747326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.761034+0000","last_change":"2024-04-25T08:36:34.758227+0000","last_active":"2024-04-25T08:36:34.761034+0000","last_peered":"2024-04-25T08:36:34.761034+0000","last_clean":"2024-04-25T08:36:34.761034+0000","last_became_active":"2024-04-25T08:36:34.757872+0000","last_became_peered":"2024-04-25T08:36:34.757872+0000","last_unstale":"2024-04-25T08:36:34.761034+0000","last_undegraded":"2024-04-25T08:36:34.761034+0000","last_fullsized":"2024-04-25T08:36:34.761034+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750496+0000","last_change":"2024-04-25T08:36:34.750496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750496+0000","last_undegraded":"2024-04-25T08:36:34.750496+0000","last_fullsized":"2024-04-25T08:36:34.750496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760454+0000","last_change":"2024-04-25T08:36:34.759067+0000","last_active":"2024-04-25T08:36:34.760454+0000","last_peered":"2024-04-25T08:36:34.760454+0000","last_clean":"2024-04-25T08:36:34.760454+0000","last_became_active":"2024-04-25T08:36:34.758669+0000","last_became_peered":"2024-04-25T08:36:34.758669+0000","last_unstale":"2024-04-25T08:36:34.760454+0000","last_undegraded":"2024-04-25T08:36:34.760454+0000","last_fullsized":"2024-04-25T08:36:34.760454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:06.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749803+0000","last_change":"2024-04-25T08:36:34.749803+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749803+0000","last_undegraded":"2024-04-25T08:36:34.749803+0000","last_fullsized":"2024-04-25T08:36:34.749803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760997+0000","last_change":"2024-04-25T08:36:34.759931+0000","last_active":"2024-04-25T08:36:34.760997+0000","last_peered":"2024-04-25T08:36:34.760997+0000","last_clean":"2024-04-25T08:36:34.760997+0000","last_became_active":"2024-04-25T08:36:34.759599+0000","last_became_peered":"2024-04-25T08:36:34.759599+0000","last_unstale":"2024-04-25T08:36:34.760997+0000","last_undegraded":"2024-04-25T08:36:34.760997+0000","last_fullsized":"2024-04-25T08:36:34.760997+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:33.870046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751089+0000","last_change":"2024-04-25T08:36:34.751089+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751089+0000","last_undegraded":"2024-04-25T08:36:34.751089+0000","last_fullsized":"2024-04-25T08:36:34.751089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783720+0000","last_change":"2024-04-25T08:36:34.753588+0000","last_active":"2024-04-25T08:36:35.783720+0000","last_peered":"2024-04-25T08:36:35.783720+0000","last_clean":"2024-04-25T08:36:35.783720+0000","last_became_active":"2024-04-25T08:36:34.753290+0000","last_became_peered":"2024-04-25T08:36:34.753290+0000","last_unstale":"2024-04-25T08:36:35.783720+0000","last_undegraded":"2024-04-25T08:36:35.783720+0000","last_fullsized":"2024-04-25T08:36:35.783720+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748432+0000","last_change":"2024-04-25T08:36:34.748432+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748432+0000","last_undegraded":"2024-04-25T08:36:34.748432+0000","last_fullsized":"2024-04-25T08:36:34.748432+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.786237+0000","last_change":"2024-04-25T08:36:34.750641+0000","last_active":"2024-04-25T08:36:35.786237+0000","last_peered":"2024-04-25T08:36:35.786237+0000","last_clean":"2024-04-25T08:36:35.786237+0000","last_became_active":"2024-04-25T08:36:34.748802+0000","last_became_peered":"2024-04-25T08:36:34.748802+0000","last_unstale":"2024-04-25T08:36:35.786237+0000","last_undegraded":"2024-04-25T08:36:35.786237+0000","last_fullsized":"2024-04-25T08:36:35.786237+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:39.476164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.773615+0000","last_change":"2024-04-25T08:36:35.768603+0000","last_active":"2024-04-25T08:36:35.773615+0000","last_peered":"2024-04-25T08:36:35.773615+0000","last_clean":"2024-04-25T08:36:35.773615+0000","last_became_active":"2024-04-25T08:36:35.766863+0000","last_became_peered":"2024-04-25T08:36:35.766863+0000","last_unstale":"2024-04-25T08:36:35.773615+0000","last_undegraded":"2024-04-25T08:36:35.773615+0000","last_fullsized":"2024-04-25T08:36:35.773615+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762382+0000","last_change":"2024-04-25T08:36:34.761126+0000","last_active":"2024-04-25T08:36:34.762382+0000","last_peered":"2024-04-25T08:36:34.762382+0000","last_clean":"2024-04-25T08:36:34.762382+0000","last_became_active":"2024-04-25T08:36:34.760848+0000","last_became_peered":"2024-04-25T08:36:34.760848+0000","last_unstale":"2024-04-25T08:36:34.762382+0000","last_undegraded":"2024-04-25T08:36:34.762382+0000","last_fullsized":"2024-04-25T08:36:34.762382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770663+0000","last_change":"2024-04-25T08:36:35.769783+0000","last_active":"2024-04-25T08:36:35.770663+0000","last_peered":"2024-04-25T08:36:35.770663+0000","last_clean":"2024-04-25T08:36:35.770663+0000","last_became_active":"2024-04-25T08:36:35.769101+0000","last_became_peered":"2024-04-25T08:36:35.769101+0000","last_unstale":"2024-04-25T08:36:35.770663+0000","last_undegraded":"2024-04-25T08:36:35.770663+0000","last_fullsized":"2024-04-25T08:36:35.770663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757823+0000","last_change":"2024-04-25T08:36:34.756585+0000","last_active":"2024-04-25T08:36:34.757823+0000","last_peered":"2024-04-25T08:36:34.757823+0000","last_clean":"2024-04-25T08:36:34.757823+0000","last_became_active":"2024-04-25T08:36:34.756301+0000","last_became_peered":"2024-04-25T08:36:34.756301+0000","last_unstale":"2024-04-25T08:36:34.757823+0000","last_undegraded":"2024-04-25T08:36:34.757823+0000","last_fullsized":"2024-04-25T08:36:34.757823+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763088+0000","last_change":"2024-04-25T08:36:34.761194+0000","last_active":"2024-04-25T08:36:34.763088+0000","last_peered":"2024-04-25T08:36:34.763088+0000","last_clean":"2024-04-25T08:36:34.763088+0000","last_became_active":"2024-04-25T08:36:34.760927+0000","last_became_peered":"2024-04-25T08:36:34.760927+0000","last_unstale":"2024-04-25T08:36:34.763088+0000","last_undegraded":"2024-04-25T08:36:34.763088+0000","last_fullsized":"2024-04-25T08:36:34.763088+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:51.400868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747795+0000","last_change":"2024-04-25T08:36:34.747795+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747795+0000","last_undegraded":"2024-04-25T08:36:34.747795+0000","last_fullsized":"2024-04-25T08:36:34.747795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760876+0000","last_change":"2024-04-25T08:36:34.759616+0000","last_active":"2024-04-25T08:36:34.760876+0000","last_peered":"2024-04-25T08:36:34.760876+0000","last_clean":"2024-04-25T08:36:34.760876+0000","last_became_active":"2024-04-25T08:36:34.759260+0000","last_became_peered":"2024-04-25T08:36:34.759260+0000","last_unstale":"2024-04-25T08:36:34.760876+0000","last_undegraded":"2024-04-25T08:36:34.760876+0000","last_fullsized":"2024-04-25T08:36:34.760876+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752657+0000","last_change":"2024-04-25T08:36:34.752657+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752657+0000","last_undegraded":"2024-04-25T08:36:34.752657+0000","last_fullsized":"2024-04-25T08:36:34.752657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.777741+0000","last_change":"2024-04-25T08:36:34.753294+0000","last_active":"2024-04-25T08:36:35.777741+0000","last_peered":"2024-04-25T08:36:35.777741+0000","last_clean":"2024-04-25T08:36:35.777741+0000","last_became_active":"2024-04-25T08:36:34.753016+0000","last_became_peered":"2024-04-25T08:36:34.753016+0000","last_unstale":"2024-04-25T08:36:35.777741+0000","last_undegraded":"2024-04-25T08:36:35.777741+0000","last_fullsized":"2024-04-25T08:36:35.777741+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770521+0000","last_change":"2024-04-25T08:36:35.767817+0000","last_active":"2024-04-25T08:36:35.770521+0000","last_peered":"2024-04-25T08:36:35.770521+0000","last_clean":"2024-04-25T08:36:35.770521+0000","last_became_active":"2024-04-25T08:36:35.767517+0000","last_became_peered":"2024-04-25T08:36:35.767517+0000","last_unstale":"2024-04-25T08:36:35.770521+0000","last_undegraded":"2024-04-25T08:36:35.770521+0000","last_fullsized":"2024-04-25T08:36:35.770521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.784098+0000","last_change":"2024-04-25T08:36:34.759604+0000","last_active":"2024-04-25T08:36:35.784098+0000","last_peered":"2024-04-25T08:36:35.784098+0000","last_clean":"2024-04-25T08:36:35.784098+0000","last_became_active":"2024-04-25T08:36:34.759303+0000","last_became_peered":"2024-04-25T08:36:34.759303+0000","last_unstale":"2024-04-25T08:36:35.784098+0000","last_undegraded":"2024-04-25T08:36:35.784098+0000","last_fullsized":"2024-04-25T08:36:35.784098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.258277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748937+0000","last_change":"2024-04-25T08:36:34.748937+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748937+0000","last_undegraded":"2024-04-25T08:36:34.748937+0000","last_fullsized":"2024-04-25T08:36:34.748937+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760131+0000","last_change":"2024-04-25T08:36:34.757404+0000","last_active":"2024-04-25T08:36:35.760131+0000","last_peered":"2024-04-25T08:36:35.760131+0000","last_clean":"2024-04-25T08:36:35.760131+0000","last_became_active":"2024-04-25T08:36:34.755586+0000","last_became_peered":"2024-04-25T08:36:34.755586+0000","last_unstale":"2024-04-25T08:36:35.760131+0000","last_undegraded":"2024-04-25T08:36:35.760131+0000","last_fullsized":"2024-04-25T08:36:35.760131+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747259+0000","last_change":"2024-04-25T08:36:34.747259+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747259+0000","last_undegraded":"2024-04-25T08:36:34.747259+0000","last_fullsized":"2024-04-25T08:36:34.747259+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.755095+0000","last_change":"2024-04-25T08:36:34.749438+0000","last_active":"2024-04-25T08:36:34.755095+0000","last_peered":"2024-04-25T08:36:34.755095+0000","last_clean":"2024-04-25T08:36:34.755095+0000","last_became_active":"2024-04-25T08:36:34.749073+0000","last_became_peered":"2024-04-25T08:36:34.749073+0000","last_unstale":"2024-04-25T08:36:34.755095+0000","last_undegraded":"2024-04-25T08:36:34.755095+0000","last_fullsized":"2024-04-25T08:36:34.755095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:25.045409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747760+0000","last_change":"2024-04-25T08:36:34.747760+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747760+0000","last_undegraded":"2024-04-25T08:36:34.747760+0000","last_fullsized":"2024-04-25T08:36:34.747760+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762766+0000","last_change":"2024-04-25T08:36:34.755127+0000","last_active":"2024-04-25T08:36:34.762766+0000","last_peered":"2024-04-25T08:36:34.762766+0000","last_clean":"2024-04-25T08:36:34.762766+0000","last_became_active":"2024-04-25T08:36:34.754781+0000","last_became_peered":"2024-04-25T08:36:34.754781+0000","last_unstale":"2024-04-25T08:36:34.762766+0000","last_undegraded":"2024-04-25T08:36:34.762766+0000","last_fullsized":"2024-04-25T08:36:34.762766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:40.479220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748664+0000","last_change":"2024-04-25T08:36:34.748664+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748664+0000","last_undegraded":"2024-04-25T08:36:34.748664+0000","last_fullsized":"2024-04-25T08:36:34.748664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753639+0000","last_change":"2024-04-25T08:36:34.756454+0000","last_active":"2024-04-25T08:36:35.753639+0000","last_peered":"2024-04-25T08:36:35.753639+0000","last_clean":"2024-04-25T08:36:35.753639+0000","last_became_active":"2024-04-25T08:36:34.755238+0000","last_became_peered":"2024-04-25T08:36:34.755238+0000","last_unstale":"2024-04-25T08:36:35.753639+0000","last_undegraded":"2024-04-25T08:36:35.753639+0000","last_fullsized":"2024-04-25T08:36:35.753639+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762995+0000","last_change":"2024-04-25T08:36:34.760255+0000","last_active":"2024-04-25T08:36:34.762995+0000","last_peered":"2024-04-25T08:36:34.762995+0000","last_clean":"2024-04-25T08:36:34.762995+0000","last_became_active":"2024-04-25T08:36:34.759634+0000","last_became_peered":"2024-04-25T08:36:34.759634+0000","last_unstale":"2024-04-25T08:36:34.762995+0000","last_undegraded":"2024-04-25T08:36:34.762995+0000","last_fullsized":"2024-04-25T08:36:34.762995+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:54.192240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751714+0000","last_change":"2024-04-25T08:36:34.751714+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751714+0000","last_undegraded":"2024-04-25T08:36:34.751714+0000","last_fullsized":"2024-04-25T08:36:34.751714+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762154+0000","last_change":"2024-04-25T08:36:34.760831+0000","last_active":"2024-04-25T08:36:34.762154+0000","last_peered":"2024-04-25T08:36:34.762154+0000","last_clean":"2024-04-25T08:36:34.762154+0000","last_became_active":"2024-04-25T08:36:34.760558+0000","last_became_peered":"2024-04-25T08:36:34.760558+0000","last_unstale":"2024-04-25T08:36:34.762154+0000","last_undegraded":"2024-04-25T08:36:34.762154+0000","last_fullsized":"2024-04-25T08:36:34.762154+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744146+0000","last_change":"2024-04-25T08:36:34.744146+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744146+0000","last_undegraded":"2024-04-25T08:36:34.744146+0000","last_fullsized":"2024-04-25T08:36:34.744146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.758450+0000","last_change":"2024-04-25T08:36:34.756998+0000","last_active":"2024-04-25T08:36:34.758450+0000","last_peered":"2024-04-25T08:36:34.758450+0000","last_clean":"2024-04-25T08:36:34.758450+0000","last_became_active":"2024-04-25T08:36:34.756057+0000","last_became_peered":"2024-04-25T08:36:34.756057+0000","last_unstale":"2024-04-25T08:36:34.758450+0000","last_undegraded":"2024-04-25T08:36:34.758450+0000","last_fullsized":"2024-04-25T08:36:34.758450+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:51.646595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749164+0000","last_change":"2024-04-25T08:36:34.749164+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749164+0000","last_undegraded":"2024-04-25T08:36:34.749164+0000","last_fullsized":"2024-04-25T08:36:34.749164+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.754613+0000","last_change":"2024-04-25T08:36:34.750357+0000","last_active":"2024-04-25T08:36:35.754613+0000","last_peered":"2024-04-25T08:36:35.754613+0000","last_clean":"2024-04-25T08:36:35.754613+0000","last_became_active":"2024-04-25T08:36:34.749684+0000","last_became_peered":"2024-04-25T08:36:34.749684+0000","last_unstale":"2024-04-25T08:36:35.754613+0000","last_undegraded":"2024-04-25T08:36:35.754613+0000","last_fullsized":"2024-04-25T08:36:35.754613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774914+0000","last_change":"2024-04-25T08:36:35.773260+0000","last_active":"2024-04-25T08:36:35.774914+0000","last_peered":"2024-04-25T08:36:35.774914+0000","last_clean":"2024-04-25T08:36:35.774914+0000","last_became_active":"2024-04-25T08:36:35.773012+0000","last_became_peered":"2024-04-25T08:36:35.773012+0000","last_unstale":"2024-04-25T08:36:35.774914+0000","last_undegraded":"2024-04-25T08:36:35.774914+0000","last_fullsized":"2024-04-25T08:36:35.774914+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757258+0000","last_change":"2024-04-25T08:36:34.756400+0000","last_active":"2024-04-25T08:36:34.757258+0000","last_peered":"2024-04-25T08:36:34.757258+0000","last_clean":"2024-04-25T08:36:34.757258+0000","last_became_active":"2024-04-25T08:36:34.756114+0000","last_became_peered":"2024-04-25T08:36:34.756114+0000","last_unstale":"2024-04-25T08:36:34.757258+0000","last_undegraded":"2024-04-25T08:36:34.757258+0000","last_fullsized":"2024-04-25T08:36:34.757258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752972+0000","last_change":"2024-04-25T08:36:34.752972+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752972+0000","last_undegraded":"2024-04-25T08:36:34.752972+0000","last_fullsized":"2024-04-25T08:36:34.752972+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759854+0000","last_change":"2024-04-25T08:36:34.757608+0000","last_active":"2024-04-25T08:36:34.759854+0000","last_peered":"2024-04-25T08:36:34.759854+0000","last_clean":"2024-04-25T08:36:34.759854+0000","last_became_active":"2024-04-25T08:36:34.756201+0000","last_became_peered":"2024-04-25T08:36:34.756201+0000","last_unstale":"2024-04-25T08:36:34.759854+0000","last_undegraded":"2024-04-25T08:36:34.759854+0000","last_fullsized":"2024-04-25T08:36:34.759854+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:09.132700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754953+0000","last_change":"2024-04-25T08:36:34.754953+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754953+0000","last_undegraded":"2024-04-25T08:36:34.754953+0000","last_fullsized":"2024-04-25T08:36:34.754953+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763179+0000","last_change":"2024-04-25T08:36:34.760614+0000","last_active":"2024-04-25T08:36:34.763179+0000","last_peered":"2024-04-25T08:36:34.763179+0000","last_clean":"2024-04-25T08:36:34.763179+0000","last_became_active":"2024-04-25T08:36:34.760357+0000","last_became_peered":"2024-04-25T08:36:34.760357+0000","last_unstale":"2024-04-25T08:36:34.763179+0000","last_undegraded":"2024-04-25T08:36:34.763179+0000","last_fullsized":"2024-04-25T08:36:34.763179+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:48.096031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768321+0000","last_change":"2024-04-25T08:36:35.766389+0000","last_active":"2024-04-25T08:36:35.768321+0000","last_peered":"2024-04-25T08:36:35.768321+0000","last_clean":"2024-04-25T08:36:35.768321+0000","last_became_active":"2024-04-25T08:36:35.766116+0000","last_became_peered":"2024-04-25T08:36:35.766116+0000","last_unstale":"2024-04-25T08:36:35.768321+0000","last_undegraded":"2024-04-25T08:36:35.768321+0000","last_fullsized":"2024-04-25T08:36:35.768321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753757+0000","last_change":"2024-04-25T08:36:34.754257+0000","last_active":"2024-04-25T08:36:35.753757+0000","last_peered":"2024-04-25T08:36:35.753757+0000","last_clean":"2024-04-25T08:36:35.753757+0000","last_became_active":"2024-04-25T08:36:34.752327+0000","last_became_peered":"2024-04-25T08:36:34.752327+0000","last_unstale":"2024-04-25T08:36:35.753757+0000","last_undegraded":"2024-04-25T08:36:35.753757+0000","last_fullsized":"2024-04-25T08:36:35.753757+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774411+0000","last_change":"2024-04-25T08:36:35.770356+0000","last_active":"2024-04-25T08:36:35.774411+0000","last_peered":"2024-04-25T08:36:35.774411+0000","last_clean":"2024-04-25T08:36:35.774411+0000","last_became_active":"2024-04-25T08:36:35.769027+0000","last_became_peered":"2024-04-25T08:36:35.769027+0000","last_unstale":"2024-04-25T08:36:35.774411+0000","last_undegraded":"2024-04-25T08:36:35.774411+0000","last_fullsized":"2024-04-25T08:36:35.774411+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763309+0000","last_change":"2024-04-25T08:36:34.762590+0000","last_active":"2024-04-25T08:36:34.763309+0000","last_peered":"2024-04-25T08:36:34.763309+0000","last_clean":"2024-04-25T08:36:34.763309+0000","last_became_active":"2024-04-25T08:36:34.762255+0000","last_became_peered":"2024-04-25T08:36:34.762255+0000","last_unstale":"2024-04-25T08:36:34.763309+0000","last_undegraded":"2024-04-25T08:36:34.763309+0000","last_fullsized":"2024-04-25T08:36:34.763309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:13.862606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774363+0000","last_change":"2024-04-25T08:36:35.769174+0000","last_active":"2024-04-25T08:36:35.774363+0000","last_peered":"2024-04-25T08:36:35.774363+0000","last_clean":"2024-04-25T08:36:35.774363+0000","last_became_active":"2024-04-25T08:36:35.768769+0000","last_became_peered":"2024-04-25T08:36:35.768769+0000","last_unstale":"2024-04-25T08:36:35.774363+0000","last_undegraded":"2024-04-25T08:36:35.774363+0000","last_fullsized":"2024-04-25T08:36:35.774363+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762841+0000","last_change":"2024-04-25T08:36:34.755845+0000","last_active":"2024-04-25T08:36:34.762841+0000","last_peered":"2024-04-25T08:36:34.762841+0000","last_clean":"2024-04-25T08:36:34.762841+0000","last_became_active":"2024-04-25T08:36:34.755540+0000","last_became_peered":"2024-04-25T08:36:34.755540+0000","last_unstale":"2024-04-25T08:36:34.762841+0000","last_undegraded":"2024-04-25T08:36:34.762841+0000","last_fullsized":"2024-04-25T08:36:34.762841+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:02.715530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760141+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:36:35.760141+0000","last_peered":"2024-04-25T08:36:35.760141+0000","last_clean":"2024-04-25T08:36:35.760141+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:36:35.760141+0000","last_undegraded":"2024-04-25T08:36:35.760141+0000","last_fullsized":"2024-04-25T08:36:35.760141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1949696,"data_stored":1216579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":1368064,"data_stored":628267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40824,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331016,"statfs":{"total":96636764160,"available":96594960384,"internally_reserved":0,"allocated":1347584,"data_stored":626051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54688,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317152,"statfs":{"total":96636764160,"available":96580763648,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54696,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317144,"statfs":{"total":96636764160,"available":96580755456,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1916928,"data_stored":1202585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:38.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.905+0000 7fb6c6e9a640 1 Processor -- start 2024-04-25T08:36:38.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.905+0000 7fb6c6e9a640 1 -- start start 2024-04-25T08:36:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c6e9a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6c015dd20 0x7fb6c00b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c6e9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e100 0x7fb6c00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c6e9a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6c015e560 0x7fb6c00d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c6e9a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c00d1c00 con 0x7fb6c015e560 2024-04-25T08:36:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c6e9a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c015eac0 con 0x7fb6c015dd20 2024-04-25T08:36:38.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c6e9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c015eda0 con 0x7fb6c015e100 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6c015e560 0x7fb6c00d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb6c015e560 0x7fb6c00d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55000/0 (socket says 172.21.15.60:55000) 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 -- 172.21.15.60:0/3235317388 learned_addr learned my addr 172.21.15.60:0/3235317388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6bffff640 1 --2- 172.21.15.60:0/3235317388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e100 0x7fb6c00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c4c36640 1 --2- 172.21.15.60:0/3235317388 >> v2:172.21.15.100:3300/0 conn(0x7fb6c015dd20 0x7fb6c00b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 -- 172.21.15.60:0/3235317388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e100 msgr2=0x7fb6c00d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 --2- 172.21.15.60:0/3235317388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e100 0x7fb6c00d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 -- 172.21.15.60:0/3235317388 >> v2:172.21.15.100:3300/0 conn(0x7fb6c015dd20 msgr2=0x7fb6c00b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 --2- 172.21.15.60:0/3235317388 >> v2:172.21.15.100:3300/0 conn(0x7fb6c015dd20 0x7fb6c00b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c5437640 1 -- 172.21.15.60:0/3235317388 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c015f080 con 0x7fb6c015e560 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.906+0000 7fb6c4c36640 1 --2- 172.21.15.60:0/3235317388 >> v2:172.21.15.100:3300/0 conn(0x7fb6c015dd20 0x7fb6c00b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.907+0000 7fb6c5437640 1 --2- 172.21.15.60:0/3235317388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015e560 0x7fb6c00d0fe0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00b6520 tx=0x7fb6b802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63c15326c3ac6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.907+0000 7fb6bf7fe640 1 -- 172.21.15.60:0/3235317388 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8036070 con 0x7fb6c015e560 2024-04-25T08:36:38.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.907+0000 7fb6bf7fe640 1 -- 172.21.15.60:0/3235317388 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b8031600 con 0x7fb6c015e560 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.907+0000 7fb6bf7fe640 1 -- 172.21.15.60:0/3235317388 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80319d0 con 0x7fb6c015e560 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/3235317388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015e560 msgr2=0x7fb6c00d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/3235317388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015e560 0x7fb6c00d0fe0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00b6520 tx=0x7fb6b802a610 comp rx=0 tx=0).stop 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/3235317388 shutdown_connections 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/3235317388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015e560 0x7fb6c00d0fe0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/3235317388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e100 0x7fb6c00d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/3235317388 >> v2:172.21.15.100:3300/0 conn(0x7fb6c015dd20 0x7fb6c00b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/3235317388 >> 172.21.15.60:0/3235317388 conn(0x7fb6c0061a20 msgr2=0x7fb6c0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/3235317388 shutdown_connections 2024-04-25T08:36:38.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.908+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/3235317388 wait complete. 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.909+0000 7fb6c6e9a640 1 Processor -- start 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.909+0000 7fb6c6e9a640 1 -- start start 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c6e9a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 0x7fb6c00c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c6e9a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6c015e100 0x7fb6c00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c6e9a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e560 0x7fb6c00ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c6e9a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00d3c20 con 0x7fb6c015dd20 2024-04-25T08:36:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c6e9a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00d3aa0 con 0x7fb6c015e100 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c6e9a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00d3da0 con 0x7fb6c015e560 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 0x7fb6c00c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 0x7fb6c00c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55012/0 (socket says 172.21.15.60:55012) 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 -- 172.21.15.60:0/2830578342 learned_addr learned my addr 172.21.15.60:0/2830578342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6bffff640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6c015e100 0x7fb6c00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c5437640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e560 0x7fb6c00ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 -- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e560 msgr2=0x7fb6c00ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e560 0x7fb6c00ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 -- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6c015e100 msgr2=0x7fb6c00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6c015e100 0x7fb6c00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 -- 172.21.15.60:0/2830578342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00c9ce0 con 0x7fb6c015dd20 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c5437640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e560 0x7fb6c00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.910+0000 7fb6c4c36640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 0x7fb6c00c97a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c7c0 tx=0x7fb6b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.911+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4015070 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.911+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4007a30 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.911+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c00b1330 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.911+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4007e00 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.911+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c00b18f0 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.912+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c0073990 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.912+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6b4018070 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.913+0000 7fb6bd7fa640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb6a406b4d0 0x7fb6a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.913+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6b408a280 con 0x7fb6c015dd20 2024-04-25T08:36:38.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.913+0000 7fb6bffff640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb6a406b4d0 0x7fb6a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:38.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.913+0000 7fb6bffff640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb6a406b4d0 0x7fb6a406d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb6c01f6a80 tx=0x7fb6ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:38.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:38.917+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6b4058f70 con 0x7fb6c015dd20 2024-04-25T08:36:39.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.095+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6c006b210 con 0x7fb6a406b4d0 2024-04-25T08:36:39.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.101+0000 7fb6bd7fa640 1 -- 172.21.15.60:0/2830578342 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248853 (secure 0 0 0) 0x7fb6c006b210 con 0x7fb6a406b4d0 2024-04-25T08:36:39.098 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:39.099 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb6a406b4d0 msgr2=0x7fb6a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb6a406b4d0 0x7fb6a406d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb6c01f6a80 tx=0x7fb6ac008040 comp rx=0 tx=0).stop 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 msgr2=0x7fb6c00c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 0x7fb6c00c97a0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c7c0 tx=0x7fb6b400cc80 comp rx=0 tx=0).stop 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 shutdown_connections 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6c015e560 0x7fb6c00ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb6a406b4d0 0x7fb6a406d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6c015e100 0x7fb6c00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 --2- 172.21.15.60:0/2830578342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6c015dd20 0x7fb6c00c97a0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 >> 172.21.15.60:0/2830578342 conn(0x7fb6c0061a20 msgr2=0x7fb6c0160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 shutdown_connections 2024-04-25T08:36:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.104+0000 7fb6c6e9a640 1 -- 172.21.15.60:0/2830578342 wait complete. 2024-04-25T08:36:39.109 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-25T08:36:37.497672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394360,"kb_used_data":11644,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754580360,"statfs":{"total":773094113280,"available":772690288640,"internally_reserved":0,"allocated":11923456,"data_stored":6149018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001159"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750939+0000","last_change":"2024-04-25T08:36:34.750939+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750939+0000","last_undegraded":"2024-04-25T08:36:34.750939+0000","last_fullsized":"2024-04-25T08:36:34.750939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766771+0000","last_change":"2024-04-25T08:36:35.765297+0000","last_active":"2024-04-25T08:36:35.766771+0000","last_peered":"2024-04-25T08:36:35.766771+0000","last_clean":"2024-04-25T08:36:35.766771+0000","last_became_active":"2024-04-25T08:36:35.763995+0000","last_became_peered":"2024-04-25T08:36:35.763995+0000","last_unstale":"2024-04-25T08:36:35.766771+0000","last_undegraded":"2024-04-25T08:36:35.766771+0000","last_fullsized":"2024-04-25T08:36:35.766771+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749594+0000","last_change":"2024-04-25T08:36:34.749594+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749594+0000","last_undegraded":"2024-04-25T08:36:34.749594+0000","last_fullsized":"2024-04-25T08:36:34.749594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752769+0000","last_change":"2024-04-25T08:36:34.752769+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752769+0000","last_undegraded":"2024-04-25T08:36:34.752769+0000","last_fullsized":"2024-04-25T08:36:34.752769+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749401+0000","last_change":"2024-04-25T08:36:34.749401+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749401+0000","last_undegraded":"2024-04-25T08:36:34.749401+0000","last_fullsized":"2024-04-25T08:36:34.749401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783055+0000","last_change":"2024-04-25T08:36:35.782370+0000","last_active":"2024-04-25T08:36:35.783055+0000","last_peered":"2024-04-25T08:36:35.783055+0000","last_clean":"2024-04-25T08:36:35.783055+0000","last_became_active":"2024-04-25T08:36:35.782129+0000","last_became_peered":"2024-04-25T08:36:35.782129+0000","last_unstale":"2024-04-25T08:36:35.783055+0000","last_undegraded":"2024-04-25T08:36:35.783055+0000","last_fullsized":"2024-04-25T08:36:35.783055+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772945+0000","last_change":"2024-04-25T08:36:35.771864+0000","last_active":"2024-04-25T08:36:35.772945+0000","last_peered":"2024-04-25T08:36:35.772945+0000","last_clean":"2024-04-25T08:36:35.772945+0000","last_became_active":"2024-04-25T08:36:35.771591+0000","last_became_peered":"2024-04-25T08:36:35.771591+0000","last_unstale":"2024-04-25T08:36:35.772945+0000","last_undegraded":"2024-04-25T08:36:35.772945+0000","last_fullsized":"2024-04-25T08:36:35.772945+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.756279+0000","last_change":"2024-04-25T08:36:34.756279+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.756279+0000","last_undegraded":"2024-04-25T08:36:34.756279+0000","last_fullsized":"2024-04-25T08:36:34.756279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770474+0000","last_change":"2024-04-25T08:36:35.768883+0000","last_active":"2024-04-25T08:36:35.770474+0000","last_peered":"2024-04-25T08:36:35.770474+0000","last_clean":"2024-04-25T08:36:35.770474+0000","last_became_active":"2024-04-25T08:36:35.768603+0000","last_became_peered":"2024-04-25T08:36:35.768603+0000","last_unstale":"2024-04-25T08:36:35.770474+0000","last_undegraded":"2024-04-25T08:36:35.770474+0000","last_fullsized":"2024-04-25T08:36:35.770474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750944+0000","last_change":"2024-04-25T08:36:34.750944+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750944+0000","last_undegraded":"2024-04-25T08:36:34.750944+0000","last_fullsized":"2024-04-25T08:36:34.750944+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774841+0000","last_change":"2024-04-25T08:36:35.772717+0000","last_active":"2024-04-25T08:36:35.774841+0000","last_peered":"2024-04-25T08:36:35.774841+0000","last_clean":"2024-04-25T08:36:35.774841+0000","last_became_active":"2024-04-25T08:36:35.772232+0000","last_became_peered":"2024-04-25T08:36:35.772232+0000","last_unstale":"2024-04-25T08:36:35.774841+0000","last_undegraded":"2024-04-25T08:36:35.774841+0000","last_fullsized":"2024-04-25T08:36:35.774841+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744825+0000","last_change":"2024-04-25T08:36:34.744825+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744825+0000","last_undegraded":"2024-04-25T08:36:34.744825+0000","last_fullsized":"2024-04-25T08:36:34.744825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749875+0000","last_change":"2024-04-25T08:36:34.749875+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749875+0000","last_undegraded":"2024-04-25T08:36:34.749875+0000","last_fullsized":"2024-04-25T08:36:34.749875+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768324+0000","last_change":"2024-04-25T08:36:35.767092+0000","last_active":"2024-04-25T08:36:35.768324+0000","last_peered":"2024-04-25T08:36:35.768324+0000","last_clean":"2024-04-25T08:36:35.768324+0000","last_became_active":"2024-04-25T08:36:35.766827+0000","last_became_peered":"2024-04-25T08:36:35.766827+0000","last_unstale":"2024-04-25T08:36:35.768324+0000","last_undegraded":"2024-04-25T08:36:35.768324+0000","last_fullsized":"2024-04-25T08:36:35.768324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.776596+0000","last_change":"2024-04-25T08:36:35.775011+0000","last_active":"2024-04-25T08:36:35.776596+0000","last_peered":"2024-04-25T08:36:35.776596+0000","last_clean":"2024-04-25T08:36:35.776596+0000","last_became_active":"2024-04-25T08:36:35.774695+0000","last_became_peered":"2024-04-25T08:36:35.774695+0000","last_unstale":"2024-04-25T08:36:35.776596+0000","last_undegraded":"2024-04-25T08:36:35.776596+0000","last_fullsized":"2024-04-25T08:36:35.776596+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.432682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768288+0000","last_change":"2024-04-25T08:36:35.766209+0000","last_active":"2024-04-25T08:36:35.768288+0000","last_peered":"2024-04-25T08:36:35.768288+0000","last_clean":"2024-04-25T08:36:35.768288+0000","last_became_active":"2024-04-25T08:36:35.765932+0000","last_became_peered":"2024-04-25T08:36:35.765932+0000","last_unstale":"2024-04-25T08:36:35.768288+0000","last_undegraded":"2024-04-25T08:36:35.768288+0000","last_fullsized":"2024-04-25T08:36:35.768288+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749979+0000","last_change":"2024-04-25T08:36:34.749979+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749979+0000","last_undegraded":"2024-04-25T08:36:34.749979+0000","last_fullsized":"2024-04-25T08:36:34.749979+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772836+0000","last_change":"2024-04-25T08:36:35.771768+0000","last_active":"2024-04-25T08:36:35.772836+0000","last_peered":"2024-04-25T08:36:35.772836+0000","last_clean":"2024-04-25T08:36:35.772836+0000","last_became_active":"2024-04-25T08:36:35.771487+0000","last_became_peered":"2024-04-25T08:36:35.771487+0000","last_unstale":"2024-04-25T08:36:35.772836+0000","last_undegraded":"2024-04-25T08:36:35.772836+0000","last_fullsized":"2024-04-25T08:36:35.772836+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748333+0000","last_change":"2024-04-25T08:36:34.748333+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748333+0000","last_undegraded":"2024-04-25T08:36:34.748333+0000","last_fullsized":"2024-04-25T08:36:34.748333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744421+0000","last_change":"2024-04-25T08:36:34.744421+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744421+0000","last_undegraded":"2024-04-25T08:36:34.744421+0000","last_fullsized":"2024-04-25T08:36:34.744421+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754852+0000","last_change":"2024-04-25T08:36:34.754852+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754852+0000","last_undegraded":"2024-04-25T08:36:34.754852+0000","last_fullsized":"2024-04-25T08:36:34.754852+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751587+0000","last_change":"2024-04-25T08:36:34.751587+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751587+0000","last_undegraded":"2024-04-25T08:36:34.751587+0000","last_fullsized":"2024-04-25T08:36:34.751587+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747496+0000","last_change":"2024-04-25T08:36:34.747496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747496+0000","last_undegraded":"2024-04-25T08:36:34.747496+0000","last_fullsized":"2024-04-25T08:36:34.747496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749242+0000","last_change":"2024-04-25T08:36:34.749242+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749242+0000","last_undegraded":"2024-04-25T08:36:34.749242+0000","last_fullsized":"2024-04-25T08:36:34.749242+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.746069+0000","last_change":"2024-04-25T08:36:34.746069+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.746069+0000","last_undegraded":"2024-04-25T08:36:34.746069+0000","last_fullsized":"2024-04-25T08:36:34.746069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.745098+0000","last_change":"2024-04-25T08:36:34.745098+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.745098+0000","last_undegraded":"2024-04-25T08:36:34.745098+0000","last_fullsized":"2024-04-25T08:36:34.745098+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774306+0000","last_change":"2024-04-25T08:36:35.768841+0000","last_active":"2024-04-25T08:36:35.774306+0000","last_peered":"2024-04-25T08:36:35.774306+0000","last_clean":"2024-04-25T08:36:35.774306+0000","last_became_active":"2024-04-25T08:36:35.768565+0000","last_became_peered":"2024-04-25T08:36:35.768565+0000","last_unstale":"2024-04-25T08:36:35.774306+0000","last_undegraded":"2024-04-25T08:36:35.774306+0000","last_fullsized":"2024-04-25T08:36:35.774306+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.769495+0000","last_change":"2024-04-25T08:36:35.764415+0000","last_active":"2024-04-25T08:36:35.769495+0000","last_peered":"2024-04-25T08:36:35.769495+0000","last_clean":"2024-04-25T08:36:35.769495+0000","last_became_active":"2024-04-25T08:36:35.764127+0000","last_became_peered":"2024-04-25T08:36:35.764127+0000","last_unstale":"2024-04-25T08:36:35.769495+0000","last_undegraded":"2024-04-25T08:36:35.769495+0000","last_fullsized":"2024-04-25T08:36:35.769495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754375+0000","last_change":"2024-04-25T08:36:34.754375+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754375+0000","last_undegraded":"2024-04-25T08:36:34.754375+0000","last_fullsized":"2024-04-25T08:36:34.754375+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744739+0000","last_change":"2024-04-25T08:36:34.744739+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744739+0000","last_undegraded":"2024-04-25T08:36:34.744739+0000","last_fullsized":"2024-04-25T08:36:34.744739+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770339+0000","last_change":"2024-04-25T08:36:35.767653+0000","last_active":"2024-04-25T08:36:35.770339+0000","last_peered":"2024-04-25T08:36:35.770339+0000","last_clean":"2024-04-25T08:36:35.770339+0000","last_became_active":"2024-04-25T08:36:35.767382+0000","last_became_peered":"2024-04-25T08:36:35.767382+0000","last_unstale":"2024-04-25T08:36:35.770339+0000","last_undegraded":"2024-04-25T08:36:35.770339+0000","last_fullsized":"2024-04-25T08:36:35.770339+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754189+0000","last_change":"2024-04-25T08:36:34.754189+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754189+0000","last_undegraded":"2024-04-25T08:36:34.754189+0000","last_fullsized":"2024-04-25T08:36:34.754189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748276+0000","last_change":"2024-04-25T08:36:34.748276+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748276+0000","last_undegraded":"2024-04-25T08:36:34.748276+0000","last_fullsized":"2024-04-25T08:36:34.748276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757104+0000","last_change":"2024-04-25T08:36:34.755252+0000","last_active":"2024-04-25T08:36:34.757104+0000","last_peered":"2024-04-25T08:36:34.757104+0000","last_clean":"2024-04-25T08:36:34.757104+0000","last_became_active":"2024-04-25T08:36:34.754950+0000","last_became_peered":"2024-04-25T08:36:34.754950+0000","last_unstale":"2024-04-25T08:36:34.757104+0000","last_undegraded":"2024-04-25T08:36:34.757104+0000","last_fullsized":"2024-04-25T08:36:34.757104+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750505+0000","last_change":"2024-04-25T08:36:34.750505+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750505+0000","last_undegraded":"2024-04-25T08:36:34.750505+0000","last_fullsized":"2024-04-25T08:36:34.750505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762949+0000","last_change":"2024-04-25T08:36:34.759886+0000","last_active":"2024-04-25T08:36:34.762949+0000","last_peered":"2024-04-25T08:36:34.762949+0000","last_clean":"2024-04-25T08:36:34.762949+0000","last_became_active":"2024-04-25T08:36:34.759596+0000","last_became_peered":"2024-04-25T08:36:34.759596+0000","last_unstale":"2024-04-25T08:36:34.762949+0000","last_undegraded":"2024-04-25T08:36:34.762949+0000","last_fullsized":"2024-04-25T08:36:34.762949+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:49.092358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744294+0000","last_change":"2024-04-25T08:36:34.744294+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744294+0000","last_undegraded":"2024-04-25T08:36:34.744294+0000","last_fullsized":"2024-04-25T08:36:34.744294+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760269+0000","last_change":"2024-04-25T08:36:34.759233+0000","last_active":"2024-04-25T08:36:34.760269+0000","last_peered":"2024-04-25T08:36:34.760269+0000","last_clean":"2024-04-25T08:36:34.760269+0000","last_became_active":"2024-04-25T08:36:34.758929+0000","last_became_peered":"2024-04-25T08:36:34.758929+0000","last_unstale":"2024-04-25T08:36:34.760269+0000","last_undegraded":"2024-04-25T08:36:34.760269+0000","last_fullsized":"2024-04-25T08:36:34.760269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.176620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774869+0000","last_change":"2024-04-25T08:36:35.773004+0000","last_active":"2024-04-25T08:36:35.774869+0000","last_peered":"2024-04-25T08:36:35.774869+0000","last_clean":"2024-04-25T08:36:35.774869+0000","last_became_active":"2024-04-25T08:36:35.772491+0000","last_became_peered":"2024-04-25T08:36:35.772491+0000","last_unstale":"2024-04-25T08:36:35.774869+0000","last_undegraded":"2024-04-25T08:36:35.774869+0000","last_fullsized":"2024-04-25T08:36:35.774869+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762413+0000","last_change":"2024-04-25T08:36:34.761740+0000","last_active":"2024-04-25T08:36:34.762413+0000","last_peered":"2024-04-25T08:36:34.762413+0000","last_clean":"2024-04-25T08:36:34.762413+0000","last_became_active":"2024-04-25T08:36:34.761387+0000","last_became_peered":"2024-04-25T08:36:34.761387+0000","last_unstale":"2024-04-25T08:36:34.762413+0000","last_undegraded":"2024-04-25T08:36:34.762413+0000","last_fullsized":"2024-04-25T08:36:34.762413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747266+0000","last_change":"2024-04-25T08:36:34.747266+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747266+0000","last_undegraded":"2024-04-25T08:36:34.747266+0000","last_fullsized":"2024-04-25T08:36:34.747266+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783225+0000","last_change":"2024-04-25T08:36:34.752807+0000","last_active":"2024-04-25T08:36:35.783225+0000","last_peered":"2024-04-25T08:36:35.783225+0000","last_clean":"2024-04-25T08:36:35.783225+0000","last_became_active":"2024-04-25T08:36:34.752501+0000","last_became_peered":"2024-04-25T08:36:34.752501+0000","last_unstale":"2024-04-25T08:36:35.783225+0000","last_undegraded":"2024-04-25T08:36:35.783225+0000","last_fullsized":"2024-04-25T08:36:35.783225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.320853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747253+0000","last_change":"2024-04-25T08:36:34.747253+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747253+0000","last_undegraded":"2024-04-25T08:36:34.747253+0000","last_fullsized":"2024-04-25T08:36:34.747253+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783449+0000","last_change":"2024-04-25T08:36:34.761095+0000","last_active":"2024-04-25T08:36:35.783449+0000","last_peered":"2024-04-25T08:36:35.783449+0000","last_clean":"2024-04-25T08:36:35.783449+0000","last_became_active":"2024-04-25T08:36:34.760735+0000","last_became_peered":"2024-04-25T08:36:34.760735+0000","last_unstale":"2024-04-25T08:36:35.783449+0000","last_undegraded":"2024-04-25T08:36:35.783449+0000","last_fullsized":"2024-04-25T08:36:35.783449+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766752+0000","last_change":"2024-04-25T08:36:35.764305+0000","last_active":"2024-04-25T08:36:35.766752+0000","last_peered":"2024-04-25T08:36:35.766752+0000","last_clean":"2024-04-25T08:36:35.766752+0000","last_became_active":"2024-04-25T08:36:35.764067+0000","last_became_peered":"2024-04-25T08:36:35.764067+0000","last_unstale":"2024-04-25T08:36:35.766752+0000","last_undegraded":"2024-04-25T08:36:35.766752+0000","last_fullsized":"2024-04-25T08:36:35.766752+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:20.920616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759815+0000","last_change":"2024-04-25T08:36:34.758831+0000","last_active":"2024-04-25T08:36:34.759815+0000","last_peered":"2024-04-25T08:36:34.759815+0000","last_clean":"2024-04-25T08:36:34.759815+0000","last_became_active":"2024-04-25T08:36:34.758382+0000","last_became_peered":"2024-04-25T08:36:34.758382+0000","last_unstale":"2024-04-25T08:36:34.759815+0000","last_undegraded":"2024-04-25T08:36:34.759815+0000","last_fullsized":"2024-04-25T08:36:34.759815+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:06.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747326+0000","last_change":"2024-04-25T08:36:34.747326+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747326+0000","last_undegraded":"2024-04-25T08:36:34.747326+0000","last_fullsized":"2024-04-25T08:36:34.747326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.761034+0000","last_change":"2024-04-25T08:36:34.758227+0000","last_active":"2024-04-25T08:36:34.761034+0000","last_peered":"2024-04-25T08:36:34.761034+0000","last_clean":"2024-04-25T08:36:34.761034+0000","last_became_active":"2024-04-25T08:36:34.757872+0000","last_became_peered":"2024-04-25T08:36:34.757872+0000","last_unstale":"2024-04-25T08:36:34.761034+0000","last_undegraded":"2024-04-25T08:36:34.761034+0000","last_fullsized":"2024-04-25T08:36:34.761034+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750496+0000","last_change":"2024-04-25T08:36:34.750496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750496+0000","last_undegraded":"2024-04-25T08:36:34.750496+0000","last_fullsized":"2024-04-25T08:36:34.750496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760454+0000","last_change":"2024-04-25T08:36:34.759067+0000","last_active":"2024-04-25T08:36:34.760454+0000","last_peered":"2024-04-25T08:36:34.760454+0000","last_clean":"2024-04-25T08:36:34.760454+0000","last_became_active":"2024-04-25T08:36:34.758669+0000","last_became_peered":"2024-04-25T08:36:34.758669+0000","last_unstale":"2024-04-25T08:36:34.760454+0000","last_undegraded":"2024-04-25T08:36:34.760454+0000","last_fullsized":"2024-04-25T08:36:34.760454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:06.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749803+0000","last_change":"2024-04-25T08:36:34.749803+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749803+0000","last_undegraded":"2024-04-25T08:36:34.749803+0000","last_fullsized":"2024-04-25T08:36:34.749803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760997+0000","last_change":"2024-04-25T08:36:34.759931+0000","last_active":"2024-04-25T08:36:34.760997+0000","last_peered":"2024-04-25T08:36:34.760997+0000","last_clean":"2024-04-25T08:36:34.760997+0000","last_became_active":"2024-04-25T08:36:34.759599+0000","last_became_peered":"2024-04-25T08:36:34.759599+0000","last_unstale":"2024-04-25T08:36:34.760997+0000","last_undegraded":"2024-04-25T08:36:34.760997+0000","last_fullsized":"2024-04-25T08:36:34.760997+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:33.870046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751089+0000","last_change":"2024-04-25T08:36:34.751089+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751089+0000","last_undegraded":"2024-04-25T08:36:34.751089+0000","last_fullsized":"2024-04-25T08:36:34.751089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783720+0000","last_change":"2024-04-25T08:36:34.753588+0000","last_active":"2024-04-25T08:36:35.783720+0000","last_peered":"2024-04-25T08:36:35.783720+0000","last_clean":"2024-04-25T08:36:35.783720+0000","last_became_active":"2024-04-25T08:36:34.753290+0000","last_became_peered":"2024-04-25T08:36:34.753290+0000","last_unstale":"2024-04-25T08:36:35.783720+0000","last_undegraded":"2024-04-25T08:36:35.783720+0000","last_fullsized":"2024-04-25T08:36:35.783720+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748432+0000","last_change":"2024-04-25T08:36:34.748432+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748432+0000","last_undegraded":"2024-04-25T08:36:34.748432+0000","last_fullsized":"2024-04-25T08:36:34.748432+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.786237+0000","last_change":"2024-04-25T08:36:34.750641+0000","last_active":"2024-04-25T08:36:35.786237+0000","last_peered":"2024-04-25T08:36:35.786237+0000","last_clean":"2024-04-25T08:36:35.786237+0000","last_became_active":"2024-04-25T08:36:34.748802+0000","last_became_peered":"2024-04-25T08:36:34.748802+0000","last_unstale":"2024-04-25T08:36:35.786237+0000","last_undegraded":"2024-04-25T08:36:35.786237+0000","last_fullsized":"2024-04-25T08:36:35.786237+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:39.476164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.773615+0000","last_change":"2024-04-25T08:36:35.768603+0000","last_active":"2024-04-25T08:36:35.773615+0000","last_peered":"2024-04-25T08:36:35.773615+0000","last_clean":"2024-04-25T08:36:35.773615+0000","last_became_active":"2024-04-25T08:36:35.766863+0000","last_became_peered":"2024-04-25T08:36:35.766863+0000","last_unstale":"2024-04-25T08:36:35.773615+0000","last_undegraded":"2024-04-25T08:36:35.773615+0000","last_fullsized":"2024-04-25T08:36:35.773615+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762382+0000","last_change":"2024-04-25T08:36:34.761126+0000","last_active":"2024-04-25T08:36:34.762382+0000","last_peered":"2024-04-25T08:36:34.762382+0000","last_clean":"2024-04-25T08:36:34.762382+0000","last_became_active":"2024-04-25T08:36:34.760848+0000","last_became_peered":"2024-04-25T08:36:34.760848+0000","last_unstale":"2024-04-25T08:36:34.762382+0000","last_undegraded":"2024-04-25T08:36:34.762382+0000","last_fullsized":"2024-04-25T08:36:34.762382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770663+0000","last_change":"2024-04-25T08:36:35.769783+0000","last_active":"2024-04-25T08:36:35.770663+0000","last_peered":"2024-04-25T08:36:35.770663+0000","last_clean":"2024-04-25T08:36:35.770663+0000","last_became_active":"2024-04-25T08:36:35.769101+0000","last_became_peered":"2024-04-25T08:36:35.769101+0000","last_unstale":"2024-04-25T08:36:35.770663+0000","last_undegraded":"2024-04-25T08:36:35.770663+0000","last_fullsized":"2024-04-25T08:36:35.770663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757823+0000","last_change":"2024-04-25T08:36:34.756585+0000","last_active":"2024-04-25T08:36:34.757823+0000","last_peered":"2024-04-25T08:36:34.757823+0000","last_clean":"2024-04-25T08:36:34.757823+0000","last_became_active":"2024-04-25T08:36:34.756301+0000","last_became_peered":"2024-04-25T08:36:34.756301+0000","last_unstale":"2024-04-25T08:36:34.757823+0000","last_undegraded":"2024-04-25T08:36:34.757823+0000","last_fullsized":"2024-04-25T08:36:34.757823+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763088+0000","last_change":"2024-04-25T08:36:34.761194+0000","last_active":"2024-04-25T08:36:34.763088+0000","last_peered":"2024-04-25T08:36:34.763088+0000","last_clean":"2024-04-25T08:36:34.763088+0000","last_became_active":"2024-04-25T08:36:34.760927+0000","last_became_peered":"2024-04-25T08:36:34.760927+0000","last_unstale":"2024-04-25T08:36:34.763088+0000","last_undegraded":"2024-04-25T08:36:34.763088+0000","last_fullsized":"2024-04-25T08:36:34.763088+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:51.400868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747795+0000","last_change":"2024-04-25T08:36:34.747795+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747795+0000","last_undegraded":"2024-04-25T08:36:34.747795+0000","last_fullsized":"2024-04-25T08:36:34.747795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760876+0000","last_change":"2024-04-25T08:36:34.759616+0000","last_active":"2024-04-25T08:36:34.760876+0000","last_peered":"2024-04-25T08:36:34.760876+0000","last_clean":"2024-04-25T08:36:34.760876+0000","last_became_active":"2024-04-25T08:36:34.759260+0000","last_became_peered":"2024-04-25T08:36:34.759260+0000","last_unstale":"2024-04-25T08:36:34.760876+0000","last_undegraded":"2024-04-25T08:36:34.760876+0000","last_fullsized":"2024-04-25T08:36:34.760876+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752657+0000","last_change":"2024-04-25T08:36:34.752657+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752657+0000","last_undegraded":"2024-04-25T08:36:34.752657+0000","last_fullsized":"2024-04-25T08:36:34.752657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.777741+0000","last_change":"2024-04-25T08:36:34.753294+0000","last_active":"2024-04-25T08:36:35.777741+0000","last_peered":"2024-04-25T08:36:35.777741+0000","last_clean":"2024-04-25T08:36:35.777741+0000","last_became_active":"2024-04-25T08:36:34.753016+0000","last_became_peered":"2024-04-25T08:36:34.753016+0000","last_unstale":"2024-04-25T08:36:35.777741+0000","last_undegraded":"2024-04-25T08:36:35.777741+0000","last_fullsized":"2024-04-25T08:36:35.777741+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770521+0000","last_change":"2024-04-25T08:36:35.767817+0000","last_active":"2024-04-25T08:36:35.770521+0000","last_peered":"2024-04-25T08:36:35.770521+0000","last_clean":"2024-04-25T08:36:35.770521+0000","last_became_active":"2024-04-25T08:36:35.767517+0000","last_became_peered":"2024-04-25T08:36:35.767517+0000","last_unstale":"2024-04-25T08:36:35.770521+0000","last_undegraded":"2024-04-25T08:36:35.770521+0000","last_fullsized":"2024-04-25T08:36:35.770521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.784098+0000","last_change":"2024-04-25T08:36:34.759604+0000","last_active":"2024-04-25T08:36:35.784098+0000","last_peered":"2024-04-25T08:36:35.784098+0000","last_clean":"2024-04-25T08:36:35.784098+0000","last_became_active":"2024-04-25T08:36:34.759303+0000","last_became_peered":"2024-04-25T08:36:34.759303+0000","last_unstale":"2024-04-25T08:36:35.784098+0000","last_undegraded":"2024-04-25T08:36:35.784098+0000","last_fullsized":"2024-04-25T08:36:35.784098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.258277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748937+0000","last_change":"2024-04-25T08:36:34.748937+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748937+0000","last_undegraded":"2024-04-25T08:36:34.748937+0000","last_fullsized":"2024-04-25T08:36:34.748937+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760131+0000","last_change":"2024-04-25T08:36:34.757404+0000","last_active":"2024-04-25T08:36:35.760131+0000","last_peered":"2024-04-25T08:36:35.760131+0000","last_clean":"2024-04-25T08:36:35.760131+0000","last_became_active":"2024-04-25T08:36:34.755586+0000","last_became_peered":"2024-04-25T08:36:34.755586+0000","last_unstale":"2024-04-25T08:36:35.760131+0000","last_undegraded":"2024-04-25T08:36:35.760131+0000","last_fullsized":"2024-04-25T08:36:35.760131+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747259+0000","last_change":"2024-04-25T08:36:34.747259+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747259+0000","last_undegraded":"2024-04-25T08:36:34.747259+0000","last_fullsized":"2024-04-25T08:36:34.747259+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.755095+0000","last_change":"2024-04-25T08:36:34.749438+0000","last_active":"2024-04-25T08:36:34.755095+0000","last_peered":"2024-04-25T08:36:34.755095+0000","last_clean":"2024-04-25T08:36:34.755095+0000","last_became_active":"2024-04-25T08:36:34.749073+0000","last_became_peered":"2024-04-25T08:36:34.749073+0000","last_unstale":"2024-04-25T08:36:34.755095+0000","last_undegraded":"2024-04-25T08:36:34.755095+0000","last_fullsized":"2024-04-25T08:36:34.755095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:25.045409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747760+0000","last_change":"2024-04-25T08:36:34.747760+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747760+0000","last_undegraded":"2024-04-25T08:36:34.747760+0000","last_fullsized":"2024-04-25T08:36:34.747760+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762766+0000","last_change":"2024-04-25T08:36:34.755127+0000","last_active":"2024-04-25T08:36:34.762766+0000","last_peered":"2024-04-25T08:36:34.762766+0000","last_clean":"2024-04-25T08:36:34.762766+0000","last_became_active":"2024-04-25T08:36:34.754781+0000","last_became_peered":"2024-04-25T08:36:34.754781+0000","last_unstale":"2024-04-25T08:36:34.762766+0000","last_undegraded":"2024-04-25T08:36:34.762766+0000","last_fullsized":"2024-04-25T08:36:34.762766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:40.479220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748664+0000","last_change":"2024-04-25T08:36:34.748664+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748664+0000","last_undegraded":"2024-04-25T08:36:34.748664+0000","last_fullsized":"2024-04-25T08:36:34.748664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753639+0000","last_change":"2024-04-25T08:36:34.756454+0000","last_active":"2024-04-25T08:36:35.753639+0000","last_peered":"2024-04-25T08:36:35.753639+0000","last_clean":"2024-04-25T08:36:35.753639+0000","last_became_active":"2024-04-25T08:36:34.755238+0000","last_became_peered":"2024-04-25T08:36:34.755238+0000","last_unstale":"2024-04-25T08:36:35.753639+0000","last_undegraded":"2024-04-25T08:36:35.753639+0000","last_fullsized":"2024-04-25T08:36:35.753639+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762995+0000","last_change":"2024-04-25T08:36:34.760255+0000","last_active":"2024-04-25T08:36:34.762995+0000","last_peered":"2024-04-25T08:36:34.762995+0000","last_clean":"2024-04-25T08:36:34.762995+0000","last_became_active":"2024-04-25T08:36:34.759634+0000","last_became_peered":"2024-04-25T08:36:34.759634+0000","last_unstale":"2024-04-25T08:36:34.762995+0000","last_undegraded":"2024-04-25T08:36:34.762995+0000","last_fullsized":"2024-04-25T08:36:34.762995+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:54.192240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751714+0000","last_change":"2024-04-25T08:36:34.751714+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751714+0000","last_undegraded":"2024-04-25T08:36:34.751714+0000","last_fullsized":"2024-04-25T08:36:34.751714+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762154+0000","last_change":"2024-04-25T08:36:34.760831+0000","last_active":"2024-04-25T08:36:34.762154+0000","last_peered":"2024-04-25T08:36:34.762154+0000","last_clean":"2024-04-25T08:36:34.762154+0000","last_became_active":"2024-04-25T08:36:34.760558+0000","last_became_peered":"2024-04-25T08:36:34.760558+0000","last_unstale":"2024-04-25T08:36:34.762154+0000","last_undegraded":"2024-04-25T08:36:34.762154+0000","last_fullsized":"2024-04-25T08:36:34.762154+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744146+0000","last_change":"2024-04-25T08:36:34.744146+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744146+0000","last_undegraded":"2024-04-25T08:36:34.744146+0000","last_fullsized":"2024-04-25T08:36:34.744146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.758450+0000","last_change":"2024-04-25T08:36:34.756998+0000","last_active":"2024-04-25T08:36:34.758450+0000","last_peered":"2024-04-25T08:36:34.758450+0000","last_clean":"2024-04-25T08:36:34.758450+0000","last_became_active":"2024-04-25T08:36:34.756057+0000","last_became_peered":"2024-04-25T08:36:34.756057+0000","last_unstale":"2024-04-25T08:36:34.758450+0000","last_undegraded":"2024-04-25T08:36:34.758450+0000","last_fullsized":"2024-04-25T08:36:34.758450+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:51.646595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749164+0000","last_change":"2024-04-25T08:36:34.749164+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749164+0000","last_undegraded":"2024-04-25T08:36:34.749164+0000","last_fullsized":"2024-04-25T08:36:34.749164+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.754613+0000","last_change":"2024-04-25T08:36:34.750357+0000","last_active":"2024-04-25T08:36:35.754613+0000","last_peered":"2024-04-25T08:36:35.754613+0000","last_clean":"2024-04-25T08:36:35.754613+0000","last_became_active":"2024-04-25T08:36:34.749684+0000","last_became_peered":"2024-04-25T08:36:34.749684+0000","last_unstale":"2024-04-25T08:36:35.754613+0000","last_undegraded":"2024-04-25T08:36:35.754613+0000","last_fullsized":"2024-04-25T08:36:35.754613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774914+0000","last_change":"2024-04-25T08:36:35.773260+0000","last_active":"2024-04-25T08:36:35.774914+0000","last_peered":"2024-04-25T08:36:35.774914+0000","last_clean":"2024-04-25T08:36:35.774914+0000","last_became_active":"2024-04-25T08:36:35.773012+0000","last_became_peered":"2024-04-25T08:36:35.773012+0000","last_unstale":"2024-04-25T08:36:35.774914+0000","last_undegraded":"2024-04-25T08:36:35.774914+0000","last_fullsized":"2024-04-25T08:36:35.774914+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757258+0000","last_change":"2024-04-25T08:36:34.756400+0000","last_active":"2024-04-25T08:36:34.757258+0000","last_peered":"2024-04-25T08:36:34.757258+0000","last_clean":"2024-04-25T08:36:34.757258+0000","last_became_active":"2024-04-25T08:36:34.756114+0000","last_became_peered":"2024-04-25T08:36:34.756114+0000","last_unstale":"2024-04-25T08:36:34.757258+0000","last_undegraded":"2024-04-25T08:36:34.757258+0000","last_fullsized":"2024-04-25T08:36:34.757258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752972+0000","last_change":"2024-04-25T08:36:34.752972+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752972+0000","last_undegraded":"2024-04-25T08:36:34.752972+0000","last_fullsized":"2024-04-25T08:36:34.752972+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759854+0000","last_change":"2024-04-25T08:36:34.757608+0000","last_active":"2024-04-25T08:36:34.759854+0000","last_peered":"2024-04-25T08:36:34.759854+0000","last_clean":"2024-04-25T08:36:34.759854+0000","last_became_active":"2024-04-25T08:36:34.756201+0000","last_became_peered":"2024-04-25T08:36:34.756201+0000","last_unstale":"2024-04-25T08:36:34.759854+0000","last_undegraded":"2024-04-25T08:36:34.759854+0000","last_fullsized":"2024-04-25T08:36:34.759854+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:09.132700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754953+0000","last_change":"2024-04-25T08:36:34.754953+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754953+0000","last_undegraded":"2024-04-25T08:36:34.754953+0000","last_fullsized":"2024-04-25T08:36:34.754953+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763179+0000","last_change":"2024-04-25T08:36:34.760614+0000","last_active":"2024-04-25T08:36:34.763179+0000","last_peered":"2024-04-25T08:36:34.763179+0000","last_clean":"2024-04-25T08:36:34.763179+0000","last_became_active":"2024-04-25T08:36:34.760357+0000","last_became_peered":"2024-04-25T08:36:34.760357+0000","last_unstale":"2024-04-25T08:36:34.763179+0000","last_undegraded":"2024-04-25T08:36:34.763179+0000","last_fullsized":"2024-04-25T08:36:34.763179+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:48.096031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768321+0000","last_change":"2024-04-25T08:36:35.766389+0000","last_active":"2024-04-25T08:36:35.768321+0000","last_peered":"2024-04-25T08:36:35.768321+0000","last_clean":"2024-04-25T08:36:35.768321+0000","last_became_active":"2024-04-25T08:36:35.766116+0000","last_became_peered":"2024-04-25T08:36:35.766116+0000","last_unstale":"2024-04-25T08:36:35.768321+0000","last_undegraded":"2024-04-25T08:36:35.768321+0000","last_fullsized":"2024-04-25T08:36:35.768321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753757+0000","last_change":"2024-04-25T08:36:34.754257+0000","last_active":"2024-04-25T08:36:35.753757+0000","last_peered":"2024-04-25T08:36:35.753757+0000","last_clean":"2024-04-25T08:36:35.753757+0000","last_became_active":"2024-04-25T08:36:34.752327+0000","last_became_peered":"2024-04-25T08:36:34.752327+0000","last_unstale":"2024-04-25T08:36:35.753757+0000","last_undegraded":"2024-04-25T08:36:35.753757+0000","last_fullsized":"2024-04-25T08:36:35.753757+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774411+0000","last_change":"2024-04-25T08:36:35.770356+0000","last_active":"2024-04-25T08:36:35.774411+0000","last_peered":"2024-04-25T08:36:35.774411+0000","last_clean":"2024-04-25T08:36:35.774411+0000","last_became_active":"2024-04-25T08:36:35.769027+0000","last_became_peered":"2024-04-25T08:36:35.769027+0000","last_unstale":"2024-04-25T08:36:35.774411+0000","last_undegraded":"2024-04-25T08:36:35.774411+0000","last_fullsized":"2024-04-25T08:36:35.774411+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763309+0000","last_change":"2024-04-25T08:36:34.762590+0000","last_active":"2024-04-25T08:36:34.763309+0000","last_peered":"2024-04-25T08:36:34.763309+0000","last_clean":"2024-04-25T08:36:34.763309+0000","last_became_active":"2024-04-25T08:36:34.762255+0000","last_became_peered":"2024-04-25T08:36:34.762255+0000","last_unstale":"2024-04-25T08:36:34.763309+0000","last_undegraded":"2024-04-25T08:36:34.763309+0000","last_fullsized":"2024-04-25T08:36:34.763309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:13.862606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774363+0000","last_change":"2024-04-25T08:36:35.769174+0000","last_active":"2024-04-25T08:36:35.774363+0000","last_peered":"2024-04-25T08:36:35.774363+0000","last_clean":"2024-04-25T08:36:35.774363+0000","last_became_active":"2024-04-25T08:36:35.768769+0000","last_became_peered":"2024-04-25T08:36:35.768769+0000","last_unstale":"2024-04-25T08:36:35.774363+0000","last_undegraded":"2024-04-25T08:36:35.774363+0000","last_fullsized":"2024-04-25T08:36:35.774363+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762841+0000","last_change":"2024-04-25T08:36:34.755845+0000","last_active":"2024-04-25T08:36:34.762841+0000","last_peered":"2024-04-25T08:36:34.762841+0000","last_clean":"2024-04-25T08:36:34.762841+0000","last_became_active":"2024-04-25T08:36:34.755540+0000","last_became_peered":"2024-04-25T08:36:34.755540+0000","last_unstale":"2024-04-25T08:36:34.762841+0000","last_undegraded":"2024-04-25T08:36:34.762841+0000","last_fullsized":"2024-04-25T08:36:34.762841+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:02.715530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760141+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:36:35.760141+0000","last_peered":"2024-04-25T08:36:35.760141+0000","last_clean":"2024-04-25T08:36:35.760141+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:36:35.760141+0000","last_undegraded":"2024-04-25T08:36:35.760141+0000","last_fullsized":"2024-04-25T08:36:35.760141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1949696,"data_stored":1216579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":1368064,"data_stored":628267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40824,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331016,"statfs":{"total":96636764160,"available":96594960384,"internally_reserved":0,"allocated":1347584,"data_stored":626051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54688,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317152,"statfs":{"total":96636764160,"available":96580763648,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54696,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317144,"statfs":{"total":96636764160,"available":96580755456,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1916928,"data_stored":1202585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:39.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:39.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.256+0000 7f7588e8f640 1 Processor -- start 2024-04-25T08:36:39.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.257+0000 7f7588e8f640 1 -- start start 2024-04-25T08:36:39.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.257+0000 7f7588e8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 0x7f75840d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.257+0000 7f7588e8f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f75840b2cb0 0x7f75840d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.257+0000 7f7588e8f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75840b97a0 con 0x7f75840b92a0 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.257+0000 7f7588e8f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758415dfc0 con 0x7f75840b2cb0 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.257+0000 7f7588e8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840d1a00 con 0x7f758415dbe0 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 0x7f75840d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 0x7f75840d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54654/0 (socket says 172.21.15.60:54654) 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 -- 172.21.15.60:0/2483603705 learned_addr learned my addr 172.21.15.60:0/2483603705 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f757bfff640 1 --2- 172.21.15.60:0/2483603705 >> v2:172.21.15.100:3300/0 conn(0x7f75840b2cb0 0x7f75840d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 -- 172.21.15.60:0/2483603705 >> v2:172.21.15.100:3300/0 conn(0x7f75840b2cb0 msgr2=0x7f75840d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 --2- 172.21.15.60:0/2483603705 >> v2:172.21.15.100:3300/0 conn(0x7f75840b2cb0 0x7f75840d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 -- 172.21.15.60:0/2483603705 >> v1:172.21.15.100:6789/0 conn(0x7f75840b92a0 legacy=0x7f75840b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f7581d9b640 1 -- 172.21.15.60:0/2483603705 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438075366 0 0) 0x7f75840b97a0 con 0x7f75840b92a0 2024-04-25T08:36:39.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.258+0000 7f758259c640 1 -- 172.21.15.60:0/2483603705 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840d1c50 con 0x7f758415dbe0 2024-04-25T08:36:39.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.259+0000 7f758259c640 1 --2- 172.21.15.60:0/2483603705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 0x7f75840d0910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7574009870 tx=0x7f757402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f9cf0fc52c1654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.259+0000 7f7581d9b640 1 -- 172.21.15.60:0/2483603705 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574037070 con 0x7f758415dbe0 2024-04-25T08:36:39.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.259+0000 7f7581d9b640 1 -- 172.21.15.60:0/2483603705 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7574032440 con 0x7f758415dbe0 2024-04-25T08:36:39.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.259+0000 7f7581d9b640 1 -- 172.21.15.60:0/2483603705 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574032810 con 0x7f758415dbe0 2024-04-25T08:36:39.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 -- 172.21.15.60:0/2483603705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 msgr2=0x7f75840d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 --2- 172.21.15.60:0/2483603705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 0x7f75840d0910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7574009870 tx=0x7f757402a270 comp rx=0 tx=0).stop 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 -- 172.21.15.60:0/2483603705 shutdown_connections 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 --2- 172.21.15.60:0/2483603705 >> v2:172.21.15.100:3300/0 conn(0x7f75840b2cb0 0x7f75840d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 --2- 172.21.15.60:0/2483603705 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758415dbe0 0x7f75840d0910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 -- 172.21.15.60:0/2483603705 >> 172.21.15.60:0/2483603705 conn(0x7f7584061a20 msgr2=0x7f7584061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 -- 172.21.15.60:0/2483603705 shutdown_connections 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.260+0000 7f7588e8f640 1 -- 172.21.15.60:0/2483603705 wait complete. 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.261+0000 7f7588e8f640 1 Processor -- start 2024-04-25T08:36:39.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.261+0000 7f7588e8f640 1 -- start start 2024-04-25T08:36:39.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7588e8f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b2cb0 0x7f75840c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7588e8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b92a0 0x7f75840cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7588e8f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 0x7f758415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7588e8f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75840b6ab0 con 0x7f758415dbe0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7588e8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75840d2210 con 0x7f75840b92a0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7588e8f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840b6c30 con 0x7f75840b2cb0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 0x7f758415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 0x7f758415f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55020/0 (socket says 172.21.15.60:55020) 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 -- 172.21.15.60:0/927854681 learned_addr learned my addr 172.21.15.60:0/927854681 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f757bfff640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b92a0 0x7f75840cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f758259c640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b2cb0 0x7f75840c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 -- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b2cb0 msgr2=0x7f75840c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b2cb0 0x7f75840c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 -- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b92a0 msgr2=0x7f75840cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b92a0 0x7f75840cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 -- 172.21.15.60:0/927854681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75841f6a90 con 0x7f758415dbe0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.262+0000 7f7582d9d640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 0x7f758415f1a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c700 tx=0x7f757c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.263+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c003da0 con 0x7f758415dbe0 2024-04-25T08:36:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.263+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c00ce10 con 0x7f758415dbe0 2024-04-25T08:36:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.263+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75841f6d70 con 0x7f758415dbe0 2024-04-25T08:36:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.263+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c005270 con 0x7f758415dbe0 2024-04-25T08:36:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.263+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75841f7330 con 0x7f758415dbe0 2024-04-25T08:36:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.264+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7584073990 con 0x7f758415dbe0 2024-04-25T08:36:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.264+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f757c005410 con 0x7f758415dbe0 2024-04-25T08:36:39.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.265+0000 7f757affd640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756406b500 0x7f756406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.265+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f757c08a020 con 0x7f758415dbe0 2024-04-25T08:36:39.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.265+0000 7f758259c640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756406b500 0x7f756406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.265+0000 7f758259c640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756406b500 0x7f756406d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.269+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f757c058750 con 0x7f758415dbe0 2024-04-25T08:36:39.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.448+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75840620e0 con 0x7f756406b500 2024-04-25T08:36:39.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.454+0000 7f757affd640 1 -- 172.21.15.60:0/927854681 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248853 (secure 0 0 0) 0x7f75840620e0 con 0x7f756406b500 2024-04-25T08:36:39.451 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:39.452 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756406b500 msgr2=0x7f756406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756406b500 0x7f756406d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).stop 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 msgr2=0x7f758415f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 0x7f758415f1a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c700 tx=0x7f757c00cbc0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 shutdown_connections 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758415dbe0 0x7f758415f1a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756406b500 0x7f756406d9c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f75840b92a0 0x7f75840cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 --2- 172.21.15.60:0/927854681 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f75840b2cb0 0x7f75840c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 >> 172.21.15.60:0/927854681 conn(0x7f7584061a20 msgr2=0x7f75840d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 shutdown_connections 2024-04-25T08:36:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.457+0000 7f7588e8f640 1 -- 172.21.15.60:0/927854681 wait complete. 2024-04-25T08:36:39.462 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-25T08:36:37.497672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394360,"kb_used_data":11644,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754580360,"statfs":{"total":773094113280,"available":772690288640,"internally_reserved":0,"allocated":11923456,"data_stored":6149018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001159"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750939+0000","last_change":"2024-04-25T08:36:34.750939+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750939+0000","last_undegraded":"2024-04-25T08:36:34.750939+0000","last_fullsized":"2024-04-25T08:36:34.750939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766771+0000","last_change":"2024-04-25T08:36:35.765297+0000","last_active":"2024-04-25T08:36:35.766771+0000","last_peered":"2024-04-25T08:36:35.766771+0000","last_clean":"2024-04-25T08:36:35.766771+0000","last_became_active":"2024-04-25T08:36:35.763995+0000","last_became_peered":"2024-04-25T08:36:35.763995+0000","last_unstale":"2024-04-25T08:36:35.766771+0000","last_undegraded":"2024-04-25T08:36:35.766771+0000","last_fullsized":"2024-04-25T08:36:35.766771+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749594+0000","last_change":"2024-04-25T08:36:34.749594+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749594+0000","last_undegraded":"2024-04-25T08:36:34.749594+0000","last_fullsized":"2024-04-25T08:36:34.749594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752769+0000","last_change":"2024-04-25T08:36:34.752769+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752769+0000","last_undegraded":"2024-04-25T08:36:34.752769+0000","last_fullsized":"2024-04-25T08:36:34.752769+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749401+0000","last_change":"2024-04-25T08:36:34.749401+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749401+0000","last_undegraded":"2024-04-25T08:36:34.749401+0000","last_fullsized":"2024-04-25T08:36:34.749401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783055+0000","last_change":"2024-04-25T08:36:35.782370+0000","last_active":"2024-04-25T08:36:35.783055+0000","last_peered":"2024-04-25T08:36:35.783055+0000","last_clean":"2024-04-25T08:36:35.783055+0000","last_became_active":"2024-04-25T08:36:35.782129+0000","last_became_peered":"2024-04-25T08:36:35.782129+0000","last_unstale":"2024-04-25T08:36:35.783055+0000","last_undegraded":"2024-04-25T08:36:35.783055+0000","last_fullsized":"2024-04-25T08:36:35.783055+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772945+0000","last_change":"2024-04-25T08:36:35.771864+0000","last_active":"2024-04-25T08:36:35.772945+0000","last_peered":"2024-04-25T08:36:35.772945+0000","last_clean":"2024-04-25T08:36:35.772945+0000","last_became_active":"2024-04-25T08:36:35.771591+0000","last_became_peered":"2024-04-25T08:36:35.771591+0000","last_unstale":"2024-04-25T08:36:35.772945+0000","last_undegraded":"2024-04-25T08:36:35.772945+0000","last_fullsized":"2024-04-25T08:36:35.772945+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.756279+0000","last_change":"2024-04-25T08:36:34.756279+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.756279+0000","last_undegraded":"2024-04-25T08:36:34.756279+0000","last_fullsized":"2024-04-25T08:36:34.756279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770474+0000","last_change":"2024-04-25T08:36:35.768883+0000","last_active":"2024-04-25T08:36:35.770474+0000","last_peered":"2024-04-25T08:36:35.770474+0000","last_clean":"2024-04-25T08:36:35.770474+0000","last_became_active":"2024-04-25T08:36:35.768603+0000","last_became_peered":"2024-04-25T08:36:35.768603+0000","last_unstale":"2024-04-25T08:36:35.770474+0000","last_undegraded":"2024-04-25T08:36:35.770474+0000","last_fullsized":"2024-04-25T08:36:35.770474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750944+0000","last_change":"2024-04-25T08:36:34.750944+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750944+0000","last_undegraded":"2024-04-25T08:36:34.750944+0000","last_fullsized":"2024-04-25T08:36:34.750944+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774841+0000","last_change":"2024-04-25T08:36:35.772717+0000","last_active":"2024-04-25T08:36:35.774841+0000","last_peered":"2024-04-25T08:36:35.774841+0000","last_clean":"2024-04-25T08:36:35.774841+0000","last_became_active":"2024-04-25T08:36:35.772232+0000","last_became_peered":"2024-04-25T08:36:35.772232+0000","last_unstale":"2024-04-25T08:36:35.774841+0000","last_undegraded":"2024-04-25T08:36:35.774841+0000","last_fullsized":"2024-04-25T08:36:35.774841+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744825+0000","last_change":"2024-04-25T08:36:34.744825+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744825+0000","last_undegraded":"2024-04-25T08:36:34.744825+0000","last_fullsized":"2024-04-25T08:36:34.744825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749875+0000","last_change":"2024-04-25T08:36:34.749875+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749875+0000","last_undegraded":"2024-04-25T08:36:34.749875+0000","last_fullsized":"2024-04-25T08:36:34.749875+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768324+0000","last_change":"2024-04-25T08:36:35.767092+0000","last_active":"2024-04-25T08:36:35.768324+0000","last_peered":"2024-04-25T08:36:35.768324+0000","last_clean":"2024-04-25T08:36:35.768324+0000","last_became_active":"2024-04-25T08:36:35.766827+0000","last_became_peered":"2024-04-25T08:36:35.766827+0000","last_unstale":"2024-04-25T08:36:35.768324+0000","last_undegraded":"2024-04-25T08:36:35.768324+0000","last_fullsized":"2024-04-25T08:36:35.768324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.776596+0000","last_change":"2024-04-25T08:36:35.775011+0000","last_active":"2024-04-25T08:36:35.776596+0000","last_peered":"2024-04-25T08:36:35.776596+0000","last_clean":"2024-04-25T08:36:35.776596+0000","last_became_active":"2024-04-25T08:36:35.774695+0000","last_became_peered":"2024-04-25T08:36:35.774695+0000","last_unstale":"2024-04-25T08:36:35.776596+0000","last_undegraded":"2024-04-25T08:36:35.776596+0000","last_fullsized":"2024-04-25T08:36:35.776596+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.432682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768288+0000","last_change":"2024-04-25T08:36:35.766209+0000","last_active":"2024-04-25T08:36:35.768288+0000","last_peered":"2024-04-25T08:36:35.768288+0000","last_clean":"2024-04-25T08:36:35.768288+0000","last_became_active":"2024-04-25T08:36:35.765932+0000","last_became_peered":"2024-04-25T08:36:35.765932+0000","last_unstale":"2024-04-25T08:36:35.768288+0000","last_undegraded":"2024-04-25T08:36:35.768288+0000","last_fullsized":"2024-04-25T08:36:35.768288+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749979+0000","last_change":"2024-04-25T08:36:34.749979+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749979+0000","last_undegraded":"2024-04-25T08:36:34.749979+0000","last_fullsized":"2024-04-25T08:36:34.749979+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772836+0000","last_change":"2024-04-25T08:36:35.771768+0000","last_active":"2024-04-25T08:36:35.772836+0000","last_peered":"2024-04-25T08:36:35.772836+0000","last_clean":"2024-04-25T08:36:35.772836+0000","last_became_active":"2024-04-25T08:36:35.771487+0000","last_became_peered":"2024-04-25T08:36:35.771487+0000","last_unstale":"2024-04-25T08:36:35.772836+0000","last_undegraded":"2024-04-25T08:36:35.772836+0000","last_fullsized":"2024-04-25T08:36:35.772836+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748333+0000","last_change":"2024-04-25T08:36:34.748333+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748333+0000","last_undegraded":"2024-04-25T08:36:34.748333+0000","last_fullsized":"2024-04-25T08:36:34.748333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744421+0000","last_change":"2024-04-25T08:36:34.744421+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744421+0000","last_undegraded":"2024-04-25T08:36:34.744421+0000","last_fullsized":"2024-04-25T08:36:34.744421+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754852+0000","last_change":"2024-04-25T08:36:34.754852+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754852+0000","last_undegraded":"2024-04-25T08:36:34.754852+0000","last_fullsized":"2024-04-25T08:36:34.754852+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751587+0000","last_change":"2024-04-25T08:36:34.751587+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751587+0000","last_undegraded":"2024-04-25T08:36:34.751587+0000","last_fullsized":"2024-04-25T08:36:34.751587+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747496+0000","last_change":"2024-04-25T08:36:34.747496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747496+0000","last_undegraded":"2024-04-25T08:36:34.747496+0000","last_fullsized":"2024-04-25T08:36:34.747496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749242+0000","last_change":"2024-04-25T08:36:34.749242+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749242+0000","last_undegraded":"2024-04-25T08:36:34.749242+0000","last_fullsized":"2024-04-25T08:36:34.749242+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.746069+0000","last_change":"2024-04-25T08:36:34.746069+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.746069+0000","last_undegraded":"2024-04-25T08:36:34.746069+0000","last_fullsized":"2024-04-25T08:36:34.746069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.745098+0000","last_change":"2024-04-25T08:36:34.745098+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.745098+0000","last_undegraded":"2024-04-25T08:36:34.745098+0000","last_fullsized":"2024-04-25T08:36:34.745098+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774306+0000","last_change":"2024-04-25T08:36:35.768841+0000","last_active":"2024-04-25T08:36:35.774306+0000","last_peered":"2024-04-25T08:36:35.774306+0000","last_clean":"2024-04-25T08:36:35.774306+0000","last_became_active":"2024-04-25T08:36:35.768565+0000","last_became_peered":"2024-04-25T08:36:35.768565+0000","last_unstale":"2024-04-25T08:36:35.774306+0000","last_undegraded":"2024-04-25T08:36:35.774306+0000","last_fullsized":"2024-04-25T08:36:35.774306+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.769495+0000","last_change":"2024-04-25T08:36:35.764415+0000","last_active":"2024-04-25T08:36:35.769495+0000","last_peered":"2024-04-25T08:36:35.769495+0000","last_clean":"2024-04-25T08:36:35.769495+0000","last_became_active":"2024-04-25T08:36:35.764127+0000","last_became_peered":"2024-04-25T08:36:35.764127+0000","last_unstale":"2024-04-25T08:36:35.769495+0000","last_undegraded":"2024-04-25T08:36:35.769495+0000","last_fullsized":"2024-04-25T08:36:35.769495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754375+0000","last_change":"2024-04-25T08:36:34.754375+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754375+0000","last_undegraded":"2024-04-25T08:36:34.754375+0000","last_fullsized":"2024-04-25T08:36:34.754375+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744739+0000","last_change":"2024-04-25T08:36:34.744739+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744739+0000","last_undegraded":"2024-04-25T08:36:34.744739+0000","last_fullsized":"2024-04-25T08:36:34.744739+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770339+0000","last_change":"2024-04-25T08:36:35.767653+0000","last_active":"2024-04-25T08:36:35.770339+0000","last_peered":"2024-04-25T08:36:35.770339+0000","last_clean":"2024-04-25T08:36:35.770339+0000","last_became_active":"2024-04-25T08:36:35.767382+0000","last_became_peered":"2024-04-25T08:36:35.767382+0000","last_unstale":"2024-04-25T08:36:35.770339+0000","last_undegraded":"2024-04-25T08:36:35.770339+0000","last_fullsized":"2024-04-25T08:36:35.770339+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754189+0000","last_change":"2024-04-25T08:36:34.754189+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754189+0000","last_undegraded":"2024-04-25T08:36:34.754189+0000","last_fullsized":"2024-04-25T08:36:34.754189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748276+0000","last_change":"2024-04-25T08:36:34.748276+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748276+0000","last_undegraded":"2024-04-25T08:36:34.748276+0000","last_fullsized":"2024-04-25T08:36:34.748276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757104+0000","last_change":"2024-04-25T08:36:34.755252+0000","last_active":"2024-04-25T08:36:34.757104+0000","last_peered":"2024-04-25T08:36:34.757104+0000","last_clean":"2024-04-25T08:36:34.757104+0000","last_became_active":"2024-04-25T08:36:34.754950+0000","last_became_peered":"2024-04-25T08:36:34.754950+0000","last_unstale":"2024-04-25T08:36:34.757104+0000","last_undegraded":"2024-04-25T08:36:34.757104+0000","last_fullsized":"2024-04-25T08:36:34.757104+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750505+0000","last_change":"2024-04-25T08:36:34.750505+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750505+0000","last_undegraded":"2024-04-25T08:36:34.750505+0000","last_fullsized":"2024-04-25T08:36:34.750505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762949+0000","last_change":"2024-04-25T08:36:34.759886+0000","last_active":"2024-04-25T08:36:34.762949+0000","last_peered":"2024-04-25T08:36:34.762949+0000","last_clean":"2024-04-25T08:36:34.762949+0000","last_became_active":"2024-04-25T08:36:34.759596+0000","last_became_peered":"2024-04-25T08:36:34.759596+0000","last_unstale":"2024-04-25T08:36:34.762949+0000","last_undegraded":"2024-04-25T08:36:34.762949+0000","last_fullsized":"2024-04-25T08:36:34.762949+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:49.092358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744294+0000","last_change":"2024-04-25T08:36:34.744294+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744294+0000","last_undegraded":"2024-04-25T08:36:34.744294+0000","last_fullsized":"2024-04-25T08:36:34.744294+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760269+0000","last_change":"2024-04-25T08:36:34.759233+0000","last_active":"2024-04-25T08:36:34.760269+0000","last_peered":"2024-04-25T08:36:34.760269+0000","last_clean":"2024-04-25T08:36:34.760269+0000","last_became_active":"2024-04-25T08:36:34.758929+0000","last_became_peered":"2024-04-25T08:36:34.758929+0000","last_unstale":"2024-04-25T08:36:34.760269+0000","last_undegraded":"2024-04-25T08:36:34.760269+0000","last_fullsized":"2024-04-25T08:36:34.760269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.176620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774869+0000","last_change":"2024-04-25T08:36:35.773004+0000","last_active":"2024-04-25T08:36:35.774869+0000","last_peered":"2024-04-25T08:36:35.774869+0000","last_clean":"2024-04-25T08:36:35.774869+0000","last_became_active":"2024-04-25T08:36:35.772491+0000","last_became_peered":"2024-04-25T08:36:35.772491+0000","last_unstale":"2024-04-25T08:36:35.774869+0000","last_undegraded":"2024-04-25T08:36:35.774869+0000","last_fullsized":"2024-04-25T08:36:35.774869+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762413+0000","last_change":"2024-04-25T08:36:34.761740+0000","last_active":"2024-04-25T08:36:34.762413+0000","last_peered":"2024-04-25T08:36:34.762413+0000","last_clean":"2024-04-25T08:36:34.762413+0000","last_became_active":"2024-04-25T08:36:34.761387+0000","last_became_peered":"2024-04-25T08:36:34.761387+0000","last_unstale":"2024-04-25T08:36:34.762413+0000","last_undegraded":"2024-04-25T08:36:34.762413+0000","last_fullsized":"2024-04-25T08:36:34.762413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747266+0000","last_change":"2024-04-25T08:36:34.747266+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747266+0000","last_undegraded":"2024-04-25T08:36:34.747266+0000","last_fullsized":"2024-04-25T08:36:34.747266+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783225+0000","last_change":"2024-04-25T08:36:34.752807+0000","last_active":"2024-04-25T08:36:35.783225+0000","last_peered":"2024-04-25T08:36:35.783225+0000","last_clean":"2024-04-25T08:36:35.783225+0000","last_became_active":"2024-04-25T08:36:34.752501+0000","last_became_peered":"2024-04-25T08:36:34.752501+0000","last_unstale":"2024-04-25T08:36:35.783225+0000","last_undegraded":"2024-04-25T08:36:35.783225+0000","last_fullsized":"2024-04-25T08:36:35.783225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.320853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747253+0000","last_change":"2024-04-25T08:36:34.747253+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747253+0000","last_undegraded":"2024-04-25T08:36:34.747253+0000","last_fullsized":"2024-04-25T08:36:34.747253+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783449+0000","last_change":"2024-04-25T08:36:34.761095+0000","last_active":"2024-04-25T08:36:35.783449+0000","last_peered":"2024-04-25T08:36:35.783449+0000","last_clean":"2024-04-25T08:36:35.783449+0000","last_became_active":"2024-04-25T08:36:34.760735+0000","last_became_peered":"2024-04-25T08:36:34.760735+0000","last_unstale":"2024-04-25T08:36:35.783449+0000","last_undegraded":"2024-04-25T08:36:35.783449+0000","last_fullsized":"2024-04-25T08:36:35.783449+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766752+0000","last_change":"2024-04-25T08:36:35.764305+0000","last_active":"2024-04-25T08:36:35.766752+0000","last_peered":"2024-04-25T08:36:35.766752+0000","last_clean":"2024-04-25T08:36:35.766752+0000","last_became_active":"2024-04-25T08:36:35.764067+0000","last_became_peered":"2024-04-25T08:36:35.764067+0000","last_unstale":"2024-04-25T08:36:35.766752+0000","last_undegraded":"2024-04-25T08:36:35.766752+0000","last_fullsized":"2024-04-25T08:36:35.766752+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:20.920616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759815+0000","last_change":"2024-04-25T08:36:34.758831+0000","last_active":"2024-04-25T08:36:34.759815+0000","last_peered":"2024-04-25T08:36:34.759815+0000","last_clean":"2024-04-25T08:36:34.759815+0000","last_became_active":"2024-04-25T08:36:34.758382+0000","last_became_peered":"2024-04-25T08:36:34.758382+0000","last_unstale":"2024-04-25T08:36:34.759815+0000","last_undegraded":"2024-04-25T08:36:34.759815+0000","last_fullsized":"2024-04-25T08:36:34.759815+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:06.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747326+0000","last_change":"2024-04-25T08:36:34.747326+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747326+0000","last_undegraded":"2024-04-25T08:36:34.747326+0000","last_fullsized":"2024-04-25T08:36:34.747326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.761034+0000","last_change":"2024-04-25T08:36:34.758227+0000","last_active":"2024-04-25T08:36:34.761034+0000","last_peered":"2024-04-25T08:36:34.761034+0000","last_clean":"2024-04-25T08:36:34.761034+0000","last_became_active":"2024-04-25T08:36:34.757872+0000","last_became_peered":"2024-04-25T08:36:34.757872+0000","last_unstale":"2024-04-25T08:36:34.761034+0000","last_undegraded":"2024-04-25T08:36:34.761034+0000","last_fullsized":"2024-04-25T08:36:34.761034+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750496+0000","last_change":"2024-04-25T08:36:34.750496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750496+0000","last_undegraded":"2024-04-25T08:36:34.750496+0000","last_fullsized":"2024-04-25T08:36:34.750496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760454+0000","last_change":"2024-04-25T08:36:34.759067+0000","last_active":"2024-04-25T08:36:34.760454+0000","last_peered":"2024-04-25T08:36:34.760454+0000","last_clean":"2024-04-25T08:36:34.760454+0000","last_became_active":"2024-04-25T08:36:34.758669+0000","last_became_peered":"2024-04-25T08:36:34.758669+0000","last_unstale":"2024-04-25T08:36:34.760454+0000","last_undegraded":"2024-04-25T08:36:34.760454+0000","last_fullsized":"2024-04-25T08:36:34.760454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:06.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749803+0000","last_change":"2024-04-25T08:36:34.749803+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749803+0000","last_undegraded":"2024-04-25T08:36:34.749803+0000","last_fullsized":"2024-04-25T08:36:34.749803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760997+0000","last_change":"2024-04-25T08:36:34.759931+0000","last_active":"2024-04-25T08:36:34.760997+0000","last_peered":"2024-04-25T08:36:34.760997+0000","last_clean":"2024-04-25T08:36:34.760997+0000","last_became_active":"2024-04-25T08:36:34.759599+0000","last_became_peered":"2024-04-25T08:36:34.759599+0000","last_unstale":"2024-04-25T08:36:34.760997+0000","last_undegraded":"2024-04-25T08:36:34.760997+0000","last_fullsized":"2024-04-25T08:36:34.760997+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:33.870046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751089+0000","last_change":"2024-04-25T08:36:34.751089+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751089+0000","last_undegraded":"2024-04-25T08:36:34.751089+0000","last_fullsized":"2024-04-25T08:36:34.751089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783720+0000","last_change":"2024-04-25T08:36:34.753588+0000","last_active":"2024-04-25T08:36:35.783720+0000","last_peered":"2024-04-25T08:36:35.783720+0000","last_clean":"2024-04-25T08:36:35.783720+0000","last_became_active":"2024-04-25T08:36:34.753290+0000","last_became_peered":"2024-04-25T08:36:34.753290+0000","last_unstale":"2024-04-25T08:36:35.783720+0000","last_undegraded":"2024-04-25T08:36:35.783720+0000","last_fullsized":"2024-04-25T08:36:35.783720+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748432+0000","last_change":"2024-04-25T08:36:34.748432+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748432+0000","last_undegraded":"2024-04-25T08:36:34.748432+0000","last_fullsized":"2024-04-25T08:36:34.748432+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.786237+0000","last_change":"2024-04-25T08:36:34.750641+0000","last_active":"2024-04-25T08:36:35.786237+0000","last_peered":"2024-04-25T08:36:35.786237+0000","last_clean":"2024-04-25T08:36:35.786237+0000","last_became_active":"2024-04-25T08:36:34.748802+0000","last_became_peered":"2024-04-25T08:36:34.748802+0000","last_unstale":"2024-04-25T08:36:35.786237+0000","last_undegraded":"2024-04-25T08:36:35.786237+0000","last_fullsized":"2024-04-25T08:36:35.786237+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:39.476164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.773615+0000","last_change":"2024-04-25T08:36:35.768603+0000","last_active":"2024-04-25T08:36:35.773615+0000","last_peered":"2024-04-25T08:36:35.773615+0000","last_clean":"2024-04-25T08:36:35.773615+0000","last_became_active":"2024-04-25T08:36:35.766863+0000","last_became_peered":"2024-04-25T08:36:35.766863+0000","last_unstale":"2024-04-25T08:36:35.773615+0000","last_undegraded":"2024-04-25T08:36:35.773615+0000","last_fullsized":"2024-04-25T08:36:35.773615+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762382+0000","last_change":"2024-04-25T08:36:34.761126+0000","last_active":"2024-04-25T08:36:34.762382+0000","last_peered":"2024-04-25T08:36:34.762382+0000","last_clean":"2024-04-25T08:36:34.762382+0000","last_became_active":"2024-04-25T08:36:34.760848+0000","last_became_peered":"2024-04-25T08:36:34.760848+0000","last_unstale":"2024-04-25T08:36:34.762382+0000","last_undegraded":"2024-04-25T08:36:34.762382+0000","last_fullsized":"2024-04-25T08:36:34.762382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770663+0000","last_change":"2024-04-25T08:36:35.769783+0000","last_active":"2024-04-25T08:36:35.770663+0000","last_peered":"2024-04-25T08:36:35.770663+0000","last_clean":"2024-04-25T08:36:35.770663+0000","last_became_active":"2024-04-25T08:36:35.769101+0000","last_became_peered":"2024-04-25T08:36:35.769101+0000","last_unstale":"2024-04-25T08:36:35.770663+0000","last_undegraded":"2024-04-25T08:36:35.770663+0000","last_fullsized":"2024-04-25T08:36:35.770663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757823+0000","last_change":"2024-04-25T08:36:34.756585+0000","last_active":"2024-04-25T08:36:34.757823+0000","last_peered":"2024-04-25T08:36:34.757823+0000","last_clean":"2024-04-25T08:36:34.757823+0000","last_became_active":"2024-04-25T08:36:34.756301+0000","last_became_peered":"2024-04-25T08:36:34.756301+0000","last_unstale":"2024-04-25T08:36:34.757823+0000","last_undegraded":"2024-04-25T08:36:34.757823+0000","last_fullsized":"2024-04-25T08:36:34.757823+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763088+0000","last_change":"2024-04-25T08:36:34.761194+0000","last_active":"2024-04-25T08:36:34.763088+0000","last_peered":"2024-04-25T08:36:34.763088+0000","last_clean":"2024-04-25T08:36:34.763088+0000","last_became_active":"2024-04-25T08:36:34.760927+0000","last_became_peered":"2024-04-25T08:36:34.760927+0000","last_unstale":"2024-04-25T08:36:34.763088+0000","last_undegraded":"2024-04-25T08:36:34.763088+0000","last_fullsized":"2024-04-25T08:36:34.763088+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:51.400868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747795+0000","last_change":"2024-04-25T08:36:34.747795+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747795+0000","last_undegraded":"2024-04-25T08:36:34.747795+0000","last_fullsized":"2024-04-25T08:36:34.747795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760876+0000","last_change":"2024-04-25T08:36:34.759616+0000","last_active":"2024-04-25T08:36:34.760876+0000","last_peered":"2024-04-25T08:36:34.760876+0000","last_clean":"2024-04-25T08:36:34.760876+0000","last_became_active":"2024-04-25T08:36:34.759260+0000","last_became_peered":"2024-04-25T08:36:34.759260+0000","last_unstale":"2024-04-25T08:36:34.760876+0000","last_undegraded":"2024-04-25T08:36:34.760876+0000","last_fullsized":"2024-04-25T08:36:34.760876+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752657+0000","last_change":"2024-04-25T08:36:34.752657+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752657+0000","last_undegraded":"2024-04-25T08:36:34.752657+0000","last_fullsized":"2024-04-25T08:36:34.752657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.777741+0000","last_change":"2024-04-25T08:36:34.753294+0000","last_active":"2024-04-25T08:36:35.777741+0000","last_peered":"2024-04-25T08:36:35.777741+0000","last_clean":"2024-04-25T08:36:35.777741+0000","last_became_active":"2024-04-25T08:36:34.753016+0000","last_became_peered":"2024-04-25T08:36:34.753016+0000","last_unstale":"2024-04-25T08:36:35.777741+0000","last_undegraded":"2024-04-25T08:36:35.777741+0000","last_fullsized":"2024-04-25T08:36:35.777741+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770521+0000","last_change":"2024-04-25T08:36:35.767817+0000","last_active":"2024-04-25T08:36:35.770521+0000","last_peered":"2024-04-25T08:36:35.770521+0000","last_clean":"2024-04-25T08:36:35.770521+0000","last_became_active":"2024-04-25T08:36:35.767517+0000","last_became_peered":"2024-04-25T08:36:35.767517+0000","last_unstale":"2024-04-25T08:36:35.770521+0000","last_undegraded":"2024-04-25T08:36:35.770521+0000","last_fullsized":"2024-04-25T08:36:35.770521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.784098+0000","last_change":"2024-04-25T08:36:34.759604+0000","last_active":"2024-04-25T08:36:35.784098+0000","last_peered":"2024-04-25T08:36:35.784098+0000","last_clean":"2024-04-25T08:36:35.784098+0000","last_became_active":"2024-04-25T08:36:34.759303+0000","last_became_peered":"2024-04-25T08:36:34.759303+0000","last_unstale":"2024-04-25T08:36:35.784098+0000","last_undegraded":"2024-04-25T08:36:35.784098+0000","last_fullsized":"2024-04-25T08:36:35.784098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.258277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748937+0000","last_change":"2024-04-25T08:36:34.748937+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748937+0000","last_undegraded":"2024-04-25T08:36:34.748937+0000","last_fullsized":"2024-04-25T08:36:34.748937+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760131+0000","last_change":"2024-04-25T08:36:34.757404+0000","last_active":"2024-04-25T08:36:35.760131+0000","last_peered":"2024-04-25T08:36:35.760131+0000","last_clean":"2024-04-25T08:36:35.760131+0000","last_became_active":"2024-04-25T08:36:34.755586+0000","last_became_peered":"2024-04-25T08:36:34.755586+0000","last_unstale":"2024-04-25T08:36:35.760131+0000","last_undegraded":"2024-04-25T08:36:35.760131+0000","last_fullsized":"2024-04-25T08:36:35.760131+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747259+0000","last_change":"2024-04-25T08:36:34.747259+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747259+0000","last_undegraded":"2024-04-25T08:36:34.747259+0000","last_fullsized":"2024-04-25T08:36:34.747259+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.755095+0000","last_change":"2024-04-25T08:36:34.749438+0000","last_active":"2024-04-25T08:36:34.755095+0000","last_peered":"2024-04-25T08:36:34.755095+0000","last_clean":"2024-04-25T08:36:34.755095+0000","last_became_active":"2024-04-25T08:36:34.749073+0000","last_became_peered":"2024-04-25T08:36:34.749073+0000","last_unstale":"2024-04-25T08:36:34.755095+0000","last_undegraded":"2024-04-25T08:36:34.755095+0000","last_fullsized":"2024-04-25T08:36:34.755095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:25.045409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747760+0000","last_change":"2024-04-25T08:36:34.747760+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747760+0000","last_undegraded":"2024-04-25T08:36:34.747760+0000","last_fullsized":"2024-04-25T08:36:34.747760+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762766+0000","last_change":"2024-04-25T08:36:34.755127+0000","last_active":"2024-04-25T08:36:34.762766+0000","last_peered":"2024-04-25T08:36:34.762766+0000","last_clean":"2024-04-25T08:36:34.762766+0000","last_became_active":"2024-04-25T08:36:34.754781+0000","last_became_peered":"2024-04-25T08:36:34.754781+0000","last_unstale":"2024-04-25T08:36:34.762766+0000","last_undegraded":"2024-04-25T08:36:34.762766+0000","last_fullsized":"2024-04-25T08:36:34.762766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:40.479220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748664+0000","last_change":"2024-04-25T08:36:34.748664+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748664+0000","last_undegraded":"2024-04-25T08:36:34.748664+0000","last_fullsized":"2024-04-25T08:36:34.748664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753639+0000","last_change":"2024-04-25T08:36:34.756454+0000","last_active":"2024-04-25T08:36:35.753639+0000","last_peered":"2024-04-25T08:36:35.753639+0000","last_clean":"2024-04-25T08:36:35.753639+0000","last_became_active":"2024-04-25T08:36:34.755238+0000","last_became_peered":"2024-04-25T08:36:34.755238+0000","last_unstale":"2024-04-25T08:36:35.753639+0000","last_undegraded":"2024-04-25T08:36:35.753639+0000","last_fullsized":"2024-04-25T08:36:35.753639+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762995+0000","last_change":"2024-04-25T08:36:34.760255+0000","last_active":"2024-04-25T08:36:34.762995+0000","last_peered":"2024-04-25T08:36:34.762995+0000","last_clean":"2024-04-25T08:36:34.762995+0000","last_became_active":"2024-04-25T08:36:34.759634+0000","last_became_peered":"2024-04-25T08:36:34.759634+0000","last_unstale":"2024-04-25T08:36:34.762995+0000","last_undegraded":"2024-04-25T08:36:34.762995+0000","last_fullsized":"2024-04-25T08:36:34.762995+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:54.192240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751714+0000","last_change":"2024-04-25T08:36:34.751714+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751714+0000","last_undegraded":"2024-04-25T08:36:34.751714+0000","last_fullsized":"2024-04-25T08:36:34.751714+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762154+0000","last_change":"2024-04-25T08:36:34.760831+0000","last_active":"2024-04-25T08:36:34.762154+0000","last_peered":"2024-04-25T08:36:34.762154+0000","last_clean":"2024-04-25T08:36:34.762154+0000","last_became_active":"2024-04-25T08:36:34.760558+0000","last_became_peered":"2024-04-25T08:36:34.760558+0000","last_unstale":"2024-04-25T08:36:34.762154+0000","last_undegraded":"2024-04-25T08:36:34.762154+0000","last_fullsized":"2024-04-25T08:36:34.762154+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744146+0000","last_change":"2024-04-25T08:36:34.744146+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744146+0000","last_undegraded":"2024-04-25T08:36:34.744146+0000","last_fullsized":"2024-04-25T08:36:34.744146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.758450+0000","last_change":"2024-04-25T08:36:34.756998+0000","last_active":"2024-04-25T08:36:34.758450+0000","last_peered":"2024-04-25T08:36:34.758450+0000","last_clean":"2024-04-25T08:36:34.758450+0000","last_became_active":"2024-04-25T08:36:34.756057+0000","last_became_peered":"2024-04-25T08:36:34.756057+0000","last_unstale":"2024-04-25T08:36:34.758450+0000","last_undegraded":"2024-04-25T08:36:34.758450+0000","last_fullsized":"2024-04-25T08:36:34.758450+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:51.646595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749164+0000","last_change":"2024-04-25T08:36:34.749164+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749164+0000","last_undegraded":"2024-04-25T08:36:34.749164+0000","last_fullsized":"2024-04-25T08:36:34.749164+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.754613+0000","last_change":"2024-04-25T08:36:34.750357+0000","last_active":"2024-04-25T08:36:35.754613+0000","last_peered":"2024-04-25T08:36:35.754613+0000","last_clean":"2024-04-25T08:36:35.754613+0000","last_became_active":"2024-04-25T08:36:34.749684+0000","last_became_peered":"2024-04-25T08:36:34.749684+0000","last_unstale":"2024-04-25T08:36:35.754613+0000","last_undegraded":"2024-04-25T08:36:35.754613+0000","last_fullsized":"2024-04-25T08:36:35.754613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774914+0000","last_change":"2024-04-25T08:36:35.773260+0000","last_active":"2024-04-25T08:36:35.774914+0000","last_peered":"2024-04-25T08:36:35.774914+0000","last_clean":"2024-04-25T08:36:35.774914+0000","last_became_active":"2024-04-25T08:36:35.773012+0000","last_became_peered":"2024-04-25T08:36:35.773012+0000","last_unstale":"2024-04-25T08:36:35.774914+0000","last_undegraded":"2024-04-25T08:36:35.774914+0000","last_fullsized":"2024-04-25T08:36:35.774914+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757258+0000","last_change":"2024-04-25T08:36:34.756400+0000","last_active":"2024-04-25T08:36:34.757258+0000","last_peered":"2024-04-25T08:36:34.757258+0000","last_clean":"2024-04-25T08:36:34.757258+0000","last_became_active":"2024-04-25T08:36:34.756114+0000","last_became_peered":"2024-04-25T08:36:34.756114+0000","last_unstale":"2024-04-25T08:36:34.757258+0000","last_undegraded":"2024-04-25T08:36:34.757258+0000","last_fullsized":"2024-04-25T08:36:34.757258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752972+0000","last_change":"2024-04-25T08:36:34.752972+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752972+0000","last_undegraded":"2024-04-25T08:36:34.752972+0000","last_fullsized":"2024-04-25T08:36:34.752972+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759854+0000","last_change":"2024-04-25T08:36:34.757608+0000","last_active":"2024-04-25T08:36:34.759854+0000","last_peered":"2024-04-25T08:36:34.759854+0000","last_clean":"2024-04-25T08:36:34.759854+0000","last_became_active":"2024-04-25T08:36:34.756201+0000","last_became_peered":"2024-04-25T08:36:34.756201+0000","last_unstale":"2024-04-25T08:36:34.759854+0000","last_undegraded":"2024-04-25T08:36:34.759854+0000","last_fullsized":"2024-04-25T08:36:34.759854+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:09.132700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754953+0000","last_change":"2024-04-25T08:36:34.754953+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754953+0000","last_undegraded":"2024-04-25T08:36:34.754953+0000","last_fullsized":"2024-04-25T08:36:34.754953+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763179+0000","last_change":"2024-04-25T08:36:34.760614+0000","last_active":"2024-04-25T08:36:34.763179+0000","last_peered":"2024-04-25T08:36:34.763179+0000","last_clean":"2024-04-25T08:36:34.763179+0000","last_became_active":"2024-04-25T08:36:34.760357+0000","last_became_peered":"2024-04-25T08:36:34.760357+0000","last_unstale":"2024-04-25T08:36:34.763179+0000","last_undegraded":"2024-04-25T08:36:34.763179+0000","last_fullsized":"2024-04-25T08:36:34.763179+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:48.096031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768321+0000","last_change":"2024-04-25T08:36:35.766389+0000","last_active":"2024-04-25T08:36:35.768321+0000","last_peered":"2024-04-25T08:36:35.768321+0000","last_clean":"2024-04-25T08:36:35.768321+0000","last_became_active":"2024-04-25T08:36:35.766116+0000","last_became_peered":"2024-04-25T08:36:35.766116+0000","last_unstale":"2024-04-25T08:36:35.768321+0000","last_undegraded":"2024-04-25T08:36:35.768321+0000","last_fullsized":"2024-04-25T08:36:35.768321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753757+0000","last_change":"2024-04-25T08:36:34.754257+0000","last_active":"2024-04-25T08:36:35.753757+0000","last_peered":"2024-04-25T08:36:35.753757+0000","last_clean":"2024-04-25T08:36:35.753757+0000","last_became_active":"2024-04-25T08:36:34.752327+0000","last_became_peered":"2024-04-25T08:36:34.752327+0000","last_unstale":"2024-04-25T08:36:35.753757+0000","last_undegraded":"2024-04-25T08:36:35.753757+0000","last_fullsized":"2024-04-25T08:36:35.753757+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774411+0000","last_change":"2024-04-25T08:36:35.770356+0000","last_active":"2024-04-25T08:36:35.774411+0000","last_peered":"2024-04-25T08:36:35.774411+0000","last_clean":"2024-04-25T08:36:35.774411+0000","last_became_active":"2024-04-25T08:36:35.769027+0000","last_became_peered":"2024-04-25T08:36:35.769027+0000","last_unstale":"2024-04-25T08:36:35.774411+0000","last_undegraded":"2024-04-25T08:36:35.774411+0000","last_fullsized":"2024-04-25T08:36:35.774411+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763309+0000","last_change":"2024-04-25T08:36:34.762590+0000","last_active":"2024-04-25T08:36:34.763309+0000","last_peered":"2024-04-25T08:36:34.763309+0000","last_clean":"2024-04-25T08:36:34.763309+0000","last_became_active":"2024-04-25T08:36:34.762255+0000","last_became_peered":"2024-04-25T08:36:34.762255+0000","last_unstale":"2024-04-25T08:36:34.763309+0000","last_undegraded":"2024-04-25T08:36:34.763309+0000","last_fullsized":"2024-04-25T08:36:34.763309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:13.862606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774363+0000","last_change":"2024-04-25T08:36:35.769174+0000","last_active":"2024-04-25T08:36:35.774363+0000","last_peered":"2024-04-25T08:36:35.774363+0000","last_clean":"2024-04-25T08:36:35.774363+0000","last_became_active":"2024-04-25T08:36:35.768769+0000","last_became_peered":"2024-04-25T08:36:35.768769+0000","last_unstale":"2024-04-25T08:36:35.774363+0000","last_undegraded":"2024-04-25T08:36:35.774363+0000","last_fullsized":"2024-04-25T08:36:35.774363+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762841+0000","last_change":"2024-04-25T08:36:34.755845+0000","last_active":"2024-04-25T08:36:34.762841+0000","last_peered":"2024-04-25T08:36:34.762841+0000","last_clean":"2024-04-25T08:36:34.762841+0000","last_became_active":"2024-04-25T08:36:34.755540+0000","last_became_peered":"2024-04-25T08:36:34.755540+0000","last_unstale":"2024-04-25T08:36:34.762841+0000","last_undegraded":"2024-04-25T08:36:34.762841+0000","last_fullsized":"2024-04-25T08:36:34.762841+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:02.715530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760141+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:36:35.760141+0000","last_peered":"2024-04-25T08:36:35.760141+0000","last_clean":"2024-04-25T08:36:35.760141+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:36:35.760141+0000","last_undegraded":"2024-04-25T08:36:35.760141+0000","last_fullsized":"2024-04-25T08:36:35.760141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1949696,"data_stored":1216579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":1368064,"data_stored":628267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40824,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331016,"statfs":{"total":96636764160,"available":96594960384,"internally_reserved":0,"allocated":1347584,"data_stored":626051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54688,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317152,"statfs":{"total":96636764160,"available":96580763648,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54696,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317144,"statfs":{"total":96636764160,"available":96580755456,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1916928,"data_stored":1202585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:39.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.750939+0000', 'last_change': '2024-04-25T08:36:34.750939+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.750939+0000', 'last_undegraded': '2024-04-25T08:36:34.750939+0000', 'last_fullsized': '2024-04-25T08:36:34.750939+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749594+0000', 'last_change': '2024-04-25T08:36:34.749594+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749594+0000', 'last_undegraded': '2024-04-25T08:36:34.749594+0000', 'last_fullsized': '2024-04-25T08:36:34.749594+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.752769+0000', 'last_change': '2024-04-25T08:36:34.752769+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.752769+0000', 'last_undegraded': '2024-04-25T08:36:34.752769+0000', 'last_fullsized': '2024-04-25T08:36:34.752769+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749401+0000', 'last_change': '2024-04-25T08:36:34.749401+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749401+0000', 'last_undegraded': '2024-04-25T08:36:34.749401+0000', 'last_fullsized': '2024-04-25T08:36:34.749401+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.756279+0000', 'last_change': '2024-04-25T08:36:34.756279+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.756279+0000', 'last_undegraded': '2024-04-25T08:36:34.756279+0000', 'last_fullsized': '2024-04-25T08:36:34.756279+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.750944+0000', 'last_change': '2024-04-25T08:36:34.750944+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.750944+0000', 'last_undegraded': '2024-04-25T08:36:34.750944+0000', 'last_fullsized': '2024-04-25T08:36:34.750944+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.744825+0000', 'last_change': '2024-04-25T08:36:34.744825+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.744825+0000', 'last_undegraded': '2024-04-25T08:36:34.744825+0000', 'last_fullsized': '2024-04-25T08:36:34.744825+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749875+0000', 'last_change': '2024-04-25T08:36:34.749875+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749875+0000', 'last_undegraded': '2024-04-25T08:36:34.749875+0000', 'last_fullsized': '2024-04-25T08:36:34.749875+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749979+0000', 'last_change': '2024-04-25T08:36:34.749979+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749979+0000', 'last_undegraded': '2024-04-25T08:36:34.749979+0000', 'last_fullsized': '2024-04-25T08:36:34.749979+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.748333+0000', 'last_change': '2024-04-25T08:36:34.748333+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.748333+0000', 'last_undegraded': '2024-04-25T08:36:34.748333+0000', 'last_fullsized': '2024-04-25T08:36:34.748333+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2024-04-25T08:36:39.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.744421+0000', 'last_change': '2024-04-25T08:36:34.744421+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.744421+0000', 'last_undegraded': '2024-04-25T08:36:34.744421+0000', 'last_fullsized': '2024-04-25T08:36:34.744421+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.754852+0000', 'last_change': '2024-04-25T08:36:34.754852+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.754852+0000', 'last_undegraded': '2024-04-25T08:36:34.754852+0000', 'last_fullsized': '2024-04-25T08:36:34.754852+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.751587+0000', 'last_change': '2024-04-25T08:36:34.751587+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.751587+0000', 'last_undegraded': '2024-04-25T08:36:34.751587+0000', 'last_fullsized': '2024-04-25T08:36:34.751587+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747496+0000', 'last_change': '2024-04-25T08:36:34.747496+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747496+0000', 'last_undegraded': '2024-04-25T08:36:34.747496+0000', 'last_fullsized': '2024-04-25T08:36:34.747496+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749242+0000', 'last_change': '2024-04-25T08:36:34.749242+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749242+0000', 'last_undegraded': '2024-04-25T08:36:34.749242+0000', 'last_fullsized': '2024-04-25T08:36:34.749242+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.746069+0000', 'last_change': '2024-04-25T08:36:34.746069+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.746069+0000', 'last_undegraded': '2024-04-25T08:36:34.746069+0000', 'last_fullsized': '2024-04-25T08:36:34.746069+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2024-04-25T08:36:39.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.745098+0000', 'last_change': '2024-04-25T08:36:34.745098+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.745098+0000', 'last_undegraded': '2024-04-25T08:36:34.745098+0000', 'last_fullsized': '2024-04-25T08:36:34.745098+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.754375+0000', 'last_change': '2024-04-25T08:36:34.754375+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.754375+0000', 'last_undegraded': '2024-04-25T08:36:34.754375+0000', 'last_fullsized': '2024-04-25T08:36:34.754375+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.744739+0000', 'last_change': '2024-04-25T08:36:34.744739+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.744739+0000', 'last_undegraded': '2024-04-25T08:36:34.744739+0000', 'last_fullsized': '2024-04-25T08:36:34.744739+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.754189+0000', 'last_change': '2024-04-25T08:36:34.754189+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.754189+0000', 'last_undegraded': '2024-04-25T08:36:34.754189+0000', 'last_fullsized': '2024-04-25T08:36:34.754189+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.748276+0000', 'last_change': '2024-04-25T08:36:34.748276+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.748276+0000', 'last_undegraded': '2024-04-25T08:36:34.748276+0000', 'last_fullsized': '2024-04-25T08:36:34.748276+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-04-25T08:36:39.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.750505+0000', 'last_change': '2024-04-25T08:36:34.750505+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.750505+0000', 'last_undegraded': '2024-04-25T08:36:34.750505+0000', 'last_fullsized': '2024-04-25T08:36:34.750505+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.744294+0000', 'last_change': '2024-04-25T08:36:34.744294+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.744294+0000', 'last_undegraded': '2024-04-25T08:36:34.744294+0000', 'last_fullsized': '2024-04-25T08:36:34.744294+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747266+0000', 'last_change': '2024-04-25T08:36:34.747266+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747266+0000', 'last_undegraded': '2024-04-25T08:36:34.747266+0000', 'last_fullsized': '2024-04-25T08:36:34.747266+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747253+0000', 'last_change': '2024-04-25T08:36:34.747253+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747253+0000', 'last_undegraded': '2024-04-25T08:36:34.747253+0000', 'last_fullsized': '2024-04-25T08:36:34.747253+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747326+0000', 'last_change': '2024-04-25T08:36:34.747326+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747326+0000', 'last_undegraded': '2024-04-25T08:36:34.747326+0000', 'last_fullsized': '2024-04-25T08:36:34.747326+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.750496+0000', 'last_change': '2024-04-25T08:36:34.750496+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.750496+0000', 'last_undegraded': '2024-04-25T08:36:34.750496+0000', 'last_fullsized': '2024-04-25T08:36:34.750496+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749803+0000', 'last_change': '2024-04-25T08:36:34.749803+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749803+0000', 'last_undegraded': '2024-04-25T08:36:34.749803+0000', 'last_fullsized': '2024-04-25T08:36:34.749803+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.751089+0000', 'last_change': '2024-04-25T08:36:34.751089+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.751089+0000', 'last_undegraded': '2024-04-25T08:36:34.751089+0000', 'last_fullsized': '2024-04-25T08:36:34.751089+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.748432+0000', 'last_change': '2024-04-25T08:36:34.748432+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.748432+0000', 'last_undegraded': '2024-04-25T08:36:34.748432+0000', 'last_fullsized': '2024-04-25T08:36:34.748432+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.752366+0000', 'last_change': '2024-04-25T08:36:34.752366+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.752366+0000', 'last_undegraded': '2024-04-25T08:36:34.752366+0000', 'last_fullsized': '2024-04-25T08:36:34.752366+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747795+0000', 'last_change': '2024-04-25T08:36:34.747795+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747795+0000', 'last_undegraded': '2024-04-25T08:36:34.747795+0000', 'last_fullsized': '2024-04-25T08:36:34.747795+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.752657+0000', 'last_change': '2024-04-25T08:36:34.752657+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.752657+0000', 'last_undegraded': '2024-04-25T08:36:34.752657+0000', 'last_fullsized': '2024-04-25T08:36:34.752657+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2024-04-25T08:36:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.748937+0000', 'last_change': '2024-04-25T08:36:34.748937+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.748937+0000', 'last_undegraded': '2024-04-25T08:36:34.748937+0000', 'last_fullsized': '2024-04-25T08:36:34.748937+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747259+0000', 'last_change': '2024-04-25T08:36:34.747259+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747259+0000', 'last_undegraded': '2024-04-25T08:36:34.747259+0000', 'last_fullsized': '2024-04-25T08:36:34.747259+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.747760+0000', 'last_change': '2024-04-25T08:36:34.747760+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.747760+0000', 'last_undegraded': '2024-04-25T08:36:34.747760+0000', 'last_fullsized': '2024-04-25T08:36:34.747760+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.748664+0000', 'last_change': '2024-04-25T08:36:34.748664+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.748664+0000', 'last_undegraded': '2024-04-25T08:36:34.748664+0000', 'last_fullsized': '2024-04-25T08:36:34.748664+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.752366+0000', 'last_change': '2024-04-25T08:36:34.752366+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.752366+0000', 'last_undegraded': '2024-04-25T08:36:34.752366+0000', 'last_fullsized': '2024-04-25T08:36:34.752366+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.751714+0000', 'last_change': '2024-04-25T08:36:34.751714+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.751714+0000', 'last_undegraded': '2024-04-25T08:36:34.751714+0000', 'last_fullsized': '2024-04-25T08:36:34.751714+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:39.473 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.744146+0000', 'last_change': '2024-04-25T08:36:34.744146+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.744146+0000', 'last_undegraded': '2024-04-25T08:36:34.744146+0000', 'last_fullsized': '2024-04-25T08:36:34.744146+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.749164+0000', 'last_change': '2024-04-25T08:36:34.749164+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.749164+0000', 'last_undegraded': '2024-04-25T08:36:34.749164+0000', 'last_fullsized': '2024-04-25T08:36:34.749164+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.752972+0000', 'last_change': '2024-04-25T08:36:34.752972+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.752972+0000', 'last_undegraded': '2024-04-25T08:36:34.752972+0000', 'last_fullsized': '2024-04-25T08:36:34.752972+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-04-25T08:36:39.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 108, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:34.754953+0000', 'last_change': '2024-04-25T08:36:34.754953+0000', 'last_active': '2024-04-25T08:36:34.737374+0000', 'last_peered': '2024-04-25T08:36:34.737374+0000', 'last_clean': '2024-04-25T08:36:34.737374+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:34.754953+0000', 'last_undegraded': '2024-04-25T08:36:34.754953+0000', 'last_fullsized': '2024-04-25T08:36:34.754953+0000', 'mapping_epoch': 108, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 108, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:34.737374+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:39.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.608+0000 7f9c04134640 1 Processor -- start 2024-04-25T08:36:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.608+0000 7f9c04134640 1 -- start start 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c04134640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc15db70 0x7f9bfc162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c04134640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9bfc0b5c30 0x7f9bfc0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c04134640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc162690 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c04134640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0ceee0 con 0x7f9bfc0b5c30 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c04134640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0d1d10 con 0x7f9bfc15db70 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c016cf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9bfc0b6cb0 0x7f9bfc0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35704/0 (socket says 172.21.15.60:35704) 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c016cf640 1 -- 172.21.15.60:0/1687880502 learned_addr learned my addr 172.21.15.60:0/1687880502 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c01ed0640 1 --2- 172.21.15.60:0/1687880502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc15db70 0x7f9bfc162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.609+0000 7f9c026d1640 1 --2- 172.21.15.60:0/1687880502 >> v2:172.21.15.100:3300/0 conn(0x7f9bfc0b5c30 0x7f9bfc0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c00ece640 1 -- 172.21.15.60:0/1687880502 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167558719 0 0) 0x7f9bfc162690 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c00ece640 1 -- 172.21.15.60:0/1687880502 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be4003520 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c01ed0640 1 -- 172.21.15.60:0/1687880502 >> v2:172.21.15.100:3300/0 conn(0x7f9bfc0b5c30 msgr2=0x7f9bfc0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c01ed0640 1 --2- 172.21.15.60:0/1687880502 >> v2:172.21.15.100:3300/0 conn(0x7f9bfc0b5c30 0x7f9bfc0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c01ed0640 1 -- 172.21.15.60:0/1687880502 >> v1:172.21.15.60:6789/0 conn(0x7f9bfc0b6cb0 legacy=0x7f9bfc0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c00ece640 1 -- 172.21.15.60:0/1687880502 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3846964371 0 0) 0x7f9be4003520 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.610+0000 7f9c01ed0640 1 -- 172.21.15.60:0/1687880502 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0d1ff0 con 0x7f9bfc15db70 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c01ed0640 1 --2- 172.21.15.60:0/1687880502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc15db70 0x7f9bfc162150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009d90 tx=0x7f9bec02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=341fc234518fb971 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c00ece640 1 -- 172.21.15.60:0/1687880502 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0001670 con 0x7f9bfc15db70 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c00ece640 1 -- 172.21.15.60:0/1687880502 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec007540 con 0x7f9bfc15db70 2024-04-25T08:36:39.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c00ece640 1 -- 172.21.15.60:0/1687880502 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0078f0 con 0x7f9bfc15db70 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 -- 172.21.15.60:0/1687880502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc15db70 msgr2=0x7f9bfc162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 --2- 172.21.15.60:0/1687880502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc15db70 0x7f9bfc162150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009d90 tx=0x7f9bec02a560 comp rx=0 tx=0).stop 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 -- 172.21.15.60:0/1687880502 shutdown_connections 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 --2- 172.21.15.60:0/1687880502 >> v2:172.21.15.100:3300/0 conn(0x7f9bfc0b5c30 0x7f9bfc0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 --2- 172.21.15.60:0/1687880502 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc15db70 0x7f9bfc162150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 -- 172.21.15.60:0/1687880502 >> 172.21.15.60:0/1687880502 conn(0x7f9bfc0d0520 msgr2=0x7f9bfc0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 -- 172.21.15.60:0/1687880502 shutdown_connections 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.611+0000 7f9c04134640 1 -- 172.21.15.60:0/1687880502 wait complete. 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.612+0000 7f9c04134640 1 Processor -- start 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 -- start start 2024-04-25T08:36:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc0b5c30 0x7f9bfc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 0x7f9bfc0cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bfc15db70 0x7f9bfc15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0ce530 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc15df50 con 0x7f9bfc15db70 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c04134640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc0cf1c0 con 0x7f9bfc0b5c30 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c016cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 0x7f9bfc0cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c016cf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 0x7f9bfc0cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55022/0 (socket says 172.21.15.60:55022) 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c016cf640 1 -- 172.21.15.60:0/2686903999 learned_addr learned my addr 172.21.15.60:0/2686903999 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c01ed0640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc0b5c30 0x7f9bfc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.613+0000 7f9c026d1640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bfc15db70 0x7f9bfc15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c016cf640 1 -- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc0b5c30 msgr2=0x7f9bfc0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c016cf640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc0b5c30 0x7f9bfc0caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c016cf640 1 -- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bfc15db70 msgr2=0x7f9bfc15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c016cf640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bfc15db70 0x7f9bfc15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c016cf640 1 -- 172.21.15.60:0/2686903999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0cdb60 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c016cf640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 0x7f9bfc0cd620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c750 tx=0x7f9bf000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0017070 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0005ad0 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0007070 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc1f3a40 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.614+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc1f4000 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.615+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc073990 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.615+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bf001a070 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.616+0000 7f9bfa7fc640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9be806b4d0 0x7f9be806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.616+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9bf0089760 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.616+0000 7f9c01ed0640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9be806b4d0 0x7f9be806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.617+0000 7f9c01ed0640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9be806b4d0 0x7f9be806d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9bec034000 tx=0x7f9bec02f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.620+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bf0057e00 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.870+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9bfc0d1820 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.870+0000 7f9bfa7fc640 1 -- 172.21.15.60:0/2686903999 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1649 (secure 0 0 0) 0x7f9bf005bea0 con 0x7f9bfc0b6cb0 2024-04-25T08:36:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:39.865 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":460,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":54},{"state_name":"creating+peering","count":43}],"num_pgs":97,"num_pools":3,"num_objects":10,"data_bytes":592162,"bytes_used":403824640,"bytes_avail":772690288640,"bytes_total":773094113280,"inactive_pgs_ratio":0.44329896569252014,"write_bytes_sec":1066,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9be806b4d0 msgr2=0x7f9be806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9be806b4d0 0x7f9be806d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9bec034000 tx=0x7f9bec02f000 comp rx=0 tx=0).stop 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 msgr2=0x7f9bfc0cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 0x7f9bfc0cd620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c750 tx=0x7f9bf000cc10 comp rx=0 tx=0).stop 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 shutdown_connections 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9be806b4d0 0x7f9be806d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bfc15db70 0x7f9bfc15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9bfc0b6cb0 0x7f9bfc0cd620 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 --2- 172.21.15.60:0/2686903999 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9bfc0b5c30 0x7f9bfc0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 >> 172.21.15.60:0/2686903999 conn(0x7f9bfc0d0520 msgr2=0x7f9bfc0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.872+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 shutdown_connections 2024-04-25T08:36:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.873+0000 7f9c04134640 1 -- 172.21.15.60:0/2686903999 wait complete. 2024-04-25T08:36:39.878 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:36:39.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:39.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.985+0000 7fadd08c4640 1 Processor -- start 2024-04-25T08:36:39.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.985+0000 7fadd08c4640 1 -- start start 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadd08c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 0x7fadcc0d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadd08c4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0d20e0 con 0x7fadcc0c7310 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadd08c4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0c7970 con 0x7fadcc0d1ca0 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadd08c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0c7730 con 0x7fadcc15ef00 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadcad9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fadcc0c7310 0x7fadcc0cc270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:35712/0 (socket says 172.21.15.60:35712) 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadcad9d640 1 -- 172.21.15.60:0/210973909 learned_addr learned my addr 172.21.15.60:0/210973909 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadca59c640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 0x7fadcc0d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762271998 0 0) 0x7fadcc0d20e0 con 0x7fadcc0c7310 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb8003520 con 0x7fadcc0c7310 2024-04-25T08:36:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3561485421 0 0) 0x7fadb8003520 con 0x7fadcc0c7310 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 msgr2=0x7fadcc0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 0x7fadcc0d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 >> v1:172.21.15.100:6789/0 conn(0x7fadcc0d1ca0 legacy=0x7fadcc0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0c7c50 con 0x7fadcc0c7310 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.986+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fadc40019f0 con 0x7fadcc0c7310 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 >> v1:172.21.15.60:6789/0 conn(0x7fadcc0c7310 legacy=0x7fadcc0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb80039e0 0x7fadb8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 0x7fadb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb8004d40 0x7fadb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0c7730 con 0x7fadb8004d40 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb800fdd0 con 0x7fadb80039e0 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb800ffd0 con 0x7fadcc15ef00 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadcad9d640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb8004d40 0x7fadb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadca59c640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb80039e0 0x7fadb8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadc3fff640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 0x7fadb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadcad9d640 1 -- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 msgr2=0x7fadb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadcad9d640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc15ef00 0x7fadb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadcad9d640 1 -- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb80039e0 msgr2=0x7fadb8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadcad9d640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb80039e0 0x7fadb8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.987+0000 7fadcad9d640 1 -- 172.21.15.60:0/210973909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8003520 con 0x7fadb8004d40 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadcad9d640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb8004d40 0x7fadb80051c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fadc400a810 tx=0x7fadb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a8173f287fc040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc400a430 con 0x7fadb8004d40 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc402f070 con 0x7fadb8004d40 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadc9d9b640 1 -- 172.21.15.60:0/210973909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc402bc50 con 0x7fadb8004d40 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadd08c4640 1 -- 172.21.15.60:0/210973909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb8004d40 msgr2=0x7fadb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadd08c4640 1 --2- 172.21.15.60:0/210973909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb8004d40 0x7fadb80051c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fadc400a810 tx=0x7fadb8005830 comp rx=0 tx=0).stop 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadcad9d640 1 -- 172.21.15.60:0/210973909 reap_dead start 2024-04-25T08:36:39.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadd08c4640 1 -- 172.21.15.60:0/210973909 shutdown_connections 2024-04-25T08:36:39.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadd08c4640 1 -- 172.21.15.60:0/210973909 >> 172.21.15.60:0/210973909 conn(0x7fadcc06f5d0 msgr2=0x7fadcc06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadd08c4640 1 -- 172.21.15.60:0/210973909 shutdown_connections 2024-04-25T08:36:39.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.988+0000 7fadd08c4640 1 -- 172.21.15.60:0/210973909 wait complete. 2024-04-25T08:36:39.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 Processor -- start 2024-04-25T08:36:39.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 -- start start 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb8004d40 0x7fadcc0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 0x7fadcc0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc0c99a0 0x7fadcc0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0c8210 con 0x7fadcc144620 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.990+0000 7fadd08c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc15f2e0 con 0x7fadb8004d40 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadd08c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0c84f0 con 0x7fadcc0c99a0 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 0x7fadcc0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 0x7fadcc0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55036/0 (socket says 172.21.15.60:55036) 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 -- 172.21.15.60:0/1565698769 learned_addr learned my addr 172.21.15.60:0/1565698769 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadcad9d640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc0c99a0 0x7fadcc0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadca59c640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb8004d40 0x7fadcc0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 -- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc0c99a0 msgr2=0x7fadcc0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc0c99a0 0x7fadcc0c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 -- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb8004d40 msgr2=0x7fadcc0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb8004d40 0x7fadcc0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 -- 172.21.15.60:0/1565698769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0c9690 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadca59c640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb8004d40 0x7fadcc0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc3fff640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 0x7fadcc0cadb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fadb400f8a0 tx=0x7fadb400fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb401a070 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4005320 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc14b850 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb40072b0 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.991+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc14be10 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.992+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc073990 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.993+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadb401d070 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.993+0000 7fadc2ffd640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadac06b500 0x7fadac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.993+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadb4089b40 con 0x7fadcc144620 2024-04-25T08:36:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.993+0000 7fadca59c640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadac06b500 0x7fadac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.994+0000 7fadca59c640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadac06b500 0x7fadac06d9c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009bc0 tx=0x7fadbc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:39.997+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadb40588c0 con 0x7fadcc144620 2024-04-25T08:36:40.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.176+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadcc0cb2f0 con 0x7fadac06b500 2024-04-25T08:36:40.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.182+0000 7fadc2ffd640 1 -- 172.21.15.60:0/1565698769 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248853 (secure 0 0 0) 0x7fadbc014bb0 con 0x7fadac06b500 2024-04-25T08:36:40.179 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:40.180 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadac06b500 msgr2=0x7fadac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadac06b500 0x7fadac06d9c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009bc0 tx=0x7fadbc008040 comp rx=0 tx=0).stop 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 msgr2=0x7fadcc0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 0x7fadcc0cadb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fadb400f8a0 tx=0x7fadb400fd60 comp rx=0 tx=0).stop 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 shutdown_connections 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fadcc0c99a0 0x7fadcc0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadcc144620 0x7fadcc0cadb0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:40.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fadac06b500 0x7fadac06d9c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:40.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 --2- 172.21.15.60:0/1565698769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb8004d40 0x7fadcc0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:40.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.184+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 >> 172.21.15.60:0/1565698769 conn(0x7fadcc06f5d0 msgr2=0x7fadcc1595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:40.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.185+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 shutdown_connections 2024-04-25T08:36:40.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:40.185+0000 7fadd08c4640 1 -- 172.21.15.60:0/1565698769 wait complete. 2024-04-25T08:36:40.190 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-25T08:36:39.498113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394360,"kb_used_data":11644,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754580360,"statfs":{"total":773094113280,"available":772690288640,"internally_reserved":0,"allocated":11923456,"data_stored":6149018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759146"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750939+0000","last_change":"2024-04-25T08:36:34.750939+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750939+0000","last_undegraded":"2024-04-25T08:36:34.750939+0000","last_fullsized":"2024-04-25T08:36:34.750939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766771+0000","last_change":"2024-04-25T08:36:35.765297+0000","last_active":"2024-04-25T08:36:35.766771+0000","last_peered":"2024-04-25T08:36:35.766771+0000","last_clean":"2024-04-25T08:36:35.766771+0000","last_became_active":"2024-04-25T08:36:35.763995+0000","last_became_peered":"2024-04-25T08:36:35.763995+0000","last_unstale":"2024-04-25T08:36:35.766771+0000","last_undegraded":"2024-04-25T08:36:35.766771+0000","last_fullsized":"2024-04-25T08:36:35.766771+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749594+0000","last_change":"2024-04-25T08:36:34.749594+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749594+0000","last_undegraded":"2024-04-25T08:36:34.749594+0000","last_fullsized":"2024-04-25T08:36:34.749594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752769+0000","last_change":"2024-04-25T08:36:34.752769+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752769+0000","last_undegraded":"2024-04-25T08:36:34.752769+0000","last_fullsized":"2024-04-25T08:36:34.752769+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749401+0000","last_change":"2024-04-25T08:36:34.749401+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749401+0000","last_undegraded":"2024-04-25T08:36:34.749401+0000","last_fullsized":"2024-04-25T08:36:34.749401+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783055+0000","last_change":"2024-04-25T08:36:35.782370+0000","last_active":"2024-04-25T08:36:35.783055+0000","last_peered":"2024-04-25T08:36:35.783055+0000","last_clean":"2024-04-25T08:36:35.783055+0000","last_became_active":"2024-04-25T08:36:35.782129+0000","last_became_peered":"2024-04-25T08:36:35.782129+0000","last_unstale":"2024-04-25T08:36:35.783055+0000","last_undegraded":"2024-04-25T08:36:35.783055+0000","last_fullsized":"2024-04-25T08:36:35.783055+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772945+0000","last_change":"2024-04-25T08:36:35.771864+0000","last_active":"2024-04-25T08:36:35.772945+0000","last_peered":"2024-04-25T08:36:35.772945+0000","last_clean":"2024-04-25T08:36:35.772945+0000","last_became_active":"2024-04-25T08:36:35.771591+0000","last_became_peered":"2024-04-25T08:36:35.771591+0000","last_unstale":"2024-04-25T08:36:35.772945+0000","last_undegraded":"2024-04-25T08:36:35.772945+0000","last_fullsized":"2024-04-25T08:36:35.772945+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.756279+0000","last_change":"2024-04-25T08:36:34.756279+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.756279+0000","last_undegraded":"2024-04-25T08:36:34.756279+0000","last_fullsized":"2024-04-25T08:36:34.756279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770474+0000","last_change":"2024-04-25T08:36:35.768883+0000","last_active":"2024-04-25T08:36:35.770474+0000","last_peered":"2024-04-25T08:36:35.770474+0000","last_clean":"2024-04-25T08:36:35.770474+0000","last_became_active":"2024-04-25T08:36:35.768603+0000","last_became_peered":"2024-04-25T08:36:35.768603+0000","last_unstale":"2024-04-25T08:36:35.770474+0000","last_undegraded":"2024-04-25T08:36:35.770474+0000","last_fullsized":"2024-04-25T08:36:35.770474+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750944+0000","last_change":"2024-04-25T08:36:34.750944+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750944+0000","last_undegraded":"2024-04-25T08:36:34.750944+0000","last_fullsized":"2024-04-25T08:36:34.750944+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774841+0000","last_change":"2024-04-25T08:36:35.772717+0000","last_active":"2024-04-25T08:36:35.774841+0000","last_peered":"2024-04-25T08:36:35.774841+0000","last_clean":"2024-04-25T08:36:35.774841+0000","last_became_active":"2024-04-25T08:36:35.772232+0000","last_became_peered":"2024-04-25T08:36:35.772232+0000","last_unstale":"2024-04-25T08:36:35.774841+0000","last_undegraded":"2024-04-25T08:36:35.774841+0000","last_fullsized":"2024-04-25T08:36:35.774841+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744825+0000","last_change":"2024-04-25T08:36:34.744825+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744825+0000","last_undegraded":"2024-04-25T08:36:34.744825+0000","last_fullsized":"2024-04-25T08:36:34.744825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749875+0000","last_change":"2024-04-25T08:36:34.749875+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749875+0000","last_undegraded":"2024-04-25T08:36:34.749875+0000","last_fullsized":"2024-04-25T08:36:34.749875+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768324+0000","last_change":"2024-04-25T08:36:35.767092+0000","last_active":"2024-04-25T08:36:35.768324+0000","last_peered":"2024-04-25T08:36:35.768324+0000","last_clean":"2024-04-25T08:36:35.768324+0000","last_became_active":"2024-04-25T08:36:35.766827+0000","last_became_peered":"2024-04-25T08:36:35.766827+0000","last_unstale":"2024-04-25T08:36:35.768324+0000","last_undegraded":"2024-04-25T08:36:35.768324+0000","last_fullsized":"2024-04-25T08:36:35.768324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.776596+0000","last_change":"2024-04-25T08:36:35.775011+0000","last_active":"2024-04-25T08:36:35.776596+0000","last_peered":"2024-04-25T08:36:35.776596+0000","last_clean":"2024-04-25T08:36:35.776596+0000","last_became_active":"2024-04-25T08:36:35.774695+0000","last_became_peered":"2024-04-25T08:36:35.774695+0000","last_unstale":"2024-04-25T08:36:35.776596+0000","last_undegraded":"2024-04-25T08:36:35.776596+0000","last_fullsized":"2024-04-25T08:36:35.776596+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.432682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768288+0000","last_change":"2024-04-25T08:36:35.766209+0000","last_active":"2024-04-25T08:36:35.768288+0000","last_peered":"2024-04-25T08:36:35.768288+0000","last_clean":"2024-04-25T08:36:35.768288+0000","last_became_active":"2024-04-25T08:36:35.765932+0000","last_became_peered":"2024-04-25T08:36:35.765932+0000","last_unstale":"2024-04-25T08:36:35.768288+0000","last_undegraded":"2024-04-25T08:36:35.768288+0000","last_fullsized":"2024-04-25T08:36:35.768288+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749979+0000","last_change":"2024-04-25T08:36:34.749979+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749979+0000","last_undegraded":"2024-04-25T08:36:34.749979+0000","last_fullsized":"2024-04-25T08:36:34.749979+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.772836+0000","last_change":"2024-04-25T08:36:35.771768+0000","last_active":"2024-04-25T08:36:35.772836+0000","last_peered":"2024-04-25T08:36:35.772836+0000","last_clean":"2024-04-25T08:36:35.772836+0000","last_became_active":"2024-04-25T08:36:35.771487+0000","last_became_peered":"2024-04-25T08:36:35.771487+0000","last_unstale":"2024-04-25T08:36:35.772836+0000","last_undegraded":"2024-04-25T08:36:35.772836+0000","last_fullsized":"2024-04-25T08:36:35.772836+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748333+0000","last_change":"2024-04-25T08:36:34.748333+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748333+0000","last_undegraded":"2024-04-25T08:36:34.748333+0000","last_fullsized":"2024-04-25T08:36:34.748333+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744421+0000","last_change":"2024-04-25T08:36:34.744421+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744421+0000","last_undegraded":"2024-04-25T08:36:34.744421+0000","last_fullsized":"2024-04-25T08:36:34.744421+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754852+0000","last_change":"2024-04-25T08:36:34.754852+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754852+0000","last_undegraded":"2024-04-25T08:36:34.754852+0000","last_fullsized":"2024-04-25T08:36:34.754852+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751587+0000","last_change":"2024-04-25T08:36:34.751587+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751587+0000","last_undegraded":"2024-04-25T08:36:34.751587+0000","last_fullsized":"2024-04-25T08:36:34.751587+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747496+0000","last_change":"2024-04-25T08:36:34.747496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747496+0000","last_undegraded":"2024-04-25T08:36:34.747496+0000","last_fullsized":"2024-04-25T08:36:34.747496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749242+0000","last_change":"2024-04-25T08:36:34.749242+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749242+0000","last_undegraded":"2024-04-25T08:36:34.749242+0000","last_fullsized":"2024-04-25T08:36:34.749242+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.746069+0000","last_change":"2024-04-25T08:36:34.746069+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.746069+0000","last_undegraded":"2024-04-25T08:36:34.746069+0000","last_fullsized":"2024-04-25T08:36:34.746069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.745098+0000","last_change":"2024-04-25T08:36:34.745098+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.745098+0000","last_undegraded":"2024-04-25T08:36:34.745098+0000","last_fullsized":"2024-04-25T08:36:34.745098+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774306+0000","last_change":"2024-04-25T08:36:35.768841+0000","last_active":"2024-04-25T08:36:35.774306+0000","last_peered":"2024-04-25T08:36:35.774306+0000","last_clean":"2024-04-25T08:36:35.774306+0000","last_became_active":"2024-04-25T08:36:35.768565+0000","last_became_peered":"2024-04-25T08:36:35.768565+0000","last_unstale":"2024-04-25T08:36:35.774306+0000","last_undegraded":"2024-04-25T08:36:35.774306+0000","last_fullsized":"2024-04-25T08:36:35.774306+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:36.331106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.769495+0000","last_change":"2024-04-25T08:36:35.764415+0000","last_active":"2024-04-25T08:36:35.769495+0000","last_peered":"2024-04-25T08:36:35.769495+0000","last_clean":"2024-04-25T08:36:35.769495+0000","last_became_active":"2024-04-25T08:36:35.764127+0000","last_became_peered":"2024-04-25T08:36:35.764127+0000","last_unstale":"2024-04-25T08:36:35.769495+0000","last_undegraded":"2024-04-25T08:36:35.769495+0000","last_fullsized":"2024-04-25T08:36:35.769495+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754375+0000","last_change":"2024-04-25T08:36:34.754375+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754375+0000","last_undegraded":"2024-04-25T08:36:34.754375+0000","last_fullsized":"2024-04-25T08:36:34.754375+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744739+0000","last_change":"2024-04-25T08:36:34.744739+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744739+0000","last_undegraded":"2024-04-25T08:36:34.744739+0000","last_fullsized":"2024-04-25T08:36:34.744739+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770339+0000","last_change":"2024-04-25T08:36:35.767653+0000","last_active":"2024-04-25T08:36:35.770339+0000","last_peered":"2024-04-25T08:36:35.770339+0000","last_clean":"2024-04-25T08:36:35.770339+0000","last_became_active":"2024-04-25T08:36:35.767382+0000","last_became_peered":"2024-04-25T08:36:35.767382+0000","last_unstale":"2024-04-25T08:36:35.770339+0000","last_undegraded":"2024-04-25T08:36:35.770339+0000","last_fullsized":"2024-04-25T08:36:35.770339+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754189+0000","last_change":"2024-04-25T08:36:34.754189+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754189+0000","last_undegraded":"2024-04-25T08:36:34.754189+0000","last_fullsized":"2024-04-25T08:36:34.754189+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748276+0000","last_change":"2024-04-25T08:36:34.748276+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748276+0000","last_undegraded":"2024-04-25T08:36:34.748276+0000","last_fullsized":"2024-04-25T08:36:34.748276+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757104+0000","last_change":"2024-04-25T08:36:34.755252+0000","last_active":"2024-04-25T08:36:34.757104+0000","last_peered":"2024-04-25T08:36:34.757104+0000","last_clean":"2024-04-25T08:36:34.757104+0000","last_became_active":"2024-04-25T08:36:34.754950+0000","last_became_peered":"2024-04-25T08:36:34.754950+0000","last_unstale":"2024-04-25T08:36:34.757104+0000","last_undegraded":"2024-04-25T08:36:34.757104+0000","last_fullsized":"2024-04-25T08:36:34.757104+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750505+0000","last_change":"2024-04-25T08:36:34.750505+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750505+0000","last_undegraded":"2024-04-25T08:36:34.750505+0000","last_fullsized":"2024-04-25T08:36:34.750505+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762949+0000","last_change":"2024-04-25T08:36:34.759886+0000","last_active":"2024-04-25T08:36:34.762949+0000","last_peered":"2024-04-25T08:36:34.762949+0000","last_clean":"2024-04-25T08:36:34.762949+0000","last_became_active":"2024-04-25T08:36:34.759596+0000","last_became_peered":"2024-04-25T08:36:34.759596+0000","last_unstale":"2024-04-25T08:36:34.762949+0000","last_undegraded":"2024-04-25T08:36:34.762949+0000","last_fullsized":"2024-04-25T08:36:34.762949+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:49.092358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744294+0000","last_change":"2024-04-25T08:36:34.744294+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744294+0000","last_undegraded":"2024-04-25T08:36:34.744294+0000","last_fullsized":"2024-04-25T08:36:34.744294+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760269+0000","last_change":"2024-04-25T08:36:34.759233+0000","last_active":"2024-04-25T08:36:34.760269+0000","last_peered":"2024-04-25T08:36:34.760269+0000","last_clean":"2024-04-25T08:36:34.760269+0000","last_became_active":"2024-04-25T08:36:34.758929+0000","last_became_peered":"2024-04-25T08:36:34.758929+0000","last_unstale":"2024-04-25T08:36:34.760269+0000","last_undegraded":"2024-04-25T08:36:34.760269+0000","last_fullsized":"2024-04-25T08:36:34.760269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:08.176620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774869+0000","last_change":"2024-04-25T08:36:35.773004+0000","last_active":"2024-04-25T08:36:35.774869+0000","last_peered":"2024-04-25T08:36:35.774869+0000","last_clean":"2024-04-25T08:36:35.774869+0000","last_became_active":"2024-04-25T08:36:35.772491+0000","last_became_peered":"2024-04-25T08:36:35.772491+0000","last_unstale":"2024-04-25T08:36:35.774869+0000","last_undegraded":"2024-04-25T08:36:35.774869+0000","last_fullsized":"2024-04-25T08:36:35.774869+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762413+0000","last_change":"2024-04-25T08:36:34.761740+0000","last_active":"2024-04-25T08:36:34.762413+0000","last_peered":"2024-04-25T08:36:34.762413+0000","last_clean":"2024-04-25T08:36:34.762413+0000","last_became_active":"2024-04-25T08:36:34.761387+0000","last_became_peered":"2024-04-25T08:36:34.761387+0000","last_unstale":"2024-04-25T08:36:34.762413+0000","last_undegraded":"2024-04-25T08:36:34.762413+0000","last_fullsized":"2024-04-25T08:36:34.762413+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747266+0000","last_change":"2024-04-25T08:36:34.747266+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747266+0000","last_undegraded":"2024-04-25T08:36:34.747266+0000","last_fullsized":"2024-04-25T08:36:34.747266+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783225+0000","last_change":"2024-04-25T08:36:34.752807+0000","last_active":"2024-04-25T08:36:35.783225+0000","last_peered":"2024-04-25T08:36:35.783225+0000","last_clean":"2024-04-25T08:36:35.783225+0000","last_became_active":"2024-04-25T08:36:34.752501+0000","last_became_peered":"2024-04-25T08:36:34.752501+0000","last_unstale":"2024-04-25T08:36:35.783225+0000","last_undegraded":"2024-04-25T08:36:35.783225+0000","last_fullsized":"2024-04-25T08:36:35.783225+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.320853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747253+0000","last_change":"2024-04-25T08:36:34.747253+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747253+0000","last_undegraded":"2024-04-25T08:36:34.747253+0000","last_fullsized":"2024-04-25T08:36:34.747253+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783449+0000","last_change":"2024-04-25T08:36:34.761095+0000","last_active":"2024-04-25T08:36:35.783449+0000","last_peered":"2024-04-25T08:36:35.783449+0000","last_clean":"2024-04-25T08:36:35.783449+0000","last_became_active":"2024-04-25T08:36:34.760735+0000","last_became_peered":"2024-04-25T08:36:34.760735+0000","last_unstale":"2024-04-25T08:36:35.783449+0000","last_undegraded":"2024-04-25T08:36:35.783449+0000","last_fullsized":"2024-04-25T08:36:35.783449+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.766752+0000","last_change":"2024-04-25T08:36:35.764305+0000","last_active":"2024-04-25T08:36:35.766752+0000","last_peered":"2024-04-25T08:36:35.766752+0000","last_clean":"2024-04-25T08:36:35.766752+0000","last_became_active":"2024-04-25T08:36:35.764067+0000","last_became_peered":"2024-04-25T08:36:35.764067+0000","last_unstale":"2024-04-25T08:36:35.766752+0000","last_undegraded":"2024-04-25T08:36:35.766752+0000","last_fullsized":"2024-04-25T08:36:35.766752+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:20.920616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759815+0000","last_change":"2024-04-25T08:36:34.758831+0000","last_active":"2024-04-25T08:36:34.759815+0000","last_peered":"2024-04-25T08:36:34.759815+0000","last_clean":"2024-04-25T08:36:34.759815+0000","last_became_active":"2024-04-25T08:36:34.758382+0000","last_became_peered":"2024-04-25T08:36:34.758382+0000","last_unstale":"2024-04-25T08:36:34.759815+0000","last_undegraded":"2024-04-25T08:36:34.759815+0000","last_fullsized":"2024-04-25T08:36:34.759815+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:06.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747326+0000","last_change":"2024-04-25T08:36:34.747326+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747326+0000","last_undegraded":"2024-04-25T08:36:34.747326+0000","last_fullsized":"2024-04-25T08:36:34.747326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.761034+0000","last_change":"2024-04-25T08:36:34.758227+0000","last_active":"2024-04-25T08:36:34.761034+0000","last_peered":"2024-04-25T08:36:34.761034+0000","last_clean":"2024-04-25T08:36:34.761034+0000","last_became_active":"2024-04-25T08:36:34.757872+0000","last_became_peered":"2024-04-25T08:36:34.757872+0000","last_unstale":"2024-04-25T08:36:34.761034+0000","last_undegraded":"2024-04-25T08:36:34.761034+0000","last_fullsized":"2024-04-25T08:36:34.761034+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.750496+0000","last_change":"2024-04-25T08:36:34.750496+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.750496+0000","last_undegraded":"2024-04-25T08:36:34.750496+0000","last_fullsized":"2024-04-25T08:36:34.750496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760454+0000","last_change":"2024-04-25T08:36:34.759067+0000","last_active":"2024-04-25T08:36:34.760454+0000","last_peered":"2024-04-25T08:36:34.760454+0000","last_clean":"2024-04-25T08:36:34.760454+0000","last_became_active":"2024-04-25T08:36:34.758669+0000","last_became_peered":"2024-04-25T08:36:34.758669+0000","last_unstale":"2024-04-25T08:36:34.760454+0000","last_undegraded":"2024-04-25T08:36:34.760454+0000","last_fullsized":"2024-04-25T08:36:34.760454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:06.312278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749803+0000","last_change":"2024-04-25T08:36:34.749803+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749803+0000","last_undegraded":"2024-04-25T08:36:34.749803+0000","last_fullsized":"2024-04-25T08:36:34.749803+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760997+0000","last_change":"2024-04-25T08:36:34.759931+0000","last_active":"2024-04-25T08:36:34.760997+0000","last_peered":"2024-04-25T08:36:34.760997+0000","last_clean":"2024-04-25T08:36:34.760997+0000","last_became_active":"2024-04-25T08:36:34.759599+0000","last_became_peered":"2024-04-25T08:36:34.759599+0000","last_unstale":"2024-04-25T08:36:34.760997+0000","last_undegraded":"2024-04-25T08:36:34.760997+0000","last_fullsized":"2024-04-25T08:36:34.760997+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:33.870046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751089+0000","last_change":"2024-04-25T08:36:34.751089+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751089+0000","last_undegraded":"2024-04-25T08:36:34.751089+0000","last_fullsized":"2024-04-25T08:36:34.751089+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.783720+0000","last_change":"2024-04-25T08:36:34.753588+0000","last_active":"2024-04-25T08:36:35.783720+0000","last_peered":"2024-04-25T08:36:35.783720+0000","last_clean":"2024-04-25T08:36:35.783720+0000","last_became_active":"2024-04-25T08:36:34.753290+0000","last_became_peered":"2024-04-25T08:36:34.753290+0000","last_unstale":"2024-04-25T08:36:35.783720+0000","last_undegraded":"2024-04-25T08:36:35.783720+0000","last_fullsized":"2024-04-25T08:36:35.783720+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748432+0000","last_change":"2024-04-25T08:36:34.748432+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748432+0000","last_undegraded":"2024-04-25T08:36:34.748432+0000","last_fullsized":"2024-04-25T08:36:34.748432+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.786237+0000","last_change":"2024-04-25T08:36:34.750641+0000","last_active":"2024-04-25T08:36:35.786237+0000","last_peered":"2024-04-25T08:36:35.786237+0000","last_clean":"2024-04-25T08:36:35.786237+0000","last_became_active":"2024-04-25T08:36:34.748802+0000","last_became_peered":"2024-04-25T08:36:34.748802+0000","last_unstale":"2024-04-25T08:36:35.786237+0000","last_undegraded":"2024-04-25T08:36:35.786237+0000","last_fullsized":"2024-04-25T08:36:35.786237+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:39.476164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.773615+0000","last_change":"2024-04-25T08:36:35.768603+0000","last_active":"2024-04-25T08:36:35.773615+0000","last_peered":"2024-04-25T08:36:35.773615+0000","last_clean":"2024-04-25T08:36:35.773615+0000","last_became_active":"2024-04-25T08:36:35.766863+0000","last_became_peered":"2024-04-25T08:36:35.766863+0000","last_unstale":"2024-04-25T08:36:35.773615+0000","last_undegraded":"2024-04-25T08:36:35.773615+0000","last_fullsized":"2024-04-25T08:36:35.773615+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:26.446290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762382+0000","last_change":"2024-04-25T08:36:34.761126+0000","last_active":"2024-04-25T08:36:34.762382+0000","last_peered":"2024-04-25T08:36:34.762382+0000","last_clean":"2024-04-25T08:36:34.762382+0000","last_became_active":"2024-04-25T08:36:34.760848+0000","last_became_peered":"2024-04-25T08:36:34.760848+0000","last_unstale":"2024-04-25T08:36:34.762382+0000","last_undegraded":"2024-04-25T08:36:34.762382+0000","last_fullsized":"2024-04-25T08:36:34.762382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770663+0000","last_change":"2024-04-25T08:36:35.769783+0000","last_active":"2024-04-25T08:36:35.770663+0000","last_peered":"2024-04-25T08:36:35.770663+0000","last_clean":"2024-04-25T08:36:35.770663+0000","last_became_active":"2024-04-25T08:36:35.769101+0000","last_became_peered":"2024-04-25T08:36:35.769101+0000","last_unstale":"2024-04-25T08:36:35.770663+0000","last_undegraded":"2024-04-25T08:36:35.770663+0000","last_fullsized":"2024-04-25T08:36:35.770663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757823+0000","last_change":"2024-04-25T08:36:34.756585+0000","last_active":"2024-04-25T08:36:34.757823+0000","last_peered":"2024-04-25T08:36:34.757823+0000","last_clean":"2024-04-25T08:36:34.757823+0000","last_became_active":"2024-04-25T08:36:34.756301+0000","last_became_peered":"2024-04-25T08:36:34.756301+0000","last_unstale":"2024-04-25T08:36:34.757823+0000","last_undegraded":"2024-04-25T08:36:34.757823+0000","last_fullsized":"2024-04-25T08:36:34.757823+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763088+0000","last_change":"2024-04-25T08:36:34.761194+0000","last_active":"2024-04-25T08:36:34.763088+0000","last_peered":"2024-04-25T08:36:34.763088+0000","last_clean":"2024-04-25T08:36:34.763088+0000","last_became_active":"2024-04-25T08:36:34.760927+0000","last_became_peered":"2024-04-25T08:36:34.760927+0000","last_unstale":"2024-04-25T08:36:34.763088+0000","last_undegraded":"2024-04-25T08:36:34.763088+0000","last_fullsized":"2024-04-25T08:36:34.763088+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:51.400868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747795+0000","last_change":"2024-04-25T08:36:34.747795+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747795+0000","last_undegraded":"2024-04-25T08:36:34.747795+0000","last_fullsized":"2024-04-25T08:36:34.747795+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.760876+0000","last_change":"2024-04-25T08:36:34.759616+0000","last_active":"2024-04-25T08:36:34.760876+0000","last_peered":"2024-04-25T08:36:34.760876+0000","last_clean":"2024-04-25T08:36:34.760876+0000","last_became_active":"2024-04-25T08:36:34.759260+0000","last_became_peered":"2024-04-25T08:36:34.759260+0000","last_unstale":"2024-04-25T08:36:34.760876+0000","last_undegraded":"2024-04-25T08:36:34.760876+0000","last_fullsized":"2024-04-25T08:36:34.760876+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752657+0000","last_change":"2024-04-25T08:36:34.752657+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752657+0000","last_undegraded":"2024-04-25T08:36:34.752657+0000","last_fullsized":"2024-04-25T08:36:34.752657+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.777741+0000","last_change":"2024-04-25T08:36:34.753294+0000","last_active":"2024-04-25T08:36:35.777741+0000","last_peered":"2024-04-25T08:36:35.777741+0000","last_clean":"2024-04-25T08:36:35.777741+0000","last_became_active":"2024-04-25T08:36:34.753016+0000","last_became_peered":"2024-04-25T08:36:34.753016+0000","last_unstale":"2024-04-25T08:36:35.777741+0000","last_undegraded":"2024-04-25T08:36:35.777741+0000","last_fullsized":"2024-04-25T08:36:35.777741+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.770521+0000","last_change":"2024-04-25T08:36:35.767817+0000","last_active":"2024-04-25T08:36:35.770521+0000","last_peered":"2024-04-25T08:36:35.770521+0000","last_clean":"2024-04-25T08:36:35.770521+0000","last_became_active":"2024-04-25T08:36:35.767517+0000","last_became_peered":"2024-04-25T08:36:35.767517+0000","last_unstale":"2024-04-25T08:36:35.770521+0000","last_undegraded":"2024-04-25T08:36:35.770521+0000","last_fullsized":"2024-04-25T08:36:35.770521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.784098+0000","last_change":"2024-04-25T08:36:34.759604+0000","last_active":"2024-04-25T08:36:35.784098+0000","last_peered":"2024-04-25T08:36:35.784098+0000","last_clean":"2024-04-25T08:36:35.784098+0000","last_became_active":"2024-04-25T08:36:34.759303+0000","last_became_peered":"2024-04-25T08:36:34.759303+0000","last_unstale":"2024-04-25T08:36:35.784098+0000","last_undegraded":"2024-04-25T08:36:35.784098+0000","last_fullsized":"2024-04-25T08:36:35.784098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.258277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748937+0000","last_change":"2024-04-25T08:36:34.748937+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748937+0000","last_undegraded":"2024-04-25T08:36:34.748937+0000","last_fullsized":"2024-04-25T08:36:34.748937+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760131+0000","last_change":"2024-04-25T08:36:34.757404+0000","last_active":"2024-04-25T08:36:35.760131+0000","last_peered":"2024-04-25T08:36:35.760131+0000","last_clean":"2024-04-25T08:36:35.760131+0000","last_became_active":"2024-04-25T08:36:34.755586+0000","last_became_peered":"2024-04-25T08:36:34.755586+0000","last_unstale":"2024-04-25T08:36:35.760131+0000","last_undegraded":"2024-04-25T08:36:35.760131+0000","last_fullsized":"2024-04-25T08:36:35.760131+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747259+0000","last_change":"2024-04-25T08:36:34.747259+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747259+0000","last_undegraded":"2024-04-25T08:36:34.747259+0000","last_fullsized":"2024-04-25T08:36:34.747259+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.755095+0000","last_change":"2024-04-25T08:36:34.749438+0000","last_active":"2024-04-25T08:36:34.755095+0000","last_peered":"2024-04-25T08:36:34.755095+0000","last_clean":"2024-04-25T08:36:34.755095+0000","last_became_active":"2024-04-25T08:36:34.749073+0000","last_became_peered":"2024-04-25T08:36:34.749073+0000","last_unstale":"2024-04-25T08:36:34.755095+0000","last_undegraded":"2024-04-25T08:36:34.755095+0000","last_fullsized":"2024-04-25T08:36:34.755095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:25.045409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.747760+0000","last_change":"2024-04-25T08:36:34.747760+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.747760+0000","last_undegraded":"2024-04-25T08:36:34.747760+0000","last_fullsized":"2024-04-25T08:36:34.747760+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762766+0000","last_change":"2024-04-25T08:36:34.755127+0000","last_active":"2024-04-25T08:36:34.762766+0000","last_peered":"2024-04-25T08:36:34.762766+0000","last_clean":"2024-04-25T08:36:34.762766+0000","last_became_active":"2024-04-25T08:36:34.754781+0000","last_became_peered":"2024-04-25T08:36:34.754781+0000","last_unstale":"2024-04-25T08:36:34.762766+0000","last_undegraded":"2024-04-25T08:36:34.762766+0000","last_fullsized":"2024-04-25T08:36:34.762766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:40.479220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.748664+0000","last_change":"2024-04-25T08:36:34.748664+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.748664+0000","last_undegraded":"2024-04-25T08:36:34.748664+0000","last_fullsized":"2024-04-25T08:36:34.748664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753639+0000","last_change":"2024-04-25T08:36:34.756454+0000","last_active":"2024-04-25T08:36:35.753639+0000","last_peered":"2024-04-25T08:36:35.753639+0000","last_clean":"2024-04-25T08:36:35.753639+0000","last_became_active":"2024-04-25T08:36:34.755238+0000","last_became_peered":"2024-04-25T08:36:34.755238+0000","last_unstale":"2024-04-25T08:36:35.753639+0000","last_undegraded":"2024-04-25T08:36:35.753639+0000","last_fullsized":"2024-04-25T08:36:35.753639+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752366+0000","last_change":"2024-04-25T08:36:34.752366+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752366+0000","last_undegraded":"2024-04-25T08:36:34.752366+0000","last_fullsized":"2024-04-25T08:36:34.752366+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762995+0000","last_change":"2024-04-25T08:36:34.760255+0000","last_active":"2024-04-25T08:36:34.762995+0000","last_peered":"2024-04-25T08:36:34.762995+0000","last_clean":"2024-04-25T08:36:34.762995+0000","last_became_active":"2024-04-25T08:36:34.759634+0000","last_became_peered":"2024-04-25T08:36:34.759634+0000","last_unstale":"2024-04-25T08:36:34.762995+0000","last_undegraded":"2024-04-25T08:36:34.762995+0000","last_fullsized":"2024-04-25T08:36:34.762995+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:54.192240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.751714+0000","last_change":"2024-04-25T08:36:34.751714+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.751714+0000","last_undegraded":"2024-04-25T08:36:34.751714+0000","last_fullsized":"2024-04-25T08:36:34.751714+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762154+0000","last_change":"2024-04-25T08:36:34.760831+0000","last_active":"2024-04-25T08:36:34.762154+0000","last_peered":"2024-04-25T08:36:34.762154+0000","last_clean":"2024-04-25T08:36:34.762154+0000","last_became_active":"2024-04-25T08:36:34.760558+0000","last_became_peered":"2024-04-25T08:36:34.760558+0000","last_unstale":"2024-04-25T08:36:34.762154+0000","last_undegraded":"2024-04-25T08:36:34.762154+0000","last_fullsized":"2024-04-25T08:36:34.762154+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.744146+0000","last_change":"2024-04-25T08:36:34.744146+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.744146+0000","last_undegraded":"2024-04-25T08:36:34.744146+0000","last_fullsized":"2024-04-25T08:36:34.744146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.758450+0000","last_change":"2024-04-25T08:36:34.756998+0000","last_active":"2024-04-25T08:36:34.758450+0000","last_peered":"2024-04-25T08:36:34.758450+0000","last_clean":"2024-04-25T08:36:34.758450+0000","last_became_active":"2024-04-25T08:36:34.756057+0000","last_became_peered":"2024-04-25T08:36:34.756057+0000","last_unstale":"2024-04-25T08:36:34.758450+0000","last_undegraded":"2024-04-25T08:36:34.758450+0000","last_fullsized":"2024-04-25T08:36:34.758450+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:51.646595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.749164+0000","last_change":"2024-04-25T08:36:34.749164+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.749164+0000","last_undegraded":"2024-04-25T08:36:34.749164+0000","last_fullsized":"2024-04-25T08:36:34.749164+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.754613+0000","last_change":"2024-04-25T08:36:34.750357+0000","last_active":"2024-04-25T08:36:35.754613+0000","last_peered":"2024-04-25T08:36:35.754613+0000","last_clean":"2024-04-25T08:36:35.754613+0000","last_became_active":"2024-04-25T08:36:34.749684+0000","last_became_peered":"2024-04-25T08:36:34.749684+0000","last_unstale":"2024-04-25T08:36:35.754613+0000","last_undegraded":"2024-04-25T08:36:35.754613+0000","last_fullsized":"2024-04-25T08:36:35.754613+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774914+0000","last_change":"2024-04-25T08:36:35.773260+0000","last_active":"2024-04-25T08:36:35.774914+0000","last_peered":"2024-04-25T08:36:35.774914+0000","last_clean":"2024-04-25T08:36:35.774914+0000","last_became_active":"2024-04-25T08:36:35.773012+0000","last_became_peered":"2024-04-25T08:36:35.773012+0000","last_unstale":"2024-04-25T08:36:35.774914+0000","last_undegraded":"2024-04-25T08:36:35.774914+0000","last_fullsized":"2024-04-25T08:36:35.774914+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.757258+0000","last_change":"2024-04-25T08:36:34.756400+0000","last_active":"2024-04-25T08:36:34.757258+0000","last_peered":"2024-04-25T08:36:34.757258+0000","last_clean":"2024-04-25T08:36:34.757258+0000","last_became_active":"2024-04-25T08:36:34.756114+0000","last_became_peered":"2024-04-25T08:36:34.756114+0000","last_unstale":"2024-04-25T08:36:34.757258+0000","last_undegraded":"2024-04-25T08:36:34.757258+0000","last_fullsized":"2024-04-25T08:36:34.757258+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:14.444631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.752972+0000","last_change":"2024-04-25T08:36:34.752972+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.752972+0000","last_undegraded":"2024-04-25T08:36:34.752972+0000","last_fullsized":"2024-04-25T08:36:34.752972+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.759854+0000","last_change":"2024-04-25T08:36:34.757608+0000","last_active":"2024-04-25T08:36:34.759854+0000","last_peered":"2024-04-25T08:36:34.759854+0000","last_clean":"2024-04-25T08:36:34.759854+0000","last_became_active":"2024-04-25T08:36:34.756201+0000","last_became_peered":"2024-04-25T08:36:34.756201+0000","last_unstale":"2024-04-25T08:36:34.759854+0000","last_undegraded":"2024-04-25T08:36:34.759854+0000","last_fullsized":"2024-04-25T08:36:34.759854+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:09.132700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":108,"state":"creating+peering","last_fresh":"2024-04-25T08:36:34.754953+0000","last_change":"2024-04-25T08:36:34.754953+0000","last_active":"2024-04-25T08:36:34.737374+0000","last_peered":"2024-04-25T08:36:34.737374+0000","last_clean":"2024-04-25T08:36:34.737374+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:34.754953+0000","last_undegraded":"2024-04-25T08:36:34.754953+0000","last_fullsized":"2024-04-25T08:36:34.754953+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763179+0000","last_change":"2024-04-25T08:36:34.760614+0000","last_active":"2024-04-25T08:36:34.763179+0000","last_peered":"2024-04-25T08:36:34.763179+0000","last_clean":"2024-04-25T08:36:34.763179+0000","last_became_active":"2024-04-25T08:36:34.760357+0000","last_became_peered":"2024-04-25T08:36:34.760357+0000","last_unstale":"2024-04-25T08:36:34.763179+0000","last_undegraded":"2024-04-25T08:36:34.763179+0000","last_fullsized":"2024-04-25T08:36:34.763179+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:48.096031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.768321+0000","last_change":"2024-04-25T08:36:35.766389+0000","last_active":"2024-04-25T08:36:35.768321+0000","last_peered":"2024-04-25T08:36:35.768321+0000","last_clean":"2024-04-25T08:36:35.768321+0000","last_became_active":"2024-04-25T08:36:35.766116+0000","last_became_peered":"2024-04-25T08:36:35.766116+0000","last_unstale":"2024-04-25T08:36:35.768321+0000","last_undegraded":"2024-04-25T08:36:35.768321+0000","last_fullsized":"2024-04-25T08:36:35.768321+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.753757+0000","last_change":"2024-04-25T08:36:34.754257+0000","last_active":"2024-04-25T08:36:35.753757+0000","last_peered":"2024-04-25T08:36:35.753757+0000","last_clean":"2024-04-25T08:36:35.753757+0000","last_became_active":"2024-04-25T08:36:34.752327+0000","last_became_peered":"2024-04-25T08:36:34.752327+0000","last_unstale":"2024-04-25T08:36:35.753757+0000","last_undegraded":"2024-04-25T08:36:35.753757+0000","last_fullsized":"2024-04-25T08:36:35.753757+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774411+0000","last_change":"2024-04-25T08:36:35.770356+0000","last_active":"2024-04-25T08:36:35.774411+0000","last_peered":"2024-04-25T08:36:35.774411+0000","last_clean":"2024-04-25T08:36:35.774411+0000","last_became_active":"2024-04-25T08:36:35.769027+0000","last_became_peered":"2024-04-25T08:36:35.769027+0000","last_unstale":"2024-04-25T08:36:35.774411+0000","last_undegraded":"2024-04-25T08:36:35.774411+0000","last_fullsized":"2024-04-25T08:36:35.774411+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.763309+0000","last_change":"2024-04-25T08:36:34.762590+0000","last_active":"2024-04-25T08:36:34.763309+0000","last_peered":"2024-04-25T08:36:34.763309+0000","last_clean":"2024-04-25T08:36:34.763309+0000","last_became_active":"2024-04-25T08:36:34.762255+0000","last_became_peered":"2024-04-25T08:36:34.762255+0000","last_unstale":"2024-04-25T08:36:34.763309+0000","last_undegraded":"2024-04-25T08:36:34.763309+0000","last_fullsized":"2024-04-25T08:36:34.763309+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:13.862606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.774363+0000","last_change":"2024-04-25T08:36:35.769174+0000","last_active":"2024-04-25T08:36:35.774363+0000","last_peered":"2024-04-25T08:36:35.774363+0000","last_clean":"2024-04-25T08:36:35.774363+0000","last_became_active":"2024-04-25T08:36:35.768769+0000","last_became_peered":"2024-04-25T08:36:35.768769+0000","last_unstale":"2024-04-25T08:36:35.774363+0000","last_undegraded":"2024-04-25T08:36:35.774363+0000","last_fullsized":"2024-04-25T08:36:35.774363+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:55.056618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:36:34.762841+0000","last_change":"2024-04-25T08:36:34.755845+0000","last_active":"2024-04-25T08:36:34.762841+0000","last_peered":"2024-04-25T08:36:34.762841+0000","last_clean":"2024-04-25T08:36:34.762841+0000","last_became_active":"2024-04-25T08:36:34.755540+0000","last_became_peered":"2024-04-25T08:36:34.755540+0000","last_unstale":"2024-04-25T08:36:34.762841+0000","last_undegraded":"2024-04-25T08:36:34.762841+0000","last_fullsized":"2024-04-25T08:36:34.762841+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:02.715530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T08:36:35.760141+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:36:35.760141+0000","last_peered":"2024-04-25T08:36:35.760141+0000","last_clean":"2024-04-25T08:36:35.760141+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:36:35.760141+0000","last_undegraded":"2024-04-25T08:36:35.760141+0000","last_fullsized":"2024-04-25T08:36:35.760141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1949696,"data_stored":1216579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46412,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325428,"statfs":{"total":96636764160,"available":96589238272,"internally_reserved":0,"allocated":1368064,"data_stored":628267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40824,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331016,"statfs":{"total":96636764160,"available":96594960384,"internally_reserved":0,"allocated":1347584,"data_stored":626051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54688,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317152,"statfs":{"total":96636764160,"available":96580763648,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54696,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317144,"statfs":{"total":96636764160,"available":96580755456,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50536,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321304,"statfs":{"total":96636764160,"available":96585015296,"internally_reserved":0,"allocated":1335296,"data_stored":618884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55260,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316580,"statfs":{"total":96636764160,"available":96580177920,"internally_reserved":0,"allocated":1916928,"data_stored":1202585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:43.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:43.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.316+0000 7fa57da7d640 1 Processor -- start 2024-04-25T08:36:43.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.316+0000 7fa57da7d640 1 -- start start 2024-04-25T08:36:43.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa57da7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa57815da70 0x7fa578161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa57da7d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa5780b9730 0x7fa5780ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa57da7d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa578161b80 0x7fa5780ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa57da7d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5781625e0 con 0x7fa57815da70 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa57da7d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa578162020 con 0x7fa5780b9730 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa57da7d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa578162300 con 0x7fa578161b80 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa57815da70 0x7fa578161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa57815da70 0x7fa578161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56424/0 (socket says 172.21.15.60:56424) 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 -- 172.21.15.60:0/1663789598 learned_addr learned my addr 172.21.15.60:0/1663789598 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa576ffd640 1 --2- 172.21.15.60:0/1663789598 >> v2:172.21.15.100:3300/0 conn(0x7fa5780b9730 0x7fa5780ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa577fff640 1 --2- 172.21.15.60:0/1663789598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa578161b80 0x7fa5780ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 -- 172.21.15.60:0/1663789598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa578161b80 msgr2=0x7fa5780ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 --2- 172.21.15.60:0/1663789598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa578161b80 0x7fa5780ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 -- 172.21.15.60:0/1663789598 >> v2:172.21.15.100:3300/0 conn(0x7fa5780b9730 msgr2=0x7fa5780ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 --2- 172.21.15.60:0/1663789598 >> v2:172.21.15.100:3300/0 conn(0x7fa5780b9730 0x7fa5780ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.317+0000 7fa5777fe640 1 -- 172.21.15.60:0/1663789598 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5780d1af0 con 0x7fa57815da70 2024-04-25T08:36:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.318+0000 7fa5777fe640 1 --2- 172.21.15.60:0/1663789598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57815da70 0x7fa578161640 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa5780c72a0 tx=0x7fa56802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed010372074ee0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.318+0000 7fa5767fc640 1 -- 172.21.15.60:0/1663789598 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568037070 con 0x7fa57815da70 2024-04-25T08:36:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.318+0000 7fa5767fc640 1 -- 172.21.15.60:0/1663789598 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa568032440 con 0x7fa57815da70 2024-04-25T08:36:43.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.318+0000 7fa5767fc640 1 -- 172.21.15.60:0/1663789598 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568032810 con 0x7fa57815da70 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 -- 172.21.15.60:0/1663789598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57815da70 msgr2=0x7fa578161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 --2- 172.21.15.60:0/1663789598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57815da70 0x7fa578161640 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa5780c72a0 tx=0x7fa56802a140 comp rx=0 tx=0).stop 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 -- 172.21.15.60:0/1663789598 shutdown_connections 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 --2- 172.21.15.60:0/1663789598 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa578161b80 0x7fa5780ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 --2- 172.21.15.60:0/1663789598 >> v2:172.21.15.100:3300/0 conn(0x7fa5780b9730 0x7fa5780ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 --2- 172.21.15.60:0/1663789598 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57815da70 0x7fa578161640 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 -- 172.21.15.60:0/1663789598 >> 172.21.15.60:0/1663789598 conn(0x7fa578061a20 msgr2=0x7fa578061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 -- 172.21.15.60:0/1663789598 shutdown_connections 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.319+0000 7fa57da7d640 1 -- 172.21.15.60:0/1663789598 wait complete. 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.320+0000 7fa57da7d640 1 Processor -- start 2024-04-25T08:36:43.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.320+0000 7fa57da7d640 1 -- start start 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa57da7d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57815da70 0x7fa57815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa57da7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 0x7fa578148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa57da7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5781587d0 0x7fa5780b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa57da7d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57806c120 con 0x7fa578161b80 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa57da7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57815de50 con 0x7fa5781587d0 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa57da7d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57806c2a0 con 0x7fa57815da70 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 0x7fa578148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 0x7fa578148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56436/0 (socket says 172.21.15.60:56436) 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 -- 172.21.15.60:0/3079257948 learned_addr learned my addr 172.21.15.60:0/3079257948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa5777fe640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57815da70 0x7fa57815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa577fff640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5781587d0 0x7fa5780b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 -- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57815da70 msgr2=0x7fa57815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57815da70 0x7fa57815dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 -- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5781587d0 msgr2=0x7fa5780b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5781587d0 0x7fa5780b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.321+0000 7fa576ffd640 1 -- 172.21.15.60:0/3079257948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa578149190 con 0x7fa578161b80 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.322+0000 7fa576ffd640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 0x7fa578148c50 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00e980 tx=0x7fa56c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.322+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c017070 con 0x7fa578161b80 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.322+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00ca00 con 0x7fa578161b80 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.322+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57815e660 con 0x7fa578161b80 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.322+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00cdd0 con 0x7fa578161b80 2024-04-25T08:36:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.322+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57815ec20 con 0x7fa578161b80 2024-04-25T08:36:43.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.323+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa578073990 con 0x7fa578161b80 2024-04-25T08:36:43.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.324+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa56c01a070 con 0x7fa578161b80 2024-04-25T08:36:43.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.324+0000 7fa55ffff640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa56406b4d0 0x7fa56406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.324+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa56c08a390 con 0x7fa578161b80 2024-04-25T08:36:43.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.324+0000 7fa5777fe640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa56406b4d0 0x7fa56406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.325+0000 7fa5777fe640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa56406b4d0 0x7fa56406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa568034000 tx=0x7fa56802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:43.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.328+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa56c059080 con 0x7fa578161b80 2024-04-25T08:36:43.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.506+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5780d3a00 con 0x7fa56406b4d0 2024-04-25T08:36:43.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.513+0000 7fa55ffff640 1 -- 172.21.15.60:0/3079257948 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252450 (secure 0 0 0) 0x7fa5780d3a00 con 0x7fa56406b4d0 2024-04-25T08:36:43.509 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:43.511 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:36:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa56406b4d0 msgr2=0x7fa56406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa56406b4d0 0x7fa56406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa568034000 tx=0x7fa56802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 msgr2=0x7fa578148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 0x7fa578148c50 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00e980 tx=0x7fa56c00ee40 comp rx=0 tx=0).stop 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 shutdown_connections 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa56406b4d0 0x7fa56406d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5781587d0 0x7fa5780b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa578161b80 0x7fa578148c50 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 --2- 172.21.15.60:0/3079257948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57815da70 0x7fa57815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 >> 172.21.15.60:0/3079257948 conn(0x7fa578061a20 msgr2=0x7fa5780b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 shutdown_connections 2024-04-25T08:36:43.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.516+0000 7fa57da7d640 1 -- 172.21.15.60:0/3079257948 wait complete. 2024-04-25T08:36:43.521 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-25T08:36:43.499171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":394548,"kb_used_data":11848,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754580172,"statfs":{"total":773094113280,"available":772690096128,"internally_reserved":0,"allocated":12132352,"data_stored":6247500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002146"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773146+0000","last_change":"2024-04-25T08:36:35.772942+0000","last_active":"2024-04-25T08:36:36.773146+0000","last_peered":"2024-04-25T08:36:36.773146+0000","last_clean":"2024-04-25T08:36:36.773146+0000","last_became_active":"2024-04-25T08:36:35.772663+0000","last_became_peered":"2024-04-25T08:36:35.772663+0000","last_unstale":"2024-04-25T08:36:36.773146+0000","last_undegraded":"2024-04-25T08:36:36.773146+0000","last_fullsized":"2024-04-25T08:36:36.773146+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:03.122631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755499+0000","last_change":"2024-04-25T08:36:35.765297+0000","last_active":"2024-04-25T08:36:36.755499+0000","last_peered":"2024-04-25T08:36:36.755499+0000","last_clean":"2024-04-25T08:36:36.755499+0000","last_became_active":"2024-04-25T08:36:35.763995+0000","last_became_peered":"2024-04-25T08:36:35.763995+0000","last_unstale":"2024-04-25T08:36:36.755499+0000","last_undegraded":"2024-04-25T08:36:36.755499+0000","last_fullsized":"2024-04-25T08:36:36.755499+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773129+0000","last_change":"2024-04-25T08:36:35.767440+0000","last_active":"2024-04-25T08:36:36.773129+0000","last_peered":"2024-04-25T08:36:36.773129+0000","last_clean":"2024-04-25T08:36:36.773129+0000","last_became_active":"2024-04-25T08:36:35.766952+0000","last_became_peered":"2024-04-25T08:36:35.766952+0000","last_unstale":"2024-04-25T08:36:36.773129+0000","last_undegraded":"2024-04-25T08:36:36.773129+0000","last_fullsized":"2024-04-25T08:36:36.773129+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.086396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559485+0000","last_change":"2024-04-25T08:36:35.761783+0000","last_active":"2024-04-25T08:36:37.559485+0000","last_peered":"2024-04-25T08:36:37.559485+0000","last_clean":"2024-04-25T08:36:37.559485+0000","last_became_active":"2024-04-25T08:36:35.761466+0000","last_became_peered":"2024-04-25T08:36:35.761466+0000","last_unstale":"2024-04-25T08:36:37.559485+0000","last_undegraded":"2024-04-25T08:36:37.559485+0000","last_fullsized":"2024-04-25T08:36:37.559485+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632564+0000","last_change":"2024-04-25T08:36:35.769085+0000","last_active":"2024-04-25T08:36:37.632564+0000","last_peered":"2024-04-25T08:36:37.632564+0000","last_clean":"2024-04-25T08:36:37.632564+0000","last_became_active":"2024-04-25T08:36:35.768780+0000","last_became_peered":"2024-04-25T08:36:35.768780+0000","last_unstale":"2024-04-25T08:36:37.632564+0000","last_undegraded":"2024-04-25T08:36:37.632564+0000","last_fullsized":"2024-04-25T08:36:37.632564+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754475+0000","last_change":"2024-04-25T08:36:35.782370+0000","last_active":"2024-04-25T08:36:36.754475+0000","last_peered":"2024-04-25T08:36:36.754475+0000","last_clean":"2024-04-25T08:36:36.754475+0000","last_became_active":"2024-04-25T08:36:35.782129+0000","last_became_peered":"2024-04-25T08:36:35.782129+0000","last_unstale":"2024-04-25T08:36:36.754475+0000","last_undegraded":"2024-04-25T08:36:36.754475+0000","last_fullsized":"2024-04-25T08:36:36.754475+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.324942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754701+0000","last_change":"2024-04-25T08:36:35.771864+0000","last_active":"2024-04-25T08:36:36.754701+0000","last_peered":"2024-04-25T08:36:36.754701+0000","last_clean":"2024-04-25T08:36:36.754701+0000","last_became_active":"2024-04-25T08:36:35.771591+0000","last_became_peered":"2024-04-25T08:36:35.771591+0000","last_unstale":"2024-04-25T08:36:36.754701+0000","last_undegraded":"2024-04-25T08:36:36.754701+0000","last_fullsized":"2024-04-25T08:36:36.754701+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753338+0000","last_change":"2024-04-25T08:36:35.774393+0000","last_active":"2024-04-25T08:36:36.753338+0000","last_peered":"2024-04-25T08:36:36.753338+0000","last_clean":"2024-04-25T08:36:36.753338+0000","last_became_active":"2024-04-25T08:36:35.774072+0000","last_became_peered":"2024-04-25T08:36:35.774072+0000","last_unstale":"2024-04-25T08:36:36.753338+0000","last_undegraded":"2024-04-25T08:36:36.753338+0000","last_fullsized":"2024-04-25T08:36:36.753338+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:33.781314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755634+0000","last_change":"2024-04-25T08:36:35.768883+0000","last_active":"2024-04-25T08:36:36.755634+0000","last_peered":"2024-04-25T08:36:36.755634+0000","last_clean":"2024-04-25T08:36:36.755634+0000","last_became_active":"2024-04-25T08:36:35.768603+0000","last_became_peered":"2024-04-25T08:36:35.768603+0000","last_unstale":"2024-04-25T08:36:36.755634+0000","last_undegraded":"2024-04-25T08:36:36.755634+0000","last_fullsized":"2024-04-25T08:36:36.755634+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:09.625971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632609+0000","last_change":"2024-04-25T08:36:35.774578+0000","last_active":"2024-04-25T08:36:37.632609+0000","last_peered":"2024-04-25T08:36:37.632609+0000","last_clean":"2024-04-25T08:36:37.632609+0000","last_became_active":"2024-04-25T08:36:35.772262+0000","last_became_peered":"2024-04-25T08:36:35.772262+0000","last_unstale":"2024-04-25T08:36:37.632609+0000","last_undegraded":"2024-04-25T08:36:37.632609+0000","last_fullsized":"2024-04-25T08:36:37.632609+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775608+0000","last_change":"2024-04-25T08:36:35.772717+0000","last_active":"2024-04-25T08:36:36.775608+0000","last_peered":"2024-04-25T08:36:36.775608+0000","last_clean":"2024-04-25T08:36:36.775608+0000","last_became_active":"2024-04-25T08:36:35.772232+0000","last_became_peered":"2024-04-25T08:36:35.772232+0000","last_unstale":"2024-04-25T08:36:36.775608+0000","last_undegraded":"2024-04-25T08:36:36.775608+0000","last_fullsized":"2024-04-25T08:36:36.775608+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632498+0000","last_change":"2024-04-25T08:36:35.769987+0000","last_active":"2024-04-25T08:36:37.632498+0000","last_peered":"2024-04-25T08:36:37.632498+0000","last_clean":"2024-04-25T08:36:37.632498+0000","last_became_active":"2024-04-25T08:36:35.768474+0000","last_became_peered":"2024-04-25T08:36:35.768474+0000","last_unstale":"2024-04-25T08:36:37.632498+0000","last_undegraded":"2024-04-25T08:36:37.632498+0000","last_fullsized":"2024-04-25T08:36:37.632498+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559354+0000","last_change":"2024-04-25T08:36:35.764248+0000","last_active":"2024-04-25T08:36:37.559354+0000","last_peered":"2024-04-25T08:36:37.559354+0000","last_clean":"2024-04-25T08:36:37.559354+0000","last_became_active":"2024-04-25T08:36:35.763926+0000","last_became_peered":"2024-04-25T08:36:35.763926+0000","last_unstale":"2024-04-25T08:36:37.559354+0000","last_undegraded":"2024-04-25T08:36:37.559354+0000","last_fullsized":"2024-04-25T08:36:37.559354+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754861+0000","last_change":"2024-04-25T08:36:35.767092+0000","last_active":"2024-04-25T08:36:36.754861+0000","last_peered":"2024-04-25T08:36:36.754861+0000","last_clean":"2024-04-25T08:36:36.754861+0000","last_became_active":"2024-04-25T08:36:35.766827+0000","last_became_peered":"2024-04-25T08:36:35.766827+0000","last_unstale":"2024-04-25T08:36:36.754861+0000","last_undegraded":"2024-04-25T08:36:36.754861+0000","last_fullsized":"2024-04-25T08:36:36.754861+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:09.625971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754201+0000","last_change":"2024-04-25T08:36:35.775011+0000","last_active":"2024-04-25T08:36:36.754201+0000","last_peered":"2024-04-25T08:36:36.754201+0000","last_clean":"2024-04-25T08:36:36.754201+0000","last_became_active":"2024-04-25T08:36:35.774695+0000","last_became_peered":"2024-04-25T08:36:35.774695+0000","last_unstale":"2024-04-25T08:36:36.754201+0000","last_undegraded":"2024-04-25T08:36:36.754201+0000","last_fullsized":"2024-04-25T08:36:36.754201+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:34.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754292+0000","last_change":"2024-04-25T08:36:35.766209+0000","last_active":"2024-04-25T08:36:36.754292+0000","last_peered":"2024-04-25T08:36:36.754292+0000","last_clean":"2024-04-25T08:36:36.754292+0000","last_became_active":"2024-04-25T08:36:35.765932+0000","last_became_peered":"2024-04-25T08:36:35.765932+0000","last_unstale":"2024-04-25T08:36:36.754292+0000","last_undegraded":"2024-04-25T08:36:36.754292+0000","last_fullsized":"2024-04-25T08:36:36.754292+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:17.413250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773360+0000","last_change":"2024-04-25T08:36:35.768549+0000","last_active":"2024-04-25T08:36:36.773360+0000","last_peered":"2024-04-25T08:36:36.773360+0000","last_clean":"2024-04-25T08:36:36.773360+0000","last_became_active":"2024-04-25T08:36:35.767808+0000","last_became_peered":"2024-04-25T08:36:35.767808+0000","last_unstale":"2024-04-25T08:36:36.773360+0000","last_undegraded":"2024-04-25T08:36:36.773360+0000","last_fullsized":"2024-04-25T08:36:36.773360+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:43.984792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755553+0000","last_change":"2024-04-25T08:36:35.771768+0000","last_active":"2024-04-25T08:36:36.755553+0000","last_peered":"2024-04-25T08:36:36.755553+0000","last_clean":"2024-04-25T08:36:36.755553+0000","last_became_active":"2024-04-25T08:36:35.771487+0000","last_became_peered":"2024-04-25T08:36:35.771487+0000","last_unstale":"2024-04-25T08:36:36.755553+0000","last_undegraded":"2024-04-25T08:36:36.755553+0000","last_fullsized":"2024-04-25T08:36:36.755553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.643034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773224+0000","last_change":"2024-04-25T08:36:35.775237+0000","last_active":"2024-04-25T08:36:36.773224+0000","last_peered":"2024-04-25T08:36:36.773224+0000","last_clean":"2024-04-25T08:36:36.773224+0000","last_became_active":"2024-04-25T08:36:35.774871+0000","last_became_peered":"2024-04-25T08:36:35.774871+0000","last_unstale":"2024-04-25T08:36:36.773224+0000","last_undegraded":"2024-04-25T08:36:36.773224+0000","last_fullsized":"2024-04-25T08:36:36.773224+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:21.642225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632499+0000","last_change":"2024-04-25T08:36:35.766200+0000","last_active":"2024-04-25T08:36:37.632499+0000","last_peered":"2024-04-25T08:36:37.632499+0000","last_clean":"2024-04-25T08:36:37.632499+0000","last_became_active":"2024-04-25T08:36:35.762501+0000","last_became_peered":"2024-04-25T08:36:35.762501+0000","last_unstale":"2024-04-25T08:36:37.632499+0000","last_undegraded":"2024-04-25T08:36:37.632499+0000","last_fullsized":"2024-04-25T08:36:37.632499+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753604+0000","last_change":"2024-04-25T08:36:35.766947+0000","last_active":"2024-04-25T08:36:36.753604+0000","last_peered":"2024-04-25T08:36:36.753604+0000","last_clean":"2024-04-25T08:36:36.753604+0000","last_became_active":"2024-04-25T08:36:35.765888+0000","last_became_peered":"2024-04-25T08:36:35.765888+0000","last_unstale":"2024-04-25T08:36:36.753604+0000","last_undegraded":"2024-04-25T08:36:36.753604+0000","last_fullsized":"2024-04-25T08:36:36.753604+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:49.545602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632765+0000","last_change":"2024-04-25T08:36:35.781810+0000","last_active":"2024-04-25T08:36:37.632765+0000","last_peered":"2024-04-25T08:36:37.632765+0000","last_clean":"2024-04-25T08:36:37.632765+0000","last_became_active":"2024-04-25T08:36:35.780738+0000","last_became_peered":"2024-04-25T08:36:35.780738+0000","last_unstale":"2024-04-25T08:36:37.632765+0000","last_undegraded":"2024-04-25T08:36:37.632765+0000","last_fullsized":"2024-04-25T08:36:37.632765+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.772985+0000","last_change":"2024-04-25T08:36:35.770246+0000","last_active":"2024-04-25T08:36:36.772985+0000","last_peered":"2024-04-25T08:36:36.772985+0000","last_clean":"2024-04-25T08:36:36.772985+0000","last_became_active":"2024-04-25T08:36:35.769772+0000","last_became_peered":"2024-04-25T08:36:35.769772+0000","last_unstale":"2024-04-25T08:36:36.772985+0000","last_undegraded":"2024-04-25T08:36:36.772985+0000","last_fullsized":"2024-04-25T08:36:36.772985+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:34.050120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632391+0000","last_change":"2024-04-25T08:36:35.778334+0000","last_active":"2024-04-25T08:36:37.632391+0000","last_peered":"2024-04-25T08:36:37.632391+0000","last_clean":"2024-04-25T08:36:37.632391+0000","last_became_active":"2024-04-25T08:36:35.778004+0000","last_became_peered":"2024-04-25T08:36:35.778004+0000","last_unstale":"2024-04-25T08:36:37.632391+0000","last_undegraded":"2024-04-25T08:36:37.632391+0000","last_fullsized":"2024-04-25T08:36:37.632391+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.324942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632607+0000","last_change":"2024-04-25T08:36:35.771141+0000","last_active":"2024-04-25T08:36:37.632607+0000","last_peered":"2024-04-25T08:36:37.632607+0000","last_clean":"2024-04-25T08:36:37.632607+0000","last_became_active":"2024-04-25T08:36:35.770800+0000","last_became_peered":"2024-04-25T08:36:35.770800+0000","last_unstale":"2024-04-25T08:36:37.632607+0000","last_undegraded":"2024-04-25T08:36:37.632607+0000","last_fullsized":"2024-04-25T08:36:37.632607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.324942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632599+0000","last_change":"2024-04-25T08:36:35.760599+0000","last_active":"2024-04-25T08:36:37.632599+0000","last_peered":"2024-04-25T08:36:37.632599+0000","last_clean":"2024-04-25T08:36:37.632599+0000","last_became_active":"2024-04-25T08:36:35.760304+0000","last_became_peered":"2024-04-25T08:36:35.760304+0000","last_unstale":"2024-04-25T08:36:37.632599+0000","last_undegraded":"2024-04-25T08:36:37.632599+0000","last_fullsized":"2024-04-25T08:36:37.632599+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775468+0000","last_change":"2024-04-25T08:36:35.768841+0000","last_active":"2024-04-25T08:36:36.775468+0000","last_peered":"2024-04-25T08:36:36.775468+0000","last_clean":"2024-04-25T08:36:36.775468+0000","last_became_active":"2024-04-25T08:36:35.768565+0000","last_became_peered":"2024-04-25T08:36:35.768565+0000","last_unstale":"2024-04-25T08:36:36.775468+0000","last_undegraded":"2024-04-25T08:36:36.775468+0000","last_fullsized":"2024-04-25T08:36:36.775468+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.432682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755047+0000","last_change":"2024-04-25T08:36:35.764415+0000","last_active":"2024-04-25T08:36:36.755047+0000","last_peered":"2024-04-25T08:36:36.755047+0000","last_clean":"2024-04-25T08:36:36.755047+0000","last_became_active":"2024-04-25T08:36:35.764127+0000","last_became_peered":"2024-04-25T08:36:35.764127+0000","last_unstale":"2024-04-25T08:36:36.755047+0000","last_undegraded":"2024-04-25T08:36:36.755047+0000","last_fullsized":"2024-04-25T08:36:36.755047+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:11.432682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753640+0000","last_change":"2024-04-25T08:36:35.766280+0000","last_active":"2024-04-25T08:36:36.753640+0000","last_peered":"2024-04-25T08:36:36.753640+0000","last_clean":"2024-04-25T08:36:36.753640+0000","last_became_active":"2024-04-25T08:36:35.765973+0000","last_became_peered":"2024-04-25T08:36:35.765973+0000","last_unstale":"2024-04-25T08:36:36.753640+0000","last_undegraded":"2024-04-25T08:36:36.753640+0000","last_fullsized":"2024-04-25T08:36:36.753640+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:50.665169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632249+0000","last_change":"2024-04-25T08:36:35.765091+0000","last_active":"2024-04-25T08:36:37.632249+0000","last_peered":"2024-04-25T08:36:37.632249+0000","last_clean":"2024-04-25T08:36:37.632249+0000","last_became_active":"2024-04-25T08:36:35.761657+0000","last_became_peered":"2024-04-25T08:36:35.761657+0000","last_unstale":"2024-04-25T08:36:37.632249+0000","last_undegraded":"2024-04-25T08:36:37.632249+0000","last_fullsized":"2024-04-25T08:36:37.632249+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.876976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754819+0000","last_change":"2024-04-25T08:36:35.767653+0000","last_active":"2024-04-25T08:36:36.754819+0000","last_peered":"2024-04-25T08:36:36.754819+0000","last_clean":"2024-04-25T08:36:36.754819+0000","last_became_active":"2024-04-25T08:36:35.767382+0000","last_became_peered":"2024-04-25T08:36:35.767382+0000","last_unstale":"2024-04-25T08:36:36.754819+0000","last_undegraded":"2024-04-25T08:36:36.754819+0000","last_fullsized":"2024-04-25T08:36:36.754819+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753801+0000","last_change":"2024-04-25T08:36:35.759860+0000","last_active":"2024-04-25T08:36:36.753801+0000","last_peered":"2024-04-25T08:36:36.753801+0000","last_clean":"2024-04-25T08:36:36.753801+0000","last_became_active":"2024-04-25T08:36:35.757908+0000","last_became_peered":"2024-04-25T08:36:35.757908+0000","last_unstale":"2024-04-25T08:36:36.753801+0000","last_undegraded":"2024-04-25T08:36:36.753801+0000","last_fullsized":"2024-04-25T08:36:36.753801+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:58.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773290+0000","last_change":"2024-04-25T08:36:35.776724+0000","last_active":"2024-04-25T08:36:36.773290+0000","last_peered":"2024-04-25T08:36:36.773290+0000","last_clean":"2024-04-25T08:36:36.773290+0000","last_became_active":"2024-04-25T08:36:35.775472+0000","last_became_peered":"2024-04-25T08:36:35.775472+0000","last_unstale":"2024-04-25T08:36:36.773290+0000","last_undegraded":"2024-04-25T08:36:36.773290+0000","last_fullsized":"2024-04-25T08:36:36.773290+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:59.981529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"109'4","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559579+0000","last_change":"2024-04-25T08:36:34.755252+0000","last_active":"2024-04-25T08:36:37.559579+0000","last_peered":"2024-04-25T08:36:37.559579+0000","last_clean":"2024-04-25T08:36:37.559579+0000","last_became_active":"2024-04-25T08:36:34.754950+0000","last_became_peered":"2024-04-25T08:36:34.754950+0000","last_unstale":"2024-04-25T08:36:37.559579+0000","last_undegraded":"2024-04-25T08:36:37.559579+0000","last_fullsized":"2024-04-25T08:36:37.559579+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753411+0000","last_change":"2024-04-25T08:36:35.775255+0000","last_active":"2024-04-25T08:36:36.753411+0000","last_peered":"2024-04-25T08:36:36.753411+0000","last_clean":"2024-04-25T08:36:36.753411+0000","last_became_active":"2024-04-25T08:36:35.774709+0000","last_became_peered":"2024-04-25T08:36:35.774709+0000","last_unstale":"2024-04-25T08:36:36.753411+0000","last_undegraded":"2024-04-25T08:36:36.753411+0000","last_fullsized":"2024-04-25T08:36:36.753411+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:42.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753750+0000","last_change":"2024-04-25T08:36:34.759886+0000","last_active":"2024-04-25T08:36:36.753750+0000","last_peered":"2024-04-25T08:36:36.753750+0000","last_clean":"2024-04-25T08:36:36.753750+0000","last_became_active":"2024-04-25T08:36:34.759596+0000","last_became_peered":"2024-04-25T08:36:34.759596+0000","last_unstale":"2024-04-25T08:36:36.753750+0000","last_undegraded":"2024-04-25T08:36:36.753750+0000","last_fullsized":"2024-04-25T08:36:36.753750+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:34.582682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632368+0000","last_change":"2024-04-25T08:36:35.784382+0000","last_active":"2024-04-25T08:36:37.632368+0000","last_peered":"2024-04-25T08:36:37.632368+0000","last_clean":"2024-04-25T08:36:37.632368+0000","last_became_active":"2024-04-25T08:36:35.784152+0000","last_became_peered":"2024-04-25T08:36:35.784152+0000","last_unstale":"2024-04-25T08:36:37.632368+0000","last_undegraded":"2024-04-25T08:36:37.632368+0000","last_fullsized":"2024-04-25T08:36:37.632368+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632479+0000","last_change":"2024-04-25T08:36:34.759233+0000","last_active":"2024-04-25T08:36:37.632479+0000","last_peered":"2024-04-25T08:36:37.632479+0000","last_clean":"2024-04-25T08:36:37.632479+0000","last_became_active":"2024-04-25T08:36:34.758929+0000","last_became_peered":"2024-04-25T08:36:34.758929+0000","last_unstale":"2024-04-25T08:36:37.632479+0000","last_undegraded":"2024-04-25T08:36:37.632479+0000","last_fullsized":"2024-04-25T08:36:37.632479+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775279+0000","last_change":"2024-04-25T08:36:35.773004+0000","last_active":"2024-04-25T08:36:36.775279+0000","last_peered":"2024-04-25T08:36:36.775279+0000","last_clean":"2024-04-25T08:36:36.775279+0000","last_became_active":"2024-04-25T08:36:35.772491+0000","last_became_peered":"2024-04-25T08:36:35.772491+0000","last_unstale":"2024-04-25T08:36:36.775279+0000","last_undegraded":"2024-04-25T08:36:36.775279+0000","last_fullsized":"2024-04-25T08:36:36.775279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632631+0000","last_change":"2024-04-25T08:36:34.761740+0000","last_active":"2024-04-25T08:36:37.632631+0000","last_peered":"2024-04-25T08:36:37.632631+0000","last_clean":"2024-04-25T08:36:37.632631+0000","last_became_active":"2024-04-25T08:36:34.761387+0000","last_became_peered":"2024-04-25T08:36:34.761387+0000","last_unstale":"2024-04-25T08:36:37.632631+0000","last_undegraded":"2024-04-25T08:36:37.632631+0000","last_fullsized":"2024-04-25T08:36:37.632631+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:39.476164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559599+0000","last_change":"2024-04-25T08:36:35.772127+0000","last_active":"2024-04-25T08:36:37.559599+0000","last_peered":"2024-04-25T08:36:37.559599+0000","last_clean":"2024-04-25T08:36:37.559599+0000","last_became_active":"2024-04-25T08:36:35.771810+0000","last_became_peered":"2024-04-25T08:36:35.771810+0000","last_unstale":"2024-04-25T08:36:37.559599+0000","last_undegraded":"2024-04-25T08:36:37.559599+0000","last_fullsized":"2024-04-25T08:36:37.559599+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.324942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754436+0000","last_change":"2024-04-25T08:36:34.752807+0000","last_active":"2024-04-25T08:36:36.754436+0000","last_peered":"2024-04-25T08:36:36.754436+0000","last_clean":"2024-04-25T08:36:36.754436+0000","last_became_active":"2024-04-25T08:36:34.752501+0000","last_became_peered":"2024-04-25T08:36:34.752501+0000","last_unstale":"2024-04-25T08:36:36.754436+0000","last_undegraded":"2024-04-25T08:36:36.754436+0000","last_fullsized":"2024-04-25T08:36:36.754436+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.320853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773398+0000","last_change":"2024-04-25T08:36:35.769214+0000","last_active":"2024-04-25T08:36:36.773398+0000","last_peered":"2024-04-25T08:36:36.773398+0000","last_clean":"2024-04-25T08:36:36.773398+0000","last_became_active":"2024-04-25T08:36:35.768497+0000","last_became_peered":"2024-04-25T08:36:35.768497+0000","last_unstale":"2024-04-25T08:36:36.773398+0000","last_undegraded":"2024-04-25T08:36:36.773398+0000","last_fullsized":"2024-04-25T08:36:36.773398+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:26.185943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775136+0000","last_change":"2024-04-25T08:36:34.761095+0000","last_active":"2024-04-25T08:36:36.775136+0000","last_peered":"2024-04-25T08:36:36.775136+0000","last_clean":"2024-04-25T08:36:36.775136+0000","last_became_active":"2024-04-25T08:36:34.760735+0000","last_became_peered":"2024-04-25T08:36:34.760735+0000","last_unstale":"2024-04-25T08:36:36.775136+0000","last_undegraded":"2024-04-25T08:36:36.775136+0000","last_fullsized":"2024-04-25T08:36:36.775136+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755653+0000","last_change":"2024-04-25T08:36:35.764305+0000","last_active":"2024-04-25T08:36:36.755653+0000","last_peered":"2024-04-25T08:36:36.755653+0000","last_clean":"2024-04-25T08:36:36.755653+0000","last_became_active":"2024-04-25T08:36:35.764067+0000","last_became_peered":"2024-04-25T08:36:35.764067+0000","last_unstale":"2024-04-25T08:36:36.755653+0000","last_undegraded":"2024-04-25T08:36:36.755653+0000","last_fullsized":"2024-04-25T08:36:36.755653+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:50.062303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632118+0000","last_change":"2024-04-25T08:36:34.758831+0000","last_active":"2024-04-25T08:36:37.632118+0000","last_peered":"2024-04-25T08:36:37.632118+0000","last_clean":"2024-04-25T08:36:37.632118+0000","last_became_active":"2024-04-25T08:36:34.758382+0000","last_became_peered":"2024-04-25T08:36:34.758382+0000","last_unstale":"2024-04-25T08:36:37.632118+0000","last_undegraded":"2024-04-25T08:36:37.632118+0000","last_fullsized":"2024-04-25T08:36:37.632118+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:58.532684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559588+0000","last_change":"2024-04-25T08:36:35.773357+0000","last_active":"2024-04-25T08:36:37.559588+0000","last_peered":"2024-04-25T08:36:37.559588+0000","last_clean":"2024-04-25T08:36:37.559588+0000","last_became_active":"2024-04-25T08:36:35.773073+0000","last_became_peered":"2024-04-25T08:36:35.773073+0000","last_unstale":"2024-04-25T08:36:37.559588+0000","last_undegraded":"2024-04-25T08:36:37.559588+0000","last_fullsized":"2024-04-25T08:36:37.559588+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559561+0000","last_change":"2024-04-25T08:36:34.758227+0000","last_active":"2024-04-25T08:36:37.559561+0000","last_peered":"2024-04-25T08:36:37.559561+0000","last_clean":"2024-04-25T08:36:37.559561+0000","last_became_active":"2024-04-25T08:36:34.757872+0000","last_became_peered":"2024-04-25T08:36:34.757872+0000","last_unstale":"2024-04-25T08:36:37.559561+0000","last_undegraded":"2024-04-25T08:36:37.559561+0000","last_fullsized":"2024-04-25T08:36:37.559561+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773131+0000","last_change":"2024-04-25T08:36:35.779985+0000","last_active":"2024-04-25T08:36:36.773131+0000","last_peered":"2024-04-25T08:36:36.773131+0000","last_clean":"2024-04-25T08:36:36.773131+0000","last_became_active":"2024-04-25T08:36:35.776392+0000","last_became_peered":"2024-04-25T08:36:35.776392+0000","last_unstale":"2024-04-25T08:36:36.773131+0000","last_undegraded":"2024-04-25T08:36:36.773131+0000","last_fullsized":"2024-04-25T08:36:36.773131+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:39.519994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632623+0000","last_change":"2024-04-25T08:36:34.759067+0000","last_active":"2024-04-25T08:36:37.632623+0000","last_peered":"2024-04-25T08:36:37.632623+0000","last_clean":"2024-04-25T08:36:37.632623+0000","last_became_active":"2024-04-25T08:36:34.758669+0000","last_became_peered":"2024-04-25T08:36:34.758669+0000","last_unstale":"2024-04-25T08:36:37.632623+0000","last_undegraded":"2024-04-25T08:36:37.632623+0000","last_fullsized":"2024-04-25T08:36:37.632623+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632385+0000","last_change":"2024-04-25T08:36:35.771331+0000","last_active":"2024-04-25T08:36:37.632385+0000","last_peered":"2024-04-25T08:36:37.632385+0000","last_clean":"2024-04-25T08:36:37.632385+0000","last_became_active":"2024-04-25T08:36:35.771020+0000","last_became_peered":"2024-04-25T08:36:35.771020+0000","last_unstale":"2024-04-25T08:36:37.632385+0000","last_undegraded":"2024-04-25T08:36:37.632385+0000","last_fullsized":"2024-04-25T08:36:37.632385+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:00.370889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.772957+0000","last_change":"2024-04-25T08:36:34.759931+0000","last_active":"2024-04-25T08:36:36.772957+0000","last_peered":"2024-04-25T08:36:36.772957+0000","last_clean":"2024-04-25T08:36:36.772957+0000","last_became_active":"2024-04-25T08:36:34.759599+0000","last_became_peered":"2024-04-25T08:36:34.759599+0000","last_unstale":"2024-04-25T08:36:36.772957+0000","last_undegraded":"2024-04-25T08:36:36.772957+0000","last_fullsized":"2024-04-25T08:36:36.772957+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:58.107799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753677+0000","last_change":"2024-04-25T08:36:35.784649+0000","last_active":"2024-04-25T08:36:36.753677+0000","last_peered":"2024-04-25T08:36:36.753677+0000","last_clean":"2024-04-25T08:36:36.753677+0000","last_became_active":"2024-04-25T08:36:35.784214+0000","last_became_peered":"2024-04-25T08:36:35.784214+0000","last_unstale":"2024-04-25T08:36:36.753677+0000","last_undegraded":"2024-04-25T08:36:36.753677+0000","last_fullsized":"2024-04-25T08:36:36.753677+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:10.004537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754442+0000","last_change":"2024-04-25T08:36:34.753588+0000","last_active":"2024-04-25T08:36:36.754442+0000","last_peered":"2024-04-25T08:36:36.754442+0000","last_clean":"2024-04-25T08:36:36.754442+0000","last_became_active":"2024-04-25T08:36:34.753290+0000","last_became_peered":"2024-04-25T08:36:34.753290+0000","last_unstale":"2024-04-25T08:36:36.754442+0000","last_undegraded":"2024-04-25T08:36:36.754442+0000","last_fullsized":"2024-04-25T08:36:36.754442+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.378269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632761+0000","last_change":"2024-04-25T08:36:35.771939+0000","last_active":"2024-04-25T08:36:37.632761+0000","last_peered":"2024-04-25T08:36:37.632761+0000","last_clean":"2024-04-25T08:36:37.632761+0000","last_became_active":"2024-04-25T08:36:35.771654+0000","last_became_peered":"2024-04-25T08:36:35.771654+0000","last_unstale":"2024-04-25T08:36:37.632761+0000","last_undegraded":"2024-04-25T08:36:37.632761+0000","last_fullsized":"2024-04-25T08:36:37.632761+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:40.152676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755019+0000","last_change":"2024-04-25T08:36:34.750641+0000","last_active":"2024-04-25T08:36:36.755019+0000","last_peered":"2024-04-25T08:36:36.755019+0000","last_clean":"2024-04-25T08:36:36.755019+0000","last_became_active":"2024-04-25T08:36:34.748802+0000","last_became_peered":"2024-04-25T08:36:34.748802+0000","last_unstale":"2024-04-25T08:36:36.755019+0000","last_undegraded":"2024-04-25T08:36:36.755019+0000","last_fullsized":"2024-04-25T08:36:36.755019+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:39.476164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775248+0000","last_change":"2024-04-25T08:36:35.768603+0000","last_active":"2024-04-25T08:36:36.775248+0000","last_peered":"2024-04-25T08:36:36.775248+0000","last_clean":"2024-04-25T08:36:36.775248+0000","last_became_active":"2024-04-25T08:36:35.766863+0000","last_became_peered":"2024-04-25T08:36:35.766863+0000","last_unstale":"2024-04-25T08:36:36.775248+0000","last_undegraded":"2024-04-25T08:36:36.775248+0000","last_fullsized":"2024-04-25T08:36:36.775248+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632814+0000","last_change":"2024-04-25T08:36:34.761126+0000","last_active":"2024-04-25T08:36:37.632814+0000","last_peered":"2024-04-25T08:36:37.632814+0000","last_clean":"2024-04-25T08:36:37.632814+0000","last_became_active":"2024-04-25T08:36:34.760848+0000","last_became_peered":"2024-04-25T08:36:34.760848+0000","last_unstale":"2024-04-25T08:36:37.632814+0000","last_undegraded":"2024-04-25T08:36:37.632814+0000","last_fullsized":"2024-04-25T08:36:37.632814+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755327+0000","last_change":"2024-04-25T08:36:35.769783+0000","last_active":"2024-04-25T08:36:36.755327+0000","last_peered":"2024-04-25T08:36:36.755327+0000","last_clean":"2024-04-25T08:36:36.755327+0000","last_became_active":"2024-04-25T08:36:35.769101+0000","last_became_peered":"2024-04-25T08:36:35.769101+0000","last_unstale":"2024-04-25T08:36:36.755327+0000","last_undegraded":"2024-04-25T08:36:36.755327+0000","last_fullsized":"2024-04-25T08:36:36.755327+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:34.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559425+0000","last_change":"2024-04-25T08:36:34.756585+0000","last_active":"2024-04-25T08:36:37.559425+0000","last_peered":"2024-04-25T08:36:37.559425+0000","last_clean":"2024-04-25T08:36:37.559425+0000","last_became_active":"2024-04-25T08:36:34.756301+0000","last_became_peered":"2024-04-25T08:36:34.756301+0000","last_unstale":"2024-04-25T08:36:37.559425+0000","last_undegraded":"2024-04-25T08:36:37.559425+0000","last_fullsized":"2024-04-25T08:36:37.559425+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753713+0000","last_change":"2024-04-25T08:36:35.757369+0000","last_active":"2024-04-25T08:36:36.753713+0000","last_peered":"2024-04-25T08:36:36.753713+0000","last_clean":"2024-04-25T08:36:36.753713+0000","last_became_active":"2024-04-25T08:36:35.757047+0000","last_became_peered":"2024-04-25T08:36:35.757047+0000","last_unstale":"2024-04-25T08:36:36.753713+0000","last_undegraded":"2024-04-25T08:36:36.753713+0000","last_fullsized":"2024-04-25T08:36:36.753713+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:05.027915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753429+0000","last_change":"2024-04-25T08:36:34.761194+0000","last_active":"2024-04-25T08:36:36.753429+0000","last_peered":"2024-04-25T08:36:36.753429+0000","last_clean":"2024-04-25T08:36:36.753429+0000","last_became_active":"2024-04-25T08:36:34.760927+0000","last_became_peered":"2024-04-25T08:36:34.760927+0000","last_unstale":"2024-04-25T08:36:36.753429+0000","last_undegraded":"2024-04-25T08:36:36.753429+0000","last_fullsized":"2024-04-25T08:36:36.753429+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:49.408800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773399+0000","last_change":"2024-04-25T08:36:35.770731+0000","last_active":"2024-04-25T08:36:36.773399+0000","last_peered":"2024-04-25T08:36:36.773399+0000","last_clean":"2024-04-25T08:36:36.773399+0000","last_became_active":"2024-04-25T08:36:35.767752+0000","last_became_peered":"2024-04-25T08:36:35.767752+0000","last_unstale":"2024-04-25T08:36:36.773399+0000","last_undegraded":"2024-04-25T08:36:36.773399+0000","last_fullsized":"2024-04-25T08:36:36.773399+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:06.311510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"109'8","reported_seq":22,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632098+0000","last_change":"2024-04-25T08:36:34.759616+0000","last_active":"2024-04-25T08:36:37.632098+0000","last_peered":"2024-04-25T08:36:37.632098+0000","last_clean":"2024-04-25T08:36:37.632098+0000","last_became_active":"2024-04-25T08:36:34.759260+0000","last_became_peered":"2024-04-25T08:36:34.759260+0000","last_unstale":"2024-04-25T08:36:37.632098+0000","last_undegraded":"2024-04-25T08:36:37.632098+0000","last_fullsized":"2024-04-25T08:36:37.632098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.257828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753947+0000","last_change":"2024-04-25T08:36:35.773611+0000","last_active":"2024-04-25T08:36:36.753947+0000","last_peered":"2024-04-25T08:36:36.753947+0000","last_clean":"2024-04-25T08:36:36.753947+0000","last_became_active":"2024-04-25T08:36:35.773312+0000","last_became_peered":"2024-04-25T08:36:35.773312+0000","last_unstale":"2024-04-25T08:36:36.753947+0000","last_undegraded":"2024-04-25T08:36:36.753947+0000","last_fullsized":"2024-04-25T08:36:36.753947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:50.133858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755663+0000","last_change":"2024-04-25T08:36:34.753294+0000","last_active":"2024-04-25T08:36:36.755663+0000","last_peered":"2024-04-25T08:36:36.755663+0000","last_clean":"2024-04-25T08:36:36.755663+0000","last_became_active":"2024-04-25T08:36:34.753016+0000","last_became_peered":"2024-04-25T08:36:34.753016+0000","last_unstale":"2024-04-25T08:36:36.755663+0000","last_undegraded":"2024-04-25T08:36:36.755663+0000","last_fullsized":"2024-04-25T08:36:36.755663+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754634+0000","last_change":"2024-04-25T08:36:35.767817+0000","last_active":"2024-04-25T08:36:36.754634+0000","last_peered":"2024-04-25T08:36:36.754634+0000","last_clean":"2024-04-25T08:36:36.754634+0000","last_became_active":"2024-04-25T08:36:35.767517+0000","last_became_peered":"2024-04-25T08:36:35.767517+0000","last_unstale":"2024-04-25T08:36:36.754634+0000","last_undegraded":"2024-04-25T08:36:36.754634+0000","last_fullsized":"2024-04-25T08:36:36.754634+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:01.503849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754516+0000","last_change":"2024-04-25T08:36:34.759604+0000","last_active":"2024-04-25T08:36:36.754516+0000","last_peered":"2024-04-25T08:36:36.754516+0000","last_clean":"2024-04-25T08:36:36.754516+0000","last_became_active":"2024-04-25T08:36:34.759303+0000","last_became_peered":"2024-04-25T08:36:34.759303+0000","last_unstale":"2024-04-25T08:36:36.754516+0000","last_undegraded":"2024-04-25T08:36:36.754516+0000","last_fullsized":"2024-04-25T08:36:36.754516+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:20.258277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632536+0000","last_change":"2024-04-25T08:36:35.769378+0000","last_active":"2024-04-25T08:36:37.632536+0000","last_peered":"2024-04-25T08:36:37.632536+0000","last_clean":"2024-04-25T08:36:37.632536+0000","last_became_active":"2024-04-25T08:36:35.768831+0000","last_became_peered":"2024-04-25T08:36:35.768831+0000","last_unstale":"2024-04-25T08:36:37.632536+0000","last_undegraded":"2024-04-25T08:36:37.632536+0000","last_fullsized":"2024-04-25T08:36:37.632536+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:22.382358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775262+0000","last_change":"2024-04-25T08:36:34.757404+0000","last_active":"2024-04-25T08:36:36.775262+0000","last_peered":"2024-04-25T08:36:36.775262+0000","last_clean":"2024-04-25T08:36:36.775262+0000","last_became_active":"2024-04-25T08:36:34.755586+0000","last_became_peered":"2024-04-25T08:36:34.755586+0000","last_unstale":"2024-04-25T08:36:36.775262+0000","last_undegraded":"2024-04-25T08:36:36.775262+0000","last_fullsized":"2024-04-25T08:36:36.775262+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.916526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773127+0000","last_change":"2024-04-25T08:36:35.772669+0000","last_active":"2024-04-25T08:36:36.773127+0000","last_peered":"2024-04-25T08:36:36.773127+0000","last_clean":"2024-04-25T08:36:36.773127+0000","last_became_active":"2024-04-25T08:36:35.772355+0000","last_became_peered":"2024-04-25T08:36:35.772355+0000","last_unstale":"2024-04-25T08:36:36.773127+0000","last_undegraded":"2024-04-25T08:36:36.773127+0000","last_fullsized":"2024-04-25T08:36:36.773127+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:28.991299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753825+0000","last_change":"2024-04-25T08:36:34.749438+0000","last_active":"2024-04-25T08:36:36.753825+0000","last_peered":"2024-04-25T08:36:36.753825+0000","last_clean":"2024-04-25T08:36:36.753825+0000","last_became_active":"2024-04-25T08:36:34.749073+0000","last_became_peered":"2024-04-25T08:36:34.749073+0000","last_unstale":"2024-04-25T08:36:36.753825+0000","last_undegraded":"2024-04-25T08:36:36.753825+0000","last_fullsized":"2024-04-25T08:36:36.753825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:02.519556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773320+0000","last_change":"2024-04-25T08:36:35.772062+0000","last_active":"2024-04-25T08:36:36.773320+0000","last_peered":"2024-04-25T08:36:36.773320+0000","last_clean":"2024-04-25T08:36:36.773320+0000","last_became_active":"2024-04-25T08:36:35.771679+0000","last_became_peered":"2024-04-25T08:36:35.771679+0000","last_unstale":"2024-04-25T08:36:36.773320+0000","last_undegraded":"2024-04-25T08:36:36.773320+0000","last_fullsized":"2024-04-25T08:36:36.773320+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:07.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753385+0000","last_change":"2024-04-25T08:36:34.755127+0000","last_active":"2024-04-25T08:36:36.753385+0000","last_peered":"2024-04-25T08:36:36.753385+0000","last_clean":"2024-04-25T08:36:36.753385+0000","last_became_active":"2024-04-25T08:36:34.754781+0000","last_became_peered":"2024-04-25T08:36:34.754781+0000","last_unstale":"2024-04-25T08:36:36.753385+0000","last_undegraded":"2024-04-25T08:36:36.753385+0000","last_fullsized":"2024-04-25T08:36:36.753385+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:18.047116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559600+0000","last_change":"2024-04-25T08:36:35.765959+0000","last_active":"2024-04-25T08:36:37.559600+0000","last_peered":"2024-04-25T08:36:37.559600+0000","last_clean":"2024-04-25T08:36:37.559600+0000","last_became_active":"2024-04-25T08:36:35.765681+0000","last_became_peered":"2024-04-25T08:36:35.765681+0000","last_unstale":"2024-04-25T08:36:37.559600+0000","last_undegraded":"2024-04-25T08:36:37.559600+0000","last_fullsized":"2024-04-25T08:36:37.559600+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:21.262366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754792+0000","last_change":"2024-04-25T08:36:34.756454+0000","last_active":"2024-04-25T08:36:36.754792+0000","last_peered":"2024-04-25T08:36:36.754792+0000","last_clean":"2024-04-25T08:36:36.754792+0000","last_became_active":"2024-04-25T08:36:34.755238+0000","last_became_peered":"2024-04-25T08:36:34.755238+0000","last_unstale":"2024-04-25T08:36:36.754792+0000","last_undegraded":"2024-04-25T08:36:36.754792+0000","last_fullsized":"2024-04-25T08:36:36.754792+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:09.185643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753490+0000","last_change":"2024-04-25T08:36:35.773402+0000","last_active":"2024-04-25T08:36:36.753490+0000","last_peered":"2024-04-25T08:36:36.753490+0000","last_clean":"2024-04-25T08:36:36.753490+0000","last_became_active":"2024-04-25T08:36:35.773069+0000","last_became_peered":"2024-04-25T08:36:35.773069+0000","last_unstale":"2024-04-25T08:36:36.753490+0000","last_undegraded":"2024-04-25T08:36:36.753490+0000","last_fullsized":"2024-04-25T08:36:36.753490+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:21.682944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753886+0000","last_change":"2024-04-25T08:36:34.760255+0000","last_active":"2024-04-25T08:36:36.753886+0000","last_peered":"2024-04-25T08:36:36.753886+0000","last_clean":"2024-04-25T08:36:36.753886+0000","last_became_active":"2024-04-25T08:36:34.759634+0000","last_became_peered":"2024-04-25T08:36:34.759634+0000","last_unstale":"2024-04-25T08:36:36.753886+0000","last_undegraded":"2024-04-25T08:36:36.753886+0000","last_fullsized":"2024-04-25T08:36:36.753886+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:38.153287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.772989+0000","last_change":"2024-04-25T08:36:35.766707+0000","last_active":"2024-04-25T08:36:36.772989+0000","last_peered":"2024-04-25T08:36:36.772989+0000","last_clean":"2024-04-25T08:36:36.772989+0000","last_became_active":"2024-04-25T08:36:35.765655+0000","last_became_peered":"2024-04-25T08:36:35.765655+0000","last_unstale":"2024-04-25T08:36:36.772989+0000","last_undegraded":"2024-04-25T08:36:36.772989+0000","last_fullsized":"2024-04-25T08:36:36.772989+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:54.510797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632727+0000","last_change":"2024-04-25T08:36:34.760831+0000","last_active":"2024-04-25T08:36:37.632727+0000","last_peered":"2024-04-25T08:36:37.632727+0000","last_clean":"2024-04-25T08:36:37.632727+0000","last_became_active":"2024-04-25T08:36:34.760558+0000","last_became_peered":"2024-04-25T08:36:34.760558+0000","last_unstale":"2024-04-25T08:36:37.632727+0000","last_undegraded":"2024-04-25T08:36:37.632727+0000","last_fullsized":"2024-04-25T08:36:37.632727+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.520022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632137+0000","last_change":"2024-04-25T08:36:35.768900+0000","last_active":"2024-04-25T08:36:37.632137+0000","last_peered":"2024-04-25T08:36:37.632137+0000","last_clean":"2024-04-25T08:36:37.632137+0000","last_became_active":"2024-04-25T08:36:35.768578+0000","last_became_peered":"2024-04-25T08:36:35.768578+0000","last_unstale":"2024-04-25T08:36:37.632137+0000","last_undegraded":"2024-04-25T08:36:37.632137+0000","last_fullsized":"2024-04-25T08:36:37.632137+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:41.198578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"110'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.773121+0000","last_change":"2024-04-25T08:36:34.756998+0000","last_active":"2024-04-25T08:36:36.773121+0000","last_peered":"2024-04-25T08:36:36.773121+0000","last_clean":"2024-04-25T08:36:36.773121+0000","last_became_active":"2024-04-25T08:36:34.756057+0000","last_became_peered":"2024-04-25T08:36:34.756057+0000","last_unstale":"2024-04-25T08:36:36.773121+0000","last_undegraded":"2024-04-25T08:36:36.773121+0000","last_fullsized":"2024-04-25T08:36:36.773121+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:44.009668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.559338+0000","last_change":"2024-04-25T08:36:35.764749+0000","last_active":"2024-04-25T08:36:37.559338+0000","last_peered":"2024-04-25T08:36:37.559338+0000","last_clean":"2024-04-25T08:36:37.559338+0000","last_became_active":"2024-04-25T08:36:35.764397+0000","last_became_peered":"2024-04-25T08:36:35.764397+0000","last_unstale":"2024-04-25T08:36:37.559338+0000","last_undegraded":"2024-04-25T08:36:37.559338+0000","last_fullsized":"2024-04-25T08:36:37.559338+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:24.496262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.755285+0000","last_change":"2024-04-25T08:36:34.750357+0000","last_active":"2024-04-25T08:36:36.755285+0000","last_peered":"2024-04-25T08:36:36.755285+0000","last_clean":"2024-04-25T08:36:36.755285+0000","last_became_active":"2024-04-25T08:36:34.749684+0000","last_became_peered":"2024-04-25T08:36:34.749684+0000","last_unstale":"2024-04-25T08:36:36.755285+0000","last_undegraded":"2024-04-25T08:36:36.755285+0000","last_fullsized":"2024-04-25T08:36:36.755285+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775350+0000","last_change":"2024-04-25T08:36:35.773260+0000","last_active":"2024-04-25T08:36:36.775350+0000","last_peered":"2024-04-25T08:36:36.775350+0000","last_clean":"2024-04-25T08:36:36.775350+0000","last_became_active":"2024-04-25T08:36:35.773012+0000","last_became_peered":"2024-04-25T08:36:35.773012+0000","last_unstale":"2024-04-25T08:36:36.775350+0000","last_undegraded":"2024-04-25T08:36:36.775350+0000","last_fullsized":"2024-04-25T08:36:36.775350+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:30.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632638+0000","last_change":"2024-04-25T08:36:34.756400+0000","last_active":"2024-04-25T08:36:37.632638+0000","last_peered":"2024-04-25T08:36:37.632638+0000","last_clean":"2024-04-25T08:36:37.632638+0000","last_became_active":"2024-04-25T08:36:34.756114+0000","last_became_peered":"2024-04-25T08:36:34.756114+0000","last_unstale":"2024-04-25T08:36:37.632638+0000","last_undegraded":"2024-04-25T08:36:37.632638+0000","last_fullsized":"2024-04-25T08:36:37.632638+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:27.630484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753577+0000","last_change":"2024-04-25T08:36:35.774222+0000","last_active":"2024-04-25T08:36:36.753577+0000","last_peered":"2024-04-25T08:36:36.753577+0000","last_clean":"2024-04-25T08:36:36.753577+0000","last_became_active":"2024-04-25T08:36:35.773918+0000","last_became_peered":"2024-04-25T08:36:35.773918+0000","last_unstale":"2024-04-25T08:36:36.753577+0000","last_undegraded":"2024-04-25T08:36:36.753577+0000","last_fullsized":"2024-04-25T08:36:36.753577+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:25.093418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:37.632263+0000","last_change":"2024-04-25T08:36:34.757608+0000","last_active":"2024-04-25T08:36:37.632263+0000","last_peered":"2024-04-25T08:36:37.632263+0000","last_clean":"2024-04-25T08:36:37.632263+0000","last_became_active":"2024-04-25T08:36:34.756201+0000","last_became_peered":"2024-04-25T08:36:34.756201+0000","last_unstale":"2024-04-25T08:36:37.632263+0000","last_undegraded":"2024-04-25T08:36:37.632263+0000","last_fullsized":"2024-04-25T08:36:37.632263+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:29.252859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753515+0000","last_change":"2024-04-25T08:36:35.772844+0000","last_active":"2024-04-25T08:36:36.753515+0000","last_peered":"2024-04-25T08:36:36.753515+0000","last_clean":"2024-04-25T08:36:36.753515+0000","last_became_active":"2024-04-25T08:36:35.772542+0000","last_became_peered":"2024-04-25T08:36:35.772542+0000","last_unstale":"2024-04-25T08:36:36.753515+0000","last_undegraded":"2024-04-25T08:36:36.753515+0000","last_fullsized":"2024-04-25T08:36:36.753515+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:40.452512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753782+0000","last_change":"2024-04-25T08:36:34.760614+0000","last_active":"2024-04-25T08:36:36.753782+0000","last_peered":"2024-04-25T08:36:36.753782+0000","last_clean":"2024-04-25T08:36:36.753782+0000","last_became_active":"2024-04-25T08:36:34.760357+0000","last_became_peered":"2024-04-25T08:36:34.760357+0000","last_unstale":"2024-04-25T08:36:36.753782+0000","last_undegraded":"2024-04-25T08:36:36.753782+0000","last_fullsized":"2024-04-25T08:36:36.753782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:11:52.472803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754548+0000","last_change":"2024-04-25T08:36:35.766389+0000","last_active":"2024-04-25T08:36:36.754548+0000","last_peered":"2024-04-25T08:36:36.754548+0000","last_clean":"2024-04-25T08:36:36.754548+0000","last_became_active":"2024-04-25T08:36:35.766116+0000","last_became_peered":"2024-04-25T08:36:35.766116+0000","last_unstale":"2024-04-25T08:36:36.754548+0000","last_undegraded":"2024-04-25T08:36:36.754548+0000","last_fullsized":"2024-04-25T08:36:36.754548+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:27.643034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754879+0000","last_change":"2024-04-25T08:36:34.754257+0000","last_active":"2024-04-25T08:36:36.754879+0000","last_peered":"2024-04-25T08:36:36.754879+0000","last_clean":"2024-04-25T08:36:36.754879+0000","last_became_active":"2024-04-25T08:36:34.752327+0000","last_became_peered":"2024-04-25T08:36:34.752327+0000","last_unstale":"2024-04-25T08:36:36.754879+0000","last_undegraded":"2024-04-25T08:36:36.754879+0000","last_fullsized":"2024-04-25T08:36:36.754879+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:04.247802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775545+0000","last_change":"2024-04-25T08:36:35.770356+0000","last_active":"2024-04-25T08:36:36.775545+0000","last_peered":"2024-04-25T08:36:36.775545+0000","last_clean":"2024-04-25T08:36:36.775545+0000","last_became_active":"2024-04-25T08:36:35.769027+0000","last_became_peered":"2024-04-25T08:36:35.769027+0000","last_unstale":"2024-04-25T08:36:36.775545+0000","last_undegraded":"2024-04-25T08:36:36.775545+0000","last_fullsized":"2024-04-25T08:36:36.775545+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:34.226575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753946+0000","last_change":"2024-04-25T08:36:34.762590+0000","last_active":"2024-04-25T08:36:36.753946+0000","last_peered":"2024-04-25T08:36:36.753946+0000","last_clean":"2024-04-25T08:36:36.753946+0000","last_became_active":"2024-04-25T08:36:34.762255+0000","last_became_peered":"2024-04-25T08:36:34.762255+0000","last_unstale":"2024-04-25T08:36:36.753946+0000","last_undegraded":"2024-04-25T08:36:36.753946+0000","last_fullsized":"2024-04-25T08:36:36.753946+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:02.692776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.775154+0000","last_change":"2024-04-25T08:36:35.769174+0000","last_active":"2024-04-25T08:36:36.775154+0000","last_peered":"2024-04-25T08:36:36.775154+0000","last_clean":"2024-04-25T08:36:36.775154+0000","last_became_active":"2024-04-25T08:36:35.768769+0000","last_became_peered":"2024-04-25T08:36:35.768769+0000","last_unstale":"2024-04-25T08:36:36.775154+0000","last_undegraded":"2024-04-25T08:36:36.775154+0000","last_fullsized":"2024-04-25T08:36:36.775154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:34.737374+0000","last_clean_scrub_stamp":"2024-04-25T08:36:34.737374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:50.324942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.753825+0000","last_change":"2024-04-25T08:36:34.755845+0000","last_active":"2024-04-25T08:36:36.753825+0000","last_peered":"2024-04-25T08:36:36.753825+0000","last_clean":"2024-04-25T08:36:36.753825+0000","last_became_active":"2024-04-25T08:36:34.755540+0000","last_became_peered":"2024-04-25T08:36:34.755540+0000","last_unstale":"2024-04-25T08:36:36.753825+0000","last_undegraded":"2024-04-25T08:36:36.753825+0000","last_fullsized":"2024-04-25T08:36:36.753825+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:33.733285+0000","last_clean_scrub_stamp":"2024-04-25T08:36:33.733285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:29.036927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":338,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T08:36:36.754417+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:36:36.754417+0000","last_peered":"2024-04-25T08:36:36.754417+0000","last_clean":"2024-04-25T08:36:36.754417+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:36:36.754417+0000","last_undegraded":"2024-04-25T08:36:36.754417+0000","last_fullsized":"2024-04-25T08:36:36.754417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542238,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41424,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330416,"statfs":{"total":96636764160,"available":96594345984,"internally_reserved":0,"allocated":1961984,"data_stored":1223271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542238,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46416,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325424,"statfs":{"total":96636764160,"available":96589234176,"internally_reserved":0,"allocated":1380352,"data_stored":634959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542238,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40828,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331012,"statfs":{"total":96636764160,"available":96594956288,"internally_reserved":0,"allocated":1359872,"data_stored":632743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50564,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321276,"statfs":{"total":96636764160,"available":96584986624,"internally_reserved":0,"allocated":1368064,"data_stored":632835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54724,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317116,"statfs":{"total":96636764160,"available":96580726784,"internally_reserved":0,"allocated":1363968,"data_stored":633325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54720,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317120,"statfs":{"total":96636764160,"available":96580730880,"internally_reserved":0,"allocated":1363968,"data_stored":632777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50560,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321280,"statfs":{"total":96636764160,"available":96584990720,"internally_reserved":0,"allocated":1363968,"data_stored":632777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1970176,"data_stored":1224813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:43.570 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:36:43.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:36:43.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.685+0000 7f05fe06f640 1 Processor -- start 2024-04-25T08:36:43.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.685+0000 7f05fe06f640 1 -- start start 2024-04-25T08:36:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05fe06f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05fe06f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 0x7f05f80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05fe06f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f815e000 con 0x7f05f8161710 2024-04-25T08:36:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05fe06f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f80d1de0 con 0x7f05f80d32f0 2024-04-25T08:36:43.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05fe06f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8161c60 con 0x7f05f815dc20 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05f7fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 0x7f05f80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:34776/0 (socket says 172.21.15.60:34776) 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05f6ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f05f8161710 0x7f05f806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:46838/0 (socket says 172.21.15.60:46838) 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05f7fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 0x7f05f80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58198/0 (socket says 172.21.15.60:58198) 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.686+0000 7f05f77fe640 1 -- 172.21.15.60:0/1124567818 learned_addr learned my addr 172.21.15.60:0/1124567818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.687+0000 7f05f77fe640 1 -- 172.21.15.60:0/1124567818 >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 msgr2=0x7f05f80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.687+0000 7f05f77fe640 1 --2- 172.21.15.60:0/1124567818 >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 0x7f05f80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.687+0000 7f05f77fe640 1 -- 172.21.15.60:0/1124567818 >> v1:172.21.15.100:6789/0 conn(0x7f05f8161710 legacy=0x7f05f806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.687+0000 7f05f77fe640 1 -- 172.21.15.60:0/1124567818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8161f40 con 0x7f05f815dc20 2024-04-25T08:36:43.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.687+0000 7f05f7fff640 1 --2- 172.21.15.60:0/1124567818 >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 0x7f05f80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.687+0000 7f05f77fe640 1 --2- 172.21.15.60:0/1124567818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f80ce670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009870 tx=0x7f05e802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d37f24eb346db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05f67fc640 1 -- 172.21.15.60:0/1124567818 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037070 con 0x7f05f815dc20 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05f67fc640 1 -- 172.21.15.60:0/1124567818 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8032440 con 0x7f05f815dc20 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05f67fc640 1 -- 172.21.15.60:0/1124567818 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8032810 con 0x7f05f815dc20 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 -- 172.21.15.60:0/1124567818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 msgr2=0x7f05f80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 --2- 172.21.15.60:0/1124567818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f80ce670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009870 tx=0x7f05e802a270 comp rx=0 tx=0).stop 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 -- 172.21.15.60:0/1124567818 shutdown_connections 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 --2- 172.21.15.60:0/1124567818 >> v2:172.21.15.100:3300/0 conn(0x7f05f80d32f0 0x7f05f80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 --2- 172.21.15.60:0/1124567818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f80ce670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 -- 172.21.15.60:0/1124567818 >> 172.21.15.60:0/1124567818 conn(0x7f05f80d0420 msgr2=0x7f05f80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 -- 172.21.15.60:0/1124567818 shutdown_connections 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.688+0000 7f05fe06f640 1 -- 172.21.15.60:0/1124567818 wait complete. 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.689+0000 7f05fe06f640 1 Processor -- start 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.689+0000 7f05fe06f640 1 -- start start 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05fe06f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 0x7f05f814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05fe06f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f81e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05fe06f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f8161710 0x7f05f81e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05fe06f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80b5e50 con 0x7f05f80d32f0 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05fe06f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8162500 con 0x7f05f8161710 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05fe06f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80b5fd0 con 0x7f05f815dc20 2024-04-25T08:36:43.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 0x7f05f814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 0x7f05f814fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56446/0 (socket says 172.21.15.60:56446) 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05f77fe640 1 -- 172.21.15.60:0/297205172 learned_addr learned my addr 172.21.15.60:0/297205172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05f6ffd640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f81e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.690+0000 7f05f7fff640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f8161710 0x7f05f81e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f77fe640 1 -- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 msgr2=0x7f05f81e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f77fe640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f81e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f77fe640 1 -- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f8161710 msgr2=0x7f05f81e9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f77fe640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f8161710 0x7f05f81e9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f77fe640 1 -- 172.21.15.60:0/297205172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f815fbe0 con 0x7f05f80d32f0 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f6ffd640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f81e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05f77fe640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 0x7f05f814fc60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009870 tx=0x7f05e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:43.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8041070 con 0x7f05f80d32f0 2024-04-25T08:36:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e803c890 con 0x7f05f80d32f0 2024-04-25T08:36:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f815fe80 con 0x7f05f80d32f0 2024-04-25T08:36:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037040 con 0x7f05f80d32f0 2024-04-25T08:36:43.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.691+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f8160440 con 0x7f05f80d32f0 2024-04-25T08:36:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.692+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f80d16b0 con 0x7f05f80d32f0 2024-04-25T08:36:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.693+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05e8044070 con 0x7f05f80d32f0 2024-04-25T08:36:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.693+0000 7f05dffff640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05e406b4d0 0x7f05e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.693+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05e80aad30 con 0x7f05f80d32f0 2024-04-25T08:36:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.693+0000 7f05f6ffd640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05e406b4d0 0x7f05e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.694+0000 7f05f6ffd640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05e406b4d0 0x7f05e406d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f05ec007fd0 tx=0x7f05ec007d60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:43.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.697+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05e8079330 con 0x7f05f80d32f0 2024-04-25T08:36:43.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.953+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f05f80739e0 con 0x7f05f80d32f0 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.955+0000 7f05dffff640 1 -- 172.21.15.60:0/297205172 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f05e807d3d0 con 0x7f05f80d32f0 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:36:43.950 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:36:43.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05e406b4d0 msgr2=0x7f05e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05e406b4d0 0x7f05e406d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f05ec007fd0 tx=0x7f05ec007d60 comp rx=0 tx=0).stop 2024-04-25T08:36:43.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 msgr2=0x7f05f814fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 0x7f05f814fc60 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009870 tx=0x7f05e80046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 shutdown_connections 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05e406b4d0 0x7f05e406d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f8161710 0x7f05f81e9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05f815dc20 0x7f05f81e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 --2- 172.21.15.60:0/297205172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05f80d32f0 0x7f05f814fc60 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 >> 172.21.15.60:0/297205172 conn(0x7f05f80d0420 msgr2=0x7f05f80cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 shutdown_connections 2024-04-25T08:36:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:43.958+0000 7f05fe06f640 1 -- 172.21.15.60:0/297205172 wait complete. 2024-04-25T08:36:43.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.070+0000 7f642bf22640 1 Processor -- start 2024-04-25T08:36:44.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.070+0000 7f642bf22640 1 -- start start 2024-04-25T08:36:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f642bf22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f642415dbe0 0x7f64240d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f642bf22640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f64240b92a0 0x7f64240d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f642bf22640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64240b97a0 con 0x7f64240b2cb0 2024-04-25T08:36:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f642bf22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642415dfc0 con 0x7f642415dbe0 2024-04-25T08:36:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f642bf22640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64240d1a00 con 0x7f64240b92a0 2024-04-25T08:36:44.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f6429cbe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f642415dbe0 0x7f64240d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f6429cbe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f642415dbe0 0x7f64240d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56456/0 (socket says 172.21.15.60:56456) 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.071+0000 7f6429cbe640 1 -- 172.21.15.60:0/1224923587 learned_addr learned my addr 172.21.15.60:0/1224923587 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.072+0000 7f6429cbe640 1 -- 172.21.15.60:0/1224923587 >> v2:172.21.15.100:3300/0 conn(0x7f64240b92a0 msgr2=0x7f64240d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.072+0000 7f642a4bf640 1 --2- 172.21.15.60:0/1224923587 >> v2:172.21.15.100:3300/0 conn(0x7f64240b92a0 0x7f64240d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.072+0000 7f6429cbe640 1 --2- 172.21.15.60:0/1224923587 >> v2:172.21.15.100:3300/0 conn(0x7f64240b92a0 0x7f64240d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.072+0000 7f6429cbe640 1 -- 172.21.15.60:0/1224923587 >> v1:172.21.15.100:6789/0 conn(0x7f64240b2cb0 legacy=0x7f64240b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.072+0000 7f6429cbe640 1 -- 172.21.15.60:0/1224923587 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240d1c50 con 0x7f642415dbe0 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.072+0000 7f6429cbe640 1 --2- 172.21.15.60:0/1224923587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642415dbe0 0x7f64240d0910 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f64240cfb90 tx=0x7f641c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1291e23e04f239a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f6428cbc640 1 -- 172.21.15.60:0/1224923587 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c037070 con 0x7f642415dbe0 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f6428cbc640 1 -- 172.21.15.60:0/1224923587 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c032440 con 0x7f642415dbe0 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f6428cbc640 1 -- 172.21.15.60:0/1224923587 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c032810 con 0x7f642415dbe0 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 -- 172.21.15.60:0/1224923587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642415dbe0 msgr2=0x7f64240d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 --2- 172.21.15.60:0/1224923587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642415dbe0 0x7f64240d0910 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f64240cfb90 tx=0x7f641c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 -- 172.21.15.60:0/1224923587 shutdown_connections 2024-04-25T08:36:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 --2- 172.21.15.60:0/1224923587 >> v2:172.21.15.100:3300/0 conn(0x7f64240b92a0 0x7f64240d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 --2- 172.21.15.60:0/1224923587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f642415dbe0 0x7f64240d0910 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 -- 172.21.15.60:0/1224923587 >> 172.21.15.60:0/1224923587 conn(0x7f6424061a20 msgr2=0x7f6424061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 -- 172.21.15.60:0/1224923587 shutdown_connections 2024-04-25T08:36:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.073+0000 7f642bf22640 1 -- 172.21.15.60:0/1224923587 wait complete. 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.074+0000 7f642bf22640 1 Processor -- start 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.074+0000 7f642bf22640 1 -- start start 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642bf22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64240b2cb0 0x7f64240c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642bf22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 0x7f64240b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642bf22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642415dbe0 0x7f64240b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642bf22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64240b6ab0 con 0x7f64240b92a0 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642bf22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64240d2210 con 0x7f642415dbe0 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642bf22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64240b6c30 con 0x7f64240b2cb0 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f64294bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 0x7f64240b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f64294bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 0x7f64240b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56470/0 (socket says 172.21.15.60:56470) 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f64294bd640 1 -- 172.21.15.60:0/2379884707 learned_addr learned my addr 172.21.15.60:0/2379884707 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f64294bd640 1 -- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64240b2cb0 msgr2=0x7f64240c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f64294bd640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64240b2cb0 0x7f64240c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.075+0000 7f642a4bf640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642415dbe0 0x7f64240b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64294bd640 1 -- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642415dbe0 msgr2=0x7f64240b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64294bd640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642415dbe0 0x7f64240b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64294bd640 1 -- 172.21.15.60:0/2379884707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240b33c0 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f642a4bf640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642415dbe0 0x7f64240b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64294bd640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 0x7f64240b3a20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f641800c400 tx=0x7f641800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418017070 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64180035f0 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180039e0 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64240cac60 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.076+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64240cb220 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.077+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6424073990 con 0x7f64240b92a0 2024-04-25T08:36:44.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.078+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f641801a070 con 0x7f64240b92a0 2024-04-25T08:36:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.078+0000 7f64227fc640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641006b4d0 0x7f641006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.078+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f64180896a0 con 0x7f64240b92a0 2024-04-25T08:36:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.078+0000 7f6429cbe640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641006b4d0 0x7f641006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.079+0000 7f6429cbe640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641006b4d0 0x7f641006d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f641c034000 tx=0x7f641c02c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.082+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f64240b33c0 con 0x7f64240b92a0 2024-04-25T08:36:44.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.298+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64240c9fb0 con 0x7f64240b92a0 2024-04-25T08:36:44.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.299+0000 7f64227fc640 1 -- 172.21.15.60:0/2379884707 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6418057d40 con 0x7f64240b92a0 2024-04-25T08:36:44.293 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:44.293 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:36:44.294 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641006b4d0 msgr2=0x7f641006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641006b4d0 0x7f641006d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f641c034000 tx=0x7f641c02c000 comp rx=0 tx=0).stop 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 msgr2=0x7f64240b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 0x7f64240b3a20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f641800c400 tx=0x7f641800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 shutdown_connections 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f641006b4d0 0x7f641006d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f642415dbe0 0x7f64240b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f64240b92a0 0x7f64240b3a20 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 --2- 172.21.15.60:0/2379884707 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f64240b2cb0 0x7f64240c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 >> 172.21.15.60:0/2379884707 conn(0x7f6424061a20 msgr2=0x7f64240d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 shutdown_connections 2024-04-25T08:36:44.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.301+0000 7f642bf22640 1 -- 172.21.15.60:0/2379884707 wait complete. 2024-04-25T08:36:44.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:36:35.118687+0000', 'modified': '2024-04-25T08:36:37.111723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8184}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8184': {'gid': 8184, 'name': 'c', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6835/4168808734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6833', 'nonce': 4168808734}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 4168808734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:36:44.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:36:44.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:36:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.824+0000 7f0026922640 1 Processor -- start 2024-04-25T08:36:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.824+0000 7f0026922640 1 -- start start 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f0026922640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00200ced80 0x7f00200d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f0026922640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f00200d1ce0 0x7f00200b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f0026922640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00200d2120 con 0x7f002015dc20 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f0026922640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00200d23a0 con 0x7f00200ced80 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f0026922640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00200d2680 con 0x7f00200d1ce0 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f001ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f002015dc20 0x7f00200ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:49062/0 (socket says 172.21.15.60:49062) 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f001ffff640 1 -- 172.21.15.60:0/2966039509 learned_addr learned my addr 172.21.15.60:0/2966039509 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f001f7fe640 1 --2- 172.21.15.60:0/2966039509 >> v2:172.21.15.60:3300/0 conn(0x7f00200ced80 0x7f00200d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.825+0000 7f0024ebf640 1 --2- 172.21.15.60:0/2966039509 >> v2:172.21.15.100:3300/0 conn(0x7f00200d1ce0 0x7f00200b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001effd640 1 -- 172.21.15.60:0/2966039509 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009368914 0 0) 0x7f0008001110 con 0x7f002015dc20 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001effd640 1 -- 172.21.15.60:0/2966039509 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f000c003520 con 0x7f002015dc20 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001f7fe640 1 -- 172.21.15.60:0/2966039509 >> v2:172.21.15.100:3300/0 conn(0x7f00200d1ce0 msgr2=0x7f00200b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001f7fe640 1 --2- 172.21.15.60:0/2966039509 >> v2:172.21.15.100:3300/0 conn(0x7f00200d1ce0 0x7f00200b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001f7fe640 1 -- 172.21.15.60:0/2966039509 >> v1:172.21.15.60:6789/0 conn(0x7f002015dc20 legacy=0x7f00200ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001f7fe640 1 -- 172.21.15.60:0/2966039509 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200d2960 con 0x7f00200ced80 2024-04-25T08:36:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001f7fe640 1 --2- 172.21.15.60:0/2966039509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200ced80 0x7f00200d3440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f00200b6050 tx=0x7f001002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2145cd7c568d6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001effd640 1 -- 172.21.15.60:0/2966039509 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010037070 con 0x7f00200ced80 2024-04-25T08:36:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001effd640 1 -- 172.21.15.60:0/2966039509 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010032440 con 0x7f00200ced80 2024-04-25T08:36:44.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.826+0000 7f001effd640 1 -- 172.21.15.60:0/2966039509 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001002ae60 con 0x7f00200ced80 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 -- 172.21.15.60:0/2966039509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200ced80 msgr2=0x7f00200d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 --2- 172.21.15.60:0/2966039509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200ced80 0x7f00200d3440 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f00200b6050 tx=0x7f001002a110 comp rx=0 tx=0).stop 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 -- 172.21.15.60:0/2966039509 shutdown_connections 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 --2- 172.21.15.60:0/2966039509 >> v2:172.21.15.100:3300/0 conn(0x7f00200d1ce0 0x7f00200b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 --2- 172.21.15.60:0/2966039509 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200ced80 0x7f00200d3440 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 -- 172.21.15.60:0/2966039509 >> 172.21.15.60:0/2966039509 conn(0x7f0020061a20 msgr2=0x7f0020061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 -- 172.21.15.60:0/2966039509 shutdown_connections 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.827+0000 7f0026922640 1 -- 172.21.15.60:0/2966039509 wait complete. 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.828+0000 7f0026922640 1 Processor -- start 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.828+0000 7f0026922640 1 -- start start 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f0026922640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00200ced80 0x7f00201f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f0026922640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 0x7f00201f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f0026922640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f002015dc20 0x7f00200c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f0026922640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200b6a30 con 0x7f00200d1ce0 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f0026922640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200d2f20 con 0x7f002015dc20 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f0026922640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200b6bb0 con 0x7f00200ced80 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 0x7f00201f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 0x7f00201f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56480/0 (socket says 172.21.15.60:56480) 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.829+0000 7f001f7fe640 1 -- 172.21.15.60:0/1185208908 learned_addr learned my addr 172.21.15.60:0/1185208908 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001ffff640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00200ced80 0x7f00201f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f0024ebf640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f002015dc20 0x7f00200c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001f7fe640 1 -- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00200ced80 msgr2=0x7f00201f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001f7fe640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00200ced80 0x7f00201f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001f7fe640 1 -- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f002015dc20 msgr2=0x7f00200c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001f7fe640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f002015dc20 0x7f00200c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001f7fe640 1 -- 172.21.15.60:0/1185208908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00201589a0 con 0x7f00200d1ce0 2024-04-25T08:36:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001f7fe640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 0x7f00201f6a50 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0010009890 tx=0x7f00100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001cff9640 1 -- 172.21.15.60:0/1185208908 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010037040 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001cff9640 1 -- 172.21.15.60:0/1185208908 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0010042070 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002015e1a0 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f001cff9640 1 -- 172.21.15.60:0/1185208908 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010030960 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.830+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0020155bf0 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.832+0000 7f001cff9640 1 -- 172.21.15.60:0/1185208908 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0010030b00 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.832+0000 7f001cff9640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f000006b4d0 0x7f000006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.832+0000 7f001cff9640 1 -- 172.21.15.60:0/1185208908 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00100aabb0 con 0x7f00200d1ce0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.832+0000 7f001ffff640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f000006b4d0 0x7f000006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.833+0000 7f001ffff640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f000006b4d0 0x7f000006d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f00080012b0 tx=0x7f0008009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.833+0000 7f0029622740 1 Processor -- start 2024-04-25T08:36:44.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.833+0000 7f0029622740 1 -- start start 2024-04-25T08:36:44.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a292f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0029622740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a294d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a2904f10 0x5647a294b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0029622740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a2867a10 con 0x5647a294cad0 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a2810b30 con 0x5647a292f230 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a2863c50 con 0x5647a2904f10 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a294d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a294d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56486/0 (socket says 172.21.15.60:56486) 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 -- 172.21.15.60:0/3168145010 learned_addr learned my addr 172.21.15.60:0/3168145010 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001ffff640 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a292f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f0024ebf640 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a2904f10 0x5647a294b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 -- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a2904f10 msgr2=0x5647a294b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a2904f10 0x5647a294b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 -- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 msgr2=0x5647a292f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a292f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 -- 172.21.15.60:0/3168145010 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a2825e10 con 0x5647a294cad0 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.834+0000 7f001f7fe640 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a294d020 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f00100091d0 tx=0x7f0010078b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bb1557a403af93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0015ffb640 1 -- 172.21.15.60:0/3168145010 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100a6020 con 0x5647a294cad0 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0015ffb640 1 -- 172.21.15.60:0/3168145010 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001007d170 con 0x5647a294cad0 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0015ffb640 1 -- 172.21.15.60:0/3168145010 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010079230 con 0x5647a294cad0 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 -- 172.21.15.60:0/3168145010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 msgr2=0x5647a294d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a294d020 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f00100091d0 tx=0x7f0010078b90 comp rx=0 tx=0).stop 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 -- 172.21.15.60:0/3168145010 shutdown_connections 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a2904f10 0x5647a294b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a294d020 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 --2- 172.21.15.60:0/3168145010 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a292f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 -- 172.21.15.60:0/3168145010 >> 172.21.15.60:0/3168145010 conn(0x5647a2901840 msgr2=0x5647a294a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 -- 172.21.15.60:0/3168145010 shutdown_connections 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.835+0000 7f0029622740 1 -- 172.21.15.60:0/3168145010 wait complete. 2024-04-25T08:36:44.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.836+0000 7f0029622740 1 -- start start 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.836+0000 7f0029622740 1 Processor -- start 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a292f230 0x5647a291da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a2942470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a2932cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a2867a10 con 0x5647a294cad0 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a28240c0 con 0x5647a291e400 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a2905cb0 con 0x5647a292f230 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0029622740 10 client.0 _ll_register_callbacks cb 0x5647a2817ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a2942470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a2942470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56494/0 (socket says 172.21.15.60:56494) 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 -- 172.21.15.60:0/3763249307 learned_addr learned my addr 172.21.15.60:0/3763249307 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001ffff640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a292f230 0x5647a291da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f0024ebf640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a2932cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 -- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a292f230 msgr2=0x5647a291da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a292f230 0x5647a291da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 -- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 msgr2=0x5647a2932cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a2932cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 -- 172.21.15.60:0/3763249307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647a257c2d0 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7f001f7fe640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a2942470 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0010000c00 tx=0x7f0010049070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d51cbc5b52434e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.837+0000 7efffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7efffeffd640 1 -- 172.21.15.60:0/3763249307 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010095070 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7efffeffd640 1 -- 172.21.15.60:0/3763249307 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0010077530 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f001007d170 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7efffeffd640 1 -- 172.21.15.60:0/3763249307 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00100a0050 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 10 client.8355 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5647a26466b0 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7efffeffd640 1 -- 172.21.15.60:0/3763249307 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001007d170 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7efffeffd640 1 -- 172.21.15.60:0/3763249307 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f001003a070 con 0x5647a294cad0 2024-04-25T08:36:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 10 client.8355 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 10 client.8355 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 20 client.8355 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2930f40 0x5647a2931320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 4 client.8355 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647a28c6920 con 0x5647a2930f40 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f001ffff640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2930f40 0x5647a2931320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7f001ffff640 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2930f40 0x5647a2931320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd67ed30ee6e793b server_cookie=69bbff5944040f6d in_seq=0 out_seq=0 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.838+0000 7efffeffd640 10 client.8355 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7efffeffd640 1 -- 172.21.15.60:0/3763249307 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647a28c6920 con 0x5647a2930f40 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7efffeffd640 10 client.8355 handle_command_reply: tid=0 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 client.8355 shutdown 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 msgr2=0x5647a2942470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a2942470 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0010000c00 tx=0x7f0010049070 comp rx=0 tx=0).stop 2024-04-25T08:36:44.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 shutdown_connections 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a2932cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2930f40 0x5647a2931320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a2942470 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 --2- 172.21.15.60:0/3763249307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a292f230 0x5647a291da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7efffeffd640 0 client.8355 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 >> 172.21.15.60:0/3763249307 conn(0x5647a2904f10 msgr2=0x5647a294de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 shutdown_connections 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.839+0000 7f0029622740 1 -- 172.21.15.60:0/3763249307 wait complete. 2024-04-25T08:36:44.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.840+0000 7f0029622740 20 client.8355 trim_cache size 0 max 16384 2024-04-25T08:36:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.855+0000 7f0029622740 1 Processor -- start 2024-04-25T08:36:44.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.855+0000 7f0029622740 1 -- start start 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a296d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f0029622740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 0x5647a2963790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a294cad0 0x5647a2963cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f0029622740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201ff680 con 0x5647a292f230 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002015f850 con 0x5647a291e400 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002015fe50 con 0x5647a294cad0 2024-04-25T08:36:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 0x5647a2963790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 0x5647a2963790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56510/0 (socket says 172.21.15.60:56510) 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 -- 172.21.15.60:0/2203931026 learned_addr learned my addr 172.21.15.60:0/2203931026 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 -- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a294cad0 msgr2=0x5647a2963cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a294cad0 0x5647a2963cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 -- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 msgr2=0x5647a296d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001ffff640 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a296d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a296d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001f7fe640 1 -- 172.21.15.60:0/2203931026 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a294cf80 con 0x5647a292f230 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.856+0000 7f001ffff640 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a296d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f001f7fe640 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 0x5647a2963790 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f00100774e0 tx=0x7f00100a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599ca4400f559d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f00157fa640 1 -- 172.21.15.60:0/2203931026 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010085070 con 0x5647a292f230 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f00157fa640 1 -- 172.21.15.60:0/2203931026 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001007d170 con 0x5647a292f230 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f00157fa640 1 -- 172.21.15.60:0/2203931026 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010095070 con 0x5647a292f230 2024-04-25T08:36:44.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 -- 172.21.15.60:0/2203931026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 msgr2=0x5647a2963790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 0x5647a2963790 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f00100774e0 tx=0x7f00100a6000 comp rx=0 tx=0).stop 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 -- 172.21.15.60:0/2203931026 shutdown_connections 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a294cad0 0x5647a2963cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a292f230 0x5647a2963790 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 --2- 172.21.15.60:0/2203931026 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a291e400 0x5647a296d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 -- 172.21.15.60:0/2203931026 >> 172.21.15.60:0/2203931026 conn(0x5647a2904f10 msgr2=0x5647a28fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 -- 172.21.15.60:0/2203931026 shutdown_connections 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.857+0000 7f0029622740 1 -- 172.21.15.60:0/2203931026 wait complete. 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.858+0000 7f0029622740 1 -- start start 2024-04-25T08:36:44.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.858+0000 7f0029622740 1 Processor -- start 2024-04-25T08:36:44.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a291e400 0x5647a2963790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a2963cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a29b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a2867a10 con 0x5647a294cad0 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a28240c0 con 0x5647a292f230 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647a291ee00 con 0x5647a291e400 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0029622740 10 client.0 _ll_register_callbacks cb 0x5647a28ffc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a29b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a29b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56514/0 (socket says 172.21.15.60:56514) 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 -- 172.21.15.60:0/944070395 learned_addr learned my addr 172.21.15.60:0/944070395 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f001ffff640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a291e400 0x5647a2963790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f001f7fe640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a2963cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 -- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a291e400 msgr2=0x5647a2963790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a291e400 0x5647a2963790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 -- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 msgr2=0x5647a2963cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a2963cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 -- 172.21.15.60:0/944070395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647a2905900 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7f0024ebf640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a29b3d20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x5647a292f750 tx=0x7f0018007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9ec8a88697acff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.859+0000 7efffeffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7efffeffd640 1 -- 172.21.15.60:0/944070395 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001801b070 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7efffeffd640 1 -- 172.21.15.60:0/944070395 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0018003ba0 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f001007d170 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7efffeffd640 1 -- 172.21.15.60:0/944070395 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0018026050 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 10 client.8361 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5647a2930c20 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7efffeffd640 1 -- 172.21.15.60:0/944070395 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001007d170 con 0x5647a294cad0 2024-04-25T08:36:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7efffeffd640 1 -- 172.21.15.60:0/944070395 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0018029020 con 0x5647a294cad0 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 10 client.8361 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 10 client.8361 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 20 client.8361 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2903ee0 0x5647a29bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 4 client.8361 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5647a2874f50 con 0x5647a2903ee0 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.860+0000 7f001ffff640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2903ee0 0x5647a29bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7efffeffd640 10 client.8361 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f001ffff640 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2903ee0 0x5647a29bfe00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f50972fcad5c13 server_cookie=9a3a279ed7183ab4 in_seq=0 out_seq=0 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7efffeffd640 1 -- 172.21.15.60:0/944070395 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5647a2874f50 con 0x5647a2903ee0 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7efffeffd640 10 client.8361 handle_command_reply: tid=0 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 client.8361 shutdown 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 msgr2=0x5647a29b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a29b3d20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x5647a292f750 tx=0x7f0018007d80 comp rx=0 tx=0).stop 2024-04-25T08:36:44.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 shutdown_connections 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5647a2903ee0 0x5647a29bfe00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5647a294cad0 0x5647a29b3d20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7efffeffd640 0 client.8361 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647a292f230 0x5647a2963cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 --2- 172.21.15.60:0/944070395 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5647a291e400 0x5647a2963790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 >> 172.21.15.60:0/944070395 conn(0x5647a2904f10 msgr2=0x5647a2941c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 shutdown_connections 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 1 -- 172.21.15.60:0/944070395 wait complete. 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.861+0000 7f0029622740 20 client.8361 trim_cache size 0 max 16384 2024-04-25T08:36:44.862 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8184, 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.0991092560000002, 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 62, 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 110, 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 110, 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.124069929999999 2024-04-25T08:36:44.863 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f000006b4d0 msgr2=0x7f000006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f000006b4d0 0x7f000006d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f00080012b0 tx=0x7f0008009040 comp rx=0 tx=0).stop 2024-04-25T08:36:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 msgr2=0x7f00201f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 0x7f00201f6a50 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f0010009890 tx=0x7f00100046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 shutdown_connections 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f000006b4d0 0x7f000006d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f002015dc20 0x7f00200c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00200d1ce0 0x7f00201f6a50 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 --2- 172.21.15.60:0/1185208908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00200ced80 0x7f00201f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 >> 172.21.15.60:0/1185208908 conn(0x7f0020061a20 msgr2=0x7f00200b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 shutdown_connections 2024-04-25T08:36:44.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.862+0000 7f0026922640 1 -- 172.21.15.60:0/1185208908 wait complete. 2024-04-25T08:36:44.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:36:44.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.973+0000 7f14dae58640 1 Processor -- start 2024-04-25T08:36:44.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.973+0000 7f14dae58640 1 -- start start 2024-04-25T08:36:44.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14dae58640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f14d40cfec0 0x7f14d40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14dae58640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14d40ce3c0 0x7f14d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14dae58640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d4071480 con 0x7f14d4154ee0 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14dae58640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d40ce830 con 0x7f14d40ce3c0 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14dae58640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d4072090 con 0x7f14d40cfec0 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14d40ce3c0 0x7f14d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f14d40ce3c0 0x7f14d406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56530/0 (socket says 172.21.15.60:56530) 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 -- 172.21.15.60:0/3516853003 learned_addr learned my addr 172.21.15.60:0/3516853003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d3fff640 1 --2- 172.21.15.60:0/3516853003 >> v2:172.21.15.100:3300/0 conn(0x7f14d40cfec0 0x7f14d40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 -- 172.21.15.60:0/3516853003 >> v2:172.21.15.100:3300/0 conn(0x7f14d40cfec0 msgr2=0x7f14d40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 --2- 172.21.15.60:0/3516853003 >> v2:172.21.15.100:3300/0 conn(0x7f14d40cfec0 0x7f14d40cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 -- 172.21.15.60:0/3516853003 >> v1:172.21.15.100:6789/0 conn(0x7f14d4154ee0 legacy=0x7f14d40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d93f5640 1 -- 172.21.15.60:0/3516853003 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d4071760 con 0x7f14d40ce3c0 2024-04-25T08:36:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.974+0000 7f14d3fff640 1 --2- 172.21.15.60:0/3516853003 >> v2:172.21.15.100:3300/0 conn(0x7f14d40cfec0 0x7f14d40cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:44.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.975+0000 7f14d93f5640 1 --2- 172.21.15.60:0/3516853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d406d6f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f14d40cf560 tx=0x7f14cc02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06dc0b2a958d3da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.975+0000 7f14d37fe640 1 -- 172.21.15.60:0/3516853003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00b2e0 con 0x7f14d40ce3c0 2024-04-25T08:36:44.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.975+0000 7f14d37fe640 1 -- 172.21.15.60:0/3516853003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc037070 con 0x7f14d40ce3c0 2024-04-25T08:36:44.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.975+0000 7f14d37fe640 1 -- 172.21.15.60:0/3516853003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00b480 con 0x7f14d40ce3c0 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 -- 172.21.15.60:0/3516853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 msgr2=0x7f14d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 --2- 172.21.15.60:0/3516853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d406d6f0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f14d40cf560 tx=0x7f14cc02c940 comp rx=0 tx=0).stop 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 -- 172.21.15.60:0/3516853003 shutdown_connections 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 --2- 172.21.15.60:0/3516853003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d406d6f0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 --2- 172.21.15.60:0/3516853003 >> v2:172.21.15.100:3300/0 conn(0x7f14d40cfec0 0x7f14d40cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 -- 172.21.15.60:0/3516853003 >> 172.21.15.60:0/3516853003 conn(0x7f14d406abe0 msgr2=0x7f14d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 -- 172.21.15.60:0/3516853003 shutdown_connections 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.976+0000 7f14dae58640 1 -- 172.21.15.60:0/3516853003 wait complete. 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.977+0000 7f14dae58640 1 Processor -- start 2024-04-25T08:36:44.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.977+0000 7f14dae58640 1 -- start start 2024-04-25T08:36:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14dae58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d41f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14dae58640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d40cfec0 0x7f14d415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14dae58640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d4154ee0 0x7f14d41536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14dae58640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d4071d20 con 0x7f14d40ce3c0 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14dae58640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d4071230 con 0x7f14d40cfec0 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14dae58640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d41552c0 con 0x7f14d4154ee0 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d41f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d41f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56544/0 (socket says 172.21.15.60:56544) 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 -- 172.21.15.60:0/2002660927 learned_addr learned my addr 172.21.15.60:0/2002660927 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d93f5640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d4154ee0 0x7f14d41536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d3fff640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d40cfec0 0x7f14d415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 -- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d4154ee0 msgr2=0x7f14d41536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d4154ee0 0x7f14d41536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 -- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d40cfec0 msgr2=0x7f14d415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d40cfec0 0x7f14d415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:44.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.978+0000 7f14d8bf4640 1 -- 172.21.15.60:0/2002660927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40a8660 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.979+0000 7f14d8bf4640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d41f3620 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f14c800c3c0 tx=0x7f14c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.979+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80052d0 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.979+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8005490 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.979+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d4157800 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.979+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8007260 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.979+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d40a8940 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.980+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d4073990 con 0x7f14d40ce3c0 2024-04-25T08:36:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.980+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14c8007400 con 0x7f14d40ce3c0 2024-04-25T08:36:44.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.981+0000 7f14d17fa640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b4d0 0x7f14b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:44.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.981+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14c8089880 con 0x7f14d40ce3c0 2024-04-25T08:36:44.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.981+0000 7f14d3fff640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b4d0 0x7f14b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:44.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.981+0000 7f14d3fff640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b4d0 0x7f14b806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f14d415aae0 tx=0x7f14c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:44.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:44.985+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14c8058560 con 0x7f14d40ce3c0 2024-04-25T08:36:45.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.206+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f14d406d360 con 0x7f14d40ce3c0 2024-04-25T08:36:45.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.207+0000 7f14d17fa640 1 -- 172.21.15.60:0/2002660927 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f14c8057e60 con 0x7f14d40ce3c0 2024-04-25T08:36:45.201 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:36:45.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b4d0 msgr2=0x7f14b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b4d0 0x7f14b806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f14d415aae0 tx=0x7f14c0008040 comp rx=0 tx=0).stop 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 msgr2=0x7f14d41f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d41f3620 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f14c800c3c0 tx=0x7f14c800c880 comp rx=0 tx=0).stop 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 shutdown_connections 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f14d4154ee0 0x7f14d41536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f14b806b4d0 0x7f14b806d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14d40cfec0 0x7f14d415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 --2- 172.21.15.60:0/2002660927 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14d40ce3c0 0x7f14d41f3620 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 >> 172.21.15.60:0/2002660927 conn(0x7f14d406abe0 msgr2=0x7f14d415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 shutdown_connections 2024-04-25T08:36:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:45.209+0000 7f14dae58640 1 -- 172.21.15.60:0/2002660927 wait complete. 2024-04-25T08:36:45.214 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:36:45.214 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:36:45.215 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:36:45.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.215 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:36:45.231 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42305sec preferred_lft 42305sec 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:36:45.232 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:45.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.233 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:36:45.294 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:36:45.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.295 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:36:45.352 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:36:45.352 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:36:45.352 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:36:45.352 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:36:45.352 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:36:45.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.353 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:36:45.353 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:45.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:36:45.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:36:45.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:36:45.353 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:45.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.438 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:36:45.453 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:36:45.453 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:36:45.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:36:45.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:36:45.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:36:45.518 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:45.576 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:36:45.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:36:45.576 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:45.631 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:36:45.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:36:45.714 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:36:45.714 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:36:45.714 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:36:45.722 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:36:45.723 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:36:45.723 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:36:45.723 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:36:45.723 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:36:45.729 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:36:45.729 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:36:45.729 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:36:45.730 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:36:45.730 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:36:45.730 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:36:45.730 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:36:45.730 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:36:45.731 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:36:45.731 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:36:45.731 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:36:45.731 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:36:45.732 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:36:45.733 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:36:45.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:45.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.800 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:36:45.863 INFO:teuthology.orchestra.run.smithi060.stdout:client7492 2024-04-25T08:36:45.864 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:36:45.864 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:45.920 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:45.920 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T08:36:45.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:45.920 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:45.921 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:45.921 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:36:45.921 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:45.921 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:45.921 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:45.921 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:45.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:45.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:45.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932303 s, 112 MB/s 2024-04-25T08:36:46.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.006 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084529 s, 124 MB/s 2024-04-25T08:36:46.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.015 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731534 s, 143 MB/s 2024-04-25T08:36:46.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710378 s, 148 MB/s 2024-04-25T08:36:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.031 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774985 s, 135 MB/s 2024-04-25T08:36:46.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.039 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698981 s, 150 MB/s 2024-04-25T08:36:46.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.047 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684418 s, 153 MB/s 2024-04-25T08:36:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861796 s, 122 MB/s 2024-04-25T08:36:46.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.064 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669724 s, 157 MB/s 2024-04-25T08:36:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669562 s, 157 MB/s 2024-04-25T08:36:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663009 s, 158 MB/s 2024-04-25T08:36:46.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582538 s, 180 MB/s 2024-04-25T08:36:46.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774364 s, 135 MB/s 2024-04-25T08:36:46.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717271 s, 146 MB/s 2024-04-25T08:36:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682326 s, 154 MB/s 2024-04-25T08:36:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.118 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619667 s, 169 MB/s 2024-04-25T08:36:46.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.126 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725629 s, 145 MB/s 2024-04-25T08:36:46.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.134 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668035 s, 157 MB/s 2024-04-25T08:36:46.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.141 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058772 s, 178 MB/s 2024-04-25T08:36:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679344 s, 154 MB/s 2024-04-25T08:36:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639138 s, 164 MB/s 2024-04-25T08:36:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655892 s, 160 MB/s 2024-04-25T08:36:46.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724455 s, 145 MB/s 2024-04-25T08:36:46.180 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.180 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.180 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781236 s, 134 MB/s 2024-04-25T08:36:46.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.188 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677037 s, 155 MB/s 2024-04-25T08:36:46.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063403 s, 165 MB/s 2024-04-25T08:36:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.203 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664894 s, 158 MB/s 2024-04-25T08:36:46.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.211 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703446 s, 149 MB/s 2024-04-25T08:36:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076157 s, 138 MB/s 2024-04-25T08:36:46.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787096 s, 133 MB/s 2024-04-25T08:36:46.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639408 s, 164 MB/s 2024-04-25T08:36:46.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.242 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059651 s, 176 MB/s 2024-04-25T08:36:46.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:46.243 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:46.243 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:46.244 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:36:46.244 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:46.244 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:46.244 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:46.244 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.308 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715116 s, 147 MB/s 2024-04-25T08:36:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.315 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057527 s, 182 MB/s 2024-04-25T08:36:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603145 s, 174 MB/s 2024-04-25T08:36:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063957 s, 164 MB/s 2024-04-25T08:36:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.336 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574666 s, 182 MB/s 2024-04-25T08:36:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632199 s, 166 MB/s 2024-04-25T08:36:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772763 s, 136 MB/s 2024-04-25T08:36:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.359 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632111 s, 166 MB/s 2024-04-25T08:36:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602188 s, 174 MB/s 2024-04-25T08:36:46.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.374 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660373 s, 159 MB/s 2024-04-25T08:36:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637027 s, 165 MB/s 2024-04-25T08:36:46.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685255 s, 153 MB/s 2024-04-25T08:36:46.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630387 s, 166 MB/s 2024-04-25T08:36:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734276 s, 143 MB/s 2024-04-25T08:36:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.412 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068925 s, 152 MB/s 2024-04-25T08:36:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.419 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641163 s, 164 MB/s 2024-04-25T08:36:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.427 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740684 s, 142 MB/s 2024-04-25T08:36:46.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.435 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686083 s, 153 MB/s 2024-04-25T08:36:46.442 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.442 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.442 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630235 s, 166 MB/s 2024-04-25T08:36:46.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.449 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598772 s, 175 MB/s 2024-04-25T08:36:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.456 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615173 s, 170 MB/s 2024-04-25T08:36:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636667 s, 165 MB/s 2024-04-25T08:36:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651455 s, 161 MB/s 2024-04-25T08:36:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.478 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654426 s, 160 MB/s 2024-04-25T08:36:46.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.486 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688427 s, 152 MB/s 2024-04-25T08:36:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669713 s, 157 MB/s 2024-04-25T08:36:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666866 s, 157 MB/s 2024-04-25T08:36:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.509 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613772 s, 171 MB/s 2024-04-25T08:36:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.516 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664825 s, 158 MB/s 2024-04-25T08:36:46.523 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643297 s, 163 MB/s 2024-04-25T08:36:46.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.532 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744875 s, 141 MB/s 2024-04-25T08:36:46.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631896 s, 166 MB/s 2024-04-25T08:36:46.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:46.541 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:46.542 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:46.542 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:36:46.542 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:46.542 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:46.542 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:46.542 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:46.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592406 s, 177 MB/s 2024-04-25T08:36:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641549 s, 163 MB/s 2024-04-25T08:36:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613951 s, 171 MB/s 2024-04-25T08:36:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655241 s, 160 MB/s 2024-04-25T08:36:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.634 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633398 s, 166 MB/s 2024-04-25T08:36:46.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.641 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622345 s, 168 MB/s 2024-04-25T08:36:46.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.648 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634437 s, 165 MB/s 2024-04-25T08:36:46.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.655 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609905 s, 172 MB/s 2024-04-25T08:36:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.663 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704787 s, 149 MB/s 2024-04-25T08:36:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.671 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593922 s, 177 MB/s 2024-04-25T08:36:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.678 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646642 s, 162 MB/s 2024-04-25T08:36:46.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.685 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599739 s, 175 MB/s 2024-04-25T08:36:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770553 s, 136 MB/s 2024-04-25T08:36:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.700 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617933 s, 170 MB/s 2024-04-25T08:36:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631569 s, 166 MB/s 2024-04-25T08:36:46.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719856 s, 146 MB/s 2024-04-25T08:36:46.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654526 s, 160 MB/s 2024-04-25T08:36:46.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.731 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694427 s, 151 MB/s 2024-04-25T08:36:46.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645715 s, 162 MB/s 2024-04-25T08:36:46.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625578 s, 168 MB/s 2024-04-25T08:36:46.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614099 s, 171 MB/s 2024-04-25T08:36:46.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719603 s, 146 MB/s 2024-04-25T08:36:46.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612316 s, 171 MB/s 2024-04-25T08:36:46.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.776 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729504 s, 144 MB/s 2024-04-25T08:36:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613552 s, 171 MB/s 2024-04-25T08:36:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.791 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633293 s, 166 MB/s 2024-04-25T08:36:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.797 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572386 s, 183 MB/s 2024-04-25T08:36:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.806 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792642 s, 132 MB/s 2024-04-25T08:36:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.814 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719257 s, 146 MB/s 2024-04-25T08:36:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.822 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663064 s, 158 MB/s 2024-04-25T08:36:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724684 s, 145 MB/s 2024-04-25T08:36:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678955 s, 154 MB/s 2024-04-25T08:36:46.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:46.839 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:46.839 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:46.839 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:36:46.839 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:46.840 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:46.840 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:46.840 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764009 s, 137 MB/s 2024-04-25T08:36:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.913 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675207 s, 155 MB/s 2024-04-25T08:36:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622695 s, 168 MB/s 2024-04-25T08:36:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723924 s, 145 MB/s 2024-04-25T08:36:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613272 s, 171 MB/s 2024-04-25T08:36:46.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645994 s, 162 MB/s 2024-04-25T08:36:46.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608347 s, 172 MB/s 2024-04-25T08:36:46.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634049 s, 165 MB/s 2024-04-25T08:36:46.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703931 s, 149 MB/s 2024-04-25T08:36:46.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.972 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622057 s, 169 MB/s 2024-04-25T08:36:46.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.979 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642755 s, 163 MB/s 2024-04-25T08:36:46.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.987 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694956 s, 151 MB/s 2024-04-25T08:36:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:46.994 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608008 s, 172 MB/s 2024-04-25T08:36:47.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062794 s, 167 MB/s 2024-04-25T08:36:47.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.010 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744325 s, 141 MB/s 2024-04-25T08:36:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737163 s, 142 MB/s 2024-04-25T08:36:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708625 s, 148 MB/s 2024-04-25T08:36:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609643 s, 172 MB/s 2024-04-25T08:36:47.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.040 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565733 s, 185 MB/s 2024-04-25T08:36:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.049 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791969 s, 132 MB/s 2024-04-25T08:36:47.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.056 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636352 s, 165 MB/s 2024-04-25T08:36:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.063 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063291 s, 166 MB/s 2024-04-25T08:36:47.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615942 s, 170 MB/s 2024-04-25T08:36:47.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.078 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653769 s, 160 MB/s 2024-04-25T08:36:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.084 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058154 s, 180 MB/s 2024-04-25T08:36:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589668 s, 178 MB/s 2024-04-25T08:36:47.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714101 s, 147 MB/s 2024-04-25T08:36:47.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776838 s, 135 MB/s 2024-04-25T08:36:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593587 s, 177 MB/s 2024-04-25T08:36:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612856 s, 171 MB/s 2024-04-25T08:36:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.129 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625493 s, 168 MB/s 2024-04-25T08:36:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.136 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619865 s, 169 MB/s 2024-04-25T08:36:47.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:47.137 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:47.137 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:47.137 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:36:47.138 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:47.138 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:47.138 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:47.138 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:47.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614474 s, 171 MB/s 2024-04-25T08:36:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.209 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071022 s, 148 MB/s 2024-04-25T08:36:47.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694776 s, 151 MB/s 2024-04-25T08:36:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066385 s, 158 MB/s 2024-04-25T08:36:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.232 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070815 s, 148 MB/s 2024-04-25T08:36:47.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651907 s, 161 MB/s 2024-04-25T08:36:47.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.246 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580386 s, 181 MB/s 2024-04-25T08:36:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577228 s, 182 MB/s 2024-04-25T08:36:47.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.261 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705816 s, 149 MB/s 2024-04-25T08:36:47.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601442 s, 174 MB/s 2024-04-25T08:36:47.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.275 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616273 s, 170 MB/s 2024-04-25T08:36:47.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.283 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753853 s, 139 MB/s 2024-04-25T08:36:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.291 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006578 s, 159 MB/s 2024-04-25T08:36:47.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604993 s, 173 MB/s 2024-04-25T08:36:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635497 s, 165 MB/s 2024-04-25T08:36:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.313 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630849 s, 166 MB/s 2024-04-25T08:36:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060178 s, 174 MB/s 2024-04-25T08:36:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593907 s, 177 MB/s 2024-04-25T08:36:47.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.333 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628678 s, 167 MB/s 2024-04-25T08:36:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578953 s, 181 MB/s 2024-04-25T08:36:47.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752109 s, 139 MB/s 2024-04-25T08:36:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.356 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702947 s, 149 MB/s 2024-04-25T08:36:47.363 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.364 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640276 s, 164 MB/s 2024-04-25T08:36:47.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.372 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744908 s, 141 MB/s 2024-04-25T08:36:47.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617342 s, 170 MB/s 2024-04-25T08:36:47.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.386 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577811 s, 181 MB/s 2024-04-25T08:36:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.394 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703481 s, 149 MB/s 2024-04-25T08:36:47.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070963 s, 148 MB/s 2024-04-25T08:36:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599067 s, 175 MB/s 2024-04-25T08:36:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.417 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070055 s, 150 MB/s 2024-04-25T08:36:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.424 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621197 s, 169 MB/s 2024-04-25T08:36:47.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.431 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586488 s, 179 MB/s 2024-04-25T08:36:47.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:47.432 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701767 s, 149 MB/s 2024-04-25T08:36:47.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613144 s, 171 MB/s 2024-04-25T08:36:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637485 s, 164 MB/s 2024-04-25T08:36:47.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639005 s, 164 MB/s 2024-04-25T08:36:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058299 s, 180 MB/s 2024-04-25T08:36:47.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662746 s, 158 MB/s 2024-04-25T08:36:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.539 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054714 s, 192 MB/s 2024-04-25T08:36:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700147 s, 150 MB/s 2024-04-25T08:36:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589422 s, 178 MB/s 2024-04-25T08:36:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.561 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619141 s, 169 MB/s 2024-04-25T08:36:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061984 s, 169 MB/s 2024-04-25T08:36:47.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.575 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607069 s, 173 MB/s 2024-04-25T08:36:47.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.583 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706625 s, 148 MB/s 2024-04-25T08:36:47.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612142 s, 171 MB/s 2024-04-25T08:36:47.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060691 s, 173 MB/s 2024-04-25T08:36:47.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638217 s, 164 MB/s 2024-04-25T08:36:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636884 s, 165 MB/s 2024-04-25T08:36:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619359 s, 169 MB/s 2024-04-25T08:36:47.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.626 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616002 s, 170 MB/s 2024-04-25T08:36:47.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770832 s, 136 MB/s 2024-04-25T08:36:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.642 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691849 s, 152 MB/s 2024-04-25T08:36:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655449 s, 160 MB/s 2024-04-25T08:36:47.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.657 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596059 s, 176 MB/s 2024-04-25T08:36:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604081 s, 174 MB/s 2024-04-25T08:36:47.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.671 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614672 s, 171 MB/s 2024-04-25T08:36:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721127 s, 145 MB/s 2024-04-25T08:36:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625105 s, 168 MB/s 2024-04-25T08:36:47.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595219 s, 176 MB/s 2024-04-25T08:36:47.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.700 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061105 s, 172 MB/s 2024-04-25T08:36:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596655 s, 176 MB/s 2024-04-25T08:36:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.714 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630673 s, 166 MB/s 2024-04-25T08:36:47.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.721 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585384 s, 179 MB/s 2024-04-25T08:36:47.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:47.722 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:47.722 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:47.722 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:36:47.723 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:47.723 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:47.723 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:47.723 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:47.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590561 s, 178 MB/s 2024-04-25T08:36:47.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762343 s, 138 MB/s 2024-04-25T08:36:47.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597747 s, 175 MB/s 2024-04-25T08:36:47.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.809 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607937 s, 172 MB/s 2024-04-25T08:36:47.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.817 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718389 s, 146 MB/s 2024-04-25T08:36:47.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654396 s, 160 MB/s 2024-04-25T08:36:47.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.831 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645842 s, 162 MB/s 2024-04-25T08:36:47.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.839 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065781 s, 159 MB/s 2024-04-25T08:36:47.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.847 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676077 s, 155 MB/s 2024-04-25T08:36:47.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.854 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634198 s, 165 MB/s 2024-04-25T08:36:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611909 s, 171 MB/s 2024-04-25T08:36:47.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.868 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589377 s, 178 MB/s 2024-04-25T08:36:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.875 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631408 s, 166 MB/s 2024-04-25T08:36:47.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711455 s, 147 MB/s 2024-04-25T08:36:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751437 s, 140 MB/s 2024-04-25T08:36:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006911 s, 152 MB/s 2024-04-25T08:36:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.906 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064173 s, 163 MB/s 2024-04-25T08:36:47.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.914 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658318 s, 159 MB/s 2024-04-25T08:36:47.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.922 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695734 s, 151 MB/s 2024-04-25T08:36:47.930 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.930 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.930 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717541 s, 146 MB/s 2024-04-25T08:36:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.938 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747268 s, 140 MB/s 2024-04-25T08:36:47.945 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.945 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.945 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063394 s, 165 MB/s 2024-04-25T08:36:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.952 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555214 s, 189 MB/s 2024-04-25T08:36:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.958 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559554 s, 187 MB/s 2024-04-25T08:36:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611569 s, 171 MB/s 2024-04-25T08:36:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.972 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598259 s, 175 MB/s 2024-04-25T08:36:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.979 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602438 s, 174 MB/s 2024-04-25T08:36:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610926 s, 172 MB/s 2024-04-25T08:36:47.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:47.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:47.994 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656814 s, 160 MB/s 2024-04-25T08:36:48.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.000 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00548137 s, 191 MB/s 2024-04-25T08:36:48.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.008 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690141 s, 152 MB/s 2024-04-25T08:36:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.015 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600513 s, 175 MB/s 2024-04-25T08:36:48.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:48.016 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:48.016 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:48.016 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:36:48.016 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:48.016 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:48.016 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:48.017 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062071 s, 169 MB/s 2024-04-25T08:36:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.087 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062297 s, 168 MB/s 2024-04-25T08:36:48.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645715 s, 162 MB/s 2024-04-25T08:36:48.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061659 s, 170 MB/s 2024-04-25T08:36:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588264 s, 178 MB/s 2024-04-25T08:36:48.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.116 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640988 s, 164 MB/s 2024-04-25T08:36:48.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.123 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606227 s, 173 MB/s 2024-04-25T08:36:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077278 s, 136 MB/s 2024-04-25T08:36:48.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.139 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640637 s, 164 MB/s 2024-04-25T08:36:48.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.147 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720142 s, 146 MB/s 2024-04-25T08:36:48.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.153 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594158 s, 176 MB/s 2024-04-25T08:36:48.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707642 s, 148 MB/s 2024-04-25T08:36:48.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600082 s, 175 MB/s 2024-04-25T08:36:48.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.176 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709538 s, 148 MB/s 2024-04-25T08:36:48.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564439 s, 186 MB/s 2024-04-25T08:36:48.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618211 s, 170 MB/s 2024-04-25T08:36:48.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.197 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595879 s, 176 MB/s 2024-04-25T08:36:48.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.204 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587572 s, 178 MB/s 2024-04-25T08:36:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569682 s, 184 MB/s 2024-04-25T08:36:48.218 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712535 s, 147 MB/s 2024-04-25T08:36:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.225 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614673 s, 171 MB/s 2024-04-25T08:36:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.234 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072982 s, 144 MB/s 2024-04-25T08:36:48.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570665 s, 184 MB/s 2024-04-25T08:36:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.248 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065959 s, 159 MB/s 2024-04-25T08:36:48.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.256 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682679 s, 154 MB/s 2024-04-25T08:36:48.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.262 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060896 s, 172 MB/s 2024-04-25T08:36:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.270 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657988 s, 159 MB/s 2024-04-25T08:36:48.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582233 s, 180 MB/s 2024-04-25T08:36:48.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.285 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726221 s, 144 MB/s 2024-04-25T08:36:48.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654727 s, 160 MB/s 2024-04-25T08:36:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620446 s, 169 MB/s 2024-04-25T08:36:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070301 s, 149 MB/s 2024-04-25T08:36:48.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:48.309 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.374 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618493 s, 170 MB/s 2024-04-25T08:36:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.380 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602116 s, 174 MB/s 2024-04-25T08:36:48.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.387 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599106 s, 175 MB/s 2024-04-25T08:36:48.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.394 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620095 s, 169 MB/s 2024-04-25T08:36:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.401 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610045 s, 172 MB/s 2024-04-25T08:36:48.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600549 s, 175 MB/s 2024-04-25T08:36:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637335 s, 165 MB/s 2024-04-25T08:36:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623325 s, 168 MB/s 2024-04-25T08:36:48.429 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.429 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.429 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593242 s, 177 MB/s 2024-04-25T08:36:48.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655602 s, 160 MB/s 2024-04-25T08:36:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610159 s, 172 MB/s 2024-04-25T08:36:48.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.450 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576238 s, 182 MB/s 2024-04-25T08:36:48.457 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.457 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.457 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567504 s, 185 MB/s 2024-04-25T08:36:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.464 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564513 s, 186 MB/s 2024-04-25T08:36:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.471 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687922 s, 152 MB/s 2024-04-25T08:36:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.479 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725546 s, 145 MB/s 2024-04-25T08:36:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.486 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582334 s, 180 MB/s 2024-04-25T08:36:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.493 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574804 s, 182 MB/s 2024-04-25T08:36:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.501 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719226 s, 146 MB/s 2024-04-25T08:36:48.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066108 s, 159 MB/s 2024-04-25T08:36:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.516 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626142 s, 167 MB/s 2024-04-25T08:36:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071852 s, 146 MB/s 2024-04-25T08:36:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.531 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613942 s, 171 MB/s 2024-04-25T08:36:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.538 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618003 s, 170 MB/s 2024-04-25T08:36:48.544 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.545 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603348 s, 174 MB/s 2024-04-25T08:36:48.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.552 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672923 s, 156 MB/s 2024-04-25T08:36:48.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.559 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614188 s, 171 MB/s 2024-04-25T08:36:48.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.566 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613566 s, 171 MB/s 2024-04-25T08:36:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597635 s, 175 MB/s 2024-04-25T08:36:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621133 s, 169 MB/s 2024-04-25T08:36:48.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.588 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641955 s, 163 MB/s 2024-04-25T08:36:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.595 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639488 s, 164 MB/s 2024-04-25T08:36:48.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:48.596 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:48.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.660 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604352 s, 174 MB/s 2024-04-25T08:36:48.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642355 s, 163 MB/s 2024-04-25T08:36:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627327 s, 167 MB/s 2024-04-25T08:36:48.681 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.681 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.681 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585123 s, 179 MB/s 2024-04-25T08:36:48.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.689 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712359 s, 147 MB/s 2024-04-25T08:36:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556968 s, 188 MB/s 2024-04-25T08:36:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.703 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632237 s, 166 MB/s 2024-04-25T08:36:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.710 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596366 s, 176 MB/s 2024-04-25T08:36:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.717 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613263 s, 171 MB/s 2024-04-25T08:36:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.724 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062756 s, 167 MB/s 2024-04-25T08:36:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.732 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675703 s, 155 MB/s 2024-04-25T08:36:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.739 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630984 s, 166 MB/s 2024-04-25T08:36:48.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603136 s, 174 MB/s 2024-04-25T08:36:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.755 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776356 s, 135 MB/s 2024-04-25T08:36:48.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.762 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654028 s, 160 MB/s 2024-04-25T08:36:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.770 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723629 s, 145 MB/s 2024-04-25T08:36:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.777 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604406 s, 173 MB/s 2024-04-25T08:36:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576538 s, 182 MB/s 2024-04-25T08:36:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.791 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602991 s, 174 MB/s 2024-04-25T08:36:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.798 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063624 s, 165 MB/s 2024-04-25T08:36:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.806 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694767 s, 151 MB/s 2024-04-25T08:36:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.814 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071531 s, 147 MB/s 2024-04-25T08:36:48.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.822 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651549 s, 161 MB/s 2024-04-25T08:36:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.829 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629815 s, 166 MB/s 2024-04-25T08:36:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.836 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626325 s, 167 MB/s 2024-04-25T08:36:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.846 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606228 s, 173 MB/s 2024-04-25T08:36:48.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.851 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.851 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614149 s, 171 MB/s 2024-04-25T08:36:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.858 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704311 s, 149 MB/s 2024-04-25T08:36:48.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.865 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608263 s, 172 MB/s 2024-04-25T08:36:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.872 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577757 s, 181 MB/s 2024-04-25T08:36:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.879 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567065 s, 185 MB/s 2024-04-25T08:36:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.886 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623629 s, 168 MB/s 2024-04-25T08:36:48.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:48.887 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581435 s, 180 MB/s 2024-04-25T08:36:48.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.958 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635084 s, 165 MB/s 2024-04-25T08:36:48.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623001 s, 168 MB/s 2024-04-25T08:36:48.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.972 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577625 s, 182 MB/s 2024-04-25T08:36:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.980 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649885 s, 161 MB/s 2024-04-25T08:36:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.987 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064018 s, 164 MB/s 2024-04-25T08:36:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:48.994 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612716 s, 171 MB/s 2024-04-25T08:36:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613619 s, 171 MB/s 2024-04-25T08:36:49.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.008 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624965 s, 168 MB/s 2024-04-25T08:36:49.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703341 s, 149 MB/s 2024-04-25T08:36:49.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.024 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751291 s, 140 MB/s 2024-04-25T08:36:49.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769652 s, 136 MB/s 2024-04-25T08:36:49.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702346 s, 149 MB/s 2024-04-25T08:36:49.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.048 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618836 s, 169 MB/s 2024-04-25T08:36:49.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.055 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552501 s, 190 MB/s 2024-04-25T08:36:49.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676131 s, 155 MB/s 2024-04-25T08:36:49.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725316 s, 145 MB/s 2024-04-25T08:36:49.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731465 s, 143 MB/s 2024-04-25T08:36:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.085 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584687 s, 179 MB/s 2024-04-25T08:36:49.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542724 s, 193 MB/s 2024-04-25T08:36:49.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607237 s, 173 MB/s 2024-04-25T08:36:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.107 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777426 s, 135 MB/s 2024-04-25T08:36:49.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.114 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584109 s, 180 MB/s 2024-04-25T08:36:49.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.122 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707966 s, 148 MB/s 2024-04-25T08:36:49.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707219 s, 148 MB/s 2024-04-25T08:36:49.137 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642332 s, 163 MB/s 2024-04-25T08:36:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.145 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632132 s, 166 MB/s 2024-04-25T08:36:49.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.151 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572672 s, 183 MB/s 2024-04-25T08:36:49.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640807 s, 164 MB/s 2024-04-25T08:36:49.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.166 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630789 s, 166 MB/s 2024-04-25T08:36:49.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.173 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614514 s, 171 MB/s 2024-04-25T08:36:49.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.181 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690219 s, 152 MB/s 2024-04-25T08:36:49.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:49.182 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:49.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627112 s, 167 MB/s 2024-04-25T08:36:49.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.255 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778073 s, 135 MB/s 2024-04-25T08:36:49.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.262 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060108 s, 174 MB/s 2024-04-25T08:36:49.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586326 s, 179 MB/s 2024-04-25T08:36:49.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.276 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655383 s, 160 MB/s 2024-04-25T08:36:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061725 s, 170 MB/s 2024-04-25T08:36:49.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.290 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609869 s, 172 MB/s 2024-04-25T08:36:49.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.297 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608198 s, 172 MB/s 2024-04-25T08:36:49.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.304 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608516 s, 172 MB/s 2024-04-25T08:36:49.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.311 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624958 s, 168 MB/s 2024-04-25T08:36:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632031 s, 166 MB/s 2024-04-25T08:36:49.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.326 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640352 s, 164 MB/s 2024-04-25T08:36:49.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.333 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613542 s, 171 MB/s 2024-04-25T08:36:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666662 s, 157 MB/s 2024-04-25T08:36:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604001 s, 174 MB/s 2024-04-25T08:36:49.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.354 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620858 s, 169 MB/s 2024-04-25T08:36:49.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578458 s, 181 MB/s 2024-04-25T08:36:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591679 s, 177 MB/s 2024-04-25T08:36:49.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659009 s, 159 MB/s 2024-04-25T08:36:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.383 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660766 s, 159 MB/s 2024-04-25T08:36:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.390 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651187 s, 161 MB/s 2024-04-25T08:36:49.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632597 s, 166 MB/s 2024-04-25T08:36:49.405 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.405 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.406 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728568 s, 144 MB/s 2024-04-25T08:36:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.413 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623585 s, 168 MB/s 2024-04-25T08:36:49.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.420 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609466 s, 172 MB/s 2024-04-25T08:36:49.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.428 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071495 s, 147 MB/s 2024-04-25T08:36:49.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.435 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636359 s, 165 MB/s 2024-04-25T08:36:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.443 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708426 s, 148 MB/s 2024-04-25T08:36:49.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.450 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613681 s, 171 MB/s 2024-04-25T08:36:49.457 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.457 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.457 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005994 s, 175 MB/s 2024-04-25T08:36:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.466 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624897 s, 168 MB/s 2024-04-25T08:36:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.472 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751389 s, 140 MB/s 2024-04-25T08:36:49.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:49.474 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701371 s, 150 MB/s 2024-04-25T08:36:49.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.548 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747405 s, 140 MB/s 2024-04-25T08:36:49.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.556 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733988 s, 143 MB/s 2024-04-25T08:36:49.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.564 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702488 s, 149 MB/s 2024-04-25T08:36:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.571 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600666 s, 175 MB/s 2024-04-25T08:36:49.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.579 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637658 s, 164 MB/s 2024-04-25T08:36:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.587 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719424 s, 146 MB/s 2024-04-25T08:36:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.593 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571474 s, 183 MB/s 2024-04-25T08:36:49.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570415 s, 184 MB/s 2024-04-25T08:36:49.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.607 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597992 s, 175 MB/s 2024-04-25T08:36:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619635 s, 169 MB/s 2024-04-25T08:36:49.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.622 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686452 s, 153 MB/s 2024-04-25T08:36:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665267 s, 158 MB/s 2024-04-25T08:36:49.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.637 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062877 s, 167 MB/s 2024-04-25T08:36:49.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697592 s, 150 MB/s 2024-04-25T08:36:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616899 s, 170 MB/s 2024-04-25T08:36:49.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.660 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738433 s, 142 MB/s 2024-04-25T08:36:49.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633102 s, 166 MB/s 2024-04-25T08:36:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614184 s, 171 MB/s 2024-04-25T08:36:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571058 s, 184 MB/s 2024-04-25T08:36:49.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.688 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646656 s, 162 MB/s 2024-04-25T08:36:49.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.695 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621193 s, 169 MB/s 2024-04-25T08:36:49.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.702 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609819 s, 172 MB/s 2024-04-25T08:36:49.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653701 s, 160 MB/s 2024-04-25T08:36:49.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634119 s, 165 MB/s 2024-04-25T08:36:49.724 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.724 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.724 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697972 s, 150 MB/s 2024-04-25T08:36:49.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.731 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609389 s, 172 MB/s 2024-04-25T08:36:49.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.739 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735777 s, 143 MB/s 2024-04-25T08:36:49.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606089 s, 173 MB/s 2024-04-25T08:36:49.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.754 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655544 s, 160 MB/s 2024-04-25T08:36:49.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610516 s, 172 MB/s 2024-04-25T08:36:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.769 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728551 s, 144 MB/s 2024-04-25T08:36:49.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:49.770 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:49.770 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:49.770 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:36:49.770 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:49.770 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:49.771 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:49.771 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.835 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617235 s, 170 MB/s 2024-04-25T08:36:49.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.842 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622176 s, 169 MB/s 2024-04-25T08:36:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.850 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071296 s, 147 MB/s 2024-04-25T08:36:49.859 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.859 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.859 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765329 s, 137 MB/s 2024-04-25T08:36:49.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.866 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667805 s, 157 MB/s 2024-04-25T08:36:49.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656798 s, 160 MB/s 2024-04-25T08:36:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633296 s, 166 MB/s 2024-04-25T08:36:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.888 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561462 s, 187 MB/s 2024-04-25T08:36:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.894 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567625 s, 185 MB/s 2024-04-25T08:36:49.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.901 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631463 s, 166 MB/s 2024-04-25T08:36:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552895 s, 190 MB/s 2024-04-25T08:36:49.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.915 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638874 s, 164 MB/s 2024-04-25T08:36:49.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.922 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592431 s, 177 MB/s 2024-04-25T08:36:49.931 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.931 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.931 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772706 s, 136 MB/s 2024-04-25T08:36:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730227 s, 144 MB/s 2024-04-25T08:36:49.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.947 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072672 s, 144 MB/s 2024-04-25T08:36:49.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.954 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615972 s, 170 MB/s 2024-04-25T08:36:49.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.963 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723625 s, 145 MB/s 2024-04-25T08:36:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.970 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607342 s, 173 MB/s 2024-04-25T08:36:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.977 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632959 s, 166 MB/s 2024-04-25T08:36:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.984 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624555 s, 168 MB/s 2024-04-25T08:36:49.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.991 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623151 s, 168 MB/s 2024-04-25T08:36:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:49.999 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712575 s, 147 MB/s 2024-04-25T08:36:50.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.006 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600411 s, 175 MB/s 2024-04-25T08:36:50.013 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.013 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.013 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609042 s, 172 MB/s 2024-04-25T08:36:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.020 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631892 s, 166 MB/s 2024-04-25T08:36:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.027 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601121 s, 174 MB/s 2024-04-25T08:36:50.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.035 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726169 s, 144 MB/s 2024-04-25T08:36:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.042 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564001 s, 186 MB/s 2024-04-25T08:36:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715729 s, 147 MB/s 2024-04-25T08:36:50.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.057 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607549 s, 173 MB/s 2024-04-25T08:36:50.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.064 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588661 s, 178 MB/s 2024-04-25T08:36:50.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:50.065 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:50.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666847 s, 157 MB/s 2024-04-25T08:36:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.137 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572446 s, 183 MB/s 2024-04-25T08:36:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.145 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741393 s, 141 MB/s 2024-04-25T08:36:50.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.153 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686917 s, 153 MB/s 2024-04-25T08:36:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744344 s, 141 MB/s 2024-04-25T08:36:50.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062208 s, 169 MB/s 2024-04-25T08:36:50.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641249 s, 164 MB/s 2024-04-25T08:36:50.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561649 s, 187 MB/s 2024-04-25T08:36:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706268 s, 148 MB/s 2024-04-25T08:36:50.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.197 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641797 s, 163 MB/s 2024-04-25T08:36:50.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.205 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063003 s, 166 MB/s 2024-04-25T08:36:50.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.213 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744188 s, 141 MB/s 2024-04-25T08:36:50.220 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.221 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695021 s, 151 MB/s 2024-04-25T08:36:50.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608733 s, 172 MB/s 2024-04-25T08:36:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752043 s, 139 MB/s 2024-04-25T08:36:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625113 s, 168 MB/s 2024-04-25T08:36:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.251 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693423 s, 151 MB/s 2024-04-25T08:36:50.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.258 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615193 s, 170 MB/s 2024-04-25T08:36:50.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.265 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631157 s, 166 MB/s 2024-04-25T08:36:50.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.272 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630671 s, 166 MB/s 2024-04-25T08:36:50.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.279 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591623 s, 177 MB/s 2024-04-25T08:36:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613894 s, 171 MB/s 2024-04-25T08:36:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.294 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633129 s, 166 MB/s 2024-04-25T08:36:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.301 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640814 s, 164 MB/s 2024-04-25T08:36:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.308 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643533 s, 163 MB/s 2024-04-25T08:36:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692996 s, 151 MB/s 2024-04-25T08:36:50.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.323 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588908 s, 178 MB/s 2024-04-25T08:36:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618607 s, 170 MB/s 2024-04-25T08:36:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600341 s, 175 MB/s 2024-04-25T08:36:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579052 s, 181 MB/s 2024-04-25T08:36:50.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627432 s, 167 MB/s 2024-04-25T08:36:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625826 s, 168 MB/s 2024-04-25T08:36:50.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:50.359 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:50.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.424 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728506 s, 144 MB/s 2024-04-25T08:36:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.435 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726116 s, 144 MB/s 2024-04-25T08:36:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613354 s, 171 MB/s 2024-04-25T08:36:50.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.447 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618596 s, 170 MB/s 2024-04-25T08:36:50.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643979 s, 163 MB/s 2024-04-25T08:36:50.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580223 s, 181 MB/s 2024-04-25T08:36:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651692 s, 161 MB/s 2024-04-25T08:36:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.476 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709018 s, 148 MB/s 2024-04-25T08:36:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644534 s, 163 MB/s 2024-04-25T08:36:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.490 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577576 s, 182 MB/s 2024-04-25T08:36:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.498 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699915 s, 150 MB/s 2024-04-25T08:36:50.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.506 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706263 s, 148 MB/s 2024-04-25T08:36:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.514 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705718 s, 149 MB/s 2024-04-25T08:36:50.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.522 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711848 s, 147 MB/s 2024-04-25T08:36:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.529 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061521 s, 170 MB/s 2024-04-25T08:36:50.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663364 s, 158 MB/s 2024-04-25T08:36:50.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.545 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076997 s, 136 MB/s 2024-04-25T08:36:50.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.552 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629007 s, 167 MB/s 2024-04-25T08:36:50.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.559 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620239 s, 169 MB/s 2024-04-25T08:36:50.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.566 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061565 s, 170 MB/s 2024-04-25T08:36:50.573 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.574 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637056 s, 165 MB/s 2024-04-25T08:36:50.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.581 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617647 s, 170 MB/s 2024-04-25T08:36:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.589 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070289 s, 149 MB/s 2024-04-25T08:36:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.595 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563818 s, 186 MB/s 2024-04-25T08:36:50.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.603 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662336 s, 158 MB/s 2024-04-25T08:36:50.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576497 s, 182 MB/s 2024-04-25T08:36:50.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.616 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586526 s, 179 MB/s 2024-04-25T08:36:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700426 s, 150 MB/s 2024-04-25T08:36:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.631 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585332 s, 179 MB/s 2024-04-25T08:36:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061588 s, 170 MB/s 2024-04-25T08:36:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613087 s, 171 MB/s 2024-04-25T08:36:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563121 s, 186 MB/s 2024-04-25T08:36:50.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:50.653 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703138 s, 149 MB/s 2024-04-25T08:36:50.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.726 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672587 s, 156 MB/s 2024-04-25T08:36:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597352 s, 176 MB/s 2024-04-25T08:36:50.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.741 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753028 s, 139 MB/s 2024-04-25T08:36:50.749 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.749 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.749 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710296 s, 148 MB/s 2024-04-25T08:36:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.757 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665434 s, 158 MB/s 2024-04-25T08:36:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.764 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611565 s, 171 MB/s 2024-04-25T08:36:50.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.772 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679889 s, 154 MB/s 2024-04-25T08:36:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.779 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619577 s, 169 MB/s 2024-04-25T08:36:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.787 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686712 s, 153 MB/s 2024-04-25T08:36:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661559 s, 159 MB/s 2024-04-25T08:36:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638039 s, 164 MB/s 2024-04-25T08:36:50.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659156 s, 159 MB/s 2024-04-25T08:36:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062253 s, 168 MB/s 2024-04-25T08:36:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638458 s, 164 MB/s 2024-04-25T08:36:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.832 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694462 s, 151 MB/s 2024-04-25T08:36:50.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.839 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633747 s, 165 MB/s 2024-04-25T08:36:50.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.847 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646684 s, 162 MB/s 2024-04-25T08:36:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685606 s, 153 MB/s 2024-04-25T08:36:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602046 s, 174 MB/s 2024-04-25T08:36:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630698 s, 166 MB/s 2024-04-25T08:36:50.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608467 s, 172 MB/s 2024-04-25T08:36:50.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620016 s, 169 MB/s 2024-04-25T08:36:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611651 s, 171 MB/s 2024-04-25T08:36:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620288 s, 169 MB/s 2024-04-25T08:36:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627343 s, 167 MB/s 2024-04-25T08:36:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740486 s, 142 MB/s 2024-04-25T08:36:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071741 s, 146 MB/s 2024-04-25T08:36:50.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708014 s, 148 MB/s 2024-04-25T08:36:50.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617602 s, 170 MB/s 2024-04-25T08:36:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561862 s, 187 MB/s 2024-04-25T08:36:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:50.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580108 s, 181 MB/s 2024-04-25T08:36:50.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:50.950 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:51.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.015 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611909 s, 171 MB/s 2024-04-25T08:36:51.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714672 s, 147 MB/s 2024-04-25T08:36:51.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654341 s, 160 MB/s 2024-04-25T08:36:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607474 s, 173 MB/s 2024-04-25T08:36:51.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592115 s, 177 MB/s 2024-04-25T08:36:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.051 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639652 s, 164 MB/s 2024-04-25T08:36:51.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695921 s, 151 MB/s 2024-04-25T08:36:51.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.066 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639131 s, 164 MB/s 2024-04-25T08:36:51.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.073 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604128 s, 174 MB/s 2024-04-25T08:36:51.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.080 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563993 s, 186 MB/s 2024-04-25T08:36:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.087 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635528 s, 165 MB/s 2024-04-25T08:36:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.094 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628497 s, 167 MB/s 2024-04-25T08:36:51.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.101 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621563 s, 169 MB/s 2024-04-25T08:36:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613146 s, 171 MB/s 2024-04-25T08:36:51.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620747 s, 169 MB/s 2024-04-25T08:36:51.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.122 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622323 s, 168 MB/s 2024-04-25T08:36:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635423 s, 165 MB/s 2024-04-25T08:36:51.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.136 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569516 s, 184 MB/s 2024-04-25T08:36:51.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.143 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601223 s, 174 MB/s 2024-04-25T08:36:51.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.150 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634226 s, 165 MB/s 2024-04-25T08:36:51.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.165 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138971 s, 75.5 MB/s 2024-04-25T08:36:51.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.172 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584846 s, 179 MB/s 2024-04-25T08:36:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.179 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061445 s, 171 MB/s 2024-04-25T08:36:51.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624786 s, 168 MB/s 2024-04-25T08:36:51.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.193 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615649 s, 170 MB/s 2024-04-25T08:36:51.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.200 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651018 s, 161 MB/s 2024-04-25T08:36:51.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585856 s, 179 MB/s 2024-04-25T08:36:51.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631878 s, 166 MB/s 2024-04-25T08:36:51.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637699 s, 164 MB/s 2024-04-25T08:36:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583375 s, 180 MB/s 2024-04-25T08:36:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563644 s, 186 MB/s 2024-04-25T08:36:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.242 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621582 s, 169 MB/s 2024-04-25T08:36:51.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:51.243 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:51.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772118 s, 136 MB/s 2024-04-25T08:36:51.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635014 s, 165 MB/s 2024-04-25T08:36:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.324 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723807 s, 145 MB/s 2024-04-25T08:36:51.331 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.332 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.332 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626033 s, 167 MB/s 2024-04-25T08:36:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.339 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006977 s, 150 MB/s 2024-04-25T08:36:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650287 s, 161 MB/s 2024-04-25T08:36:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.353 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575085 s, 182 MB/s 2024-04-25T08:36:51.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618119 s, 170 MB/s 2024-04-25T08:36:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.367 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599491 s, 175 MB/s 2024-04-25T08:36:51.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618448 s, 170 MB/s 2024-04-25T08:36:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604674 s, 173 MB/s 2024-04-25T08:36:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613004 s, 171 MB/s 2024-04-25T08:36:51.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652691 s, 161 MB/s 2024-04-25T08:36:51.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549028 s, 191 MB/s 2024-04-25T08:36:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653979 s, 160 MB/s 2024-04-25T08:36:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.417 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639866 s, 164 MB/s 2024-04-25T08:36:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.425 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718313 s, 146 MB/s 2024-04-25T08:36:51.432 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.432 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.432 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637689 s, 164 MB/s 2024-04-25T08:36:51.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.440 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739705 s, 142 MB/s 2024-04-25T08:36:51.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.447 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573537 s, 183 MB/s 2024-04-25T08:36:51.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645555 s, 162 MB/s 2024-04-25T08:36:51.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619042 s, 169 MB/s 2024-04-25T08:36:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616456 s, 170 MB/s 2024-04-25T08:36:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.475 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584866 s, 179 MB/s 2024-04-25T08:36:51.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631574 s, 166 MB/s 2024-04-25T08:36:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574084 s, 183 MB/s 2024-04-25T08:36:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.496 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600644 s, 175 MB/s 2024-04-25T08:36:51.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604392 s, 173 MB/s 2024-04-25T08:36:51.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.514 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761266 s, 138 MB/s 2024-04-25T08:36:51.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.518 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614158 s, 171 MB/s 2024-04-25T08:36:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062994 s, 166 MB/s 2024-04-25T08:36:51.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.534 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075537 s, 139 MB/s 2024-04-25T08:36:51.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:51.535 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:51.535 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:51.535 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:36:51.535 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:51.535 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:51.536 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:51.536 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.601 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734603 s, 143 MB/s 2024-04-25T08:36:51.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.608 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578112 s, 181 MB/s 2024-04-25T08:36:51.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.615 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636141 s, 165 MB/s 2024-04-25T08:36:51.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.622 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663701 s, 158 MB/s 2024-04-25T08:36:51.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.631 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740458 s, 142 MB/s 2024-04-25T08:36:51.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623742 s, 168 MB/s 2024-04-25T08:36:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636092 s, 165 MB/s 2024-04-25T08:36:51.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061976 s, 169 MB/s 2024-04-25T08:36:51.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.661 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.661 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744947 s, 141 MB/s 2024-04-25T08:36:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.668 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618936 s, 169 MB/s 2024-04-25T08:36:51.676 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.676 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.676 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741351 s, 141 MB/s 2024-04-25T08:36:51.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.684 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719881 s, 146 MB/s 2024-04-25T08:36:51.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.691 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610407 s, 172 MB/s 2024-04-25T08:36:51.698 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.698 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.698 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064073 s, 164 MB/s 2024-04-25T08:36:51.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.705 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568555 s, 184 MB/s 2024-04-25T08:36:51.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.712 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613658 s, 171 MB/s 2024-04-25T08:36:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.719 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588166 s, 178 MB/s 2024-04-25T08:36:51.725 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.725 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.726 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579079 s, 181 MB/s 2024-04-25T08:36:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669454 s, 157 MB/s 2024-04-25T08:36:51.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.740 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619381 s, 169 MB/s 2024-04-25T08:36:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.747 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571272 s, 184 MB/s 2024-04-25T08:36:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.754 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064933 s, 161 MB/s 2024-04-25T08:36:51.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.762 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694415 s, 151 MB/s 2024-04-25T08:36:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.771 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763172 s, 137 MB/s 2024-04-25T08:36:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.778 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063004 s, 166 MB/s 2024-04-25T08:36:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730084 s, 144 MB/s 2024-04-25T08:36:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650723 s, 161 MB/s 2024-04-25T08:36:51.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.801 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636639 s, 165 MB/s 2024-04-25T08:36:51.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654222 s, 160 MB/s 2024-04-25T08:36:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633159 s, 166 MB/s 2024-04-25T08:36:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640678 s, 164 MB/s 2024-04-25T08:36:51.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061483 s, 171 MB/s 2024-04-25T08:36:51.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:51.831 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:51.831 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:51.831 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:36:51.831 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:51.831 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:51.832 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:51.832 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:51.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736872 s, 142 MB/s 2024-04-25T08:36:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624302 s, 168 MB/s 2024-04-25T08:36:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679795 s, 154 MB/s 2024-04-25T08:36:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720152 s, 146 MB/s 2024-04-25T08:36:51.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642435 s, 163 MB/s 2024-04-25T08:36:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627816 s, 167 MB/s 2024-04-25T08:36:51.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586143 s, 179 MB/s 2024-04-25T08:36:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603278 s, 174 MB/s 2024-04-25T08:36:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652758 s, 161 MB/s 2024-04-25T08:36:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619456 s, 169 MB/s 2024-04-25T08:36:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.971 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736653 s, 142 MB/s 2024-04-25T08:36:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.978 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624035 s, 168 MB/s 2024-04-25T08:36:51.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643792 s, 163 MB/s 2024-04-25T08:36:51.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:51.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:51.993 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647164 s, 162 MB/s 2024-04-25T08:36:52.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563595 s, 186 MB/s 2024-04-25T08:36:52.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.007 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062356 s, 168 MB/s 2024-04-25T08:36:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.014 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634317 s, 165 MB/s 2024-04-25T08:36:52.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.021 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586363 s, 179 MB/s 2024-04-25T08:36:52.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.027 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577472 s, 182 MB/s 2024-04-25T08:36:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.035 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638818 s, 164 MB/s 2024-04-25T08:36:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742042 s, 141 MB/s 2024-04-25T08:36:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.049 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570874 s, 184 MB/s 2024-04-25T08:36:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.057 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061768 s, 170 MB/s 2024-04-25T08:36:52.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.064 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695388 s, 151 MB/s 2024-04-25T08:36:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701979 s, 149 MB/s 2024-04-25T08:36:52.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626969 s, 167 MB/s 2024-04-25T08:36:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.087 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062453 s, 168 MB/s 2024-04-25T08:36:52.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.094 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675544 s, 155 MB/s 2024-04-25T08:36:52.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.101 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055652 s, 188 MB/s 2024-04-25T08:36:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.107 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571211 s, 184 MB/s 2024-04-25T08:36:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.114 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620848 s, 169 MB/s 2024-04-25T08:36:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592422 s, 177 MB/s 2024-04-25T08:36:52.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:52.122 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:52.123 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:52.123 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:36:52.123 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:52.123 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:52.123 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:52.123 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.188 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635722 s, 165 MB/s 2024-04-25T08:36:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606137 s, 173 MB/s 2024-04-25T08:36:52.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.203 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716808 s, 146 MB/s 2024-04-25T08:36:52.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619533 s, 169 MB/s 2024-04-25T08:36:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622927 s, 168 MB/s 2024-04-25T08:36:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611917 s, 171 MB/s 2024-04-25T08:36:52.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.232 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720652 s, 146 MB/s 2024-04-25T08:36:52.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.239 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649523 s, 161 MB/s 2024-04-25T08:36:52.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.246 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580948 s, 180 MB/s 2024-04-25T08:36:52.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576897 s, 182 MB/s 2024-04-25T08:36:52.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.260 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628249 s, 167 MB/s 2024-04-25T08:36:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741159 s, 141 MB/s 2024-04-25T08:36:52.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.275 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607376 s, 173 MB/s 2024-04-25T08:36:52.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.283 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637409 s, 165 MB/s 2024-04-25T08:36:52.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.290 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706798 s, 148 MB/s 2024-04-25T08:36:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708695 s, 148 MB/s 2024-04-25T08:36:52.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614701 s, 171 MB/s 2024-04-25T08:36:52.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.313 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639022 s, 164 MB/s 2024-04-25T08:36:52.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074563 s, 141 MB/s 2024-04-25T08:36:52.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.328 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057053 s, 184 MB/s 2024-04-25T08:36:52.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.335 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647007 s, 162 MB/s 2024-04-25T08:36:52.342 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.342 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.342 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622493 s, 168 MB/s 2024-04-25T08:36:52.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.353 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906277 s, 116 MB/s 2024-04-25T08:36:52.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.359 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580765 s, 181 MB/s 2024-04-25T08:36:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666832 s, 157 MB/s 2024-04-25T08:36:52.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726795 s, 144 MB/s 2024-04-25T08:36:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594353 s, 176 MB/s 2024-04-25T08:36:52.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.390 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730063 s, 144 MB/s 2024-04-25T08:36:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645652 s, 162 MB/s 2024-04-25T08:36:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.405 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709795 s, 148 MB/s 2024-04-25T08:36:52.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.412 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620072 s, 169 MB/s 2024-04-25T08:36:52.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.420 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741866 s, 141 MB/s 2024-04-25T08:36:52.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:52.422 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.486 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565175 s, 186 MB/s 2024-04-25T08:36:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.493 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645371 s, 162 MB/s 2024-04-25T08:36:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.501 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716222 s, 146 MB/s 2024-04-25T08:36:52.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.509 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699303 s, 150 MB/s 2024-04-25T08:36:52.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647424 s, 162 MB/s 2024-04-25T08:36:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.525 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704716 s, 149 MB/s 2024-04-25T08:36:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.532 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632398 s, 166 MB/s 2024-04-25T08:36:52.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.539 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062562 s, 168 MB/s 2024-04-25T08:36:52.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720786 s, 145 MB/s 2024-04-25T08:36:52.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.553 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568788 s, 184 MB/s 2024-04-25T08:36:52.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.561 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698041 s, 150 MB/s 2024-04-25T08:36:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727565 s, 144 MB/s 2024-04-25T08:36:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638962 s, 164 MB/s 2024-04-25T08:36:52.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596323 s, 176 MB/s 2024-04-25T08:36:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058856 s, 178 MB/s 2024-04-25T08:36:52.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.598 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661754 s, 158 MB/s 2024-04-25T08:36:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.606 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756291 s, 139 MB/s 2024-04-25T08:36:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625201 s, 168 MB/s 2024-04-25T08:36:52.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.622 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711457 s, 147 MB/s 2024-04-25T08:36:52.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570809 s, 184 MB/s 2024-04-25T08:36:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.636 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630874 s, 166 MB/s 2024-04-25T08:36:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.642 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058381 s, 180 MB/s 2024-04-25T08:36:52.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713897 s, 147 MB/s 2024-04-25T08:36:52.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.658 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702722 s, 149 MB/s 2024-04-25T08:36:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709405 s, 148 MB/s 2024-04-25T08:36:52.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645136 s, 163 MB/s 2024-04-25T08:36:52.681 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.681 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.682 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070182 s, 149 MB/s 2024-04-25T08:36:52.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.689 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700154 s, 150 MB/s 2024-04-25T08:36:52.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.697 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716896 s, 146 MB/s 2024-04-25T08:36:52.704 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.705 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631666 s, 166 MB/s 2024-04-25T08:36:52.711 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.712 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622183 s, 169 MB/s 2024-04-25T08:36:52.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.720 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694637 s, 151 MB/s 2024-04-25T08:36:52.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:52.721 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:52.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066648 s, 157 MB/s 2024-04-25T08:36:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578258 s, 181 MB/s 2024-04-25T08:36:52.800 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.800 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.800 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630848 s, 166 MB/s 2024-04-25T08:36:52.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654581 s, 160 MB/s 2024-04-25T08:36:52.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.818 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886765 s, 118 MB/s 2024-04-25T08:36:52.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.825 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692422 s, 151 MB/s 2024-04-25T08:36:52.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065045 s, 161 MB/s 2024-04-25T08:36:52.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.840 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639496 s, 164 MB/s 2024-04-25T08:36:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715227 s, 147 MB/s 2024-04-25T08:36:52.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.855 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619475 s, 169 MB/s 2024-04-25T08:36:52.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.862 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632177 s, 166 MB/s 2024-04-25T08:36:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693289 s, 151 MB/s 2024-04-25T08:36:52.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.878 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712113 s, 147 MB/s 2024-04-25T08:36:52.888 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.888 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.888 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916753 s, 114 MB/s 2024-04-25T08:36:52.895 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.896 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654403 s, 160 MB/s 2024-04-25T08:36:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.903 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615447 s, 170 MB/s 2024-04-25T08:36:52.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.910 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650446 s, 161 MB/s 2024-04-25T08:36:52.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.917 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579989 s, 181 MB/s 2024-04-25T08:36:52.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.925 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070188 s, 149 MB/s 2024-04-25T08:36:52.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.932 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643556 s, 163 MB/s 2024-04-25T08:36:52.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565018 s, 186 MB/s 2024-04-25T08:36:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115413 s, 90.9 MB/s 2024-04-25T08:36:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.959 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724969 s, 145 MB/s 2024-04-25T08:36:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.967 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719893 s, 146 MB/s 2024-04-25T08:36:52.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.975 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623837 s, 168 MB/s 2024-04-25T08:36:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699584 s, 150 MB/s 2024-04-25T08:36:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604106 s, 174 MB/s 2024-04-25T08:36:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:52.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055927 s, 187 MB/s 2024-04-25T08:36:53.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.002 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575154 s, 182 MB/s 2024-04-25T08:36:53.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.010 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612544 s, 171 MB/s 2024-04-25T08:36:53.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614193 s, 171 MB/s 2024-04-25T08:36:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.024 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574629 s, 182 MB/s 2024-04-25T08:36:53.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:53.026 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:53.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.090 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615395 s, 170 MB/s 2024-04-25T08:36:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.097 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063722 s, 165 MB/s 2024-04-25T08:36:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659986 s, 159 MB/s 2024-04-25T08:36:53.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585222 s, 179 MB/s 2024-04-25T08:36:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.120 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778673 s, 135 MB/s 2024-04-25T08:36:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663304 s, 158 MB/s 2024-04-25T08:36:53.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.135 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632497 s, 166 MB/s 2024-04-25T08:36:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.145 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889561 s, 118 MB/s 2024-04-25T08:36:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.154 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753333 s, 139 MB/s 2024-04-25T08:36:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.160 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587847 s, 178 MB/s 2024-04-25T08:36:53.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.167 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632675 s, 166 MB/s 2024-04-25T08:36:53.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673023 s, 156 MB/s 2024-04-25T08:36:53.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613395 s, 171 MB/s 2024-04-25T08:36:53.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073411 s, 143 MB/s 2024-04-25T08:36:53.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652731 s, 161 MB/s 2024-04-25T08:36:53.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.209 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091362 s, 115 MB/s 2024-04-25T08:36:53.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.215 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609856 s, 172 MB/s 2024-04-25T08:36:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601822 s, 174 MB/s 2024-04-25T08:36:53.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.230 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707539 s, 148 MB/s 2024-04-25T08:36:53.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.238 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722803 s, 145 MB/s 2024-04-25T08:36:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.244 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584704 s, 179 MB/s 2024-04-25T08:36:53.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.254 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844774 s, 124 MB/s 2024-04-25T08:36:53.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.261 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641381 s, 163 MB/s 2024-04-25T08:36:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635095 s, 165 MB/s 2024-04-25T08:36:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.276 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642658 s, 163 MB/s 2024-04-25T08:36:53.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705666 s, 149 MB/s 2024-04-25T08:36:53.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.291 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612041 s, 171 MB/s 2024-04-25T08:36:53.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669028 s, 157 MB/s 2024-04-25T08:36:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.306 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635381 s, 165 MB/s 2024-04-25T08:36:53.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715986 s, 146 MB/s 2024-04-25T08:36:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638446 s, 164 MB/s 2024-04-25T08:36:53.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732 s, 143 MB/s 2024-04-25T08:36:53.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:53.330 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:53.330 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:53.330 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:36:53.330 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:53.330 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:53.330 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:53.331 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717581 s, 146 MB/s 2024-04-25T08:36:53.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575123 s, 182 MB/s 2024-04-25T08:36:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638874 s, 164 MB/s 2024-04-25T08:36:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.416 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060968 s, 172 MB/s 2024-04-25T08:36:53.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601877 s, 174 MB/s 2024-04-25T08:36:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.430 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617726 s, 170 MB/s 2024-04-25T08:36:53.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644187 s, 163 MB/s 2024-04-25T08:36:53.445 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.446 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741219 s, 141 MB/s 2024-04-25T08:36:53.453 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.453 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.453 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625609 s, 168 MB/s 2024-04-25T08:36:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.460 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582771 s, 180 MB/s 2024-04-25T08:36:53.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.466 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606004 s, 173 MB/s 2024-04-25T08:36:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.474 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677327 s, 155 MB/s 2024-04-25T08:36:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602405 s, 174 MB/s 2024-04-25T08:36:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.488 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631285 s, 166 MB/s 2024-04-25T08:36:53.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.496 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715753 s, 146 MB/s 2024-04-25T08:36:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682728 s, 154 MB/s 2024-04-25T08:36:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613493 s, 171 MB/s 2024-04-25T08:36:53.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653063 s, 161 MB/s 2024-04-25T08:36:53.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065727 s, 160 MB/s 2024-04-25T08:36:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608359 s, 172 MB/s 2024-04-25T08:36:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889949 s, 118 MB/s 2024-04-25T08:36:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742703 s, 141 MB/s 2024-04-25T08:36:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644657 s, 163 MB/s 2024-04-25T08:36:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.565 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575418 s, 182 MB/s 2024-04-25T08:36:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.572 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634591 s, 165 MB/s 2024-04-25T08:36:53.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892918 s, 117 MB/s 2024-04-25T08:36:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680636 s, 154 MB/s 2024-04-25T08:36:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685563 s, 153 MB/s 2024-04-25T08:36:53.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.604 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575609 s, 182 MB/s 2024-04-25T08:36:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.611 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615212 s, 170 MB/s 2024-04-25T08:36:53.618 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658785 s, 159 MB/s 2024-04-25T08:36:53.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727783 s, 144 MB/s 2024-04-25T08:36:53.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:53.628 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:53.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.695 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088809 s, 118 MB/s 2024-04-25T08:36:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.702 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646601 s, 162 MB/s 2024-04-25T08:36:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628191 s, 167 MB/s 2024-04-25T08:36:53.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584796 s, 179 MB/s 2024-04-25T08:36:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637916 s, 164 MB/s 2024-04-25T08:36:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.731 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684177 s, 153 MB/s 2024-04-25T08:36:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645182 s, 163 MB/s 2024-04-25T08:36:53.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.747 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765111 s, 137 MB/s 2024-04-25T08:36:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.754 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630402 s, 166 MB/s 2024-04-25T08:36:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.764 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910773 s, 115 MB/s 2024-04-25T08:36:53.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.773 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727474 s, 144 MB/s 2024-04-25T08:36:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.779 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560547 s, 187 MB/s 2024-04-25T08:36:53.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.787 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692872 s, 151 MB/s 2024-04-25T08:36:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627558 s, 167 MB/s 2024-04-25T08:36:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624656 s, 168 MB/s 2024-04-25T08:36:53.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594494 s, 176 MB/s 2024-04-25T08:36:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695882 s, 151 MB/s 2024-04-25T08:36:53.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.826 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886018 s, 118 MB/s 2024-04-25T08:36:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650984 s, 161 MB/s 2024-04-25T08:36:53.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.840 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625823 s, 168 MB/s 2024-04-25T08:36:53.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663831 s, 158 MB/s 2024-04-25T08:36:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.856 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712434 s, 147 MB/s 2024-04-25T08:36:53.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.863 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635793 s, 165 MB/s 2024-04-25T08:36:53.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.873 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872725 s, 120 MB/s 2024-04-25T08:36:53.880 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.880 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.880 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628972 s, 167 MB/s 2024-04-25T08:36:53.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.887 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061608 s, 170 MB/s 2024-04-25T08:36:53.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.894 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572335 s, 183 MB/s 2024-04-25T08:36:53.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.901 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644007 s, 163 MB/s 2024-04-25T08:36:53.909 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.909 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.909 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696745 s, 150 MB/s 2024-04-25T08:36:53.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.917 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733587 s, 143 MB/s 2024-04-25T08:36:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.925 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704644 s, 149 MB/s 2024-04-25T08:36:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:53.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925692 s, 113 MB/s 2024-04-25T08:36:53.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:53.936 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:53.936 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:53.936 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:36:53.936 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:53.936 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:53.936 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:53.937 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713602 s, 147 MB/s 2024-04-25T08:36:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647663 s, 162 MB/s 2024-04-25T08:36:54.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647454 s, 162 MB/s 2024-04-25T08:36:54.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.025 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751715 s, 139 MB/s 2024-04-25T08:36:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717775 s, 146 MB/s 2024-04-25T08:36:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721707 s, 145 MB/s 2024-04-25T08:36:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562171 s, 187 MB/s 2024-04-25T08:36:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.054 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581577 s, 180 MB/s 2024-04-25T08:36:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.061 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644428 s, 163 MB/s 2024-04-25T08:36:54.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.071 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870745 s, 120 MB/s 2024-04-25T08:36:54.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631878 s, 166 MB/s 2024-04-25T08:36:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631435 s, 166 MB/s 2024-04-25T08:36:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.093 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692653 s, 151 MB/s 2024-04-25T08:36:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.100 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619381 s, 169 MB/s 2024-04-25T08:36:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646563 s, 162 MB/s 2024-04-25T08:36:54.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613797 s, 171 MB/s 2024-04-25T08:36:54.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.125 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911538 s, 115 MB/s 2024-04-25T08:36:54.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071485 s, 147 MB/s 2024-04-25T08:36:54.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.141 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720475 s, 146 MB/s 2024-04-25T08:36:54.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635534 s, 165 MB/s 2024-04-25T08:36:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.155 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626648 s, 167 MB/s 2024-04-25T08:36:54.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.163 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065562 s, 160 MB/s 2024-04-25T08:36:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722344 s, 145 MB/s 2024-04-25T08:36:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751543 s, 140 MB/s 2024-04-25T08:36:54.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583896 s, 180 MB/s 2024-04-25T08:36:54.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816149 s, 128 MB/s 2024-04-25T08:36:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646243 s, 162 MB/s 2024-04-25T08:36:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.209 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609441 s, 172 MB/s 2024-04-25T08:36:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070236 s, 149 MB/s 2024-04-25T08:36:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562336 s, 186 MB/s 2024-04-25T08:36:54.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.232 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725766 s, 144 MB/s 2024-04-25T08:36:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.239 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639178 s, 164 MB/s 2024-04-25T08:36:54.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:54.240 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:54.241 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:54.241 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:36:54.241 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:54.241 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:54.241 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:54.241 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626515 s, 167 MB/s 2024-04-25T08:36:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776463 s, 135 MB/s 2024-04-25T08:36:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.320 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569298 s, 184 MB/s 2024-04-25T08:36:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609649 s, 172 MB/s 2024-04-25T08:36:54.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.335 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640144 s, 164 MB/s 2024-04-25T08:36:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772664 s, 136 MB/s 2024-04-25T08:36:54.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072726 s, 144 MB/s 2024-04-25T08:36:54.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.360 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754938 s, 139 MB/s 2024-04-25T08:36:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.367 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587828 s, 178 MB/s 2024-04-25T08:36:54.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.377 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888575 s, 118 MB/s 2024-04-25T08:36:54.385 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.385 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.385 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722212 s, 145 MB/s 2024-04-25T08:36:54.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.391 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587046 s, 179 MB/s 2024-04-25T08:36:54.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.399 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708629 s, 148 MB/s 2024-04-25T08:36:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.407 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645708 s, 162 MB/s 2024-04-25T08:36:54.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.414 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.414 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061872 s, 169 MB/s 2024-04-25T08:36:54.421 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.421 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.421 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642842 s, 163 MB/s 2024-04-25T08:36:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591778 s, 177 MB/s 2024-04-25T08:36:54.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.436 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689315 s, 152 MB/s 2024-04-25T08:36:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698928 s, 150 MB/s 2024-04-25T08:36:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.452 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698789 s, 150 MB/s 2024-04-25T08:36:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066666 s, 157 MB/s 2024-04-25T08:36:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755317 s, 139 MB/s 2024-04-25T08:36:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.474 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571139 s, 184 MB/s 2024-04-25T08:36:54.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662623 s, 158 MB/s 2024-04-25T08:36:54.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.488 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596652 s, 176 MB/s 2024-04-25T08:36:54.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713349 s, 147 MB/s 2024-04-25T08:36:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.505 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741821 s, 141 MB/s 2024-04-25T08:36:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058407 s, 180 MB/s 2024-04-25T08:36:54.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644491 s, 163 MB/s 2024-04-25T08:36:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.527 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727199 s, 144 MB/s 2024-04-25T08:36:54.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652318 s, 161 MB/s 2024-04-25T08:36:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.542 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659442 s, 159 MB/s 2024-04-25T08:36:54.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:54.544 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:54.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006347 s, 165 MB/s 2024-04-25T08:36:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.616 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064926 s, 162 MB/s 2024-04-25T08:36:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.622 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.623 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585365 s, 179 MB/s 2024-04-25T08:36:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.630 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703026 s, 149 MB/s 2024-04-25T08:36:54.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633831 s, 165 MB/s 2024-04-25T08:36:54.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635808 s, 165 MB/s 2024-04-25T08:36:54.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060996 s, 172 MB/s 2024-04-25T08:36:54.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590602 s, 178 MB/s 2024-04-25T08:36:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621572 s, 169 MB/s 2024-04-25T08:36:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592279 s, 177 MB/s 2024-04-25T08:36:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638699 s, 164 MB/s 2024-04-25T08:36:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.688 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716978 s, 146 MB/s 2024-04-25T08:36:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.695 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642275 s, 163 MB/s 2024-04-25T08:36:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.703 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707091 s, 148 MB/s 2024-04-25T08:36:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.711 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642955 s, 163 MB/s 2024-04-25T08:36:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.718 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638336 s, 164 MB/s 2024-04-25T08:36:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.725 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617627 s, 170 MB/s 2024-04-25T08:36:54.732 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669701 s, 157 MB/s 2024-04-25T08:36:54.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.741 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742849 s, 141 MB/s 2024-04-25T08:36:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621734 s, 169 MB/s 2024-04-25T08:36:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.756 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007429 s, 141 MB/s 2024-04-25T08:36:54.763 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.764 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657233 s, 160 MB/s 2024-04-25T08:36:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.771 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705695 s, 149 MB/s 2024-04-25T08:36:54.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.779 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645892 s, 162 MB/s 2024-04-25T08:36:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620951 s, 169 MB/s 2024-04-25T08:36:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725259 s, 145 MB/s 2024-04-25T08:36:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645763 s, 162 MB/s 2024-04-25T08:36:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.809 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618628 s, 170 MB/s 2024-04-25T08:36:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581471 s, 180 MB/s 2024-04-25T08:36:54.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061976 s, 169 MB/s 2024-04-25T08:36:54.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617202 s, 170 MB/s 2024-04-25T08:36:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662943 s, 158 MB/s 2024-04-25T08:36:54.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:54.839 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731404 s, 143 MB/s 2024-04-25T08:36:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581017 s, 180 MB/s 2024-04-25T08:36:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645092 s, 163 MB/s 2024-04-25T08:36:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753535 s, 139 MB/s 2024-04-25T08:36:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699778 s, 150 MB/s 2024-04-25T08:36:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.943 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642603 s, 163 MB/s 2024-04-25T08:36:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.950 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630877 s, 166 MB/s 2024-04-25T08:36:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623826 s, 168 MB/s 2024-04-25T08:36:54.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.964 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646263 s, 162 MB/s 2024-04-25T08:36:54.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.972 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643994 s, 163 MB/s 2024-04-25T08:36:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.979 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701935 s, 149 MB/s 2024-04-25T08:36:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624774 s, 168 MB/s 2024-04-25T08:36:54.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:54.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:54.993 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617469 s, 170 MB/s 2024-04-25T08:36:55.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670091 s, 156 MB/s 2024-04-25T08:36:55.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.008 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646814 s, 162 MB/s 2024-04-25T08:36:55.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616909 s, 170 MB/s 2024-04-25T08:36:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.024 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760593 s, 138 MB/s 2024-04-25T08:36:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.032 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714255 s, 147 MB/s 2024-04-25T08:36:55.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.039 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065279 s, 161 MB/s 2024-04-25T08:36:55.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.046 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624877 s, 168 MB/s 2024-04-25T08:36:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.054 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647089 s, 162 MB/s 2024-04-25T08:36:55.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746041 s, 141 MB/s 2024-04-25T08:36:55.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.069 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583808 s, 180 MB/s 2024-04-25T08:36:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.076 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615528 s, 170 MB/s 2024-04-25T08:36:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643235 s, 163 MB/s 2024-04-25T08:36:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638942 s, 164 MB/s 2024-04-25T08:36:55.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.098 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627377 s, 167 MB/s 2024-04-25T08:36:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623288 s, 168 MB/s 2024-04-25T08:36:55.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601465 s, 174 MB/s 2024-04-25T08:36:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.119 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644352 s, 163 MB/s 2024-04-25T08:36:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.126 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610666 s, 172 MB/s 2024-04-25T08:36:55.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.134 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709789 s, 148 MB/s 2024-04-25T08:36:55.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:55.135 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:36:55.135 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:36:55.135 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:36:55.135 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:36:55.135 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:36:55.136 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:36:55.136 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:36:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562627 s, 186 MB/s 2024-04-25T08:36:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708967 s, 148 MB/s 2024-04-25T08:36:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595642 s, 176 MB/s 2024-04-25T08:36:55.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702728 s, 149 MB/s 2024-04-25T08:36:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.229 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626429 s, 167 MB/s 2024-04-25T08:36:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612512 s, 171 MB/s 2024-04-25T08:36:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570011 s, 184 MB/s 2024-04-25T08:36:55.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.250 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641295 s, 164 MB/s 2024-04-25T08:36:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.257 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063629 s, 165 MB/s 2024-04-25T08:36:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.264 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601124 s, 174 MB/s 2024-04-25T08:36:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.271 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617669 s, 170 MB/s 2024-04-25T08:36:55.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.279 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658975 s, 159 MB/s 2024-04-25T08:36:55.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633342 s, 166 MB/s 2024-04-25T08:36:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.294 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733993 s, 143 MB/s 2024-04-25T08:36:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707057 s, 148 MB/s 2024-04-25T08:36:55.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671575 s, 156 MB/s 2024-04-25T08:36:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570685 s, 184 MB/s 2024-04-25T08:36:55.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.323 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606549 s, 173 MB/s 2024-04-25T08:36:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616135 s, 170 MB/s 2024-04-25T08:36:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603342 s, 174 MB/s 2024-04-25T08:36:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.345 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062562 s, 168 MB/s 2024-04-25T08:36:55.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.353 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738401 s, 142 MB/s 2024-04-25T08:36:55.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739341 s, 142 MB/s 2024-04-25T08:36:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.368 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588877 s, 178 MB/s 2024-04-25T08:36:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624563 s, 168 MB/s 2024-04-25T08:36:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578083 s, 181 MB/s 2024-04-25T08:36:55.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648285 s, 162 MB/s 2024-04-25T08:36:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609877 s, 172 MB/s 2024-04-25T08:36:55.405 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.406 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597949 s, 175 MB/s 2024-04-25T08:36:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063451 s, 165 MB/s 2024-04-25T08:36:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719003 s, 146 MB/s 2024-04-25T08:36:55.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:36:55.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:36:55.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619944 s, 169 MB/s 2024-04-25T08:36:55.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:55.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:36:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 Processor -- start 2024-04-25T08:36:55.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 -- start start 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c340cc4f0 0x7f6c340cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c340cdb20 0x7f6c340cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6c3415b270 0x7f6c3415b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340ce4c0 con 0x7f6c340cdb20 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c34071270 con 0x7f6c3415b270 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.580+0000 7f6c3a61f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34072280 con 0x7f6c340cc4f0 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c340cdb20 0x7f6c340cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6c340cdb20 0x7f6c340cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41694/0 (socket says 172.21.15.60:41694) 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 -- 172.21.15.60:0/3475127865 learned_addr learned my addr 172.21.15.60:0/3475127865 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3475127865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c340cc4f0 0x7f6c340cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3475127865 >> v2:172.21.15.100:3300/0 conn(0x7f6c3415b270 0x7f6c3415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 -- 172.21.15.60:0/3475127865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c340cc4f0 msgr2=0x7f6c340cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3475127865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c340cc4f0 0x7f6c340cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 -- 172.21.15.60:0/3475127865 >> v2:172.21.15.100:3300/0 conn(0x7f6c3415b270 msgr2=0x7f6c3415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3475127865 >> v2:172.21.15.100:3300/0 conn(0x7f6c3415b270 0x7f6c3415b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c337fe640 1 -- 172.21.15.60:0/3475127865 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340714c0 con 0x7f6c340cdb20 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.581+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3475127865 >> v2:172.21.15.100:3300/0 conn(0x7f6c3415b270 0x7f6c3415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3475127865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cdb20 0x7f6c340cdf80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f6c34158aa0 tx=0x7f6c2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbe81bdc17888b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c32ffd640 1 -- 172.21.15.60:0/3475127865 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28037070 con 0x7f6c340cdb20 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c32ffd640 1 -- 172.21.15.60:0/3475127865 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28032440 con 0x7f6c340cdb20 2024-04-25T08:36:55.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c32ffd640 1 -- 172.21.15.60:0/3475127865 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28032810 con 0x7f6c340cdb20 2024-04-25T08:36:55.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3475127865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cdb20 msgr2=0x7f6c340cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3475127865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cdb20 0x7f6c340cdf80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f6c34158aa0 tx=0x7f6c2802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3475127865 shutdown_connections 2024-04-25T08:36:55.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3475127865 >> v2:172.21.15.100:3300/0 conn(0x7f6c3415b270 0x7f6c3415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3475127865 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cdb20 0x7f6c340cdf80 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3475127865 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c340cc4f0 0x7f6c340cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3475127865 >> 172.21.15.60:0/3475127865 conn(0x7f6c3406abe0 msgr2=0x7f6c3406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3475127865 shutdown_connections 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.582+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3475127865 wait complete. 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 Processor -- start 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 -- start start 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 0x7f6c341466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c340cdb20 0x7f6c3406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c3415b270 0x7f6c3406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34071d60 con 0x7f6c340cc4f0 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34071a80 con 0x7f6c340cdb20 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.584+0000 7f6c3a61f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3415bbc0 con 0x7f6c3415b270 2024-04-25T08:36:55.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 0x7f6c341466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 0x7f6c341466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41706/0 (socket says 172.21.15.60:41706) 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 -- 172.21.15.60:0/3368609429 learned_addr learned my addr 172.21.15.60:0/3368609429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c340cdb20 0x7f6c3406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c3415b270 0x7f6c3406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 -- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c3415b270 msgr2=0x7f6c3406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c3415b270 0x7f6c3406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 -- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c340cdb20 msgr2=0x7f6c3406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c340cdb20 0x7f6c3406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 -- 172.21.15.60:0/3368609429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34146f60 con 0x7f6c340cc4f0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 0x7f6c341466f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c550 tx=0x7f6c2400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c30ff9640 1 -- 172.21.15.60:0/3368609429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24015070 con 0x7f6c340cc4f0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c30ff9640 1 -- 172.21.15.60:0/3368609429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24004220 con 0x7f6c340cc4f0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c34147240 con 0x7f6c340cc4f0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c30ff9640 1 -- 172.21.15.60:0/3368609429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24003610 con 0x7f6c340cc4f0 2024-04-25T08:36:55.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.585+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c34147800 con 0x7f6c340cc4f0 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.587+0000 7f6c30ff9640 1 -- 172.21.15.60:0/3368609429 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c240037b0 con 0x7f6c340cc4f0 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.587+0000 7f6c30ff9640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c2006b4d0 0x7f6c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.587+0000 7f6c30ff9640 1 -- 172.21.15.60:0/3368609429 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c2408a050 con 0x7f6c340cc4f0 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.587+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c2006b4d0 0x7f6c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.588+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c2006b4d0 0x7f6c2006d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6c28034000 tx=0x7f6c2802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.588+0000 7f6c3d3fe740 1 Processor -- start 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.588+0000 7f6c3d3fe740 1 -- start start 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742491200 0x563742493360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x5637424915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637423f5000 con 0x5637424bbcc0 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374239df20 con 0x563742491200 2024-04-25T08:36:55.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637423f10a0 con 0x5637424cd8d0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424bc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41720/0 (socket says 172.21.15.60:41720) 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 -- 172.21.15.60:0/379540513 learned_addr learned my addr 172.21.15.60:0/379540513 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c337fe640 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x5637424915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c33fff640 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742491200 0x563742493360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 -- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 msgr2=0x5637424915e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x5637424915e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 -- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742491200 msgr2=0x563742493360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742491200 0x563742493360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.589+0000 7f6c38bbc640 1 -- 172.21.15.60:0/379540513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637421d5ba0 con 0x5637424bbcc0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424bc140 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00d590 tx=0x7f6c2c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e66ad25d460778 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c19ffb640 1 -- 172.21.15.60:0/379540513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00c850 con 0x5637424bbcc0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c19ffb640 1 -- 172.21.15.60:0/379540513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c00c9f0 con 0x5637424bbcc0 2024-04-25T08:36:55.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c19ffb640 1 -- 172.21.15.60:0/379540513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00cdc0 con 0x5637424bbcc0 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/379540513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 msgr2=0x5637424bc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424bc140 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00d590 tx=0x7f6c2c00da50 comp rx=0 tx=0).stop 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/379540513 shutdown_connections 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424bc140 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x5637424915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/379540513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563742491200 0x563742493360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/379540513 >> 172.21.15.60:0/379540513 conn(0x563742459050 msgr2=0x5637424d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/379540513 shutdown_connections 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.590+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/379540513 wait complete. 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.591+0000 7f6c3d3fe740 1 -- start start 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.591+0000 7f6c3d3fe740 1 Processor -- start 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424bd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424ab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424cd8d0 0x5637424e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563742393610 con 0x5637424bbcc0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637424b9810 con 0x5637424cd8d0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637424bc650 con 0x563742491200 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c3d3fe740 10 client.0 _ll_register_callbacks cb 0x5637423abf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41732/0 (socket says 172.21.15.60:41732) 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 -- 172.21.15.60:0/3288813277 learned_addr learned my addr 172.21.15.60:0/3288813277 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424bd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424cd8d0 0x5637424e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 -- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 msgr2=0x5637424bd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424bd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 -- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424cd8d0 msgr2=0x5637424e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424cd8d0 0x5637424e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.592+0000 7f6c337fe640 1 -- 172.21.15.60:0/3288813277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637420c9490 con 0x5637424bbcc0 2024-04-25T08:36:55.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424ab550 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6c2803e040 tx=0x7f6c28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8385228ea52d3b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c12ffd640 1 -- 172.21.15.60:0/3288813277 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2804e020 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c12ffd640 1 -- 172.21.15.60:0/3288813277 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c28003610 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c2c00c9f0 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c12ffd640 1 -- 172.21.15.60:0/3288813277 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c28058050 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 10 client.8379 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x563742236330 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c12ffd640 1 -- 172.21.15.60:0/3288813277 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c00c9f0 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c12ffd640 1 -- 172.21.15.60:0/3288813277 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c2805b020 con 0x5637424bbcc0 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 10 client.8379 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 10 client.8379 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 20 client.8379 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424da840 0x5637424cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 4 client.8379 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:55.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563742402540 con 0x5637424da840 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.593+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424da840 0x5637424cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c12ffd640 10 client.8379 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424da840 0x5637424cf800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e5e203991e8854 server_cookie=dca8fba2c1a098f3 in_seq=0 out_seq=0 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c12ffd640 1 -- 172.21.15.60:0/3288813277 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563742402540 con 0x5637424da840 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c12ffd640 10 client.8379 handle_command_reply: tid=0 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 client.8379 shutdown 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 msgr2=0x5637424ab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424ab550 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6c2803e040 tx=0x7f6c28004730 comp rx=0 tx=0).stop 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 shutdown_connections 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424cd8d0 0x5637424e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424da840 0x5637424cf800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424bbcc0 0x5637424ab550 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3288813277 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424bd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c12ffd640 0 client.8379 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 >> 172.21.15.60:0/3288813277 conn(0x563742459050 msgr2=0x5637424b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.594+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 shutdown_connections 2024-04-25T08:36:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.595+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3288813277 wait complete. 2024-04-25T08:36:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.595+0000 7f6c3d3fe740 20 client.8379 trim_cache size 0 max 16384 2024-04-25T08:36:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 Processor -- start 2024-04-25T08:36:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 -- start start 2024-04-25T08:36:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 0x5637424d74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34200380 con 0x5637424cd8d0 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341ff600 con 0x5637424bbcc0 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.610+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34156e40 con 0x563742491200 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 0x5637424d74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 0x5637424d74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41740/0 (socket says 172.21.15.60:41740) 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 -- 172.21.15.60:0/3307279654 learned_addr learned my addr 172.21.15.60:0/3307279654 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c33fff640 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 -- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 msgr2=0x5637424ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 -- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 msgr2=0x5637424d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 -- 172.21.15.60:0/3307279654 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637420a2610 con 0x5637424cd8d0 2024-04-25T08:36:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c337fe640 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 0x5637424d74e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c007a00 tx=0x7f6c2c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684747581e1e5aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c197fa640 1 -- 172.21.15.60:0/3307279654 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c026070 con 0x5637424cd8d0 2024-04-25T08:36:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c197fa640 1 -- 172.21.15.60:0/3307279654 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c007c70 con 0x5637424cd8d0 2024-04-25T08:36:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.611+0000 7f6c197fa640 1 -- 172.21.15.60:0/3307279654 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00ca50 con 0x5637424cd8d0 2024-04-25T08:36:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3307279654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 msgr2=0x5637424d74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 0x5637424d74e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c007a00 tx=0x7f6c2c00a000 comp rx=0 tx=0).stop 2024-04-25T08:36:55.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3307279654 shutdown_connections 2024-04-25T08:36:55.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5637424cd8d0 0x5637424d74e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/3307279654 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563742491200 0x5637424ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3307279654 >> 172.21.15.60:0/3307279654 conn(0x563742459050 msgr2=0x5637424b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3307279654 shutdown_connections 2024-04-25T08:36:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/3307279654 wait complete. 2024-04-25T08:36:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.612+0000 7f6c3d3fe740 1 -- start start 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 Processor -- start 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 0x5637424ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x563742545530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637424aba90 con 0x563742491200 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34157440 con 0x5637424bbcc0 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563742542ea0 con 0x5637424cd8d0 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c3d3fe740 10 client.0 _ll_register_callbacks cb 0x7f6c2c007bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 0x5637424ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.613+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 0x5637424ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41752/0 (socket says 172.21.15.60:41752) 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 -- 172.21.15.60:0/987977477 learned_addr learned my addr 172.21.15.60:0/987977477 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c38bbc640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x563742545530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c337fe640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 -- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 msgr2=0x563742545530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x563742545530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 -- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 msgr2=0x5637424cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 -- 172.21.15.60:0/987977477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637425429c0 con 0x563742491200 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c33fff640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 0x5637424ccbf0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6c24058480 tx=0x7f6c2401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f26e6af494fef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c12ffd640 1 -- 172.21.15.60:0/987977477 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2401e040 con 0x563742491200 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c12ffd640 1 -- 172.21.15.60:0/987977477 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c2401ac40 con 0x563742491200 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c2c007c70 con 0x563742491200 2024-04-25T08:36:55.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c12ffd640 1 -- 172.21.15.60:0/987977477 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c2401ae60 con 0x563742491200 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c3d3fe740 10 client.8385 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5637424f8fd0 con 0x563742491200 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.614+0000 7f6c12ffd640 1 -- 172.21.15.60:0/987977477 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c007c70 con 0x563742491200 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c12ffd640 1 -- 172.21.15.60:0/987977477 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c24057bd0 con 0x563742491200 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c3d3fe740 10 client.8385 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c3d3fe740 10 client.8385 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c3d3fe740 20 client.8385 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424b71a0 0x563742545a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c3d3fe740 4 client.8385 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5637424be9c0 con 0x5637424b71a0 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c337fe640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424b71a0 0x563742545a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c12ffd640 10 client.8385 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.615+0000 7f6c337fe640 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424b71a0 0x563742545a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=808f65940a706832 server_cookie=1a6c5e0b17736e95 in_seq=0 out_seq=0 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c12ffd640 1 -- 172.21.15.60:0/987977477 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5637424be9c0 con 0x5637424b71a0 2024-04-25T08:36:55.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c12ffd640 10 client.8385 handle_command_reply: tid=0 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 client.8385 shutdown 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 msgr2=0x5637424ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 0x5637424ccbf0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6c24058480 tx=0x7f6c2401b040 comp rx=0 tx=0).stop 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 shutdown_connections 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5637424cd8d0 0x563742545530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5637424bbcc0 0x5637424cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x5637424b71a0 0x563742545a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c12ffd640 0 client.8385 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 --2- 172.21.15.60:0/987977477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563742491200 0x5637424ccbf0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 >> 172.21.15.60:0/987977477 conn(0x563742459050 msgr2=0x5637424ef590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 shutdown_connections 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 1 -- 172.21.15.60:0/987977477 wait complete. 2024-04-25T08:36:55.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.619+0000 7f6c3d3fe740 20 client.8385 trim_cache size 0 max 16384 2024-04-25T08:36:55.618 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:55.618 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:36:55.618 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "ff3e52b2-4827-46da-a403-d2d18d6ee93a", 2024-04-25T08:36:55.618 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:36:55.618 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c2006b4d0 msgr2=0x7f6c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c2006b4d0 0x7f6c2006d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6c28034000 tx=0x7f6c2802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 msgr2=0x7f6c341466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 0x7f6c341466f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c550 tx=0x7f6c2400ca10 comp rx=0 tx=0).stop 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 shutdown_connections 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c3415b270 0x7f6c3406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6c2006b4d0 0x7f6c2006d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c340cdb20 0x7f6c3406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 --2- 172.21.15.60:0/3368609429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6c340cc4f0 0x7f6c341466f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 >> 172.21.15.60:0/3368609429 conn(0x7f6c3406abe0 msgr2=0x7f6c3415a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 shutdown_connections 2024-04-25T08:36:55.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.620+0000 7f6c3a61f640 1 -- 172.21.15.60:0/3368609429 wait complete. 2024-04-25T08:36:55.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:55.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T08:36:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.752+0000 7fd012f13640 1 Processor -- start 2024-04-25T08:36:55.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.752+0000 7fd012f13640 1 -- start start 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd012f13640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd012f13640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd00c0ccdd0 0x7fd00c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd012f13640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd00c0cdcd0 0x7fd00c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd012f13640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c0d0450 con 0x7fd00c0cdcd0 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd012f13640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c0cb190 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd012f13640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0cb470 con 0x7fd00c15bb20 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd00c0cdcd0 0x7fd00c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd00c0cdcd0 0x7fd00c0cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41764/0 (socket says 172.21.15.60:41764) 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 -- 172.21.15.60:0/3075958849 learned_addr learned my addr 172.21.15.60:0/3075958849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd00bfff640 1 --2- 172.21.15.60:0/3075958849 >> v2:172.21.15.100:3300/0 conn(0x7fd00c0ccdd0 0x7fd00c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd010caf640 1 --2- 172.21.15.60:0/3075958849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 -- 172.21.15.60:0/3075958849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 msgr2=0x7fd00c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 --2- 172.21.15.60:0/3075958849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 -- 172.21.15.60:0/3075958849 >> v2:172.21.15.100:3300/0 conn(0x7fd00c0ccdd0 msgr2=0x7fd00c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 --2- 172.21.15.60:0/3075958849 >> v2:172.21.15.100:3300/0 conn(0x7fd00c0ccdd0 0x7fd00c0cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.754+0000 7fd0114b0640 1 -- 172.21.15.60:0/3075958849 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0cb750 con 0x7fd00c0cdcd0 2024-04-25T08:36:55.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.755+0000 7fd0114b0640 1 --2- 172.21.15.60:0/3075958849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c0cff10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0d1860 tx=0x7fd00402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4705c6527c28dacc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd00b7fe640 1 -- 172.21.15.60:0/3075958849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004036070 con 0x7fd00c0cdcd0 2024-04-25T08:36:55.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd00b7fe640 1 -- 172.21.15.60:0/3075958849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004031600 con 0x7fd00c0cdcd0 2024-04-25T08:36:55.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd00b7fe640 1 -- 172.21.15.60:0/3075958849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0040319d0 con 0x7fd00c0cdcd0 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 -- 172.21.15.60:0/3075958849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0cdcd0 msgr2=0x7fd00c0cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 --2- 172.21.15.60:0/3075958849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c0cff10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0d1860 tx=0x7fd00402a610 comp rx=0 tx=0).stop 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 -- 172.21.15.60:0/3075958849 shutdown_connections 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 --2- 172.21.15.60:0/3075958849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c0cff10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 --2- 172.21.15.60:0/3075958849 >> v2:172.21.15.100:3300/0 conn(0x7fd00c0ccdd0 0x7fd00c0cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 --2- 172.21.15.60:0/3075958849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 -- 172.21.15.60:0/3075958849 >> 172.21.15.60:0/3075958849 conn(0x7fd00c0b2390 msgr2=0x7fd00c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 -- 172.21.15.60:0/3075958849 shutdown_connections 2024-04-25T08:36:55.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.756+0000 7fd012f13640 1 -- 172.21.15.60:0/3075958849 wait complete. 2024-04-25T08:36:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.758+0000 7fd012f13640 1 Processor -- start 2024-04-25T08:36:55.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.758+0000 7fd012f13640 1 -- start start 2024-04-25T08:36:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd012f13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 0x7fd00c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd012f13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd012f13640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd012f13640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0cc770 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd012f13640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0cbd10 con 0x7fd00c0cdcd0 2024-04-25T08:36:55.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd012f13640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c15bf00 con 0x7fd00c15bb20 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 0x7fd00c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 0x7fd00c154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41780/0 (socket says 172.21.15.60:41780) 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 -- 172.21.15.60:0/3390520311 learned_addr learned my addr 172.21.15.60:0/3390520311 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd0114b0640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd00bfff640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 -- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 msgr2=0x7fd00c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 -- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00c0cdcd0 msgr2=0x7fd00c1f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c1f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 -- 172.21.15.60:0/3390520311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c154e40 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd010caf640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 0x7fd00c154900 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e9b0 tx=0x7fd00000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.760+0000 7fd0097fa640 1 -- 172.21.15.60:0/3390520311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000017070 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.761+0000 7fd0097fa640 1 -- 172.21.15.60:0/3390520311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00000c9a0 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.761+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c0b4920 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.761+0000 7fd0097fa640 1 -- 172.21.15.60:0/3390520311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000cd90 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.761+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0b51c0 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.762+0000 7fd0097fa640 1 -- 172.21.15.60:0/3390520311 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd00001a070 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.763+0000 7fd0097fa640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.763+0000 7fd0097fa640 1 -- 172.21.15.60:0/3390520311 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd00008a290 con 0x7fd00c0ccdd0 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.763+0000 7fd00bfff640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 Processor -- start 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 -- start start 2024-04-25T08:36:55.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd00bfff640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcff006b4d0 0x7fcff006d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd00c15c3c0 tx=0x7fcff8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x56272900de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x56272902c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56272902cdf0 0x562728fe4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728f48b90 con 0x562728fe7e40 2024-04-25T08:36:55.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728ef1e40 con 0x56272902cdf0 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.764+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562728f44c30 con 0x56272900da90 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x56272902c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x56272902c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41796/0 (socket says 172.21.15.60:41796) 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56272902cdf0 0x562728fe4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd00bfff640 1 -- 172.21.15.60:0/2728675381 learned_addr learned my addr 172.21.15.60:0/2728675381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd010caf640 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x56272900de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 -- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 msgr2=0x56272900de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x56272900de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 -- 172.21.15.60:0/2728675381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 msgr2=0x56272902c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x56272902c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 -- 172.21.15.60:0/2728675381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562728d277d0 con 0x56272902cdf0 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd00bfff640 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x56272902c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fd0114b0640 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56272902cdf0 0x562728fe4d00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd004030000 tx=0x7fd00402e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810b2e1505d260fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.760 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fcff67fc640 1 -- 172.21.15.60:0/2728675381 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004042070 con 0x56272902cdf0 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fcff67fc640 1 -- 172.21.15.60:0/2728675381 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004037040 con 0x56272902cdf0 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.765+0000 7fcff67fc640 1 -- 172.21.15.60:0/2728675381 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004036040 con 0x56272902cdf0 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 -- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56272902cdf0 msgr2=0x562728fe4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56272902cdf0 0x562728fe4d00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd004030000 tx=0x7fd00402e000 comp rx=0 tx=0).stop 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 -- 172.21.15.60:0/2728675381 shutdown_connections 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56272902cdf0 0x562728fe4d00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x56272902c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2728675381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x56272900de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 -- 172.21.15.60:0/2728675381 >> 172.21.15.60:0/2728675381 conn(0x562728fad870 msgr2=0x56272902b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 -- 172.21.15.60:0/2728675381 shutdown_connections 2024-04-25T08:36:55.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.766+0000 7fd015c0c740 1 -- 172.21.15.60:0/2728675381 wait complete. 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.767+0000 7fd015c0c740 1 -- start start 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.767+0000 7fd015c0c740 1 Processor -- start 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe7e40 0x562728fff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 0x562729001110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562728fe5430 0x562728fe58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728fe22a0 con 0x56272900da90 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562728f053e0 con 0x562728fe7e40 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562728fe6dd0 con 0x562728fe5430 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd015c0c740 10 client.0 _ll_register_callbacks cb 0x562728f04db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 0x562729001110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 0x562729001110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41804/0 (socket says 172.21.15.60:41804) 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 -- 172.21.15.60:0/2489672457 learned_addr learned my addr 172.21.15.60:0/2489672457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd010caf640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe7e40 0x562728fff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd0114b0640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562728fe5430 0x562728fe58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 -- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562728fe5430 msgr2=0x562728fe58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562728fe5430 0x562728fe58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 -- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe7e40 msgr2=0x562728fff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe7e40 0x562728fff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 -- 172.21.15.60:0/2489672457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562728c1d0f0 con 0x56272900da90 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fd00bfff640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 0x562729001110 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcff8011f70 tx=0x7fcff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a057fd6b601fb397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.769+0000 7fcfeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/2489672457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8027070 con 0x56272900da90 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/2489672457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcff8014440 con 0x56272900da90 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd004037040 con 0x56272900da90 2024-04-25T08:36:55.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/2489672457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcff8032050 con 0x56272900da90 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 10 client.8397 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x562728d87f60 con 0x56272900da90 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/2489672457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004037040 con 0x56272900da90 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/2489672457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcff8035020 con 0x56272900da90 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 10 client.8397 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 10 client.8397 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 20 client.8397 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729006990 0x562729006df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 4 client.8397 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562728f560d0 con 0x562729006990 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.770+0000 7fd010caf640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729006990 0x562729006df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.771+0000 7fcfeb7fe640 10 client.8397 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.771+0000 7fd010caf640 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729006990 0x562729006df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485155618634e376 server_cookie=4396f94fcb390b68 in_seq=0 out_seq=0 2024-04-25T08:36:55.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/2489672457 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562728f560d0 con 0x562729006990 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fcfeb7fe640 10 client.8397 handle_command_reply: tid=0 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 client.8397 shutdown 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 msgr2=0x562729001110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 0x562729001110 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fcff8011f70 tx=0x7fcff80046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 shutdown_connections 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x562728fe5430 0x562728fe58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729006990 0x562729006df0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56272900da90 0x562729001110 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 --2- 172.21.15.60:0/2489672457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe7e40 0x562728fff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 >> 172.21.15.60:0/2489672457 conn(0x562728fad870 msgr2=0x56272902ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fcfeb7fe640 0 client.8397 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 shutdown_connections 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 1 -- 172.21.15.60:0/2489672457 wait complete. 2024-04-25T08:36:55.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.772+0000 7fd015c0c740 20 client.8397 trim_cache size 0 max 16384 2024-04-25T08:36:55.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.788+0000 7fd015c0c740 1 Processor -- start 2024-04-25T08:36:55.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.788+0000 7fd015c0c740 1 -- start start 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729006d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728fe12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562729007970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd015c0c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1fc050 con 0x562728fe7e40 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0c6bf0 con 0x562728fe5430 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0b3d10 con 0x56272900da90 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728fe12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728fe12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41810/0 (socket says 172.21.15.60:41810) 2024-04-25T08:36:55.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 -- 172.21.15.60:0/1117495450 learned_addr learned my addr 172.21.15.60:0/1117495450 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd010caf640 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729006d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd0114b0640 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562729007970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 -- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 msgr2=0x562729007970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562729007970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 -- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 msgr2=0x562729006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729006d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd00bfff640 1 -- 172.21.15.60:0/1117495450 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562728fffb80 con 0x562728fe7e40 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.789+0000 7fd010caf640 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729006d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.790+0000 7fd00bfff640 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728fe12c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcff80046b0 tx=0x7fcff8024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea445a51b1fd927e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.790+0000 7fcff5ffb640 1 -- 172.21.15.60:0/1117495450 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff803a070 con 0x562728fe7e40 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.790+0000 7fcff5ffb640 1 -- 172.21.15.60:0/1117495450 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8014890 con 0x562728fe7e40 2024-04-25T08:36:55.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.790+0000 7fcff5ffb640 1 -- 172.21.15.60:0/1117495450 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8027040 con 0x562728fe7e40 2024-04-25T08:36:55.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 -- 172.21.15.60:0/1117495450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 msgr2=0x562728fe12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728fe12c0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcff80046b0 tx=0x7fcff8024000 comp rx=0 tx=0).stop 2024-04-25T08:36:55.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 -- 172.21.15.60:0/1117495450 shutdown_connections 2024-04-25T08:36:55.785 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562729007970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728fe12c0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 --2- 172.21.15.60:0/1117495450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729006d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 -- 172.21.15.60:0/1117495450 >> 172.21.15.60:0/1117495450 conn(0x562728fad870 msgr2=0x562728fdbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 -- 172.21.15.60:0/1117495450 shutdown_connections 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 -- 172.21.15.60:0/1117495450 wait complete. 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.791+0000 7fd015c0c740 1 -- start start 2024-04-25T08:36:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.792+0000 7fd015c0c740 1 Processor -- start 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728ffc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562728ffcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562729000b10 con 0x562728fe7e40 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0b4310 con 0x562728fe5430 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56272900c9e0 con 0x56272900da90 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd015c0c740 10 client.0 _ll_register_callbacks cb 0x7fcff8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728ffc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728ffc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41826/0 (socket says 172.21.15.60:41826) 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 -- 172.21.15.60:0/4231099284 learned_addr learned my addr 172.21.15.60:0/4231099284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.789 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd010caf640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd0114b0640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562728ffcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 -- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 msgr2=0x562728ffcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562728ffcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 -- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 msgr2=0x562729003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd00bfff640 1 -- 172.21.15.60:0/4231099284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562728bf61b0 con 0x562728fe7e40 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.793+0000 7fd010caf640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd00bfff640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728ffc5d0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcff8000c00 tx=0x7fcff8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdace916cf80323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fcfeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/4231099284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8045070 con 0x562728fe7e40 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/4231099284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcff803f650 con 0x562728fe7e40 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcff8014890 con 0x562728fe7e40 2024-04-25T08:36:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/4231099284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcff8050050 con 0x562728fe7e40 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 10 client.8403 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x562729002e20 con 0x562728fe7e40 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/4231099284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8014890 con 0x562728fe7e40 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/4231099284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcff8053020 con 0x562728fe7e40 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 10 client.8403 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 10 client.8403 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 20 client.8403 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729094620 0x56272900ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 4 client.8403 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.794+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x562728f50420 con 0x562729094620 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.795+0000 7fd010caf640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729094620 0x56272900ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.795+0000 7fcfeb7fe640 10 client.8403 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.795+0000 7fd010caf640 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729094620 0x56272900ba50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878928015d2069db server_cookie=b6d0d02b32e537ca in_seq=0 out_seq=0 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.856+0000 7fcfeb7fe640 1 -- 172.21.15.60:0/4231099284 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x562728f50420 con 0x562729094620 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.856+0000 7fcfeb7fe640 10 client.8403 handle_command_reply: tid=0 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.856+0000 7fd015c0c740 1 client.8403 shutdown 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.856+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 msgr2=0x562728ffc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.856+0000 7fd015c0c740 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728ffc5d0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcff8000c00 tx=0x7fcff8019000 comp rx=0 tx=0).stop 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 shutdown_connections 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56272900da90 0x562728ffcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x562729094620 0x56272900ba50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562728fe7e40 0x562728ffc5d0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 --2- 172.21.15.60:0/4231099284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562728fe5430 0x562729003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fcfeb7fe640 0 client.8403 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 >> 172.21.15.60:0/4231099284 conn(0x562728fad870 msgr2=0x562728fe6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 shutdown_connections 2024-04-25T08:36:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 1 -- 172.21.15.60:0/4231099284 wait complete. 2024-04-25T08:36:55.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd015c0c740 20 client.8403 trim_cache size 0 max 16384 2024-04-25T08:36:55.852 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:55.852 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:36:55.852 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:55.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcff006b4d0 msgr2=0x7fcff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcff006b4d0 0x7fcff006d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd00c15c3c0 tx=0x7fcff8009190 comp rx=0 tx=0).stop 2024-04-25T08:36:55.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 msgr2=0x7fd00c154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 0x7fd00c154900 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e9b0 tx=0x7fd00000ee70 comp rx=0 tx=0).stop 2024-04-25T08:36:55.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 shutdown_connections 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd00c15bb20 0x7fd00c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcff006b4d0 0x7fcff006d990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd00c0cdcd0 0x7fd00c1f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.857+0000 7fd012f13640 1 --2- 172.21.15.60:0/3390520311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd00c0ccdd0 0x7fd00c154900 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.858+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 >> 172.21.15.60:0/3390520311 conn(0x7fd00c0b2390 msgr2=0x7fd00c06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.858+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 shutdown_connections 2024-04-25T08:36:55.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.858+0000 7fd012f13640 1 -- 172.21.15.60:0/3390520311 wait complete. 2024-04-25T08:36:55.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:55.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:36:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.971+0000 7f43248dd640 1 Processor -- start 2024-04-25T08:36:55.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.971+0000 7f43248dd640 1 -- start start 2024-04-25T08:36:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f43248dd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f432015b7d0 0x7f43200cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f43248dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43200b5620 0x7f43200b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f43248dd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432006cf50 con 0x7f432006c2d0 2024-04-25T08:36:55.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f43248dd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43200b5fc0 con 0x7f432015b7d0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f43248dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200ce530 con 0x7f43200b5620 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f432015b7d0 0x7f43200cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f432006c2d0 0x7f432006c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50628/0 (socket says 172.21.15.60:50628) 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431ed9d640 1 -- 172.21.15.60:0/4023045280 learned_addr learned my addr 172.21.15.60:0/4023045280 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431dd9b640 1 --2- 172.21.15.60:0/4023045280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43200b5620 0x7f43200b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431e59c640 1 -- 172.21.15.60:0/4023045280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43200b5620 msgr2=0x7f43200b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431e59c640 1 --2- 172.21.15.60:0/4023045280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43200b5620 0x7f43200b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431e59c640 1 -- 172.21.15.60:0/4023045280 >> v1:172.21.15.60:6789/0 conn(0x7f432006c2d0 legacy=0x7f432006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431d59a640 1 -- 172.21.15.60:0/4023045280 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229831816 0 0) 0x7f432006cf50 con 0x7f432006c2d0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.972+0000 7f431e59c640 1 -- 172.21.15.60:0/4023045280 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200ce810 con 0x7f432015b7d0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.973+0000 7f431e59c640 1 --2- 172.21.15.60:0/4023045280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432015b7d0 0x7f43200cf010 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f43100097a0 tx=0x7f431002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f186e0dbd67626 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.973+0000 7f431d59a640 1 -- 172.21.15.60:0/4023045280 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310037070 con 0x7f432015b7d0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.973+0000 7f431d59a640 1 -- 172.21.15.60:0/4023045280 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310032440 con 0x7f432015b7d0 2024-04-25T08:36:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.973+0000 7f431d59a640 1 -- 172.21.15.60:0/4023045280 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310032810 con 0x7f432015b7d0 2024-04-25T08:36:55.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 -- 172.21.15.60:0/4023045280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432015b7d0 msgr2=0x7f43200cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 --2- 172.21.15.60:0/4023045280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432015b7d0 0x7f43200cf010 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f43100097a0 tx=0x7f431002a180 comp rx=0 tx=0).stop 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 -- 172.21.15.60:0/4023045280 shutdown_connections 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 --2- 172.21.15.60:0/4023045280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43200b5620 0x7f43200b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 --2- 172.21.15.60:0/4023045280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432015b7d0 0x7f43200cf010 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 -- 172.21.15.60:0/4023045280 >> 172.21.15.60:0/4023045280 conn(0x7f43200d0800 msgr2=0x7f43200d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 -- 172.21.15.60:0/4023045280 shutdown_connections 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.974+0000 7f43248dd640 1 -- 172.21.15.60:0/4023045280 wait complete. 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.975+0000 7f43248dd640 1 Processor -- start 2024-04-25T08:36:55.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.975+0000 7f43248dd640 1 -- start start 2024-04-25T08:36:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f43248dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f432006c2d0 0x7f4320157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f43248dd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 0x7f43200c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f43248dd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432015b7d0 0x7f43200ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f43248dd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200cf550 con 0x7f43200b5620 2024-04-25T08:36:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f43248dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432006d230 con 0x7f432006c2d0 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f43248dd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200cf6d0 con 0x7f432015b7d0 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 0x7f43200c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 0x7f43200c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41832/0 (socket says 172.21.15.60:41832) 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 -- 172.21.15.60:0/1429210145 learned_addr learned my addr 172.21.15.60:0/1429210145 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431e59c640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f432006c2d0 0x7f4320157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431ed9d640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432015b7d0 0x7f43200ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 -- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432015b7d0 msgr2=0x7f43200ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432015b7d0 0x7f43200ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 -- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f432006c2d0 msgr2=0x7f4320157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f432006c2d0 0x7f4320157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 -- 172.21.15.60:0/1429210145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43201579d0 con 0x7f43200b5620 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431ed9d640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432015b7d0 0x7f43200ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.976+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 0x7f43200c9ac0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f431400d800 tx=0x7f431400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.977+0000 7f430effd640 1 -- 172.21.15.60:0/1429210145 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400b6c0 con 0x7f43200b5620 2024-04-25T08:36:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.977+0000 7f430effd640 1 -- 172.21.15.60:0/1429210145 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314019070 con 0x7f43200b5620 2024-04-25T08:36:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.977+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43200cc910 con 0x7f43200b5620 2024-04-25T08:36:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.977+0000 7f430effd640 1 -- 172.21.15.60:0/1429210145 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400ba80 con 0x7f43200b5620 2024-04-25T08:36:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.977+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43200cced0 con 0x7f43200b5620 2024-04-25T08:36:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.978+0000 7f430effd640 1 -- 172.21.15.60:0/1429210145 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f431400bc20 con 0x7f43200b5620 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.978+0000 7f430effd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f430006b4d0 0x7f430006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.979+0000 7f430effd640 1 -- 172.21.15.60:0/1429210145 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f431408a7b0 con 0x7f43200b5620 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.979+0000 7f431e59c640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f430006b4d0 0x7f430006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.979+0000 7f431e59c640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f430006b4d0 0x7f430006d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4310034000 tx=0x7f431002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.979+0000 7f43275db740 1 Processor -- start 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 -- start start 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 0x563db3465320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462ef0 0x563db3465860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3435770 0x563db3438f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db339eda0 con 0x563db3462b10 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db3347b80 con 0x563db3462ef0 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563db339ae40 con 0x563db3435770 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.980+0000 7f431e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 0x563db3465320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 0x563db3465320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41846/0 (socket says 172.21.15.60:41846) 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 -- 172.21.15.60:0/1179642735 learned_addr learned my addr 172.21.15.60:0/1179642735 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431ed9d640 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3435770 0x563db3438f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462ef0 0x563db3465860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 -- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3435770 msgr2=0x563db3438f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3435770 0x563db3438f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 -- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462ef0 msgr2=0x563db3465860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462ef0 0x563db3465860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 -- 172.21.15.60:0/1179642735 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563db3351f10 con 0x563db3462b10 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462ef0 0x563db3465860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f431e59c640 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 0x563db3465320 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f431002a1c0 tx=0x7f431002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19344e158d13ee38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f42fffff640 1 -- 172.21.15.60:0/1179642735 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431004f070 con 0x563db3462b10 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f42fffff640 1 -- 172.21.15.60:0/1179642735 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310038040 con 0x563db3462b10 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.981+0000 7f42fffff640 1 -- 172.21.15.60:0/1179642735 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310037050 con 0x563db3462b10 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 -- 172.21.15.60:0/1179642735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 msgr2=0x563db3465320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 0x563db3465320 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f431002a1c0 tx=0x7f431002c000 comp rx=0 tx=0).stop 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 -- 172.21.15.60:0/1179642735 shutdown_connections 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3435770 0x563db3438f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462ef0 0x563db3465860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 --2- 172.21.15.60:0/1179642735 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462b10 0x563db3465320 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 -- 172.21.15.60:0/1179642735 >> 172.21.15.60:0/1179642735 conn(0x563db34379e0 msgr2=0x563db3480320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 -- 172.21.15.60:0/1179642735 shutdown_connections 2024-04-25T08:36:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.982+0000 7f43275db740 1 -- 172.21.15.60:0/1179642735 wait complete. 2024-04-25T08:36:55.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.983+0000 7f43275db740 1 -- start start 2024-04-25T08:36:55.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.983+0000 7f43275db740 1 Processor -- start 2024-04-25T08:36:55.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.983+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3482580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db3481450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3482ac0 0x563db3481990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db343ab90 con 0x563db3462ef0 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db333d4b0 con 0x563db3462b10 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563db343ae70 con 0x563db3482ac0 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 10 client.0 _ll_register_callbacks cb 0x563db337d180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db3481450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3482580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db3481450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41858/0 (socket says 172.21.15.60:41858) 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 -- 172.21.15.60:0/708486992 learned_addr learned my addr 172.21.15.60:0/708486992 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431ed9d640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3482ac0 0x563db3481990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 -- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3482ac0 msgr2=0x563db3481990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3482ac0 0x563db3481990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 -- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 msgr2=0x563db3482580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3482580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 -- 172.21.15.60:0/708486992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563db317d7d0 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431e59c640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3482580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f431dd9b640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db3481450 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4314058dc0 tx=0x7f43140595f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2657caae82188c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f42fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f42fd7fa640 1 -- 172.21.15.60:0/708486992 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314087060 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f42fd7fa640 1 -- 172.21.15.60:0/708486992 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43140554a0 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4310038040 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f42fd7fa640 1 -- 172.21.15.60:0/708486992 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4314070070 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 10 client.8418 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.984+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x563db30b33f0 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f42fd7fa640 1 -- 172.21.15.60:0/708486992 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310038040 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f42fd7fa640 1 -- 172.21.15.60:0/708486992 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4314013070 con 0x563db3462ef0 2024-04-25T08:36:55.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f43275db740 10 client.8418 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f43275db740 10 client.8418 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f43275db740 20 client.8418 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f43275db740 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db347bd10 0x563db347c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f43275db740 4 client.8418 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563db33ac2e0 con 0x563db347bd10 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f431e59c640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db347bd10 0x563db347c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f42fd7fa640 10 client.8418 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.985+0000 7f431e59c640 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db347bd10 0x563db347c140 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676e7dd2f0550a55 server_cookie=2ef7d3cee506da50 in_seq=0 out_seq=0 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f42fd7fa640 1 -- 172.21.15.60:0/708486992 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563db33ac2e0 con 0x563db347bd10 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f42fd7fa640 10 client.8418 handle_command_reply: tid=0 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 client.8418 shutdown 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 msgr2=0x563db3481450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db3481450 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4314058dc0 tx=0x7f43140595f0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 shutdown_connections 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3482ac0 0x563db3481990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db347bd10 0x563db347c140 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db3481450 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 --2- 172.21.15.60:0/708486992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3482580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f42fd7fa640 0 client.8418 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 >> 172.21.15.60:0/708486992 conn(0x563db3435770 msgr2=0x563db3434600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 shutdown_connections 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 1 -- 172.21.15.60:0/708486992 wait complete. 2024-04-25T08:36:55.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:55.986+0000 7f43275db740 20 client.8418 trim_cache size 0 max 16384 2024-04-25T08:36:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.001+0000 7f43275db740 1 Processor -- start 2024-04-25T08:36:55.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.001+0000 7f43275db740 1 -- start start 2024-04-25T08:36:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3460ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462ef0 0x563db347ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f43275db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 0x563db347d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f43275db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43202000b0 con 0x563db3482ac0 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201ff330 con 0x563db3462b10 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200c8350 con 0x563db3462ef0 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 0x563db347d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 0x563db347d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41862/0 (socket says 172.21.15.60:41862) 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 -- 172.21.15.60:0/3327507419 learned_addr learned my addr 172.21.15.60:0/3327507419 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431dd9b640 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462ef0 0x563db347ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431e59c640 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3460ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 -- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462ef0 msgr2=0x563db347ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462ef0 0x563db347ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 -- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 msgr2=0x563db3460ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:55.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3460ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 -- 172.21.15.60:0/3327507419 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563db30730f0 con 0x563db3482ac0 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.002+0000 7f431ed9d640 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 0x563db347d230 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4318001cb0 tx=0x7f4318002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5bfb5b4e3843c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f42ff7fe640 1 -- 172.21.15.60:0/3327507419 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318007890 con 0x563db3482ac0 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f42ff7fe640 1 -- 172.21.15.60:0/3327507419 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431801a020 con 0x563db3482ac0 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f42ff7fe640 1 -- 172.21.15.60:0/3327507419 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318015610 con 0x563db3482ac0 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 -- 172.21.15.60:0/3327507419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 msgr2=0x563db347d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 0x563db347d230 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4318001cb0 tx=0x7f4318002170 comp rx=0 tx=0).stop 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 -- 172.21.15.60:0/3327507419 shutdown_connections 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3482ac0 0x563db347d230 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462ef0 0x563db347ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 --2- 172.21.15.60:0/3327507419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3462b10 0x563db3460ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 -- 172.21.15.60:0/3327507419 >> 172.21.15.60:0/3327507419 conn(0x563db3435770 msgr2=0x563db3469000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:55.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 -- 172.21.15.60:0/3327507419 shutdown_connections 2024-04-25T08:36:55.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.003+0000 7f43275db740 1 -- 172.21.15.60:0/3327507419 wait complete. 2024-04-25T08:36:56.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.004+0000 7f43275db740 1 -- start start 2024-04-25T08:36:56.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.004+0000 7f43275db740 1 Processor -- start 2024-04-25T08:36:56.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462b10 0x563db3468490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db34689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3482ac0 0x563db3499810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200c8950 con 0x563db3462ef0 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563db343ca80 con 0x563db3482ac0 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x563db3461870 con 0x563db3462b10 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 10 client.0 _ll_register_callbacks cb 0x7f4310035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db34689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db34689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41878/0 (socket says 172.21.15.60:41878) 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 -- 172.21.15.60:0/1897957234 learned_addr learned my addr 172.21.15.60:0/1897957234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431ed9d640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3482ac0 0x563db3499810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431e59c640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462b10 0x563db3468490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 -- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462b10 msgr2=0x563db3468490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462b10 0x563db3468490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 -- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3482ac0 msgr2=0x563db3499810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3482ac0 0x563db3499810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 -- 172.21.15.60:0/1897957234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563db347b670 con 0x563db3462ef0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f431dd9b640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db34689d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4314058b30 tx=0x7f4314088ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e14428a6ffc6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f42fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f42fd7fa640 1 -- 172.21.15.60:0/1897957234 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431407b070 con 0x563db3462ef0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f42fd7fa640 1 -- 172.21.15.60:0/1897957234 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4314058280 con 0x563db3462ef0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.005+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f431801a020 con 0x563db3462ef0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f42fd7fa640 1 -- 172.21.15.60:0/1897957234 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4314066050 con 0x563db3462ef0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 10 client.8424 fetch_fsmap learned FSMap version 62 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x563db3435340 con 0x563db3462ef0 2024-04-25T08:36:56.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f42fd7fa640 1 -- 172.21.15.60:0/1897957234 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431801a020 con 0x563db3462ef0 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f42fd7fa640 1 -- 172.21.15.60:0/1897957234 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4314077040 con 0x563db3462ef0 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 10 client.8424 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 10 client.8424 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 20 client.8424 populate_metadata read hostname 'smithi060' 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db3461170 0x563db3499d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 4 client.8424 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563db343a7d0 con 0x563db3461170 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f431e59c640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db3461170 0x563db3499d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f42fd7fa640 10 client.8424 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f431e59c640 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db3461170 0x563db3499d50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d50938fb4e3838 server_cookie=2619265ae1e61009 in_seq=0 out_seq=0 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f42fd7fa640 1 -- 172.21.15.60:0/1897957234 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x563db343a7d0 con 0x563db3461170 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f42fd7fa640 10 client.8424 handle_command_reply: tid=0 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.006+0000 7f43275db740 1 client.8424 shutdown 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 msgr2=0x563db34689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db34689d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f4314058b30 tx=0x7f4314088ab0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 shutdown_connections 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563db3482ac0 0x563db3499810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x563db3461170 0x563db3499d50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563db3462ef0 0x563db34689d0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 --2- 172.21.15.60:0/1897957234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x563db3462b10 0x563db3468490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f42fd7fa640 0 client.8424 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 >> 172.21.15.60:0/1897957234 conn(0x563db3435770 msgr2=0x563db3469000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 shutdown_connections 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 1 -- 172.21.15.60:0/1897957234 wait complete. 2024-04-25T08:36:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43275db740 20 client.8424 trim_cache size 0 max 16384 2024-04-25T08:36:56.007 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:36:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:36:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:36:56.007 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f430006b4d0 msgr2=0x7f430006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f430006b4d0 0x7f430006d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4310034000 tx=0x7f431002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 msgr2=0x7f43200c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 0x7f43200c9ac0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f431400d800 tx=0x7f431400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 shutdown_connections 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432015b7d0 0x7f43200ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f43200b5620 0x7f43200c9ac0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f430006b4d0 0x7f430006d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 --2- 172.21.15.60:0/1429210145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f432006c2d0 0x7f4320157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.007+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 >> 172.21.15.60:0/1429210145 conn(0x7f43200d0800 msgr2=0x7f432015f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.008+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 shutdown_connections 2024-04-25T08:36:56.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.008+0000 7f43248dd640 1 -- 172.21.15.60:0/1429210145 wait complete. 2024-04-25T08:36:56.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:56.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.118+0000 7f94a4534640 1 Processor -- start 2024-04-25T08:36:56.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.118+0000 7f94a4534640 1 -- start start 2024-04-25T08:36:56.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a4534640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 0x7f949c168ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a4534640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c0ceee0 con 0x7f949c15db70 2024-04-25T08:36:56.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a4534640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c0d1d10 con 0x7f949c0b6cb0 2024-04-25T08:36:56.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a4534640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0d3800 con 0x7f949c0b5c30 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a22d0640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f949c15db70 0x7f949c162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50644/0 (socket says 172.21.15.60:50644) 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a22d0640 1 -- 172.21.15.60:0/1486097649 learned_addr learned my addr 172.21.15.60:0/1486097649 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.119+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 0x7f949c168ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388591826 0 0) 0x7f949c0ceee0 con 0x7f949c15db70 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9484003520 con 0x7f949c15db70 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574327203 0 0) 0x7f949c0d1d10 con 0x7f949c0b6cb0 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f949c0ceee0 con 0x7f949c0b6cb0 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3641665275 0 0) 0x7f9484003520 con 0x7f949c15db70 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 msgr2=0x7f949c168ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 0x7f949c168ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 >> v1:172.21.15.100:6789/0 conn(0x7f949c0b6cb0 legacy=0x7f949c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0d1ff0 con 0x7f949c15db70 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f9494001670 con 0x7f949c15db70 2024-04-25T08:36:56.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 >> v1:172.21.15.60:6789/0 conn(0x7f949c15db70 legacy=0x7f949c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 0x7f9484003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94840040d0 0x7f9484004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9484004d40 0x7f94840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0d3800 con 0x7f9484004d40 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948400baa0 con 0x7f94840040d0 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948400bca0 con 0x7f949c0b5c30 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a22d0640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 0x7f9484003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9484004d40 0x7f94840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a1acf640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94840040d0 0x7f9484004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a2ad1640 1 -- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 msgr2=0x7f9484003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c0b5c30 0x7f9484003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a2ad1640 1 -- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94840040d0 msgr2=0x7f9484004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94840040d0 0x7f9484004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.120+0000 7f94a2ad1640 1 -- 172.21.15.60:0/1486097649 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484003520 con 0x7f9484004d40 2024-04-25T08:36:56.117 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.121+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9484004d40 0x7f94840051c0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f948c009f90 tx=0x7f9484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a11371ab8466e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.121+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c009c10 con 0x7f9484004d40 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.121+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c036070 con 0x7f9484004d40 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.121+0000 7f94a12ce640 1 -- 172.21.15.60:0/1486097649 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c031810 con 0x7f9484004d40 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a4534640 1 -- 172.21.15.60:0/1486097649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9484004d40 msgr2=0x7f94840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a4534640 1 --2- 172.21.15.60:0/1486097649 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9484004d40 0x7f94840051c0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f948c009f90 tx=0x7f9484005830 comp rx=0 tx=0).stop 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a2ad1640 1 -- 172.21.15.60:0/1486097649 reap_dead start 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a4534640 1 -- 172.21.15.60:0/1486097649 shutdown_connections 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a4534640 1 -- 172.21.15.60:0/1486097649 >> 172.21.15.60:0/1486097649 conn(0x7f949c0d0520 msgr2=0x7f949c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a4534640 1 -- 172.21.15.60:0/1486097649 shutdown_connections 2024-04-25T08:36:56.118 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.122+0000 7f94a4534640 1 -- 172.21.15.60:0/1486097649 wait complete. 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.123+0000 7f94a4534640 1 Processor -- start 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.123+0000 7f94a4534640 1 -- start start 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a4534640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9484004d40 0x7f949c0cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a4534640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c160760 0x7f949c160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a4534640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 0x7f949c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a4534640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0ce530 con 0x7f949c0c7830 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a4534640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15df50 con 0x7f9484004d40 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a4534640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0cf1c0 con 0x7f949c160760 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 0x7f949c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 0x7f949c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41900/0 (socket says 172.21.15.60:41900) 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 -- 172.21.15.60:0/3835075307 learned_addr learned my addr 172.21.15.60:0/3835075307 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a22d0640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9484004d40 0x7f949c0cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a1acf640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c160760 0x7f949c160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 -- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c160760 msgr2=0x7f949c160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c160760 0x7f949c160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 -- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9484004d40 msgr2=0x7f949c0cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9484004d40 0x7f949c0cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.124+0000 7f94a2ad1640 1 -- 172.21.15.60:0/3835075307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0c9280 con 0x7f949c0c7830 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.125+0000 7f94a2ad1640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 0x7f949c0c7cb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f948c009f90 tx=0x7f948c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.125+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c009c10 con 0x7f949c0c7830 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.125+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c045430 con 0x7f949c0c7830 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.125+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c045820 con 0x7f949c0c7830 2024-04-25T08:36:56.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.125+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c0cad40 con 0x7f949c0c7830 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.125+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c0cb2c0 con 0x7f949c0c7830 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.126+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c073990 con 0x7f949c0c7830 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.126+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f948c005d10 con 0x7f949c0c7830 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.127+0000 7f949affd640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f948806b4d0 0x7f948806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.127+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f948c0ab5a0 con 0x7f949c0c7830 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.127+0000 7f94a22d0640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f948806b4d0 0x7f948806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.127+0000 7f94a22d0640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f948806b4d0 0x7f948806d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f949c1f6b60 tx=0x7f9494001e40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.131+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f949c0c9280 con 0x7f949c0c7830 2024-04-25T08:36:56.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.349+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f949c161da0 con 0x7f949c0c7830 2024-04-25T08:36:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.349+0000 7f949affd640 1 -- 172.21.15.60:0/3835075307 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f948c07ae20 con 0x7f949c0c7830 2024-04-25T08:36:56.343 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:56.343 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:36:56.343 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f948806b4d0 msgr2=0x7f948806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f948806b4d0 0x7f948806d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f949c1f6b60 tx=0x7f9494001e40 comp rx=0 tx=0).stop 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 msgr2=0x7f949c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 0x7f949c0c7cb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f948c009f90 tx=0x7f948c009750 comp rx=0 tx=0).stop 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 shutdown_connections 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f949c0c7830 0x7f949c0c7cb0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f949c160760 0x7f949c160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f948806b4d0 0x7f948806d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 --2- 172.21.15.60:0/3835075307 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9484004d40 0x7f949c0cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 >> 172.21.15.60:0/3835075307 conn(0x7f949c0d0520 msgr2=0x7f949c15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:56.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 shutdown_connections 2024-04-25T08:36:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.351+0000 7f94a4534640 1 -- 172.21.15.60:0/3835075307 wait complete. 2024-04-25T08:36:56.355 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:36:56.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:56.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.462+0000 7f1fa7409640 1 Processor -- start 2024-04-25T08:36:56.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.462+0000 7f1fa7409640 1 -- start start 2024-04-25T08:36:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa7409640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa00b6cb0 0x7f1fa00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa7409640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fa00b5a60 0x7f1fa00d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa7409640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa00ceee0 con 0x7f1fa015db70 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa7409640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa00d1d10 con 0x7f1fa00b6cb0 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa7409640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa00d1ff0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa49a4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1fa00b6cb0 0x7f1fa00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa51a5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1fa015db70 0x7f1fa0162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50658/0 (socket says 172.21.15.60:50658) 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa51a5640 1 -- 172.21.15.60:0/816450541 learned_addr learned my addr 172.21.15.60:0/816450541 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa59a6640 1 --2- 172.21.15.60:0/816450541 >> v2:172.21.15.100:3300/0 conn(0x7f1fa00b5a60 0x7f1fa00d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa49a4640 1 -- 172.21.15.60:0/816450541 >> v2:172.21.15.100:3300/0 conn(0x7f1fa00b5a60 msgr2=0x7f1fa00d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa49a4640 1 --2- 172.21.15.60:0/816450541 >> v2:172.21.15.100:3300/0 conn(0x7f1fa00b5a60 0x7f1fa00d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa49a4640 1 -- 172.21.15.60:0/816450541 >> v1:172.21.15.60:6789/0 conn(0x7f1fa015db70 legacy=0x7f1fa0162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.463+0000 7f1fa49a4640 1 -- 172.21.15.60:0/816450541 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00d22d0 con 0x7f1fa00b6cb0 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa49a4640 1 --2- 172.21.15.60:0/816450541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b6cb0 0x7f1fa00b5520 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00cfa10 tx=0x7f1f9002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97578d798cb67867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1f97fff640 1 -- 172.21.15.60:0/816450541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90037070 con 0x7f1fa00b6cb0 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1f97fff640 1 -- 172.21.15.60:0/816450541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f90032440 con 0x7f1fa00b6cb0 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1f97fff640 1 -- 172.21.15.60:0/816450541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9002ae60 con 0x7f1fa00b6cb0 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa7409640 1 -- 172.21.15.60:0/816450541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b6cb0 msgr2=0x7f1fa00b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa7409640 1 --2- 172.21.15.60:0/816450541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b6cb0 0x7f1fa00b5520 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00cfa10 tx=0x7f1f9002a110 comp rx=0 tx=0).stop 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa7409640 1 -- 172.21.15.60:0/816450541 shutdown_connections 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa7409640 1 --2- 172.21.15.60:0/816450541 >> v2:172.21.15.100:3300/0 conn(0x7f1fa00b5a60 0x7f1fa00d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa7409640 1 --2- 172.21.15.60:0/816450541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b6cb0 0x7f1fa00b5520 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.464+0000 7f1fa7409640 1 -- 172.21.15.60:0/816450541 >> 172.21.15.60:0/816450541 conn(0x7f1fa00d0520 msgr2=0x7f1fa00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:56.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.465+0000 7f1fa7409640 1 -- 172.21.15.60:0/816450541 shutdown_connections 2024-04-25T08:36:56.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.465+0000 7f1fa7409640 1 -- 172.21.15.60:0/816450541 wait complete. 2024-04-25T08:36:56.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 Processor -- start 2024-04-25T08:36:56.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 -- start start 2024-04-25T08:36:56.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 0x7f1fa015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fa00b6cb0 0x7f1fa00c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fa015db70 0x7f1fa0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00cf1c0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00ce530 con 0x7f1fa015db70 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa7409640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00cf340 con 0x7f1fa00b6cb0 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.466+0000 7f1fa51a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 0x7f1fa015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 0x7f1fa015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41914/0 (socket says 172.21.15.60:41914) 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 -- 172.21.15.60:0/3291149620 learned_addr learned my addr 172.21.15.60:0/3291149620 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa59a6640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fa015db70 0x7f1fa0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa49a4640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fa00b6cb0 0x7f1fa00c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 -- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fa00b6cb0 msgr2=0x7f1fa00c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fa00b6cb0 0x7f1fa00c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 -- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fa015db70 msgr2=0x7f1fa0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fa015db70 0x7f1fa0156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 -- 172.21.15.60:0/3291149620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0156960 con 0x7f1fa00b5a60 2024-04-25T08:36:56.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa49a4640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fa00b6cb0 0x7f1fa00c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa51a5640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 0x7f1fa015e1c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c3c0 tx=0x7f1f9c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c017070 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c005ce0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa0160780 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0072f0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.467+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa0160d40 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.468+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa015c8f0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.469+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f9c01a070 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.469+0000 7f1f95ffb640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f8406b4d0 0x7f1f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.469+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f9c0896c0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.469+0000 7f1fa49a4640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f8406b4d0 0x7f1f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:56.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.470+0000 7f1fa49a4640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f8406b4d0 0x7f1f8406d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1f90034000 tx=0x7f1f9002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:56.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.474+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fa015c8f0 con 0x7f1fa00b5a60 2024-04-25T08:36:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.742+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1fa00cf950 con 0x7f1fa00b5a60 2024-04-25T08:36:56.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.743+0000 7f1f95ffb640 1 -- 172.21.15.60:0/3291149620 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f1fa00cf950 con 0x7f1fa00b5a60 2024-04-25T08:36:56.737 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:56.737 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":477,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":796,"data_bytes":811214270,"bytes_used":1903509504,"bytes_avail":771190603776,"bytes_total":773094113280,"write_bytes_sec":67538882,"read_op_per_sec":0,"write_op_per_sec":65},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f8406b4d0 msgr2=0x7f1f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f8406b4d0 0x7f1f8406d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1f90034000 tx=0x7f1f9002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 msgr2=0x7f1fa015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 0x7f1fa015e1c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c3c0 tx=0x7f1f9c00c880 comp rx=0 tx=0).stop 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 shutdown_connections 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1f8406b4d0 0x7f1f8406d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fa015db70 0x7f1fa0156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1fa00b6cb0 0x7f1fa00c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 --2- 172.21.15.60:0/3291149620 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1fa00b5a60 0x7f1fa015e1c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 >> 172.21.15.60:0/3291149620 conn(0x7f1fa00d0520 msgr2=0x7f1fa015c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 shutdown_connections 2024-04-25T08:36:56.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:56.745+0000 7f1fa7409640 1 -- 172.21.15.60:0/3291149620 wait complete. 2024-04-25T08:36:56.750 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:36:57.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:57.261 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:57.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:57.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.870+0000 7f6a90334640 1 Processor -- start 2024-04-25T08:36:57.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.870+0000 7f6a90334640 1 -- start start 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a90334640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a8815db70 0x7f6a88162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a90334640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b5c30 0x7f6a880d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a90334640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a88162690 con 0x7f6a880b6cb0 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a90334640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880ceee0 con 0x7f6a8815db70 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a90334640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880d1d10 con 0x7f6a880b5c30 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a8e0d0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6a8815db70 0x7f6a88162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a8d8cf640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6a880b6cb0 0x7f6a880b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50672/0 (socket says 172.21.15.60:50672) 2024-04-25T08:36:57.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.871+0000 7f6a8d8cf640 1 -- 172.21.15.60:0/929634517 learned_addr learned my addr 172.21.15.60:0/929634517 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8e8d1640 1 --2- 172.21.15.60:0/929634517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b5c30 0x7f6a880d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8e0d0640 1 -- 172.21.15.60:0/929634517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b5c30 msgr2=0x7f6a880d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8e0d0640 1 --2- 172.21.15.60:0/929634517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b5c30 0x7f6a880d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8e0d0640 1 -- 172.21.15.60:0/929634517 >> v1:172.21.15.60:6789/0 conn(0x7f6a880b6cb0 legacy=0x7f6a880b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8e0d0640 1 -- 172.21.15.60:0/929634517 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880d1ff0 con 0x7f6a8815db70 2024-04-25T08:36:57.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8d0ce640 1 -- 172.21.15.60:0/929634517 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945094444 0 0) 0x7f6a88162690 con 0x7f6a880b6cb0 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.872+0000 7f6a8e0d0640 1 --2- 172.21.15.60:0/929634517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a88162150 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6a800097a0 tx=0x7f6a8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e661482660105a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a8d0ce640 1 -- 172.21.15.60:0/929634517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037070 con 0x7f6a8815db70 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a8d0ce640 1 -- 172.21.15.60:0/929634517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80032440 con 0x7f6a8815db70 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a8d0ce640 1 -- 172.21.15.60:0/929634517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80032810 con 0x7f6a8815db70 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 -- 172.21.15.60:0/929634517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 msgr2=0x7f6a88162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 --2- 172.21.15.60:0/929634517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a88162150 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6a800097a0 tx=0x7f6a8002a180 comp rx=0 tx=0).stop 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 -- 172.21.15.60:0/929634517 shutdown_connections 2024-04-25T08:36:57.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 --2- 172.21.15.60:0/929634517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b5c30 0x7f6a880d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:57.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 --2- 172.21.15.60:0/929634517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a88162150 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:57.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 -- 172.21.15.60:0/929634517 >> 172.21.15.60:0/929634517 conn(0x7f6a880d0520 msgr2=0x7f6a880d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:57.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 -- 172.21.15.60:0/929634517 shutdown_connections 2024-04-25T08:36:57.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.873+0000 7f6a90334640 1 -- 172.21.15.60:0/929634517 wait complete. 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.874+0000 7f6a90334640 1 Processor -- start 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 -- start start 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a880b5c30 0x7f6a881519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b6cb0 0x7f6a88159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a8815a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880ce530 con 0x7f6a8815db70 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8815df50 con 0x7f6a880b5c30 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a90334640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880cf1c0 con 0x7f6a880b6cb0 2024-04-25T08:36:57.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a8e8d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a8815a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a8e8d1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a8815a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41922/0 (socket says 172.21.15.60:41922) 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.875+0000 7f6a8e8d1640 1 -- 172.21.15.60:0/922882279 learned_addr learned my addr 172.21.15.60:0/922882279 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8d8cf640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b6cb0 0x7f6a88159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e0d0640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a880b5c30 0x7f6a881519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e8d1640 1 -- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b6cb0 msgr2=0x7f6a88159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e8d1640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b6cb0 0x7f6a88159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e8d1640 1 -- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a880b5c30 msgr2=0x7f6a881519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e8d1640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a880b5c30 0x7f6a881519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e8d1640 1 -- 172.21.15.60:0/922882279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8815ef20 con 0x7f6a8815db70 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a8e8d1640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a8815a070 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007de0 tx=0x7f6a8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84014620 con 0x7f6a8815db70 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a84019070 con 0x7f6a8815db70 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84014a20 con 0x7f6a8815db70 2024-04-25T08:36:57.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8815f200 con 0x7f6a8815db70 2024-04-25T08:36:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.876+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8815f7c0 con 0x7f6a8815db70 2024-04-25T08:36:57.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.877+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a88073990 con 0x7f6a8815db70 2024-04-25T08:36:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.878+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a84014bc0 con 0x7f6a8815db70 2024-04-25T08:36:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.878+0000 7f6a7effd640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a6c06b4d0 0x7f6a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.878+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a8408ac70 con 0x7f6a8815db70 2024-04-25T08:36:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.878+0000 7f6a8e0d0640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a6c06b4d0 0x7f6a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:57.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.879+0000 7f6a8e0d0640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a6c06b4d0 0x7f6a6c06d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a80034000 tx=0x7f6a8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:57.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:57.883+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a840599f0 con 0x7f6a8815db70 2024-04-25T08:36:58.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.101+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a880d1820 con 0x7f6a8815db70 2024-04-25T08:36:58.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.102+0000 7f6a7effd640 1 -- 172.21.15.60:0/922882279 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6a840592f0 con 0x7f6a8815db70 2024-04-25T08:36:58.096 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:58.096 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:36:58.096 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:36:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a6c06b4d0 msgr2=0x7f6a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a6c06b4d0 0x7f6a6c06d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a80034000 tx=0x7f6a8002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:58.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 msgr2=0x7f6a8815a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a8815a070 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6a84007de0 tx=0x7f6a8400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 shutdown_connections 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6a8815db70 0x7f6a8815a070 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6a880b6cb0 0x7f6a88159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6a6c06b4d0 0x7f6a6c06d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 --2- 172.21.15.60:0/922882279 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a880b5c30 0x7f6a881519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 >> 172.21.15.60:0/922882279 conn(0x7f6a880d0520 msgr2=0x7f6a880d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 shutdown_connections 2024-04-25T08:36:58.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.104+0000 7f6a90334640 1 -- 172.21.15.60:0/922882279 wait complete. 2024-04-25T08:36:58.109 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:36:58.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:58.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.214+0000 7f42edbe7640 1 Processor -- start 2024-04-25T08:36:58.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.214+0000 7f42edbe7640 1 -- start start 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42edbe7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 0x7f42e80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42edbe7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e80b92a0 0x7f42e80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42edbe7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e80b97a0 con 0x7f42e80b2cb0 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42edbe7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e815dfc0 con 0x7f42e815dbe0 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42edbe7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80d1a00 con 0x7f42e80b92a0 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42e77fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 0x7f42e80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e80b92a0 0x7f42e80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:58.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42e6ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f42e80b2cb0 0x7f42e80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:45436/0 (socket says 172.21.15.60:45436) 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42e6ffd640 1 -- 172.21.15.60:0/4093847489 learned_addr learned my addr 172.21.15.60:0/4093847489 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.215+0000 7f42e77fe640 1 --2- 172.21.15.60:0/4093847489 >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 0x7f42e80d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:36600/0 (socket says 172.21.15.60:36600) 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.216+0000 7f42e7fff640 1 -- 172.21.15.60:0/4093847489 >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 msgr2=0x7f42e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.216+0000 7f42e7fff640 1 --2- 172.21.15.60:0/4093847489 >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 0x7f42e80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.216+0000 7f42e7fff640 1 -- 172.21.15.60:0/4093847489 >> v1:172.21.15.100:6789/0 conn(0x7f42e80b2cb0 legacy=0x7f42e80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.216+0000 7f42e7fff640 1 -- 172.21.15.60:0/4093847489 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e80d1c50 con 0x7f42e80b92a0 2024-04-25T08:36:58.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.216+0000 7f42e77fe640 1 --2- 172.21.15.60:0/4093847489 >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 0x7f42e80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:58.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.216+0000 7f42e7fff640 1 --2- 172.21.15.60:0/4093847489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e80b92a0 0x7f42e80d0e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009d90 tx=0x7f42e002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a68f0fbd5a906c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42e67fc640 1 -- 172.21.15.60:0/4093847489 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0007740 con 0x7f42e80b92a0 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42e67fc640 1 -- 172.21.15.60:0/4093847489 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e00078e0 con 0x7f42e80b92a0 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42e67fc640 1 -- 172.21.15.60:0/4093847489 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0007cb0 con 0x7f42e80b92a0 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 -- 172.21.15.60:0/4093847489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e80b92a0 msgr2=0x7f42e80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 --2- 172.21.15.60:0/4093847489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e80b92a0 0x7f42e80d0e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009d90 tx=0x7f42e002a660 comp rx=0 tx=0).stop 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 -- 172.21.15.60:0/4093847489 shutdown_connections 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 --2- 172.21.15.60:0/4093847489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e80b92a0 0x7f42e80d0e50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 --2- 172.21.15.60:0/4093847489 >> v2:172.21.15.100:3300/0 conn(0x7f42e815dbe0 0x7f42e80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 -- 172.21.15.60:0/4093847489 >> 172.21.15.60:0/4093847489 conn(0x7f42e8061a20 msgr2=0x7f42e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 -- 172.21.15.60:0/4093847489 shutdown_connections 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.217+0000 7f42edbe7640 1 -- 172.21.15.60:0/4093847489 wait complete. 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.218+0000 7f42edbe7640 1 Processor -- start 2024-04-25T08:36:58.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 -- start start 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e80b2cb0 0x7f42e814ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 0x7f42e815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e815dbe0 0x7f42e80b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80b6ab0 con 0x7f42e80b92a0 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80d2210 con 0x7f42e80b2cb0 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42edbe7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e80b6c30 con 0x7f42e815dbe0 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.219+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 0x7f42e815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 0x7f42e815fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41930/0 (socket says 172.21.15.60:41930) 2024-04-25T08:36:58.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 -- 172.21.15.60:0/1832850658 learned_addr learned my addr 172.21.15.60:0/1832850658 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e7fff640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e815dbe0 0x7f42e80b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e77fe640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e80b2cb0 0x7f42e814ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 -- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e815dbe0 msgr2=0x7f42e80b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e815dbe0 0x7f42e80b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 -- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e80b2cb0 msgr2=0x7f42e814ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e80b2cb0 0x7f42e814ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 -- 172.21.15.60:0/1832850658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e814d360 con 0x7f42e80b92a0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42e6ffd640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 0x7f42e815fbd0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c440 tx=0x7f42dc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc017070 con 0x7f42e80b92a0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc005ce0 con 0x7f42e80b92a0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0072f0 con 0x7f42e80b92a0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e80cba60 con 0x7f42e80b92a0 2024-04-25T08:36:58.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.220+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e81609c0 con 0x7f42e80b92a0 2024-04-25T08:36:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.221+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e8073990 con 0x7f42e80b92a0 2024-04-25T08:36:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.222+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42dc01a070 con 0x7f42e80b92a0 2024-04-25T08:36:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.222+0000 7f42cffff640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f42d406b4d0 0x7f42d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.222+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42dc089790 con 0x7f42e80b92a0 2024-04-25T08:36:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.223+0000 7f42e77fe640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f42d406b4d0 0x7f42d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:58.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.223+0000 7f42e77fe640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f42d406b4d0 0x7f42d406d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f42e814d5f0 tx=0x7f42d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:58.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.227+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42e814d360 con 0x7f42e80b92a0 2024-04-25T08:36:58.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.479+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f42e8154bb0 con 0x7f42e80b92a0 2024-04-25T08:36:58.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.479+0000 7f42cffff640 1 -- 172.21.15.60:0/1832850658 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f42dc058470 con 0x7f42e80b92a0 2024-04-25T08:36:58.474 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:58.474 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":479,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2504073216,"bytes_avail":770590040064,"bytes_total":773094113280,"write_bytes_sec":89651605,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f42d406b4d0 msgr2=0x7f42d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f42d406b4d0 0x7f42d406d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f42e814d5f0 tx=0x7f42d8008040 comp rx=0 tx=0).stop 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 msgr2=0x7f42e815fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 0x7f42e815fbd0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c440 tx=0x7f42dc001e70 comp rx=0 tx=0).stop 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 shutdown_connections 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42e815dbe0 0x7f42e80b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42e80b92a0 0x7f42e815fbd0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f42d406b4d0 0x7f42d406d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 --2- 172.21.15.60:0/1832850658 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42e80b2cb0 0x7f42e814ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 >> 172.21.15.60:0/1832850658 conn(0x7f42e8061a20 msgr2=0x7f42e815ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.481+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 shutdown_connections 2024-04-25T08:36:58.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:58.482+0000 7f42edbe7640 1 -- 172.21.15.60:0/1832850658 wait complete. 2024-04-25T08:36:58.487 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:36:59.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:59.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.608+0000 7faa91de0640 1 Processor -- start 2024-04-25T08:36:59.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.608+0000 7faa91de0640 1 -- start start 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa91de0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa8c154ee0 0x7faa8c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa91de0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c071480 con 0x7faa8c0ce590 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa91de0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c071760 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa91de0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c072090 con 0x7faa8c154ee0 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa8bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faa8c0ce590 0x7faa8c06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50684/0 (socket says 172.21.15.60:50684) 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa8bfff640 1 -- 172.21.15.60:0/2214832422 learned_addr learned my addr 172.21.15.60:0/2214832422 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa8b7fe640 1 --2- 172.21.15.60:0/2214832422 >> v2:172.21.15.100:3300/0 conn(0x7faa8c154ee0 0x7faa8c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa8a7fc640 1 -- 172.21.15.60:0/2214832422 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178713332 0 0) 0x7faa8c071480 con 0x7faa8c0ce590 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.609+0000 7faa8a7fc640 1 -- 172.21.15.60:0/2214832422 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa74003520 con 0x7faa8c0ce590 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.610+0000 7faa8a7fc640 1 -- 172.21.15.60:0/2214832422 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555854279 0 0) 0x7faa74003520 con 0x7faa8c0ce590 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.610+0000 7faa8b7fe640 1 -- 172.21.15.60:0/2214832422 >> v1:172.21.15.100:6789/0 conn(0x7faa8c0cfcf0 legacy=0x7faa8c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.610+0000 7faa8b7fe640 1 -- 172.21.15.60:0/2214832422 >> v1:172.21.15.60:6789/0 conn(0x7faa8c0ce590 legacy=0x7faa8c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.610+0000 7faa8b7fe640 1 -- 172.21.15.60:0/2214832422 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c071a40 con 0x7faa8c154ee0 2024-04-25T08:36:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.610+0000 7faa8b7fe640 1 --2- 172.21.15.60:0/2214832422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0cf7b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48c4469f790cb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa8a7fc640 1 -- 172.21.15.60:0/2214832422 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c037070 con 0x7faa8c154ee0 2024-04-25T08:36:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa8a7fc640 1 -- 172.21.15.60:0/2214832422 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c02ac50 con 0x7faa8c154ee0 2024-04-25T08:36:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa8a7fc640 1 -- 172.21.15.60:0/2214832422 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa840019f0 con 0x7faa8c154ee0 2024-04-25T08:36:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 -- 172.21.15.60:0/2214832422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 msgr2=0x7faa8c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 --2- 172.21.15.60:0/2214832422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0cf7b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faa7c0097a0 tx=0x7faa7c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 -- 172.21.15.60:0/2214832422 shutdown_connections 2024-04-25T08:36:59.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 --2- 172.21.15.60:0/2214832422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0cf7b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 -- 172.21.15.60:0/2214832422 >> 172.21.15.60:0/2214832422 conn(0x7faa8c06abe0 msgr2=0x7faa8c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:59.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 -- 172.21.15.60:0/2214832422 shutdown_connections 2024-04-25T08:36:59.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.611+0000 7faa91de0640 1 -- 172.21.15.60:0/2214832422 wait complete. 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.612+0000 7faa91de0640 1 Processor -- start 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 -- start start 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8c0ce590 0x7faa8c1f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 0x7faa8c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1552c0 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c071230 con 0x7faa8c154ee0 2024-04-25T08:36:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.613+0000 7faa91de0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c155440 con 0x7faa8c0ce590 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 0x7faa8c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 0x7faa8c14b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41946/0 (socket says 172.21.15.60:41946) 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 -- 172.21.15.60:0/1072101074 learned_addr learned my addr 172.21.15.60:0/1072101074 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8bfff640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8b7fe640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8c0ce590 0x7faa8c1f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 -- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8c0ce590 msgr2=0x7faa8c1f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8c0ce590 0x7faa8c1f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 -- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 msgr2=0x7faa8c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 -- 172.21.15.60:0/1072101074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c1f7ac0 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8b7fe640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8c0ce590 0x7faa8c1f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa8affd640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 0x7faa8c14b1e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c480 tx=0x7faa8000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:59.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa800052d0 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80005470 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c0afdc0 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80012870 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.614+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c0b0380 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.616+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c073990 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.616+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa80012a10 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.616+0000 7faa73fff640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.617+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa80089ee0 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.617+0000 7faa8b7fe640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.617+0000 7faa8b7fe640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa7806b4d0 0x7faa7806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faa7c034000 tx=0x7faa7c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:59.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.621+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa8c1f7ac0 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.841+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa8c06d360 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.842+0000 7faa73fff640 1 -- 172.21.15.60:0/1072101074 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7faa80058c60 con 0x7faa8c0cfcf0 2024-04-25T08:36:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:36:59.837 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:36:59.837 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:36:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa7806b4d0 msgr2=0x7faa7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa7806b4d0 0x7faa7806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faa7c034000 tx=0x7faa7c02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:59.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 msgr2=0x7faa8c14b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 0x7faa8c14b1e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c480 tx=0x7faa8000c940 comp rx=0 tx=0).stop 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 shutdown_connections 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa8c154ee0 0x7faa8c0b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa8c0cfcf0 0x7faa8c14b1e0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 --2- 172.21.15.60:0/1072101074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faa8c0ce590 0x7faa8c1f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 >> 172.21.15.60:0/1072101074 conn(0x7faa8c06abe0 msgr2=0x7faa8c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 shutdown_connections 2024-04-25T08:36:59.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.844+0000 7faa91de0640 1 -- 172.21.15.60:0/1072101074 wait complete. 2024-04-25T08:36:59.849 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:36:59.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.955+0000 7f58daf4c640 1 Processor -- start 2024-04-25T08:36:59.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.955+0000 7f58daf4c640 1 -- start start 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58daf4c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d415df00 0x7f58d40cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58daf4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40701e0 0x7f58d40cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58daf4c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d406f6c0 con 0x7f58d406e410 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58daf4c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d406f950 con 0x7f58d415df00 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58daf4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40cdf00 con 0x7f58d40701e0 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58d8ce8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f58d415df00 0x7f58d40cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58d94e9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f58d406e410 0x7f58d406e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50690/0 (socket says 172.21.15.60:50690) 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.956+0000 7f58d94e9640 1 -- 172.21.15.60:0/1527069848 learned_addr learned my addr 172.21.15.60:0/1527069848 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d3fff640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40701e0 0x7f58d40cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d8ce8640 1 -- 172.21.15.60:0/1527069848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40701e0 msgr2=0x7f58d40cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d8ce8640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40701e0 0x7f58d40cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d8ce8640 1 -- 172.21.15.60:0/1527069848 >> v1:172.21.15.60:6789/0 conn(0x7f58d406e410 legacy=0x7f58d406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d8ce8640 1 -- 172.21.15.60:0/1527069848 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40cece0 con 0x7f58d415df00 2024-04-25T08:36:59.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d3fff640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40701e0 0x7f58d40cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:59.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d8ce8640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415df00 0x7f58d40cd110 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009d90 tx=0x7f58cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f52ee295dda730e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:59.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.957+0000 7f58d37fe640 1 -- 172.21.15.60:0/1527069848 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc007740 con 0x7f58d415df00 2024-04-25T08:36:59.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58d37fe640 1 -- 172.21.15.60:0/1527069848 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc0078e0 con 0x7f58d415df00 2024-04-25T08:36:59.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58d37fe640 1 -- 172.21.15.60:0/1527069848 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc007cb0 con 0x7f58d415df00 2024-04-25T08:36:59.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 -- 172.21.15.60:0/1527069848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415df00 msgr2=0x7f58d40cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415df00 0x7f58d40cd110 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009d90 tx=0x7f58cc02a660 comp rx=0 tx=0).stop 2024-04-25T08:36:59.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 -- 172.21.15.60:0/1527069848 shutdown_connections 2024-04-25T08:36:59.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d40701e0 0x7f58d40cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1527069848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d415df00 0x7f58d40cd110 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 -- 172.21.15.60:0/1527069848 >> 172.21.15.60:0/1527069848 conn(0x7f58d40b2290 msgr2=0x7f58d40b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 -- 172.21.15.60:0/1527069848 shutdown_connections 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.958+0000 7f58daf4c640 1 -- 172.21.15.60:0/1527069848 wait complete. 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.959+0000 7f58daf4c640 1 Processor -- start 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.959+0000 7f58daf4c640 1 -- start start 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58daf4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d406e410 0x7f58d41f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58daf4c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 0x7f58d406ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58daf4c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415df00 0x7f58d41f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58daf4c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40cdb90 con 0x7f58d40701e0 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58daf4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40cf4e0 con 0x7f58d406e410 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58daf4c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40ccc00 con 0x7f58d415df00 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 0x7f58d406ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 0x7f58d406ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41964/0 (socket says 172.21.15.60:41964) 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.960+0000 7f58d3fff640 1 -- 172.21.15.60:0/1480671144 learned_addr learned my addr 172.21.15.60:0/1480671144 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d94e9640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415df00 0x7f58d41f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d8ce8640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d406e410 0x7f58d41f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d3fff640 1 -- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415df00 msgr2=0x7f58d41f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d3fff640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415df00 0x7f58d41f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d3fff640 1 -- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d406e410 msgr2=0x7f58d41f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d3fff640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d406e410 0x7f58d41f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d3fff640 1 -- 172.21.15.60:0/1480671144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d41f7890 con 0x7f58d40701e0 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d8ce8640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d406e410 0x7f58d41f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d3fff640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 0x7f58d406ae50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c7f0 tx=0x7f58c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8015070 con 0x7f58d40701e0 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8007a30 con 0x7f58d40701e0 2024-04-25T08:36:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d41f7b70 con 0x7f58d40701e0 2024-04-25T08:36:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8007e00 con 0x7f58d40701e0 2024-04-25T08:36:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.961+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d41f3db0 con 0x7f58d40701e0 2024-04-25T08:36:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.962+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d4073990 con 0x7f58d40701e0 2024-04-25T08:36:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.963+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58c8018070 con 0x7f58d40701e0 2024-04-25T08:36:59.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.963+0000 7f58d17fa640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f58c006b4d0 0x7f58c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.963+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58c808a310 con 0x7f58d40701e0 2024-04-25T08:36:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.963+0000 7f58d8ce8640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f58c006b4d0 0x7f58c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:59.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.964+0000 7f58d8ce8640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f58c006b4d0 0x7f58c006d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f58cc034000 tx=0x7f58cc033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:59.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:36:59.967+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58c8059000 con 0x7f58d40701e0 2024-04-25T08:37:00.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.220+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f58d41f38c0 con 0x7f58d40701e0 2024-04-25T08:37:00.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.220+0000 7f58d17fa640 1 -- 172.21.15.60:0/1480671144 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f58c8058900 con 0x7f58d40701e0 2024-04-25T08:37:00.215 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:00.215 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":480,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2504073216,"bytes_avail":770590040064,"bytes_total":773094113280,"write_bytes_sec":88166740,"read_op_per_sec":0,"write_op_per_sec":85},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f58c006b4d0 msgr2=0x7f58c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f58c006b4d0 0x7f58c006d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f58cc034000 tx=0x7f58cc033040 comp rx=0 tx=0).stop 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 msgr2=0x7f58d406ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 0x7f58d406ae50 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f58c800c7f0 tx=0x7f58c800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 shutdown_connections 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58d415df00 0x7f58d41f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58d40701e0 0x7f58d406ae50 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f58c006b4d0 0x7f58c006d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 --2- 172.21.15.60:0/1480671144 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58d406e410 0x7f58d41f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 >> 172.21.15.60:0/1480671144 conn(0x7f58d40b2290 msgr2=0x7f58d40cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 shutdown_connections 2024-04-25T08:37:00.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:00.222+0000 7f58daf4c640 1 -- 172.21.15.60:0/1480671144 wait complete. 2024-04-25T08:37:00.227 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:01.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.349+0000 7fa58161f640 1 Processor -- start 2024-04-25T08:37:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.349+0000 7fa58161f640 1 -- start start 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa58161f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa57c15dbe0 0x7fa57c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa58161f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa58161f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c0b97a0 con 0x7fa57c0b92a0 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa58161f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57c15dfc0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa58161f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0d1a00 con 0x7fa57c0b2cb0 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa57c15dbe0 0x7fa57c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa57c15dbe0 0x7fa57c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41966/0 (socket says 172.21.15.60:41966) 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 -- 172.21.15.60:0/4209802327 learned_addr learned my addr 172.21.15.60:0/4209802327 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57a7fc640 1 --2- 172.21.15.60:0/4209802327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 -- 172.21.15.60:0/4209802327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 msgr2=0x7fa57c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 --2- 172.21.15.60:0/4209802327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 -- 172.21.15.60:0/4209802327 >> v1:172.21.15.100:6789/0 conn(0x7fa57c0b92a0 legacy=0x7fa57c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa579ffb640 1 -- 172.21.15.60:0/4209802327 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000840097 0 0) 0x7fa57c0b97a0 con 0x7fa57c0b92a0 2024-04-25T08:37:01.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.350+0000 7fa57affd640 1 -- 172.21.15.60:0/4209802327 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c0d1c50 con 0x7fa57c15dbe0 2024-04-25T08:37:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.351+0000 7fa57affd640 1 --2- 172.21.15.60:0/4209802327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0d0910 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009870 tx=0x7fa56c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f4c3d0a4f70123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:01.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.351+0000 7fa579ffb640 1 -- 172.21.15.60:0/4209802327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c037070 con 0x7fa57c15dbe0 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.351+0000 7fa579ffb640 1 -- 172.21.15.60:0/4209802327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c032440 con 0x7fa57c15dbe0 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.351+0000 7fa579ffb640 1 -- 172.21.15.60:0/4209802327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c032810 con 0x7fa57c15dbe0 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 -- 172.21.15.60:0/4209802327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 msgr2=0x7fa57c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 --2- 172.21.15.60:0/4209802327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0d0910 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009870 tx=0x7fa56c02a270 comp rx=0 tx=0).stop 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 -- 172.21.15.60:0/4209802327 shutdown_connections 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 --2- 172.21.15.60:0/4209802327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 --2- 172.21.15.60:0/4209802327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0d0910 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 -- 172.21.15.60:0/4209802327 >> 172.21.15.60:0/4209802327 conn(0x7fa57c061a20 msgr2=0x7fa57c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 -- 172.21.15.60:0/4209802327 shutdown_connections 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.352+0000 7fa58161f640 1 -- 172.21.15.60:0/4209802327 wait complete. 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.353+0000 7fa58161f640 1 Processor -- start 2024-04-25T08:37:01.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.353+0000 7fa58161f640 1 -- start start 2024-04-25T08:37:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa58161f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa58161f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa57c0b92a0 0x7fa57c0d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa58161f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa58161f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0b6ab0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa58161f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0d2210 con 0x7fa57c0b92a0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa58161f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c0b6c30 con 0x7fa57c0b2cb0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa57b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa57b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41980/0 (socket says 172.21.15.60:41980) 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa57b7fe640 1 -- 172.21.15.60:0/3442355659 learned_addr learned my addr 172.21.15.60:0/3442355659 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa57affd640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.354+0000 7fa57a7fc640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa57c0b92a0 0x7fa57c0d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa57b7fe640 1 -- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 msgr2=0x7fa57c0b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa57b7fe640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa57b7fe640 1 -- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa57c0b92a0 msgr2=0x7fa57c0d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa57b7fe640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa57c0b92a0 0x7fa57c0d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa57b7fe640 1 -- 172.21.15.60:0/3442355659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c1f3fa0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa57b7fe640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0ccf30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c730 tx=0x7fa57400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:01.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574003da0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57400ce40 con 0x7fa57c15dbe0 2024-04-25T08:37:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa574005270 con 0x7fa57c15dbe0 2024-04-25T08:37:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c0b44b0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.355+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c0b4a70 con 0x7fa57c15dbe0 2024-04-25T08:37:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.356+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c073990 con 0x7fa57c15dbe0 2024-04-25T08:37:01.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.357+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa574005410 con 0x7fa57c15dbe0 2024-04-25T08:37:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.357+0000 7fa5737fe640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa55c06b4d0 0x7fa55c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.357+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa57408a0c0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.357+0000 7fa57affd640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa55c06b4d0 0x7fa55c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.358+0000 7fa57affd640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa55c06b4d0 0x7fa55c06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa56c034000 tx=0x7fa56c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:01.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.362+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5740587f0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.581+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa57c15d0c0 con 0x7fa57c15dbe0 2024-04-25T08:37:01.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.581+0000 7fa5737fe640 1 -- 172.21.15.60:0/3442355659 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa57405c890 con 0x7fa57c15dbe0 2024-04-25T08:37:01.576 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:01.576 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:01.576 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa55c06b4d0 msgr2=0x7fa55c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa55c06b4d0 0x7fa55c06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa56c034000 tx=0x7fa56c02e000 comp rx=0 tx=0).stop 2024-04-25T08:37:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 msgr2=0x7fa57c0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0ccf30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa57400c730 tx=0x7fa57400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 shutdown_connections 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa57c15dbe0 0x7fa57c0ccf30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa55c06b4d0 0x7fa55c06d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa57c0b92a0 0x7fa57c0d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 --2- 172.21.15.60:0/3442355659 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa57c0b2cb0 0x7fa57c0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 >> 172.21.15.60:0/3442355659 conn(0x7fa57c061a20 msgr2=0x7fa57c160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 shutdown_connections 2024-04-25T08:37:01.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.583+0000 7fa58161f640 1 -- 172.21.15.60:0/3442355659 wait complete. 2024-04-25T08:37:01.589 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:01.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.695+0000 7f00a4ef5640 1 Processor -- start 2024-04-25T08:37:01.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.695+0000 7f00a4ef5640 1 -- start start 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f00a4ef5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a015dc90 0x7f00a01625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f00a4ef5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00a00cc8e0 0x7f00a015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f00a4ef5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a015f390 con 0x7f00a00d1810 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f00a4ef5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a00cce50 con 0x7f00a00cc8e0 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f00a4ef5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a0161af0 con 0x7f00a015dc90 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f0097fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00a00cc8e0 0x7f00a015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f0097fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f00a00cc8e0 0x7f00a015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36976/0 (socket says 172.21.15.60:36976) 2024-04-25T08:37:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f0097fff640 1 -- 172.21.15.60:0/928793447 learned_addr learned my addr 172.21.15.60:0/928793447 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.696+0000 7f009e59c640 1 --2- 172.21.15.60:0/928793447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a015dc90 0x7f00a01625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f0097fff640 1 -- 172.21.15.60:0/928793447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a015dc90 msgr2=0x7f00a01625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f0097fff640 1 --2- 172.21.15.60:0/928793447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a015dc90 0x7f00a01625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f0097fff640 1 -- 172.21.15.60:0/928793447 >> v1:172.21.15.100:6789/0 conn(0x7f00a00d1810 legacy=0x7f00a00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f0097fff640 1 -- 172.21.15.60:0/928793447 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0161dd0 con 0x7f00a00cc8e0 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f0097fff640 1 --2- 172.21.15.60:0/928793447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a00cc8e0 0x7f00a015ee50 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0088009870 tx=0x7f008802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf93b38c95b63fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f009dd9b640 1 -- 172.21.15.60:0/928793447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088037070 con 0x7f00a00cc8e0 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f009dd9b640 1 -- 172.21.15.60:0/928793447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088032440 con 0x7f00a00cc8e0 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.697+0000 7f009dd9b640 1 -- 172.21.15.60:0/928793447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088032810 con 0x7f00a00cc8e0 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 -- 172.21.15.60:0/928793447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a00cc8e0 msgr2=0x7f00a015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/928793447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a00cc8e0 0x7f00a015ee50 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0088009870 tx=0x7f008802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 -- 172.21.15.60:0/928793447 shutdown_connections 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/928793447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a00cc8e0 0x7f00a015ee50 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/928793447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a015dc90 0x7f00a01625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 -- 172.21.15.60:0/928793447 >> 172.21.15.60:0/928793447 conn(0x7f00a006f590 msgr2=0x7f00a006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 -- 172.21.15.60:0/928793447 shutdown_connections 2024-04-25T08:37:01.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.698+0000 7f00a4ef5640 1 -- 172.21.15.60:0/928793447 wait complete. 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.699+0000 7f00a4ef5640 1 Processor -- start 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.699+0000 7f00a4ef5640 1 -- start start 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f00a4ef5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00a00cc8e0 0x7f00a00cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f00a4ef5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a00d1810 0x7f00a00b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f00a4ef5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 0x7f00a0151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f00a4ef5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a015e070 con 0x7f00a015dc90 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f00a4ef5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00cd130 con 0x7f00a00cc8e0 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f00a4ef5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a015e1f0 con 0x7f00a00d1810 2024-04-25T08:37:01.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f009ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 0x7f00a0151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f009ed9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 0x7f00a0151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36982/0 (socket says 172.21.15.60:36982) 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f009ed9d640 1 -- 172.21.15.60:0/4236174817 learned_addr learned my addr 172.21.15.60:0/4236174817 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f009e59c640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00a00cc8e0 0x7f00a00cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.700+0000 7f0097fff640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a00d1810 0x7f00a00b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f009ed9d640 1 -- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a00d1810 msgr2=0x7f00a00b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f009ed9d640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a00d1810 0x7f00a00b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f009ed9d640 1 -- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00a00cc8e0 msgr2=0x7f00a00cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f009ed9d640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00a00cc8e0 0x7f00a00cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f009ed9d640 1 -- 172.21.15.60:0/4236174817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0151c80 con 0x7f00a015dc90 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f0097fff640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a00d1810 0x7f00a00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f009ed9d640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 0x7f00a0151740 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f009800c760 tx=0x7f009800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098003b70 con 0x7f00a015dc90 2024-04-25T08:37:01.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098003d30 con 0x7f00a015dc90 2024-04-25T08:37:01.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a00b5620 con 0x7f00a015dc90 2024-04-25T08:37:01.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800aa90 con 0x7f00a015dc90 2024-04-25T08:37:01.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.701+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a00c8ba0 con 0x7f00a015dc90 2024-04-25T08:37:01.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.702+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a0073990 con 0x7f00a015dc90 2024-04-25T08:37:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.702+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f009800ac30 con 0x7f00a015dc90 2024-04-25T08:37:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.703+0000 7f0096ffd640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f008006b500 0x7f008006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.703+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f009808a110 con 0x7f00a015dc90 2024-04-25T08:37:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.703+0000 7f009e59c640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f008006b500 0x7f008006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.704+0000 7f009e59c640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f008006b500 0x7f008006d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0090009b40 tx=0x7f0090009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:01.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.707+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00a015f390 con 0x7f00a015dc90 2024-04-25T08:37:01.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.965+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f00a00cf6b0 con 0x7f00a015dc90 2024-04-25T08:37:01.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.965+0000 7f0096ffd640 1 -- 172.21.15.60:0/4236174817 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f0098058e90 con 0x7f00a015dc90 2024-04-25T08:37:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:01.960 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":482,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280,"write_bytes_sec":88169719,"read_op_per_sec":0,"write_op_per_sec":86},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f008006b500 msgr2=0x7f008006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f008006b500 0x7f008006d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0090009b40 tx=0x7f0090009190 comp rx=0 tx=0).stop 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 msgr2=0x7f00a0151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 0x7f00a0151740 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f009800c760 tx=0x7f009800cc20 comp rx=0 tx=0).stop 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 shutdown_connections 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f00a015dc90 0x7f00a0151740 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f00a00d1810 0x7f00a00b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f008006b500 0x7f008006d9c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 --2- 172.21.15.60:0/4236174817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00a00cc8e0 0x7f00a00cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 >> 172.21.15.60:0/4236174817 conn(0x7f00a006f590 msgr2=0x7f00a006f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 shutdown_connections 2024-04-25T08:37:01.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:01.967+0000 7f00a4ef5640 1 -- 172.21.15.60:0/4236174817 wait complete. 2024-04-25T08:37:01.973 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:02.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 Processor -- start 2024-04-25T08:37:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 -- start start 2024-04-25T08:37:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd6013d190 0x7fcd600cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd600cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd60071ed0 con 0x7fcd6015b320 2024-04-25T08:37:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6013d570 con 0x7fcd6013d190 2024-04-25T08:37:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.094+0000 7fcd649b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6015a840 con 0x7fcd60071940 2024-04-25T08:37:03.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5e59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcd6013d190 0x7fcd600cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd57fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcd6015b320 0x7fcd60071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34616/0 (socket says 172.21.15.60:34616) 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd57fff640 1 -- 172.21.15.60:0/521005086 learned_addr learned my addr 172.21.15.60:0/521005086 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5ed9d640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd600cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5e59c640 1 -- 172.21.15.60:0/521005086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 msgr2=0x7fcd600cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5e59c640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd600cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5e59c640 1 -- 172.21.15.60:0/521005086 >> v1:172.21.15.60:6789/0 conn(0x7fcd6015b320 legacy=0x7fcd60071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5dd9b640 1 -- 172.21.15.60:0/521005086 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890226509 0 0) 0x7fcd60071ed0 con 0x7fcd6015b320 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5e59c640 1 -- 172.21.15.60:0/521005086 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6015ab20 con 0x7fcd6013d190 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.095+0000 7fcd5ed9d640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd600cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.096+0000 7fcd5e59c640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd600cbfb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097a0 tx=0x7fcd5002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aaa7c86b30d5db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.096+0000 7fcd5dd9b640 1 -- 172.21.15.60:0/521005086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50037070 con 0x7fcd6013d190 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.096+0000 7fcd5dd9b640 1 -- 172.21.15.60:0/521005086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5002ae50 con 0x7fcd6013d190 2024-04-25T08:37:03.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.096+0000 7fcd5dd9b640 1 -- 172.21.15.60:0/521005086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50032660 con 0x7fcd6013d190 2024-04-25T08:37:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.096+0000 7fcd649b0640 1 -- 172.21.15.60:0/521005086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 msgr2=0x7fcd600cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.096+0000 7fcd649b0640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd600cbfb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097a0 tx=0x7fcd5002a100 comp rx=0 tx=0).stop 2024-04-25T08:37:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.097+0000 7fcd649b0640 1 -- 172.21.15.60:0/521005086 shutdown_connections 2024-04-25T08:37:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.097+0000 7fcd649b0640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd600cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.097+0000 7fcd649b0640 1 --2- 172.21.15.60:0/521005086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd600cbfb0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.097+0000 7fcd649b0640 1 -- 172.21.15.60:0/521005086 >> 172.21.15.60:0/521005086 conn(0x7fcd6006abe0 msgr2=0x7fcd6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.097+0000 7fcd649b0640 1 -- 172.21.15.60:0/521005086 shutdown_connections 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.097+0000 7fcd649b0640 1 -- 172.21.15.60:0/521005086 wait complete. 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.098+0000 7fcd649b0640 1 Processor -- start 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.098+0000 7fcd649b0640 1 -- start start 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd649b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd60139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd649b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd6013b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd649b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6015b320 0x7fcd600cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd649b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600cf7a0 con 0x7fcd6013d190 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd649b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600cf620 con 0x7fcd6015b320 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd649b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600cf920 con 0x7fcd60071940 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd6013b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd6013b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36986/0 (socket says 172.21.15.60:36986) 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 -- 172.21.15.60:0/2826577625 learned_addr learned my addr 172.21.15.60:0/2826577625 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:03.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd5e59c640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd60139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd5ed9d640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6015b320 0x7fcd600cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 -- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 msgr2=0x7fcd60139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd60139d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 -- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6015b320 msgr2=0x7fcd600cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6015b320 0x7fcd600cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 -- 172.21.15.60:0/2826577625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6013c750 con 0x7fcd6013d190 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd5ed9d640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6015b320 0x7fcd600cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.099+0000 7fcd57fff640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd6013b840 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800c430 tx=0x7fcd4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.100+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480074e0 con 0x7fcd6013d190 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.100+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd480047e0 con 0x7fcd6013d190 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.100+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd601f2ff0 con 0x7fcd6013d190 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.100+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004bd0 con 0x7fcd6013d190 2024-04-25T08:37:03.096 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.100+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd601f35b0 con 0x7fcd6013d190 2024-04-25T08:37:03.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.101+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd60073990 con 0x7fcd6013d190 2024-04-25T08:37:03.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.101+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd48011070 con 0x7fcd6013d190 2024-04-25T08:37:03.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.102+0000 7fcd56ffd640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd4006b4d0 0x7fcd4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.102+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd4808a000 con 0x7fcd6013d190 2024-04-25T08:37:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.102+0000 7fcd5e59c640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd4006b4d0 0x7fcd4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.098 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.103+0000 7fcd5e59c640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd4006b4d0 0x7fcd4006d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd50034000 tx=0x7fcd5002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:03.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.106+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd6013c750 con 0x7fcd6013d190 2024-04-25T08:37:03.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.326+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd6006e800 con 0x7fcd6013d190 2024-04-25T08:37:03.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.327+0000 7fcd56ffd640 1 -- 172.21.15.60:0/2826577625 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fcd48058cf0 con 0x7fcd6013d190 2024-04-25T08:37:03.321 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:03.321 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:03.322 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:03.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd4006b4d0 msgr2=0x7fcd4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd4006b4d0 0x7fcd4006d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd50034000 tx=0x7fcd5002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 msgr2=0x7fcd6013b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd6013b840 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800c430 tx=0x7fcd4800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 shutdown_connections 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcd4006b4d0 0x7fcd4006d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd6015b320 0x7fcd600cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcd6013d190 0x7fcd6013b840 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 --2- 172.21.15.60:0/2826577625 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd60071940 0x7fcd60139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 >> 172.21.15.60:0/2826577625 conn(0x7fcd6006abe0 msgr2=0x7fcd6006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 shutdown_connections 2024-04-25T08:37:03.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.329+0000 7fcd649b0640 1 -- 172.21.15.60:0/2826577625 wait complete. 2024-04-25T08:37:03.334 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:03.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:03.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.440+0000 7f661f13b640 1 Processor -- start 2024-04-25T08:37:03.435 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.440+0000 7f661f13b640 1 -- start start 2024-04-25T08:37:03.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661f13b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66180b96d0 0x7f66180d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661f13b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66180d0fe0 0x7f66180d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661f13b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815e810 con 0x7f661815dd20 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661f13b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66180d1c00 con 0x7f66180d0fe0 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661f13b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66180d1420 con 0x7f66180b96d0 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661d6d8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f66180d0fe0 0x7f66180d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661ced7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f661815dd20 0x7f661815e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34626/0 (socket says 172.21.15.60:34626) 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661ced7640 1 -- 172.21.15.60:0/2559811726 learned_addr learned my addr 172.21.15.60:0/2559811726 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f6617fff640 1 --2- 172.21.15.60:0/2559811726 >> v2:172.21.15.100:3300/0 conn(0x7f66180b96d0 0x7f66180d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f66177fe640 1 -- 172.21.15.60:0/2559811726 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813688125 0 0) 0x7f661815e810 con 0x7f661815dd20 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f66177fe640 1 -- 172.21.15.60:0/2559811726 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6600003520 con 0x7f661815dd20 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661d6d8640 1 -- 172.21.15.60:0/2559811726 >> v2:172.21.15.100:3300/0 conn(0x7f66180b96d0 msgr2=0x7f66180d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661d6d8640 1 --2- 172.21.15.60:0/2559811726 >> v2:172.21.15.100:3300/0 conn(0x7f66180b96d0 0x7f66180d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661d6d8640 1 -- 172.21.15.60:0/2559811726 >> v1:172.21.15.60:6789/0 conn(0x7f661815dd20 legacy=0x7f661815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.441+0000 7f661d6d8640 1 -- 172.21.15.60:0/2559811726 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661815eac0 con 0x7f66180d0fe0 2024-04-25T08:37:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.442+0000 7f661d6d8640 1 --2- 172.21.15.60:0/2559811726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180d0fe0 0x7f66180d2370 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f66180b5d20 tx=0x7f661002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9175380107c01c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.442+0000 7f66177fe640 1 -- 172.21.15.60:0/2559811726 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610007740 con 0x7f66180d0fe0 2024-04-25T08:37:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.442+0000 7f66177fe640 1 -- 172.21.15.60:0/2559811726 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66100078e0 con 0x7f66180d0fe0 2024-04-25T08:37:03.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.442+0000 7f66177fe640 1 -- 172.21.15.60:0/2559811726 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610007cb0 con 0x7f66180d0fe0 2024-04-25T08:37:03.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 -- 172.21.15.60:0/2559811726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180d0fe0 msgr2=0x7f66180d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 --2- 172.21.15.60:0/2559811726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180d0fe0 0x7f66180d2370 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f66180b5d20 tx=0x7f661002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 -- 172.21.15.60:0/2559811726 shutdown_connections 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 --2- 172.21.15.60:0/2559811726 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180d0fe0 0x7f66180d2370 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 --2- 172.21.15.60:0/2559811726 >> v2:172.21.15.100:3300/0 conn(0x7f66180b96d0 0x7f66180d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 -- 172.21.15.60:0/2559811726 >> 172.21.15.60:0/2559811726 conn(0x7f6618061a20 msgr2=0x7f6618061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 -- 172.21.15.60:0/2559811726 shutdown_connections 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.443+0000 7f661f13b640 1 -- 172.21.15.60:0/2559811726 wait complete. 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.444+0000 7f661f13b640 1 Processor -- start 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.444+0000 7f661f13b640 1 -- start start 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661f13b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 0x7f66180c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661f13b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66180d0fe0 0x7f66180b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661f13b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f661815dd20 0x7f6618151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661f13b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66180d1e50 con 0x7f66180b96d0 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661f13b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661815f080 con 0x7f66180d0fe0 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661f13b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180d28b0 con 0x7f661815dd20 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 0x7f66180c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 0x7f66180c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37002/0 (socket says 172.21.15.60:37002) 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 -- 172.21.15.60:0/3015882236 learned_addr learned my addr 172.21.15.60:0/3015882236 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f6617fff640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66180d0fe0 0x7f66180b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661d6d8640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f661815dd20 0x7f6618151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 -- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f661815dd20 msgr2=0x7f6618151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f661815dd20 0x7f6618151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 -- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66180d0fe0 msgr2=0x7f66180b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66180d0fe0 0x7f66180b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f661ced7640 1 -- 172.21.15.60:0/3015882236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180c8ee0 con 0x7f66180b96d0 2024-04-25T08:37:03.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.445+0000 7f6617fff640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66180d0fe0 0x7f66180b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:03.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.446+0000 7f661ced7640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 0x7f66180c8690 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f660800c390 tx=0x7f660800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:03.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.446+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608017070 con 0x7f66180b96d0 2024-04-25T08:37:03.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.446+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608005ce0 con 0x7f66180b96d0 2024-04-25T08:37:03.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.446+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66180c91c0 con 0x7f66180b96d0 2024-04-25T08:37:03.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.446+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080072f0 con 0x7f66180b96d0 2024-04-25T08:37:03.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.446+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66180c9780 con 0x7f66180b96d0 2024-04-25T08:37:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.447+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6618073990 con 0x7f66180b96d0 2024-04-25T08:37:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.447+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f660801a070 con 0x7f66180b96d0 2024-04-25T08:37:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.448+0000 7f66157fa640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f660406b500 0x7f660406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.448+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f66080896f0 con 0x7f66180b96d0 2024-04-25T08:37:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.448+0000 7f6617fff640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f660406b500 0x7f660406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:03.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.448+0000 7f6617fff640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f660406b500 0x7f660406d9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f66180b2940 tx=0x7f660c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:03.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.452+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6608058470 con 0x7f66180b96d0 2024-04-25T08:37:03.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.705+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f66180cabc0 con 0x7f66180b96d0 2024-04-25T08:37:03.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.706+0000 7f66157fa640 1 -- 172.21.15.60:0/3015882236 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f6608057d70 con 0x7f66180b96d0 2024-04-25T08:37:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:03.700 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":484,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280,"write_bytes_sec":88169719,"read_op_per_sec":0,"write_op_per_sec":86},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:36:15.492518+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f660406b500 msgr2=0x7f660406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f660406b500 0x7f660406d9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f66180b2940 tx=0x7f660c008040 comp rx=0 tx=0).stop 2024-04-25T08:37:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 msgr2=0x7f66180c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 0x7f66180c8690 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f660800c390 tx=0x7f660800c850 comp rx=0 tx=0).stop 2024-04-25T08:37:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 shutdown_connections 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f661815dd20 0x7f6618151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f660406b500 0x7f660406d9c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66180d0fe0 0x7f66180b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 --2- 172.21.15.60:0/3015882236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66180b96d0 0x7f66180c8690 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 >> 172.21.15.60:0/3015882236 conn(0x7f6618061a20 msgr2=0x7f66180b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 shutdown_connections 2024-04-25T08:37:03.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:03.708+0000 7f661f13b640 1 -- 172.21.15.60:0/3015882236 wait complete. 2024-04-25T08:37:03.713 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:04.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:04.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.834+0000 7fe225ecd640 1 Processor -- start 2024-04-25T08:37:04.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.834+0000 7fe225ecd640 1 -- start start 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe225ecd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe2200b2cb0 0x7fe2200d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe225ecd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2200b97a0 con 0x7fe2200d1020 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe225ecd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2200d1c50 con 0x7fe22015dbe0 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe225ecd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22015dfc0 con 0x7fe2200b2cb0 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe2200d1020 0x7fe2200d2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34630/0 (socket says 172.21.15.60:34630) 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21ffff640 1 -- 172.21.15.60:0/3671689252 learned_addr learned my addr 172.21.15.60:0/3671689252 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21effd640 1 --2- 172.21.15.60:0/3671689252 >> v2:172.21.15.100:3300/0 conn(0x7fe2200b2cb0 0x7fe2200d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956282120 0 0) 0x7fe2200b97a0 con 0x7fe2200d1020 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe208003520 con 0x7fe2200d1020 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1602114346 0 0) 0x7fe208003520 con 0x7fe2200d1020 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 >> v2:172.21.15.100:3300/0 conn(0x7fe2200b2cb0 msgr2=0x7fe2200d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21e7fc640 1 --2- 172.21.15.60:0/3671689252 >> v2:172.21.15.100:3300/0 conn(0x7fe2200b2cb0 0x7fe2200d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:04.832 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.835+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 >> v1:172.21.15.100:6789/0 conn(0x7fe22015dbe0 legacy=0x7fe2200d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200d1f30 con 0x7fe2200d1020 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe2180019f0 con 0x7fe2200d1020 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 >> v1:172.21.15.60:6789/0 conn(0x7fe2200d1020 legacy=0x7fe2200d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2080039b0 0x7fe208003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe208004800 0x7fe208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe20800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20800c030 con 0x7fe2080051c0 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20800c1e0 con 0x7fe208004800 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20800c3e0 con 0x7fe2080039b0 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21ffff640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe20800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21f7fe640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2080039b0 0x7fe208003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21effd640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe208004800 0x7fe208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21ffff640 1 -- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2080039b0 msgr2=0x7fe208003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21ffff640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe2080039b0 0x7fe208003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21ffff640 1 -- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe208004800 msgr2=0x7fe208004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:04.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21ffff640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe208004800 0x7fe208004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.836+0000 7fe21ffff640 1 -- 172.21.15.60:0/3671689252 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe208003520 con 0x7fe2080051c0 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe21ffff640 1 -- 172.21.15.60:0/3671689252 reap_dead start 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe21ffff640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe20800b9c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe21800a860 tx=0x7fe208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458ad742264c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800a480 con 0x7fe2080051c0 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21802f070 con 0x7fe2080051c0 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe21e7fc640 1 -- 172.21.15.60:0/3671689252 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21802bcc0 con 0x7fe2080051c0 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe225ecd640 1 -- 172.21.15.60:0/3671689252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 msgr2=0x7fe20800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.837+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe20800b9c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe21800a860 tx=0x7fe208005830 comp rx=0 tx=0).stop 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.838+0000 7fe225ecd640 1 -- 172.21.15.60:0/3671689252 shutdown_connections 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.838+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3671689252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe20800b9c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.838+0000 7fe225ecd640 1 -- 172.21.15.60:0/3671689252 >> 172.21.15.60:0/3671689252 conn(0x7fe220061a20 msgr2=0x7fe220061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.838+0000 7fe225ecd640 1 -- 172.21.15.60:0/3671689252 shutdown_connections 2024-04-25T08:37:04.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.838+0000 7fe225ecd640 1 -- 172.21.15.60:0/3671689252 wait complete. 2024-04-25T08:37:04.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.839+0000 7fe225ecd640 1 Processor -- start 2024-04-25T08:37:04.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.839+0000 7fe225ecd640 1 -- start start 2024-04-25T08:37:04.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe225ecd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe2200b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe225ecd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22014f740 0x7fe22015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe225ecd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22015ecf0 0x7fe22014ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe225ecd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2200b92a0 con 0x7fe2080051c0 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe225ecd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2200d1a00 con 0x7fe22015ecf0 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe225ecd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2200b9420 con 0x7fe22014f740 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe2200b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe2200b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37026/0 (socket says 172.21.15.60:37026) 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 -- 172.21.15.60:0/3750129025 learned_addr learned my addr 172.21.15.60:0/3750129025 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21effd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22014f740 0x7fe22015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21ffff640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22015ecf0 0x7fe22014ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 -- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22014f740 msgr2=0x7fe22015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22014f740 0x7fe22015e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 -- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22015ecf0 msgr2=0x7fe22014ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22015ecf0 0x7fe22014ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 -- 172.21.15.60:0/3750129025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220160d90 con 0x7fe2080051c0 2024-04-25T08:37:04.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21effd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22014f740 0x7fe22015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.840+0000 7fe21f7fe640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe2200b52a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe21000f620 tx=0x7fe21000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.841+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100052e0 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.841+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210018070 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.841+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2200cbdf0 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.841+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210003810 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.841+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2200b4aa0 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.842+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe220073990 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.842+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2100039b0 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.843+0000 7fe207fff640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe20c06b4d0 0x7fe20c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.843+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe21008a0b0 con 0x7fe2080051c0 2024-04-25T08:37:04.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.843+0000 7fe21effd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe20c06b4d0 0x7fe20c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:04.840 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.843+0000 7fe21effd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe20c06b4d0 0x7fe20c06d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe2200cddf0 tx=0x7fe214008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:04.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:04.847+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe210058e30 con 0x7fe2080051c0 2024-04-25T08:37:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.070+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe22015f0d0 con 0x7fe2080051c0 2024-04-25T08:37:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.071+0000 7fe207fff640 1 -- 172.21.15.60:0/3750129025 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe210058730 con 0x7fe2080051c0 2024-04-25T08:37:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:05.065 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe20c06b4d0 msgr2=0x7fe20c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe20c06b4d0 0x7fe20c06d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe2200cddf0 tx=0x7fe214008040 comp rx=0 tx=0).stop 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 msgr2=0x7fe2200b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe2200b52a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe21000f620 tx=0x7fe21000fae0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 shutdown_connections 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe20c06b4d0 0x7fe20c06d990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe22015ecf0 0x7fe22014ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe22014f740 0x7fe22015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 --2- 172.21.15.60:0/3750129025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2080051c0 0x7fe2200b52a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.072+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 >> 172.21.15.60:0/3750129025 conn(0x7fe220061a20 msgr2=0x7fe2200d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.073+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 shutdown_connections 2024-04-25T08:37:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.073+0000 7fe225ecd640 1 -- 172.21.15.60:0/3750129025 wait complete. 2024-04-25T08:37:05.078 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:05.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.184+0000 7fd80055c640 1 Processor -- start 2024-04-25T08:37:05.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.184+0000 7fd80055c640 1 -- start start 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.185+0000 7fd80055c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7f815dc70 0x7fd7f80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.185+0000 7fd80055c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7f806ce10 0x7fd7f80cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.185+0000 7fd80055c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f806c220 con 0x7fd7f809a450 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.185+0000 7fd80055c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80d0c90 con 0x7fd7f815dc70 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.185+0000 7fd80055c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80d0080 con 0x7fd7f806ce10 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7f815dc70 0x7fd7f80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7f815dc70 0x7fd7f80d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37032/0 (socket says 172.21.15.60:37032) 2024-04-25T08:37:05.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 -- 172.21.15.60:0/805616904 learned_addr learned my addr 172.21.15.60:0/805616904 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7feaf9640 1 --2- 172.21.15.60:0/805616904 >> v2:172.21.15.100:3300/0 conn(0x7fd7f806ce10 0x7fd7f80cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 -- 172.21.15.60:0/805616904 >> v2:172.21.15.100:3300/0 conn(0x7fd7f806ce10 msgr2=0x7fd7f80cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 --2- 172.21.15.60:0/805616904 >> v2:172.21.15.100:3300/0 conn(0x7fd7f806ce10 0x7fd7f80cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 -- 172.21.15.60:0/805616904 >> v1:172.21.15.100:6789/0 conn(0x7fd7f809a450 legacy=0x7fd7f806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7fe2f8640 1 -- 172.21.15.60:0/805616904 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80d0360 con 0x7fd7f815dc70 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.186+0000 7fd7feaf9640 1 --2- 172.21.15.60:0/805616904 >> v2:172.21.15.100:3300/0 conn(0x7fd7f806ce10 0x7fd7f80cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd7fe2f8640 1 --2- 172.21.15.60:0/805616904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f80d1120 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80b7720 tx=0x7fd7f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fa29ff67d0ab3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd7fd2f6640 1 -- 172.21.15.60:0/805616904 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0037070 con 0x7fd7f815dc70 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd7fd2f6640 1 -- 172.21.15.60:0/805616904 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f0032440 con 0x7fd7f815dc70 2024-04-25T08:37:05.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd7fd2f6640 1 -- 172.21.15.60:0/805616904 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0032810 con 0x7fd7f815dc70 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 -- 172.21.15.60:0/805616904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 msgr2=0x7fd7f80d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 --2- 172.21.15.60:0/805616904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f80d1120 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80b7720 tx=0x7fd7f002a230 comp rx=0 tx=0).stop 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 -- 172.21.15.60:0/805616904 shutdown_connections 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 --2- 172.21.15.60:0/805616904 >> v2:172.21.15.100:3300/0 conn(0x7fd7f806ce10 0x7fd7f80cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 --2- 172.21.15.60:0/805616904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f80d1120 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 -- 172.21.15.60:0/805616904 >> 172.21.15.60:0/805616904 conn(0x7fd7f80b3270 msgr2=0x7fd7f80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.187+0000 7fd80055c640 1 -- 172.21.15.60:0/805616904 shutdown_connections 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.188+0000 7fd80055c640 1 -- 172.21.15.60:0/805616904 wait complete. 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 Processor -- start 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 -- start start 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f806ce10 0x7fd7f81f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7f809a450 0x7fd7f81482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f815e050 con 0x7fd7f815dc70 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80d0920 con 0x7fd7f806ce10 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd80055c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80cbdd0 con 0x7fd7f809a450 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.189+0000 7fd7feaf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37034/0 (socket says 172.21.15.60:37034) 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 -- 172.21.15.60:0/3772338353 learned_addr learned my addr 172.21.15.60:0/3772338353 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7fe2f8640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f806ce10 0x7fd7f81f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7fdaf7640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7f809a450 0x7fd7f81482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 -- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7f809a450 msgr2=0x7fd7f81482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7f809a450 0x7fd7f81482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 -- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f806ce10 msgr2=0x7fd7f81f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f806ce10 0x7fd7f81f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 -- 172.21.15.60:0/3772338353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8148830 con 0x7fd7f815dc70 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7fe2f8640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f806ce10 0x7fd7f81f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7feaf9640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f815b470 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800cef0 tx=0x7fd7e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e801a070 con 0x7fd7f815dc70 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800d860 con 0x7fd7f815dc70 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f815b9b0 con 0x7fd7f815dc70 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800dc50 con 0x7fd7f815dc70 2024-04-25T08:37:05.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.190+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f8158b80 con 0x7fd7f815dc70 2024-04-25T08:37:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.191+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8158dc0 con 0x7fd7f815dc70 2024-04-25T08:37:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.192+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7e801d070 con 0x7fd7f815dc70 2024-04-25T08:37:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.192+0000 7fd7f6ffd640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7e406b4d0 0x7fd7e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.192+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7e808ac60 con 0x7fd7f815dc70 2024-04-25T08:37:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.192+0000 7fd7fe2f8640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7e406b4d0 0x7fd7e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.193+0000 7fd7fe2f8640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7e406b4d0 0x7fd7e406d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0034000 tx=0x7fd7f002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.190 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.197+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7e8059a10 con 0x7fd7f815dc70 2024-04-25T08:37:05.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.447+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd7f815ad40 con 0x7fd7f815dc70 2024-04-25T08:37:05.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.447+0000 7fd7f6ffd640 1 -- 172.21.15.60:0/3772338353 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fd7e8059310 con 0x7fd7f815dc70 2024-04-25T08:37:05.441 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:05.442 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":486,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280,"write_bytes_sec":44217795,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:37:03.504101+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:05.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7e406b4d0 msgr2=0x7fd7e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7e406b4d0 0x7fd7e406d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0034000 tx=0x7fd7f002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 msgr2=0x7fd7f815b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f815b470 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800cef0 tx=0x7fd7e800a710 comp rx=0 tx=0).stop 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 shutdown_connections 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7f815dc70 0x7fd7f815b470 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7f809a450 0x7fd7f81482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7e406b4d0 0x7fd7e406d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 --2- 172.21.15.60:0/3772338353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f806ce10 0x7fd7f81f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 >> 172.21.15.60:0/3772338353 conn(0x7fd7f80b3270 msgr2=0x7fd7f80c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 shutdown_connections 2024-04-25T08:37:05.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.449+0000 7fd80055c640 1 -- 172.21.15.60:0/3772338353 wait complete. 2024-04-25T08:37:05.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T08:37:05.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:05.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T08:37:05.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.564+0000 7f8f2206b640 1 Processor -- start 2024-04-25T08:37:05.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.564+0000 7f8f2206b640 1 -- start start 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f2206b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f1c0b9750 0x7f8f1c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f2206b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f1c0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f2206b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c0cf060 con 0x7f8f1c15da90 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f2206b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1c162600 con 0x7f8f1c0b9750 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f2206b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c161d70 con 0x7f8f1c0cebb0 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f1b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8f1c15da90 0x7f8f1c161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34638/0 (socket says 172.21.15.60:34638) 2024-04-25T08:37:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2484449997 learned_addr learned my addr 172.21.15.60:0/2484449997 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606278285 0 0) 0x7f8f1c0cf060 con 0x7f8f1c15da90 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f04003520 con 0x7f8f1c15da90 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f1c0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.565+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2484449997 >> v2:172.21.15.100:3300/0 conn(0x7f8f1c0b9750 0x7f8f1c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2008551115 0 0) 0x7f8f04003520 con 0x7f8f1c15da90 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 msgr2=0x7f8f1c0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f1c0d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 >> v2:172.21.15.100:3300/0 conn(0x7f8f1c0b9750 msgr2=0x7f8f1c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 --2- 172.21.15.60:0/2484449997 >> v2:172.21.15.100:3300/0 conn(0x7f8f1c0b9750 0x7f8f1c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c162040 con 0x7f8f1c15da90 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f8f0c001670 con 0x7f8f1c15da90 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 >> v1:172.21.15.60:6789/0 conn(0x7f8f1c15da90 legacy=0x7f8f1c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f04004d40 0x7f8f040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c162600 con 0x7f8f040039b0 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c161d70 con 0x7f8f04004d40 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0400b9f0 con 0x7f8f1c0cebb0 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f04004d40 0x7f8f040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 msgr2=0x7f8f04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f04004d40 msgr2=0x7f8f040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f04004d40 0x7f8f040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2484449997 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04003520 con 0x7f8f040039b0 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f04004d40 0x7f8f040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.566+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c0cebb0 0x7f8f04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f04003dd0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c007fd0 tx=0x7f8f04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f602f3da1fa6d97f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c007d50 con 0x7f8f040039b0 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c005ad0 con 0x7f8f040039b0 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f1a7fc640 1 -- 172.21.15.60:0/2484449997 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c007070 con 0x7f8f040039b0 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f2206b640 1 -- 172.21.15.60:0/2484449997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 msgr2=0x7f8f04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f2206b640 1 --2- 172.21.15.60:0/2484449997 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f04003dd0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c007fd0 tx=0x7f8f04005830 comp rx=0 tx=0).stop 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.567+0000 7f8f1bfff640 1 -- 172.21.15.60:0/2484449997 reap_dead start 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.568+0000 7f8f2206b640 1 -- 172.21.15.60:0/2484449997 shutdown_connections 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.568+0000 7f8f2206b640 1 -- 172.21.15.60:0/2484449997 >> 172.21.15.60:0/2484449997 conn(0x7f8f1c061a20 msgr2=0x7f8f1c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.568+0000 7f8f2206b640 1 -- 172.21.15.60:0/2484449997 shutdown_connections 2024-04-25T08:37:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.568+0000 7f8f2206b640 1 -- 172.21.15.60:0/2484449997 wait complete. 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.569+0000 7f8f2206b640 1 Processor -- start 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.569+0000 7f8f2206b640 1 -- start start 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f1c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1c1f0ce0 0x7f8f1c1f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c1f7100 0x7f8f1c1f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0d1df0 con 0x7f8f040039b0 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0d1b10 con 0x7f8f1c1f0ce0 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c15de70 con 0x7f8f1c1f7100 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f1c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f1c0cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37056/0 (socket says 172.21.15.60:37056) 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/4186355511 learned_addr learned my addr 172.21.15.60:0/4186355511 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1affd640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1c1f0ce0 0x7f8f1c1f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c1f7100 0x7f8f1c1f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c1f7100 msgr2=0x7f8f1c1f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c1f7100 0x7f8f1c1f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1c1f0ce0 msgr2=0x7f8f1c1f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1c1f0ce0 0x7f8f1c1f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/4186355511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c0cd8a0 con 0x7f8f040039b0 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f1c0cd360 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c300 tx=0x7f8f1c162210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f03fff640 1 -- 172.21.15.60:0/4186355511 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c042020 con 0x7f8f040039b0 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f03fff640 1 -- 172.21.15.60:0/4186355511 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c007270 con 0x7f8f040039b0 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f03fff640 1 -- 172.21.15.60:0/4186355511 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c001cb0 con 0x7f8f040039b0 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c160640 con 0x7f8f040039b0 2024-04-25T08:37:05.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.570+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c160c00 con 0x7f8f040039b0 2024-04-25T08:37:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.572+0000 7f8f03fff640 1 -- 172.21.15.60:0/4186355511 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f0c045020 con 0x7f8f040039b0 2024-04-25T08:37:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.572+0000 7f8f03fff640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f0806b4d0 0x7f8f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.572+0000 7f8f03fff640 1 -- 172.21.15.60:0/4186355511 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f0c0aaee0 con 0x7f8f040039b0 2024-04-25T08:37:05.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.573+0000 7f8f1affd640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f0806b4d0 0x7f8f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.573+0000 7f8f1affd640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f0806b4d0 0x7f8f0806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0ca190 tx=0x7f8f10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.573+0000 7f8f24d6b740 1 Processor -- start 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.573+0000 7f8f24d6b740 1 -- start start 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a44e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a44e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 0x558a4a421f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4a387da0 con 0x558a4a41e770 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4a330b80 con 0x558a4a44bef0 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a4a383e40 con 0x558a4a44bb10 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 0x558a4a421f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 0x558a4a421f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37068/0 (socket says 172.21.15.60:37068) 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.574+0000 7f8f1bfff640 1 -- 172.21.15.60:0/766662355 learned_addr learned my addr 172.21.15.60:0/766662355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a44e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1affd640 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a44e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1bfff640 1 -- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 msgr2=0x558a4a44e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a44e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1bfff640 1 -- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 msgr2=0x558a4a44e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a44e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1bfff640 1 -- 172.21.15.60:0/766662355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a4a33af10 con 0x558a4a41e770 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 0x558a4a421f80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0d0ef0 tx=0x7f8f14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314f1294375dd0a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f017fa640 1 -- 172.21.15.60:0/766662355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1401a070 con 0x558a4a41e770 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f017fa640 1 -- 172.21.15.60:0/766662355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14016630 con 0x558a4a41e770 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.575+0000 7f8f017fa640 1 -- 172.21.15.60:0/766662355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f140169e0 con 0x558a4a41e770 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 -- 172.21.15.60:0/766662355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 msgr2=0x558a4a421f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 0x558a4a421f80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0d0ef0 tx=0x7f8f14009750 comp rx=0 tx=0).stop 2024-04-25T08:37:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 -- 172.21.15.60:0/766662355 shutdown_connections 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a41e770 0x558a4a421f80 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a44e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/766662355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a44e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 -- 172.21.15.60:0/766662355 >> 172.21.15.60:0/766662355 conn(0x558a4a4209e0 msgr2=0x558a4a469320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 -- 172.21.15.60:0/766662355 shutdown_connections 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 -- 172.21.15.60:0/766662355 wait complete. 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 -- start start 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.576+0000 7f8f24d6b740 1 Processor -- start 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a462470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a4629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a463d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4a423b90 con 0x558a4a44bb10 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4a3264b0 con 0x558a4a44bef0 2024-04-25T08:37:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a4a423e70 con 0x558a4a46c9c0 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f24d6b740 10 client.0 _ll_register_callbacks cb 0x558a4a366180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a462470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a462470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37080/0 (socket says 172.21.15.60:37080) 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2547137728 learned_addr learned my addr 172.21.15.60:0/2547137728 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a463d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a4629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 msgr2=0x558a4a463d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a463d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 msgr2=0x558a4a4629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a4629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2547137728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a4a1667d0 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.577+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a4629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a462470 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0a9280 tx=0x7f8f0c079bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9b0fd687ef102d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8efa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8efa7fc640 1 -- 172.21.15.60:0/2547137728 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0a7060 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8efa7fc640 1 -- 172.21.15.60:0/2547137728 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f0c0a2be0 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f14016630 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8efa7fc640 1 -- 172.21.15.60:0/2547137728 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f0c081be0 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 10 client.8520 fetch_fsmap learned FSMap version 62 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x558a4a09c3f0 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8efa7fc640 1 -- 172.21.15.60:0/2547137728 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14016630 con 0x558a4a44bb10 2024-04-25T08:37:05.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8efa7fc640 1 -- 172.21.15.60:0/2547137728 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f0c04f060 con 0x558a4a44bb10 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 10 client.8520 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 10 client.8520 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 20 client.8520 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a4611b0 0x558a4a461590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 4 client.8520 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a4a3952e0 con 0x558a4a4611b0 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.578+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a4611b0 0x558a4a461590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a4611b0 0x558a4a461590 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14854f42261805c7 server_cookie=894905b70d0aa443 in_seq=0 out_seq=0 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8efa7fc640 10 client.8520 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8efa7fc640 1 -- 172.21.15.60:0/2547137728 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a4a3952e0 con 0x558a4a4611b0 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8efa7fc640 10 client.8520 handle_command_reply: tid=0 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 client.8520 shutdown 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 msgr2=0x558a4a462470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a462470 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0a9280 tx=0x7f8f0c079bd0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 shutdown_connections 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a463d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a4629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a4611b0 0x558a4a461590 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2547137728 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a462470 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8efa7fc640 0 client.8520 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 >> 172.21.15.60:0/2547137728 conn(0x558a4a41e770 msgr2=0x558a4a4232b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 shutdown_connections 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.579+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2547137728 wait complete. 2024-04-25T08:37:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.580+0000 7f8f24d6b740 20 client.8520 trim_cache size 0 max 16384 2024-04-25T08:37:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.594+0000 7f8f24d6b740 1 Processor -- start 2024-04-25T08:37:05.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 -- start start 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a4232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a465ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a44a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c200920 con 0x558a4a44bb10 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c1ffb00 con 0x558a4a44bef0 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c1fef50 con 0x558a4a46c9c0 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a4232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f1b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a4232b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37086/0 (socket says 172.21.15.60:37086) 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.595+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2950103388 learned_addr learned my addr 172.21.15.60:0/2950103388 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a465ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a44a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 msgr2=0x558a4a44a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a44a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 msgr2=0x558a4a465ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a465ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1b7fe640 1 -- 172.21.15.60:0/2950103388 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a4a05c0f0 con 0x558a4a44bb10 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1affd640 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a465ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a4232b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0a2e30 tx=0x7f8f0c08e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb8f594d125b20b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f00ff9640 1 -- 172.21.15.60:0/2950103388 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0792a0 con 0x558a4a44bb10 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f00ff9640 1 -- 172.21.15.60:0/2950103388 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c007660 con 0x558a4a44bb10 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.596+0000 7f8f00ff9640 1 -- 172.21.15.60:0/2950103388 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c081990 con 0x558a4a44bb10 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2950103388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 msgr2=0x558a4a4232b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a4232b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0a2e30 tx=0x7f8f0c08e000 comp rx=0 tx=0).stop 2024-04-25T08:37:05.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2950103388 shutdown_connections 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a46c9c0 0x558a4a44a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a465ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/2950103388 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a44bb10 0x558a4a4232b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2950103388 >> 172.21.15.60:0/2950103388 conn(0x558a4a41e770 msgr2=0x558a4a4247a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2950103388 shutdown_connections 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 -- 172.21.15.60:0/2950103388 wait complete. 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 -- start start 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.597+0000 7f8f24d6b740 1 Processor -- start 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a4d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a41f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 0x558a4a44a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c1ff500 con 0x558a4a46c9c0 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4a461590 con 0x558a4a44bef0 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a4a478860 con 0x558a4a44bb10 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f24d6b740 10 client.0 _ll_register_callbacks cb 0x7f8f14009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:05.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 0x558a4a44a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 0x558a4a44a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37092/0 (socket says 172.21.15.60:37092) 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 -- 172.21.15.60:0/1571062866 learned_addr learned my addr 172.21.15.60:0/1571062866 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a4d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1affd640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a41f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 -- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 msgr2=0x558a4a4d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a4d2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 -- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 msgr2=0x558a4a41f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.598+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a41f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f1bfff640 1 -- 172.21.15.60:0/1571062866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a4a449600 con 0x558a4a46c9c0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f1bfff640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 0x558a4a44a700 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8f14019000 tx=0x7f8f14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42de08a0f299eb14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8efa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8efa7fc640 1 -- 172.21.15.60:0/1571062866 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1402f060 con 0x558a4a46c9c0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8efa7fc640 1 -- 172.21.15.60:0/1571062866 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f14004450 con 0x558a4a46c9c0 2024-04-25T08:37:05.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f0c007660 con 0x558a4a46c9c0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8efa7fc640 1 -- 172.21.15.60:0/1571062866 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f14035050 con 0x558a4a46c9c0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 10 client.8526 fetch_fsmap learned FSMap version 62 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x558a4a466810 con 0x558a4a46c9c0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8efa7fc640 1 -- 172.21.15.60:0/1571062866 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c007660 con 0x558a4a46c9c0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8efa7fc640 1 -- 172.21.15.60:0/1571062866 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f14038020 con 0x558a4a46c9c0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 10 client.8526 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 10 client.8526 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 20 client.8526 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a44ac40 0x558a4a477c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 4 client.8526 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x558a4a38f630 con 0x558a4a44ac40 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.599+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a44ac40 0x558a4a477c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8efa7fc640 10 client.8526 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f1b7fe640 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a44ac40 0x558a4a477c70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c8b02cec2afb69 server_cookie=5ecc6d2982f53255 in_seq=0 out_seq=0 2024-04-25T08:37:05.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8efa7fc640 1 -- 172.21.15.60:0/1571062866 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x558a4a38f630 con 0x558a4a44ac40 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8efa7fc640 10 client.8526 handle_command_reply: tid=0 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 client.8526 shutdown 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 msgr2=0x558a4a44a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 0x558a4a44a700 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8f14019000 tx=0x7f8f14004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 shutdown_connections 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x558a4a44ac40 0x558a4a477c70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558a4a46c9c0 0x558a4a44a700 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558a4a44bef0 0x558a4a41f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 --2- 172.21.15.60:0/1571062866 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558a4a44bb10 0x558a4a4d2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 >> 172.21.15.60:0/1571062866 conn(0x558a4a41e770 msgr2=0x558a4a46ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8efa7fc640 0 client.8526 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 shutdown_connections 2024-04-25T08:37:05.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 1 -- 172.21.15.60:0/1571062866 wait complete. 2024-04-25T08:37:05.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.600+0000 7f8f24d6b740 20 client.8526 trim_cache size 0 max 16384 2024-04-25T08:37:05.604 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:05.604 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:37:05.605 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f0806b4d0 msgr2=0x7f8f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f0806b4d0 0x7f8f0806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0ca190 tx=0x7f8f10008040 comp rx=0 tx=0).stop 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 msgr2=0x7f8f1c0cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f1c0cd360 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c300 tx=0x7f8f1c162210 comp rx=0 tx=0).stop 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 shutdown_connections 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8f1c1f7100 0x7f8f1c1f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8f0806b4d0 0x7f8f0806d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f1c1f0ce0 0x7f8f1c1f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 --2- 172.21.15.60:0/4186355511 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f040039b0 0x7f8f1c0cd360 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 >> 172.21.15.60:0/4186355511 conn(0x7f8f1c061a20 msgr2=0x7f8f1c15cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 shutdown_connections 2024-04-25T08:37:05.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.601+0000 7f8f2206b640 1 -- 172.21.15.60:0/4186355511 wait complete. 2024-04-25T08:37:05.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:05.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:37:05.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.716+0000 7fe09d14c640 1 Processor -- start 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.716+0000 7fe09d14c640 1 -- start start 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe09d14c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe09815de80 0x7fe0980cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe09d14c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe0980cefb0 0x7fe0980cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe09d14c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0980afc80 con 0x7fe0980cf980 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe09d14c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0980ce4d0 con 0x7fe09815de80 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe09d14c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0980ce7b0 con 0x7fe0980cefb0 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe096d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe09815de80 0x7fe0980cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe096d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe09815de80 0x7fe0980cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37102/0 (socket says 172.21.15.60:37102) 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.717+0000 7fe096d9d640 1 -- 172.21.15.60:0/1205753402 learned_addr learned my addr 172.21.15.60:0/1205753402 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe09659c640 1 --2- 172.21.15.60:0/1205753402 >> v2:172.21.15.100:3300/0 conn(0x7fe0980cefb0 0x7fe0980cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe096d9d640 1 -- 172.21.15.60:0/1205753402 >> v2:172.21.15.100:3300/0 conn(0x7fe0980cefb0 msgr2=0x7fe0980cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe096d9d640 1 --2- 172.21.15.60:0/1205753402 >> v2:172.21.15.100:3300/0 conn(0x7fe0980cefb0 0x7fe0980cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe096d9d640 1 -- 172.21.15.60:0/1205753402 >> v1:172.21.15.100:6789/0 conn(0x7fe0980cf980 legacy=0x7fe09806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe096d9d640 1 -- 172.21.15.60:0/1205753402 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980cea90 con 0x7fe09815de80 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe09659c640 1 --2- 172.21.15.60:0/1205753402 >> v2:172.21.15.100:3300/0 conn(0x7fe0980cefb0 0x7fe0980cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe096d9d640 1 --2- 172.21.15.60:0/1205753402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe0980cadb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe098062500 tx=0x7fe09002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5d7603526b05ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe095d9b640 1 -- 172.21.15.60:0/1205753402 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090036070 con 0x7fe09815de80 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe095d9b640 1 -- 172.21.15.60:0/1205753402 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe090031600 con 0x7fe09815de80 2024-04-25T08:37:05.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.718+0000 7fe095d9b640 1 -- 172.21.15.60:0/1205753402 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900319d0 con 0x7fe09815de80 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 -- 172.21.15.60:0/1205753402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 msgr2=0x7fe0980cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 --2- 172.21.15.60:0/1205753402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe0980cadb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe098062500 tx=0x7fe09002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 -- 172.21.15.60:0/1205753402 shutdown_connections 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 --2- 172.21.15.60:0/1205753402 >> v2:172.21.15.100:3300/0 conn(0x7fe0980cefb0 0x7fe0980cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 --2- 172.21.15.60:0/1205753402 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe0980cadb0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 -- 172.21.15.60:0/1205753402 >> 172.21.15.60:0/1205753402 conn(0x7fe0980b2390 msgr2=0x7fe0980b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 -- 172.21.15.60:0/1205753402 shutdown_connections 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.719+0000 7fe09d14c640 1 -- 172.21.15.60:0/1205753402 wait complete. 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 Processor -- start 2024-04-25T08:37:05.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 -- start start 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0980cefb0 0x7fe098154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0980cf980 0x7fe09814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe098156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980b3030 con 0x7fe09815de80 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980b2eb0 con 0x7fe0980cefb0 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09d14c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980c5600 con 0x7fe0980cf980 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe098156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe098156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37108/0 (socket says 172.21.15.60:37108) 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.721+0000 7fe09759e640 1 -- 172.21.15.60:0/584836008 learned_addr learned my addr 172.21.15.60:0/584836008 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe096d9d640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0980cefb0 0x7fe098154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09659c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0980cf980 0x7fe09814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09759e640 1 -- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0980cf980 msgr2=0x7fe09814d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09759e640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0980cf980 0x7fe09814d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09759e640 1 -- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0980cefb0 msgr2=0x7fe098154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09759e640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0980cefb0 0x7fe098154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09759e640 1 -- 172.21.15.60:0/584836008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098157890 con 0x7fe09815de80 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09759e640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe098156570 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe08800c410 tx=0x7fe08800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe07f7fe640 1 -- 172.21.15.60:0/584836008 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088011790 con 0x7fe09815de80 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe07f7fe640 1 -- 172.21.15.60:0/584836008 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe088017070 con 0x7fe09815de80 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0980c7d00 con 0x7fe09815de80 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe07f7fe640 1 -- 172.21.15.60:0/584836008 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088011bb0 con 0x7fe09815de80 2024-04-25T08:37:05.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.722+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0980c7fa0 con 0x7fe09815de80 2024-04-25T08:37:05.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.724+0000 7fe07f7fe640 1 -- 172.21.15.60:0/584836008 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe08801c460 con 0x7fe09815de80 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.724+0000 7fe07f7fe640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe08406b4d0 0x7fe08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.724+0000 7fe07f7fe640 1 -- 172.21.15.60:0/584836008 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe088089ff0 con 0x7fe09815de80 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.725+0000 7fe096d9d640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe08406b4d0 0x7fe08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.725+0000 7fe096d9d640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe08406b4d0 0x7fe08406d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe090033000 tx=0x7fe090009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.725+0000 7fe09fe4f740 1 Processor -- start 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 -- start start 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d6334d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d6334d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6331d770 0x555d63320f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d63286da0 con 0x555d6334ab10 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6322fb80 con 0x555d6331d770 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d63282e40 con 0x555d6334aef0 2024-04-25T08:37:05.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d6334d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d6334d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37124/0 (socket says 172.21.15.60:37124) 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.726+0000 7fe096d9d640 1 -- 172.21.15.60:0/1382961159 learned_addr learned my addr 172.21.15.60:0/1382961159 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe096d9d640 1 -- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 msgr2=0x555d6334d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe09659c640 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d6334d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe09759e640 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6331d770 0x555d63320f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe096d9d640 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d6334d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe096d9d640 1 -- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6331d770 msgr2=0x555d63320f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe096d9d640 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6331d770 0x555d63320f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe096d9d640 1 -- 172.21.15.60:0/1382961159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d63239f10 con 0x555d6334ab10 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe096d9d640 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d6334d320 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe090034040 tx=0x7fe09002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63ad2aba6d15fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe07cff9640 1 -- 172.21.15.60:0/1382961159 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09004c070 con 0x555d6334ab10 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe07cff9640 1 -- 172.21.15.60:0/1382961159 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09003cb90 con 0x555d6334ab10 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.727+0000 7fe07cff9640 1 -- 172.21.15.60:0/1382961159 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090036040 con 0x555d6334ab10 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 -- 172.21.15.60:0/1382961159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 msgr2=0x555d6334d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d6334d320 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe090034040 tx=0x7fe09002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 -- 172.21.15.60:0/1382961159 shutdown_connections 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6331d770 0x555d63320f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d6334d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/1382961159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d6334d320 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 -- 172.21.15.60:0/1382961159 >> 172.21.15.60:0/1382961159 conn(0x555d6331f9e0 msgr2=0x555d63368320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 -- 172.21.15.60:0/1382961159 shutdown_connections 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 -- 172.21.15.60:0/1382961159 wait complete. 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 -- start start 2024-04-25T08:37:05.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.728+0000 7fe09fe4f740 1 Processor -- start 2024-04-25T08:37:05.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63341850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63378070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d6336bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d63322b90 con 0x555d6334aef0 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d632254b0 con 0x555d6334ab10 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d63322e70 con 0x555d63350f90 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09fe4f740 10 client.0 _ll_register_callbacks cb 0x555d63265180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63378070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63341850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63378070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37128/0 (socket says 172.21.15.60:37128) 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09659c640 1 -- 172.21.15.60:0/2902077123 learned_addr learned my addr 172.21.15.60:0/2902077123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09759e640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d6336bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.729+0000 7fe09659c640 1 -- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 msgr2=0x555d6336bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09659c640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d6336bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09659c640 1 -- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 msgr2=0x555d63341850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09659c640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63341850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09659c640 1 -- 172.21.15.60:0/2902077123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d630657d0 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe096d9d640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63341850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09659c640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63378070 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe0981f6bb0 tx=0x7fe08c0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb132ce0c3ac984 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe075ffb640 1 -- 172.21.15.60:0/2902077123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c014660 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe075ffb640 1 -- 172.21.15.60:0/2902077123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe08c014800 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe09003cb90 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe075ffb640 1 -- 172.21.15.60:0/2902077123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe08c024050 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 10 client.8538 fetch_fsmap learned FSMap version 62 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x555d62f9b3f0 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe075ffb640 1 -- 172.21.15.60:0/2902077123 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09003cb90 con 0x555d6334aef0 2024-04-25T08:37:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe075ffb640 1 -- 172.21.15.60:0/2902077123 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe08c014a00 con 0x555d6334aef0 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 10 client.8538 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 10 client.8538 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 20 client.8538 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6331cad0 0x555d63369450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 4 client.8538 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.730+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d632942e0 con 0x555d6331cad0 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.731+0000 7fe096d9d640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6331cad0 0x555d63369450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.731+0000 7fe096d9d640 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6331cad0 0x555d63369450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9bad61eebbb083 server_cookie=565b9e055859dd45 in_seq=0 out_seq=0 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.731+0000 7fe075ffb640 10 client.8538 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.731+0000 7fe075ffb640 1 -- 172.21.15.60:0/2902077123 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555d632942e0 con 0x555d6331cad0 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.731+0000 7fe075ffb640 10 client.8538 handle_command_reply: tid=0 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.731+0000 7fe09fe4f740 1 client.8538 shutdown 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 msgr2=0x555d63378070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63378070 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe0981f6bb0 tx=0x7fe08c0109e0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 shutdown_connections 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d6336bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6331cad0 0x555d63369450 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63378070 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2902077123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63341850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe075ffb640 0 client.8538 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 >> 172.21.15.60:0/2902077123 conn(0x555d6331d770 msgr2=0x555d6331e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 shutdown_connections 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2902077123 wait complete. 2024-04-25T08:37:05.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.732+0000 7fe09fe4f740 20 client.8538 trim_cache size 0 max 16384 2024-04-25T08:37:05.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.747+0000 7fe09fe4f740 1 Processor -- start 2024-04-25T08:37:05.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.747+0000 7fe09fe4f740 1 -- start start 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d63340770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d63340cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d63350f90 0x555d63320f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe098200380 con 0x555d6334ab10 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981ff520 con 0x555d63350f90 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980c9400 con 0x555d6334aef0 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d63340770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d63340770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37140/0 (socket says 172.21.15.60:37140) 2024-04-25T08:37:05.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 -- 172.21.15.60:0/2154348835 learned_addr learned my addr 172.21.15.60:0/2154348835 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09659c640 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d63340cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe09759e640 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d63350f90 0x555d63320f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 -- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 msgr2=0x555d63340cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d63340cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 -- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d63350f90 msgr2=0x555d63320f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d63350f90 0x555d63320f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 -- 172.21.15.60:0/2154348835 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d62f5b0f0 con 0x555d6334ab10 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe096d9d640 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d63340770 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe09002a730 tx=0x7fe09003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f35732b1f70a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe077fff640 1 -- 172.21.15.60:0/2154348835 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09004c070 con 0x555d6334ab10 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe077fff640 1 -- 172.21.15.60:0/2154348835 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09004d040 con 0x555d6334ab10 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.748+0000 7fe077fff640 1 -- 172.21.15.60:0/2154348835 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090036040 con 0x555d6334ab10 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2154348835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 msgr2=0x555d63340770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d63340770 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe09002a730 tx=0x7fe09003b000 comp rx=0 tx=0).stop 2024-04-25T08:37:05.749 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2154348835 shutdown_connections 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d63350f90 0x555d63320f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d6334aef0 0x555d63340cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/2154348835 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334ab10 0x555d63340770 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2154348835 >> 172.21.15.60:0/2154348835 conn(0x555d6331d770 msgr2=0x555d6331e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2154348835 shutdown_connections 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.749+0000 7fe09fe4f740 1 -- 172.21.15.60:0/2154348835 wait complete. 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 -- start start 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 Processor -- start 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63364fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63365520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d63368320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d6336b950 con 0x555d6334aef0 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981fef20 con 0x555d6334ab10 2024-04-25T08:37:05.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d6334d2e0 con 0x555d63350f90 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09fe4f740 10 client.0 _ll_register_callbacks cb 0x7fe090034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63365520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63365520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37152/0 (socket says 172.21.15.60:37152) 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.750+0000 7fe09659c640 1 -- 172.21.15.60:0/695470121 learned_addr learned my addr 172.21.15.60:0/695470121 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe096d9d640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63364fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09759e640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d63368320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09659c640 1 -- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 msgr2=0x555d63368320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09659c640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d63368320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09659c640 1 -- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 msgr2=0x555d63364fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09659c640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63364fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09659c640 1 -- 172.21.15.60:0/695470121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d6336bdb0 con 0x555d6334aef0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09659c640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63365520 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe08c006fd0 tx=0x7fe08c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2ed1e51af38897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:05.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe075ffb640 1 -- 172.21.15.60:0/695470121 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c014a70 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe075ffb640 1 -- 172.21.15.60:0/695470121 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe08c00ecc0 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe09004d040 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe075ffb640 1 -- 172.21.15.60:0/695470121 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe08c038050 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09fe4f740 10 client.8544 fetch_fsmap learned FSMap version 62 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x555d633820d0 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe075ffb640 1 -- 172.21.15.60:0/695470121 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09004d040 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe075ffb640 1 -- 172.21.15.60:0/695470121 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe08c03b020 con 0x555d6334aef0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09fe4f740 10 client.8544 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09fe4f740 10 client.8544 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.751+0000 7fe09fe4f740 20 client.8544 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6333b560 0x555d63368860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 4 client.8544 mds_command: new command op to 8184 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 --> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d6328e630 con 0x555d6333b560 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe096d9d640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6333b560 0x555d63368860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe075ffb640 10 client.8544 ms_handle_connect on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe096d9d640 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6333b560 0x555d63368860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b100f5e2281c7c14 server_cookie=ca7320746ee65f54 in_seq=0 out_seq=0 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe075ffb640 1 -- 172.21.15.60:0/695470121 <== mds.0 v2:172.21.15.60:6833/4168808734 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555d6328e630 con 0x555d6333b560 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe075ffb640 10 client.8544 handle_command_reply: tid=0 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 client.8544 shutdown 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 msgr2=0x555d63365520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63365520 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe08c006fd0 tx=0x7fe08c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 shutdown_connections 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555d63350f90 0x555d63368320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:6833/4168808734,v1:172.21.15.60:6835/4168808734] conn(0x555d6333b560 0x555d63368860 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555d6334aef0 0x555d63365520 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe075ffb640 0 client.8544 ms_handle_reset on v2:172.21.15.60:6833/4168808734 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 --2- 172.21.15.60:0/695470121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555d6334ab10 0x555d63364fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 >> 172.21.15.60:0/695470121 conn(0x555d6331d770 msgr2=0x555d63378050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 shutdown_connections 2024-04-25T08:37:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 1 -- 172.21.15.60:0/695470121 wait complete. 2024-04-25T08:37:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.752+0000 7fe09fe4f740 20 client.8544 trim_cache size 0 max 16384 2024-04-25T08:37:05.754 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:05.754 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:37:05.754 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:37:05.754 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe08406b4d0 msgr2=0x7fe08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe08406b4d0 0x7fe08406d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe090033000 tx=0x7fe090009000 comp rx=0 tx=0).stop 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 msgr2=0x7fe098156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe098156570 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe08800c410 tx=0x7fe08800c8d0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 shutdown_connections 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe09815de80 0x7fe098156570 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0980cf980 0x7fe09814d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe08406b4d0 0x7fe08406d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 --2- 172.21.15.60:0/584836008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0980cefb0 0x7fe098154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 >> 172.21.15.60:0/584836008 conn(0x7fe0980b2390 msgr2=0x7fe0980b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 shutdown_connections 2024-04-25T08:37:05.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.753+0000 7fe09d14c640 1 -- 172.21.15.60:0/584836008 wait complete. 2024-04-25T08:37:05.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:05.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.865+0000 7f5d77238640 1 Processor -- start 2024-04-25T08:37:05.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.865+0000 7f5d77238640 1 -- start start 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d77238640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5d7015dd30 0x7f5d700b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d77238640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d700d1df0 con 0x7f5d700b9770 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d77238640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d700d20d0 con 0x7f5d700d0ad0 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d77238640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d700d23b0 con 0x7f5d7015dd30 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d6ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5d700b9770 0x7f5d700d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34644/0 (socket says 172.21.15.60:34644) 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d6ffff640 1 -- 172.21.15.60:0/2706054172 learned_addr learned my addr 172.21.15.60:0/2706054172 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d74fd4640 1 --2- 172.21.15.60:0/2706054172 >> v2:172.21.15.100:3300/0 conn(0x7f5d7015dd30 0x7f5d700b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.866+0000 7f5d6f7fe640 1 -- 172.21.15.60:0/2706054172 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364783826 0 0) 0x7f5d700d1df0 con 0x7f5d700b9770 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.867+0000 7f5d6f7fe640 1 -- 172.21.15.60:0/2706054172 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d5c003520 con 0x7f5d700b9770 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.867+0000 7f5d74fd4640 1 -- 172.21.15.60:0/2706054172 >> v1:172.21.15.100:6789/0 conn(0x7f5d700d0ad0 legacy=0x7f5d700b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.867+0000 7f5d74fd4640 1 -- 172.21.15.60:0/2706054172 >> v1:172.21.15.60:6789/0 conn(0x7f5d700b9770 legacy=0x7f5d700d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.867+0000 7f5d6f7fe640 1 -- 172.21.15.60:0/2706054172 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3582305986 0 0) 0x7f5d5c003520 con 0x7f5d700b9770 2024-04-25T08:37:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.867+0000 7f5d74fd4640 1 -- 172.21.15.60:0/2706054172 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700d2690 con 0x7f5d7015dd30 2024-04-25T08:37:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d74fd4640 1 --2- 172.21.15.60:0/2706054172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7015dd30 0x7f5d700b9230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5d680097a0 tx=0x7f5d6802a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26eb57bdae20020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d6f7fe640 1 -- 172.21.15.60:0/2706054172 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68037070 con 0x7f5d7015dd30 2024-04-25T08:37:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d6f7fe640 1 -- 172.21.15.60:0/2706054172 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6802ab30 con 0x7f5d7015dd30 2024-04-25T08:37:05.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d6f7fe640 1 -- 172.21.15.60:0/2706054172 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60001b70 con 0x7f5d7015dd30 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 -- 172.21.15.60:0/2706054172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7015dd30 msgr2=0x7f5d700b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 --2- 172.21.15.60:0/2706054172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7015dd30 0x7f5d700b9230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5d680097a0 tx=0x7f5d6802a0a0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 -- 172.21.15.60:0/2706054172 shutdown_connections 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 --2- 172.21.15.60:0/2706054172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d7015dd30 0x7f5d700b9230 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 -- 172.21.15.60:0/2706054172 >> 172.21.15.60:0/2706054172 conn(0x7f5d70061c30 msgr2=0x7f5d7006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 -- 172.21.15.60:0/2706054172 shutdown_connections 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.868+0000 7f5d77238640 1 -- 172.21.15.60:0/2706054172 wait complete. 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.869+0000 7f5d77238640 1 Processor -- start 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 -- start start 2024-04-25T08:37:05.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d700b9770 0x7f5d701f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d700d0ad0 0x7f5d700b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 0x7f5d700cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700d1b90 con 0x7f5d7015dd30 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700d1a10 con 0x7f5d700d0ad0 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.870+0000 7f5d77238640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d700d2e90 con 0x7f5d700b9770 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 0x7f5d700cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 0x7f5d700cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37156/0 (socket says 172.21.15.60:37156) 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 -- 172.21.15.60:0/2672452994 learned_addr learned my addr 172.21.15.60:0/2672452994 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d74fd4640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d700b9770 0x7f5d701f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d6ffff640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d700d0ad0 0x7f5d700b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 -- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d700b9770 msgr2=0x7f5d701f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d700b9770 0x7f5d701f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 -- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d700d0ad0 msgr2=0x7f5d700b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:05.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d700d0ad0 0x7f5d700b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 -- 172.21.15.60:0/2672452994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700cb990 con 0x7f5d7015dd30 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d757d5640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 0x7f5d700cd0b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5d58001eb0 tx=0x7f5d5800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58017070 con 0x7f5d7015dd30 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d580052d0 con 0x7f5d7015dd30 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58003810 con 0x7f5d7015dd30 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d700cbc70 con 0x7f5d7015dd30 2024-04-25T08:37:05.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.871+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d700cc230 con 0x7f5d7015dd30 2024-04-25T08:37:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.872+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d70073990 con 0x7f5d7015dd30 2024-04-25T08:37:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.873+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d5801a070 con 0x7f5d7015dd30 2024-04-25T08:37:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.873+0000 7f5d6d7fa640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d5006b4d0 0x7f5d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.873+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d58089710 con 0x7f5d7015dd30 2024-04-25T08:37:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.873+0000 7f5d74fd4640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d5006b4d0 0x7f5d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:05.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.874+0000 7f5d74fd4640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d5006b4d0 0x7f5d5006d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5d68034000 tx=0x7f5d6802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:05.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:05.878+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d58058400 con 0x7f5d7015dd30 2024-04-25T08:37:06.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.095+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d700d4100 con 0x7f5d7015dd30 2024-04-25T08:37:06.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.096+0000 7f5d6d7fa640 1 -- 172.21.15.60:0/2672452994 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5d58057d00 con 0x7f5d7015dd30 2024-04-25T08:37:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:06.091 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:06.091 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:06.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d5006b4d0 msgr2=0x7f5d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d5006b4d0 0x7f5d5006d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5d68034000 tx=0x7f5d6802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:06.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 msgr2=0x7f5d700cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 0x7f5d700cd0b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5d58001eb0 tx=0x7f5d5800c890 comp rx=0 tx=0).stop 2024-04-25T08:37:06.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 shutdown_connections 2024-04-25T08:37:06.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5d7015dd30 0x7f5d700cd0b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5d5006b4d0 0x7f5d5006d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d700d0ad0 0x7f5d700b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 --2- 172.21.15.60:0/2672452994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5d700b9770 0x7f5d701f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 >> 172.21.15.60:0/2672452994 conn(0x7f5d70061c30 msgr2=0x7f5d700c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:06.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 shutdown_connections 2024-04-25T08:37:06.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.098+0000 7f5d77238640 1 -- 172.21.15.60:0/2672452994 wait complete. 2024-04-25T08:37:06.103 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:06.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:06.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.210+0000 7f663accd640 1 Processor -- start 2024-04-25T08:37:06.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.210+0000 7f663accd640 1 -- start start 2024-04-25T08:37:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f663accd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f663415dbe0 0x7f66340d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f663accd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66340b2cb0 0x7f66340d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f663accd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f663accd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663415dfc0 con 0x7f663415dbe0 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f663accd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66340d1a00 con 0x7f66340b2cb0 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f663accd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b97a0 con 0x7f66340b92a0 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f663415dbe0 0x7f66340d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f663415dbe0 0x7f66340d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37168/0 (socket says 172.21.15.60:37168) 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 -- 172.21.15.60:0/3636929076 learned_addr learned my addr 172.21.15.60:0/3636929076 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 -- 172.21.15.60:0/3636929076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 msgr2=0x7f66340b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 --2- 172.21.15.60:0/3636929076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 -- 172.21.15.60:0/3636929076 >> v2:172.21.15.100:3300/0 conn(0x7f66340b2cb0 msgr2=0x7f66340d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 --2- 172.21.15.60:0/3636929076 >> v2:172.21.15.100:3300/0 conn(0x7f66340b2cb0 0x7f66340d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.211+0000 7f6638a69640 1 -- 172.21.15.60:0/3636929076 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340d1c50 con 0x7f663415dbe0 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f6638a69640 1 --2- 172.21.15.60:0/3636929076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f66340d0910 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f663406cff0 tx=0x7f662402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ede4a1c6c2ec586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f66337fe640 1 -- 172.21.15.60:0/3636929076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624037070 con 0x7f663415dbe0 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f66337fe640 1 -- 172.21.15.60:0/3636929076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624032440 con 0x7f663415dbe0 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f66337fe640 1 -- 172.21.15.60:0/3636929076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624032810 con 0x7f663415dbe0 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 -- 172.21.15.60:0/3636929076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 msgr2=0x7f66340d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 --2- 172.21.15.60:0/3636929076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f66340d0910 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f663406cff0 tx=0x7f662402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 -- 172.21.15.60:0/3636929076 shutdown_connections 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 --2- 172.21.15.60:0/3636929076 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 --2- 172.21.15.60:0/3636929076 >> v2:172.21.15.100:3300/0 conn(0x7f66340b2cb0 0x7f66340d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 --2- 172.21.15.60:0/3636929076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f66340d0910 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 -- 172.21.15.60:0/3636929076 >> 172.21.15.60:0/3636929076 conn(0x7f6634061a20 msgr2=0x7f6634061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 -- 172.21.15.60:0/3636929076 shutdown_connections 2024-04-25T08:37:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.212+0000 7f663accd640 1 -- 172.21.15.60:0/3636929076 wait complete. 2024-04-25T08:37:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.214+0000 7f663accd640 1 Processor -- start 2024-04-25T08:37:06.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.214+0000 7f663accd640 1 -- start start 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66340b2cb0 0x7f66340b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f663415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b6ab0 con 0x7f663415dbe0 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340d2210 con 0x7f66340b2cb0 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b6c30 con 0x7f66340b92a0 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f663415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f663415f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37182/0 (socket says 172.21.15.60:37182) 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 -- 172.21.15.60:0/4209577841 learned_addr learned my addr 172.21.15.60:0/4209577841 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f6633fff640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:06.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f6638a69640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66340b2cb0 0x7f66340b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 -- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 msgr2=0x7f66340c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 -- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66340b2cb0 msgr2=0x7f66340b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66340b2cb0 0x7f66340b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 -- 172.21.15.60:0/4209577841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340b32e0 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663926a640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f663415f1a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f662c00ba10 tx=0x7f662c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c019070 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00ca80 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66340b35c0 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c014440 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.215+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66340b3b80 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.216+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6634073990 con 0x7f663415dbe0 2024-04-25T08:37:06.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.217+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f662c01c070 con 0x7f663415dbe0 2024-04-25T08:37:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.217+0000 7f66317fa640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661406b4d0 0x7f661406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.217+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f662c089e50 con 0x7f663415dbe0 2024-04-25T08:37:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.217+0000 7f6638a69640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661406b4d0 0x7f661406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:06.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.218+0000 7f6638a69640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661406b4d0 0x7f661406d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6624034000 tx=0x7f662402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:06.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.222+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f662c0584d0 con 0x7f663415dbe0 2024-04-25T08:37:06.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.469+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6634160f60 con 0x7f663415dbe0 2024-04-25T08:37:06.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.470+0000 7f66317fa640 1 -- 172.21.15.60:0/4209577841 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f662c05c570 con 0x7f663415dbe0 2024-04-25T08:37:06.464 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:06.464 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280,"write_bytes_sec":44218230,"read_op_per_sec":0,"write_op_per_sec":43},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:37:03.504101+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.471+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661406b4d0 msgr2=0x7f661406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.471+0000 7f663accd640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661406b4d0 0x7f661406d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6624034000 tx=0x7f662402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.471+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 msgr2=0x7f663415f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.471+0000 7f663accd640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f663415f1a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f662c00ba10 tx=0x7f662c00be10 comp rx=0 tx=0).stop 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 shutdown_connections 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f663415dbe0 0x7f663415f1a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66340b92a0 0x7f66340c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f661406b4d0 0x7f661406d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 --2- 172.21.15.60:0/4209577841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f66340b2cb0 0x7f66340b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:06.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 >> 172.21.15.60:0/4209577841 conn(0x7f6634061a20 msgr2=0x7f663415b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:06.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 shutdown_connections 2024-04-25T08:37:06.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:06.472+0000 7f663accd640 1 -- 172.21.15.60:0/4209577841 wait complete. 2024-04-25T08:37:06.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T08:37:07.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:07.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.599+0000 7f74b32e1640 1 Processor -- start 2024-04-25T08:37:07.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.599+0000 7f74b32e1640 1 -- start start 2024-04-25T08:37:07.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b32e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74ac15dc20 0x7f74ac1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b32e1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f74ac0b93a0 0x7f74ac0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b32e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0cec20 0x7f74ac06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b32e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac15e000 con 0x7f74ac15dc20 2024-04-25T08:37:07.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b32e1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ac0b6bb0 con 0x7f74ac0b93a0 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b32e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0b6030 con 0x7f74ac0cec20 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74ac15dc20 0x7f74ac1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f74ac15dc20 0x7f74ac1616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37196/0 (socket says 172.21.15.60:37196) 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 -- 172.21.15.60:0/3041323067 learned_addr learned my addr 172.21.15.60:0/3041323067 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b187e640 1 --2- 172.21.15.60:0/3041323067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0cec20 0x7f74ac06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b087c640 1 --2- 172.21.15.60:0/3041323067 >> v2:172.21.15.100:3300/0 conn(0x7f74ac0b93a0 0x7f74ac0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 -- 172.21.15.60:0/3041323067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0cec20 msgr2=0x7f74ac06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 --2- 172.21.15.60:0/3041323067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0cec20 0x7f74ac06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 -- 172.21.15.60:0/3041323067 >> v2:172.21.15.100:3300/0 conn(0x7f74ac0b93a0 msgr2=0x7f74ac0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 --2- 172.21.15.60:0/3041323067 >> v2:172.21.15.100:3300/0 conn(0x7f74ac0b93a0 0x7f74ac0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.600+0000 7f74b107d640 1 -- 172.21.15.60:0/3041323067 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac161de0 con 0x7f74ac15dc20 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.601+0000 7f74b107d640 1 --2- 172.21.15.60:0/3041323067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac1616b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0d4b10 tx=0x7f74a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a4c4f6466e90da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.601+0000 7f74a3fff640 1 -- 172.21.15.60:0/3041323067 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8037070 con 0x7f74ac15dc20 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.601+0000 7f74a3fff640 1 -- 172.21.15.60:0/3041323067 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8032440 con 0x7f74ac15dc20 2024-04-25T08:37:07.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.601+0000 7f74a3fff640 1 -- 172.21.15.60:0/3041323067 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8032810 con 0x7f74ac15dc20 2024-04-25T08:37:07.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 -- 172.21.15.60:0/3041323067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 msgr2=0x7f74ac1616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 --2- 172.21.15.60:0/3041323067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac1616b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0d4b10 tx=0x7f74a802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 -- 172.21.15.60:0/3041323067 shutdown_connections 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 --2- 172.21.15.60:0/3041323067 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0cec20 0x7f74ac06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 --2- 172.21.15.60:0/3041323067 >> v2:172.21.15.100:3300/0 conn(0x7f74ac0b93a0 0x7f74ac0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 --2- 172.21.15.60:0/3041323067 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac1616b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 -- 172.21.15.60:0/3041323067 >> 172.21.15.60:0/3041323067 conn(0x7f74ac0d0490 msgr2=0x7f74ac0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 -- 172.21.15.60:0/3041323067 shutdown_connections 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.602+0000 7f74b32e1640 1 -- 172.21.15.60:0/3041323067 wait complete. 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.603+0000 7f74b32e1640 1 Processor -- start 2024-04-25T08:37:07.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.603+0000 7f74b32e1640 1 -- start start 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b32e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0b93a0 0x7f74ac160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b32e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ac0cec20 0x7f74ac156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b32e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b32e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac06c610 con 0x7f74ac15dc20 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b32e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac1623a0 con 0x7f74ac0cec20 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b32e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac06c790 con 0x7f74ac0b93a0 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac145f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37206/0 (socket says 172.21.15.60:37206) 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 -- 172.21.15.60:0/4048423132 learned_addr learned my addr 172.21.15.60:0/4048423132 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b107d640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0b93a0 0x7f74ac160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b087c640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ac0cec20 0x7f74ac156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 -- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0b93a0 msgr2=0x7f74ac160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0b93a0 0x7f74ac160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 -- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ac0cec20 msgr2=0x7f74ac156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ac0cec20 0x7f74ac156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b187e640 1 -- 172.21.15.60:0/4048423132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac1e80e0 con 0x7f74ac15dc20 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.604+0000 7f74b087c640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ac0cec20 0x7f74ac156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.605+0000 7f74b187e640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac145f00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f74a4007e40 tx=0x7f74a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.605+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4014620 con 0x7f74ac15dc20 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.605+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a4019070 con 0x7f74ac15dc20 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.605+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac1e83c0 con 0x7f74ac15dc20 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.605+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4014a20 con 0x7f74ac15dc20 2024-04-25T08:37:07.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.605+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac1e8980 con 0x7f74ac15dc20 2024-04-25T08:37:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.606+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac073990 con 0x7f74ac15dc20 2024-04-25T08:37:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.606+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74a4014bc0 con 0x7f74ac15dc20 2024-04-25T08:37:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.607+0000 7f74a1ffb640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f749006b4d0 0x7f749006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.607+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f74a408acf0 con 0x7f74ac15dc20 2024-04-25T08:37:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.607+0000 7f74b107d640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f749006b4d0 0x7f749006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.608+0000 7f74b107d640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f749006b4d0 0x7f749006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f74a8034000 tx=0x7f74a802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:07.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.611+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74a408f070 con 0x7f74ac15dc20 2024-04-25T08:37:07.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.832+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74ac1e8b50 con 0x7f74ac15dc20 2024-04-25T08:37:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.833+0000 7f74a1ffb640 1 -- 172.21.15.60:0/4048423132 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f74a40599c0 con 0x7f74ac15dc20 2024-04-25T08:37:07.827 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:07.827 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:07.827 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f749006b4d0 msgr2=0x7f749006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f749006b4d0 0x7f749006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f74a8034000 tx=0x7f74a802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 msgr2=0x7f74ac145f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac145f00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f74a4007e40 tx=0x7f74a400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 shutdown_connections 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f74ac15dc20 0x7f74ac145f00 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f749006b4d0 0x7f749006d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74ac0cec20 0x7f74ac156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 --2- 172.21.15.60:0/4048423132 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f74ac0b93a0 0x7f74ac160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 >> 172.21.15.60:0/4048423132 conn(0x7f74ac0d0490 msgr2=0x7f74ac0b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 shutdown_connections 2024-04-25T08:37:07.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.835+0000 7f74b32e1640 1 -- 172.21.15.60:0/4048423132 wait complete. 2024-04-25T08:37:07.840 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:07.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.945+0000 7ff69cfb0640 1 Processor -- start 2024-04-25T08:37:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.945+0000 7ff69cfb0640 1 -- start start 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69cfb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 0x7ff6980c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69cfb0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69815dfe0 con 0x7ff6980d1dd0 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69cfb0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6980d00c0 con 0x7ff6980ceb20 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69cfb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980d03a0 con 0x7ff69815dc00 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff696d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff6980d1dd0 0x7ff6980ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34658/0 (socket says 172.21.15.60:34658) 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff696d9d640 1 -- 172.21.15.60:0/259492998 learned_addr learned my addr 172.21.15.60:0/259492998 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69659c640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 0x7ff6980c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541611815 0 0) 0x7ff688001490 con 0x7ff6980d1dd0 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.946+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff684003520 con 0x7ff6980d1dd0 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128498982 0 0) 0x7ff6980d00c0 con 0x7ff6980ceb20 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff688001490 con 0x7ff6980ceb20 2024-04-25T08:37:07.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187485529 0 0) 0x7ff684003520 con 0x7ff6980d1dd0 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 msgr2=0x7ff6980c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 0x7ff6980c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 >> v1:172.21.15.100:6789/0 conn(0x7ff6980ceb20 legacy=0x7ff6980d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6980cc440 con 0x7ff6980d1dd0 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ff688001bc0 con 0x7ff6980d1dd0 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 >> v1:172.21.15.60:6789/0 conn(0x7ff6980d1dd0 legacy=0x7ff6980ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff684003b70 0x7ff684003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff684004800 0x7ff684004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 0x7ff6840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980d03a0 con 0x7ff684003b70 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68400ba70 con 0x7ff684004800 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68400bc40 con 0x7ff69815dc00 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69659c640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff684003b70 0x7ff684003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff695d9b640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff684004800 0x7ff684004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff696d9d640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 0x7ff6840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69659c640 1 -- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 msgr2=0x7ff6840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69659c640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff69815dc00 0x7ff6840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69659c640 1 -- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff684004800 msgr2=0x7ff684004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69659c640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff684004800 0x7ff684004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.947+0000 7ff69659c640 1 -- 172.21.15.60:0/259492998 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684003520 con 0x7ff684003b70 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.948+0000 7ff69659c640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff684003b70 0x7ff684003f70 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff690009ac0 tx=0x7ff684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527ab8cca424d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.948+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690036070 con 0x7ff684003b70 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.948+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff690031400 con 0x7ff684003b70 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.948+0000 7ff69559a640 1 -- 172.21.15.60:0/259492998 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690009690 con 0x7ff684003b70 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff69cfb0640 1 -- 172.21.15.60:0/259492998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff684003b70 msgr2=0x7ff684003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/259492998 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff684003b70 0x7ff684003f70 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff690009ac0 tx=0x7ff684005830 comp rx=0 tx=0).stop 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff696d9d640 1 -- 172.21.15.60:0/259492998 reap_dead start 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff69cfb0640 1 -- 172.21.15.60:0/259492998 shutdown_connections 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff69cfb0640 1 -- 172.21.15.60:0/259492998 >> 172.21.15.60:0/259492998 conn(0x7ff69806f5d0 msgr2=0x7ff69806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:07.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff69cfb0640 1 -- 172.21.15.60:0/259492998 shutdown_connections 2024-04-25T08:37:07.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.949+0000 7ff69cfb0640 1 -- 172.21.15.60:0/259492998 wait complete. 2024-04-25T08:37:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.950+0000 7ff69cfb0640 1 Processor -- start 2024-04-25T08:37:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.950+0000 7ff69cfb0640 1 -- start start 2024-04-25T08:37:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69cfb0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff684003b70 0x7ff698148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69cfb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 0x7ff6980cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69cfb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff698144fc0 0x7ff698145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69cfb0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980cfd90 con 0x7ff6980b44a0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69cfb0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980cc130 con 0x7ff698144fc0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69cfb0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6980cdd70 con 0x7ff684003b70 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 0x7ff6980cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 0x7ff6980cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37212/0 (socket says 172.21.15.60:37212) 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 -- 172.21.15.60:0/3450420129 learned_addr learned my addr 172.21.15.60:0/3450420129 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff69659c640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff684003b70 0x7ff698148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff696d9d640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff698144fc0 0x7ff698145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 -- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff684003b70 msgr2=0x7ff698148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff684003b70 0x7ff698148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 -- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff698144fc0 msgr2=0x7ff698145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff698144fc0 0x7ff698145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 -- 172.21.15.60:0/3450420129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698148dd0 con 0x7ff6980b44a0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.951+0000 7ff695d9b640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 0x7ff6980cb3c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff68000f7d0 tx=0x7ff68000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:07.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.952+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000fe80 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.952+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680018070 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.952+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6980b4a10 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.952+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000d6d0 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.952+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6980b6b20 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.953+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff698073990 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.953+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff68000d870 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.954+0000 7ff68effd640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff67806b500 0x7ff67806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.954+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff68008a150 con 0x7ff6980b44a0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.954+0000 7ff69659c640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff67806b500 0x7ff67806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:07.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.954+0000 7ff69659c640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff67806b500 0x7ff67806d9c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff690033000 tx=0x7ff690008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:07.958+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6980b4a10 con 0x7ff6980b44a0 2024-04-25T08:37:08.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.212+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff6980c66c0 con 0x7ff6980b44a0 2024-04-25T08:37:08.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.212+0000 7ff68effd640 1 -- 172.21.15.60:0/3450420129 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1593 (secure 0 0 0) 0x7ff698148dd0 con 0x7ff6980b44a0 2024-04-25T08:37:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:08.207 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":488,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280,"write_bytes_sec":4862,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:37:03.504101+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:08.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff67806b500 msgr2=0x7ff67806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff67806b500 0x7ff67806d9c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff690033000 tx=0x7ff690008030 comp rx=0 tx=0).stop 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 msgr2=0x7ff6980cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 0x7ff6980cb3c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff68000f7d0 tx=0x7ff68000fc90 comp rx=0 tx=0).stop 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 shutdown_connections 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff67806b500 0x7ff67806d9c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff698144fc0 0x7ff698145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6980b44a0 0x7ff6980cb3c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 --2- 172.21.15.60:0/3450420129 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff684003b70 0x7ff698148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 >> 172.21.15.60:0/3450420129 conn(0x7ff69806f5d0 msgr2=0x7ff6980b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 shutdown_connections 2024-04-25T08:37:08.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:08.214+0000 7ff69cfb0640 1 -- 172.21.15.60:0/3450420129 wait complete. 2024-04-25T08:37:08.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T08:37:09.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:09.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.341+0000 7fa724fe7640 1 Processor -- start 2024-04-25T08:37:09.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.341+0000 7fa724fe7640 1 -- start start 2024-04-25T08:37:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa724fe7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa72006ce10 0x7fa7200d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa724fe7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72006c220 con 0x7fa72015dc70 2024-04-25T08:37:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa724fe7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200d0080 con 0x7fa72009a450 2024-04-25T08:37:09.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa724fe7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7200d0c90 con 0x7fa72006ce10 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa717fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa72006ce10 0x7fa7200d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa71e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa72015dc70 0x7fa72006c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34670/0 (socket says 172.21.15.60:34670) 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa71e59c640 1 -- 172.21.15.60:0/3328099125 learned_addr learned my addr 172.21.15.60:0/3328099125 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa717fff640 1 -- 172.21.15.60:0/3328099125 >> v1:172.21.15.100:6789/0 conn(0x7fa72009a450 legacy=0x7fa7200cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa717fff640 1 -- 172.21.15.60:0/3328099125 >> v1:172.21.15.60:6789/0 conn(0x7fa72015dc70 legacy=0x7fa72006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa71dd9b640 1 -- 172.21.15.60:0/3328099125 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852244001 0 0) 0x7fa72006c220 con 0x7fa72015dc70 2024-04-25T08:37:09.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.342+0000 7fa717fff640 1 -- 172.21.15.60:0/3328099125 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200d0360 con 0x7fa72006ce10 2024-04-25T08:37:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.343+0000 7fa717fff640 1 --2- 172.21.15.60:0/3328099125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72006ce10 0x7fa7200d1120 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa7080097a0 tx=0x7fa70802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32106c5ae06d11d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.343+0000 7fa71dd9b640 1 -- 172.21.15.60:0/3328099125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708037070 con 0x7fa72006ce10 2024-04-25T08:37:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.343+0000 7fa71dd9b640 1 -- 172.21.15.60:0/3328099125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708032440 con 0x7fa72006ce10 2024-04-25T08:37:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.343+0000 7fa71dd9b640 1 -- 172.21.15.60:0/3328099125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708032810 con 0x7fa72006ce10 2024-04-25T08:37:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 -- 172.21.15.60:0/3328099125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72006ce10 msgr2=0x7fa7200d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3328099125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72006ce10 0x7fa7200d1120 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa7080097a0 tx=0x7fa70802a230 comp rx=0 tx=0).stop 2024-04-25T08:37:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 -- 172.21.15.60:0/3328099125 shutdown_connections 2024-04-25T08:37:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3328099125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72006ce10 0x7fa7200d1120 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 -- 172.21.15.60:0/3328099125 >> 172.21.15.60:0/3328099125 conn(0x7fa7200b3270 msgr2=0x7fa7200b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 -- 172.21.15.60:0/3328099125 shutdown_connections 2024-04-25T08:37:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.344+0000 7fa724fe7640 1 -- 172.21.15.60:0/3328099125 wait complete. 2024-04-25T08:37:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.345+0000 7fa724fe7640 1 Processor -- start 2024-04-25T08:37:09.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.345+0000 7fa724fe7640 1 -- start start 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa724fe7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72006ce10 0x7fa7201f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa724fe7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 0x7fa720070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa724fe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa72015dc70 0x7fa720070870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa724fe7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72015e050 con 0x7fa72009a450 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa724fe7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200d0920 con 0x7fa72015dc70 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa724fe7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200cf9c0 con 0x7fa72006ce10 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 0x7fa720070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 0x7fa720070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37232/0 (socket says 172.21.15.60:37232) 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 -- 172.21.15.60:0/3383079812 learned_addr learned my addr 172.21.15.60:0/3383079812 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:09.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa71e59c640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72006ce10 0x7fa7201f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa71ed9d640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa72015dc70 0x7fa720070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 -- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72006ce10 msgr2=0x7fa7201f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72006ce10 0x7fa7201f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 -- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa72015dc70 msgr2=0x7fa720070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa72015dc70 0x7fa720070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa717fff640 1 -- 172.21.15.60:0/3383079812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7201f4390 con 0x7fa72009a450 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.346+0000 7fa71ed9d640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa72015dc70 0x7fa720070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.347+0000 7fa717fff640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 0x7fa720070330 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa708035040 tx=0x7fa7080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.347+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708037050 con 0x7fa72009a450 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.347+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708038040 con 0x7fa72009a450 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.347+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708004470 con 0x7fa72009a450 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.347+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7201f4670 con 0x7fa72009a450 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.347+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7200cd0e0 con 0x7fa72009a450 2024-04-25T08:37:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.348+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa720073990 con 0x7fa72009a450 2024-04-25T08:37:09.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.348+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa708030880 con 0x7fa72009a450 2024-04-25T08:37:09.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.349+0000 7fa716ffd640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa70006b4d0 0x7fa70006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.349+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fa7080aafb0 con 0x7fa72009a450 2024-04-25T08:37:09.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.349+0000 7fa71e59c640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa70006b4d0 0x7fa70006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.349+0000 7fa71e59c640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa70006b4d0 0x7fa70006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa710000f60 tx=0x7fa710007e40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:09.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.353+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa708079c80 con 0x7fa72009a450 2024-04-25T08:37:09.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.570+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7200c9860 con 0x7fa72009a450 2024-04-25T08:37:09.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.571+0000 7fa716ffd640 1 -- 172.21.15.60:0/3383079812 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa708079580 con 0x7fa72009a450 2024-04-25T08:37:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:09.565 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:09.566 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa70006b4d0 msgr2=0x7fa70006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa70006b4d0 0x7fa70006d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa710000f60 tx=0x7fa710007e40 comp rx=0 tx=0).stop 2024-04-25T08:37:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 msgr2=0x7fa720070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 0x7fa720070330 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa708035040 tx=0x7fa7080046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 shutdown_connections 2024-04-25T08:37:09.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa70006b4d0 0x7fa70006d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa72015dc70 0x7fa720070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa72009a450 0x7fa720070330 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 --2- 172.21.15.60:0/3383079812 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa72006ce10 0x7fa7201f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 >> 172.21.15.60:0/3383079812 conn(0x7fa7200b3270 msgr2=0x7fa7200cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:09.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 shutdown_connections 2024-04-25T08:37:09.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.573+0000 7fa724fe7640 1 -- 172.21.15.60:0/3383079812 wait complete. 2024-04-25T08:37:09.578 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:09.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.684+0000 7f86e415c640 1 Processor -- start 2024-04-25T08:37:09.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.685+0000 7f86e415c640 1 -- start start 2024-04-25T08:37:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.685+0000 7f86e415c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f86dc0d2e90 0x7f86dc0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.685+0000 7f86e415c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0b9940 con 0x7f86dc0d0900 2024-04-25T08:37:09.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.685+0000 7f86e415c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0d1df0 con 0x7f86dc15dd30 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.685+0000 7f86e415c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc0d0d40 con 0x7f86dc0d2e90 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.685+0000 7f86e26f9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f86dc0d0900 0x7f86dc0b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34680/0 (socket says 172.21.15.60:34680) 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e26f9640 1 -- 172.21.15.60:0/2833905642 learned_addr learned my addr 172.21.15.60:0/2833905642 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e16f7640 1 --2- 172.21.15.60:0/2833905642 >> v2:172.21.15.100:3300/0 conn(0x7f86dc0d2e90 0x7f86dc0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995824762 0 0) 0x7f86dc0b9940 con 0x7f86dc0d0900 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c8003520 con 0x7f86dc0d0900 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443923321 0 0) 0x7f86dc0d1df0 con 0x7f86dc15dd30 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc0b9940 con 0x7f86dc15dd30 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e16f7640 1 -- 172.21.15.60:0/2833905642 >> v1:172.21.15.100:6789/0 conn(0x7f86dc15dd30 legacy=0x7f86dc0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e16f7640 1 -- 172.21.15.60:0/2833905642 >> v1:172.21.15.60:6789/0 conn(0x7f86dc0d0900 legacy=0x7f86dc0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.686+0000 7f86e16f7640 1 -- 172.21.15.60:0/2833905642 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0d20d0 con 0x7f86dc0d2e90 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e16f7640 1 --2- 172.21.15.60:0/2833905642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc0d2e90 0x7f86dc0d03c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009790 tx=0x7f86cc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d4a5e3185cc936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc037070 con 0x7f86dc0d2e90 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc02ae50 con 0x7f86dc0d2e90 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e0ef6640 1 -- 172.21.15.60:0/2833905642 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc032640 con 0x7f86dc0d2e90 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e415c640 1 -- 172.21.15.60:0/2833905642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc0d2e90 msgr2=0x7f86dc0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e415c640 1 --2- 172.21.15.60:0/2833905642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc0d2e90 0x7f86dc0d03c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009790 tx=0x7f86cc02a100 comp rx=0 tx=0).stop 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e415c640 1 -- 172.21.15.60:0/2833905642 shutdown_connections 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e415c640 1 --2- 172.21.15.60:0/2833905642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc0d2e90 0x7f86dc0d03c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.687+0000 7f86e415c640 1 -- 172.21.15.60:0/2833905642 >> 172.21.15.60:0/2833905642 conn(0x7f86dc061c30 msgr2=0x7f86dc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.688+0000 7f86e415c640 1 -- 172.21.15.60:0/2833905642 shutdown_connections 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.688+0000 7f86e415c640 1 -- 172.21.15.60:0/2833905642 wait complete. 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 Processor -- start 2024-04-25T08:37:09.683 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 -- start start 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86dc0d0900 0x7f86dc0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 0x7f86dc152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc15dd30 0x7f86dc0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0d2970 con 0x7f86dc0d2e90 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0d2690 con 0x7f86dc15dd30 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.689+0000 7f86e415c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0d1a10 con 0x7f86dc0d0900 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 0x7f86dc152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 0x7f86dc152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37246/0 (socket says 172.21.15.60:37246) 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 -- 172.21.15.60:0/2522913333 learned_addr learned my addr 172.21.15.60:0/2522913333 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e1ef8640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86dc0d0900 0x7f86dc0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e26f9640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc15dd30 0x7f86dc0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 -- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86dc0d0900 msgr2=0x7f86dc0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86dc0d0900 0x7f86dc0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 -- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc15dd30 msgr2=0x7f86dc0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc15dd30 0x7f86dc0b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 -- 172.21.15.60:0/2522913333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc15f7a0 con 0x7f86dc0d2e90 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e1ef8640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86dc0d0900 0x7f86dc0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e26f9640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc15dd30 0x7f86dc0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e16f7640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 0x7f86dc152a80 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f86cc02a140 tx=0x7f86cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc037040 con 0x7f86dc0d2e90 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc005990 con 0x7f86dc0d2e90 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc15fa80 con 0x7f86dc0d2e90 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc005d60 con 0x7f86dc0d2e90 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.690+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc160040 con 0x7f86dc0d2e90 2024-04-25T08:37:09.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.691+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc073990 con 0x7f86dc0d2e90 2024-04-25T08:37:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.692+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86cc03a070 con 0x7f86dc0d2e90 2024-04-25T08:37:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.692+0000 7f86d27fc640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86c006b500 0x7f86c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.692+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f86cc0aa4b0 con 0x7f86dc0d2e90 2024-04-25T08:37:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.693+0000 7f86e1ef8640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86c006b500 0x7f86c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:09.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.693+0000 7f86e1ef8640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86c006b500 0x7f86c006d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f86d80016b0 tx=0x7f86d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:09.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.696+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86cc04a070 con 0x7f86dc0d2e90 2024-04-25T08:37:09.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.948+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f86dc15d530 con 0x7f86dc0d2e90 2024-04-25T08:37:09.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.949+0000 7f86d27fc640 1 -- 172.21.15.60:0/2522913333 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1593 (secure 0 0 0) 0x7f86cc041020 con 0x7f86dc0d2e90 2024-04-25T08:37:09.943 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:09.943 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280,"write_bytes_sec":4862,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:37:03.504101+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.950+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86c006b500 msgr2=0x7f86c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.950+0000 7f86e415c640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86c006b500 0x7f86c006d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f86d80016b0 tx=0x7f86d8009040 comp rx=0 tx=0).stop 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.950+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 msgr2=0x7f86dc152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.950+0000 7f86e415c640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 0x7f86dc152a80 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f86cc02a140 tx=0x7f86cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 shutdown_connections 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f86c006b500 0x7f86c006d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f86dc15dd30 0x7f86dc0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f86dc0d2e90 0x7f86dc152a80 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 --2- 172.21.15.60:0/2522913333 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f86dc0d0900 0x7f86dc0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 >> 172.21.15.60:0/2522913333 conn(0x7f86dc061c30 msgr2=0x7f86dc0c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 shutdown_connections 2024-04-25T08:37:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:09.951+0000 7f86e415c640 1 -- 172.21.15.60:0/2522913333 wait complete. 2024-04-25T08:37:09.956 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T08:37:10.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:11.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.077+0000 7f1335709640 1 Processor -- start 2024-04-25T08:37:11.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.077+0000 7f1335709640 1 -- start start 2024-04-25T08:37:11.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.078+0000 7f1335709640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1330158220 0x7f133015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.078+0000 7f1335709640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133015f180 con 0x7f13300af720 2024-04-25T08:37:11.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.078+0000 7f1335709640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13301625d0 con 0x7f13300afba0 2024-04-25T08:37:11.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.078+0000 7f1335709640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13300b3670 con 0x7f1330158220 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.078+0000 7f132e7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f13300af720 0x7f13301617b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34696/0 (socket says 172.21.15.60:34696) 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.078+0000 7f132e7fc640 1 -- 172.21.15.60:0/1207158444 learned_addr learned my addr 172.21.15.60:0/1207158444 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132effd640 1 --2- 172.21.15.60:0/1207158444 >> v2:172.21.15.100:3300/0 conn(0x7f1330158220 0x7f133015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841770847 0 0) 0x7f1318001110 con 0x7f13300af720 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c003520 con 0x7f13300af720 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1386894243 0 0) 0x7f131c003520 con 0x7f13300af720 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 >> v2:172.21.15.100:3300/0 conn(0x7f1330158220 msgr2=0x7f133015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 --2- 172.21.15.60:0/1207158444 >> v2:172.21.15.100:3300/0 conn(0x7f1330158220 0x7f133015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 >> v1:172.21.15.100:6789/0 conn(0x7f13300afba0 legacy=0x7f1330161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133006fe00 con 0x7f13300af720 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1318001840 con 0x7f13300af720 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 >> v1:172.21.15.60:6789/0 conn(0x7f13300af720 legacy=0x7f13301617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131c0039b0 0x7f131c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f131c004800 0x7f131c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f131c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00c030 con 0x7f131c0051c0 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00c1e0 con 0x7f131c0039b0 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00c3e0 con 0x7f131c004800 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132f7fe640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f131c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132effd640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131c0039b0 0x7f131c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.079+0000 7f132e7fc640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f131c004800 0x7f131c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 -- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f131c004800 msgr2=0x7f131c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f131c004800 0x7f131c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 -- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131c0039b0 msgr2=0x7f131c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f131c0039b0 0x7f131c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 -- 172.21.15.60:0/1207158444 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c003520 con 0x7f131c0051c0 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 -- 172.21.15.60:0/1207158444 reap_dead start 2024-04-25T08:37:11.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132f7fe640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f131c00b9c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1320001e50 tx=0x7f131c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc545f075d07c129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320001a70 con 0x7f131c0051c0 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320003970 con 0x7f131c0051c0 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.080+0000 7f132dffb640 1 -- 172.21.15.60:0/1207158444 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320003d60 con 0x7f131c0051c0 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 -- 172.21.15.60:0/1207158444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 msgr2=0x7f131c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f131c00b9c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1320001e50 tx=0x7f131c005830 comp rx=0 tx=0).stop 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 -- 172.21.15.60:0/1207158444 shutdown_connections 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 --2- 172.21.15.60:0/1207158444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f131c00b9c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 -- 172.21.15.60:0/1207158444 >> 172.21.15.60:0/1207158444 conn(0x7f13300b2370 msgr2=0x7f13300b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 -- 172.21.15.60:0/1207158444 shutdown_connections 2024-04-25T08:37:11.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.081+0000 7f1335709640 1 -- 172.21.15.60:0/1207158444 wait complete. 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.082+0000 7f1335709640 1 Processor -- start 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 -- start start 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f1330158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13301f3c70 0x7f13300cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13300cffc0 0x7f133013e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13300703c0 con 0x7f131c0051c0 2024-04-25T08:37:11.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1330070a10 con 0x7f13301f3c70 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f1335709640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13300706a0 con 0x7f13300cffc0 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f132effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f1330158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f132effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f1330158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37262/0 (socket says 172.21.15.60:37262) 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.083+0000 7f132effd640 1 -- 172.21.15.60:0/994331645 learned_addr learned my addr 172.21.15.60:0/994331645 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f132effd640 1 -- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13300cffc0 msgr2=0x7f133013e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f132effd640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13300cffc0 0x7f133013e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f132effd640 1 -- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13301f3c70 msgr2=0x7f13300cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f132effd640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13301f3c70 0x7f13300cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f132effd640 1 -- 172.21.15.60:0/994331645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133015bb80 con 0x7f131c0051c0 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f132effd640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f1330158a60 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1328011910 tx=0x7f1328011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800faf0 con 0x7f131c0051c0 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132800fc90 con 0x7f131c0051c0 2024-04-25T08:37:11.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133015be20 con 0x7f131c0051c0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328014440 con 0x7f131c0051c0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.084+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133015c3e0 con 0x7f131c0051c0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.085+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1330073990 con 0x7f131c0051c0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.086+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1328012040 con 0x7f131c0051c0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.086+0000 7f13277fe640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f131006b500 0x7f131006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.086+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f132808a980 con 0x7f131c0051c0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.086+0000 7f132e7fc640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f131006b500 0x7f131006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.087+0000 7f132e7fc640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f131006b500 0x7f131006d9c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f13180012b0 tx=0x7f1318000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.084 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.090+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1328019030 con 0x7f131c0051c0 2024-04-25T08:37:11.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.309+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f133006baa0 con 0x7f131c0051c0 2024-04-25T08:37:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.309+0000 7f13277fe640 1 -- 172.21.15.60:0/994331645 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1328059650 con 0x7f131c0051c0 2024-04-25T08:37:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:11.304 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:11.304 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f131006b500 msgr2=0x7f131006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f131006b500 0x7f131006d9c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f13180012b0 tx=0x7f1318000ed0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 msgr2=0x7f1330158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f1330158a60 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1328011910 tx=0x7f1328011dd0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 shutdown_connections 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13300cffc0 0x7f133013e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f131006b500 0x7f131006d9c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13301f3c70 0x7f13300cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 --2- 172.21.15.60:0/994331645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f131c0051c0 0x7f1330158a60 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 >> 172.21.15.60:0/994331645 conn(0x7f13300b2370 msgr2=0x7f13300d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.311+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 shutdown_connections 2024-04-25T08:37:11.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.312+0000 7f1335709640 1 -- 172.21.15.60:0/994331645 wait complete. 2024-04-25T08:37:11.317 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:37:11.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.422+0000 7f0c1af5c640 1 Processor -- start 2024-04-25T08:37:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.422+0000 7f0c1af5c640 1 -- start start 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c1af5c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c14144750 0x7f0c1406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c1af5c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0c1406eb10 0x7f0c140ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c1af5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140cee60 0x7f0c1415f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c1af5c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c14144b30 con 0x7f0c14144750 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c1af5c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140712d0 con 0x7f0c1406eb10 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c1af5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140715b0 con 0x7f0c140cee60 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c18cf8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c14144750 0x7f0c1406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c18cf8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c14144750 0x7f0c1406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37274/0 (socket says 172.21.15.60:37274) 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.423+0000 7f0c18cf8640 1 -- 172.21.15.60:0/863993420 learned_addr learned my addr 172.21.15.60:0/863993420 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c18cf8640 1 -- 172.21.15.60:0/863993420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140cee60 msgr2=0x7f0c1415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c194f9640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140cee60 0x7f0c1415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c13fff640 1 --2- 172.21.15.60:0/863993420 >> v2:172.21.15.100:3300/0 conn(0x7f0c1406eb10 0x7f0c140ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c18cf8640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140cee60 0x7f0c1415f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c18cf8640 1 -- 172.21.15.60:0/863993420 >> v2:172.21.15.100:3300/0 conn(0x7f0c1406eb10 msgr2=0x7f0c140ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c18cf8640 1 --2- 172.21.15.60:0/863993420 >> v2:172.21.15.100:3300/0 conn(0x7f0c1406eb10 0x7f0c140ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c18cf8640 1 -- 172.21.15.60:0/863993420 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14071890 con 0x7f0c14144750 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c13fff640 1 --2- 172.21.15.60:0/863993420 >> v2:172.21.15.100:3300/0 conn(0x7f0c1406eb10 0x7f0c140ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c194f9640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140cee60 0x7f0c1415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.424+0000 7f0c18cf8640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c14144750 0x7f0c1406e5d0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f0c141606b0 tx=0x7f0c0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2da65a175584ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c137fe640 1 -- 172.21.15.60:0/863993420 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04037070 con 0x7f0c14144750 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c137fe640 1 -- 172.21.15.60:0/863993420 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04032440 con 0x7f0c14144750 2024-04-25T08:37:11.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c137fe640 1 -- 172.21.15.60:0/863993420 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04032810 con 0x7f0c14144750 2024-04-25T08:37:11.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 -- 172.21.15.60:0/863993420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c14144750 msgr2=0x7f0c1406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c14144750 0x7f0c1406e5d0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f0c141606b0 tx=0x7f0c0402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 -- 172.21.15.60:0/863993420 shutdown_connections 2024-04-25T08:37:11.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c140cee60 0x7f0c1415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/863993420 >> v2:172.21.15.100:3300/0 conn(0x7f0c1406eb10 0x7f0c140ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/863993420 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c14144750 0x7f0c1406e5d0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 -- 172.21.15.60:0/863993420 >> 172.21.15.60:0/863993420 conn(0x7f0c1406abe0 msgr2=0x7f0c1406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 -- 172.21.15.60:0/863993420 shutdown_connections 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.425+0000 7f0c1af5c640 1 -- 172.21.15.60:0/863993420 wait complete. 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.426+0000 7f0c1af5c640 1 Processor -- start 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.426+0000 7f0c1af5c640 1 -- start start 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c1af5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1406eb10 0x7f0c1413c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c1af5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 0x7f0c140ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c1af5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14144750 0x7f0c141f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c1af5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14145450 con 0x7f0c140cee60 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c1af5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1406f020 con 0x7f0c14144750 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c1af5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141455d0 con 0x7f0c1406eb10 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 0x7f0c140ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 0x7f0c140ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:37280/0 (socket says 172.21.15.60:37280) 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.427+0000 7f0c13fff640 1 -- 172.21.15.60:0/2061276646 learned_addr learned my addr 172.21.15.60:0/2061276646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c194f9640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14144750 0x7f0c141f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c18cf8640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1406eb10 0x7f0c1413c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c13fff640 1 -- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1406eb10 msgr2=0x7f0c1413c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c13fff640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1406eb10 0x7f0c1413c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c13fff640 1 -- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14144750 msgr2=0x7f0c141f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c13fff640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14144750 0x7f0c141f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c13fff640 1 -- 172.21.15.60:0/2061276646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140addc0 con 0x7f0c140cee60 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c194f9640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14144750 0x7f0c141f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c13fff640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 0x7f0c140ad2c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c7f0 tx=0x7f0c0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08015070 con 0x7f0c140cee60 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c08007820 con 0x7f0c140cee60 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c140ae0a0 con 0x7f0c140cee60 2024-04-25T08:37:11.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08007c10 con 0x7f0c140cee60 2024-04-25T08:37:11.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.428+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c140b2c80 con 0x7f0c140cee60 2024-04-25T08:37:11.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.429+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c14073990 con 0x7f0c140cee60 2024-04-25T08:37:11.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.430+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c08018070 con 0x7f0c140cee60 2024-04-25T08:37:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.430+0000 7f0c117fa640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0006b4d0 0x7f0c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.430+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0c0808a490 con 0x7f0c140cee60 2024-04-25T08:37:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.430+0000 7f0c18cf8640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0006b4d0 0x7f0c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.431+0000 7f0c18cf8640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0006b4d0 0x7f0c0006d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0c04034000 tx=0x7f0c0402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.434+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c08058a90 con 0x7f0c140cee60 2024-04-25T08:37:11.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.683+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0c1415b630 con 0x7f0c140cee60 2024-04-25T08:37:11.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.683+0000 7f0c117fa640 1 -- 172.21.15.60:0/2061276646 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1506 (secure 0 0 0) 0x7f0c08077080 con 0x7f0c140cee60 2024-04-25T08:37:11.677 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:11.677 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":112,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076559729,"bytes_used":2574385152,"bytes_avail":770519728128,"bytes_total":773094113280},"fsmap":{"epoch":62,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8184}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T08:37:09.505383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:11.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0006b4d0 msgr2=0x7f0c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0006b4d0 0x7f0c0006d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0c04034000 tx=0x7f0c0402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:11.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 msgr2=0x7f0c140ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 0x7f0c140ad2c0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c7f0 tx=0x7f0c0800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 shutdown_connections 2024-04-25T08:37:11.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c0006b4d0 0x7f0c0006d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c14144750 0x7f0c141f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c140cee60 0x7f0c140ad2c0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 --2- 172.21.15.60:0/2061276646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c1406eb10 0x7f0c1413c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 >> 172.21.15.60:0/2061276646 conn(0x7f0c1406abe0 msgr2=0x7f0c140b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:11.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 shutdown_connections 2024-04-25T08:37:11.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.685+0000 7f0c1af5c640 1 -- 172.21.15.60:0/2061276646 wait complete. 2024-04-25T08:37:11.690 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:11.690 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:37:11.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:37:11.728 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:37:11.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:37:11.773 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:11.773 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:11.791 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:11.791 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:11.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:11.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.993+0000 7fafb3a5a640 1 Processor -- start 2024-04-25T08:37:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.993+0000 7fafb3a5a640 1 -- start start 2024-04-25T08:37:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.994+0000 7fafb3a5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 0x7fafac162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.994+0000 7fafb3a5a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fafac0b5c30 0x7fafac0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.994+0000 7fafb3a5a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac162690 con 0x7fafac0b6cb0 2024-04-25T08:37:11.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.994+0000 7fafb3a5a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafac0ceee0 con 0x7fafac0b5c30 2024-04-25T08:37:11.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.994+0000 7fafb3a5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0d1d10 con 0x7fafac15db70 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb1ff7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fafac0b5c30 0x7fafac0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb17f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 0x7fafac162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb17f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 0x7fafac162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:34846/0 (socket says 172.21.15.60:34846) 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb17f6640 1 -- 172.21.15.60:0/3837492509 learned_addr learned my addr 172.21.15.60:0/3837492509 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb0ff5640 1 --1- 172.21.15.60:0/3837492509 >> v1:172.21.15.100:6789/0 conn(0x7fafac0b6cb0 0x7fafac0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:53564/0 (socket says 172.21.15.60:53564) 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb1ff7640 1 -- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 msgr2=0x7fafac162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb1ff7640 1 --2- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 0x7fafac162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb1ff7640 1 -- 172.21.15.60:0/3837492509 >> v1:172.21.15.100:6789/0 conn(0x7fafac0b6cb0 legacy=0x7fafac0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafa3fff640 1 -- 172.21.15.60:0/3837492509 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940797832 0 0) 0x7fafac162690 con 0x7fafac0b6cb0 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb1ff7640 1 -- 172.21.15.60:0/3837492509 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0d1ff0 con 0x7fafac0b5c30 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.995+0000 7fafb17f6640 1 --2- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 0x7fafac162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.996+0000 7fafb1ff7640 1 --2- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b5c30 0x7fafac0d27f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009d90 tx=0x7fafa402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99a587091df54e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.996+0000 7fafa3fff640 1 -- 172.21.15.60:0/3837492509 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4031600 con 0x7fafac0b5c30 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.996+0000 7fafa3fff640 1 -- 172.21.15.60:0/3837492509 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40317a0 con 0x7fafac0b5c30 2024-04-25T08:37:11.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.996+0000 7fafa3fff640 1 -- 172.21.15.60:0/3837492509 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4031b70 con 0x7fafac0b5c30 2024-04-25T08:37:11.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 -- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b5c30 msgr2=0x7fafac0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:11.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b5c30 0x7fafac0d27f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009d90 tx=0x7fafa402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 -- 172.21.15.60:0/3837492509 shutdown_connections 2024-04-25T08:37:11.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b5c30 0x7fafac0d27f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/3837492509 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac15db70 0x7fafac162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:11.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 -- 172.21.15.60:0/3837492509 >> 172.21.15.60:0/3837492509 conn(0x7fafac0d0520 msgr2=0x7fafac0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:11.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 -- 172.21.15.60:0/3837492509 shutdown_connections 2024-04-25T08:37:11.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.997+0000 7fafb3a5a640 1 -- 172.21.15.60:0/3837492509 wait complete. 2024-04-25T08:37:11.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.998+0000 7fafb3a5a640 1 Processor -- start 2024-04-25T08:37:11.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.998+0000 7fafb3a5a640 1 -- start start 2024-04-25T08:37:11.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb3a5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac0b5c30 0x7fafac145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb3a5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b6cb0 0x7fafac143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb3a5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 0x7fafac15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb3a5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0ce530 con 0x7fafac15db70 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb3a5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac15df50 con 0x7fafac0b6cb0 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb3a5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0cf1c0 con 0x7fafac0b5c30 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 0x7fafac15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 0x7fafac15c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33354/0 (socket says 172.21.15.60:33354) 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 -- 172.21.15.60:0/847685230 learned_addr learned my addr 172.21.15.60:0/847685230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb17f6640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac0b5c30 0x7fafac145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb0ff5640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b6cb0 0x7fafac143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 -- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac0b5c30 msgr2=0x7fafac145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac0b5c30 0x7fafac145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 -- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b6cb0 msgr2=0x7fafac143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b6cb0 0x7fafac143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 -- 172.21.15.60:0/847685230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac145960 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb17f6640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac0b5c30 0x7fafac145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:11.999+0000 7fafb1ff7640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 0x7fafac15c170 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009d90 tx=0x7fafa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.000+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4040070 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.000+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa4030b60 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.000+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac144380 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.000+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4009450 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.000+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac144940 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.001+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac073990 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.001+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafa4039040 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.002+0000 7fafa1ffb640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf9006b500 0x7faf9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.002+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fafa40ab6d0 con 0x7fafac15db70 2024-04-25T08:37:12.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.002+0000 7fafb17f6640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf9006b500 0x7faf9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.002+0000 7fafb17f6640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf9006b500 0x7faf9006d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009b80 tx=0x7fafa8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.006+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafa404be20 con 0x7fafac15db70 2024-04-25T08:37:12.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.188+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafac0d1820 con 0x7fafac15db70 2024-04-25T08:37:12.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.190+0000 7fafa1ffb640 1 -- 172.21.15.60:0/847685230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16921 (secure 0 0 0) 0x7fafa407a300 con 0x7fafac15db70 2024-04-25T08:37:12.184 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:07.134750+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf9006b500 msgr2=0x7faf9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf9006b500 0x7faf9006d9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009b80 tx=0x7fafa8008040 comp rx=0 tx=0).stop 2024-04-25T08:37:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 msgr2=0x7fafac15c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 0x7fafac15c170 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009d90 tx=0x7fafa4004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 shutdown_connections 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fafac15db70 0x7fafac15c170 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7faf9006b500 0x7faf9006d9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafac0b6cb0 0x7fafac143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 --2- 172.21.15.60:0/847685230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fafac0b5c30 0x7fafac145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 >> 172.21.15.60:0/847685230 conn(0x7fafac0d0520 msgr2=0x7fafac0d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 shutdown_connections 2024-04-25T08:37:12.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.192+0000 7fafb3a5a640 1 -- 172.21.15.60:0/847685230 wait complete. 2024-04-25T08:37:12.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:36:33.401380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:36:34.086710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:37:12.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:37:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.307+0000 7f2777770640 1 Processor -- start 2024-04-25T08:37:12.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.307+0000 7f2777770640 1 -- start start 2024-04-25T08:37:12.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2777770640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f277015b7d0 0x7f27700cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2777770640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277006cf50 con 0x7f277006c9e0 2024-04-25T08:37:12.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2777770640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27700ce530 con 0x7f27700b5620 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2777770640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700ce810 con 0x7f277015b7d0 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f277550c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f277015b7d0 0x7f27700cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2775d0d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f277006c9e0 0x7f27700d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42392/0 (socket says 172.21.15.60:42392) 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2775d0d640 1 -- 172.21.15.60:0/1173255733 learned_addr learned my addr 172.21.15.60:0/1173255733 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f277550c640 1 -- 172.21.15.60:0/1173255733 >> v1:172.21.15.100:6789/0 conn(0x7f27700b5620 legacy=0x7f277006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f277550c640 1 -- 172.21.15.60:0/1173255733 >> v1:172.21.15.60:6789/0 conn(0x7f277006c9e0 legacy=0x7f27700d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f277550c640 1 -- 172.21.15.60:0/1173255733 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700ceaf0 con 0x7f277015b7d0 2024-04-25T08:37:12.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.308+0000 7f2767fff640 1 -- 172.21.15.60:0/1173255733 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856185120 0 0) 0x7f277006cf50 con 0x7f277006c9e0 2024-04-25T08:37:12.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.309+0000 7f277550c640 1 --2- 172.21.15.60:0/1173255733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277015b7d0 0x7f27700cf010 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f276c0097a0 tx=0x7f276c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5149ea205261b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.309+0000 7f2767fff640 1 -- 172.21.15.60:0/1173255733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c037070 con 0x7f277015b7d0 2024-04-25T08:37:12.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.309+0000 7f2767fff640 1 -- 172.21.15.60:0/1173255733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c032440 con 0x7f277015b7d0 2024-04-25T08:37:12.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.309+0000 7f2767fff640 1 -- 172.21.15.60:0/1173255733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c032810 con 0x7f277015b7d0 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.309+0000 7f2777770640 1 -- 172.21.15.60:0/1173255733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277015b7d0 msgr2=0x7f27700cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.310+0000 7f2777770640 1 --2- 172.21.15.60:0/1173255733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277015b7d0 0x7f27700cf010 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f276c0097a0 tx=0x7f276c02a230 comp rx=0 tx=0).stop 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.310+0000 7f2777770640 1 -- 172.21.15.60:0/1173255733 shutdown_connections 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.310+0000 7f2777770640 1 --2- 172.21.15.60:0/1173255733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277015b7d0 0x7f27700cf010 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.310+0000 7f2777770640 1 -- 172.21.15.60:0/1173255733 >> 172.21.15.60:0/1173255733 conn(0x7f27700d0800 msgr2=0x7f27700d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.310+0000 7f2777770640 1 -- 172.21.15.60:0/1173255733 shutdown_connections 2024-04-25T08:37:12.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.310+0000 7f2777770640 1 -- 172.21.15.60:0/1173255733 wait complete. 2024-04-25T08:37:12.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.311+0000 7f2777770640 1 Processor -- start 2024-04-25T08:37:12.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.311+0000 7f2777770640 1 -- start start 2024-04-25T08:37:12.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.311+0000 7f2777770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 0x7f27700c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2777770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27700b5620 0x7f27700ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2777770640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277015b7d0 0x7f2770150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2777770640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27700cf6d0 con 0x7f277006c9e0 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2777770640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27700cf550 con 0x7f277015b7d0 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2777770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277015bbb0 con 0x7f27700b5620 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 0x7f27700c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 0x7f27700c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33370/0 (socket says 172.21.15.60:33370) 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 -- 172.21.15.60:0/1325003743 learned_addr learned my addr 172.21.15.60:0/1325003743 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2775d0d640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277015b7d0 0x7f2770150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2774d0b640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27700b5620 0x7f27700ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 -- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27700b5620 msgr2=0x7f27700ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27700b5620 0x7f27700ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 -- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277015b7d0 msgr2=0x7f2770150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277015b7d0 0x7f2770150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 -- 172.21.15.60:0/1325003743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700c7b50 con 0x7f277006c9e0 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2775d0d640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277015b7d0 0x7f2770150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2774d0b640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27700b5620 0x7f27700ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:12.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f277550c640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 0x7f27700c8bf0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098c0 tx=0x7f276c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c037040 con 0x7f277006c9e0 2024-04-25T08:37:12.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c042070 con 0x7f277006c9e0 2024-04-25T08:37:12.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27700c7e30 con 0x7f277006c9e0 2024-04-25T08:37:12.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.312+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c030a70 con 0x7f277006c9e0 2024-04-25T08:37:12.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.313+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27700c83f0 con 0x7f277006c9e0 2024-04-25T08:37:12.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.314+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2770073a00 con 0x7f277006c9e0 2024-04-25T08:37:12.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.314+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f276c030c10 con 0x7f277006c9e0 2024-04-25T08:37:12.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.314+0000 7f2765ffb640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f275406b4d0 0x7f275406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.314+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f276c0aaea0 con 0x7f277006c9e0 2024-04-25T08:37:12.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.315+0000 7f2774d0b640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f275406b4d0 0x7f275406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.315+0000 7f2774d0b640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f275406b4d0 0x7f275406d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f27700d2b50 tx=0x7f2760009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.319+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f276c045360 con 0x7f277006c9e0 2024-04-25T08:37:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.501+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27700d27c0 con 0x7f277006c9e0 2024-04-25T08:37:12.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.501+0000 7f2765ffb640 1 -- 172.21.15.60:0/1325003743 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f276c079b70 con 0x7f277006c9e0 2024-04-25T08:37:12.495 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:37:12.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f275406b4d0 msgr2=0x7f275406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f275406b4d0 0x7f275406d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f27700d2b50 tx=0x7f2760009040 comp rx=0 tx=0).stop 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 msgr2=0x7f27700c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 0x7f27700c8bf0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f276c0098c0 tx=0x7f276c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 shutdown_connections 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f275406b4d0 0x7f275406d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f277015b7d0 0x7f2770150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f27700b5620 0x7f27700ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 --2- 172.21.15.60:0/1325003743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f277006c9e0 0x7f27700c8bf0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 >> 172.21.15.60:0/1325003743 conn(0x7f27700d0800 msgr2=0x7f27700b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 shutdown_connections 2024-04-25T08:37:12.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.503+0000 7f2777770640 1 -- 172.21.15.60:0/1325003743 wait complete. 2024-04-25T08:37:12.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.618+0000 7fded606b640 1 Processor -- start 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.618+0000 7fded606b640 1 -- start start 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.618+0000 7fded606b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fded00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.618+0000 7fded606b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded00c75e0 con 0x7fded00c70d0 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.618+0000 7fded606b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded00c78c0 con 0x7fded00cfd10 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.618+0000 7fded606b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c8680 con 0x7fded015bb50 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdeceffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fded00c70d0 0x7fded00cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42394/0 (socket says 172.21.15.60:42394) 2024-04-25T08:37:12.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdeceffd640 1 -- 172.21.15.60:0/1493804818 learned_addr learned my addr 172.21.15.60:0/1493804818 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdecf7fe640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fded00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75675255 0 0) 0x7fded00c75e0 con 0x7fded00c70d0 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb8003520 con 0x7fded00c70d0 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852739816 0 0) 0x7fded00c78c0 con 0x7fded00cfd10 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded00c75e0 con 0x7fded00cfd10 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2436656430 0 0) 0x7fdeb8003520 con 0x7fded00c70d0 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 msgr2=0x7fded00c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fded00c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 >> v1:172.21.15.100:6789/0 conn(0x7fded00cfd10 legacy=0x7fded00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00c7ba0 con 0x7fded00c70d0 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fdec4001670 con 0x7fded00c70d0 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdecf7fe640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fded00c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 >> v1:172.21.15.60:6789/0 conn(0x7fded00c70d0 legacy=0x7fded00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fdeb8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80040b0 0x7fdeb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeb8004d40 0x7fdeb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c8680 con 0x7fdeb8004d40 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb800ba70 con 0x7fdeb80040b0 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb800bc70 con 0x7fded015bb50 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdecffff640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeb8004d40 0x7fdeb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdecf7fe640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fdeb8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.619+0000 7fdeceffd640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80040b0 0x7fdeb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdecffff640 1 -- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 msgr2=0x7fdeb8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdecffff640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fded015bb50 0x7fdeb8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdecffff640 1 -- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80040b0 msgr2=0x7fdeb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdecffff640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdeb80040b0 0x7fdeb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdecffff640 1 -- 172.21.15.60:0/1493804818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8003520 con 0x7fdeb8004d40 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdecffff640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeb8004d40 0x7fdeb80051c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdec8001fa0 tx=0x7fdeb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f07057d700483a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8001bf0 con 0x7fdeb8004d40 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8003b70 con 0x7fdeb8004d40 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.620+0000 7fdece7fc640 1 -- 172.21.15.60:0/1493804818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec80050d0 con 0x7fdeb8004d40 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fded606b640 1 -- 172.21.15.60:0/1493804818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeb8004d40 msgr2=0x7fdeb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fded606b640 1 --2- 172.21.15.60:0/1493804818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdeb8004d40 0x7fdeb80051c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fdec8001fa0 tx=0x7fdeb8005830 comp rx=0 tx=0).stop 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fdecffff640 1 -- 172.21.15.60:0/1493804818 reap_dead start 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fded606b640 1 -- 172.21.15.60:0/1493804818 shutdown_connections 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fded606b640 1 -- 172.21.15.60:0/1493804818 >> 172.21.15.60:0/1493804818 conn(0x7fded006f500 msgr2=0x7fded006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fded606b640 1 -- 172.21.15.60:0/1493804818 shutdown_connections 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.621+0000 7fded606b640 1 -- 172.21.15.60:0/1493804818 wait complete. 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.622+0000 7fded606b640 1 Processor -- start 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.622+0000 7fded606b640 1 -- start start 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fded606b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdeb8004d40 0x7fded01f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fded606b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded015c980 0x7fded01f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fded606b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 0x7fded0141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fded606b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded015f6d0 con 0x7fded0141960 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fded606b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c8160 con 0x7fded015c980 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fded606b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded015f850 con 0x7fdeb8004d40 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fdecffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 0x7fded0141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fdecffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 0x7fded0141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33388/0 (socket says 172.21.15.60:33388) 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.623+0000 7fdecffff640 1 -- 172.21.15.60:0/1051675204 learned_addr learned my addr 172.21.15.60:0/1051675204 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecffff640 1 -- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdeb8004d40 msgr2=0x7fded01f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdeceffd640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded015c980 0x7fded01f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecf7fe640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdeb8004d40 0x7fded01f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecffff640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdeb8004d40 0x7fded01f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecffff640 1 -- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded015c980 msgr2=0x7fded01f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecffff640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded015c980 0x7fded01f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecffff640 1 -- 172.21.15.60:0/1051675204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded015cf20 con 0x7fded0141960 2024-04-25T08:37:12.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdecffff640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 0x7fded0141e10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fdec8033040 tx=0x7fdec8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8001bf0 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8043070 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded015d200 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8005610 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.624+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded015d7c0 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.625+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded0073a00 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.625+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdec803a040 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.626+0000 7fdeb7fff640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdebc06b4d0 0x7fdebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.626+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fdec80aabf0 con 0x7fded0141960 2024-04-25T08:37:12.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.626+0000 7fdecf7fe640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdebc06b4d0 0x7fdebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.626+0000 7fdecf7fe640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdebc06b4d0 0x7fdebc06d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009a10 tx=0x7fdec0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.630+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdec80799e0 con 0x7fded0141960 2024-04-25T08:37:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.811+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fded0156580 con 0x7fded0141960 2024-04-25T08:37:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.811+0000 7fdeb7fff640 1 -- 172.21.15.60:0/1051675204 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdec80792e0 con 0x7fded0141960 2024-04-25T08:37:12.805 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:37:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdebc06b4d0 msgr2=0x7fdebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdebc06b4d0 0x7fdebc06d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009a10 tx=0x7fdec0008040 comp rx=0 tx=0).stop 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 msgr2=0x7fded0141e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 0x7fded0141e10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fdec8033040 tx=0x7fdec8001650 comp rx=0 tx=0).stop 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 shutdown_connections 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fded0141960 0x7fded0141e10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdebc06b4d0 0x7fdebc06d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fded015c980 0x7fded01f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 --2- 172.21.15.60:0/1051675204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdeb8004d40 0x7fded01f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 >> 172.21.15.60:0/1051675204 conn(0x7fded006f500 msgr2=0x7fded015f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 shutdown_connections 2024-04-25T08:37:12.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.813+0000 7fded606b640 1 -- 172.21.15.60:0/1051675204 wait complete. 2024-04-25T08:37:12.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:37:12.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.928+0000 7fdd69770640 1 Processor -- start 2024-04-25T08:37:12.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.928+0000 7fdd69770640 1 -- start start 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.928+0000 7fdd69770640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd64144d20 0x7fdd640ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.928+0000 7fdd69770640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdd641454b0 0x7fdd64070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd69770640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd640ceba0 con 0x7fdd64071300 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd69770640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640b4830 con 0x7fdd64144d20 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd69770640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd64071730 con 0x7fdd641454b0 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd62ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd64144d20 0x7fdd640ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd637fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdd64071300 0x7fdd6415f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42410/0 (socket says 172.21.15.60:42410) 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd62ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdd64144d20 0x7fdd640ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33402/0 (socket says 172.21.15.60:33402) 2024-04-25T08:37:12.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd637fe640 1 -- 172.21.15.60:0/4079341941 learned_addr learned my addr 172.21.15.60:0/4079341941 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd627fc640 1 --2- 172.21.15.60:0/4079341941 >> v2:172.21.15.100:3300/0 conn(0x7fdd641454b0 0x7fdd64070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd62ffd640 1 -- 172.21.15.60:0/4079341941 >> v2:172.21.15.100:3300/0 conn(0x7fdd641454b0 msgr2=0x7fdd64070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd62ffd640 1 --2- 172.21.15.60:0/4079341941 >> v2:172.21.15.100:3300/0 conn(0x7fdd641454b0 0x7fdd64070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd62ffd640 1 -- 172.21.15.60:0/4079341941 >> v1:172.21.15.60:6789/0 conn(0x7fdd64071300 legacy=0x7fdd6415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd62ffd640 1 -- 172.21.15.60:0/4079341941 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640cee80 con 0x7fdd64144d20 2024-04-25T08:37:12.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.929+0000 7fdd61ffb640 1 -- 172.21.15.60:0/4079341941 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145761550 0 0) 0x7fdd640ceba0 con 0x7fdd64071300 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd62ffd640 1 --2- 172.21.15.60:0/4079341941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640ce350 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdd64145940 tx=0x7fdd5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbdbc53663996f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd61ffb640 1 -- 172.21.15.60:0/4079341941 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54037070 con 0x7fdd64144d20 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd61ffb640 1 -- 172.21.15.60:0/4079341941 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54032440 con 0x7fdd64144d20 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd61ffb640 1 -- 172.21.15.60:0/4079341941 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54032810 con 0x7fdd64144d20 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 -- 172.21.15.60:0/4079341941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 msgr2=0x7fdd640ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 --2- 172.21.15.60:0/4079341941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640ce350 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdd64145940 tx=0x7fdd5402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 -- 172.21.15.60:0/4079341941 shutdown_connections 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 --2- 172.21.15.60:0/4079341941 >> v2:172.21.15.100:3300/0 conn(0x7fdd641454b0 0x7fdd64070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 --2- 172.21.15.60:0/4079341941 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640ce350 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 -- 172.21.15.60:0/4079341941 >> 172.21.15.60:0/4079341941 conn(0x7fdd640b3530 msgr2=0x7fdd640b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:12.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 -- 172.21.15.60:0/4079341941 shutdown_connections 2024-04-25T08:37:12.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.930+0000 7fdd69770640 1 -- 172.21.15.60:0/4079341941 wait complete. 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 Processor -- start 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 -- start start 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd64071300 0x7fdd640cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd641454b0 0x7fdd64141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640ce890 con 0x7fdd64144d20 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640cf440 con 0x7fdd64071300 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.932+0000 7fdd69770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd64145100 con 0x7fdd641454b0 2024-04-25T08:37:12.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33406/0 (socket says 172.21.15.60:33406) 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 -- 172.21.15.60:0/727862517 learned_addr learned my addr 172.21.15.60:0/727862517 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd637fe640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd641454b0 0x7fdd64141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd62ffd640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd64071300 0x7fdd640cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 -- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd641454b0 msgr2=0x7fdd64141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd641454b0 0x7fdd64141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 -- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd64071300 msgr2=0x7fdd640cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd64071300 0x7fdd640cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 -- 172.21.15.60:0/727862517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640cba30 con 0x7fdd64144d20 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd627fc640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640caae0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d800 tx=0x7fdd5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800b6c0 con 0x7fdd64144d20 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58019070 con 0x7fdd64144d20 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd640cbd10 con 0x7fdd64144d20 2024-04-25T08:37:12.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800ba80 con 0x7fdd64144d20 2024-04-25T08:37:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.933+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd640cc2d0 con 0x7fdd64144d20 2024-04-25T08:37:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.934+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd64072fc0 con 0x7fdd64144d20 2024-04-25T08:37:12.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.935+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd5800bc20 con 0x7fdd64144d20 2024-04-25T08:37:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.935+0000 7fdd4b7fe640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdd5006b4d0 0x7fdd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.935+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fdd5808a7e0 con 0x7fdd64144d20 2024-04-25T08:37:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.935+0000 7fdd62ffd640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdd5006b4d0 0x7fdd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:12.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.936+0000 7fdd62ffd640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdd5006b4d0 0x7fdd5006d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd54034000 tx=0x7fdd5402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:12.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:12.939+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd64072fc0 con 0x7fdd64144d20 2024-04-25T08:37:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.118+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd64073a00 con 0x7fdd64144d20 2024-04-25T08:37:13.112 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.118+0000 7fdd4b7fe640 1 -- 172.21.15.60:0/727862517 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd58059540 con 0x7fdd64144d20 2024-04-25T08:37:13.112 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:37:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdd5006b4d0 msgr2=0x7fdd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdd5006b4d0 0x7fdd5006d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd54034000 tx=0x7fdd5402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 msgr2=0x7fdd640caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640caae0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d800 tx=0x7fdd5800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 shutdown_connections 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdd641454b0 0x7fdd64141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdd64144d20 0x7fdd640caae0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdd5006b4d0 0x7fdd5006d990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 --2- 172.21.15.60:0/727862517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd64071300 0x7fdd640cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 >> 172.21.15.60:0/727862517 conn(0x7fdd640b3530 msgr2=0x7fdd640b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 shutdown_connections 2024-04-25T08:37:13.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.120+0000 7fdd69770640 1 -- 172.21.15.60:0/727862517 wait complete. 2024-04-25T08:37:13.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:37:13.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.234+0000 7f37faf14640 1 Processor -- start 2024-04-25T08:37:13.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.234+0000 7f37faf14640 1 -- start start 2024-04-25T08:37:13.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37faf14640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37f41610f0 0x7f37f406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37faf14640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f37f41616f0 0x7f37f4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37faf14640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f40c6960 con 0x7f37f40ce0a0 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37faf14640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f41614d0 con 0x7f37f41616f0 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37faf14640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f4162260 con 0x7f37f41610f0 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37f3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f37f41616f0 0x7f37f4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37f94b1640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f37f40ce0a0 0x7f37f40d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42420/0 (socket says 172.21.15.60:42420) 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.235+0000 7f37f94b1640 1 -- 172.21.15.60:0/1951962722 learned_addr learned my addr 172.21.15.60:0/1951962722 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f8cb0640 1 --2- 172.21.15.60:0/1951962722 >> v2:172.21.15.100:3300/0 conn(0x7f37f41610f0 0x7f37f406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f3fff640 1 -- 172.21.15.60:0/1951962722 >> v2:172.21.15.100:3300/0 conn(0x7f37f41610f0 msgr2=0x7f37f406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f3fff640 1 --2- 172.21.15.60:0/1951962722 >> v2:172.21.15.100:3300/0 conn(0x7f37f41610f0 0x7f37f406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f3fff640 1 -- 172.21.15.60:0/1951962722 >> v1:172.21.15.60:6789/0 conn(0x7f37f40ce0a0 legacy=0x7f37f40d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f37fe640 1 -- 172.21.15.60:0/1951962722 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154507008 0 0) 0x7f37f40c6960 con 0x7f37f40ce0a0 2024-04-25T08:37:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f3fff640 1 -- 172.21.15.60:0/1951962722 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f4162540 con 0x7f37f41616f0 2024-04-25T08:37:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.236+0000 7f37f8cb0640 1 --2- 172.21.15.60:0/1951962722 >> v2:172.21.15.100:3300/0 conn(0x7f37f41610f0 0x7f37f406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37f3fff640 1 --2- 172.21.15.60:0/1951962722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f41616f0 0x7f37f4161b50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f37f4071fb0 tx=0x7f37e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a938417b33c79952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37f37fe640 1 -- 172.21.15.60:0/1951962722 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0037070 con 0x7f37f41616f0 2024-04-25T08:37:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37f37fe640 1 -- 172.21.15.60:0/1951962722 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e0032440 con 0x7f37f41616f0 2024-04-25T08:37:13.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37f37fe640 1 -- 172.21.15.60:0/1951962722 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0032810 con 0x7f37f41616f0 2024-04-25T08:37:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 -- 172.21.15.60:0/1951962722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f41616f0 msgr2=0x7f37f4161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 --2- 172.21.15.60:0/1951962722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f41616f0 0x7f37f4161b50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f37f4071fb0 tx=0x7f37e002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 -- 172.21.15.60:0/1951962722 shutdown_connections 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 --2- 172.21.15.60:0/1951962722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f41616f0 0x7f37f4161b50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 --2- 172.21.15.60:0/1951962722 >> v2:172.21.15.100:3300/0 conn(0x7f37f41610f0 0x7f37f406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 -- 172.21.15.60:0/1951962722 >> 172.21.15.60:0/1951962722 conn(0x7f37f406f500 msgr2=0x7f37f406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 -- 172.21.15.60:0/1951962722 shutdown_connections 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.237+0000 7f37faf14640 1 -- 172.21.15.60:0/1951962722 wait complete. 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.238+0000 7f37faf14640 1 Processor -- start 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.238+0000 7f37faf14640 1 -- start start 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37faf14640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 0x7f37f40b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37faf14640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37f41610f0 0x7f37f4144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37faf14640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f41616f0 0x7f37f4147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37faf14640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40cf690 con 0x7f37f40ce0a0 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37faf14640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40cfce0 con 0x7f37f41616f0 2024-04-25T08:37:13.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37faf14640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f406c700 con 0x7f37f41610f0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37f8cb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 0x7f37f40b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37f8cb0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 0x7f37f40b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33420/0 (socket says 172.21.15.60:33420) 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.239+0000 7f37f8cb0640 1 -- 172.21.15.60:0/1961620775 learned_addr learned my addr 172.21.15.60:0/1961620775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f8cb0640 1 -- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37f41610f0 msgr2=0x7f37f4144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f8cb0640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37f41610f0 0x7f37f4144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f8cb0640 1 -- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f41616f0 msgr2=0x7f37f4147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f8cb0640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f41616f0 0x7f37f4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f8cb0640 1 -- 172.21.15.60:0/1961620775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f41449d0 con 0x7f37f40ce0a0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f8cb0640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 0x7f37f40b45d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c820 tx=0x7f37e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8015070 con 0x7f37f40ce0a0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8007a30 con 0x7f37f40ce0a0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f4147c70 con 0x7f37f40ce0a0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8007e00 con 0x7f37f40ce0a0 2024-04-25T08:37:13.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.240+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f40b6300 con 0x7f37f40ce0a0 2024-04-25T08:37:13.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.241+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f4073a00 con 0x7f37f40ce0a0 2024-04-25T08:37:13.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.241+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37e8018070 con 0x7f37f40ce0a0 2024-04-25T08:37:13.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.242+0000 7f37f17fa640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f37d806b500 0x7f37d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:13.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.242+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f37e808a340 con 0x7f37f40ce0a0 2024-04-25T08:37:13.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.242+0000 7f37f3fff640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f37d806b500 0x7f37d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:13.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.243+0000 7f37f3fff640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f37d806b500 0x7f37d806d9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f37e0034000 tx=0x7f37e002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:13.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.246+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37e8059010 con 0x7f37f40ce0a0 2024-04-25T08:37:13.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.442+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37f40d25a0 con 0x7f37f40ce0a0 2024-04-25T08:37:13.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.442+0000 7f37f17fa640 1 -- 172.21.15.60:0/1961620775 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7f37e8058910 con 0x7f37f40ce0a0 2024-04-25T08:37:13.436 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:37:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f37d806b500 msgr2=0x7f37d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f37d806b500 0x7f37d806d9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f37e0034000 tx=0x7f37e002c000 comp rx=0 tx=0).stop 2024-04-25T08:37:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 msgr2=0x7f37f40b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:13.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 0x7f37f40b45d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c820 tx=0x7f37e800cc20 comp rx=0 tx=0).stop 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 shutdown_connections 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f37d806b500 0x7f37d806d9c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37f41616f0 0x7f37f4147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f37f41610f0 0x7f37f4144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 --2- 172.21.15.60:0/1961620775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f37f40ce0a0 0x7f37f40b45d0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 >> 172.21.15.60:0/1961620775 conn(0x7f37f406f500 msgr2=0x7f37f40c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 shutdown_connections 2024-04-25T08:37:13.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:13.444+0000 7f37faf14640 1 -- 172.21.15.60:0/1961620775 wait complete. 2024-04-25T08:37:13.449 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:37:13.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:13.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:37:14.047 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:37:14.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:14.047 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:37:14.067 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:37:14.067 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:14.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:37:14.084 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:14.084 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:14.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:37:14.123 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:37:14.123 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:14.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:37:14.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:14.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.270+0000 7f4ad09b0640 1 Processor -- start 2024-04-25T08:37:14.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.270+0000 7f4ad09b0640 1 -- start start 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4ad09b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4acc071ac0 0x7f4acc0ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4ad09b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc15ec60 con 0x7f4acc154ed0 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4ad09b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4acc15ef40 con 0x7f4acc06c1d0 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4ad09b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4acc06db90 con 0x7f4acc071ac0 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4ac3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4acc071ac0 0x7f4acc0ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4aca59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4acc154ed0 0x7f4acc0aced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42436/0 (socket says 172.21.15.60:42436) 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.271+0000 7f4aca59c640 1 -- 172.21.15.60:0/1405006799 learned_addr learned my addr 172.21.15.60:0/1405006799 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.272+0000 7f4ac3fff640 1 -- 172.21.15.60:0/1405006799 >> v1:172.21.15.100:6789/0 conn(0x7f4acc06c1d0 legacy=0x7f4acc06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.272+0000 7f4ac3fff640 1 -- 172.21.15.60:0/1405006799 >> v1:172.21.15.60:6789/0 conn(0x7f4acc154ed0 legacy=0x7f4acc0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.272+0000 7f4ac3fff640 1 -- 172.21.15.60:0/1405006799 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc06de70 con 0x7f4acc071ac0 2024-04-25T08:37:14.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.272+0000 7f4ac3fff640 1 --2- 172.21.15.60:0/1405006799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc071ac0 0x7f4acc0ad5e0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11393c26445369f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.272+0000 7f4ac9d9b640 1 -- 172.21.15.60:0/1405006799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4037070 con 0x7f4acc071ac0 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ac9d9b640 1 -- 172.21.15.60:0/1405006799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab4032440 con 0x7f4acc071ac0 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ac9d9b640 1 -- 172.21.15.60:0/1405006799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4032810 con 0x7f4acc071ac0 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 -- 172.21.15.60:0/1405006799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc071ac0 msgr2=0x7f4acc0ad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/1405006799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc071ac0 0x7f4acc0ad5e0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 -- 172.21.15.60:0/1405006799 shutdown_connections 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/1405006799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc071ac0 0x7f4acc0ad5e0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 -- 172.21.15.60:0/1405006799 >> 172.21.15.60:0/1405006799 conn(0x7f4acc0af670 msgr2=0x7f4acc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 -- 172.21.15.60:0/1405006799 shutdown_connections 2024-04-25T08:37:14.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.273+0000 7f4ad09b0640 1 -- 172.21.15.60:0/1405006799 wait complete. 2024-04-25T08:37:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.274+0000 7f4ad09b0640 1 Processor -- start 2024-04-25T08:37:14.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 -- start start 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4acc071ac0 0x7f4acc144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4acc154ed0 0x7f4acc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 0x7f4acc1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc06b830 con 0x7f4acc0cea60 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc06b6b0 con 0x7f4acc071ac0 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ad09b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc06b9b0 con 0x7f4acc154ed0 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4acad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 0x7f4acc1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4acad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 0x7f4acc1f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33448/0 (socket says 172.21.15.60:33448) 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4acad9d640 1 -- 172.21.15.60:0/2623662600 learned_addr learned my addr 172.21.15.60:0/2623662600 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4aca59c640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4acc071ac0 0x7f4acc144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.275+0000 7f4ac3fff640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4acc154ed0 0x7f4acc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4acad9d640 1 -- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4acc154ed0 msgr2=0x7f4acc0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4acad9d640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4acc154ed0 0x7f4acc0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4acad9d640 1 -- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4acc071ac0 msgr2=0x7f4acc144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4acad9d640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4acc071ac0 0x7f4acc144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4acad9d640 1 -- 172.21.15.60:0/2623662600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc1541f0 con 0x7f4acc0cea60 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4acad9d640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 0x7f4acc1f3c70 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c790 tx=0x7f4ac400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4003b70 con 0x7f4acc0cea60 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4003d30 con 0x7f4acc0cea60 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc1544d0 con 0x7f4acc0cea60 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400aa30 con 0x7f4acc0cea60 2024-04-25T08:37:14.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.276+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc0ce4c0 con 0x7f4acc0cea60 2024-04-25T08:37:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.277+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc0b3c90 con 0x7f4acc0cea60 2024-04-25T08:37:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.278+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ac400abd0 con 0x7f4acc0cea60 2024-04-25T08:37:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.278+0000 7f4ac2ffd640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.278+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4ac408a0e0 con 0x7f4acc0cea60 2024-04-25T08:37:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.279+0000 7f4aca59c640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.279+0000 7f4aca59c640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4aac06b500 0x7f4aac06d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005880 tx=0x7f4abc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.283+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac40586d0 con 0x7f4acc0cea60 2024-04-25T08:37:14.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.502+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4acc0cc420 con 0x7f4acc0cea60 2024-04-25T08:37:14.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.502+0000 7f4ac2ffd640 1 -- 172.21.15.60:0/2623662600 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4acc0cc420 con 0x7f4acc0cea60 2024-04-25T08:37:14.497 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:14.497 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:14.497 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:14.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4aac06b500 msgr2=0x7f4aac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4aac06b500 0x7f4aac06d9c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005880 tx=0x7f4abc009040 comp rx=0 tx=0).stop 2024-04-25T08:37:14.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 msgr2=0x7f4acc1f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 0x7f4acc1f3c70 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c790 tx=0x7f4ac400cc50 comp rx=0 tx=0).stop 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 shutdown_connections 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4acc0cea60 0x7f4acc1f3c70 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4acc154ed0 0x7f4acc0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 --2- 172.21.15.60:0/2623662600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4acc071ac0 0x7f4acc144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 >> 172.21.15.60:0/2623662600 conn(0x7f4acc06c1d0 msgr2=0x7f4acc15fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 shutdown_connections 2024-04-25T08:37:14.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.504+0000 7f4ad09b0640 1 -- 172.21.15.60:0/2623662600 wait complete. 2024-04-25T08:37:14.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 Processor -- start 2024-04-25T08:37:14.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 -- start start 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9c4015db70 0x7f9c40162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c400b5a60 0x7f9c4006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c40162690 con 0x7f9c400b5a60 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c400ceee0 con 0x7f9c4015db70 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.617+0000 7f9c470e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400d1d10 con 0x7f9c400b6cb0 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c400b5a60 0x7f9c4006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9c400b5a60 0x7f9c4006c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33456/0 (socket says 172.21.15.60:33456) 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 -- 172.21.15.60:0/4194989198 learned_addr learned my addr 172.21.15.60:0/4194989198 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:14.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c44e7d640 1 --2- 172.21.15.60:0/4194989198 >> v2:172.21.15.100:3300/0 conn(0x7f9c4015db70 0x7f9c40162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 -- 172.21.15.60:0/4194989198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 msgr2=0x7f9c400b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c37fff640 1 --2- 172.21.15.60:0/4194989198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 --2- 172.21.15.60:0/4194989198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 -- 172.21.15.60:0/4194989198 >> v2:172.21.15.100:3300/0 conn(0x7f9c4015db70 msgr2=0x7f9c40162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 --2- 172.21.15.60:0/4194989198 >> v2:172.21.15.100:3300/0 conn(0x7f9c4015db70 0x7f9c40162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c4567e640 1 -- 172.21.15.60:0/4194989198 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400d1ff0 con 0x7f9c400b5a60 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.618+0000 7f9c44e7d640 1 --2- 172.21.15.60:0/4194989198 >> v2:172.21.15.100:3300/0 conn(0x7f9c4015db70 0x7f9c40162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c4567e640 1 --2- 172.21.15.60:0/4194989198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c4006c2d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b6570 tx=0x7f9c3802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63730bca9e4bb64a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c377fe640 1 -- 172.21.15.60:0/4194989198 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38036070 con 0x7f9c400b5a60 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c377fe640 1 -- 172.21.15.60:0/4194989198 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c38031600 con 0x7f9c400b5a60 2024-04-25T08:37:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c377fe640 1 -- 172.21.15.60:0/4194989198 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c380319d0 con 0x7f9c400b5a60 2024-04-25T08:37:14.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 -- 172.21.15.60:0/4194989198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 msgr2=0x7f9c4006c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 --2- 172.21.15.60:0/4194989198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c4006c2d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b6570 tx=0x7f9c3802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 -- 172.21.15.60:0/4194989198 shutdown_connections 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 --2- 172.21.15.60:0/4194989198 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c4006c2d0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 --2- 172.21.15.60:0/4194989198 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 --2- 172.21.15.60:0/4194989198 >> v2:172.21.15.100:3300/0 conn(0x7f9c4015db70 0x7f9c40162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 -- 172.21.15.60:0/4194989198 >> 172.21.15.60:0/4194989198 conn(0x7f9c400d0520 msgr2=0x7f9c400d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 -- 172.21.15.60:0/4194989198 shutdown_connections 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.619+0000 7f9c470e1640 1 -- 172.21.15.60:0/4194989198 wait complete. 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.620+0000 7f9c470e1640 1 Processor -- start 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 -- start start 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c40159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4015db70 0x7f9c4015f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400ce530 con 0x7f9c400b5a60 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4015df50 con 0x7f9c4015db70 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c470e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400cf1c0 con 0x7f9c400b6cb0 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.621+0000 7f9c44e7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c40159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c40159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33470/0 (socket says 172.21.15.60:33470) 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 -- 172.21.15.60:0/3204496951 learned_addr learned my addr 172.21.15.60:0/3204496951 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c4567e640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4015db70 0x7f9c4015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 -- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 msgr2=0x7f9c400cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c37fff640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 -- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4015db70 msgr2=0x7f9c4015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4015db70 0x7f9c4015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 -- 172.21.15.60:0/3204496951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c40159540 con 0x7f9c400b5a60 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c4567e640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4015db70 0x7f9c4015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c37fff640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:14.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c44e7d640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c40159000 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c7f0 tx=0x7f9c3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c015070 con 0x7f9c400b5a60 2024-04-25T08:37:14.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c007820 con 0x7f9c400b5a60 2024-04-25T08:37:14.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c007bd0 con 0x7f9c400b5a60 2024-04-25T08:37:14.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4015e170 con 0x7f9c400b5a60 2024-04-25T08:37:14.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.622+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4015ea10 con 0x7f9c400b5a60 2024-04-25T08:37:14.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.623+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c40073990 con 0x7f9c400b5a60 2024-04-25T08:37:14.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.624+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c3c018070 con 0x7f9c400b5a60 2024-04-25T08:37:14.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.624+0000 7f9c357fa640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c2c06b4d0 0x7f9c2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.624+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f9c3c08a530 con 0x7f9c400b5a60 2024-04-25T08:37:14.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.624+0000 7f9c37fff640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c2c06b4d0 0x7f9c2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.625+0000 7f9c37fff640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c2c06b4d0 0x7f9c2c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006fd0 tx=0x7f9c30008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.629+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c40159540 con 0x7f9c400b5a60 2024-04-25T08:37:14.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.813+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c400d1820 con 0x7f9c400b5a60 2024-04-25T08:37:14.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.814+0000 7f9c357fa640 1 -- 172.21.15.60:0/3204496951 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f9c3c0591f0 con 0x7f9c400b5a60 2024-04-25T08:37:14.808 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:14.809 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:07.134750+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c2c06b4d0 msgr2=0x7f9c2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c2c06b4d0 0x7f9c2c06d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9c30006fd0 tx=0x7f9c30008040 comp rx=0 tx=0).stop 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 msgr2=0x7f9c40159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c40159000 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c7f0 tx=0x7f9c3c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 shutdown_connections 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9c2c06b4d0 0x7f9c2c06d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c4015db70 0x7f9c4015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9c400b6cb0 0x7f9c400cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 --2- 172.21.15.60:0/3204496951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9c400b5a60 0x7f9c40159000 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 >> 172.21.15.60:0/3204496951 conn(0x7f9c400d0520 msgr2=0x7f9c400b8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 shutdown_connections 2024-04-25T08:37:14.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.816+0000 7f9c470e1640 1 -- 172.21.15.60:0/3204496951 wait complete. 2024-04-25T08:37:14.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:36:33.401380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:36:34.086710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:37:14.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:37:14.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 Processor -- start 2024-04-25T08:37:14.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 -- start start 2024-04-25T08:37:14.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f342c15bb20 0x7f342c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f342c0ccdd0 0x7f342c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0cb190 con 0x7f342c0cdcd0 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0cb470 con 0x7f342c0ccdd0 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.932+0000 7f3432b22640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0cb750 con 0x7f342c15bb20 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f342c0ccdd0 0x7f342c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f342c0ccdd0 0x7f342c0cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33472/0 (socket says 172.21.15.60:33472) 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 -- 172.21.15.60:0/3135001157 learned_addr learned my addr 172.21.15.60:0/3135001157 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f34308be640 1 --2- 172.21.15.60:0/3135001157 >> v2:172.21.15.100:3300/0 conn(0x7f342c15bb20 0x7f342c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 -- 172.21.15.60:0/3135001157 >> v2:172.21.15.100:3300/0 conn(0x7f342c15bb20 msgr2=0x7f342c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 --2- 172.21.15.60:0/3135001157 >> v2:172.21.15.100:3300/0 conn(0x7f342c15bb20 0x7f342c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 -- 172.21.15.60:0/3135001157 >> v1:172.21.15.100:6789/0 conn(0x7f342c0cdcd0 legacy=0x7f342c0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.933+0000 7f342bfff640 1 -- 172.21.15.60:0/3135001157 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0cba30 con 0x7f342c0ccdd0 2024-04-25T08:37:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f342bfff640 1 --2- 172.21.15.60:0/3135001157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c0ccdd0 0x7f342c0cf9d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f342c0d1800 tx=0x7f341802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a8b7cbc0d4ee21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f342b7fe640 1 -- 172.21.15.60:0/3135001157 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418037070 con 0x7f342c0ccdd0 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f342b7fe640 1 -- 172.21.15.60:0/3135001157 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418032440 con 0x7f342c0ccdd0 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f342b7fe640 1 -- 172.21.15.60:0/3135001157 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418032810 con 0x7f342c0ccdd0 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 -- 172.21.15.60:0/3135001157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c0ccdd0 msgr2=0x7f342c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 --2- 172.21.15.60:0/3135001157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c0ccdd0 0x7f342c0cf9d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f342c0d1800 tx=0x7f341802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 -- 172.21.15.60:0/3135001157 shutdown_connections 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 --2- 172.21.15.60:0/3135001157 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c0ccdd0 0x7f342c0cf9d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 --2- 172.21.15.60:0/3135001157 >> v2:172.21.15.100:3300/0 conn(0x7f342c15bb20 0x7f342c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 -- 172.21.15.60:0/3135001157 >> 172.21.15.60:0/3135001157 conn(0x7f342c0b2390 msgr2=0x7f342c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 -- 172.21.15.60:0/3135001157 shutdown_connections 2024-04-25T08:37:14.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.934+0000 7f3432b22640 1 -- 172.21.15.60:0/3135001157 wait complete. 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.935+0000 7f3432b22640 1 Processor -- start 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 -- start start 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f342c0ccdd0 0x7f342c155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f342c0cdcd0 0x7f342c0c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 0x7f342c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15bf00 con 0x7f342c15bb20 2024-04-25T08:37:14.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0cc770 con 0x7f342c0ccdd0 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.936+0000 7f3432b22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15c080 con 0x7f342c0cdcd0 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 0x7f342c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 0x7f342c0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33474/0 (socket says 172.21.15.60:33474) 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 -- 172.21.15.60:0/2254795513 learned_addr learned my addr 172.21.15.60:0/2254795513 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f342bfff640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f342c0cdcd0 0x7f342c0c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34308be640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f342c0ccdd0 0x7f342c155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 -- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f342c0cdcd0 msgr2=0x7f342c0c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f342c0cdcd0 0x7f342c0c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 -- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f342c0ccdd0 msgr2=0x7f342c155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f342c0ccdd0 0x7f342c155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 -- 172.21.15.60:0/2254795513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0b6210 con 0x7f342c15bb20 2024-04-25T08:37:14.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34310bf640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 0x7f342c0c5b40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f342400c780 tx=0x7f342400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424004b60 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3424004d00 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400aa90 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c0c6170 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.937+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c0c6730 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.938+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c073a00 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.939+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f342400ac30 con 0x7f342c15bb20 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.939+0000 7f34297fa640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f341006b4d0 0x7f341006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:14.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.939+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f342408a4a0 con 0x7f342c15bb20 2024-04-25T08:37:14.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.939+0000 7f34308be640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f341006b4d0 0x7f341006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:14.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.940+0000 7f34308be640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f341006b4d0 0x7f341006d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f342c0afc60 tx=0x7f3420008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:14.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:14.943+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3424020070 con 0x7f342c15bb20 2024-04-25T08:37:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.126+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f342c156940 con 0x7f342c15bb20 2024-04-25T08:37:15.120 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.126+0000 7f34297fa640 1 -- 172.21.15.60:0/2254795513 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3424058b20 con 0x7f342c15bb20 2024-04-25T08:37:15.121 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:37:15.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f341006b4d0 msgr2=0x7f341006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f341006b4d0 0x7f341006d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f342c0afc60 tx=0x7f3420008040 comp rx=0 tx=0).stop 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 msgr2=0x7f342c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 0x7f342c0c5b40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f342400c780 tx=0x7f342400cc40 comp rx=0 tx=0).stop 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 shutdown_connections 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f342c15bb20 0x7f342c0c5b40 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f342c0cdcd0 0x7f342c0c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f341006b4d0 0x7f341006d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 --2- 172.21.15.60:0/2254795513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f342c0ccdd0 0x7f342c155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 >> 172.21.15.60:0/2254795513 conn(0x7f342c0b2390 msgr2=0x7f342c15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 shutdown_connections 2024-04-25T08:37:15.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.128+0000 7f3432b22640 1 -- 172.21.15.60:0/2254795513 wait complete. 2024-04-25T08:37:15.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:37:15.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.243+0000 7f698bb09640 1 Processor -- start 2024-04-25T08:37:15.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.243+0000 7f698bb09640 1 -- start start 2024-04-25T08:37:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698bb09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698415dac0 0x7f69840d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698bb09640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69840d1f30 0x7f69840d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698bb09640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69840d34c0 0x7f69840d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698bb09640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840d3030 con 0x7f69840d34c0 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698bb09640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840d2420 con 0x7f69840d1f30 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698bb09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840d2700 con 0x7f698415dac0 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69840d34c0 0x7f69840d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f69840d34c0 0x7f69840d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33484/0 (socket says 172.21.15.60:33484) 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 -- 172.21.15.60:0/1307574785 learned_addr learned my addr 172.21.15.60:0/1307574785 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f69898a5640 1 --2- 172.21.15.60:0/1307574785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698415dac0 0x7f69840d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f69890a4640 1 --2- 172.21.15.60:0/1307574785 >> v2:172.21.15.100:3300/0 conn(0x7f69840d1f30 0x7f69840d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 -- 172.21.15.60:0/1307574785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698415dac0 msgr2=0x7f69840d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 --2- 172.21.15.60:0/1307574785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698415dac0 0x7f69840d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 -- 172.21.15.60:0/1307574785 >> v2:172.21.15.100:3300/0 conn(0x7f69840d1f30 msgr2=0x7f69840d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.244+0000 7f698a0a6640 1 --2- 172.21.15.60:0/1307574785 >> v2:172.21.15.100:3300/0 conn(0x7f69840d1f30 0x7f69840d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.245+0000 7f698a0a6640 1 -- 172.21.15.60:0/1307574785 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840d29e0 con 0x7f69840d34c0 2024-04-25T08:37:15.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.245+0000 7f698a0a6640 1 --2- 172.21.15.60:0/1307574785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840d08e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f69840b6550 tx=0x7f697402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e5a6dde4c7433d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.245+0000 7f69888a3640 1 -- 172.21.15.60:0/1307574785 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974036070 con 0x7f69840d34c0 2024-04-25T08:37:15.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.245+0000 7f69888a3640 1 -- 172.21.15.60:0/1307574785 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974031600 con 0x7f69840d34c0 2024-04-25T08:37:15.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.245+0000 7f69888a3640 1 -- 172.21.15.60:0/1307574785 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740319d0 con 0x7f69840d34c0 2024-04-25T08:37:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 -- 172.21.15.60:0/1307574785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 msgr2=0x7f69840d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 --2- 172.21.15.60:0/1307574785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840d08e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f69840b6550 tx=0x7f697402a730 comp rx=0 tx=0).stop 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 -- 172.21.15.60:0/1307574785 shutdown_connections 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 --2- 172.21.15.60:0/1307574785 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840d08e0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 --2- 172.21.15.60:0/1307574785 >> v2:172.21.15.100:3300/0 conn(0x7f69840d1f30 0x7f69840d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 --2- 172.21.15.60:0/1307574785 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f698415dac0 0x7f69840d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 -- 172.21.15.60:0/1307574785 >> 172.21.15.60:0/1307574785 conn(0x7f6984061c30 msgr2=0x7f698406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 -- 172.21.15.60:0/1307574785 shutdown_connections 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.246+0000 7f698bb09640 1 -- 172.21.15.60:0/1307574785 wait complete. 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.247+0000 7f698bb09640 1 Processor -- start 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.247+0000 7f698bb09640 1 -- start start 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698bb09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69840d1f30 0x7f6984146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698bb09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698bb09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698415dac0 0x7f6984152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698bb09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b97d0 con 0x7f69840d34c0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698bb09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698415dea0 con 0x7f698415dac0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698bb09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b9950 con 0x7f69840d1f30 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33498/0 (socket says 172.21.15.60:33498) 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 -- 172.21.15.60:0/815928839 learned_addr learned my addr 172.21.15.60:0/815928839 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69898a5640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69840d1f30 0x7f6984146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f698a0a6640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698415dac0 0x7f6984152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 -- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69840d1f30 msgr2=0x7f6984146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69840d1f30 0x7f6984146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 -- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698415dac0 msgr2=0x7f6984152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698415dac0 0x7f6984152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.248+0000 7f69890a4640 1 -- 172.21.15.60:0/815928839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840c8f20 con 0x7f69840d34c0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.249+0000 7f69890a4640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840caaa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f697800d7d0 tx=0x7f697800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.249+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800b6c0 con 0x7f69840d34c0 2024-04-25T08:37:15.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.249+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6978019070 con 0x7f69840d34c0 2024-04-25T08:37:15.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.249+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69840c9200 con 0x7f69840d34c0 2024-04-25T08:37:15.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.249+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800bac0 con 0x7f69840d34c0 2024-04-25T08:37:15.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.249+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69840c97c0 con 0x7f69840d34c0 2024-04-25T08:37:15.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.250+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6984073a00 con 0x7f69840d34c0 2024-04-25T08:37:15.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.250+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f697800bc60 con 0x7f69840d34c0 2024-04-25T08:37:15.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.251+0000 7f69827fc640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f697006b500 0x7f697006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.251+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f697808a680 con 0x7f69840d34c0 2024-04-25T08:37:15.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.251+0000 7f69898a5640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f697006b500 0x7f697006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.251+0000 7f69898a5640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f697006b500 0x7f697006d9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f697c007920 tx=0x7f697c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.255+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6978058d00 con 0x7f69840d34c0 2024-04-25T08:37:15.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.433+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69840b89a0 con 0x7f69840d34c0 2024-04-25T08:37:15.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.433+0000 7f69827fc640 1 -- 172.21.15.60:0/815928839 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6978088e20 con 0x7f69840d34c0 2024-04-25T08:37:15.427 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:37:15.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f697006b500 msgr2=0x7f697006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f697006b500 0x7f697006d9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f697c007920 tx=0x7f697c008040 comp rx=0 tx=0).stop 2024-04-25T08:37:15.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 msgr2=0x7f69840caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840caaa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f697800d7d0 tx=0x7f697800dc90 comp rx=0 tx=0).stop 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 shutdown_connections 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f697006b500 0x7f697006d9c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f698415dac0 0x7f6984152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69840d34c0 0x7f69840caaa0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 --2- 172.21.15.60:0/815928839 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69840d1f30 0x7f6984146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 >> 172.21.15.60:0/815928839 conn(0x7f6984061c30 msgr2=0x7f69840ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 shutdown_connections 2024-04-25T08:37:15.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.435+0000 7f698bb09640 1 -- 172.21.15.60:0/815928839 wait complete. 2024-04-25T08:37:15.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:37:15.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.549+0000 7f0d12083640 1 Processor -- start 2024-04-25T08:37:15.544 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.549+0000 7f0d12083640 1 -- start start 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d12083640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d0c1610f0 0x7f0d0c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d12083640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d0c0ce7b0 0x7f0d0c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d12083640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0c0c6960 con 0x7f0d0c1616f0 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d12083640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c1614d0 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d12083640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c162260 con 0x7f0d0c1610f0 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d0bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d0c0ce7b0 0x7f0d0c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d0bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0d0c0ce7b0 0x7f0d0c161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33506/0 (socket says 172.21.15.60:33506) 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.550+0000 7f0d0bfff640 1 -- 172.21.15.60:0/1848126504 learned_addr learned my addr 172.21.15.60:0/1848126504 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.551+0000 7f0d0b7fe640 1 --2- 172.21.15.60:0/1848126504 >> v2:172.21.15.100:3300/0 conn(0x7f0d0c1610f0 0x7f0d0c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.551+0000 7f0d0bfff640 1 -- 172.21.15.60:0/1848126504 >> v2:172.21.15.100:3300/0 conn(0x7f0d0c1610f0 msgr2=0x7f0d0c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:15.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.551+0000 7f0d0bfff640 1 --2- 172.21.15.60:0/1848126504 >> v2:172.21.15.100:3300/0 conn(0x7f0d0c1610f0 0x7f0d0c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.551+0000 7f0d0bfff640 1 -- 172.21.15.60:0/1848126504 >> v1:172.21.15.100:6789/0 conn(0x7f0d0c1616f0 legacy=0x7f0d0c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.551+0000 7f0d0bfff640 1 -- 172.21.15.60:0/1848126504 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c162540 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.551+0000 7f0d0b7fe640 1 --2- 172.21.15.60:0/1848126504 >> v2:172.21.15.100:3300/0 conn(0x7f0d0c1610f0 0x7f0d0c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d0bfff640 1 --2- 172.21.15.60:0/1848126504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c161b50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c071fb0 tx=0x7f0d0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b2f333e42e4f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d0a7fc640 1 -- 172.21.15.60:0/1848126504 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04036070 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d0a7fc640 1 -- 172.21.15.60:0/1848126504 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04031600 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d0a7fc640 1 -- 172.21.15.60:0/1848126504 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040319d0 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 -- 172.21.15.60:0/1848126504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 msgr2=0x7f0d0c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 --2- 172.21.15.60:0/1848126504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c161b50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c071fb0 tx=0x7f0d0402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 -- 172.21.15.60:0/1848126504 shutdown_connections 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 --2- 172.21.15.60:0/1848126504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c161b50 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 --2- 172.21.15.60:0/1848126504 >> v2:172.21.15.100:3300/0 conn(0x7f0d0c1610f0 0x7f0d0c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 -- 172.21.15.60:0/1848126504 >> 172.21.15.60:0/1848126504 conn(0x7f0d0c06f500 msgr2=0x7f0d0c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 -- 172.21.15.60:0/1848126504 shutdown_connections 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.552+0000 7f0d12083640 1 -- 172.21.15.60:0/1848126504 wait complete. 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.553+0000 7f0d12083640 1 Processor -- start 2024-04-25T08:37:15.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 -- start start 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c0b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d0c1610f0 0x7f0d0c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d0c1616f0 0x7f0d0c0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0cf690 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0cfce0 con 0x7f0d0c1610f0 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.554+0000 7f0d12083640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c06c700 con 0x7f0d0c1616f0 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c0b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c0b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33516/0 (socket says 172.21.15.60:33516) 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 -- 172.21.15.60:0/2577929180 learned_addr learned my addr 172.21.15.60:0/2577929180 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0affd640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d0c1610f0 0x7f0d0c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0bfff640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d0c1616f0 0x7f0d0c0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 -- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d0c1616f0 msgr2=0x7f0d0c0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d0c1616f0 0x7f0d0c0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 -- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d0c1610f0 msgr2=0x7f0d0c14d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d0c1610f0 0x7f0d0c14d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 -- 172.21.15.60:0/2577929180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c0b56c0 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0affd640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d0c1610f0 0x7f0d0c14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0bfff640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d0c1616f0 0x7f0d0c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d0b7fe640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c0b6760 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00c820 tx=0x7f0cfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc015070 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc007a30 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c0b59a0 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc007e00 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.555+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c0b5f60 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.556+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c0cef40 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.557+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cfc018070 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.557+0000 7f0cf3fff640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf806b4d0 0x7f0cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.557+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0cfc08a390 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.557+0000 7f0d0affd640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf806b4d0 0x7f0cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.558+0000 7f0d0affd640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf806b4d0 0x7f0cf806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0d00006fd0 tx=0x7f0d00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.561+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d0c0cef40 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.744+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d0c073a00 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.744+0000 7f0cf3fff640 1 -- 172.21.15.60:0/2577929180 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0cfc059060 con 0x7f0d0c0ce7b0 2024-04-25T08:37:15.738 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:37:15.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf806b4d0 msgr2=0x7f0cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf806b4d0 0x7f0cf806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0d00006fd0 tx=0x7f0d00008040 comp rx=0 tx=0).stop 2024-04-25T08:37:15.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 msgr2=0x7f0d0c0b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c0b6760 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00c820 tx=0x7f0cfc00cc20 comp rx=0 tx=0).stop 2024-04-25T08:37:15.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 shutdown_connections 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0d0c1616f0 0x7f0d0c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cf806b4d0 0x7f0cf806d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d0c1610f0 0x7f0d0c14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 --2- 172.21.15.60:0/2577929180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d0c0ce7b0 0x7f0d0c0b6760 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 >> 172.21.15.60:0/2577929180 conn(0x7f0d0c06f500 msgr2=0x7f0d0c0b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 shutdown_connections 2024-04-25T08:37:15.741 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.746+0000 7f0d12083640 1 -- 172.21.15.60:0/2577929180 wait complete. 2024-04-25T08:37:15.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:15.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.859+0000 7fa459f89640 1 Processor -- start 2024-04-25T08:37:15.854 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.859+0000 7fa459f89640 1 -- start start 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa459f89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45415dc70 0x7fa4540d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa459f89640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa45409a450 0x7fa4540cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa459f89640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45406c220 con 0x7fa45406c700 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa459f89640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45409a8b0 con 0x7fa45409a450 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa459f89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540d0c90 con 0x7fa45415dc70 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa452ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa45409a450 0x7fa4540cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa452ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa45409a450 0x7fa4540cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33524/0 (socket says 172.21.15.60:33524) 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.860+0000 7fa452ffd640 1 -- 172.21.15.60:0/3807378655 learned_addr learned my addr 172.21.15.60:0/3807378655 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.861+0000 7fa4537fe640 1 --2- 172.21.15.60:0/3807378655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45415dc70 0x7fa4540d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.861+0000 7fa452ffd640 1 -- 172.21.15.60:0/3807378655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45415dc70 msgr2=0x7fa4540d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.861+0000 7fa452ffd640 1 --2- 172.21.15.60:0/3807378655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45415dc70 0x7fa4540d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.861+0000 7fa452ffd640 1 -- 172.21.15.60:0/3807378655 >> v1:172.21.15.100:6789/0 conn(0x7fa45406c700 legacy=0x7fa4540cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.861+0000 7fa452ffd640 1 -- 172.21.15.60:0/3807378655 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4540d0080 con 0x7fa45409a450 2024-04-25T08:37:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa452ffd640 1 --2- 172.21.15.60:0/3807378655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45409a450 0x7fa4540cf9c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa448009870 tx=0x7fa44802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2b7302966d392c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa4527fc640 1 -- 172.21.15.60:0/3807378655 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448037070 con 0x7fa45409a450 2024-04-25T08:37:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa4527fc640 1 -- 172.21.15.60:0/3807378655 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448032440 con 0x7fa45409a450 2024-04-25T08:37:15.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa4527fc640 1 -- 172.21.15.60:0/3807378655 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448032810 con 0x7fa45409a450 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 -- 172.21.15.60:0/3807378655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45409a450 msgr2=0x7fa4540cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 --2- 172.21.15.60:0/3807378655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45409a450 0x7fa4540cf9c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa448009870 tx=0x7fa44802a270 comp rx=0 tx=0).stop 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 -- 172.21.15.60:0/3807378655 shutdown_connections 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 --2- 172.21.15.60:0/3807378655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45409a450 0x7fa4540cf9c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 --2- 172.21.15.60:0/3807378655 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45415dc70 0x7fa4540d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 -- 172.21.15.60:0/3807378655 >> 172.21.15.60:0/3807378655 conn(0x7fa4540b3270 msgr2=0x7fa4540b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 -- 172.21.15.60:0/3807378655 shutdown_connections 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.862+0000 7fa459f89640 1 -- 172.21.15.60:0/3807378655 wait complete. 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.863+0000 7fa459f89640 1 Processor -- start 2024-04-25T08:37:15.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 -- start start 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa45406c700 0x7fa4540c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45409a450 0x7fa454156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 0x7fa4541f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540d0920 con 0x7fa45415dc70 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4540d0640 con 0x7fa45406c700 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa459f89640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45415e050 con 0x7fa45409a450 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa453fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 0x7fa4541f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa453fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 0x7fa4541f4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33532/0 (socket says 172.21.15.60:33532) 2024-04-25T08:37:15.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.864+0000 7fa453fff640 1 -- 172.21.15.60:0/1001946149 learned_addr learned my addr 172.21.15.60:0/1001946149 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa4537fe640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa45406c700 0x7fa4540c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa452ffd640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45409a450 0x7fa454156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa453fff640 1 -- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45409a450 msgr2=0x7fa454156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa453fff640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45409a450 0x7fa454156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa453fff640 1 -- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa45406c700 msgr2=0x7fa4540c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa453fff640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa45406c700 0x7fa4540c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa453fff640 1 -- 172.21.15.60:0/1001946149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4541f5020 con 0x7fa45415dc70 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa4537fe640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa45406c700 0x7fa4540c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa453fff640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 0x7fa4541f4750 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c6b0 tx=0x7fa44c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c003b70 con 0x7fa45415dc70 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c003d30 con 0x7fa45415dc70 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4541f5300 con 0x7fa45415dc70 2024-04-25T08:37:15.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c005310 con 0x7fa45415dc70 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.865+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4541f58c0 con 0x7fa45415dc70 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.866+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa454073990 con 0x7fa45415dc70 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.867+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa44c0054b0 con 0x7fa45415dc70 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.867+0000 7fa43bfff640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa44006b4d0 0x7fa44006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.867+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fa44c08a1c0 con 0x7fa45415dc70 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.867+0000 7fa4537fe640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa44006b4d0 0x7fa44006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:15.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.868+0000 7fa4537fe640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa44006b4d0 0x7fa44006d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa444006fd0 tx=0x7fa444008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:15.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:15.871+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa44c058f20 con 0x7fa45415dc70 2024-04-25T08:37:16.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.091+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4540d1370 con 0x7fa45415dc70 2024-04-25T08:37:16.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.091+0000 7fa43bfff640 1 -- 172.21.15.60:0/1001946149 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa44c058820 con 0x7fa45415dc70 2024-04-25T08:37:16.087 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:16.087 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa44006b4d0 msgr2=0x7fa44006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa44006b4d0 0x7fa44006d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa444006fd0 tx=0x7fa444008040 comp rx=0 tx=0).stop 2024-04-25T08:37:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 msgr2=0x7fa4541f4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 0x7fa4541f4750 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c6b0 tx=0x7fa44c00cb70 comp rx=0 tx=0).stop 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 shutdown_connections 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa45415dc70 0x7fa4541f4750 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa45409a450 0x7fa454156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa44006b4d0 0x7fa44006d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 --2- 172.21.15.60:0/1001946149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa45406c700 0x7fa4540c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 >> 172.21.15.60:0/1001946149 conn(0x7fa4540b3270 msgr2=0x7fa454070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:16.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 shutdown_connections 2024-04-25T08:37:16.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.093+0000 7fa459f89640 1 -- 172.21.15.60:0/1001946149 wait complete. 2024-04-25T08:37:16.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:16.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.206+0000 7ff647ccd640 1 Processor -- start 2024-04-25T08:37:16.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.206+0000 7ff647ccd640 1 -- start start 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff647ccd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6400b2cb0 0x7ff6400d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff647ccd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400d1020 0x7ff6400d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff647ccd640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6400b97a0 con 0x7ff64015dbe0 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff647ccd640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64015dfc0 con 0x7ff6400b2cb0 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff647ccd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400d1a00 con 0x7ff6400d1020 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff645268640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff6400b2cb0 0x7ff6400d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff645a69640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff64015dbe0 0x7ff6400d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42448/0 (socket says 172.21.15.60:42448) 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.207+0000 7ff645a69640 1 -- 172.21.15.60:0/1474500712 learned_addr learned my addr 172.21.15.60:0/1474500712 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff64626a640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400d1020 0x7ff6400d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff645268640 1 -- 172.21.15.60:0/1474500712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400d1020 msgr2=0x7ff6400d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff645268640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400d1020 0x7ff6400d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff645268640 1 -- 172.21.15.60:0/1474500712 >> v1:172.21.15.60:6789/0 conn(0x7ff64015dbe0 legacy=0x7ff6400d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff645268640 1 -- 172.21.15.60:0/1474500712 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400d1c50 con 0x7ff6400b2cb0 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff644a67640 1 -- 172.21.15.60:0/1474500712 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876314169 0 0) 0x7ff6400b97a0 con 0x7ff64015dbe0 2024-04-25T08:37:16.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff64626a640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400d1020 0x7ff6400d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:16.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.208+0000 7ff645268640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6400b2cb0 0x7ff6400d2730 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff6300097a0 tx=0x7ff63002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663424eb39f89e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff644a67640 1 -- 172.21.15.60:0/1474500712 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630037070 con 0x7ff6400b2cb0 2024-04-25T08:37:16.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff644a67640 1 -- 172.21.15.60:0/1474500712 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff630032440 con 0x7ff6400b2cb0 2024-04-25T08:37:16.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff644a67640 1 -- 172.21.15.60:0/1474500712 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff630032810 con 0x7ff6400b2cb0 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 -- 172.21.15.60:0/1474500712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6400b2cb0 msgr2=0x7ff6400d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6400b2cb0 0x7ff6400d2730 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff6300097a0 tx=0x7ff63002a180 comp rx=0 tx=0).stop 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 -- 172.21.15.60:0/1474500712 shutdown_connections 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400d1020 0x7ff6400d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1474500712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6400b2cb0 0x7ff6400d2730 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 -- 172.21.15.60:0/1474500712 >> 172.21.15.60:0/1474500712 conn(0x7ff640061a20 msgr2=0x7ff640061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 -- 172.21.15.60:0/1474500712 shutdown_connections 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.209+0000 7ff647ccd640 1 -- 172.21.15.60:0/1474500712 wait complete. 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.210+0000 7ff647ccd640 1 Processor -- start 2024-04-25T08:37:16.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.210+0000 7ff647ccd640 1 -- start start 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff647ccd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400b2cb0 0x7ff64015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff647ccd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6400d1020 0x7ff6400cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff647ccd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 0x7ff6401f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff647ccd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400b92a0 con 0x7ff64015dbe0 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff647ccd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400d2210 con 0x7ff6400d1020 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff647ccd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400b9420 con 0x7ff6400b2cb0 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff64626a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 0x7ff6401f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff64626a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 0x7ff6401f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33552/0 (socket says 172.21.15.60:33552) 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff64626a640 1 -- 172.21.15.60:0/1743672069 learned_addr learned my addr 172.21.15.60:0/1743672069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff645a69640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400b2cb0 0x7ff64015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff64626a640 1 -- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400b2cb0 msgr2=0x7ff64015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.211+0000 7ff64626a640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400b2cb0 0x7ff64015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff64626a640 1 -- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6400d1020 msgr2=0x7ff6400cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff645268640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6400d1020 0x7ff6400cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff64626a640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6400d1020 0x7ff6400cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff64626a640 1 -- 172.21.15.60:0/1743672069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400cc410 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff645a69640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400b2cb0 0x7ff64015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff64626a640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 0x7ff6401f3730 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00ce80 tx=0x7ff63c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c01a070 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00d880 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6400cc6f0 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00dc70 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.212+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6400cc9d0 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.213+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff640073990 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.213+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff63c01d070 con 0x7ff64015dbe0 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.214+0000 7ff6367fc640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff62406b4d0 0x7ff62406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.214+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff63c08abe0 con 0x7ff64015dbe0 2024-04-25T08:37:16.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.214+0000 7ff645a69640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff62406b4d0 0x7ff62406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.215+0000 7ff645a69640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff62406b4d0 0x7ff62406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f60 tx=0x7ff638009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.212 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.218+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff63c0591b0 con 0x7ff64015dbe0 2024-04-25T08:37:16.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.398+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff640155350 con 0x7ff64015dbe0 2024-04-25T08:37:16.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.399+0000 7ff6367fc640 1 -- 172.21.15.60:0/1743672069 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16921 (secure 0 0 0) 0x7ff63c05d250 con 0x7ff64015dbe0 2024-04-25T08:37:16.394 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:16.394 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:07.134750+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:16.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff62406b4d0 msgr2=0x7ff62406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff62406b4d0 0x7ff62406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f60 tx=0x7ff638009040 comp rx=0 tx=0).stop 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 msgr2=0x7ff6401f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 0x7ff6401f3730 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00ce80 tx=0x7ff63c00a710 comp rx=0 tx=0).stop 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 shutdown_connections 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff64015dbe0 0x7ff6401f3730 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff62406b4d0 0x7ff62406d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6400d1020 0x7ff6400cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 --2- 172.21.15.60:0/1743672069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6400b2cb0 0x7ff64015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 >> 172.21.15.60:0/1743672069 conn(0x7ff640061a20 msgr2=0x7ff6400b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 shutdown_connections 2024-04-25T08:37:16.396 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.401+0000 7ff647ccd640 1 -- 172.21.15.60:0/1743672069 wait complete. 2024-04-25T08:37:16.407 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:37:16.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.514+0000 7fc44a3d5640 1 Processor -- start 2024-04-25T08:37:16.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.514+0000 7fc44a3d5640 1 -- start start 2024-04-25T08:37:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc44a3d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc44415dc00 0x7fc4440c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc44a3d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc4440ceb20 0x7fc4440d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc44a3d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44415dfe0 con 0x7fc4440d1c00 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc44a3d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440d2090 con 0x7fc4440ceb20 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc44a3d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440d00c0 con 0x7fc44415dc00 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc4440ceb20 0x7fc4440d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc4440ceb20 0x7fc4440d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33566/0 (socket says 172.21.15.60:33566) 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 -- 172.21.15.60:0/1284395867 learned_addr learned my addr 172.21.15.60:0/1284395867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 -- 172.21.15.60:0/1284395867 >> v2:172.21.15.100:3300/0 conn(0x7fc44415dc00 msgr2=0x7fc4440c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc443fff640 1 --2- 172.21.15.60:0/1284395867 >> v2:172.21.15.100:3300/0 conn(0x7fc44415dc00 0x7fc4440c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 --2- 172.21.15.60:0/1284395867 >> v2:172.21.15.100:3300/0 conn(0x7fc44415dc00 0x7fc4440c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 -- 172.21.15.60:0/1284395867 >> v1:172.21.15.100:6789/0 conn(0x7fc4440d1c00 legacy=0x7fc4440ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc4437fe640 1 -- 172.21.15.60:0/1284395867 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440d03a0 con 0x7fc4440ceb20 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.515+0000 7fc443fff640 1 --2- 172.21.15.60:0/1284395867 >> v2:172.21.15.100:3300/0 conn(0x7fc44415dc00 0x7fc4440c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.516+0000 7fc4437fe640 1 --2- 172.21.15.60:0/1284395867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4440d16c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc44406efe0 tx=0x7fc43802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac81124069a90648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.516+0000 7fc442ffd640 1 -- 172.21.15.60:0/1284395867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438037070 con 0x7fc4440ceb20 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.516+0000 7fc442ffd640 1 -- 172.21.15.60:0/1284395867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438032440 con 0x7fc4440ceb20 2024-04-25T08:37:16.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.516+0000 7fc442ffd640 1 -- 172.21.15.60:0/1284395867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438032810 con 0x7fc4440ceb20 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.516+0000 7fc44a3d5640 1 -- 172.21.15.60:0/1284395867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 msgr2=0x7fc4440d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.516+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/1284395867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4440d16c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc44406efe0 tx=0x7fc43802a230 comp rx=0 tx=0).stop 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.517+0000 7fc44a3d5640 1 -- 172.21.15.60:0/1284395867 shutdown_connections 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.517+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/1284395867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4440d16c0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.517+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/1284395867 >> v2:172.21.15.100:3300/0 conn(0x7fc44415dc00 0x7fc4440c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.517+0000 7fc44a3d5640 1 -- 172.21.15.60:0/1284395867 >> 172.21.15.60:0/1284395867 conn(0x7fc44406f5d0 msgr2=0x7fc44406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.517+0000 7fc44a3d5640 1 -- 172.21.15.60:0/1284395867 shutdown_connections 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.517+0000 7fc44a3d5640 1 -- 172.21.15.60:0/1284395867 wait complete. 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.518+0000 7fc44a3d5640 1 Processor -- start 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.518+0000 7fc44a3d5640 1 -- start start 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4441575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440d1c00 0x7fc4440b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44415dc00 0x7fc4440ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cc130 con 0x7fc4440ceb20 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cc720 con 0x7fc4440d1c00 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cfd90 con 0x7fc44415dc00 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4441575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4441575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33582/0 (socket says 172.21.15.60:33582) 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 -- 172.21.15.60:0/2480126131 learned_addr learned my addr 172.21.15.60:0/2480126131 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc4437fe640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440d1c00 0x7fc4440b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc448972640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44415dc00 0x7fc4440ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 -- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44415dc00 msgr2=0x7fc4440ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44415dc00 0x7fc4440ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 -- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440d1c00 msgr2=0x7fc4440b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440d1c00 0x7fc4440b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 -- 172.21.15.60:0/2480126131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4441e9080 con 0x7fc4440ceb20 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc443fff640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4441575a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c7f0 tx=0x7fc43400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434015070 con 0x7fc4440ceb20 2024-04-25T08:37:16.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4340079d0 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4441e9320 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.519+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434007da0 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.520+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4440b3080 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.521+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444073990 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.521+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc434018070 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.522+0000 7fc440ff9640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc43006b4d0 0x7fc43006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.522+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc43408a2f0 con 0x7fc4440ceb20 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.522+0000 7fc4437fe640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc43006b4d0 0x7fc43006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.522+0000 7fc4437fe640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc43006b4d0 0x7fc43006d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc438034000 tx=0x7fc43802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.526+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc434058fc0 con 0x7fc4440ceb20 2024-04-25T08:37:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.742+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4440c66c0 con 0x7fc4440ceb20 2024-04-25T08:37:16.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.742+0000 7fc440ff9640 1 -- 172.21.15.60:0/2480126131 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc4340588c0 con 0x7fc4440ceb20 2024-04-25T08:37:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:16.737 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:16.737 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:16.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc43006b4d0 msgr2=0x7fc43006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc43006b4d0 0x7fc43006d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc438034000 tx=0x7fc43802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:16.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 msgr2=0x7fc4441575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.739 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4441575a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c7f0 tx=0x7fc43400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 shutdown_connections 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc44415dc00 0x7fc4440ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc43006b4d0 0x7fc43006d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4440d1c00 0x7fc4440b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 --2- 172.21.15.60:0/2480126131 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4440ceb20 0x7fc4441575a0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 >> 172.21.15.60:0/2480126131 conn(0x7fc44406f5d0 msgr2=0x7fc4440ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 shutdown_connections 2024-04-25T08:37:16.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.744+0000 7fc44a3d5640 1 -- 172.21.15.60:0/2480126131 wait complete. 2024-04-25T08:37:16.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.856+0000 7f1063ce4640 1 Processor -- start 2024-04-25T08:37:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.856+0000 7f1063ce4640 1 -- start start 2024-04-25T08:37:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1063ce4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f105c0d11b0 0x7f105c168880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1063ce4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c15e810 con 0x7f105c15dd20 2024-04-25T08:37:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1063ce4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c15eac0 con 0x7f105c0b96d0 2024-04-25T08:37:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1063ce4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0d1c00 con 0x7f105c0d11b0 2024-04-25T08:37:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1062281640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f105c0d11b0 0x7f105c168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1061a80640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f105c15dd20 0x7f105c15e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42458/0 (socket says 172.21.15.60:42458) 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1061a80640 1 -- 172.21.15.60:0/347520584 learned_addr learned my addr 172.21.15.60:0/347520584 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1062281640 1 -- 172.21.15.60:0/347520584 >> v1:172.21.15.100:6789/0 conn(0x7f105c0b96d0 legacy=0x7f105c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1062281640 1 -- 172.21.15.60:0/347520584 >> v1:172.21.15.60:6789/0 conn(0x7f105c15dd20 legacy=0x7f105c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1060a7e640 1 -- 172.21.15.60:0/347520584 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065438060 0 0) 0x7f105c15e810 con 0x7f105c15dd20 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.857+0000 7f1062281640 1 -- 172.21.15.60:0/347520584 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c15eda0 con 0x7f105c0d11b0 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.858+0000 7f1062281640 1 --2- 172.21.15.60:0/347520584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0d11b0 0x7f105c168880 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1054009d90 tx=0x7f105402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e40bbff9f0fc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.858+0000 7f1060a7e640 1 -- 172.21.15.60:0/347520584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054007740 con 0x7f105c0d11b0 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.858+0000 7f1060a7e640 1 -- 172.21.15.60:0/347520584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540078e0 con 0x7f105c0d11b0 2024-04-25T08:37:16.853 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.858+0000 7f1060a7e640 1 -- 172.21.15.60:0/347520584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054007cb0 con 0x7f105c0d11b0 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 -- 172.21.15.60:0/347520584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0d11b0 msgr2=0x7f105c168880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 --2- 172.21.15.60:0/347520584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0d11b0 0x7f105c168880 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1054009d90 tx=0x7f105402a660 comp rx=0 tx=0).stop 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 -- 172.21.15.60:0/347520584 shutdown_connections 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 --2- 172.21.15.60:0/347520584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0d11b0 0x7f105c168880 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 -- 172.21.15.60:0/347520584 >> 172.21.15.60:0/347520584 conn(0x7f105c061a20 msgr2=0x7f105c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 -- 172.21.15.60:0/347520584 shutdown_connections 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.859+0000 7f1063ce4640 1 -- 172.21.15.60:0/347520584 wait complete. 2024-04-25T08:37:16.856 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.860+0000 7f1063ce4640 1 Processor -- start 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.860+0000 7f1063ce4640 1 -- start start 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1063ce4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 0x7f105c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1063ce4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c0d11b0 0x7f105c14d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1063ce4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15dd20 0x7f105c14edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1063ce4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0d3380 con 0x7f105c0b96d0 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1063ce4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0d1e50 con 0x7f105c0d11b0 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1063ce4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0d3500 con 0x7f105c15dd20 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 0x7f105c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 0x7f105c14cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33602/0 (socket says 172.21.15.60:33602) 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 -- 172.21.15.60:0/615625347 learned_addr learned my addr 172.21.15.60:0/615625347 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1062281640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15dd20 0x7f105c14edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f106127f640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c0d11b0 0x7f105c14d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 -- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15dd20 msgr2=0x7f105c14edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15dd20 0x7f105c14edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 -- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c0d11b0 msgr2=0x7f105c14d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c0d11b0 0x7f105c14d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 -- 172.21.15.60:0/615625347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c14dab0 con 0x7f105c0b96d0 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1062281640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15dd20 0x7f105c14edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f106127f640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c0d11b0 0x7f105c14d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.861+0000 7f1061a80640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 0x7f105c14cd80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f105800c3f0 tx=0x7f105800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.862+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10580074e0 con 0x7f105c0b96d0 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.862+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10580047e0 con 0x7f105c0b96d0 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.862+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c14dd30 con 0x7f105c0b96d0 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.862+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058004c10 con 0x7f105c0b96d0 2024-04-25T08:37:16.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.862+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c14e2f0 con 0x7f105c0b96d0 2024-04-25T08:37:16.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.863+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c161250 con 0x7f105c0b96d0 2024-04-25T08:37:16.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.863+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1058011070 con 0x7f105c0b96d0 2024-04-25T08:37:16.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.864+0000 7f10527fc640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f104006b4d0 0x7f104006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:16.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.864+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f1058089fb0 con 0x7f105c0b96d0 2024-04-25T08:37:16.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.864+0000 7f106127f640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f104006b4d0 0x7f104006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:16.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.864+0000 7f106127f640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f104006b4d0 0x7f104006d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f104c007fd0 tx=0x7f104c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:16.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:16.868+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f105c161250 con 0x7f105c0b96d0 2024-04-25T08:37:17.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.088+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f105c073990 con 0x7f105c0b96d0 2024-04-25T08:37:17.083 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.089+0000 7f10527fc640 1 -- 172.21.15.60:0/615625347 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f105c14dab0 con 0x7f105c0b96d0 2024-04-25T08:37:17.083 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:17.083 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:35.118687+0000","modified":"2024-04-25T08:36:37.111723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8184":{"gid":8184,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6835/4168808734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":4168808734},{"type":"v1","addr":"172.21.15.60:6835","nonce":4168808734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:37:17.084 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 62 2024-04-25T08:37:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f104006b4d0 msgr2=0x7f104006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f104006b4d0 0x7f104006d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f104c007fd0 tx=0x7f104c009040 comp rx=0 tx=0).stop 2024-04-25T08:37:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 msgr2=0x7f105c14cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 0x7f105c14cd80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f105800c3f0 tx=0x7f105800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 shutdown_connections 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f105c15dd20 0x7f105c14edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f104006b4d0 0x7f104006d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f105c0d11b0 0x7f105c14d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 --2- 172.21.15.60:0/615625347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f105c0b96d0 0x7f105c14cd80 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 >> 172.21.15.60:0/615625347 conn(0x7f105c061a20 msgr2=0x7f105c061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 shutdown_connections 2024-04-25T08:37:17.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.091+0000 7f1063ce4640 1 -- 172.21.15.60:0/615625347 wait complete. 2024-04-25T08:37:17.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:17.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.204+0000 7f50daacb640 1 Processor -- start 2024-04-25T08:37:17.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.204+0000 7f50daacb640 1 -- start start 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50daacb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50d40b2cb0 0x7f50d40d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50daacb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f50d40d1020 0x7f50d40d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50daacb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d40b97a0 con 0x7f50d415dbe0 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50daacb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d415dfc0 con 0x7f50d40b2cb0 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50daacb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d40d1a00 con 0x7f50d40d1020 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d3fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f50d40b2cb0 0x7f50d40d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d8867640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f50d415dbe0 0x7f50d40d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42466/0 (socket says 172.21.15.60:42466) 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d8867640 1 -- 172.21.15.60:0/830525330 learned_addr learned my addr 172.21.15.60:0/830525330 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d3fff640 1 -- 172.21.15.60:0/830525330 >> v2:172.21.15.100:3300/0 conn(0x7f50d40d1020 msgr2=0x7f50d40d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d3fff640 1 --2- 172.21.15.60:0/830525330 >> v2:172.21.15.100:3300/0 conn(0x7f50d40d1020 0x7f50d40d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d3fff640 1 -- 172.21.15.60:0/830525330 >> v1:172.21.15.60:6789/0 conn(0x7f50d415dbe0 legacy=0x7f50d40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d37fe640 1 -- 172.21.15.60:0/830525330 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439908537 0 0) 0x7f50d40b97a0 con 0x7f50d415dbe0 2024-04-25T08:37:17.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.205+0000 7f50d3fff640 1 -- 172.21.15.60:0/830525330 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40d1c50 con 0x7f50d40b2cb0 2024-04-25T08:37:17.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.206+0000 7f50d3fff640 1 --2- 172.21.15.60:0/830525330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40b2cb0 0x7f50d40d2730 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f50d40cf570 tx=0x7f50c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f972782dae9c683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.206+0000 7f50d37fe640 1 -- 172.21.15.60:0/830525330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037070 con 0x7f50d40b2cb0 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.206+0000 7f50d37fe640 1 -- 172.21.15.60:0/830525330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8032440 con 0x7f50d40b2cb0 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.206+0000 7f50d37fe640 1 -- 172.21.15.60:0/830525330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8032810 con 0x7f50d40b2cb0 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 -- 172.21.15.60:0/830525330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40b2cb0 msgr2=0x7f50d40d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 --2- 172.21.15.60:0/830525330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40b2cb0 0x7f50d40d2730 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f50d40cf570 tx=0x7f50c802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 -- 172.21.15.60:0/830525330 shutdown_connections 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 --2- 172.21.15.60:0/830525330 >> v2:172.21.15.100:3300/0 conn(0x7f50d40d1020 0x7f50d40d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 --2- 172.21.15.60:0/830525330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40b2cb0 0x7f50d40d2730 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 -- 172.21.15.60:0/830525330 >> 172.21.15.60:0/830525330 conn(0x7f50d4061a20 msgr2=0x7f50d4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 -- 172.21.15.60:0/830525330 shutdown_connections 2024-04-25T08:37:17.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.207+0000 7f50daacb640 1 -- 172.21.15.60:0/830525330 wait complete. 2024-04-25T08:37:17.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.208+0000 7f50daacb640 1 Processor -- start 2024-04-25T08:37:17.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.208+0000 7f50daacb640 1 -- start start 2024-04-25T08:37:17.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50daacb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d40b2cb0 0x7f50d40b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50daacb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 0x7f50d40b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50daacb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50d415dbe0 0x7f50d4150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50daacb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b92a0 con 0x7f50d40d1020 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50daacb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40d2210 con 0x7f50d40b2cb0 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50daacb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d40b9420 con 0x7f50d415dbe0 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 0x7f50d40b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 0x7f50d40b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33614/0 (socket says 172.21.15.60:33614) 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 -- 172.21.15.60:0/1643551700 learned_addr learned my addr 172.21.15.60:0/1643551700 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d8867640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d40b2cb0 0x7f50d40b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d9068640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50d415dbe0 0x7f50d4150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 -- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50d415dbe0 msgr2=0x7f50d4150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50d415dbe0 0x7f50d4150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 -- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d40b2cb0 msgr2=0x7f50d40b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d40b2cb0 0x7f50d40b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d3fff640 1 -- 172.21.15.60:0/1643551700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d40b5a40 con 0x7f50d40d1020 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.209+0000 7f50d9068640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50d415dbe0 0x7f50d4150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.210+0000 7f50d3fff640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 0x7f50d40b5500 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f50c8035040 tx=0x7f50c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.210+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037040 con 0x7f50d40d1020 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.210+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8004230 con 0x7f50d40d1020 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.210+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d415e530 con 0x7f50d40d1020 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.210+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8032e20 con 0x7f50d40d1020 2024-04-25T08:37:17.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.210+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d415eaf0 con 0x7f50d40d1020 2024-04-25T08:37:17.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.211+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4073990 con 0x7f50d40d1020 2024-04-25T08:37:17.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.211+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50c8030770 con 0x7f50d40d1020 2024-04-25T08:37:17.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.212+0000 7f50d17fa640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50c006b4d0 0x7f50c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.212+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f50c80ab5c0 con 0x7f50d40d1020 2024-04-25T08:37:17.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.212+0000 7f50d8867640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50c006b4d0 0x7f50c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.212+0000 7f50d8867640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50c006b4d0 0x7f50c006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f50c4007fd0 tx=0x7f50c4007de0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.216+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50c807a290 con 0x7f50d40d1020 2024-04-25T08:37:17.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.396+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d4061f30 con 0x7f50d40d1020 2024-04-25T08:37:17.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.397+0000 7f50d17fa640 1 -- 172.21.15.60:0/1643551700 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16921 (secure 0 0 0) 0x7f50c8079b90 con 0x7f50d40d1020 2024-04-25T08:37:17.391 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:17.392 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":112,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:07.134750+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:33.401380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:34.086710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.c","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50c006b4d0 msgr2=0x7f50c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50c006b4d0 0x7f50c006d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f50c4007fd0 tx=0x7f50c4007de0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 msgr2=0x7f50d40b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 0x7f50d40b5500 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f50c8035040 tx=0x7f50c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 shutdown_connections 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f50d415dbe0 0x7f50d4150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f50d40d1020 0x7f50d40b5500 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f50c006b4d0 0x7f50c006d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 --2- 172.21.15.60:0/1643551700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50d40b2cb0 0x7f50d40b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.394 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 >> 172.21.15.60:0/1643551700 conn(0x7f50d4061a20 msgr2=0x7f50d41591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:17.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 shutdown_connections 2024-04-25T08:37:17.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.399+0000 7f50daacb640 1 -- 172.21.15.60:0/1643551700 wait complete. 2024-04-25T08:37:17.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:37:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 Processor -- start 2024-04-25T08:37:17.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 -- start start 2024-04-25T08:37:17.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f18f015bb50 0x7f18f00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f18f00c70d0 0x7f18f00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00cfb40 0x7f18f0070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f00c8680 con 0x7f18f015bb50 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f00c75e0 con 0x7f18f00c70d0 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.513+0000 7f18f497d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00c78c0 con 0x7f18f00cfb40 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f18f015bb50 0x7f18f00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f18f015bb50 0x7f18f00c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33624/0 (socket says 172.21.15.60:33624) 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 -- 172.21.15.60:0/3236498644 learned_addr learned my addr 172.21.15.60:0/3236498644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 -- 172.21.15.60:0/3236498644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00cfb40 msgr2=0x7f18f0070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18e7fff640 1 --2- 172.21.15.60:0/3236498644 >> v2:172.21.15.100:3300/0 conn(0x7f18f00c70d0 0x7f18f00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18eed9d640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00cfb40 0x7f18f0070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00cfb40 0x7f18f0070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 -- 172.21.15.60:0/3236498644 >> v2:172.21.15.100:3300/0 conn(0x7f18f00c70d0 msgr2=0x7f18f00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 --2- 172.21.15.60:0/3236498644 >> v2:172.21.15.100:3300/0 conn(0x7f18f00c70d0 0x7f18f00cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18ee59c640 1 -- 172.21.15.60:0/3236498644 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00c7ba0 con 0x7f18f015bb50 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18e7fff640 1 --2- 172.21.15.60:0/3236498644 >> v2:172.21.15.100:3300/0 conn(0x7f18f00c70d0 0x7f18f00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.514+0000 7f18eed9d640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00cfb40 0x7f18f0070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18ee59c640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f015bb50 0x7f18f00c6b90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f18f006f220 tx=0x7f18e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6218775ae08973 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18edd9b640 1 -- 172.21.15.60:0/3236498644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0037070 con 0x7f18f015bb50 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18edd9b640 1 -- 172.21.15.60:0/3236498644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e0032440 con 0x7f18f015bb50 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18edd9b640 1 -- 172.21.15.60:0/3236498644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0032810 con 0x7f18f015bb50 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 -- 172.21.15.60:0/3236498644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f015bb50 msgr2=0x7f18f00c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f015bb50 0x7f18f00c6b90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f18f006f220 tx=0x7f18e002a270 comp rx=0 tx=0).stop 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 -- 172.21.15.60:0/3236498644 shutdown_connections 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00cfb40 0x7f18f0070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 --2- 172.21.15.60:0/3236498644 >> v2:172.21.15.100:3300/0 conn(0x7f18f00c70d0 0x7f18f00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 --2- 172.21.15.60:0/3236498644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f015bb50 0x7f18f00c6b90 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 -- 172.21.15.60:0/3236498644 >> 172.21.15.60:0/3236498644 conn(0x7f18f006f500 msgr2=0x7f18f006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 -- 172.21.15.60:0/3236498644 shutdown_connections 2024-04-25T08:37:17.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.515+0000 7f18f497d640 1 -- 172.21.15.60:0/3236498644 wait complete. 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.517+0000 7f18f497d640 1 Processor -- start 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.517+0000 7f18f497d640 1 -- start start 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.517+0000 7f18f497d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00c70d0 0x7f18f00c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.517+0000 7f18f497d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 0x7f18f015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.517+0000 7f18f497d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f015bb50 0x7f18f0157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.517+0000 7f18f497d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f015f6d0 con 0x7f18f00cfb40 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18f497d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00c8160 con 0x7f18f015bb50 2024-04-25T08:37:17.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18f497d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f015f850 con 0x7f18f00c70d0 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 0x7f18f015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 0x7f18f015a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33640/0 (socket says 172.21.15.60:33640) 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 -- 172.21.15.60:0/1371309113 learned_addr learned my addr 172.21.15.60:0/1371309113 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18ee59c640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00c70d0 0x7f18f00c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18eed9d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f015bb50 0x7f18f0157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 -- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00c70d0 msgr2=0x7f18f00c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00c70d0 0x7f18f00c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 -- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f015bb50 msgr2=0x7f18f0157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f015bb50 0x7f18f0157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 -- 172.21.15.60:0/1371309113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f015ae10 con 0x7f18f00cfb40 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18ee59c640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00c70d0 0x7f18f00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18eed9d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f015bb50 0x7f18f0157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e7fff640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 0x7f18f015a8d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c820 tx=0x7f18d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8015070 con 0x7f18f00cfb40 2024-04-25T08:37:17.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8007830 con 0x7f18f00cfb40 2024-04-25T08:37:17.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8007c20 con 0x7f18f00cfb40 2024-04-25T08:37:17.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f00ca010 con 0x7f18f00cfb40 2024-04-25T08:37:17.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.518+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f00ca850 con 0x7f18f00cfb40 2024-04-25T08:37:17.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.519+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f0073a00 con 0x7f18f00cfb40 2024-04-25T08:37:17.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.520+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18d8018070 con 0x7f18f00cfb40 2024-04-25T08:37:17.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.520+0000 7f18e6ffd640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f18d006b4d0 0x7f18d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.521+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f18d808a150 con 0x7f18f00cfb40 2024-04-25T08:37:17.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.521+0000 7f18ee59c640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f18d006b4d0 0x7f18d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.521+0000 7f18ee59c640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f18d006b4d0 0x7f18d006d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f18e0034000 tx=0x7f18e002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.525+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18f00ca010 con 0x7f18f00cfb40 2024-04-25T08:37:17.740 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.746+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f18f00b28c0 con 0x7f18f00cfb40 2024-04-25T08:37:17.740 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:37:17.746+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:37:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.748+0000 7f18e6ffd640 1 -- 172.21.15.60:0/1371309113 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f18d8058eb0 con 0x7f18f00cfb40 2024-04-25T08:37:17.742 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f18d006b4d0 msgr2=0x7f18d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f18d006b4d0 0x7f18d006d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f18e0034000 tx=0x7f18e002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 msgr2=0x7f18f015a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 0x7f18f015a8d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c820 tx=0x7f18d800cce0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 shutdown_connections 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f18d006b4d0 0x7f18d006d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18f015bb50 0x7f18f0157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.750+0000 7f18f497d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18f00cfb40 0x7f18f015a8d0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.751+0000 7f18f497d640 1 --2- 172.21.15.60:0/1371309113 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18f00c70d0 0x7f18f00c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.751+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 >> 172.21.15.60:0/1371309113 conn(0x7f18f006f500 msgr2=0x7f18f00b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.751+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 shutdown_connections 2024-04-25T08:37:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.751+0000 7f18f497d640 1 -- 172.21.15.60:0/1371309113 wait complete. 2024-04-25T08:37:17.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:37:17.784 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:37:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 Processor -- start 2024-04-25T08:37:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 -- start start 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2900ced80 0x7fe2900d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe290161710 0x7fe2900d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29015e000 con 0x7fe29015dc20 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900d1de0 con 0x7fe2900ced80 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.867+0000 7fe2952a3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe290161c60 con 0x7fe290161710 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2900ced80 0x7fe2900d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe2900ced80 0x7fe2900d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33684/0 (socket says 172.21.15.60:33684) 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 -- 172.21.15.60:0/106924948 learned_addr learned my addr 172.21.15.60:0/106924948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 -- 172.21.15.60:0/106924948 >> v2:172.21.15.100:3300/0 conn(0x7fe290161710 msgr2=0x7fe2900d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28f7fe640 1 --2- 172.21.15.60:0/106924948 >> v2:172.21.15.100:3300/0 conn(0x7fe290161710 0x7fe2900d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.863 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 --2- 172.21.15.60:0/106924948 >> v2:172.21.15.100:3300/0 conn(0x7fe290161710 0x7fe2900d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 -- 172.21.15.60:0/106924948 >> v1:172.21.15.100:6789/0 conn(0x7fe29015dc20 legacy=0x7fe2900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.868+0000 7fe28e7fc640 1 -- 172.21.15.60:0/106924948 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe290161f40 con 0x7fe2900ced80 2024-04-25T08:37:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe28e7fc640 1 --2- 172.21.15.60:0/106924948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900ced80 0x7fe2900d22e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe2900b6570 tx=0x7fe28002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8beacc6399470b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe28dffb640 1 -- 172.21.15.60:0/106924948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280037070 con 0x7fe2900ced80 2024-04-25T08:37:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe28dffb640 1 -- 172.21.15.60:0/106924948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe280032440 con 0x7fe2900ced80 2024-04-25T08:37:17.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe28dffb640 1 -- 172.21.15.60:0/106924948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280032810 con 0x7fe2900ced80 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 -- 172.21.15.60:0/106924948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900ced80 msgr2=0x7fe2900d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 --2- 172.21.15.60:0/106924948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900ced80 0x7fe2900d22e0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe2900b6570 tx=0x7fe28002a230 comp rx=0 tx=0).stop 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 -- 172.21.15.60:0/106924948 shutdown_connections 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 --2- 172.21.15.60:0/106924948 >> v2:172.21.15.100:3300/0 conn(0x7fe290161710 0x7fe2900d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 --2- 172.21.15.60:0/106924948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2900ced80 0x7fe2900d22e0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 -- 172.21.15.60:0/106924948 >> 172.21.15.60:0/106924948 conn(0x7fe2900d0420 msgr2=0x7fe2900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 -- 172.21.15.60:0/106924948 shutdown_connections 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.869+0000 7fe2952a3640 1 -- 172.21.15.60:0/106924948 wait complete. 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 Processor -- start 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 -- start start 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2900ced80 0x7fe2900c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe29015dc20 0x7fe2901e9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 0x7fe2900caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900b5e50 con 0x7fe290161710 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290162500 con 0x7fe2900ced80 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe2952a3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900b5fd0 con 0x7fe29015dc20 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 0x7fe2900caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 0x7fe2900caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33692/0 (socket says 172.21.15.60:33692) 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.871+0000 7fe28f7fe640 1 -- 172.21.15.60:0/354101348 learned_addr learned my addr 172.21.15.60:0/354101348 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28effd640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2900ced80 0x7fe2900c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28f7fe640 1 -- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe29015dc20 msgr2=0x7fe2901e9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28f7fe640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe29015dc20 0x7fe2901e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28f7fe640 1 -- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2900ced80 msgr2=0x7fe2900c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28f7fe640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2900ced80 0x7fe2900c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28f7fe640 1 -- 172.21.15.60:0/354101348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900cdf60 con 0x7fe290161710 2024-04-25T08:37:17.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28effd640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2900ced80 0x7fe2900c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe28f7fe640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 0x7fe2900caa70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe28800cc80 tx=0x7fe28800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800ce30 con 0x7fe290161710 2024-04-25T08:37:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288003990 con 0x7fe290161710 2024-04-25T08:37:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2900cd9a0 con 0x7fe290161710 2024-04-25T08:37:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288003d60 con 0x7fe290161710 2024-04-25T08:37:17.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.872+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2901e9140 con 0x7fe290161710 2024-04-25T08:37:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.873+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290161450 con 0x7fe290161710 2024-04-25T08:37:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.873+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe288003f00 con 0x7fe290161710 2024-04-25T08:37:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.874+0000 7fe2877fe640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe27006b4d0 0x7fe27006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.874+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fe28808a7f0 con 0x7fe290161710 2024-04-25T08:37:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.874+0000 7fe28effd640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe27006b4d0 0x7fe27006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:17.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.875+0000 7fe28effd640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe27006b4d0 0x7fe27006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe27800a810 tx=0x7fe278008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:17.878+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe290161450 con 0x7fe290161710 2024-04-25T08:37:18.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.098+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe290073a00 con 0x7fe290161710 2024-04-25T08:37:18.743 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.749+0000 7fe2877fe640 1 -- 172.21.15.60:0/354101348 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe288014030 con 0x7fe290161710 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe27006b4d0 msgr2=0x7fe27006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe27006b4d0 0x7fe27006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe27800a810 tx=0x7fe278008040 comp rx=0 tx=0).stop 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 msgr2=0x7fe2900caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 0x7fe2900caa70 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe28800cc80 tx=0x7fe28800a5c0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 shutdown_connections 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe290161710 0x7fe2900caa70 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.747 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe29015dc20 0x7fe2901e9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe27006b4d0 0x7fe27006d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 --2- 172.21.15.60:0/354101348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe2900ced80 0x7fe2900c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 >> 172.21.15.60:0/354101348 conn(0x7fe2900d0420 msgr2=0x7fe2900b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:18.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 shutdown_connections 2024-04-25T08:37:18.748 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.753+0000 7fe2952a3640 1 -- 172.21.15.60:0/354101348 wait complete. 2024-04-25T08:37:18.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:37:18.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.921+0000 7fef73a07640 1 Processor -- start 2024-04-25T08:37:18.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.921+0000 7fef73a07640 1 -- start start 2024-04-25T08:37:18.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef73a07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fef6c15b7d0 0x7fef6c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:18.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef73a07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef6c06c9e0 0x7fef6c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:18.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef73a07640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c06cf50 con 0x7fef6c0b5620 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef73a07640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c0ce530 con 0x7fef6c06c9e0 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef73a07640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c0ce810 con 0x7fef6c15b7d0 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef71fa4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef6c06c9e0 0x7fef6c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef70fa2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef6c0b5620 0x7fef6c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42496/0 (socket says 172.21.15.60:42496) 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef70fa2640 1 -- 172.21.15.60:0/1380020225 learned_addr learned my addr 172.21.15.60:0/1380020225 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef717a3640 1 --2- 172.21.15.60:0/1380020225 >> v2:172.21.15.100:3300/0 conn(0x7fef6c15b7d0 0x7fef6c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef71fa4640 1 -- 172.21.15.60:0/1380020225 >> v2:172.21.15.100:3300/0 conn(0x7fef6c15b7d0 msgr2=0x7fef6c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef71fa4640 1 --2- 172.21.15.60:0/1380020225 >> v2:172.21.15.100:3300/0 conn(0x7fef6c15b7d0 0x7fef6c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef71fa4640 1 -- 172.21.15.60:0/1380020225 >> v1:172.21.15.60:6789/0 conn(0x7fef6c0b5620 legacy=0x7fef6c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef71fa4640 1 -- 172.21.15.60:0/1380020225 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0ceaf0 con 0x7fef6c06c9e0 2024-04-25T08:37:18.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.922+0000 7fef63fff640 1 -- 172.21.15.60:0/1380020225 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092994438 0 0) 0x7fef6c06cf50 con 0x7fef6c0b5620 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.923+0000 7fef71fa4640 1 --2- 172.21.15.60:0/1380020225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c06c9e0 0x7fef6c0b5ab0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fef6c08c0d0 tx=0x7fef6402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e788d3cef1f3e678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.923+0000 7fef63fff640 1 -- 172.21.15.60:0/1380020225 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64007740 con 0x7fef6c06c9e0 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.923+0000 7fef63fff640 1 -- 172.21.15.60:0/1380020225 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef640078e0 con 0x7fef6c06c9e0 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.923+0000 7fef63fff640 1 -- 172.21.15.60:0/1380020225 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64007cb0 con 0x7fef6c06c9e0 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 -- 172.21.15.60:0/1380020225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c06c9e0 msgr2=0x7fef6c0b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 --2- 172.21.15.60:0/1380020225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c06c9e0 0x7fef6c0b5ab0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fef6c08c0d0 tx=0x7fef6402a680 comp rx=0 tx=0).stop 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 -- 172.21.15.60:0/1380020225 shutdown_connections 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 --2- 172.21.15.60:0/1380020225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c06c9e0 0x7fef6c0b5ab0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 --2- 172.21.15.60:0/1380020225 >> v2:172.21.15.100:3300/0 conn(0x7fef6c15b7d0 0x7fef6c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 -- 172.21.15.60:0/1380020225 >> 172.21.15.60:0/1380020225 conn(0x7fef6c0d0800 msgr2=0x7fef6c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 -- 172.21.15.60:0/1380020225 shutdown_connections 2024-04-25T08:37:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.924+0000 7fef73a07640 1 -- 172.21.15.60:0/1380020225 wait complete. 2024-04-25T08:37:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.925+0000 7fef73a07640 1 Processor -- start 2024-04-25T08:37:18.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.925+0000 7fef73a07640 1 -- start start 2024-04-25T08:37:18.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef73a07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef6c06c9e0 0x7fef6c1f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef73a07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0b5620 0x7fef6c159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef73a07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 0x7fef6c0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef73a07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0cf6d0 con 0x7fef6c15b7d0 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef73a07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0cf550 con 0x7fef6c0b5620 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef73a07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c15bbb0 con 0x7fef6c06c9e0 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 0x7fef6c0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 0x7fef6c0cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33710/0 (socket says 172.21.15.60:33710) 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 -- 172.21.15.60:0/4227881276 learned_addr learned my addr 172.21.15.60:0/4227881276 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef70fa2640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0b5620 0x7fef6c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef717a3640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef6c06c9e0 0x7fef6c1f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 -- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef6c06c9e0 msgr2=0x7fef6c1f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef6c06c9e0 0x7fef6c1f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 -- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0b5620 msgr2=0x7fef6c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0b5620 0x7fef6c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 -- 172.21.15.60:0/4227881276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c1f43e0 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef70fa2640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0b5620 0x7fef6c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef717a3640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef6c06c9e0 0x7fef6c1f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef71fa4640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 0x7fef6c0cd170 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fef64009e50 tx=0x7fef64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.926+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64040070 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.927+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef64031bf0 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.927+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c159560 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.927+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64009450 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.927+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c0cba90 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.928+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c1e65a0 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.928+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef64039040 con 0x7fef6c15b7d0 2024-04-25T08:37:18.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.929+0000 7fef61ffb640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef5006b4d0 0x7fef5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:18.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.929+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fef640abbb0 con 0x7fef6c15b7d0 2024-04-25T08:37:18.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.929+0000 7fef717a3640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef5006b4d0 0x7fef5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:18.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.929+0000 7fef717a3640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef5006b4d0 0x7fef5006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef68006fd0 tx=0x7fef68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:18.933+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef6c1e65a0 con 0x7fef6c15b7d0 2024-04-25T08:37:19.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.112+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef6c073a00 con 0x7fef6c15b7d0 2024-04-25T08:37:19.745 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.750+0000 7fef61ffb640 1 -- 172.21.15.60:0/4227881276 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7fef64043360 con 0x7fef6c15b7d0 2024-04-25T08:37:19.745 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef5006b4d0 msgr2=0x7fef5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef5006b4d0 0x7fef5006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef68006fd0 tx=0x7fef68008040 comp rx=0 tx=0).stop 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 msgr2=0x7fef6c0cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 0x7fef6c0cd170 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fef64009e50 tx=0x7fef64004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 shutdown_connections 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef6c15b7d0 0x7fef6c0cd170 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef5006b4d0 0x7fef5006d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef6c0b5620 0x7fef6c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.750 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 --2- 172.21.15.60:0/4227881276 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef6c06c9e0 0x7fef6c1f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.754+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 >> 172.21.15.60:0/4227881276 conn(0x7fef6c0d0800 msgr2=0x7fef6c160550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:19.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.755+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 shutdown_connections 2024-04-25T08:37:19.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.755+0000 7fef73a07640 1 -- 172.21.15.60:0/4227881276 wait complete. 2024-04-25T08:37:19.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:37:19.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.880+0000 7fec6b53b640 1 Processor -- start 2024-04-25T08:37:19.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.880+0000 7fec6b53b640 1 -- start start 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec6b53b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec6406c9e0 0x7fec640b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec6b53b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6406cf50 con 0x7fec6415b7d0 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec6b53b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec640ce530 con 0x7fec640b5620 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec6b53b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec640ce810 con 0x7fec6406c9e0 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec692d7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fec6415b7d0 0x7fec640cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42512/0 (socket says 172.21.15.60:42512) 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec692d7640 1 -- 172.21.15.60:0/1761038512 learned_addr learned my addr 172.21.15.60:0/1761038512 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.881+0000 7fec69ad8640 1 --2- 172.21.15.60:0/1761038512 >> v2:172.21.15.60:3300/0 conn(0x7fec6406c9e0 0x7fec640b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.882+0000 7fec63fff640 1 -- 172.21.15.60:0/1761038512 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104972915 0 0) 0x7fec6406cf50 con 0x7fec6415b7d0 2024-04-25T08:37:19.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.882+0000 7fec63fff640 1 -- 172.21.15.60:0/1761038512 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec4c003520 con 0x7fec6415b7d0 2024-04-25T08:37:19.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.882+0000 7fec69ad8640 1 -- 172.21.15.60:0/1761038512 >> v1:172.21.15.100:6789/0 conn(0x7fec640b5620 legacy=0x7fec6406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.882+0000 7fec69ad8640 1 -- 172.21.15.60:0/1761038512 >> v1:172.21.15.60:6789/0 conn(0x7fec6415b7d0 legacy=0x7fec640cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.882+0000 7fec69ad8640 1 -- 172.21.15.60:0/1761038512 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640ceaf0 con 0x7fec6406c9e0 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec69ad8640 1 --2- 172.21.15.60:0/1761038512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406c9e0 0x7fec640b5ab0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fec54009d90 tx=0x7fec5402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9677faece575bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec63fff640 1 -- 172.21.15.60:0/1761038512 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54007740 con 0x7fec6406c9e0 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec63fff640 1 -- 172.21.15.60:0/1761038512 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec540078e0 con 0x7fec6406c9e0 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec63fff640 1 -- 172.21.15.60:0/1761038512 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54007cb0 con 0x7fec6406c9e0 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 -- 172.21.15.60:0/1761038512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406c9e0 msgr2=0x7fec640b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 --2- 172.21.15.60:0/1761038512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406c9e0 0x7fec640b5ab0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fec54009d90 tx=0x7fec5402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 -- 172.21.15.60:0/1761038512 shutdown_connections 2024-04-25T08:37:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 --2- 172.21.15.60:0/1761038512 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec6406c9e0 0x7fec640b5ab0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 -- 172.21.15.60:0/1761038512 >> 172.21.15.60:0/1761038512 conn(0x7fec640d0800 msgr2=0x7fec640d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 -- 172.21.15.60:0/1761038512 shutdown_connections 2024-04-25T08:37:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.883+0000 7fec6b53b640 1 -- 172.21.15.60:0/1761038512 wait complete. 2024-04-25T08:37:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.884+0000 7fec6b53b640 1 Processor -- start 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 -- start start 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6406c9e0 0x7fec640aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 0x7fec64150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6415b7d0 0x7fec640cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6415bd30 con 0x7fec640b5620 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6415bbb0 con 0x7fec6406c9e0 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec6b53b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec640d1f40 con 0x7fec6415b7d0 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec68ad6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 0x7fec64150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec68ad6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 0x7fec64150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33732/0 (socket says 172.21.15.60:33732) 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.885+0000 7fec68ad6640 1 -- 172.21.15.60:0/3654090901 learned_addr learned my addr 172.21.15.60:0/3654090901 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:19.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec692d7640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6406c9e0 0x7fec640aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec69ad8640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6415b7d0 0x7fec640cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec68ad6640 1 -- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6415b7d0 msgr2=0x7fec640cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec68ad6640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6415b7d0 0x7fec640cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec68ad6640 1 -- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6406c9e0 msgr2=0x7fec640aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec68ad6640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6406c9e0 0x7fec640aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec68ad6640 1 -- 172.21.15.60:0/3654090901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640cbf80 con 0x7fec640b5620 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec68ad6640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 0x7fec64150f80 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c400 tx=0x7fec5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec580052d0 con 0x7fec640b5620 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec58005490 con 0x7fec640b5620 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec640c77c0 con 0x7fec640b5620 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58007260 con 0x7fec640b5620 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.886+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641514c0 con 0x7fec640b5620 2024-04-25T08:37:19.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.887+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64073a00 con 0x7fec640b5620 2024-04-25T08:37:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.888+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec58007400 con 0x7fec640b5620 2024-04-25T08:37:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.888+0000 7fec61ffb640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec5006b4d0 0x7fec5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.888+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5778+0+0 (secure 0 0 0) 0x7fec58089720 con 0x7fec640b5620 2024-04-25T08:37:19.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.888+0000 7fec692d7640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec5006b4d0 0x7fec5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:19.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.889+0000 7fec692d7640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec5006b4d0 0x7fec5006d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0011e0 tx=0x7fec5c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:19.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:19.892+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec580585e0 con 0x7fec640b5620 2024-04-25T08:37:20.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.080+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fec6415fe10 con 0x7fec640b5620 2024-04-25T08:37:20.746 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.752+0000 7fec61ffb640 1 -- 172.21.15.60:0/3654090901 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7fec58057ee0 con 0x7fec640b5620 2024-04-25T08:37:20.747 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:37:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec5006b4d0 msgr2=0x7fec5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec5006b4d0 0x7fec5006d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0011e0 tx=0x7fec5c009040 comp rx=0 tx=0).stop 2024-04-25T08:37:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 msgr2=0x7fec64150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 0x7fec64150f80 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c400 tx=0x7fec5800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 shutdown_connections 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec6415b7d0 0x7fec640cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec640b5620 0x7fec64150f80 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec5006b4d0 0x7fec5006d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 --2- 172.21.15.60:0/3654090901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec6406c9e0 0x7fec640aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.756+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 >> 172.21.15.60:0/3654090901 conn(0x7fec640d0800 msgr2=0x7fec640b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.757+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 shutdown_connections 2024-04-25T08:37:20.752 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.757+0000 7fec6b53b640 1 -- 172.21.15.60:0/3654090901 wait complete. 2024-04-25T08:37:20.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:37:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.886+0000 7f85cc6e4640 1 Processor -- start 2024-04-25T08:37:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.887+0000 7f85cc6e4640 1 -- start start 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.887+0000 7f85cc6e4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85c40ce230 0x7f85c40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.887+0000 7f85cc6e4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c4072280 con 0x7f85c415b270 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.887+0000 7f85cc6e4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c40714c0 con 0x7f85c40cc4f0 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.887+0000 7f85cc6e4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c4071270 con 0x7f85c40ce230 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.888+0000 7f85c9c7f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f85c40ce230 0x7f85c40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.888+0000 7f85cac81640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f85c415b270 0x7f85c415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:42520/0 (socket says 172.21.15.60:42520) 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.888+0000 7f85cac81640 1 -- 172.21.15.60:0/704541519 learned_addr learned my addr 172.21.15.60:0/704541519 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.888+0000 7f85c9c7f640 1 -- 172.21.15.60:0/704541519 >> v1:172.21.15.100:6789/0 conn(0x7f85c40cc4f0 legacy=0x7f85c40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.888+0000 7f85c9c7f640 1 -- 172.21.15.60:0/704541519 >> v1:172.21.15.60:6789/0 conn(0x7f85c415b270 legacy=0x7f85c415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.888+0000 7f85c9c7f640 1 -- 172.21.15.60:0/704541519 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40717a0 con 0x7f85c40ce230 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85c9c7f640 1 --2- 172.21.15.60:0/704541519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c40ce230 0x7f85c40cc8d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f85b40097a0 tx=0x7f85b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6827dcda818ce3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85c947e640 1 -- 172.21.15.60:0/704541519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4037070 con 0x7f85c40ce230 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85c947e640 1 -- 172.21.15.60:0/704541519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4032440 con 0x7f85c40ce230 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85c947e640 1 -- 172.21.15.60:0/704541519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4032810 con 0x7f85c40ce230 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 -- 172.21.15.60:0/704541519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c40ce230 msgr2=0x7f85c40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/704541519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c40ce230 0x7f85c40cc8d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f85b40097a0 tx=0x7f85b402a230 comp rx=0 tx=0).stop 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 -- 172.21.15.60:0/704541519 shutdown_connections 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/704541519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c40ce230 0x7f85c40cc8d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 -- 172.21.15.60:0/704541519 >> 172.21.15.60:0/704541519 conn(0x7f85c406abe0 msgr2=0x7f85c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 -- 172.21.15.60:0/704541519 shutdown_connections 2024-04-25T08:37:20.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.889+0000 7f85cc6e4640 1 -- 172.21.15.60:0/704541519 wait complete. 2024-04-25T08:37:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 Processor -- start 2024-04-25T08:37:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 -- start start 2024-04-25T08:37:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c40cc4f0 0x7f85c40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c40ce230 0x7f85c413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 0x7f85c414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:20.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c406e8a0 con 0x7f85c415b270 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4071d60 con 0x7f85c40cc4f0 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cc6e4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40ae500 con 0x7f85c40ce230 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cac81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 0x7f85c414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cac81640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 0x7f85c414e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33748/0 (socket says 172.21.15.60:33748) 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.891+0000 7f85cac81640 1 -- 172.21.15.60:0/1520778788 learned_addr learned my addr 172.21.15.60:0/1520778788 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85ca480640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c40cc4f0 0x7f85c40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85c9c7f640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c40ce230 0x7f85c413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cac81640 1 -- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c40ce230 msgr2=0x7f85c413d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cac81640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c40ce230 0x7f85c413d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cac81640 1 -- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c40cc4f0 msgr2=0x7f85c40b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cac81640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c40cc4f0 0x7f85c40b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:20.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cac81640 1 -- 172.21.15.60:0/1520778788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40af6b0 con 0x7f85c415b270 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85ca480640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c40cc4f0 0x7f85c40b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cac81640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 0x7f85c414e720 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f85c000cbc0 tx=0x7f85c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0017070 con 0x7f85c415b270 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c000ad40 con 0x7f85c415b270 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c40af990 con 0x7f85c415b270 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0012600 con 0x7f85c415b270 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.892+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c40aff50 con 0x7f85c415b270 2024-04-25T08:37:20.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.893+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c4073a00 con 0x7f85c415b270 2024-04-25T08:37:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.894+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85c001a070 con 0x7f85c415b270 2024-04-25T08:37:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.894+0000 7f85baffd640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85a806b4d0 0x7f85a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.894+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f85c0089de0 con 0x7f85c415b270 2024-04-25T08:37:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.894+0000 7f85ca480640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85a806b4d0 0x7f85a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:20.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.895+0000 7f85ca480640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85a806b4d0 0x7f85a806d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f85c415dc70 tx=0x7f85bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:20.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:20.898+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85c0058700 con 0x7f85c415b270 2024-04-25T08:37:21.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.080+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f85c40cdb80 con 0x7f85c415b270 2024-04-25T08:37:21.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.083+0000 7f85baffd640 1 -- 172.21.15.60:0/1520778788 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85c0075e20 con 0x7f85c415b270 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85a806b4d0 msgr2=0x7f85a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85a806b4d0 0x7f85a806d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f85c415dc70 tx=0x7f85bc009040 comp rx=0 tx=0).stop 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 msgr2=0x7f85c414e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 0x7f85c414e720 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f85c000cbc0 tx=0x7f85c000a430 comp rx=0 tx=0).stop 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 shutdown_connections 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f85c415b270 0x7f85c414e720 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f85c40ce230 0x7f85c413d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f85a806b4d0 0x7f85a806d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 --2- 172.21.15.60:0/1520778788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85c40cc4f0 0x7f85c40b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 >> 172.21.15.60:0/1520778788 conn(0x7f85c406abe0 msgr2=0x7f85c4152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 shutdown_connections 2024-04-25T08:37:21.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.086+0000 7f85cc6e4640 1 -- 172.21.15.60:0/1520778788 wait complete. 2024-04-25T08:37:21.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T08:37:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.198+0000 7f97a421f640 1 Processor -- start 2024-04-25T08:37:21.192 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.198+0000 7f97a421f640 1 -- start start 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a421f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0b2cb0 0x7f979c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a421f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f979c0d1020 0x7f979c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a421f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979c0b97a0 con 0x7f979c15dbe0 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a421f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979c15dfc0 con 0x7f979c0d1020 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a421f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0d1a00 con 0x7f979c0b2cb0 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f979c0d1020 0x7f979c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f979c0d1020 0x7f979c0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33756/0 (socket says 172.21.15.60:33756) 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 -- 172.21.15.60:0/3067196767 learned_addr learned my addr 172.21.15.60:0/3067196767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a17ba640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0b2cb0 0x7f979c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 -- 172.21.15.60:0/3067196767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0b2cb0 msgr2=0x7f979c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0b2cb0 0x7f979c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 -- 172.21.15.60:0/3067196767 >> v1:172.21.15.100:6789/0 conn(0x7f979c15dbe0 legacy=0x7f979c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a27bc640 1 -- 172.21.15.60:0/3067196767 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0d1c50 con 0x7f979c0d1020 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.199+0000 7f97a17ba640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0b2cb0 0x7f979c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.200+0000 7f97a27bc640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c0d1020 0x7f979c0d2c70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f9794009e50 tx=0x7f979402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b307ed41ceb61fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.200+0000 7f97a0fb9640 1 -- 172.21.15.60:0/3067196767 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794036070 con 0x7f979c0d1020 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.200+0000 7f97a0fb9640 1 -- 172.21.15.60:0/3067196767 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794031600 con 0x7f979c0d1020 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.200+0000 7f97a0fb9640 1 -- 172.21.15.60:0/3067196767 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940319d0 con 0x7f979c0d1020 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 -- 172.21.15.60:0/3067196767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c0d1020 msgr2=0x7f979c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c0d1020 0x7f979c0d2c70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f9794009e50 tx=0x7f979402a710 comp rx=0 tx=0).stop 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 -- 172.21.15.60:0/3067196767 shutdown_connections 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c0d1020 0x7f979c0d2c70 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 --2- 172.21.15.60:0/3067196767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0b2cb0 0x7f979c0d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 -- 172.21.15.60:0/3067196767 >> 172.21.15.60:0/3067196767 conn(0x7f979c061a20 msgr2=0x7f979c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 -- 172.21.15.60:0/3067196767 shutdown_connections 2024-04-25T08:37:21.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.201+0000 7f97a421f640 1 -- 172.21.15.60:0/3067196767 wait complete. 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.202+0000 7f97a421f640 1 Processor -- start 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.202+0000 7f97a421f640 1 -- start start 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0b2cb0 0x7f979c0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0d1020 0x7f979c0cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 0x7f979c0b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0b92a0 con 0x7f979c15dbe0 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0d2210 con 0x7f979c0b2cb0 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979c0b9420 con 0x7f979c0d1020 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 0x7f979c0b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 0x7f979c0b7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:33766/0 (socket says 172.21.15.60:33766) 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 -- 172.21.15.60:0/2309322321 learned_addr learned my addr 172.21.15.60:0/2309322321 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a17ba640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0d1020 0x7f979c0cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a1fbb640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0b2cb0 0x7f979c0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 -- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0d1020 msgr2=0x7f979c0cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0d1020 0x7f979c0cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 -- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0b2cb0 msgr2=0x7f979c0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0b2cb0 0x7f979c0c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 -- 172.21.15.60:0/2309322321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0cc800 con 0x7f979c15dbe0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a17ba640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0d1020 0x7f979c0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a1fbb640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0b2cb0 0x7f979c0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a27bc640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 0x7f979c0b7cc0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f9794034040 tx=0x7f9794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794036050 con 0x7f979c15dbe0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794042070 con 0x7f979c15dbe0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.203+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c0ccae0 con 0x7f979c15dbe0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.204+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794009510 con 0x7f979c15dbe0 2024-04-25T08:37:21.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.204+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c0cd0a0 con 0x7f979c15dbe0 2024-04-25T08:37:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.204+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c073990 con 0x7f979c15dbe0 2024-04-25T08:37:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.205+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9794007bd0 con 0x7f979c15dbe0 2024-04-25T08:37:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.205+0000 7f97927fc640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f978006b4d0 0x7f978006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.206+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f97940aaf50 con 0x7f979c15dbe0 2024-04-25T08:37:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.206+0000 7f97a1fbb640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f978006b4d0 0x7f978006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.206+0000 7f97a1fbb640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f978006b4d0 0x7f978006d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f97980015e0 tx=0x7f9798009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:21.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.210+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9794079fd0 con 0x7f979c15dbe0 2024-04-25T08:37:21.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.461+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f979c15d1f0 con 0x7f979c15dbe0 2024-04-25T08:37:21.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.761+0000 7f97927fc640 1 -- 172.21.15.60:0/2309322321 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v396) v1 ==== 155+0+0 (secure 0 0 0) 0x7f97940798d0 con 0x7f979c15dbe0 2024-04-25T08:37:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f978006b4d0 msgr2=0x7f978006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f978006b4d0 0x7f978006d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f97980015e0 tx=0x7f9798009040 comp rx=0 tx=0).stop 2024-04-25T08:37:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 msgr2=0x7f979c0b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 0x7f979c0b7cc0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f9794034040 tx=0x7f9794004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 shutdown_connections 2024-04-25T08:37:21.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f979c15dbe0 0x7f979c0b7cc0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f979c0d1020 0x7f979c0cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f978006b4d0 0x7f978006d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 --2- 172.21.15.60:0/2309322321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f979c0b2cb0 0x7f979c0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 >> 172.21.15.60:0/2309322321 conn(0x7f979c061a20 msgr2=0x7f979c158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 shutdown_connections 2024-04-25T08:37:21.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.764+0000 7f97a421f640 1 -- 172.21.15.60:0/2309322321 wait complete. 2024-04-25T08:37:21.770 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:37:21.770 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:37:21.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.880+0000 7f5f2685d640 1 Processor -- start 2024-04-25T08:37:21.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.880+0000 7f5f2685d640 1 -- start start 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.880+0000 7f5f2685d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f2015db60 0x7f5f200ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.880+0000 7f5f2685d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f200d0b10 0x7f5f200d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f2685d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f200cec00 con 0x7f5f200b5f00 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f2685d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200ce930 con 0x7f5f2015db60 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f2685d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f200b6450 con 0x7f5f200d0b10 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f2015db60 0x7f5f200ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f2015db60 0x7f5f200ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41314/0 (socket says 172.21.15.60:41314) 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 -- 172.21.15.60:0/3197170798 learned_addr learned my addr 172.21.15.60:0/3197170798 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f24dfa640 1 --2- 172.21.15.60:0/3197170798 >> v2:172.21.15.100:3300/0 conn(0x7f5f200d0b10 0x7f5f200d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 -- 172.21.15.60:0/3197170798 >> v2:172.21.15.100:3300/0 conn(0x7f5f200d0b10 msgr2=0x7f5f200d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:21.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 --2- 172.21.15.60:0/3197170798 >> v2:172.21.15.100:3300/0 conn(0x7f5f200d0b10 0x7f5f200d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 -- 172.21.15.60:0/3197170798 >> v1:172.21.15.100:6789/0 conn(0x7f5f200b5f00 legacy=0x7f5f200d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.881+0000 7f5f1ffff640 1 -- 172.21.15.60:0/3197170798 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f200ceee0 con 0x7f5f2015db60 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.882+0000 7f5f1ffff640 1 --2- 172.21.15.60:0/3197170798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f2015db60 0x7f5f200ce3f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7562b567715e2712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.882+0000 7f5f1effd640 1 -- 172.21.15.60:0/3197170798 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10037070 con 0x7f5f2015db60 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.882+0000 7f5f1effd640 1 -- 172.21.15.60:0/3197170798 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10032440 con 0x7f5f2015db60 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.882+0000 7f5f1effd640 1 -- 172.21.15.60:0/3197170798 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10032810 con 0x7f5f2015db60 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.882+0000 7f5f2685d640 1 -- 172.21.15.60:0/3197170798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f2015db60 msgr2=0x7f5f200ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.882+0000 7f5f2685d640 1 --2- 172.21.15.60:0/3197170798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f2015db60 0x7f5f200ce3f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009870 tx=0x7f5f1002a210 comp rx=0 tx=0).stop 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.883+0000 7f5f2685d640 1 -- 172.21.15.60:0/3197170798 shutdown_connections 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.883+0000 7f5f2685d640 1 --2- 172.21.15.60:0/3197170798 >> v2:172.21.15.100:3300/0 conn(0x7f5f200d0b10 0x7f5f200d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.883+0000 7f5f2685d640 1 --2- 172.21.15.60:0/3197170798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f2015db60 0x7f5f200ce3f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.883+0000 7f5f2685d640 1 -- 172.21.15.60:0/3197170798 >> 172.21.15.60:0/3197170798 conn(0x7f5f20061600 msgr2=0x7f5f20061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.883+0000 7f5f2685d640 1 -- 172.21.15.60:0/3197170798 shutdown_connections 2024-04-25T08:37:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.883+0000 7f5f2685d640 1 -- 172.21.15.60:0/3197170798 wait complete. 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.884+0000 7f5f2685d640 1 Processor -- start 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.884+0000 7f5f2685d640 1 -- start start 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f200b5f00 0x7f5f2015f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 0x7f5f200aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2015db60 0x7f5f2014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2015df40 con 0x7f5f200d0b10 2024-04-25T08:37:21.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200d1d50 con 0x7f5f2015db60 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2015e0c0 con 0x7f5f200b5f00 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 0x7f5f200aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 0x7f5f200aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41318/0 (socket says 172.21.15.60:41318) 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 -- 172.21.15.60:0/1291199595 learned_addr learned my addr 172.21.15.60:0/1291199595 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f24dfa640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2015db60 0x7f5f2014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1ffff640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f200b5f00 0x7f5f2015f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 -- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f200b5f00 msgr2=0x7f5f2015f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f200b5f00 0x7f5f2015f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 -- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2015db60 msgr2=0x7f5f2014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2015db60 0x7f5f2014deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 -- 172.21.15.60:0/1291199595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2014e3f0 con 0x7f5f200d0b10 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1f7fe640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 0x7f5f200aa3c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c430 tx=0x7f5f1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:21.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f140052d0 con 0x7f5f200d0b10 2024-04-25T08:37:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14005490 con 0x7f5f200d0b10 2024-04-25T08:37:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14007260 con 0x7f5f200d0b10 2024-04-25T08:37:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.885+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2015fc50 con 0x7f5f200d0b10 2024-04-25T08:37:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.886+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f20160210 con 0x7f5f200d0b10 2024-04-25T08:37:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.887+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f20073270 con 0x7f5f200d0b10 2024-04-25T08:37:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.887+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f14007400 con 0x7f5f200d0b10 2024-04-25T08:37:21.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.887+0000 7f5f1cff9640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:21.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.888+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5f14089560 con 0x7f5f200d0b10 2024-04-25T08:37:21.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.888+0000 7f5f1ffff640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:21.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.888+0000 7f5f1ffff640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:21.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:21.892+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f2014e3f0 con 0x7f5f200d0b10 2024-04-25T08:37:22.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.074+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f20073990 con 0x7f5f200d0b10 2024-04-25T08:37:22.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.075+0000 7f5f1cff9640 1 -- 172.21.15.60:0/1291199595 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+13291 (secure 0 0 0) 0x7f5f14057e70 con 0x7f5f200d0b10 2024-04-25T08:37:22.069 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:22.070 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":115,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:20.751621+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6835/4168808734":"2024-04-26T08:37:17.747477+0000","172.21.15.60:6833/4168808734":"2024-04-26T08:37:17.747477+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.077+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f0c06b500 msgr2=0x7f5f0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.077+0000 7f5f2685d640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034000 tx=0x7f5f1002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.077+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 msgr2=0x7f5f200aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.077+0000 7f5f2685d640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 0x7f5f200aa3c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c430 tx=0x7f5f1400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.077+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 shutdown_connections 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.077+0000 7f5f2685d640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f0c06b500 0x7f5f0c06d9c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.078+0000 7f5f2685d640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f2015db60 0x7f5f2014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.078+0000 7f5f2685d640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f200d0b10 0x7f5f200aa3c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.078+0000 7f5f2685d640 1 --2- 172.21.15.60:0/1291199595 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f200b5f00 0x7f5f2015f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.078+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 >> 172.21.15.60:0/1291199595 conn(0x7f5f20061600 msgr2=0x7f5f200d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:22.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.078+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 shutdown_connections 2024-04-25T08:37:22.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.078+0000 7f5f2685d640 1 -- 172.21.15.60:0/1291199595 wait complete. 2024-04-25T08:37:22.083 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:37:22.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:37:22.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.204+0000 7fc8182b5640 1 Processor -- start 2024-04-25T08:37:22.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.204+0000 7fc8182b5640 1 -- start start 2024-04-25T08:37:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc8182b5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc81015dd80 0x7fc8100b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc8182b5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc8100ad570 0x7fc81006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc8182b5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8100b3930 con 0x7fc81006abe0 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc8182b5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81006e410 con 0x7fc81015dd80 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc8182b5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8100b3c10 con 0x7fc8100ad570 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc815850640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc81015dd80 0x7fc8100b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc815850640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc81015dd80 0x7fc8100b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41320/0 (socket says 172.21.15.60:41320) 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.205+0000 7fc815850640 1 -- 172.21.15.60:0/2668367066 learned_addr learned my addr 172.21.15.60:0/2668367066 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.206+0000 7fc816852640 1 --2- 172.21.15.60:0/2668367066 >> v2:172.21.15.100:3300/0 conn(0x7fc8100ad570 0x7fc81006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.206+0000 7fc815850640 1 -- 172.21.15.60:0/2668367066 >> v2:172.21.15.100:3300/0 conn(0x7fc8100ad570 msgr2=0x7fc81006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.206+0000 7fc815850640 1 --2- 172.21.15.60:0/2668367066 >> v2:172.21.15.100:3300/0 conn(0x7fc8100ad570 0x7fc81006ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.206+0000 7fc815850640 1 -- 172.21.15.60:0/2668367066 >> v1:172.21.15.100:6789/0 conn(0x7fc81006abe0 legacy=0x7fc81006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.206+0000 7fc815850640 1 -- 172.21.15.60:0/2668367066 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81006e660 con 0x7fc81015dd80 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.207+0000 7fc815850640 1 --2- 172.21.15.60:0/2668367066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc8100b5290 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc810072fe0 tx=0x7fc80002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7407a76f77a9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.207+0000 7fc81504f640 1 -- 172.21.15.60:0/2668367066 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800037070 con 0x7fc81015dd80 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.207+0000 7fc81504f640 1 -- 172.21.15.60:0/2668367066 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc800032440 con 0x7fc81015dd80 2024-04-25T08:37:22.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.207+0000 7fc81504f640 1 -- 172.21.15.60:0/2668367066 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800032810 con 0x7fc81015dd80 2024-04-25T08:37:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 -- 172.21.15.60:0/2668367066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 msgr2=0x7fc8100b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 --2- 172.21.15.60:0/2668367066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc8100b5290 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc810072fe0 tx=0x7fc80002a270 comp rx=0 tx=0).stop 2024-04-25T08:37:22.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 -- 172.21.15.60:0/2668367066 shutdown_connections 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 --2- 172.21.15.60:0/2668367066 >> v2:172.21.15.100:3300/0 conn(0x7fc8100ad570 0x7fc81006ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 --2- 172.21.15.60:0/2668367066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc8100b5290 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 -- 172.21.15.60:0/2668367066 >> 172.21.15.60:0/2668367066 conn(0x7fc8100b5af0 msgr2=0x7fc8100b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 -- 172.21.15.60:0/2668367066 shutdown_connections 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.208+0000 7fc8182b5640 1 -- 172.21.15.60:0/2668367066 wait complete. 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.209+0000 7fc8182b5640 1 Processor -- start 2024-04-25T08:37:22.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.209+0000 7fc8182b5640 1 -- start start 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.210+0000 7fc8182b5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81006abe0 0x7fc81015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.210+0000 7fc8182b5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8100ad570 0x7fc81009bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.210+0000 7fc8182b5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc810150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.210+0000 7fc8182b5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810070280 con 0x7fc81015dd80 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.210+0000 7fc8182b5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8100b57d0 con 0x7fc8100ad570 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.210+0000 7fc8182b5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc810070400 con 0x7fc81006abe0 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc810150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc810150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41328/0 (socket says 172.21.15.60:41328) 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 -- 172.21.15.60:0/179298291 learned_addr learned my addr 172.21.15.60:0/179298291 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816051640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81006abe0 0x7fc81015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc815850640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8100ad570 0x7fc81009bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 -- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81006abe0 msgr2=0x7fc81015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81006abe0 0x7fc81015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 -- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8100ad570 msgr2=0x7fc81009bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8100ad570 0x7fc81009bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 -- 172.21.15.60:0/179298291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8101f7c70 con 0x7fc81015dd80 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc815850640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8100ad570 0x7fc81009bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.211+0000 7fc816852640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc810150930 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc80800efc0 tx=0x7fc80800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.212+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808018020 con 0x7fc81015dd80 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.212+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808014480 con 0x7fc81015dd80 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.212+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808014870 con 0x7fc81015dd80 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.212+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc810160140 con 0x7fc81015dd80 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.212+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc810160700 con 0x7fc81015dd80 2024-04-25T08:37:22.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.213+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810073a00 con 0x7fc81015dd80 2024-04-25T08:37:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.214+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc808005d10 con 0x7fc81015dd80 2024-04-25T08:37:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.214+0000 7fc806ffd640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f406b500 0x7fc7f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.215+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc80808aac0 con 0x7fc81015dd80 2024-04-25T08:37:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.215+0000 7fc816051640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f406b500 0x7fc7f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.215+0000 7fc816051640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f406b500 0x7fc7f406d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc80c001780 tx=0x7fc80c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:22.214 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.220+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc808022070 con 0x7fc81015dd80 2024-04-25T08:37:22.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.404+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8100b0200 con 0x7fc81015dd80 2024-04-25T08:37:22.399 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.404+0000 7fc806ffd640 1 -- 172.21.15.60:0/179298291 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8101f7c70 con 0x7fc81015dd80 2024-04-25T08:37:22.399 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f406b500 msgr2=0x7fc7f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f406b500 0x7fc7f406d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc80c001780 tx=0x7fc80c009040 comp rx=0 tx=0).stop 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 msgr2=0x7fc810150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc810150930 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc80800efc0 tx=0x7fc80800c490 comp rx=0 tx=0).stop 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 shutdown_connections 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc81015dd80 0x7fc810150930 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f406b500 0x7fc7f406d9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8100ad570 0x7fc81009bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.401 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 --2- 172.21.15.60:0/179298291 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81006abe0 0x7fc81015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 >> 172.21.15.60:0/179298291 conn(0x7fc8100b5af0 msgr2=0x7fc8100b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:22.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 shutdown_connections 2024-04-25T08:37:22.402 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.406+0000 7fc8182b5640 1 -- 172.21.15.60:0/179298291 wait complete. 2024-04-25T08:37:22.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T08:37:22.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.519+0000 7f96c5770640 1 Processor -- start 2024-04-25T08:37:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.519+0000 7f96c5770640 1 -- start start 2024-04-25T08:37:22.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96c5770640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f96c015bf50 0x7f96c00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96c5770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c015c400 0x7f96c00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96c5770640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c015c930 con 0x7f96c01594f0 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96c5770640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00d12d0 con 0x7f96c015bf50 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96c5770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01598d0 con 0x7f96c015c400 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96be7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f96c015bf50 0x7f96c00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96be7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f96c015bf50 0x7f96c00d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41338/0 (socket says 172.21.15.60:41338) 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96be7fc640 1 -- 172.21.15.60:0/931926963 learned_addr learned my addr 172.21.15.60:0/931926963 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.520+0000 7f96bf7fe640 1 --2- 172.21.15.60:0/931926963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c015c400 0x7f96c00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96be7fc640 1 -- 172.21.15.60:0/931926963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c015c400 msgr2=0x7f96c00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96be7fc640 1 --2- 172.21.15.60:0/931926963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c015c400 0x7f96c00d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96be7fc640 1 -- 172.21.15.60:0/931926963 >> v1:172.21.15.100:6789/0 conn(0x7f96c01594f0 legacy=0x7f96c00b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96be7fc640 1 -- 172.21.15.60:0/931926963 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c015cc10 con 0x7f96c015bf50 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96be7fc640 1 --2- 172.21.15.60:0/931926963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c00d0850 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f96a80098a0 tx=0x7f96a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca265f2811cc5968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:22.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96bdffb640 1 -- 172.21.15.60:0/931926963 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8037070 con 0x7f96c015bf50 2024-04-25T08:37:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.521+0000 7f96bdffb640 1 -- 172.21.15.60:0/931926963 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8032440 con 0x7f96c015bf50 2024-04-25T08:37:22.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96bdffb640 1 -- 172.21.15.60:0/931926963 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8032810 con 0x7f96c015bf50 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 -- 172.21.15.60:0/931926963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 msgr2=0x7f96c00d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 --2- 172.21.15.60:0/931926963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c00d0850 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f96a80098a0 tx=0x7f96a802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 -- 172.21.15.60:0/931926963 shutdown_connections 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 --2- 172.21.15.60:0/931926963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c00d0850 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 --2- 172.21.15.60:0/931926963 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c015c400 0x7f96c00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 -- 172.21.15.60:0/931926963 >> 172.21.15.60:0/931926963 conn(0x7f96c006f520 msgr2=0x7f96c006f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 -- 172.21.15.60:0/931926963 shutdown_connections 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.522+0000 7f96c5770640 1 -- 172.21.15.60:0/931926963 wait complete. 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.523+0000 7f96c5770640 1 Processor -- start 2024-04-25T08:37:22.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.523+0000 7f96c5770640 1 -- start start 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96c5770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c01594f0 0x7f96c0154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96c5770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c0151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96c5770640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96c015c400 0x7f96c015ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96c5770640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0070e10 con 0x7f96c015bf50 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96c5770640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00b4c60 con 0x7f96c015c400 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96c5770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0070f90 con 0x7f96c01594f0 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96be7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c0151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96be7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c0151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41340/0 (socket says 172.21.15.60:41340) 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.524+0000 7f96be7fc640 1 -- 172.21.15.60:0/3904237327 learned_addr learned my addr 172.21.15.60:0/3904237327 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96be7fc640 1 -- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c01594f0 msgr2=0x7f96c0154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96be7fc640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c01594f0 0x7f96c0154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96bf7fe640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96c015c400 0x7f96c015ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96be7fc640 1 -- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96c015c400 msgr2=0x7f96c015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96be7fc640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96c015c400 0x7f96c015ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96be7fc640 1 -- 172.21.15.60:0/3904237327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c015efb0 con 0x7f96c015bf50 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96bf7fe640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96c015c400 0x7f96c015ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96be7fc640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c0151a90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f96a80090b0 tx=0x7f96a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8037040 con 0x7f96c015bf50 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8004230 con 0x7f96c015bf50 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c015da10 con 0x7f96c015bf50 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8003610 con 0x7f96c015bf50 2024-04-25T08:37:22.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.525+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c015df90 con 0x7f96c015bf50 2024-04-25T08:37:22.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.526+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c0073990 con 0x7f96c015bf50 2024-04-25T08:37:22.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.527+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96a80037b0 con 0x7f96c015bf50 2024-04-25T08:37:22.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.527+0000 7f96b77fe640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96a006b4d0 0x7f96a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:22.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.527+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f96a80aaee0 con 0x7f96c015bf50 2024-04-25T08:37:22.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.527+0000 7f96beffd640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96a006b4d0 0x7f96a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:22.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.528+0000 7f96beffd640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96a006b4d0 0x7f96a006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8006bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:22.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.531+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96a8079ff0 con 0x7f96c015bf50 2024-04-25T08:37:22.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:22.790+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f96c00700e0 con 0x7f96c015bf50 2024-04-25T08:37:23.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.770+0000 7f96b77fe640 1 -- 172.21.15.60:0/3904237327 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v398) v1 ==== 169+0+0 (secure 0 0 0) 0x7f96a80798f0 con 0x7f96c015bf50 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96a006b4d0 msgr2=0x7f96a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96a006b4d0 0x7f96a006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8006bc0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 msgr2=0x7f96c0151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c0151a90 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f96a80090b0 tx=0x7f96a80046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 shutdown_connections 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f96a006b4d0 0x7f96a006d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96c015c400 0x7f96c015ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f96c015bf50 0x7f96c0151a90 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 --2- 172.21.15.60:0/3904237327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f96c01594f0 0x7f96c0154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 >> 172.21.15.60:0/3904237327 conn(0x7f96c006f520 msgr2=0x7f96c00b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 shutdown_connections 2024-04-25T08:37:23.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.773+0000 7f96c5770640 1 -- 172.21.15.60:0/3904237327 wait complete. 2024-04-25T08:37:23.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:37:23.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.911+0000 7f3902bb6640 1 Processor -- start 2024-04-25T08:37:23.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.911+0000 7f3902bb6640 1 -- start start 2024-04-25T08:37:23.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3902bb6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38fc06c330 0x7f38fc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3902bb6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 0x7f38fc0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3902bb6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0d1d50 con 0x7f38fc158230 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3902bb6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc158610 con 0x7f38fc06c330 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3902bb6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc06bea0 con 0x7f38fc06c7e0 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3900952640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f38fc158230 0x7f38fc0d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52364/0 (socket says 172.21.15.60:52364) 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3900952640 1 -- 172.21.15.60:0/3464044637 learned_addr learned my addr 172.21.15.60:0/3464044637 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f3fff640 1 --2- 172.21.15.60:0/3464044637 >> v2:172.21.15.100:3300/0 conn(0x7f38fc06c330 0x7f38fc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f3901153640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 0x7f38fc0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096826493 0 0) 0x7f38fc0d1d50 con 0x7f38fc158230 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4003520 con 0x7f38fc158230 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976617874 0 0) 0x7f38e4003520 con 0x7f38fc158230 2024-04-25T08:37:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 msgr2=0x7f38fc0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 0x7f38fc0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 >> v2:172.21.15.100:3300/0 conn(0x7f38fc06c330 msgr2=0x7f38fc0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 --2- 172.21.15.60:0/3464044637 >> v2:172.21.15.100:3300/0 conn(0x7f38fc06c330 0x7f38fc0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.912+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0d2020 con 0x7f38fc158230 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f38f8001670 con 0x7f38fc158230 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 >> v1:172.21.15.60:6789/0 conn(0x7f38fc158230 legacy=0x7f38fc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 0x7f38e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38e4003f30 0x7f38e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4004d40 0x7f38e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc158610 con 0x7f38e4003f30 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc06bea0 con 0x7f38e4004d40 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400b9c0 con 0x7f38fc06c7e0 2024-04-25T08:37:23.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f3fff640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38e4003f30 0x7f38e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f3901153640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4004d40 0x7f38e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f3900952640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 0x7f38e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f3fff640 1 -- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 msgr2=0x7f38e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f3fff640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc06c7e0 0x7f38e40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f3fff640 1 -- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4004d40 msgr2=0x7f38e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f3fff640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4004d40 0x7f38e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.913+0000 7f38f3fff640 1 -- 172.21.15.60:0/3464044637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4003520 con 0x7f38e4003f30 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f38f3fff640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38e4003f30 0x7f38e4004800 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009ab0 tx=0x7f38e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ec1ab005e82217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec036070 con 0x7f38e4003f30 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec005990 con 0x7f38e4003f30 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f38f37fe640 1 -- 172.21.15.60:0/3464044637 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec009680 con 0x7f38e4003f30 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f3902bb6640 1 -- 172.21.15.60:0/3464044637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38e4003f30 msgr2=0x7f38e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3464044637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38e4003f30 0x7f38e4004800 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009ab0 tx=0x7f38e4005830 comp rx=0 tx=0).stop 2024-04-25T08:37:23.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f3901153640 1 -- 172.21.15.60:0/3464044637 reap_dead start 2024-04-25T08:37:23.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f3902bb6640 1 -- 172.21.15.60:0/3464044637 shutdown_connections 2024-04-25T08:37:23.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.914+0000 7f3902bb6640 1 -- 172.21.15.60:0/3464044637 >> 172.21.15.60:0/3464044637 conn(0x7f38fc061600 msgr2=0x7f38fc0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:23.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.915+0000 7f3902bb6640 1 -- 172.21.15.60:0/3464044637 shutdown_connections 2024-04-25T08:37:23.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.915+0000 7f3902bb6640 1 -- 172.21.15.60:0/3464044637 wait complete. 2024-04-25T08:37:23.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.916+0000 7f3902bb6640 1 Processor -- start 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.916+0000 7f3902bb6640 1 -- start start 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4003f30 0x7f38fc0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc0b27e0 0x7f38fc0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 0x7f38fc15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0d25e0 con 0x7f38fc15ae80 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc06bb30 con 0x7f38e4003f30 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc15e320 con 0x7f38fc0b27e0 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 0x7f38fc15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 0x7f38fc15b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41356/0 (socket says 172.21.15.60:41356) 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 -- 172.21.15.60:0/3847957124 learned_addr learned my addr 172.21.15.60:0/3847957124 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f38f3fff640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc0b27e0 0x7f38fc0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3900952640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4003f30 0x7f38fc0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 -- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc0b27e0 msgr2=0x7f38fc0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc0b27e0 0x7f38fc0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 -- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4003f30 msgr2=0x7f38fc0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4003f30 0x7f38fc0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 -- 172.21.15.60:0/3847957124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0b31a0 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3900952640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4003f30 0x7f38fc0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3901153640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 0x7f38fc15b2e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0d2860 tx=0x7f38f400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400fe10 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4015430 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0b3460 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4015820 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.917+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0b3a20 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.919+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc06de50 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.919+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38f40159c0 con 0x7f38fc15ae80 2024-04-25T08:37:23.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.919+0000 7f38f17fa640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e806b4d0 0x7f38e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:23.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.919+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f38f408ad60 con 0x7f38fc15ae80 2024-04-25T08:37:23.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.920+0000 7f3900952640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e806b4d0 0x7f38e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:23.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.920+0000 7f3900952640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e806b4d0 0x7f38e806d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f38f8001eb0 tx=0x7f38f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:23.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:23.923+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38f4019030 con 0x7f38fc15ae80 2024-04-25T08:37:24.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.103+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38fc073a00 con 0x7f38fc15ae80 2024-04-25T08:37:24.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.106+0000 7f38f17fa640 1 -- 172.21.15.60:0/3847957124 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38f4010040 con 0x7f38fc15ae80 2024-04-25T08:37:24.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e806b4d0 msgr2=0x7f38e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e806b4d0 0x7f38e806d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f38f8001eb0 tx=0x7f38f8009040 comp rx=0 tx=0).stop 2024-04-25T08:37:24.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 msgr2=0x7f38fc15b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 0x7f38fc15b2e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0d2860 tx=0x7f38f400f580 comp rx=0 tx=0).stop 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 shutdown_connections 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38fc15ae80 0x7f38fc15b2e0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38fc0b27e0 0x7f38fc0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f38e806b4d0 0x7f38e806d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 --2- 172.21.15.60:0/3847957124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38e4003f30 0x7f38fc0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 >> 172.21.15.60:0/3847957124 conn(0x7f38fc061600 msgr2=0x7f38fc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 shutdown_connections 2024-04-25T08:37:24.103 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.108+0000 7f3902bb6640 1 -- 172.21.15.60:0/3847957124 wait complete. 2024-04-25T08:37:24.113 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:37:24.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:37:24.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:37:24.204 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:37:24.204 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:24.204 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:24.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:37:24.220 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:37:24.220 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:24.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:37:24.239 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:24.239 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:24.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:37:24.276 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:37:24.276 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:24.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:37:24.296 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:37:24.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:24.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.422+0000 7ff59b246640 1 Processor -- start 2024-04-25T08:37:24.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.422+0000 7ff59b246640 1 -- start start 2024-04-25T08:37:24.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.423+0000 7ff59b246640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 0x7ff59406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.423+0000 7ff59b246640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff59415d370 0x7ff59415d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.423+0000 7ff59b246640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5941626e0 con 0x7ff5940ce330 2024-04-25T08:37:24.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.423+0000 7ff59b246640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59406b500 con 0x7ff59415d370 2024-04-25T08:37:24.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.423+0000 7ff59b246640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594070270 con 0x7ff59406abe0 2024-04-25T08:37:24.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 0x7ff59406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 0x7ff59406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:39306/0 (socket says 172.21.15.60:39306) 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff5997e3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff59415d370 0x7ff59415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 -- 172.21.15.60:0/2880607995 learned_addr learned my addr 172.21.15.60:0/2880607995 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff593fff640 1 --1- 172.21.15.60:0/2880607995 >> v1:172.21.15.100:6789/0 conn(0x7ff5940ce330 0x7ff59406fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:58622/0 (socket says 172.21.15.60:58622) 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff5937fe640 1 -- 172.21.15.60:0/2880607995 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088166536 0 0) 0x7ff5941626e0 con 0x7ff5940ce330 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff5937fe640 1 -- 172.21.15.60:0/2880607995 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff584003520 con 0x7ff5940ce330 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 -- 172.21.15.60:0/2880607995 >> v2:172.21.15.100:3300/0 conn(0x7ff59415d370 msgr2=0x7ff59415d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:24.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 --2- 172.21.15.60:0/2880607995 >> v2:172.21.15.100:3300/0 conn(0x7ff59415d370 0x7ff59415d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 -- 172.21.15.60:0/2880607995 >> v1:172.21.15.100:6789/0 conn(0x7ff5940ce330 legacy=0x7ff59406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.424+0000 7ff598fe2640 1 -- 172.21.15.60:0/2880607995 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940cd270 con 0x7ff59406abe0 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.425+0000 7ff598fe2640 1 --2- 172.21.15.60:0/2880607995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 0x7ff59406afc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff588009870 tx=0x7ff58802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b80c13456171b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.425+0000 7ff5937fe640 1 -- 172.21.15.60:0/2880607995 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588037070 con 0x7ff59406abe0 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.425+0000 7ff5937fe640 1 -- 172.21.15.60:0/2880607995 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff588032440 con 0x7ff59406abe0 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.425+0000 7ff5937fe640 1 -- 172.21.15.60:0/2880607995 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588032810 con 0x7ff59406abe0 2024-04-25T08:37:24.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 -- 172.21.15.60:0/2880607995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 msgr2=0x7ff59406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 --2- 172.21.15.60:0/2880607995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 0x7ff59406afc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff588009870 tx=0x7ff58802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 -- 172.21.15.60:0/2880607995 shutdown_connections 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 --2- 172.21.15.60:0/2880607995 >> v2:172.21.15.100:3300/0 conn(0x7ff59415d370 0x7ff59415d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 --2- 172.21.15.60:0/2880607995 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59406abe0 0x7ff59406afc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 -- 172.21.15.60:0/2880607995 >> 172.21.15.60:0/2880607995 conn(0x7ff5940b2220 msgr2=0x7ff5940b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 -- 172.21.15.60:0/2880607995 shutdown_connections 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.426+0000 7ff59b246640 1 -- 172.21.15.60:0/2880607995 wait complete. 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.427+0000 7ff59b246640 1 Processor -- start 2024-04-25T08:37:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.427+0000 7ff59b246640 1 -- start start 2024-04-25T08:37:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff59b246640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff59406abe0 0x7ff59415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff59b246640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 0x7ff594149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff59b246640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59415d370 0x7ff5941577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff59b246640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cdb10 con 0x7ff5940ce330 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff59b246640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5940cd830 con 0x7ff59406abe0 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff59b246640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594161780 con 0x7ff59415d370 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 0x7ff594149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 0x7ff594149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41358/0 (socket says 172.21.15.60:41358) 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 -- 172.21.15.60:0/3486462136 learned_addr learned my addr 172.21.15.60:0/3486462136 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff598fe2640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff59406abe0 0x7ff59415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff5997e3640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59415d370 0x7ff5941577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 -- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59415d370 msgr2=0x7ff5941577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59415d370 0x7ff5941577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 -- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff59406abe0 msgr2=0x7ff59415d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff59406abe0 0x7ff59415d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 -- 172.21.15.60:0/3486462136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59414a140 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.428+0000 7ff593fff640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 0x7ff594149c00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c460 tx=0x7ff58000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.429+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580018070 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.429+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5800074a0 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.429+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5941ea350 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.429+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580004a20 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.429+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5940b4840 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.430+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5940afe50 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.430+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff58001b070 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.431+0000 7ff5917fa640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff57806b500 0x7ff57806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.431+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff580089230 con 0x7ff5940ce330 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.431+0000 7ff598fe2640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff57806b500 0x7ff57806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.431+0000 7ff598fe2640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff57806b500 0x7ff57806d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff588034000 tx=0x7ff58802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:24.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.435+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff580007650 con 0x7ff5940ce330 2024-04-25T08:37:24.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.653+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5940b6670 con 0x7ff5940ce330 2024-04-25T08:37:24.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.654+0000 7ff5917fa640 1 -- 172.21.15.60:0/3486462136 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ff580057c90 con 0x7ff5940ce330 2024-04-25T08:37:24.648 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:24.649 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/345268928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":345268928},{"type":"v1","addr":"172.21.15.100:6837","nonce":345268928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7360,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3061794317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3061794317},{"type":"v1","addr":"172.21.15.100:6836","nonce":3061794317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":8181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3284461419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3284461419},{"type":"v1","addr":"172.21.15.60:6834","nonce":3284461419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8715,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/577565329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":577565329},{"type":"v1","addr":"172.21.15.60:6835","nonce":577565329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[]} 2024-04-25T08:37:24.649 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 64 2024-04-25T08:37:24.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff57806b500 msgr2=0x7ff57806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff57806b500 0x7ff57806d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff588034000 tx=0x7ff58802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:24.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 msgr2=0x7ff594149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 0x7ff594149c00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c460 tx=0x7ff58000c920 comp rx=0 tx=0).stop 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 shutdown_connections 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff59415d370 0x7ff5941577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff5940ce330 0x7ff594149c00 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff57806b500 0x7ff57806d9c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 --2- 172.21.15.60:0/3486462136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff59406abe0 0x7ff59415d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 >> 172.21.15.60:0/3486462136 conn(0x7ff5940b2220 msgr2=0x7ff59406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 shutdown_connections 2024-04-25T08:37:24.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.656+0000 7ff59b246640 1 -- 172.21.15.60:0/3486462136 wait complete. 2024-04-25T08:37:24.663 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:37:24.663 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:37:24.663 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:37:24.663 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:37:24.663 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:37:24.663 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:37:24.664 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:37:24.664 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:37:24.664 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:37:24.664 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:37:24.664 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:37:24.664 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:37:24.664 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:37:24.665 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:37:24.665 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:37:24.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:37:24.672 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:37:24.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:37:24.673 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:37:24.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:37:24.676 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:37:24.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:37:24.677 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:37:24.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:37:24.715 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:37:24.716 DEBUG:teuthology.parallel:result is None 2024-04-25T08:37:24.716 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:37:24.716 DEBUG:teuthology.parallel:result is None 2024-04-25T08:37:24.719 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:37:24.719 DEBUG:teuthology.parallel:result is None 2024-04-25T08:37:24.719 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:37:24.719 DEBUG:teuthology.parallel:result is None 2024-04-25T08:37:24.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:37:24.791 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:37:24.792 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:37:24.792 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:37:24.798+0000 7f31714bf640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T08:37:24.802 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:37:24.802 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:37:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.832+0000 7fccb6491640 1 Processor -- start 2024-04-25T08:37:24.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.832+0000 7fccb6491640 1 -- start start 2024-04-25T08:37:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.833+0000 7fccb6491640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fccb00ced80 0x7fccb00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.833+0000 7fccb6491640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb015e000 con 0x7fccb015dc20 2024-04-25T08:37:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.833+0000 7fccb6491640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb00d1de0 con 0x7fccb0161710 2024-04-25T08:37:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.833+0000 7fccb6491640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb0161c60 con 0x7fccb00ced80 2024-04-25T08:37:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.833+0000 7fccaffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fccb015dc20 0x7fccb00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52388/0 (socket says 172.21.15.60:52388) 2024-04-25T08:37:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.833+0000 7fccaffff640 1 -- 172.21.15.60:0/3598944627 learned_addr learned my addr 172.21.15.60:0/3598944627 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaf7fe640 1 --2- 172.21.15.60:0/3598944627 >> v2:172.21.15.100:3300/0 conn(0x7fccb00ced80 0x7fccb00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaeffd640 1 -- 172.21.15.60:0/3598944627 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952370108 0 0) 0x7fccb015e000 con 0x7fccb015dc20 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaeffd640 1 -- 172.21.15.60:0/3598944627 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc9c003520 con 0x7fccb015dc20 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaf7fe640 1 -- 172.21.15.60:0/3598944627 >> v1:172.21.15.100:6789/0 conn(0x7fccb0161710 legacy=0x7fccb00d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaeffd640 1 -- 172.21.15.60:0/3598944627 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545473635 0 0) 0x7fccb00d1de0 con 0x7fccb0161710 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaf7fe640 1 -- 172.21.15.60:0/3598944627 >> v1:172.21.15.60:6789/0 conn(0x7fccb015dc20 legacy=0x7fccb00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.834+0000 7fccaf7fe640 1 -- 172.21.15.60:0/3598944627 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0161f40 con 0x7fccb00ced80 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccaf7fe640 1 --2- 172.21.15.60:0/3598944627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb00ced80 0x7fccb00d22e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097a0 tx=0x7fcc9802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4513432ed329a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccaeffd640 1 -- 172.21.15.60:0/3598944627 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc98037070 con 0x7fccb00ced80 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccaeffd640 1 -- 172.21.15.60:0/3598944627 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9802ac30 con 0x7fccb00ced80 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccaeffd640 1 -- 172.21.15.60:0/3598944627 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00015f0 con 0x7fccb00ced80 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 -- 172.21.15.60:0/3598944627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb00ced80 msgr2=0x7fccb00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 --2- 172.21.15.60:0/3598944627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb00ced80 0x7fccb00d22e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcc980097a0 tx=0x7fcc9802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 -- 172.21.15.60:0/3598944627 shutdown_connections 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 --2- 172.21.15.60:0/3598944627 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb00ced80 0x7fccb00d22e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 -- 172.21.15.60:0/3598944627 >> 172.21.15.60:0/3598944627 conn(0x7fccb00d0420 msgr2=0x7fccb00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 -- 172.21.15.60:0/3598944627 shutdown_connections 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.835+0000 7fccb6491640 1 -- 172.21.15.60:0/3598944627 wait complete. 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 Processor -- start 2024-04-25T08:37:24.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 -- start start 2024-04-25T08:37:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 0x7fccb015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccb015dc20 0x7fccb00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb0161710 0x7fccb01e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00b5e50 con 0x7fccb00ced80 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0162500 con 0x7fccb0161710 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.837+0000 7fccb6491640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00b5fd0 con 0x7fccb015dc20 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 0x7fccb015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 0x7fccb015a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41414/0 (socket says 172.21.15.60:41414) 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 -- 172.21.15.60:0/337544538 learned_addr learned my addr 172.21.15.60:0/337544538 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaf7fe640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccb015dc20 0x7fccb00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccb4a2e640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb0161710 0x7fccb01e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 -- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccb015dc20 msgr2=0x7fccb00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccb015dc20 0x7fccb00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 -- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb0161710 msgr2=0x7fccb01e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:24.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb0161710 0x7fccb01e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 -- 172.21.15.60:0/337544538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00cafb0 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccb4a2e640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb0161710 0x7fccb01e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccaffff640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 0x7fccb015a030 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c780 tx=0x7fcca000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0018070 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca00074e0 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb00c8710 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.838+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00049e0 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.839+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb00c8cd0 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.840+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb0073a00 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.840+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcca001b070 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.840+0000 7fccacff9640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.840+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcca0089660 con 0x7fccb00ced80 2024-04-25T08:37:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.841+0000 7fccaf7fe640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:24.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.841+0000 7fccaf7fe640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc9006b4d0 0x7fcc9006d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcc98034000 tx=0x7fcc9802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:24.839 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:24.845+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcca0058000 con 0x7fccb00ced80 2024-04-25T08:37:24.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T08:37:24.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T08:37:24.879 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T08:37:25.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.037+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fccb00b6660 con 0x7fccb00ced80 2024-04-25T08:37:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.783+0000 7fccacff9640 1 -- 172.21.15.60:0/337544538 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcca005c0a0 con 0x7fccb00ced80 2024-04-25T08:37:25.777 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:37:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.786+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc9006b4d0 msgr2=0x7fcc9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.786+0000 7fccb6491640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc9006b4d0 0x7fcc9006d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcc98034000 tx=0x7fcc9802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.786+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 msgr2=0x7fccb015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.786+0000 7fccb6491640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 0x7fccb015a030 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c780 tx=0x7fcca000cc40 comp rx=0 tx=0).stop 2024-04-25T08:37:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 shutdown_connections 2024-04-25T08:37:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc9006b4d0 0x7fcc9006d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fccb0161710 0x7fccb01e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fccb015dc20 0x7fccb00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 --2- 172.21.15.60:0/337544538 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fccb00ced80 0x7fccb015a030 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 >> 172.21.15.60:0/337544538 conn(0x7fccb00d0420 msgr2=0x7fccb00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 shutdown_connections 2024-04-25T08:37:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.787+0000 7fccb6491640 1 -- 172.21.15.60:0/337544538 wait complete. 2024-04-25T08:37:25.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:37:25.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 Processor -- start 2024-04-25T08:37:25.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 -- start start 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6cbc15dd20 0x7f6cbc0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cbc15e100 0x7f6cbc0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc15eac0 con 0x7f6cbc15e590 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0d1c00 con 0x7f6cbc15e100 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.912+0000 7f6cc101f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc15eda0 con 0x7f6cbc15dd20 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cba59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cbc15e100 0x7f6cbc0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cbb59e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6cbc15e590 0x7f6cbc0d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52402/0 (socket says 172.21.15.60:52402) 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cba59c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6cbc15e100 0x7f6cbc0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41416/0 (socket says 172.21.15.60:41416) 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cbb59e640 1 -- 172.21.15.60:0/1574462076 learned_addr learned my addr 172.21.15.60:0/1574462076 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cba59c640 1 -- 172.21.15.60:0/1574462076 >> v2:172.21.15.100:3300/0 conn(0x7f6cbc15dd20 msgr2=0x7f6cbc0b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cbad9d640 1 --2- 172.21.15.60:0/1574462076 >> v2:172.21.15.100:3300/0 conn(0x7f6cbc15dd20 0x7f6cbc0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:25.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cb9d9b640 1 -- 172.21.15.60:0/1574462076 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959016421 0 0) 0x7f6cbc15eac0 con 0x7f6cbc15e590 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cba59c640 1 --2- 172.21.15.60:0/1574462076 >> v2:172.21.15.100:3300/0 conn(0x7f6cbc15dd20 0x7f6cbc0b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cba59c640 1 -- 172.21.15.60:0/1574462076 >> v1:172.21.15.60:6789/0 conn(0x7f6cbc15e590 legacy=0x7f6cbc0d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.913+0000 7f6cba59c640 1 -- 172.21.15.60:0/1574462076 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc15f080 con 0x7f6cbc15e100 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cba59c640 1 --2- 172.21.15.60:0/1574462076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15e100 0x7f6cbc0d0aa0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0b6520 tx=0x7f6cb002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56163bfbc320a9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cb9d9b640 1 -- 172.21.15.60:0/1574462076 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0037070 con 0x7f6cbc15e100 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cb9d9b640 1 -- 172.21.15.60:0/1574462076 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0032440 con 0x7f6cbc15e100 2024-04-25T08:37:25.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cb9d9b640 1 -- 172.21.15.60:0/1574462076 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb002ae60 con 0x7f6cbc15e100 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 -- 172.21.15.60:0/1574462076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15e100 msgr2=0x7f6cbc0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 --2- 172.21.15.60:0/1574462076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15e100 0x7f6cbc0d0aa0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0b6520 tx=0x7f6cb002a110 comp rx=0 tx=0).stop 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 -- 172.21.15.60:0/1574462076 shutdown_connections 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 --2- 172.21.15.60:0/1574462076 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15e100 0x7f6cbc0d0aa0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 --2- 172.21.15.60:0/1574462076 >> v2:172.21.15.100:3300/0 conn(0x7f6cbc15dd20 0x7f6cbc0b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 -- 172.21.15.60:0/1574462076 >> 172.21.15.60:0/1574462076 conn(0x7f6cbc061a20 msgr2=0x7f6cbc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 -- 172.21.15.60:0/1574462076 shutdown_connections 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.914+0000 7f6cc101f640 1 -- 172.21.15.60:0/1574462076 wait complete. 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.916+0000 7f6cc101f640 1 Processor -- start 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.916+0000 7f6cc101f640 1 -- start start 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cc101f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 0x7f6cbc0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cc101f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cbc15e100 0x7f6cbc1f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:25.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cc101f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cbc15e590 0x7f6cbc0d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cc101f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0d3500 con 0x7f6cbc15dd20 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cc101f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0d3380 con 0x7f6cbc15e590 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cc101f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0d3680 con 0x7f6cbc15e100 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 0x7f6cbc0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 0x7f6cbc0cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41420/0 (socket says 172.21.15.60:41420) 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 -- 172.21.15.60:0/900639593 learned_addr learned my addr 172.21.15.60:0/900639593 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cba59c640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cbc15e100 0x7f6cbc1f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbb59e640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cbc15e590 0x7f6cbc0d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 -- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cbc15e100 msgr2=0x7f6cbc1f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cbc15e100 0x7f6cbc1f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 -- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cbc15e590 msgr2=0x7f6cbc0d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:25.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cbc15e590 0x7f6cbc0d3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 -- 172.21.15.60:0/900639593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0cd670 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbb59e640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cbc15e590 0x7f6cbc0d3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6cbad9d640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 0x7f6cbc0cd130 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400c7f0 tx=0x7f6cb400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.917+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4015070 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.918+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4004240 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.918+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc1f3360 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.918+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4007620 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.918+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0cae30 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.919+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc073990 con 0x7f6cbc15dd20 2024-04-25T08:37:25.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.919+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cb40079d0 con 0x7f6cbc15dd20 2024-04-25T08:37:25.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.920+0000 7f6ca37fe640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ca806b4d0 0x7f6ca806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:25.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.920+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6cb408a390 con 0x7f6cbc15dd20 2024-04-25T08:37:25.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.920+0000 7f6cba59c640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ca806b4d0 0x7f6ca806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:25.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.920+0000 7f6cba59c640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ca806b4d0 0x7f6ca806d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0034000 tx=0x7f6cb002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:25.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:25.924+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cb4058d90 con 0x7f6cbc15dd20 2024-04-25T08:37:26.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.175+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6cbc0b9740 con 0x7f6cbc15dd20 2024-04-25T08:37:26.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.175+0000 7f6ca37fe640 1 -- 172.21.15.60:0/900639593 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6cb405ce30 con 0x7f6cbc15dd20 2024-04-25T08:37:26.169 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:26.169 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.170 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:37:26.171 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.172 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.173 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:37:26.174 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.175 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.176 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:37:26.177 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:37:26.178 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:37:26.179 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:37:26.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.181 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:26.182 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:37:26.183 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ca806b4d0 msgr2=0x7f6ca806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ca806b4d0 0x7f6ca806d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0034000 tx=0x7f6cb002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:26.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 msgr2=0x7f6cbc0cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 0x7f6cbc0cd130 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400c7f0 tx=0x7f6cb400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 shutdown_connections 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6ca806b4d0 0x7f6ca806d990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cbc15e590 0x7f6cbc0d3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6cbc15e100 0x7f6cbc1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 --2- 172.21.15.60:0/900639593 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6cbc15dd20 0x7f6cbc0cd130 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 >> 172.21.15.60:0/900639593 conn(0x7f6cbc061a20 msgr2=0x7f6cbc0b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 shutdown_connections 2024-04-25T08:37:26.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.177+0000 7f6cc101f640 1 -- 172.21.15.60:0/900639593 wait complete. 2024-04-25T08:37:26.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.292+0000 7f41bb5ab640 1 Processor -- start 2024-04-25T08:37:26.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.292+0000 7f41bb5ab640 1 -- start start 2024-04-25T08:37:26.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41bb5ab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41b40cfec0 0x7f41b40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41bb5ab640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f41b40ce3c0 0x7f41b406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41bb5ab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b4071480 con 0x7f41b4154ee0 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41bb5ab640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b40ce830 con 0x7f41b40cfec0 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41bb5ab640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b4072090 con 0x7f41b40ce3c0 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41b40cfec0 0x7f41b40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f41b40cfec0 0x7f41b40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41428/0 (socket says 172.21.15.60:41428) 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 -- 172.21.15.60:0/961397385 learned_addr learned my addr 172.21.15.60:0/961397385 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b9b48640 1 --2- 172.21.15.60:0/961397385 >> v2:172.21.15.100:3300/0 conn(0x7f41b40ce3c0 0x7f41b406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 -- 172.21.15.60:0/961397385 >> v2:172.21.15.100:3300/0 conn(0x7f41b40ce3c0 msgr2=0x7f41b406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 --2- 172.21.15.60:0/961397385 >> v2:172.21.15.100:3300/0 conn(0x7f41b40ce3c0 0x7f41b406d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 -- 172.21.15.60:0/961397385 >> v1:172.21.15.100:6789/0 conn(0x7f41b4154ee0 legacy=0x7f41b40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:26.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.293+0000 7f41b8b46640 1 -- 172.21.15.60:0/961397385 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4071760 con 0x7f41b40cfec0 2024-04-25T08:37:26.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.294+0000 7f41b8b46640 1 --2- 172.21.15.60:0/961397385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40cfec0 0x7f41b40cde80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f41b406f240 tx=0x7f41a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5007656f6b0cb96b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.289 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.294+0000 7f41abfff640 1 -- 172.21.15.60:0/961397385 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4037070 con 0x7f41b40cfec0 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.294+0000 7f41abfff640 1 -- 172.21.15.60:0/961397385 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a4032440 con 0x7f41b40cfec0 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.294+0000 7f41abfff640 1 -- 172.21.15.60:0/961397385 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4032810 con 0x7f41b40cfec0 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 -- 172.21.15.60:0/961397385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40cfec0 msgr2=0x7f41b40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/961397385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40cfec0 0x7f41b40cde80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f41b406f240 tx=0x7f41a402a230 comp rx=0 tx=0).stop 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 -- 172.21.15.60:0/961397385 shutdown_connections 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/961397385 >> v2:172.21.15.100:3300/0 conn(0x7f41b40ce3c0 0x7f41b406d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/961397385 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40cfec0 0x7f41b40cde80 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 -- 172.21.15.60:0/961397385 >> 172.21.15.60:0/961397385 conn(0x7f41b406abe0 msgr2=0x7f41b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 -- 172.21.15.60:0/961397385 shutdown_connections 2024-04-25T08:37:26.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.295+0000 7f41bb5ab640 1 -- 172.21.15.60:0/961397385 wait complete. 2024-04-25T08:37:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.296+0000 7f41bb5ab640 1 Processor -- start 2024-04-25T08:37:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.296+0000 7f41bb5ab640 1 -- start start 2024-04-25T08:37:26.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41bb5ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 0x7f41b4148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41bb5ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41b40cfec0 0x7f41b40cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41bb5ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41b4154ee0 0x7f41b40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41bb5ab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b4071d20 con 0x7f41b40ce3c0 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41bb5ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b4071230 con 0x7f41b4154ee0 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41bb5ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41552c0 con 0x7f41b40cfec0 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 0x7f41b4148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 0x7f41b4148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41436/0 (socket says 172.21.15.60:41436) 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 -- 172.21.15.60:0/1253210130 learned_addr learned my addr 172.21.15.60:0/1253210130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b8b46640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41b40cfec0 0x7f41b40cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9b48640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41b4154ee0 0x7f41b40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 -- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41b40cfec0 msgr2=0x7f41b40cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41b40cfec0 0x7f41b40cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 -- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41b4154ee0 msgr2=0x7f41b40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41b4154ee0 0x7f41b40cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 -- 172.21.15.60:0/1253210130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4148660 con 0x7f41b40ce3c0 2024-04-25T08:37:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.297+0000 7f41b9347640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 0x7f41b4148120 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b420 tx=0x7f41b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.298+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0011830 con 0x7f41b40ce3c0 2024-04-25T08:37:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.298+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0016070 con 0x7f41b40ce3c0 2024-04-25T08:37:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.298+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b4158470 con 0x7f41b40ce3c0 2024-04-25T08:37:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.298+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b0011c10 con 0x7f41b40ce3c0 2024-04-25T08:37:26.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.298+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b4158a30 con 0x7f41b40ce3c0 2024-04-25T08:37:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.299+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b4073990 con 0x7f41b40ce3c0 2024-04-25T08:37:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.299+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41b001b460 con 0x7f41b40ce3c0 2024-04-25T08:37:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.299+0000 7f41a9ffb640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f419806b4d0 0x7f419806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.300+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f41b00899e0 con 0x7f41b40ce3c0 2024-04-25T08:37:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.300+0000 7f41b8b46640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f419806b4d0 0x7f419806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.300+0000 7f41b8b46640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f419806b4d0 0x7f419806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f41a4034000 tx=0x7f41a402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.304+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41b0020020 con 0x7f41b40ce3c0 2024-04-25T08:37:26.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.484+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41b40cbe80 con 0x7f41b40ce3c0 2024-04-25T08:37:26.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.485+0000 7f41a9ffb640 1 -- 172.21.15.60:0/1253210130 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+13162 (secure 0 0 0) 0x7f41b4148660 con 0x7f41b40ce3c0 2024-04-25T08:37:26.479 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:26.479 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":119,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:25.795427+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:26.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f419806b4d0 msgr2=0x7f419806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f419806b4d0 0x7f419806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f41a4034000 tx=0x7f41a402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 msgr2=0x7f41b4148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 0x7f41b4148120 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b420 tx=0x7f41b000b8e0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 shutdown_connections 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f419806b4d0 0x7f419806d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41b4154ee0 0x7f41b40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41b40cfec0 0x7f41b40cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 --2- 172.21.15.60:0/1253210130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41b40ce3c0 0x7f41b4148120 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 >> 172.21.15.60:0/1253210130 conn(0x7f41b406abe0 msgr2=0x7f41b40af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 shutdown_connections 2024-04-25T08:37:26.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.487+0000 7f41bb5ab640 1 -- 172.21.15.60:0/1253210130 wait complete. 2024-04-25T08:37:26.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:37:26.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:37:26.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.601+0000 7f11d3d5c640 1 Processor -- start 2024-04-25T08:37:26.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.602+0000 7f11d3d5c640 1 -- start start 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.602+0000 7f11d3d5c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11cc1610f0 0x7f11cc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.602+0000 7f11d3d5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 0x7f11cc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.602+0000 7f11d3d5c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0c6960 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.602+0000 7f11d3d5c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc1614d0 con 0x7f11cc1610f0 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.602+0000 7f11d3d5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc162260 con 0x7f11cc1616f0 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 0x7f11cc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 0x7f11cc161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:39390/0 (socket says 172.21.15.60:39390) 2024-04-25T08:37:26.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 -- 172.21.15.60:0/681937299 learned_addr learned my addr 172.21.15.60:0/681937299 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d1af8640 1 --2- 172.21.15.60:0/681937299 >> v2:172.21.15.100:3300/0 conn(0x7f11cc1610f0 0x7f11cc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 -- 172.21.15.60:0/681937299 >> v2:172.21.15.100:3300/0 conn(0x7f11cc1610f0 msgr2=0x7f11cc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 --2- 172.21.15.60:0/681937299 >> v2:172.21.15.100:3300/0 conn(0x7f11cc1610f0 0x7f11cc06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 -- 172.21.15.60:0/681937299 >> v1:172.21.15.100:6789/0 conn(0x7f11cc0ce0a0 legacy=0x7f11cc0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.603+0000 7f11d12f7640 1 -- 172.21.15.60:0/681937299 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc162540 con 0x7f11cc1616f0 2024-04-25T08:37:26.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.604+0000 7f11d12f7640 1 --2- 172.21.15.60:0/681937299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 0x7f11cc161b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009870 tx=0x7f11bc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700365041dadfa4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.604+0000 7f11d0af6640 1 -- 172.21.15.60:0/681937299 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037070 con 0x7f11cc1616f0 2024-04-25T08:37:26.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.604+0000 7f11d0af6640 1 -- 172.21.15.60:0/681937299 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc032440 con 0x7f11cc1616f0 2024-04-25T08:37:26.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.604+0000 7f11d0af6640 1 -- 172.21.15.60:0/681937299 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc032810 con 0x7f11cc1616f0 2024-04-25T08:37:26.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.604+0000 7f11d3d5c640 1 -- 172.21.15.60:0/681937299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 msgr2=0x7f11cc161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.604+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/681937299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 0x7f11cc161b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009870 tx=0x7f11bc02a270 comp rx=0 tx=0).stop 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.605+0000 7f11d3d5c640 1 -- 172.21.15.60:0/681937299 shutdown_connections 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.605+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/681937299 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1616f0 0x7f11cc161b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.605+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/681937299 >> v2:172.21.15.100:3300/0 conn(0x7f11cc1610f0 0x7f11cc06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.605+0000 7f11d3d5c640 1 -- 172.21.15.60:0/681937299 >> 172.21.15.60:0/681937299 conn(0x7f11cc06f500 msgr2=0x7f11cc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.605+0000 7f11d3d5c640 1 -- 172.21.15.60:0/681937299 shutdown_connections 2024-04-25T08:37:26.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.605+0000 7f11d3d5c640 1 -- 172.21.15.60:0/681937299 wait complete. 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 Processor -- start 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 -- start start 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 0x7f11cc14a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1610f0 0x7f11cc160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11cc1616f0 0x7f11cc159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0cf690 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0cfce0 con 0x7f11cc1616f0 2024-04-25T08:37:26.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.606+0000 7f11d3d5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc06c700 con 0x7f11cc1610f0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 0x7f11cc14a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 0x7f11cc14a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41438/0 (socket says 172.21.15.60:41438) 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 -- 172.21.15.60:0/3709257909 learned_addr learned my addr 172.21.15.60:0/3709257909 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d22f9640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11cc1616f0 0x7f11cc159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d12f7640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1610f0 0x7f11cc160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 -- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1610f0 msgr2=0x7f11cc160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1610f0 0x7f11cc160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 -- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11cc1616f0 msgr2=0x7f11cc159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11cc1616f0 0x7f11cc159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 -- 172.21.15.60:0/3709257909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0b4a70 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d1af8640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 0x7f11cc14a460 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f11c800e9b0 tx=0x7f11c800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8017070 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c800c9a0 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc0b4d50 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c800cd90 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.607+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc0b5310 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.608+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc073a00 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.609+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11c801a070 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.609+0000 7f11c27fc640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.609+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f11c808a370 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.610+0000 7f11d12f7640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.610+0000 7f11d12f7640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11b006b4d0 0x7f11b006d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f11bc034000 tx=0x7f11bc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.614+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11c8058d60 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.796+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11cc0cf150 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.796+0000 7f11c27fc640 1 -- 172.21.15.60:0/3709257909 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11c805ce00 con 0x7f11cc0ce0a0 2024-04-25T08:37:26.790 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11b006b4d0 msgr2=0x7f11b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11b006b4d0 0x7f11b006d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f11bc034000 tx=0x7f11bc02e000 comp rx=0 tx=0).stop 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 msgr2=0x7f11cc14a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 0x7f11cc14a460 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f11c800e9b0 tx=0x7f11c800ee70 comp rx=0 tx=0).stop 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 shutdown_connections 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11b006b4d0 0x7f11b006d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11cc1616f0 0x7f11cc159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f11cc1610f0 0x7f11cc160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 --2- 172.21.15.60:0/3709257909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f11cc0ce0a0 0x7f11cc14a460 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 >> 172.21.15.60:0/3709257909 conn(0x7f11cc06f500 msgr2=0x7f11cc0d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:26.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 shutdown_connections 2024-04-25T08:37:26.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.798+0000 7f11d3d5c640 1 -- 172.21.15.60:0/3709257909 wait complete. 2024-04-25T08:37:26.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:37:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.911+0000 7f4ebe1b0640 1 Processor -- start 2024-04-25T08:37:26.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.911+0000 7f4ebe1b0640 1 -- start start 2024-04-25T08:37:26.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4ebe1b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4ebe1b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4eb80b92a0 0x7f4eb80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4ebe1b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80b97a0 con 0x7f4eb80b2cb0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4ebe1b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb815dfc0 con 0x7f4eb80b92a0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4ebe1b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80d1a00 con 0x7f4eb815dbe0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f4eb80b92a0 0x7f4eb80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4eb80b2cb0 0x7f4eb80b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52414/0 (socket says 172.21.15.60:52414) 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb6ffd640 1 -- 172.21.15.60:0/449539939 learned_addr learned my addr 172.21.15.60:0/449539939 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb77fe640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb7fff640 1 -- 172.21.15.60:0/449539939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 msgr2=0x7f4eb80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb7fff640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb7fff640 1 -- 172.21.15.60:0/449539939 >> v1:172.21.15.60:6789/0 conn(0x7f4eb80b2cb0 legacy=0x7f4eb80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb7fff640 1 -- 172.21.15.60:0/449539939 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80d1c50 con 0x7f4eb80b92a0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb67fc640 1 -- 172.21.15.60:0/449539939 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868932310 0 0) 0x7f4eb80b97a0 con 0x7f4eb80b2cb0 2024-04-25T08:37:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.912+0000 7f4eb77fe640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.913+0000 7f4eb7fff640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80d0e50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009d90 tx=0x7f4eb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fca873c13bfd1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.913+0000 7f4eb67fc640 1 -- 172.21.15.60:0/449539939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007740 con 0x7f4eb80b92a0 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.913+0000 7f4eb67fc640 1 -- 172.21.15.60:0/449539939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb00078e0 con 0x7f4eb80b92a0 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.913+0000 7f4eb67fc640 1 -- 172.21.15.60:0/449539939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0007cb0 con 0x7f4eb80b92a0 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/449539939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 msgr2=0x7f4eb80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80d0e50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009d90 tx=0x7f4eb002a660 comp rx=0 tx=0).stop 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/449539939 shutdown_connections 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80d0e50 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/449539939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/449539939 >> 172.21.15.60:0/449539939 conn(0x7f4eb8061a20 msgr2=0x7f4eb8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:26.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/449539939 shutdown_connections 2024-04-25T08:37:26.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.914+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/449539939 wait complete. 2024-04-25T08:37:26.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.915+0000 7f4ebe1b0640 1 Processor -- start 2024-04-25T08:37:26.910 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.915+0000 7f4ebe1b0640 1 -- start start 2024-04-25T08:37:26.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4ebe1b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb80b2cb0 0x7f4eb80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4ebe1b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4ebe1b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb8160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4ebe1b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80b6ab0 con 0x7f4eb80b92a0 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4ebe1b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80d2210 con 0x7f4eb80b2cb0 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4ebe1b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb80b6c30 con 0x7f4eb815dbe0 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41466/0 (socket says 172.21.15.60:41466) 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 -- 172.21.15.60:0/1962797164 learned_addr learned my addr 172.21.15.60:0/1962797164 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb7fff640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb8160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb77fe640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb80b2cb0 0x7f4eb80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 -- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 msgr2=0x7f4eb8160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb8160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 -- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb80b2cb0 msgr2=0x7f4eb80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb80b2cb0 0x7f4eb80c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 -- 172.21.15.60:0/1962797164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80b3c80 con 0x7f4eb80b92a0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb7fff640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb8160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb77fe640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb80b2cb0 0x7f4eb80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.916+0000 7f4eb6ffd640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80b3270 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00c500 tx=0x7f4eac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.917+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac018070 con 0x7f4eb80b92a0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.917+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac007580 con 0x7f4eb80b92a0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.917+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb80b3f60 con 0x7f4eb80b92a0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.917+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac004a20 con 0x7f4eb80b92a0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.917+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb80b4520 con 0x7f4eb80b92a0 2024-04-25T08:37:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.918+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb8073990 con 0x7f4eb80b92a0 2024-04-25T08:37:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.918+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4eac01b070 con 0x7f4eb80b92a0 2024-04-25T08:37:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.919+0000 7f4e9ffff640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ea406b500 0x7f4ea406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.919+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4eac0892b0 con 0x7f4eb80b92a0 2024-04-25T08:37:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.919+0000 7f4eb77fe640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ea406b500 0x7f4ea406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.919+0000 7f4eb77fe640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ea406b500 0x7f4ea406d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8006fd0 tx=0x7f4ea8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:26.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:26.923+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4eac058360 con 0x7f4eb80b92a0 2024-04-25T08:37:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.105+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4eb806b080 con 0x7f4eb80b92a0 2024-04-25T08:37:27.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.105+0000 7f4e9ffff640 1 -- 172.21.15.60:0/1962797164 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v119) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4eac057c60 con 0x7f4eb80b92a0 2024-04-25T08:37:27.099 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:27.100 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:37:27.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:37:27.100 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:37:27.100 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:37:27.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:27.100 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ea406b500 msgr2=0x7f4ea406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ea406b500 0x7f4ea406d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8006fd0 tx=0x7f4ea8008040 comp rx=0 tx=0).stop 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 msgr2=0x7f4eb80b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80b3270 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00c500 tx=0x7f4eac00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 shutdown_connections 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eb815dbe0 0x7f4eb8160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eb80b92a0 0x7f4eb80b3270 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ea406b500 0x7f4ea406d9c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 --2- 172.21.15.60:0/1962797164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eb80b2cb0 0x7f4eb80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 >> 172.21.15.60:0/1962797164 conn(0x7f4eb8061a20 msgr2=0x7f4eb80b83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 shutdown_connections 2024-04-25T08:37:27.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.107+0000 7f4ebe1b0640 1 -- 172.21.15.60:0/1962797164 wait complete. 2024-04-25T08:37:27.112 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:37:27.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:37:27.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.223+0000 7efc939d5640 1 Processor -- start 2024-04-25T08:37:27.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.223+0000 7efc939d5640 1 -- start start 2024-04-25T08:37:27.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc939d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc8c1610f0 0x7efc8c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc939d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c0c6960 con 0x7efc8c0ce7b0 2024-04-25T08:37:27.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc939d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8c162260 con 0x7efc8c1616f0 2024-04-25T08:37:27.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc939d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8c1614d0 con 0x7efc8c1610f0 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91771640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc8c1610f0 0x7efc8c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91771640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efc8c1610f0 0x7efc8c06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41476/0 (socket says 172.21.15.60:41476) 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91f72640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efc8c0ce7b0 0x7efc8c0d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52422/0 (socket says 172.21.15.60:52422) 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91771640 1 -- 172.21.15.60:0/3835763319 learned_addr learned my addr 172.21.15.60:0/3835763319 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91771640 1 -- 172.21.15.60:0/3835763319 >> v1:172.21.15.100:6789/0 conn(0x7efc8c1616f0 legacy=0x7efc8c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91771640 1 -- 172.21.15.60:0/3835763319 >> v1:172.21.15.60:6789/0 conn(0x7efc8c0ce7b0 legacy=0x7efc8c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc83fff640 1 -- 172.21.15.60:0/3835763319 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189725154 0 0) 0x7efc8c0c6960 con 0x7efc8c0ce7b0 2024-04-25T08:37:27.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.224+0000 7efc91771640 1 -- 172.21.15.60:0/3835763319 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c162540 con 0x7efc8c1610f0 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc91771640 1 --2- 172.21.15.60:0/3835763319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c06c1c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7efc88009870 tx=0x7efc8802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd764a5a4c51cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc83fff640 1 -- 172.21.15.60:0/3835763319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88037070 con 0x7efc8c1610f0 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc83fff640 1 -- 172.21.15.60:0/3835763319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc88032440 con 0x7efc8c1610f0 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc83fff640 1 -- 172.21.15.60:0/3835763319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88032810 con 0x7efc8c1610f0 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc939d5640 1 -- 172.21.15.60:0/3835763319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 msgr2=0x7efc8c06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc939d5640 1 --2- 172.21.15.60:0/3835763319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c06c1c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7efc88009870 tx=0x7efc8802a2a0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.225+0000 7efc939d5640 1 -- 172.21.15.60:0/3835763319 shutdown_connections 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.226+0000 7efc939d5640 1 --2- 172.21.15.60:0/3835763319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c06c1c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.226+0000 7efc939d5640 1 -- 172.21.15.60:0/3835763319 >> 172.21.15.60:0/3835763319 conn(0x7efc8c06f500 msgr2=0x7efc8c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.226+0000 7efc939d5640 1 -- 172.21.15.60:0/3835763319 shutdown_connections 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.226+0000 7efc939d5640 1 -- 172.21.15.60:0/3835763319 wait complete. 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 Processor -- start 2024-04-25T08:37:27.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 -- start start 2024-04-25T08:37:27.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc8c0ce7b0 0x7efc8c0d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8c1616f0 0x7efc8c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0cf690 con 0x7efc8c1610f0 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c0cfce0 con 0x7efc8c0ce7b0 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.227+0000 7efc939d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c06c700 con 0x7efc8c1616f0 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c14aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41492/0 (socket says 172.21.15.60:41492) 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 -- 172.21.15.60:0/917404727 learned_addr learned my addr 172.21.15.60:0/917404727 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc91f72640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8c1616f0 0x7efc8c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc91771640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc8c0ce7b0 0x7efc8c0d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 -- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8c1616f0 msgr2=0x7efc8c14b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8c1616f0 0x7efc8c14b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 -- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc8c0ce7b0 msgr2=0x7efc8c0d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc8c0ce7b0 0x7efc8c0d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 -- 172.21.15.60:0/917404727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c0b0400 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc90f70640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c14aeb0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c430 tx=0x7efc7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c011790 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c017070 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c011bb0 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.228+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c0b06e0 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.229+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c0b0ca0 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.229+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8c073a00 con 0x7efc8c1610f0 2024-04-25T08:37:27.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.230+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc7c01c460 con 0x7efc8c1610f0 2024-04-25T08:37:27.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.230+0000 7efc81ffb640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc7006b4d0 0x7efc7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.230+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efc7c089f80 con 0x7efc8c1610f0 2024-04-25T08:37:27.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.231+0000 7efc91771640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc7006b4d0 0x7efc7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.231+0000 7efc91771640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc7006b4d0 0x7efc7006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efc88034000 tx=0x7efc8802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:27.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.235+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc7c058f30 con 0x7efc8c1610f0 2024-04-25T08:37:27.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:27.390 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:27.408 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.414+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efc8c0d25a0 con 0x7efc8c1610f0 2024-04-25T08:37:27.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.784+0000 7efc81ffb640 1 -- 172.21.15.60:0/917404727 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v120) v1 ==== 138+0+0 (secure 0 0 0) 0x7efc7c058830 con 0x7efc8c1610f0 2024-04-25T08:37:27.779 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:37:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.787+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc7006b4d0 msgr2=0x7efc7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.787+0000 7efc939d5640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc7006b4d0 0x7efc7006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efc88034000 tx=0x7efc8802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 msgr2=0x7efc8c14aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c14aeb0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c430 tx=0x7efc7c00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 shutdown_connections 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc8c1616f0 0x7efc8c14b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efc8c1610f0 0x7efc8c14aeb0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efc7006b4d0 0x7efc7006d990 secure :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7efc88034000 tx=0x7efc8802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 --2- 172.21.15.60:0/917404727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc8c0ce7b0 0x7efc8c0d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 >> 172.21.15.60:0/917404727 conn(0x7efc8c06f500 msgr2=0x7efc8c0c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 shutdown_connections 2024-04-25T08:37:27.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.788+0000 7efc939d5640 1 -- 172.21.15.60:0/917404727 wait complete. 2024-04-25T08:37:27.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:37:27.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.913+0000 7ff75a69a640 1 Processor -- start 2024-04-25T08:37:27.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.914+0000 7ff75a69a640 1 -- start start 2024-04-25T08:37:27.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.914+0000 7ff75a69a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 0x7ff7540cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.914+0000 7ff75a69a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff754072280 con 0x7ff7540ce230 2024-04-25T08:37:27.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.914+0000 7ff75a69a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7540714c0 con 0x7ff7540cc4f0 2024-04-25T08:37:27.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.914+0000 7ff75a69a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75415bdc0 con 0x7ff75415b980 2024-04-25T08:37:27.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff7537fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff7540ce230 0x7ff75415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52438/0 (socket says 172.21.15.60:52438) 2024-04-25T08:37:27.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff7537fe640 1 -- 172.21.15.60:0/517183311 learned_addr learned my addr 172.21.15.60:0/517183311 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff758c37640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 0x7ff7540cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170409855 0 0) 0x7ff754072280 con 0x7ff7540ce230 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff73c005730 con 0x7ff7540ce230 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254240005 0 0) 0x7ff73c005730 con 0x7ff7540ce230 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 msgr2=0x7ff7540cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 0x7ff7540cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 >> v1:172.21.15.100:6789/0 conn(0x7ff7540cc4f0 legacy=0x7ff7540cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7540717a0 con 0x7ff7540ce230 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ff748001670 con 0x7ff7540ce230 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 >> v1:172.21.15.60:6789/0 conn(0x7ff7540ce230 legacy=0x7ff75415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff73c005bf0 0x7ff73c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 0x7ff73c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff73c006f50 0x7ff73c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75415bdc0 con 0x7ff73c005bf0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c00dc80 con 0x7ff73c006f50 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c00de80 con 0x7ff75415b980 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff753fff640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff73c005bf0 0x7ff73c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff758c37640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff73c006f50 0x7ff73c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.915+0000 7ff7537fe640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 0x7ff73c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff753fff640 1 -- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 msgr2=0x7ff73c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff753fff640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff75415b980 0x7ff73c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff753fff640 1 -- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff73c006f50 msgr2=0x7ff73c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff753fff640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff73c006f50 0x7ff73c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff753fff640 1 -- 172.21.15.60:0/517183311 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c005730 con 0x7ff73c005bf0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff753fff640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff73c005bf0 0x7ff73c006010 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff744001720 tx=0x7ff73c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db250c53ce07fd8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.916+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744001370 con 0x7ff73c005bf0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7440052d0 con 0x7ff73c005bf0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff752ffd640 1 -- 172.21.15.60:0/517183311 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74402abc0 con 0x7ff73c005bf0 2024-04-25T08:37:27.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff75a69a640 1 -- 172.21.15.60:0/517183311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff73c005bf0 msgr2=0x7ff73c006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff75a69a640 1 --2- 172.21.15.60:0/517183311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff73c005bf0 0x7ff73c006010 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff744001720 tx=0x7ff73c007a40 comp rx=0 tx=0).stop 2024-04-25T08:37:27.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff758c37640 1 -- 172.21.15.60:0/517183311 reap_dead start 2024-04-25T08:37:27.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff75a69a640 1 -- 172.21.15.60:0/517183311 shutdown_connections 2024-04-25T08:37:27.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff75a69a640 1 -- 172.21.15.60:0/517183311 >> 172.21.15.60:0/517183311 conn(0x7ff75406abe0 msgr2=0x7ff75406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:27.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff75a69a640 1 -- 172.21.15.60:0/517183311 shutdown_connections 2024-04-25T08:37:27.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.917+0000 7ff75a69a640 1 -- 172.21.15.60:0/517183311 wait complete. 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.918+0000 7ff75a69a640 1 Processor -- start 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 -- start start 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff73c005bf0 0x7ff754154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 0x7ff7541532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff754153830 0x7ff7540b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754071d60 con 0x7ff754143100 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754071270 con 0x7ff754153830 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff75a69a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75406e8a0 con 0x7ff73c005bf0 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff7537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 0x7ff7541532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff7537fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 0x7ff7541532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41508/0 (socket says 172.21.15.60:41508) 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.919+0000 7ff7537fe640 1 -- 172.21.15.60:0/1687913250 learned_addr learned my addr 172.21.15.60:0/1687913250 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff7537fe640 1 -- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff73c005bf0 msgr2=0x7ff754154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff753fff640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff73c005bf0 0x7ff754154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff758c37640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff754153830 0x7ff7540b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff7537fe640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff73c005bf0 0x7ff754154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff7537fe640 1 -- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff754153830 msgr2=0x7ff7540b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff7537fe640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff754153830 0x7ff7540b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff7537fe640 1 -- 172.21.15.60:0/1687913250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754153db0 con 0x7ff754143100 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff753fff640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff73c005bf0 0x7ff754154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff7537fe640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 0x7ff7541532f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff748001b80 tx=0x7ff748007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800f960 con 0x7ff754143100 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748018070 con 0x7ff754143100 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff754154090 con 0x7ff754143100 2024-04-25T08:37:27.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800fd60 con 0x7ff754143100 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.920+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff754154650 con 0x7ff754143100 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.921+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff754073a00 con 0x7ff754143100 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.921+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff74801d460 con 0x7ff754143100 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.922+0000 7ff750ff9640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff74006b500 0x7ff74006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.922+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff74808a160 con 0x7ff754143100 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.922+0000 7ff753fff640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff74006b500 0x7ff74006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:27.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.923+0000 7ff753fff640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff74006b500 0x7ff74006d9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff744032000 tx=0x7ff74402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:27.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:27.926+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7480590c0 con 0x7ff754143100 2024-04-25T08:37:28.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.107+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff75406e520 con 0x7ff754143100 2024-04-25T08:37:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.790+0000 7ff750ff9640 1 -- 172.21.15.60:0/1687913250 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v121) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff7480589c0 con 0x7ff754143100 2024-04-25T08:37:28.784 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:37:28.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.795+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff74006b500 msgr2=0x7ff74006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:28.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.795+0000 7ff75a69a640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff74006b500 0x7ff74006d9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff744032000 tx=0x7ff74402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:28.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.795+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 msgr2=0x7ff7541532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:28.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.795+0000 7ff75a69a640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 0x7ff7541532f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff748001b80 tx=0x7ff748007720 comp rx=0 tx=0).stop 2024-04-25T08:37:28.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.796+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 shutdown_connections 2024-04-25T08:37:28.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.796+0000 7ff75a69a640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff74006b500 0x7ff74006d9c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.796+0000 7ff75a69a640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff754153830 0x7ff7540b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.796+0000 7ff75a69a640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff754143100 0x7ff7541532f0 secure :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff748001b80 tx=0x7ff748007720 comp rx=0 tx=0).stop 2024-04-25T08:37:28.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.796+0000 7ff75a69a640 1 --2- 172.21.15.60:0/1687913250 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff73c005bf0 0x7ff754154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.796+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 >> 172.21.15.60:0/1687913250 conn(0x7ff75406abe0 msgr2=0x7ff75415b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:28.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.801+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 shutdown_connections 2024-04-25T08:37:28.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.805+0000 7ff75a69a640 1 -- 172.21.15.60:0/1687913250 wait complete. 2024-04-25T08:37:28.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:37:28.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.933+0000 7fb67645a640 1 Processor -- start 2024-04-25T08:37:28.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.933+0000 7fb67645a640 1 -- start start 2024-04-25T08:37:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb67645a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb67015b7d0 0x7fb6700cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb67645a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 0x7fb6700b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb67645a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67006cf50 con 0x7fb6700b5620 2024-04-25T08:37:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb67645a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6700ce530 con 0x7fb67015b7d0 2024-04-25T08:37:28.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb67645a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700ce810 con 0x7fb67006c9e0 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb66f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb6700b5620 0x7fb67006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52452/0 (socket says 172.21.15.60:52452) 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb66f7fe640 1 -- 172.21.15.60:0/2779604583 learned_addr learned my addr 172.21.15.60:0/2779604583 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb66ffff640 1 --2- 172.21.15.60:0/2779604583 >> v2:172.21.15.100:3300/0 conn(0x7fb67015b7d0 0x7fb6700cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 0x7fb6700b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.934+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562583789 0 0) 0x7fb67006cf50 con 0x7fb6700b5620 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb658003520 con 0x7fb6700b5620 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1046857635 0 0) 0x7fb658003520 con 0x7fb6700b5620 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 msgr2=0x7fb6700b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 0x7fb6700b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 >> v2:172.21.15.100:3300/0 conn(0x7fb67015b7d0 msgr2=0x7fb6700cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 --2- 172.21.15.60:0/2779604583 >> v2:172.21.15.100:3300/0 conn(0x7fb67015b7d0 0x7fb6700cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700ceaf0 con 0x7fb6700b5620 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb660001690 con 0x7fb6700b5620 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 >> v1:172.21.15.60:6789/0 conn(0x7fb6700b5620 legacy=0x7fb67006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6580039b0 0x7fb658003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 0x7fb658004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb658004d40 0x7fb6580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700ce530 con 0x7fb658004d40 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700ce810 con 0x7fb6580039b0 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65800b9c0 con 0x7fb67006c9e0 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb658004d40 0x7fb6580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66ffff640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6580039b0 0x7fb658003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb66f7fe640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 0x7fb658004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb6749f7640 1 -- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 msgr2=0x7fb658004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb67006c9e0 0x7fb658004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb6749f7640 1 -- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6580039b0 msgr2=0x7fb658003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6580039b0 0x7fb658003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.935+0000 7fb6749f7640 1 -- 172.21.15.60:0/2779604583 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658003520 con 0x7fb658004d40 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.936+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb658004d40 0x7fb6580051c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb66800e170 tx=0x7fb658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4524f8c567fe0577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.936+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668034020 con 0x7fb658004d40 2024-04-25T08:37:28.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.936+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6680305f0 con 0x7fb658004d40 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.936+0000 7fb66effd640 1 -- 172.21.15.60:0/2779604583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668009c10 con 0x7fb658004d40 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb67645a640 1 -- 172.21.15.60:0/2779604583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb658004d40 msgr2=0x7fb6580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb67645a640 1 --2- 172.21.15.60:0/2779604583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb658004d40 0x7fb6580051c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb66800e170 tx=0x7fb658005830 comp rx=0 tx=0).stop 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb6749f7640 1 -- 172.21.15.60:0/2779604583 reap_dead start 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb67645a640 1 -- 172.21.15.60:0/2779604583 shutdown_connections 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb67645a640 1 -- 172.21.15.60:0/2779604583 >> 172.21.15.60:0/2779604583 conn(0x7fb6700d0800 msgr2=0x7fb6700d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb67645a640 1 -- 172.21.15.60:0/2779604583 shutdown_connections 2024-04-25T08:37:28.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.937+0000 7fb67645a640 1 -- 172.21.15.60:0/2779604583 wait complete. 2024-04-25T08:37:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.938+0000 7fb67645a640 1 Processor -- start 2024-04-25T08:37:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.938+0000 7fb67645a640 1 -- start start 2024-04-25T08:37:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb67645a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6700b2880 0x7fb6700c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb67645a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 0x7fb6700c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb67645a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6700ca250 0x7fb6700cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb67645a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700cf6d0 con 0x7fb6700c98b0 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb67645a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700cf550 con 0x7fb6700ca250 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb67645a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67015bbb0 con 0x7fb6700b2880 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 0x7fb6700c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 0x7fb6700c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41520/0 (socket says 172.21.15.60:41520) 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 -- 172.21.15.60:0/2578800230 learned_addr learned my addr 172.21.15.60:0/2578800230 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66ffff640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6700b2880 0x7fb6700c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6700ca250 0x7fb6700cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 -- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6700b2880 msgr2=0x7fb6700c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6700b2880 0x7fb6700c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 -- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6700ca250 msgr2=0x7fb6700cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:28.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6700ca250 0x7fb6700cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 -- 172.21.15.60:0/2578800230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700cb1f0 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb6749f7640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6700ca250 0x7fb6700cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.939+0000 7fb66f7fe640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 0x7fb6700c9d10 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb6600072f0 tx=0x7fb6600076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.940+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000f9c0 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.940+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660018070 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.940+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6700b85d0 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.940+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000fda0 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.940+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6700cb490 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.941+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6700d0c10 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.941+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb66001d460 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.942+0000 7fb66cff9640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb65c06b4d0 0x7fb65c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.942+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb66008a3d0 con 0x7fb6700c98b0 2024-04-25T08:37:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.942+0000 7fb66ffff640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb65c06b4d0 0x7fb65c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.943+0000 7fb66ffff640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb65c06b4d0 0x7fb65c06d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb664009bc0 tx=0x7fb664009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:28.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:28.947+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6700d0c10 con 0x7fb6700c98b0 2024-04-25T08:37:29.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.164+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb670073a00 con 0x7fb6700c98b0 2024-04-25T08:37:29.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.805+0000 7fb66cff9640 1 -- 172.21.15.60:0/2578800230 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb6600591b0 con 0x7fb6700c98b0 2024-04-25T08:37:29.800 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T08:37:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.817+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb65c06b4d0 msgr2=0x7fb65c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.817+0000 7fb67645a640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb65c06b4d0 0x7fb65c06d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb664009bc0 tx=0x7fb664009190 comp rx=0 tx=0).stop 2024-04-25T08:37:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.817+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 msgr2=0x7fb6700c9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:29.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.817+0000 7fb67645a640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 0x7fb6700c9d10 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb6600072f0 tx=0x7fb6600076f0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.818+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 shutdown_connections 2024-04-25T08:37:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.818+0000 7fb67645a640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb65c06b4d0 0x7fb65c06d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.818+0000 7fb67645a640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6700ca250 0x7fb6700cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.818+0000 7fb67645a640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6700c98b0 0x7fb6700c9d10 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.818+0000 7fb67645a640 1 --2- 172.21.15.60:0/2578800230 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb6700b2880 0x7fb6700c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.818+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 >> 172.21.15.60:0/2578800230 conn(0x7fb6700d0800 msgr2=0x7fb6700b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:29.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.819+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 shutdown_connections 2024-04-25T08:37:29.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.819+0000 7fb67645a640 1 -- 172.21.15.60:0/2578800230 wait complete. 2024-04-25T08:37:29.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.942+0000 7f95ce883640 1 Processor -- start 2024-04-25T08:37:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.942+0000 7f95ce883640 1 -- start start 2024-04-25T08:37:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95ce883640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95c8164970 0x7f95c8166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95ce883640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c8144b30 con 0x7f95c8144750 2024-04-25T08:37:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95ce883640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c80712d0 con 0x7f95c80ce920 2024-04-25T08:37:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95ce883640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c80715b0 con 0x7f95c8164970 2024-04-25T08:37:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95cce20640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95c8164970 0x7f95c8166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:29.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95cce20640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f95c8164970 0x7f95c8166d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41522/0 (socket says 172.21.15.60:41522) 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95cce20640 1 -- 172.21.15.60:0/2085668574 learned_addr learned my addr 172.21.15.60:0/2085668574 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95cce20640 1 -- 172.21.15.60:0/2085668574 >> v1:172.21.15.100:6789/0 conn(0x7f95c80ce920 legacy=0x7f95c80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95cce20640 1 -- 172.21.15.60:0/2085668574 >> v1:172.21.15.60:6789/0 conn(0x7f95c8144750 legacy=0x7f95c806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95cce20640 1 -- 172.21.15.60:0/2085668574 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8071890 con 0x7f95c8164970 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.943+0000 7f95c6ffd640 1 -- 172.21.15.60:0/2085668574 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967145378 0 0) 0x7f95c8144b30 con 0x7f95c8144750 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.944+0000 7f95cce20640 1 --2- 172.21.15.60:0/2085668574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8166d60 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009e50 tx=0x7f95c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa678fb6c07caa3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.944+0000 7f95c6ffd640 1 -- 172.21.15.60:0/2085668574 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0036070 con 0x7f95c8164970 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.944+0000 7f95c6ffd640 1 -- 172.21.15.60:0/2085668574 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c0031600 con 0x7f95c8164970 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.944+0000 7f95c6ffd640 1 -- 172.21.15.60:0/2085668574 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c00319d0 con 0x7f95c8164970 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 -- 172.21.15.60:0/2085668574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 msgr2=0x7f95c8166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 --2- 172.21.15.60:0/2085668574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8166d60 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009e50 tx=0x7f95c002a730 comp rx=0 tx=0).stop 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 -- 172.21.15.60:0/2085668574 shutdown_connections 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 --2- 172.21.15.60:0/2085668574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8166d60 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 -- 172.21.15.60:0/2085668574 >> 172.21.15.60:0/2085668574 conn(0x7f95c806abe0 msgr2=0x7f95c806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:29.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 -- 172.21.15.60:0/2085668574 shutdown_connections 2024-04-25T08:37:29.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.945+0000 7f95ce883640 1 -- 172.21.15.60:0/2085668574 wait complete. 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.946+0000 7f95ce883640 1 Processor -- start 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.946+0000 7f95ce883640 1 -- start start 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95ce883640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80ce920 0x7f95c81547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95ce883640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c8144750 0x7f95c8157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95ce883640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95ce883640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c8145450 con 0x7f95c8164970 2024-04-25T08:37:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95ce883640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c806f020 con 0x7f95c80ce920 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95ce883640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81455d0 con 0x7f95c8144750 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41536/0 (socket says 172.21.15.60:41536) 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 -- 172.21.15.60:0/677946783 learned_addr learned my addr 172.21.15.60:0/677946783 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95c7fff640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80ce920 0x7f95c81547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95c77fe640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c8144750 0x7f95c8157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 -- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c8144750 msgr2=0x7f95c8157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c8144750 0x7f95c8157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 -- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80ce920 msgr2=0x7f95c81547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80ce920 0x7f95c81547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95cce20640 1 -- 172.21.15.60:0/677946783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8158160 con 0x7f95c8164970 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.947+0000 7f95c77fe640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c8144750 0x7f95c8157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.948+0000 7f95cce20640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8155c50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009e50 tx=0x7f95c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.948+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0041020 con 0x7f95c8164970 2024-04-25T08:37:29.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.948+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c00077a0 con 0x7f95c8164970 2024-04-25T08:37:29.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.948+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c8158440 con 0x7f95c8164970 2024-04-25T08:37:29.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.948+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0036040 con 0x7f95c8164970 2024-04-25T08:37:29.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.948+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c81384f0 con 0x7f95c8164970 2024-04-25T08:37:29.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.949+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c8073990 con 0x7f95c8164970 2024-04-25T08:37:29.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.949+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95c0044020 con 0x7f95c8164970 2024-04-25T08:37:29.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.950+0000 7f95c4ff9640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95b406b500 0x7f95b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:29.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.950+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95c00aafd0 con 0x7f95c8164970 2024-04-25T08:37:29.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.950+0000 7f95c7fff640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95b406b500 0x7f95b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:29.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.950+0000 7f95c7fff640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95b406b500 0x7f95b406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f95c8155bb0 tx=0x7f95bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:29.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:29.954+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f95c004b030 con 0x7f95c8164970 2024-04-25T08:37:30.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.139+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95c80d0510 con 0x7f95c8164970 2024-04-25T08:37:30.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.141+0000 7f95c4ff9640 1 -- 172.21.15.60:0/677946783 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16953 (secure 0 0 0) 0x7f95c0079cc0 con 0x7f95c8164970 2024-04-25T08:37:30.135 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:30.135 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":122,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:29.166293+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95b406b500 msgr2=0x7f95b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95b406b500 0x7f95b406d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f95c8155bb0 tx=0x7f95bc009040 comp rx=0 tx=0).stop 2024-04-25T08:37:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 msgr2=0x7f95c8155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8155c50 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009e50 tx=0x7f95c0004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 shutdown_connections 2024-04-25T08:37:30.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95c8164970 0x7f95c8155c50 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f95c8144750 0x7f95c8157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f95b406b500 0x7f95b406d9c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 --2- 172.21.15.60:0/677946783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95c80ce920 0x7f95c81547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 >> 172.21.15.60:0/677946783 conn(0x7f95c806abe0 msgr2=0x7f95c806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 shutdown_connections 2024-04-25T08:37:30.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.143+0000 7f95ce883640 1 -- 172.21.15.60:0/677946783 wait complete. 2024-04-25T08:37:30.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.255+0000 7f9ed469e640 1 Processor -- start 2024-04-25T08:37:30.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.255+0000 7f9ed469e640 1 -- start start 2024-04-25T08:37:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed469e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc15dc20 0x7f9ecc1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed469e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ecc0cedf0 0x7f9ecc0d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed469e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc15e000 con 0x7f9ecc0b93a0 2024-04-25T08:37:30.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed469e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc0b6bb0 con 0x7f9ecc0cedf0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed469e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0b6030 con 0x7f9ecc15dc20 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed2c3b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ecc0cedf0 0x7f9ecc0d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed1c39640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f9ecc0b93a0 0x7f9ecc0ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52460/0 (socket says 172.21.15.60:52460) 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed1c39640 1 -- 172.21.15.60:0/1830930980 learned_addr learned my addr 172.21.15.60:0/1830930980 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed243a640 1 --2- 172.21.15.60:0/1830930980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc15dc20 0x7f9ecc1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/1830930980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc15dc20 msgr2=0x7f9ecc1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.256+0000 7f9ed2c3b640 1 --2- 172.21.15.60:0/1830930980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc15dc20 0x7f9ecc1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/1830930980 >> v1:172.21.15.60:6789/0 conn(0x7f9ecc0b93a0 legacy=0x7f9ecc0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed1438640 1 -- 172.21.15.60:0/1830930980 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969200089 0 0) 0x7f9ebc001110 con 0x7f9ecc0b93a0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/1830930980 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc161de0 con 0x7f9ecc0cedf0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed2c3b640 1 --2- 172.21.15.60:0/1830930980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc0d2510 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009dc0 tx=0x7f9ec402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531399d20b624884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed1438640 1 -- 172.21.15.60:0/1830930980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec402ae60 con 0x7f9ecc0cedf0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed1438640 1 -- 172.21.15.60:0/1830930980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4007740 con 0x7f9ecc0cedf0 2024-04-25T08:37:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.257+0000 7f9ed1438640 1 -- 172.21.15.60:0/1830930980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4007b10 con 0x7f9ecc0cedf0 2024-04-25T08:37:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 -- 172.21.15.60:0/1830930980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc0cedf0 msgr2=0x7f9ecc0d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 --2- 172.21.15.60:0/1830930980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc0d2510 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009dc0 tx=0x7f9ec402a680 comp rx=0 tx=0).stop 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 -- 172.21.15.60:0/1830930980 shutdown_connections 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 --2- 172.21.15.60:0/1830930980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc0d2510 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 --2- 172.21.15.60:0/1830930980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc15dc20 0x7f9ecc1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 -- 172.21.15.60:0/1830930980 >> 172.21.15.60:0/1830930980 conn(0x7f9ecc0d0490 msgr2=0x7f9ecc0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 -- 172.21.15.60:0/1830930980 shutdown_connections 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.258+0000 7f9ed469e640 1 -- 172.21.15.60:0/1830930980 wait complete. 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.259+0000 7f9ed469e640 1 Processor -- start 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.259+0000 7f9ed469e640 1 -- start start 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed469e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b93a0 0x7f9ecc15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed469e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed469e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 0x7f9ecc14ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed469e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0d2a50 con 0x7f9ecc15dc20 2024-04-25T08:37:30.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed469e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1623a0 con 0x7f9ecc0cedf0 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed469e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0d2bd0 con 0x7f9ecc0b93a0 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 0x7f9ecc14ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 0x7f9ecc14ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41540/0 (socket says 172.21.15.60:41540) 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/529389174 learned_addr learned my addr 172.21.15.60:0/529389174 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed1c39640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed243a640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b93a0 0x7f9ecc15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b93a0 msgr2=0x7f9ecc15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b93a0 0x7f9ecc15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0cedf0 msgr2=0x7f9ecc157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.260+0000 7f9ed2c3b640 1 -- 172.21.15.60:0/529389174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc15e670 con 0x7f9ecc15dc20 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.261+0000 7f9ed2c3b640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 0x7f9ecc14ccf0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009ee0 tx=0x7f9ec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.261+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4040070 con 0x7f9ecc15dc20 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.261+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec402ae60 con 0x7f9ecc15dc20 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.261+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc0cbd40 con 0x7f9ecc15dc20 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.261+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4009450 con 0x7f9ecc15dc20 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.261+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc0cc300 con 0x7f9ecc15dc20 2024-04-25T08:37:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.262+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc0cbb30 con 0x7f9ecc15dc20 2024-04-25T08:37:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.262+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ec4039040 con 0x7f9ecc15dc20 2024-04-25T08:37:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.263+0000 7f9ec2ffd640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.263+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ec40aba00 con 0x7f9ecc15dc20 2024-04-25T08:37:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.263+0000 7f9ed243a640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.263+0000 7f9ed243a640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9eb006b4d0 0x7f9eb006d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8006fd0 tx=0x7f9ec8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.267+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ec407a250 con 0x7f9ecc15dc20 2024-04-25T08:37:30.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.448+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ecc073990 con 0x7f9ecc15dc20 2024-04-25T08:37:30.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.449+0000 7f9ec2ffd640 1 -- 172.21.15.60:0/529389174 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16953 (secure 0 0 0) 0x7f9ec407e2f0 con 0x7f9ecc15dc20 2024-04-25T08:37:30.443 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:30.443 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":122,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:29.166293+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9eb006b4d0 msgr2=0x7f9eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9eb006b4d0 0x7f9eb006d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8006fd0 tx=0x7f9ec8008040 comp rx=0 tx=0).stop 2024-04-25T08:37:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 msgr2=0x7f9ecc14ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 0x7f9ecc14ccf0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009ee0 tx=0x7f9ec4004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:30.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 shutdown_connections 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ecc15dc20 0x7f9ecc14ccf0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9eb006b4d0 0x7f9eb006d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ecc0cedf0 0x7f9ecc157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 --2- 172.21.15.60:0/529389174 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ecc0b93a0 0x7f9ecc15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 >> 172.21.15.60:0/529389174 conn(0x7f9ecc0d0490 msgr2=0x7f9ecc0b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 shutdown_connections 2024-04-25T08:37:30.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.451+0000 7f9ed469e640 1 -- 172.21.15.60:0/529389174 wait complete. 2024-04-25T08:37:30.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:37:30.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.566+0000 7fd6029ce640 1 Processor -- start 2024-04-25T08:37:30.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.566+0000 7fd6029ce640 1 -- start start 2024-04-25T08:37:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd6029ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd5fc1610f0 0x7fd5fc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd6029ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5fc161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd6029ce640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0c6960 con 0x7fd5fc1616f0 2024-04-25T08:37:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd6029ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc1614d0 con 0x7fd5fc1610f0 2024-04-25T08:37:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd6029ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc162260 con 0x7fd5fc0ce7b0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd5fb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd5fc1616f0 0x7fd5fc0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:52472/0 (socket says 172.21.15.60:52472) 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.567+0000 7fd5fb7fe640 1 -- 172.21.15.60:0/3199812611 learned_addr learned my addr 172.21.15.60:0/3199812611 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd600f6b640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5fc161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646045448 0 0) 0x7fd5fc0c6960 con 0x7fd5fc1616f0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e4003520 con 0x7fd5fc1616f0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/3199812611 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc1610f0 0x7fd5fc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1187839826 0 0) 0x7fd5e4003520 con 0x7fd5fc1616f0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 msgr2=0x7fd5fc161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5fc161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc1610f0 msgr2=0x7fd5fc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 --2- 172.21.15.60:0/3199812611 >> v2:172.21.15.100:3300/0 conn(0x7fd5fc1610f0 0x7fd5fc06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc162540 con 0x7fd5fc1616f0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fd5f0001670 con 0x7fd5fc1616f0 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 >> v1:172.21.15.60:6789/0 conn(0x7fd5fc1616f0 legacy=0x7fd5fc0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e40039b0 0x7fd5e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e4004800 0x7fd5e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc1614d0 con 0x7fd5e40039b0 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc162260 con 0x7fd5e4004800 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e400b9c0 con 0x7fd5fc0ce7b0 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e40039b0 0x7fd5e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd5fb7fe640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e4004800 0x7fd5e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.568+0000 7fd600f6b640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5fbfff640 1 -- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 msgr2=0x7fd5e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5fbfff640 1 -- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e4004800 msgr2=0x7fd5e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e4004800 0x7fd5e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5fbfff640 1 -- 172.21.15.60:0/3199812611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4003520 con 0x7fd5e40039b0 2024-04-25T08:37:30.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd600f6b640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc0ce7b0 0x7fd5e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e40039b0 0x7fd5e4003dd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ac0 tx=0x7fd5e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9690473bda2ab154 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec036070 con 0x7fd5e40039b0 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec031400 con 0x7fd5e40039b0 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.569+0000 7fd5faffd640 1 -- 172.21.15.60:0/3199812611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec009690 con 0x7fd5e40039b0 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd6029ce640 1 -- 172.21.15.60:0/3199812611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e40039b0 msgr2=0x7fd5e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd6029ce640 1 --2- 172.21.15.60:0/3199812611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5e40039b0 0x7fd5e4003dd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009ac0 tx=0x7fd5e4005830 comp rx=0 tx=0).stop 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd600f6b640 1 -- 172.21.15.60:0/3199812611 reap_dead start 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd6029ce640 1 -- 172.21.15.60:0/3199812611 shutdown_connections 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd6029ce640 1 -- 172.21.15.60:0/3199812611 >> 172.21.15.60:0/3199812611 conn(0x7fd5fc06f500 msgr2=0x7fd5fc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd6029ce640 1 -- 172.21.15.60:0/3199812611 shutdown_connections 2024-04-25T08:37:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.570+0000 7fd6029ce640 1 -- 172.21.15.60:0/3199812611 wait complete. 2024-04-25T08:37:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.571+0000 7fd6029ce640 1 Processor -- start 2024-04-25T08:37:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.571+0000 7fd6029ce640 1 -- start start 2024-04-25T08:37:30.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd6029ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e40039b0 0x7fd5fc144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd6029ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc1477d0 0x7fd5fc0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd6029ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 0x7fd5fc15f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd6029ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cf690 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd6029ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cfce0 con 0x7fd5e40039b0 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd6029ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc06c700 con 0x7fd5fc1477d0 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd600f6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 0x7fd5fc15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd600f6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 0x7fd5fc15f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:41548/0 (socket says 172.21.15.60:41548) 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd600f6b640 1 -- 172.21.15.60:0/1301124335 learned_addr learned my addr 172.21.15.60:0/1301124335 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd5fb7fe640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc1477d0 0x7fd5fc0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.572+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e40039b0 0x7fd5fc144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd600f6b640 1 -- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc1477d0 msgr2=0x7fd5fc0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd600f6b640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc1477d0 0x7fd5fc0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd600f6b640 1 -- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e40039b0 msgr2=0x7fd5fc144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd600f6b640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e40039b0 0x7fd5fc144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd600f6b640 1 -- 172.21.15.60:0/1301124335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc144790 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e40039b0 0x7fd5fc144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd600f6b640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 0x7fd5fc15f550 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0cf120 tx=0x7fd5f4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f401a030 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4010e00 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4003bd0 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0b4e20 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.573+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0b53e0 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.574+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc0cf150 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.574+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5f4005d10 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.575+0000 7fd5f8ff9640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e806b4d0 0x7fd5e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.575+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5f408adb0 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.575+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e806b4d0 0x7fd5e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:30.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.576+0000 7fd5fbfff640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e806b4d0 0x7fd5e806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec033000 tx=0x7fd5ec008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:30.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.579+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5f4024070 con 0x7fd5fc15f0c0 2024-04-25T08:37:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:30.804+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd5fc0739e0 con 0x7fd5fc15f0c0 2024-04-25T08:37:31.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T08:37:31.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T08:37:31.593 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T08:37:31.779 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-25T08:37:31.785+0000 7f01e029f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:37:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.806+0000 7fd5f8ff9640 1 -- 172.21.15.60:0/1301124335 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd5f4059b30 con 0x7fd5fc15f0c0 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e806b4d0 msgr2=0x7fd5e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e806b4d0 0x7fd5e806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec033000 tx=0x7fd5ec008030 comp rx=0 tx=0).stop 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 msgr2=0x7fd5fc15f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 0x7fd5fc15f550 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0cf120 tx=0x7fd5f4010590 comp rx=0 tx=0).stop 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 shutdown_connections 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5fc15f0c0 0x7fd5fc15f550 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5fc1477d0 0x7fd5fc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5e806b4d0 0x7fd5e806d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 --2- 172.21.15.60:0/1301124335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5e40039b0 0x7fd5fc144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 >> 172.21.15.60:0/1301124335 conn(0x7fd5fc06f500 msgr2=0x7fd5fc06fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:31.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 shutdown_connections 2024-04-25T08:37:31.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.809+0000 7fd6029ce640 1 -- 172.21.15.60:0/1301124335 wait complete. 2024-04-25T08:37:31.817 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:37:31.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:37:31.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.935+0000 7fc66334f640 1 Processor -- start 2024-04-25T08:37:31.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.935+0000 7fc66334f640 1 -- start start 2024-04-25T08:37:31.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc66334f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc65c15dac0 0x7fc65c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:31.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc66334f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c0d0ab0 0x7fc65c0b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:31.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc66334f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c0d2420 con 0x7fc65c0d1f30 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc66334f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c0d3030 con 0x7fc65c15dac0 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc66334f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0d2700 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc65c15dac0 0x7fc65c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc65c15dac0 0x7fc65c0d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53370/0 (socket says 172.21.15.60:53370) 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 -- 172.21.15.60:0/1643552396 learned_addr learned my addr 172.21.15.60:0/1643552396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6618ec640 1 --2- 172.21.15.60:0/1643552396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c0d0ab0 0x7fc65c0b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 -- 172.21.15.60:0/1643552396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c0d0ab0 msgr2=0x7fc65c0b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 --2- 172.21.15.60:0/1643552396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c0d0ab0 0x7fc65c0b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 -- 172.21.15.60:0/1643552396 >> v1:172.21.15.100:6789/0 conn(0x7fc65c0d1f30 legacy=0x7fc65c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:31.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.936+0000 7fc6610eb640 1 -- 172.21.15.60:0/1643552396 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0d29e0 con 0x7fc65c15dac0 2024-04-25T08:37:31.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.937+0000 7fc6610eb640 1 --2- 172.21.15.60:0/1643552396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c15dac0 0x7fc65c0d19f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc658009870 tx=0x7fc65802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12034523e153bc89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:31.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.937+0000 7fc653fff640 1 -- 172.21.15.60:0/1643552396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658037070 con 0x7fc65c15dac0 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.937+0000 7fc653fff640 1 -- 172.21.15.60:0/1643552396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658032440 con 0x7fc65c15dac0 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.937+0000 7fc653fff640 1 -- 172.21.15.60:0/1643552396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658032810 con 0x7fc65c15dac0 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 -- 172.21.15.60:0/1643552396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c15dac0 msgr2=0x7fc65c0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 --2- 172.21.15.60:0/1643552396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c15dac0 0x7fc65c0d19f0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fc658009870 tx=0x7fc65802a2a0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 -- 172.21.15.60:0/1643552396 shutdown_connections 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 --2- 172.21.15.60:0/1643552396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c0d0ab0 0x7fc65c0b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 --2- 172.21.15.60:0/1643552396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c15dac0 0x7fc65c0d19f0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 -- 172.21.15.60:0/1643552396 >> 172.21.15.60:0/1643552396 conn(0x7fc65c061c30 msgr2=0x7fc65c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 -- 172.21.15.60:0/1643552396 shutdown_connections 2024-04-25T08:37:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.938+0000 7fc66334f640 1 -- 172.21.15.60:0/1643552396 wait complete. 2024-04-25T08:37:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.939+0000 7fc66334f640 1 Processor -- start 2024-04-25T08:37:31.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.939+0000 7fc66334f640 1 -- start start 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc66334f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 0x7fc65c151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc66334f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc65c0d1f30 0x7fc65c0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc66334f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c15dac0 0x7fc65c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc66334f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0b97d0 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc66334f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c15dea0 con 0x7fc65c0d1f30 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc66334f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0b9950 con 0x7fc65c15dac0 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 0x7fc65c151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 0x7fc65c151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53380/0 (socket says 172.21.15.60:53380) 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 -- 172.21.15.60:0/4082644852 learned_addr learned my addr 172.21.15.60:0/4082644852 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6618ec640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c15dac0 0x7fc65c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 -- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c15dac0 msgr2=0x7fc65c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c15dac0 0x7fc65c0cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 -- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc65c0d1f30 msgr2=0x7fc65c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:31.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc65c0d1f30 0x7fc65c0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 -- 172.21.15.60:0/4082644852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c151640 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc6610eb640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 0x7fc65c151100 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc658009870 tx=0x7fc6580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658041020 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.940+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6580306c0 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.941+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c0b39d0 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.941+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658037040 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.941+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c0b3f90 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.942+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c073a00 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.942+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc658044020 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.943+0000 7fc651ffb640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc64006b4d0 0x7fc64006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.943+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6580aac10 con 0x7fc65c0d0ab0 2024-04-25T08:37:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.943+0000 7fc6608ea640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc64006b4d0 0x7fc64006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:31.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.943+0000 7fc6608ea640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc64006b4d0 0x7fc64006d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0c9a40 tx=0x7fc64c007a00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:31.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:31.947+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc65804b030 con 0x7fc65c0d0ab0 2024-04-25T08:37:32.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.170+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc65c0caa80 con 0x7fc65c0d0ab0 2024-04-25T08:37:32.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.811+0000 7fc651ffb640 1 -- 172.21.15.60:0/4082644852 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc6580798f0 con 0x7fc65c0d0ab0 2024-04-25T08:37:32.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc64006b4d0 msgr2=0x7fc64006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc64006b4d0 0x7fc64006d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0c9a40 tx=0x7fc64c007a00 comp rx=0 tx=0).stop 2024-04-25T08:37:32.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 msgr2=0x7fc65c151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 0x7fc65c151100 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc658009870 tx=0x7fc6580046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 shutdown_connections 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc65c15dac0 0x7fc65c0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc64006b4d0 0x7fc64006d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc65c0d1f30 0x7fc65c0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 --2- 172.21.15.60:0/4082644852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc65c0d0ab0 0x7fc65c151100 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 >> 172.21.15.60:0/4082644852 conn(0x7fc65c061c30 msgr2=0x7fc65c0c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 shutdown_connections 2024-04-25T08:37:32.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.814+0000 7fc66334f640 1 -- 172.21.15.60:0/4082644852 wait complete. 2024-04-25T08:37:32.822 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:37:32.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.935+0000 7ff36835c640 1 Processor -- start 2024-04-25T08:37:32.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.935+0000 7ff36835c640 1 -- start start 2024-04-25T08:37:32.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.936+0000 7ff36835c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 0x7ff3600ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.936+0000 7ff36835c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3600cd710 con 0x7ff36015f5f0 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.936+0000 7ff36835c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff360099b70 con 0x7ff36015ba00 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.936+0000 7ff36835c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff360099e50 con 0x7ff3601602e0 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3668f9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff36015f5f0 0x7ff3600c52d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44164/0 (socket says 172.21.15.60:44164) 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3668f9640 1 -- 172.21.15.60:0/3495849370 learned_addr learned my addr 172.21.15.60:0/3495849370 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3658f7640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 0x7ff3600ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342044862 0 0) 0x7ff358001490 con 0x7ff36015f5f0 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff34c003520 con 0x7ff36015f5f0 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1973090777 0 0) 0x7ff34c003520 con 0x7ff36015f5f0 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 msgr2=0x7ff3600ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 0x7ff3600ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 >> v1:172.21.15.100:6789/0 conn(0x7ff36015ba00 legacy=0x7ff36009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36009a130 con 0x7ff36015f5f0 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ff358001bc0 con 0x7ff36015f5f0 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 >> v1:172.21.15.60:6789/0 conn(0x7ff36015f5f0 legacy=0x7ff3600c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff34c0039e0 0x7ff34c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c004800 0x7ff34c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 0x7ff34c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff360099e50 con 0x7ff34c0039e0 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c00baf0 con 0x7ff34c004800 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c00bcf0 con 0x7ff3601602e0 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3660f8640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff34c0039e0 0x7ff34c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.937+0000 7ff3668f9640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 0x7ff34c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3658f7640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c004800 0x7ff34c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3660f8640 1 -- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 msgr2=0x7ff34c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3660f8640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff3601602e0 0x7ff34c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3660f8640 1 -- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c004800 msgr2=0x7ff34c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3660f8640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c004800 0x7ff34c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3660f8640 1 -- 172.21.15.60:0/3495849370 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c003520 con 0x7ff34c0039e0 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.938+0000 7ff3660f8640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff34c0039e0 0x7ff34c003dc0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7ff35c001860 tx=0x7ff34c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f920a7efb96b9ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c037070 con 0x7ff34c0039e0 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c02ac30 con 0x7ff34c0039e0 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff3650f6640 1 -- 172.21.15.60:0/3495849370 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c001430 con 0x7ff34c0039e0 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff36835c640 1 -- 172.21.15.60:0/3495849370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff34c0039e0 msgr2=0x7ff34c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff36835c640 1 --2- 172.21.15.60:0/3495849370 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff34c0039e0 0x7ff34c003dc0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7ff35c001860 tx=0x7ff34c005830 comp rx=0 tx=0).stop 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff3668f9640 1 -- 172.21.15.60:0/3495849370 reap_dead start 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff36835c640 1 -- 172.21.15.60:0/3495849370 shutdown_connections 2024-04-25T08:37:32.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff36835c640 1 -- 172.21.15.60:0/3495849370 >> 172.21.15.60:0/3495849370 conn(0x7ff3600b3270 msgr2=0x7ff3600b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff36835c640 1 -- 172.21.15.60:0/3495849370 shutdown_connections 2024-04-25T08:37:32.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.939+0000 7ff36835c640 1 -- 172.21.15.60:0/3495849370 wait complete. 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 Processor -- start 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 -- start start 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0039e0 0x7ff3601f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff36015a630 0x7ff36015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 0x7ff3600b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3600c5b60 con 0x7ff3600b6980 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3600c59e0 con 0x7ff34c0039e0 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff36835c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3600c5ce0 con 0x7ff36015a630 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff3668f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 0x7ff3600b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff3668f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 0x7ff3600b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53408/0 (socket says 172.21.15.60:53408) 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.941+0000 7ff3668f9640 1 -- 172.21.15.60:0/542229714 learned_addr learned my addr 172.21.15.60:0/542229714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:32.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3658f7640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff36015a630 0x7ff36015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3660f8640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0039e0 0x7ff3601f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3668f9640 1 -- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff36015a630 msgr2=0x7ff36015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3668f9640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff36015a630 0x7ff36015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3668f9640 1 -- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0039e0 msgr2=0x7ff3601f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3668f9640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0039e0 0x7ff3601f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3668f9640 1 -- 172.21.15.60:0/542229714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3600b5bc0 con 0x7ff3600b6980 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3658f7640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff36015a630 0x7ff36015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff3668f9640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 0x7ff3600b6e30 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ff3600b7830 tx=0x7ff35800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff358007800 con 0x7ff3600b6980 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff358018070 con 0x7ff3600b6980 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3600b5ea0 con 0x7ff3600b6980 2024-04-25T08:37:32.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3580133f0 con 0x7ff3600b6980 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.942+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3600b6460 con 0x7ff3600b6980 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.943+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff360073990 con 0x7ff3600b6980 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.943+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff358013590 con 0x7ff3600b6980 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.944+0000 7ff356ffd640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff34406b4d0 0x7ff34406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.944+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff35808a890 con 0x7ff3600b6980 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.944+0000 7ff3660f8640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff34406b4d0 0x7ff34406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.945+0000 7ff3660f8640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff34406b4d0 0x7ff34406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff35c034000 tx=0x7ff35c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:32.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:32.948+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3600cd710 con 0x7ff3600b6980 2024-04-25T08:37:33.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.166+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3600b40a0 con 0x7ff3600b6980 2024-04-25T08:37:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.167+0000 7ff356ffd640 1 -- 172.21.15.60:0/542229714 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff358059570 con 0x7ff3600b6980 2024-04-25T08:37:33.161 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:33.161 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:33.161 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff34406b4d0 msgr2=0x7ff34406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff34406b4d0 0x7ff34406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff35c034000 tx=0x7ff35c009030 comp rx=0 tx=0).stop 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 msgr2=0x7ff3600b6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 0x7ff3600b6e30 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ff3600b7830 tx=0x7ff35800af40 comp rx=0 tx=0).stop 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 shutdown_connections 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3600b6980 0x7ff3600b6e30 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff36015a630 0x7ff36015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff34406b4d0 0x7ff34406d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 --2- 172.21.15.60:0/542229714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff34c0039e0 0x7ff3601f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 >> 172.21.15.60:0/542229714 conn(0x7ff3600b3270 msgr2=0x7ff3600702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:33.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 shutdown_connections 2024-04-25T08:37:33.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.169+0000 7ff36835c640 1 -- 172.21.15.60:0/542229714 wait complete. 2024-04-25T08:37:33.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:37:33.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 Processor -- start 2024-04-25T08:37:33.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 -- start start 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18158220 0x7f5e1815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5e180afba0 0x7f5e18161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1815f180 con 0x7f5e180af720 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180b3670 con 0x7f5e180afba0 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.281+0000 7f5e1de88640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e18070a10 con 0x7f5e18158220 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e16ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e180af720 0x7f5e181617b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44176/0 (socket says 172.21.15.60:44176) 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e16ffd640 1 -- 172.21.15.60:0/4157907930 learned_addr learned my addr 172.21.15.60:0/4157907930 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e177fe640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18158220 0x7f5e1815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e17fff640 1 --2- 172.21.15.60:0/4157907930 >> v2:172.21.15.100:3300/0 conn(0x7f5e180afba0 0x7f5e18161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e167fc640 1 -- 172.21.15.60:0/4157907930 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139388933 0 0) 0x7f5e0c001110 con 0x7f5e180af720 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e167fc640 1 -- 172.21.15.60:0/4157907930 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e00003520 con 0x7f5e180af720 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e167fc640 1 -- 172.21.15.60:0/4157907930 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553779175 0 0) 0x7f5e00003520 con 0x7f5e180af720 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e17fff640 1 -- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18158220 msgr2=0x7f5e1815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e17fff640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18158220 0x7f5e1815ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e17fff640 1 -- 172.21.15.60:0/4157907930 >> v1:172.21.15.60:6789/0 conn(0x7f5e180af720 legacy=0x7f5e181617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e17fff640 1 -- 172.21.15.60:0/4157907930 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180b0030 con 0x7f5e180afba0 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.282+0000 7f5e177fe640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18158220 0x7f5e1815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.283+0000 7f5e17fff640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180afba0 0x7f5e18161ec0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1718086bf9420c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.283+0000 7f5e167fc640 1 -- 172.21.15.60:0/4157907930 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c001840 con 0x7f5e180afba0 2024-04-25T08:37:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.283+0000 7f5e167fc640 1 -- 172.21.15.60:0/4157907930 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10007540 con 0x7f5e180afba0 2024-04-25T08:37:33.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.283+0000 7f5e167fc640 1 -- 172.21.15.60:0/4157907930 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10007910 con 0x7f5e180afba0 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 -- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180afba0 msgr2=0x7f5e18161ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180afba0 0x7f5e18161ec0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 -- 172.21.15.60:0/4157907930 shutdown_connections 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180afba0 0x7f5e18161ec0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 --2- 172.21.15.60:0/4157907930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e18158220 0x7f5e1815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 -- 172.21.15.60:0/4157907930 >> 172.21.15.60:0/4157907930 conn(0x7f5e180b2370 msgr2=0x7f5e180b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 -- 172.21.15.60:0/4157907930 shutdown_connections 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.284+0000 7f5e1de88640 1 -- 172.21.15.60:0/4157907930 wait complete. 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.285+0000 7f5e1de88640 1 Processor -- start 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.285+0000 7f5e1de88640 1 -- start start 2024-04-25T08:37:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e1de88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180af720 0x7f5e18155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e1de88640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180afba0 0x7f5e1814a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e1de88640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 0x7f5e18156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e1de88640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180703c0 con 0x7f5e18158220 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e1de88640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180700e0 con 0x7f5e180af720 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e1de88640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180706a0 con 0x7f5e180afba0 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 0x7f5e18156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 0x7f5e18156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53420/0 (socket says 172.21.15.60:53420) 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 -- 172.21.15.60:0/3330490344 learned_addr learned my addr 172.21.15.60:0/3330490344 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e177fe640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180af720 0x7f5e18155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e16ffd640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180afba0 0x7f5e1814a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 -- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180afba0 msgr2=0x7f5e1814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180afba0 0x7f5e1814a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 -- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180af720 msgr2=0x7f5e18155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180af720 0x7f5e18155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 -- 172.21.15.60:0/3330490344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18155830 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.286+0000 7f5e17fff640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 0x7f5e18156b40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.287+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10040070 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.287+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10031b40 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.287+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e180cc320 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.287+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10007e90 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.287+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e18155ab0 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.288+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18073990 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.288+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e10039040 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.289+0000 7f5dfffff640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.289+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e100ab5c0 con 0x7f5e18158220 2024-04-25T08:37:33.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.289+0000 7f5e177fe640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.289+0000 7f5e177fe640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e0406b4d0 0x7f5e0406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009560 tx=0x7f5e08009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.293+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e18155830 con 0x7f5e18158220 2024-04-25T08:37:33.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.474+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e18072a00 con 0x7f5e18158220 2024-04-25T08:37:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.475+0000 7f5dfffff640 1 -- 172.21.15.60:0/3330490344 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f5e10079cf0 con 0x7f5e18158220 2024-04-25T08:37:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:33.469 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:30.801169+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:37:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e0406b4d0 msgr2=0x7f5e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e0406b4d0 0x7f5e0406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009560 tx=0x7f5e08009190 comp rx=0 tx=0).stop 2024-04-25T08:37:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 msgr2=0x7f5e18156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 0x7f5e18156b40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e10004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 shutdown_connections 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e18158220 0x7f5e18156b40 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e180afba0 0x7f5e1814a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 --2- 172.21.15.60:0/3330490344 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e180af720 0x7f5e18155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 >> 172.21.15.60:0/3330490344 conn(0x7f5e180b2370 msgr2=0x7f5e180d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 shutdown_connections 2024-04-25T08:37:33.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.477+0000 7f5e1de88640 1 -- 172.21.15.60:0/3330490344 wait complete. 2024-04-25T08:37:33.482 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:37:33.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:37:33.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.589+0000 7ff9a499e640 1 Processor -- start 2024-04-25T08:37:33.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.589+0000 7ff9a499e640 1 -- start start 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.590+0000 7ff9a499e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 0x7ff9a006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.590+0000 7ff9a499e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff9a00700a0 0x7ff9a015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.590+0000 7ff9a499e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a01626e0 con 0x7ff9a006abe0 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.590+0000 7ff9a499e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a00cd270 con 0x7ff9a00700a0 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.590+0000 7ff9a499e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00cd550 con 0x7ff9a00ce330 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9a006abe0 0x7ff9a006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44192/0 (socket says 172.21.15.60:44192) 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99e59c640 1 -- 172.21.15.60:0/3725515196 learned_addr learned my addr 172.21.15.60:0/3725515196 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587050123 0 0) 0x7ff9a01626e0 con 0x7ff9a006abe0 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99dd9b640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 0x7ff9a006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff98c003520 con 0x7ff9a006abe0 2024-04-25T08:37:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99ed9d640 1 --2- 172.21.15.60:0/3725515196 >> v2:172.21.15.100:3300/0 conn(0x7ff9a00700a0 0x7ff9a015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2551411400 0 0) 0x7ff98c003520 con 0x7ff9a006abe0 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 msgr2=0x7ff9a006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 0x7ff9a006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 >> v2:172.21.15.100:3300/0 conn(0x7ff9a00700a0 msgr2=0x7ff9a015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 --2- 172.21.15.60:0/3725515196 >> v2:172.21.15.100:3300/0 conn(0x7ff9a00700a0 0x7ff9a015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00cd830 con 0x7ff9a006abe0 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7ff998001670 con 0x7ff9a006abe0 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 >> v1:172.21.15.60:6789/0 conn(0x7ff9a006abe0 legacy=0x7ff9a006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 0x7ff98c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff98c003f30 0x7ff98c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff98c004d40 0x7ff98c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00cd270 con 0x7ff98c004d40 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00cd550 con 0x7ff98c003f30 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c00ba10 con 0x7ff9a00ce330 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99ed9d640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff98c004d40 0x7ff98c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99e59c640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 0x7ff98c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.591+0000 7ff99dd9b640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff98c003f30 0x7ff98c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99ed9d640 1 -- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 msgr2=0x7ff98c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99ed9d640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9a00ce330 0x7ff98c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99ed9d640 1 -- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff98c003f30 msgr2=0x7ff98c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99ed9d640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff98c003f30 0x7ff98c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99ed9d640 1 -- 172.21.15.60:0/3725515196 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c003520 con 0x7ff98c004d40 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99ed9d640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff98c004d40 0x7ff98c0051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff990009f90 tx=0x7ff98c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f88763e92213ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990009c10 con 0x7ff98c004d40 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.592+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff990036070 con 0x7ff98c004d40 2024-04-25T08:37:33.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff99d59a640 1 -- 172.21.15.60:0/3725515196 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990031840 con 0x7ff98c004d40 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff9a499e640 1 -- 172.21.15.60:0/3725515196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff98c004d40 msgr2=0x7ff98c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff9a499e640 1 --2- 172.21.15.60:0/3725515196 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff98c004d40 0x7ff98c0051c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff990009f90 tx=0x7ff98c005830 comp rx=0 tx=0).stop 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff99ed9d640 1 -- 172.21.15.60:0/3725515196 reap_dead start 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff9a499e640 1 -- 172.21.15.60:0/3725515196 shutdown_connections 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff9a499e640 1 -- 172.21.15.60:0/3725515196 >> 172.21.15.60:0/3725515196 conn(0x7ff9a00b2220 msgr2=0x7ff9a00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff9a499e640 1 -- 172.21.15.60:0/3725515196 shutdown_connections 2024-04-25T08:37:33.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.593+0000 7ff9a499e640 1 -- 172.21.15.60:0/3725515196 wait complete. 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.594+0000 7ff9a499e640 1 Processor -- start 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 -- start start 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff98c004d40 0x7ff9a01f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 0x7ff9a00cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a00b4810 0x7ff9a00cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006fb60 con 0x7ff9a01f5080 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0161780 con 0x7ff9a00b4810 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff9a499e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006fce0 con 0x7ff98c004d40 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff99dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 0x7ff9a00cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff99dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 0x7ff9a00cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53424/0 (socket says 172.21.15.60:53424) 2024-04-25T08:37:33.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff99dd9b640 1 -- 172.21.15.60:0/1989401396 learned_addr learned my addr 172.21.15.60:0/1989401396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff99e59c640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff98c004d40 0x7ff9a01f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.595+0000 7ff99ed9d640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a00b4810 0x7ff9a00cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99dd9b640 1 -- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff98c004d40 msgr2=0x7ff9a01f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99dd9b640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff98c004d40 0x7ff9a01f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99dd9b640 1 -- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a00b4810 msgr2=0x7ff9a00cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99dd9b640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a00b4810 0x7ff9a00cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99dd9b640 1 -- 172.21.15.60:0/1989401396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a01f8820 con 0x7ff9a01f5080 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99e59c640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff98c004d40 0x7ff9a01f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff99dd9b640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 0x7ff9a00cbd70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff9880118e0 tx=0x7ff988011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800faf0 con 0x7ff9a01f5080 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98800fc90 con 0x7ff9a01f5080 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a01f42f0 con 0x7ff9a01f5080 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988014440 con 0x7ff9a01f5080 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.596+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a01f45a0 con 0x7ff9a01f5080 2024-04-25T08:37:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.597+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a00afe50 con 0x7ff9a01f5080 2024-04-25T08:37:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.597+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff988012040 con 0x7ff9a01f5080 2024-04-25T08:37:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.598+0000 7ff996ffd640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff98006b4d0 0x7ff98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.598+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff98808ab70 con 0x7ff9a01f5080 2024-04-25T08:37:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.598+0000 7ff99e59c640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff98006b4d0 0x7ff98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.598+0000 7ff99e59c640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff98006b4d0 0x7ff98006d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff998000fb0 tx=0x7ff998009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.602+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9a01f8820 con 0x7ff9a01f5080 2024-04-25T08:37:33.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.779+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9a00b6670 con 0x7ff98006b4d0 2024-04-25T08:37:33.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.786+0000 7ff996ffd640 1 -- 172.21.15.60:0/1989401396 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7ff9a00b6670 con 0x7ff98006b4d0 2024-04-25T08:37:33.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff98006b4d0 msgr2=0x7ff98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff98006b4d0 0x7ff98006d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff998000fb0 tx=0x7ff998009040 comp rx=0 tx=0).stop 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 msgr2=0x7ff9a00cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 0x7ff9a00cbd70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff9880118e0 tx=0x7ff988011da0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 shutdown_connections 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff98006b4d0 0x7ff98006d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9a00b4810 0x7ff9a00cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9a01f5080 0x7ff9a00cbd70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 --2- 172.21.15.60:0/1989401396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff98c004d40 0x7ff9a01f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 >> 172.21.15.60:0/1989401396 conn(0x7ff9a00b2220 msgr2=0x7ff9a0160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 shutdown_connections 2024-04-25T08:37:33.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.789+0000 7ff9a499e640 1 -- 172.21.15.60:0/1989401396 wait complete. 2024-04-25T08:37:33.794 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-25T08:37:33.510830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422148,"kb_used_data":13060,"kb_used_omap":12,"kb_used_meta":408947,"kb_avail":754552572,"statfs":{"total":773094113280,"available":772661833728,"internally_reserved":0,"allocated":13373440,"data_stored":6835739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":418762288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.724457"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.807386+0000","last_change":"2024-04-25T08:37:29.805394+0000","last_active":"2024-04-25T08:37:29.807386+0000","last_peered":"2024-04-25T08:37:29.807386+0000","last_clean":"2024-04-25T08:37:29.807386+0000","last_became_active":"2024-04-25T08:37:29.805096+0000","last_became_peered":"2024-04-25T08:37:29.805096+0000","last_unstale":"2024-04-25T08:37:29.807386+0000","last_undegraded":"2024-04-25T08:37:29.807386+0000","last_fullsized":"2024-04-25T08:37:29.807386+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:39.062426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820911+0000","last_change":"2024-04-25T08:37:29.819145+0000","last_active":"2024-04-25T08:37:29.820911+0000","last_peered":"2024-04-25T08:37:29.820911+0000","last_clean":"2024-04-25T08:37:29.820911+0000","last_became_active":"2024-04-25T08:37:29.817988+0000","last_became_peered":"2024-04-25T08:37:29.817988+0000","last_unstale":"2024-04-25T08:37:29.820911+0000","last_undegraded":"2024-04-25T08:37:29.820911+0000","last_fullsized":"2024-04-25T08:37:29.820911+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:13.708035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819438+0000","last_change":"2024-04-25T08:37:29.816297+0000","last_active":"2024-04-25T08:37:29.819438+0000","last_peered":"2024-04-25T08:37:29.819438+0000","last_clean":"2024-04-25T08:37:29.819438+0000","last_became_active":"2024-04-25T08:37:29.815841+0000","last_became_peered":"2024-04-25T08:37:29.815841+0000","last_unstale":"2024-04-25T08:37:29.819438+0000","last_undegraded":"2024-04-25T08:37:29.819438+0000","last_fullsized":"2024-04-25T08:37:29.819438+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819582+0000","last_change":"2024-04-25T08:37:29.815164+0000","last_active":"2024-04-25T08:37:29.819582+0000","last_peered":"2024-04-25T08:37:29.819582+0000","last_clean":"2024-04-25T08:37:29.819582+0000","last_became_active":"2024-04-25T08:37:29.814481+0000","last_became_peered":"2024-04-25T08:37:29.814481+0000","last_unstale":"2024-04-25T08:37:29.819582+0000","last_undegraded":"2024-04-25T08:37:29.819582+0000","last_fullsized":"2024-04-25T08:37:29.819582+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:15.690894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.806474+0000","last_change":"2024-04-25T08:37:29.800742+0000","last_active":"2024-04-25T08:37:29.806474+0000","last_peered":"2024-04-25T08:37:29.806474+0000","last_clean":"2024-04-25T08:37:29.806474+0000","last_became_active":"2024-04-25T08:37:29.800414+0000","last_became_peered":"2024-04-25T08:37:29.800414+0000","last_unstale":"2024-04-25T08:37:29.806474+0000","last_undegraded":"2024-04-25T08:37:29.806474+0000","last_fullsized":"2024-04-25T08:37:29.806474+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:53.160556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819332+0000","last_change":"2024-04-25T08:37:29.816221+0000","last_active":"2024-04-25T08:37:29.819332+0000","last_peered":"2024-04-25T08:37:29.819332+0000","last_clean":"2024-04-25T08:37:29.819332+0000","last_became_active":"2024-04-25T08:37:29.815558+0000","last_became_peered":"2024-04-25T08:37:29.815558+0000","last_unstale":"2024-04-25T08:37:29.819332+0000","last_undegraded":"2024-04-25T08:37:29.819332+0000","last_fullsized":"2024-04-25T08:37:29.819332+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:43.591982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819990+0000","last_change":"2024-04-25T08:37:29.818826+0000","last_active":"2024-04-25T08:37:29.819990+0000","last_peered":"2024-04-25T08:37:29.819990+0000","last_clean":"2024-04-25T08:37:29.819990+0000","last_became_active":"2024-04-25T08:37:29.818170+0000","last_became_peered":"2024-04-25T08:37:29.818170+0000","last_unstale":"2024-04-25T08:37:29.819990+0000","last_undegraded":"2024-04-25T08:37:29.819990+0000","last_fullsized":"2024-04-25T08:37:29.819990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:38.631283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815350+0000","last_change":"2024-04-25T08:37:29.812644+0000","last_active":"2024-04-25T08:37:29.815350+0000","last_peered":"2024-04-25T08:37:29.815350+0000","last_clean":"2024-04-25T08:37:29.815350+0000","last_became_active":"2024-04-25T08:37:29.812327+0000","last_became_peered":"2024-04-25T08:37:29.812327+0000","last_unstale":"2024-04-25T08:37:29.815350+0000","last_undegraded":"2024-04-25T08:37:29.815350+0000","last_fullsized":"2024-04-25T08:37:29.815350+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:39.062426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815854+0000","last_change":"2024-04-25T08:37:29.814674+0000","last_active":"2024-04-25T08:37:29.815854+0000","last_peered":"2024-04-25T08:37:29.815854+0000","last_clean":"2024-04-25T08:37:29.815854+0000","last_became_active":"2024-04-25T08:37:29.814368+0000","last_became_peered":"2024-04-25T08:37:29.814368+0000","last_unstale":"2024-04-25T08:37:29.815854+0000","last_undegraded":"2024-04-25T08:37:29.815854+0000","last_fullsized":"2024-04-25T08:37:29.815854+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.171300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820141+0000","last_change":"2024-04-25T08:37:29.814965+0000","last_active":"2024-04-25T08:37:29.820141+0000","last_peered":"2024-04-25T08:37:29.820141+0000","last_clean":"2024-04-25T08:37:29.820141+0000","last_became_active":"2024-04-25T08:37:29.814694+0000","last_became_peered":"2024-04-25T08:37:29.814694+0000","last_unstale":"2024-04-25T08:37:29.820141+0000","last_undegraded":"2024-04-25T08:37:29.820141+0000","last_fullsized":"2024-04-25T08:37:29.820141+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:30.169153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811689+0000","last_change":"2024-04-25T08:37:29.810114+0000","last_active":"2024-04-25T08:37:29.811689+0000","last_peered":"2024-04-25T08:37:29.811689+0000","last_clean":"2024-04-25T08:37:29.811689+0000","last_became_active":"2024-04-25T08:37:29.809857+0000","last_became_peered":"2024-04-25T08:37:29.809857+0000","last_unstale":"2024-04-25T08:37:29.811689+0000","last_undegraded":"2024-04-25T08:37:29.811689+0000","last_fullsized":"2024-04-25T08:37:29.811689+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:26.707410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.809032+0000","last_change":"2024-04-25T08:37:29.805126+0000","last_active":"2024-04-25T08:37:29.809032+0000","last_peered":"2024-04-25T08:37:29.809032+0000","last_clean":"2024-04-25T08:37:29.809032+0000","last_became_active":"2024-04-25T08:37:29.803937+0000","last_became_peered":"2024-04-25T08:37:29.803937+0000","last_unstale":"2024-04-25T08:37:29.809032+0000","last_undegraded":"2024-04-25T08:37:29.809032+0000","last_fullsized":"2024-04-25T08:37:29.809032+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:11.881085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.810034+0000","last_change":"2024-04-25T08:37:29.808286+0000","last_active":"2024-04-25T08:37:29.810034+0000","last_peered":"2024-04-25T08:37:29.810034+0000","last_clean":"2024-04-25T08:37:29.810034+0000","last_became_active":"2024-04-25T08:37:29.807915+0000","last_became_peered":"2024-04-25T08:37:29.807915+0000","last_unstale":"2024-04-25T08:37:29.810034+0000","last_undegraded":"2024-04-25T08:37:29.810034+0000","last_fullsized":"2024-04-25T08:37:29.810034+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:26.707410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.823623+0000","last_change":"2024-04-25T08:37:29.822699+0000","last_active":"2024-04-25T08:37:29.823623+0000","last_peered":"2024-04-25T08:37:29.823623+0000","last_clean":"2024-04-25T08:37:29.823623+0000","last_became_active":"2024-04-25T08:37:29.820717+0000","last_became_peered":"2024-04-25T08:37:29.820717+0000","last_unstale":"2024-04-25T08:37:29.823623+0000","last_undegraded":"2024-04-25T08:37:29.823623+0000","last_fullsized":"2024-04-25T08:37:29.823623+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811721+0000","last_change":"2024-04-25T08:37:29.809932+0000","last_active":"2024-04-25T08:37:29.811721+0000","last_peered":"2024-04-25T08:37:29.811721+0000","last_clean":"2024-04-25T08:37:29.811721+0000","last_became_active":"2024-04-25T08:37:29.809603+0000","last_became_peered":"2024-04-25T08:37:29.809603+0000","last_unstale":"2024-04-25T08:37:29.811721+0000","last_undegraded":"2024-04-25T08:37:29.811721+0000","last_fullsized":"2024-04-25T08:37:29.811721+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:18.794051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.808829+0000","last_change":"2024-04-25T08:37:29.805773+0000","last_active":"2024-04-25T08:37:29.808829+0000","last_peered":"2024-04-25T08:37:29.808829+0000","last_clean":"2024-04-25T08:37:29.808829+0000","last_became_active":"2024-04-25T08:37:29.805097+0000","last_became_peered":"2024-04-25T08:37:29.805097+0000","last_unstale":"2024-04-25T08:37:29.808829+0000","last_undegraded":"2024-04-25T08:37:29.808829+0000","last_fullsized":"2024-04-25T08:37:29.808829+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:52.013023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.823458+0000","last_change":"2024-04-25T08:37:29.818124+0000","last_active":"2024-04-25T08:37:29.823458+0000","last_peered":"2024-04-25T08:37:29.823458+0000","last_clean":"2024-04-25T08:37:29.823458+0000","last_became_active":"2024-04-25T08:37:29.817812+0000","last_became_peered":"2024-04-25T08:37:29.817812+0000","last_unstale":"2024-04-25T08:37:29.823458+0000","last_undegraded":"2024-04-25T08:37:29.823458+0000","last_fullsized":"2024-04-25T08:37:29.823458+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820888+0000","last_change":"2024-04-25T08:37:29.818854+0000","last_active":"2024-04-25T08:37:29.820888+0000","last_peered":"2024-04-25T08:37:29.820888+0000","last_clean":"2024-04-25T08:37:29.820888+0000","last_became_active":"2024-04-25T08:37:29.817586+0000","last_became_peered":"2024-04-25T08:37:29.817586+0000","last_unstale":"2024-04-25T08:37:29.820888+0000","last_undegraded":"2024-04-25T08:37:29.820888+0000","last_fullsized":"2024-04-25T08:37:29.820888+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:23.074832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808714+0000","last_change":"2024-04-25T08:37:29.812146+0000","last_active":"2024-04-25T08:37:30.808714+0000","last_peered":"2024-04-25T08:37:30.808714+0000","last_clean":"2024-04-25T08:37:30.808714+0000","last_became_active":"2024-04-25T08:37:29.811887+0000","last_became_peered":"2024-04-25T08:37:29.811887+0000","last_unstale":"2024-04-25T08:37:30.808714+0000","last_undegraded":"2024-04-25T08:37:30.808714+0000","last_fullsized":"2024-04-25T08:37:30.808714+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811818+0000","last_change":"2024-04-25T08:37:29.809752+0000","last_active":"2024-04-25T08:37:29.811818+0000","last_peered":"2024-04-25T08:37:29.811818+0000","last_clean":"2024-04-25T08:37:29.811818+0000","last_became_active":"2024-04-25T08:37:29.809457+0000","last_became_peered":"2024-04-25T08:37:29.809457+0000","last_unstale":"2024-04-25T08:37:29.811818+0000","last_undegraded":"2024-04-25T08:37:29.811818+0000","last_fullsized":"2024-04-25T08:37:29.811818+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:53.585442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813020+0000","last_change":"2024-04-25T08:37:29.810594+0000","last_active":"2024-04-25T08:37:29.813020+0000","last_peered":"2024-04-25T08:37:29.813020+0000","last_clean":"2024-04-25T08:37:29.813020+0000","last_became_active":"2024-04-25T08:37:29.810293+0000","last_became_peered":"2024-04-25T08:37:29.810293+0000","last_unstale":"2024-04-25T08:37:29.813020+0000","last_undegraded":"2024-04-25T08:37:29.813020+0000","last_fullsized":"2024-04-25T08:37:29.813020+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814004+0000","last_change":"2024-04-25T08:37:29.812391+0000","last_active":"2024-04-25T08:37:29.814004+0000","last_peered":"2024-04-25T08:37:29.814004+0000","last_clean":"2024-04-25T08:37:29.814004+0000","last_became_active":"2024-04-25T08:37:29.812085+0000","last_became_peered":"2024-04-25T08:37:29.812085+0000","last_unstale":"2024-04-25T08:37:29.814004+0000","last_undegraded":"2024-04-25T08:37:29.814004+0000","last_fullsized":"2024-04-25T08:37:29.814004+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:33.914976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815883+0000","last_change":"2024-04-25T08:37:29.813955+0000","last_active":"2024-04-25T08:37:29.815883+0000","last_peered":"2024-04-25T08:37:29.815883+0000","last_clean":"2024-04-25T08:37:29.815883+0000","last_became_active":"2024-04-25T08:37:29.813684+0000","last_became_peered":"2024-04-25T08:37:29.813684+0000","last_unstale":"2024-04-25T08:37:29.815883+0000","last_undegraded":"2024-04-25T08:37:29.815883+0000","last_fullsized":"2024-04-25T08:37:29.815883+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:33.568662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437680+0000","last_change":"2024-04-25T08:37:29.815329+0000","last_active":"2024-04-25T08:37:31.437680+0000","last_peered":"2024-04-25T08:37:31.437680+0000","last_clean":"2024-04-25T08:37:31.437680+0000","last_became_active":"2024-04-25T08:37:29.814718+0000","last_became_peered":"2024-04-25T08:37:29.814718+0000","last_unstale":"2024-04-25T08:37:31.437680+0000","last_undegraded":"2024-04-25T08:37:31.437680+0000","last_fullsized":"2024-04-25T08:37:31.437680+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:48.559466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.438059+0000","last_change":"2024-04-25T08:37:29.818063+0000","last_active":"2024-04-25T08:37:31.438059+0000","last_peered":"2024-04-25T08:37:31.438059+0000","last_clean":"2024-04-25T08:37:31.438059+0000","last_became_active":"2024-04-25T08:37:29.817777+0000","last_became_peered":"2024-04-25T08:37:29.817777+0000","last_unstale":"2024-04-25T08:37:31.438059+0000","last_undegraded":"2024-04-25T08:37:31.438059+0000","last_fullsized":"2024-04-25T08:37:31.438059+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:54.030197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811751+0000","last_change":"2024-04-25T08:37:29.808092+0000","last_active":"2024-04-25T08:37:29.811751+0000","last_peered":"2024-04-25T08:37:29.811751+0000","last_clean":"2024-04-25T08:37:29.811751+0000","last_became_active":"2024-04-25T08:37:29.807774+0000","last_became_peered":"2024-04-25T08:37:29.807774+0000","last_unstale":"2024-04-25T08:37:29.811751+0000","last_undegraded":"2024-04-25T08:37:29.811751+0000","last_fullsized":"2024-04-25T08:37:29.811751+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:48.310586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808868+0000","last_change":"2024-04-25T08:37:29.811940+0000","last_active":"2024-04-25T08:37:30.808868+0000","last_peered":"2024-04-25T08:37:30.808868+0000","last_clean":"2024-04-25T08:37:30.808868+0000","last_became_active":"2024-04-25T08:37:29.811495+0000","last_became_peered":"2024-04-25T08:37:29.811495+0000","last_unstale":"2024-04-25T08:37:30.808868+0000","last_undegraded":"2024-04-25T08:37:30.808868+0000","last_fullsized":"2024-04-25T08:37:30.808868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:48.559466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819950+0000","last_change":"2024-04-25T08:37:29.819029+0000","last_active":"2024-04-25T08:37:29.819950+0000","last_peered":"2024-04-25T08:37:29.819950+0000","last_clean":"2024-04-25T08:37:29.819950+0000","last_became_active":"2024-04-25T08:37:29.817916+0000","last_became_peered":"2024-04-25T08:37:29.817916+0000","last_unstale":"2024-04-25T08:37:29.819950+0000","last_undegraded":"2024-04-25T08:37:29.819950+0000","last_fullsized":"2024-04-25T08:37:29.819950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:53.401391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.818372+0000","last_change":"2024-04-25T08:37:29.815591+0000","last_active":"2024-04-25T08:37:29.818372+0000","last_peered":"2024-04-25T08:37:29.818372+0000","last_clean":"2024-04-25T08:37:29.818372+0000","last_became_active":"2024-04-25T08:37:29.815268+0000","last_became_peered":"2024-04-25T08:37:29.815268+0000","last_unstale":"2024-04-25T08:37:29.818372+0000","last_undegraded":"2024-04-25T08:37:29.818372+0000","last_fullsized":"2024-04-25T08:37:29.818372+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:24.305617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.822858+0000","last_change":"2024-04-25T08:37:29.817058+0000","last_active":"2024-04-25T08:37:29.822858+0000","last_peered":"2024-04-25T08:37:29.822858+0000","last_clean":"2024-04-25T08:37:29.822858+0000","last_became_active":"2024-04-25T08:37:29.816740+0000","last_became_peered":"2024-04-25T08:37:29.816740+0000","last_unstale":"2024-04-25T08:37:29.822858+0000","last_undegraded":"2024-04-25T08:37:29.822858+0000","last_fullsized":"2024-04-25T08:37:29.822858+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:12.773879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437790+0000","last_change":"2024-04-25T08:37:29.814763+0000","last_active":"2024-04-25T08:37:31.437790+0000","last_peered":"2024-04-25T08:37:31.437790+0000","last_clean":"2024-04-25T08:37:31.437790+0000","last_became_active":"2024-04-25T08:37:29.814456+0000","last_became_peered":"2024-04-25T08:37:29.814456+0000","last_unstale":"2024-04-25T08:37:31.437790+0000","last_undegraded":"2024-04-25T08:37:31.437790+0000","last_fullsized":"2024-04-25T08:37:31.437790+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:24.135064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437613+0000","last_change":"2024-04-25T08:37:29.815101+0000","last_active":"2024-04-25T08:37:31.437613+0000","last_peered":"2024-04-25T08:37:31.437613+0000","last_clean":"2024-04-25T08:37:31.437613+0000","last_became_active":"2024-04-25T08:37:29.814141+0000","last_became_peered":"2024-04-25T08:37:29.814141+0000","last_unstale":"2024-04-25T08:37:31.437613+0000","last_undegraded":"2024-04-25T08:37:31.437613+0000","last_fullsized":"2024-04-25T08:37:31.437613+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813549+0000","last_change":"2024-04-25T08:37:29.811608+0000","last_active":"2024-04-25T08:37:29.813549+0000","last_peered":"2024-04-25T08:37:29.813549+0000","last_clean":"2024-04-25T08:37:29.813549+0000","last_became_active":"2024-04-25T08:37:29.810656+0000","last_became_peered":"2024-04-25T08:37:29.810656+0000","last_unstale":"2024-04-25T08:37:29.813549+0000","last_undegraded":"2024-04-25T08:37:29.813549+0000","last_fullsized":"2024-04-25T08:37:29.813549+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:30.098003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"122'4","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.809109+0000","last_change":"2024-04-25T08:37:28.808094+0000","last_active":"2024-04-25T08:37:30.809109+0000","last_peered":"2024-04-25T08:37:30.809109+0000","last_clean":"2024-04-25T08:37:30.809109+0000","last_became_active":"2024-04-25T08:37:28.807758+0000","last_became_peered":"2024-04-25T08:37:28.807758+0000","last_unstale":"2024-04-25T08:37:30.809109+0000","last_undegraded":"2024-04-25T08:37:30.809109+0000","last_fullsized":"2024-04-25T08:37:30.809109+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:33.377078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820231+0000","last_change":"2024-04-25T08:37:29.819619+0000","last_active":"2024-04-25T08:37:29.820231+0000","last_peered":"2024-04-25T08:37:29.820231+0000","last_clean":"2024-04-25T08:37:29.820231+0000","last_became_active":"2024-04-25T08:37:29.818993+0000","last_became_peered":"2024-04-25T08:37:29.818993+0000","last_unstale":"2024-04-25T08:37:29.820231+0000","last_undegraded":"2024-04-25T08:37:29.820231+0000","last_fullsized":"2024-04-25T08:37:29.820231+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:25.579263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.823563+0000","last_change":"2024-04-25T08:37:28.805556+0000","last_active":"2024-04-25T08:37:29.823563+0000","last_peered":"2024-04-25T08:37:29.823563+0000","last_clean":"2024-04-25T08:37:29.823563+0000","last_became_active":"2024-04-25T08:37:28.805190+0000","last_became_peered":"2024-04-25T08:37:28.805190+0000","last_unstale":"2024-04-25T08:37:29.823563+0000","last_undegraded":"2024-04-25T08:37:29.823563+0000","last_fullsized":"2024-04-25T08:37:29.823563+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:46.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437947+0000","last_change":"2024-04-25T08:37:29.821571+0000","last_active":"2024-04-25T08:37:31.437947+0000","last_peered":"2024-04-25T08:37:31.437947+0000","last_clean":"2024-04-25T08:37:31.437947+0000","last_became_active":"2024-04-25T08:37:29.821234+0000","last_became_peered":"2024-04-25T08:37:29.821234+0000","last_unstale":"2024-04-25T08:37:31.437947+0000","last_undegraded":"2024-04-25T08:37:31.437947+0000","last_fullsized":"2024-04-25T08:37:31.437947+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:15.690894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.824329+0000","last_change":"2024-04-25T08:37:28.809776+0000","last_active":"2024-04-25T08:37:29.824329+0000","last_peered":"2024-04-25T08:37:29.824329+0000","last_clean":"2024-04-25T08:37:29.824329+0000","last_became_active":"2024-04-25T08:37:28.809459+0000","last_became_peered":"2024-04-25T08:37:28.809459+0000","last_unstale":"2024-04-25T08:37:29.824329+0000","last_undegraded":"2024-04-25T08:37:29.824329+0000","last_fullsized":"2024-04-25T08:37:29.824329+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:22.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.812991+0000","last_change":"2024-04-25T08:37:29.810633+0000","last_active":"2024-04-25T08:37:29.812991+0000","last_peered":"2024-04-25T08:37:29.812991+0000","last_clean":"2024-04-25T08:37:29.812991+0000","last_became_active":"2024-04-25T08:37:29.810344+0000","last_became_peered":"2024-04-25T08:37:29.810344+0000","last_unstale":"2024-04-25T08:37:29.812991+0000","last_undegraded":"2024-04-25T08:37:29.812991+0000","last_fullsized":"2024-04-25T08:37:29.812991+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:48.559466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.827933+0000","last_change":"2024-04-25T08:37:28.810195+0000","last_active":"2024-04-25T08:37:29.827933+0000","last_peered":"2024-04-25T08:37:29.827933+0000","last_clean":"2024-04-25T08:37:29.827933+0000","last_became_active":"2024-04-25T08:37:28.809854+0000","last_became_peered":"2024-04-25T08:37:28.809854+0000","last_unstale":"2024-04-25T08:37:29.827933+0000","last_undegraded":"2024-04-25T08:37:29.827933+0000","last_fullsized":"2024-04-25T08:37:29.827933+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:38.057904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.804716+0000","last_change":"2024-04-25T08:37:29.802001+0000","last_active":"2024-04-25T08:37:29.804716+0000","last_peered":"2024-04-25T08:37:29.804716+0000","last_clean":"2024-04-25T08:37:29.804716+0000","last_became_active":"2024-04-25T08:37:29.801679+0000","last_became_peered":"2024-04-25T08:37:29.801679+0000","last_unstale":"2024-04-25T08:37:29.804716+0000","last_undegraded":"2024-04-25T08:37:29.804716+0000","last_fullsized":"2024-04-25T08:37:29.804716+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:53.160556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.822354+0000","last_change":"2024-04-25T08:37:28.807402+0000","last_active":"2024-04-25T08:37:29.822354+0000","last_peered":"2024-04-25T08:37:29.822354+0000","last_clean":"2024-04-25T08:37:29.822354+0000","last_became_active":"2024-04-25T08:37:28.806817+0000","last_became_peered":"2024-04-25T08:37:28.806817+0000","last_unstale":"2024-04-25T08:37:29.822354+0000","last_undegraded":"2024-04-25T08:37:29.822354+0000","last_fullsized":"2024-04-25T08:37:29.822354+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:27.918282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.810682+0000","last_change":"2024-04-25T08:37:29.808485+0000","last_active":"2024-04-25T08:37:29.810682+0000","last_peered":"2024-04-25T08:37:29.810682+0000","last_clean":"2024-04-25T08:37:29.810682+0000","last_became_active":"2024-04-25T08:37:29.808117+0000","last_became_peered":"2024-04-25T08:37:29.808117+0000","last_unstale":"2024-04-25T08:37:29.810682+0000","last_undegraded":"2024-04-25T08:37:29.810682+0000","last_fullsized":"2024-04-25T08:37:29.810682+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:47.389963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.824957+0000","last_change":"2024-04-25T08:37:28.811481+0000","last_active":"2024-04-25T08:37:29.824957+0000","last_peered":"2024-04-25T08:37:29.824957+0000","last_clean":"2024-04-25T08:37:29.824957+0000","last_became_active":"2024-04-25T08:37:28.809575+0000","last_became_peered":"2024-04-25T08:37:28.809575+0000","last_unstale":"2024-04-25T08:37:29.824957+0000","last_undegraded":"2024-04-25T08:37:29.824957+0000","last_fullsized":"2024-04-25T08:37:29.824957+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:37.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811831+0000","last_change":"2024-04-25T08:37:29.810618+0000","last_active":"2024-04-25T08:37:29.811831+0000","last_peered":"2024-04-25T08:37:29.811831+0000","last_clean":"2024-04-25T08:37:29.811831+0000","last_became_active":"2024-04-25T08:37:29.810305+0000","last_became_peered":"2024-04-25T08:37:29.810305+0000","last_unstale":"2024-04-25T08:37:29.811831+0000","last_undegraded":"2024-04-25T08:37:29.811831+0000","last_fullsized":"2024-04-25T08:37:29.811831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:53.160556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437791+0000","last_change":"2024-04-25T08:37:28.806715+0000","last_active":"2024-04-25T08:37:31.437791+0000","last_peered":"2024-04-25T08:37:31.437791+0000","last_clean":"2024-04-25T08:37:31.437791+0000","last_became_active":"2024-04-25T08:37:28.805236+0000","last_became_peered":"2024-04-25T08:37:28.805236+0000","last_unstale":"2024-04-25T08:37:31.437791+0000","last_undegraded":"2024-04-25T08:37:31.437791+0000","last_fullsized":"2024-04-25T08:37:31.437791+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:38.057904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813154+0000","last_change":"2024-04-25T08:37:29.811481+0000","last_active":"2024-04-25T08:37:29.813154+0000","last_peered":"2024-04-25T08:37:29.813154+0000","last_clean":"2024-04-25T08:37:29.813154+0000","last_became_active":"2024-04-25T08:37:29.811168+0000","last_became_peered":"2024-04-25T08:37:29.811168+0000","last_unstale":"2024-04-25T08:37:29.813154+0000","last_undegraded":"2024-04-25T08:37:29.813154+0000","last_fullsized":"2024-04-25T08:37:29.813154+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:39.330257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.796625+0000","last_change":"2024-04-25T08:37:28.812056+0000","last_active":"2024-04-25T08:37:29.796625+0000","last_peered":"2024-04-25T08:37:29.796625+0000","last_clean":"2024-04-25T08:37:29.796625+0000","last_became_active":"2024-04-25T08:37:28.811767+0000","last_became_peered":"2024-04-25T08:37:28.811767+0000","last_unstale":"2024-04-25T08:37:29.796625+0000","last_undegraded":"2024-04-25T08:37:29.796625+0000","last_fullsized":"2024-04-25T08:37:29.796625+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:17.850069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808384+0000","last_change":"2024-04-25T08:37:29.818925+0000","last_active":"2024-04-25T08:37:30.808384+0000","last_peered":"2024-04-25T08:37:30.808384+0000","last_clean":"2024-04-25T08:37:30.808384+0000","last_became_active":"2024-04-25T08:37:29.818194+0000","last_became_peered":"2024-04-25T08:37:29.818194+0000","last_unstale":"2024-04-25T08:37:30.808384+0000","last_undegraded":"2024-04-25T08:37:30.808384+0000","last_fullsized":"2024-04-25T08:37:30.808384+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:13.708035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.809204+0000","last_change":"2024-04-25T08:37:28.805159+0000","last_active":"2024-04-25T08:37:30.809204+0000","last_peered":"2024-04-25T08:37:30.809204+0000","last_clean":"2024-04-25T08:37:30.809204+0000","last_became_active":"2024-04-25T08:37:28.804900+0000","last_became_peered":"2024-04-25T08:37:28.804900+0000","last_unstale":"2024-04-25T08:37:30.809204+0000","last_undegraded":"2024-04-25T08:37:30.809204+0000","last_fullsized":"2024-04-25T08:37:30.809204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:52.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437867+0000","last_change":"2024-04-25T08:37:29.821883+0000","last_active":"2024-04-25T08:37:31.437867+0000","last_peered":"2024-04-25T08:37:31.437867+0000","last_clean":"2024-04-25T08:37:31.437867+0000","last_became_active":"2024-04-25T08:37:29.821534+0000","last_became_peered":"2024-04-25T08:37:29.821534+0000","last_unstale":"2024-04-25T08:37:31.437867+0000","last_undegraded":"2024-04-25T08:37:31.437867+0000","last_fullsized":"2024-04-25T08:37:31.437867+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.171300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437929+0000","last_change":"2024-04-25T08:37:28.804893+0000","last_active":"2024-04-25T08:37:31.437929+0000","last_peered":"2024-04-25T08:37:31.437929+0000","last_clean":"2024-04-25T08:37:31.437929+0000","last_became_active":"2024-04-25T08:37:28.804561+0000","last_became_peered":"2024-04-25T08:37:28.804561+0000","last_unstale":"2024-04-25T08:37:31.437929+0000","last_undegraded":"2024-04-25T08:37:31.437929+0000","last_fullsized":"2024-04-25T08:37:31.437929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:25.702889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820183+0000","last_change":"2024-04-25T08:37:29.814910+0000","last_active":"2024-04-25T08:37:29.820183+0000","last_peered":"2024-04-25T08:37:29.820183+0000","last_clean":"2024-04-25T08:37:29.820183+0000","last_became_active":"2024-04-25T08:37:29.814598+0000","last_became_peered":"2024-04-25T08:37:29.814598+0000","last_unstale":"2024-04-25T08:37:29.820183+0000","last_undegraded":"2024-04-25T08:37:29.820183+0000","last_fullsized":"2024-04-25T08:37:29.820183+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:39.330257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"122'3","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437857+0000","last_change":"2024-04-25T08:37:28.809333+0000","last_active":"2024-04-25T08:37:31.437857+0000","last_peered":"2024-04-25T08:37:31.437857+0000","last_clean":"2024-04-25T08:37:31.437857+0000","last_became_active":"2024-04-25T08:37:28.808954+0000","last_became_peered":"2024-04-25T08:37:28.808954+0000","last_unstale":"2024-04-25T08:37:31.437857+0000","last_undegraded":"2024-04-25T08:37:31.437857+0000","last_fullsized":"2024-04-25T08:37:31.437857+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:46.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.438114+0000","last_change":"2024-04-25T08:37:29.816262+0000","last_active":"2024-04-25T08:37:31.438114+0000","last_peered":"2024-04-25T08:37:31.438114+0000","last_clean":"2024-04-25T08:37:31.438114+0000","last_became_active":"2024-04-25T08:37:29.814603+0000","last_became_peered":"2024-04-25T08:37:29.814603+0000","last_unstale":"2024-04-25T08:37:31.438114+0000","last_undegraded":"2024-04-25T08:37:31.438114+0000","last_fullsized":"2024-04-25T08:37:31.438114+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:01.817130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"122'6","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.835313+0000","last_change":"2024-04-25T08:37:28.812343+0000","last_active":"2024-04-25T08:37:29.835313+0000","last_peered":"2024-04-25T08:37:29.835313+0000","last_clean":"2024-04-25T08:37:29.835313+0000","last_became_active":"2024-04-25T08:37:28.811991+0000","last_became_peered":"2024-04-25T08:37:28.811991+0000","last_unstale":"2024-04-25T08:37:29.835313+0000","last_undegraded":"2024-04-25T08:37:29.835313+0000","last_fullsized":"2024-04-25T08:37:29.835313+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:29.093482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808984+0000","last_change":"2024-04-25T08:37:29.811347+0000","last_active":"2024-04-25T08:37:30.808984+0000","last_peered":"2024-04-25T08:37:30.808984+0000","last_clean":"2024-04-25T08:37:30.808984+0000","last_became_active":"2024-04-25T08:37:29.811091+0000","last_became_peered":"2024-04-25T08:37:29.811091+0000","last_unstale":"2024-04-25T08:37:30.808984+0000","last_undegraded":"2024-04-25T08:37:30.808984+0000","last_fullsized":"2024-04-25T08:37:30.808984+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.830197+0000","last_change":"2024-04-25T08:37:28.809181+0000","last_active":"2024-04-25T08:37:29.830197+0000","last_peered":"2024-04-25T08:37:29.830197+0000","last_clean":"2024-04-25T08:37:29.830197+0000","last_became_active":"2024-04-25T08:37:28.808882+0000","last_became_peered":"2024-04-25T08:37:28.808882+0000","last_unstale":"2024-04-25T08:37:29.830197+0000","last_undegraded":"2024-04-25T08:37:29.830197+0000","last_fullsized":"2024-04-25T08:37:29.830197+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:33.377078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.812169+0000","last_change":"2024-04-25T08:37:29.809581+0000","last_active":"2024-04-25T08:37:29.812169+0000","last_peered":"2024-04-25T08:37:29.812169+0000","last_clean":"2024-04-25T08:37:29.812169+0000","last_became_active":"2024-04-25T08:37:29.809209+0000","last_became_peered":"2024-04-25T08:37:29.809209+0000","last_unstale":"2024-04-25T08:37:29.812169+0000","last_undegraded":"2024-04-25T08:37:29.812169+0000","last_fullsized":"2024-04-25T08:37:29.812169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:33.914976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813988+0000","last_change":"2024-04-25T08:37:28.804189+0000","last_active":"2024-04-25T08:37:29.813988+0000","last_peered":"2024-04-25T08:37:29.813988+0000","last_clean":"2024-04-25T08:37:29.813988+0000","last_became_active":"2024-04-25T08:37:28.803605+0000","last_became_peered":"2024-04-25T08:37:28.803605+0000","last_unstale":"2024-04-25T08:37:29.813988+0000","last_undegraded":"2024-04-25T08:37:29.813988+0000","last_fullsized":"2024-04-25T08:37:29.813988+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:17.789530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.807225+0000","last_change":"2024-04-25T08:37:29.803410+0000","last_active":"2024-04-25T08:37:29.807225+0000","last_peered":"2024-04-25T08:37:29.807225+0000","last_clean":"2024-04-25T08:37:29.807225+0000","last_became_active":"2024-04-25T08:37:29.803072+0000","last_became_peered":"2024-04-25T08:37:29.803072+0000","last_unstale":"2024-04-25T08:37:29.807225+0000","last_undegraded":"2024-04-25T08:37:29.807225+0000","last_fullsized":"2024-04-25T08:37:29.807225+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.497389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814231+0000","last_change":"2024-04-25T08:37:28.808072+0000","last_active":"2024-04-25T08:37:29.814231+0000","last_peered":"2024-04-25T08:37:29.814231+0000","last_clean":"2024-04-25T08:37:29.814231+0000","last_became_active":"2024-04-25T08:37:28.807797+0000","last_became_peered":"2024-04-25T08:37:28.807797+0000","last_unstale":"2024-04-25T08:37:29.814231+0000","last_undegraded":"2024-04-25T08:37:29.814231+0000","last_fullsized":"2024-04-25T08:37:29.814231+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:25.702889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437787+0000","last_change":"2024-04-25T08:37:29.820198+0000","last_active":"2024-04-25T08:37:31.437787+0000","last_peered":"2024-04-25T08:37:31.437787+0000","last_clean":"2024-04-25T08:37:31.437787+0000","last_became_active":"2024-04-25T08:37:29.818785+0000","last_became_peered":"2024-04-25T08:37:29.818785+0000","last_unstale":"2024-04-25T08:37:31.437787+0000","last_undegraded":"2024-04-25T08:37:31.437787+0000","last_fullsized":"2024-04-25T08:37:31.437787+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:33.568662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.825674+0000","last_change":"2024-04-25T08:37:28.815584+0000","last_active":"2024-04-25T08:37:29.825674+0000","last_peered":"2024-04-25T08:37:29.825674+0000","last_clean":"2024-04-25T08:37:29.825674+0000","last_became_active":"2024-04-25T08:37:28.815258+0000","last_became_peered":"2024-04-25T08:37:28.815258+0000","last_unstale":"2024-04-25T08:37:29.825674+0000","last_undegraded":"2024-04-25T08:37:29.825674+0000","last_fullsized":"2024-04-25T08:37:29.825674+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:17.789530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.821135+0000","last_change":"2024-04-25T08:37:29.817652+0000","last_active":"2024-04-25T08:37:29.821135+0000","last_peered":"2024-04-25T08:37:29.821135+0000","last_clean":"2024-04-25T08:37:29.821135+0000","last_became_active":"2024-04-25T08:37:29.817382+0000","last_became_peered":"2024-04-25T08:37:29.817382+0000","last_unstale":"2024-04-25T08:37:29.821135+0000","last_undegraded":"2024-04-25T08:37:29.821135+0000","last_fullsized":"2024-04-25T08:37:29.821135+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:34.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.826862+0000","last_change":"2024-04-25T08:37:28.814257+0000","last_active":"2024-04-25T08:37:29.826862+0000","last_peered":"2024-04-25T08:37:29.826862+0000","last_clean":"2024-04-25T08:37:29.826862+0000","last_became_active":"2024-04-25T08:37:28.813933+0000","last_became_peered":"2024-04-25T08:37:28.813933+0000","last_unstale":"2024-04-25T08:37:29.826862+0000","last_undegraded":"2024-04-25T08:37:29.826862+0000","last_fullsized":"2024-04-25T08:37:29.826862+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:02.224856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.808586+0000","last_change":"2024-04-25T08:37:29.799100+0000","last_active":"2024-04-25T08:37:29.808586+0000","last_peered":"2024-04-25T08:37:29.808586+0000","last_clean":"2024-04-25T08:37:29.808586+0000","last_became_active":"2024-04-25T08:37:29.798576+0000","last_became_peered":"2024-04-25T08:37:29.798576+0000","last_unstale":"2024-04-25T08:37:29.808586+0000","last_undegraded":"2024-04-25T08:37:29.808586+0000","last_fullsized":"2024-04-25T08:37:29.808586+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:40.306970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.827152+0000","last_change":"2024-04-25T08:37:28.808855+0000","last_active":"2024-04-25T08:37:29.827152+0000","last_peered":"2024-04-25T08:37:29.827152+0000","last_clean":"2024-04-25T08:37:29.827152+0000","last_became_active":"2024-04-25T08:37:28.808555+0000","last_became_peered":"2024-04-25T08:37:28.808555+0000","last_unstale":"2024-04-25T08:37:29.827152+0000","last_undegraded":"2024-04-25T08:37:29.827152+0000","last_fullsized":"2024-04-25T08:37:29.827152+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:12.703514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.438149+0000","last_change":"2024-04-25T08:37:29.817384+0000","last_active":"2024-04-25T08:37:31.438149+0000","last_peered":"2024-04-25T08:37:31.438149+0000","last_clean":"2024-04-25T08:37:31.438149+0000","last_became_active":"2024-04-25T08:37:29.816712+0000","last_became_peered":"2024-04-25T08:37:29.816712+0000","last_unstale":"2024-04-25T08:37:31.438149+0000","last_undegraded":"2024-04-25T08:37:31.438149+0000","last_fullsized":"2024-04-25T08:37:31.438149+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.033461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.805757+0000","last_change":"2024-04-25T08:37:28.816285+0000","last_active":"2024-04-25T08:37:29.805757+0000","last_peered":"2024-04-25T08:37:29.805757+0000","last_clean":"2024-04-25T08:37:29.805757+0000","last_became_active":"2024-04-25T08:37:28.816043+0000","last_became_peered":"2024-04-25T08:37:28.816043+0000","last_unstale":"2024-04-25T08:37:29.805757+0000","last_undegraded":"2024-04-25T08:37:29.805757+0000","last_fullsized":"2024-04-25T08:37:29.805757+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:11.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.806377+0000","last_change":"2024-04-25T08:37:29.799578+0000","last_active":"2024-04-25T08:37:29.806377+0000","last_peered":"2024-04-25T08:37:29.806377+0000","last_clean":"2024-04-25T08:37:29.806377+0000","last_became_active":"2024-04-25T08:37:29.799302+0000","last_became_peered":"2024-04-25T08:37:29.799302+0000","last_unstale":"2024-04-25T08:37:29.806377+0000","last_undegraded":"2024-04-25T08:37:29.806377+0000","last_fullsized":"2024-04-25T08:37:29.806377+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:34.239796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.827353+0000","last_change":"2024-04-25T08:37:28.811977+0000","last_active":"2024-04-25T08:37:29.827353+0000","last_peered":"2024-04-25T08:37:29.827353+0000","last_clean":"2024-04-25T08:37:29.827353+0000","last_became_active":"2024-04-25T08:37:28.811682+0000","last_became_peered":"2024-04-25T08:37:28.811682+0000","last_unstale":"2024-04-25T08:37:29.827353+0000","last_undegraded":"2024-04-25T08:37:29.827353+0000","last_fullsized":"2024-04-25T08:37:29.827353+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:37.886123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820442+0000","last_change":"2024-04-25T08:37:29.815322+0000","last_active":"2024-04-25T08:37:29.820442+0000","last_peered":"2024-04-25T08:37:29.820442+0000","last_clean":"2024-04-25T08:37:29.820442+0000","last_became_active":"2024-04-25T08:37:29.815003+0000","last_became_peered":"2024-04-25T08:37:29.815003+0000","last_unstale":"2024-04-25T08:37:29.820442+0000","last_undegraded":"2024-04-25T08:37:29.820442+0000","last_fullsized":"2024-04-25T08:37:29.820442+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:30.169153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.802411+0000","last_change":"2024-04-25T08:37:28.811694+0000","last_active":"2024-04-25T08:37:29.802411+0000","last_peered":"2024-04-25T08:37:29.802411+0000","last_clean":"2024-04-25T08:37:29.802411+0000","last_became_active":"2024-04-25T08:37:28.811423+0000","last_became_peered":"2024-04-25T08:37:28.811423+0000","last_unstale":"2024-04-25T08:37:29.802411+0000","last_undegraded":"2024-04-25T08:37:29.802411+0000","last_fullsized":"2024-04-25T08:37:29.802411+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:29.093482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.816668+0000","last_change":"2024-04-25T08:37:29.813018+0000","last_active":"2024-04-25T08:37:29.816668+0000","last_peered":"2024-04-25T08:37:29.816668+0000","last_clean":"2024-04-25T08:37:29.816668+0000","last_became_active":"2024-04-25T08:37:29.812733+0000","last_became_peered":"2024-04-25T08:37:29.812733+0000","last_unstale":"2024-04-25T08:37:29.816668+0000","last_undegraded":"2024-04-25T08:37:29.816668+0000","last_fullsized":"2024-04-25T08:37:29.816668+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:47.389963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814415+0000","last_change":"2024-04-25T08:37:28.809836+0000","last_active":"2024-04-25T08:37:29.814415+0000","last_peered":"2024-04-25T08:37:29.814415+0000","last_clean":"2024-04-25T08:37:29.814415+0000","last_became_active":"2024-04-25T08:37:28.809525+0000","last_became_peered":"2024-04-25T08:37:28.809525+0000","last_unstale":"2024-04-25T08:37:29.814415+0000","last_undegraded":"2024-04-25T08:37:29.814415+0000","last_fullsized":"2024-04-25T08:37:29.814415+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:32.910454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437764+0000","last_change":"2024-04-25T08:37:29.812775+0000","last_active":"2024-04-25T08:37:31.437764+0000","last_peered":"2024-04-25T08:37:31.437764+0000","last_clean":"2024-04-25T08:37:31.437764+0000","last_became_active":"2024-04-25T08:37:29.808458+0000","last_became_peered":"2024-04-25T08:37:29.808458+0000","last_unstale":"2024-04-25T08:37:31.437764+0000","last_undegraded":"2024-04-25T08:37:31.437764+0000","last_fullsized":"2024-04-25T08:37:31.437764+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.803981+0000","last_change":"2024-04-25T08:37:28.809691+0000","last_active":"2024-04-25T08:37:29.803981+0000","last_peered":"2024-04-25T08:37:29.803981+0000","last_clean":"2024-04-25T08:37:29.803981+0000","last_became_active":"2024-04-25T08:37:28.809249+0000","last_became_peered":"2024-04-25T08:37:28.809249+0000","last_unstale":"2024-04-25T08:37:29.803981+0000","last_undegraded":"2024-04-25T08:37:29.803981+0000","last_fullsized":"2024-04-25T08:37:29.803981+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:29.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814345+0000","last_change":"2024-04-25T08:37:29.812638+0000","last_active":"2024-04-25T08:37:29.814345+0000","last_peered":"2024-04-25T08:37:29.814345+0000","last_clean":"2024-04-25T08:37:29.814345+0000","last_became_active":"2024-04-25T08:37:29.809364+0000","last_became_peered":"2024-04-25T08:37:29.809364+0000","last_unstale":"2024-04-25T08:37:29.814345+0000","last_undegraded":"2024-04-25T08:37:29.814345+0000","last_fullsized":"2024-04-25T08:37:29.814345+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:19.665018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.829973+0000","last_change":"2024-04-25T08:37:28.803503+0000","last_active":"2024-04-25T08:37:29.829973+0000","last_peered":"2024-04-25T08:37:29.829973+0000","last_clean":"2024-04-25T08:37:29.829973+0000","last_became_active":"2024-04-25T08:37:28.802463+0000","last_became_peered":"2024-04-25T08:37:28.802463+0000","last_unstale":"2024-04-25T08:37:29.829973+0000","last_undegraded":"2024-04-25T08:37:29.829973+0000","last_fullsized":"2024-04-25T08:37:29.829973+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:06.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813301+0000","last_change":"2024-04-25T08:37:29.811329+0000","last_active":"2024-04-25T08:37:29.813301+0000","last_peered":"2024-04-25T08:37:29.813301+0000","last_clean":"2024-04-25T08:37:29.813301+0000","last_became_active":"2024-04-25T08:37:29.811024+0000","last_became_peered":"2024-04-25T08:37:29.811024+0000","last_unstale":"2024-04-25T08:37:29.813301+0000","last_undegraded":"2024-04-25T08:37:29.813301+0000","last_fullsized":"2024-04-25T08:37:29.813301+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:53.473865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"123'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.810581+0000","last_change":"2024-04-25T08:37:28.809149+0000","last_active":"2024-04-25T08:37:30.810581+0000","last_peered":"2024-04-25T08:37:30.810581+0000","last_clean":"2024-04-25T08:37:30.810581+0000","last_became_active":"2024-04-25T08:37:28.808873+0000","last_became_peered":"2024-04-25T08:37:28.808873+0000","last_unstale":"2024-04-25T08:37:30.810581+0000","last_undegraded":"2024-04-25T08:37:30.810581+0000","last_fullsized":"2024-04-25T08:37:30.810581+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:38.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815627+0000","last_change":"2024-04-25T08:37:29.813671+0000","last_active":"2024-04-25T08:37:29.815627+0000","last_peered":"2024-04-25T08:37:29.815627+0000","last_clean":"2024-04-25T08:37:29.815627+0000","last_became_active":"2024-04-25T08:37:29.813363+0000","last_became_peered":"2024-04-25T08:37:29.813363+0000","last_unstale":"2024-04-25T08:37:29.815627+0000","last_undegraded":"2024-04-25T08:37:29.815627+0000","last_fullsized":"2024-04-25T08:37:29.815627+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:24.135064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808970+0000","last_change":"2024-04-25T08:37:28.809947+0000","last_active":"2024-04-25T08:37:30.808970+0000","last_peered":"2024-04-25T08:37:30.808970+0000","last_clean":"2024-04-25T08:37:30.808970+0000","last_became_active":"2024-04-25T08:37:28.809692+0000","last_became_peered":"2024-04-25T08:37:28.809692+0000","last_unstale":"2024-04-25T08:37:30.808970+0000","last_undegraded":"2024-04-25T08:37:30.808970+0000","last_fullsized":"2024-04-25T08:37:30.808970+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:29.093482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819067+0000","last_change":"2024-04-25T08:37:29.816751+0000","last_active":"2024-04-25T08:37:29.819067+0000","last_peered":"2024-04-25T08:37:29.819067+0000","last_clean":"2024-04-25T08:37:29.819067+0000","last_became_active":"2024-04-25T08:37:29.816395+0000","last_became_peered":"2024-04-25T08:37:29.816395+0000","last_unstale":"2024-04-25T08:37:29.819067+0000","last_undegraded":"2024-04-25T08:37:29.819067+0000","last_fullsized":"2024-04-25T08:37:29.819067+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:04.238401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.796328+0000","last_change":"2024-04-25T08:37:28.814614+0000","last_active":"2024-04-25T08:37:29.796328+0000","last_peered":"2024-04-25T08:37:29.796328+0000","last_clean":"2024-04-25T08:37:29.796328+0000","last_became_active":"2024-04-25T08:37:28.813602+0000","last_became_peered":"2024-04-25T08:37:28.813602+0000","last_unstale":"2024-04-25T08:37:29.796328+0000","last_undegraded":"2024-04-25T08:37:29.796328+0000","last_fullsized":"2024-04-25T08:37:29.796328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:23.987856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.808805+0000","last_change":"2024-04-25T08:37:29.804329+0000","last_active":"2024-04-25T08:37:29.808805+0000","last_peered":"2024-04-25T08:37:29.808805+0000","last_clean":"2024-04-25T08:37:29.808805+0000","last_became_active":"2024-04-25T08:37:29.803921+0000","last_became_peered":"2024-04-25T08:37:29.803921+0000","last_unstale":"2024-04-25T08:37:29.808805+0000","last_undegraded":"2024-04-25T08:37:29.808805+0000","last_fullsized":"2024-04-25T08:37:29.808805+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:10.303139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.798114+0000","last_change":"2024-04-25T08:37:28.805295+0000","last_active":"2024-04-25T08:37:29.798114+0000","last_peered":"2024-04-25T08:37:29.798114+0000","last_clean":"2024-04-25T08:37:29.798114+0000","last_became_active":"2024-04-25T08:37:28.804997+0000","last_became_peered":"2024-04-25T08:37:28.804997+0000","last_unstale":"2024-04-25T08:37:29.798114+0000","last_undegraded":"2024-04-25T08:37:29.798114+0000","last_fullsized":"2024-04-25T08:37:29.798114+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:56.626573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811792+0000","last_change":"2024-04-25T08:37:29.810461+0000","last_active":"2024-04-25T08:37:29.811792+0000","last_peered":"2024-04-25T08:37:29.811792+0000","last_clean":"2024-04-25T08:37:29.811792+0000","last_became_active":"2024-04-25T08:37:29.810119+0000","last_became_peered":"2024-04-25T08:37:29.810119+0000","last_unstale":"2024-04-25T08:37:29.811792+0000","last_undegraded":"2024-04-25T08:37:29.811792+0000","last_fullsized":"2024-04-25T08:37:29.811792+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:57.631095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.797376+0000","last_change":"2024-04-25T08:37:28.807200+0000","last_active":"2024-04-25T08:37:29.797376+0000","last_peered":"2024-04-25T08:37:29.797376+0000","last_clean":"2024-04-25T08:37:29.797376+0000","last_became_active":"2024-04-25T08:37:28.805882+0000","last_became_peered":"2024-04-25T08:37:28.805882+0000","last_unstale":"2024-04-25T08:37:29.797376+0000","last_undegraded":"2024-04-25T08:37:29.797376+0000","last_fullsized":"2024-04-25T08:37:29.797376+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:45.694831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819577+0000","last_change":"2024-04-25T08:37:29.815381+0000","last_active":"2024-04-25T08:37:29.819577+0000","last_peered":"2024-04-25T08:37:29.819577+0000","last_clean":"2024-04-25T08:37:29.819577+0000","last_became_active":"2024-04-25T08:37:29.814953+0000","last_became_peered":"2024-04-25T08:37:29.814953+0000","last_unstale":"2024-04-25T08:37:29.819577+0000","last_undegraded":"2024-04-25T08:37:29.819577+0000","last_fullsized":"2024-04-25T08:37:29.819577+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:54.030197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.795421+0000","last_change":"2024-04-25T08:37:28.808866+0000","last_active":"2024-04-25T08:37:29.795421+0000","last_peered":"2024-04-25T08:37:29.795421+0000","last_clean":"2024-04-25T08:37:29.795421+0000","last_became_active":"2024-04-25T08:37:28.808571+0000","last_became_peered":"2024-04-25T08:37:28.808571+0000","last_unstale":"2024-04-25T08:37:29.795421+0000","last_undegraded":"2024-04-25T08:37:29.795421+0000","last_fullsized":"2024-04-25T08:37:29.795421+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:52.605914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.812499+0000","last_change":"2024-04-25T08:37:29.809812+0000","last_active":"2024-04-25T08:37:29.812499+0000","last_peered":"2024-04-25T08:37:29.812499+0000","last_clean":"2024-04-25T08:37:29.812499+0000","last_became_active":"2024-04-25T08:37:29.809502+0000","last_became_peered":"2024-04-25T08:37:29.809502+0000","last_unstale":"2024-04-25T08:37:29.812499+0000","last_undegraded":"2024-04-25T08:37:29.812499+0000","last_fullsized":"2024-04-25T08:37:29.812499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:34.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.804126+0000","last_change":"2024-04-25T08:37:28.811212+0000","last_active":"2024-04-25T08:37:29.804126+0000","last_peered":"2024-04-25T08:37:29.804126+0000","last_clean":"2024-04-25T08:37:29.804126+0000","last_became_active":"2024-04-25T08:37:28.810583+0000","last_became_peered":"2024-04-25T08:37:28.810583+0000","last_unstale":"2024-04-25T08:37:29.804126+0000","last_undegraded":"2024-04-25T08:37:29.804126+0000","last_fullsized":"2024-04-25T08:37:29.804126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:37.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813041+0000","last_change":"2024-04-25T08:37:29.811744+0000","last_active":"2024-04-25T08:37:29.813041+0000","last_peered":"2024-04-25T08:37:29.813041+0000","last_clean":"2024-04-25T08:37:29.813041+0000","last_became_active":"2024-04-25T08:37:29.811318+0000","last_became_peered":"2024-04-25T08:37:29.811318+0000","last_unstale":"2024-04-25T08:37:29.813041+0000","last_undegraded":"2024-04-25T08:37:29.813041+0000","last_fullsized":"2024-04-25T08:37:29.813041+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"122'8","reported_seq":20,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.830291+0000","last_change":"2024-04-25T08:37:28.813007+0000","last_active":"2024-04-25T08:37:29.830291+0000","last_peered":"2024-04-25T08:37:29.830291+0000","last_clean":"2024-04-25T08:37:29.830291+0000","last_became_active":"2024-04-25T08:37:28.812716+0000","last_became_peered":"2024-04-25T08:37:28.812716+0000","last_unstale":"2024-04-25T08:37:29.830291+0000","last_undegraded":"2024-04-25T08:37:29.830291+0000","last_fullsized":"2024-04-25T08:37:29.830291+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:52.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":360,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.804270+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:37:29.804270+0000","last_peered":"2024-04-25T08:37:29.804270+0000","last_clean":"2024-04-25T08:37:29.804270+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:37:29.804270+0000","last_undegraded":"2024-04-25T08:37:29.804270+0000","last_fullsized":"2024-04-25T08:37:29.804270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45740,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326100,"statfs":{"total":96636764160,"available":96589926400,"internally_reserved":0,"allocated":2125824,"data_stored":1297479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46564,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325276,"statfs":{"total":96636764160,"available":96589082624,"internally_reserved":0,"allocated":1531904,"data_stored":713609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63204,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308636,"statfs":{"total":96636764160,"available":96572043264,"internally_reserved":0,"allocated":1523712,"data_stored":706495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50728,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321112,"statfs":{"total":96636764160,"available":96584818688,"internally_reserved":0,"allocated":1527808,"data_stored":707947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54884,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316956,"statfs":{"total":96636764160,"available":96580562944,"internally_reserved":0,"allocated":1531904,"data_stored":707981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54872,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316968,"statfs":{"total":96636764160,"available":96580575232,"internally_reserved":0,"allocated":1515520,"data_stored":706335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50716,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321124,"statfs":{"total":96636764160,"available":96584830976,"internally_reserved":0,"allocated":1519616,"data_stored":706357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55440,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316400,"statfs":{"total":96636764160,"available":96579993600,"internally_reserved":0,"allocated":2097152,"data_stored":1289536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:37:33.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:37:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 Processor -- start 2024-04-25T08:37:33.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 -- start start 2024-04-25T08:37:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4cc0b5f00 0x7fd4cc0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0cec00 con 0x7fd4cc15df40 2024-04-25T08:37:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0ce930 con 0x7fd4cc0b5f00 2024-04-25T08:37:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.942+0000 7fd4d2b7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0b6450 con 0x7fd4cc15db60 2024-04-25T08:37:33.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4cc0b5f00 0x7fd4cc0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd4cc0b5f00 0x7fd4cc0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53440/0 (socket says 172.21.15.60:53440) 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 -- 172.21.15.60:0/1656707988 learned_addr learned my addr 172.21.15.60:0/1656707988 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4d091a640 1 --2- 172.21.15.60:0/1656707988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 -- 172.21.15.60:0/1656707988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 msgr2=0x7fd4cc0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 --2- 172.21.15.60:0/1656707988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 -- 172.21.15.60:0/1656707988 >> v1:172.21.15.100:6789/0 conn(0x7fd4cc15df40 legacy=0x7fd4cc161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.943+0000 7fd4cbfff640 1 -- 172.21.15.60:0/1656707988 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0ceee0 con 0x7fd4cc0b5f00 2024-04-25T08:37:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4cbfff640 1 --2- 172.21.15.60:0/1656707988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0d0400 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098a0 tx=0x7fd4c002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4601c7192566a6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4cb7fe640 1 -- 172.21.15.60:0/1656707988 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037070 con 0x7fd4cc0b5f00 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4cb7fe640 1 -- 172.21.15.60:0/1656707988 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c0032440 con 0x7fd4cc0b5f00 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4cb7fe640 1 -- 172.21.15.60:0/1656707988 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0032810 con 0x7fd4cc0b5f00 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/1656707988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc0b5f00 msgr2=0x7fd4cc0d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/1656707988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0d0400 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098a0 tx=0x7fd4c002a200 comp rx=0 tx=0).stop 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/1656707988 shutdown_connections 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/1656707988 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0d0400 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/1656707988 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/1656707988 >> 172.21.15.60:0/1656707988 conn(0x7fd4cc061600 msgr2=0x7fd4cc061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/1656707988 shutdown_connections 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.944+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/1656707988 wait complete. 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 Processor -- start 2024-04-25T08:37:33.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 -- start start 2024-04-25T08:37:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 0x7fd4cc160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d3280 con 0x7fd4cc15df40 2024-04-25T08:37:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d1d50 con 0x7fd4cc0b5f00 2024-04-25T08:37:33.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.946+0000 7fd4d2b7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d3400 con 0x7fd4cc15db60 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 0x7fd4cc160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 0x7fd4cc160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53444/0 (socket says 172.21.15.60:53444) 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 -- 172.21.15.60:0/2744051674 learned_addr learned my addr 172.21.15.60:0/2744051674 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4cbfff640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d091a640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 -- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 msgr2=0x7fd4cc160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 -- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4cc0b5f00 msgr2=0x7fd4cc0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 -- 172.21.15.60:0/2744051674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc1612f0 con 0x7fd4cc15df40 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d111b640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 0x7fd4cc160db0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c400 tx=0x7fd4bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc017070 con 0x7fd4cc15df40 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc0052d0 con 0x7fd4cc15df40 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc0cbb90 con 0x7fd4cc15df40 2024-04-25T08:37:33.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc003810 con 0x7fd4cc15df40 2024-04-25T08:37:33.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.947+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc0cc150 con 0x7fd4cc15df40 2024-04-25T08:37:33.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.948+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc073270 con 0x7fd4cc15df40 2024-04-25T08:37:33.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.949+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4bc01a070 con 0x7fd4cc15df40 2024-04-25T08:37:33.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.949+0000 7fd4c97fa640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd4b806b4d0 0x7fd4b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:33.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.950+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4bc089650 con 0x7fd4cc15df40 2024-04-25T08:37:33.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.950+0000 7fd4d091a640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd4b806b4d0 0x7fd4b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:33.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.950+0000 7fd4d091a640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd4b806b4d0 0x7fd4b806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4003b80 tx=0x7fd4c40098d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:33.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:33.954+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4bc057cf0 con 0x7fd4cc15df40 2024-04-25T08:37:34.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.130+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4cc0d1860 con 0x7fd4b806b4d0 2024-04-25T08:37:34.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.137+0000 7fd4c97fa640 1 -- 172.21.15.60:0/2744051674 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7fd4cc0d1860 con 0x7fd4b806b4d0 2024-04-25T08:37:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:37:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd4b806b4d0 msgr2=0x7fd4b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd4b806b4d0 0x7fd4b806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4003b80 tx=0x7fd4c40098d0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 msgr2=0x7fd4cc160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 0x7fd4cc160db0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c400 tx=0x7fd4bc00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 shutdown_connections 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd4cc15df40 0x7fd4cc160db0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd4cc15db60 0x7fd4cc160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd4b806b4d0 0x7fd4b806d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 --2- 172.21.15.60:0/2744051674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4cc0b5f00 0x7fd4cc0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 >> 172.21.15.60:0/2744051674 conn(0x7fd4cc061600 msgr2=0x7fd4cc0b38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 shutdown_connections 2024-04-25T08:37:34.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.140+0000 7fd4d2b7e640 1 -- 172.21.15.60:0/2744051674 wait complete. 2024-04-25T08:37:34.145 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-25T08:37:33.510830+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422148,"kb_used_data":13060,"kb_used_omap":12,"kb_used_meta":408947,"kb_avail":754552572,"statfs":{"total":773094113280,"available":772661833728,"internally_reserved":0,"allocated":13373440,"data_stored":6835739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":418762288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.724457"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.807386+0000","last_change":"2024-04-25T08:37:29.805394+0000","last_active":"2024-04-25T08:37:29.807386+0000","last_peered":"2024-04-25T08:37:29.807386+0000","last_clean":"2024-04-25T08:37:29.807386+0000","last_became_active":"2024-04-25T08:37:29.805096+0000","last_became_peered":"2024-04-25T08:37:29.805096+0000","last_unstale":"2024-04-25T08:37:29.807386+0000","last_undegraded":"2024-04-25T08:37:29.807386+0000","last_fullsized":"2024-04-25T08:37:29.807386+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:39.062426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820911+0000","last_change":"2024-04-25T08:37:29.819145+0000","last_active":"2024-04-25T08:37:29.820911+0000","last_peered":"2024-04-25T08:37:29.820911+0000","last_clean":"2024-04-25T08:37:29.820911+0000","last_became_active":"2024-04-25T08:37:29.817988+0000","last_became_peered":"2024-04-25T08:37:29.817988+0000","last_unstale":"2024-04-25T08:37:29.820911+0000","last_undegraded":"2024-04-25T08:37:29.820911+0000","last_fullsized":"2024-04-25T08:37:29.820911+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:13.708035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819438+0000","last_change":"2024-04-25T08:37:29.816297+0000","last_active":"2024-04-25T08:37:29.819438+0000","last_peered":"2024-04-25T08:37:29.819438+0000","last_clean":"2024-04-25T08:37:29.819438+0000","last_became_active":"2024-04-25T08:37:29.815841+0000","last_became_peered":"2024-04-25T08:37:29.815841+0000","last_unstale":"2024-04-25T08:37:29.819438+0000","last_undegraded":"2024-04-25T08:37:29.819438+0000","last_fullsized":"2024-04-25T08:37:29.819438+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819582+0000","last_change":"2024-04-25T08:37:29.815164+0000","last_active":"2024-04-25T08:37:29.819582+0000","last_peered":"2024-04-25T08:37:29.819582+0000","last_clean":"2024-04-25T08:37:29.819582+0000","last_became_active":"2024-04-25T08:37:29.814481+0000","last_became_peered":"2024-04-25T08:37:29.814481+0000","last_unstale":"2024-04-25T08:37:29.819582+0000","last_undegraded":"2024-04-25T08:37:29.819582+0000","last_fullsized":"2024-04-25T08:37:29.819582+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:15.690894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.806474+0000","last_change":"2024-04-25T08:37:29.800742+0000","last_active":"2024-04-25T08:37:29.806474+0000","last_peered":"2024-04-25T08:37:29.806474+0000","last_clean":"2024-04-25T08:37:29.806474+0000","last_became_active":"2024-04-25T08:37:29.800414+0000","last_became_peered":"2024-04-25T08:37:29.800414+0000","last_unstale":"2024-04-25T08:37:29.806474+0000","last_undegraded":"2024-04-25T08:37:29.806474+0000","last_fullsized":"2024-04-25T08:37:29.806474+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:53.160556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819332+0000","last_change":"2024-04-25T08:37:29.816221+0000","last_active":"2024-04-25T08:37:29.819332+0000","last_peered":"2024-04-25T08:37:29.819332+0000","last_clean":"2024-04-25T08:37:29.819332+0000","last_became_active":"2024-04-25T08:37:29.815558+0000","last_became_peered":"2024-04-25T08:37:29.815558+0000","last_unstale":"2024-04-25T08:37:29.819332+0000","last_undegraded":"2024-04-25T08:37:29.819332+0000","last_fullsized":"2024-04-25T08:37:29.819332+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:43.591982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819990+0000","last_change":"2024-04-25T08:37:29.818826+0000","last_active":"2024-04-25T08:37:29.819990+0000","last_peered":"2024-04-25T08:37:29.819990+0000","last_clean":"2024-04-25T08:37:29.819990+0000","last_became_active":"2024-04-25T08:37:29.818170+0000","last_became_peered":"2024-04-25T08:37:29.818170+0000","last_unstale":"2024-04-25T08:37:29.819990+0000","last_undegraded":"2024-04-25T08:37:29.819990+0000","last_fullsized":"2024-04-25T08:37:29.819990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:38.631283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815350+0000","last_change":"2024-04-25T08:37:29.812644+0000","last_active":"2024-04-25T08:37:29.815350+0000","last_peered":"2024-04-25T08:37:29.815350+0000","last_clean":"2024-04-25T08:37:29.815350+0000","last_became_active":"2024-04-25T08:37:29.812327+0000","last_became_peered":"2024-04-25T08:37:29.812327+0000","last_unstale":"2024-04-25T08:37:29.815350+0000","last_undegraded":"2024-04-25T08:37:29.815350+0000","last_fullsized":"2024-04-25T08:37:29.815350+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:39.062426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815854+0000","last_change":"2024-04-25T08:37:29.814674+0000","last_active":"2024-04-25T08:37:29.815854+0000","last_peered":"2024-04-25T08:37:29.815854+0000","last_clean":"2024-04-25T08:37:29.815854+0000","last_became_active":"2024-04-25T08:37:29.814368+0000","last_became_peered":"2024-04-25T08:37:29.814368+0000","last_unstale":"2024-04-25T08:37:29.815854+0000","last_undegraded":"2024-04-25T08:37:29.815854+0000","last_fullsized":"2024-04-25T08:37:29.815854+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.171300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820141+0000","last_change":"2024-04-25T08:37:29.814965+0000","last_active":"2024-04-25T08:37:29.820141+0000","last_peered":"2024-04-25T08:37:29.820141+0000","last_clean":"2024-04-25T08:37:29.820141+0000","last_became_active":"2024-04-25T08:37:29.814694+0000","last_became_peered":"2024-04-25T08:37:29.814694+0000","last_unstale":"2024-04-25T08:37:29.820141+0000","last_undegraded":"2024-04-25T08:37:29.820141+0000","last_fullsized":"2024-04-25T08:37:29.820141+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:30.169153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811689+0000","last_change":"2024-04-25T08:37:29.810114+0000","last_active":"2024-04-25T08:37:29.811689+0000","last_peered":"2024-04-25T08:37:29.811689+0000","last_clean":"2024-04-25T08:37:29.811689+0000","last_became_active":"2024-04-25T08:37:29.809857+0000","last_became_peered":"2024-04-25T08:37:29.809857+0000","last_unstale":"2024-04-25T08:37:29.811689+0000","last_undegraded":"2024-04-25T08:37:29.811689+0000","last_fullsized":"2024-04-25T08:37:29.811689+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:26.707410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.809032+0000","last_change":"2024-04-25T08:37:29.805126+0000","last_active":"2024-04-25T08:37:29.809032+0000","last_peered":"2024-04-25T08:37:29.809032+0000","last_clean":"2024-04-25T08:37:29.809032+0000","last_became_active":"2024-04-25T08:37:29.803937+0000","last_became_peered":"2024-04-25T08:37:29.803937+0000","last_unstale":"2024-04-25T08:37:29.809032+0000","last_undegraded":"2024-04-25T08:37:29.809032+0000","last_fullsized":"2024-04-25T08:37:29.809032+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:11.881085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.810034+0000","last_change":"2024-04-25T08:37:29.808286+0000","last_active":"2024-04-25T08:37:29.810034+0000","last_peered":"2024-04-25T08:37:29.810034+0000","last_clean":"2024-04-25T08:37:29.810034+0000","last_became_active":"2024-04-25T08:37:29.807915+0000","last_became_peered":"2024-04-25T08:37:29.807915+0000","last_unstale":"2024-04-25T08:37:29.810034+0000","last_undegraded":"2024-04-25T08:37:29.810034+0000","last_fullsized":"2024-04-25T08:37:29.810034+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:26.707410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.823623+0000","last_change":"2024-04-25T08:37:29.822699+0000","last_active":"2024-04-25T08:37:29.823623+0000","last_peered":"2024-04-25T08:37:29.823623+0000","last_clean":"2024-04-25T08:37:29.823623+0000","last_became_active":"2024-04-25T08:37:29.820717+0000","last_became_peered":"2024-04-25T08:37:29.820717+0000","last_unstale":"2024-04-25T08:37:29.823623+0000","last_undegraded":"2024-04-25T08:37:29.823623+0000","last_fullsized":"2024-04-25T08:37:29.823623+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811721+0000","last_change":"2024-04-25T08:37:29.809932+0000","last_active":"2024-04-25T08:37:29.811721+0000","last_peered":"2024-04-25T08:37:29.811721+0000","last_clean":"2024-04-25T08:37:29.811721+0000","last_became_active":"2024-04-25T08:37:29.809603+0000","last_became_peered":"2024-04-25T08:37:29.809603+0000","last_unstale":"2024-04-25T08:37:29.811721+0000","last_undegraded":"2024-04-25T08:37:29.811721+0000","last_fullsized":"2024-04-25T08:37:29.811721+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:18.794051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.808829+0000","last_change":"2024-04-25T08:37:29.805773+0000","last_active":"2024-04-25T08:37:29.808829+0000","last_peered":"2024-04-25T08:37:29.808829+0000","last_clean":"2024-04-25T08:37:29.808829+0000","last_became_active":"2024-04-25T08:37:29.805097+0000","last_became_peered":"2024-04-25T08:37:29.805097+0000","last_unstale":"2024-04-25T08:37:29.808829+0000","last_undegraded":"2024-04-25T08:37:29.808829+0000","last_fullsized":"2024-04-25T08:37:29.808829+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:52.013023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.823458+0000","last_change":"2024-04-25T08:37:29.818124+0000","last_active":"2024-04-25T08:37:29.823458+0000","last_peered":"2024-04-25T08:37:29.823458+0000","last_clean":"2024-04-25T08:37:29.823458+0000","last_became_active":"2024-04-25T08:37:29.817812+0000","last_became_peered":"2024-04-25T08:37:29.817812+0000","last_unstale":"2024-04-25T08:37:29.823458+0000","last_undegraded":"2024-04-25T08:37:29.823458+0000","last_fullsized":"2024-04-25T08:37:29.823458+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.819438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820888+0000","last_change":"2024-04-25T08:37:29.818854+0000","last_active":"2024-04-25T08:37:29.820888+0000","last_peered":"2024-04-25T08:37:29.820888+0000","last_clean":"2024-04-25T08:37:29.820888+0000","last_became_active":"2024-04-25T08:37:29.817586+0000","last_became_peered":"2024-04-25T08:37:29.817586+0000","last_unstale":"2024-04-25T08:37:29.820888+0000","last_undegraded":"2024-04-25T08:37:29.820888+0000","last_fullsized":"2024-04-25T08:37:29.820888+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:23.074832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808714+0000","last_change":"2024-04-25T08:37:29.812146+0000","last_active":"2024-04-25T08:37:30.808714+0000","last_peered":"2024-04-25T08:37:30.808714+0000","last_clean":"2024-04-25T08:37:30.808714+0000","last_became_active":"2024-04-25T08:37:29.811887+0000","last_became_peered":"2024-04-25T08:37:29.811887+0000","last_unstale":"2024-04-25T08:37:30.808714+0000","last_undegraded":"2024-04-25T08:37:30.808714+0000","last_fullsized":"2024-04-25T08:37:30.808714+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811818+0000","last_change":"2024-04-25T08:37:29.809752+0000","last_active":"2024-04-25T08:37:29.811818+0000","last_peered":"2024-04-25T08:37:29.811818+0000","last_clean":"2024-04-25T08:37:29.811818+0000","last_became_active":"2024-04-25T08:37:29.809457+0000","last_became_peered":"2024-04-25T08:37:29.809457+0000","last_unstale":"2024-04-25T08:37:29.811818+0000","last_undegraded":"2024-04-25T08:37:29.811818+0000","last_fullsized":"2024-04-25T08:37:29.811818+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:53.585442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813020+0000","last_change":"2024-04-25T08:37:29.810594+0000","last_active":"2024-04-25T08:37:29.813020+0000","last_peered":"2024-04-25T08:37:29.813020+0000","last_clean":"2024-04-25T08:37:29.813020+0000","last_became_active":"2024-04-25T08:37:29.810293+0000","last_became_peered":"2024-04-25T08:37:29.810293+0000","last_unstale":"2024-04-25T08:37:29.813020+0000","last_undegraded":"2024-04-25T08:37:29.813020+0000","last_fullsized":"2024-04-25T08:37:29.813020+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814004+0000","last_change":"2024-04-25T08:37:29.812391+0000","last_active":"2024-04-25T08:37:29.814004+0000","last_peered":"2024-04-25T08:37:29.814004+0000","last_clean":"2024-04-25T08:37:29.814004+0000","last_became_active":"2024-04-25T08:37:29.812085+0000","last_became_peered":"2024-04-25T08:37:29.812085+0000","last_unstale":"2024-04-25T08:37:29.814004+0000","last_undegraded":"2024-04-25T08:37:29.814004+0000","last_fullsized":"2024-04-25T08:37:29.814004+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:33.914976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815883+0000","last_change":"2024-04-25T08:37:29.813955+0000","last_active":"2024-04-25T08:37:29.815883+0000","last_peered":"2024-04-25T08:37:29.815883+0000","last_clean":"2024-04-25T08:37:29.815883+0000","last_became_active":"2024-04-25T08:37:29.813684+0000","last_became_peered":"2024-04-25T08:37:29.813684+0000","last_unstale":"2024-04-25T08:37:29.815883+0000","last_undegraded":"2024-04-25T08:37:29.815883+0000","last_fullsized":"2024-04-25T08:37:29.815883+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:33.568662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437680+0000","last_change":"2024-04-25T08:37:29.815329+0000","last_active":"2024-04-25T08:37:31.437680+0000","last_peered":"2024-04-25T08:37:31.437680+0000","last_clean":"2024-04-25T08:37:31.437680+0000","last_became_active":"2024-04-25T08:37:29.814718+0000","last_became_peered":"2024-04-25T08:37:29.814718+0000","last_unstale":"2024-04-25T08:37:31.437680+0000","last_undegraded":"2024-04-25T08:37:31.437680+0000","last_fullsized":"2024-04-25T08:37:31.437680+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:48.559466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.438059+0000","last_change":"2024-04-25T08:37:29.818063+0000","last_active":"2024-04-25T08:37:31.438059+0000","last_peered":"2024-04-25T08:37:31.438059+0000","last_clean":"2024-04-25T08:37:31.438059+0000","last_became_active":"2024-04-25T08:37:29.817777+0000","last_became_peered":"2024-04-25T08:37:29.817777+0000","last_unstale":"2024-04-25T08:37:31.438059+0000","last_undegraded":"2024-04-25T08:37:31.438059+0000","last_fullsized":"2024-04-25T08:37:31.438059+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:54.030197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811751+0000","last_change":"2024-04-25T08:37:29.808092+0000","last_active":"2024-04-25T08:37:29.811751+0000","last_peered":"2024-04-25T08:37:29.811751+0000","last_clean":"2024-04-25T08:37:29.811751+0000","last_became_active":"2024-04-25T08:37:29.807774+0000","last_became_peered":"2024-04-25T08:37:29.807774+0000","last_unstale":"2024-04-25T08:37:29.811751+0000","last_undegraded":"2024-04-25T08:37:29.811751+0000","last_fullsized":"2024-04-25T08:37:29.811751+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:48.310586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808868+0000","last_change":"2024-04-25T08:37:29.811940+0000","last_active":"2024-04-25T08:37:30.808868+0000","last_peered":"2024-04-25T08:37:30.808868+0000","last_clean":"2024-04-25T08:37:30.808868+0000","last_became_active":"2024-04-25T08:37:29.811495+0000","last_became_peered":"2024-04-25T08:37:29.811495+0000","last_unstale":"2024-04-25T08:37:30.808868+0000","last_undegraded":"2024-04-25T08:37:30.808868+0000","last_fullsized":"2024-04-25T08:37:30.808868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:48.559466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819950+0000","last_change":"2024-04-25T08:37:29.819029+0000","last_active":"2024-04-25T08:37:29.819950+0000","last_peered":"2024-04-25T08:37:29.819950+0000","last_clean":"2024-04-25T08:37:29.819950+0000","last_became_active":"2024-04-25T08:37:29.817916+0000","last_became_peered":"2024-04-25T08:37:29.817916+0000","last_unstale":"2024-04-25T08:37:29.819950+0000","last_undegraded":"2024-04-25T08:37:29.819950+0000","last_fullsized":"2024-04-25T08:37:29.819950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:53.401391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.818372+0000","last_change":"2024-04-25T08:37:29.815591+0000","last_active":"2024-04-25T08:37:29.818372+0000","last_peered":"2024-04-25T08:37:29.818372+0000","last_clean":"2024-04-25T08:37:29.818372+0000","last_became_active":"2024-04-25T08:37:29.815268+0000","last_became_peered":"2024-04-25T08:37:29.815268+0000","last_unstale":"2024-04-25T08:37:29.818372+0000","last_undegraded":"2024-04-25T08:37:29.818372+0000","last_fullsized":"2024-04-25T08:37:29.818372+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:24.305617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.822858+0000","last_change":"2024-04-25T08:37:29.817058+0000","last_active":"2024-04-25T08:37:29.822858+0000","last_peered":"2024-04-25T08:37:29.822858+0000","last_clean":"2024-04-25T08:37:29.822858+0000","last_became_active":"2024-04-25T08:37:29.816740+0000","last_became_peered":"2024-04-25T08:37:29.816740+0000","last_unstale":"2024-04-25T08:37:29.822858+0000","last_undegraded":"2024-04-25T08:37:29.822858+0000","last_fullsized":"2024-04-25T08:37:29.822858+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:12.773879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437790+0000","last_change":"2024-04-25T08:37:29.814763+0000","last_active":"2024-04-25T08:37:31.437790+0000","last_peered":"2024-04-25T08:37:31.437790+0000","last_clean":"2024-04-25T08:37:31.437790+0000","last_became_active":"2024-04-25T08:37:29.814456+0000","last_became_peered":"2024-04-25T08:37:29.814456+0000","last_unstale":"2024-04-25T08:37:31.437790+0000","last_undegraded":"2024-04-25T08:37:31.437790+0000","last_fullsized":"2024-04-25T08:37:31.437790+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:24.135064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437613+0000","last_change":"2024-04-25T08:37:29.815101+0000","last_active":"2024-04-25T08:37:31.437613+0000","last_peered":"2024-04-25T08:37:31.437613+0000","last_clean":"2024-04-25T08:37:31.437613+0000","last_became_active":"2024-04-25T08:37:29.814141+0000","last_became_peered":"2024-04-25T08:37:29.814141+0000","last_unstale":"2024-04-25T08:37:31.437613+0000","last_undegraded":"2024-04-25T08:37:31.437613+0000","last_fullsized":"2024-04-25T08:37:31.437613+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:28.098789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813549+0000","last_change":"2024-04-25T08:37:29.811608+0000","last_active":"2024-04-25T08:37:29.813549+0000","last_peered":"2024-04-25T08:37:29.813549+0000","last_clean":"2024-04-25T08:37:29.813549+0000","last_became_active":"2024-04-25T08:37:29.810656+0000","last_became_peered":"2024-04-25T08:37:29.810656+0000","last_unstale":"2024-04-25T08:37:29.813549+0000","last_undegraded":"2024-04-25T08:37:29.813549+0000","last_fullsized":"2024-04-25T08:37:29.813549+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:30.098003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"122'4","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.809109+0000","last_change":"2024-04-25T08:37:28.808094+0000","last_active":"2024-04-25T08:37:30.809109+0000","last_peered":"2024-04-25T08:37:30.809109+0000","last_clean":"2024-04-25T08:37:30.809109+0000","last_became_active":"2024-04-25T08:37:28.807758+0000","last_became_peered":"2024-04-25T08:37:28.807758+0000","last_unstale":"2024-04-25T08:37:30.809109+0000","last_undegraded":"2024-04-25T08:37:30.809109+0000","last_fullsized":"2024-04-25T08:37:30.809109+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:33.377078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820231+0000","last_change":"2024-04-25T08:37:29.819619+0000","last_active":"2024-04-25T08:37:29.820231+0000","last_peered":"2024-04-25T08:37:29.820231+0000","last_clean":"2024-04-25T08:37:29.820231+0000","last_became_active":"2024-04-25T08:37:29.818993+0000","last_became_peered":"2024-04-25T08:37:29.818993+0000","last_unstale":"2024-04-25T08:37:29.820231+0000","last_undegraded":"2024-04-25T08:37:29.820231+0000","last_fullsized":"2024-04-25T08:37:29.820231+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:25.579263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.823563+0000","last_change":"2024-04-25T08:37:28.805556+0000","last_active":"2024-04-25T08:37:29.823563+0000","last_peered":"2024-04-25T08:37:29.823563+0000","last_clean":"2024-04-25T08:37:29.823563+0000","last_became_active":"2024-04-25T08:37:28.805190+0000","last_became_peered":"2024-04-25T08:37:28.805190+0000","last_unstale":"2024-04-25T08:37:29.823563+0000","last_undegraded":"2024-04-25T08:37:29.823563+0000","last_fullsized":"2024-04-25T08:37:29.823563+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:46.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437947+0000","last_change":"2024-04-25T08:37:29.821571+0000","last_active":"2024-04-25T08:37:31.437947+0000","last_peered":"2024-04-25T08:37:31.437947+0000","last_clean":"2024-04-25T08:37:31.437947+0000","last_became_active":"2024-04-25T08:37:29.821234+0000","last_became_peered":"2024-04-25T08:37:29.821234+0000","last_unstale":"2024-04-25T08:37:31.437947+0000","last_undegraded":"2024-04-25T08:37:31.437947+0000","last_fullsized":"2024-04-25T08:37:31.437947+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:15.690894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.824329+0000","last_change":"2024-04-25T08:37:28.809776+0000","last_active":"2024-04-25T08:37:29.824329+0000","last_peered":"2024-04-25T08:37:29.824329+0000","last_clean":"2024-04-25T08:37:29.824329+0000","last_became_active":"2024-04-25T08:37:28.809459+0000","last_became_peered":"2024-04-25T08:37:28.809459+0000","last_unstale":"2024-04-25T08:37:29.824329+0000","last_undegraded":"2024-04-25T08:37:29.824329+0000","last_fullsized":"2024-04-25T08:37:29.824329+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:22.070311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.812991+0000","last_change":"2024-04-25T08:37:29.810633+0000","last_active":"2024-04-25T08:37:29.812991+0000","last_peered":"2024-04-25T08:37:29.812991+0000","last_clean":"2024-04-25T08:37:29.812991+0000","last_became_active":"2024-04-25T08:37:29.810344+0000","last_became_peered":"2024-04-25T08:37:29.810344+0000","last_unstale":"2024-04-25T08:37:29.812991+0000","last_undegraded":"2024-04-25T08:37:29.812991+0000","last_fullsized":"2024-04-25T08:37:29.812991+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:48.559466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.827933+0000","last_change":"2024-04-25T08:37:28.810195+0000","last_active":"2024-04-25T08:37:29.827933+0000","last_peered":"2024-04-25T08:37:29.827933+0000","last_clean":"2024-04-25T08:37:29.827933+0000","last_became_active":"2024-04-25T08:37:28.809854+0000","last_became_peered":"2024-04-25T08:37:28.809854+0000","last_unstale":"2024-04-25T08:37:29.827933+0000","last_undegraded":"2024-04-25T08:37:29.827933+0000","last_fullsized":"2024-04-25T08:37:29.827933+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:38.057904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.804716+0000","last_change":"2024-04-25T08:37:29.802001+0000","last_active":"2024-04-25T08:37:29.804716+0000","last_peered":"2024-04-25T08:37:29.804716+0000","last_clean":"2024-04-25T08:37:29.804716+0000","last_became_active":"2024-04-25T08:37:29.801679+0000","last_became_peered":"2024-04-25T08:37:29.801679+0000","last_unstale":"2024-04-25T08:37:29.804716+0000","last_undegraded":"2024-04-25T08:37:29.804716+0000","last_fullsized":"2024-04-25T08:37:29.804716+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:53.160556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.822354+0000","last_change":"2024-04-25T08:37:28.807402+0000","last_active":"2024-04-25T08:37:29.822354+0000","last_peered":"2024-04-25T08:37:29.822354+0000","last_clean":"2024-04-25T08:37:29.822354+0000","last_became_active":"2024-04-25T08:37:28.806817+0000","last_became_peered":"2024-04-25T08:37:28.806817+0000","last_unstale":"2024-04-25T08:37:29.822354+0000","last_undegraded":"2024-04-25T08:37:29.822354+0000","last_fullsized":"2024-04-25T08:37:29.822354+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:27.918282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.810682+0000","last_change":"2024-04-25T08:37:29.808485+0000","last_active":"2024-04-25T08:37:29.810682+0000","last_peered":"2024-04-25T08:37:29.810682+0000","last_clean":"2024-04-25T08:37:29.810682+0000","last_became_active":"2024-04-25T08:37:29.808117+0000","last_became_peered":"2024-04-25T08:37:29.808117+0000","last_unstale":"2024-04-25T08:37:29.810682+0000","last_undegraded":"2024-04-25T08:37:29.810682+0000","last_fullsized":"2024-04-25T08:37:29.810682+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:47.389963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.824957+0000","last_change":"2024-04-25T08:37:28.811481+0000","last_active":"2024-04-25T08:37:29.824957+0000","last_peered":"2024-04-25T08:37:29.824957+0000","last_clean":"2024-04-25T08:37:29.824957+0000","last_became_active":"2024-04-25T08:37:28.809575+0000","last_became_peered":"2024-04-25T08:37:28.809575+0000","last_unstale":"2024-04-25T08:37:29.824957+0000","last_undegraded":"2024-04-25T08:37:29.824957+0000","last_fullsized":"2024-04-25T08:37:29.824957+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:37.083005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811831+0000","last_change":"2024-04-25T08:37:29.810618+0000","last_active":"2024-04-25T08:37:29.811831+0000","last_peered":"2024-04-25T08:37:29.811831+0000","last_clean":"2024-04-25T08:37:29.811831+0000","last_became_active":"2024-04-25T08:37:29.810305+0000","last_became_peered":"2024-04-25T08:37:29.810305+0000","last_unstale":"2024-04-25T08:37:29.811831+0000","last_undegraded":"2024-04-25T08:37:29.811831+0000","last_fullsized":"2024-04-25T08:37:29.811831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:53.160556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437791+0000","last_change":"2024-04-25T08:37:28.806715+0000","last_active":"2024-04-25T08:37:31.437791+0000","last_peered":"2024-04-25T08:37:31.437791+0000","last_clean":"2024-04-25T08:37:31.437791+0000","last_became_active":"2024-04-25T08:37:28.805236+0000","last_became_peered":"2024-04-25T08:37:28.805236+0000","last_unstale":"2024-04-25T08:37:31.437791+0000","last_undegraded":"2024-04-25T08:37:31.437791+0000","last_fullsized":"2024-04-25T08:37:31.437791+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:38.057904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813154+0000","last_change":"2024-04-25T08:37:29.811481+0000","last_active":"2024-04-25T08:37:29.813154+0000","last_peered":"2024-04-25T08:37:29.813154+0000","last_clean":"2024-04-25T08:37:29.813154+0000","last_became_active":"2024-04-25T08:37:29.811168+0000","last_became_peered":"2024-04-25T08:37:29.811168+0000","last_unstale":"2024-04-25T08:37:29.813154+0000","last_undegraded":"2024-04-25T08:37:29.813154+0000","last_fullsized":"2024-04-25T08:37:29.813154+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:39.330257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.796625+0000","last_change":"2024-04-25T08:37:28.812056+0000","last_active":"2024-04-25T08:37:29.796625+0000","last_peered":"2024-04-25T08:37:29.796625+0000","last_clean":"2024-04-25T08:37:29.796625+0000","last_became_active":"2024-04-25T08:37:28.811767+0000","last_became_peered":"2024-04-25T08:37:28.811767+0000","last_unstale":"2024-04-25T08:37:29.796625+0000","last_undegraded":"2024-04-25T08:37:29.796625+0000","last_fullsized":"2024-04-25T08:37:29.796625+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:17.850069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808384+0000","last_change":"2024-04-25T08:37:29.818925+0000","last_active":"2024-04-25T08:37:30.808384+0000","last_peered":"2024-04-25T08:37:30.808384+0000","last_clean":"2024-04-25T08:37:30.808384+0000","last_became_active":"2024-04-25T08:37:29.818194+0000","last_became_peered":"2024-04-25T08:37:29.818194+0000","last_unstale":"2024-04-25T08:37:30.808384+0000","last_undegraded":"2024-04-25T08:37:30.808384+0000","last_fullsized":"2024-04-25T08:37:30.808384+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:13.708035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.809204+0000","last_change":"2024-04-25T08:37:28.805159+0000","last_active":"2024-04-25T08:37:30.809204+0000","last_peered":"2024-04-25T08:37:30.809204+0000","last_clean":"2024-04-25T08:37:30.809204+0000","last_became_active":"2024-04-25T08:37:28.804900+0000","last_became_peered":"2024-04-25T08:37:28.804900+0000","last_unstale":"2024-04-25T08:37:30.809204+0000","last_undegraded":"2024-04-25T08:37:30.809204+0000","last_fullsized":"2024-04-25T08:37:30.809204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:52.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437867+0000","last_change":"2024-04-25T08:37:29.821883+0000","last_active":"2024-04-25T08:37:31.437867+0000","last_peered":"2024-04-25T08:37:31.437867+0000","last_clean":"2024-04-25T08:37:31.437867+0000","last_became_active":"2024-04-25T08:37:29.821534+0000","last_became_peered":"2024-04-25T08:37:29.821534+0000","last_unstale":"2024-04-25T08:37:31.437867+0000","last_undegraded":"2024-04-25T08:37:31.437867+0000","last_fullsized":"2024-04-25T08:37:31.437867+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:57.171300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437929+0000","last_change":"2024-04-25T08:37:28.804893+0000","last_active":"2024-04-25T08:37:31.437929+0000","last_peered":"2024-04-25T08:37:31.437929+0000","last_clean":"2024-04-25T08:37:31.437929+0000","last_became_active":"2024-04-25T08:37:28.804561+0000","last_became_peered":"2024-04-25T08:37:28.804561+0000","last_unstale":"2024-04-25T08:37:31.437929+0000","last_undegraded":"2024-04-25T08:37:31.437929+0000","last_fullsized":"2024-04-25T08:37:31.437929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:25.702889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820183+0000","last_change":"2024-04-25T08:37:29.814910+0000","last_active":"2024-04-25T08:37:29.820183+0000","last_peered":"2024-04-25T08:37:29.820183+0000","last_clean":"2024-04-25T08:37:29.820183+0000","last_became_active":"2024-04-25T08:37:29.814598+0000","last_became_peered":"2024-04-25T08:37:29.814598+0000","last_unstale":"2024-04-25T08:37:29.820183+0000","last_undegraded":"2024-04-25T08:37:29.820183+0000","last_fullsized":"2024-04-25T08:37:29.820183+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:39.330257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"122'3","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437857+0000","last_change":"2024-04-25T08:37:28.809333+0000","last_active":"2024-04-25T08:37:31.437857+0000","last_peered":"2024-04-25T08:37:31.437857+0000","last_clean":"2024-04-25T08:37:31.437857+0000","last_became_active":"2024-04-25T08:37:28.808954+0000","last_became_peered":"2024-04-25T08:37:28.808954+0000","last_unstale":"2024-04-25T08:37:31.437857+0000","last_undegraded":"2024-04-25T08:37:31.437857+0000","last_fullsized":"2024-04-25T08:37:31.437857+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:46.385442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.438114+0000","last_change":"2024-04-25T08:37:29.816262+0000","last_active":"2024-04-25T08:37:31.438114+0000","last_peered":"2024-04-25T08:37:31.438114+0000","last_clean":"2024-04-25T08:37:31.438114+0000","last_became_active":"2024-04-25T08:37:29.814603+0000","last_became_peered":"2024-04-25T08:37:29.814603+0000","last_unstale":"2024-04-25T08:37:31.438114+0000","last_undegraded":"2024-04-25T08:37:31.438114+0000","last_fullsized":"2024-04-25T08:37:31.438114+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:01.817130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"122'6","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.835313+0000","last_change":"2024-04-25T08:37:28.812343+0000","last_active":"2024-04-25T08:37:29.835313+0000","last_peered":"2024-04-25T08:37:29.835313+0000","last_clean":"2024-04-25T08:37:29.835313+0000","last_became_active":"2024-04-25T08:37:28.811991+0000","last_became_peered":"2024-04-25T08:37:28.811991+0000","last_unstale":"2024-04-25T08:37:29.835313+0000","last_undegraded":"2024-04-25T08:37:29.835313+0000","last_fullsized":"2024-04-25T08:37:29.835313+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:29.093482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808984+0000","last_change":"2024-04-25T08:37:29.811347+0000","last_active":"2024-04-25T08:37:30.808984+0000","last_peered":"2024-04-25T08:37:30.808984+0000","last_clean":"2024-04-25T08:37:30.808984+0000","last_became_active":"2024-04-25T08:37:29.811091+0000","last_became_peered":"2024-04-25T08:37:29.811091+0000","last_unstale":"2024-04-25T08:37:30.808984+0000","last_undegraded":"2024-04-25T08:37:30.808984+0000","last_fullsized":"2024-04-25T08:37:30.808984+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.830197+0000","last_change":"2024-04-25T08:37:28.809181+0000","last_active":"2024-04-25T08:37:29.830197+0000","last_peered":"2024-04-25T08:37:29.830197+0000","last_clean":"2024-04-25T08:37:29.830197+0000","last_became_active":"2024-04-25T08:37:28.808882+0000","last_became_peered":"2024-04-25T08:37:28.808882+0000","last_unstale":"2024-04-25T08:37:29.830197+0000","last_undegraded":"2024-04-25T08:37:29.830197+0000","last_fullsized":"2024-04-25T08:37:29.830197+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:33.377078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.812169+0000","last_change":"2024-04-25T08:37:29.809581+0000","last_active":"2024-04-25T08:37:29.812169+0000","last_peered":"2024-04-25T08:37:29.812169+0000","last_clean":"2024-04-25T08:37:29.812169+0000","last_became_active":"2024-04-25T08:37:29.809209+0000","last_became_peered":"2024-04-25T08:37:29.809209+0000","last_unstale":"2024-04-25T08:37:29.812169+0000","last_undegraded":"2024-04-25T08:37:29.812169+0000","last_fullsized":"2024-04-25T08:37:29.812169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:33.914976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813988+0000","last_change":"2024-04-25T08:37:28.804189+0000","last_active":"2024-04-25T08:37:29.813988+0000","last_peered":"2024-04-25T08:37:29.813988+0000","last_clean":"2024-04-25T08:37:29.813988+0000","last_became_active":"2024-04-25T08:37:28.803605+0000","last_became_peered":"2024-04-25T08:37:28.803605+0000","last_unstale":"2024-04-25T08:37:29.813988+0000","last_undegraded":"2024-04-25T08:37:29.813988+0000","last_fullsized":"2024-04-25T08:37:29.813988+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:17.789530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.807225+0000","last_change":"2024-04-25T08:37:29.803410+0000","last_active":"2024-04-25T08:37:29.807225+0000","last_peered":"2024-04-25T08:37:29.807225+0000","last_clean":"2024-04-25T08:37:29.807225+0000","last_became_active":"2024-04-25T08:37:29.803072+0000","last_became_peered":"2024-04-25T08:37:29.803072+0000","last_unstale":"2024-04-25T08:37:29.807225+0000","last_undegraded":"2024-04-25T08:37:29.807225+0000","last_fullsized":"2024-04-25T08:37:29.807225+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:09.497389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814231+0000","last_change":"2024-04-25T08:37:28.808072+0000","last_active":"2024-04-25T08:37:29.814231+0000","last_peered":"2024-04-25T08:37:29.814231+0000","last_clean":"2024-04-25T08:37:29.814231+0000","last_became_active":"2024-04-25T08:37:28.807797+0000","last_became_peered":"2024-04-25T08:37:28.807797+0000","last_unstale":"2024-04-25T08:37:29.814231+0000","last_undegraded":"2024-04-25T08:37:29.814231+0000","last_fullsized":"2024-04-25T08:37:29.814231+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:25.702889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437787+0000","last_change":"2024-04-25T08:37:29.820198+0000","last_active":"2024-04-25T08:37:31.437787+0000","last_peered":"2024-04-25T08:37:31.437787+0000","last_clean":"2024-04-25T08:37:31.437787+0000","last_became_active":"2024-04-25T08:37:29.818785+0000","last_became_peered":"2024-04-25T08:37:29.818785+0000","last_unstale":"2024-04-25T08:37:31.437787+0000","last_undegraded":"2024-04-25T08:37:31.437787+0000","last_fullsized":"2024-04-25T08:37:31.437787+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:33.568662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.825674+0000","last_change":"2024-04-25T08:37:28.815584+0000","last_active":"2024-04-25T08:37:29.825674+0000","last_peered":"2024-04-25T08:37:29.825674+0000","last_clean":"2024-04-25T08:37:29.825674+0000","last_became_active":"2024-04-25T08:37:28.815258+0000","last_became_peered":"2024-04-25T08:37:28.815258+0000","last_unstale":"2024-04-25T08:37:29.825674+0000","last_undegraded":"2024-04-25T08:37:29.825674+0000","last_fullsized":"2024-04-25T08:37:29.825674+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:17.789530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.821135+0000","last_change":"2024-04-25T08:37:29.817652+0000","last_active":"2024-04-25T08:37:29.821135+0000","last_peered":"2024-04-25T08:37:29.821135+0000","last_clean":"2024-04-25T08:37:29.821135+0000","last_became_active":"2024-04-25T08:37:29.817382+0000","last_became_peered":"2024-04-25T08:37:29.817382+0000","last_unstale":"2024-04-25T08:37:29.821135+0000","last_undegraded":"2024-04-25T08:37:29.821135+0000","last_fullsized":"2024-04-25T08:37:29.821135+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:34.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.826862+0000","last_change":"2024-04-25T08:37:28.814257+0000","last_active":"2024-04-25T08:37:29.826862+0000","last_peered":"2024-04-25T08:37:29.826862+0000","last_clean":"2024-04-25T08:37:29.826862+0000","last_became_active":"2024-04-25T08:37:28.813933+0000","last_became_peered":"2024-04-25T08:37:28.813933+0000","last_unstale":"2024-04-25T08:37:29.826862+0000","last_undegraded":"2024-04-25T08:37:29.826862+0000","last_fullsized":"2024-04-25T08:37:29.826862+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:02.224856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.808586+0000","last_change":"2024-04-25T08:37:29.799100+0000","last_active":"2024-04-25T08:37:29.808586+0000","last_peered":"2024-04-25T08:37:29.808586+0000","last_clean":"2024-04-25T08:37:29.808586+0000","last_became_active":"2024-04-25T08:37:29.798576+0000","last_became_peered":"2024-04-25T08:37:29.798576+0000","last_unstale":"2024-04-25T08:37:29.808586+0000","last_undegraded":"2024-04-25T08:37:29.808586+0000","last_fullsized":"2024-04-25T08:37:29.808586+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:40.306970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"122'1","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.827152+0000","last_change":"2024-04-25T08:37:28.808855+0000","last_active":"2024-04-25T08:37:29.827152+0000","last_peered":"2024-04-25T08:37:29.827152+0000","last_clean":"2024-04-25T08:37:29.827152+0000","last_became_active":"2024-04-25T08:37:28.808555+0000","last_became_peered":"2024-04-25T08:37:28.808555+0000","last_unstale":"2024-04-25T08:37:29.827152+0000","last_undegraded":"2024-04-25T08:37:29.827152+0000","last_fullsized":"2024-04-25T08:37:29.827152+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:12.703514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.438149+0000","last_change":"2024-04-25T08:37:29.817384+0000","last_active":"2024-04-25T08:37:31.438149+0000","last_peered":"2024-04-25T08:37:31.438149+0000","last_clean":"2024-04-25T08:37:31.438149+0000","last_became_active":"2024-04-25T08:37:29.816712+0000","last_became_peered":"2024-04-25T08:37:29.816712+0000","last_unstale":"2024-04-25T08:37:31.438149+0000","last_undegraded":"2024-04-25T08:37:31.438149+0000","last_fullsized":"2024-04-25T08:37:31.438149+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.033461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.805757+0000","last_change":"2024-04-25T08:37:28.816285+0000","last_active":"2024-04-25T08:37:29.805757+0000","last_peered":"2024-04-25T08:37:29.805757+0000","last_clean":"2024-04-25T08:37:29.805757+0000","last_became_active":"2024-04-25T08:37:28.816043+0000","last_became_peered":"2024-04-25T08:37:28.816043+0000","last_unstale":"2024-04-25T08:37:29.805757+0000","last_undegraded":"2024-04-25T08:37:29.805757+0000","last_fullsized":"2024-04-25T08:37:29.805757+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:11.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.806377+0000","last_change":"2024-04-25T08:37:29.799578+0000","last_active":"2024-04-25T08:37:29.806377+0000","last_peered":"2024-04-25T08:37:29.806377+0000","last_clean":"2024-04-25T08:37:29.806377+0000","last_became_active":"2024-04-25T08:37:29.799302+0000","last_became_peered":"2024-04-25T08:37:29.799302+0000","last_unstale":"2024-04-25T08:37:29.806377+0000","last_undegraded":"2024-04-25T08:37:29.806377+0000","last_fullsized":"2024-04-25T08:37:29.806377+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:34.239796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.827353+0000","last_change":"2024-04-25T08:37:28.811977+0000","last_active":"2024-04-25T08:37:29.827353+0000","last_peered":"2024-04-25T08:37:29.827353+0000","last_clean":"2024-04-25T08:37:29.827353+0000","last_became_active":"2024-04-25T08:37:28.811682+0000","last_became_peered":"2024-04-25T08:37:28.811682+0000","last_unstale":"2024-04-25T08:37:29.827353+0000","last_undegraded":"2024-04-25T08:37:29.827353+0000","last_fullsized":"2024-04-25T08:37:29.827353+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:37.886123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.820442+0000","last_change":"2024-04-25T08:37:29.815322+0000","last_active":"2024-04-25T08:37:29.820442+0000","last_peered":"2024-04-25T08:37:29.820442+0000","last_clean":"2024-04-25T08:37:29.820442+0000","last_became_active":"2024-04-25T08:37:29.815003+0000","last_became_peered":"2024-04-25T08:37:29.815003+0000","last_unstale":"2024-04-25T08:37:29.820442+0000","last_undegraded":"2024-04-25T08:37:29.820442+0000","last_fullsized":"2024-04-25T08:37:29.820442+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:30.169153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.802411+0000","last_change":"2024-04-25T08:37:28.811694+0000","last_active":"2024-04-25T08:37:29.802411+0000","last_peered":"2024-04-25T08:37:29.802411+0000","last_clean":"2024-04-25T08:37:29.802411+0000","last_became_active":"2024-04-25T08:37:28.811423+0000","last_became_peered":"2024-04-25T08:37:28.811423+0000","last_unstale":"2024-04-25T08:37:29.802411+0000","last_undegraded":"2024-04-25T08:37:29.802411+0000","last_fullsized":"2024-04-25T08:37:29.802411+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:29.093482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.816668+0000","last_change":"2024-04-25T08:37:29.813018+0000","last_active":"2024-04-25T08:37:29.816668+0000","last_peered":"2024-04-25T08:37:29.816668+0000","last_clean":"2024-04-25T08:37:29.816668+0000","last_became_active":"2024-04-25T08:37:29.812733+0000","last_became_peered":"2024-04-25T08:37:29.812733+0000","last_unstale":"2024-04-25T08:37:29.816668+0000","last_undegraded":"2024-04-25T08:37:29.816668+0000","last_fullsized":"2024-04-25T08:37:29.816668+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:47.389963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814415+0000","last_change":"2024-04-25T08:37:28.809836+0000","last_active":"2024-04-25T08:37:29.814415+0000","last_peered":"2024-04-25T08:37:29.814415+0000","last_clean":"2024-04-25T08:37:29.814415+0000","last_became_active":"2024-04-25T08:37:28.809525+0000","last_became_peered":"2024-04-25T08:37:28.809525+0000","last_unstale":"2024-04-25T08:37:29.814415+0000","last_undegraded":"2024-04-25T08:37:29.814415+0000","last_fullsized":"2024-04-25T08:37:29.814415+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:32.910454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:31.437764+0000","last_change":"2024-04-25T08:37:29.812775+0000","last_active":"2024-04-25T08:37:31.437764+0000","last_peered":"2024-04-25T08:37:31.437764+0000","last_clean":"2024-04-25T08:37:31.437764+0000","last_became_active":"2024-04-25T08:37:29.808458+0000","last_became_peered":"2024-04-25T08:37:29.808458+0000","last_unstale":"2024-04-25T08:37:31.437764+0000","last_undegraded":"2024-04-25T08:37:31.437764+0000","last_fullsized":"2024-04-25T08:37:31.437764+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.803981+0000","last_change":"2024-04-25T08:37:28.809691+0000","last_active":"2024-04-25T08:37:29.803981+0000","last_peered":"2024-04-25T08:37:29.803981+0000","last_clean":"2024-04-25T08:37:29.803981+0000","last_became_active":"2024-04-25T08:37:28.809249+0000","last_became_peered":"2024-04-25T08:37:28.809249+0000","last_unstale":"2024-04-25T08:37:29.803981+0000","last_undegraded":"2024-04-25T08:37:29.803981+0000","last_fullsized":"2024-04-25T08:37:29.803981+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:29.164631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.814345+0000","last_change":"2024-04-25T08:37:29.812638+0000","last_active":"2024-04-25T08:37:29.814345+0000","last_peered":"2024-04-25T08:37:29.814345+0000","last_clean":"2024-04-25T08:37:29.814345+0000","last_became_active":"2024-04-25T08:37:29.809364+0000","last_became_peered":"2024-04-25T08:37:29.809364+0000","last_unstale":"2024-04-25T08:37:29.814345+0000","last_undegraded":"2024-04-25T08:37:29.814345+0000","last_fullsized":"2024-04-25T08:37:29.814345+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:19.665018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"122'2","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.829973+0000","last_change":"2024-04-25T08:37:28.803503+0000","last_active":"2024-04-25T08:37:29.829973+0000","last_peered":"2024-04-25T08:37:29.829973+0000","last_clean":"2024-04-25T08:37:29.829973+0000","last_became_active":"2024-04-25T08:37:28.802463+0000","last_became_peered":"2024-04-25T08:37:28.802463+0000","last_unstale":"2024-04-25T08:37:29.829973+0000","last_undegraded":"2024-04-25T08:37:29.829973+0000","last_fullsized":"2024-04-25T08:37:29.829973+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:06.271361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813301+0000","last_change":"2024-04-25T08:37:29.811329+0000","last_active":"2024-04-25T08:37:29.813301+0000","last_peered":"2024-04-25T08:37:29.813301+0000","last_clean":"2024-04-25T08:37:29.813301+0000","last_became_active":"2024-04-25T08:37:29.811024+0000","last_became_peered":"2024-04-25T08:37:29.811024+0000","last_unstale":"2024-04-25T08:37:29.813301+0000","last_undegraded":"2024-04-25T08:37:29.813301+0000","last_fullsized":"2024-04-25T08:37:29.813301+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:53.473865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"123'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.810581+0000","last_change":"2024-04-25T08:37:28.809149+0000","last_active":"2024-04-25T08:37:30.810581+0000","last_peered":"2024-04-25T08:37:30.810581+0000","last_clean":"2024-04-25T08:37:30.810581+0000","last_became_active":"2024-04-25T08:37:28.808873+0000","last_became_peered":"2024-04-25T08:37:28.808873+0000","last_unstale":"2024-04-25T08:37:30.810581+0000","last_undegraded":"2024-04-25T08:37:30.810581+0000","last_fullsized":"2024-04-25T08:37:30.810581+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:38.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.815627+0000","last_change":"2024-04-25T08:37:29.813671+0000","last_active":"2024-04-25T08:37:29.815627+0000","last_peered":"2024-04-25T08:37:29.815627+0000","last_clean":"2024-04-25T08:37:29.815627+0000","last_became_active":"2024-04-25T08:37:29.813363+0000","last_became_peered":"2024-04-25T08:37:29.813363+0000","last_unstale":"2024-04-25T08:37:29.815627+0000","last_undegraded":"2024-04-25T08:37:29.815627+0000","last_fullsized":"2024-04-25T08:37:29.815627+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:24.135064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T08:37:30.808970+0000","last_change":"2024-04-25T08:37:28.809947+0000","last_active":"2024-04-25T08:37:30.808970+0000","last_peered":"2024-04-25T08:37:30.808970+0000","last_clean":"2024-04-25T08:37:30.808970+0000","last_became_active":"2024-04-25T08:37:28.809692+0000","last_became_peered":"2024-04-25T08:37:28.809692+0000","last_unstale":"2024-04-25T08:37:30.808970+0000","last_undegraded":"2024-04-25T08:37:30.808970+0000","last_fullsized":"2024-04-25T08:37:30.808970+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:29.093482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819067+0000","last_change":"2024-04-25T08:37:29.816751+0000","last_active":"2024-04-25T08:37:29.819067+0000","last_peered":"2024-04-25T08:37:29.819067+0000","last_clean":"2024-04-25T08:37:29.819067+0000","last_became_active":"2024-04-25T08:37:29.816395+0000","last_became_peered":"2024-04-25T08:37:29.816395+0000","last_unstale":"2024-04-25T08:37:29.819067+0000","last_undegraded":"2024-04-25T08:37:29.819067+0000","last_fullsized":"2024-04-25T08:37:29.819067+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:04.238401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.796328+0000","last_change":"2024-04-25T08:37:28.814614+0000","last_active":"2024-04-25T08:37:29.796328+0000","last_peered":"2024-04-25T08:37:29.796328+0000","last_clean":"2024-04-25T08:37:29.796328+0000","last_became_active":"2024-04-25T08:37:28.813602+0000","last_became_peered":"2024-04-25T08:37:28.813602+0000","last_unstale":"2024-04-25T08:37:29.796328+0000","last_undegraded":"2024-04-25T08:37:29.796328+0000","last_fullsized":"2024-04-25T08:37:29.796328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:23.987856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.808805+0000","last_change":"2024-04-25T08:37:29.804329+0000","last_active":"2024-04-25T08:37:29.808805+0000","last_peered":"2024-04-25T08:37:29.808805+0000","last_clean":"2024-04-25T08:37:29.808805+0000","last_became_active":"2024-04-25T08:37:29.803921+0000","last_became_peered":"2024-04-25T08:37:29.803921+0000","last_unstale":"2024-04-25T08:37:29.808805+0000","last_undegraded":"2024-04-25T08:37:29.808805+0000","last_fullsized":"2024-04-25T08:37:29.808805+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:10.303139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.798114+0000","last_change":"2024-04-25T08:37:28.805295+0000","last_active":"2024-04-25T08:37:29.798114+0000","last_peered":"2024-04-25T08:37:29.798114+0000","last_clean":"2024-04-25T08:37:29.798114+0000","last_became_active":"2024-04-25T08:37:28.804997+0000","last_became_peered":"2024-04-25T08:37:28.804997+0000","last_unstale":"2024-04-25T08:37:29.798114+0000","last_undegraded":"2024-04-25T08:37:29.798114+0000","last_fullsized":"2024-04-25T08:37:29.798114+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:56.626573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.811792+0000","last_change":"2024-04-25T08:37:29.810461+0000","last_active":"2024-04-25T08:37:29.811792+0000","last_peered":"2024-04-25T08:37:29.811792+0000","last_clean":"2024-04-25T08:37:29.811792+0000","last_became_active":"2024-04-25T08:37:29.810119+0000","last_became_peered":"2024-04-25T08:37:29.810119+0000","last_unstale":"2024-04-25T08:37:29.811792+0000","last_undegraded":"2024-04-25T08:37:29.811792+0000","last_fullsized":"2024-04-25T08:37:29.811792+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:57.631095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.797376+0000","last_change":"2024-04-25T08:37:28.807200+0000","last_active":"2024-04-25T08:37:29.797376+0000","last_peered":"2024-04-25T08:37:29.797376+0000","last_clean":"2024-04-25T08:37:29.797376+0000","last_became_active":"2024-04-25T08:37:28.805882+0000","last_became_peered":"2024-04-25T08:37:28.805882+0000","last_unstale":"2024-04-25T08:37:29.797376+0000","last_undegraded":"2024-04-25T08:37:29.797376+0000","last_fullsized":"2024-04-25T08:37:29.797376+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:45.694831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.819577+0000","last_change":"2024-04-25T08:37:29.815381+0000","last_active":"2024-04-25T08:37:29.819577+0000","last_peered":"2024-04-25T08:37:29.819577+0000","last_clean":"2024-04-25T08:37:29.819577+0000","last_became_active":"2024-04-25T08:37:29.814953+0000","last_became_peered":"2024-04-25T08:37:29.814953+0000","last_unstale":"2024-04-25T08:37:29.819577+0000","last_undegraded":"2024-04-25T08:37:29.819577+0000","last_fullsized":"2024-04-25T08:37:29.819577+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:54.030197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.795421+0000","last_change":"2024-04-25T08:37:28.808866+0000","last_active":"2024-04-25T08:37:29.795421+0000","last_peered":"2024-04-25T08:37:29.795421+0000","last_clean":"2024-04-25T08:37:29.795421+0000","last_became_active":"2024-04-25T08:37:28.808571+0000","last_became_peered":"2024-04-25T08:37:28.808571+0000","last_unstale":"2024-04-25T08:37:29.795421+0000","last_undegraded":"2024-04-25T08:37:29.795421+0000","last_fullsized":"2024-04-25T08:37:29.795421+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:52.605914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.812499+0000","last_change":"2024-04-25T08:37:29.809812+0000","last_active":"2024-04-25T08:37:29.812499+0000","last_peered":"2024-04-25T08:37:29.812499+0000","last_clean":"2024-04-25T08:37:29.812499+0000","last_became_active":"2024-04-25T08:37:29.809502+0000","last_became_peered":"2024-04-25T08:37:29.809502+0000","last_unstale":"2024-04-25T08:37:29.812499+0000","last_undegraded":"2024-04-25T08:37:29.812499+0000","last_fullsized":"2024-04-25T08:37:29.812499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:34.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.804126+0000","last_change":"2024-04-25T08:37:28.811212+0000","last_active":"2024-04-25T08:37:29.804126+0000","last_peered":"2024-04-25T08:37:29.804126+0000","last_clean":"2024-04-25T08:37:29.804126+0000","last_became_active":"2024-04-25T08:37:28.810583+0000","last_became_peered":"2024-04-25T08:37:28.810583+0000","last_unstale":"2024-04-25T08:37:29.804126+0000","last_undegraded":"2024-04-25T08:37:29.804126+0000","last_fullsized":"2024-04-25T08:37:29.804126+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:37.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.813041+0000","last_change":"2024-04-25T08:37:29.811744+0000","last_active":"2024-04-25T08:37:29.813041+0000","last_peered":"2024-04-25T08:37:29.813041+0000","last_clean":"2024-04-25T08:37:29.813041+0000","last_became_active":"2024-04-25T08:37:29.811318+0000","last_became_peered":"2024-04-25T08:37:29.811318+0000","last_unstale":"2024-04-25T08:37:29.813041+0000","last_undegraded":"2024-04-25T08:37:29.813041+0000","last_fullsized":"2024-04-25T08:37:29.813041+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:28.786042+0000","last_clean_scrub_stamp":"2024-04-25T08:37:28.786042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:23.039968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"122'8","reported_seq":20,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.830291+0000","last_change":"2024-04-25T08:37:28.813007+0000","last_active":"2024-04-25T08:37:29.830291+0000","last_peered":"2024-04-25T08:37:29.830291+0000","last_clean":"2024-04-25T08:37:29.830291+0000","last_became_active":"2024-04-25T08:37:28.812716+0000","last_became_peered":"2024-04-25T08:37:28.812716+0000","last_unstale":"2024-04-25T08:37:29.830291+0000","last_undegraded":"2024-04-25T08:37:29.830291+0000","last_fullsized":"2024-04-25T08:37:29.830291+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:37:27.781521+0000","last_clean_scrub_stamp":"2024-04-25T08:37:27.781521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:52.469343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":360,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T08:37:29.804270+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:37:29.804270+0000","last_peered":"2024-04-25T08:37:29.804270+0000","last_clean":"2024-04-25T08:37:29.804270+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:37:29.804270+0000","last_undegraded":"2024-04-25T08:37:29.804270+0000","last_fullsized":"2024-04-25T08:37:29.804270+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45740,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326100,"statfs":{"total":96636764160,"available":96589926400,"internally_reserved":0,"allocated":2125824,"data_stored":1297479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46564,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325276,"statfs":{"total":96636764160,"available":96589082624,"internally_reserved":0,"allocated":1531904,"data_stored":713609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63204,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308636,"statfs":{"total":96636764160,"available":96572043264,"internally_reserved":0,"allocated":1523712,"data_stored":706495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50728,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321112,"statfs":{"total":96636764160,"available":96584818688,"internally_reserved":0,"allocated":1527808,"data_stored":707947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54884,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316956,"statfs":{"total":96636764160,"available":96580562944,"internally_reserved":0,"allocated":1531904,"data_stored":707981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54872,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316968,"statfs":{"total":96636764160,"available":96580575232,"internally_reserved":0,"allocated":1515520,"data_stored":706335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50716,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321124,"statfs":{"total":96636764160,"available":96584830976,"internally_reserved":0,"allocated":1519616,"data_stored":706357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55440,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316400,"statfs":{"total":96636764160,"available":96579993600,"internally_reserved":0,"allocated":2097152,"data_stored":1289536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:37:34.149 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:37:34.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:37:34.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.300+0000 7fea14838640 1 Processor -- start 2024-04-25T08:37:34.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.300+0000 7fea14838640 1 -- start start 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea14838640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fea1015b7d0 0x7fea100cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea14838640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea100b5620 0x7fea100b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea14838640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea1006c2d0 0x7fea1006c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea14838640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea100b5fc0 con 0x7fea1006c2d0 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea14838640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1006cca0 con 0x7fea1015b7d0 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea14838640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1006cf50 con 0x7fea100b5620 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea1006c2d0 0x7fea1006c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fea1006c2d0 0x7fea1006c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53448/0 (socket says 172.21.15.60:53448) 2024-04-25T08:37:34.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 -- 172.21.15.60:0/3265598668 learned_addr learned my addr 172.21.15.60:0/3265598668 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea07fff640 1 --2- 172.21.15.60:0/3265598668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea100b5620 0x7fea100b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0e59c640 1 --2- 172.21.15.60:0/3265598668 >> v2:172.21.15.100:3300/0 conn(0x7fea1015b7d0 0x7fea100cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 -- 172.21.15.60:0/3265598668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea100b5620 msgr2=0x7fea100b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 --2- 172.21.15.60:0/3265598668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea100b5620 0x7fea100b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 -- 172.21.15.60:0/3265598668 >> v2:172.21.15.100:3300/0 conn(0x7fea1015b7d0 msgr2=0x7fea100cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 --2- 172.21.15.60:0/3265598668 >> v2:172.21.15.100:3300/0 conn(0x7fea1015b7d0 0x7fea100cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0ed9d640 1 -- 172.21.15.60:0/3265598668 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea100ce530 con 0x7fea1006c2d0 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.301+0000 7fea0e59c640 1 --2- 172.21.15.60:0/3265598668 >> v2:172.21.15.100:3300/0 conn(0x7fea1015b7d0 0x7fea100cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.302+0000 7fea0ed9d640 1 --2- 172.21.15.60:0/3265598668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea1006c2d0 0x7fea1006c760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fea1008c0d0 tx=0x7fea0802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0613f213f94a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.302+0000 7fea0dd9b640 1 -- 172.21.15.60:0/3265598668 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08036070 con 0x7fea1006c2d0 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.302+0000 7fea0dd9b640 1 -- 172.21.15.60:0/3265598668 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea08031600 con 0x7fea1006c2d0 2024-04-25T08:37:34.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.302+0000 7fea0dd9b640 1 -- 172.21.15.60:0/3265598668 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea080319d0 con 0x7fea1006c2d0 2024-04-25T08:37:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.302+0000 7fea14838640 1 -- 172.21.15.60:0/3265598668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea1006c2d0 msgr2=0x7fea1006c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.302+0000 7fea14838640 1 --2- 172.21.15.60:0/3265598668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea1006c2d0 0x7fea1006c760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fea1008c0d0 tx=0x7fea0802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 -- 172.21.15.60:0/3265598668 shutdown_connections 2024-04-25T08:37:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 --2- 172.21.15.60:0/3265598668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea1006c2d0 0x7fea1006c760 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 --2- 172.21.15.60:0/3265598668 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea100b5620 0x7fea100b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 --2- 172.21.15.60:0/3265598668 >> v2:172.21.15.100:3300/0 conn(0x7fea1015b7d0 0x7fea100cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 -- 172.21.15.60:0/3265598668 >> 172.21.15.60:0/3265598668 conn(0x7fea100d0800 msgr2=0x7fea100d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 -- 172.21.15.60:0/3265598668 shutdown_connections 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.303+0000 7fea14838640 1 -- 172.21.15.60:0/3265598668 wait complete. 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.304+0000 7fea14838640 1 Processor -- start 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.304+0000 7fea14838640 1 -- start start 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea1006c2d0 0x7fea101593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 0x7fea100c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea1015b7d0 0x7fea101f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1006d230 con 0x7fea100b5620 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea100ceaf0 con 0x7fea1006c2d0 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea100cf550 con 0x7fea1015b7d0 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 0x7fea100c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 0x7fea100c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53460/0 (socket says 172.21.15.60:53460) 2024-04-25T08:37:34.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 -- 172.21.15.60:0/868816243 learned_addr learned my addr 172.21.15.60:0/868816243 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea0ed9d640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea1015b7d0 0x7fea101f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea0e59c640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea1006c2d0 0x7fea101593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 -- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea1015b7d0 msgr2=0x7fea101f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea1015b7d0 0x7fea101f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 -- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea1006c2d0 msgr2=0x7fea101593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea1006c2d0 0x7fea101593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 -- 172.21.15.60:0/868816243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea101f4180 con 0x7fea100b5620 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea0e59c640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea1006c2d0 0x7fea101593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea07fff640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 0x7fea100c9ef0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d800 tx=0x7fe9f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8018020 con 0x7fea100b5620 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f80096a0 con 0x7fea100b5620 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea101f4460 con 0x7fea100b5620 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8009a70 con 0x7fea100b5620 2024-04-25T08:37:34.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.305+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea100ca430 con 0x7fea100b5620 2024-04-25T08:37:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.307+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea10073a00 con 0x7fea100b5620 2024-04-25T08:37:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.307+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9f801b020 con 0x7fea100b5620 2024-04-25T08:37:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.307+0000 7fea06ffd640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9f006b4d0 0x7fe9f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.308+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9f808a4d0 con 0x7fea100b5620 2024-04-25T08:37:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.308+0000 7fea0e59c640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9f006b4d0 0x7fe9f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.308+0000 7fea0e59c640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9f006b4d0 0x7fe9f006d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fea0000a990 tx=0x7fea00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:34.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.312+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9f8022030 con 0x7fea100b5620 2024-04-25T08:37:34.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.565+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fea100b8f90 con 0x7fea100b5620 2024-04-25T08:37:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.567+0000 7fea06ffd640 1 -- 172.21.15.60:0/868816243 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe9f8059250 con 0x7fea100b5620 2024-04-25T08:37:34.561 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:37:34.561 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:37:34.561 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:37:34.561 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:37:34.561 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:37:34.562 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:37:34.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9f006b4d0 msgr2=0x7fe9f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9f006b4d0 0x7fe9f006d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fea0000a990 tx=0x7fea00008040 comp rx=0 tx=0).stop 2024-04-25T08:37:34.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 msgr2=0x7fea100c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 0x7fea100c9ef0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d800 tx=0x7fe9f800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.564 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 shutdown_connections 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fea1015b7d0 0x7fea101f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fea100b5620 0x7fea100c9ef0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9f006b4d0 0x7fe9f006d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 --2- 172.21.15.60:0/868816243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea1006c2d0 0x7fea101593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.569+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 >> 172.21.15.60:0/868816243 conn(0x7fea100d0800 msgr2=0x7fea100d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.570+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 shutdown_connections 2024-04-25T08:37:34.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.570+0000 7fea14838640 1 -- 172.21.15.60:0/868816243 wait complete. 2024-04-25T08:37:34.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:34.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.682+0000 7fd778a1b640 1 Processor -- start 2024-04-25T08:37:34.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.682+0000 7fd778a1b640 1 -- start start 2024-04-25T08:37:34.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd778a1b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd774162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd778a1b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7740b6cb0 0x7fd7740b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd778a1b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7740b5a60 0x7fd77406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd778a1b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd774162690 con 0x7fd7740b5a60 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd778a1b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7740ceee0 con 0x7fd7740b6cb0 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd778a1b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740d1d10 con 0x7fd77415db70 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd772d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7740b5a60 0x7fd77406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd772d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7740b5a60 0x7fd77406c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53464/0 (socket says 172.21.15.60:53464) 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.683+0000 7fd772d9d640 1 -- 172.21.15.60:0/2139825541 learned_addr learned my addr 172.21.15.60:0/2139825541 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd77259c640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd774162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd771d9b640 1 --2- 172.21.15.60:0/2139825541 >> v2:172.21.15.100:3300/0 conn(0x7fd7740b6cb0 0x7fd7740b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd772d9d640 1 -- 172.21.15.60:0/2139825541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 msgr2=0x7fd774162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd772d9d640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd774162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd772d9d640 1 -- 172.21.15.60:0/2139825541 >> v2:172.21.15.100:3300/0 conn(0x7fd7740b6cb0 msgr2=0x7fd7740b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd772d9d640 1 --2- 172.21.15.60:0/2139825541 >> v2:172.21.15.100:3300/0 conn(0x7fd7740b6cb0 0x7fd7740b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd772d9d640 1 -- 172.21.15.60:0/2139825541 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740d1ff0 con 0x7fd7740b5a60 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd77259c640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd774162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.684+0000 7fd772d9d640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77406c2d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd7740d4b60 tx=0x7fd76402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee95e8e9adb535b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd77159a640 1 -- 172.21.15.60:0/2139825541 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764036070 con 0x7fd7740b5a60 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd77159a640 1 -- 172.21.15.60:0/2139825541 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764031600 con 0x7fd7740b5a60 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd77159a640 1 -- 172.21.15.60:0/2139825541 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640319d0 con 0x7fd7740b5a60 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 -- 172.21.15.60:0/2139825541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 msgr2=0x7fd77406c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77406c2d0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd7740d4b60 tx=0x7fd76402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 -- 172.21.15.60:0/2139825541 shutdown_connections 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77406c2d0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 --2- 172.21.15.60:0/2139825541 >> v2:172.21.15.100:3300/0 conn(0x7fd7740b6cb0 0x7fd7740b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 --2- 172.21.15.60:0/2139825541 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd774162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 -- 172.21.15.60:0/2139825541 >> 172.21.15.60:0/2139825541 conn(0x7fd7740d0520 msgr2=0x7fd7740d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:34.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 -- 172.21.15.60:0/2139825541 shutdown_connections 2024-04-25T08:37:34.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.685+0000 7fd778a1b640 1 -- 172.21.15.60:0/2139825541 wait complete. 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.686+0000 7fd778a1b640 1 Processor -- start 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 -- start start 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7740b6cb0 0x7fd774160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd77415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740ce530 con 0x7fd7740b5a60 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77415df50 con 0x7fd7740b6cb0 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd778a1b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740cf1c0 con 0x7fd77415db70 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd77259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd77259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77415a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53474/0 (socket says 172.21.15.60:53474) 2024-04-25T08:37:34.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.687+0000 7fd77259c640 1 -- 172.21.15.60:0/1653544748 learned_addr learned my addr 172.21.15.60:0/1653544748 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd772d9d640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd77415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd771d9b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7740b6cb0 0x7fd774160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd77259c640 1 -- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 msgr2=0x7fd77415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd77259c640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd77415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd77259c640 1 -- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7740b6cb0 msgr2=0x7fd774160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd77259c640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7740b6cb0 0x7fd774160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd77259c640 1 -- 172.21.15.60:0/1653544748 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774160fa0 con 0x7fd7740b5a60 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd771d9b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7740b6cb0 0x7fd774160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd77259c640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77415a580 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d7d0 tx=0x7fd76c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c004030 con 0x7fd7740b5a60 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c0041d0 con 0x7fd7740b5a60 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77415e320 con 0x7fd7740b5a60 2024-04-25T08:37:34.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c009640 con 0x7fd7740b5a60 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.688+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77415e8e0 con 0x7fd7740b5a60 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.689+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774073990 con 0x7fd7740b5a60 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.690+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd76c0097e0 con 0x7fd7740b5a60 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.690+0000 7fd76affd640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd75406b4d0 0x7fd75406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.690+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd76c08a4a0 con 0x7fd7740b5a60 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.690+0000 7fd771d9b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd75406b4d0 0x7fd75406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:34.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.691+0000 7fd771d9b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd75406b4d0 0x7fd75406d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd774162200 tx=0x7fd75c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:34.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.694+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd76c059220 con 0x7fd7740b5a60 2024-04-25T08:37:34.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.918+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7740d1820 con 0x7fd7740b5a60 2024-04-25T08:37:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.919+0000 7fd76affd640 1 -- 172.21.15.60:0/1653544748 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd76c058b20 con 0x7fd7740b5a60 2024-04-25T08:37:34.913 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:34.913 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:34.913 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:34.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd75406b4d0 msgr2=0x7fd75406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd75406b4d0 0x7fd75406d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd774162200 tx=0x7fd75c008040 comp rx=0 tx=0).stop 2024-04-25T08:37:34.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 msgr2=0x7fd77415a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:34.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77415a580 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d7d0 tx=0x7fd76c00dc90 comp rx=0 tx=0).stop 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 shutdown_connections 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd77415db70 0x7fd77415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd75406b4d0 0x7fd75406d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7740b6cb0 0x7fd774160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 --2- 172.21.15.60:0/1653544748 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7740b5a60 0x7fd77415a580 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 >> 172.21.15.60:0/1653544748 conn(0x7fd7740d0520 msgr2=0x7fd77406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 shutdown_connections 2024-04-25T08:37:34.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:34.921+0000 7fd778a1b640 1 -- 172.21.15.60:0/1653544748 wait complete. 2024-04-25T08:37:35.328 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:37:29.166595+0000', 'modified': '2024-04-25T08:37:32.172373+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8805}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8805': {'gid': 8805, 'name': 'd', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6837/2648832765', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6835', 'nonce': 2648832765}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 2648832765}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:37:35.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:37:35.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:37:35.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.438+0000 7f58a057e640 1 Processor -- start 2024-04-25T08:37:35.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.438+0000 7f58a057e640 1 -- start start 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.439+0000 7f58a057e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 0x7f5898168670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.439+0000 7f58a057e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58980d2420 con 0x7f58980d34c0 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.439+0000 7f58a057e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58980d2700 con 0x7f589815dac0 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.439+0000 7f58a057e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58980d3030 con 0x7f58980d0ab0 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.439+0000 7f589db19640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f58980d34c0 0x7f58980d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44196/0 (socket says 172.21.15.60:44196) 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.439+0000 7f589db19640 1 -- 172.21.15.60:0/3352725626 learned_addr learned my addr 172.21.15.60:0/3352725626 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821255698 0 0) 0x7f58980d2420 con 0x7f58980d34c0 2024-04-25T08:37:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5884003520 con 0x7f58980d34c0 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589eb1b640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 0x7f5898168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 525896012 0 0) 0x7f5884003520 con 0x7f58980d34c0 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 msgr2=0x7f5898168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 0x7f5898168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 >> v1:172.21.15.100:6789/0 conn(0x7f589815dac0 legacy=0x7f58980d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980d29e0 con 0x7f58980d34c0 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f58880017f0 con 0x7f58980d34c0 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 >> v1:172.21.15.60:6789/0 conn(0x7f58980d34c0 legacy=0x7f58980d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58840039e0 0x7f5884003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5884004800 0x7f5884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 0x7f58840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980d3030 con 0x7f58840039e0 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588400bb00 con 0x7f5884004800 2024-04-25T08:37:35.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588400bd00 con 0x7f58980d0ab0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589e31a640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58840039e0 0x7f5884003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589eb1b640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 0x7f58840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589db19640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5884004800 0x7f5884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589e31a640 1 -- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 msgr2=0x7f58840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.440+0000 7f589e31a640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980d0ab0 0x7f58840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589e31a640 1 -- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5884004800 msgr2=0x7f5884004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589e31a640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5884004800 0x7f5884004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589e31a640 1 -- 172.21.15.60:0/3352725626 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884003520 con 0x7f58840039e0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589db19640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5884004800 0x7f5884004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589e31a640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58840039e0 0x7f5884003e20 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5894001720 tx=0x7f5884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b34a3ba6d249b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894001370 con 0x7f58840039e0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58940052d0 con 0x7f58840039e0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.441+0000 7f589d318640 1 -- 172.21.15.60:0/3352725626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589402ab90 con 0x7f58840039e0 2024-04-25T08:37:35.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f58a057e640 1 -- 172.21.15.60:0/3352725626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58840039e0 msgr2=0x7f5884003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f58a057e640 1 --2- 172.21.15.60:0/3352725626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58840039e0 0x7f5884003e20 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5894001720 tx=0x7f5884005830 comp rx=0 tx=0).stop 2024-04-25T08:37:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f589eb1b640 1 -- 172.21.15.60:0/3352725626 reap_dead start 2024-04-25T08:37:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f58a057e640 1 -- 172.21.15.60:0/3352725626 shutdown_connections 2024-04-25T08:37:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f58a057e640 1 -- 172.21.15.60:0/3352725626 >> 172.21.15.60:0/3352725626 conn(0x7f5898061c30 msgr2=0x7f589806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f58a057e640 1 -- 172.21.15.60:0/3352725626 shutdown_connections 2024-04-25T08:37:35.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.442+0000 7f58a057e640 1 -- 172.21.15.60:0/3352725626 wait complete. 2024-04-25T08:37:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.443+0000 7f58a057e640 1 Processor -- start 2024-04-25T08:37:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.443+0000 7f58a057e640 1 -- start start 2024-04-25T08:37:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f58a057e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58840039e0 0x7f58981606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f58a057e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 0x7f5898151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f58a057e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980c8650 0x7f58980c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f58a057e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589815dea0 con 0x7f58981517e0 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f58a057e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980d2100 con 0x7f58840039e0 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f58a057e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58980b97d0 con 0x7f58980c8650 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 0x7f5898151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 0x7f5898151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53488/0 (socket says 172.21.15.60:53488) 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 -- 172.21.15.60:0/3288215457 learned_addr learned my addr 172.21.15.60:0/3288215457 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 -- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980c8650 msgr2=0x7f58980c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589eb1b640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980c8650 0x7f58980c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589e31a640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58840039e0 0x7f58981606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980c8650 0x7f58980c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 -- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58840039e0 msgr2=0x7f58981606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58840039e0 0x7f58981606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.444+0000 7f589db19640 1 -- 172.21.15.60:0/3288215457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589815e0c0 con 0x7f58981517e0 2024-04-25T08:37:35.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.445+0000 7f589db19640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 0x7f5898151c60 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f588800e780 tx=0x7f588800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.445+0000 7f588effd640 1 -- 172.21.15.60:0/3288215457 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58880070d0 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.445+0000 7f588effd640 1 -- 172.21.15.60:0/3288215457 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5888018070 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.445+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58981521a0 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.445+0000 7f588effd640 1 -- 172.21.15.60:0/3288215457 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58880074d0 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.445+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58980c95f0 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.446+0000 7f588effd640 1 -- 172.21.15.60:0/3288215457 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58880057a0 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.447+0000 7f588effd640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f587c06b4d0 0x7f587c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.447+0000 7f588effd640 1 -- 172.21.15.60:0/3288215457 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f588808a280 con 0x7f58981517e0 2024-04-25T08:37:35.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.447+0000 7f589e31a640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f587c06b4d0 0x7f587c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.448+0000 7f589e31a640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f587c06b4d0 0x7f587c06d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5894032000 tx=0x7f589402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.448+0000 7f58a3280740 1 Processor -- start 2024-04-25T08:37:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.448+0000 7f58a3280740 1 -- start start 2024-04-25T08:37:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b23610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b40ad0 0x55ea40b41020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f58a3280740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 0x55ea40b3f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f58a3280740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40a5ba10 con 0x55ea40af8f10 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40a04b30 con 0x55ea40b40ad0 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea40a57c50 con 0x55ea40b23230 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 0x55ea40b3f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 0x55ea40b3f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53490/0 (socket says 172.21.15.60:53490) 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 -- 172.21.15.60:0/2974180602 learned_addr learned my addr 172.21.15.60:0/2974180602 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589e31a640 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b23610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589db19640 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b40ad0 0x55ea40b41020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 -- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 msgr2=0x55ea40b23610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b23610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 -- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b40ad0 msgr2=0x55ea40b41020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b40ad0 0x55ea40b41020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 -- 172.21.15.60:0/2974180602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea40a19e10 con 0x55ea40af8f10 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 0x55ea40b3f190 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f589806d010 tx=0x7f5890009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de7b9c3f90bc174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.449+0000 7f587bfff640 1 -- 172.21.15.60:0/2974180602 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003b90 con 0x55ea40af8f10 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f587bfff640 1 -- 172.21.15.60:0/2974180602 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589001d070 con 0x55ea40af8f10 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f587bfff640 1 -- 172.21.15.60:0/2974180602 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890004030 con 0x55ea40af8f10 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 -- 172.21.15.60:0/2974180602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 msgr2=0x55ea40b3f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 0x55ea40b3f190 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f589806d010 tx=0x7f5890009750 comp rx=0 tx=0).stop 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 -- 172.21.15.60:0/2974180602 shutdown_connections 2024-04-25T08:37:35.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40af8f10 0x55ea40b3f190 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b40ad0 0x55ea40b41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 --2- 172.21.15.60:0/2974180602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b23610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 -- 172.21.15.60:0/2974180602 >> 172.21.15.60:0/2974180602 conn(0x55ea40af5840 msgr2=0x55ea40b3e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 -- 172.21.15.60:0/2974180602 shutdown_connections 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.450+0000 7f58a3280740 1 -- 172.21.15.60:0/2974180602 wait complete. 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.451+0000 7f58a3280740 1 -- start start 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.451+0000 7f58a3280740 1 Processor -- start 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b23230 0x55ea40b270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 0x55ea40b35cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40af9cb0 con 0x55ea40b4e790 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40a180c0 con 0x55ea40b23230 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea40af3770 con 0x55ea40b40ad0 2024-04-25T08:37:35.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f58a3280740 10 client.0 _ll_register_callbacks cb 0x55ea40a0bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 0x55ea40b35cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 0x55ea40b35cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53502/0 (socket says 172.21.15.60:53502) 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 -- 172.21.15.60:0/2497591586 learned_addr learned my addr 172.21.15.60:0/2497591586 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589db19640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589e31a640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b23230 0x55ea40b270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 -- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 msgr2=0x55ea40b357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 -- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b23230 msgr2=0x55ea40b270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b23230 0x55ea40b270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 -- 172.21.15.60:0/2497591586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea407702d0 con 0x55ea40b4e790 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589db19640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.452+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 0x55ea40b35cf0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5890019000 tx=0x7f5890004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6566e533f7098d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58797fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58797fa640 1 -- 172.21.15.60:0/2497591586 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003990 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58797fa640 1 -- 172.21.15.60:0/2497591586 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5890003b30 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f589001d070 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58797fa640 1 -- 172.21.15.60:0/2497591586 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5890031050 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 10 client.8937 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ea4083a6b0 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58797fa640 1 -- 172.21.15.60:0/2497591586 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589001d070 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58797fa640 1 -- 172.21.15.60:0/2497591586 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5890034020 con 0x55ea40b4e790 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 10 client.8937 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 10 client.8937 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 20 client.8937 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40af2670 0x55ea40b255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 4 client.8937 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea40aba920 con 0x55ea40af2670 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.453+0000 7f589e31a640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40af2670 0x55ea40b255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.454+0000 7f589e31a640 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40af2670 0x55ea40b255c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57022be6c319459f server_cookie=16d0f01b8343d381 in_seq=0 out_seq=0 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.454+0000 7f58797fa640 10 client.8937 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58797fa640 1 -- 172.21.15.60:0/2497591586 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea40aba920 con 0x55ea40af2670 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58797fa640 10 client.8937 handle_command_reply: tid=0 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 client.8937 shutdown 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 msgr2=0x55ea40b35cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 0x55ea40b35cf0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5890019000 tx=0x7f5890004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 shutdown_connections 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40af2670 0x55ea40b255c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b4e790 0x55ea40b35cf0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58797fa640 0 client.8937 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 --2- 172.21.15.60:0/2497591586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40b23230 0x55ea40b270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 >> 172.21.15.60:0/2497591586 conn(0x55ea40af8f10 msgr2=0x55ea40b3d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 shutdown_connections 2024-04-25T08:37:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 1 -- 172.21.15.60:0/2497591586 wait complete. 2024-04-25T08:37:35.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.455+0000 7f58a3280740 20 client.8937 trim_cache size 0 max 16384 2024-04-25T08:37:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.471+0000 7f58a3280740 1 Processor -- start 2024-04-25T08:37:35.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.471+0000 7f58a3280740 1 -- start start 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b3f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f58a3280740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 0x55ea40af92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f58a3280740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981ffc20 con 0x55ea40b40ad0 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589815f8c0 con 0x55ea40af8f10 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58981ff620 con 0x55ea40b23230 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f589eb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 0x55ea40af92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f589eb1b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 0x55ea40af92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53504/0 (socket says 172.21.15.60:53504) 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.472+0000 7f589eb1b640 1 -- 172.21.15.60:0/2988360474 learned_addr learned my addr 172.21.15.60:0/2988360474 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f589eb1b640 1 -- 172.21.15.60:0/2988360474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 msgr2=0x55ea40b3f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b3f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f589eb1b640 1 -- 172.21.15.60:0/2988360474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 msgr2=0x55ea40b26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f589eb1b640 1 -- 172.21.15.60:0/2988360474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea40b23840 con 0x55ea40b40ad0 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f589eb1b640 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 0x55ea40af92f0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f589001a040 tx=0x7f5890022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830e6bd8435d0669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f587b7fe640 1 -- 172.21.15.60:0/2988360474 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003990 con 0x55ea40b40ad0 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f587b7fe640 1 -- 172.21.15.60:0/2988360474 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890039070 con 0x55ea40b40ad0 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f587b7fe640 1 -- 172.21.15.60:0/2988360474 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890003d90 con 0x55ea40b40ad0 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 -- 172.21.15.60:0/2988360474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 msgr2=0x55ea40af92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 0x55ea40af92f0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f589001a040 tx=0x7f5890022000 comp rx=0 tx=0).stop 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 -- 172.21.15.60:0/2988360474 shutdown_connections 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b40ad0 0x55ea40af92f0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b23230 0x55ea40b3f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 --2- 172.21.15.60:0/2988360474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 -- 172.21.15.60:0/2988360474 >> 172.21.15.60:0/2988360474 conn(0x55ea40b406f0 msgr2=0x55ea40b22630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.473+0000 7f58a3280740 1 -- 172.21.15.60:0/2988360474 shutdown_connections 2024-04-25T08:37:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.474+0000 7f58a3280740 1 -- 172.21.15.60:0/2988360474 wait complete. 2024-04-25T08:37:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.474+0000 7f58a3280740 1 -- start start 2024-04-25T08:37:35.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.474+0000 7f58a3280740 1 Processor -- start 2024-04-25T08:37:35.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 0x55ea40ba6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40af24f0 con 0x55ea40b23230 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40b35290 con 0x55ea40af8f10 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea40af2af0 con 0x55ea40b40ad0 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58a3280740 10 client.0 _ll_register_callbacks cb 0x55ea40af3c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 0x55ea40ba6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 0x55ea40ba6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53506/0 (socket says 172.21.15.60:53506) 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 -- 172.21.15.60:0/1127120221 learned_addr learned my addr 172.21.15.60:0/1127120221 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589eb1b640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589e31a640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 -- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 msgr2=0x55ea40b385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 -- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 msgr2=0x55ea40b339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 -- 172.21.15.60:0/1127120221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea40b1e7b0 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f589db19640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 0x55ea40ba6a60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5888058560 tx=0x7f5888059090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a372f44f92420254 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.475+0000 7f58797fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58797fa640 1 -- 172.21.15.60:0/1127120221 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888085020 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58797fa640 1 -- 172.21.15.60:0/1127120221 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5888059b90 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5890039070 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58797fa640 1 -- 172.21.15.60:0/1127120221 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5888077050 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 10 client.8943 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55ea40b1f0b0 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58797fa640 1 -- 172.21.15.60:0/1127120221 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890039070 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58797fa640 1 -- 172.21.15.60:0/1127120221 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58880771f0 con 0x55ea40b23230 2024-04-25T08:37:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 10 client.8943 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 10 client.8943 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 20 client.8943 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40b4e790 0x55ea40af6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 4 client.8943 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ea40a68f50 con 0x55ea40b4e790 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.476+0000 7f589e31a640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40b4e790 0x55ea40af6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58797fa640 10 client.8943 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f589e31a640 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40b4e790 0x55ea40af6cb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b671ce32f96c7c13 server_cookie=2dafc956a8d828aa in_seq=0 out_seq=0 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58797fa640 1 -- 172.21.15.60:0/1127120221 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55ea40a68f50 con 0x55ea40b4e790 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58797fa640 10 client.8943 handle_command_reply: tid=0 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 client.8943 shutdown 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 msgr2=0x55ea40ba6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 0x55ea40ba6a60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5888058560 tx=0x7f5888059090 comp rx=0 tx=0).stop 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 shutdown_connections 2024-04-25T08:37:35.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55ea40b4e790 0x55ea40af6cb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55ea40b40ad0 0x55ea40b385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ea40b23230 0x55ea40ba6a60 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 --2- 172.21.15.60:0/1127120221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ea40af8f10 0x55ea40b339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58797fa640 0 client.8943 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 >> 172.21.15.60:0/1127120221 conn(0x55ea40b406f0 msgr2=0x55ea40af92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 shutdown_connections 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 1 -- 172.21.15.60:0/1127120221 wait complete. 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.477+0000 7f58a3280740 20 client.8943 trim_cache size 0 max 16384 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:35.477 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 8805, 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 5.6762573349999998, 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 71, 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 123, 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 123, 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 10.667825176999999 2024-04-25T08:37:35.478 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f587c06b4d0 msgr2=0x7f587c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f587c06b4d0 0x7f587c06d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5894032000 tx=0x7f589402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 msgr2=0x7f5898151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 0x7f5898151c60 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f588800e780 tx=0x7f588800ec40 comp rx=0 tx=0).stop 2024-04-25T08:37:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 shutdown_connections 2024-04-25T08:37:35.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58980c8650 0x7f58980c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58981517e0 0x7f5898151c60 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f587c06b4d0 0x7f587c06d990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 --2- 172.21.15.60:0/3288215457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58840039e0 0x7f58981606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 >> 172.21.15.60:0/3288215457 conn(0x7f5898061c30 msgr2=0x7f58980b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 shutdown_connections 2024-04-25T08:37:35.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.478+0000 7f58a057e640 1 -- 172.21.15.60:0/3288215457 wait complete. 2024-04-25T08:37:35.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:37:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.590+0000 7f4b37d5c640 1 Processor -- start 2024-04-25T08:37:35.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.590+0000 7f4b37d5c640 1 -- start start 2024-04-25T08:37:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.590+0000 7f4b37d5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3015dd30 0x7f4b300b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b37d5c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b300b9770 0x7f4b300d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b37d5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b300d1df0 con 0x7f4b300d2e90 2024-04-25T08:37:35.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b37d5c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300b7a50 con 0x7f4b300b9770 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b37d5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d20d0 con 0x7f4b3015dd30 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b362f9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4b300d2e90 0x7f4b300b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44208/0 (socket says 172.21.15.60:44208) 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b362f9640 1 -- 172.21.15.60:0/3227805869 learned_addr learned my addr 172.21.15.60:0/3227805869 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b35af8640 1 --2- 172.21.15.60:0/3227805869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3015dd30 0x7f4b300b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b352f7640 1 --2- 172.21.15.60:0/3227805869 >> v2:172.21.15.100:3300/0 conn(0x7f4b300b9770 0x7f4b300d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b34af6640 1 -- 172.21.15.60:0/3227805869 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98879783 0 0) 0x7f4b300d1df0 con 0x7f4b300d2e90 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b34af6640 1 -- 172.21.15.60:0/3227805869 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b1c003520 con 0x7f4b300d2e90 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b35af8640 1 -- 172.21.15.60:0/3227805869 >> v2:172.21.15.100:3300/0 conn(0x7f4b300b9770 msgr2=0x7f4b300d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b35af8640 1 --2- 172.21.15.60:0/3227805869 >> v2:172.21.15.100:3300/0 conn(0x7f4b300b9770 0x7f4b300d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b35af8640 1 -- 172.21.15.60:0/3227805869 >> v1:172.21.15.60:6789/0 conn(0x7f4b300d2e90 legacy=0x7f4b300b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.591+0000 7f4b35af8640 1 -- 172.21.15.60:0/3227805869 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300d23b0 con 0x7f4b3015dd30 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.592+0000 7f4b35af8640 1 --2- 172.21.15.60:0/3227805869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3015dd30 0x7f4b300b9230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009d90 tx=0x7f4b2802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d581e7bf31399c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.592+0000 7f4b34af6640 1 -- 172.21.15.60:0/3227805869 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28031600 con 0x7f4b3015dd30 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.592+0000 7f4b34af6640 1 -- 172.21.15.60:0/3227805869 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b280317a0 con 0x7f4b3015dd30 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.592+0000 7f4b34af6640 1 -- 172.21.15.60:0/3227805869 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28031b50 con 0x7f4b3015dd30 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 -- 172.21.15.60:0/3227805869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3015dd30 msgr2=0x7f4b300b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/3227805869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3015dd30 0x7f4b300b9230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009d90 tx=0x7f4b2802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 -- 172.21.15.60:0/3227805869 shutdown_connections 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/3227805869 >> v2:172.21.15.100:3300/0 conn(0x7f4b300b9770 0x7f4b300d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/3227805869 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b3015dd30 0x7f4b300b9230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 -- 172.21.15.60:0/3227805869 >> 172.21.15.60:0/3227805869 conn(0x7f4b30061c30 msgr2=0x7f4b3006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 -- 172.21.15.60:0/3227805869 shutdown_connections 2024-04-25T08:37:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.593+0000 7f4b37d5c640 1 -- 172.21.15.60:0/3227805869 wait complete. 2024-04-25T08:37:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.594+0000 7f4b37d5c640 1 Processor -- start 2024-04-25T08:37:35.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.594+0000 7f4b37d5c640 1 -- start start 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b37d5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b300b9770 0x7f4b300b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b37d5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b300d2e90 0x7f4b30160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b37d5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 0x7f4b300caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b37d5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d1a10 con 0x7f4b3015dd30 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b37d5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d2970 con 0x7f4b300d2e90 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b37d5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300d1b90 con 0x7f4b300b9770 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 0x7f4b300caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 0x7f4b300caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:53520/0 (socket says 172.21.15.60:53520) 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 -- 172.21.15.60:0/1171501 learned_addr learned my addr 172.21.15.60:0/1171501 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b35af8640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b300b9770 0x7f4b300b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b352f7640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b300d2e90 0x7f4b30160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 -- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b300b9770 msgr2=0x7f4b300b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b300b9770 0x7f4b300b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 -- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b300d2e90 msgr2=0x7f4b30160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b300d2e90 0x7f4b30160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 -- 172.21.15.60:0/1171501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3014c940 con 0x7f4b3015dd30 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.595+0000 7f4b362f9640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 0x7f4b300caa80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c430 tx=0x7f4b2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.596+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c017070 con 0x7f4b3015dd30 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.596+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c005ce0 con 0x7f4b3015dd30 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.596+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3014cc20 con 0x7f4b3015dd30 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.596+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c007310 con 0x7f4b3015dd30 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.596+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3014d1e0 con 0x7f4b3015dd30 2024-04-25T08:37:35.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.597+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b300c8090 con 0x7f4b3015dd30 2024-04-25T08:37:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.597+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b2c01a070 con 0x7f4b3015dd30 2024-04-25T08:37:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.598+0000 7f4b267fc640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b1406b4d0 0x7f4b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.598+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b2c0897f0 con 0x7f4b3015dd30 2024-04-25T08:37:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.598+0000 7f4b35af8640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b1406b4d0 0x7f4b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:35.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.598+0000 7f4b35af8640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b1406b4d0 0x7f4b1406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4b28034000 tx=0x7f4b2802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.602+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b300c8090 con 0x7f4b3015dd30 2024-04-25T08:37:35.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.819+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b300b83a0 con 0x7f4b3015dd30 2024-04-25T08:37:35.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.819+0000 7f4b267fc640 1 -- 172.21.15.60:0/1171501 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4b2c057df0 con 0x7f4b3015dd30 2024-04-25T08:37:35.814 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b1406b4d0 msgr2=0x7f4b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b1406b4d0 0x7f4b1406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4b28034000 tx=0x7f4b2802f000 comp rx=0 tx=0).stop 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 msgr2=0x7f4b300caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 0x7f4b300caa80 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c430 tx=0x7f4b2c00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 shutdown_connections 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b3015dd30 0x7f4b300caa80 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4b1406b4d0 0x7f4b1406d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b300d2e90 0x7f4b30160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 --2- 172.21.15.60:0/1171501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4b300b9770 0x7f4b300b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 >> 172.21.15.60:0/1171501 conn(0x7f4b30061c30 msgr2=0x7f4b300c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 shutdown_connections 2024-04-25T08:37:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:35.821+0000 7f4b37d5c640 1 -- 172.21.15.60:0/1171501 wait complete. 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:37:35.827 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:37:35.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:35.828 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:37:35.843 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:37:35.843 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:37:35.844 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42254sec preferred_lft 42254sec 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:37:35.845 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:37:35.846 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.846 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:37:35.846 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:37:35.846 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:37:35.846 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:37:35.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:35.846 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:37:35.906 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:37:35.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:35.908 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:37:35.965 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:37:35.965 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:37:35.965 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:37:35.965 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:37:35.965 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:37:35.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:35.965 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:37:35.966 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:35.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:37:35.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:37:35.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:37:35.966 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:36.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.050 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:37:36.066 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:37:36.066 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:37:36.066 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:37:36.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:37:36.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:37:36.130 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:37:36.185 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:37:36.186 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:37:36.186 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:37:36.240 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:37:36.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:37:36.322 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:37:36.323 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:37:36.323 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:37:36.331 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:37:36.331 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:37:36.331 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:37:36.331 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:37:36.332 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:37:36.350 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:37:36.351 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:37:36.351 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:37:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:37:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:37:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:37:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:37:36.352 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:37:36.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:37:36.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.418 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:37:36.481 INFO:teuthology.orchestra.run.smithi060.stdout:client8952 2024-04-25T08:37:36.482 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:36.482 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:36.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:36.538 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880547 s, 119 MB/s 2024-04-25T08:37:36.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076453 s, 137 MB/s 2024-04-25T08:37:36.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705564 s, 149 MB/s 2024-04-25T08:37:36.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.637 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699382 s, 150 MB/s 2024-04-25T08:37:36.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786554 s, 133 MB/s 2024-04-25T08:37:36.653 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.653 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.653 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680714 s, 154 MB/s 2024-04-25T08:37:36.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.662 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081155 s, 129 MB/s 2024-04-25T08:37:36.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680983 s, 154 MB/s 2024-04-25T08:37:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.677 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677475 s, 155 MB/s 2024-04-25T08:37:36.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730056 s, 144 MB/s 2024-04-25T08:37:36.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676417 s, 155 MB/s 2024-04-25T08:37:36.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.702 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780381 s, 134 MB/s 2024-04-25T08:37:36.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.710 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719104 s, 146 MB/s 2024-04-25T08:37:36.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.718 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696121 s, 151 MB/s 2024-04-25T08:37:36.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.726 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722263 s, 145 MB/s 2024-04-25T08:37:36.734 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.734 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.734 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722874 s, 145 MB/s 2024-04-25T08:37:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.742 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728116 s, 144 MB/s 2024-04-25T08:37:36.749 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.750 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.750 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653592 s, 160 MB/s 2024-04-25T08:37:36.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.758 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728601 s, 144 MB/s 2024-04-25T08:37:36.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.765 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573906 s, 183 MB/s 2024-04-25T08:37:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.771 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609204 s, 172 MB/s 2024-04-25T08:37:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.780 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728342 s, 144 MB/s 2024-04-25T08:37:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576768 s, 182 MB/s 2024-04-25T08:37:36.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699221 s, 150 MB/s 2024-04-25T08:37:36.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647082 s, 162 MB/s 2024-04-25T08:37:36.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.809 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659023 s, 159 MB/s 2024-04-25T08:37:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604453 s, 173 MB/s 2024-04-25T08:37:36.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660106 s, 159 MB/s 2024-04-25T08:37:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.831 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629438 s, 167 MB/s 2024-04-25T08:37:36.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.838 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591404 s, 177 MB/s 2024-04-25T08:37:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616085 s, 170 MB/s 2024-04-25T08:37:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.853 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696736 s, 150 MB/s 2024-04-25T08:37:36.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:36.854 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:36.854 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:36.854 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:37:36.854 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:36.855 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:36.855 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:36.855 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658011 s, 159 MB/s 2024-04-25T08:37:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731053 s, 143 MB/s 2024-04-25T08:37:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705186 s, 149 MB/s 2024-04-25T08:37:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675633 s, 155 MB/s 2024-04-25T08:37:36.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645775 s, 162 MB/s 2024-04-25T08:37:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.958 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621633 s, 169 MB/s 2024-04-25T08:37:36.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.966 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704626 s, 149 MB/s 2024-04-25T08:37:36.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.973 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624701 s, 168 MB/s 2024-04-25T08:37:36.980 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.980 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.980 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642182 s, 163 MB/s 2024-04-25T08:37:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660038 s, 159 MB/s 2024-04-25T08:37:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601698 s, 174 MB/s 2024-04-25T08:37:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631481 s, 166 MB/s 2024-04-25T08:37:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560663 s, 187 MB/s 2024-04-25T08:37:37.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686504 s, 153 MB/s 2024-04-25T08:37:37.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558399 s, 188 MB/s 2024-04-25T08:37:37.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637258 s, 165 MB/s 2024-04-25T08:37:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.037 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595319 s, 176 MB/s 2024-04-25T08:37:37.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660485 s, 159 MB/s 2024-04-25T08:37:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.052 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674885 s, 155 MB/s 2024-04-25T08:37:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600257 s, 175 MB/s 2024-04-25T08:37:37.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.066 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582446 s, 180 MB/s 2024-04-25T08:37:37.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.074 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069941 s, 150 MB/s 2024-04-25T08:37:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065695 s, 160 MB/s 2024-04-25T08:37:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568683 s, 184 MB/s 2024-04-25T08:37:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618457 s, 170 MB/s 2024-04-25T08:37:37.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648858 s, 162 MB/s 2024-04-25T08:37:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603145 s, 174 MB/s 2024-04-25T08:37:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.116 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622296 s, 169 MB/s 2024-04-25T08:37:37.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658182 s, 159 MB/s 2024-04-25T08:37:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593057 s, 177 MB/s 2024-04-25T08:37:37.137 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.137 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.137 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618381 s, 170 MB/s 2024-04-25T08:37:37.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.144 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564453 s, 186 MB/s 2024-04-25T08:37:37.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:37.145 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.211 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751324 s, 140 MB/s 2024-04-25T08:37:37.218 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700232 s, 150 MB/s 2024-04-25T08:37:37.225 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616592 s, 170 MB/s 2024-04-25T08:37:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.233 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657207 s, 160 MB/s 2024-04-25T08:37:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583807 s, 180 MB/s 2024-04-25T08:37:37.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579532 s, 181 MB/s 2024-04-25T08:37:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594244 s, 176 MB/s 2024-04-25T08:37:37.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.261 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061175 s, 171 MB/s 2024-04-25T08:37:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619798 s, 169 MB/s 2024-04-25T08:37:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.275 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692612 s, 151 MB/s 2024-04-25T08:37:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.283 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638824 s, 164 MB/s 2024-04-25T08:37:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.290 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619282 s, 169 MB/s 2024-04-25T08:37:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701102 s, 150 MB/s 2024-04-25T08:37:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570108 s, 184 MB/s 2024-04-25T08:37:37.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.312 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710398 s, 148 MB/s 2024-04-25T08:37:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.320 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695696 s, 151 MB/s 2024-04-25T08:37:37.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634566 s, 165 MB/s 2024-04-25T08:37:37.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.335 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645598 s, 162 MB/s 2024-04-25T08:37:37.342 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703001 s, 149 MB/s 2024-04-25T08:37:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.350 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623501 s, 168 MB/s 2024-04-25T08:37:37.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.357 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625638 s, 168 MB/s 2024-04-25T08:37:37.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.364 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613157 s, 171 MB/s 2024-04-25T08:37:37.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.371 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639646 s, 164 MB/s 2024-04-25T08:37:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.378 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620624 s, 169 MB/s 2024-04-25T08:37:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.385 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610651 s, 172 MB/s 2024-04-25T08:37:37.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.392 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624578 s, 168 MB/s 2024-04-25T08:37:37.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.400 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653906 s, 160 MB/s 2024-04-25T08:37:37.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.407 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625121 s, 168 MB/s 2024-04-25T08:37:37.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.413 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544617 s, 193 MB/s 2024-04-25T08:37:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.420 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634556 s, 165 MB/s 2024-04-25T08:37:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.428 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709104 s, 148 MB/s 2024-04-25T08:37:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.435 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056871 s, 184 MB/s 2024-04-25T08:37:37.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:37.436 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:37.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725899 s, 144 MB/s 2024-04-25T08:37:37.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633542 s, 166 MB/s 2024-04-25T08:37:37.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639627 s, 164 MB/s 2024-04-25T08:37:37.523 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568745 s, 184 MB/s 2024-04-25T08:37:37.530 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.531 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611167 s, 172 MB/s 2024-04-25T08:37:37.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.538 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625263 s, 168 MB/s 2024-04-25T08:37:37.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.545 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644398 s, 163 MB/s 2024-04-25T08:37:37.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.552 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607061 s, 173 MB/s 2024-04-25T08:37:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.560 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725217 s, 145 MB/s 2024-04-25T08:37:37.566 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.567 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554828 s, 189 MB/s 2024-04-25T08:37:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618361 s, 170 MB/s 2024-04-25T08:37:37.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723876 s, 145 MB/s 2024-04-25T08:37:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569367 s, 184 MB/s 2024-04-25T08:37:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701409 s, 149 MB/s 2024-04-25T08:37:37.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700273 s, 150 MB/s 2024-04-25T08:37:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.611 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606974 s, 173 MB/s 2024-04-25T08:37:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698246 s, 150 MB/s 2024-04-25T08:37:37.625 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.626 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556813 s, 188 MB/s 2024-04-25T08:37:37.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.634 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766544 s, 137 MB/s 2024-04-25T08:37:37.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.641 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621498 s, 169 MB/s 2024-04-25T08:37:37.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.648 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601335 s, 174 MB/s 2024-04-25T08:37:37.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.655 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627247 s, 167 MB/s 2024-04-25T08:37:37.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.662 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559169 s, 188 MB/s 2024-04-25T08:37:37.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697419 s, 150 MB/s 2024-04-25T08:37:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058 s, 181 MB/s 2024-04-25T08:37:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.684 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600661 s, 175 MB/s 2024-04-25T08:37:37.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.691 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647525 s, 162 MB/s 2024-04-25T08:37:37.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722462 s, 145 MB/s 2024-04-25T08:37:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607378 s, 173 MB/s 2024-04-25T08:37:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572512 s, 183 MB/s 2024-04-25T08:37:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.721 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712931 s, 147 MB/s 2024-04-25T08:37:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621716 s, 169 MB/s 2024-04-25T08:37:37.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:37.729 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641336 s, 163 MB/s 2024-04-25T08:37:37.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.801 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587113 s, 179 MB/s 2024-04-25T08:37:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062329 s, 168 MB/s 2024-04-25T08:37:37.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063099 s, 166 MB/s 2024-04-25T08:37:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673061 s, 156 MB/s 2024-04-25T08:37:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575544 s, 182 MB/s 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582637 s, 180 MB/s 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709538 s, 148 MB/s 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611324 s, 172 MB/s 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603275 s, 174 MB/s 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634367 s, 165 MB/s 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622361 s, 168 MB/s 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.881 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682594 s, 154 MB/s 2024-04-25T08:37:37.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.887 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.887 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605059 s, 173 MB/s 2024-04-25T08:37:37.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.895 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638926 s, 164 MB/s 2024-04-25T08:37:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.902 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063635 s, 165 MB/s 2024-04-25T08:37:37.909 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.909 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.909 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590787 s, 177 MB/s 2024-04-25T08:37:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611888 s, 171 MB/s 2024-04-25T08:37:37.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.923 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591232 s, 177 MB/s 2024-04-25T08:37:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629402 s, 167 MB/s 2024-04-25T08:37:37.938 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.938 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.938 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764337 s, 137 MB/s 2024-04-25T08:37:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.946 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692126 s, 152 MB/s 2024-04-25T08:37:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.954 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696698 s, 151 MB/s 2024-04-25T08:37:37.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.961 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642028 s, 163 MB/s 2024-04-25T08:37:37.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.969 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638726 s, 164 MB/s 2024-04-25T08:37:37.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707445 s, 148 MB/s 2024-04-25T08:37:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.983 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622881 s, 168 MB/s 2024-04-25T08:37:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.990 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599048 s, 175 MB/s 2024-04-25T08:37:37.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:37.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:37.998 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673731 s, 156 MB/s 2024-04-25T08:37:38.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.006 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708921 s, 148 MB/s 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590383 s, 178 MB/s 2024-04-25T08:37:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.020 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576697 s, 182 MB/s 2024-04-25T08:37:38.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:38.021 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:38.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623025 s, 168 MB/s 2024-04-25T08:37:38.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627503 s, 167 MB/s 2024-04-25T08:37:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632439 s, 166 MB/s 2024-04-25T08:37:38.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734618 s, 143 MB/s 2024-04-25T08:37:38.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.114 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574858 s, 182 MB/s 2024-04-25T08:37:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730723 s, 143 MB/s 2024-04-25T08:37:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658245 s, 159 MB/s 2024-04-25T08:37:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722143 s, 145 MB/s 2024-04-25T08:37:38.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.145 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575924 s, 182 MB/s 2024-04-25T08:37:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645153 s, 163 MB/s 2024-04-25T08:37:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634281 s, 165 MB/s 2024-04-25T08:37:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.166 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621001 s, 169 MB/s 2024-04-25T08:37:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625714 s, 168 MB/s 2024-04-25T08:37:38.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.181 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632661 s, 166 MB/s 2024-04-25T08:37:38.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.188 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611806 s, 171 MB/s 2024-04-25T08:37:38.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596392 s, 176 MB/s 2024-04-25T08:37:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062019 s, 169 MB/s 2024-04-25T08:37:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752208 s, 139 MB/s 2024-04-25T08:37:38.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630058 s, 166 MB/s 2024-04-25T08:37:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629922 s, 166 MB/s 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605166 s, 173 MB/s 2024-04-25T08:37:38.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.239 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646413 s, 162 MB/s 2024-04-25T08:37:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.245 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585149 s, 179 MB/s 2024-04-25T08:37:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718363 s, 146 MB/s 2024-04-25T08:37:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.261 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714274 s, 147 MB/s 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706372 s, 148 MB/s 2024-04-25T08:37:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.278 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744974 s, 141 MB/s 2024-04-25T08:37:38.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582749 s, 180 MB/s 2024-04-25T08:37:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.293 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736733 s, 142 MB/s 2024-04-25T08:37:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.300 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661976 s, 158 MB/s 2024-04-25T08:37:38.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.307 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600861 s, 175 MB/s 2024-04-25T08:37:38.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590882 s, 177 MB/s 2024-04-25T08:37:38.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:38.315 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:38.315 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:38.316 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:37:38.316 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:38.316 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:38.316 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:38.316 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:38.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656363 s, 160 MB/s 2024-04-25T08:37:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711772 s, 147 MB/s 2024-04-25T08:37:38.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065421 s, 160 MB/s 2024-04-25T08:37:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633136 s, 166 MB/s 2024-04-25T08:37:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.412 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070078 s, 150 MB/s 2024-04-25T08:37:38.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.419 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630781 s, 166 MB/s 2024-04-25T08:37:38.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583947 s, 180 MB/s 2024-04-25T08:37:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.434 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699151 s, 150 MB/s 2024-04-25T08:37:38.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.441 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581524 s, 180 MB/s 2024-04-25T08:37:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.448 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599243 s, 175 MB/s 2024-04-25T08:37:38.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.455 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703563 s, 149 MB/s 2024-04-25T08:37:38.462 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.462 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.462 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578708 s, 181 MB/s 2024-04-25T08:37:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.470 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647144 s, 162 MB/s 2024-04-25T08:37:38.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615549 s, 170 MB/s 2024-04-25T08:37:38.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.484 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689958 s, 152 MB/s 2024-04-25T08:37:38.491 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.491 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.491 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562477 s, 186 MB/s 2024-04-25T08:37:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062541 s, 168 MB/s 2024-04-25T08:37:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006474 s, 162 MB/s 2024-04-25T08:37:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595742 s, 176 MB/s 2024-04-25T08:37:38.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.522 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871555 s, 120 MB/s 2024-04-25T08:37:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611663 s, 171 MB/s 2024-04-25T08:37:38.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608994 s, 172 MB/s 2024-04-25T08:37:38.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628118 s, 167 MB/s 2024-04-25T08:37:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700739 s, 150 MB/s 2024-04-25T08:37:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.559 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695075 s, 151 MB/s 2024-04-25T08:37:38.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.567 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709843 s, 148 MB/s 2024-04-25T08:37:38.573 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.574 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.574 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582873 s, 180 MB/s 2024-04-25T08:37:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.581 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629387 s, 167 MB/s 2024-04-25T08:37:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701822 s, 149 MB/s 2024-04-25T08:37:38.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708914 s, 148 MB/s 2024-04-25T08:37:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682652 s, 154 MB/s 2024-04-25T08:37:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712942 s, 147 MB/s 2024-04-25T08:37:38.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:38.614 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637683 s, 164 MB/s 2024-04-25T08:37:38.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704478 s, 149 MB/s 2024-04-25T08:37:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562276 s, 186 MB/s 2024-04-25T08:37:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.700 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621111 s, 169 MB/s 2024-04-25T08:37:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615255 s, 170 MB/s 2024-04-25T08:37:38.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.714 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632857 s, 166 MB/s 2024-04-25T08:37:38.722 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.722 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.722 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710747 s, 148 MB/s 2024-04-25T08:37:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627406 s, 167 MB/s 2024-04-25T08:37:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068945 s, 152 MB/s 2024-04-25T08:37:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.745 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628974 s, 167 MB/s 2024-04-25T08:37:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592454 s, 177 MB/s 2024-04-25T08:37:38.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659522 s, 159 MB/s 2024-04-25T08:37:38.765 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.765 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.765 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058455 s, 179 MB/s 2024-04-25T08:37:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.772 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587378 s, 179 MB/s 2024-04-25T08:37:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.779 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601741 s, 174 MB/s 2024-04-25T08:37:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.788 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744019 s, 141 MB/s 2024-04-25T08:37:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064133 s, 164 MB/s 2024-04-25T08:37:38.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590955 s, 177 MB/s 2024-04-25T08:37:38.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573392 s, 183 MB/s 2024-04-25T08:37:38.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.815 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557723 s, 188 MB/s 2024-04-25T08:37:38.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712779 s, 147 MB/s 2024-04-25T08:37:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.831 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729921 s, 144 MB/s 2024-04-25T08:37:38.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.839 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719104 s, 146 MB/s 2024-04-25T08:37:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.846 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632473 s, 166 MB/s 2024-04-25T08:37:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.854 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709362 s, 148 MB/s 2024-04-25T08:37:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.863 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718651 s, 146 MB/s 2024-04-25T08:37:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682998 s, 154 MB/s 2024-04-25T08:37:38.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.878 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688768 s, 152 MB/s 2024-04-25T08:37:38.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.885 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622457 s, 168 MB/s 2024-04-25T08:37:38.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.893 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687976 s, 152 MB/s 2024-04-25T08:37:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.900 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636672 s, 165 MB/s 2024-04-25T08:37:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690231 s, 152 MB/s 2024-04-25T08:37:38.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:38.909 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:38.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.974 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067545 s, 155 MB/s 2024-04-25T08:37:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.981 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605337 s, 173 MB/s 2024-04-25T08:37:38.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.988 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596386 s, 176 MB/s 2024-04-25T08:37:38.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:38.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:38.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674861 s, 155 MB/s 2024-04-25T08:37:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630791 s, 166 MB/s 2024-04-25T08:37:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.011 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729286 s, 144 MB/s 2024-04-25T08:37:39.017 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.017 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593435 s, 177 MB/s 2024-04-25T08:37:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.025 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624662 s, 168 MB/s 2024-04-25T08:37:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734982 s, 143 MB/s 2024-04-25T08:37:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668272 s, 157 MB/s 2024-04-25T08:37:39.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.047 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560153 s, 187 MB/s 2024-04-25T08:37:39.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.055 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703702 s, 149 MB/s 2024-04-25T08:37:39.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662495 s, 158 MB/s 2024-04-25T08:37:39.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697082 s, 150 MB/s 2024-04-25T08:37:39.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.077 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608116 s, 172 MB/s 2024-04-25T08:37:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740333 s, 142 MB/s 2024-04-25T08:37:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.093 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647953 s, 162 MB/s 2024-04-25T08:37:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.101 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704663 s, 149 MB/s 2024-04-25T08:37:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701742 s, 149 MB/s 2024-04-25T08:37:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578073 s, 181 MB/s 2024-04-25T08:37:39.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.123 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708141 s, 148 MB/s 2024-04-25T08:37:39.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.131 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666968 s, 157 MB/s 2024-04-25T08:37:39.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623358 s, 168 MB/s 2024-04-25T08:37:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015842 s, 66.2 MB/s 2024-04-25T08:37:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.162 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063393 s, 165 MB/s 2024-04-25T08:37:39.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.170 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737588 s, 142 MB/s 2024-04-25T08:37:39.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.178 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617215 s, 170 MB/s 2024-04-25T08:37:39.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700913 s, 150 MB/s 2024-04-25T08:37:39.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007103 s, 148 MB/s 2024-04-25T08:37:39.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.201 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005785 s, 181 MB/s 2024-04-25T08:37:39.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691964 s, 152 MB/s 2024-04-25T08:37:39.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.215 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633962 s, 165 MB/s 2024-04-25T08:37:39.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:39.216 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:39.216 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:39.216 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:37:39.216 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:39.216 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:39.217 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:39.217 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598679 s, 175 MB/s 2024-04-25T08:37:39.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.290 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713031 s, 147 MB/s 2024-04-25T08:37:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058325 s, 180 MB/s 2024-04-25T08:37:39.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729339 s, 144 MB/s 2024-04-25T08:37:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586611 s, 179 MB/s 2024-04-25T08:37:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.320 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703373 s, 149 MB/s 2024-04-25T08:37:39.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695347 s, 151 MB/s 2024-04-25T08:37:39.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.335 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.335 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709422 s, 148 MB/s 2024-04-25T08:37:39.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685802 s, 153 MB/s 2024-04-25T08:37:39.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718869 s, 146 MB/s 2024-04-25T08:37:39.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614082 s, 171 MB/s 2024-04-25T08:37:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621643 s, 169 MB/s 2024-04-25T08:37:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643607 s, 163 MB/s 2024-04-25T08:37:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631626 s, 166 MB/s 2024-04-25T08:37:39.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.387 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647229 s, 162 MB/s 2024-04-25T08:37:39.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723573 s, 145 MB/s 2024-04-25T08:37:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625576 s, 168 MB/s 2024-04-25T08:37:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.409 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590859 s, 177 MB/s 2024-04-25T08:37:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.416 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643987 s, 163 MB/s 2024-04-25T08:37:39.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627588 s, 167 MB/s 2024-04-25T08:37:39.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.431 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727974 s, 144 MB/s 2024-04-25T08:37:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.439 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699052 s, 150 MB/s 2024-04-25T08:37:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598199 s, 175 MB/s 2024-04-25T08:37:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713248 s, 147 MB/s 2024-04-25T08:37:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630815 s, 166 MB/s 2024-04-25T08:37:39.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.469 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677435 s, 155 MB/s 2024-04-25T08:37:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627122 s, 167 MB/s 2024-04-25T08:37:39.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.484 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705305 s, 149 MB/s 2024-04-25T08:37:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.493 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785141 s, 134 MB/s 2024-04-25T08:37:39.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.500 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637904 s, 164 MB/s 2024-04-25T08:37:39.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.508 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707415 s, 148 MB/s 2024-04-25T08:37:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.516 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631932 s, 166 MB/s 2024-04-25T08:37:39.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:39.517 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:39.581 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659405 s, 159 MB/s 2024-04-25T08:37:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.589 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641909 s, 163 MB/s 2024-04-25T08:37:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597342 s, 176 MB/s 2024-04-25T08:37:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564979 s, 186 MB/s 2024-04-25T08:37:39.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553749 s, 189 MB/s 2024-04-25T08:37:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.616 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640645 s, 164 MB/s 2024-04-25T08:37:39.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.624 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659547 s, 159 MB/s 2024-04-25T08:37:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.632 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723824 s, 145 MB/s 2024-04-25T08:37:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.639 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671687 s, 156 MB/s 2024-04-25T08:37:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.647 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746131 s, 141 MB/s 2024-04-25T08:37:39.654 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.654 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.654 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057674 s, 182 MB/s 2024-04-25T08:37:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657314 s, 160 MB/s 2024-04-25T08:37:39.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.669 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626996 s, 167 MB/s 2024-04-25T08:37:39.675 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.676 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.676 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059587 s, 176 MB/s 2024-04-25T08:37:39.683 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.684 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692946 s, 151 MB/s 2024-04-25T08:37:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.690 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587303 s, 179 MB/s 2024-04-25T08:37:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.698 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632055 s, 166 MB/s 2024-04-25T08:37:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.705 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652495 s, 161 MB/s 2024-04-25T08:37:39.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.713 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709759 s, 148 MB/s 2024-04-25T08:37:39.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.721 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705467 s, 149 MB/s 2024-04-25T08:37:39.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.729 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071235 s, 147 MB/s 2024-04-25T08:37:39.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705465 s, 149 MB/s 2024-04-25T08:37:39.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.745 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730513 s, 144 MB/s 2024-04-25T08:37:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713715 s, 147 MB/s 2024-04-25T08:37:39.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654214 s, 160 MB/s 2024-04-25T08:37:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614469 s, 171 MB/s 2024-04-25T08:37:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.776 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007308 s, 143 MB/s 2024-04-25T08:37:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621562 s, 169 MB/s 2024-04-25T08:37:39.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.790 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637545 s, 164 MB/s 2024-04-25T08:37:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.797 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613858 s, 171 MB/s 2024-04-25T08:37:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.805 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727637 s, 144 MB/s 2024-04-25T08:37:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00987539 s, 106 MB/s 2024-04-25T08:37:39.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:39.817 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:39.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650248 s, 161 MB/s 2024-04-25T08:37:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657503 s, 159 MB/s 2024-04-25T08:37:39.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657873 s, 159 MB/s 2024-04-25T08:37:39.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599564 s, 175 MB/s 2024-04-25T08:37:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670402 s, 156 MB/s 2024-04-25T08:37:39.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618949 s, 169 MB/s 2024-04-25T08:37:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710556 s, 148 MB/s 2024-04-25T08:37:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635712 s, 165 MB/s 2024-04-25T08:37:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634983 s, 165 MB/s 2024-04-25T08:37:39.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.948 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601458 s, 174 MB/s 2024-04-25T08:37:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.955 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635684 s, 165 MB/s 2024-04-25T08:37:39.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.962 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600261 s, 175 MB/s 2024-04-25T08:37:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.969 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060475 s, 173 MB/s 2024-04-25T08:37:39.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570444 s, 184 MB/s 2024-04-25T08:37:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.983 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630064 s, 166 MB/s 2024-04-25T08:37:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.990 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601263 s, 174 MB/s 2024-04-25T08:37:39.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:39.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:39.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614773 s, 171 MB/s 2024-04-25T08:37:40.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.005 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.005 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650371 s, 161 MB/s 2024-04-25T08:37:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.012 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601819 s, 174 MB/s 2024-04-25T08:37:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.019 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671227 s, 156 MB/s 2024-04-25T08:37:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622714 s, 168 MB/s 2024-04-25T08:37:40.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.035 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737094 s, 142 MB/s 2024-04-25T08:37:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579078 s, 181 MB/s 2024-04-25T08:37:40.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.049 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635463 s, 165 MB/s 2024-04-25T08:37:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.056 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701837 s, 149 MB/s 2024-04-25T08:37:40.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727078 s, 144 MB/s 2024-04-25T08:37:40.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.071 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572041 s, 183 MB/s 2024-04-25T08:37:40.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.078 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580319 s, 181 MB/s 2024-04-25T08:37:40.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.085 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622396 s, 168 MB/s 2024-04-25T08:37:40.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.093 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707466 s, 148 MB/s 2024-04-25T08:37:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.100 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591776 s, 177 MB/s 2024-04-25T08:37:40.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703341 s, 149 MB/s 2024-04-25T08:37:40.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:40.109 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:40.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731776 s, 143 MB/s 2024-04-25T08:37:40.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731057 s, 143 MB/s 2024-04-25T08:37:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587253 s, 179 MB/s 2024-04-25T08:37:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.197 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604866 s, 173 MB/s 2024-04-25T08:37:40.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.204 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.204 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592083 s, 177 MB/s 2024-04-25T08:37:40.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565642 s, 185 MB/s 2024-04-25T08:37:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.218 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635808 s, 165 MB/s 2024-04-25T08:37:40.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605643 s, 173 MB/s 2024-04-25T08:37:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714533 s, 147 MB/s 2024-04-25T08:37:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625082 s, 168 MB/s 2024-04-25T08:37:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626467 s, 167 MB/s 2024-04-25T08:37:40.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.255 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682545 s, 154 MB/s 2024-04-25T08:37:40.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.262 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641698 s, 163 MB/s 2024-04-25T08:37:40.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.269 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652454 s, 161 MB/s 2024-04-25T08:37:40.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710406 s, 148 MB/s 2024-04-25T08:37:40.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.285 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709435 s, 148 MB/s 2024-04-25T08:37:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597788 s, 175 MB/s 2024-04-25T08:37:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587221 s, 179 MB/s 2024-04-25T08:37:40.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581068 s, 180 MB/s 2024-04-25T08:37:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.313 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617992 s, 170 MB/s 2024-04-25T08:37:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.320 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634437 s, 165 MB/s 2024-04-25T08:37:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628815 s, 167 MB/s 2024-04-25T08:37:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.334 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615081 s, 170 MB/s 2024-04-25T08:37:40.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.341 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629934 s, 166 MB/s 2024-04-25T08:37:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724953 s, 145 MB/s 2024-04-25T08:37:40.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.357 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717176 s, 146 MB/s 2024-04-25T08:37:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.364 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611846 s, 171 MB/s 2024-04-25T08:37:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.372 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661919 s, 158 MB/s 2024-04-25T08:37:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624595 s, 168 MB/s 2024-04-25T08:37:40.385 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.385 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.385 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579698 s, 181 MB/s 2024-04-25T08:37:40.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.392 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616232 s, 170 MB/s 2024-04-25T08:37:40.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.399 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603547 s, 174 MB/s 2024-04-25T08:37:40.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:40.400 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:40.400 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:40.401 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:37:40.401 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:40.401 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:40.401 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:40.401 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:40.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.467 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746661 s, 140 MB/s 2024-04-25T08:37:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.475 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721715 s, 145 MB/s 2024-04-25T08:37:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642132 s, 163 MB/s 2024-04-25T08:37:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562479 s, 186 MB/s 2024-04-25T08:37:40.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667621 s, 157 MB/s 2024-04-25T08:37:40.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619705 s, 169 MB/s 2024-04-25T08:37:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731293 s, 143 MB/s 2024-04-25T08:37:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633477 s, 166 MB/s 2024-04-25T08:37:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583568 s, 180 MB/s 2024-04-25T08:37:40.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063457 s, 165 MB/s 2024-04-25T08:37:40.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627555 s, 167 MB/s 2024-04-25T08:37:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602331 s, 174 MB/s 2024-04-25T08:37:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567441 s, 185 MB/s 2024-04-25T08:37:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.562 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739186 s, 142 MB/s 2024-04-25T08:37:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707644 s, 148 MB/s 2024-04-25T08:37:40.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.578 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694428 s, 151 MB/s 2024-04-25T08:37:40.585 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.585 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.586 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688022 s, 152 MB/s 2024-04-25T08:37:40.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.593 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631526 s, 166 MB/s 2024-04-25T08:37:40.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.601 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.601 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724332 s, 145 MB/s 2024-04-25T08:37:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641145 s, 164 MB/s 2024-04-25T08:37:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.615 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586596 s, 179 MB/s 2024-04-25T08:37:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725585 s, 145 MB/s 2024-04-25T08:37:40.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.631 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671206 s, 156 MB/s 2024-04-25T08:37:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613142 s, 171 MB/s 2024-04-25T08:37:40.646 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.646 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.646 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717939 s, 146 MB/s 2024-04-25T08:37:40.653 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.653 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.653 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634438 s, 165 MB/s 2024-04-25T08:37:40.661 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.661 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.661 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740359 s, 142 MB/s 2024-04-25T08:37:40.668 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.669 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650909 s, 161 MB/s 2024-04-25T08:37:40.675 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.675 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.675 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595503 s, 176 MB/s 2024-04-25T08:37:40.683 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.683 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.683 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648277 s, 162 MB/s 2024-04-25T08:37:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.691 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721094 s, 145 MB/s 2024-04-25T08:37:40.698 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.698 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.698 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621324 s, 169 MB/s 2024-04-25T08:37:40.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:40.699 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:40.699 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:40.699 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:37:40.699 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:40.699 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:40.699 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:40.700 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:40.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.765 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.765 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621257 s, 169 MB/s 2024-04-25T08:37:40.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.772 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596073 s, 176 MB/s 2024-04-25T08:37:40.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.778 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616718 s, 170 MB/s 2024-04-25T08:37:40.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666719 s, 157 MB/s 2024-04-25T08:37:40.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703942 s, 149 MB/s 2024-04-25T08:37:40.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.801 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625531 s, 168 MB/s 2024-04-25T08:37:40.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604394 s, 173 MB/s 2024-04-25T08:37:40.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.815 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555947 s, 189 MB/s 2024-04-25T08:37:40.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.822 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064416 s, 163 MB/s 2024-04-25T08:37:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.828 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607817 s, 173 MB/s 2024-04-25T08:37:40.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.835 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609895 s, 172 MB/s 2024-04-25T08:37:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.843 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667003 s, 157 MB/s 2024-04-25T08:37:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.850 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624398 s, 168 MB/s 2024-04-25T08:37:40.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.857 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621941 s, 169 MB/s 2024-04-25T08:37:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.864 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578138 s, 181 MB/s 2024-04-25T08:37:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726026 s, 144 MB/s 2024-04-25T08:37:40.879 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.879 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.879 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063974 s, 164 MB/s 2024-04-25T08:37:40.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.886 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618474 s, 170 MB/s 2024-04-25T08:37:40.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.894 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700769 s, 150 MB/s 2024-04-25T08:37:40.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.901 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629771 s, 167 MB/s 2024-04-25T08:37:40.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600005 s, 175 MB/s 2024-04-25T08:37:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.915 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061982 s, 169 MB/s 2024-04-25T08:37:40.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.922 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617755 s, 170 MB/s 2024-04-25T08:37:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584572 s, 179 MB/s 2024-04-25T08:37:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621638 s, 169 MB/s 2024-04-25T08:37:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699528 s, 150 MB/s 2024-04-25T08:37:40.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635451 s, 165 MB/s 2024-04-25T08:37:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.959 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625838 s, 168 MB/s 2024-04-25T08:37:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581846 s, 180 MB/s 2024-04-25T08:37:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.972 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596052 s, 176 MB/s 2024-04-25T08:37:40.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.979 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585958 s, 179 MB/s 2024-04-25T08:37:40.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:40.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:40.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627324 s, 167 MB/s 2024-04-25T08:37:40.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:40.987 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:40.987 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:40.987 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:37:40.987 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:40.987 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:40.987 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:40.988 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:41.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.054 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.054 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721092 s, 145 MB/s 2024-04-25T08:37:41.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.061 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.061 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621642 s, 169 MB/s 2024-04-25T08:37:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.068 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623106 s, 168 MB/s 2024-04-25T08:37:41.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621121 s, 169 MB/s 2024-04-25T08:37:41.082 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.082 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.082 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599974 s, 175 MB/s 2024-04-25T08:37:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.089 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606816 s, 173 MB/s 2024-04-25T08:37:41.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.097 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662131 s, 158 MB/s 2024-04-25T08:37:41.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705421 s, 149 MB/s 2024-04-25T08:37:41.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633575 s, 166 MB/s 2024-04-25T08:37:41.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.119 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607856 s, 173 MB/s 2024-04-25T08:37:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060232 s, 174 MB/s 2024-04-25T08:37:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656213 s, 160 MB/s 2024-04-25T08:37:41.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.140 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629342 s, 167 MB/s 2024-04-25T08:37:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.147 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649457 s, 161 MB/s 2024-04-25T08:37:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.154 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569405 s, 184 MB/s 2024-04-25T08:37:41.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635447 s, 165 MB/s 2024-04-25T08:37:41.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571592 s, 183 MB/s 2024-04-25T08:37:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698363 s, 150 MB/s 2024-04-25T08:37:41.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611996 s, 171 MB/s 2024-04-25T08:37:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.191 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660648 s, 159 MB/s 2024-04-25T08:37:41.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648167 s, 162 MB/s 2024-04-25T08:37:41.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.205 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623554 s, 168 MB/s 2024-04-25T08:37:41.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.212 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582503 s, 180 MB/s 2024-04-25T08:37:41.218 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.218 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.218 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566103 s, 185 MB/s 2024-04-25T08:37:41.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657692 s, 159 MB/s 2024-04-25T08:37:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.233 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613016 s, 171 MB/s 2024-04-25T08:37:41.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.240 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617242 s, 170 MB/s 2024-04-25T08:37:41.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565387 s, 185 MB/s 2024-04-25T08:37:41.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.254 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607545 s, 173 MB/s 2024-04-25T08:37:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.261 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698837 s, 150 MB/s 2024-04-25T08:37:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623688 s, 168 MB/s 2024-04-25T08:37:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703965 s, 149 MB/s 2024-04-25T08:37:41.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:41.278 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:41.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.343 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657167 s, 160 MB/s 2024-04-25T08:37:41.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.350 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625408 s, 168 MB/s 2024-04-25T08:37:41.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636299 s, 165 MB/s 2024-04-25T08:37:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708082 s, 148 MB/s 2024-04-25T08:37:41.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.374 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073683 s, 142 MB/s 2024-04-25T08:37:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636528 s, 165 MB/s 2024-04-25T08:37:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.388 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626479 s, 167 MB/s 2024-04-25T08:37:41.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560965 s, 187 MB/s 2024-04-25T08:37:41.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.403 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742792 s, 141 MB/s 2024-04-25T08:37:41.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116599 s, 89.9 MB/s 2024-04-25T08:37:41.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715229 s, 147 MB/s 2024-04-25T08:37:41.432 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.432 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.432 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743581 s, 141 MB/s 2024-04-25T08:37:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.439 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635742 s, 165 MB/s 2024-04-25T08:37:41.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.447 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700374 s, 150 MB/s 2024-04-25T08:37:41.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618135 s, 170 MB/s 2024-04-25T08:37:41.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624819 s, 168 MB/s 2024-04-25T08:37:41.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.469 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071341 s, 147 MB/s 2024-04-25T08:37:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074769 s, 140 MB/s 2024-04-25T08:37:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.485 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694105 s, 151 MB/s 2024-04-25T08:37:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.493 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738171 s, 142 MB/s 2024-04-25T08:37:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.500 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619304 s, 169 MB/s 2024-04-25T08:37:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.508 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710422 s, 148 MB/s 2024-04-25T08:37:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.515 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062938 s, 167 MB/s 2024-04-25T08:37:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.522 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593224 s, 177 MB/s 2024-04-25T08:37:41.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.530 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.530 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659225 s, 159 MB/s 2024-04-25T08:37:41.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609387 s, 172 MB/s 2024-04-25T08:37:41.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.544 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616247 s, 170 MB/s 2024-04-25T08:37:41.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.550 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585625 s, 179 MB/s 2024-04-25T08:37:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619604 s, 169 MB/s 2024-04-25T08:37:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.565 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619197 s, 169 MB/s 2024-04-25T08:37:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578469 s, 181 MB/s 2024-04-25T08:37:41.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.579 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666093 s, 157 MB/s 2024-04-25T08:37:41.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:41.580 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:41.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623509 s, 168 MB/s 2024-04-25T08:37:41.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654466 s, 160 MB/s 2024-04-25T08:37:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060653 s, 173 MB/s 2024-04-25T08:37:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601383 s, 174 MB/s 2024-04-25T08:37:41.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710273 s, 148 MB/s 2024-04-25T08:37:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058364 s, 180 MB/s 2024-04-25T08:37:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599136 s, 175 MB/s 2024-04-25T08:37:41.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622317 s, 168 MB/s 2024-04-25T08:37:41.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110689 s, 94.7 MB/s 2024-04-25T08:37:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074788 s, 140 MB/s 2024-04-25T08:37:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770843 s, 136 MB/s 2024-04-25T08:37:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579446 s, 181 MB/s 2024-04-25T08:37:41.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701183 s, 150 MB/s 2024-04-25T08:37:41.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.745 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607254 s, 173 MB/s 2024-04-25T08:37:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.752 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061043 s, 172 MB/s 2024-04-25T08:37:41.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.759 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649439 s, 161 MB/s 2024-04-25T08:37:41.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.766 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629088 s, 167 MB/s 2024-04-25T08:37:41.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730178 s, 144 MB/s 2024-04-25T08:37:41.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634228 s, 165 MB/s 2024-04-25T08:37:41.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.789 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062316 s, 168 MB/s 2024-04-25T08:37:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.796 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627666 s, 167 MB/s 2024-04-25T08:37:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614428 s, 171 MB/s 2024-04-25T08:37:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635404 s, 165 MB/s 2024-04-25T08:37:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.819 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732945 s, 143 MB/s 2024-04-25T08:37:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697342 s, 150 MB/s 2024-04-25T08:37:41.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582013 s, 180 MB/s 2024-04-25T08:37:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.840 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605148 s, 173 MB/s 2024-04-25T08:37:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700529 s, 150 MB/s 2024-04-25T08:37:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628219 s, 167 MB/s 2024-04-25T08:37:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711101 s, 147 MB/s 2024-04-25T08:37:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599767 s, 175 MB/s 2024-04-25T08:37:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611364 s, 172 MB/s 2024-04-25T08:37:41.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:41.878 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:41.878 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:41.879 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:37:41.879 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:41.879 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:41.879 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:41.879 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660331 s, 159 MB/s 2024-04-25T08:37:41.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.952 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708089 s, 148 MB/s 2024-04-25T08:37:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.959 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063684 s, 165 MB/s 2024-04-25T08:37:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.967 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654492 s, 160 MB/s 2024-04-25T08:37:41.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.974 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061484 s, 171 MB/s 2024-04-25T08:37:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725025 s, 145 MB/s 2024-04-25T08:37:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060332 s, 174 MB/s 2024-04-25T08:37:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:41.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715811 s, 146 MB/s 2024-04-25T08:37:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672649 s, 156 MB/s 2024-04-25T08:37:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.012 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669693 s, 157 MB/s 2024-04-25T08:37:42.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.020 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713199 s, 147 MB/s 2024-04-25T08:37:42.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.027 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634047 s, 165 MB/s 2024-04-25T08:37:42.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.035 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065738 s, 160 MB/s 2024-04-25T08:37:42.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.042 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621727 s, 169 MB/s 2024-04-25T08:37:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729593 s, 144 MB/s 2024-04-25T08:37:42.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.057 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624691 s, 168 MB/s 2024-04-25T08:37:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.064 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605184 s, 173 MB/s 2024-04-25T08:37:42.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.071 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629008 s, 167 MB/s 2024-04-25T08:37:42.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.078 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600659 s, 175 MB/s 2024-04-25T08:37:42.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715036 s, 147 MB/s 2024-04-25T08:37:42.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.093 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647439 s, 162 MB/s 2024-04-25T08:37:42.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.100 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573198 s, 183 MB/s 2024-04-25T08:37:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740679 s, 142 MB/s 2024-04-25T08:37:42.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.116 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069843 s, 150 MB/s 2024-04-25T08:37:42.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.123 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633126 s, 166 MB/s 2024-04-25T08:37:42.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.132 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704346 s, 149 MB/s 2024-04-25T08:37:42.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.140 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698867 s, 150 MB/s 2024-04-25T08:37:42.147 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.147 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.147 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658646 s, 159 MB/s 2024-04-25T08:37:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.154 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616013 s, 170 MB/s 2024-04-25T08:37:42.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633964 s, 165 MB/s 2024-04-25T08:37:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583645 s, 180 MB/s 2024-04-25T08:37:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.176 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721149 s, 145 MB/s 2024-04-25T08:37:42.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:42.177 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:42.177 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:42.177 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:37:42.178 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:42.178 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:42.178 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:42.178 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:42.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.244 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639358 s, 164 MB/s 2024-04-25T08:37:42.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.251 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583672 s, 180 MB/s 2024-04-25T08:37:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.258 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659757 s, 159 MB/s 2024-04-25T08:37:42.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.265 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611683 s, 171 MB/s 2024-04-25T08:37:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.273 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713612 s, 147 MB/s 2024-04-25T08:37:42.280 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.280 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664742 s, 158 MB/s 2024-04-25T08:37:42.287 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.287 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.287 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061391 s, 171 MB/s 2024-04-25T08:37:42.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.295 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595707 s, 176 MB/s 2024-04-25T08:37:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.302 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665806 s, 157 MB/s 2024-04-25T08:37:42.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.309 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626828 s, 167 MB/s 2024-04-25T08:37:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652527 s, 161 MB/s 2024-04-25T08:37:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.323 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563639 s, 186 MB/s 2024-04-25T08:37:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631627 s, 166 MB/s 2024-04-25T08:37:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600298 s, 175 MB/s 2024-04-25T08:37:42.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.345 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.345 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709834 s, 148 MB/s 2024-04-25T08:37:42.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.352 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601261 s, 174 MB/s 2024-04-25T08:37:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.360 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741804 s, 141 MB/s 2024-04-25T08:37:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.367 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586776 s, 179 MB/s 2024-04-25T08:37:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631557 s, 166 MB/s 2024-04-25T08:37:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058442 s, 179 MB/s 2024-04-25T08:37:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637361 s, 165 MB/s 2024-04-25T08:37:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624157 s, 168 MB/s 2024-04-25T08:37:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573896 s, 183 MB/s 2024-04-25T08:37:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638738 s, 164 MB/s 2024-04-25T08:37:42.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.417 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618208 s, 170 MB/s 2024-04-25T08:37:42.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.424 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060319 s, 174 MB/s 2024-04-25T08:37:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.431 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634468 s, 165 MB/s 2024-04-25T08:37:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.438 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594073 s, 177 MB/s 2024-04-25T08:37:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.445 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663972 s, 158 MB/s 2024-04-25T08:37:42.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.453 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639176 s, 164 MB/s 2024-04-25T08:37:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753046 s, 139 MB/s 2024-04-25T08:37:42.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063672 s, 165 MB/s 2024-04-25T08:37:42.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:42.469 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:42.469 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:42.469 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:37:42.469 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:42.469 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:42.470 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:42.470 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734467 s, 143 MB/s 2024-04-25T08:37:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628573 s, 167 MB/s 2024-04-25T08:37:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.551 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716106 s, 146 MB/s 2024-04-25T08:37:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.558 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648422 s, 162 MB/s 2024-04-25T08:37:42.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.566 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653773 s, 160 MB/s 2024-04-25T08:37:42.573 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.573 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.573 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617906 s, 170 MB/s 2024-04-25T08:37:42.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.580 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625027 s, 168 MB/s 2024-04-25T08:37:42.586 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.586 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.586 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569519 s, 184 MB/s 2024-04-25T08:37:42.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.593 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.593 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596563 s, 176 MB/s 2024-04-25T08:37:42.600 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.600 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.600 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620917 s, 169 MB/s 2024-04-25T08:37:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623006 s, 168 MB/s 2024-04-25T08:37:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621261 s, 169 MB/s 2024-04-25T08:37:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598313 s, 175 MB/s 2024-04-25T08:37:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597555 s, 175 MB/s 2024-04-25T08:37:42.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584466 s, 179 MB/s 2024-04-25T08:37:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.643 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714775 s, 147 MB/s 2024-04-25T08:37:42.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617638 s, 170 MB/s 2024-04-25T08:37:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.657 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594316 s, 176 MB/s 2024-04-25T08:37:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.664 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653601 s, 160 MB/s 2024-04-25T08:37:42.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.672 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626581 s, 167 MB/s 2024-04-25T08:37:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.678 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622085 s, 169 MB/s 2024-04-25T08:37:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628539 s, 167 MB/s 2024-04-25T08:37:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.692 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055669 s, 188 MB/s 2024-04-25T08:37:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618334 s, 170 MB/s 2024-04-25T08:37:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715541 s, 147 MB/s 2024-04-25T08:37:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691181 s, 152 MB/s 2024-04-25T08:37:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680668 s, 154 MB/s 2024-04-25T08:37:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634115 s, 165 MB/s 2024-04-25T08:37:42.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573465 s, 183 MB/s 2024-04-25T08:37:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568782 s, 184 MB/s 2024-04-25T08:37:42.750 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616143 s, 170 MB/s 2024-04-25T08:37:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.759 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700253 s, 150 MB/s 2024-04-25T08:37:42.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:42.760 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:42.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.825 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623197 s, 168 MB/s 2024-04-25T08:37:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601632 s, 174 MB/s 2024-04-25T08:37:42.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.840 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711122 s, 147 MB/s 2024-04-25T08:37:42.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.850 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630211 s, 166 MB/s 2024-04-25T08:37:42.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.854 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644238 s, 163 MB/s 2024-04-25T08:37:42.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557043 s, 188 MB/s 2024-04-25T08:37:42.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.868 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637335 s, 165 MB/s 2024-04-25T08:37:42.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.875 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639831 s, 164 MB/s 2024-04-25T08:37:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715419 s, 147 MB/s 2024-04-25T08:37:42.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625386 s, 168 MB/s 2024-04-25T08:37:42.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633261 s, 166 MB/s 2024-04-25T08:37:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063018 s, 166 MB/s 2024-04-25T08:37:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661852 s, 158 MB/s 2024-04-25T08:37:42.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591729 s, 177 MB/s 2024-04-25T08:37:42.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.926 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637923 s, 164 MB/s 2024-04-25T08:37:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648288 s, 162 MB/s 2024-04-25T08:37:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.941 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660072 s, 159 MB/s 2024-04-25T08:37:42.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.948 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060856 s, 172 MB/s 2024-04-25T08:37:42.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.956 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621115 s, 169 MB/s 2024-04-25T08:37:42.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.962 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631474 s, 166 MB/s 2024-04-25T08:37:42.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.971 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707917 s, 148 MB/s 2024-04-25T08:37:42.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.978 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623995 s, 168 MB/s 2024-04-25T08:37:42.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701671 s, 149 MB/s 2024-04-25T08:37:42.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:42.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:42.993 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696462 s, 151 MB/s 2024-04-25T08:37:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653809 s, 160 MB/s 2024-04-25T08:37:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747937 s, 140 MB/s 2024-04-25T08:37:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.017 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665543 s, 158 MB/s 2024-04-25T08:37:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582582 s, 180 MB/s 2024-04-25T08:37:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.031 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657965 s, 159 MB/s 2024-04-25T08:37:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066356 s, 158 MB/s 2024-04-25T08:37:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598662 s, 175 MB/s 2024-04-25T08:37:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.053 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715998 s, 146 MB/s 2024-04-25T08:37:43.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:43.055 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.120 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615029 s, 170 MB/s 2024-04-25T08:37:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.126 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574442 s, 183 MB/s 2024-04-25T08:37:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.134 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065908 s, 159 MB/s 2024-04-25T08:37:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.141 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656922 s, 160 MB/s 2024-04-25T08:37:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609454 s, 172 MB/s 2024-04-25T08:37:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637171 s, 165 MB/s 2024-04-25T08:37:43.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597404 s, 176 MB/s 2024-04-25T08:37:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.169 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615277 s, 170 MB/s 2024-04-25T08:37:43.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.178 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706372 s, 148 MB/s 2024-04-25T08:37:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734504 s, 143 MB/s 2024-04-25T08:37:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763529 s, 137 MB/s 2024-04-25T08:37:43.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657426 s, 159 MB/s 2024-04-25T08:37:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716811 s, 146 MB/s 2024-04-25T08:37:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616125 s, 170 MB/s 2024-04-25T08:37:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606378 s, 173 MB/s 2024-04-25T08:37:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.231 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624021 s, 168 MB/s 2024-04-25T08:37:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.238 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619039 s, 169 MB/s 2024-04-25T08:37:43.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.246 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625502 s, 168 MB/s 2024-04-25T08:37:43.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705955 s, 149 MB/s 2024-04-25T08:37:43.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.260 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615874 s, 170 MB/s 2024-04-25T08:37:43.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619764 s, 169 MB/s 2024-04-25T08:37:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.276 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719945 s, 146 MB/s 2024-04-25T08:37:43.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738212 s, 142 MB/s 2024-04-25T08:37:43.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.290 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.290 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569446 s, 184 MB/s 2024-04-25T08:37:43.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624124 s, 168 MB/s 2024-04-25T08:37:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652768 s, 161 MB/s 2024-04-25T08:37:43.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.312 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642194 s, 163 MB/s 2024-04-25T08:37:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.319 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566569 s, 185 MB/s 2024-04-25T08:37:43.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.326 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622116 s, 169 MB/s 2024-04-25T08:37:43.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.334 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734503 s, 143 MB/s 2024-04-25T08:37:43.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.341 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606238 s, 173 MB/s 2024-04-25T08:37:43.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665609 s, 158 MB/s 2024-04-25T08:37:43.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:43.350 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660384 s, 159 MB/s 2024-04-25T08:37:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711095 s, 147 MB/s 2024-04-25T08:37:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.430 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675761 s, 155 MB/s 2024-04-25T08:37:43.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.438 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.438 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709831 s, 148 MB/s 2024-04-25T08:37:43.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.446 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672858 s, 156 MB/s 2024-04-25T08:37:43.453 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.453 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.453 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614918 s, 171 MB/s 2024-04-25T08:37:43.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.460 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623414 s, 168 MB/s 2024-04-25T08:37:43.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740065 s, 142 MB/s 2024-04-25T08:37:43.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.476 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061926 s, 169 MB/s 2024-04-25T08:37:43.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.483 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059906 s, 175 MB/s 2024-04-25T08:37:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600804 s, 175 MB/s 2024-04-25T08:37:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.496 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632499 s, 166 MB/s 2024-04-25T08:37:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627556 s, 167 MB/s 2024-04-25T08:37:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612773 s, 171 MB/s 2024-04-25T08:37:43.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.518 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623191 s, 168 MB/s 2024-04-25T08:37:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704337 s, 149 MB/s 2024-04-25T08:37:43.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642644 s, 163 MB/s 2024-04-25T08:37:43.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.541 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728256 s, 144 MB/s 2024-04-25T08:37:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.548 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618942 s, 169 MB/s 2024-04-25T08:37:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.555 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058955 s, 178 MB/s 2024-04-25T08:37:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.562 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646022 s, 162 MB/s 2024-04-25T08:37:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.569 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616591 s, 170 MB/s 2024-04-25T08:37:43.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.576 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566779 s, 185 MB/s 2024-04-25T08:37:43.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716229 s, 146 MB/s 2024-04-25T08:37:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.591 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644673 s, 163 MB/s 2024-04-25T08:37:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.599 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714065 s, 147 MB/s 2024-04-25T08:37:43.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.606 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611216 s, 172 MB/s 2024-04-25T08:37:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701863 s, 149 MB/s 2024-04-25T08:37:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064232 s, 163 MB/s 2024-04-25T08:37:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630022 s, 166 MB/s 2024-04-25T08:37:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614446 s, 171 MB/s 2024-04-25T08:37:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069589 s, 151 MB/s 2024-04-25T08:37:43.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:43.644 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:43.644 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:43.644 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:37:43.645 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:43.645 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:43.645 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:43.645 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:43.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.708 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578705 s, 181 MB/s 2024-04-25T08:37:43.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587733 s, 178 MB/s 2024-04-25T08:37:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072139 s, 145 MB/s 2024-04-25T08:37:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655968 s, 160 MB/s 2024-04-25T08:37:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711014 s, 147 MB/s 2024-04-25T08:37:43.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.746 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.746 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651628 s, 161 MB/s 2024-04-25T08:37:43.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.754 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071476 s, 147 MB/s 2024-04-25T08:37:43.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618298 s, 170 MB/s 2024-04-25T08:37:43.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597824 s, 175 MB/s 2024-04-25T08:37:43.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669063 s, 157 MB/s 2024-04-25T08:37:43.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631198 s, 166 MB/s 2024-04-25T08:37:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.790 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642108 s, 163 MB/s 2024-04-25T08:37:43.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.797 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621245 s, 169 MB/s 2024-04-25T08:37:43.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.805 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717817 s, 146 MB/s 2024-04-25T08:37:43.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.813 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712253 s, 147 MB/s 2024-04-25T08:37:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.820 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614471 s, 171 MB/s 2024-04-25T08:37:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.827 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622138 s, 169 MB/s 2024-04-25T08:37:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.834 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637814 s, 164 MB/s 2024-04-25T08:37:43.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.841 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063421 s, 165 MB/s 2024-04-25T08:37:43.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.849 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649723 s, 161 MB/s 2024-04-25T08:37:43.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.857 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739219 s, 142 MB/s 2024-04-25T08:37:43.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.865 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707475 s, 148 MB/s 2024-04-25T08:37:43.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.872 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.872 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621541 s, 169 MB/s 2024-04-25T08:37:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.879 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647181 s, 162 MB/s 2024-04-25T08:37:43.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.886 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612188 s, 171 MB/s 2024-04-25T08:37:43.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.894 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646908 s, 162 MB/s 2024-04-25T08:37:43.902 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.902 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.902 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720886 s, 145 MB/s 2024-04-25T08:37:43.909 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.909 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.909 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063299 s, 166 MB/s 2024-04-25T08:37:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.917 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720561 s, 146 MB/s 2024-04-25T08:37:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.924 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577388 s, 182 MB/s 2024-04-25T08:37:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.931 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619988 s, 169 MB/s 2024-04-25T08:37:43.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:43.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:43.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714879 s, 147 MB/s 2024-04-25T08:37:43.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721727 s, 145 MB/s 2024-04-25T08:37:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.014 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654261 s, 160 MB/s 2024-04-25T08:37:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.022 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072552 s, 145 MB/s 2024-04-25T08:37:44.029 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.029 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.029 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617654 s, 170 MB/s 2024-04-25T08:37:44.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.036 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600712 s, 175 MB/s 2024-04-25T08:37:44.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.044 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715209 s, 147 MB/s 2024-04-25T08:37:44.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.051 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634015 s, 165 MB/s 2024-04-25T08:37:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731905 s, 143 MB/s 2024-04-25T08:37:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.067 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696129 s, 151 MB/s 2024-04-25T08:37:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743842 s, 141 MB/s 2024-04-25T08:37:44.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656626 s, 160 MB/s 2024-04-25T08:37:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.090 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066338 s, 158 MB/s 2024-04-25T08:37:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.098 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720954 s, 145 MB/s 2024-04-25T08:37:44.106 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.106 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.106 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643282 s, 163 MB/s 2024-04-25T08:37:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569692 s, 184 MB/s 2024-04-25T08:37:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.120 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644304 s, 163 MB/s 2024-04-25T08:37:44.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.127 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653282 s, 161 MB/s 2024-04-25T08:37:44.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.135 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652154 s, 161 MB/s 2024-04-25T08:37:44.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.142 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640574 s, 164 MB/s 2024-04-25T08:37:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.149 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630691 s, 166 MB/s 2024-04-25T08:37:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.156 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062946 s, 167 MB/s 2024-04-25T08:37:44.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.163 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.163 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607567 s, 173 MB/s 2024-04-25T08:37:44.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.170 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617257 s, 170 MB/s 2024-04-25T08:37:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.179 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720341 s, 146 MB/s 2024-04-25T08:37:44.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.185 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598427 s, 175 MB/s 2024-04-25T08:37:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.193 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617694 s, 170 MB/s 2024-04-25T08:37:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629907 s, 166 MB/s 2024-04-25T08:37:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622219 s, 169 MB/s 2024-04-25T08:37:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590292 s, 178 MB/s 2024-04-25T08:37:44.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712558 s, 147 MB/s 2024-04-25T08:37:44.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.230 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.230 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707248 s, 148 MB/s 2024-04-25T08:37:44.237 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.237 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.237 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660961 s, 159 MB/s 2024-04-25T08:37:44.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:44.238 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:44.238 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:44.239 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:37:44.239 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:44.239 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:44.239 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:44.239 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:44.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.304 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654152 s, 160 MB/s 2024-04-25T08:37:44.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573625 s, 183 MB/s 2024-04-25T08:37:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.317 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598416 s, 175 MB/s 2024-04-25T08:37:44.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.324 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599703 s, 175 MB/s 2024-04-25T08:37:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.331 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656843 s, 160 MB/s 2024-04-25T08:37:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758891 s, 138 MB/s 2024-04-25T08:37:44.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.348 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655548 s, 160 MB/s 2024-04-25T08:37:44.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.355 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638723 s, 164 MB/s 2024-04-25T08:37:44.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.362 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599411 s, 175 MB/s 2024-04-25T08:37:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.369 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640152 s, 164 MB/s 2024-04-25T08:37:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.377 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703347 s, 149 MB/s 2024-04-25T08:37:44.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.384 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611891 s, 171 MB/s 2024-04-25T08:37:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.391 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627408 s, 167 MB/s 2024-04-25T08:37:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.399 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712936 s, 147 MB/s 2024-04-25T08:37:44.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.407 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706995 s, 148 MB/s 2024-04-25T08:37:44.414 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.414 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.414 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627944 s, 167 MB/s 2024-04-25T08:37:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.422 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740591 s, 142 MB/s 2024-04-25T08:37:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.430 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630933 s, 166 MB/s 2024-04-25T08:37:44.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612027 s, 171 MB/s 2024-04-25T08:37:44.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600209 s, 175 MB/s 2024-04-25T08:37:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611844 s, 171 MB/s 2024-04-25T08:37:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.459 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714802 s, 147 MB/s 2024-04-25T08:37:44.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.466 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608546 s, 172 MB/s 2024-04-25T08:37:44.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.474 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725334 s, 145 MB/s 2024-04-25T08:37:44.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.481 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607887 s, 172 MB/s 2024-04-25T08:37:44.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612889 s, 171 MB/s 2024-04-25T08:37:44.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564767 s, 186 MB/s 2024-04-25T08:37:44.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.502 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658289 s, 159 MB/s 2024-04-25T08:37:44.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711078 s, 147 MB/s 2024-04-25T08:37:44.516 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604267 s, 174 MB/s 2024-04-25T08:37:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609712 s, 172 MB/s 2024-04-25T08:37:44.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.531 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623795 s, 168 MB/s 2024-04-25T08:37:44.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:44.532 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727178 s, 144 MB/s 2024-04-25T08:37:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.604 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626673 s, 167 MB/s 2024-04-25T08:37:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634366 s, 165 MB/s 2024-04-25T08:37:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626896 s, 167 MB/s 2024-04-25T08:37:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.625 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575356 s, 182 MB/s 2024-04-25T08:37:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.633 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071396 s, 147 MB/s 2024-04-25T08:37:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.640 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583121 s, 180 MB/s 2024-04-25T08:37:44.647 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.647 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.647 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062906 s, 167 MB/s 2024-04-25T08:37:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.654 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.655 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637238 s, 165 MB/s 2024-04-25T08:37:44.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.662 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643181 s, 163 MB/s 2024-04-25T08:37:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749263 s, 140 MB/s 2024-04-25T08:37:44.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.677 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609095 s, 172 MB/s 2024-04-25T08:37:44.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.685 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623495 s, 168 MB/s 2024-04-25T08:37:44.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.692 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006683 s, 157 MB/s 2024-04-25T08:37:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606313 s, 173 MB/s 2024-04-25T08:37:44.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668232 s, 157 MB/s 2024-04-25T08:37:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.714 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631486 s, 166 MB/s 2024-04-25T08:37:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618852 s, 169 MB/s 2024-04-25T08:37:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623515 s, 168 MB/s 2024-04-25T08:37:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.736 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645772 s, 162 MB/s 2024-04-25T08:37:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618306 s, 170 MB/s 2024-04-25T08:37:44.750 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706201 s, 148 MB/s 2024-04-25T08:37:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.758 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708839 s, 148 MB/s 2024-04-25T08:37:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.765 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597471 s, 176 MB/s 2024-04-25T08:37:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.773 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691157 s, 152 MB/s 2024-04-25T08:37:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.781 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705183 s, 149 MB/s 2024-04-25T08:37:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.788 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612048 s, 171 MB/s 2024-04-25T08:37:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603748 s, 174 MB/s 2024-04-25T08:37:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669851 s, 157 MB/s 2024-04-25T08:37:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.812 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786691 s, 133 MB/s 2024-04-25T08:37:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.819 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610128 s, 172 MB/s 2024-04-25T08:37:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.826 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064308 s, 163 MB/s 2024-04-25T08:37:44.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:44.827 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:44.827 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:44.828 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:37:44.828 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:44.828 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:44.828 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:44.828 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:44.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.929 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623067 s, 168 MB/s 2024-04-25T08:37:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609172 s, 172 MB/s 2024-04-25T08:37:44.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664157 s, 158 MB/s 2024-04-25T08:37:44.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600907 s, 174 MB/s 2024-04-25T08:37:44.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.958 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653624 s, 160 MB/s 2024-04-25T08:37:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.966 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664961 s, 158 MB/s 2024-04-25T08:37:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.974 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734216 s, 143 MB/s 2024-04-25T08:37:44.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695874 s, 151 MB/s 2024-04-25T08:37:44.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.991 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757341 s, 138 MB/s 2024-04-25T08:37:44.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:44.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:44.998 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658392 s, 159 MB/s 2024-04-25T08:37:45.005 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.005 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.005 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627631 s, 167 MB/s 2024-04-25T08:37:45.012 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.012 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.012 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602753 s, 174 MB/s 2024-04-25T08:37:45.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.019 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585537 s, 179 MB/s 2024-04-25T08:37:45.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067222 s, 156 MB/s 2024-04-25T08:37:45.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061167 s, 171 MB/s 2024-04-25T08:37:45.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.041 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.041 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701579 s, 149 MB/s 2024-04-25T08:37:45.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.048 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637657 s, 164 MB/s 2024-04-25T08:37:45.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.055 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603275 s, 174 MB/s 2024-04-25T08:37:45.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.063 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672794 s, 156 MB/s 2024-04-25T08:37:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610818 s, 172 MB/s 2024-04-25T08:37:45.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.078 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676782 s, 155 MB/s 2024-04-25T08:37:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.084 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588999 s, 178 MB/s 2024-04-25T08:37:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060833 s, 172 MB/s 2024-04-25T08:37:45.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060889 s, 172 MB/s 2024-04-25T08:37:45.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565804 s, 185 MB/s 2024-04-25T08:37:45.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635621 s, 165 MB/s 2024-04-25T08:37:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.119 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614259 s, 171 MB/s 2024-04-25T08:37:45.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.127 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634599 s, 165 MB/s 2024-04-25T08:37:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600512 s, 175 MB/s 2024-04-25T08:37:45.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.142 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731969 s, 143 MB/s 2024-04-25T08:37:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566979 s, 185 MB/s 2024-04-25T08:37:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.156 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638218 s, 164 MB/s 2024-04-25T08:37:45.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:45.157 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706108 s, 149 MB/s 2024-04-25T08:37:45.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663308 s, 158 MB/s 2024-04-25T08:37:45.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071333 s, 147 MB/s 2024-04-25T08:37:45.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.205 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584525 s, 179 MB/s 2024-04-25T08:37:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.212 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621994 s, 169 MB/s 2024-04-25T08:37:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654002 s, 160 MB/s 2024-04-25T08:37:45.225 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574464 s, 183 MB/s 2024-04-25T08:37:45.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.233 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.233 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598084 s, 175 MB/s 2024-04-25T08:37:45.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.240 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603376 s, 174 MB/s 2024-04-25T08:37:45.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639154 s, 164 MB/s 2024-04-25T08:37:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.254 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627248 s, 167 MB/s 2024-04-25T08:37:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.261 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606221 s, 173 MB/s 2024-04-25T08:37:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635534 s, 165 MB/s 2024-04-25T08:37:45.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.275 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607297 s, 173 MB/s 2024-04-25T08:37:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.282 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614481 s, 171 MB/s 2024-04-25T08:37:45.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599086 s, 175 MB/s 2024-04-25T08:37:45.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.297 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.297 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648096 s, 162 MB/s 2024-04-25T08:37:45.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.303 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574541 s, 183 MB/s 2024-04-25T08:37:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625355 s, 168 MB/s 2024-04-25T08:37:45.318 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.318 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.318 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714056 s, 147 MB/s 2024-04-25T08:37:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.325 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601899 s, 174 MB/s 2024-04-25T08:37:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.332 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608787 s, 172 MB/s 2024-04-25T08:37:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.339 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618391 s, 170 MB/s 2024-04-25T08:37:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615316 s, 170 MB/s 2024-04-25T08:37:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.353 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570453 s, 184 MB/s 2024-04-25T08:37:45.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072444 s, 145 MB/s 2024-04-25T08:37:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698006 s, 150 MB/s 2024-04-25T08:37:45.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.377 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724306 s, 145 MB/s 2024-04-25T08:37:45.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.384 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612323 s, 171 MB/s 2024-04-25T08:37:45.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.392 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740635 s, 142 MB/s 2024-04-25T08:37:45.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.399 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573988 s, 183 MB/s 2024-04-25T08:37:45.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.407 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615516 s, 170 MB/s 2024-04-25T08:37:45.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:45.407 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:45.407 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:45.407 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:37:45.408 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:45.408 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:45.408 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:45.408 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:45.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.472 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744476 s, 141 MB/s 2024-04-25T08:37:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.479 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607976 s, 172 MB/s 2024-04-25T08:37:45.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.486 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.487 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634187 s, 165 MB/s 2024-04-25T08:37:45.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.494 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060153 s, 174 MB/s 2024-04-25T08:37:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706799 s, 148 MB/s 2024-04-25T08:37:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.509 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674641 s, 155 MB/s 2024-04-25T08:37:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.516 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570046 s, 184 MB/s 2024-04-25T08:37:45.523 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.523 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.523 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646966 s, 162 MB/s 2024-04-25T08:37:45.530 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.530 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.530 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609097 s, 172 MB/s 2024-04-25T08:37:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747024 s, 140 MB/s 2024-04-25T08:37:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.546 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701284 s, 150 MB/s 2024-04-25T08:37:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065632 s, 160 MB/s 2024-04-25T08:37:45.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.562 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748066 s, 140 MB/s 2024-04-25T08:37:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.569 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627201 s, 167 MB/s 2024-04-25T08:37:45.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.577 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749699 s, 140 MB/s 2024-04-25T08:37:45.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.585 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.585 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626209 s, 167 MB/s 2024-04-25T08:37:45.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.592 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612363 s, 171 MB/s 2024-04-25T08:37:45.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.598 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587622 s, 178 MB/s 2024-04-25T08:37:45.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586289 s, 179 MB/s 2024-04-25T08:37:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660568 s, 159 MB/s 2024-04-25T08:37:45.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716243 s, 146 MB/s 2024-04-25T08:37:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629071 s, 167 MB/s 2024-04-25T08:37:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.636 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072493 s, 145 MB/s 2024-04-25T08:37:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710525 s, 148 MB/s 2024-04-25T08:37:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581847 s, 180 MB/s 2024-04-25T08:37:45.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.657 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564193 s, 186 MB/s 2024-04-25T08:37:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.665 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624234 s, 168 MB/s 2024-04-25T08:37:45.672 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.672 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.672 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665946 s, 157 MB/s 2024-04-25T08:37:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615733 s, 170 MB/s 2024-04-25T08:37:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583592 s, 180 MB/s 2024-04-25T08:37:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746689 s, 140 MB/s 2024-04-25T08:37:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578416 s, 181 MB/s 2024-04-25T08:37:45.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:45.702 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:37:45.702 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:37:45.702 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:37:45.702 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:37:45.702 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:37:45.702 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:37:45.703 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:37:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671205 s, 156 MB/s 2024-04-25T08:37:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623127 s, 168 MB/s 2024-04-25T08:37:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617283 s, 170 MB/s 2024-04-25T08:37:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.789 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610864 s, 172 MB/s 2024-04-25T08:37:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.796 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061817 s, 170 MB/s 2024-04-25T08:37:45.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.804 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071003 s, 148 MB/s 2024-04-25T08:37:45.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.812 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.812 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006547 s, 160 MB/s 2024-04-25T08:37:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653235 s, 161 MB/s 2024-04-25T08:37:45.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.827 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628422 s, 167 MB/s 2024-04-25T08:37:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623981 s, 168 MB/s 2024-04-25T08:37:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064141 s, 163 MB/s 2024-04-25T08:37:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639446 s, 164 MB/s 2024-04-25T08:37:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.856 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721298 s, 145 MB/s 2024-04-25T08:37:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653165 s, 161 MB/s 2024-04-25T08:37:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057929 s, 181 MB/s 2024-04-25T08:37:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.878 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642439 s, 163 MB/s 2024-04-25T08:37:45.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.886 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071879 s, 146 MB/s 2024-04-25T08:37:45.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.894 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070304 s, 149 MB/s 2024-04-25T08:37:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.901 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635633 s, 165 MB/s 2024-04-25T08:37:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619035 s, 169 MB/s 2024-04-25T08:37:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.916 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706566 s, 148 MB/s 2024-04-25T08:37:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.924 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716234 s, 146 MB/s 2024-04-25T08:37:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.932 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694358 s, 151 MB/s 2024-04-25T08:37:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620308 s, 169 MB/s 2024-04-25T08:37:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.946 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618588 s, 170 MB/s 2024-04-25T08:37:45.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.953 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591695 s, 177 MB/s 2024-04-25T08:37:45.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.960 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636245 s, 165 MB/s 2024-04-25T08:37:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718341 s, 146 MB/s 2024-04-25T08:37:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715662 s, 147 MB/s 2024-04-25T08:37:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.984 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683248 s, 153 MB/s 2024-04-25T08:37:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666385 s, 157 MB/s 2024-04-25T08:37:45.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:37:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:37:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570962 s, 184 MB/s 2024-04-25T08:37:45.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:45.999 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:37:46.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.150+0000 7fd028a07640 1 Processor -- start 2024-04-25T08:37:46.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.150+0000 7fd028a07640 1 -- start start 2024-04-25T08:37:46.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd028a07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0240d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd028a07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd0240ced80 0x7fd0240b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd028a07640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240d2120 con 0x7fd0240d1ce0 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd028a07640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0240d23a0 con 0x7fd0240ced80 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd028a07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240d2680 con 0x7fd02415dc20 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd022d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd0240ced80 0x7fd0240b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd02259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0240d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd01bfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd0240d1ce0 0x7fd0240ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57168/0 (socket says 172.21.15.60:57168) 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd01bfff640 1 -- 172.21.15.60:0/3076246482 learned_addr learned my addr 172.21.15.60:0/3076246482 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.151+0000 7fd02259c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0240d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45046/0 (socket says 172.21.15.60:45046) 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd02259c640 1 -- 172.21.15.60:0/3076246482 >> v2:172.21.15.100:3300/0 conn(0x7fd0240ced80 msgr2=0x7fd0240b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd02259c640 1 --2- 172.21.15.60:0/3076246482 >> v2:172.21.15.100:3300/0 conn(0x7fd0240ced80 0x7fd0240b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd02259c640 1 -- 172.21.15.60:0/3076246482 >> v1:172.21.15.100:6789/0 conn(0x7fd0240d1ce0 legacy=0x7fd0240ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd02259c640 1 -- 172.21.15.60:0/3076246482 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240d2960 con 0x7fd02415dc20 2024-04-25T08:37:46.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd022d9d640 1 --2- 172.21.15.60:0/3076246482 >> v2:172.21.15.100:3300/0 conn(0x7fd0240ced80 0x7fd0240b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd02259c640 1 --2- 172.21.15.60:0/3076246482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0240d3440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097d0 tx=0x7fd01402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0c162e16da4c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd021d9b640 1 -- 172.21.15.60:0/3076246482 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014037070 con 0x7fd02415dc20 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.152+0000 7fd021d9b640 1 -- 172.21.15.60:0/3076246482 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01402ae40 con 0x7fd02415dc20 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd021d9b640 1 -- 172.21.15.60:0/3076246482 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014032660 con 0x7fd02415dc20 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 -- 172.21.15.60:0/3076246482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 msgr2=0x7fd0240d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 --2- 172.21.15.60:0/3076246482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0240d3440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd0140097d0 tx=0x7fd01402a130 comp rx=0 tx=0).stop 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 -- 172.21.15.60:0/3076246482 shutdown_connections 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 --2- 172.21.15.60:0/3076246482 >> v2:172.21.15.100:3300/0 conn(0x7fd0240ced80 0x7fd0240b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 --2- 172.21.15.60:0/3076246482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0240d3440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 -- 172.21.15.60:0/3076246482 >> 172.21.15.60:0/3076246482 conn(0x7fd024061a20 msgr2=0x7fd024061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 -- 172.21.15.60:0/3076246482 shutdown_connections 2024-04-25T08:37:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.153+0000 7fd028a07640 1 -- 172.21.15.60:0/3076246482 wait complete. 2024-04-25T08:37:46.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.154+0000 7fd028a07640 1 Processor -- start 2024-04-25T08:37:46.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 -- start start 2024-04-25T08:37:46.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0240ced80 0x7fd02415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 0x7fd0240c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0241607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240b6a30 con 0x7fd0240d1ce0 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240d2f20 con 0x7fd0240ced80 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd028a07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240b6bb0 con 0x7fd02415dc20 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 0x7fd0240c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 0x7fd0240c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54258/0 (socket says 172.21.15.60:54258) 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.155+0000 7fd01bfff640 1 -- 172.21.15.60:0/1298095838 learned_addr learned my addr 172.21.15.60:0/1298095838 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd02259c640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0240ced80 0x7fd02415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd022d9d640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0241607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01bfff640 1 -- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 msgr2=0x7fd0241607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0241607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01bfff640 1 -- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0240ced80 msgr2=0x7fd02415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0240ced80 0x7fd02415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01bfff640 1 -- 172.21.15.60:0/1298095838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024148870 con 0x7fd0240d1ce0 2024-04-25T08:37:46.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 0x7fd0240c6450 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00c400 tx=0x7fd00c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01affd640 1 -- 172.21.15.60:0/1298095838 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c017070 con 0x7fd0240d1ce0 2024-04-25T08:37:46.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01affd640 1 -- 172.21.15.60:0/1298095838 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c005ce0 con 0x7fd0240d1ce0 2024-04-25T08:37:46.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd024148b50 con 0x7fd0240d1ce0 2024-04-25T08:37:46.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd01affd640 1 -- 172.21.15.60:0/1298095838 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0072f0 con 0x7fd0240d1ce0 2024-04-25T08:37:46.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.156+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd024149110 con 0x7fd0240d1ce0 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.158+0000 7fd01affd640 1 -- 172.21.15.60:0/1298095838 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd00c01a070 con 0x7fd0240d1ce0 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.158+0000 7fd01affd640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd00406b4d0 0x7fd00406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.158+0000 7fd01affd640 1 -- 172.21.15.60:0/1298095838 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd00c089790 con 0x7fd0240d1ce0 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.159+0000 7fd02259c640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd00406b4d0 0x7fd00406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.159+0000 7fd02259c640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd00406b4d0 0x7fd00406d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd014034000 tx=0x7fd01402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.159+0000 7fd02b7e8740 1 Processor -- start 2024-04-25T08:37:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.159+0000 7fd02b7e8740 1 -- start start 2024-04-25T08:37:46.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e2453200 0x5621e2455360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e24535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 0x5621e247e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e23b7000 con 0x5621e247dcc0 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e235ff20 con 0x5621e248f8d0 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e23b30a0 con 0x5621e2453200 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 0x5621e247e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 0x5621e247e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54274/0 (socket says 172.21.15.60:54274) 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd022d9d640 1 -- 172.21.15.60:0/1623484081 learned_addr learned my addr 172.21.15.60:0/1623484081 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd02259c640 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e2453200 0x5621e2455360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e24535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd01bfff640 1 -- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e2453200 msgr2=0x5621e2455360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e2453200 0x5621e2455360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd01bfff640 1 -- 172.21.15.60:0/1623484081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 msgr2=0x5621e247e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.160+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 0x5621e247e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd01bfff640 1 -- 172.21.15.60:0/1623484081 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621e2197ba0 con 0x5621e248f8d0 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd022d9d640 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 0x5621e247e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd01bfff640 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e24535e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd00c087a60 tx=0x7fd00c057790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a16e1a140164d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd003fff640 1 -- 172.21.15.60:0/1623484081 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c056000 con 0x5621e248f8d0 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd003fff640 1 -- 172.21.15.60:0/1623484081 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c054370 con 0x5621e248f8d0 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd003fff640 1 -- 172.21.15.60:0/1623484081 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0076d0 con 0x5621e248f8d0 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd02b7e8740 1 -- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 msgr2=0x5621e24535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.161+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e24535e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd00c087a60 tx=0x7fd00c057790 comp rx=0 tx=0).stop 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 -- 172.21.15.60:0/1623484081 shutdown_connections 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e247dcc0 0x5621e247e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e24535e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/1623484081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e2453200 0x5621e2455360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 -- 172.21.15.60:0/1623484081 >> 172.21.15.60:0/1623484081 conn(0x5621e241b050 msgr2=0x5621e2498cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 -- 172.21.15.60:0/1623484081 shutdown_connections 2024-04-25T08:37:46.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 -- 172.21.15.60:0/1623484081 wait complete. 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 -- start start 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.162+0000 7fd02b7e8740 1 Processor -- start 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e2491c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e2492150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e2355610 con 0x5621e248f8d0 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e247b810 con 0x5621e2453200 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e2481370 con 0x5621e247dcc0 2024-04-25T08:37:46.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02b7e8740 10 client.0 _ll_register_callbacks cb 0x5621e236df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54284/0 (socket says 172.21.15.60:54284) 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd022d9d640 1 -- 172.21.15.60:0/3320995849 learned_addr learned my addr 172.21.15.60:0/3320995849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd01bfff640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e2492150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.163+0000 7fd02259c640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e2491c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd022d9d640 1 -- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 msgr2=0x5621e2492150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd022d9d640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e2492150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd022d9d640 1 -- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 msgr2=0x5621e2491c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd022d9d640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e2491c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd022d9d640 1 -- 172.21.15.60:0/3320995849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621e208b490 con 0x5621e248f8d0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd022d9d640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24abff0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd01c007220 tx=0x7fd01c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19548675133d1874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd0017fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd0017fa640 1 -- 172.21.15.60:0/3320995849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c01f680 con 0x5621e248f8d0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd0017fa640 1 -- 172.21.15.60:0/3320995849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd01c01f820 con 0x5621e248f8d0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd00c054370 con 0x5621e248f8d0 2024-04-25T08:37:46.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd0017fa640 1 -- 172.21.15.60:0/3320995849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01c027050 con 0x5621e248f8d0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 10 client.8961 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5621e21f8330 con 0x5621e248f8d0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd0017fa640 1 -- 172.21.15.60:0/3320995849 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c054370 con 0x5621e248f8d0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd0017fa640 1 -- 172.21.15.60:0/3320995849 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd01c01fa20 con 0x5621e248f8d0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 10 client.8961 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 10 client.8961 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 20 client.8961 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e24944f0 0x5621e24ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 4 client.8961 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.164+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621e23c4540 con 0x5621e24944f0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.165+0000 7fd02259c640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e24944f0 0x5621e24ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.165+0000 7fd02259c640 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e24944f0 0x5621e24ac530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0c9081ff981f3a server_cookie=45971ae2f0713c99 in_seq=0 out_seq=0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.165+0000 7fd0017fa640 10 client.8961 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.165+0000 7fd0017fa640 1 -- 172.21.15.60:0/3320995849 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621e23c4540 con 0x5621e24944f0 2024-04-25T08:37:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.165+0000 7fd0017fa640 10 client.8961 handle_command_reply: tid=0 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 client.8961 shutdown 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 msgr2=0x5621e24abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24abff0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd01c007220 tx=0x7fd01c012670 comp rx=0 tx=0).stop 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 shutdown_connections 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e24944f0 0x5621e24ac530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24abff0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e2492150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd0017fa640 0 client.8961 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/3320995849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e2491c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 >> 172.21.15.60:0/3320995849 conn(0x5621e241b050 msgr2=0x5621e247ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 shutdown_connections 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 1 -- 172.21.15.60:0/3320995849 wait complete. 2024-04-25T08:37:46.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.166+0000 7fd02b7e8740 20 client.8961 trim_cache size 0 max 16384 2024-04-25T08:37:46.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.181+0000 7fd02b7e8740 1 Processor -- start 2024-04-25T08:37:46.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.181+0000 7fd02b7e8740 1 -- start start 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 0x5621e247b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e24816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e247c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0242005e0 con 0x5621e2453200 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0241ff7e0 con 0x5621e248f8d0 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240cb7d0 con 0x5621e247dcc0 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 0x5621e247b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 0x5621e247b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54286/0 (socket says 172.21.15.60:54286) 2024-04-25T08:37:46.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 -- 172.21.15.60:0/435457429 learned_addr learned my addr 172.21.15.60:0/435457429 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd01bfff640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e24816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd022d9d640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e247c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 -- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 msgr2=0x5621e24816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e24816f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 -- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 msgr2=0x5621e247c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e247c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 -- 172.21.15.60:0/435457429 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621e247a8a0 con 0x5621e2453200 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd022d9d640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e247c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd01bfff640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e24816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd02259c640 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 0x5621e247b4f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd0140098f0 tx=0x7fd01402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81097e26284e38e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.182+0000 7fd0037fe640 1 -- 172.21.15.60:0/435457429 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014037040 con 0x5621e2453200 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd0037fe640 1 -- 172.21.15.60:0/435457429 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0140328f0 con 0x5621e2453200 2024-04-25T08:37:46.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd0037fe640 1 -- 172.21.15.60:0/435457429 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014032ca0 con 0x5621e2453200 2024-04-25T08:37:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 -- 172.21.15.60:0/435457429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 msgr2=0x5621e247b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 0x5621e247b4f0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd0140098f0 tx=0x7fd01402c000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 -- 172.21.15.60:0/435457429 shutdown_connections 2024-04-25T08:37:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e248f8d0 0x5621e247c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e24816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.183 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/435457429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e2453200 0x5621e247b4f0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 -- 172.21.15.60:0/435457429 >> 172.21.15.60:0/435457429 conn(0x5621e241b050 msgr2=0x5621e2491e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 -- 172.21.15.60:0/435457429 shutdown_connections 2024-04-25T08:37:46.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.183+0000 7fd02b7e8740 1 -- 172.21.15.60:0/435457429 wait complete. 2024-04-25T08:37:46.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.184+0000 7fd02b7e8740 1 -- start start 2024-04-25T08:37:46.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.184+0000 7fd02b7e8740 1 Processor -- start 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e24b6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e248ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0241ff1e0 con 0x5621e248f8d0 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e24953d0 con 0x5621e2453200 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e24b6870 con 0x5621e247dcc0 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 10 client.0 _ll_register_callbacks cb 0x7fd00c00ceb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54302/0 (socket says 172.21.15.60:54302) 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 -- 172.21.15.60:0/2927645777 learned_addr learned my addr 172.21.15.60:0/2927645777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02259c640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e24b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd01bfff640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e248ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 -- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 msgr2=0x5621e248ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e248ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 -- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 msgr2=0x5621e24b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e24b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 -- 172.21.15.60:0/2927645777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621e2064610 con 0x5621e248f8d0 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02259c640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e24b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd022d9d640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24b2520 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x5621e247f5c0 tx=0x7fd01c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5f3771d6e922a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd0017fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd0017fa640 1 -- 172.21.15.60:0/2927645777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c01f480 con 0x5621e248f8d0 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd0017fa640 1 -- 172.21.15.60:0/2927645777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd01c01f620 con 0x5621e248f8d0 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0140328f0 con 0x5621e248f8d0 2024-04-25T08:37:46.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd0017fa640 1 -- 172.21.15.60:0/2927645777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01c03b050 con 0x5621e248f8d0 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 10 client.8967 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.185+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5621e248f650 con 0x5621e248f8d0 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd0017fa640 1 -- 172.21.15.60:0/2927645777 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0140328f0 con 0x5621e248f8d0 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd0017fa640 1 -- 172.21.15.60:0/2927645777 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd01c03e020 con 0x5621e248f8d0 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02b7e8740 10 client.8967 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02b7e8740 10 client.8967 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02b7e8740 20 client.8967 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e2503540 0x5621e24b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02b7e8740 4 client.8967 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5621e23be890 con 0x5621e2503540 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02259c640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e2503540 0x5621e24b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd0017fa640 10 client.8967 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.187 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.186+0000 7fd02259c640 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e2503540 0x5621e24b2a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e404b36e7888 server_cookie=516fd847c602b6f2 in_seq=0 out_seq=0 2024-04-25T08:37:46.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.188+0000 7fd0017fa640 1 -- 172.21.15.60:0/2927645777 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5621e23be890 con 0x5621e2503540 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.188+0000 7fd0017fa640 10 client.8967 handle_command_reply: tid=0 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.188+0000 7fd02b7e8740 1 client.8967 shutdown 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 msgr2=0x5621e24b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24b2520 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x5621e247f5c0 tx=0x7fd01c004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 shutdown_connections 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x5621e2503540 0x5621e24b2a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5621e248f8d0 0x5621e24b2520 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5621e247dcc0 0x5621e248ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd0017fa640 0 client.8967 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 --2- 172.21.15.60:0/2927645777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5621e2453200 0x5621e24b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 >> 172.21.15.60:0/2927645777 conn(0x5621e241b050 msgr2=0x5621e2481900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 shutdown_connections 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 1 -- 172.21.15.60:0/2927645777 wait complete. 2024-04-25T08:37:46.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd02b7e8740 20 client.8967 trim_cache size 0 max 16384 2024-04-25T08:37:46.190 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:37:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "27b078d0-e100-49f7-b1e4-4af885c05c8c", 2024-04-25T08:37:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:37:46.190 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd00406b4d0 msgr2=0x7fd00406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd00406b4d0 0x7fd00406d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd014034000 tx=0x7fd01402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 msgr2=0x7fd0240c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 0x7fd0240c6450 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00c400 tx=0x7fd00c00c800 comp rx=0 tx=0).stop 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 shutdown_connections 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd02415dc20 0x7fd0241607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0240d1ce0 0x7fd0240c6450 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd00406b4d0 0x7fd00406d990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 --2- 172.21.15.60:0/1298095838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0240ced80 0x7fd02415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 >> 172.21.15.60:0/1298095838 conn(0x7fd024061a20 msgr2=0x7fd0240c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.189+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 shutdown_connections 2024-04-25T08:37:46.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.190+0000 7fd028a07640 1 -- 172.21.15.60:0/1298095838 wait complete. 2024-04-25T08:37:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:46.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T08:37:46.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.316+0000 7f5eb7e07640 1 Processor -- start 2024-04-25T08:37:46.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.316+0000 7f5eb7e07640 1 -- start start 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb7e07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5eb0158230 0x7f5eb00d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb7e07640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00d2020 con 0x7f5eb00d1b80 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb7e07640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00d2300 con 0x7f5eb00b4e60 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb7e07640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb0158610 con 0x7f5eb0158230 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb5ba3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5eb0158230 0x7f5eb00d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb5ba3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5eb0158230 0x7f5eb00d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54312/0 (socket says 172.21.15.60:54312) 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb53a2640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5eb00d1b80 0x7f5eb006c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56200/0 (socket says 172.21.15.60:56200) 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/424214068 learned_addr learned my addr 172.21.15.60:0/424214068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/424214068 >> v1:172.21.15.100:6789/0 conn(0x7f5eb00b4e60 legacy=0x7f5eb006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/424214068 >> v1:172.21.15.60:6789/0 conn(0x7f5eb00d1b80 legacy=0x7f5eb006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.317+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/424214068 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb006bb30 con 0x7f5eb0158230 2024-04-25T08:37:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.318+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/424214068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb00d1640 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009870 tx=0x7f5eac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8045bf94c534ad8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.318+0000 7f5eb4ba1640 1 -- 172.21.15.60:0/424214068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac037070 con 0x7f5eb0158230 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.318+0000 7f5eb4ba1640 1 -- 172.21.15.60:0/424214068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac032440 con 0x7f5eb0158230 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.318+0000 7f5eb4ba1640 1 -- 172.21.15.60:0/424214068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac032810 con 0x7f5eb0158230 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 -- 172.21.15.60:0/424214068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 msgr2=0x7f5eb00d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/424214068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb00d1640 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009870 tx=0x7f5eac02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 -- 172.21.15.60:0/424214068 shutdown_connections 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/424214068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb00d1640 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 -- 172.21.15.60:0/424214068 >> 172.21.15.60:0/424214068 conn(0x7f5eb0061600 msgr2=0x7f5eb00619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 -- 172.21.15.60:0/424214068 shutdown_connections 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.319+0000 7f5eb7e07640 1 -- 172.21.15.60:0/424214068 wait complete. 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.320+0000 7f5eb7e07640 1 Processor -- start 2024-04-25T08:37:46.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.320+0000 7f5eb7e07640 1 -- start start 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb7e07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5eb00b4e60 0x7f5eb00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb7e07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eb00d1b80 0x7f5eb00cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb7e07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb014bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb7e07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb006ca40 con 0x7f5eb0158230 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb7e07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb006bea0 con 0x7f5eb00d1b80 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb7e07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb015e320 con 0x7f5eb00b4e60 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb014bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb014bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54328/0 (socket says 172.21.15.60:54328) 2024-04-25T08:37:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 -- 172.21.15.60:0/4288898095 learned_addr learned my addr 172.21.15.60:0/4288898095 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eb00d1b80 0x7f5eb00cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5eb00b4e60 0x7f5eb00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 -- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5eb00b4e60 msgr2=0x7f5eb00caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5eb00b4e60 0x7f5eb00caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 -- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eb00d1b80 msgr2=0x7f5eb00cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eb00d1b80 0x7f5eb00cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 -- 172.21.15.60:0/4288898095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00cdba0 con 0x7f5eb0158230 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.321+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb014bfe0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c700 tx=0x7f5ea800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.322+0000 7f5ea67fc640 1 -- 172.21.15.60:0/4288898095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8016020 con 0x7f5eb0158230 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.322+0000 7f5ea67fc640 1 -- 172.21.15.60:0/4288898095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800aa50 con 0x7f5eb0158230 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.322+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb00cb080 con 0x7f5eb0158230 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.322+0000 7f5ea67fc640 1 -- 172.21.15.60:0/4288898095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800ce10 con 0x7f5eb0158230 2024-04-25T08:37:46.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.322+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb00cb640 con 0x7f5eb0158230 2024-04-25T08:37:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.323+0000 7f5ea67fc640 1 -- 172.21.15.60:0/4288898095 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ea8019020 con 0x7f5eb0158230 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.324+0000 7f5ea67fc640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9406b4d0 0x7f5e9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.324+0000 7f5ea67fc640 1 -- 172.21.15.60:0/4288898095 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ea8089ef0 con 0x7f5eb0158230 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.324+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9406b4d0 0x7f5e9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.325+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9406b4d0 0x7f5e9406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.325+0000 7f5ebabe5740 1 Processor -- start 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.325+0000 7f5ebabe5740 1 -- start start 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 0x55df71522e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7155d370 0x55df71521080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7154b760 0x55df7154bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df71483b90 con 0x55df71520ca0 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df7142ce40 con 0x55df7155d370 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df7147fc30 con 0x55df7154b760 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 0x55df71522e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 0x55df71522e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54330/0 (socket says 172.21.15.60:54330) 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/286767024 learned_addr learned my addr 172.21.15.60:0/286767024 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7155d370 0x55df71521080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7154b760 0x55df7154bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7154b760 msgr2=0x55df7154bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7154b760 0x55df7154bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7155d370 msgr2=0x55df71521080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7155d370 0x55df71521080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 -- 172.21.15.60:0/286767024 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df712627d0 con 0x55df71520ca0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7155d370 0x55df71521080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 0x55df71522e00 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5eac035040 tx=0x7f5eac02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d89d47ff19f7e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5e937fe640 1 -- 172.21.15.60:0/286767024 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac037040 con 0x55df71520ca0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5e937fe640 1 -- 172.21.15.60:0/286767024 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac032df0 con 0x55df71520ca0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.326+0000 7f5e937fe640 1 -- 172.21.15.60:0/286767024 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac047630 con 0x55df71520ca0 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 -- 172.21.15.60:0/286767024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 msgr2=0x55df71522e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 0x55df71522e00 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5eac035040 tx=0x7f5eac02c000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 -- 172.21.15.60:0/286767024 shutdown_connections 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7154b760 0x55df7154bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7155d370 0x55df71521080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/286767024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df71520ca0 0x55df71522e00 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 -- 172.21.15.60:0/286767024 >> 172.21.15.60:0/286767024 conn(0x55df714e8870 msgr2=0x55df71566760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 -- 172.21.15.60:0/286767024 shutdown_connections 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.327+0000 7f5ebabe5740 1 -- 172.21.15.60:0/286767024 wait complete. 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.328+0000 7f5ebabe5740 1 -- start start 2024-04-25T08:37:46.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.328+0000 7f5ebabe5740 1 Processor -- start 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df7154e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df71560a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df714403e0 con 0x55df7154b760 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df715492b0 con 0x55df71520ca0 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df7155fdf0 con 0x55df7155d370 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 10 client.0 _ll_register_callbacks cb 0x55df7143fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df7154e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb5ba3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df7154e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54332/0 (socket says 172.21.15.60:54332) 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 -- 172.21.15.60:0/4264808719 learned_addr learned my addr 172.21.15.60:0/4264808719 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.328 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df71560a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 -- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 msgr2=0x55df71560a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df71560a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 -- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 msgr2=0x55df7153ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 -- 172.21.15.60:0/4264808719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df711580f0 con 0x55df7154b760 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df7154e080 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00b3390 tx=0x7f5ea0012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9dfb3bc93e268d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5e90ff9640 1 -- 172.21.15.60:0/4264808719 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0019020 con 0x55df7154b760 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5e90ff9640 1 -- 172.21.15.60:0/4264808719 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ea00072e0 con 0x55df7154b760 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.329+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5eac032df0 con 0x55df7154b760 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5e90ff9640 1 -- 172.21.15.60:0/4264808719 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea0023050 con 0x55df7154b760 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 10 client.8982 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:46.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55df712c2f60 con 0x55df7154b760 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5e90ff9640 1 -- 172.21.15.60:0/4264808719 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac032df0 con 0x55df7154b760 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5e90ff9640 1 -- 172.21.15.60:0/4264808719 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5ea0026020 con 0x55df7154b760 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 10 client.8982 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 10 client.8982 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 20 client.8982 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df71579d50 0x55df7157a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 4 client.8982 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df714910d0 con 0x55df71579d50 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df71579d50 0x55df7157a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5e90ff9640 10 client.8982 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.330+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df71579d50 0x55df7157a190 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f2668c448e4edc server_cookie=c7f5c806926a82fa in_seq=0 out_seq=0 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5e90ff9640 1 -- 172.21.15.60:0/4264808719 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df714910d0 con 0x55df71579d50 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5e90ff9640 10 client.8982 handle_command_reply: tid=0 2024-04-25T08:37:46.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 client.8982 shutdown 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 msgr2=0x55df7154e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df7154e080 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00b3390 tx=0x7f5ea0012690 comp rx=0 tx=0).stop 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 shutdown_connections 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df71579d50 0x55df7157a190 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df71560a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df7154e080 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/4264808719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5e90ff9640 0 client.8982 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 >> 172.21.15.60:0/4264808719 conn(0x55df714e8870 msgr2=0x55df71548aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 shutdown_connections 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 1 -- 172.21.15.60:0/4264808719 wait complete. 2024-04-25T08:37:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.331+0000 7f5ebabe5740 20 client.8982 trim_cache size 0 max 16384 2024-04-25T08:37:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.346+0000 7f5ebabe5740 1 Processor -- start 2024-04-25T08:37:46.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.346+0000 7f5ebabe5740 1 -- start start 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df71575e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df7155ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0200690 con 0x55df7154b760 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01ff890 con 0x55df71520ca0 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb015a8a0 con 0x55df7155d370 2024-04-25T08:37:46.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df71575e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df71575e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54334/0 (socket says 172.21.15.60:54334) 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 -- 172.21.15.60:0/376538396 learned_addr learned my addr 172.21.15.60:0/376538396 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df7155ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 -- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 msgr2=0x55df7155ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df7155ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 -- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 msgr2=0x55df7153cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 -- 172.21.15.60:0/376538396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df711311b0 con 0x55df7154b760 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df7155ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.347+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df71575e30 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00079a0 tx=0x7f5ea000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c279b2087d5297c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5e92ffd640 1 -- 172.21.15.60:0/376538396 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea002c070 con 0x55df7154b760 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5e92ffd640 1 -- 172.21.15.60:0/376538396 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea00057a0 con 0x55df7154b760 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5e92ffd640 1 -- 172.21.15.60:0/376538396 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0019070 con 0x55df7154b760 2024-04-25T08:37:46.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 -- 172.21.15.60:0/376538396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 msgr2=0x55df71575e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df71575e30 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00079a0 tx=0x7f5ea000c000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 -- 172.21.15.60:0/376538396 shutdown_connections 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df7155d370 0x55df7155ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7154b760 0x55df71575e30 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/376538396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df71520ca0 0x55df7153cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 -- 172.21.15.60:0/376538396 >> 172.21.15.60:0/376538396 conn(0x55df714e8870 msgr2=0x55df71547f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 -- 172.21.15.60:0/376538396 shutdown_connections 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.348+0000 7f5ebabe5740 1 -- 172.21.15.60:0/376538396 wait complete. 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.349+0000 7f5ebabe5740 1 -- start start 2024-04-25T08:37:46.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.349+0000 7f5ebabe5740 1 Processor -- start 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df71520ca0 0x55df715d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7154b760 0x55df7154c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 0x55df7154c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb015aea0 con 0x55df7155d370 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df715809d0 con 0x55df7154b760 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df715d1c00 con 0x55df71520ca0 2024-04-25T08:37:46.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5ebabe5740 10 client.0 _ll_register_callbacks cb 0x7f5ea0001300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 0x55df7154c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 0x55df7154c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54342/0 (socket says 172.21.15.60:54342) 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 -- 172.21.15.60:0/2588816935 learned_addr learned my addr 172.21.15.60:0/2588816935 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb53a2640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7154b760 0x55df7154c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df71520ca0 0x55df715d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 -- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df71520ca0 msgr2=0x55df715d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df71520ca0 0x55df715d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 -- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7154b760 msgr2=0x55df7154c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7154b760 0x55df7154c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.350+0000 7f5eb63a4640 1 -- 172.21.15.60:0/2588816935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df7155df70 con 0x55df7155d370 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5eb63a4640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 0x55df7154c6b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80881f0 tx=0x7f5ea805c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffa19b81975efd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:46.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5e90ff9640 1 -- 172.21.15.60:0/2588816935 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8085020 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5e90ff9640 1 -- 172.21.15.60:0/2588816935 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ea80606c0 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ea00057a0 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5e90ff9640 1 -- 172.21.15.60:0/2588816935 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea80608e0 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 10 client.8988 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55df715d19a0 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5e90ff9640 1 -- 172.21.15.60:0/2588816935 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea00057a0 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5e90ff9640 1 -- 172.21.15.60:0/2588816935 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5ea8081b50 con 0x55df7155d370 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 10 client.8988 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 10 client.8988 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 20 client.8988 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df7153d5c0 0x55df715d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 4 client.8988 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55df7148b420 con 0x55df7153d5c0 2024-04-25T08:37:46.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.351+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df7153d5c0 0x55df715d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5e90ff9640 10 client.8988 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5eb5ba3640 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df7153d5c0 0x55df715d0890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5e14086374dfe7 server_cookie=82f8ded035f72a03 in_seq=0 out_seq=0 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5e90ff9640 1 -- 172.21.15.60:0/2588816935 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55df7148b420 con 0x55df7153d5c0 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5e90ff9640 10 client.8988 handle_command_reply: tid=0 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 client.8988 shutdown 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 msgr2=0x55df7154c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 0x55df7154c6b0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80881f0 tx=0x7f5ea805c6d0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 shutdown_connections 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55df7153d5c0 0x55df715d0890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55df7155d370 0x55df7154c6b0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df7154b760 0x55df7154c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5e90ff9640 0 client.8988 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 --2- 172.21.15.60:0/2588816935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55df71520ca0 0x55df715d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 >> 172.21.15.60:0/2588816935 conn(0x55df714e8870 msgr2=0x55df7154de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 shutdown_connections 2024-04-25T08:37:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 1 -- 172.21.15.60:0/2588816935 wait complete. 2024-04-25T08:37:46.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.352+0000 7f5ebabe5740 20 client.8988 trim_cache size 0 max 16384 2024-04-25T08:37:46.353 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:46.353 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:37:46.353 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9406b4d0 msgr2=0x7f5e9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9406b4d0 0x7f5e9406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 msgr2=0x7f5eb014bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb014bfe0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c700 tx=0x7f5ea800cbc0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 shutdown_connections 2024-04-25T08:37:46.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5eb0158230 0x7f5eb014bfe0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e9406b4d0 0x7f5e9406d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5eb00d1b80 0x7f5eb00cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 --2- 172.21.15.60:0/4288898095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5eb00b4e60 0x7f5eb00caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 >> 172.21.15.60:0/4288898095 conn(0x7f5eb0061600 msgr2=0x7f5eb01570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 shutdown_connections 2024-04-25T08:37:46.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.353+0000 7f5eb7e07640 1 -- 172.21.15.60:0/4288898095 wait complete. 2024-04-25T08:37:46.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:46.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T08:37:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.466+0000 7fa2c86f5640 1 Processor -- start 2024-04-25T08:37:46.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.466+0000 7fa2c86f5640 1 -- start start 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c86f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2c006c9e0 0x7fa2c00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c86f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c006cf50 con 0x7fa2c015b7d0 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c86f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00ce530 con 0x7fa2c00b5620 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c86f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c00ce810 con 0x7fa2c006c9e0 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c6491640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa2c015b7d0 0x7fa2c00cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56212/0 (socket says 172.21.15.60:56212) 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c6491640 1 -- 172.21.15.60:0/656884421 learned_addr learned my addr 172.21.15.60:0/656884421 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/656884421 >> v2:172.21.15.100:3300/0 conn(0x7fa2c006c9e0 0x7fa2c00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c548f640 1 -- 172.21.15.60:0/656884421 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103588892 0 0) 0x7fa2c006cf50 con 0x7fa2c015b7d0 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.467+0000 7fa2c548f640 1 -- 172.21.15.60:0/656884421 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ac003520 con 0x7fa2c015b7d0 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.468+0000 7fa2c6c92640 1 -- 172.21.15.60:0/656884421 >> v1:172.21.15.100:6789/0 conn(0x7fa2c00b5620 legacy=0x7fa2c006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.468+0000 7fa2c548f640 1 -- 172.21.15.60:0/656884421 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587838637 0 0) 0x7fa2c00ce530 con 0x7fa2c00b5620 2024-04-25T08:37:46.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.468+0000 7fa2c6c92640 1 -- 172.21.15.60:0/656884421 >> v1:172.21.15.60:6789/0 conn(0x7fa2c015b7d0 legacy=0x7fa2c00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.468+0000 7fa2c6c92640 1 -- 172.21.15.60:0/656884421 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00ceaf0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.468+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/656884421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c006c9e0 0x7fa2c00b5ab0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24eebdbad819f097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c548f640 1 -- 172.21.15.60:0/656884421 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8007740 con 0x7fa2c006c9e0 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c548f640 1 -- 172.21.15.60:0/656884421 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b80078e0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c548f640 1 -- 172.21.15.60:0/656884421 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8007cb0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 -- 172.21.15.60:0/656884421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c006c9e0 msgr2=0x7fa2c00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/656884421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c006c9e0 0x7fa2c00b5ab0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 -- 172.21.15.60:0/656884421 shutdown_connections 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/656884421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c006c9e0 0x7fa2c00b5ab0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 -- 172.21.15.60:0/656884421 >> 172.21.15.60:0/656884421 conn(0x7fa2c00d0800 msgr2=0x7fa2c00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 -- 172.21.15.60:0/656884421 shutdown_connections 2024-04-25T08:37:46.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.469+0000 7fa2c86f5640 1 -- 172.21.15.60:0/656884421 wait complete. 2024-04-25T08:37:46.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.470+0000 7fa2c86f5640 1 Processor -- start 2024-04-25T08:37:46.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.470+0000 7fa2c86f5640 1 -- start start 2024-04-25T08:37:46.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c86f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 0x7fa2c0145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c86f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c00b5620 0x7fa2c01e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c86f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2c015b7d0 0x7fa2c00ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c86f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c015bd30 con 0x7fa2c006c9e0 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c86f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c015bbb0 con 0x7fa2c00b5620 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c86f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00d1f40 con 0x7fa2c015b7d0 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c6491640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 0x7fa2c0145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c6491640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 0x7fa2c0145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54356/0 (socket says 172.21.15.60:54356) 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c6491640 1 -- 172.21.15.60:0/136133731 learned_addr learned my addr 172.21.15.60:0/136133731 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c00b5620 0x7fa2c01e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.471+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2c015b7d0 0x7fa2c00ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c6491640 1 -- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2c015b7d0 msgr2=0x7fa2c00ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c6491640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2c015b7d0 0x7fa2c00ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c6491640 1 -- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c00b5620 msgr2=0x7fa2c01e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c6491640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c00b5620 0x7fa2c01e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c6491640 1 -- 172.21.15.60:0/136133731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c01461c0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c6491640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 0x7fa2c0145be0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c390 tx=0x7fa2bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2b6ffd640 1 -- 172.21.15.60:0/136133731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc017070 con 0x7fa2c006c9e0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2b6ffd640 1 -- 172.21.15.60:0/136133731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc005ce0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c01464a0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2b6ffd640 1 -- 172.21.15.60:0/136133731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0072f0 con 0x7fa2c006c9e0 2024-04-25T08:37:46.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.472+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0146a60 con 0x7fa2c006c9e0 2024-04-25T08:37:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.473+0000 7fa2b6ffd640 1 -- 172.21.15.60:0/136133731 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2bc01a070 con 0x7fa2c006c9e0 2024-04-25T08:37:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.474+0000 7fa2b6ffd640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a406b4d0 0x7fa2a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.474+0000 7fa2b6ffd640 1 -- 172.21.15.60:0/136133731 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2bc089840 con 0x7fa2c006c9e0 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.474+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a406b4d0 0x7fa2a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.474+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a406b4d0 0x7fa2a406d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0148d30 tx=0x7fa2b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.475+0000 7fa2cb3f8740 1 Processor -- start 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.475+0000 7fa2cb3f8740 1 -- start start 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 0x55714b8d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfef0 0x55714b8d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 0x55714b8a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714b80bda0 con 0x55714b8a2770 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714b7b4b80 con 0x55714b8cfb10 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55714b807e40 con 0x55714b8cfef0 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 0x55714b8a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 0x55714b8d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.472 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c5c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfef0 0x55714b8d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 0x55714b8a5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54358/0 (socket says 172.21.15.60:54358) 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 -- 172.21.15.60:0/4241203533 learned_addr learned my addr 172.21.15.60:0/4241203533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 0x55714b8d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40952/0 (socket says 172.21.15.60:40952) 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 -- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfef0 msgr2=0x55714b8d2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfef0 0x55714b8d2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 -- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 msgr2=0x55714b8d2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 0x55714b8d2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 -- 172.21.15.60:0/4241203533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714b7bef10 con 0x55714b8a2770 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6491640 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 0x55714b8d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.476+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 0x55714b8a5f80 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d857d4fdfb64f9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2a3fff640 1 -- 172.21.15.60:0/4241203533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003b90 con 0x55714b8a2770 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2a3fff640 1 -- 172.21.15.60:0/4241203533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8003d30 con 0x55714b8a2770 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2a3fff640 1 -- 172.21.15.60:0/4241203533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004230 con 0x55714b8a2770 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/4241203533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 msgr2=0x55714b8a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 0x55714b8a5f80 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/4241203533 shutdown_connections 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8a2770 0x55714b8a5f80 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfef0 0x55714b8d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/4241203533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfb10 0x55714b8d2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/4241203533 >> 172.21.15.60:0/4241203533 conn(0x55714b8a49e0 msgr2=0x55714b8ed320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/4241203533 shutdown_connections 2024-04-25T08:37:46.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.477+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/4241203533 wait complete. 2024-04-25T08:37:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.478+0000 7fa2cb3f8740 1 -- start start 2024-04-25T08:37:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.478+0000 7fa2cb3f8740 1 Processor -- start 2024-04-25T08:37:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 0x55714b8a3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714b8a7b90 con 0x55714b8d5050 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714b7aa4b0 con 0x55714b8cfef0 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55714b8a7e70 con 0x55714b8cfb10 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2cb3f8740 10 client.0 _ll_register_callbacks cb 0x55714b7ea180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 0x55714b8a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 0x55714b8a3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54364/0 (socket says 172.21.15.60:54364) 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2546776091 learned_addr learned my addr 172.21.15.60:0/2546776091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6491640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 msgr2=0x55714b8c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 msgr2=0x55714b8d4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8d4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2546776091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55714b5ea7d0 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 0x55714b8a3400 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d173c4cfc930bb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.479+0000 7fa2a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2546776091 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8054060 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2546776091 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2b8004440 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2b8003d30 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2546776091 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b805a050 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 10 client.9000 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55714b5203f0 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2546776091 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8003d30 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2546776091 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2b805d020 con 0x55714b8d5050 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 10 client.9000 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 10 client.9000 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 20 client.9000 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:46.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8cb060 0x55714b8cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 4 client.9000 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55714b8192e0 con 0x55714b8cb060 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.480+0000 7fa2c6491640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8cb060 0x55714b8cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2a17fa640 10 client.9000 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2c6491640 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8cb060 0x55714b8cb470 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b54722a1cdedc2 server_cookie=83eced4ddfed93c4 in_seq=0 out_seq=0 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2546776091 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55714b8192e0 con 0x55714b8cb060 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2a17fa640 10 client.9000 handle_command_reply: tid=0 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 client.9000 shutdown 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 msgr2=0x55714b8a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 0x55714b8a3400 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009d90 tx=0x7fa2b8004ab0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 shutdown_connections 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8cb060 0x55714b8cb470 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8d5050 0x55714b8a3400 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8d4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2a17fa640 0 client.9000 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2546776091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.481+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 >> 172.21.15.60:0/2546776091 conn(0x55714b8a2770 msgr2=0x55714b8cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.482+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 shutdown_connections 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.482+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2546776091 wait complete. 2024-04-25T08:37:46.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.482+0000 7fa2cb3f8740 20 client.9000 trim_cache size 0 max 16384 2024-04-25T08:37:46.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 Processor -- start 2024-04-25T08:37:46.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 -- start start 2024-04-25T08:37:46.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cb060 0x55714b8d58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 0x55714b8ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0200140 con 0x55714b8cfb10 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0148270 con 0x55714b8cfef0 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.497+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00c7ab0 con 0x55714b8cb060 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 0x55714b8ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 0x55714b8ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54366/0 (socket says 172.21.15.60:54366) 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 -- 172.21.15.60:0/3145166818 learned_addr learned my addr 172.21.15.60:0/3145166818 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c6491640 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cb060 0x55714b8d58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 -- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cb060 msgr2=0x55714b8d58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cb060 0x55714b8d58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 -- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 msgr2=0x55714b8cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 -- 172.21.15.60:0/3145166818 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714b4e00f0 con 0x55714b8cfb10 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2c5c90640 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 0x55714b8ced50 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005210 tx=0x7fa2b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5dd38dc064a713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2a37fe640 1 -- 172.21.15.60:0/3145166818 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0003800 con 0x55714b8cfb10 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2a37fe640 1 -- 172.21.15.60:0/3145166818 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0027070 con 0x55714b8cfb10 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.498+0000 7fa2a37fe640 1 -- 172.21.15.60:0/3145166818 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00039a0 con 0x55714b8cfb10 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/3145166818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 msgr2=0x55714b8ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 0x55714b8ced50 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005210 tx=0x7fa2b0011000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/3145166818 shutdown_connections 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cfef0 0x55714b8cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfb10 0x55714b8ced50 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/3145166818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cb060 0x55714b8d58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/3145166818 >> 172.21.15.60:0/3145166818 conn(0x55714b8a2770 msgr2=0x55714b90fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/3145166818 shutdown_connections 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/3145166818 wait complete. 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 -- start start 2024-04-25T08:37:46.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.499+0000 7fa2cb3f8740 1 Processor -- start 2024-04-25T08:37:46.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cb060 0x55714b8f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 0x55714b8c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0147c70 con 0x55714b8cfef0 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714b8cabe0 con 0x55714b8cb060 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55714b8c4d60 con 0x55714b8cfb10 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2cb3f8740 10 client.0 _ll_register_callbacks cb 0x7fa2c0062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c6c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 0x55714b8c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c6491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cb060 0x55714b8f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c5c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c6c92640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 0x55714b8c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54374/0 (socket says 172.21.15.60:54374) 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2922310328 learned_addr learned my addr 172.21.15.60:0/2922310328 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c6491640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cb060 0x55714b8f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:40982/0 (socket says 172.21.15.60:40982) 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.500+0000 7fa2c5c90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45192/0 (socket says 172.21.15.60:45192) 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 msgr2=0x55714b8a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cb060 msgr2=0x55714b8f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cb060 0x55714b8f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2c6c92640 1 -- 172.21.15.60:0/2922310328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55714b8f0930 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2c6c92640 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 0x55714b8c41c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8002230 tx=0x7fa2b802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee9b2e1e618b2df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2922310328 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8004690 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2922310328 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2b80078f0 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2b0027070 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2922310328 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2b806a050 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 10 client.9006 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55714b8e3030 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2922310328 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0027070 con 0x55714b8cfef0 2024-04-25T08:37:46.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2922310328 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2b806d020 con 0x55714b8cfef0 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 10 client.9006 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 10 client.9006 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 20 client.9006 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8d5050 0x55714b957a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 4 client.9006 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.501+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55714b813630 con 0x55714b8d5050 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2c6491640 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8d5050 0x55714b957a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2a17fa640 10 client.9006 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2c6491640 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8d5050 0x55714b957a50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a572dbb199c4280 server_cookie=abdd9682921506b4 in_seq=0 out_seq=0 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2a17fa640 1 -- 172.21.15.60:0/2922310328 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55714b813630 con 0x55714b8d5050 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2a17fa640 10 client.9006 handle_command_reply: tid=0 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 client.9006 shutdown 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 msgr2=0x55714b8c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 0x55714b8c41c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8002230 tx=0x7fa2b802f000 comp rx=0 tx=0).stop 2024-04-25T08:37:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 shutdown_connections 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55714b8d5050 0x55714b957a50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55714b8cfef0 0x55714b8c41c0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55714b8cfb10 0x55714b8a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2a17fa640 0 client.9006 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 --2- 172.21.15.60:0/2922310328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55714b8cb060 0x55714b8f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 >> 172.21.15.60:0/2922310328 conn(0x55714b8a2770 msgr2=0x55714b8d57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 shutdown_connections 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.502+0000 7fa2cb3f8740 1 -- 172.21.15.60:0/2922310328 wait complete. 2024-04-25T08:37:46.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2cb3f8740 20 client.9006 trim_cache size 0 max 16384 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout: "27b078d0-e100-49f7-b1e4-4af885c05c8c": { 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "27b078d0-e100-49f7-b1e4-4af885c05c8c", 2024-04-25T08:37:46.505 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T08:37:46.506 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:46.506 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:37:46.506 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a406b4d0 msgr2=0x7fa2a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a406b4d0 0x7fa2a406d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0148d30 tx=0x7fa2b0009040 comp rx=0 tx=0).stop 2024-04-25T08:37:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 msgr2=0x7fa2c0145be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 0x7fa2c0145be0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c390 tx=0x7fa2bc00c850 comp rx=0 tx=0).stop 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 shutdown_connections 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa2c015b7d0 0x7fa2c00ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa2a406b4d0 0x7fa2a406d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2c00b5620 0x7fa2c01e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 --2- 172.21.15.60:0/136133731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa2c006c9e0 0x7fa2c0145be0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 >> 172.21.15.60:0/136133731 conn(0x7fa2c00d0800 msgr2=0x7fa2c00b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 shutdown_connections 2024-04-25T08:37:46.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.503+0000 7fa2c86f5640 1 -- 172.21.15.60:0/136133731 wait complete. 2024-04-25T08:37:46.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:46.609 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.615+0000 7f3ee1770640 1 Processor -- start 2024-04-25T08:37:46.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.615+0000 7f3ee1770640 1 -- start start 2024-04-25T08:37:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3ee1770640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3edc15db60 0x7f3edc0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3ee1770640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3edc0b5f00 0x7f3edc0d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3ee1770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc15df40 0x7f3edc0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3ee1770640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc0ce930 con 0x7f3edc0b5f00 2024-04-25T08:37:46.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3ee1770640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3edc0b6450 con 0x7f3edc15db60 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3ee1770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc0cec00 con 0x7f3edc15df40 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3eda7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3edc0b5f00 0x7f3edc0d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3eda7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3edc0b5f00 0x7f3edc0d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54376/0 (socket says 172.21.15.60:54376) 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3eda7fc640 1 -- 172.21.15.60:0/1518509237 learned_addr learned my addr 172.21.15.60:0/1518509237 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3edaffd640 1 --2- 172.21.15.60:0/1518509237 >> v2:172.21.15.100:3300/0 conn(0x7f3edc15db60 0x7f3edc0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.616+0000 7f3edb7fe640 1 --2- 172.21.15.60:0/1518509237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc15df40 0x7f3edc0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3eda7fc640 1 -- 172.21.15.60:0/1518509237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc15df40 msgr2=0x7f3edc0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3eda7fc640 1 --2- 172.21.15.60:0/1518509237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc15df40 0x7f3edc0d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3eda7fc640 1 -- 172.21.15.60:0/1518509237 >> v2:172.21.15.100:3300/0 conn(0x7f3edc15db60 msgr2=0x7f3edc0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3eda7fc640 1 --2- 172.21.15.60:0/1518509237 >> v2:172.21.15.100:3300/0 conn(0x7f3edc15db60 0x7f3edc0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3eda7fc640 1 -- 172.21.15.60:0/1518509237 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc0ceee0 con 0x7f3edc0b5f00 2024-04-25T08:37:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.617+0000 7f3eda7fc640 1 --2- 172.21.15.60:0/1518509237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc0b5f00 0x7f3edc0d0400 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3edc062440 tx=0x7f3ed002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f8c2a5f987067a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ed9ffb640 1 -- 172.21.15.60:0/1518509237 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0037070 con 0x7f3edc0b5f00 2024-04-25T08:37:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ed9ffb640 1 -- 172.21.15.60:0/1518509237 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0032440 con 0x7f3edc0b5f00 2024-04-25T08:37:46.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ed9ffb640 1 -- 172.21.15.60:0/1518509237 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0032810 con 0x7f3edc0b5f00 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 -- 172.21.15.60:0/1518509237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc0b5f00 msgr2=0x7f3edc0d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 --2- 172.21.15.60:0/1518509237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc0b5f00 0x7f3edc0d0400 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3edc062440 tx=0x7f3ed002a180 comp rx=0 tx=0).stop 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 -- 172.21.15.60:0/1518509237 shutdown_connections 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 --2- 172.21.15.60:0/1518509237 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc15df40 0x7f3edc0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 --2- 172.21.15.60:0/1518509237 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc0b5f00 0x7f3edc0d0400 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 --2- 172.21.15.60:0/1518509237 >> v2:172.21.15.100:3300/0 conn(0x7f3edc15db60 0x7f3edc0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 -- 172.21.15.60:0/1518509237 >> 172.21.15.60:0/1518509237 conn(0x7f3edc061600 msgr2=0x7f3edc061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 -- 172.21.15.60:0/1518509237 shutdown_connections 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.618+0000 7f3ee1770640 1 -- 172.21.15.60:0/1518509237 wait complete. 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.619+0000 7f3ee1770640 1 Processor -- start 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.619+0000 7f3ee1770640 1 -- start start 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3ee1770640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc0b5f00 0x7f3edc150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3ee1770640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3edc15db60 0x7f3edc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3ee1770640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 0x7f3edc1603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3ee1770640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc161950 con 0x7f3edc15df40 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3ee1770640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc0d1d50 con 0x7f3edc15db60 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3ee1770640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc161ad0 con 0x7f3edc0b5f00 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3edb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 0x7f3edc1603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3edb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 0x7f3edc1603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54390/0 (socket says 172.21.15.60:54390) 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3edb7fe640 1 -- 172.21.15.60:0/2078464665 learned_addr learned my addr 172.21.15.60:0/2078464665 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3edaffd640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc0b5f00 0x7f3edc150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.620+0000 7f3eda7fc640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3edc15db60 0x7f3edc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3edb7fe640 1 -- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc0b5f00 msgr2=0x7f3edc150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3edb7fe640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc0b5f00 0x7f3edc150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3edb7fe640 1 -- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3edc15db60 msgr2=0x7f3edc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3edb7fe640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3edc15db60 0x7f3edc0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3edb7fe640 1 -- 172.21.15.60:0/2078464665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3edc0cd160 con 0x7f3edc15df40 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3edb7fe640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 0x7f3edc1603c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00efc0 tx=0x7f3ecc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc018020 con 0x7f3edc15df40 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc014480 con 0x7f3edc15df40 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc151390 con 0x7f3edc15df40 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc014870 con 0x7f3edc15df40 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.621+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc0cd6b0 con 0x7f3edc15df40 2024-04-25T08:37:46.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.622+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc073270 con 0x7f3edc15df40 2024-04-25T08:37:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.623+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ecc005d10 con 0x7f3edc15df40 2024-04-25T08:37:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.623+0000 7f3ec37fe640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.623+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ecc08ace0 con 0x7f3edc15df40 2024-04-25T08:37:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.623+0000 7f3edaffd640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.624+0000 7f3edaffd640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0cc6f0 tx=0x7f3ed4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.628+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ecc022070 con 0x7f3edc15df40 2024-04-25T08:37:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.844+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3edc0d1860 con 0x7f3edc15df40 2024-04-25T08:37:46.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.845+0000 7f3ec37fe640 1 -- 172.21.15.60:0/2078464665 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3ecc010040 con 0x7f3edc15df40 2024-04-25T08:37:46.839 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:46.839 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:46.839 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ec806b4d0 msgr2=0x7f3ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ec806b4d0 0x7f3ec806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0cc6f0 tx=0x7f3ed4009190 comp rx=0 tx=0).stop 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 msgr2=0x7f3edc1603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 0x7f3edc1603c0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00efc0 tx=0x7f3ecc00c490 comp rx=0 tx=0).stop 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 shutdown_connections 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3edc15df40 0x7f3edc1603c0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ec806b4d0 0x7f3ec806d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3edc15db60 0x7f3edc0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 --2- 172.21.15.60:0/2078464665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3edc0b5f00 0x7f3edc150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.846+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 >> 172.21.15.60:0/2078464665 conn(0x7f3edc061600 msgr2=0x7f3edc0b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.847+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 shutdown_connections 2024-04-25T08:37:46.842 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.847+0000 7f3ee1770640 1 -- 172.21.15.60:0/2078464665 wait complete. 2024-04-25T08:37:46.851 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:46.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:46.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.957+0000 7f6141534640 1 Processor -- start 2024-04-25T08:37:46.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.958+0000 7f6141534640 1 -- start start 2024-04-25T08:37:46.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.958+0000 7f6141534640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f613c0d2e90 0x7f613c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.958+0000 7f6141534640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 0x7f613c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.958+0000 7f6141534640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c0b9940 con 0x7f613c15dd30 2024-04-25T08:37:46.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.958+0000 7f6141534640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c0d0d40 con 0x7f613c0d2e90 2024-04-25T08:37:46.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.958+0000 7f6141534640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0d1df0 con 0x7f613c0d0900 2024-04-25T08:37:46.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 0x7f613c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613a7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f613c0d2e90 0x7f613c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613affd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f613c15dd30 0x7f613c0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:57176/0 (socket says 172.21.15.60:57176) 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613a7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f613c0d2e90 0x7f613c0d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:41016/0 (socket says 172.21.15.60:41016) 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613b7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 0x7f613c0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45222/0 (socket says 172.21.15.60:45222) 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613affd640 1 -- 172.21.15.60:0/1764642554 learned_addr learned my addr 172.21.15.60:0/1764642554 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613b7fe640 1 -- 172.21.15.60:0/1764642554 >> v2:172.21.15.100:3300/0 conn(0x7f613c0d2e90 msgr2=0x7f613c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613b7fe640 1 --2- 172.21.15.60:0/1764642554 >> v2:172.21.15.100:3300/0 conn(0x7f613c0d2e90 0x7f613c0d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613b7fe640 1 -- 172.21.15.60:0/1764642554 >> v1:172.21.15.100:6789/0 conn(0x7f613c15dd30 legacy=0x7f613c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.959+0000 7f613b7fe640 1 -- 172.21.15.60:0/1764642554 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0d20d0 con 0x7f613c0d0900 2024-04-25T08:37:46.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f613b7fe640 1 --2- 172.21.15.60:0/1764642554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 0x7f613c0b6a40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f612c009d90 tx=0x7f612c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5af4bcdb894083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6139ffb640 1 -- 172.21.15.60:0/1764642554 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c031600 con 0x7f613c0d0900 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6139ffb640 1 -- 172.21.15.60:0/1764642554 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c0317a0 con 0x7f613c0d0900 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6139ffb640 1 -- 172.21.15.60:0/1764642554 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c031b70 con 0x7f613c0d0900 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 -- 172.21.15.60:0/1764642554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 msgr2=0x7f613c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 --2- 172.21.15.60:0/1764642554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 0x7f613c0b6a40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f612c009d90 tx=0x7f612c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 -- 172.21.15.60:0/1764642554 shutdown_connections 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 --2- 172.21.15.60:0/1764642554 >> v2:172.21.15.100:3300/0 conn(0x7f613c0d2e90 0x7f613c0d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 --2- 172.21.15.60:0/1764642554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d0900 0x7f613c0b6a40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 -- 172.21.15.60:0/1764642554 >> 172.21.15.60:0/1764642554 conn(0x7f613c061c30 msgr2=0x7f613c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 -- 172.21.15.60:0/1764642554 shutdown_connections 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.960+0000 7f6141534640 1 -- 172.21.15.60:0/1764642554 wait complete. 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 Processor -- start 2024-04-25T08:37:46.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 -- start start 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 0x7f613c15b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d2e90 0x7f613c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f613c15dd30 0x7f613c0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0d2970 con 0x7f613c0d0900 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0d2690 con 0x7f613c15dd30 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.962+0000 7f6141534640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0d1a10 con 0x7f613c0d2e90 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 0x7f613c15b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 0x7f613c15b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54400/0 (socket says 172.21.15.60:54400) 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d2e90 0x7f613c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613affd640 1 -- 172.21.15.60:0/2234680907 learned_addr learned my addr 172.21.15.60:0/2234680907 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613b7fe640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f613c15dd30 0x7f613c0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 -- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f613c15dd30 msgr2=0x7f613c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f613c15dd30 0x7f613c0cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 -- 172.21.15.60:0/2234680907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 msgr2=0x7f613c15b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 0x7f613c15b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 -- 172.21.15.60:0/2234680907 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0b5590 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613affd640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 0x7f613c15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f613a7fc640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d2e90 0x7f613c0b4bb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f612400e840 tx=0x7f612400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124017070 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612400c940 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c0b5870 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400ccf0 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.963+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c1f3a60 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.964+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c08bbd0 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.965+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f612401a070 con 0x7f613c0d2e90 2024-04-25T08:37:46.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.965+0000 7f61337fe640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f611c06b4d0 0x7f611c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:46.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.965+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f612408a2c0 con 0x7f613c0d2e90 2024-04-25T08:37:46.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.966+0000 7f613affd640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f611c06b4d0 0x7f611c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:46.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.966+0000 7f613affd640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f611c06b4d0 0x7f611c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f613c15c2f0 tx=0x7f6134009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:46.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:46.971+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f613c08bbd0 con 0x7f613c0d2e90 2024-04-25T08:37:47.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.221+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f613c073990 con 0x7f613c0d2e90 2024-04-25T08:37:47.215 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.222+0000 7f61337fe640 1 -- 172.21.15.60:0/2234680907 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f6124058960 con 0x7f613c0d2e90 2024-04-25T08:37:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:47.216 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":755,"data_bytes":769051682,"bytes_used":1857429504,"bytes_avail":771236683776,"bytes_total":773094113280,"write_bytes_sec":64029799,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T08:37:09.505383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f611c06b4d0 msgr2=0x7f611c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f611c06b4d0 0x7f611c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f613c15c2f0 tx=0x7f6134009040 comp rx=0 tx=0).stop 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d2e90 msgr2=0x7f613c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d2e90 0x7f613c0b4bb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f612400e840 tx=0x7f612400ed00 comp rx=0 tx=0).stop 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 shutdown_connections 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f611c06b4d0 0x7f611c06d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f613c15dd30 0x7f613c0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f613c0d2e90 0x7f613c0b4bb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:47.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 --2- 172.21.15.60:0/2234680907 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f613c0d0900 0x7f613c15b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 >> 172.21.15.60:0/2234680907 conn(0x7f613c061c30 msgr2=0x7f613c0d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 shutdown_connections 2024-04-25T08:37:47.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:47.224+0000 7f6141534640 1 -- 172.21.15.60:0/2234680907 wait complete. 2024-04-25T08:37:47.228 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:48.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:48.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.348+0000 7fbf6c93b640 1 Processor -- start 2024-04-25T08:37:48.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.348+0000 7fbf6c93b640 1 -- start start 2024-04-25T08:37:48.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.349+0000 7fbf6c93b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 0x7fbf680d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.349+0000 7fbf6c93b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680b6220 con 0x7fbf680b2cb0 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.349+0000 7fbf6c93b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680b97a0 con 0x7fbf680b92a0 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.349+0000 7fbf6c93b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6815dfc0 con 0x7fbf6815dbe0 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.349+0000 7fbf65d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf680b2cb0 0x7fbf680b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56216/0 (socket says 172.21.15.60:56216) 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.349+0000 7fbf65d9b640 1 -- 172.21.15.60:0/3586576182 learned_addr learned my addr 172.21.15.60:0/3586576182 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6659c640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 0x7fbf680d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119233206 0 0) 0x7fbf680b6220 con 0x7fbf680b2cb0 2024-04-25T08:37:48.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf50003520 con 0x7fbf680b2cb0 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2932568998 0 0) 0x7fbf50003520 con 0x7fbf680b2cb0 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 msgr2=0x7fbf680d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 0x7fbf680d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 >> v1:172.21.15.100:6789/0 conn(0x7fbf680b92a0 legacy=0x7fbf680b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680d1c50 con 0x7fbf680b2cb0 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fbf58001670 con 0x7fbf680b2cb0 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 >> v1:172.21.15.60:6789/0 conn(0x7fbf680b2cb0 legacy=0x7fbf680b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 0x7fbf500039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf50004d40 0x7fbf500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000b9f0 con 0x7fbf50003f20 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000bba0 con 0x7fbf50004d40 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000bda0 con 0x7fbf6815dbe0 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf6659c640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 0x7fbf500039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.350+0000 7fbf66d9d640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf50004d40 0x7fbf500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf65d9b640 1 -- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 msgr2=0x7fbf500039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815dbe0 0x7fbf500039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf65d9b640 1 -- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf50004d40 msgr2=0x7fbf500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf50004d40 0x7fbf500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf65d9b640 1 -- 172.21.15.60:0/3586576182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50003520 con 0x7fbf50003f20 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf66d9d640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf50004d40 0x7fbf500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf50004800 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c240 tx=0x7fbf50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e039991c4a1e3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:48.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58007d30 con 0x7fbf50003f20 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58005af0 con 0x7fbf50003f20 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.351+0000 7fbf6559a640 1 -- 172.21.15.60:0/3586576182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58007070 con 0x7fbf50003f20 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf6c93b640 1 -- 172.21.15.60:0/3586576182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 msgr2=0x7fbf50004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/3586576182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf50004800 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c240 tx=0x7fbf50005830 comp rx=0 tx=0).stop 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf66d9d640 1 -- 172.21.15.60:0/3586576182 reap_dead start 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf6c93b640 1 -- 172.21.15.60:0/3586576182 shutdown_connections 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf6c93b640 1 -- 172.21.15.60:0/3586576182 >> 172.21.15.60:0/3586576182 conn(0x7fbf68061a20 msgr2=0x7fbf68061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf6c93b640 1 -- 172.21.15.60:0/3586576182 shutdown_connections 2024-04-25T08:37:48.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.352+0000 7fbf6c93b640 1 -- 172.21.15.60:0/3586576182 wait complete. 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 Processor -- start 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 -- start start 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf6815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf680b5cd0 0x7fbf68153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815c7d0 0x7fbf6815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680d2210 con 0x7fbf50003f20 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680d1a00 con 0x7fbf680b5cd0 2024-04-25T08:37:48.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.354+0000 7fbf6c93b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680b6ab0 con 0x7fbf6815c7d0 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf6815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf6815f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54428/0 (socket says 172.21.15.60:54428) 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 -- 172.21.15.60:0/1191253948 learned_addr learned my addr 172.21.15.60:0/1191253948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf680b5cd0 0x7fbf68153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf66d9d640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815c7d0 0x7fbf6815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 -- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815c7d0 msgr2=0x7fbf6815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815c7d0 0x7fbf6815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 -- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf680b5cd0 msgr2=0x7fbf68153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf680b5cd0 0x7fbf68153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 -- 172.21.15.60:0/1191253948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6815eb90 con 0x7fbf50003f20 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6659c640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf6815f950 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0119d0 tx=0x7fbf5c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00fb50 con 0x7fbf50003f20 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c00fcf0 con 0x7fbf50003f20 2024-04-25T08:37:48.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6815ee70 con 0x7fbf50003f20 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6815f430 con 0x7fbf50003f20 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.355+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00fea0 con 0x7fbf50003f20 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.356+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf68073990 con 0x7fbf50003f20 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.357+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf5c012040 con 0x7fbf50003f20 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.357+0000 7fbf56ffd640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b500 0x7fbf4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.357+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf5c08ab20 con 0x7fbf50003f20 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.358+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b500 0x7fbf4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.358+0000 7fbf65d9b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b500 0x7fbf4806d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbf58032000 tx=0x7fbf58009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:48.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.362+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf5c019030 con 0x7fbf50003f20 2024-04-25T08:37:48.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.582+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf680b8240 con 0x7fbf50003f20 2024-04-25T08:37:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.583+0000 7fbf56ffd640 1 -- 172.21.15.60:0/1191253948 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fbf5c059780 con 0x7fbf50003f20 2024-04-25T08:37:48.577 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:48.577 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:48.577 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b500 msgr2=0x7fbf4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b500 0x7fbf4806d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbf58032000 tx=0x7fbf58009030 comp rx=0 tx=0).stop 2024-04-25T08:37:48.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 msgr2=0x7fbf6815f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf6815f950 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0119d0 tx=0x7fbf5c011e90 comp rx=0 tx=0).stop 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 shutdown_connections 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf6815c7d0 0x7fbf6815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b500 0x7fbf4806d9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf680b5cd0 0x7fbf68153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 --2- 172.21.15.60:0/1191253948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf50003f20 0x7fbf6815f950 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 >> 172.21.15.60:0/1191253948 conn(0x7fbf68061a20 msgr2=0x7fbf6815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 shutdown_connections 2024-04-25T08:37:48.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.585+0000 7fbf6c93b640 1 -- 172.21.15.60:0/1191253948 wait complete. 2024-04-25T08:37:48.590 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:48.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:48.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.695+0000 7f3552438640 1 Processor -- start 2024-04-25T08:37:48.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f3552438640 1 -- start start 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f3552438640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f354c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f3552438640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c161960 con 0x7f354c06b1a0 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f3552438640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c161c40 con 0x7f354c0cffa0 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f3552438640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0d0430 con 0x7f354c06b580 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f354bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f354c06b1a0 0x7f354c06f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56218/0 (socket says 172.21.15.60:56218) 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.696+0000 7f354bfff640 1 -- 172.21.15.60:0/3895596744 learned_addr learned my addr 172.21.15.60:0/3895596744 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354b7fe640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f354c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491368192 0 0) 0x7f354c161960 con 0x7f354c06b1a0 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3534003520 con 0x7f354c06b1a0 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 724810610 0 0) 0x7f3534003520 con 0x7f354c06b1a0 2024-04-25T08:37:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 msgr2=0x7f354c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f354c0cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 >> v1:172.21.15.100:6789/0 conn(0x7f354c0cffa0 legacy=0x7f354c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c161f20 con 0x7f354c06b1a0 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3540001670 con 0x7f354c06b1a0 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354b7fe640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f354c0cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 >> v1:172.21.15.60:6789/0 conn(0x7f354c06b1a0 legacy=0x7f354c06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35340039e0 0x7f3534003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f3534004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3534004d40 0x7f35340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0d0430 con 0x7f35340039e0 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353400ba10 con 0x7f3534004d40 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353400bc10 con 0x7f354c06b580 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354bfff640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35340039e0 0x7f3534003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354b7fe640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f3534004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f35509d5640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3534004d40 0x7f35340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354bfff640 1 -- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 msgr2=0x7f3534004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354bfff640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c06b580 0x7f3534004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354bfff640 1 -- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3534004d40 msgr2=0x7f35340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.697+0000 7f354bfff640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3534004d40 0x7f35340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.698+0000 7f354bfff640 1 -- 172.21.15.60:0/3895596744 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534003520 con 0x7f35340039e0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.698+0000 7f354bfff640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35340039e0 0x7f3534003dc0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f354000c190 tx=0x7f3534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9645a2dd1443cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.698+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540007cc0 con 0x7f35340039e0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.698+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3540005af0 con 0x7f35340039e0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.698+0000 7f354affd640 1 -- 172.21.15.60:0/3895596744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540007070 con 0x7f35340039e0 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f3552438640 1 -- 172.21.15.60:0/3895596744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35340039e0 msgr2=0x7f3534003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f3552438640 1 --2- 172.21.15.60:0/3895596744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35340039e0 0x7f3534003dc0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f354000c190 tx=0x7f3534005830 comp rx=0 tx=0).stop 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f35509d5640 1 -- 172.21.15.60:0/3895596744 reap_dead start 2024-04-25T08:37:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f3552438640 1 -- 172.21.15.60:0/3895596744 shutdown_connections 2024-04-25T08:37:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f3552438640 1 -- 172.21.15.60:0/3895596744 >> 172.21.15.60:0/3895596744 conn(0x7f354c0b2220 msgr2=0x7f354c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f3552438640 1 -- 172.21.15.60:0/3895596744 shutdown_connections 2024-04-25T08:37:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.699+0000 7f3552438640 1 -- 172.21.15.60:0/3895596744 wait complete. 2024-04-25T08:37:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.700+0000 7f3552438640 1 Processor -- start 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.700+0000 7f3552438640 1 -- start start 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f3552438640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 0x7f354c154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f3552438640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1536b0 0x7f354c1e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f3552438640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c0c4890 0x7f354c0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f3552438640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0700d0 con 0x7f354c1532d0 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f3552438640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0cbd50 con 0x7f354c1536b0 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f3552438640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c162720 con 0x7f354c0c4890 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 0x7f354c154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 0x7f354c154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54454/0 (socket says 172.21.15.60:54454) 2024-04-25T08:37:48.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 -- 172.21.15.60:0/1342663489 learned_addr learned my addr 172.21.15.60:0/1342663489 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f35509d5640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c0c4890 0x7f354c0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354b7fe640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1536b0 0x7f354c1e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 -- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c0c4890 msgr2=0x7f354c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c0c4890 0x7f354c0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 -- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1536b0 msgr2=0x7f354c1e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1536b0 0x7f354c1e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354bfff640 1 -- 172.21.15.60:0/1342663489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c1e91f0 con 0x7f354c1532d0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f354b7fe640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1536b0 0x7f354c1e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.701+0000 7f35509d5640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c0c4890 0x7f354c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.702+0000 7f354bfff640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 0x7f354c154fa0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f354002aca0 tx=0x7f35400019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.702+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540043070 con 0x7f354c1532d0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.702+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3540038040 con 0x7f354c1532d0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.702+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c1e94d0 con 0x7f354c1532d0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.702+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540007070 con 0x7f354c1532d0 2024-04-25T08:37:48.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.702+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c155530 con 0x7f354c1532d0 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.703+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c073990 con 0x7f354c1532d0 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.703+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3540007210 con 0x7f354c1532d0 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.704+0000 7f3548ff9640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f353806b500 0x7f353806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.704+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35400a9d40 con 0x7f354c1532d0 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.704+0000 7f354b7fe640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f353806b500 0x7f353806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.704+0000 7f354b7fe640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f353806b500 0x7f353806d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f354c161190 tx=0x7f353c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:48.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.708+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3540078a30 con 0x7f354c1532d0 2024-04-25T08:37:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.957+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f354c155930 con 0x7f354c1532d0 2024-04-25T08:37:48.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.957+0000 7f3548ff9640 1 -- 172.21.15.60:0/1342663489 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f3540078330 con 0x7f354c1532d0 2024-04-25T08:37:48.951 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:48.951 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":529,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1010,"data_bytes":1036438562,"bytes_used":2459377664,"bytes_avail":770634735616,"bytes_total":773094113280,"write_bytes_sec":86312226,"read_op_per_sec":0,"write_op_per_sec":84},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T08:37:09.505383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f353806b500 msgr2=0x7f353806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f353806b500 0x7f353806d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f354c161190 tx=0x7f353c008040 comp rx=0 tx=0).stop 2024-04-25T08:37:48.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 msgr2=0x7f354c154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 0x7f354c154fa0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f354002aca0 tx=0x7f35400019c0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 shutdown_connections 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f354c0c4890 0x7f354c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f353806b500 0x7f353806d9c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f354c1536b0 0x7f354c1e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 --2- 172.21.15.60:0/1342663489 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f354c1532d0 0x7f354c154fa0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 >> 172.21.15.60:0/1342663489 conn(0x7f354c0b2220 msgr2=0x7f354c0cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 shutdown_connections 2024-04-25T08:37:48.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:48.959+0000 7f3552438640 1 -- 172.21.15.60:0/1342663489 wait complete. 2024-04-25T08:37:48.964 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:49.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.085+0000 7fed1c01c640 1 Processor -- start 2024-04-25T08:37:50.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.085+0000 7fed1c01c640 1 -- start start 2024-04-25T08:37:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1c01c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed1415dd30 0x7fed140b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1c01c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140d0ad0 0x7fed140b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1c01c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140d1df0 con 0x7fed140b9770 2024-04-25T08:37:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1c01c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed140d20d0 con 0x7fed1415dd30 2024-04-25T08:37:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1c01c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed140d23b0 con 0x7fed140d0ad0 2024-04-25T08:37:50.081 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed1415dd30 0x7fed140b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fed1415dd30 0x7fed140b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54456/0 (socket says 172.21.15.60:54456) 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 -- 172.21.15.60:0/3829887227 learned_addr learned my addr 172.21.15.60:0/3829887227 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 -- 172.21.15.60:0/3829887227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140d0ad0 msgr2=0x7fed140b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1a5b9640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140d0ad0 0x7fed140b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140d0ad0 0x7fed140b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 -- 172.21.15.60:0/3829887227 >> v1:172.21.15.100:6789/0 conn(0x7fed140b9770 legacy=0x7fed140d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed19db8640 1 -- 172.21.15.60:0/3829887227 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140d2690 con 0x7fed1415dd30 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.086+0000 7fed1a5b9640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140d0ad0 0x7fed140b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.087+0000 7fed19db8640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed140b9230 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fed10009870 tx=0x7fed1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5902809802bdacc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.087+0000 7fed18db6640 1 -- 172.21.15.60:0/3829887227 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10037070 con 0x7fed1415dd30 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.087+0000 7fed18db6640 1 -- 172.21.15.60:0/3829887227 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10032440 con 0x7fed1415dd30 2024-04-25T08:37:50.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.087+0000 7fed18db6640 1 -- 172.21.15.60:0/3829887227 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10032810 con 0x7fed1415dd30 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 -- 172.21.15.60:0/3829887227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 msgr2=0x7fed140b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed140b9230 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fed10009870 tx=0x7fed1002a180 comp rx=0 tx=0).stop 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 -- 172.21.15.60:0/3829887227 shutdown_connections 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140d0ad0 0x7fed140b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3829887227 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed140b9230 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 -- 172.21.15.60:0/3829887227 >> 172.21.15.60:0/3829887227 conn(0x7fed14061c30 msgr2=0x7fed1406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 -- 172.21.15.60:0/3829887227 shutdown_connections 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.088+0000 7fed1c01c640 1 -- 172.21.15.60:0/3829887227 wait complete. 2024-04-25T08:37:50.086 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.089+0000 7fed1c01c640 1 Processor -- start 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.089+0000 7fed1c01c640 1 -- start start 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1c01c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140b9770 0x7fed14152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1c01c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed140d0ad0 0x7fed1415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1c01c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed14153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1c01c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed140d1b90 con 0x7fed1415dd30 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1c01c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed140d1a10 con 0x7fed140d0ad0 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1c01c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed140d2e90 con 0x7fed140b9770 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed14153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed14153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54470/0 (socket says 172.21.15.60:54470) 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 -- 172.21.15.60:0/3852365614 learned_addr learned my addr 172.21.15.60:0/3852365614 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed19db8640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140b9770 0x7fed14152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed195b7640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed140d0ad0 0x7fed1415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 -- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140b9770 msgr2=0x7fed14152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140b9770 0x7fed14152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 -- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed140d0ad0 msgr2=0x7fed1415f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.090+0000 7fed1a5b9640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed140d0ad0 0x7fed1415f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed1a5b9640 1 -- 172.21.15.60:0/3852365614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14160050 con 0x7fed1415dd30 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed1a5b9640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed14153db0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00cef0 tx=0x7fed0c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c01a070 con 0x7fed1415dd30 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c00d860 con 0x7fed1415dd30 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed14160330 con 0x7fed1415dd30 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00dc50 con 0x7fed1415dd30 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.091+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed141608f0 con 0x7fed1415dd30 2024-04-25T08:37:50.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.092+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed140d47e0 con 0x7fed1415dd30 2024-04-25T08:37:50.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.092+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed0c01d070 con 0x7fed1415dd30 2024-04-25T08:37:50.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.093+0000 7fed0a7fc640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecf806b4d0 0x7fecf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.093+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed0c08ab50 con 0x7fed1415dd30 2024-04-25T08:37:50.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.093+0000 7fed19db8640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecf806b4d0 0x7fecf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.094+0000 7fed19db8640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecf806b4d0 0x7fecf806d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fed10034000 tx=0x7fed1002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:50.091 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.097+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed140d47e0 con 0x7fed1415dd30 2024-04-25T08:37:50.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.319+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed14073990 con 0x7fed1415dd30 2024-04-25T08:37:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.320+0000 7fed0a7fc640 1 -- 172.21.15.60:0/3852365614 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fed0c0592b0 con 0x7fed1415dd30 2024-04-25T08:37:50.314 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:50.314 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:50.314 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecf806b4d0 msgr2=0x7fecf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecf806b4d0 0x7fecf806d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fed10034000 tx=0x7fed1002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 msgr2=0x7fed14153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed14153db0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00cef0 tx=0x7fed0c00a710 comp rx=0 tx=0).stop 2024-04-25T08:37:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 shutdown_connections 2024-04-25T08:37:50.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1415dd30 0x7fed14153db0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecf806b4d0 0x7fecf806d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed140d0ad0 0x7fed1415f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 --2- 172.21.15.60:0/3852365614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed140b9770 0x7fed14152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 >> 172.21.15.60:0/3852365614 conn(0x7fed14061c30 msgr2=0x7fed140c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 shutdown_connections 2024-04-25T08:37:50.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.322+0000 7fed1c01c640 1 -- 172.21.15.60:0/3852365614 wait complete. 2024-04-25T08:37:50.327 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:50.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.432+0000 7f1ce38b3640 1 Processor -- start 2024-04-25T08:37:50.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.432+0000 7f1ce38b3640 1 -- start start 2024-04-25T08:37:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.433+0000 7f1ce38b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1cdc15f5f0 0x7f1cdc15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.433+0000 7f1ce38b3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0cd710 con 0x7f1cdc1602e0 2024-04-25T08:37:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.433+0000 7f1ce38b3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc099b70 con 0x7f1cdc15ba00 2024-04-25T08:37:50.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.433+0000 7f1ce38b3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc15ff80 con 0x7f1cdc15f5f0 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.433+0000 7f1ce0e4e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1cdc1602e0 0x7f1cdc0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:56220/0 (socket says 172.21.15.60:56220) 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.433+0000 7f1ce0e4e640 1 -- 172.21.15.60:0/1110333153 learned_addr learned my addr 172.21.15.60:0/1110333153 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255785077 0 0) 0x7f1cd0001110 con 0x7f1cdc1602e0 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce1e50640 1 --2- 172.21.15.60:0/1110333153 >> v2:172.21.15.100:3300/0 conn(0x7f1cdc15f5f0 0x7f1cdc15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc4003520 con 0x7f1cdc1602e0 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750419264 0 0) 0x7f1cdc099b70 con 0x7f1cdc15ba00 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd0001110 con 0x7f1cdc15ba00 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862119510 0 0) 0x7f1cc4003520 con 0x7f1cdc1602e0 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 >> v2:172.21.15.100:3300/0 conn(0x7f1cdc15f5f0 msgr2=0x7f1cdc15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 --2- 172.21.15.60:0/1110333153 >> v2:172.21.15.100:3300/0 conn(0x7f1cdc15f5f0 0x7f1cdc15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 >> v1:172.21.15.100:6789/0 conn(0x7f1cdc15ba00 legacy=0x7f1cdc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc099e50 con 0x7f1cdc1602e0 2024-04-25T08:37:50.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce1e50640 1 --2- 172.21.15.60:0/1110333153 >> v2:172.21.15.100:3300/0 conn(0x7f1cdc15f5f0 0x7f1cdc15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1cd00017c0 con 0x7f1cdc1602e0 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 >> v1:172.21.15.60:6789/0 conn(0x7f1cdc1602e0 legacy=0x7f1cdc0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cc4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc4005280 0x7f1cc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc15ff80 con 0x7f1cc4003b90 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc400c1c0 con 0x7f1cc4005280 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc400c3c0 con 0x7f1cc4004800 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce164f640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cc4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce0e4e640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce1e50640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc4005280 0x7f1cc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce164f640 1 -- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cc4004800 msgr2=0x7f1cc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce164f640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.432 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce164f640 1 -- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc4005280 msgr2=0x7f1cc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce164f640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cc4005280 0x7f1cc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce164f640 1 -- 172.21.15.60:0/1110333153 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4003520 con 0x7f1cc4003b90 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce1e50640 1 -- 172.21.15.60:0/1110333153 reap_dead start 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.434+0000 7f1ce0e4e640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cc4004800 0x7f1cc4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.435+0000 7f1ce164f640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cc4003f90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40018d0 tx=0x7f1cc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30490265d1559850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.435+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40014f0 con 0x7f1cc4003b90 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.435+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40052d0 con 0x7f1cc4003b90 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.435+0000 7f1cdbfff640 1 -- 172.21.15.60:0/1110333153 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd402ac40 con 0x7f1cc4003b90 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 -- 172.21.15.60:0/1110333153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 msgr2=0x7f1cc4003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cc4003f90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40018d0 tx=0x7f1cc4005830 comp rx=0 tx=0).stop 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 -- 172.21.15.60:0/1110333153 shutdown_connections 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/1110333153 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cc4003f90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.433 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 -- 172.21.15.60:0/1110333153 >> 172.21.15.60:0/1110333153 conn(0x7f1cdc0b3270 msgr2=0x7f1cdc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 -- 172.21.15.60:0/1110333153 shutdown_connections 2024-04-25T08:37:50.434 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.436+0000 7f1ce38b3640 1 -- 172.21.15.60:0/1110333153 wait complete. 2024-04-25T08:37:50.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.437+0000 7f1ce38b3640 1 Processor -- start 2024-04-25T08:37:50.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.437+0000 7f1ce38b3640 1 -- start start 2024-04-25T08:37:50.436 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce38b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cdc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce38b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cdc0b4330 0x7f1cdc0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce38b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cdc06f4d0 0x7f1cdc0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce38b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc15bde0 con 0x7f1cc4003b90 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce38b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc15fa40 con 0x7f1cdc06f4d0 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce38b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b2430 con 0x7f1cdc0b4330 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cdc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cdc155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54494/0 (socket says 172.21.15.60:54494) 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 -- 172.21.15.60:0/655372247 learned_addr learned my addr 172.21.15.60:0/655372247 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce0e4e640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cdc0b4330 0x7f1cdc0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce1e50640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cdc06f4d0 0x7f1cdc0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 -- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cdc0b4330 msgr2=0x7f1cdc0c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cdc0b4330 0x7f1cdc0c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 -- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cdc06f4d0 msgr2=0x7f1cdc0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cdc06f4d0 0x7f1cdc0cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.437 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 -- 172.21.15.60:0/655372247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0b4d50 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce1e50640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cdc06f4d0 0x7f1cdc0cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.438+0000 7f1ce164f640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cdc155530 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1cd402a6d0 tx=0x7f1cd40012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.439+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4037040 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.439+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd4005480 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.439+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0b5030 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.439+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40014f0 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.439+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0b5310 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.440+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc073990 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.440+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cd4042020 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.441+0000 7f1cd9ffb640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1cc806b4d0 0x7f1cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.441+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cd40a9910 con 0x7f1cc4003b90 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.441+0000 7f1ce0e4e640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1cc806b4d0 0x7f1cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:50.438 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.442+0000 7f1ce0e4e640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1cc806b4d0 0x7f1cc806d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0b1a80 tx=0x7f1cd0007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.445+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cd40785f0 con 0x7f1cc4003b90 2024-04-25T08:37:50.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.692+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1cdc0c9d80 con 0x7f1cc4003b90 2024-04-25T08:37:50.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.693+0000 7f1cd9ffb640 1 -- 172.21.15.60:0/655372247 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f1cd4077ef0 con 0x7f1cc4003b90 2024-04-25T08:37:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:50.687 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":531,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1010,"data_bytes":1036438562,"bytes_used":2459377664,"bytes_avail":770634735616,"bytes_total":773094113280,"write_bytes_sec":86311973,"read_op_per_sec":0,"write_op_per_sec":84},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T08:37:09.505383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1cc806b4d0 msgr2=0x7f1cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1cc806b4d0 0x7f1cc806d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0b1a80 tx=0x7f1cd0007040 comp rx=0 tx=0).stop 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 msgr2=0x7f1cdc155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cdc155530 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1cd402a6d0 tx=0x7f1cd40012d0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 shutdown_connections 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1cc806b4d0 0x7f1cc806d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1cdc06f4d0 0x7f1cdc0cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1cdc0b4330 0x7f1cdc0c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 --2- 172.21.15.60:0/655372247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1cc4003b90 0x7f1cdc155530 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 >> 172.21.15.60:0/655372247 conn(0x7f1cdc0b3270 msgr2=0x7f1cdc159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 shutdown_connections 2024-04-25T08:37:50.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:50.695+0000 7f1ce38b3640 1 -- 172.21.15.60:0/655372247 wait complete. 2024-04-25T08:37:50.699 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:51.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:51.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.819+0000 7f5225f53640 1 Processor -- start 2024-04-25T08:37:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.820+0000 7f5225f53640 1 -- start start 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.820+0000 7f5225f53640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f52200b2cb0 0x7f52200d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.820+0000 7f5225f53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52200d1020 0x7f52200d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.820+0000 7f5225f53640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52200b97a0 con 0x7f522015dbe0 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.820+0000 7f5225f53640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522015dfc0 con 0x7f52200d1020 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.820+0000 7f5225f53640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52200d1a00 con 0x7f52200b2cb0 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52200d1020 0x7f52200d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f52200d1020 0x7f52200d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55620/0 (socket says 172.21.15.60:55620) 2024-04-25T08:37:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 -- 172.21.15.60:0/821372906 learned_addr learned my addr 172.21.15.60:0/821372906 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521effd640 1 --2- 172.21.15.60:0/821372906 >> v2:172.21.15.100:3300/0 conn(0x7f52200b2cb0 0x7f52200d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 -- 172.21.15.60:0/821372906 >> v2:172.21.15.100:3300/0 conn(0x7f52200b2cb0 msgr2=0x7f52200d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 --2- 172.21.15.60:0/821372906 >> v2:172.21.15.100:3300/0 conn(0x7f52200b2cb0 0x7f52200d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 -- 172.21.15.60:0/821372906 >> v1:172.21.15.100:6789/0 conn(0x7f522015dbe0 legacy=0x7f52200d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.821+0000 7f521ffff640 1 -- 172.21.15.60:0/821372906 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200d1c50 con 0x7f52200d1020 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.822+0000 7f521ffff640 1 --2- 172.21.15.60:0/821372906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f52200d2c70 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f522006cff0 tx=0x7f521802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f65fab8e2c8defd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.822+0000 7f521e7fc640 1 -- 172.21.15.60:0/821372906 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218036070 con 0x7f52200d1020 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.822+0000 7f521e7fc640 1 -- 172.21.15.60:0/821372906 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5218031600 con 0x7f52200d1020 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.822+0000 7f521e7fc640 1 -- 172.21.15.60:0/821372906 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52180319d0 con 0x7f52200d1020 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.822+0000 7f5225f53640 1 -- 172.21.15.60:0/821372906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 msgr2=0x7f52200d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.822+0000 7f5225f53640 1 --2- 172.21.15.60:0/821372906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f52200d2c70 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f522006cff0 tx=0x7f521802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.823+0000 7f5225f53640 1 -- 172.21.15.60:0/821372906 shutdown_connections 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.823+0000 7f5225f53640 1 --2- 172.21.15.60:0/821372906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f52200d2c70 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.823+0000 7f5225f53640 1 --2- 172.21.15.60:0/821372906 >> v2:172.21.15.100:3300/0 conn(0x7f52200b2cb0 0x7f52200d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.823+0000 7f5225f53640 1 -- 172.21.15.60:0/821372906 >> 172.21.15.60:0/821372906 conn(0x7f5220061a20 msgr2=0x7f5220061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.823+0000 7f5225f53640 1 -- 172.21.15.60:0/821372906 shutdown_connections 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.823+0000 7f5225f53640 1 -- 172.21.15.60:0/821372906 wait complete. 2024-04-25T08:37:51.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.824+0000 7f5225f53640 1 Processor -- start 2024-04-25T08:37:51.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.824+0000 7f5225f53640 1 -- start start 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f5225f53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52200b2cb0 0x7f52200cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f5225f53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f522015e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f5225f53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f522015dbe0 0x7f52200b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f5225f53640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200b92a0 con 0x7f52200d1020 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f5225f53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200d2210 con 0x7f522015dbe0 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f5225f53640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52200b9420 con 0x7f52200b2cb0 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f522015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:51.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f522015e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55636/0 (socket says 172.21.15.60:55636) 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 -- 172.21.15.60:0/3274473234 learned_addr learned my addr 172.21.15.60:0/3274473234 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521f7fe640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52200b2cb0 0x7f52200cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521ffff640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f522015dbe0 0x7f52200b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 -- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52200b2cb0 msgr2=0x7f52200cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52200b2cb0 0x7f52200cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 -- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f522015dbe0 msgr2=0x7f52200b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f522015dbe0 0x7f52200b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 -- 172.21.15.60:0/3274473234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200ccec0 con 0x7f52200d1020 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521f7fe640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52200b2cb0 0x7f52200cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.825+0000 7f521effd640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f522015e740 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f521400d7d0 tx=0x7f521400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:51.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.826+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400b6c0 con 0x7f52200d1020 2024-04-25T08:37:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.826+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214019070 con 0x7f52200d1020 2024-04-25T08:37:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.826+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52200cd1a0 con 0x7f52200d1020 2024-04-25T08:37:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.826+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400bac0 con 0x7f52200d1020 2024-04-25T08:37:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.826+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52200cd760 con 0x7f52200d1020 2024-04-25T08:37:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.827+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5220073990 con 0x7f52200d1020 2024-04-25T08:37:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.827+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f521400bc60 con 0x7f52200d1020 2024-04-25T08:37:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.828+0000 7f5207fff640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f520c06b4d0 0x7f520c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.828+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f521408a7e0 con 0x7f52200d1020 2024-04-25T08:37:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.828+0000 7f521f7fe640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f520c06b4d0 0x7f520c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:51.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.828+0000 7f521f7fe640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f520c06b4d0 0x7f520c06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5210006fd0 tx=0x7f5210008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:51.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:51.832+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f521401c360 con 0x7f52200d1020 2024-04-25T08:37:52.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.049+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52200d3400 con 0x7f52200d1020 2024-04-25T08:37:52.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.049+0000 7f5207fff640 1 -- 172.21.15.60:0/3274473234 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5214059560 con 0x7f52200d1020 2024-04-25T08:37:52.043 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:52.044 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:52.044 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f520c06b4d0 msgr2=0x7f520c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f520c06b4d0 0x7f520c06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5210006fd0 tx=0x7f5210008040 comp rx=0 tx=0).stop 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 msgr2=0x7f522015e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f522015e740 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f521400d7d0 tx=0x7f521400dc90 comp rx=0 tx=0).stop 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 shutdown_connections 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f520c06b4d0 0x7f520c06d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f522015dbe0 0x7f52200b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f52200d1020 0x7f522015e740 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 --2- 172.21.15.60:0/3274473234 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f52200b2cb0 0x7f52200cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 >> 172.21.15.60:0/3274473234 conn(0x7f5220061a20 msgr2=0x7f52200b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 shutdown_connections 2024-04-25T08:37:52.046 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.051+0000 7f5225f53640 1 -- 172.21.15.60:0/3274473234 wait complete. 2024-04-25T08:37:52.056 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:52.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:52.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.161+0000 7f9454ff5640 1 Processor -- start 2024-04-25T08:37:52.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.161+0000 7f9454ff5640 1 -- start start 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f9454ff5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f945015a790 0x7f94500d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f9454ff5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f945006fea0 0x7f94500d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f9454ff5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94500b3710 con 0x7f94500af9c0 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f9454ff5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500cf0e0 con 0x7f945006fea0 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f9454ff5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500cf3c0 con 0x7f945015a790 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f9445d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f945006fea0 0x7f94500d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:52.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f944ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f94500af9c0 0x7f9450072300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59058/0 (socket says 172.21.15.60:59058) 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.162+0000 7f944ed9d640 1 -- 172.21.15.60:0/423818160 learned_addr learned my addr 172.21.15.60:0/423818160 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f944e59c640 1 --2- 172.21.15.60:0/423818160 >> v2:172.21.15.100:3300/0 conn(0x7f945015a790 0x7f94500d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f9445d9b640 1 -- 172.21.15.60:0/423818160 >> v2:172.21.15.100:3300/0 conn(0x7f945015a790 msgr2=0x7f94500d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f9445d9b640 1 --2- 172.21.15.60:0/423818160 >> v2:172.21.15.100:3300/0 conn(0x7f945015a790 0x7f94500d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f9445d9b640 1 -- 172.21.15.60:0/423818160 >> v1:172.21.15.60:6789/0 conn(0x7f94500af9c0 legacy=0x7f9450072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f9445d9b640 1 -- 172.21.15.60:0/423818160 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500cf6a0 con 0x7f945006fea0 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f944dd9b640 1 -- 172.21.15.60:0/423818160 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236659187 0 0) 0x7f94500b3710 con 0x7f94500af9c0 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f9445d9b640 1 --2- 172.21.15.60:0/423818160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94500d08d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f94500b2e90 tx=0x7f943802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31fa904faa72f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.163+0000 7f944dd9b640 1 -- 172.21.15.60:0/423818160 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438037070 con 0x7f945006fea0 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f944dd9b640 1 -- 172.21.15.60:0/423818160 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438032440 con 0x7f945006fea0 2024-04-25T08:37:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f944dd9b640 1 -- 172.21.15.60:0/423818160 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438032810 con 0x7f945006fea0 2024-04-25T08:37:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 -- 172.21.15.60:0/423818160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 msgr2=0x7f94500d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 --2- 172.21.15.60:0/423818160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94500d08d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f94500b2e90 tx=0x7f943802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 -- 172.21.15.60:0/423818160 shutdown_connections 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 --2- 172.21.15.60:0/423818160 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94500d08d0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 --2- 172.21.15.60:0/423818160 >> v2:172.21.15.100:3300/0 conn(0x7f945015a790 0x7f94500d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 -- 172.21.15.60:0/423818160 >> 172.21.15.60:0/423818160 conn(0x7f94500b2370 msgr2=0x7f94500b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 -- 172.21.15.60:0/423818160 shutdown_connections 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.164+0000 7f9454ff5640 1 -- 172.21.15.60:0/423818160 wait complete. 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 Processor -- start 2024-04-25T08:37:52.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 -- start start 2024-04-25T08:37:52.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94501ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:52.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94500af9c0 0x7f945015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f945015a790 0x7f945015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015ab70 con 0x7f945006fea0 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500cf980 con 0x7f94500af9c0 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f9454ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945015acf0 con 0x7f945015a790 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f944e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94501ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f944e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94501ed3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55654/0 (socket says 172.21.15.60:55654) 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.166+0000 7f944e59c640 1 -- 172.21.15.60:0/3876796849 learned_addr learned my addr 172.21.15.60:0/3876796849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f9445d9b640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94500af9c0 0x7f945015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944ed9d640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f945015a790 0x7f945015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944e59c640 1 -- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f945015a790 msgr2=0x7f945015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944e59c640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f945015a790 0x7f945015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944e59c640 1 -- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94500af9c0 msgr2=0x7f945015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944e59c640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94500af9c0 0x7f945015d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944e59c640 1 -- 172.21.15.60:0/3876796849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94501f3f40 con 0x7f945006fea0 2024-04-25T08:37:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f9445d9b640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94500af9c0 0x7f945015d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f944e59c640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94501ed3b0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f944000d830 tx=0x7f944000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000b6c0 con 0x7f945006fea0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440019070 con 0x7f945006fea0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9450153d40 con 0x7f945006fea0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000ba80 con 0x7f945006fea0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.167+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9450151420 con 0x7f945006fea0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.168+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9450073990 con 0x7f945006fea0 2024-04-25T08:37:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.169+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f944001e460 con 0x7f945006fea0 2024-04-25T08:37:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.169+0000 7f94477fe640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.169+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f944008a9b0 con 0x7f945006fea0 2024-04-25T08:37:52.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.169+0000 7f9445d9b640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:52.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.170+0000 7f9445d9b640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f943006b4d0 0x7f943006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9438034000 tx=0x7f943802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:52.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.173+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9440059730 con 0x7f945006fea0 2024-04-25T08:37:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.429+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9450156c10 con 0x7f945006fea0 2024-04-25T08:37:52.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.429+0000 7f94477fe640 1 -- 172.21.15.60:0/3876796849 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f9440059030 con 0x7f945006fea0 2024-04-25T08:37:52.424 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:52.424 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":533,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2589949952,"bytes_avail":770504163328,"bytes_total":773094113280,"write_bytes_sec":89656045,"read_op_per_sec":0,"write_op_per_sec":88},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T08:37:09.505383+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.431+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f943006b4d0 msgr2=0x7f943006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.431+0000 7f9454ff5640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f943006b4d0 0x7f943006d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9438034000 tx=0x7f943802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.431+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 msgr2=0x7f94501ed3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.431+0000 7f9454ff5640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94501ed3b0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f944000d830 tx=0x7f944000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 shutdown_connections 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f945015a790 0x7f945015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94500af9c0 0x7f945015d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 --2- 172.21.15.60:0/3876796849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f945006fea0 0x7f94501ed3b0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 >> 172.21.15.60:0/3876796849 conn(0x7f94500b2370 msgr2=0x7f9450159760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 shutdown_connections 2024-04-25T08:37:52.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:52.432+0000 7f9454ff5640 1 -- 172.21.15.60:0/3876796849 wait complete. 2024-04-25T08:37:52.436 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:53.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.557+0000 7fd9860f5640 1 Processor -- start 2024-04-25T08:37:53.551 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.557+0000 7fd9860f5640 1 -- start start 2024-04-25T08:37:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd9860f5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd9800b2cb0 0x7fd9800d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd9860f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd9800d1020 0x7fd9800d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd9860f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9800b97a0 con 0x7fd98015dbe0 2024-04-25T08:37:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd9860f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98015dfc0 con 0x7fd9800d1020 2024-04-25T08:37:53.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd9860f5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9800d1a00 con 0x7fd9800b2cb0 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd98015dbe0 0x7fd9800d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59064/0 (socket says 172.21.15.60:59064) 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97f7fe640 1 -- 172.21.15.60:0/3945181858 learned_addr learned my addr 172.21.15.60:0/3945181858 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97ffff640 1 --2- 172.21.15.60:0/3945181858 >> v2:172.21.15.60:3300/0 conn(0x7fd9800d1020 0x7fd9800d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97effd640 1 --2- 172.21.15.60:0/3945181858 >> v2:172.21.15.100:3300/0 conn(0x7fd9800b2cb0 0x7fd9800d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97ffff640 1 -- 172.21.15.60:0/3945181858 >> v2:172.21.15.100:3300/0 conn(0x7fd9800b2cb0 msgr2=0x7fd9800d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97ffff640 1 --2- 172.21.15.60:0/3945181858 >> v2:172.21.15.100:3300/0 conn(0x7fd9800b2cb0 0x7fd9800d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97ffff640 1 -- 172.21.15.60:0/3945181858 >> v1:172.21.15.60:6789/0 conn(0x7fd98015dbe0 legacy=0x7fd9800d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97ffff640 1 -- 172.21.15.60:0/3945181858 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9800d1c50 con 0x7fd9800d1020 2024-04-25T08:37:53.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.558+0000 7fd97e7fc640 1 -- 172.21.15.60:0/3945181858 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160867112 0 0) 0x7fd9800b97a0 con 0x7fd98015dbe0 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.559+0000 7fd97ffff640 1 --2- 172.21.15.60:0/3945181858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9800d2c70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd98006cff0 tx=0x7fd97802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d22f923f19b2899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.559+0000 7fd97e7fc640 1 -- 172.21.15.60:0/3945181858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978007740 con 0x7fd9800d1020 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.559+0000 7fd97e7fc640 1 -- 172.21.15.60:0/3945181858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9780078e0 con 0x7fd9800d1020 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.559+0000 7fd97e7fc640 1 -- 172.21.15.60:0/3945181858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978007cb0 con 0x7fd9800d1020 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 -- 172.21.15.60:0/3945181858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 msgr2=0x7fd9800d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 --2- 172.21.15.60:0/3945181858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9800d2c70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd98006cff0 tx=0x7fd97802a680 comp rx=0 tx=0).stop 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 -- 172.21.15.60:0/3945181858 shutdown_connections 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 --2- 172.21.15.60:0/3945181858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9800d2c70 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 --2- 172.21.15.60:0/3945181858 >> v2:172.21.15.100:3300/0 conn(0x7fd9800b2cb0 0x7fd9800d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 -- 172.21.15.60:0/3945181858 >> 172.21.15.60:0/3945181858 conn(0x7fd980061a20 msgr2=0x7fd980061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 -- 172.21.15.60:0/3945181858 shutdown_connections 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.560+0000 7fd9860f5640 1 -- 172.21.15.60:0/3945181858 wait complete. 2024-04-25T08:37:53.555 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.561+0000 7fd9860f5640 1 Processor -- start 2024-04-25T08:37:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.561+0000 7fd9860f5640 1 -- start start 2024-04-25T08:37:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd9860f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9800b2cb0 0x7fd9801533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd9860f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9801538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd9860f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98015dbe0 0x7fd9800b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd9860f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800b92a0 con 0x7fd9800d1020 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd9860f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800d2210 con 0x7fd9800b2cb0 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd9860f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9800b9420 con 0x7fd98015dbe0 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9801538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9801538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55660/0 (socket says 172.21.15.60:55660) 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97effd640 1 -- 172.21.15.60:0/4052832614 learned_addr learned my addr 172.21.15.60:0/4052832614 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97f7fe640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9800b2cb0 0x7fd9801533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97ffff640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98015dbe0 0x7fd9800b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97effd640 1 -- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98015dbe0 msgr2=0x7fd9800b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.562+0000 7fd97effd640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98015dbe0 0x7fd9800b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd97effd640 1 -- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9800b2cb0 msgr2=0x7fd9801533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.558 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd97effd640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9800b2cb0 0x7fd9801533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd97effd640 1 -- 172.21.15.60:0/4052832614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980160680 con 0x7fd9800d1020 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd97effd640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9801538e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd97400d7d0 tx=0x7fd97400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400b6c0 con 0x7fd9800d1020 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974019070 con 0x7fd9800d1020 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9801f7a00 con 0x7fd9800d1020 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400bac0 con 0x7fd9800d1020 2024-04-25T08:37:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.563+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98015e710 con 0x7fd9800d1020 2024-04-25T08:37:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.564+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980073990 con 0x7fd9800d1020 2024-04-25T08:37:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.564+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd97400bc60 con 0x7fd9800d1020 2024-04-25T08:37:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.565+0000 7fd967fff640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd96c06b4d0 0x7fd96c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.565+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd97408a790 con 0x7fd9800d1020 2024-04-25T08:37:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.565+0000 7fd97f7fe640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd96c06b4d0 0x7fd96c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.560 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.565+0000 7fd97f7fe640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd96c06b4d0 0x7fd96c06d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd970007fd0 tx=0x7fd970007de0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:53.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.569+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd974058da0 con 0x7fd9800d1020 2024-04-25T08:37:53.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.786+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd980155390 con 0x7fd9800d1020 2024-04-25T08:37:53.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.787+0000 7fd967fff640 1 -- 172.21.15.60:0/4052832614 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd97405ce40 con 0x7fd9800d1020 2024-04-25T08:37:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:53.781 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:53.782 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd96c06b4d0 msgr2=0x7fd96c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd96c06b4d0 0x7fd96c06d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd970007fd0 tx=0x7fd970007de0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 msgr2=0x7fd9801538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9801538e0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd97400d7d0 tx=0x7fd97400dc90 comp rx=0 tx=0).stop 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 shutdown_connections 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd98015dbe0 0x7fd9800b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9800d1020 0x7fd9801538e0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd96c06b4d0 0x7fd96c06d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 --2- 172.21.15.60:0/4052832614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9800b2cb0 0x7fd9801533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 >> 172.21.15.60:0/4052832614 conn(0x7fd980061a20 msgr2=0x7fd9800c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:53.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 shutdown_connections 2024-04-25T08:37:53.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.789+0000 7fd9860f5640 1 -- 172.21.15.60:0/4052832614 wait complete. 2024-04-25T08:37:53.794 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:53.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:53.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.901+0000 7f2ad784a640 1 Processor -- start 2024-04-25T08:37:53.895 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.901+0000 7f2ad784a640 1 -- start start 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad784a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 0x7f2ad00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad784a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad784a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00b97a0 con 0x7f2ad00b92a0 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad784a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad015dfc0 con 0x7f2ad015dbe0 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad784a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00d1a00 con 0x7f2ad00b2cb0 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad4de5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad55e6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 0x7f2ad00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad4de5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad00d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:50362/0 (socket says 172.21.15.60:50362) 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad4de5640 1 -- 172.21.15.60:0/2973198424 learned_addr learned my addr 172.21.15.60:0/2973198424 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:53.896 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.902+0000 7f2ad55e6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 0x7f2ad00d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:59642/0 (socket says 172.21.15.60:59642) 2024-04-25T08:37:53.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ad4de5640 1 -- 172.21.15.60:0/2973198424 >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 msgr2=0x7f2ad00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:53.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ad4de5640 1 --2- 172.21.15.60:0/2973198424 >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 0x7f2ad00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.898 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ad4de5640 1 -- 172.21.15.60:0/2973198424 >> v1:172.21.15.100:6789/0 conn(0x7f2ad00b92a0 legacy=0x7f2ad00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ac7fff640 1 -- 172.21.15.60:0/2973198424 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469119969 0 0) 0x7f2ad00b97a0 con 0x7f2ad00b92a0 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ad4de5640 1 -- 172.21.15.60:0/2973198424 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00d1c50 con 0x7f2ad00b2cb0 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ad55e6640 1 --2- 172.21.15.60:0/2973198424 >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 0x7f2ad00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.903+0000 7f2ad4de5640 1 --2- 172.21.15.60:0/2973198424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad00d0e50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009870 tx=0x7f2ac002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22f038390661279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ac7fff640 1 -- 172.21.15.60:0/2973198424 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037070 con 0x7f2ad00b2cb0 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ac7fff640 1 -- 172.21.15.60:0/2973198424 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac0032440 con 0x7f2ad00b2cb0 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ac7fff640 1 -- 172.21.15.60:0/2973198424 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0032810 con 0x7f2ad00b2cb0 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 -- 172.21.15.60:0/2973198424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 msgr2=0x7f2ad00d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 --2- 172.21.15.60:0/2973198424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad00d0e50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009870 tx=0x7f2ac002a250 comp rx=0 tx=0).stop 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 -- 172.21.15.60:0/2973198424 shutdown_connections 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 --2- 172.21.15.60:0/2973198424 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad00d0e50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 --2- 172.21.15.60:0/2973198424 >> v2:172.21.15.100:3300/0 conn(0x7f2ad015dbe0 0x7f2ad00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 -- 172.21.15.60:0/2973198424 >> 172.21.15.60:0/2973198424 conn(0x7f2ad0061a20 msgr2=0x7f2ad0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 -- 172.21.15.60:0/2973198424 shutdown_connections 2024-04-25T08:37:53.899 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.904+0000 7f2ad784a640 1 -- 172.21.15.60:0/2973198424 wait complete. 2024-04-25T08:37:53.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.905+0000 7f2ad784a640 1 Processor -- start 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 -- start start 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 0x7f2ad015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad015dbe0 0x7f2ad00ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b6ab0 con 0x7f2ad00b92a0 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00d2210 con 0x7f2ad015dbe0 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad784a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b6c30 con 0x7f2ad00b2cb0 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad4de5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 0x7f2ad015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad4de5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 0x7f2ad015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55664/0 (socket says 172.21.15.60:55664) 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.906+0000 7f2ad4de5640 1 -- 172.21.15.60:0/3971704533 learned_addr learned my addr 172.21.15.60:0/3971704533 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad5de7640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad015dbe0 0x7f2ad00ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad55e6640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad4de5640 1 -- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 msgr2=0x7f2ad015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad4de5640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad4de5640 1 -- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad015dbe0 msgr2=0x7f2ad00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad4de5640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad015dbe0 0x7f2ad00ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad4de5640 1 -- 172.21.15.60:0/3971704533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad015faa0 con 0x7f2ad00b92a0 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad4de5640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 0x7f2ad015b420 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009870 tx=0x7f2ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0037040 con 0x7f2ad00b92a0 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac0004250 con 0x7f2ad00b92a0 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad015fd80 con 0x7f2ad00b92a0 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003610 con 0x7f2ad00b92a0 2024-04-25T08:37:53.903 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.907+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad00b4600 con 0x7f2ad00b92a0 2024-04-25T08:37:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.908+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad0073990 con 0x7f2ad00b92a0 2024-04-25T08:37:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.909+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ac00037b0 con 0x7f2ad00b92a0 2024-04-25T08:37:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.909+0000 7f2ac5ffb640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ab406b4d0 0x7f2ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.909+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ac00ab180 con 0x7f2ad00b92a0 2024-04-25T08:37:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.909+0000 7f2ad55e6640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ab406b4d0 0x7f2ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:53.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.910+0000 7f2ad55e6640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ab406b4d0 0x7f2ab406d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009560 tx=0x7f2acc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:53.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:53.914+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ac007a020 con 0x7f2ad00b92a0 2024-04-25T08:37:54.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.163+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2ad015c5a0 con 0x7f2ad00b92a0 2024-04-25T08:37:54.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.163+0000 7f2ac5ffb640 1 -- 172.21.15.60:0/3971704533 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f2ac0079920 con 0x7f2ad00b92a0 2024-04-25T08:37:54.158 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:54.158 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":534,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2589949952,"bytes_avail":770504163328,"bytes_total":773094113280,"write_bytes_sec":49836185,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T08:37:53.515641+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ab406b4d0 msgr2=0x7f2ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ab406b4d0 0x7f2ab406d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009560 tx=0x7f2acc008040 comp rx=0 tx=0).stop 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 msgr2=0x7f2ad015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 0x7f2ad015b420 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009870 tx=0x7f2ac00046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 shutdown_connections 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2ab406b4d0 0x7f2ab406d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad015dbe0 0x7f2ad00ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2ad00b92a0 0x7f2ad015b420 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 --2- 172.21.15.60:0/3971704533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2ad00b2cb0 0x7f2ad015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 >> 172.21.15.60:0/3971704533 conn(0x7f2ad0061a20 msgr2=0x7f2ad00b84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 shutdown_connections 2024-04-25T08:37:54.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.166+0000 7f2ad784a640 1 -- 172.21.15.60:0/3971704533 wait complete. 2024-04-25T08:37:54.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T08:37:54.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:54.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T08:37:54.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.280+0000 7f3d773c8640 1 Processor -- start 2024-04-25T08:37:54.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.280+0000 7f3d773c8640 1 -- start start 2024-04-25T08:37:54.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d773c8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d7015dc20 0x7f3d700ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d773c8640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7006cda0 con 0x7f3d70161710 2024-04-25T08:37:54.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d773c8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7015e000 con 0x7f3d700d32f0 2024-04-25T08:37:54.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d773c8640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d700d1de0 con 0x7f3d7015dc20 2024-04-25T08:37:54.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d75164640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d7015dc20 0x7f3d700ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d74963640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3d70161710 0x7f3d7006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59074/0 (socket says 172.21.15.60:59074) 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d75164640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3d7015dc20 0x7f3d700ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55672/0 (socket says 172.21.15.60:55672) 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d74963640 1 -- 172.21.15.60:0/1305760928 learned_addr learned my addr 172.21.15.60:0/1305760928 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d75164640 1 -- 172.21.15.60:0/1305760928 >> v1:172.21.15.100:6789/0 conn(0x7f3d700d32f0 legacy=0x7f3d7006c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d75164640 1 -- 172.21.15.60:0/1305760928 >> v1:172.21.15.60:6789/0 conn(0x7f3d70161710 legacy=0x7f3d7006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d75164640 1 -- 172.21.15.60:0/1305760928 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70161c60 con 0x7f3d7015dc20 2024-04-25T08:37:54.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.281+0000 7f3d67fff640 1 -- 172.21.15.60:0/1305760928 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197490765 0 0) 0x7f3d7006cda0 con 0x7f3d70161710 2024-04-25T08:37:54.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.282+0000 7f3d75164640 1 --2- 172.21.15.60:0/1305760928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d7015dc20 0x7f3d700ce670 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097a0 tx=0x7f3d6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d8f894b7ed26ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.282+0000 7f3d67fff640 1 -- 172.21.15.60:0/1305760928 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c037070 con 0x7f3d7015dc20 2024-04-25T08:37:54.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.282+0000 7f3d67fff640 1 -- 172.21.15.60:0/1305760928 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c032440 con 0x7f3d7015dc20 2024-04-25T08:37:54.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.282+0000 7f3d67fff640 1 -- 172.21.15.60:0/1305760928 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c032810 con 0x7f3d7015dc20 2024-04-25T08:37:54.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 -- 172.21.15.60:0/1305760928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d7015dc20 msgr2=0x7f3d700ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 --2- 172.21.15.60:0/1305760928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d7015dc20 0x7f3d700ce670 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097a0 tx=0x7f3d6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 -- 172.21.15.60:0/1305760928 shutdown_connections 2024-04-25T08:37:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 --2- 172.21.15.60:0/1305760928 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d7015dc20 0x7f3d700ce670 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 -- 172.21.15.60:0/1305760928 >> 172.21.15.60:0/1305760928 conn(0x7f3d700d0420 msgr2=0x7f3d700d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 -- 172.21.15.60:0/1305760928 shutdown_connections 2024-04-25T08:37:54.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.283+0000 7f3d773c8640 1 -- 172.21.15.60:0/1305760928 wait complete. 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.284+0000 7f3d773c8640 1 Processor -- start 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.284+0000 7f3d773c8640 1 -- start start 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d700d32f0 0x7f3d700c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7015dc20 0x7f3d7014b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 0x7f3d7014bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d70162500 con 0x7f3d70161710 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d70162220 con 0x7f3d700d32f0 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d700b5e50 con 0x7f3d7015dc20 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 0x7f3d7014bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 0x7f3d7014bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55674/0 (socket says 172.21.15.60:55674) 2024-04-25T08:37:54.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 -- 172.21.15.60:0/3053481944 learned_addr learned my addr 172.21.15.60:0/3053481944 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75164640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d700d32f0 0x7f3d700c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d74963640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7015dc20 0x7f3d7014b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 -- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7015dc20 msgr2=0x7f3d7014b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7015dc20 0x7f3d7014b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 -- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d700d32f0 msgr2=0x7f3d700c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d700d32f0 0x7f3d700c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 -- 172.21.15.60:0/3053481944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d701e9ed0 con 0x7f3d70161710 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75164640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d700d32f0 0x7f3d700c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d75965640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 0x7f3d7014bdb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800ba10 tx=0x7f3d6800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d65ffb640 1 -- 172.21.15.60:0/3053481944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68003b70 con 0x7f3d70161710 2024-04-25T08:37:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d65ffb640 1 -- 172.21.15.60:0/3053481944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68003d10 con 0x7f3d70161710 2024-04-25T08:37:54.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.285+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d701e9910 con 0x7f3d70161710 2024-04-25T08:37:54.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.286+0000 7f3d65ffb640 1 -- 172.21.15.60:0/3053481944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800cae0 con 0x7f3d70161710 2024-04-25T08:37:54.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.286+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d701498f0 con 0x7f3d70161710 2024-04-25T08:37:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.288+0000 7f3d65ffb640 1 -- 172.21.15.60:0/3053481944 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d6800cc80 con 0x7f3d70161710 2024-04-25T08:37:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.288+0000 7f3d65ffb640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d5406b4d0 0x7f3d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.288+0000 7f3d65ffb640 1 -- 172.21.15.60:0/3053481944 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d6808a090 con 0x7f3d70161710 2024-04-25T08:37:54.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.288+0000 7f3d75164640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d5406b4d0 0x7f3d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 Processor -- start 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 -- start start 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d75164640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d5406b4d0 0x7f3d5406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c034000 tx=0x7f3d6c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe770de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe770c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe7736be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe766eb90 con 0x55dbe770bca0 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe7617e40 con 0x55dbe7736760 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.289+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbe766ac30 con 0x55dbe7748370 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe770de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe770de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55676/0 (socket says 172.21.15.60:55676) 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 -- 172.21.15.60:0/777394844 learned_addr learned my addr 172.21.15.60:0/777394844 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d74963640 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe770c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 -- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 msgr2=0x55dbe770c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75965640 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe7736be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe770c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 -- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 msgr2=0x55dbe7736be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe7736be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 -- 172.21.15.60:0/777394844 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbe744d7d0 con 0x55dbe770bca0 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d74963640 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe770c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d75164640 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe770de00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02a7d0 tx=0x7f3d6c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e049282244b9214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d5affd640 1 -- 172.21.15.60:0/777394844 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c04e070 con 0x55dbe770bca0 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d5affd640 1 -- 172.21.15.60:0/777394844 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c032d90 con 0x55dbe770bca0 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.290+0000 7f3d5affd640 1 -- 172.21.15.60:0/777394844 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c037040 con 0x55dbe770bca0 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/777394844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 msgr2=0x55dbe770de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe770de00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02a7d0 tx=0x7f3d6c02c000 comp rx=0 tx=0).stop 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/777394844 shutdown_connections 2024-04-25T08:37:54.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe7736be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe770c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/777394844 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe770de00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/777394844 >> 172.21.15.60:0/777394844 conn(0x55dbe76d3870 msgr2=0x55dbe7751760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/777394844 shutdown_connections 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/777394844 wait complete. 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 -- start start 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.291+0000 7f3d7a1a4740 1 Processor -- start 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7733840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe774be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe774c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe762b3e0 con 0x55dbe770bca0 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe77342b0 con 0x55dbe7736760 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbe7734100 con 0x55dbe7748370 2024-04-25T08:37:54.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d7a1a4740 10 client.0 _ll_register_callbacks cb 0x55dbe762adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d75164640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7733840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d75164640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7733840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55688/0 (socket says 172.21.15.60:55688) 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.292+0000 7f3d75164640 1 -- 172.21.15.60:0/2056436641 learned_addr learned my addr 172.21.15.60:0/2056436641 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d74963640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe774be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75965640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe774c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75164640 1 -- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 msgr2=0x55dbe774c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75164640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe774c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75164640 1 -- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 msgr2=0x55dbe774be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75164640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe774be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75164640 1 -- 172.21.15.60:0/2056436641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbe73430f0 con 0x55dbe770bca0 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d75164640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7733840 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009560 tx=0x7f3d6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657b7b4005c510d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d4ffff640 1 -- 172.21.15.60:0/2056436641 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c04e070 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d4ffff640 1 -- 172.21.15.60:0/2056436641 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d6c03c910 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d6c032d90 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d4ffff640 1 -- 172.21.15.60:0/2056436641 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d6c03cb10 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d7a1a4740 10 client.9081 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55dbe74adf60 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d4ffff640 1 -- 172.21.15.60:0/2056436641 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c032d90 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d4ffff640 1 -- 172.21.15.60:0/2056436641 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d6c03ccb0 con 0x55dbe770bca0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d7a1a4740 10 client.9081 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d7a1a4740 10 client.9081 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.293+0000 7f3d7a1a4740 20 client.9081 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.294+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77388b0 0x55dbe7738c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.294+0000 7f3d7a1a4740 4 client.9081 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.294+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbe767c0d0 con 0x55dbe77388b0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.294+0000 7f3d74963640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77388b0 0x55dbe7738c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.294+0000 7f3d4ffff640 10 client.9081 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.294+0000 7f3d74963640 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77388b0 0x55dbe7738c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4802d0ec41e79ad server_cookie=5bf945c7a4c4e8c1 in_seq=0 out_seq=0 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d4ffff640 1 -- 172.21.15.60:0/2056436641 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dbe767c0d0 con 0x55dbe77388b0 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d4ffff640 10 client.9081 handle_command_reply: tid=0 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 client.9081 shutdown 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 msgr2=0x55dbe7733840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7733840 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009560 tx=0x7f3d6c004730 comp rx=0 tx=0).stop 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 shutdown_connections 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77388b0 0x55dbe7738c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe774c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe774be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d4ffff640 0 client.9081 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/2056436641 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7733840 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 >> 172.21.15.60:0/2056436641 conn(0x55dbe76d3870 msgr2=0x55dbe77346b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 shutdown_connections 2024-04-25T08:37:54.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/2056436641 wait complete. 2024-04-25T08:37:54.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.295+0000 7f3d7a1a4740 20 client.9081 trim_cache size 0 max 16384 2024-04-25T08:37:54.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.310+0000 7f3d7a1a4740 1 Processor -- start 2024-04-25T08:37:54.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.310+0000 7f3d7a1a4740 1 -- start start 2024-04-25T08:37:54.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.310+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe770bca0 0x55dbe7705cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 0x55dbe7739f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7748370 0x55dbe773a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d70200350 con 0x55dbe7736760 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d701ff4d0 con 0x55dbe7748370 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7014aff0 con 0x55dbe770bca0 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 0x55dbe7739f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 0x55dbe7739f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55694/0 (socket says 172.21.15.60:55694) 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 -- 172.21.15.60:0/349706867 learned_addr learned my addr 172.21.15.60:0/349706867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 -- 172.21.15.60:0/349706867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe770bca0 msgr2=0x55dbe7705cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe770bca0 0x55dbe7705cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 -- 172.21.15.60:0/349706867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7748370 msgr2=0x55dbe773a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7748370 0x55dbe773a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 -- 172.21.15.60:0/349706867 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbe731c1b0 con 0x55dbe7736760 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d74963640 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 0x55dbe7739f90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3d7014fef0 tx=0x7f3d60007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747d193730127047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d5a7fc640 1 -- 172.21.15.60:0/349706867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60017430 con 0x55dbe7736760 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d5a7fc640 1 -- 172.21.15.60:0/349706867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6001c020 con 0x55dbe7736760 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.311+0000 7f3d5a7fc640 1 -- 172.21.15.60:0/349706867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600175d0 con 0x55dbe7736760 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/349706867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 msgr2=0x55dbe7739f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 0x55dbe7739f90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3d7014fef0 tx=0x7f3d60007b90 comp rx=0 tx=0).stop 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/349706867 shutdown_connections 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7748370 0x55dbe773a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe7736760 0x55dbe7739f90 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/349706867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe770bca0 0x55dbe7705cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/349706867 >> 172.21.15.60:0/349706867 conn(0x55dbe76d3870 msgr2=0x55dbe774ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/349706867 shutdown_connections 2024-04-25T08:37:54.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/349706867 wait complete. 2024-04-25T08:37:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 -- start start 2024-04-25T08:37:54.309 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.312+0000 7f3d7a1a4740 1 Processor -- start 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7737800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe77bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe7739d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d701fef00 con 0x55dbe770bca0 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbe7702b40 con 0x55dbe7736760 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbe7773180 con 0x55dbe7748370 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d7a1a4740 10 client.0 _ll_register_callbacks cb 0x7f3d6c04c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d75164640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7737800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d75164640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7737800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55700/0 (socket says 172.21.15.60:55700) 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.313+0000 7f3d75164640 1 -- 172.21.15.60:0/3622720563 learned_addr learned my addr 172.21.15.60:0/3622720563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d74963640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe77bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75965640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe7739d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75164640 1 -- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 msgr2=0x55dbe7739d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75164640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe7739d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75164640 1 -- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 msgr2=0x55dbe77bb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75164640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe77bb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75164640 1 -- 172.21.15.60:0/3622720563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbe7773920 con 0x55dbe770bca0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75965640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe7739d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d75164640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7737800 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02a220 tx=0x7f3d6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f365f27f869ad4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d4ffff640 1 -- 172.21.15.60:0/3622720563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c062020 con 0x55dbe770bca0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d4ffff640 1 -- 172.21.15.60:0/3622720563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d6c054870 con 0x55dbe770bca0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d6001c020 con 0x55dbe770bca0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d4ffff640 1 -- 172.21.15.60:0/3622720563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d6c06c050 con 0x55dbe770bca0 2024-04-25T08:37:54.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d7a1a4740 10 client.9087 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55dbe773a8c0 con 0x55dbe770bca0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d4ffff640 1 -- 172.21.15.60:0/3622720563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6001c020 con 0x55dbe770bca0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.314+0000 7f3d4ffff640 1 -- 172.21.15.60:0/3622720563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d6c06f020 con 0x55dbe770bca0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 10 client.9087 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 10 client.9087 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 20 client.9087 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77535b0 0x55dbe773a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 4 client.9087 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55dbe7676420 con 0x55dbe77535b0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d74963640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77535b0 0x55dbe773a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d74963640 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77535b0 0x55dbe773a270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5214b3ca51fdf2d4 server_cookie=5a14cf23b004af68 in_seq=0 out_seq=0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d4ffff640 10 client.9087 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d4ffff640 1 -- 172.21.15.60:0/3622720563 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55dbe7676420 con 0x55dbe77535b0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d4ffff640 10 client.9087 handle_command_reply: tid=0 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 1 client.9087 shutdown 2024-04-25T08:37:54.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.315+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 msgr2=0x55dbe7737800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7737800 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c02a220 tx=0x7f3d6c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 shutdown_connections 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55dbe77535b0 0x55dbe773a270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55dbe7748370 0x55dbe7739d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dbe7736760 0x55dbe77bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 --2- 172.21.15.60:0/3622720563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55dbe770bca0 0x55dbe7737800 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 >> 172.21.15.60:0/3622720563 conn(0x55dbe76d3870 msgr2=0x55dbe7705cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d4ffff640 0 client.9087 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 shutdown_connections 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 1 -- 172.21.15.60:0/3622720563 wait complete. 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d7a1a4740 20 client.9087 trim_cache size 0 max 16384 2024-04-25T08:37:54.316 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:54.317 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:37:54.317 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d5406b4d0 msgr2=0x7f3d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d5406b4d0 0x7f3d5406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c034000 tx=0x7f3d6c02e000 comp rx=0 tx=0).stop 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 msgr2=0x7f3d7014bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 0x7f3d7014bdb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800ba10 tx=0x7f3d6800bed0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 shutdown_connections 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d70161710 0x7f3d7014bdb0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3d7015dc20 0x7f3d7014b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d5406b4d0 0x7f3d5406d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 --2- 172.21.15.60:0/3053481944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d700d32f0 0x7f3d700c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 >> 172.21.15.60:0/3053481944 conn(0x7f3d700d0420 msgr2=0x7f3d700c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 shutdown_connections 2024-04-25T08:37:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.316+0000 7f3d773c8640 1 -- 172.21.15.60:0/3053481944 wait complete. 2024-04-25T08:37:54.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:54.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T08:37:54.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.429+0000 7f5e8e6db640 1 Processor -- start 2024-04-25T08:37:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.429+0000 7f5e8e6db640 1 -- start start 2024-04-25T08:37:54.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e8e6db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 0x7f5e880ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e8e6db640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8806cda0 con 0x7f5e88161710 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e8e6db640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8815e000 con 0x7f5e880d32f0 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e8e6db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880d1de0 con 0x7f5e8815dc20 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e877fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5e88161710 0x7f5e8806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59076/0 (socket says 172.21.15.60:59076) 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e877fe640 1 -- 172.21.15.60:0/780660479 learned_addr learned my addr 172.21.15.60:0/780660479 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.430+0000 7f5e87fff640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 0x7f5e880ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341327995 0 0) 0x7f5e8806cda0 con 0x7f5e88161710 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003520 con 0x7f5e88161710 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739001510 0 0) 0x7f5e8815e000 con 0x7f5e880d32f0 2024-04-25T08:37:54.428 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e8806cda0 con 0x7f5e880d32f0 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 297273505 0 0) 0x7f5e74003520 con 0x7f5e88161710 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 msgr2=0x7f5e880ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 0x7f5e880ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 >> v1:172.21.15.100:6789/0 conn(0x7f5e880d32f0 legacy=0x7f5e8806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e88161c60 con 0x7f5e88161710 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f5e70001670 con 0x7f5e88161710 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 >> v1:172.21.15.60:6789/0 conn(0x7f5e88161710 legacy=0x7f5e8806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74003b70 0x7f5e74003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 0x7f5e74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e74004d40 0x7f5e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880d1de0 con 0x7f5e74004d40 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400ba70 con 0x7f5e74003b70 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400bc70 con 0x7f5e8815dc20 2024-04-25T08:37:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e74004d40 0x7f5e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e87fff640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74003b70 0x7f5e74003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e877fe640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 0x7f5e74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e8cc78640 1 -- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 msgr2=0x7f5e74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e8815dc20 0x7f5e74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e8cc78640 1 -- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74003b70 msgr2=0x7f5e74003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74003b70 0x7f5e74003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.431+0000 7f5e8cc78640 1 -- 172.21.15.60:0/780660479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74003520 con 0x7f5e74004d40 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.432+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e74004d40 0x7f5e740051c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5e80001fa0 tx=0x7f5e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12f4948aa2eaf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.432+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80001bf0 con 0x7f5e74004d40 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.432+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80003b70 con 0x7f5e74004d40 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.432+0000 7f5e86ffd640 1 -- 172.21.15.60:0/780660479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800050d0 con 0x7f5e74004d40 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8e6db640 1 -- 172.21.15.60:0/780660479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e74004d40 msgr2=0x7f5e740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/780660479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e74004d40 0x7f5e740051c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5e80001fa0 tx=0x7f5e74005830 comp rx=0 tx=0).stop 2024-04-25T08:37:54.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8cc78640 1 -- 172.21.15.60:0/780660479 reap_dead start 2024-04-25T08:37:54.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8e6db640 1 -- 172.21.15.60:0/780660479 shutdown_connections 2024-04-25T08:37:54.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8e6db640 1 -- 172.21.15.60:0/780660479 >> 172.21.15.60:0/780660479 conn(0x7f5e880d0420 msgr2=0x7f5e880d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8e6db640 1 -- 172.21.15.60:0/780660479 shutdown_connections 2024-04-25T08:37:54.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.433+0000 7f5e8e6db640 1 -- 172.21.15.60:0/780660479 wait complete. 2024-04-25T08:37:54.431 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.434+0000 7f5e8e6db640 1 Processor -- start 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.434+0000 7f5e8e6db640 1 -- start start 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8e6db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004d40 0x7f5e8814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8e6db640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e880aa380 0x7f5e880aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8e6db640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 0x7f5e8814c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8e6db640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88162500 con 0x7f5e8814c4e0 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8e6db640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88162220 con 0x7f5e74004d40 2024-04-25T08:37:54.440 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8e6db640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880b5e50 con 0x7f5e880aa380 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 0x7f5e8814c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 0x7f5e8814c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55714/0 (socket says 172.21.15.60:55714) 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 -- 172.21.15.60:0/4065725854 learned_addr learned my addr 172.21.15.60:0/4065725854 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e877fe640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e880aa380 0x7f5e880aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e87fff640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004d40 0x7f5e8814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 -- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e880aa380 msgr2=0x7f5e880aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e880aa380 0x7f5e880aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 -- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004d40 msgr2=0x7f5e8814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004d40 0x7f5e8814d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.435+0000 7f5e8cc78640 1 -- 172.21.15.60:0/4065725854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8814cea0 con 0x7f5e8814c4e0 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.436+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 0x7f5e8814c960 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005040 tx=0x7f5e80001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.436+0000 7f5e84ff9640 1 -- 172.21.15.60:0/4065725854 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80001bf0 con 0x7f5e8814c4e0 2024-04-25T08:37:54.441 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.436+0000 7f5e84ff9640 1 -- 172.21.15.60:0/4065725854 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80044070 con 0x7f5e8814c4e0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.436+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e880cac80 con 0x7f5e8814c4e0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.436+0000 7f5e84ff9640 1 -- 172.21.15.60:0/4065725854 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8003b670 con 0x7f5e8814c4e0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.436+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e880cb240 con 0x7f5e8814c4e0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.437+0000 7f5e84ff9640 1 -- 172.21.15.60:0/4065725854 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e8003b810 con 0x7f5e8814c4e0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.438+0000 7f5e84ff9640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6806b500 0x7f5e6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.438+0000 7f5e84ff9640 1 -- 172.21.15.60:0/4065725854 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e800aaa00 con 0x7f5e8814c4e0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.438+0000 7f5e87fff640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6806b500 0x7f5e6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.439+0000 7f5e87fff640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6806b500 0x7f5e6806d9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009bc0 tx=0x7f5e78009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.439+0000 7f5e913dd740 1 Processor -- start 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.439+0000 7f5e913dd740 1 -- start start 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 0x55cd5a789320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786ef0 0x55cd5a789860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.442 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a759770 0x55cd5a75cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e913dd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a6c2da0 con 0x55cd5a786b10 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a66bb80 con 0x55cd5a759770 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a6bee40 con 0x55cd5a786ef0 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 0x55cd5a789320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 0x55cd5a789320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55716/0 (socket says 172.21.15.60:55716) 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 -- 172.21.15.60:0/919292437 learned_addr learned my addr 172.21.15.60:0/919292437 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e877fe640 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786ef0 0x55cd5a789860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a759770 0x55cd5a75cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 -- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786ef0 msgr2=0x55cd5a789860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786ef0 0x55cd5a789860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 -- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a759770 msgr2=0x55cd5a75cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a759770 0x55cd5a75cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 -- 172.21.15.60:0/919292437 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5a675f10 con 0x55cd5a786b10 2024-04-25T08:37:54.443 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e877fe640 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786ef0 0x55cd5a789860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e87fff640 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 0x55cd5a789320 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006d40 tx=0x7f5e78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cd424e58466cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.440+0000 7f5e7dffb640 1 -- 172.21.15.60:0/919292437 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7801b070 con 0x55cd5a786b10 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e7dffb640 1 -- 172.21.15.60:0/919292437 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78003da0 con 0x55cd5a786b10 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e7dffb640 1 -- 172.21.15.60:0/919292437 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78022630 con 0x55cd5a786b10 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 -- 172.21.15.60:0/919292437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 msgr2=0x55cd5a789320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 0x55cd5a789320 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006d40 tx=0x7f5e78017000 comp rx=0 tx=0).stop 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 -- 172.21.15.60:0/919292437 shutdown_connections 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a759770 0x55cd5a75cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786ef0 0x55cd5a789860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 --2- 172.21.15.60:0/919292437 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786b10 0x55cd5a789320 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 -- 172.21.15.60:0/919292437 >> 172.21.15.60:0/919292437 conn(0x55cd5a75b9e0 msgr2=0x55cd5a7a4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 -- 172.21.15.60:0/919292437 shutdown_connections 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.441+0000 7f5e913dd740 1 -- 172.21.15.60:0/919292437 wait complete. 2024-04-25T08:37:54.444 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.442+0000 7f5e913dd740 1 -- start start 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.442+0000 7f5e913dd740 1 Processor -- start 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786b10 0x55cd5a784e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786ef0 0x55cd5a7aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 0x55cd5a78a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a75eb90 con 0x55cd5a7a99a0 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a6614b0 con 0x55cd5a786ef0 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a75ee70 con 0x55cd5a786b10 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 10 client.0 _ll_register_callbacks cb 0x55cd5a6a1180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 0x55cd5a78a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 0x55cd5a78a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55730/0 (socket says 172.21.15.60:55730) 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 -- 172.21.15.60:0/1158850128 learned_addr learned my addr 172.21.15.60:0/1158850128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e87fff640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786b10 0x55cd5a784e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.445 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e877fe640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786ef0 0x55cd5a7aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 -- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786b10 msgr2=0x55cd5a784e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786b10 0x55cd5a784e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 -- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786ef0 msgr2=0x55cd5a7aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786ef0 0x55cd5a7aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 -- 172.21.15.60:0/1158850128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5a4a17d0 con 0x55cd5a7a99a0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 0x55cd5a78a5c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5e800812f0 tx=0x7f5e80078a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997f1c06ea181804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e66ffd640 1 -- 172.21.15.60:0/1158850128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800a8060 con 0x55cd5a7a99a0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e66ffd640 1 -- 172.21.15.60:0/1158850128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e80077430 con 0x55cd5a7a99a0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.443+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e78003da0 con 0x55cd5a7a99a0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e66ffd640 1 -- 172.21.15.60:0/1158850128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e80081820 con 0x55cd5a7a99a0 2024-04-25T08:37:54.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 10 client.9099 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55cd5a3d73f0 con 0x55cd5a7a99a0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e66ffd640 1 -- 172.21.15.60:0/1158850128 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78003da0 con 0x55cd5a7a99a0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e66ffd640 1 -- 172.21.15.60:0/1158850128 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5e8003bb00 con 0x55cd5a7a99a0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 10 client.9099 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 10 client.9099 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 20 client.9099 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a7a7af0 0x55cd5a7c1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 4 client.9099 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5a6d02e0 con 0x55cd5a7a7af0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.444+0000 7f5e87fff640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a7a7af0 0x55cd5a7c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e66ffd640 10 client.9099 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e87fff640 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a7a7af0 0x55cd5a7c1850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51376982d52c674 server_cookie=af0bfdf2bc6d4057 in_seq=0 out_seq=0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e66ffd640 1 -- 172.21.15.60:0/1158850128 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd5a6d02e0 con 0x55cd5a7a7af0 2024-04-25T08:37:54.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e66ffd640 10 client.9099 handle_command_reply: tid=0 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 client.9099 shutdown 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 msgr2=0x55cd5a78a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 0x55cd5a78a5c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5e800812f0 tx=0x7f5e80078a90 comp rx=0 tx=0).stop 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 shutdown_connections 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a7a7af0 0x55cd5a7c1850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a7a99a0 0x55cd5a78a5c0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786ef0 0x55cd5a7aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e66ffd640 0 client.9099 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1158850128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a786b10 0x55cd5a784e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 >> 172.21.15.60:0/1158850128 conn(0x55cd5a759770 msgr2=0x55cd5a758600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 shutdown_connections 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.445+0000 7f5e913dd740 1 -- 172.21.15.60:0/1158850128 wait complete. 2024-04-25T08:37:54.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.446+0000 7f5e913dd740 20 client.9099 trim_cache size 0 max 16384 2024-04-25T08:37:54.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.461+0000 7f5e913dd740 1 Processor -- start 2024-04-25T08:37:54.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.461+0000 7f5e913dd740 1 -- start start 2024-04-25T08:37:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.461+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a75edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.461+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a75e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e913dd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88200ac0 con 0x55cd5a786ef0 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e881f7980 con 0x55cd5a786b10 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880cc410 con 0x55cd5a7a99a0 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55740/0 (socket says 172.21.15.60:55740) 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 -- 172.21.15.60:0/1213690953 learned_addr learned my addr 172.21.15.60:0/1213690953 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e87fff640 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a75edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a75e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 -- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 msgr2=0x55cd5a75e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a75e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 -- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 msgr2=0x55cd5a75edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a75edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 -- 172.21.15.60:0/1213690953 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5a3970f0 con 0x55cd5a786ef0 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e87fff640 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a75edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e877fe640 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c6070 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007d90 tx=0x7f5e70000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35521c1fee213b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e7d7fa640 1 -- 172.21.15.60:0/1213690953 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70004c70 con 0x55cd5a786ef0 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e7d7fa640 1 -- 172.21.15.60:0/1213690953 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70004e10 con 0x55cd5a786ef0 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.462+0000 7f5e7d7fa640 1 -- 172.21.15.60:0/1213690953 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70017650 con 0x55cd5a786ef0 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 -- 172.21.15.60:0/1213690953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 msgr2=0x55cd5a7c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c6070 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007d90 tx=0x7f5e70000f80 comp rx=0 tx=0).stop 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 -- 172.21.15.60:0/1213690953 shutdown_connections 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a75e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c6070 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 --2- 172.21.15.60:0/1213690953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a75edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 -- 172.21.15.60:0/1213690953 >> 172.21.15.60:0/1213690953 conn(0x55cd5a759770 msgr2=0x55cd5a7aa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 -- 172.21.15.60:0/1213690953 shutdown_connections 2024-04-25T08:37:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 -- 172.21.15.60:0/1213690953 wait complete. 2024-04-25T08:37:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.463+0000 7f5e913dd740 1 -- start start 2024-04-25T08:37:54.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 Processor -- start 2024-04-25T08:37:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a80e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a7c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a7ac770 con 0x55cd5a786ef0 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88150040 con 0x55cd5a786b10 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5a6844b0 con 0x55cd5a7a99a0 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e913dd740 10 client.0 _ll_register_callbacks cb 0x7f5e780098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.464+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55746/0 (socket says 172.21.15.60:55746) 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 -- 172.21.15.60:0/2165280757 learned_addr learned my addr 172.21.15.60:0/2165280757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e8cc78640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a7c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e87fff640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a80e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 -- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 msgr2=0x55cd5a7c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a7c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 -- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 msgr2=0x55cd5a80e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a80e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 -- 172.21.15.60:0/2165280757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5a80e650 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e877fe640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c5ca0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007490 tx=0x7f5e88161e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248ce4eaa9ded413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e66ffd640 1 -- 172.21.15.60:0/2165280757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70004c70 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e66ffd640 1 -- 172.21.15.60:0/2165280757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e70027990 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e70004e10 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e66ffd640 1 -- 172.21.15.60:0/2165280757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e7002e050 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e913dd740 10 client.9105 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55cd5a8127f0 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.465+0000 7f5e66ffd640 1 -- 172.21.15.60:0/2165280757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70004e10 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e66ffd640 1 -- 172.21.15.60:0/2165280757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5e70031020 con 0x55cd5a786ef0 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 10 client.9105 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 10 client.9105 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:54.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 20 client.9105 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a785370 0x55cd5a783200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 4 client.9105 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd5a6ca630 con 0x55cd5a785370 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e87fff640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a785370 0x55cd5a783200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e87fff640 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a785370 0x55cd5a783200 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a17a02bc1cc4424 server_cookie=6f4591eec3cb3678 in_seq=0 out_seq=0 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e66ffd640 10 client.9105 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e66ffd640 1 -- 172.21.15.60:0/2165280757 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x55cd5a6ca630 con 0x55cd5a785370 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e66ffd640 10 client.9105 handle_command_reply: tid=0 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.466+0000 7f5e913dd740 1 client.9105 shutdown 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 msgr2=0x55cd5a7c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c5ca0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f5e70007490 tx=0x7f5e88161e30 comp rx=0 tx=0).stop 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 shutdown_connections 2024-04-25T08:37:54.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55cd5a785370 0x55cd5a783200 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55cd5a7a99a0 0x55cd5a7c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55cd5a786ef0 0x55cd5a7c5ca0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e66ffd640 0 client.9105 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 --2- 172.21.15.60:0/2165280757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cd5a786b10 0x55cd5a80e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 >> 172.21.15.60:0/2165280757 conn(0x55cd5a759770 msgr2=0x55cd5a789310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 shutdown_connections 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 1 -- 172.21.15.60:0/2165280757 wait complete. 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e913dd740 20 client.9105 trim_cache size 0 max 16384 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:37:54.467 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:37:54.468 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6806b500 msgr2=0x7f5e6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6806b500 0x7f5e6806d9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009bc0 tx=0x7f5e78009190 comp rx=0 tx=0).stop 2024-04-25T08:37:54.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 msgr2=0x7f5e8814c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 0x7f5e8814c960 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005040 tx=0x7f5e80001650 comp rx=0 tx=0).stop 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 shutdown_connections 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e8814c4e0 0x7f5e8814c960 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e880aa380 0x7f5e880aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e6806b500 0x7f5e6806d9c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 --2- 172.21.15.60:0/4065725854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e74004d40 0x7f5e8814d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 >> 172.21.15.60:0/4065725854 conn(0x7f5e880d0420 msgr2=0x7f5e880b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 shutdown_connections 2024-04-25T08:37:54.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.467+0000 7f5e8e6db640 1 -- 172.21.15.60:0/4065725854 wait complete. 2024-04-25T08:37:54.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:54.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.579+0000 7f5965313640 1 Processor -- start 2024-04-25T08:37:54.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.579+0000 7f5965313640 1 -- start start 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f5965313640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596015b9c0 0x7f596015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f5965313640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f596006abe0 0x7f596006b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f5965313640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59600cfa60 con 0x7f5960072620 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f5965313640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600731a0 con 0x7f596006abe0 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f5965313640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596006b580 con 0x7f596015b9c0 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f595f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5960072620 0x7f5960072a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59092/0 (socket says 172.21.15.60:59092) 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.580+0000 7f595f7fe640 1 -- 172.21.15.60:0/3634179797 learned_addr learned my addr 172.21.15.60:0/3634179797 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595effd640 1 --2- 172.21.15.60:0/3634179797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596015b9c0 0x7f596015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595e7fc640 1 --2- 172.21.15.60:0/3634179797 >> v2:172.21.15.100:3300/0 conn(0x7f596006abe0 0x7f596006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595dffb640 1 -- 172.21.15.60:0/3634179797 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79667468 0 0) 0x7f59600cfa60 con 0x7f5960072620 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595dffb640 1 -- 172.21.15.60:0/3634179797 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f594c003520 con 0x7f5960072620 2024-04-25T08:37:54.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595effd640 1 -- 172.21.15.60:0/3634179797 >> v2:172.21.15.100:3300/0 conn(0x7f596006abe0 msgr2=0x7f596006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595effd640 1 --2- 172.21.15.60:0/3634179797 >> v2:172.21.15.100:3300/0 conn(0x7f596006abe0 0x7f596006b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595effd640 1 -- 172.21.15.60:0/3634179797 >> v1:172.21.15.60:6789/0 conn(0x7f5960072620 legacy=0x7f5960072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595dffb640 1 -- 172.21.15.60:0/3634179797 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111217441 0 0) 0x7f594c003520 con 0x7f5960072620 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.581+0000 7f595effd640 1 -- 172.21.15.60:0/3634179797 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600cfd40 con 0x7f596015b9c0 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f595effd640 1 --2- 172.21.15.60:0/3634179797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596015b9c0 0x7f596015bda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f59580097a0 tx=0x7f595802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae512ad7ec65aa6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f595dffb640 1 -- 172.21.15.60:0/3634179797 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958037070 con 0x7f596015b9c0 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f595dffb640 1 -- 172.21.15.60:0/3634179797 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595802aba0 con 0x7f596015b9c0 2024-04-25T08:37:54.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f595dffb640 1 -- 172.21.15.60:0/3634179797 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500019f0 con 0x7f596015b9c0 2024-04-25T08:37:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f5965313640 1 -- 172.21.15.60:0/3634179797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596015b9c0 msgr2=0x7f596015bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f5965313640 1 --2- 172.21.15.60:0/3634179797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596015b9c0 0x7f596015bda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f59580097a0 tx=0x7f595802a110 comp rx=0 tx=0).stop 2024-04-25T08:37:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f5965313640 1 -- 172.21.15.60:0/3634179797 shutdown_connections 2024-04-25T08:37:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f5965313640 1 --2- 172.21.15.60:0/3634179797 >> v2:172.21.15.100:3300/0 conn(0x7f596006abe0 0x7f596006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f5965313640 1 --2- 172.21.15.60:0/3634179797 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596015b9c0 0x7f596015bda0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.582+0000 7f5965313640 1 -- 172.21.15.60:0/3634179797 >> 172.21.15.60:0/3634179797 conn(0x7f59600b2370 msgr2=0x7f59600b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.583+0000 7f5965313640 1 -- 172.21.15.60:0/3634179797 shutdown_connections 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.583+0000 7f5965313640 1 -- 172.21.15.60:0/3634179797 wait complete. 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.584+0000 7f5965313640 1 Processor -- start 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.584+0000 7f5965313640 1 -- start start 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596006abe0 0x7f596013b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5960072620 0x7f5960154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 0x7f596013bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600cf710 con 0x7f596015b9c0 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600d0300 con 0x7f5960072620 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596015c2e0 con 0x7f596006abe0 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 0x7f596013bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 0x7f596013bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55754/0 (socket says 172.21.15.60:55754) 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 -- 172.21.15.60:0/2443037946 learned_addr learned my addr 172.21.15.60:0/2443037946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595effd640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596006abe0 0x7f596013b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595e7fc640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5960072620 0x7f5960154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 -- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596006abe0 msgr2=0x7f596013b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596006abe0 0x7f596013b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 -- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5960072620 msgr2=0x7f5960154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5960072620 0x7f5960154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 -- 172.21.15.60:0/2443037946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960153720 con 0x7f596015b9c0 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595e7fc640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5960072620 0x7f5960154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595effd640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596006abe0 0x7f596013b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:54.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f595f7fe640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 0x7f596013bb30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5950001b60 tx=0x7f595000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950017070 con 0x7f596015b9c0 2024-04-25T08:37:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5950012400 con 0x7f596015b9c0 2024-04-25T08:37:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500127b0 con 0x7f596015b9c0 2024-04-25T08:37:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59601539c0 con 0x7f596015b9c0 2024-04-25T08:37:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.585+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5960153f80 con 0x7f596015b9c0 2024-04-25T08:37:54.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.587+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5960073990 con 0x7f596015b9c0 2024-04-25T08:37:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.587+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f595001a070 con 0x7f596015b9c0 2024-04-25T08:37:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.587+0000 7f59577fe640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f594006b4d0 0x7f594006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.588+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f595008a1b0 con 0x7f596015b9c0 2024-04-25T08:37:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.588+0000 7f595effd640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f594006b4d0 0x7f594006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.588+0000 7f595effd640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f594006b4d0 0x7f594006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5958034000 tx=0x7f595802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.592+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5950058860 con 0x7f596015b9c0 2024-04-25T08:37:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.809+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59601e1910 con 0x7f596015b9c0 2024-04-25T08:37:54.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.809+0000 7f59577fe640 1 -- 172.21.15.60:0/2443037946 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f595005c900 con 0x7f596015b9c0 2024-04-25T08:37:54.804 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:54.804 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:54.804 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:54.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f594006b4d0 msgr2=0x7f594006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f594006b4d0 0x7f594006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5958034000 tx=0x7f595802e000 comp rx=0 tx=0).stop 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 msgr2=0x7f596013bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 0x7f596013bb30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5950001b60 tx=0x7f595000a430 comp rx=0 tx=0).stop 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 shutdown_connections 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f596015b9c0 0x7f596013bb30 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f594006b4d0 0x7f594006d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5960072620 0x7f5960154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 --2- 172.21.15.60:0/2443037946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f596006abe0 0x7f596013b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 >> 172.21.15.60:0/2443037946 conn(0x7f59600b2370 msgr2=0x7f59601514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 shutdown_connections 2024-04-25T08:37:54.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.811+0000 7f5965313640 1 -- 172.21.15.60:0/2443037946 wait complete. 2024-04-25T08:37:54.817 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:54.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.922+0000 7f84d73a5640 1 Processor -- start 2024-04-25T08:37:54.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.922+0000 7f84d73a5640 1 -- start start 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d73a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 0x7f84d00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d73a5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d015e810 con 0x7f84d015dd20 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d73a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d015eac0 con 0x7f84d00d0fe0 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d73a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00b9b30 con 0x7f84d00b96d0 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d5141640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f84d015dd20 0x7f84d015e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59102/0 (socket says 172.21.15.60:59102) 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d5141640 1 -- 172.21.15.60:0/1254268281 learned_addr learned my addr 172.21.15.60:0/1254268281 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84d4940640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 0x7f84d00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.923+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895254300 0 0) 0x7f84d015e810 con 0x7f84d015dd20 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b8003520 con 0x7f84d015dd20 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928919915 0 0) 0x7f84d015eac0 con 0x7f84d00d0fe0 2024-04-25T08:37:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d015e810 con 0x7f84d00d0fe0 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544161097 0 0) 0x7f84b8003520 con 0x7f84d015dd20 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 msgr2=0x7f84d00d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 0x7f84d00d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 >> v1:172.21.15.100:6789/0 conn(0x7f84d00d0fe0 legacy=0x7f84d00d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d015eda0 con 0x7f84d015dd20 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f84c80015f0 con 0x7f84d015dd20 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 >> v1:172.21.15.60:6789/0 conn(0x7f84d015dd20 legacy=0x7f84d015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b8003b70 0x7f84b8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 0x7f84b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b8004d40 0x7f84b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00b9b30 con 0x7f84b8004d40 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b800bb50 con 0x7f84b8003b70 2024-04-25T08:37:54.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b800bd20 con 0x7f84d00b96d0 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5942640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b8004d40 0x7f84b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d4940640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 0x7f84b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5141640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b8003b70 0x7f84b8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5942640 1 -- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 msgr2=0x7f84b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5942640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84d00b96d0 0x7f84b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5942640 1 -- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b8003b70 msgr2=0x7f84b8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5942640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b8003b70 0x7f84b8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.924+0000 7f84d5942640 1 -- 172.21.15.60:0/1254268281 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8003520 con 0x7f84b8004d40 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.925+0000 7f84d5942640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b8004d40 0x7f84b80051c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f84c0002000 tx=0x7f84b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc34e466782d475f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.925+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0001bf0 con 0x7f84b8004d40 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.925+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c0003b70 con 0x7f84b8004d40 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.925+0000 7f84cffff640 1 -- 172.21.15.60:0/1254268281 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00050d0 con 0x7f84b8004d40 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d73a5640 1 -- 172.21.15.60:0/1254268281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b8004d40 msgr2=0x7f84b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1254268281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84b8004d40 0x7f84b80051c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f84c0002000 tx=0x7f84b8005830 comp rx=0 tx=0).stop 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d5942640 1 -- 172.21.15.60:0/1254268281 reap_dead start 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d73a5640 1 -- 172.21.15.60:0/1254268281 shutdown_connections 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d73a5640 1 -- 172.21.15.60:0/1254268281 >> 172.21.15.60:0/1254268281 conn(0x7f84d0061a20 msgr2=0x7f84d0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d73a5640 1 -- 172.21.15.60:0/1254268281 shutdown_connections 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.926+0000 7f84d73a5640 1 -- 172.21.15.60:0/1254268281 wait complete. 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.927+0000 7f84d73a5640 1 Processor -- start 2024-04-25T08:37:54.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.927+0000 7f84d73a5640 1 -- start start 2024-04-25T08:37:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d73a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84b8004d40 0x7f84d00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d73a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 0x7f84d00c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d73a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d0156fc0 0x7f84d00d2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d73a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00d1e50 con 0x7f84d0157820 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d73a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00d1c00 con 0x7f84d0156fc0 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d73a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00d0aa0 con 0x7f84b8004d40 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 0x7f84d00c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 0x7f84d00c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55778/0 (socket says 172.21.15.60:55778) 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 -- 172.21.15.60:0/1634549204 learned_addr learned my addr 172.21.15.60:0/1634549204 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d5141640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84b8004d40 0x7f84d00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d5942640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d0156fc0 0x7f84d00d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 -- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84b8004d40 msgr2=0x7f84d00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84b8004d40 0x7f84d00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 -- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d0156fc0 msgr2=0x7f84d00d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d0156fc0 0x7f84d00d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:54.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 -- 172.21.15.60:0/1634549204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00cc280 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84d4940640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 0x7f84d00c8510 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f84c4011a00 tx=0x7f84c4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.928+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400fb50 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.929+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c400fcf0 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.929+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d0157ca0 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.929+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400fea0 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.929+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d00cc800 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.930+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d0073990 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.930+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84c4012040 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.931+0000 7f84cdffb640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84bc06b500 0x7f84bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.931+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84c408a920 con 0x7f84d0157820 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.931+0000 7f84d5141640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84bc06b500 0x7f84bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.931+0000 7f84d5141640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84bc06b500 0x7f84bc06d9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f84c8007b80 tx=0x7f84c8001cd0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:54.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:54.935+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84c4019030 con 0x7f84d0157820 2024-04-25T08:37:55.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.185+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f84d015ffe0 con 0x7f84d0157820 2024-04-25T08:37:55.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.185+0000 7f84cdffb640 1 -- 172.21.15.60:0/1634549204 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f84c4058fa0 con 0x7f84d0157820 2024-04-25T08:37:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:55.180 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":535,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561783,"bytes_used":2589949952,"bytes_avail":770504163328,"bytes_total":773094113280,"write_bytes_sec":49836185,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T08:37:53.515641+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T08:37:55.181 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84bc06b500 msgr2=0x7f84bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84bc06b500 0x7f84bc06d9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f84c8007b80 tx=0x7f84c8001cd0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 msgr2=0x7f84d00c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 0x7f84d00c8510 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f84c4011a00 tx=0x7f84c4011ec0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 shutdown_connections 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f84bc06b500 0x7f84bc06d9c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d0156fc0 0x7f84d00d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f84d0157820 0x7f84d00c8510 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 --2- 172.21.15.60:0/1634549204 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f84b8004d40 0x7f84d00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 >> 172.21.15.60:0/1634549204 conn(0x7f84d0061a20 msgr2=0x7f84d006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 shutdown_connections 2024-04-25T08:37:55.182 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.187+0000 7f84d73a5640 1 -- 172.21.15.60:0/1634549204 wait complete. 2024-04-25T08:37:55.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T08:37:55.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:55.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T08:37:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.301+0000 7f697b9d5640 1 Processor -- start 2024-04-25T08:37:55.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.302+0000 7f697b9d5640 1 -- start start 2024-04-25T08:37:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.302+0000 7f697b9d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6974070dc0 0x7f697406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.302+0000 7f697b9d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f697415deb0 0x7f697415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.302+0000 7f697b9d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69740d1770 con 0x7f69740711a0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.302+0000 7f697b9d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69740b0eb0 con 0x7f697415deb0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.302+0000 7f697b9d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69741622b0 con 0x7f6974070dc0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6978f70640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f69740711a0 0x7f69740b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59112/0 (socket says 172.21.15.60:59112) 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6978f70640 1 -- 172.21.15.60:0/1227816857 learned_addr learned my addr 172.21.15.60:0/1227816857 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6979771640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6974070dc0 0x7f697406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6979f72640 1 --2- 172.21.15.60:0/1227816857 >> v2:172.21.15.100:3300/0 conn(0x7f697415deb0 0x7f697415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931250411 0 0) 0x7f69740d1770 con 0x7f69740711a0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6960003520 con 0x7f69740711a0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977356696 0 0) 0x7f6960003520 con 0x7f69740711a0 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6974070dc0 msgr2=0x7f697406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6974070dc0 0x7f697406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 >> v2:172.21.15.100:3300/0 conn(0x7f697415deb0 msgr2=0x7f697415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 --2- 172.21.15.60:0/1227816857 >> v2:172.21.15.100:3300/0 conn(0x7f697415deb0 0x7f697415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974162740 con 0x7f69740711a0 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6964001670 con 0x7f69740711a0 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 >> v1:172.21.15.60:6789/0 conn(0x7f69740711a0 legacy=0x7f69740b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69600039b0 0x7f6960003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6960004800 0x7f6960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69600051c0 0x7f696000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740b0eb0 con 0x7f6960004800 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69741622b0 con 0x7f69600051c0 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696000bfe0 con 0x7f69600039b0 2024-04-25T08:37:55.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6979f72640 1 --2- 172.21.15.60:0/1227816857 >> v2:172.21.15.100:3300/0 conn(0x7f697415deb0 0x7f697415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6979771640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6974070dc0 0x7f697406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6978f70640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6960004800 0x7f6960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6979f72640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69600051c0 0x7f696000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.303+0000 7f6979771640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69600039b0 0x7f6960003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6978f70640 1 -- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69600039b0 msgr2=0x7f6960003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6978f70640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69600039b0 0x7f6960003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6978f70640 1 -- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69600051c0 msgr2=0x7f696000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6978f70640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69600051c0 0x7f696000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6978f70640 1 -- 172.21.15.60:0/1227816857 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960003520 con 0x7f6960004800 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6979f72640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69600051c0 0x7f696000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f6978f70640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6960004800 0x7f6960004c80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f696400c290 tx=0x7f6960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2581e76c512c997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964007d50 con 0x7f6960004800 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964005af0 con 0x7f6960004800 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.304+0000 7f696bfff640 1 -- 172.21.15.60:0/1227816857 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964007070 con 0x7f6960004800 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f697b9d5640 1 -- 172.21.15.60:0/1227816857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6960004800 msgr2=0x7f6960004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f697b9d5640 1 --2- 172.21.15.60:0/1227816857 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6960004800 0x7f6960004c80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f696400c290 tx=0x7f6960005830 comp rx=0 tx=0).stop 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f6979f72640 1 -- 172.21.15.60:0/1227816857 reap_dead start 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f697b9d5640 1 -- 172.21.15.60:0/1227816857 shutdown_connections 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f697b9d5640 1 -- 172.21.15.60:0/1227816857 >> 172.21.15.60:0/1227816857 conn(0x7f69740b3530 msgr2=0x7f69740b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f697b9d5640 1 -- 172.21.15.60:0/1227816857 shutdown_connections 2024-04-25T08:37:55.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.305+0000 7f697b9d5640 1 -- 172.21.15.60:0/1227816857 wait complete. 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.306+0000 7f697b9d5640 1 Processor -- start 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.306+0000 7f697b9d5640 1 -- start start 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f697b9d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6960004800 0x7f697406bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f697b9d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 0x7f697414e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f697b9d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69741f6e30 0x7f69741f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f697b9d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6974161c60 con 0x7f697414df70 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f697b9d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69740b10f0 con 0x7f69741f6e30 2024-04-25T08:37:55.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f697b9d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6974161f40 con 0x7f6960004800 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 0x7f697414e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 0x7f697414e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55792/0 (socket says 172.21.15.60:55792) 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 -- 172.21.15.60:0/3308715646 learned_addr learned my addr 172.21.15.60:0/3308715646 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6979f72640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69741f6e30 0x7f69741f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6979771640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6960004800 0x7f697406bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 -- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6960004800 msgr2=0x7f697406bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6960004800 0x7f697406bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 -- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69741f6e30 msgr2=0x7f69741f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69741f6e30 0x7f69741f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.307+0000 7f6978f70640 1 -- 172.21.15.60:0/3308715646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697414e930 con 0x7f697414df70 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.308+0000 7f6978f70640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 0x7f697414e3f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6964034040 tx=0x7f6974162220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.308+0000 7f6969ffb640 1 -- 172.21.15.60:0/3308715646 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964042070 con 0x7f697414df70 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.308+0000 7f6969ffb640 1 -- 172.21.15.60:0/3308715646 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6964007480 con 0x7f697414df70 2024-04-25T08:37:55.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.308+0000 7f6969ffb640 1 -- 172.21.15.60:0/3308715646 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6964007d50 con 0x7f697414df70 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.308+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69741f7b60 con 0x7f697414df70 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.308+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69740caf30 con 0x7f697414df70 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.309+0000 7f6969ffb640 1 -- 172.21.15.60:0/3308715646 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6964045070 con 0x7f697414df70 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.310+0000 7f6969ffb640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f695806b500 0x7f695806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.310+0000 7f6969ffb640 1 -- 172.21.15.60:0/3308715646 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69640aad00 con 0x7f697414df70 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.310+0000 7f6979771640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f695806b500 0x7f695806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.311+0000 7f6979771640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f695806b500 0x7f695806d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6974070a00 tx=0x7f6970008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.311+0000 7f697e7bc740 1 Processor -- start 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.311+0000 7f697e7bc740 1 -- start start 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.311+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06290320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.311+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b06290860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 0x559b06263f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f697e7bc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b061c9da0 con 0x559b06260770 2024-04-25T08:37:55.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b06172b80 con 0x559b0628def0 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b061c5e40 con 0x559b0628db10 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 0x559b06263f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 0x559b06263f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55798/0 (socket says 172.21.15.60:55798) 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 -- 172.21.15.60:0/2651257095 learned_addr learned my addr 172.21.15.60:0/2651257095 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979771640 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06290320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6978f70640 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b06290860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 -- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 msgr2=0x559b06290320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06290320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 -- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 msgr2=0x559b06290860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b06290860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 -- 172.21.15.60:0/2651257095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0617cf10 con 0x559b06260770 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6979f72640 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 0x559b06263f80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f696c009a40 tx=0x7f696c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc9bac60ae71755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6956ffd640 1 -- 172.21.15.60:0/2651257095 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c01b020 con 0x559b06260770 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6956ffd640 1 -- 172.21.15.60:0/2651257095 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c017470 con 0x559b06260770 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.312+0000 7f6956ffd640 1 -- 172.21.15.60:0/2651257095 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c017860 con 0x559b06260770 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 -- 172.21.15.60:0/2651257095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 msgr2=0x559b06263f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 0x559b06263f80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f696c009a40 tx=0x7f696c009750 comp rx=0 tx=0).stop 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 -- 172.21.15.60:0/2651257095 shutdown_connections 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b06260770 0x559b06263f80 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b06290860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2651257095 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06290320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 -- 172.21.15.60:0/2651257095 >> 172.21.15.60:0/2651257095 conn(0x559b062629e0 msgr2=0x559b062ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 -- 172.21.15.60:0/2651257095 shutdown_connections 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.313+0000 7f697e7bc740 1 -- 172.21.15.60:0/2651257095 wait complete. 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.314+0000 7f697e7bc740 1 -- start start 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.314+0000 7f697e7bc740 1 Processor -- start 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.314+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06288220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.314+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b062a5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.314+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 0x559b062a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f697e7bc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b06265b90 con 0x559b062a7b60 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b061684b0 con 0x559b0628def0 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b06265e70 con 0x559b0628db10 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f697e7bc740 10 client.0 _ll_register_callbacks cb 0x559b061a8180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 0x559b062a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 0x559b062a7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55808/0 (socket says 172.21.15.60:55808) 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 -- 172.21.15.60:0/2004224150 learned_addr learned my addr 172.21.15.60:0/2004224150 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979771640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06288220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6978f70640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b062a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 -- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 msgr2=0x559b06288220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06288220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 -- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 msgr2=0x559b062a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b062a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 -- 172.21.15.60:0/2004224150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b05fa87d0 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f6979f72640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 0x559b062a7fe0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f696c01a040 tx=0x7f696c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f132268462ed42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f694ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f694ffff640 1 -- 172.21.15.60:0/2004224150 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c01b040 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f694ffff640 1 -- 172.21.15.60:0/2004224150 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f696c020420 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f696c017470 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f694ffff640 1 -- 172.21.15.60:0/2004224150 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696c0048a0 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.315+0000 7f697e7bc740 10 client.9129 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x559b05ede3f0 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f694ffff640 1 -- 172.21.15.60:0/2004224150 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c017470 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f694ffff640 1 -- 172.21.15.60:0/2004224150 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f696c02b680 con 0x559b062a7b60 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 10 client.9129 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 10 client.9129 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 20 client.9129 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062ad530 0x559b062ad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 4 client.9129 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b061d72e0 con 0x559b062ad530 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f6979771640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062ad530 0x559b062ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f6979771640 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062ad530 0x559b062ad940 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9cc855bfd6ac55 server_cookie=c7bd50dd9d1e15e6 in_seq=0 out_seq=0 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.316+0000 7f694ffff640 10 client.9129 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f694ffff640 1 -- 172.21.15.60:0/2004224150 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b061d72e0 con 0x559b062ad530 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f694ffff640 10 client.9129 handle_command_reply: tid=0 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 client.9129 shutdown 2024-04-25T08:37:55.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 msgr2=0x559b062a7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 0x559b062a7fe0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f696c01a040 tx=0x7f696c004a40 comp rx=0 tx=0).stop 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 shutdown_connections 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062ad530 0x559b062ad940 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b062a7b60 0x559b062a7fe0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628def0 0x559b062a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f694ffff640 0 client.9129 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2004224150 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b06288220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 >> 172.21.15.60:0/2004224150 conn(0x559b06260770 msgr2=0x559b062652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 shutdown_connections 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.317+0000 7f697e7bc740 1 -- 172.21.15.60:0/2004224150 wait complete. 2024-04-25T08:37:55.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.318+0000 7f697e7bc740 20 client.9129 trim_cache size 0 max 16384 2024-04-25T08:37:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.332+0000 7f697e7bc740 1 Processor -- start 2024-04-25T08:37:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.332+0000 7f697e7bc740 1 -- start start 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b062cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b062a7b60 0x559b062ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f697e7bc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b062948c0 con 0x559b0628def0 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b062a79e0 con 0x559b062a7b60 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6974200be0 con 0x559b0628db10 2024-04-25T08:37:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f6978f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f6978f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55814/0 (socket says 172.21.15.60:55814) 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f6978f70640 1 -- 172.21.15.60:0/2906940462 learned_addr learned my addr 172.21.15.60:0/2906940462 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f6979771640 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b062cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.333+0000 7f6979f72640 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b062a7b60 0x559b062ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f6978f70640 1 -- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 msgr2=0x559b062cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f6978f70640 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b062cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f6978f70640 1 -- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b062a7b60 msgr2=0x559b062ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f6978f70640 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b062a7b60 0x559b062ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f6978f70640 1 -- 172.21.15.60:0/2906940462 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b062a5cc0 con 0x559b0628def0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f6978f70640 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062aafe0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f6964081560 tx=0x7f6964001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b8698c1b51dfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f69567fc640 1 -- 172.21.15.60:0/2906940462 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640a7070 con 0x559b0628def0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f69567fc640 1 -- 172.21.15.60:0/2906940462 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696407d4d0 con 0x559b0628def0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f69567fc640 1 -- 172.21.15.60:0/2906940462 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696404c040 con 0x559b0628def0 2024-04-25T08:37:55.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 -- 172.21.15.60:0/2906940462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 msgr2=0x559b062aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062aafe0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f6964081560 tx=0x7f6964001910 comp rx=0 tx=0).stop 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 -- 172.21.15.60:0/2906940462 shutdown_connections 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b062a7b60 0x559b062ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062aafe0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2906940462 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b0628db10 0x559b062cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 -- 172.21.15.60:0/2906940462 >> 172.21.15.60:0/2906940462 conn(0x559b06260770 msgr2=0x559b06288030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 -- 172.21.15.60:0/2906940462 shutdown_connections 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.334+0000 7f697e7bc740 1 -- 172.21.15.60:0/2906940462 wait complete. 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.335+0000 7f697e7bc740 1 -- start start 2024-04-25T08:37:55.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.335+0000 7f697e7bc740 1 Processor -- start 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628db10 0x559b0628ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b062a7b60 0x559b062ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69741ff1a0 con 0x559b0628def0 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69741ffda0 con 0x559b0628db10 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69741ff7a0 con 0x559b062a7b60 2024-04-25T08:37:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f697e7bc740 10 client.0 _ll_register_callbacks cb 0x7f696c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062938b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55818/0 (socket says 172.21.15.60:55818) 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 -- 172.21.15.60:0/2270338853 learned_addr learned my addr 172.21.15.60:0/2270338853 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6979771640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628db10 0x559b0628ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6979f72640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b062a7b60 0x559b062ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 -- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b062a7b60 msgr2=0x559b062ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b062a7b60 0x559b062ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 -- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628db10 msgr2=0x559b0628ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628db10 0x559b0628ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6978f70640 1 -- 172.21.15.60:0/2270338853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b05e9e0f0 con 0x559b0628def0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.336+0000 7f6979771640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628db10 0x559b0628ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f6978f70640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062938b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f696402ae50 tx=0x7f69640a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b8f392801cc705 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f694ffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:55.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f694ffff640 1 -- 172.21.15.60:0/2270338853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640a7070 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f694ffff640 1 -- 172.21.15.60:0/2270338853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6964078880 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f696407d4d0 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f694ffff640 1 -- 172.21.15.60:0/2270338853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f696404c040 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 10 client.9135 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x559b06293df0 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f694ffff640 1 -- 172.21.15.60:0/2270338853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696407d4d0 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f694ffff640 1 -- 172.21.15.60:0/2270338853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6964075180 con 0x559b0628def0 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 10 client.9135 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 10 client.9135 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 20 client.9135 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062a7260 0x559b062bd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 4 client.9135 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x559b062acbd0 con 0x559b062a7260 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.337+0000 7f6979771640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062a7260 0x559b062bd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f6979771640 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062a7260 0x559b062bd200 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7dd5fe96e21909 server_cookie=5aa7d9091895fcd6 in_seq=0 out_seq=0 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f694ffff640 10 client.9135 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f694ffff640 1 -- 172.21.15.60:0/2270338853 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x559b062acbd0 con 0x559b062a7260 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f694ffff640 10 client.9135 handle_command_reply: tid=0 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 client.9135 shutdown 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 msgr2=0x559b062938b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062938b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f696402ae50 tx=0x7f69640a9000 comp rx=0 tx=0).stop 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 shutdown_connections 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x559b062a7260 0x559b062bd200 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x559b062a7b60 0x559b062ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f694ffff640 0 client.9135 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559b0628def0 0x559b062938b0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 --2- 172.21.15.60:0/2270338853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559b0628db10 0x559b0628ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 >> 172.21.15.60:0/2270338853 conn(0x559b06260770 msgr2=0x559b062610b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 shutdown_connections 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 1 -- 172.21.15.60:0/2270338853 wait complete. 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.338+0000 7f697e7bc740 20 client.9135 trim_cache size 0 max 16384 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:37:55.339 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f695806b500 msgr2=0x7f695806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f695806b500 0x7f695806d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6974070a00 tx=0x7f6970008040 comp rx=0 tx=0).stop 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 msgr2=0x7f697414e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 0x7f697414e3f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f6964034040 tx=0x7f6974162220 comp rx=0 tx=0).stop 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 shutdown_connections 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f695806b500 0x7f695806d9c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69741f6e30 0x7f69741f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f697414df70 0x7f697414e3f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 --2- 172.21.15.60:0/3308715646 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6960004800 0x7f697406bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 >> 172.21.15.60:0/3308715646 conn(0x7f69740b3530 msgr2=0x7f69740d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 shutdown_connections 2024-04-25T08:37:55.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.339+0000 7f697b9d5640 1 -- 172.21.15.60:0/3308715646 wait complete. 2024-04-25T08:37:55.343 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:37:55.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T08:37:55.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.452+0000 7f9b8ae31640 1 Processor -- start 2024-04-25T08:37:55.446 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.452+0000 7f9b8ae31640 1 -- start start 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b8ae31640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b8415b7d0 0x7f9b840cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b8ae31640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9b8406c9e0 0x7f9b840b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b8ae31640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8406cf50 con 0x7f9b840b5620 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b8ae31640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b840ce530 con 0x7f9b8406c9e0 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b8ae31640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b840ce810 con 0x7f9b8415b7d0 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9b8406c9e0 0x7f9b840b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.448 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9b8406c9e0 0x7f9b840b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55822/0 (socket says 172.21.15.60:55822) 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 -- 172.21.15.60:0/36372038 learned_addr learned my addr 172.21.15.60:0/36372038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 -- 172.21.15.60:0/36372038 >> v2:172.21.15.100:3300/0 conn(0x7f9b8415b7d0 msgr2=0x7f9b840cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 --2- 172.21.15.60:0/36372038 >> v2:172.21.15.100:3300/0 conn(0x7f9b8415b7d0 0x7f9b840cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 -- 172.21.15.60:0/36372038 >> v1:172.21.15.100:6789/0 conn(0x7f9b840b5620 legacy=0x7f9b8406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.453+0000 7f9b893ce640 1 -- 172.21.15.60:0/36372038 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840ceaf0 con 0x7f9b8406c9e0 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.454+0000 7f9b893ce640 1 --2- 172.21.15.60:0/36372038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b840b5ab0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f9b8408c0d0 tx=0x7f9b7c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137c0ba49bd0e0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.454+0000 7f9b837fe640 1 -- 172.21.15.60:0/36372038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c036070 con 0x7f9b8406c9e0 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.454+0000 7f9b837fe640 1 -- 172.21.15.60:0/36372038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c031600 con 0x7f9b8406c9e0 2024-04-25T08:37:55.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.454+0000 7f9b837fe640 1 -- 172.21.15.60:0/36372038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0319d0 con 0x7f9b8406c9e0 2024-04-25T08:37:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 -- 172.21.15.60:0/36372038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 msgr2=0x7f9b840b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/36372038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b840b5ab0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f9b8408c0d0 tx=0x7f9b7c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 -- 172.21.15.60:0/36372038 shutdown_connections 2024-04-25T08:37:55.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/36372038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b840b5ab0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/36372038 >> v2:172.21.15.100:3300/0 conn(0x7f9b8415b7d0 0x7f9b840cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 -- 172.21.15.60:0/36372038 >> 172.21.15.60:0/36372038 conn(0x7f9b840d0800 msgr2=0x7f9b840d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 -- 172.21.15.60:0/36372038 shutdown_connections 2024-04-25T08:37:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.455+0000 7f9b8ae31640 1 -- 172.21.15.60:0/36372038 wait complete. 2024-04-25T08:37:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.456+0000 7f9b8ae31640 1 Processor -- start 2024-04-25T08:37:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.456+0000 7f9b8ae31640 1 -- start start 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b8ae31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b84150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b8ae31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b840b5620 0x7f9b840b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b8ae31640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b8415b7d0 0x7f9b840b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b8ae31640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cf6d0 con 0x7f9b8406c9e0 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b8ae31640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cf550 con 0x7f9b840b5620 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b8ae31640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8415bbb0 con 0x7f9b8415b7d0 2024-04-25T08:37:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b84150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b84150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55824/0 (socket says 172.21.15.60:55824) 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 -- 172.21.15.60:0/2057822050 learned_addr learned my addr 172.21.15.60:0/2057822050 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b83fff640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b840b5620 0x7f9b840b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b893ce640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b8415b7d0 0x7f9b840b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 -- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b8415b7d0 msgr2=0x7f9b840b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b8415b7d0 0x7f9b840b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 -- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b840b5620 msgr2=0x7f9b840b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b840b5620 0x7f9b840b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.457+0000 7f9b88bcd640 1 -- 172.21.15.60:0/2057822050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840cad10 con 0x7f9b8406c9e0 2024-04-25T08:37:55.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.458+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b84150a30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b410 tx=0x7f9b7400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.458+0000 7f9b817fa640 1 -- 172.21.15.60:0/2057822050 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74015020 con 0x7f9b8406c9e0 2024-04-25T08:37:55.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.458+0000 7f9b817fa640 1 -- 172.21.15.60:0/2057822050 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74011850 con 0x7f9b8406c9e0 2024-04-25T08:37:55.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.458+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b840cafb0 con 0x7f9b8406c9e0 2024-04-25T08:37:55.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.458+0000 7f9b817fa640 1 -- 172.21.15.60:0/2057822050 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74011c40 con 0x7f9b8406c9e0 2024-04-25T08:37:55.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.458+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b840cb570 con 0x7f9b8406c9e0 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.460+0000 7f9b817fa640 1 -- 172.21.15.60:0/2057822050 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b74018020 con 0x7f9b8406c9e0 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.460+0000 7f9b817fa640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b7006b4d0 0x7f9b7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.460+0000 7f9b817fa640 1 -- 172.21.15.60:0/2057822050 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b7408a1f0 con 0x7f9b8406c9e0 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.460+0000 7f9b83fff640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b7006b4d0 0x7f9b7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.461+0000 7f9b8dc0d740 1 Processor -- start 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.461+0000 7f9b8dc0d740 1 -- start start 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.461+0000 7f9b83fff640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b7006b4d0 0x7f9b7006d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9b78007fd0 tx=0x7f9b78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.461+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eb10 0x55f33c221320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 0x55f33c221860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c1f1770 0x55f33c1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33c15ada0 con 0x55f33c21eef0 2024-04-25T08:37:55.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33c103b80 con 0x55f33c1f1770 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33c156e40 con 0x55f33c21eb10 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 0x55f33c221860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 0x55f33c221860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55830/0 (socket says 172.21.15.60:55830) 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 -- 172.21.15.60:0/4268169930 learned_addr learned my addr 172.21.15.60:0/4268169930 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eb10 0x55f33c221320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b893ce640 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c1f1770 0x55f33c1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 -- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eb10 msgr2=0x55f33c221320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eb10 0x55f33c221320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 -- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c1f1770 msgr2=0x55f33c1f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c1f1770 0x55f33c1f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 -- 172.21.15.60:0/4268169930 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33c10df10 con 0x55f33c21eef0 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b893ce640 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c1f1770 0x55f33c1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b83fff640 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 0x55f33c221860 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001df0 tx=0x7f9b78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbff4cc5fc31f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b6a7fc640 1 -- 172.21.15.60:0/4268169930 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780035f0 con 0x55f33c21eef0 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b6a7fc640 1 -- 172.21.15.60:0/4268169930 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78003790 con 0x55f33c21eef0 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.462+0000 7f9b6a7fc640 1 -- 172.21.15.60:0/4268169930 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003b40 con 0x55f33c21eef0 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/4268169930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 msgr2=0x55f33c221860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 0x55f33c221860 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001df0 tx=0x7f9b78011000 comp rx=0 tx=0).stop 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/4268169930 shutdown_connections 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c1f1770 0x55f33c1f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eef0 0x55f33c221860 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/4268169930 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eb10 0x55f33c221320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/4268169930 >> 172.21.15.60:0/4268169930 conn(0x55f33c1f39e0 msgr2=0x55f33c23c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/4268169930 shutdown_connections 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.463+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/4268169930 wait complete. 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 -- start start 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 Processor -- start 2024-04-25T08:37:55.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21eb10 0x55f33c24f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c222f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 0x55f33c223460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33c1f6b90 con 0x55f33c21c600 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33c0f94b0 con 0x55f33c21eb10 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.464+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33c1f6e70 con 0x55f33c21eef0 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b8dc0d740 10 client.0 _ll_register_callbacks cb 0x55f33c139180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 0x55f33c223460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 0x55f33c223460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55832/0 (socket says 172.21.15.60:55832) 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 -- 172.21.15.60:0/157922841 learned_addr learned my addr 172.21.15.60:0/157922841 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21eb10 0x55f33c24f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b83fff640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c222f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 -- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 msgr2=0x55f33c222f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c222f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 -- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21eb10 msgr2=0x55f33c24f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21eb10 0x55f33c24f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 -- 172.21.15.60:0/157922841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f33bf397d0 con 0x55f33c21c600 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b893ce640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 0x55f33c223460 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c02ace0 tx=0x7f9b7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f5c88aecaee27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b637fe640 1 -- 172.21.15.60:0/157922841 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c036040 con 0x55f33c21c600 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b637fe640 1 -- 172.21.15.60:0/157922841 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b7c031b10 con 0x55f33c21c600 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b78003790 con 0x55f33c21c600 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b637fe640 1 -- 172.21.15.60:0/157922841 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b7c04d050 con 0x55f33c21c600 2024-04-25T08:37:55.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b8dc0d740 10 client.9147 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.465+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55f33be6f3f0 con 0x55f33c21c600 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b637fe640 1 -- 172.21.15.60:0/157922841 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78003790 con 0x55f33c21c600 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b637fe640 1 -- 172.21.15.60:0/157922841 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b7c050020 con 0x55f33c21c600 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b8dc0d740 10 client.9147 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b8dc0d740 10 client.9147 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b8dc0d740 20 client.9147 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c24fdb0 0x55f33c2501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b8dc0d740 4 client.9147 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f33c1682e0 con 0x55f33c24fdb0 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c24fdb0 0x55f33c2501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b637fe640 10 client.9147 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.466+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c24fdb0 0x55f33c2501c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271527c71f78ffe9 server_cookie=59d32de9878b24bb in_seq=0 out_seq=0 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b637fe640 1 -- 172.21.15.60:0/157922841 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f33c1682e0 con 0x55f33c24fdb0 2024-04-25T08:37:55.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b637fe640 10 client.9147 handle_command_reply: tid=0 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 client.9147 shutdown 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 msgr2=0x55f33c223460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 0x55f33c223460 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c02ace0 tx=0x7f9b7c004ab0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 shutdown_connections 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c24fdb0 0x55f33c2501c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21c600 0x55f33c223460 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c222f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b637fe640 0 client.9147 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/157922841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21eb10 0x55f33c24f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 >> 172.21.15.60:0/157922841 conn(0x55f33c1f1770 msgr2=0x55f33c2212e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 shutdown_connections 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/157922841 wait complete. 2024-04-25T08:37:55.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.467+0000 7f9b8dc0d740 20 client.9147 trim_cache size 0 max 16384 2024-04-25T08:37:55.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.482+0000 7f9b8dc0d740 1 Processor -- start 2024-04-25T08:37:55.476 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.482+0000 7f9b8dc0d740 1 -- start start 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c1f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c2212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b842005e0 con 0x55f33c21eb10 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841ff810 con 0x55f33c21c600 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841f79b0 con 0x55f33c21eef0 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c1f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c1f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55838/0 (socket says 172.21.15.60:55838) 2024-04-25T08:37:55.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 -- 172.21.15.60:0/2910510591 learned_addr learned my addr 172.21.15.60:0/2910510591 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 -- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 msgr2=0x55f33c2212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c2212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 -- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 msgr2=0x55f33c24d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.483+0000 7f9b83fff640 1 -- 172.21.15.60:0/2910510591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33be2f0f0 con 0x55f33c21eb10 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b83fff640 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c1f23c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9b78014000 tx=0x7f9b78017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8570c40d46eda72d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b69ffb640 1 -- 172.21.15.60:0/2910510591 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78003c30 con 0x55f33c21eb10 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b69ffb640 1 -- 172.21.15.60:0/2910510591 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78003df0 con 0x55f33c21eb10 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b69ffb640 1 -- 172.21.15.60:0/2910510591 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78021690 con 0x55f33c21eb10 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/2910510591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 msgr2=0x55f33c1f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c1f23c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9b78014000 tx=0x7f9b78017070 comp rx=0 tx=0).stop 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/2910510591 shutdown_connections 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c2212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c1f23c0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/2910510591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/2910510591 >> 172.21.15.60:0/2910510591 conn(0x55f33c1f1770 msgr2=0x55f33c225260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/2910510591 shutdown_connections 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.484+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/2910510591 wait complete. 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.485+0000 7f9b8dc0d740 1 -- start start 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.485+0000 7f9b8dc0d740 1 Processor -- start 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c24c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c24c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33c2392c0 con 0x55f33c21eb10 2024-04-25T08:37:55.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841ff210 con 0x55f33c21c600 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f33c24f490 con 0x55f33c21eef0 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b8dc0d740 10 client.0 _ll_register_callbacks cb 0x7f9b78001300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c24c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c24c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55848/0 (socket says 172.21.15.60:55848) 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 -- 172.21.15.60:0/1712291173 learned_addr learned my addr 172.21.15.60:0/1712291173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 -- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 msgr2=0x55f33c24c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c24c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 -- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 msgr2=0x55f33c24b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 -- 172.21.15.60:0/1712291173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f33c24b5b0 con 0x55f33c21eb10 2024-04-25T08:37:55.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b83fff640 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c24c2a0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800eb80 tx=0x7f9b78013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aaa0a2318582a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.486+0000 7f9b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b637fe640 1 -- 172.21.15.60:0/1712291173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78021860 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b637fe640 1 -- 172.21.15.60:0/1712291173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b78021a00 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b78003df0 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b637fe640 1 -- 172.21.15.60:0/1712291173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b7803b050 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 10 client.9153 fetch_fsmap learned FSMap version 71 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55f33c24cd20 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b637fe640 1 -- 172.21.15.60:0/1712291173 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78003df0 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b637fe640 1 -- 172.21.15.60:0/1712291173 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9b7803e020 con 0x55f33c21eb10 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 10 client.9153 fetch_fsmap finished waiting for FSMap version 71 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 10 client.9153 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:37:55.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 20 client.9153 populate_metadata read hostname 'smithi060' 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c1f0600 0x55f33c1f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 4 client.9153 mds_command: new command op to 8805 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 --> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f33c162630 con 0x55f33c1f0600 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.487+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c1f0600 0x55f33c1f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b88bcd640 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c1f0600 0x55f33c1f0a30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5290b5ebeb15b6b server_cookie=6e13ee4504b1e81a in_seq=0 out_seq=0 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b637fe640 10 client.9153 ms_handle_connect on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b637fe640 1 -- 172.21.15.60:0/1712291173 <== mds.0 v2:172.21.15.100:6835/2648832765 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f33c162630 con 0x55f33c1f0600 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b637fe640 10 client.9153 handle_command_reply: tid=0 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 client.9153 shutdown 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 msgr2=0x55f33c24c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c24c2a0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800eb80 tx=0x7f9b78013000 comp rx=0 tx=0).stop 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 shutdown_connections 2024-04-25T08:37:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:6835/2648832765,v1:172.21.15.100:6837/2648832765] conn(0x55f33c1f0600 0x55f33c1f0a30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55f33c21eef0 0x55f33c24c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f33c21eb10 0x55f33c24c2a0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b637fe640 0 client.9153 ms_handle_reset on v2:172.21.15.100:6835/2648832765 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 --2- 172.21.15.60:0/1712291173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f33c21c600 0x55f33c24b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 >> 172.21.15.60:0/1712291173 conn(0x55f33c1f1770 msgr2=0x55f33c25ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 shutdown_connections 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.488+0000 7f9b8dc0d740 1 -- 172.21.15.60:0/1712291173 wait complete. 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8dc0d740 20 client.9153 trim_cache size 0 max 16384 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "no active scrubs running", 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": {} 2024-04-25T08:37:55.489 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:37:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b7006b4d0 msgr2=0x7f9b7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b7006b4d0 0x7f9b7006d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9b78007fd0 tx=0x7f9b78009040 comp rx=0 tx=0).stop 2024-04-25T08:37:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 msgr2=0x7f9b84150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b84150a30 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b410 tx=0x7f9b7400b8d0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 shutdown_connections 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b8415b7d0 0x7f9b840b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b7006b4d0 0x7f9b7006d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b840b5620 0x7f9b840b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 --2- 172.21.15.60:0/2057822050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b8406c9e0 0x7f9b84150a30 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 >> 172.21.15.60:0/2057822050 conn(0x7f9b840d0800 msgr2=0x7f9b8406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 shutdown_connections 2024-04-25T08:37:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.489+0000 7f9b8ae31640 1 -- 172.21.15.60:0/2057822050 wait complete. 2024-04-25T08:37:55.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:37:55.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:55.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.601+0000 7f110c038640 1 Processor -- start 2024-04-25T08:37:55.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.601+0000 7f110c038640 1 -- start start 2024-04-25T08:37:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.601+0000 7f110c038640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f110415dd20 0x7f11040b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f110c038640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11040d11b0 0x7f11040d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f110c038640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110415eac0 con 0x7f110415e100 2024-04-25T08:37:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f110c038640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11040d1c00 con 0x7f110415dd20 2024-04-25T08:37:55.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f110c038640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110415eda0 con 0x7f11040d11b0 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f110415dd20 0x7f11040b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f110415dd20 0x7f11040b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55862/0 (socket says 172.21.15.60:55862) 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 -- 172.21.15.60:0/763702193 learned_addr learned my addr 172.21.15.60:0/763702193 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f110a5d5640 1 --2- 172.21.15.60:0/763702193 >> v2:172.21.15.100:3300/0 conn(0x7f11040d11b0 0x7f11040d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 -- 172.21.15.60:0/763702193 >> v2:172.21.15.100:3300/0 conn(0x7f11040d11b0 msgr2=0x7f11040d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 --2- 172.21.15.60:0/763702193 >> v2:172.21.15.100:3300/0 conn(0x7f11040d11b0 0x7f11040d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 -- 172.21.15.60:0/763702193 >> v1:172.21.15.100:6789/0 conn(0x7f110415e100 legacy=0x7f11040d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.602+0000 7f1109dd4640 1 -- 172.21.15.60:0/763702193 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110415f080 con 0x7f110415dd20 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.603+0000 7f1109dd4640 1 --2- 172.21.15.60:0/763702193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11040b96d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f11040d1990 tx=0x7f10fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639fd41d15bfd5d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.603+0000 7f1108dd2640 1 -- 172.21.15.60:0/763702193 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc036070 con 0x7f110415dd20 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.603+0000 7f1108dd2640 1 -- 172.21.15.60:0/763702193 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc031600 con 0x7f110415dd20 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.603+0000 7f1108dd2640 1 -- 172.21.15.60:0/763702193 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0319d0 con 0x7f110415dd20 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.603+0000 7f110c038640 1 -- 172.21.15.60:0/763702193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 msgr2=0x7f11040b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.603+0000 7f110c038640 1 --2- 172.21.15.60:0/763702193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11040b96d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f11040d1990 tx=0x7f10fc02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.604+0000 7f110c038640 1 -- 172.21.15.60:0/763702193 shutdown_connections 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.604+0000 7f110c038640 1 --2- 172.21.15.60:0/763702193 >> v2:172.21.15.100:3300/0 conn(0x7f11040d11b0 0x7f11040d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.604+0000 7f110c038640 1 --2- 172.21.15.60:0/763702193 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11040b96d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.604+0000 7f110c038640 1 -- 172.21.15.60:0/763702193 >> 172.21.15.60:0/763702193 conn(0x7f1104061a20 msgr2=0x7f1104061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.604+0000 7f110c038640 1 -- 172.21.15.60:0/763702193 shutdown_connections 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.604+0000 7f110c038640 1 -- 172.21.15.60:0/763702193 wait complete. 2024-04-25T08:37:55.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 Processor -- start 2024-04-25T08:37:55.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 -- start start 2024-04-25T08:37:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11040d11b0 0x7f11040cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11041520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110415e100 0x7f11040c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110415e710 con 0x7f110415dd20 2024-04-25T08:37:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.605+0000 7f110c038640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110415e590 con 0x7f11040d11b0 2024-04-25T08:37:55.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f110c038640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11040cbbd0 con 0x7f110415e100 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11041520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11041520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55866/0 (socket says 172.21.15.60:55866) 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 -- 172.21.15.60:0/1946455762 learned_addr learned my addr 172.21.15.60:0/1946455762 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f1109dd4640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11040d11b0 0x7f11040cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f110a5d5640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110415e100 0x7f11040c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 -- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110415e100 msgr2=0x7f11040c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110415e100 0x7f11040c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 -- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11040d11b0 msgr2=0x7f11040cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11040d11b0 0x7f11040cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 -- 172.21.15.60:0/1946455762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040cafc0 con 0x7f110415dd20 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11095d3640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11041520b0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c400 tx=0x7f10f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8017070 con 0x7f110415dd20 2024-04-25T08:37:55.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f80052d0 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040cb2a0 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003830 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.606+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11040cb860 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.607+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1104073990 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.608+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10f801a070 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.608+0000 7f11027fc640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f10f006b4d0 0x7f10f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.608+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10f8089660 con 0x7f110415dd20 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.608+0000 7f1109dd4640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f10f006b4d0 0x7f10f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.609+0000 7f1109dd4640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f10f006b4d0 0x7f10f006d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f10fc033000 tx=0x7f10fc009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.613+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10f8057d00 con 0x7f110415dd20 2024-04-25T08:37:55.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.831+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11040d0df0 con 0x7f110415dd20 2024-04-25T08:37:55.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.831+0000 7f11027fc640 1 -- 172.21.15.60:0/1946455762 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f10f8087e20 con 0x7f110415dd20 2024-04-25T08:37:55.826 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:55.826 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:55.826 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f10f006b4d0 msgr2=0x7f10f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f10f006b4d0 0x7f10f006d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f10fc033000 tx=0x7f10fc009000 comp rx=0 tx=0).stop 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 msgr2=0x7f11041520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11041520b0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c400 tx=0x7f10f800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 shutdown_connections 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f110415e100 0x7f11040c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f110415dd20 0x7f11041520b0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f10f006b4d0 0x7f10f006d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 --2- 172.21.15.60:0/1946455762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11040d11b0 0x7f11040cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 >> 172.21.15.60:0/1946455762 conn(0x7f1104061a20 msgr2=0x7f11040cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 shutdown_connections 2024-04-25T08:37:55.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.834+0000 7f110c038640 1 -- 172.21.15.60:0/1946455762 wait complete. 2024-04-25T08:37:55.838 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:55.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:55.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 Processor -- start 2024-04-25T08:37:55.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 -- start start 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcad00b2cb0 0x7fcad00d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00d1020 0x7fcad00d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad00b97a0 con 0x7fcad015dbe0 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad015dfc0 con 0x7fcad00b2cb0 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.944+0000 7fcad5b5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00d1a00 con 0x7fcad00d1020 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcaceffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcad00b2cb0 0x7fcad00d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcacf7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcad015dbe0 0x7fcad00d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59122/0 (socket says 172.21.15.60:59122) 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcacf7fe640 1 -- 172.21.15.60:0/1686943301 learned_addr learned my addr 172.21.15.60:0/1686943301 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcacffff640 1 --2- 172.21.15.60:0/1686943301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00d1020 0x7fcad00d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcaceffd640 1 -- 172.21.15.60:0/1686943301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00d1020 msgr2=0x7fcad00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcaceffd640 1 --2- 172.21.15.60:0/1686943301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00d1020 0x7fcad00d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcaceffd640 1 -- 172.21.15.60:0/1686943301 >> v1:172.21.15.60:6789/0 conn(0x7fcad015dbe0 legacy=0x7fcad00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcaceffd640 1 -- 172.21.15.60:0/1686943301 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00d1c50 con 0x7fcad00b2cb0 2024-04-25T08:37:55.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.945+0000 7fcace7fc640 1 -- 172.21.15.60:0/1686943301 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152975478 0 0) 0x7fcad00b97a0 con 0x7fcad015dbe0 2024-04-25T08:37:55.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcaceffd640 1 --2- 172.21.15.60:0/1686943301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00b2cb0 0x7fcad00d2730 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fcac40097a0 tx=0x7fcac402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ddae5aa250111c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcace7fc640 1 -- 172.21.15.60:0/1686943301 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037070 con 0x7fcad00b2cb0 2024-04-25T08:37:55.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcace7fc640 1 -- 172.21.15.60:0/1686943301 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac4032440 con 0x7fcad00b2cb0 2024-04-25T08:37:55.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcace7fc640 1 -- 172.21.15.60:0/1686943301 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4032810 con 0x7fcad00b2cb0 2024-04-25T08:37:55.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 -- 172.21.15.60:0/1686943301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00b2cb0 msgr2=0x7fcad00d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/1686943301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00b2cb0 0x7fcad00d2730 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fcac40097a0 tx=0x7fcac402a180 comp rx=0 tx=0).stop 2024-04-25T08:37:55.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 -- 172.21.15.60:0/1686943301 shutdown_connections 2024-04-25T08:37:55.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/1686943301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00d1020 0x7fcad00d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/1686943301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00b2cb0 0x7fcad00d2730 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 -- 172.21.15.60:0/1686943301 >> 172.21.15.60:0/1686943301 conn(0x7fcad0061a20 msgr2=0x7fcad0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 -- 172.21.15.60:0/1686943301 shutdown_connections 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.946+0000 7fcad5b5c640 1 -- 172.21.15.60:0/1686943301 wait complete. 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.947+0000 7fcad5b5c640 1 Processor -- start 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 -- start start 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00b2cb0 0x7fcad00b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 0x7fcad015e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad015dbe0 0x7fcad0157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00b92a0 con 0x7fcad00d1020 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00d2210 con 0x7fcad015dbe0 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcad5b5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00b9420 con 0x7fcad00b2cb0 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.948+0000 7fcaceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 0x7fcad015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 0x7fcad015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55892/0 (socket says 172.21.15.60:55892) 2024-04-25T08:37:55.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 -- 172.21.15.60:0/354762955 learned_addr learned my addr 172.21.15.60:0/354762955 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcacf7fe640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00b2cb0 0x7fcad00b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcacffff640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad015dbe0 0x7fcad0157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 -- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00b2cb0 msgr2=0x7fcad00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00b2cb0 0x7fcad00b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 -- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad015dbe0 msgr2=0x7fcad0157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad015dbe0 0x7fcad0157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 -- 172.21.15.60:0/354762955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad014d370 con 0x7fcad00d1020 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcaceffd640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 0x7fcad015e150 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fcac4000c00 tx=0x7fcac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037040 con 0x7fcad00d1020 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac4042070 con 0x7fcad00d1020 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad0156890 con 0x7fcad00d1020 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4030bb0 con 0x7fcad00d1020 2024-04-25T08:37:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.949+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad0156b30 con 0x7fcad00d1020 2024-04-25T08:37:55.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.950+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad0073990 con 0x7fcad00d1020 2024-04-25T08:37:55.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.951+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcac4047460 con 0x7fcad00d1020 2024-04-25T08:37:55.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.951+0000 7fcab7fff640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcabc06b500 0x7fcabc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:55.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.951+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcac40aaf40 con 0x7fcad00d1020 2024-04-25T08:37:55.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.951+0000 7fcacf7fe640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcabc06b500 0x7fcabc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:55.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.952+0000 7fcacf7fe640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcabc06b500 0x7fcabc06d9c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcac0000f60 tx=0x7fcac0007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:55.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:55.956+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcac4032e20 con 0x7fcad00d1020 2024-04-25T08:37:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.207+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcad01552a0 con 0x7fcad00d1020 2024-04-25T08:37:56.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.207+0000 7fcab7fff640 1 -- 172.21.15.60:0/354762955 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fcac4079c30 con 0x7fcad00d1020 2024-04-25T08:37:56.201 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:56.201 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":536,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2589966336,"bytes_avail":770504146944,"bytes_total":773094113280,"write_bytes_sec":49837432,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T08:37:53.515641+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T08:37:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcabc06b500 msgr2=0x7fcabc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcabc06b500 0x7fcabc06d9c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcac0000f60 tx=0x7fcac0007c00 comp rx=0 tx=0).stop 2024-04-25T08:37:56.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 msgr2=0x7fcad015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 0x7fcad015e150 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fcac4000c00 tx=0x7fcac40046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 shutdown_connections 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcabc06b500 0x7fcabc06d9c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcad015dbe0 0x7fcad0157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcad00d1020 0x7fcad015e150 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 --2- 172.21.15.60:0/354762955 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcad00b2cb0 0x7fcad00b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 >> 172.21.15.60:0/354762955 conn(0x7fcad0061a20 msgr2=0x7fcad00b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 shutdown_connections 2024-04-25T08:37:56.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:56.209+0000 7fcad5b5c640 1 -- 172.21.15.60:0/354762955 wait complete. 2024-04-25T08:37:56.214 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T08:37:57.215 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 Processor -- start 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 -- start start 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f758c15a790 0x7f758c0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f758c06fea0 0x7f758c0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758c0b3710 con 0x7f758c0af9c0 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0cf0e0 con 0x7f758c06fea0 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.335+0000 7f7591b5c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0cf3c0 con 0x7f758c15a790 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f758c06fea0 0x7f758c0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f758c06fea0 0x7f758c0d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55906/0 (socket says 172.21.15.60:55906) 2024-04-25T08:37:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 -- 172.21.15.60:0/528789154 learned_addr learned my addr 172.21.15.60:0/528789154 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758b7fe640 1 --2- 172.21.15.60:0/528789154 >> v2:172.21.15.100:3300/0 conn(0x7f758c15a790 0x7f758c0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 -- 172.21.15.60:0/528789154 >> v2:172.21.15.100:3300/0 conn(0x7f758c15a790 msgr2=0x7f758c0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 --2- 172.21.15.60:0/528789154 >> v2:172.21.15.100:3300/0 conn(0x7f758c15a790 0x7f758c0d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 -- 172.21.15.60:0/528789154 >> v1:172.21.15.100:6789/0 conn(0x7f758c0af9c0 legacy=0x7f758c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:57.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.336+0000 7f758affd640 1 -- 172.21.15.60:0/528789154 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0cf6a0 con 0x7f758c06fea0 2024-04-25T08:37:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f758affd640 1 --2- 172.21.15.60:0/528789154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c06fea0 0x7f758c0d08d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f758c070780 tx=0x7f757c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9efdb587c978397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f758a7fc640 1 -- 172.21.15.60:0/528789154 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c037070 con 0x7f758c06fea0 2024-04-25T08:37:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f758a7fc640 1 -- 172.21.15.60:0/528789154 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c032440 con 0x7f758c06fea0 2024-04-25T08:37:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f758a7fc640 1 -- 172.21.15.60:0/528789154 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c032810 con 0x7f758c06fea0 2024-04-25T08:37:57.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 -- 172.21.15.60:0/528789154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c06fea0 msgr2=0x7f758c0d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 --2- 172.21.15.60:0/528789154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c06fea0 0x7f758c0d08d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f758c070780 tx=0x7f757c02a200 comp rx=0 tx=0).stop 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 -- 172.21.15.60:0/528789154 shutdown_connections 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 --2- 172.21.15.60:0/528789154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c06fea0 0x7f758c0d08d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 --2- 172.21.15.60:0/528789154 >> v2:172.21.15.100:3300/0 conn(0x7f758c15a790 0x7f758c0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 -- 172.21.15.60:0/528789154 >> 172.21.15.60:0/528789154 conn(0x7f758c0b2370 msgr2=0x7f758c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.337+0000 7f7591b5c640 1 -- 172.21.15.60:0/528789154 shutdown_connections 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.338+0000 7f7591b5c640 1 -- 172.21.15.60:0/528789154 wait complete. 2024-04-25T08:37:57.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.339+0000 7f7591b5c640 1 Processor -- start 2024-04-25T08:37:57.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.339+0000 7f7591b5c640 1 -- start start 2024-04-25T08:37:57.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758c06fea0 0x7f758c151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 0x7f758c1f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758c15a790 0x7f758c1e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15ab70 con 0x7f758c0af9c0 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0cf980 con 0x7f758c15a790 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15acf0 con 0x7f758c06fea0 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 0x7f758c1f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 0x7f758c1f15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55914/0 (socket says 172.21.15.60:55914) 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 -- 172.21.15.60:0/625683942 learned_addr learned my addr 172.21.15.60:0/625683942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758b7fe640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758c06fea0 0x7f758c151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758bfff640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758c15a790 0x7f758c1e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 -- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758c06fea0 msgr2=0x7f758c151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758c06fea0 0x7f758c151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 -- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758c15a790 msgr2=0x7f758c1e4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758c15a790 0x7f758c1e4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 -- 172.21.15.60:0/625683942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c152080 con 0x7f758c0af9c0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758b7fe640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758c06fea0 0x7f758c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758bfff640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758c15a790 0x7f758c1e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f758affd640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 0x7f758c1f15b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f757c000c00 tx=0x7f757c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c041020 con 0x7f758c0af9c0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c0306c0 con 0x7f758c0af9c0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.340+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c1558f0 con 0x7f758c0af9c0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.341+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c037040 con 0x7f758c0af9c0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.341+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c1e34f0 con 0x7f758c0af9c0 2024-04-25T08:37:57.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.342+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c1572e0 con 0x7f758c0af9c0 2024-04-25T08:37:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.342+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f757c044020 con 0x7f758c0af9c0 2024-04-25T08:37:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.342+0000 7f7583fff640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756c06b4d0 0x7f756c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.343+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f757c0aac00 con 0x7f758c0af9c0 2024-04-25T08:37:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.343+0000 7f758b7fe640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756c06b4d0 0x7f756c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.343+0000 7f758b7fe640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756c06b4d0 0x7f756c06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7574006fd0 tx=0x7f7574008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:57.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.347+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f758c1572e0 con 0x7f758c0af9c0 2024-04-25T08:37:57.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:57.529 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:57.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.569+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f758c1f1af0 con 0x7f758c0af9c0 2024-04-25T08:37:57.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.569+0000 7f7583fff640 1 -- 172.21.15.60:0/625683942 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f757c04b030 con 0x7f758c0af9c0 2024-04-25T08:37:57.563 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:57.563 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:57.564 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:57.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.571+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756c06b4d0 msgr2=0x7f756c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.571+0000 7f7591b5c640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756c06b4d0 0x7f756c06d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7574006fd0 tx=0x7f7574008040 comp rx=0 tx=0).stop 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.571+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 msgr2=0x7f758c1f15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.571+0000 7f7591b5c640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 0x7f758c1f15b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f757c000c00 tx=0x7f757c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 shutdown_connections 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f756c06b4d0 0x7f756c06d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f758c15a790 0x7f758c1e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f758c0af9c0 0x7f758c1f15b0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 --2- 172.21.15.60:0/625683942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f758c06fea0 0x7f758c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 >> 172.21.15.60:0/625683942 conn(0x7f758c0b2370 msgr2=0x7f758c158600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 shutdown_connections 2024-04-25T08:37:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.572+0000 7f7591b5c640 1 -- 172.21.15.60:0/625683942 wait complete. 2024-04-25T08:37:57.576 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:57.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:57.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.731+0000 7f579c5d5640 1 Processor -- start 2024-04-25T08:37:57.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.731+0000 7f579c5d5640 1 -- start start 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579c5d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f579415dc90 0x7f57941625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579c5d5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57940cc8e0 0x7f579415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579c5d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579415f390 con 0x7f57940d1810 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579c5d5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57940cce50 con 0x7f57940cc8e0 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579c5d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5794161af0 con 0x7f579415dc90 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57940cc8e0 0x7f579415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57940cc8e0 0x7f579415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55926/0 (socket says 172.21.15.60:55926) 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 -- 172.21.15.60:0/3100369858 learned_addr learned my addr 172.21.15.60:0/3100369858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:57.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579a371640 1 --2- 172.21.15.60:0/3100369858 >> v2:172.21.15.100:3300/0 conn(0x7f579415dc90 0x7f57941625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 -- 172.21.15.60:0/3100369858 >> v2:172.21.15.100:3300/0 conn(0x7f579415dc90 msgr2=0x7f57941625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 --2- 172.21.15.60:0/3100369858 >> v2:172.21.15.100:3300/0 conn(0x7f579415dc90 0x7f57941625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 -- 172.21.15.60:0/3100369858 >> v1:172.21.15.100:6789/0 conn(0x7f57940d1810 legacy=0x7f57940cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f5799b70640 1 -- 172.21.15.60:0/3100369858 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794161dd0 con 0x7f57940cc8e0 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.732+0000 7f579a371640 1 --2- 172.21.15.60:0/3100369858 >> v2:172.21.15.100:3300/0 conn(0x7f579415dc90 0x7f57941625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.733+0000 7f5799b70640 1 --2- 172.21.15.60:0/3100369858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579415ee50 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f579406b820 tx=0x7f579002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1709c9662ae4046a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.733+0000 7f579936f640 1 -- 172.21.15.60:0/3100369858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790037070 con 0x7f57940cc8e0 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.733+0000 7f579936f640 1 -- 172.21.15.60:0/3100369858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790032440 con 0x7f57940cc8e0 2024-04-25T08:37:57.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.733+0000 7f579936f640 1 -- 172.21.15.60:0/3100369858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790032810 con 0x7f57940cc8e0 2024-04-25T08:37:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 -- 172.21.15.60:0/3100369858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 msgr2=0x7f579415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 --2- 172.21.15.60:0/3100369858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579415ee50 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f579406b820 tx=0x7f579002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 -- 172.21.15.60:0/3100369858 shutdown_connections 2024-04-25T08:37:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 --2- 172.21.15.60:0/3100369858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579415ee50 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 --2- 172.21.15.60:0/3100369858 >> v2:172.21.15.100:3300/0 conn(0x7f579415dc90 0x7f57941625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 -- 172.21.15.60:0/3100369858 >> 172.21.15.60:0/3100369858 conn(0x7f579406f590 msgr2=0x7f579406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 -- 172.21.15.60:0/3100369858 shutdown_connections 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.734+0000 7f579c5d5640 1 -- 172.21.15.60:0/3100369858 wait complete. 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.735+0000 7f579c5d5640 1 Processor -- start 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.735+0000 7f579c5d5640 1 -- start start 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579c5d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579c5d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57940d1810 0x7f57941f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579c5d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f579415dc90 0x7f57941f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579c5d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579415e070 con 0x7f57940cc8e0 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579c5d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940cd130 con 0x7f57940d1810 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579c5d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579415e1f0 con 0x7f579415dc90 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579406caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55932/0 (socket says 172.21.15.60:55932) 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 -- 172.21.15.60:0/405063006 learned_addr learned my addr 172.21.15.60:0/405063006 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f5799b70640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57940d1810 0x7f57941f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579ab72640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f579415dc90 0x7f57941f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 -- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f579415dc90 msgr2=0x7f57941f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f579415dc90 0x7f57941f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 -- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57940d1810 msgr2=0x7f57941f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57940d1810 0x7f57941f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 -- 172.21.15.60:0/405063006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57941f7dd0 con 0x7f57940cc8e0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f5799b70640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57940d1810 0x7f57941f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579ab72640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f579415dc90 0x7f57941f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f579a371640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579406caf0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f578400c7f0 tx=0x7f578400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.736+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784015070 con 0x7f57940cc8e0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.737+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5784007820 con 0x7f57940cc8e0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.737+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784007bd0 con 0x7f57940cc8e0 2024-04-25T08:37:57.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.737+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579414e170 con 0x7f57940cc8e0 2024-04-25T08:37:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.737+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579414e730 con 0x7f57940cc8e0 2024-04-25T08:37:57.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.738+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5794073990 con 0x7f57940cc8e0 2024-04-25T08:37:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.738+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5784018070 con 0x7f57940cc8e0 2024-04-25T08:37:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.738+0000 7f578affd640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f577806b4d0 0x7f577806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.738+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f578408a430 con 0x7f57940cc8e0 2024-04-25T08:37:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.739+0000 7f5799b70640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f577806b4d0 0x7f577806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:57.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.739+0000 7f5799b70640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f577806b4d0 0x7f577806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5790034000 tx=0x7f579002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:57.736 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.743+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57840591b0 con 0x7f57940cc8e0 2024-04-25T08:37:57.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.995+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f57940cf6b0 con 0x7f57940cc8e0 2024-04-25T08:37:57.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.996+0000 7f578affd640 1 -- 172.21.15.60:0/405063006 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f5784058ab0 con 0x7f57940cc8e0 2024-04-25T08:37:57.990 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:57.990 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":538,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2589966336,"bytes_avail":770504146944,"bytes_total":773094113280,"write_bytes_sec":49837432,"read_op_per_sec":0,"write_op_per_sec":49},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T08:37:53.515641+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f577806b4d0 msgr2=0x7f577806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f577806b4d0 0x7f577806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5790034000 tx=0x7f579002e000 comp rx=0 tx=0).stop 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 msgr2=0x7f579406caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579406caf0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f578400c7f0 tx=0x7f578400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 shutdown_connections 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f579415dc90 0x7f57941f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f577806b4d0 0x7f577806d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57940d1810 0x7f57941f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 --2- 172.21.15.60:0/405063006 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57940cc8e0 0x7f579406caf0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:57.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 >> 172.21.15.60:0/405063006 conn(0x7f579406f590 msgr2=0x7f579406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:57.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 shutdown_connections 2024-04-25T08:37:57.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:57.998+0000 7f579c5d5640 1 -- 172.21.15.60:0/405063006 wait complete. 2024-04-25T08:37:58.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T08:37:59.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:37:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.124+0000 7fb82279e640 1 Processor -- start 2024-04-25T08:37:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.124+0000 7fb82279e640 1 -- start start 2024-04-25T08:37:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb82279e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 0x7fb81c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb82279e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 0x7fb81c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb82279e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0d1df0 con 0x7fb81c0b9770 2024-04-25T08:37:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb82279e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c0d20d0 con 0x7fb81c15dd30 2024-04-25T08:37:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb82279e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0d23b0 con 0x7fb81c0d0ad0 2024-04-25T08:37:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb81bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 0x7fb81c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb820d3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 0x7fb81c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.124 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb820d3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 0x7fb81c0b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:50714/0 (socket says 172.21.15.60:50714) 2024-04-25T08:37:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb81bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 0x7fb81c0b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:59960/0 (socket says 172.21.15.60:59960) 2024-04-25T08:37:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.125+0000 7fb820d3b640 1 -- 172.21.15.60:0/1965606421 learned_addr learned my addr 172.21.15.60:0/1965606421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:59.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.126+0000 7fb820d3b640 1 -- 172.21.15.60:0/1965606421 >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 msgr2=0x7fb81c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.126+0000 7fb820d3b640 1 --2- 172.21.15.60:0/1965606421 >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 0x7fb81c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.126+0000 7fb820d3b640 1 -- 172.21.15.60:0/1965606421 >> v1:172.21.15.100:6789/0 conn(0x7fb81c0b9770 legacy=0x7fb81c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.126+0000 7fb820d3b640 1 -- 172.21.15.60:0/1965606421 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0d2690 con 0x7fb81c0d0ad0 2024-04-25T08:37:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.126+0000 7fb81bfff640 1 --2- 172.21.15.60:0/1965606421 >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 0x7fb81c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb820d3b640 1 --2- 172.21.15.60:0/1965606421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 0x7fb81c0b6a40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb814009e50 tx=0x7fb81402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77062fc9923faf14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:59.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb81affd640 1 -- 172.21.15.60:0/1965606421 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814036070 con 0x7fb81c0d0ad0 2024-04-25T08:37:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb81affd640 1 -- 172.21.15.60:0/1965606421 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814031600 con 0x7fb81c0d0ad0 2024-04-25T08:37:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb81affd640 1 -- 172.21.15.60:0/1965606421 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140319d0 con 0x7fb81c0d0ad0 2024-04-25T08:37:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 -- 172.21.15.60:0/1965606421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 msgr2=0x7fb81c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 --2- 172.21.15.60:0/1965606421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 0x7fb81c0b6a40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb814009e50 tx=0x7fb81402a680 comp rx=0 tx=0).stop 2024-04-25T08:37:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 -- 172.21.15.60:0/1965606421 shutdown_connections 2024-04-25T08:37:59.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 --2- 172.21.15.60:0/1965606421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c0d0ad0 0x7fb81c0b6a40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 --2- 172.21.15.60:0/1965606421 >> v2:172.21.15.100:3300/0 conn(0x7fb81c15dd30 0x7fb81c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 -- 172.21.15.60:0/1965606421 >> 172.21.15.60:0/1965606421 conn(0x7fb81c061c30 msgr2=0x7fb81c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 -- 172.21.15.60:0/1965606421 shutdown_connections 2024-04-25T08:37:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.127+0000 7fb82279e640 1 -- 172.21.15.60:0/1965606421 wait complete. 2024-04-25T08:37:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 Processor -- start 2024-04-25T08:37:59.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 -- start start 2024-04-25T08:37:59.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 0x7fb81c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81c0d0ad0 0x7fb81c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c15dd30 0x7fb81c0aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0d1b90 con 0x7fb81c0b9770 2024-04-25T08:37:59.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0d1a10 con 0x7fb81c0d0ad0 2024-04-25T08:37:59.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.129+0000 7fb82279e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0d2e90 con 0x7fb81c15dd30 2024-04-25T08:37:59.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 0x7fb81c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 0x7fb81c14d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55948/0 (socket says 172.21.15.60:55948) 2024-04-25T08:37:59.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 -- 172.21.15.60:0/1581992335 learned_addr learned my addr 172.21.15.60:0/1581992335 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:59.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb820d3b640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c15dd30 0x7fb81c0aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81b7fe640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81c0d0ad0 0x7fb81c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 -- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c15dd30 msgr2=0x7fb81c0aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c15dd30 0x7fb81c0aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 -- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81c0d0ad0 msgr2=0x7fb81c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81c0d0ad0 0x7fb81c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 -- 172.21.15.60:0/1581992335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0cb4d0 con 0x7fb81c0b9770 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81b7fe640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81c0d0ad0 0x7fb81c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:59.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb81bfff640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 0x7fb81c14d200 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e960 tx=0x7fb80c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c017070 con 0x7fb81c0b9770 2024-04-25T08:37:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c00ca00 con 0x7fb81c0b9770 2024-04-25T08:37:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c0cb7b0 con 0x7fb81c0b9770 2024-04-25T08:37:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.130+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00cdd0 con 0x7fb81c0b9770 2024-04-25T08:37:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.131+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0cbd70 con 0x7fb81c0b9770 2024-04-25T08:37:59.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.131+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c073990 con 0x7fb81c0b9770 2024-04-25T08:37:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.132+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb80c01a070 con 0x7fb81c0b9770 2024-04-25T08:37:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.132+0000 7fb818ff9640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.132+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb80c08a530 con 0x7fb81c0b9770 2024-04-25T08:37:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.133+0000 7fb81b7fe640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.133+0000 7fb81b7fe640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0cf510 tx=0x7fb804009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:59.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.137+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb80c058be0 con 0x7fb81c0b9770 2024-04-25T08:37:59.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.354+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb81c0c7080 con 0x7fb81c0b9770 2024-04-25T08:37:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.355+0000 7fb818ff9640 1 -- 172.21.15.60:0/1581992335 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb80c05cc80 con 0x7fb81c0b9770 2024-04-25T08:37:59.349 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:59.349 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:37:59.349 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:37:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb7fc06b4d0 msgr2=0x7fb7fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0cf510 tx=0x7fb804009040 comp rx=0 tx=0).stop 2024-04-25T08:37:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 msgr2=0x7fb81c14d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 0x7fb81c14d200 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e960 tx=0x7fb80c00ee20 comp rx=0 tx=0).stop 2024-04-25T08:37:59.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 shutdown_connections 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb81c15dd30 0x7fb81c0aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb81c0d0ad0 0x7fb81c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 --2- 172.21.15.60:0/1581992335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb81c0b9770 0x7fb81c14d200 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 >> 172.21.15.60:0/1581992335 conn(0x7fb81c061c30 msgr2=0x7fb81c0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 shutdown_connections 2024-04-25T08:37:59.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.357+0000 7fb82279e640 1 -- 172.21.15.60:0/1581992335 wait complete. 2024-04-25T08:37:59.362 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:37:59.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:37:59.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.468+0000 7fbad7c09640 1 Processor -- start 2024-04-25T08:37:59.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.468+0000 7fbad7c09640 1 -- start start 2024-04-25T08:37:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad7c09640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbad00b2cb0 0x7fbad00d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad7c09640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00b97a0 con 0x7fbad015dbe0 2024-04-25T08:37:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad7c09640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad00d1c50 con 0x7fbad00d1020 2024-04-25T08:37:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad7c09640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad015dfc0 con 0x7fbad00b2cb0 2024-04-25T08:37:59.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad51a4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbad00b2cb0 0x7fbad00d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad59a5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbad015dbe0 0x7fbad00d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59126/0 (socket says 172.21.15.60:59126) 2024-04-25T08:37:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad59a5640 1 -- 172.21.15.60:0/1756582126 learned_addr learned my addr 172.21.15.60:0/1756582126 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:37:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad51a4640 1 -- 172.21.15.60:0/1756582126 >> v1:172.21.15.100:6789/0 conn(0x7fbad00d1020 legacy=0x7fbad00d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad51a4640 1 -- 172.21.15.60:0/1756582126 >> v1:172.21.15.60:6789/0 conn(0x7fbad015dbe0 legacy=0x7fbad00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad49a3640 1 -- 172.21.15.60:0/1756582126 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189387197 0 0) 0x7fbad00b97a0 con 0x7fbad015dbe0 2024-04-25T08:37:59.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.469+0000 7fbad51a4640 1 -- 172.21.15.60:0/1756582126 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00d1f30 con 0x7fbad00b2cb0 2024-04-25T08:37:59.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.470+0000 7fbad51a4640 1 --2- 172.21.15.60:0/1756582126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad00d2730 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fbac00097a0 tx=0x7fbac002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b659e42235bc61b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.470+0000 7fbad49a3640 1 -- 172.21.15.60:0/1756582126 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0037070 con 0x7fbad00b2cb0 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.470+0000 7fbad49a3640 1 -- 172.21.15.60:0/1756582126 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac0032440 con 0x7fbad00b2cb0 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.470+0000 7fbad49a3640 1 -- 172.21.15.60:0/1756582126 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0032810 con 0x7fbad00b2cb0 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.470+0000 7fbad7c09640 1 -- 172.21.15.60:0/1756582126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 msgr2=0x7fbad00d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.470+0000 7fbad7c09640 1 --2- 172.21.15.60:0/1756582126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad00d2730 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fbac00097a0 tx=0x7fbac002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.471+0000 7fbad7c09640 1 -- 172.21.15.60:0/1756582126 shutdown_connections 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.471+0000 7fbad7c09640 1 --2- 172.21.15.60:0/1756582126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad00d2730 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.471+0000 7fbad7c09640 1 -- 172.21.15.60:0/1756582126 >> 172.21.15.60:0/1756582126 conn(0x7fbad0061a20 msgr2=0x7fbad0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.471+0000 7fbad7c09640 1 -- 172.21.15.60:0/1756582126 shutdown_connections 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.471+0000 7fbad7c09640 1 -- 172.21.15.60:0/1756582126 wait complete. 2024-04-25T08:37:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 Processor -- start 2024-04-25T08:37:59.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 -- start start 2024-04-25T08:37:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad00d1020 0x7fbad00b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbad015dbe0 0x7fbad01605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00b92a0 con 0x7fbad00b2cb0 2024-04-25T08:37:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00d1a00 con 0x7fbad00d1020 2024-04-25T08:37:59.469 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.472+0000 7fbad7c09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad00b9420 con 0x7fbad015dbe0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad0151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55980/0 (socket says 172.21.15.60:55980) 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 -- 172.21.15.60:0/2123924031 learned_addr learned my addr 172.21.15.60:0/2123924031 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 -- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbad015dbe0 msgr2=0x7fbad01605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbad015dbe0 0x7fbad01605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 -- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad00d1020 msgr2=0x7fbad00b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad00d1020 0x7fbad00b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 -- 172.21.15.60:0/2123924031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00b31c0 con 0x7fbad00b2cb0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad59a5640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad0151e90 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00c460 tx=0x7fbacc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc018070 con 0x7fbad00b2cb0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc007500 con 0x7fbad00b2cb0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad00b34a0 con 0x7fbad00b2cb0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc004a20 con 0x7fbad00b2cb0 2024-04-25T08:37:59.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.473+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad00b3a60 con 0x7fbad00b2cb0 2024-04-25T08:37:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.474+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad0073990 con 0x7fbad00b2cb0 2024-04-25T08:37:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.475+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbacc01b070 con 0x7fbad00b2cb0 2024-04-25T08:37:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.475+0000 7fbac67fc640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.475+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbacc0895d0 con 0x7fbad00b2cb0 2024-04-25T08:37:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.475+0000 7fbad51a4640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:59.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.476+0000 7fbad51a4640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbab406b4d0 0x7fbab406d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbac0034000 tx=0x7fbac002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:59.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.479+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbacc058350 con 0x7fbad00b2cb0 2024-04-25T08:37:59.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.733+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbad00c9f80 con 0x7fbad00b2cb0 2024-04-25T08:37:59.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.734+0000 7fbac67fc640 1 -- 172.21.15.60:0/2123924031 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fbacc057c50 con 0x7fbad00b2cb0 2024-04-25T08:37:59.728 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:37:59.728 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":540,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076566130,"bytes_used":2589966336,"bytes_avail":770504146944,"bytes_total":773094113280,"write_bytes_sec":25626457,"read_op_per_sec":0,"write_op_per_sec":25},"fsmap":{"epoch":71,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":8805}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T08:37:57.516705+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbab406b4d0 msgr2=0x7fbab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbab406b4d0 0x7fbab406d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbac0034000 tx=0x7fbac002c000 comp rx=0 tx=0).stop 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 msgr2=0x7fbad0151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad0151e90 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00c460 tx=0x7fbacc00c920 comp rx=0 tx=0).stop 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 shutdown_connections 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbad015dbe0 0x7fbad01605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbab406b4d0 0x7fbab406d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad00d1020 0x7fbad00b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 --2- 172.21.15.60:0/2123924031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbad00b2cb0 0x7fbad0151e90 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 >> 172.21.15.60:0/2123924031 conn(0x7fbad0061a20 msgr2=0x7fbad00b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:59.730 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 shutdown_connections 2024-04-25T08:37:59.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:37:59.736+0000 7fbad7c09640 1 -- 172.21.15.60:0/2123924031 wait complete. 2024-04-25T08:37:59.740 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:37:59.740 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:37:59.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:37:59.789 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:37:59.832 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:37:59.833 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:59.833 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:59.849 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:37:59.849 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:37:59.904 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:00.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.051+0000 7f45e5d0a640 1 Processor -- start 2024-04-25T08:38:00.045 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.051+0000 7f45e5d0a640 1 -- start start 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45e5d0a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f45e015da40 0x7f45e006bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45e5d0a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45e006c2f0 0x7f45e00ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45e5d0a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e006ad10 con 0x7f45e006f700 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45e5d0a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e006b920 con 0x7f45e006c2f0 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45e5d0a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e006aff0 con 0x7f45e015da40 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45e006c2f0 0x7f45e00ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f45e006c2f0 0x7f45e00ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55984/0 (socket says 172.21.15.60:55984) 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 -- 172.21.15.60:0/3047629972 learned_addr learned my addr 172.21.15.60:0/3047629972 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.047 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45dffff640 1 --1- 172.21.15.60:0/3047629972 >> v1:172.21.15.60:6789/0 conn(0x7f45e006f700 0x7f45e00b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59132/0 (socket says 172.21.15.60:59132) 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45df7fe640 1 --2- 172.21.15.60:0/3047629972 >> v2:172.21.15.100:3300/0 conn(0x7f45e015da40 0x7f45e006bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 -- 172.21.15.60:0/3047629972 >> v2:172.21.15.100:3300/0 conn(0x7f45e015da40 msgr2=0x7f45e006bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 --2- 172.21.15.60:0/3047629972 >> v2:172.21.15.100:3300/0 conn(0x7f45e015da40 0x7f45e006bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 -- 172.21.15.60:0/3047629972 >> v1:172.21.15.60:6789/0 conn(0x7f45e006f700 legacy=0x7f45e00b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.052+0000 7f45deffd640 1 -- 172.21.15.60:0/3047629972 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e006b2d0 con 0x7f45e006c2f0 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.053+0000 7f45deffd640 1 --2- 172.21.15.60:0/3047629972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e006c2f0 0x7f45e00ace60 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009870 tx=0x7f45c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5802fe6a790f6b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.053+0000 7f45de7fc640 1 -- 172.21.15.60:0/3047629972 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8037070 con 0x7f45e006c2f0 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.053+0000 7f45de7fc640 1 -- 172.21.15.60:0/3047629972 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8032440 con 0x7f45e006c2f0 2024-04-25T08:38:00.048 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.053+0000 7f45de7fc640 1 -- 172.21.15.60:0/3047629972 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8032810 con 0x7f45e006c2f0 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 -- 172.21.15.60:0/3047629972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e006c2f0 msgr2=0x7f45e00ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/3047629972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e006c2f0 0x7f45e00ace60 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009870 tx=0x7f45c802a180 comp rx=0 tx=0).stop 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 -- 172.21.15.60:0/3047629972 shutdown_connections 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/3047629972 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e006c2f0 0x7f45e00ace60 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/3047629972 >> v2:172.21.15.100:3300/0 conn(0x7f45e015da40 0x7f45e006bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 -- 172.21.15.60:0/3047629972 >> 172.21.15.60:0/3047629972 conn(0x7f45e00cfe10 msgr2=0x7f45e00d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 -- 172.21.15.60:0/3047629972 shutdown_connections 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.054+0000 7f45e5d0a640 1 -- 172.21.15.60:0/3047629972 wait complete. 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.055+0000 7f45e5d0a640 1 Processor -- start 2024-04-25T08:38:00.051 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.055+0000 7f45e5d0a640 1 -- start start 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45e5d0a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e006f700 0x7f45e014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45e5d0a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 0x7f45e015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45e5d0a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e014b3b0 0x7f45e015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45e5d0a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e006ec70 con 0x7f45e015da40 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45e5d0a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e015de20 con 0x7f45e014b3b0 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45e5d0a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00b3590 con 0x7f45e006f700 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 0x7f45e015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 0x7f45e015bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:55988/0 (socket says 172.21.15.60:55988) 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 -- 172.21.15.60:0/1267488358 learned_addr learned my addr 172.21.15.60:0/1267488358 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45dffff640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e014b3b0 0x7f45e015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45df7fe640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e006f700 0x7f45e014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.052 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 -- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e006f700 msgr2=0x7f45e014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e006f700 0x7f45e014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 -- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e014b3b0 msgr2=0x7f45e015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e014b3b0 0x7f45e015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 -- 172.21.15.60:0/1267488358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e0159b30 con 0x7f45e015da40 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.056+0000 7f45deffd640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 0x7f45e015bb60 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f45c8000c00 tx=0x7f45c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.057+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8037040 con 0x7f45e015da40 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.057+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8042070 con 0x7f45e015da40 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.057+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80309a0 con 0x7f45e015da40 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.057+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e0159e10 con 0x7f45e015da40 2024-04-25T08:38:00.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.057+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e015a3d0 con 0x7f45e015da40 2024-04-25T08:38:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.058+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0073990 con 0x7f45e015da40 2024-04-25T08:38:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.058+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45c8030b40 con 0x7f45e015da40 2024-04-25T08:38:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.058+0000 7f45d7fff640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45c006b4d0 0x7f45c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.059+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45c80aac20 con 0x7f45e015da40 2024-04-25T08:38:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.059+0000 7f45df7fe640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45c006b4d0 0x7f45c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.059+0000 7f45df7fe640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45c006b4d0 0x7f45c006d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f45e0159110 tx=0x7f45d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.063+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45c80792b0 con 0x7f45e015da40 2024-04-25T08:38:00.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.246+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45e00617b0 con 0x7f45e015da40 2024-04-25T08:38:00.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.247+0000 7f45d7fff640 1 -- 172.21.15.60:0/1267488358 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f45c807d350 con 0x7f45e015da40 2024-04-25T08:38:00.241 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:00.241 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:30.801169+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45c006b4d0 msgr2=0x7f45c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45c006b4d0 0x7f45c006d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f45e0159110 tx=0x7f45d0008040 comp rx=0 tx=0).stop 2024-04-25T08:38:00.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 msgr2=0x7f45e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 0x7f45e015bb60 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f45c8000c00 tx=0x7f45c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 shutdown_connections 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f45c006b4d0 0x7f45c006d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45e014b3b0 0x7f45e015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f45e015da40 0x7f45e015bb60 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 --2- 172.21.15.60:0/1267488358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45e006f700 0x7f45e014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 >> 172.21.15.60:0/1267488358 conn(0x7f45e00cfe10 msgr2=0x7f45e00aea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 shutdown_connections 2024-04-25T08:38:00.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.249+0000 7f45e5d0a640 1 -- 172.21.15.60:0/1267488358 wait complete. 2024-04-25T08:38:00.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:37:27.446495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:37:28.138802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:00.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.364+0000 7f8ef02cb640 1 Processor -- start 2024-04-25T08:38:00.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.364+0000 7f8ef02cb640 1 -- start start 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8ef02cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8ef02cb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ee80cfd10 0x7f8ee80cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8ef02cb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee80c75e0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8ef02cb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee80c8680 con 0x7f8ee80cfd10 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8ef02cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c78c0 con 0x7f8ee815bb50 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8eee868640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ee80cfd10 0x7f8ee80cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.365+0000 7f8eee868640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8ee80cfd10 0x7f8ee80cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56002/0 (socket says 172.21.15.60:56002) 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee868640 1 -- 172.21.15.60:0/4225429251 learned_addr learned my addr 172.21.15.60:0/4225429251 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee067640 1 --2- 172.21.15.60:0/4225429251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee868640 1 -- 172.21.15.60:0/4225429251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 msgr2=0x7f8ee80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee868640 1 --2- 172.21.15.60:0/4225429251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee80c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee868640 1 -- 172.21.15.60:0/4225429251 >> v1:172.21.15.100:6789/0 conn(0x7f8ee80c70d0 legacy=0x7f8ee80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee868640 1 -- 172.21.15.60:0/4225429251 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80c7ba0 con 0x7f8ee80cfd10 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.366+0000 7f8eee868640 1 --2- 172.21.15.60:0/4225429251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cbf20 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009e50 tx=0x7f8ed802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b6068fa1fbb810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8eed065640 1 -- 172.21.15.60:0/4225429251 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8036070 con 0x7f8ee80cfd10 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8eed065640 1 -- 172.21.15.60:0/4225429251 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed8031600 con 0x7f8ee80cfd10 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8eed065640 1 -- 172.21.15.60:0/4225429251 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80319d0 con 0x7f8ee80cfd10 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 -- 172.21.15.60:0/4225429251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80cfd10 msgr2=0x7f8ee80cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/4225429251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cbf20 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009e50 tx=0x7f8ed802a750 comp rx=0 tx=0).stop 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 -- 172.21.15.60:0/4225429251 shutdown_connections 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/4225429251 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cbf20 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/4225429251 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee80c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 -- 172.21.15.60:0/4225429251 >> 172.21.15.60:0/4225429251 conn(0x7f8ee806f500 msgr2=0x7f8ee806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 -- 172.21.15.60:0/4225429251 shutdown_connections 2024-04-25T08:38:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.367+0000 7f8ef02cb640 1 -- 172.21.15.60:0/4225429251 wait complete. 2024-04-25T08:38:00.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.368+0000 7f8ef02cb640 1 Processor -- start 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.368+0000 7f8ef02cb640 1 -- start start 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8ef02cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 0x7f8ee80b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8ef02cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8ef02cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8ef02cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee815f6d0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8ef02cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c8160 con 0x7f8ee80cfd10 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8ef02cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee815f850 con 0x7f8ee815bb50 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8eee067640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 0x7f8ee80b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8eee067640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 0x7f8ee80b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56006/0 (socket says 172.21.15.60:56006) 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8eee067640 1 -- 172.21.15.60:0/2797128341 learned_addr learned my addr 172.21.15.60:0/2797128341 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8eee868640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.369+0000 7f8eed866640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8eee067640 1 -- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 msgr2=0x7f8ee815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.365 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8eee067640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee815eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8eee067640 1 -- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ee80cfd10 msgr2=0x7f8ee80cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8eee067640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8eee067640 1 -- 172.21.15.60:0/2797128341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80b4520 con 0x7f8ee80c70d0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8eee067640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 0x7f8ee80b6f80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c7f0 tx=0x7f8ee000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0015070 con 0x7f8ee80c70d0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee0004220 con 0x7f8ee80c70d0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee80b4800 con 0x7f8ee80c70d0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0007620 con 0x7f8ee80c70d0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.370+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee80b4dc0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.366 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.371+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee80d05b0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.372+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ee00079d0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.372+0000 7f8ee6ffd640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.372+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ee008a350 con 0x7f8ee80c70d0 2024-04-25T08:38:00.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.372+0000 7f8eed866640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.373+0000 7f8eed866640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ed406b4d0 0x7f8ed406d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80c7510 tx=0x7f8edc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.376+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ee80d05b0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.555+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ee8073a00 con 0x7f8ee80c70d0 2024-04-25T08:38:00.550 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.556+0000 7f8ee6ffd640 1 -- 172.21.15.60:0/2797128341 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ee00589f0 con 0x7f8ee80c70d0 2024-04-25T08:38:00.550 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ed406b4d0 msgr2=0x7f8ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ed406b4d0 0x7f8ed406d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80c7510 tx=0x7f8edc009040 comp rx=0 tx=0).stop 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 msgr2=0x7f8ee80b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 0x7f8ee80b6f80 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c7f0 tx=0x7f8ee000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 shutdown_connections 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8ee815bb50 0x7f8ee815eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ed406b4d0 0x7f8ed406d990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8ee80cfd10 0x7f8ee80cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 --2- 172.21.15.60:0/2797128341 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ee80c70d0 0x7f8ee80b6f80 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 >> 172.21.15.60:0/2797128341 conn(0x7f8ee806f500 msgr2=0x7f8ee806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 shutdown_connections 2024-04-25T08:38:00.552 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.558+0000 7f8ef02cb640 1 -- 172.21.15.60:0/2797128341 wait complete. 2024-04-25T08:38:00.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:38:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.672+0000 7f80d345d640 1 Processor -- start 2024-04-25T08:38:00.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.672+0000 7f80d345d640 1 -- start start 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d345d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d345d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80cc0d1f30 0x7f80cc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d345d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80cc0d34c0 0x7f80cc0d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d345d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc0d3030 con 0x7f80cc0d34c0 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d345d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc0d2420 con 0x7f80cc0d1f30 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d345d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0d2700 con 0x7f80cc15dac0 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80cc0d34c0 0x7f80cc0d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f80cc0d34c0 0x7f80cc0d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56022/0 (socket says 172.21.15.60:56022) 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 -- 172.21.15.60:0/2092532152 learned_addr learned my addr 172.21.15.60:0/2092532152 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d11f9640 1 --2- 172.21.15.60:0/2092532152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d09f8640 1 --2- 172.21.15.60:0/2092532152 >> v2:172.21.15.100:3300/0 conn(0x7f80cc0d1f30 0x7f80cc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 -- 172.21.15.60:0/2092532152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 msgr2=0x7f80cc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 --2- 172.21.15.60:0/2092532152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 -- 172.21.15.60:0/2092532152 >> v2:172.21.15.100:3300/0 conn(0x7f80cc0d1f30 msgr2=0x7f80cc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 --2- 172.21.15.60:0/2092532152 >> v2:172.21.15.100:3300/0 conn(0x7f80cc0d1f30 0x7f80cc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.673+0000 7f80d19fa640 1 -- 172.21.15.60:0/2092532152 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0d29e0 con 0x7f80cc0d34c0 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.674+0000 7f80d19fa640 1 --2- 172.21.15.60:0/2092532152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d34c0 0x7f80cc0d08e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0b6550 tx=0x7f80c402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927c87b01172f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.674+0000 7f80c3fff640 1 -- 172.21.15.60:0/2092532152 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4036070 con 0x7f80cc0d34c0 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.674+0000 7f80c3fff640 1 -- 172.21.15.60:0/2092532152 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c4031600 con 0x7f80cc0d34c0 2024-04-25T08:38:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.674+0000 7f80c3fff640 1 -- 172.21.15.60:0/2092532152 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c40319d0 con 0x7f80cc0d34c0 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.674+0000 7f80d345d640 1 -- 172.21.15.60:0/2092532152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d34c0 msgr2=0x7f80cc0d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.674+0000 7f80d345d640 1 --2- 172.21.15.60:0/2092532152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d34c0 0x7f80cc0d08e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0b6550 tx=0x7f80c402a730 comp rx=0 tx=0).stop 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 -- 172.21.15.60:0/2092532152 shutdown_connections 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 --2- 172.21.15.60:0/2092532152 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d34c0 0x7f80cc0d08e0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 --2- 172.21.15.60:0/2092532152 >> v2:172.21.15.100:3300/0 conn(0x7f80cc0d1f30 0x7f80cc0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 --2- 172.21.15.60:0/2092532152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 -- 172.21.15.60:0/2092532152 >> 172.21.15.60:0/2092532152 conn(0x7f80cc061c30 msgr2=0x7f80cc06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 -- 172.21.15.60:0/2092532152 shutdown_connections 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.675+0000 7f80d345d640 1 -- 172.21.15.60:0/2092532152 wait complete. 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.676+0000 7f80d345d640 1 Processor -- start 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.676+0000 7f80d345d640 1 -- start start 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.676+0000 7f80d345d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 0x7f80cc0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.676+0000 7f80d345d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc0d34c0 0x7f80cc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d345d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d345d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0b97d0 con 0x7f80cc0d1f30 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d345d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15dea0 con 0x7f80cc0d34c0 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d345d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0b9950 con 0x7f80cc15dac0 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 0x7f80cc0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 0x7f80cc0c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56038/0 (socket says 172.21.15.60:56038) 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 -- 172.21.15.60:0/1614149421 learned_addr learned my addr 172.21.15.60:0/1614149421 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d09f8640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc0d34c0 0x7f80cc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d19fa640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 -- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 msgr2=0x7f80cc1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 -- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc0d34c0 msgr2=0x7f80cc152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc0d34c0 0x7f80cc152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 -- 172.21.15.60:0/1614149421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc154780 con 0x7f80cc0d1f30 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d19fa640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d11f9640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 0x7f80cc0c9fc0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f80c800d5f0 tx=0x7f80c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80125f0 con 0x7f80cc0d1f30 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c8019070 con 0x7f80cc0d1f30 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80129d0 con 0x7f80cc0d1f30 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.677+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc154a60 con 0x7f80cc0d1f30 2024-04-25T08:38:00.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.678+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc0c86a0 con 0x7f80cc0d1f30 2024-04-25T08:38:00.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.679+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc073a00 con 0x7f80cc0d1f30 2024-04-25T08:38:00.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.679+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80c8012b70 con 0x7f80cc0d1f30 2024-04-25T08:38:00.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.679+0000 7f80c1ffb640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b006b4d0 0x7f80b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.680+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80c808a730 con 0x7f80cc0d1f30 2024-04-25T08:38:00.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.680+0000 7f80d09f8640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b006b4d0 0x7f80b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.680+0000 7f80d09f8640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b006b4d0 0x7f80b006d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0ca500 tx=0x7f80bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.684+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80c8059380 con 0x7f80cc0d1f30 2024-04-25T08:38:00.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.864+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80cc0b86c0 con 0x7f80cc0d1f30 2024-04-25T08:38:00.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.864+0000 7f80c1ffb640 1 -- 172.21.15.60:0/1614149421 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80c8058c80 con 0x7f80cc0d1f30 2024-04-25T08:38:00.858 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b006b4d0 msgr2=0x7f80b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b006b4d0 0x7f80b006d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0ca500 tx=0x7f80bc008040 comp rx=0 tx=0).stop 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 msgr2=0x7f80cc0c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 0x7f80cc0c9fc0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f80c800d5f0 tx=0x7f80c800dab0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 shutdown_connections 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f80cc15dac0 0x7f80cc1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.860 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f80b006b4d0 0x7f80b006d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80cc0d34c0 0x7f80cc152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 --2- 172.21.15.60:0/1614149421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80cc0d1f30 0x7f80cc0c9fc0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 >> 172.21.15.60:0/1614149421 conn(0x7f80cc061c30 msgr2=0x7f80cc0b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 shutdown_connections 2024-04-25T08:38:00.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.866+0000 7f80d345d640 1 -- 172.21.15.60:0/1614149421 wait complete. 2024-04-25T08:38:00.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:38:00.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.980+0000 7fd0afba5640 1 Processor -- start 2024-04-25T08:38:00.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.981+0000 7fd0afba5640 1 -- start start 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.981+0000 7fd0afba5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0a80cadb0 0x7fd0a80cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.981+0000 7fd0afba5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a80cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.981+0000 7fd0afba5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a80afc80 con 0x7fd0a815de80 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.981+0000 7fd0afba5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a80cb240 con 0x7fd0a80cadb0 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.981+0000 7fd0afba5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80ce4d0 con 0x7fd0a80ccfe0 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0a80cadb0 0x7fd0a80cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0a80cadb0 0x7fd0a80cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56046/0 (socket says 172.21.15.60:56046) 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 -- 172.21.15.60:0/1697365685 learned_addr learned my addr 172.21.15.60:0/1697365685 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ae142640 1 --2- 172.21.15.60:0/1697365685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a80cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 -- 172.21.15.60:0/1697365685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 msgr2=0x7fd0a80cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 --2- 172.21.15.60:0/1697365685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a80cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 -- 172.21.15.60:0/1697365685 >> v1:172.21.15.100:6789/0 conn(0x7fd0a815de80 legacy=0x7fd0a80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:00.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.982+0000 7fd0ad140640 1 -- 172.21.15.60:0/1697365685 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a80ce7b0 con 0x7fd0a80cadb0 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0ad140640 1 --2- 172.21.15.60:0/1697365685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80cf6c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009870 tx=0x7fd09c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20093d937d2ea9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0ac93f640 1 -- 172.21.15.60:0/1697365685 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c037070 con 0x7fd0a80cadb0 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0ac93f640 1 -- 172.21.15.60:0/1697365685 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c032440 con 0x7fd0a80cadb0 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0ac93f640 1 -- 172.21.15.60:0/1697365685 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c032810 con 0x7fd0a80cadb0 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 -- 172.21.15.60:0/1697365685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 msgr2=0x7fd0a80cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1697365685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80cf6c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009870 tx=0x7fd09c02a210 comp rx=0 tx=0).stop 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 -- 172.21.15.60:0/1697365685 shutdown_connections 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1697365685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80cf6c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1697365685 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a80cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 -- 172.21.15.60:0/1697365685 >> 172.21.15.60:0/1697365685 conn(0x7fd0a80b2390 msgr2=0x7fd0a80b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 -- 172.21.15.60:0/1697365685 shutdown_connections 2024-04-25T08:38:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.983+0000 7fd0afba5640 1 -- 172.21.15.60:0/1697365685 wait complete. 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 Processor -- start 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 -- start start 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a8159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a815de80 0x7fd0a815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a815e260 con 0x7fd0a80cadb0 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80cfc00 con 0x7fd0a815de80 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0afba5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80b2eb0 con 0x7fd0a80ccfe0 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0ad941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0ad941640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56054/0 (socket says 172.21.15.60:56054) 2024-04-25T08:38:00.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.985+0000 7fd0ad941640 1 -- 172.21.15.60:0/1591639849 learned_addr learned my addr 172.21.15.60:0/1591639849 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ae142640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a815de80 0x7fd0a815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad140640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a8159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad941640 1 -- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 msgr2=0x7fd0a8159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad941640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a8159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad941640 1 -- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a815de80 msgr2=0x7fd0a815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad941640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a815de80 0x7fd0a815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad941640 1 -- 172.21.15.60:0/1591639849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a8158fa0 con 0x7fd0a80cadb0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0ad941640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80c9830 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000c460 tx=0x7fd0a000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0016020 con 0x7fd0a80cadb0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a0012870 con 0x7fd0a80cadb0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a815b150 con 0x7fd0a80cadb0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0012c60 con 0x7fd0a80cadb0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.986+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a81594f0 con 0x7fd0a80cadb0 2024-04-25T08:38:00.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.987+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a8073a00 con 0x7fd0a80cadb0 2024-04-25T08:38:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.987+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0a0019020 con 0x7fd0a80cadb0 2024-04-25T08:38:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.988+0000 7fd0a67fc640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd09406b4d0 0x7fd09406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.988+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0a0089e10 con 0x7fd0a80cadb0 2024-04-25T08:38:00.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.988+0000 7fd0ad140640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd09406b4d0 0x7fd09406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:00.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.989+0000 7fd0ad140640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd09406b4d0 0x7fd09406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd09c034000 tx=0x7fd09c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:00.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:00.992+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0a0020070 con 0x7fd0a80cadb0 2024-04-25T08:38:01.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.173+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0a80b63b0 con 0x7fd0a80cadb0 2024-04-25T08:38:01.167 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.173+0000 7fd0a67fc640 1 -- 172.21.15.60:0/1591639849 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0a0058b00 con 0x7fd0a80cadb0 2024-04-25T08:38:01.168 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd09406b4d0 msgr2=0x7fd09406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd09406b4d0 0x7fd09406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd09c034000 tx=0x7fd09c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 msgr2=0x7fd0a80c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80c9830 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000c460 tx=0x7fd0a000c920 comp rx=0 tx=0).stop 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 shutdown_connections 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd09406b4d0 0x7fd09406d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0a815de80 0x7fd0a815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd0a80ccfe0 0x7fd0a8159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 --2- 172.21.15.60:0/1591639849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0a80cadb0 0x7fd0a80c9830 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 >> 172.21.15.60:0/1591639849 conn(0x7fd0a80b2390 msgr2=0x7fd0a815ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:01.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 shutdown_connections 2024-04-25T08:38:01.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.176+0000 7fd0afba5640 1 -- 172.21.15.60:0/1591639849 wait complete. 2024-04-25T08:38:01.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:38:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.288+0000 7ff24f5e7640 1 Processor -- start 2024-04-25T08:38:01.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 -- start start 2024-04-25T08:38:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2481610f0 0x7ff24806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2481616f0 0x7ff248161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2480ce0a0 0x7ff2480ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2481614d0 con 0x7ff2481610f0 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2480c6960 con 0x7ff2481616f0 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24f5e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff248162260 con 0x7ff2480ce0a0 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.289+0000 7ff24d383640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2481610f0 0x7ff24806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2481610f0 0x7ff24806c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56060/0 (socket says 172.21.15.60:56060) 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 -- 172.21.15.60:0/1159484294 learned_addr learned my addr 172.21.15.60:0/1159484294 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 -- 172.21.15.60:0/1159484294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2480ce0a0 msgr2=0x7ff2480ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 --2- 172.21.15.60:0/1159484294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2480ce0a0 0x7ff2480ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 -- 172.21.15.60:0/1159484294 >> v2:172.21.15.100:3300/0 conn(0x7ff2481616f0 msgr2=0x7ff248161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24cb82640 1 --2- 172.21.15.60:0/1159484294 >> v2:172.21.15.100:3300/0 conn(0x7ff2481616f0 0x7ff248161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 --2- 172.21.15.60:0/1159484294 >> v2:172.21.15.100:3300/0 conn(0x7ff2481616f0 0x7ff248161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24d383640 1 -- 172.21.15.60:0/1159484294 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff248162540 con 0x7ff2481610f0 2024-04-25T08:38:01.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.290+0000 7ff24cb82640 1 --2- 172.21.15.60:0/1159484294 >> v2:172.21.15.100:3300/0 conn(0x7ff2481616f0 0x7ff248161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24d383640 1 --2- 172.21.15.60:0/1159484294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481610f0 0x7ff24806c1c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff24806f220 tx=0x7ff24002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7eab1b042d0721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff247fff640 1 -- 172.21.15.60:0/1159484294 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240037070 con 0x7ff2481610f0 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff247fff640 1 -- 172.21.15.60:0/1159484294 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff240032440 con 0x7ff2481610f0 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff247fff640 1 -- 172.21.15.60:0/1159484294 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240032810 con 0x7ff2481610f0 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 -- 172.21.15.60:0/1159484294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481610f0 msgr2=0x7ff24806c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/1159484294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481610f0 0x7ff24806c1c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff24806f220 tx=0x7ff24002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 -- 172.21.15.60:0/1159484294 shutdown_connections 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/1159484294 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2480ce0a0 0x7ff2480ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/1159484294 >> v2:172.21.15.100:3300/0 conn(0x7ff2481616f0 0x7ff248161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/1159484294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481610f0 0x7ff24806c1c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 -- 172.21.15.60:0/1159484294 >> 172.21.15.60:0/1159484294 conn(0x7ff24806f500 msgr2=0x7ff24806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:01.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 -- 172.21.15.60:0/1159484294 shutdown_connections 2024-04-25T08:38:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.291+0000 7ff24f5e7640 1 -- 172.21.15.60:0/1159484294 wait complete. 2024-04-25T08:38:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.292+0000 7ff24f5e7640 1 Processor -- start 2024-04-25T08:38:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 -- start start 2024-04-25T08:38:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2480ce0a0 0x7ff2480aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2481610f0 0x7ff2480ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 0x7ff24814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480cf690 con 0x7ff2481616f0 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2480cfce0 con 0x7ff2480ce0a0 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24f5e7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24806c700 con 0x7ff2481610f0 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24db84640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 0x7ff24814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24db84640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 0x7ff24814af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:56062/0 (socket says 172.21.15.60:56062) 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.293+0000 7ff24db84640 1 -- 172.21.15.60:0/417799079 learned_addr learned my addr 172.21.15.60:0/417799079 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24cb82640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2481610f0 0x7ff2480ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24db84640 1 -- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2481610f0 msgr2=0x7ff2480ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24db84640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2481610f0 0x7ff2480ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24db84640 1 -- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2480ce0a0 msgr2=0x7ff2480aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24d383640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2480ce0a0 0x7ff2480aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:01.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24db84640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2480ce0a0 0x7ff2480aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24db84640 1 -- 172.21.15.60:0/417799079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480ad300 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24db84640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 0x7ff24814af70 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff23800b970 tx=0x7ff23800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238019070 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23800cb60 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2480ad5e0 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238014440 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.294+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2480adba0 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.295+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff248073a00 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.295+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff23801c070 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.296+0000 7ff245ffb640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff23406b500 0x7ff23406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.296+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff23808a0c0 con 0x7ff2481616f0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.296+0000 7ff24d383640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff23406b500 0x7ff23406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.297+0000 7ff24d383640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff23406b500 0x7ff23406d9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff240034000 tx=0x7ff24002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.300+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff238058db0 con 0x7ff2481616f0 2024-04-25T08:38:01.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.493+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff2480cf150 con 0x7ff2481616f0 2024-04-25T08:38:01.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.493+0000 7ff245ffb640 1 -- 172.21.15.60:0/417799079 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff2380586b0 con 0x7ff2481616f0 2024-04-25T08:38:01.487 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:38:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff23406b500 msgr2=0x7ff23406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff23406b500 0x7ff23406d9c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff240034000 tx=0x7ff24002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 msgr2=0x7ff24814af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 0x7ff24814af70 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff23800b970 tx=0x7ff23800be30 comp rx=0 tx=0).stop 2024-04-25T08:38:01.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 shutdown_connections 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2481616f0 0x7ff24814af70 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2481610f0 0x7ff2480ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff23406b500 0x7ff23406d9c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 --2- 172.21.15.60:0/417799079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2480ce0a0 0x7ff2480aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 >> 172.21.15.60:0/417799079 conn(0x7ff24806f500 msgr2=0x7ff2480d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 shutdown_connections 2024-04-25T08:38:01.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:01.495+0000 7ff24f5e7640 1 -- 172.21.15.60:0/417799079 wait complete. 2024-04-25T08:38:01.500 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:38:01.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:01.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:02.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:38:02.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:02.122 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:38:02.138 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:38:02.138 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:02.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:02.153 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:38:02.153 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:02.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:02.193 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:38:02.193 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:02.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:02.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.339+0000 7f05c1c38640 1 Processor -- start 2024-04-25T08:38:02.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.340+0000 7f05c1c38640 1 -- start start 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.340+0000 7f05c1c38640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05bc0b2cb0 0x7f05bc0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.340+0000 7f05c1c38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.340+0000 7f05c1c38640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0b97a0 con 0x7f05bc15dbe0 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.340+0000 7f05c1c38640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc15dfc0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.340+0000 7f05c1c38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0d1a00 con 0x7f05bc0d1020 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05bbfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05bc0b2cb0 0x7f05bc0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f05bc0b2cb0 0x7f05bc0d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:59032/0 (socket says 172.21.15.60:59032) 2024-04-25T08:38:02.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05bbfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:39892/0 (socket says 172.21.15.60:39892) 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05bb7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f05bc15dbe0 0x7f05bc0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:42902/0 (socket says 172.21.15.60:42902) 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 -- 172.21.15.60:0/2006729292 learned_addr learned my addr 172.21.15.60:0/2006729292 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 -- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 msgr2=0x7f05bc0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 --2- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 -- 172.21.15.60:0/2006729292 >> v1:172.21.15.100:6789/0 conn(0x7f05bc15dbe0 legacy=0x7f05bc0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05ba7fc640 1 -- 172.21.15.60:0/2006729292 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134250774 0 0) 0x7f05bc0b97a0 con 0x7f05bc15dbe0 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05baffd640 1 -- 172.21.15.60:0/2006729292 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0d1c50 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.341+0000 7f05bbfff640 1 --2- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.342+0000 7f05baffd640 1 --2- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0d2730 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009870 tx=0x7f05b002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ecf9fb306bfa42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.342+0000 7f05ba7fc640 1 -- 172.21.15.60:0/2006729292 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0037070 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.342+0000 7f05ba7fc640 1 -- 172.21.15.60:0/2006729292 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0032440 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.342+0000 7f05ba7fc640 1 -- 172.21.15.60:0/2006729292 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0032810 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 -- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 msgr2=0x7f05bc0d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 --2- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0d2730 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009870 tx=0x7f05b002a270 comp rx=0 tx=0).stop 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 -- 172.21.15.60:0/2006729292 shutdown_connections 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 --2- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0d2730 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 --2- 172.21.15.60:0/2006729292 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 -- 172.21.15.60:0/2006729292 >> 172.21.15.60:0/2006729292 conn(0x7f05bc061a20 msgr2=0x7f05bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:02.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 -- 172.21.15.60:0/2006729292 shutdown_connections 2024-04-25T08:38:02.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.343+0000 7f05c1c38640 1 -- 172.21.15.60:0/2006729292 wait complete. 2024-04-25T08:38:02.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.344+0000 7f05c1c38640 1 Processor -- start 2024-04-25T08:38:02.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.344+0000 7f05c1c38640 1 -- start start 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05c1c38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05c1c38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05c1c38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 0x7f05bc146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05c1c38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b92a0 con 0x7f05bc15dbe0 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05c1c38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0d2210 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05c1c38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b9420 con 0x7f05bc0d1020 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 0x7f05bc146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05baffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bbfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 0x7f05bc146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58130/0 (socket says 172.21.15.60:58130) 2024-04-25T08:38:02.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bbfff640 1 -- 172.21.15.60:0/1945171540 learned_addr learned my addr 172.21.15.60:0/1945171540 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 -- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 msgr2=0x7f05bc0b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 -- 172.21.15.60:0/1945171540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 msgr2=0x7f05bc146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 0x7f05bc146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 -- 172.21.15.60:0/1945171540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0cae30 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bbfff640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 0x7f05bc146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.345+0000 7f05bb7fe640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0cbed0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c410 tx=0x7f05ac00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.346+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0074e0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.346+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0047e0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.346+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac004bd0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.346+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc0cb110 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.346+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0cb6d0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.347+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc073990 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.347+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05ac011070 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.347+0000 7f05a3fff640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05a806b4d0 0x7f05a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.348+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05ac089eb0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.348+0000 7f05baffd640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05a806b4d0 0x7f05a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.348+0000 7f05baffd640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05a806b4d0 0x7f05a806d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f05b0034000 tx=0x7f05b002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:02.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.352+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05ac058570 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.575+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05bc1553f0 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.575+0000 7f05a3fff640 1 -- 172.21.15.60:0/1945171540 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f05ac05c610 con 0x7f05bc0b2cb0 2024-04-25T08:38:02.571 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:02.571 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05a806b4d0 msgr2=0x7f05a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05a806b4d0 0x7f05a806d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f05b0034000 tx=0x7f05b002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 msgr2=0x7f05bc0cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0cbed0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c410 tx=0x7f05ac00c8d0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 shutdown_connections 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f05bc15dbe0 0x7f05bc146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f05bc0d1020 0x7f05bc0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f05a806b4d0 0x7f05a806d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 --2- 172.21.15.60:0/1945171540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05bc0b2cb0 0x7f05bc0cbed0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 >> 172.21.15.60:0/1945171540 conn(0x7f05bc061a20 msgr2=0x7f05bc0b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 shutdown_connections 2024-04-25T08:38:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.577+0000 7f05c1c38640 1 -- 172.21.15.60:0/1945171540 wait complete. 2024-04-25T08:38:02.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:02.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.690+0000 7feecb6e1640 1 Processor -- start 2024-04-25T08:38:02.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.690+0000 7feecb6e1640 1 -- start start 2024-04-25T08:38:02.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feecb6e1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feec40ce590 0x7feec4165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feecb6e1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4071480 con 0x7feec4154ee0 2024-04-25T08:38:02.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feecb6e1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4071760 con 0x7feec40cfec0 2024-04-25T08:38:02.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feecb6e1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec4072090 con 0x7feec40ce590 2024-04-25T08:38:02.685 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec9c7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feec40ce590 0x7feec4165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec947d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7feec4154ee0 0x7feec40cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50638/0 (socket says 172.21.15.60:50638) 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec9c7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7feec40ce590 0x7feec4165900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58132/0 (socket says 172.21.15.60:58132) 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec947d640 1 -- 172.21.15.60:0/2598338825 learned_addr learned my addr 172.21.15.60:0/2598338825 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec9c7e640 1 -- 172.21.15.60:0/2598338825 >> v1:172.21.15.100:6789/0 conn(0x7feec40cfec0 legacy=0x7feec40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec9c7e640 1 -- 172.21.15.60:0/2598338825 >> v1:172.21.15.60:6789/0 conn(0x7feec4154ee0 legacy=0x7feec40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feec9c7e640 1 -- 172.21.15.60:0/2598338825 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4071a40 con 0x7feec40ce590 2024-04-25T08:38:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.691+0000 7feebbfff640 1 -- 172.21.15.60:0/2598338825 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474673433 0 0) 0x7feec4071480 con 0x7feec4154ee0 2024-04-25T08:38:02.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.692+0000 7feec9c7e640 1 --2- 172.21.15.60:0/2598338825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4165900 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7feec0009d90 tx=0x7feec002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba151426937d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:02.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.692+0000 7feebbfff640 1 -- 172.21.15.60:0/2598338825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0007740 con 0x7feec40ce590 2024-04-25T08:38:02.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.692+0000 7feebbfff640 1 -- 172.21.15.60:0/2598338825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec00078e0 con 0x7feec40ce590 2024-04-25T08:38:02.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.692+0000 7feebbfff640 1 -- 172.21.15.60:0/2598338825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0007cb0 con 0x7feec40ce590 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 -- 172.21.15.60:0/2598338825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 msgr2=0x7feec4165900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 --2- 172.21.15.60:0/2598338825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4165900 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7feec0009d90 tx=0x7feec002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 -- 172.21.15.60:0/2598338825 shutdown_connections 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 --2- 172.21.15.60:0/2598338825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4165900 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 -- 172.21.15.60:0/2598338825 >> 172.21.15.60:0/2598338825 conn(0x7feec406abe0 msgr2=0x7feec406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 -- 172.21.15.60:0/2598338825 shutdown_connections 2024-04-25T08:38:02.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.693+0000 7feecb6e1640 1 -- 172.21.15.60:0/2598338825 wait complete. 2024-04-25T08:38:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.694+0000 7feecb6e1640 1 Processor -- start 2024-04-25T08:38:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.694+0000 7feecb6e1640 1 -- start start 2024-04-25T08:38:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feecb6e1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feecb6e1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feec40cfec0 0x7feec415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feecb6e1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4154ee0 0x7feec40b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feecb6e1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec41552c0 con 0x7feec40ce590 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feecb6e1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4071230 con 0x7feec4154ee0 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feecb6e1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec4155440 con 0x7feec40cfec0 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58136/0 (socket says 172.21.15.60:58136) 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 -- 172.21.15.60:0/3226287369 learned_addr learned my addr 172.21.15.60:0/3226287369 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec9c7e640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4154ee0 0x7feec40b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec8c7c640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feec40cfec0 0x7feec415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 -- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feec40cfec0 msgr2=0x7feec415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feec40cfec0 0x7feec415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 -- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4154ee0 msgr2=0x7feec40b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4154ee0 0x7feec40b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 -- 172.21.15.60:0/3226287369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec41589c0 con 0x7feec40ce590 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.695+0000 7feec947d640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4145ed0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c3f0 tx=0x7feebc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.696+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0074e0 con 0x7feec40ce590 2024-04-25T08:38:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.696+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc0047e0 con 0x7feec40ce590 2024-04-25T08:38:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.696+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec4158ca0 con 0x7feec40ce590 2024-04-25T08:38:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.696+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc004c10 con 0x7feec40ce590 2024-04-25T08:38:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.696+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec4159260 con 0x7feec40ce590 2024-04-25T08:38:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.697+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4073990 con 0x7feec40ce590 2024-04-25T08:38:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.697+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feebc011070 con 0x7feec40ce590 2024-04-25T08:38:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.697+0000 7feeb9ffb640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feea806b4d0 0x7feea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.698+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feebc089e30 con 0x7feec40ce590 2024-04-25T08:38:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.698+0000 7feec8c7c640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feea806b4d0 0x7feea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.698+0000 7feec8c7c640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feea806b4d0 0x7feea806d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7feec415b5f0 tx=0x7feeb4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.702+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feebc0584d0 con 0x7feec40ce590 2024-04-25T08:38:02.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.882+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feec40b3770 con 0x7feec40ce590 2024-04-25T08:38:02.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.883+0000 7feeb9ffb640 1 -- 172.21.15.60:0/3226287369 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16940 (secure 0 0 0) 0x7feebc05c570 con 0x7feec40ce590 2024-04-25T08:38:02.877 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:02.878 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:30.801169+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feea806b4d0 msgr2=0x7feea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feea806b4d0 0x7feea806d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7feec415b5f0 tx=0x7feeb4009040 comp rx=0 tx=0).stop 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 msgr2=0x7feec4145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4145ed0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c3f0 tx=0x7feebc00c8b0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 shutdown_connections 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7feea806b4d0 0x7feea806d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feec4154ee0 0x7feec40b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7feec40cfec0 0x7feec415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 --2- 172.21.15.60:0/3226287369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feec40ce590 0x7feec4145ed0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 >> 172.21.15.60:0/3226287369 conn(0x7feec406abe0 msgr2=0x7feec406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 shutdown_connections 2024-04-25T08:38:02.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:02.885+0000 7feecb6e1640 1 -- 172.21.15.60:0/3226287369 wait complete. 2024-04-25T08:38:02.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:37:27.446495+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:37:28.138802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:02.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:02.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 Processor -- start 2024-04-25T08:38:02.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 -- start start 2024-04-25T08:38:03.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a9415dc20 0x7f3a940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a94161710 0x7f3a940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9415e000 con 0x7f3a940b9330 2024-04-25T08:38:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940d1de0 con 0x7f3a9415dc20 2024-04-25T08:38:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.001+0000 7f3a9aa5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a94161c60 con 0x7f3a94161710 2024-04-25T08:38:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a9415dc20 0x7f3a940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3a9415dc20 0x7f3a940ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58150/0 (socket says 172.21.15.60:58150) 2024-04-25T08:38:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a98ffa640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3a940b9330 0x7f3a9406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50648/0 (socket says 172.21.15.60:50648) 2024-04-25T08:38:03.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 -- 172.21.15.60:0/3466419271 learned_addr learned my addr 172.21.15.60:0/3466419271 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a937fe640 1 --2- 172.21.15.60:0/3466419271 >> v2:172.21.15.100:3300/0 conn(0x7f3a94161710 0x7f3a940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 -- 172.21.15.60:0/3466419271 >> v2:172.21.15.100:3300/0 conn(0x7f3a94161710 msgr2=0x7f3a940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 --2- 172.21.15.60:0/3466419271 >> v2:172.21.15.100:3300/0 conn(0x7f3a94161710 0x7f3a940cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 -- 172.21.15.60:0/3466419271 >> v1:172.21.15.60:6789/0 conn(0x7f3a940b9330 legacy=0x7f3a9406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.002+0000 7f3a93fff640 1 -- 172.21.15.60:0/3466419271 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94161f40 con 0x7f3a9415dc20 2024-04-25T08:38:03.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a93fff640 1 --2- 172.21.15.60:0/3466419271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9415dc20 0x7f3a940ce670 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3a9406da90 tx=0x7f3a8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc61445aba3277c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a92ffd640 1 -- 172.21.15.60:0/3466419271 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84037070 con 0x7f3a9415dc20 2024-04-25T08:38:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a92ffd640 1 -- 172.21.15.60:0/3466419271 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84032440 con 0x7f3a9415dc20 2024-04-25T08:38:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a92ffd640 1 -- 172.21.15.60:0/3466419271 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84032810 con 0x7f3a9415dc20 2024-04-25T08:38:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/3466419271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9415dc20 msgr2=0x7f3a940ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/3466419271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9415dc20 0x7f3a940ce670 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3a9406da90 tx=0x7f3a8402a270 comp rx=0 tx=0).stop 2024-04-25T08:38:03.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/3466419271 shutdown_connections 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/3466419271 >> v2:172.21.15.100:3300/0 conn(0x7f3a94161710 0x7f3a940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/3466419271 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9415dc20 0x7f3a940ce670 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/3466419271 >> 172.21.15.60:0/3466419271 conn(0x7f3a940d0420 msgr2=0x7f3a940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/3466419271 shutdown_connections 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.003+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/3466419271 wait complete. 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 Processor -- start 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 -- start start 2024-04-25T08:38:03.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 0x7f3a9415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9415dc20 0x7f3a940b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a94161710 0x7f3a9415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940b5e50 con 0x7f3a940b9330 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a94162500 con 0x7f3a9415dc20 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.005+0000 7f3a9aa5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940b5fd0 con 0x7f3a94161710 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 0x7f3a9415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 0x7f3a9415d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58160/0 (socket says 172.21.15.60:58160) 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 -- 172.21.15.60:0/1702841508 learned_addr learned my addr 172.21.15.60:0/1702841508 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a98ffa640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a94161710 0x7f3a9415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a937fe640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9415dc20 0x7f3a940b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 -- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a94161710 msgr2=0x7f3a9415c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a94161710 0x7f3a9415c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 -- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9415dc20 msgr2=0x7f3a940b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9415dc20 0x7f3a940b8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 -- 172.21.15.60:0/1702841508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9415ce80 con 0x7f3a940b9330 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a937fe640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9415dc20 0x7f3a940b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a98ffa640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a94161710 0x7f3a9415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a93fff640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 0x7f3a9415d0d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3a8402a2b0 tx=0x7f3a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84037040 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84042070 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9415eec0 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84030bb0 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.006+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a941e86e0 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.007+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9415f3e0 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.008+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a84047460 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.008+0000 7f3a90ff9640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.008+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a840ab030 con 0x7f3a940b9330 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.008+0000 7f3a937fe640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.009+0000 7f3a937fe640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3a8006b4d0 0x7f3a8006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3a940b5d30 tx=0x7f3a88008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.012+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a84045360 con 0x7f3a940b9330 2024-04-25T08:38:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.191+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a94073a00 con 0x7f3a940b9330 2024-04-25T08:38:03.185 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.191+0000 7f3a90ff9640 1 -- 172.21.15.60:0/1702841508 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a84079d20 con 0x7f3a940b9330 2024-04-25T08:38:03.185 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3a8006b4d0 msgr2=0x7f3a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3a8006b4d0 0x7f3a8006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3a940b5d30 tx=0x7f3a88008040 comp rx=0 tx=0).stop 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 msgr2=0x7f3a9415d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 0x7f3a9415d0d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3a8402a2b0 tx=0x7f3a840046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 shutdown_connections 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3a94161710 0x7f3a9415c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a9415dc20 0x7f3a940b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 --2- 172.21.15.60:0/1702841508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a940b9330 0x7f3a9415d0d0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.193+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 >> 172.21.15.60:0/1702841508 conn(0x7f3a940d0420 msgr2=0x7f3a940cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.194+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 shutdown_connections 2024-04-25T08:38:03.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.194+0000 7f3a9aa5d640 1 -- 172.21.15.60:0/1702841508 wait complete. 2024-04-25T08:38:03.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:38:03.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.309+0000 7fa0cdf3b640 1 Processor -- start 2024-04-25T08:38:03.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.309+0000 7fa0cdf3b640 1 -- start start 2024-04-25T08:38:03.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0cdf3b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c815dc20 0x7fa0c80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0cdf3b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c806cda0 con 0x7fa0c80d32f0 2024-04-25T08:38:03.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0cdf3b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c815e000 con 0x7fa0c8161710 2024-04-25T08:38:03.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0cdf3b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c80d1de0 con 0x7fa0c815dc20 2024-04-25T08:38:03.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0c77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c815dc20 0x7fa0c80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0c7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa0c80d32f0 0x7fa0c806c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50654/0 (socket says 172.21.15.60:50654) 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0c7fff640 1 -- 172.21.15.60:0/131408983 learned_addr learned my addr 172.21.15.60:0/131408983 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0c67fc640 1 -- 172.21.15.60:0/131408983 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106900882 0 0) 0x7fa0c806cda0 con 0x7fa0c80d32f0 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.310+0000 7fa0c67fc640 1 -- 172.21.15.60:0/131408983 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b0003520 con 0x7fa0c80d32f0 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.311+0000 7fa0c77fe640 1 -- 172.21.15.60:0/131408983 >> v1:172.21.15.100:6789/0 conn(0x7fa0c8161710 legacy=0x7fa0c806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.311+0000 7fa0c77fe640 1 -- 172.21.15.60:0/131408983 >> v1:172.21.15.60:6789/0 conn(0x7fa0c80d32f0 legacy=0x7fa0c806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.311+0000 7fa0c77fe640 1 -- 172.21.15.60:0/131408983 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c8161c60 con 0x7fa0c815dc20 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.311+0000 7fa0c77fe640 1 --2- 172.21.15.60:0/131408983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c815dc20 0x7fa0c80ce670 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80097a0 tx=0x7fa0b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7884636c95640f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.311+0000 7fa0c67fc640 1 -- 172.21.15.60:0/131408983 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8037070 con 0x7fa0c815dc20 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.311+0000 7fa0c67fc640 1 -- 172.21.15.60:0/131408983 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8032440 con 0x7fa0c815dc20 2024-04-25T08:38:03.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0c67fc640 1 -- 172.21.15.60:0/131408983 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80327f0 con 0x7fa0c815dc20 2024-04-25T08:38:03.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/131408983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c815dc20 msgr2=0x7fa0c80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/131408983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c815dc20 0x7fa0c80ce670 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80097a0 tx=0x7fa0b802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/131408983 shutdown_connections 2024-04-25T08:38:03.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/131408983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c815dc20 0x7fa0c80ce670 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/131408983 >> 172.21.15.60:0/131408983 conn(0x7fa0c80d0420 msgr2=0x7fa0c80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/131408983 shutdown_connections 2024-04-25T08:38:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.312+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/131408983 wait complete. 2024-04-25T08:38:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.313+0000 7fa0cdf3b640 1 Processor -- start 2024-04-25T08:38:03.308 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.313+0000 7fa0cdf3b640 1 -- start start 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0cdf3b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c80d32f0 0x7fa0c80aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0cdf3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c815dc20 0x7fa0c8152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0cdf3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 0x7fa0c80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0cdf3b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8162500 con 0x7fa0c8161710 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0cdf3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8162220 con 0x7fa0c815dc20 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0cdf3b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c80b5e50 con 0x7fa0c80d32f0 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 0x7fa0c80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 0x7fa0c80ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58184/0 (socket says 172.21.15.60:58184) 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0c7fff640 1 -- 172.21.15.60:0/249467725 learned_addr learned my addr 172.21.15.60:0/249467725 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:03.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0c77fe640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c80d32f0 0x7fa0c80aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.314+0000 7fa0c6ffd640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c815dc20 0x7fa0c8152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c7fff640 1 -- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c80d32f0 msgr2=0x7fa0c80aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c7fff640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c80d32f0 0x7fa0c80aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c7fff640 1 -- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c815dc20 msgr2=0x7fa0c8152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c7fff640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c815dc20 0x7fa0c8152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c7fff640 1 -- 172.21.15.60:0/249467725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80ac080 con 0x7fa0c8161710 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c77fe640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c80d32f0 0x7fa0c80aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0c7fff640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 0x7fa0c80ac860 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c9c0 tx=0x7fa0c000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0017070 con 0x7fa0c8161710 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c000ab70 con 0x7fa0c8161710 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c81532e0 con 0x7fa0c8161710 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0012400 con 0x7fa0c8161710 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.315+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c80cc850 con 0x7fa0c8161710 2024-04-25T08:38:03.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.316+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c8073a00 con 0x7fa0c8161710 2024-04-25T08:38:03.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.317+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0c001a070 con 0x7fa0c8161710 2024-04-25T08:38:03.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.317+0000 7fa0affff640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0b406b4d0 0x7fa0b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.317+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0c0089ed0 con 0x7fa0c8161710 2024-04-25T08:38:03.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.317+0000 7fa0c77fe640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0b406b4d0 0x7fa0b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.318+0000 7fa0c77fe640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0b406b4d0 0x7fa0b406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8034000 tx=0x7fa0b802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.321+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0c0058580 con 0x7fa0c8161710 2024-04-25T08:38:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.502+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0c80ca790 con 0x7fa0c8161710 2024-04-25T08:38:03.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.503+0000 7fa0affff640 1 -- 172.21.15.60:0/249467725 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0c005c620 con 0x7fa0c8161710 2024-04-25T08:38:03.497 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0b406b4d0 msgr2=0x7fa0b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0b406b4d0 0x7fa0b406d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8034000 tx=0x7fa0b802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 msgr2=0x7fa0c80ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 0x7fa0c80ac860 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c9c0 tx=0x7fa0c000ce80 comp rx=0 tx=0).stop 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 shutdown_connections 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c8161710 0x7fa0c80ac860 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0b406b4d0 0x7fa0b406d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c815dc20 0x7fa0c8152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 --2- 172.21.15.60:0/249467725 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c80d32f0 0x7fa0c80aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 >> 172.21.15.60:0/249467725 conn(0x7fa0c80d0420 msgr2=0x7fa0c80c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 shutdown_connections 2024-04-25T08:38:03.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.505+0000 7fa0cdf3b640 1 -- 172.21.15.60:0/249467725 wait complete. 2024-04-25T08:38:03.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:38:03.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.621+0000 7ff147bf9640 1 Processor -- start 2024-04-25T08:38:03.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.621+0000 7ff147bf9640 1 -- start start 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.621+0000 7ff147bf9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff14015dac0 0x7ff1400d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff147bf9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1400d2420 con 0x7ff1400d0ab0 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff147bf9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1400d2700 con 0x7ff1400d1f30 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff147bf9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1400d3030 con 0x7ff14015dac0 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff145995640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff14015dac0 0x7ff1400d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff146196640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff1400d0ab0 0x7ff1400b6990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50664/0 (socket says 172.21.15.60:50664) 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff145995640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff14015dac0 0x7ff1400d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58198/0 (socket says 172.21.15.60:58198) 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff146196640 1 -- 172.21.15.60:0/1177932626 learned_addr learned my addr 172.21.15.60:0/1177932626 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:03.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff145995640 1 -- 172.21.15.60:0/1177932626 >> v1:172.21.15.100:6789/0 conn(0x7ff1400d1f30 legacy=0x7ff1400d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff145995640 1 -- 172.21.15.60:0/1177932626 >> v1:172.21.15.60:6789/0 conn(0x7ff1400d0ab0 legacy=0x7ff1400b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff145995640 1 -- 172.21.15.60:0/1177932626 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400d29e0 con 0x7ff14015dac0 2024-04-25T08:38:03.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.622+0000 7ff144993640 1 -- 172.21.15.60:0/1177932626 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196080462 0 0) 0x7ff1400d2420 con 0x7ff1400d0ab0 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff145995640 1 --2- 172.21.15.60:0/1177932626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400d19f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0097a0 tx=0x7ff13c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0032d7d0171224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff144993640 1 -- 172.21.15.60:0/1177932626 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c037070 con 0x7ff14015dac0 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff144993640 1 -- 172.21.15.60:0/1177932626 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c032440 con 0x7ff14015dac0 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff144993640 1 -- 172.21.15.60:0/1177932626 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c032810 con 0x7ff14015dac0 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff147bf9640 1 -- 172.21.15.60:0/1177932626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 msgr2=0x7ff1400d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff147bf9640 1 --2- 172.21.15.60:0/1177932626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400d19f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0097a0 tx=0x7ff13c02a270 comp rx=0 tx=0).stop 2024-04-25T08:38:03.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff147bf9640 1 -- 172.21.15.60:0/1177932626 shutdown_connections 2024-04-25T08:38:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff147bf9640 1 --2- 172.21.15.60:0/1177932626 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400d19f0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.623+0000 7ff147bf9640 1 -- 172.21.15.60:0/1177932626 >> 172.21.15.60:0/1177932626 conn(0x7ff140061c30 msgr2=0x7ff14006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.624+0000 7ff147bf9640 1 -- 172.21.15.60:0/1177932626 shutdown_connections 2024-04-25T08:38:03.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.624+0000 7ff147bf9640 1 -- 172.21.15.60:0/1177932626 wait complete. 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 Processor -- start 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 -- start start 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400d0ab0 0x7ff14015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1400d1f30 0x7ff14015ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1400b97d0 con 0x7ff14015dac0 2024-04-25T08:38:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14015dea0 con 0x7ff1400d0ab0 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.625+0000 7ff147bf9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1400b9950 con 0x7ff1400d1f30 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58202/0 (socket says 172.21.15.60:58202) 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 -- 172.21.15.60:0/3896173284 learned_addr learned my addr 172.21.15.60:0/3896173284 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 -- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1400d1f30 msgr2=0x7ff14015ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1400d1f30 0x7ff14015ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff145995640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400d0ab0 0x7ff14015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 -- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400d0ab0 msgr2=0x7ff14015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400d0ab0 0x7ff14015ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 -- 172.21.15.60:0/3896173284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14015b180 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff146196640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400b7800 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff13800c7a0 tx=0x7ff13800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138018070 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13800a640 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1401601a0 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800aab0 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.626+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff140160760 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.627+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff140073a00 con 0x7ff14015dac0 2024-04-25T08:38:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.628+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff13801b070 con 0x7ff14015dac0 2024-04-25T08:38:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.628+0000 7ff1367fc640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff12406b4d0 0x7ff12406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.628+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1380899b0 con 0x7ff14015dac0 2024-04-25T08:38:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.628+0000 7ff145995640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff12406b4d0 0x7ff12406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.629+0000 7ff145995640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff12406b4d0 0x7ff12406d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff13c034000 tx=0x7ff13c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.632+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1380586a0 con 0x7ff14015dac0 2024-04-25T08:38:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.812+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff14006b210 con 0x7ff14015dac0 2024-04-25T08:38:03.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.813+0000 7ff1367fc640 1 -- 172.21.15.60:0/3896173284 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff138057fa0 con 0x7ff14015dac0 2024-04-25T08:38:03.807 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff12406b4d0 msgr2=0x7ff12406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff12406b4d0 0x7ff12406d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff13c034000 tx=0x7ff13c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 msgr2=0x7ff1400b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400b7800 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff13800c7a0 tx=0x7ff13800cc60 comp rx=0 tx=0).stop 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 shutdown_connections 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff14015dac0 0x7ff1400b7800 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff1400d1f30 0x7ff14015ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff12406b4d0 0x7ff12406d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 --2- 172.21.15.60:0/3896173284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1400d0ab0 0x7ff14015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 >> 172.21.15.60:0/3896173284 conn(0x7ff140061c30 msgr2=0x7ff1400b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 shutdown_connections 2024-04-25T08:38:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.815+0000 7ff147bf9640 1 -- 172.21.15.60:0/3896173284 wait complete. 2024-04-25T08:38:03.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:03.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.927+0000 7f4fca542640 1 Processor -- start 2024-04-25T08:38:03.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.927+0000 7f4fca542640 1 -- start start 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fca542640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc415dd30 0x7f4fc40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fca542640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc40d1df0 con 0x7f4fc40b9770 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fca542640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc40d20d0 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fca542640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40d23b0 con 0x7f4fc415dd30 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc37fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4fc40b9770 0x7f4fc40d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50670/0 (socket says 172.21.15.60:50670) 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc37fe640 1 -- 172.21.15.60:0/1572345543 learned_addr learned my addr 172.21.15.60:0/1572345543 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc3fff640 1 --2- 172.21.15.60:0/1572345543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc415dd30 0x7f4fc40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275193729 0 0) 0x7f4fc40d1df0 con 0x7f4fc40b9770 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb0003520 con 0x7f4fc40b9770 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724638589 0 0) 0x7f4fc40d20d0 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc40d1df0 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc3fff640 1 -- 172.21.15.60:0/1572345543 >> v1:172.21.15.100:6789/0 conn(0x7f4fc40d0ad0 legacy=0x7f4fc40b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc3fff640 1 -- 172.21.15.60:0/1572345543 >> v1:172.21.15.60:6789/0 conn(0x7f4fc40b9770 legacy=0x7f4fc40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.928+0000 7f4fc3fff640 1 -- 172.21.15.60:0/1572345543 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40d2690 con 0x7f4fc415dd30 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.929+0000 7f4fc3fff640 1 --2- 172.21.15.60:0/1572345543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc415dd30 0x7f4fc40b9230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0097d0 tx=0x7f4fac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5603614017bf3b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.929+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac037070 con 0x7f4fc415dd30 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.929+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac032440 con 0x7f4fc415dd30 2024-04-25T08:38:03.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.929+0000 7f4fc2ffd640 1 -- 172.21.15.60:0/1572345543 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac032810 con 0x7f4fc415dd30 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 -- 172.21.15.60:0/1572345543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc415dd30 msgr2=0x7f4fc40b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 --2- 172.21.15.60:0/1572345543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc415dd30 0x7f4fc40b9230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0097d0 tx=0x7f4fac02a180 comp rx=0 tx=0).stop 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 -- 172.21.15.60:0/1572345543 shutdown_connections 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 --2- 172.21.15.60:0/1572345543 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc415dd30 0x7f4fc40b9230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 -- 172.21.15.60:0/1572345543 >> 172.21.15.60:0/1572345543 conn(0x7f4fc4061c30 msgr2=0x7f4fc406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 -- 172.21.15.60:0/1572345543 shutdown_connections 2024-04-25T08:38:03.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.930+0000 7f4fca542640 1 -- 172.21.15.60:0/1572345543 wait complete. 2024-04-25T08:38:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.931+0000 7f4fca542640 1 Processor -- start 2024-04-25T08:38:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.931+0000 7f4fca542640 1 -- start start 2024-04-25T08:38:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fca542640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc40b9770 0x7f4fc40cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fca542640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 0x7f4fc4158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fca542640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc415dd30 0x7f4fc4158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fca542640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40d1b90 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fca542640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40d1a10 con 0x7f4fc415dd30 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fca542640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40d2e90 con 0x7f4fc40b9770 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 0x7f4fc4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 0x7f4fc4158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58208/0 (socket says 172.21.15.60:58208) 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 -- 172.21.15.60:0/291870206 learned_addr learned my addr 172.21.15.60:0/291870206 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc3fff640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc40b9770 0x7f4fc40cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc8adf640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc415dd30 0x7f4fc4158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 -- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc40b9770 msgr2=0x7f4fc40cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc40b9770 0x7f4fc40cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 -- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc415dd30 msgr2=0x7f4fc4158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc415dd30 0x7f4fc4158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.932+0000 7f4fc37fe640 1 -- 172.21.15.60:0/291870206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40cc300 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.933+0000 7f4fc37fe640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 0x7f4fc4158970 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c4d0 tx=0x7f4fb400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.933+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4017070 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.933+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4007480 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.933+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc40cc5e0 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.933+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40059a0 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.933+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40ccba0 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.934+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc40cd080 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.934+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fb401a070 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.935+0000 7f4fc0ff9640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4fa406b4d0 0x7f4fa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.935+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4fb4089660 con 0x7f4fc40d0ad0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.935+0000 7f4fc3fff640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4fa406b4d0 0x7f4fa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:03.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.935+0000 7f4fc3fff640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4fa406b4d0 0x7f4fa406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4fac034000 tx=0x7f4fac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:03.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:03.939+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fc40cd080 con 0x7f4fc40d0ad0 2024-04-25T08:38:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.158+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc4073990 con 0x7f4fc40d0ad0 2024-04-25T08:38:04.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.158+0000 7f4fc0ff9640 1 -- 172.21.15.60:0/291870206 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4fb4057cf0 con 0x7f4fc40d0ad0 2024-04-25T08:38:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:04.153 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:38:04.153 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4fa406b4d0 msgr2=0x7f4fa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4fa406b4d0 0x7f4fa406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4fac034000 tx=0x7f4fac02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 msgr2=0x7f4fc4158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 0x7f4fc4158970 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400c4d0 tx=0x7f4fb400c990 comp rx=0 tx=0).stop 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 shutdown_connections 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4fa406b4d0 0x7f4fa406d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4fc415dd30 0x7f4fc4158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4fc40d0ad0 0x7f4fc4158970 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 --2- 172.21.15.60:0/291870206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4fc40b9770 0x7f4fc40cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 >> 172.21.15.60:0/291870206 conn(0x7f4fc4061c30 msgr2=0x7f4fc415ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 shutdown_connections 2024-04-25T08:38:04.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.161+0000 7f4fca542640 1 -- 172.21.15.60:0/291870206 wait complete. 2024-04-25T08:38:04.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 Processor -- start 2024-04-25T08:38:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 -- start start 2024-04-25T08:38:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f554015ba00 0x7f554009a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 0x7f554015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400cd710 con 0x7f55401602e0 2024-04-25T08:38:04.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400cd4b0 con 0x7f554015ba00 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.274+0000 7f554624d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5540099b70 con 0x7f554015f5f0 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 0x7f554015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f554015ba00 0x7f554009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 0x7f554015fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:39978/0 (socket says 172.21.15.60:39978) 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 -- 172.21.15.60:0/1520316091 learned_addr learned my addr 172.21.15.60:0/1520316091 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 -- 172.21.15.60:0/1520316091 >> v2:172.21.15.100:3300/0 conn(0x7f554015ba00 msgr2=0x7f554009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 --2- 172.21.15.60:0/1520316091 >> v2:172.21.15.100:3300/0 conn(0x7f554015ba00 0x7f554009a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 -- 172.21.15.60:0/1520316091 >> v1:172.21.15.100:6789/0 conn(0x7f55401602e0 legacy=0x7f55400ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553ffff640 1 -- 172.21.15.60:0/1520316091 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5540099e50 con 0x7f554015f5f0 2024-04-25T08:38:04.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.275+0000 7f553f7fe640 1 --2- 172.21.15.60:0/1520316091 >> v2:172.21.15.100:3300/0 conn(0x7f554015ba00 0x7f554009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.276+0000 7f553ffff640 1 --2- 172.21.15.60:0/1520316091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 0x7f554015fa80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5538009e50 tx=0x7f553802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77b3a121859cf31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.276+0000 7f553e7fc640 1 -- 172.21.15.60:0/1520316091 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538036070 con 0x7f554015f5f0 2024-04-25T08:38:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.276+0000 7f553e7fc640 1 -- 172.21.15.60:0/1520316091 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5538031600 con 0x7f554015f5f0 2024-04-25T08:38:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.276+0000 7f553e7fc640 1 -- 172.21.15.60:0/1520316091 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55380319d0 con 0x7f554015f5f0 2024-04-25T08:38:04.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 -- 172.21.15.60:0/1520316091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 msgr2=0x7f554015fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 --2- 172.21.15.60:0/1520316091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 0x7f554015fa80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5538009e50 tx=0x7f553802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 -- 172.21.15.60:0/1520316091 shutdown_connections 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 --2- 172.21.15.60:0/1520316091 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f554015f5f0 0x7f554015fa80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 --2- 172.21.15.60:0/1520316091 >> v2:172.21.15.100:3300/0 conn(0x7f554015ba00 0x7f554009a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 -- 172.21.15.60:0/1520316091 >> 172.21.15.60:0/1520316091 conn(0x7f55400b3270 msgr2=0x7f55400b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 -- 172.21.15.60:0/1520316091 shutdown_connections 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.277+0000 7f554624d640 1 -- 172.21.15.60:0/1520316091 wait complete. 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.278+0000 7f554624d640 1 Processor -- start 2024-04-25T08:38:04.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.278+0000 7f554624d640 1 -- start start 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f554624d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554015ba00 0x7f554006bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f554624d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 0x7f554006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f554624d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401602e0 0x7f554015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f554624d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554015bde0 con 0x7f554015f5f0 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f554624d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554009ae70 con 0x7f554015ba00 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f554624d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554006c850 con 0x7f55401602e0 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 0x7f554006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 0x7f554006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58220/0 (socket says 172.21.15.60:58220) 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 -- 172.21.15.60:0/1002208968 learned_addr learned my addr 172.21.15.60:0/1002208968 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553f7fe640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554015ba00 0x7f554006bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553ffff640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401602e0 0x7f554015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 -- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401602e0 msgr2=0x7f554015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401602e0 0x7f554015d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 -- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554015ba00 msgr2=0x7f554006bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554015ba00 0x7f554006bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.279+0000 7f553effd640 1 -- 172.21.15.60:0/1002208968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554006cb20 con 0x7f554015f5f0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.280+0000 7f553effd640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 0x7f554006c310 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f553400c430 tx=0x7f553400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.280+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340052d0 con 0x7f554015f5f0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.280+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534005490 con 0x7f554015f5f0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.280+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534007260 con 0x7f554015f5f0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.280+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554006ce00 con 0x7f554015f5f0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.280+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554006d3c0 con 0x7f554015f5f0 2024-04-25T08:38:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.281+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55400b26f0 con 0x7f554015f5f0 2024-04-25T08:38:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.281+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5534007400 con 0x7f554015f5f0 2024-04-25T08:38:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.282+0000 7f5527fff640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f552c06b4d0 0x7f552c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.282+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5534089810 con 0x7f554015f5f0 2024-04-25T08:38:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.282+0000 7f553f7fe640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f552c06b4d0 0x7f552c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.283+0000 7f553f7fe640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f552c06b4d0 0x7f552c06d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5530009560 tx=0x7f5530009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.286+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55400b26f0 con 0x7f554015f5f0 2024-04-25T08:38:04.463 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.469+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5540073990 con 0x7f554015f5f0 2024-04-25T08:38:04.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.470+0000 7f5527fff640 1 -- 172.21.15.60:0/1002208968 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f55400cd710 con 0x7f554015f5f0 2024-04-25T08:38:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:04.464 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:30.801169+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f552c06b4d0 msgr2=0x7f552c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f552c06b4d0 0x7f552c06d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5530009560 tx=0x7f5530009190 comp rx=0 tx=0).stop 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 msgr2=0x7f554006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 0x7f554006c310 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f553400c430 tx=0x7f553400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 shutdown_connections 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f55401602e0 0x7f554015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f554015f5f0 0x7f554006c310 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f552c06b4d0 0x7f552c06d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 --2- 172.21.15.60:0/1002208968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f554015ba00 0x7f554006bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.472+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 >> 172.21.15.60:0/1002208968 conn(0x7f55400b3270 msgr2=0x7f5540070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.473+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 shutdown_connections 2024-04-25T08:38:04.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.473+0000 7f554624d640 1 -- 172.21.15.60:0/1002208968 wait complete. 2024-04-25T08:38:04.478 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:38:04.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:04.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.585+0000 7f92a137e640 1 Processor -- start 2024-04-25T08:38:04.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.585+0000 7f92a137e640 1 -- start start 2024-04-25T08:38:04.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f92a137e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f929c0b96d0 0x7f929c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f92a137e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f929c0d0fe0 0x7f929c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f92a137e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c15e810 con 0x7f929c15dd20 2024-04-25T08:38:04.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f92a137e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c0d1c00 con 0x7f929c0b96d0 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f92a137e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c0d1420 con 0x7f929c0d0fe0 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f929c0b96d0 0x7f929c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f929c0b96d0 0x7f929c0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58226/0 (socket says 172.21.15.60:58226) 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 -- 172.21.15.60:0/3510404775 learned_addr learned my addr 172.21.15.60:0/3510404775 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929b7fe640 1 --2- 172.21.15.60:0/3510404775 >> v2:172.21.15.100:3300/0 conn(0x7f929c0d0fe0 0x7f929c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 -- 172.21.15.60:0/3510404775 >> v2:172.21.15.100:3300/0 conn(0x7f929c0d0fe0 msgr2=0x7f929c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 --2- 172.21.15.60:0/3510404775 >> v2:172.21.15.100:3300/0 conn(0x7f929c0d0fe0 0x7f929c0d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 -- 172.21.15.60:0/3510404775 >> v1:172.21.15.100:6789/0 conn(0x7f929c15dd20 legacy=0x7f929c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:04.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.586+0000 7f929a7fc640 1 -- 172.21.15.60:0/3510404775 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c15eac0 con 0x7f929c0b96d0 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.587+0000 7f929a7fc640 1 --2- 172.21.15.60:0/3510404775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c0b96d0 0x7f929c0d0aa0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f929c0d1990 tx=0x7f929002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c41f8384d17bd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.587+0000 7f9299ffb640 1 -- 172.21.15.60:0/3510404775 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290037070 con 0x7f929c0b96d0 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.587+0000 7f9299ffb640 1 -- 172.21.15.60:0/3510404775 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9290032440 con 0x7f929c0b96d0 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.587+0000 7f9299ffb640 1 -- 172.21.15.60:0/3510404775 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290032810 con 0x7f929c0b96d0 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 -- 172.21.15.60:0/3510404775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c0b96d0 msgr2=0x7f929c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 --2- 172.21.15.60:0/3510404775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c0b96d0 0x7f929c0d0aa0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f929c0d1990 tx=0x7f929002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 -- 172.21.15.60:0/3510404775 shutdown_connections 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 --2- 172.21.15.60:0/3510404775 >> v2:172.21.15.100:3300/0 conn(0x7f929c0d0fe0 0x7f929c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 --2- 172.21.15.60:0/3510404775 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c0b96d0 0x7f929c0d0aa0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 -- 172.21.15.60:0/3510404775 >> 172.21.15.60:0/3510404775 conn(0x7f929c061a20 msgr2=0x7f929c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 -- 172.21.15.60:0/3510404775 shutdown_connections 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.588+0000 7f92a137e640 1 -- 172.21.15.60:0/3510404775 wait complete. 2024-04-25T08:38:04.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.589+0000 7f92a137e640 1 Processor -- start 2024-04-25T08:38:04.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.589+0000 7f92a137e640 1 -- start start 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f92a137e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f929c0b96d0 0x7f929c158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f92a137e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f929c0d0fe0 0x7f929c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f92a137e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 0x7f929c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f92a137e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0d1e50 con 0x7f929c15dd20 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f92a137e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c15f080 con 0x7f929c0d0fe0 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f92a137e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c0d28b0 con 0x7f929c0b96d0 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 0x7f929c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 0x7f929c0cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58240/0 (socket says 172.21.15.60:58240) 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 -- 172.21.15.60:0/1534974697 learned_addr learned my addr 172.21.15.60:0/1534974697 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929affd640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f929c0b96d0 0x7f929c158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929a7fc640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f929c0d0fe0 0x7f929c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 -- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f929c0b96d0 msgr2=0x7f929c158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f929c0b96d0 0x7f929c158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 -- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f929c0d0fe0 msgr2=0x7f929c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f929c0d0fe0 0x7f929c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 -- 172.21.15.60:0/1534974697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c158740 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.590+0000 7f929b7fe640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 0x7f929c0cc690 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f929400cda0 tx=0x7f929400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.591+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294003b90 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.591+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9294003d30 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.591+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9294004240 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.591+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c0c8aa0 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.591+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c0c9060 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.592+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c073990 con 0x7f929c15dd20 2024-04-25T08:38:04.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.592+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92940043e0 con 0x7f929c15dd20 2024-04-25T08:38:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.593+0000 7f92837fe640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f928806b4d0 0x7f928806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.593+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f929408ad20 con 0x7f929c15dd20 2024-04-25T08:38:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.593+0000 7f929affd640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f928806b4d0 0x7f928806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.593+0000 7f929affd640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f928806b4d0 0x7f928806d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f928c006fd0 tx=0x7f928c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.597+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92940593c0 con 0x7f929c15dd20 2024-04-25T08:38:04.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.812+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f929c0d3500 con 0x7f929c15dd20 2024-04-25T08:38:04.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.813+0000 7f92837fe640 1 -- 172.21.15.60:0/1534974697 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f929405d460 con 0x7f929c15dd20 2024-04-25T08:38:04.807 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:04.807 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:38:04.807 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f928806b4d0 msgr2=0x7f928806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f928806b4d0 0x7f928806d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f928c006fd0 tx=0x7f928c009060 comp rx=0 tx=0).stop 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 msgr2=0x7f929c0cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 0x7f929c0cc690 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f929400cda0 tx=0x7f929400a660 comp rx=0 tx=0).stop 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 shutdown_connections 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f929c15dd20 0x7f929c0cc690 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f928806b4d0 0x7f928806d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f929c0d0fe0 0x7f929c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 --2- 172.21.15.60:0/1534974697 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f929c0b96d0 0x7f929c158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 >> 172.21.15.60:0/1534974697 conn(0x7f929c061a20 msgr2=0x7f929c061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:04.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 shutdown_connections 2024-04-25T08:38:04.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.815+0000 7f92a137e640 1 -- 172.21.15.60:0/1534974697 wait complete. 2024-04-25T08:38:04.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:04.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.927+0000 7fa0c9a5d640 1 Processor -- start 2024-04-25T08:38:04.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.927+0000 7fa0c9a5d640 1 -- start start 2024-04-25T08:38:04.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c9a5d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c9a5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c9a5d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c9a5d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c415dfc0 con 0x7fa0c415dbe0 2024-04-25T08:38:04.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c9a5d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40d1a00 con 0x7fa0c40b92a0 2024-04-25T08:38:04.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c9a5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b97a0 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58248/0 (socket says 172.21.15.60:58248) 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/3930545337 learned_addr learned my addr 172.21.15.60:0/3930545337 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/3930545337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b2cb0 msgr2=0x7fa0c40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 --2- 172.21.15.60:0/3930545337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/3930545337 >> v2:172.21.15.100:3300/0 conn(0x7fa0c40b92a0 msgr2=0x7fa0c40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 --2- 172.21.15.60:0/3930545337 >> v2:172.21.15.100:3300/0 conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.928+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/3930545337 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40d1c50 con 0x7fa0c415dbe0 2024-04-25T08:38:04.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.929+0000 7fa0c2ffd640 1 --2- 172.21.15.60:0/3930545337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40d0910 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa0c406c640 tx=0x7fa0b402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb9fc9e4e08e4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.929+0000 7fa0c27fc640 1 -- 172.21.15.60:0/3930545337 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4037070 con 0x7fa0c415dbe0 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.929+0000 7fa0c27fc640 1 -- 172.21.15.60:0/3930545337 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4032440 con 0x7fa0c415dbe0 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.929+0000 7fa0c27fc640 1 -- 172.21.15.60:0/3930545337 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4032810 con 0x7fa0c415dbe0 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/3930545337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c415dbe0 msgr2=0x7fa0c40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/3930545337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40d0910 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa0c406c640 tx=0x7fa0b402a200 comp rx=0 tx=0).stop 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/3930545337 shutdown_connections 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/3930545337 >> v2:172.21.15.100:3300/0 conn(0x7fa0c40b92a0 0x7fa0c40b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/3930545337 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b2cb0 0x7fa0c40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/3930545337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40d0910 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/3930545337 >> 172.21.15.60:0/3930545337 conn(0x7fa0c4061a20 msgr2=0x7fa0c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/3930545337 shutdown_connections 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.930+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/3930545337 wait complete. 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.931+0000 7fa0c9a5d640 1 Processor -- start 2024-04-25T08:38:04.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.931+0000 7fa0c9a5d640 1 -- start start 2024-04-25T08:38:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 0x7fa0c40cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b92a0 0x7fa0c40caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b6ab0 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40d2210 con 0x7fa0c415dbe0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b6c30 con 0x7fa0c40b92a0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 0x7fa0c40cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 0x7fa0c40cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58264/0 (socket says 172.21.15.60:58264) 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/1838963417 learned_addr learned my addr 172.21.15.60:0/1838963417 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0bbfff640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b92a0 0x7fa0c40caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c37fe640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b92a0 msgr2=0x7fa0c40caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b92a0 0x7fa0c40caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c415dbe0 msgr2=0x7fa0c40cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 -- 172.21.15.60:0/1838963417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40cc260 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c37fe640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c2ffd640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 0x7fa0c40cd6c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4041070 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b403c890 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4037040 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40cdd60 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.932+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40b3660 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.934+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4073990 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.934+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0b4044070 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.934+0000 7fa0c0ff9640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0a406b500 0x7fa0a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.934+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0b40aadb0 con 0x7fa0c40b2cb0 2024-04-25T08:38:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.935+0000 7fa0bbfff640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0a406b500 0x7fa0a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.935+0000 7fa0bbfff640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0a406b500 0x7fa0a406d9c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4152560 tx=0x7fa0ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:04.939+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0c40cc260 con 0x7fa0c40b2cb0 2024-04-25T08:38:05.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.156+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0c415af70 con 0x7fa0c40b2cb0 2024-04-25T08:38:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.157+0000 7fa0c0ff9640 1 -- 172.21.15.60:0/1838963417 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa0b4079c50 con 0x7fa0c40b2cb0 2024-04-25T08:38:05.151 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:05.151 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":71,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:37:29.166595+0000","modified":"2024-04-25T08:37:32.172373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8805},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8805":{"gid":8805,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6837/2648832765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":2648832765},{"type":"v1","addr":"172.21.15.100:6837","nonce":2648832765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:38:05.151 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 71 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0a406b500 msgr2=0x7fa0a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0a406b500 0x7fa0a406d9c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4152560 tx=0x7fa0ac008040 comp rx=0 tx=0).stop 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 msgr2=0x7fa0c40cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 0x7fa0c40cd6c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4000c00 tx=0x7fa0b40046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 shutdown_connections 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa0a406b500 0x7fa0a406d9c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0c415dbe0 0x7fa0c40cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0c40b92a0 0x7fa0c40caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 --2- 172.21.15.60:0/1838963417 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c40b2cb0 0x7fa0c40cd6c0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 >> 172.21.15.60:0/1838963417 conn(0x7fa0c4061a20 msgr2=0x7fa0c415bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:05.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 shutdown_connections 2024-04-25T08:38:05.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.159+0000 7fa0c9a5d640 1 -- 172.21.15.60:0/1838963417 wait complete. 2024-04-25T08:38:05.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.271+0000 7f48bca4d640 1 Processor -- start 2024-04-25T08:38:05.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.271+0000 7f48bca4d640 1 -- start start 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48bca4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b815dd30 0x7f48b80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48bca4d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f48b80d0ad0 0x7f48b80b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48bca4d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b80d1df0 con 0x7f48b80b9770 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48bca4d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b80d20d0 con 0x7f48b80d0ad0 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48bca4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d23b0 con 0x7f48b815dd30 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48b5d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f48b80b9770 0x7f48b80d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50672/0 (socket says 172.21.15.60:50672) 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48b5d9b640 1 -- 172.21.15.60:0/2356639825 learned_addr learned my addr 172.21.15.60:0/2356639825 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48b659c640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b815dd30 0x7f48b80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48b6d9d640 1 --2- 172.21.15.60:0/2356639825 >> v2:172.21.15.100:3300/0 conn(0x7f48b80d0ad0 0x7f48b80b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365378723 0 0) 0x7f48b80d1df0 con 0x7f48b80b9770 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.272+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a0003520 con 0x7f48b80b9770 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3870188913 0 0) 0x7f48a0003520 con 0x7f48b80b9770 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b815dd30 msgr2=0x7f48b80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b815dd30 0x7f48b80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 >> v2:172.21.15.100:3300/0 conn(0x7f48b80d0ad0 msgr2=0x7f48b80b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 --2- 172.21.15.60:0/2356639825 >> v2:172.21.15.100:3300/0 conn(0x7f48b80d0ad0 0x7f48b80b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b80d2690 con 0x7f48b80b9770 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f48ac0017f0 con 0x7f48b80b9770 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 >> v1:172.21.15.60:6789/0 conn(0x7f48b80b9770 legacy=0x7f48b80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a00039b0 0x7f48a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a00043a0 0x7f48a0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a0004f90 0x7f48a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d20d0 con 0x7f48a00043a0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d23b0 con 0x7f48a0004f90 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a000bf30 con 0x7f48a00039b0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b659c640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b815dd30 0x7f48b80b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a00043a0 0x7f48a0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b6d9d640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a0004f90 0x7f48a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b659c640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a00039b0 0x7f48a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b5d9b640 1 -- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a00039b0 msgr2=0x7f48a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48a00039b0 0x7f48a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b5d9b640 1 -- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a0004f90 msgr2=0x7f48a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a0004f90 0x7f48a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.273+0000 7f48b5d9b640 1 -- 172.21.15.60:0/2356639825 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0003520 con 0x7f48a00043a0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a00043a0 0x7f48a0004880 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c240 tx=0x7f48a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733c8fa64d057814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac007d30 con 0x7f48a00043a0 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac007270 con 0x7f48a00043a0 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48b559a640 1 -- 172.21.15.60:0/2356639825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0057a0 con 0x7f48a00043a0 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48bca4d640 1 -- 172.21.15.60:0/2356639825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a00043a0 msgr2=0x7f48a0004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48bca4d640 1 --2- 172.21.15.60:0/2356639825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48a00043a0 0x7f48a0004880 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c240 tx=0x7f48a0005830 comp rx=0 tx=0).stop 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.274+0000 7f48b6d9d640 1 -- 172.21.15.60:0/2356639825 reap_dead start 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.275+0000 7f48bca4d640 1 -- 172.21.15.60:0/2356639825 shutdown_connections 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.275+0000 7f48bca4d640 1 -- 172.21.15.60:0/2356639825 >> 172.21.15.60:0/2356639825 conn(0x7f48b8061c30 msgr2=0x7f48b806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.275+0000 7f48bca4d640 1 -- 172.21.15.60:0/2356639825 shutdown_connections 2024-04-25T08:38:05.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.275+0000 7f48bca4d640 1 -- 172.21.15.60:0/2356639825 wait complete. 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.276+0000 7f48bca4d640 1 Processor -- start 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.276+0000 7f48bca4d640 1 -- start start 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a00043a0 0x7f48b814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 0x7f48b80ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b81f5800 0x7f48b80cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d1b90 con 0x7f48b81f5380 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d1a10 con 0x7f48a00043a0 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80d2e90 con 0x7f48b81f5800 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 0x7f48b80ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 0x7f48b80ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58288/0 (socket says 172.21.15.60:58288) 2024-04-25T08:38:05.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 -- 172.21.15.60:0/1478107016 learned_addr learned my addr 172.21.15.60:0/1478107016 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b659c640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a00043a0 0x7f48b814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b6d9d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b81f5800 0x7f48b80cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 -- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b81f5800 msgr2=0x7f48b80cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b81f5800 0x7f48b80cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 -- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a00043a0 msgr2=0x7f48b814def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a00043a0 0x7f48b814def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 -- 172.21.15.60:0/1478107016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b814e430 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b659c640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a00043a0 0x7f48b814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48b5d9b640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 0x7f48b80ccc50 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000ef0 tx=0x7f48b80d2860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac042070 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac0059b0 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b81f5db0 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.277+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac005da0 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.278+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b81f6370 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.278+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b8073990 con 0x7f48b81f5380 2024-04-25T08:38:05.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.279+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48ac045070 con 0x7f48b81f5380 2024-04-25T08:38:05.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.279+0000 7f48a6ffd640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f489806b4d0 0x7f489806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.279+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f48ac0aae90 con 0x7f48b81f5380 2024-04-25T08:38:05.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.279+0000 7f48b659c640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f489806b4d0 0x7f489806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.280+0000 7f48b659c640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f489806b4d0 0x7f489806d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f48b80b8cf0 tx=0x7f48a800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.283+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48ac04c030 con 0x7f48b81f5380 2024-04-25T08:38:05.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.464+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48b80d0610 con 0x7f48b81f5380 2024-04-25T08:38:05.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.465+0000 7f48a6ffd640 1 -- 172.21.15.60:0/1478107016 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f48ac079cf0 con 0x7f48b81f5380 2024-04-25T08:38:05.459 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:05.459 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":123,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:37:30.801169+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:37:27.446495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:37:28.138802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:05.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f489806b4d0 msgr2=0x7f489806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f489806b4d0 0x7f489806d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f48b80b8cf0 tx=0x7f48a800b040 comp rx=0 tx=0).stop 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 msgr2=0x7f48b80ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 0x7f48b80ccc50 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f48ac000ef0 tx=0x7f48b80d2860 comp rx=0 tx=0).stop 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 shutdown_connections 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f48b81f5800 0x7f48b80cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f48b81f5380 0x7f48b80ccc50 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f489806b4d0 0x7f489806d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 --2- 172.21.15.60:0/1478107016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f48a00043a0 0x7f48b814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 >> 172.21.15.60:0/1478107016 conn(0x7f48b8061c30 msgr2=0x7f48b815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 shutdown_connections 2024-04-25T08:38:05.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.467+0000 7f48bca4d640 1 -- 172.21.15.60:0/1478107016 wait complete. 2024-04-25T08:38:05.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:38:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.581+0000 7f43333d5640 1 Processor -- start 2024-04-25T08:38:05.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.581+0000 7f43333d5640 1 -- start start 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.582+0000 7f43333d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f432c1610f0 0x7f432c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.582+0000 7f43333d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c0c6960 con 0x7f432c1616f0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.582+0000 7f43333d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c162260 con 0x7f432c0ce7b0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.582+0000 7f43333d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c1614d0 con 0x7f432c1610f0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.582+0000 7f4330970640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f432c1616f0 0x7f432c0ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50678/0 (socket says 172.21.15.60:50678) 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.582+0000 7f4330970640 1 -- 172.21.15.60:0/3368879293 learned_addr learned my addr 172.21.15.60:0/3368879293 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4331171640 1 --2- 172.21.15.60:0/3368879293 >> v2:172.21.15.100:3300/0 conn(0x7f432c1610f0 0x7f432c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792195643 0 0) 0x7f432c0c6960 con 0x7f432c1616f0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4314003520 con 0x7f432c1616f0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034715992 0 0) 0x7f4314003520 con 0x7f432c1616f0 2024-04-25T08:38:05.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 >> v2:172.21.15.100:3300/0 conn(0x7f432c1610f0 msgr2=0x7f432c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 --2- 172.21.15.60:0/3368879293 >> v2:172.21.15.100:3300/0 conn(0x7f432c1610f0 0x7f432c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 >> v1:172.21.15.100:6789/0 conn(0x7f432c0ce7b0 legacy=0x7f432c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c162540 con 0x7f432c1616f0 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f4320001670 con 0x7f432c1616f0 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 >> v1:172.21.15.60:6789/0 conn(0x7f432c1616f0 legacy=0x7f432c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140039b0 0x7f4314003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4314004800 0x7f4314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140051c0 0x7f431400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c1614d0 con 0x7f4314004800 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431400c060 con 0x7f43140039b0 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431400c260 con 0x7f43140051c0 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4330970640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4314004800 0x7f4314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4331171640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140039b0 0x7f4314003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4331972640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140051c0 0x7f431400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4330970640 1 -- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140051c0 msgr2=0x7f431400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4330970640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f43140051c0 0x7f431400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4330970640 1 -- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140039b0 msgr2=0x7f4314003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4330970640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43140039b0 0x7f4314003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4330970640 1 -- 172.21.15.60:0/3368879293 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314003520 con 0x7f4314004800 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.583+0000 7f4331972640 1 -- 172.21.15.60:0/3368879293 reap_dead start 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.584+0000 7f4330970640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4314004800 0x7f4314004c80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f432000c210 tx=0x7f4314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9252ef533c734a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.584+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320007d10 con 0x7f4314004800 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.584+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320005ad0 con 0x7f4314004800 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.584+0000 7f432bfff640 1 -- 172.21.15.60:0/3368879293 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320007070 con 0x7f4314004800 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 -- 172.21.15.60:0/3368879293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4314004800 msgr2=0x7f4314004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4314004800 0x7f4314004c80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f432000c210 tx=0x7f4314005830 comp rx=0 tx=0).stop 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 -- 172.21.15.60:0/3368879293 shutdown_connections 2024-04-25T08:38:05.581 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 --2- 172.21.15.60:0/3368879293 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4314004800 0x7f4314004c80 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 -- 172.21.15.60:0/3368879293 >> 172.21.15.60:0/3368879293 conn(0x7f432c06f500 msgr2=0x7f432c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 -- 172.21.15.60:0/3368879293 shutdown_connections 2024-04-25T08:38:05.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.585+0000 7f43333d5640 1 -- 172.21.15.60:0/3368879293 wait complete. 2024-04-25T08:38:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.586+0000 7f43333d5640 1 Processor -- start 2024-04-25T08:38:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.586+0000 7f43333d5640 1 -- start start 2024-04-25T08:38:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f43333d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4314004800 0x7f432c0acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f43333d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 0x7f432c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f43333d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432c15bce0 0x7f432c0af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f43333d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0cf690 con 0x7f432c1f3a40 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f43333d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0cfce0 con 0x7f4314004800 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f43333d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c161b50 con 0x7f432c15bce0 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 0x7f432c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 0x7f432c15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58306/0 (socket says 172.21.15.60:58306) 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 -- 172.21.15.60:0/2817030834 learned_addr learned my addr 172.21.15.60:0/2817030834 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4331171640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4314004800 0x7f432c0acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4331972640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432c15bce0 0x7f432c0af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 -- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432c15bce0 msgr2=0x7f432c0af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432c15bce0 0x7f432c0af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 -- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4314004800 msgr2=0x7f432c0acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4314004800 0x7f432c0acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4330970640 1 -- 172.21.15.60:0/2817030834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c15ba90 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.587+0000 7f4331171640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4314004800 0x7f432c0acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.588+0000 7f4330970640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 0x7f432c15b550 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f432002adb0 tx=0x7f432c162430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.588+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320037040 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.588+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4320041070 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.588+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c0ad330 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.588+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43200047e0 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.588+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c0ad8f0 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.589+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c073a00 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.589+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4320004980 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.590+0000 7f4329ffb640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f431806b500 0x7f431806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.590+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43200a9c60 con 0x7f432c1f3a40 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.590+0000 7f4331171640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f431806b500 0x7f431806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.591+0000 7f4331171640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f431806b500 0x7f431806d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f432c068c30 tx=0x7f431c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.594+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4320030380 con 0x7f432c1f3a40 2024-04-25T08:38:05.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.818+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f432c0cf150 con 0x7f432c1f3a40 2024-04-25T08:38:05.813 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:38:05.819+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:38:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.821+0000 7f4329ffb640 1 -- 172.21.15.60:0/2817030834 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f43200789d0 con 0x7f432c1f3a40 2024-04-25T08:38:05.814 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f431806b500 msgr2=0x7f431806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f431806b500 0x7f431806d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f432c068c30 tx=0x7f431c00b040 comp rx=0 tx=0).stop 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 msgr2=0x7f432c15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 0x7f432c15b550 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f432002adb0 tx=0x7f432c162430 comp rx=0 tx=0).stop 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 shutdown_connections 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f432c15bce0 0x7f432c0af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f432c1f3a40 0x7f432c15b550 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f431806b500 0x7f431806d9c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 --2- 172.21.15.60:0/2817030834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4314004800 0x7f432c0acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.823+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 >> 172.21.15.60:0/2817030834 conn(0x7f432c06f500 msgr2=0x7f432c0b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.824+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 shutdown_connections 2024-04-25T08:38:05.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.824+0000 7f43333d5640 1 -- 172.21.15.60:0/2817030834 wait complete. 2024-04-25T08:38:05.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:38:05.857 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:38:05.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.938+0000 7fb1d4a38640 1 Processor -- start 2024-04-25T08:38:05.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.938+0000 7fb1d4a38640 1 -- start start 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1d4a38640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1d006ccd0 0x7fb1d00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1d4a38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d015deb0 0x7fb1d015e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1d4a38640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d00d1770 con 0x7fb1d0070dc0 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1d4a38640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d01622b0 con 0x7fb1d006ccd0 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1d4a38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00b10f0 con 0x7fb1d015deb0 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1d006ccd0 0x7fb1d00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb1d006ccd0 0x7fb1d00b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58316/0 (socket says 172.21.15.60:58316) 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 -- 172.21.15.60:0/2873667771 learned_addr learned my addr 172.21.15.60:0/2873667771 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1ced9d640 1 --2- 172.21.15.60:0/2873667771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d015deb0 0x7fb1d015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 -- 172.21.15.60:0/2873667771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d015deb0 msgr2=0x7fb1d015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 --2- 172.21.15.60:0/2873667771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d015deb0 0x7fb1d015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 -- 172.21.15.60:0/2873667771 >> v1:172.21.15.100:6789/0 conn(0x7fb1d0070dc0 legacy=0x7fb1d00711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:05.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.939+0000 7fb1c7fff640 1 -- 172.21.15.60:0/2873667771 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0162740 con 0x7fb1d006ccd0 2024-04-25T08:38:05.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.940+0000 7fb1c7fff640 1 --2- 172.21.15.60:0/2873667771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d006ccd0 0x7fb1d00b07a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009870 tx=0x7fb1b802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c055e4a88cd9a7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.940+0000 7fb1cdd9b640 1 -- 172.21.15.60:0/2873667771 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8037070 con 0x7fb1d006ccd0 2024-04-25T08:38:05.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.940+0000 7fb1cdd9b640 1 -- 172.21.15.60:0/2873667771 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b802ae10 con 0x7fb1d006ccd0 2024-04-25T08:38:05.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.940+0000 7fb1cdd9b640 1 -- 172.21.15.60:0/2873667771 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8032660 con 0x7fb1d006ccd0 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.940+0000 7fb1d4a38640 1 -- 172.21.15.60:0/2873667771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d006ccd0 msgr2=0x7fb1d00b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.940+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/2873667771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d006ccd0 0x7fb1d00b07a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009870 tx=0x7fb1b802a100 comp rx=0 tx=0).stop 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.941+0000 7fb1d4a38640 1 -- 172.21.15.60:0/2873667771 shutdown_connections 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.941+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/2873667771 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d006ccd0 0x7fb1d00b07a0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.941+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/2873667771 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d015deb0 0x7fb1d015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.941+0000 7fb1d4a38640 1 -- 172.21.15.60:0/2873667771 >> 172.21.15.60:0/2873667771 conn(0x7fb1d00b3530 msgr2=0x7fb1d00b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.941+0000 7fb1d4a38640 1 -- 172.21.15.60:0/2873667771 shutdown_connections 2024-04-25T08:38:05.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.941+0000 7fb1d4a38640 1 -- 172.21.15.60:0/2873667771 wait complete. 2024-04-25T08:38:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.942+0000 7fb1d4a38640 1 Processor -- start 2024-04-25T08:38:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.942+0000 7fb1d4a38640 1 -- start start 2024-04-25T08:38:05.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1d4a38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d006ccd0 0x7fb1d01f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1d4a38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d0070dc0 0x7fb1d0148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1d4a38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 0x7fb1d00d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1d4a38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0161f40 con 0x7fb1d015deb0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1d4a38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0161c60 con 0x7fb1d0070dc0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1d4a38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00b5840 con 0x7fb1d006ccd0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 0x7fb1d00d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 0x7fb1d00d0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58318/0 (socket says 172.21.15.60:58318) 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 -- 172.21.15.60:0/1404149653 learned_addr learned my addr 172.21.15.60:0/1404149653 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ce59c640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d006ccd0 0x7fb1d01f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1c7fff640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d0070dc0 0x7fb1d0148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 -- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d006ccd0 msgr2=0x7fb1d01f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d006ccd0 0x7fb1d01f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 -- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d0070dc0 msgr2=0x7fb1d0148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d0070dc0 0x7fb1d0148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 -- 172.21.15.60:0/1404149653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d01f7cd0 con 0x7fb1d015deb0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.943+0000 7fb1ced9d640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 0x7fb1d00d0520 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8007e60 tx=0x7fb1c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.944+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8014420 con 0x7fb1d015deb0 2024-04-25T08:38:05.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.944+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8019070 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.944+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d00a8960 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.944+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8014820 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.944+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d0149780 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.945+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d0073a00 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.945+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1c80149c0 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.945+0000 7fb1c6ffd640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.946+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb1c808aa70 con 0x7fb1d015deb0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.946+0000 7fb1ce59c640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:05.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.946+0000 7fb1ce59c640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb1b006b4d0 0x7fb1b006d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004f60 tx=0x7fb1c0007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:05.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:05.950+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1c8059150 con 0x7fb1d015deb0 2024-04-25T08:38:06.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.171+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb1d0149030 con 0x7fb1d015deb0 2024-04-25T08:38:06.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.822+0000 7fb1c6ffd640 1 -- 172.21.15.60:0/1404149653 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb1c805d1f0 con 0x7fb1d015deb0 2024-04-25T08:38:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb1b006b4d0 msgr2=0x7fb1b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb1b006b4d0 0x7fb1b006d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004f60 tx=0x7fb1c0007e30 comp rx=0 tx=0).stop 2024-04-25T08:38:06.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 msgr2=0x7fb1d00d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 0x7fb1d00d0520 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8007e60 tx=0x7fb1c800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 shutdown_connections 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb1d015deb0 0x7fb1d00d0520 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb1b006b4d0 0x7fb1b006d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1d0070dc0 0x7fb1d0148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 --2- 172.21.15.60:0/1404149653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb1d006ccd0 0x7fb1d01f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 >> 172.21.15.60:0/1404149653 conn(0x7fb1d00b3530 msgr2=0x7fb1d00b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 shutdown_connections 2024-04-25T08:38:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.825+0000 7fb1d4a38640 1 -- 172.21.15.60:0/1404149653 wait complete. 2024-04-25T08:38:06.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:38:06.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.953+0000 7fc49b07c640 1 Processor -- start 2024-04-25T08:38:06.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.953+0000 7fc49b07c640 1 -- start start 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.953+0000 7fc49b07c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc4940cd4c0 0x7fc4940cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.953+0000 7fc49b07c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc49415b1c0 0x7fc49415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc49b07c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49415e6e0 con 0x7fc49415e1a0 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc49b07c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49415e9c0 con 0x7fc4940cd4c0 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc49b07c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49415a9c0 con 0x7fc49415b1c0 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc4940cd4c0 0x7fc4940cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc4940cd4c0 0x7fc4940cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58328/0 (socket says 172.21.15.60:58328) 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 -- 172.21.15.60:0/1469163693 learned_addr learned my addr 172.21.15.60:0/1469163693 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:06.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc493fff640 1 --2- 172.21.15.60:0/1469163693 >> v2:172.21.15.100:3300/0 conn(0x7fc49415b1c0 0x7fc49415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 -- 172.21.15.60:0/1469163693 >> v2:172.21.15.100:3300/0 conn(0x7fc49415b1c0 msgr2=0x7fc49415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 --2- 172.21.15.60:0/1469163693 >> v2:172.21.15.100:3300/0 conn(0x7fc49415b1c0 0x7fc49415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 -- 172.21.15.60:0/1469163693 >> v1:172.21.15.100:6789/0 conn(0x7fc49415e1a0 legacy=0x7fc494150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc498e18640 1 -- 172.21.15.60:0/1469163693 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49415aca0 con 0x7fc4940cd4c0 2024-04-25T08:38:06.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.954+0000 7fc493fff640 1 --2- 172.21.15.60:0/1469163693 >> v2:172.21.15.100:3300/0 conn(0x7fc49415b1c0 0x7fc49415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc498e18640 1 --2- 172.21.15.60:0/1469163693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4940cd4c0 0x7fc4940cd8a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fc4940cdde0 tx=0x7fc48402b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04919611d215f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc4937fe640 1 -- 172.21.15.60:0/1469163693 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48400b040 con 0x7fc4940cd4c0 2024-04-25T08:38:06.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc4937fe640 1 -- 172.21.15.60:0/1469163693 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc484030710 con 0x7fc4940cd4c0 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc4937fe640 1 -- 172.21.15.60:0/1469163693 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc484030ae0 con 0x7fc4940cd4c0 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc49b07c640 1 -- 172.21.15.60:0/1469163693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4940cd4c0 msgr2=0x7fc4940cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc49b07c640 1 --2- 172.21.15.60:0/1469163693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4940cd4c0 0x7fc4940cd8a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fc4940cdde0 tx=0x7fc48402b5e0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc49b07c640 1 -- 172.21.15.60:0/1469163693 shutdown_connections 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc49b07c640 1 --2- 172.21.15.60:0/1469163693 >> v2:172.21.15.100:3300/0 conn(0x7fc49415b1c0 0x7fc49415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc49b07c640 1 --2- 172.21.15.60:0/1469163693 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc4940cd4c0 0x7fc4940cd8a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.955+0000 7fc49b07c640 1 -- 172.21.15.60:0/1469163693 >> 172.21.15.60:0/1469163693 conn(0x7fc49406abe0 msgr2=0x7fc49406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.956+0000 7fc49b07c640 1 -- 172.21.15.60:0/1469163693 shutdown_connections 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.956+0000 7fc49b07c640 1 -- 172.21.15.60:0/1469163693 wait complete. 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.957+0000 7fc49b07c640 1 Processor -- start 2024-04-25T08:38:06.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.957+0000 7fc49b07c640 1 -- start start 2024-04-25T08:38:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4940cd4c0 0x7fc494141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49415b1c0 0x7fc49414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 0x7fc494150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49406ee20 con 0x7fc49415e1a0 2024-04-25T08:38:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49406eca0 con 0x7fc49415b1c0 2024-04-25T08:38:06.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941512a0 con 0x7fc4940cd4c0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 0x7fc494150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 0x7fc494150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58332/0 (socket says 172.21.15.60:58332) 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 -- 172.21.15.60:0/60205515 learned_addr learned my addr 172.21.15.60:0/60205515 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc498e18640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4940cd4c0 0x7fc494141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc493fff640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49415b1c0 0x7fc49414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 -- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4940cd4c0 msgr2=0x7fc494141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4940cd4c0 0x7fc494141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 -- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49415b1c0 msgr2=0x7fc49414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49415b1c0 0x7fc49414fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 -- 172.21.15.60:0/60205515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4941f50d0 con 0x7fc49415e1a0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc499619640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 0x7fc494150060 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00b9e0 tx=0x7fc48c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c003b70 con 0x7fc49415e1a0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c003d10 con 0x7fc49415e1a0 2024-04-25T08:38:06.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4941f53b0 con 0x7fc49415e1a0 2024-04-25T08:38:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.958+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00cb60 con 0x7fc49415e1a0 2024-04-25T08:38:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.959+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4941f5970 con 0x7fc49415e1a0 2024-04-25T08:38:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.959+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4940af4b0 con 0x7fc49415e1a0 2024-04-25T08:38:06.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.960+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc48c00cd00 con 0x7fc49415e1a0 2024-04-25T08:38:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.960+0000 7fc4917fa640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.960+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc48c08a190 con 0x7fc49415e1a0 2024-04-25T08:38:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.961+0000 7fc498e18640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:06.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.961+0000 7fc498e18640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc47406b4d0 0x7fc47406d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc484035000 tx=0x7fc48400c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:06.965+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4940af4b0 con 0x7fc49415e1a0 2024-04-25T08:38:07.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.145+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc494073a00 con 0x7fc49415e1a0 2024-04-25T08:38:07.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.823+0000 7fc4917fa640 1 -- 172.21.15.60:0/60205515 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc48c058ec0 con 0x7fc49415e1a0 2024-04-25T08:38:07.818 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:38:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.827+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc47406b4d0 msgr2=0x7fc47406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:07.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.827+0000 7fc49b07c640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc47406b4d0 0x7fc47406d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc484035000 tx=0x7fc48400c040 comp rx=0 tx=0).stop 2024-04-25T08:38:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.827+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 msgr2=0x7fc494150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:07.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.827+0000 7fc49b07c640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 0x7fc494150060 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00b9e0 tx=0x7fc48c00bea0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.828+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 shutdown_connections 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.828+0000 7fc49b07c640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc49415e1a0 0x7fc494150060 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.828+0000 7fc49b07c640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.828+0000 7fc49b07c640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc49415b1c0 0x7fc49414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.828+0000 7fc49b07c640 1 --2- 172.21.15.60:0/60205515 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc4940cd4c0 0x7fc494141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.828+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 >> 172.21.15.60:0/60205515 conn(0x7fc49406abe0 msgr2=0x7fc49414dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.829+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 shutdown_connections 2024-04-25T08:38:07.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.829+0000 7fc49b07c640 1 -- 172.21.15.60:0/60205515 wait complete. 2024-04-25T08:38:07.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:38:07.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 Processor -- start 2024-04-25T08:38:07.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 -- start start 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f038c15bb50 0x7f038c0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f038c0c70d0 0x7f038c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038c0c75e0 con 0x7f038c0cfb40 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c0c8680 con 0x7f038c15bb50 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.957+0000 7f039236a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c0c78c0 con 0x7f038c0c70d0 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f038c15bb50 0x7f038c0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f038c15bb50 0x7f038c0c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58340/0 (socket says 172.21.15.60:58340) 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 -- 172.21.15.60:0/1087339583 learned_addr learned my addr 172.21.15.60:0/1087339583 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 -- 172.21.15.60:0/1087339583 >> v2:172.21.15.100:3300/0 conn(0x7f038c0c70d0 msgr2=0x7f038c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:07.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 --2- 172.21.15.60:0/1087339583 >> v2:172.21.15.100:3300/0 conn(0x7f038c0c70d0 0x7f038c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 -- 172.21.15.60:0/1087339583 >> v1:172.21.15.100:6789/0 conn(0x7f038c0cfb40 legacy=0x7f038c070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:07.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.958+0000 7f038bfff640 1 -- 172.21.15.60:0/1087339583 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0c7ba0 con 0x7f038c15bb50 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f038bfff640 1 --2- 172.21.15.60:0/1087339583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0c6b90 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f038c06f220 tx=0x7f037c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9187da2195e69301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f038affd640 1 -- 172.21.15.60:0/1087339583 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c037070 con 0x7f038c15bb50 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f038affd640 1 -- 172.21.15.60:0/1087339583 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c032440 con 0x7f038c15bb50 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f038affd640 1 -- 172.21.15.60:0/1087339583 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c032810 con 0x7f038c15bb50 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f039236a640 1 -- 172.21.15.60:0/1087339583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 msgr2=0x7f038c0c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f039236a640 1 --2- 172.21.15.60:0/1087339583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0c6b90 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f038c06f220 tx=0x7f037c02a2c0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f039236a640 1 -- 172.21.15.60:0/1087339583 shutdown_connections 2024-04-25T08:38:07.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f039236a640 1 --2- 172.21.15.60:0/1087339583 >> v2:172.21.15.100:3300/0 conn(0x7f038c0c70d0 0x7f038c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f039236a640 1 --2- 172.21.15.60:0/1087339583 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0c6b90 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.959+0000 7f039236a640 1 -- 172.21.15.60:0/1087339583 >> 172.21.15.60:0/1087339583 conn(0x7f038c06f500 msgr2=0x7f038c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:07.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.960+0000 7f039236a640 1 -- 172.21.15.60:0/1087339583 shutdown_connections 2024-04-25T08:38:07.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.960+0000 7f039236a640 1 -- 172.21.15.60:0/1087339583 wait complete. 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.961+0000 7f039236a640 1 Processor -- start 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.961+0000 7f039236a640 1 -- start start 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f039236a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f038c0c70d0 0x7f038c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f039236a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038c0cfb40 0x7f038c0b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f039236a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f039236a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c15f6d0 con 0x7f038c15bb50 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f039236a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c0c8160 con 0x7f038c0cfb40 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f039236a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c15f850 con 0x7f038c0c70d0 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58350/0 (socket says 172.21.15.60:58350) 2024-04-25T08:38:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 -- 172.21.15.60:0/2941308643 learned_addr learned my addr 172.21.15.60:0/2941308643 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f038bfff640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f038c0c70d0 0x7f038c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f038b7fe640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038c0cfb40 0x7f038c0b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 -- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f038c0c70d0 msgr2=0x7f038c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f038c0c70d0 0x7f038c0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 -- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038c0cfb40 msgr2=0x7f038c0b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038c0cfb40 0x7f038c0b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 -- 172.21.15.60:0/2941308643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0b7030 con 0x7f038c15bb50 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f038b7fe640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038c0cfb40 0x7f038c0b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f038bfff640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f038c0c70d0 0x7f038c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0390907640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0b69d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f038400c780 tx=0x7f038400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384017070 con 0x7f038c15bb50 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.962+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0384003da0 con 0x7f038c15bb50 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.963+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c0b7310 con 0x7f038c15bb50 2024-04-25T08:38:07.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.963+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03840052b0 con 0x7f038c15bb50 2024-04-25T08:38:07.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.963+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c1584a0 con 0x7f038c15bb50 2024-04-25T08:38:07.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.964+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c073a00 con 0x7f038c15bb50 2024-04-25T08:38:07.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.964+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f038401a070 con 0x7f038c15bb50 2024-04-25T08:38:07.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.965+0000 7f0388ff9640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f037806b4d0 0x7f037806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:07.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.965+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f03840898b0 con 0x7f038c15bb50 2024-04-25T08:38:07.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.965+0000 7f038bfff640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f037806b4d0 0x7f037806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:07.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.965+0000 7f038bfff640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f037806b4d0 0x7f037806d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f037c034000 tx=0x7f037c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:07.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:07.969+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0384058700 con 0x7f038c15bb50 2024-04-25T08:38:08.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.150+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f038c158670 con 0x7f038c15bb50 2024-04-25T08:38:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.829+0000 7f0388ff9640 1 -- 172.21.15.60:0/2941308643 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0384058000 con 0x7f038c15bb50 2024-04-25T08:38:08.823 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:38:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f037806b4d0 msgr2=0x7f037806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f037806b4d0 0x7f037806d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f037c034000 tx=0x7f037c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:08.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 msgr2=0x7f038c0b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0b69d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f038400c780 tx=0x7f038400cc40 comp rx=0 tx=0).stop 2024-04-25T08:38:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 shutdown_connections 2024-04-25T08:38:08.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f038c15bb50 0x7f038c0b69d0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f037806b4d0 0x7f037806d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038c0cfb40 0x7f038c0b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 --2- 172.21.15.60:0/2941308643 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f038c0c70d0 0x7f038c0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 >> 172.21.15.60:0/2941308643 conn(0x7f038c06f500 msgr2=0x7f038c06f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.833+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 shutdown_connections 2024-04-25T08:38:08.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.834+0000 7f039236a640 1 -- 172.21.15.60:0/2941308643 wait complete. 2024-04-25T08:38:08.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:38:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.962+0000 7f5bc2fb7640 1 Processor -- start 2024-04-25T08:38:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.962+0000 7f5bc2fb7640 1 -- start start 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bc2fb7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5bbc0ced80 0x7f5bbc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bc2fb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bc2fb7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc15e000 con 0x7f5bbc15dc20 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bc2fb7640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0d1de0 con 0x7f5bbc0ced80 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bc2fb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc161c60 con 0x7f5bbc161710 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bbbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5bbc0ced80 0x7f5bbc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bbbfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5bbc0ced80 0x7f5bbc0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58354/0 (socket says 172.21.15.60:58354) 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.963+0000 7f5bbbfff640 1 -- 172.21.15.60:0/3529213389 learned_addr learned my addr 172.21.15.60:0/3529213389 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bc1554640 1 --2- 172.21.15.60:0/3529213389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bbbfff640 1 -- 172.21.15.60:0/3529213389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 msgr2=0x7f5bbc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bbbfff640 1 --2- 172.21.15.60:0/3529213389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bbbfff640 1 -- 172.21.15.60:0/3529213389 >> v1:172.21.15.100:6789/0 conn(0x7f5bbc15dc20 legacy=0x7f5bbc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bbbfff640 1 -- 172.21.15.60:0/3529213389 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc161f40 con 0x7f5bbc0ced80 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bbbfff640 1 --2- 172.21.15.60:0/3529213389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc0d22e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009870 tx=0x7f5bac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ebbf03ece69a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.964+0000 7f5bbb7fe640 1 -- 172.21.15.60:0/3529213389 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac037070 con 0x7f5bbc0ced80 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bbb7fe640 1 -- 172.21.15.60:0/3529213389 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac032440 con 0x7f5bbc0ced80 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bbb7fe640 1 -- 172.21.15.60:0/3529213389 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac032810 con 0x7f5bbc0ced80 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/3529213389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 msgr2=0x7f5bbc0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/3529213389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc0d22e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009870 tx=0x7f5bac02a210 comp rx=0 tx=0).stop 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/3529213389 shutdown_connections 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/3529213389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc0d22e0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/3529213389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/3529213389 >> 172.21.15.60:0/3529213389 conn(0x7f5bbc0d0420 msgr2=0x7f5bbc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/3529213389 shutdown_connections 2024-04-25T08:38:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.965+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/3529213389 wait complete. 2024-04-25T08:38:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.966+0000 7f5bc2fb7640 1 Processor -- start 2024-04-25T08:38:08.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.966+0000 7f5bc2fb7640 1 -- start start 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc2fb7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc1f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc2fb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bbc15dc20 0x7f5bbc0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc2fb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc2fb7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b5e50 con 0x7f5bbc0ced80 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc2fb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc162500 con 0x7f5bbc15dc20 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc2fb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b5fd0 con 0x7f5bbc161710 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc0d53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc1f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc0d53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc1f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58366/0 (socket says 172.21.15.60:58366) 2024-04-25T08:38:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.967+0000 7f5bc0d53640 1 -- 172.21.15.60:0/4074351182 learned_addr learned my addr 172.21.15.60:0/4074351182 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc1554640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bbbfff640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bbc15dc20 0x7f5bbc0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc0d53640 1 -- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 msgr2=0x7f5bbc0c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc0d53640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc0d53640 1 -- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bbc15dc20 msgr2=0x7f5bbc0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc0d53640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bbc15dc20 0x7f5bbc0c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc0d53640 1 -- 172.21.15.60:0/4074351182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc15fe50 con 0x7f5bbc0ced80 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bbbfff640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bbc15dc20 0x7f5bbc0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc0d53640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc1f6b60 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400ba80 tx=0x7f5bb400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400cc00 con 0x7f5bbc0ced80 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400cda0 con 0x7f5bbc0ced80 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc160130 con 0x7f5bbc0ced80 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4003bb0 con 0x7f5bbc0ced80 2024-04-25T08:38:08.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.968+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc1606f0 con 0x7f5bbc0ced80 2024-04-25T08:38:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.969+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc0b2ac0 con 0x7f5bbc0ced80 2024-04-25T08:38:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.969+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bb4014440 con 0x7f5bbc0ced80 2024-04-25T08:38:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.970+0000 7f5bb97fa640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.970+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5bb4089ff0 con 0x7f5bbc0ced80 2024-04-25T08:38:08.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.970+0000 7f5bbbfff640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.971+0000 7f5bbbfff640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5bac034000 tx=0x7f5bac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:08.974+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bbc0b2ac0 con 0x7f5bbc0ced80 2024-04-25T08:38:09.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.157+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5bbc073a00 con 0x7f5bbc0ced80 2024-04-25T08:38:09.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.160+0000 7f5bb97fa640 1 -- 172.21.15.60:0/4074351182 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5bb4058fe0 con 0x7f5bbc0ced80 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5b9c06b4d0 msgr2=0x7f5b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5bac034000 tx=0x7f5bac02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 msgr2=0x7f5bbc1f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc1f6b60 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400ba80 tx=0x7f5bb400bf40 comp rx=0 tx=0).stop 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 shutdown_connections 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5bbc161710 0x7f5bbc0c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5bbc15dc20 0x7f5bbc0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 --2- 172.21.15.60:0/4074351182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bbc0ced80 0x7f5bbc1f6b60 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 >> 172.21.15.60:0/4074351182 conn(0x7f5bbc0d0420 msgr2=0x7f5bbc0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 shutdown_connections 2024-04-25T08:38:09.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.162+0000 7f5bc2fb7640 1 -- 172.21.15.60:0/4074351182 wait complete. 2024-04-25T08:38:09.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T08:38:09.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.274+0000 7f418eb9e640 1 Processor -- start 2024-04-25T08:38:09.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.274+0000 7f418eb9e640 1 -- start start 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f418eb9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 0x7f41880b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f418eb9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41880ceee0 con 0x7f41880b5a60 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f418eb9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41880d1d10 con 0x7f418815db70 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f418eb9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41880d1ff0 con 0x7f41880b6cb0 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f418d13b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f41880b5a60 0x7f41880d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50694/0 (socket says 172.21.15.60:50694) 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f418d13b640 1 -- 172.21.15.60:0/788573362 learned_addr learned my addr 172.21.15.60:0/788573362 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f4187fff640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 0x7f41880b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618359066 0 0) 0x7f41880ceee0 con 0x7f41880b5a60 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4178003520 con 0x7f41880b5a60 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964430979 0 0) 0x7f41880d1d10 con 0x7f418815db70 2024-04-25T08:38:09.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41880ceee0 con 0x7f418815db70 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191700705 0 0) 0x7f4178003520 con 0x7f41880b5a60 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 msgr2=0x7f41880b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 0x7f41880b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 >> v1:172.21.15.100:6789/0 conn(0x7f418815db70 legacy=0x7f4188162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41880d22d0 con 0x7f41880b5a60 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.275+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f41800019f0 con 0x7f41880b5a60 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 >> v1:172.21.15.60:6789/0 conn(0x7f41880b5a60 legacy=0x7f41880d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 0x7f4178003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41780040d0 0x7f4178004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4178004d40 0x7f41780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41880d1ff0 con 0x7f41780040d0 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417800baa0 con 0x7f4178004d40 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417800bca0 con 0x7f41880b6cb0 2024-04-25T08:38:09.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f4187fff640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41780040d0 0x7f4178004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f418c93a640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 0x7f4178003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f418d13b640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4178004d40 0x7f41780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f4187fff640 1 -- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 msgr2=0x7f4178003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f4187fff640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f41880b6cb0 0x7f4178003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f4187fff640 1 -- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4178004d40 msgr2=0x7f41780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f4187fff640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4178004d40 0x7f41780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f4187fff640 1 -- 172.21.15.60:0/788573362 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4178003520 con 0x7f41780040d0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.276+0000 7f418d13b640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4178004d40 0x7f41780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f4187fff640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41780040d0 0x7f4178004800 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4174009a90 tx=0x7f4178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c94734759e79a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174036070 con 0x7f41780040d0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4174031400 con 0x7f41780040d0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f41877fe640 1 -- 172.21.15.60:0/788573362 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174009690 con 0x7f41780040d0 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f418eb9e640 1 -- 172.21.15.60:0/788573362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41780040d0 msgr2=0x7f4178004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f418eb9e640 1 --2- 172.21.15.60:0/788573362 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41780040d0 0x7f4178004800 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4174009a90 tx=0x7f4178005830 comp rx=0 tx=0).stop 2024-04-25T08:38:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f418d13b640 1 -- 172.21.15.60:0/788573362 reap_dead start 2024-04-25T08:38:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f418eb9e640 1 -- 172.21.15.60:0/788573362 shutdown_connections 2024-04-25T08:38:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f418eb9e640 1 -- 172.21.15.60:0/788573362 >> 172.21.15.60:0/788573362 conn(0x7f41880d0520 msgr2=0x7f41880d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.277+0000 7f418eb9e640 1 -- 172.21.15.60:0/788573362 shutdown_connections 2024-04-25T08:38:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.278+0000 7f418eb9e640 1 -- 172.21.15.60:0/788573362 wait complete. 2024-04-25T08:38:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.279+0000 7f418eb9e640 1 Processor -- start 2024-04-25T08:38:09.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.279+0000 7f418eb9e640 1 -- start start 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41780040d0 0x7f4188158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 0x7f4188160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f418815e160 0x7f418815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41880cf1c0 con 0x7f41881602e0 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41880ce530 con 0x7f41780040d0 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41880cf340 con 0x7f418815e160 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 0x7f4188160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 0x7f4188160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58380/0 (socket says 172.21.15.60:58380) 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 -- 172.21.15.60:0/237995185 learned_addr learned my addr 172.21.15.60:0/237995185 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418c93a640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41780040d0 0x7f4188158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418d13b640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f418815e160 0x7f418815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 -- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f418815e160 msgr2=0x7f418815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f418815e160 0x7f418815e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 -- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41780040d0 msgr2=0x7f4188158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41780040d0 0x7f4188158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 -- 172.21.15.60:0/237995185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188158da0 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f4187fff640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 0x7f4188160760 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4174000c00 tx=0x7f4174004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174040020 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4174003c60 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4174036040 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.280+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418815ee70 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.281+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418815f430 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.281+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4188073990 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.282+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4174043020 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.282+0000 7f41857fa640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f416c06b4d0 0x7f416c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.282+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f41740a9fc0 con 0x7f41881602e0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.283+0000 7f418c93a640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f416c06b4d0 0x7f416c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.279 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.283+0000 7f418c93a640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f416c06b4d0 0x7f416c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f41880cde70 tx=0x7f417c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.287+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f417404a070 con 0x7f41881602e0 2024-04-25T08:38:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.581+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f41880d1820 con 0x7f41881602e0 2024-04-25T08:38:09.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.831+0000 7f41857fa640 1 -- 172.21.15.60:0/237995185 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v437) v1 ==== 166+0+0 (secure 0 0 0) 0x7f4188158da0 con 0x7f41881602e0 2024-04-25T08:38:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f416c06b4d0 msgr2=0x7f416c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f416c06b4d0 0x7f416c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f41880cde70 tx=0x7f417c009040 comp rx=0 tx=0).stop 2024-04-25T08:38:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 msgr2=0x7f4188160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 0x7f4188160760 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4174000c00 tx=0x7f4174004730 comp rx=0 tx=0).stop 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 shutdown_connections 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f418815e160 0x7f418815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f41881602e0 0x7f4188160760 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f416c06b4d0 0x7f416c06d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 --2- 172.21.15.60:0/237995185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41780040d0 0x7f4188158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 >> 172.21.15.60:0/237995185 conn(0x7f41880d0520 msgr2=0x7f41880c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 shutdown_connections 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.833+0000 7f418eb9e640 1 -- 172.21.15.60:0/237995185 wait complete. 2024-04-25T08:38:09.839 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:38:09.839 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:38:09.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:09.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.947+0000 7f223858b640 1 Processor -- start 2024-04-25T08:38:09.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.947+0000 7f223858b640 1 -- start start 2024-04-25T08:38:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f223858b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f22300d2e90 0x7f22300d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f223858b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 0x7f22300b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f223858b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300b9940 con 0x7f223015dd30 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f223858b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300d0d40 con 0x7f22300d2e90 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f223858b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300d1df0 con 0x7f22300d0900 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2236327640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f223015dd30 0x7f22300b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50702/0 (socket says 172.21.15.60:50702) 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2236327640 1 -- 172.21.15.60:0/323189158 learned_addr learned my addr 172.21.15.60:0/323189158 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235b26640 1 --2- 172.21.15.60:0/323189158 >> v2:172.21.15.100:3300/0 conn(0x7f22300d2e90 0x7f22300d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2236b28640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 0x7f22300b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367980673 0 0) 0x7f22300b9940 con 0x7f223015dd30 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f221c003520 con 0x7f223015dd30 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2668166381 0 0) 0x7f221c003520 con 0x7f223015dd30 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 msgr2=0x7f22300b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 0x7f22300b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 >> v2:172.21.15.100:3300/0 conn(0x7f22300d2e90 msgr2=0x7f22300d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 --2- 172.21.15.60:0/323189158 >> v2:172.21.15.100:3300/0 conn(0x7f22300d2e90 0x7f22300d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300d20d0 con 0x7f223015dd30 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f222c001670 con 0x7f223015dd30 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 >> v1:172.21.15.60:6789/0 conn(0x7f223015dd30 legacy=0x7f22300b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 0x7f221c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c003f30 0x7f221c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d40 0x7f221c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300d0d40 con 0x7f221c004d40 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300d1df0 con 0x7f221c003f30 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c00ba10 con 0x7f22300d0900 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2236b28640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d40 0x7f221c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.948+0000 7f2236327640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 0x7f221c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2235b26640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c003f30 0x7f221c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2236b28640 1 -- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 msgr2=0x7f221c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2236b28640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300d0900 0x7f221c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2236b28640 1 -- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c003f30 msgr2=0x7f221c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2236b28640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c003f30 0x7f221c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2236b28640 1 -- 172.21.15.60:0/323189158 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c003520 con 0x7f221c004d40 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2235b26640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c003f30 0x7f221c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.949+0000 7f2236b28640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d40 0x7f221c0051c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f222800e170 tx=0x7f221c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc80c9e2676a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228034020 con 0x7f221c004d40 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22280305f0 con 0x7f221c004d40 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f2235325640 1 -- 172.21.15.60:0/323189158 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228009c10 con 0x7f221c004d40 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f223858b640 1 -- 172.21.15.60:0/323189158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d40 msgr2=0x7f221c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f223858b640 1 --2- 172.21.15.60:0/323189158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221c004d40 0x7f221c0051c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f222800e170 tx=0x7f221c005830 comp rx=0 tx=0).stop 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f2236b28640 1 -- 172.21.15.60:0/323189158 reap_dead start 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f223858b640 1 -- 172.21.15.60:0/323189158 shutdown_connections 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f223858b640 1 -- 172.21.15.60:0/323189158 >> 172.21.15.60:0/323189158 conn(0x7f2230061c30 msgr2=0x7f223006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f223858b640 1 -- 172.21.15.60:0/323189158 shutdown_connections 2024-04-25T08:38:09.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.950+0000 7f223858b640 1 -- 172.21.15.60:0/323189158 wait complete. 2024-04-25T08:38:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.951+0000 7f223858b640 1 Processor -- start 2024-04-25T08:38:09.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.951+0000 7f223858b640 1 -- start start 2024-04-25T08:38:09.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f223858b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c004d40 0x7f2230149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f223858b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 0x7f22300caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f223858b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300cafc0 0x7f2230160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f223858b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300d2970 con 0x7f22301f3810 2024-04-25T08:38:09.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f223858b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300d2690 con 0x7f221c004d40 2024-04-25T08:38:09.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f223858b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300d1a10 con 0x7f22300cafc0 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f2235b26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 0x7f22300caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f2235b26640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 0x7f22300caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58400/0 (socket says 172.21.15.60:58400) 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f2235b26640 1 -- 172.21.15.60:0/605712774 learned_addr learned my addr 172.21.15.60:0/605712774 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f2236b28640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300cafc0 0x7f2230160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.952+0000 7f2236327640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c004d40 0x7f2230149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2235b26640 1 -- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300cafc0 msgr2=0x7f2230160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2235b26640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300cafc0 0x7f2230160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2235b26640 1 -- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c004d40 msgr2=0x7f2230149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2235b26640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c004d40 0x7f2230149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2235b26640 1 -- 172.21.15.60:0/605712774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300b3f70 con 0x7f22301f3810 2024-04-25T08:38:09.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2236327640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c004d40 0x7f2230149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2235b26640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 0x7f22300caa80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2220011600 tx=0x7f2220011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222000f8c0 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222000fa60 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22300b4250 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222000fe50 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.953+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22300b4810 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.954+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230073990 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.955+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2220015460 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.955+0000 7f2226ffd640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f221406b4d0 0x7f221406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.955+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f222008a8e0 con 0x7f22301f3810 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.955+0000 7f2236327640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f221406b4d0 0x7f221406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.956+0000 7f2236327640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f221406b4d0 0x7f221406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f222c001050 tx=0x7f222c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:09.959+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2220019030 con 0x7f22301f3810 2024-04-25T08:38:10.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.140+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22300cb3a0 con 0x7f22301f3810 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.141+0000 7f2226ffd640 1 -- 172.21.15.60:0/605712774 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+13293 (secure 0 0 0) 0x7f2220059960 con 0x7f22301f3810 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":126,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:38:08.825318+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6837/2648832765":"2024-04-26T08:38:05.820258+0000","172.21.15.100:6835/2648832765":"2024-04-26T08:38:05.820258+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f221406b4d0 msgr2=0x7f221406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f221406b4d0 0x7f221406d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f222c001050 tx=0x7f222c009040 comp rx=0 tx=0).stop 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 msgr2=0x7f22300caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 0x7f22300caa80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2220011600 tx=0x7f2220011ac0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 shutdown_connections 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22300cafc0 0x7f2230160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22301f3810 0x7f22300caa80 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f221406b4d0 0x7f221406d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 --2- 172.21.15.60:0/605712774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221c004d40 0x7f2230149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 >> 172.21.15.60:0/605712774 conn(0x7f2230061c30 msgr2=0x7f22300c7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 shutdown_connections 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.143+0000 7f223858b640 1 -- 172.21.15.60:0/605712774 wait complete. 2024-04-25T08:38:10.148 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:10.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:10.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.259+0000 7f25bf493640 1 Processor -- start 2024-04-25T08:38:10.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.259+0000 7f25bf493640 1 -- start start 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bf493640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25b80b38d0 0x7f25b80b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bf493640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b815e020 con 0x7f25b806e360 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bf493640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b806ac80 con 0x7f25b806bd20 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bf493640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b806b890 con 0x7f25b80b38d0 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bca2e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f25b80b38d0 0x7f25b80b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bd22f640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f25b806e360 0x7f25b806e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50708/0 (socket says 172.21.15.60:50708) 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bd22f640 1 -- 172.21.15.60:0/1588822638 learned_addr learned my addr 172.21.15.60:0/1588822638 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bca2e640 1 -- 172.21.15.60:0/1588822638 >> v1:172.21.15.100:6789/0 conn(0x7f25b806bd20 legacy=0x7f25b80acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bca2e640 1 -- 172.21.15.60:0/1588822638 >> v1:172.21.15.60:6789/0 conn(0x7f25b806e360 legacy=0x7f25b806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.260+0000 7f25bca2e640 1 -- 172.21.15.60:0/1588822638 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b806af60 con 0x7f25b80b38d0 2024-04-25T08:38:10.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.261+0000 7f25bca2e640 1 --2- 172.21.15.60:0/1588822638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b3d10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0097a0 tx=0x7f25ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185856fdbe0d7a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.261+0000 7f25b7fff640 1 -- 172.21.15.60:0/1588822638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac037070 con 0x7f25b80b38d0 2024-04-25T08:38:10.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.261+0000 7f25b7fff640 1 -- 172.21.15.60:0/1588822638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac032440 con 0x7f25b80b38d0 2024-04-25T08:38:10.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.261+0000 7f25b7fff640 1 -- 172.21.15.60:0/1588822638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac032810 con 0x7f25b80b38d0 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 -- 172.21.15.60:0/1588822638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 msgr2=0x7f25b80b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 --2- 172.21.15.60:0/1588822638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b3d10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0097a0 tx=0x7f25ac02a270 comp rx=0 tx=0).stop 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 -- 172.21.15.60:0/1588822638 shutdown_connections 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 --2- 172.21.15.60:0/1588822638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b3d10 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 -- 172.21.15.60:0/1588822638 >> 172.21.15.60:0/1588822638 conn(0x7f25b80cfec0 msgr2=0x7f25b80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 -- 172.21.15.60:0/1588822638 shutdown_connections 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.262+0000 7f25bf493640 1 -- 172.21.15.60:0/1588822638 wait complete. 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.263+0000 7f25bf493640 1 Processor -- start 2024-04-25T08:38:10.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.263+0000 7f25bf493640 1 -- start start 2024-04-25T08:38:10.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bf493640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25b806bd20 0x7f25b814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bf493640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b806e360 0x7f25b80affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bf493640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bf493640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b806ee50 con 0x7f25b80b38d0 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bf493640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b806b520 con 0x7f25b806e360 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bf493640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80b2fd0 con 0x7f25b806bd20 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58414/0 (socket says 172.21.15.60:58414) 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 -- 172.21.15.60:0/4214807767 learned_addr learned my addr 172.21.15.60:0/4214807767 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bd22f640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25b806bd20 0x7f25b814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bca2e640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b806e360 0x7f25b80affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 -- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25b806bd20 msgr2=0x7f25b814d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25b806bd20 0x7f25b814d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 -- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b806e360 msgr2=0x7f25b80affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b806e360 0x7f25b80affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bda30640 1 -- 172.21.15.60:0/4214807767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80ae190 con 0x7f25b80b38d0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bd22f640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25b806bd20 0x7f25b814d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.264+0000 7f25bca2e640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b806e360 0x7f25b80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.265+0000 7f25bda30640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b0520 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b440 tx=0x7f25b000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.265+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0015020 con 0x7f25b80b38d0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.265+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b0011830 con 0x7f25b80b38d0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.265+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b80ae470 con 0x7f25b80b38d0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.265+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0011c20 con 0x7f25b80b38d0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.265+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b80aea30 con 0x7f25b80b38d0 2024-04-25T08:38:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.266+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8073a00 con 0x7f25b80b38d0 2024-04-25T08:38:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.266+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25b0018020 con 0x7f25b80b38d0 2024-04-25T08:38:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.267+0000 7f25b5ffb640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.267+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f25b0089c40 con 0x7f25b80b38d0 2024-04-25T08:38:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.267+0000 7f25bd22f640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.268+0000 7f25bd22f640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f25a80012e0 tx=0x7f25a8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.271+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25b001f030 con 0x7f25b80b38d0 2024-04-25T08:38:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.453+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25b80cc450 con 0x7f25b80b38d0 2024-04-25T08:38:10.447 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.453+0000 7f25b5ffb640 1 -- 172.21.15.60:0/4214807767 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25b00172c0 con 0x7f25b80b38d0 2024-04-25T08:38:10.447 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 msgr2=0x7f25a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f25a80012e0 tx=0x7f25a8009040 comp rx=0 tx=0).stop 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 msgr2=0x7f25b80b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b0520 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b440 tx=0x7f25b000b900 comp rx=0 tx=0).stop 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 shutdown_connections 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25b80b38d0 0x7f25b80b0520 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25b806e360 0x7f25b80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.455+0000 7f25bf493640 1 --2- 172.21.15.60:0/4214807767 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f25b806bd20 0x7f25b814d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.456+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 >> 172.21.15.60:0/4214807767 conn(0x7f25b80cfec0 msgr2=0x7f25b806efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.456+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 shutdown_connections 2024-04-25T08:38:10.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.456+0000 7f25bf493640 1 -- 172.21.15.60:0/4214807767 wait complete. 2024-04-25T08:38:10.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T08:38:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.568+0000 7f4597bf9640 1 Processor -- start 2024-04-25T08:38:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.568+0000 7f4597bf9640 1 -- start start 2024-04-25T08:38:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4597bf9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 0x7f4590162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4597bf9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4590162690 con 0x7f45900b5c30 2024-04-25T08:38:10.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4597bf9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45900ceee0 con 0x7f45900b6cb0 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4597bf9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45900d1d10 con 0x7f459015db70 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4596196640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f45900b5c30 0x7f45900d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50720/0 (socket says 172.21.15.60:50720) 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4596196640 1 -- 172.21.15.60:0/556092161 learned_addr learned my addr 172.21.15.60:0/556092161 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4595995640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 0x7f4590162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302823807 0 0) 0x7f4590162690 con 0x7f45900b5c30 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f457c003520 con 0x7f45900b5c30 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1349575751 0 0) 0x7f457c003520 con 0x7f45900b5c30 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 msgr2=0x7f4590162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 0x7f4590162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 >> v1:172.21.15.100:6789/0 conn(0x7f45900b6cb0 legacy=0x7f45900b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.566 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45900d1ff0 con 0x7f45900b5c30 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f45880019f0 con 0x7f45900b5c30 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 >> v1:172.21.15.60:6789/0 conn(0x7f45900b5c30 legacy=0x7f45900d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 0x7f457c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f457c003f60 0x7f457c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f457c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45900d1d10 con 0x7f457c004d40 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c00bb50 con 0x7f457c003f60 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.569+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457c00bd50 con 0x7f459015db70 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4596196640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f457c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4595995640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 0x7f457c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4595194640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f457c003f60 0x7f457c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4596196640 1 -- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 msgr2=0x7f457c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4596196640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f459015db70 0x7f457c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4596196640 1 -- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f457c003f60 msgr2=0x7f457c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4596196640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f457c003f60 0x7f457c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.570+0000 7f4596196640 1 -- 172.21.15.60:0/556092161 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c003520 con 0x7f457c004d40 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4596196640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f457c0051c0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f458800a810 tx=0x7f457c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec5268427b612bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800a460 con 0x7f457c004d40 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458802f070 con 0x7f457c004d40 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4594993640 1 -- 172.21.15.60:0/556092161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458802bc90 con 0x7f457c004d40 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4597bf9640 1 -- 172.21.15.60:0/556092161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 msgr2=0x7f457c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4597bf9640 1 --2- 172.21.15.60:0/556092161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f457c0051c0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f458800a810 tx=0x7f457c005830 comp rx=0 tx=0).stop 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4596196640 1 -- 172.21.15.60:0/556092161 reap_dead start 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4597bf9640 1 -- 172.21.15.60:0/556092161 shutdown_connections 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4597bf9640 1 -- 172.21.15.60:0/556092161 >> 172.21.15.60:0/556092161 conn(0x7f45900d0520 msgr2=0x7f45900d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4597bf9640 1 -- 172.21.15.60:0/556092161 shutdown_connections 2024-04-25T08:38:10.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.571+0000 7f4597bf9640 1 -- 172.21.15.60:0/556092161 wait complete. 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 Processor -- start 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 -- start start 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f45901f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590146900 0x7f4590143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4590144130 0x7f4590160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45900ce530 con 0x7f457c004d40 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459015df50 con 0x7f4590146900 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4597bf9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45900cf1c0 con 0x7f4590144130 2024-04-25T08:38:10.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4595995640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f45901f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4595995640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f45901f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:58430/0 (socket says 172.21.15.60:58430) 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.573+0000 7f4595995640 1 -- 172.21.15.60:0/1218972058 learned_addr learned my addr 172.21.15.60:0/1218972058 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4596196640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4590144130 0x7f4590160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595194640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590146900 0x7f4590143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595995640 1 -- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4590144130 msgr2=0x7f4590160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595995640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4590144130 0x7f4590160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595995640 1 -- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590146900 msgr2=0x7f4590143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595995640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590146900 0x7f4590143bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595995640 1 -- 172.21.15.60:0/1218972058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459015f9a0 con 0x7f457c004d40 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4596196640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4590144130 0x7f4590160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595194640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590146900 0x7f4590143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4595995640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f45901f6750 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f458c0119a0 tx=0x7f458c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c00faf0 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c00fc90 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c014440 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459015fc80 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.574+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590160240 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.575+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4590073990 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.576+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f458c012040 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.576+0000 7f45867fc640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f457406b4d0 0x7f457406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.576+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f458c08a6d0 con 0x7f457c004d40 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.576+0000 7f4595194640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f457406b4d0 0x7f457406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.577+0000 7f4595194640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f457406b4d0 0x7f457406d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4590161320 tx=0x7f4580009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.580+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f458c019030 con 0x7f457c004d40 2024-04-25T08:38:10.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:10.833+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f45900d1820 con 0x7f457c004d40 2024-04-25T08:38:11.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.836+0000 7f45867fc640 1 -- 172.21.15.60:0/1218972058 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v439) v1 ==== 169+0+0 (secure 0 0 0) 0x7f458c059100 con 0x7f457c004d40 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f457406b4d0 msgr2=0x7f457406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f457406b4d0 0x7f457406d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4590161320 tx=0x7f4580009040 comp rx=0 tx=0).stop 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 msgr2=0x7f45901f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f45901f6750 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f458c0119a0 tx=0x7f458c011e60 comp rx=0 tx=0).stop 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 shutdown_connections 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4590144130 0x7f4590160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f457406b4d0 0x7f457406d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4590146900 0x7f4590143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 --2- 172.21.15.60:0/1218972058 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f457c004d40 0x7f45901f6750 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 >> 172.21.15.60:0/1218972058 conn(0x7f45900d0520 msgr2=0x7f459015b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.839+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 shutdown_connections 2024-04-25T08:38:11.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.840+0000 7f4597bf9640 1 -- 172.21.15.60:0/1218972058 wait complete. 2024-04-25T08:38:11.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:38:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.965+0000 7f5cae513640 1 Processor -- start 2024-04-25T08:38:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.965+0000 7f5cae513640 1 -- start start 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cae513640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ca815dc20 0x7f5ca80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cae513640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ca8161710 0x7f5ca80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cae513640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca80b9330 0x7f5ca806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cae513640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca815e000 con 0x7f5ca815dc20 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cae513640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca80d1de0 con 0x7f5ca8161710 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cae513640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8161c60 con 0x7f5ca80b9330 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ca815dc20 0x7f5ca80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5ca815dc20 0x7f5ca80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59104/0 (socket says 172.21.15.60:59104) 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 -- 172.21.15.60:0/103570733 learned_addr learned my addr 172.21.15.60:0/103570733 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5cacab0640 1 --2- 172.21.15.60:0/103570733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca80b9330 0x7f5ca806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca77fe640 1 --2- 172.21.15.60:0/103570733 >> v2:172.21.15.100:3300/0 conn(0x7f5ca8161710 0x7f5ca80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 -- 172.21.15.60:0/103570733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca80b9330 msgr2=0x7f5ca806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 --2- 172.21.15.60:0/103570733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca80b9330 0x7f5ca806bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 -- 172.21.15.60:0/103570733 >> v2:172.21.15.100:3300/0 conn(0x7f5ca8161710 msgr2=0x7f5ca80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 --2- 172.21.15.60:0/103570733 >> v2:172.21.15.100:3300/0 conn(0x7f5ca8161710 0x7f5ca80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.966+0000 7f5ca7fff640 1 -- 172.21.15.60:0/103570733 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8161f40 con 0x7f5ca815dc20 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.967+0000 7f5ca7fff640 1 --2- 172.21.15.60:0/103570733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca815dc20 0x7f5ca80ce670 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f5ca806da90 tx=0x7f5c9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554a9e8756b287c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.967+0000 7f5ca6ffd640 1 -- 172.21.15.60:0/103570733 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98037070 con 0x7f5ca815dc20 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.967+0000 7f5ca6ffd640 1 -- 172.21.15.60:0/103570733 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c98032440 con 0x7f5ca815dc20 2024-04-25T08:38:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.967+0000 7f5ca6ffd640 1 -- 172.21.15.60:0/103570733 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98032810 con 0x7f5ca815dc20 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.967+0000 7f5cae513640 1 -- 172.21.15.60:0/103570733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca815dc20 msgr2=0x7f5ca80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.967+0000 7f5cae513640 1 --2- 172.21.15.60:0/103570733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca815dc20 0x7f5ca80ce670 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f5ca806da90 tx=0x7f5c9802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 -- 172.21.15.60:0/103570733 shutdown_connections 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 --2- 172.21.15.60:0/103570733 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca80b9330 0x7f5ca806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 --2- 172.21.15.60:0/103570733 >> v2:172.21.15.100:3300/0 conn(0x7f5ca8161710 0x7f5ca80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 --2- 172.21.15.60:0/103570733 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca815dc20 0x7f5ca80ce670 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 -- 172.21.15.60:0/103570733 >> 172.21.15.60:0/103570733 conn(0x7f5ca80d0420 msgr2=0x7f5ca80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 -- 172.21.15.60:0/103570733 shutdown_connections 2024-04-25T08:38:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.968+0000 7f5cae513640 1 -- 172.21.15.60:0/103570733 wait complete. 2024-04-25T08:38:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.969+0000 7f5cae513640 1 Processor -- start 2024-04-25T08:38:11.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.969+0000 7f5cae513640 1 -- start start 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cae513640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca80b9330 0x7f5ca80cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cae513640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca815dc20 0x7f5ca81f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cae513640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 0x7f5ca81f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cae513640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80b5e50 con 0x7f5ca8161710 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cae513640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca8162500 con 0x7f5ca80b9330 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cae513640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca80b5fd0 con 0x7f5ca815dc20 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 0x7f5ca81f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 0x7f5ca81f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59120/0 (socket says 172.21.15.60:59120) 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 -- 172.21.15.60:0/3269302354 learned_addr learned my addr 172.21.15.60:0/3269302354 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5ca77fe640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca815dc20 0x7f5ca81f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5ca7fff640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca80b9330 0x7f5ca80cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 -- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca815dc20 msgr2=0x7f5ca81f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca815dc20 0x7f5ca81f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 -- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca80b9330 msgr2=0x7f5ca80cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca80b9330 0x7f5ca80cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 -- 172.21.15.60:0/3269302354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8160b90 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.970+0000 7f5cacab0640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 0x7f5ca81f3dd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c970 tx=0x7f5ca000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.971+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0015070 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.971+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca0003b90 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.971+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca80cd980 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.971+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000d040 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.971+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca81602f0 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.972+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca8073a00 con 0x7f5ca8161710 2024-04-25T08:38:11.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.972+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ca000a790 con 0x7f5ca8161710 2024-04-25T08:38:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.972+0000 7f5ca4ff9640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c9406b4d0 0x7f5c9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.973+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5ca0089de0 con 0x7f5ca8161710 2024-04-25T08:38:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.973+0000 7f5ca7fff640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c9406b4d0 0x7f5c9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.973+0000 7f5ca7fff640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c9406b4d0 0x7f5c9406d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5c98034000 tx=0x7f5c9802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:11.977+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ca0058810 con 0x7f5ca8161710 2024-04-25T08:38:12.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.157+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5ca80d16b0 con 0x7f5ca8161710 2024-04-25T08:38:12.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.160+0000 7f5ca4ff9640 1 -- 172.21.15.60:0/3269302354 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5ca005c8b0 con 0x7f5ca8161710 2024-04-25T08:38:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c9406b4d0 msgr2=0x7f5c9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c9406b4d0 0x7f5c9406d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5c98034000 tx=0x7f5c9802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 msgr2=0x7f5ca81f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 0x7f5ca81f3dd0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c970 tx=0x7f5ca000ce30 comp rx=0 tx=0).stop 2024-04-25T08:38:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 shutdown_connections 2024-04-25T08:38:12.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ca8161710 0x7f5ca81f3dd0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5ca815dc20 0x7f5ca81f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5c9406b4d0 0x7f5c9406d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 --2- 172.21.15.60:0/3269302354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca80b9330 0x7f5ca80cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 >> 172.21.15.60:0/3269302354 conn(0x7f5ca80d0420 msgr2=0x7f5ca80c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 shutdown_connections 2024-04-25T08:38:12.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.162+0000 7f5cae513640 1 -- 172.21.15.60:0/3269302354 wait complete. 2024-04-25T08:38:12.167 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:38:12.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:12.202 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:12.257 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:38:12.257 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:38:12.257 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:12.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:12.273 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:38:12.273 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:12.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:12.291 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:38:12.291 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:12.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:12.329 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:38:12.329 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:12.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:12.348 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:38:12.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.476+0000 7fa6c0de7640 1 Processor -- start 2024-04-25T08:38:12.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.476+0000 7fa6c0de7640 1 -- start start 2024-04-25T08:38:12.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.477+0000 7fa6c0de7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 0x7fa6bc0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.477+0000 7fa6c0de7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa6bc06ce10 0x7fa6bc0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.477+0000 7fa6c0de7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6bc06c220 con 0x7fa6bc09a450 2024-04-25T08:38:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.477+0000 7fa6c0de7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6bc09a8b0 con 0x7fa6bc06ce10 2024-04-25T08:38:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.477+0000 7fa6c0de7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0d0c90 con 0x7fa6bc15dc70 2024-04-25T08:38:12.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 0x7fa6bc0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6bad9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa6bc06ce10 0x7fa6bc0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 0x7fa6bc0d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54016/0 (socket says 172.21.15.60:54016) 2024-04-25T08:38:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 -- 172.21.15.60:0/4132652781 learned_addr learned my addr 172.21.15.60:0/4132652781 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6b9d9b640 1 --1- 172.21.15.60:0/4132652781 >> v1:172.21.15.100:6789/0 conn(0x7fa6bc09a450 0x7fa6bc06c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59794/0 (socket says 172.21.15.60:59794) 2024-04-25T08:38:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 -- 172.21.15.60:0/4132652781 >> v2:172.21.15.100:3300/0 conn(0x7fa6bc06ce10 msgr2=0x7fa6bc0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 --2- 172.21.15.60:0/4132652781 >> v2:172.21.15.100:3300/0 conn(0x7fa6bc06ce10 0x7fa6bc0cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 -- 172.21.15.60:0/4132652781 >> v1:172.21.15.100:6789/0 conn(0x7fa6bc09a450 legacy=0x7fa6bc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.478+0000 7fa6ba59c640 1 -- 172.21.15.60:0/4132652781 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0d0080 con 0x7fa6bc15dc70 2024-04-25T08:38:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6ba59c640 1 --2- 172.21.15.60:0/4132652781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 0x7fa6bc0d1120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009870 tx=0x7fa6b402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a6b42f046572ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6b959a640 1 -- 172.21.15.60:0/4132652781 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4037070 con 0x7fa6bc15dc70 2024-04-25T08:38:12.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6b959a640 1 -- 172.21.15.60:0/4132652781 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4032440 con 0x7fa6bc15dc70 2024-04-25T08:38:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6b959a640 1 -- 172.21.15.60:0/4132652781 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4032810 con 0x7fa6bc15dc70 2024-04-25T08:38:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 -- 172.21.15.60:0/4132652781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 msgr2=0x7fa6bc0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/4132652781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 0x7fa6bc0d1120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009870 tx=0x7fa6b402a270 comp rx=0 tx=0).stop 2024-04-25T08:38:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 -- 172.21.15.60:0/4132652781 shutdown_connections 2024-04-25T08:38:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/4132652781 >> v2:172.21.15.100:3300/0 conn(0x7fa6bc06ce10 0x7fa6bc0cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/4132652781 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc15dc70 0x7fa6bc0d1120 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 -- 172.21.15.60:0/4132652781 >> 172.21.15.60:0/4132652781 conn(0x7fa6bc0b3270 msgr2=0x7fa6bc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 -- 172.21.15.60:0/4132652781 shutdown_connections 2024-04-25T08:38:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.479+0000 7fa6c0de7640 1 -- 172.21.15.60:0/4132652781 wait complete. 2024-04-25T08:38:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 Processor -- start 2024-04-25T08:38:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 -- start start 2024-04-25T08:38:12.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc06ce10 0x7fa6bc14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 0x7fa6bc0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6bc15dc70 0x7fa6bc0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0d0920 con 0x7fa6bc09a450 2024-04-25T08:38:12.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc0d0640 con 0x7fa6bc15dc70 2024-04-25T08:38:12.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.481+0000 7fa6c0de7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc15e050 con 0x7fa6bc06ce10 2024-04-25T08:38:12.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 0x7fa6bc0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 0x7fa6bc0c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59124/0 (socket says 172.21.15.60:59124) 2024-04-25T08:38:12.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 -- 172.21.15.60:0/2234330525 learned_addr learned my addr 172.21.15.60:0/2234330525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:12.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6ba59c640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc06ce10 0x7fa6bc14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6bad9d640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6bc15dc70 0x7fa6bc0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 -- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc06ce10 msgr2=0x7fa6bc14cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc06ce10 0x7fa6bc14cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 -- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6bc15dc70 msgr2=0x7fa6bc0c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6bc15dc70 0x7fa6bc0c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 -- 172.21.15.60:0/2234330525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc1e8c70 con 0x7fa6bc09a450 2024-04-25T08:38:12.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b9d9b640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 0x7fa6bc0c6600 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c400 tx=0x7fa6a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40074e0 con 0x7fa6bc09a450 2024-04-25T08:38:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a4007680 con 0x7fa6bc09a450 2024-04-25T08:38:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc1e80f0 con 0x7fa6bc09a450 2024-04-25T08:38:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4004a00 con 0x7fa6bc09a450 2024-04-25T08:38:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.482+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc1e86b0 con 0x7fa6bc09a450 2024-04-25T08:38:12.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.483+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc073990 con 0x7fa6bc09a450 2024-04-25T08:38:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.484+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6a401a020 con 0x7fa6bc09a450 2024-04-25T08:38:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.484+0000 7fa6b2ffd640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa69c06b4d0 0x7fa69c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.484+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa6a40898d0 con 0x7fa6bc09a450 2024-04-25T08:38:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.485+0000 7fa6ba59c640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa69c06b4d0 0x7fa69c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.485+0000 7fa6ba59c640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa69c06b4d0 0x7fa69c06d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4034000 tx=0x7fa6b402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.489+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6a4058820 con 0x7fa6bc09a450 2024-04-25T08:38:12.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.710+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6bc0b5b70 con 0x7fa6bc09a450 2024-04-25T08:38:12.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.711+0000 7fa6b2ffd640 1 -- 172.21.15.60:0/2234330525 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fa6a4058120 con 0x7fa6bc09a450 2024-04-25T08:38:12.705 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:12.705 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6836/3874211946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":3874211946},{"type":"v1","addr":"172.21.15.100:6836","nonce":3874211946}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7792,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1222403636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":1222403636},{"type":"v1","addr":"172.21.15.60:6835","nonce":1222403636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":8182,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1880003188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1880003188},{"type":"v1","addr":"172.21.15.100:6837","nonce":1880003188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":8790,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/3591750657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3591750657},{"type":"v1","addr":"172.21.15.60:6834","nonce":3591750657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2024-04-25T08:38:12.706 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 73 2024-04-25T08:38:12.707 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa69c06b4d0 msgr2=0x7fa69c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa69c06b4d0 0x7fa69c06d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4034000 tx=0x7fa6b402e000 comp rx=0 tx=0).stop 2024-04-25T08:38:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 msgr2=0x7fa6bc0c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 0x7fa6bc0c6600 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c400 tx=0x7fa6a400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 shutdown_connections 2024-04-25T08:38:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa69c06b4d0 0x7fa69c06d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa6bc15dc70 0x7fa6bc0c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa6bc09a450 0x7fa6bc0c6600 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 --2- 172.21.15.60:0/2234330525 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa6bc06ce10 0x7fa6bc14cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 >> 172.21.15.60:0/2234330525 conn(0x7fa6bc0b3270 msgr2=0x7fa6bc0b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 shutdown_connections 2024-04-25T08:38:12.709 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.712+0000 7fa6c0de7640 1 -- 172.21.15.60:0/2234330525 wait complete. 2024-04-25T08:38:12.718 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:38:12.718 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:38:12.718 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:38:12.718 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:12.718 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:38:12.719 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:38:12.719 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:38:12.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:12.719 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:38:12.719 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:38:12.719 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:38:12.719 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:12.719 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:38:12.720 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:38:12.720 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:38:12.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:12.728 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:38:12.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:38:12.729 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:38:12.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:38:12.731 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:38:12.731 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:38:12.732 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:38:12.732 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:38:12.770 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:38:12.770 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:12.771 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:38:12.771 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:12.774 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:38:12.774 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:12.775 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:38:12.775 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:12.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:38:12.846 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:38:12.846 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:38:12.853 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:38:12.854 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:38:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.888+0000 7f1a5da3a640 1 Processor -- start 2024-04-25T08:38:12.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.888+0000 7f1a5da3a640 1 -- start start 2024-04-25T08:38:12.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a5da3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a5da3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a58161710 0x7f1a580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a5da3a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5815e000 con 0x7f1a580b9330 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a4ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a58161710 0x7f1a580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a5da3a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a580d1de0 con 0x7f1a58161710 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a4ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a58161710 0x7f1a580cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59152/0 (socket says 172.21.15.60:59152) 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a4ffff640 1 -- 172.21.15.60:0/1837996342 learned_addr learned my addr 172.21.15.60:0/1837996342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a577fe640 1 --1- 172.21.15.60:0/1837996342 >> v1:172.21.15.60:6789/0 conn(0x7f1a580b9330 0x7f1a5806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38624/0 (socket says 172.21.15.60:38624) 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.889+0000 7f1a5da3a640 1 -- 172.21.15.60:0/1837996342 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a58161c60 con 0x7f1a5815dc20 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a56ffd640 1 --2- 172.21.15.60:0/1837996342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a567fc640 1 -- 172.21.15.60:0/1837996342 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210595511 0 0) 0x7f1a5815e000 con 0x7f1a580b9330 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a567fc640 1 -- 172.21.15.60:0/1837996342 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a44003520 con 0x7f1a580b9330 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a4ffff640 1 -- 172.21.15.60:0/1837996342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 msgr2=0x7f1a580ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a4ffff640 1 --2- 172.21.15.60:0/1837996342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a4ffff640 1 -- 172.21.15.60:0/1837996342 >> v1:172.21.15.60:6789/0 conn(0x7f1a580b9330 legacy=0x7f1a5806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.890+0000 7f1a4ffff640 1 -- 172.21.15.60:0/1837996342 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a58161f40 con 0x7f1a58161710 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a4ffff640 1 --2- 172.21.15.60:0/1837996342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a580cebb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009870 tx=0x7f1a4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64057509b4796016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a567fc640 1 -- 172.21.15.60:0/1837996342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40037070 con 0x7f1a58161710 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a567fc640 1 -- 172.21.15.60:0/1837996342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40032440 con 0x7f1a58161710 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a567fc640 1 -- 172.21.15.60:0/1837996342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a400327f0 con 0x7f1a58161710 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a5da3a640 1 -- 172.21.15.60:0/1837996342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 msgr2=0x7f1a580cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/1837996342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a580cebb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009870 tx=0x7f1a4002a270 comp rx=0 tx=0).stop 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a5da3a640 1 -- 172.21.15.60:0/1837996342 shutdown_connections 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/1837996342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a580cebb0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/1837996342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.891+0000 7f1a5da3a640 1 -- 172.21.15.60:0/1837996342 >> 172.21.15.60:0/1837996342 conn(0x7f1a580d0420 msgr2=0x7f1a580d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.892+0000 7f1a5da3a640 1 -- 172.21.15.60:0/1837996342 shutdown_connections 2024-04-25T08:38:12.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.892+0000 7f1a5da3a640 1 -- 172.21.15.60:0/1837996342 wait complete. 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.893+0000 7f1a5da3a640 1 Processor -- start 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.893+0000 7f1a5da3a640 1 -- start start 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.893+0000 7f1a5da3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a580b9330 0x7f1a581517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.893+0000 7f1a5da3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.893+0000 7f1a5da3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a58159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a5da3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580b5e50 con 0x7f1a58161710 2024-04-25T08:38:12.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a5da3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a58162500 con 0x7f1a580b9330 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a5da3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580b5fd0 con 0x7f1a5815dc20 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a58159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a58159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59160/0 (socket says 172.21.15.60:59160) 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 -- 172.21.15.60:0/3557458171 learned_addr learned my addr 172.21.15.60:0/3557458171 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a4ffff640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a56ffd640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a580b9330 0x7f1a581517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 -- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 msgr2=0x7f1a580cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 -- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a580b9330 msgr2=0x7f1a581517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a580b9330 0x7f1a581517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 -- 172.21.15.60:0/3557458171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580cc6d0 con 0x7f1a58161710 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a577fe640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a58159c60 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c700 tx=0x7f1a5000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50018070 con 0x7f1a58161710 2024-04-25T08:38:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a50004d60 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a580cc9b0 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000a880 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.894+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a580ccf70 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.895+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a58073a00 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.896+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a5001b070 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.896+0000 7f1a54ff9640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a3806b4d0 0x7f1a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.896+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1a50089480 con 0x7f1a58161710 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.896+0000 7f1a56ffd640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a3806b4d0 0x7f1a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.897+0000 7f1a56ffd640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a3806b4d0 0x7f1a3806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009aa0 tx=0x7f1a48009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:12.901+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a50057eb0 con 0x7f1a58161710 2024-04-25T08:38:13.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.094+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1a580d0a80 con 0x7f1a58161710 2024-04-25T08:38:13.841 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.847+0000 7f1a54ff9640 1 -- 172.21.15.60:0/3557458171 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a5005bf50 con 0x7f1a58161710 2024-04-25T08:38:13.841 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:38:13.845 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a3806b4d0 msgr2=0x7f1a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a3806b4d0 0x7f1a3806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1a48009aa0 tx=0x7f1a48009190 comp rx=0 tx=0).stop 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 msgr2=0x7f1a58159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a58159c60 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c700 tx=0x7f1a5000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 shutdown_connections 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a58161710 0x7f1a58159c60 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a5815dc20 0x7f1a580cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a3806b4d0 0x7f1a3806d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 --2- 172.21.15.60:0/3557458171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a580b9330 0x7f1a581517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 >> 172.21.15.60:0/3557458171 conn(0x7f1a580d0420 msgr2=0x7f1a580b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 shutdown_connections 2024-04-25T08:38:13.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.851+0000 7f1a5da3a640 1 -- 172.21.15.60:0/3557458171 wait complete. 2024-04-25T08:38:13.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:38:13.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.968+0000 7f666b8ee640 1 Processor -- start 2024-04-25T08:38:13.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.968+0000 7f666b8ee640 1 -- start start 2024-04-25T08:38:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666b8ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66640d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666b8ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f66640d0900 0x7f66640b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666b8ee640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66640b9940 con 0x7f666415dd30 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666b8ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66640d0d70 con 0x7f66640d0900 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666b8ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640d1df0 con 0x7f66640d2e90 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666968a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f666415dd30 0x7f66640b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38626/0 (socket says 172.21.15.60:38626) 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f666968a640 1 -- 172.21.15.60:0/2988167321 learned_addr learned my addr 172.21.15.60:0/2988167321 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f6669e8b640 1 --2- 172.21.15.60:0/2988167321 >> v2:172.21.15.100:3300/0 conn(0x7f66640d0900 0x7f66640b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f6668e89640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66640d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983423904 0 0) 0x7f66640b9940 con 0x7f666415dd30 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6650003520 con 0x7f666415dd30 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441359576 0 0) 0x7f6650003520 con 0x7f666415dd30 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 msgr2=0x7f66640d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66640d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 >> v2:172.21.15.100:3300/0 conn(0x7f66640d0900 msgr2=0x7f66640b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 --2- 172.21.15.60:0/2988167321 >> v2:172.21.15.100:3300/0 conn(0x7f66640d0900 0x7f66640b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66640d20d0 con 0x7f666415dd30 2024-04-25T08:38:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f6660001670 con 0x7f666415dd30 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 >> v1:172.21.15.60:6789/0 conn(0x7f666415dd30 legacy=0x7f66640b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66500039b0 0x7f6650003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650004800 0x7f6650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640d0d70 con 0x7f66500039b0 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640d1df0 con 0x7f6650004800 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.969+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665000b9c0 con 0x7f66640d2e90 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66500039b0 0x7f6650003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f6668e89640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650004800 0x7f6650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f6669e8b640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 -- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 msgr2=0x7f66500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 -- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650004800 msgr2=0x7f6650004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6650004800 0x7f6650004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 -- 172.21.15.60:0/2988167321 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650003520 con 0x7f66500039b0 2024-04-25T08:38:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f6669e8b640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66640d2e90 0x7f66500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.970+0000 7f666968a640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66500039b0 0x7f6650003dd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f666000c2c0 tx=0x7f6650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5a68b85fb960a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660007d50 con 0x7f66500039b0 2024-04-25T08:38:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666002e070 con 0x7f66500039b0 2024-04-25T08:38:13.967 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f665bfff640 1 -- 172.21.15.60:0/2988167321 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660005d30 con 0x7f66500039b0 2024-04-25T08:38:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f666b8ee640 1 -- 172.21.15.60:0/2988167321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66500039b0 msgr2=0x7f6650003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2988167321 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66500039b0 0x7f6650003dd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f666000c2c0 tx=0x7f6650005830 comp rx=0 tx=0).stop 2024-04-25T08:38:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f6669e8b640 1 -- 172.21.15.60:0/2988167321 reap_dead start 2024-04-25T08:38:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f666b8ee640 1 -- 172.21.15.60:0/2988167321 shutdown_connections 2024-04-25T08:38:13.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f666b8ee640 1 -- 172.21.15.60:0/2988167321 >> 172.21.15.60:0/2988167321 conn(0x7f6664061c30 msgr2=0x7f666406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f666b8ee640 1 -- 172.21.15.60:0/2988167321 shutdown_connections 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.971+0000 7f666b8ee640 1 -- 172.21.15.60:0/2988167321 wait complete. 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.972+0000 7f666b8ee640 1 Processor -- start 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 -- start start 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66500039b0 0x7f666414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 0x7f6664158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f666415b470 0x7f666415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640d2970 con 0x7f6664158760 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640d2690 con 0x7f666415b470 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f666b8ee640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66640d1a10 con 0x7f66500039b0 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f6668e89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 0x7f6664158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f6668e89640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 0x7f6664158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59176/0 (socket says 172.21.15.60:59176) 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.973+0000 7f6668e89640 1 -- 172.21.15.60:0/2292733057 learned_addr learned my addr 172.21.15.60:0/2292733057 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f666968a640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66500039b0 0x7f666414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6668e89640 1 -- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66500039b0 msgr2=0x7f666414dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6668e89640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66500039b0 0x7f666414dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6669e8b640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f666415b470 0x7f666415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6668e89640 1 -- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f666415b470 msgr2=0x7f666415b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6668e89640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f666415b470 0x7f666415b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6668e89640 1 -- 172.21.15.60:0/2292733057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664156f40 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6668e89640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 0x7f6664158be0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6654011940 tx=0x7f6654011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654005ba0 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654005d40 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66641571e0 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400fb20 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.974+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66641577a0 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.975+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6664073990 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.976+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6654012040 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.976+0000 7f6659ffb640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f664806b500 0x7f664806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.976+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f665408a680 con 0x7f6664158760 2024-04-25T08:38:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.976+0000 7f666968a640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f664806b500 0x7f664806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.977+0000 7f666968a640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f664806b500 0x7f664806d9c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6660033000 tx=0x7f6660009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:13.980+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6654018030 con 0x7f6664158760 2024-04-25T08:38:14.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.226+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6664160c70 con 0x7f6664158760 2024-04-25T08:38:14.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.227+0000 7f6659ffb640 1 -- 172.21.15.60:0/2292733057 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6654059070 con 0x7f6664158760 2024-04-25T08:38:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:14.221 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:38:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:38:14.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:38:14.222 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:38:14.223 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.227 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.228 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:38:14.229 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:38:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:38:14.230 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.231 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.233 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:38:14.234 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:38:14.235 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f664806b500 msgr2=0x7f664806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f664806b500 0x7f664806d9c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6660033000 tx=0x7f6660009030 comp rx=0 tx=0).stop 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 msgr2=0x7f6664158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 0x7f6664158be0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6654011940 tx=0x7f6654011e00 comp rx=0 tx=0).stop 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 shutdown_connections 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f664806b500 0x7f664806d9c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f666415b470 0x7f666415b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6664158760 0x7f6664158be0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 --2- 172.21.15.60:0/2292733057 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f66500039b0 0x7f666414dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 >> 172.21.15.60:0/2292733057 conn(0x7f6664061c30 msgr2=0x7f666406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 shutdown_connections 2024-04-25T08:38:14.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.229+0000 7f666b8ee640 1 -- 172.21.15.60:0/2292733057 wait complete. 2024-04-25T08:38:14.238 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:14.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.345+0000 7fa95d19e640 1 Processor -- start 2024-04-25T08:38:14.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.345+0000 7fa95d19e640 1 -- start start 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95d19e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9580d1020 0x7fa9580d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95d19e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9580b97a0 con 0x7fa9580b2cb0 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95d19e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9580d1c50 con 0x7fa95815dbe0 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95d19e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95815dfc0 con 0x7fa9580d1020 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9580d1020 0x7fa9580d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9580d1020 0x7fa9580d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59178/0 (socket says 172.21.15.60:59178) 2024-04-25T08:38:14.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa94ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa9580b2cb0 0x7fa9580d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38634/0 (socket says 172.21.15.60:38634) 2024-04-25T08:38:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95759e640 1 -- 172.21.15.60:0/321052001 learned_addr learned my addr 172.21.15.60:0/321052001 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95759e640 1 -- 172.21.15.60:0/321052001 >> v1:172.21.15.100:6789/0 conn(0x7fa95815dbe0 legacy=0x7fa9580d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95759e640 1 -- 172.21.15.60:0/321052001 >> v1:172.21.15.60:6789/0 conn(0x7fa9580b2cb0 legacy=0x7fa9580d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95759e640 1 -- 172.21.15.60:0/321052001 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580d1f30 con 0x7fa9580d1020 2024-04-25T08:38:14.341 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.346+0000 7fa95659c640 1 -- 172.21.15.60:0/321052001 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796577691 0 0) 0x7fa9580b97a0 con 0x7fa9580b2cb0 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.347+0000 7fa95759e640 1 --2- 172.21.15.60:0/321052001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580d1020 0x7fa9580d2e40 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa950009e50 tx=0x7fa95002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9903524ef192c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.347+0000 7fa95659c640 1 -- 172.21.15.60:0/321052001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950036070 con 0x7fa9580d1020 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.347+0000 7fa95659c640 1 -- 172.21.15.60:0/321052001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa950031600 con 0x7fa9580d1020 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.347+0000 7fa95659c640 1 -- 172.21.15.60:0/321052001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9500319d0 con 0x7fa9580d1020 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.347+0000 7fa95d19e640 1 -- 172.21.15.60:0/321052001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580d1020 msgr2=0x7fa9580d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.347+0000 7fa95d19e640 1 --2- 172.21.15.60:0/321052001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580d1020 0x7fa9580d2e40 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa950009e50 tx=0x7fa95002a760 comp rx=0 tx=0).stop 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.348+0000 7fa95d19e640 1 -- 172.21.15.60:0/321052001 shutdown_connections 2024-04-25T08:38:14.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.348+0000 7fa95d19e640 1 --2- 172.21.15.60:0/321052001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580d1020 0x7fa9580d2e40 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.348+0000 7fa95d19e640 1 -- 172.21.15.60:0/321052001 >> 172.21.15.60:0/321052001 conn(0x7fa958061a20 msgr2=0x7fa958061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.348+0000 7fa95d19e640 1 -- 172.21.15.60:0/321052001 shutdown_connections 2024-04-25T08:38:14.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.348+0000 7fa95d19e640 1 -- 172.21.15.60:0/321052001 wait complete. 2024-04-25T08:38:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.349+0000 7fa95d19e640 1 Processor -- start 2024-04-25T08:38:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.349+0000 7fa95d19e640 1 -- start start 2024-04-25T08:38:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95d19e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 0x7fa95815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95d19e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9580d1020 0x7fa9580b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95d19e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95815dbe0 0x7fa9580b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95d19e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580b92a0 con 0x7fa9580b2cb0 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95d19e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580d1a00 con 0x7fa95815dbe0 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95d19e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580b9420 con 0x7fa9580d1020 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 0x7fa95815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 0x7fa95815eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59190/0 (socket says 172.21.15.60:59190) 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 -- 172.21.15.60:0/168557335 learned_addr learned my addr 172.21.15.60:0/168557335 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa94ffff640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9580d1020 0x7fa9580b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa95759e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95815dbe0 0x7fa9580b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 -- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9580d1020 msgr2=0x7fa9580b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9580d1020 0x7fa9580b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 -- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95815dbe0 msgr2=0x7fa9580b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95815dbe0 0x7fa9580b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 -- 172.21.15.60:0/168557335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95815f310 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa956d9d640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 0x7fa95815eac0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa94800b3f0 tx=0x7fa94800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948015020 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.350+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa948011830 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.351+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948011c20 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.351+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95815f5f0 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.351+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95815fbb0 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.352+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958073990 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.352+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa948018020 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.352+0000 7fa94f7fe640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93806b4d0 0x7fa93806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.352+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa948089d20 con 0x7fa9580b2cb0 2024-04-25T08:38:14.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.353+0000 7fa94ffff640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93806b4d0 0x7fa93806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.353+0000 7fa94ffff640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93806b4d0 0x7fa93806d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa9580d33a0 tx=0x7fa940007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.357+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa94801f030 con 0x7fa9580b2cb0 2024-04-25T08:38:14.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.537+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa958062050 con 0x7fa9580b2cb0 2024-04-25T08:38:14.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.538+0000 7fa94f7fe640 1 -- 172.21.15.60:0/168557335 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+13162 (secure 0 0 0) 0x7fa9480172c0 con 0x7fa9580b2cb0 2024-04-25T08:38:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:14.532 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":130,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:38:13.855457+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:14.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93806b4d0 msgr2=0x7fa93806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93806b4d0 0x7fa93806d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa9580d33a0 tx=0x7fa940007b50 comp rx=0 tx=0).stop 2024-04-25T08:38:14.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 msgr2=0x7fa95815eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 0x7fa95815eac0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa94800b3f0 tx=0x7fa94800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 shutdown_connections 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93806b4d0 0x7fa93806d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa95815dbe0 0x7fa9580b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9580d1020 0x7fa9580b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 --2- 172.21.15.60:0/168557335 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9580b2cb0 0x7fa95815eac0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 >> 172.21.15.60:0/168557335 conn(0x7fa958061a20 msgr2=0x7fa958158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 shutdown_connections 2024-04-25T08:38:14.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.540+0000 7fa95d19e640 1 -- 172.21.15.60:0/168557335 wait complete. 2024-04-25T08:38:14.545 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:14.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:14.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.656+0000 7fbc8fc5a640 1 Processor -- start 2024-04-25T08:38:14.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.656+0000 7fbc8fc5a640 1 -- start start 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8fc5a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc880ced80 0x7fbc880d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8fc5a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8815e000 con 0x7fbc88161710 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8fc5a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880d1de0 con 0x7fbc8815dc20 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8fc5a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc88161c60 con 0x7fbc880ced80 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8e1f7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbc88161710 0x7fbc880d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38644/0 (socket says 172.21.15.60:38644) 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8e1f7640 1 -- 172.21.15.60:0/2376179987 learned_addr learned my addr 172.21.15.60:0/2376179987 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.657+0000 7fbc8d1f5640 1 --2- 172.21.15.60:0/2376179987 >> v2:172.21.15.100:3300/0 conn(0x7fbc880ced80 0x7fbc880d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.658+0000 7fbc8c9f4640 1 -- 172.21.15.60:0/2376179987 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330462004 0 0) 0x7fbc8815e000 con 0x7fbc88161710 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.658+0000 7fbc8c9f4640 1 -- 172.21.15.60:0/2376179987 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc74003520 con 0x7fbc88161710 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.658+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/2376179987 >> v1:172.21.15.100:6789/0 conn(0x7fbc8815dc20 legacy=0x7fbc880ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.658+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/2376179987 >> v1:172.21.15.60:6789/0 conn(0x7fbc88161710 legacy=0x7fbc880d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.658+0000 7fbc8c9f4640 1 -- 172.21.15.60:0/2376179987 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3089122078 0 0) 0x7fbc74003520 con 0x7fbc88161710 2024-04-25T08:38:14.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.658+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/2376179987 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88161f40 con 0x7fbc880ced80 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8d1f5640 1 --2- 172.21.15.60:0/2376179987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc880d22e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbc780097a0 tx=0x7fbc7802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98efc28865dc7d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8c9f4640 1 -- 172.21.15.60:0/2376179987 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78037070 con 0x7fbc880ced80 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8c9f4640 1 -- 172.21.15.60:0/2376179987 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7802ac50 con 0x7fbc880ced80 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8c9f4640 1 -- 172.21.15.60:0/2376179987 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800019f0 con 0x7fbc880ced80 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/2376179987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 msgr2=0x7fbc880d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/2376179987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc880d22e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbc780097a0 tx=0x7fbc7802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/2376179987 shutdown_connections 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/2376179987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc880d22e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.659+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/2376179987 >> 172.21.15.60:0/2376179987 conn(0x7fbc880d0420 msgr2=0x7fbc880d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.660+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/2376179987 shutdown_connections 2024-04-25T08:38:14.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.660+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/2376179987 wait complete. 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 Processor -- start 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 -- start start 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc881f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 0x7fbc880c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc88161710 0x7fbc880c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b5e50 con 0x7fbc8815dc20 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88162500 con 0x7fbc880ced80 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.661+0000 7fbc8fc5a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b5fd0 con 0x7fbc88161710 2024-04-25T08:38:14.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 0x7fbc880c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 0x7fbc880c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59202/0 (socket says 172.21.15.60:59202) 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/342124686 learned_addr learned my addr 172.21.15.60:0/342124686 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8e1f7640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc88161710 0x7fbc880c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d9f6640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc881f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc88161710 msgr2=0x7fbc880c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc88161710 0x7fbc880c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 msgr2=0x7fbc881f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc881f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 -- 172.21.15.60:0/342124686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc881f33f0 con 0x7fbc8815dc20 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d9f6640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc881f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8d1f5640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 0x7fbc880c84c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbc78035040 tx=0x7fbc780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78041070 con 0x7fbc8815dc20 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7803c460 con 0x7fbc8815dc20 2024-04-25T08:38:14.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78037040 con 0x7fbc8815dc20 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8815f1e0 con 0x7fbc8815dc20 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.662+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8815f7a0 con 0x7fbc8815dc20 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.663+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc88073a00 con 0x7fbc8815dc20 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.664+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc78044070 con 0x7fbc8815dc20 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.664+0000 7fbc7e7fc640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.664+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbc780aa680 con 0x7fbc8815dc20 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.664+0000 7fbc8d9f6640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.665+0000 7fbc8d9f6640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007fd0 tx=0x7fbc84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.669+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc780796c0 con 0x7fbc8815dc20 2024-04-25T08:38:14.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.849+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc880d16b0 con 0x7fbc8815dc20 2024-04-25T08:38:14.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.850+0000 7fbc7e7fc640 1 -- 172.21.15.60:0/342124686 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc78078fc0 con 0x7fbc8815dc20 2024-04-25T08:38:14.844 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbc6c06b4d0 msgr2=0x7fbc6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbc84007fd0 tx=0x7fbc84009040 comp rx=0 tx=0).stop 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 msgr2=0x7fbc880c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 0x7fbc880c84c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbc78035040 tx=0x7fbc780046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 shutdown_connections 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbc88161710 0x7fbc880c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbc8815dc20 0x7fbc880c84c0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 --2- 172.21.15.60:0/342124686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc880ced80 0x7fbc881f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 >> 172.21.15.60:0/342124686 conn(0x7fbc880d0420 msgr2=0x7fbc880d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 shutdown_connections 2024-04-25T08:38:14.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.852+0000 7fbc8fc5a640 1 -- 172.21.15.60:0/342124686 wait complete. 2024-04-25T08:38:14.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:38:14.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.963+0000 7fcef68f5640 1 Processor -- start 2024-04-25T08:38:14.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fcef68f5640 1 -- start start 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fcef68f5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcef01602e0 0x7fcef00ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fcef68f5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef00cd710 con 0x7fcef015ba00 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fcef68f5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef0099b70 con 0x7fcef015f5f0 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fcef68f5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef0099e50 con 0x7fcef01602e0 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fceef7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcef01602e0 0x7fcef00ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fceeffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcef015ba00 0x7fcef009a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38654/0 (socket says 172.21.15.60:38654) 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.964+0000 7fceeffff640 1 -- 172.21.15.60:0/2790991295 learned_addr learned my addr 172.21.15.60:0/2790991295 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.965+0000 7fceef7fe640 1 -- 172.21.15.60:0/2790991295 >> v1:172.21.15.100:6789/0 conn(0x7fcef015f5f0 legacy=0x7fcef00c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.965+0000 7fceef7fe640 1 -- 172.21.15.60:0/2790991295 >> v1:172.21.15.60:6789/0 conn(0x7fcef015ba00 legacy=0x7fcef009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.965+0000 7fceef7fe640 1 -- 172.21.15.60:0/2790991295 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef009a130 con 0x7fcef01602e0 2024-04-25T08:38:14.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.965+0000 7fceeeffd640 1 -- 172.21.15.60:0/2790991295 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801674439 0 0) 0x7fced8001110 con 0x7fcef015ba00 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.965+0000 7fceef7fe640 1 --2- 172.21.15.60:0/2790991295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef01602e0 0x7fcef00ccda0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fcee00097a0 tx=0x7fcee002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec70b08c030336b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fceeeffd640 1 -- 172.21.15.60:0/2790991295 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0037070 con 0x7fcef01602e0 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fceeeffd640 1 -- 172.21.15.60:0/2790991295 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee002adc0 con 0x7fcef01602e0 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fceeeffd640 1 -- 172.21.15.60:0/2790991295 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0032660 con 0x7fcef01602e0 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 -- 172.21.15.60:0/2790991295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef01602e0 msgr2=0x7fcef00ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 --2- 172.21.15.60:0/2790991295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef01602e0 0x7fcef00ccda0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fcee00097a0 tx=0x7fcee002a0b0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 -- 172.21.15.60:0/2790991295 shutdown_connections 2024-04-25T08:38:14.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 --2- 172.21.15.60:0/2790991295 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef01602e0 0x7fcef00ccda0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 -- 172.21.15.60:0/2790991295 >> 172.21.15.60:0/2790991295 conn(0x7fcef00b3270 msgr2=0x7fcef00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 -- 172.21.15.60:0/2790991295 shutdown_connections 2024-04-25T08:38:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.966+0000 7fcef68f5640 1 -- 172.21.15.60:0/2790991295 wait complete. 2024-04-25T08:38:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.967+0000 7fcef68f5640 1 Processor -- start 2024-04-25T08:38:14.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 -- start start 2024-04-25T08:38:14.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef015ba00 0x7fcef01f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 0x7fcef00c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef01602e0 0x7fcef006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00cd460 con 0x7fcef015f5f0 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00cd2e0 con 0x7fcef015ba00 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fcef68f5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00c59e0 con 0x7fcef01602e0 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 0x7fcef00c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fceef7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 0x7fcef00c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59228/0 (socket says 172.21.15.60:59228) 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.968+0000 7fceef7fe640 1 -- 172.21.15.60:0/1323762020 learned_addr learned my addr 172.21.15.60:0/1323762020 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fcef4e92640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef01602e0 0x7fcef006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceeffff640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef015ba00 0x7fcef01f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceef7fe640 1 -- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef01602e0 msgr2=0x7fcef006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceef7fe640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef01602e0 0x7fcef006d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceef7fe640 1 -- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef015ba00 msgr2=0x7fcef01f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceef7fe640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef015ba00 0x7fcef01f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceef7fe640 1 -- 172.21.15.60:0/1323762020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef015ce70 con 0x7fcef015f5f0 2024-04-25T08:38:14.964 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceef7fe640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 0x7fcef00c99d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fcee0035040 tx=0x7fcee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0042020 con 0x7fcef015f5f0 2024-04-25T08:38:14.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee003d690 con 0x7fcef015f5f0 2024-04-25T08:38:14.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0037040 con 0x7fcef015f5f0 2024-04-25T08:38:14.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef015d150 con 0x7fcef015f5f0 2024-04-25T08:38:14.965 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.969+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef006cc80 con 0x7fcef015f5f0 2024-04-25T08:38:14.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.970+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef0073990 con 0x7fcef015f5f0 2024-04-25T08:38:14.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.971+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcee0045020 con 0x7fcef015f5f0 2024-04-25T08:38:14.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.971+0000 7fceecff9640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fced006b500 0x7fced006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.971+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcee00aa850 con 0x7fcef015f5f0 2024-04-25T08:38:14.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.971+0000 7fceeffff640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fced006b500 0x7fced006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.966 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.972+0000 7fceeffff640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fced006b500 0x7fced006d9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fced8000f60 tx=0x7fced8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.969 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:14.975+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcee004c030 con 0x7fcef015f5f0 2024-04-25T08:38:15.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.160+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcef00c5350 con 0x7fcef015f5f0 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.161+0000 7fceecff9640 1 -- 172.21.15.60:0/1323762020 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v130) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcee0079920 con 0x7fcef015f5f0 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:15.155 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fced006b500 msgr2=0x7fced006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fced006b500 0x7fced006d9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fced8000f60 tx=0x7fced8009040 comp rx=0 tx=0).stop 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 msgr2=0x7fcef00c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 0x7fcef00c99d0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fcee0035040 tx=0x7fcee00046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 shutdown_connections 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcef01602e0 0x7fcef006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcef015f5f0 0x7fcef00c99d0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.157 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fced006b500 0x7fced006d9c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 --2- 172.21.15.60:0/1323762020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcef015ba00 0x7fcef01f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 >> 172.21.15.60:0/1323762020 conn(0x7fcef00b3270 msgr2=0x7fcef0070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 shutdown_connections 2024-04-25T08:38:15.158 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.163+0000 7fcef68f5640 1 -- 172.21.15.60:0/1323762020 wait complete. 2024-04-25T08:38:15.168 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:38:15.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.279+0000 7f172e796640 1 Processor -- start 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.279+0000 7f172e796640 1 -- start start 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172e796640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f172815b7d0 0x7f17280cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172e796640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17280b5620 0x7f17280b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172e796640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172806cf50 con 0x7f172806c2d0 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172e796640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17280ce530 con 0x7f172815b7d0 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172e796640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17280ce810 con 0x7f17280b5620 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f1727fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f172815b7d0 0x7f17280cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172cd33640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f172806c2d0 0x7f172806c740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38656/0 (socket says 172.21.15.60:38656) 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f1727fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f172815b7d0 0x7f17280cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59238/0 (socket says 172.21.15.60:59238) 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.280+0000 7f172cd33640 1 -- 172.21.15.60:0/2701362238 learned_addr learned my addr 172.21.15.60:0/2701362238 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f17277fe640 1 --2- 172.21.15.60:0/2701362238 >> v2:172.21.15.100:3300/0 conn(0x7f17280b5620 0x7f17280b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1727fff640 1 -- 172.21.15.60:0/2701362238 >> v2:172.21.15.100:3300/0 conn(0x7f17280b5620 msgr2=0x7f17280b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1727fff640 1 --2- 172.21.15.60:0/2701362238 >> v2:172.21.15.100:3300/0 conn(0x7f17280b5620 0x7f17280b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1727fff640 1 -- 172.21.15.60:0/2701362238 >> v1:172.21.15.60:6789/0 conn(0x7f172806c2d0 legacy=0x7f172806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1727fff640 1 -- 172.21.15.60:0/2701362238 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280ceaf0 con 0x7f172815b7d0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1726ffd640 1 -- 172.21.15.60:0/2701362238 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065769574 0 0) 0x7f172806cf50 con 0x7f172806c2d0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f17277fe640 1 --2- 172.21.15.60:0/2701362238 >> v2:172.21.15.100:3300/0 conn(0x7f17280b5620 0x7f17280b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1727fff640 1 --2- 172.21.15.60:0/2701362238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f172815b7d0 0x7f17280cf010 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f17280ce180 tx=0x7f171002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755a46b39e480b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.281+0000 7f1726ffd640 1 -- 172.21.15.60:0/2701362238 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710037070 con 0x7f172815b7d0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f1726ffd640 1 -- 172.21.15.60:0/2701362238 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710032440 con 0x7f172815b7d0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f1726ffd640 1 -- 172.21.15.60:0/2701362238 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710032810 con 0x7f172815b7d0 2024-04-25T08:38:15.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 -- 172.21.15.60:0/2701362238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f172815b7d0 msgr2=0x7f17280cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 --2- 172.21.15.60:0/2701362238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f172815b7d0 0x7f17280cf010 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f17280ce180 tx=0x7f171002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 -- 172.21.15.60:0/2701362238 shutdown_connections 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 --2- 172.21.15.60:0/2701362238 >> v2:172.21.15.100:3300/0 conn(0x7f17280b5620 0x7f17280b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 --2- 172.21.15.60:0/2701362238 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f172815b7d0 0x7f17280cf010 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 -- 172.21.15.60:0/2701362238 >> 172.21.15.60:0/2701362238 conn(0x7f17280d0800 msgr2=0x7f17280d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 -- 172.21.15.60:0/2701362238 shutdown_connections 2024-04-25T08:38:15.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.282+0000 7f172e796640 1 -- 172.21.15.60:0/2701362238 wait complete. 2024-04-25T08:38:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.283+0000 7f172e796640 1 Processor -- start 2024-04-25T08:38:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.283+0000 7f172e796640 1 -- start start 2024-04-25T08:38:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f172e796640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f172806c2d0 0x7f17280c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f172e796640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 0x7f17280c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f172e796640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f172815b7d0 0x7f17280c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f172e796640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280cf6d0 con 0x7f17280b5620 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f172e796640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280cf550 con 0x7f172806c2d0 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f172e796640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172815bbb0 con 0x7f172815b7d0 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f17277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 0x7f17280c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f17277fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 0x7f17280c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59246/0 (socket says 172.21.15.60:59246) 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f17277fe640 1 -- 172.21.15.60:0/2024368563 learned_addr learned my addr 172.21.15.60:0/2024368563 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.284+0000 7f1727fff640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f172806c2d0 0x7f17280c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f172cd33640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f172815b7d0 0x7f17280c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f17277fe640 1 -- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f172815b7d0 msgr2=0x7f17280c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f17277fe640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f172815b7d0 0x7f17280c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f17277fe640 1 -- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f172806c2d0 msgr2=0x7f17280c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f17277fe640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f172806c2d0 0x7f17280c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f17277fe640 1 -- 172.21.15.60:0/2024368563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280c88a0 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f1727fff640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f172806c2d0 0x7f17280c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f17277fe640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 0x7f17280c7ad0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f171800c7f0 tx=0x7f171800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718015070 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718007a30 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17280c8b80 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718007e00 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.285+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17280c9140 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.286+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728073a00 con 0x7f17280b5620 2024-04-25T08:38:15.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.287+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1718018070 con 0x7f17280b5620 2024-04-25T08:38:15.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.287+0000 7f1724ff9640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f170806b4d0 0x7f170806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.287+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1718089ff0 con 0x7f17280b5620 2024-04-25T08:38:15.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.287+0000 7f1727fff640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f170806b4d0 0x7f170806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.288+0000 7f1727fff640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f170806b4d0 0x7f170806d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1710034000 tx=0x7f171002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.292+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1718059030 con 0x7f17280b5620 2024-04-25T08:38:15.468 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.474+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f172815c2c0 con 0x7f17280b5620 2024-04-25T08:38:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.849+0000 7f1724ff9640 1 -- 172.21.15.60:0/2024368563 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v131) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1718058930 con 0x7f17280b5620 2024-04-25T08:38:15.844 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:38:15.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f170806b4d0 msgr2=0x7f170806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f170806b4d0 0x7f170806d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1710034000 tx=0x7f171002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:15.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 msgr2=0x7f17280c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.847 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 0x7f17280c7ad0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f171800c7f0 tx=0x7f171800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 shutdown_connections 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f172815b7d0 0x7f17280c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17280b5620 0x7f17280c7ad0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f170806b4d0 0x7f170806d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 --2- 172.21.15.60:0/2024368563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f172806c2d0 0x7f17280c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.852+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 >> 172.21.15.60:0/2024368563 conn(0x7f17280d0800 msgr2=0x7f17281609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:15.848 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.853+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 shutdown_connections 2024-04-25T08:38:15.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.853+0000 7f172e796640 1 -- 172.21.15.60:0/2024368563 wait complete. 2024-04-25T08:38:15.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:38:15.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.977+0000 7f0307cfc640 1 Processor -- start 2024-04-25T08:38:15.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.978+0000 7f0307cfc640 1 -- start start 2024-04-25T08:38:15.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0307cfc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03000cc4f0 0x7f03000cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0307cfc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f030015b980 0x7f03000cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0307cfc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0300072280 con 0x7f03000cdb20 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0307cfc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0300071270 con 0x7f030015b980 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0307cfc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030015bd80 con 0x7f03000cc4f0 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f030015b980 0x7f03000cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f030015b980 0x7f03000cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59248/0 (socket says 172.21.15.60:59248) 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 -- 172.21.15.60:0/4223749354 learned_addr learned my addr 172.21.15.60:0/4223749354 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0305a98640 1 --2- 172.21.15.60:0/4223749354 >> v2:172.21.15.100:3300/0 conn(0x7f03000cc4f0 0x7f03000cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 -- 172.21.15.60:0/4223749354 >> v2:172.21.15.100:3300/0 conn(0x7f03000cc4f0 msgr2=0x7f03000cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:15.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 --2- 172.21.15.60:0/4223749354 >> v2:172.21.15.100:3300/0 conn(0x7f03000cc4f0 0x7f03000cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 -- 172.21.15.60:0/4223749354 >> v1:172.21.15.100:6789/0 conn(0x7f03000cdb20 legacy=0x7f030015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.979+0000 7f0306299640 1 -- 172.21.15.60:0/4223749354 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000714c0 con 0x7f030015b980 2024-04-25T08:38:15.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.980+0000 7f0306299640 1 --2- 172.21.15.60:0/4223749354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030015b980 0x7f03000cdfb0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f03000cd290 tx=0x7f02f802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3cb3138331ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.980+0000 7f0304a96640 1 -- 172.21.15.60:0/4223749354 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8036070 con 0x7f030015b980 2024-04-25T08:38:15.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.980+0000 7f0304a96640 1 -- 172.21.15.60:0/4223749354 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8031600 con 0x7f030015b980 2024-04-25T08:38:15.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.980+0000 7f0304a96640 1 -- 172.21.15.60:0/4223749354 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80319d0 con 0x7f030015b980 2024-04-25T08:38:15.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 -- 172.21.15.60:0/4223749354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030015b980 msgr2=0x7f03000cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 --2- 172.21.15.60:0/4223749354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030015b980 0x7f03000cdfb0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f03000cd290 tx=0x7f02f802a730 comp rx=0 tx=0).stop 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 -- 172.21.15.60:0/4223749354 shutdown_connections 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 --2- 172.21.15.60:0/4223749354 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f030015b980 0x7f03000cdfb0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 --2- 172.21.15.60:0/4223749354 >> v2:172.21.15.100:3300/0 conn(0x7f03000cc4f0 0x7f03000cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 -- 172.21.15.60:0/4223749354 >> 172.21.15.60:0/4223749354 conn(0x7f030006abe0 msgr2=0x7f030006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 -- 172.21.15.60:0/4223749354 shutdown_connections 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.981+0000 7f0307cfc640 1 -- 172.21.15.60:0/4223749354 wait complete. 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.982+0000 7f0307cfc640 1 Processor -- start 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.982+0000 7f0307cfc640 1 -- start start 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 0x7f03001e3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03000cdb20 0x7f03001e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f030015b980 0x7f03001f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300071d60 con 0x7f03000cc4f0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300071a80 con 0x7f03000cdb20 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000ae500 con 0x7f030015b980 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 0x7f03001e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 0x7f03001e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59256/0 (socket says 172.21.15.60:59256) 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 -- 172.21.15.60:0/2552445192 learned_addr learned my addr 172.21.15.60:0/2552445192 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0306299640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f030015b980 0x7f03001f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305297640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03000cdb20 0x7f03001e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 -- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f030015b980 msgr2=0x7f03001f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f030015b980 0x7f03001f16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 -- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03000cdb20 msgr2=0x7f03001e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03000cdb20 0x7f03001e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 -- 172.21.15.60:0/2552445192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300141e70 con 0x7f03000cc4f0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0305a98640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 0x7f03001e3160 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00d5d0 tx=0x7f02fc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0125f0 con 0x7f03000cc4f0 2024-04-25T08:38:15.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc019070 con 0x7f03000cc4f0 2024-04-25T08:38:15.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0300142150 con 0x7f03000cc4f0 2024-04-25T08:38:15.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0129f0 con 0x7f03000cc4f0 2024-04-25T08:38:15.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.983+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0300142710 con 0x7f03000cc4f0 2024-04-25T08:38:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.984+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0300073a00 con 0x7f03000cc4f0 2024-04-25T08:38:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.985+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02fc012b90 con 0x7f03000cc4f0 2024-04-25T08:38:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.985+0000 7f02f67fc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02e406b4d0 0x7f02e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.986+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f02fc08a5f0 con 0x7f03000cc4f0 2024-04-25T08:38:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.986+0000 7f0305297640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02e406b4d0 0x7f02e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.986+0000 7f0305297640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02e406b4d0 0x7f02e406d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0300146010 tx=0x7f02f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:15.990+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02fc0594c0 con 0x7f03000cc4f0 2024-04-25T08:38:16.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.170+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f03000b4320 con 0x7f03000cc4f0 2024-04-25T08:38:16.846 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.852+0000 7f02f67fc640 1 -- 172.21.15.60:0/2552445192 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v132) v1 ==== 170+0+0 (secure 0 0 0) 0x7f02fc058dc0 con 0x7f03000cc4f0 2024-04-25T08:38:16.846 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:38:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02e406b4d0 msgr2=0x7f02e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.849 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02e406b4d0 0x7f02e406d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0300146010 tx=0x7f02f0009040 comp rx=0 tx=0).stop 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 msgr2=0x7f03001e3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 0x7f03001e3160 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00d5d0 tx=0x7f02fc00da90 comp rx=0 tx=0).stop 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 shutdown_connections 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f030015b980 0x7f03001f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f02e406b4d0 0x7f02e406d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03000cdb20 0x7f03001e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 --2- 172.21.15.60:0/2552445192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f03000cc4f0 0x7f03001e3160 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 >> 172.21.15.60:0/2552445192 conn(0x7f030006abe0 msgr2=0x7f030006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.855+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 shutdown_connections 2024-04-25T08:38:16.850 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.856+0000 7f0307cfc640 1 -- 172.21.15.60:0/2552445192 wait complete. 2024-04-25T08:38:16.874 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:38:16.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.989+0000 7f4c5adc1640 1 Processor -- start 2024-04-25T08:38:16.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.989+0000 7f4c5adc1640 1 -- start start 2024-04-25T08:38:16.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5adc1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d34c0 0x7f4c540d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5adc1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4c540d08e0 0x7f4c540b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5adc1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c540d2420 con 0x7f4c5415dac0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5adc1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c540d3030 con 0x7f4c540d08e0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5adc1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540d2700 con 0x7f4c540d34c0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c58b5d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4c5415dac0 0x7f4c540d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38666/0 (socket says 172.21.15.60:38666) 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c58b5d640 1 -- 172.21.15.60:0/276800206 learned_addr learned my addr 172.21.15.60:0/276800206 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c53fff640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d34c0 0x7f4c540d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5935e640 1 --2- 172.21.15.60:0/276800206 >> v2:172.21.15.100:3300/0 conn(0x7f4c540d08e0 0x7f4c540b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c537fe640 1 -- 172.21.15.60:0/276800206 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210517614 0 0) 0x7f4c540d2420 con 0x7f4c5415dac0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c537fe640 1 -- 172.21.15.60:0/276800206 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c3c003520 con 0x7f4c5415dac0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c537fe640 1 -- 172.21.15.60:0/276800206 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3939503674 0 0) 0x7f4c3c003520 con 0x7f4c5415dac0 2024-04-25T08:38:16.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5935e640 1 -- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d34c0 msgr2=0x7f4c540d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:16.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5935e640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d34c0 0x7f4c540d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5935e640 1 -- 172.21.15.60:0/276800206 >> v1:172.21.15.60:6789/0 conn(0x7f4c5415dac0 legacy=0x7f4c540d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c5935e640 1 -- 172.21.15.60:0/276800206 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540d29e0 con 0x7f4c540d08e0 2024-04-25T08:38:16.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.990+0000 7f4c53fff640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d34c0 0x7f4c540d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:16.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.991+0000 7f4c5935e640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c540d08e0 0x7f4c540b6990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19415bb6bed38375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.991+0000 7f4c537fe640 1 -- 172.21.15.60:0/276800206 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44001670 con 0x7f4c540d08e0 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.991+0000 7f4c537fe640 1 -- 172.21.15.60:0/276800206 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c007540 con 0x7f4c540d08e0 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.991+0000 7f4c537fe640 1 -- 172.21.15.60:0/276800206 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c007910 con 0x7f4c540d08e0 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 -- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c540d08e0 msgr2=0x7f4c540b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c540d08e0 0x7f4c540b6990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c02a680 comp rx=0 tx=0).stop 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 -- 172.21.15.60:0/276800206 shutdown_connections 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c540d08e0 0x7f4c540b6990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/276800206 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d34c0 0x7f4c540d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 -- 172.21.15.60:0/276800206 >> 172.21.15.60:0/276800206 conn(0x7f4c54061c30 msgr2=0x7f4c5406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 -- 172.21.15.60:0/276800206 shutdown_connections 2024-04-25T08:38:16.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.992+0000 7f4c5adc1640 1 -- 172.21.15.60:0/276800206 wait complete. 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.993+0000 7f4c5adc1640 1 Processor -- start 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.993+0000 7f4c5adc1640 1 -- start start 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5adc1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d08e0 0x7f4c540cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5adc1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 0x7f4c5415f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5adc1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c5415dac0 0x7f4c5415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5adc1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5415dea0 con 0x7f4c540d34c0 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5adc1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540d2100 con 0x7f4c5415dac0 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5adc1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540b97d0 con 0x7f4c540d08e0 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 0x7f4c5415f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 0x7f4c5415f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59262/0 (socket says 172.21.15.60:59262) 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c53fff640 1 -- 172.21.15.60:0/2123677801 learned_addr learned my addr 172.21.15.60:0/2123677801 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c5935e640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c5415dac0 0x7f4c5415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.994+0000 7f4c58b5d640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d08e0 0x7f4c540cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c53fff640 1 -- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d08e0 msgr2=0x7f4c540cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c53fff640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d08e0 0x7f4c540cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c53fff640 1 -- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c5415dac0 msgr2=0x7f4c5415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c53fff640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c5415dac0 0x7f4c5415f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c53fff640 1 -- 172.21.15.60:0/2123677801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540cafa0 con 0x7f4c540d34c0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c53fff640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 0x7f4c5415f070 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e9a0 tx=0x7f4c4800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800ca00 con 0x7f4c540d34c0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4800cba0 con 0x7f4c540d34c0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c540b3e00 con 0x7f4c540d34c0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48005990 con 0x7f4c540d34c0 2024-04-25T08:38:16.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.995+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c540b46a0 con 0x7f4c540d34c0 2024-04-25T08:38:16.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.996+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c540b3fd0 con 0x7f4c540d34c0 2024-04-25T08:38:16.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.997+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c48005b30 con 0x7f4c540d34c0 2024-04-25T08:38:16.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.997+0000 7f4c517fa640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c4006b500 0x7f4c4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.997+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f4c4808a650 con 0x7f4c540d34c0 2024-04-25T08:38:16.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.998+0000 7f4c58b5d640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c4006b500 0x7f4c4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:16.998+0000 7f4c58b5d640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c4006b500 0x7f4c4006d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005880 tx=0x7f4c440057f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:16.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.002+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c48017020 con 0x7f4c540d34c0 2024-04-25T08:38:17.213 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.219+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c54073a00 con 0x7f4c540d34c0 2024-04-25T08:38:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.872+0000 7f4c517fa640 1 -- 172.21.15.60:0/2123677801 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4c48058d50 con 0x7f4c540d34c0 2024-04-25T08:38:17.866 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T08:38:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.878+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c4006b500 msgr2=0x7f4c4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.878+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c4006b500 0x7f4c4006d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005880 tx=0x7f4c440057f0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.878+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 msgr2=0x7f4c5415f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.878+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 0x7f4c5415f070 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e9a0 tx=0x7f4c4800ee60 comp rx=0 tx=0).stop 2024-04-25T08:38:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.879+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 shutdown_connections 2024-04-25T08:38:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.879+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4c4006b500 0x7f4c4006d9c0 secure :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005880 tx=0x7f4c440057f0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.879+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c5415dac0 0x7f4c5415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.879+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c540d34c0 0x7f4c5415f070 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.879+0000 7f4c5adc1640 1 --2- 172.21.15.60:0/2123677801 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4c540d08e0 0x7f4c540cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.879+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 >> 172.21.15.60:0/2123677801 conn(0x7f4c54061c30 msgr2=0x7f4c540b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:17.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.882+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 shutdown_connections 2024-04-25T08:38:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:17.889+0000 7f4c5adc1640 1 -- 172.21.15.60:0/2123677801 wait complete. 2024-04-25T08:38:17.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.011+0000 7f28114b3640 1 Processor -- start 2024-04-25T08:38:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.011+0000 7f28114b3640 1 -- start start 2024-04-25T08:38:18.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f28114b3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f280c0d2e90 0x7f280c0d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f28114b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f280c0d0900 0x7f280c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f28114b3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c0b9940 con 0x7f280c15dd30 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f28114b3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c0d0d40 con 0x7f280c0d2e90 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f28114b3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c0d1df0 con 0x7f280c0d0900 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f280a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f280c0d2e90 0x7f280c0d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f280a7fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f280c0d2e90 0x7f280c0d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59270/0 (socket says 172.21.15.60:59270) 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f280a7fc640 1 -- 172.21.15.60:0/1279093374 learned_addr learned my addr 172.21.15.60:0/1279093374 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.012+0000 7f280b7fe640 1 --2- 172.21.15.60:0/1279093374 >> v2:172.21.15.100:3300/0 conn(0x7f280c0d0900 0x7f280c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f280a7fc640 1 -- 172.21.15.60:0/1279093374 >> v2:172.21.15.100:3300/0 conn(0x7f280c0d0900 msgr2=0x7f280c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f280a7fc640 1 --2- 172.21.15.60:0/1279093374 >> v2:172.21.15.100:3300/0 conn(0x7f280c0d0900 0x7f280c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f280a7fc640 1 -- 172.21.15.60:0/1279093374 >> v1:172.21.15.100:6789/0 conn(0x7f280c15dd30 legacy=0x7f280c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f280a7fc640 1 -- 172.21.15.60:0/1279093374 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c0d20d0 con 0x7f280c0d2e90 2024-04-25T08:38:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f280a7fc640 1 --2- 172.21.15.60:0/1279093374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c0d2e90 0x7f280c0d03c0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f280c0ce1d0 tx=0x7f280002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109411338f9aa60b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f2809ffb640 1 -- 172.21.15.60:0/1279093374 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800004030 con 0x7f280c0d2e90 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f2809ffb640 1 -- 172.21.15.60:0/1279093374 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28000041d0 con 0x7f280c0d2e90 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.013+0000 7f2809ffb640 1 -- 172.21.15.60:0/1279093374 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800003610 con 0x7f280c0d2e90 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 -- 172.21.15.60:0/1279093374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c0d2e90 msgr2=0x7f280c0d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 --2- 172.21.15.60:0/1279093374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c0d2e90 0x7f280c0d03c0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f280c0ce1d0 tx=0x7f280002a130 comp rx=0 tx=0).stop 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 -- 172.21.15.60:0/1279093374 shutdown_connections 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 --2- 172.21.15.60:0/1279093374 >> v2:172.21.15.100:3300/0 conn(0x7f280c0d0900 0x7f280c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 --2- 172.21.15.60:0/1279093374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c0d2e90 0x7f280c0d03c0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 -- 172.21.15.60:0/1279093374 >> 172.21.15.60:0/1279093374 conn(0x7f280c061c30 msgr2=0x7f280c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 -- 172.21.15.60:0/1279093374 shutdown_connections 2024-04-25T08:38:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.014+0000 7f28114b3640 1 -- 172.21.15.60:0/1279093374 wait complete. 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.015+0000 7f28114b3640 1 Processor -- start 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 -- start start 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c0d0900 0x7f280c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280c0d2e90 0x7f280c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 0x7f280c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0d2970 con 0x7f280c15dd30 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0d2690 con 0x7f280c0d0900 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f28114b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0d1a10 con 0x7f280c0d2e90 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 0x7f280c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f280b7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 0x7f280c157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59286/0 (socket says 172.21.15.60:59286) 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.016+0000 7f280b7fe640 1 -- 172.21.15.60:0/2930953089 learned_addr learned my addr 172.21.15.60:0/2930953089 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280a7fc640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280c0d2e90 0x7f280c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280affd640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c0d0900 0x7f280c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280b7fe640 1 -- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280c0d2e90 msgr2=0x7f280c156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280b7fe640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280c0d2e90 0x7f280c156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280b7fe640 1 -- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c0d0900 msgr2=0x7f280c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280b7fe640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c0d0900 0x7f280c14ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280b7fe640 1 -- 172.21.15.60:0/2930953089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c155180 con 0x7f280c15dd30 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280affd640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c0d0900 0x7f280c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f280b7fe640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 0x7f280c157380 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f27fc007e70 tx=0x7f27fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc014620 con 0x7f280c15dd30 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc019070 con 0x7f280c15dd30 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c157b20 con 0x7f280c15dd30 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc014a20 con 0x7f280c15dd30 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.017+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c1580e0 con 0x7f280c15dd30 2024-04-25T08:38:18.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.018+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c1588b0 con 0x7f280c15dd30 2024-04-25T08:38:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.019+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27fc014bc0 con 0x7f280c15dd30 2024-04-25T08:38:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.019+0000 7f27f37fe640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27f806b4d0 0x7f27f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.019+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27fc08acb0 con 0x7f280c15dd30 2024-04-25T08:38:18.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.019+0000 7f280affd640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27f806b4d0 0x7f27f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.020+0000 7f280affd640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27f806b4d0 0x7f27f806d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f28040015e0 tx=0x7f2804009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.023+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27fc0599a0 con 0x7f280c15dd30 2024-04-25T08:38:18.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.203+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f280c073990 con 0x7f280c15dd30 2024-04-25T08:38:18.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.204+0000 7f27f37fe640 1 -- 172.21.15.60:0/2930953089 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f27fc0592a0 con 0x7f280c15dd30 2024-04-25T08:38:18.198 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:18.199 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":133,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:38:17.221419+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27f806b4d0 msgr2=0x7f27f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27f806b4d0 0x7f27f806d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f28040015e0 tx=0x7f2804009040 comp rx=0 tx=0).stop 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 msgr2=0x7f280c157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 0x7f280c157380 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f27fc007e70 tx=0x7f27fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 shutdown_connections 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f280c15dd30 0x7f280c157380 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f280c0d2e90 0x7f280c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f27f806b4d0 0x7f27f806d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 --2- 172.21.15.60:0/2930953089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f280c0d0900 0x7f280c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 >> 172.21.15.60:0/2930953089 conn(0x7f280c061c30 msgr2=0x7f280c0c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 shutdown_connections 2024-04-25T08:38:18.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.207+0000 7f28114b3640 1 -- 172.21.15.60:0/2930953089 wait complete. 2024-04-25T08:38:18.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.319+0000 7fc71f0bc640 1 Processor -- start 2024-04-25T08:38:18.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.319+0000 7fc71f0bc640 1 -- start start 2024-04-25T08:38:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71f0bc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc71815dd30 0x7fc7180b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71f0bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7180b9770 0x7fc7180d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71f0bc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7180d2e90 0x7fc7180d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71f0bc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180d1df0 con 0x7fc7180d2e90 2024-04-25T08:38:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71f0bc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180d20d0 con 0x7fc71815dd30 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71f0bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180d23b0 con 0x7fc7180b9770 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71d659640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7180d2e90 0x7fc7180d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71d659640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc7180d2e90 0x7fc7180d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59292/0 (socket says 172.21.15.60:59292) 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.320+0000 7fc71d659640 1 -- 172.21.15.60:0/1683498731 learned_addr learned my addr 172.21.15.60:0/1683498731 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71ce58640 1 --2- 172.21.15.60:0/1683498731 >> v2:172.21.15.100:3300/0 conn(0x7fc71815dd30 0x7fc7180b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc717fff640 1 --2- 172.21.15.60:0/1683498731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7180b9770 0x7fc7180d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71d659640 1 -- 172.21.15.60:0/1683498731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7180b9770 msgr2=0x7fc7180d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71d659640 1 --2- 172.21.15.60:0/1683498731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7180b9770 0x7fc7180d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71d659640 1 -- 172.21.15.60:0/1683498731 >> v2:172.21.15.100:3300/0 conn(0x7fc71815dd30 msgr2=0x7fc7180b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71d659640 1 --2- 172.21.15.60:0/1683498731 >> v2:172.21.15.100:3300/0 conn(0x7fc71815dd30 0x7fc7180b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71d659640 1 -- 172.21.15.60:0/1683498731 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180d2690 con 0x7fc7180d2e90 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.321+0000 7fc71d659640 1 --2- 172.21.15.60:0/1683498731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180d2e90 0x7fc7180d0900 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc71806d000 tx=0x7fc70002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35f946b22965cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc7177fe640 1 -- 172.21.15.60:0/1683498731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700037070 con 0x7fc7180d2e90 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc7177fe640 1 -- 172.21.15.60:0/1683498731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc700032440 con 0x7fc7180d2e90 2024-04-25T08:38:18.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc7177fe640 1 -- 172.21.15.60:0/1683498731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700032810 con 0x7fc7180d2e90 2024-04-25T08:38:18.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1683498731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180d2e90 msgr2=0x7fc7180d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1683498731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180d2e90 0x7fc7180d0900 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fc71806d000 tx=0x7fc70002a140 comp rx=0 tx=0).stop 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1683498731 shutdown_connections 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1683498731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180d2e90 0x7fc7180d0900 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1683498731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc7180b9770 0x7fc7180d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1683498731 >> v2:172.21.15.100:3300/0 conn(0x7fc71815dd30 0x7fc7180b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1683498731 >> 172.21.15.60:0/1683498731 conn(0x7fc718061c30 msgr2=0x7fc71806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1683498731 shutdown_connections 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.322+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1683498731 wait complete. 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.323+0000 7fc71f0bc640 1 Processor -- start 2024-04-25T08:38:18.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.323+0000 7fc71f0bc640 1 -- start start 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71f0bc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 0x7fc718159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71f0bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180d2e90 0x7fc71815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71f0bc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc71815dd30 0x7fc718152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71f0bc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180d1b90 con 0x7fc7180b9770 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71f0bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180d1a10 con 0x7fc7180d2e90 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71f0bc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7180b5430 con 0x7fc71815dd30 2024-04-25T08:38:18.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71ce58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 0x7fc718159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71ce58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 0x7fc718159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59306/0 (socket says 172.21.15.60:59306) 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.324+0000 7fc71ce58640 1 -- 172.21.15.60:0/1065433969 learned_addr learned my addr 172.21.15.60:0/1065433969 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc717fff640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180d2e90 0x7fc71815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71d659640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc71815dd30 0x7fc718152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71ce58640 1 -- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc71815dd30 msgr2=0x7fc718152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71ce58640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc71815dd30 0x7fc718152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71ce58640 1 -- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180d2e90 msgr2=0x7fc71815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71ce58640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180d2e90 0x7fc71815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71ce58640 1 -- 172.21.15.60:0/1065433969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7181555a0 con 0x7fc7180b9770 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71ce58640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 0x7fc718159b50 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fc710007e40 tx=0x7fc71000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710014620 con 0x7fc7180b9770 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710019070 con 0x7fc7180b9770 2024-04-25T08:38:18.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710014a00 con 0x7fc7180b9770 2024-04-25T08:38:18.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc718157830 con 0x7fc7180b9770 2024-04-25T08:38:18.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.325+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718157df0 con 0x7fc7180b9770 2024-04-25T08:38:18.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.326+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc718073990 con 0x7fc7180b9770 2024-04-25T08:38:18.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.327+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc710014ba0 con 0x7fc7180b9770 2024-04-25T08:38:18.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.327+0000 7fc7157fa640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6f806b4d0 0x7fc6f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.327+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc71008ab00 con 0x7fc7180b9770 2024-04-25T08:38:18.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.327+0000 7fc717fff640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6f806b4d0 0x7fc6f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.328+0000 7fc717fff640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6f806b4d0 0x7fc6f806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc718156e70 tx=0x7fc708008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.331+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc710059880 con 0x7fc7180b9770 2024-04-25T08:38:18.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.512+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc71806cfd0 con 0x7fc7180b9770 2024-04-25T08:38:18.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.514+0000 7fc7157fa640 1 -- 172.21.15.60:0/1065433969 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fc710059180 con 0x7fc7180b9770 2024-04-25T08:38:18.508 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:18.508 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":133,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:38:17.221419+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:18.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.515+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6f806b4d0 msgr2=0x7fc6f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.515+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6f806b4d0 0x7fc6f806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc718156e70 tx=0x7fc708008040 comp rx=0 tx=0).stop 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.515+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 msgr2=0x7fc718159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.515+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 0x7fc718159b50 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fc710007e40 tx=0x7fc71000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 shutdown_connections 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc71815dd30 0x7fc718152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc6f806b4d0 0x7fc6f806d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7180d2e90 0x7fc71815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 --2- 172.21.15.60:0/1065433969 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7180b9770 0x7fc718159b50 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 >> 172.21.15.60:0/1065433969 conn(0x7fc718061c30 msgr2=0x7fc7180b6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 shutdown_connections 2024-04-25T08:38:18.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.516+0000 7fc71f0bc640 1 -- 172.21.15.60:0/1065433969 wait complete. 2024-04-25T08:38:18.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:38:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.630+0000 7f0cff4bd640 1 Processor -- start 2024-04-25T08:38:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.630+0000 7f0cff4bd640 1 -- start start 2024-04-25T08:38:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cff4bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0cf815dc20 0x7f0cf80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cff4bd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cff4bd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf815e000 con 0x7f0cf8161710 2024-04-25T08:38:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cff4bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf80d1de0 con 0x7f0cf815dc20 2024-04-25T08:38:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cff4bd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8161c60 con 0x7f0cf80d32f0 2024-04-25T08:38:18.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cfd259640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0cf815dc20 0x7f0cf80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cfda5a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cfca58640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0cf8161710 0x7f0cf806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59840/0 (socket says 172.21.15.60:59840) 2024-04-25T08:38:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cfca58640 1 -- 172.21.15.60:0/2089715259 learned_addr learned my addr 172.21.15.60:0/2089715259 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.631+0000 7f0cfd259640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0cf815dc20 0x7f0cf80ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58580/0 (socket says 172.21.15.60:58580) 2024-04-25T08:38:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.632+0000 7f0cfd259640 1 -- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 msgr2=0x7f0cf80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.632+0000 7f0cfd259640 1 --2- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.632+0000 7f0cfd259640 1 -- 172.21.15.60:0/2089715259 >> v1:172.21.15.100:6789/0 conn(0x7f0cf8161710 legacy=0x7f0cf806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.632+0000 7f0ceffff640 1 -- 172.21.15.60:0/2089715259 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979868133 0 0) 0x7f0cf815e000 con 0x7f0cf8161710 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.632+0000 7f0cfd259640 1 -- 172.21.15.60:0/2089715259 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8161f40 con 0x7f0cf815dc20 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.632+0000 7f0cfda5a640 1 --2- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cfd259640 1 --2- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf815dc20 0x7f0cf80ce670 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40097a0 tx=0x7f0cf402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a814e4df474420a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0ceffff640 1 -- 172.21.15.60:0/2089715259 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4037070 con 0x7f0cf815dc20 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0ceffff640 1 -- 172.21.15.60:0/2089715259 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4032440 con 0x7f0cf815dc20 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0ceffff640 1 -- 172.21.15.60:0/2089715259 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4032810 con 0x7f0cf815dc20 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 -- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf815dc20 msgr2=0x7f0cf80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf815dc20 0x7f0cf80ce670 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40097a0 tx=0x7f0cf402a180 comp rx=0 tx=0).stop 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 -- 172.21.15.60:0/2089715259 shutdown_connections 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/2089715259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf815dc20 0x7f0cf80ce670 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 -- 172.21.15.60:0/2089715259 >> 172.21.15.60:0/2089715259 conn(0x7f0cf80d0420 msgr2=0x7f0cf80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.633+0000 7f0cff4bd640 1 -- 172.21.15.60:0/2089715259 shutdown_connections 2024-04-25T08:38:18.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.634+0000 7f0cff4bd640 1 -- 172.21.15.60:0/2089715259 wait complete. 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 Processor -- start 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 -- start start 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 0x7f0cf8156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf8161710 0x7f0cf815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b5e50 con 0x7f0cf815dc20 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8162500 con 0x7f0cf8161710 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cff4bd640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80b5fd0 con 0x7f0cf80d32f0 2024-04-25T08:38:18.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.635+0000 7f0cfca58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 0x7f0cf8156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 0x7f0cf8156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59310/0 (socket says 172.21.15.60:59310) 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 -- 172.21.15.60:0/512158703 learned_addr learned my addr 172.21.15.60:0/512158703 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfd259640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfda5a640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf8161710 0x7f0cf815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 -- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 msgr2=0x7f0cf815eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf815eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 -- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf8161710 msgr2=0x7f0cf815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf8161710 0x7f0cf815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 -- 172.21.15.60:0/512158703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf81567c0 con 0x7f0cf815dc20 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cfca58640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 0x7f0cf8156280 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c460 tx=0x7f0ce800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018070 con 0x7f0cf815dc20 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce8007500 con 0x7f0cf815dc20 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf80b1f20 con 0x7f0cf815dc20 2024-04-25T08:38:18.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8004a20 con 0x7f0cf815dc20 2024-04-25T08:38:18.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.636+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf80b24e0 con 0x7f0cf815dc20 2024-04-25T08:38:18.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.637+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf8073a00 con 0x7f0cf815dc20 2024-04-25T08:38:18.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.638+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ce8004bc0 con 0x7f0cf815dc20 2024-04-25T08:38:18.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.638+0000 7f0cedffb640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cdc06b4d0 0x7f0cdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.638+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ce8089700 con 0x7f0cf815dc20 2024-04-25T08:38:18.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.638+0000 7f0cfd259640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cdc06b4d0 0x7f0cdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.639+0000 7f0cfd259640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cdc06b4d0 0x7f0cdc06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4034000 tx=0x7f0cf402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.642+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ce8057c90 con 0x7f0cf815dc20 2024-04-25T08:38:18.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.858+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0cf80cdf00 con 0x7f0cf815dc20 2024-04-25T08:38:18.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.876+0000 7f0cedffb640 1 -- 172.21.15.60:0/512158703 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0ce805bd30 con 0x7f0cf815dc20 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.879+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cdc06b4d0 msgr2=0x7f0cdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.879+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cdc06b4d0 0x7f0cdc06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4034000 tx=0x7f0cf402e000 comp rx=0 tx=0).stop 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.879+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 msgr2=0x7f0cf8156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 0x7f0cf8156280 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800c460 tx=0x7f0ce800c920 comp rx=0 tx=0).stop 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 shutdown_connections 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0cdc06b4d0 0x7f0cdc06d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cf8161710 0x7f0cf815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0cf815dc20 0x7f0cf8156280 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 --2- 172.21.15.60:0/512158703 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0cf80d32f0 0x7f0cf815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 >> 172.21.15.60:0/512158703 conn(0x7f0cf80d0420 msgr2=0x7f0cf80d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 shutdown_connections 2024-04-25T08:38:18.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.880+0000 7f0cff4bd640 1 -- 172.21.15.60:0/512158703 wait complete. 2024-04-25T08:38:18.884 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:38:18.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:38:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.994+0000 7f20cbb90640 1 Processor -- start 2024-04-25T08:38:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.995+0000 7f20cbb90640 1 -- start start 2024-04-25T08:38:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.995+0000 7f20cbb90640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f20c41446d0 0x7f20c40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.995+0000 7f20cbb90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.995+0000 7f20cbb90640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40cccc0 con 0x7f20c4144d50 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.995+0000 7f20cbb90640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c40b2b20 con 0x7f20c41446d0 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.995+0000 7f20cbb90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c406d400 con 0x7f20c406e8a0 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c992c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f20c41446d0 0x7f20c40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c912b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f20c4144d50 0x7f20c40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38670/0 (socket says 172.21.15.60:38670) 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c912b640 1 -- 172.21.15.60:0/222431365 learned_addr learned my addr 172.21.15.60:0/222431365 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20ca12d640 1 --2- 172.21.15.60:0/222431365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c992c640 1 -- 172.21.15.60:0/222431365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 msgr2=0x7f20c40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c992c640 1 --2- 172.21.15.60:0/222431365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c992c640 1 -- 172.21.15.60:0/222431365 >> v1:172.21.15.60:6789/0 conn(0x7f20c4144d50 legacy=0x7f20c40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c892a640 1 -- 172.21.15.60:0/222431365 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389940862 0 0) 0x7f20c40cccc0 con 0x7f20c4144d50 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.996+0000 7f20c992c640 1 -- 172.21.15.60:0/222431365 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40cc140 con 0x7f20c41446d0 2024-04-25T08:38:18.991 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20c992c640 1 --2- 172.21.15.60:0/222431365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c41446d0 0x7f20c40b3f20 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f20c00097a0 tx=0x7f20c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edfd2db7599a745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20c892a640 1 -- 172.21.15.60:0/222431365 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0037070 con 0x7f20c41446d0 2024-04-25T08:38:18.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20c892a640 1 -- 172.21.15.60:0/222431365 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c0032440 con 0x7f20c41446d0 2024-04-25T08:38:18.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20c892a640 1 -- 172.21.15.60:0/222431365 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0032810 con 0x7f20c41446d0 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 -- 172.21.15.60:0/222431365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c41446d0 msgr2=0x7f20c40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 --2- 172.21.15.60:0/222431365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c41446d0 0x7f20c40b3f20 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f20c00097a0 tx=0x7f20c002a180 comp rx=0 tx=0).stop 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 -- 172.21.15.60:0/222431365 shutdown_connections 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 --2- 172.21.15.60:0/222431365 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 --2- 172.21.15.60:0/222431365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c41446d0 0x7f20c40b3f20 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 -- 172.21.15.60:0/222431365 >> 172.21.15.60:0/222431365 conn(0x7f20c406abe0 msgr2=0x7f20c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 -- 172.21.15.60:0/222431365 shutdown_connections 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.997+0000 7f20cbb90640 1 -- 172.21.15.60:0/222431365 wait complete. 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 Processor -- start 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 -- start start 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c41f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c41446d0 0x7f20c40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 0x7f20c40b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc700 con 0x7f20c4144d50 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b49a0 con 0x7f20c41446d0 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20cbb90640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40cc9e0 con 0x7f20c406e8a0 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20ca12d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 0x7f20c40b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20ca12d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 0x7f20c40b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59340/0 (socket says 172.21.15.60:59340) 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:18.999+0000 7f20ca12d640 1 -- 172.21.15.60:0/3663382470 learned_addr learned my addr 172.21.15.60:0/3663382470 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20c992c640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c41f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ca12d640 1 -- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 msgr2=0x7f20c41f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20c912b640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c41446d0 0x7f20c40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ca12d640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c41f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ca12d640 1 -- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c41446d0 msgr2=0x7f20c40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ca12d640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c41446d0 0x7f20c40b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ca12d640 1 -- 172.21.15.60:0/3663382470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40c49d0 con 0x7f20c4144d50 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ca12d640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 0x7f20c40b1810 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007e30 tx=0x7f20bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc014620 con 0x7f20c4144d50 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc019070 con 0x7f20c4144d50 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c40ad630 con 0x7f20c4144d50 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc014a00 con 0x7f20c4144d50 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.000+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40adbf0 con 0x7f20c4144d50 2024-04-25T08:38:18.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.001+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c4073a00 con 0x7f20c4144d50 2024-04-25T08:38:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.002+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20bc014ba0 con 0x7f20c4144d50 2024-04-25T08:38:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.002+0000 7f20ba7fc640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.002+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20bc08ac30 con 0x7f20c4144d50 2024-04-25T08:38:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.002+0000 7f20c992c640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.003+0000 7f20c992c640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20a806b4d0 0x7f20a806d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f20c0034000 tx=0x7f20c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:19.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.007+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20bc059920 con 0x7f20c4144d50 2024-04-25T08:38:19.217 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.223+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f20c406c250 con 0x7f20c4144d50 2024-04-25T08:38:19.844 INFO:tasks.ceph.mds.d.smithi100.stderr:2024-04-25T08:38:19.851+0000 7fef26121640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:38:19.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.882+0000 7f20ba7fc640 1 -- 172.21.15.60:0/3663382470 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20bc059220 con 0x7f20c4144d50 2024-04-25T08:38:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20a806b4d0 msgr2=0x7f20a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20a806b4d0 0x7f20a806d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f20c0034000 tx=0x7f20c002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 msgr2=0x7f20c40b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:19.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 0x7f20c40b1810 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f20bc007e30 tx=0x7f20bc00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 shutdown_connections 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20c4144d50 0x7f20c40b1810 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20a806b4d0 0x7f20a806d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20c41446d0 0x7f20c40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 --2- 172.21.15.60:0/3663382470 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20c406e8a0 0x7f20c41f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 >> 172.21.15.60:0/3663382470 conn(0x7f20c406abe0 msgr2=0x7f20c4143500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 shutdown_connections 2024-04-25T08:38:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:19.885+0000 7f20cbb90640 1 -- 172.21.15.60:0/3663382470 wait complete. 2024-04-25T08:38:19.893 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:38:19.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:20.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.008+0000 7fcc69fb7640 1 Processor -- start 2024-04-25T08:38:20.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.008+0000 7fcc69fb7640 1 -- start start 2024-04-25T08:38:20.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.009+0000 7fcc69fb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 0x7fcc640d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.009+0000 7fcc69fb7640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6406c220 con 0x7fcc6409a450 2024-04-25T08:38:20.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.009+0000 7fcc69fb7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640d0080 con 0x7fcc6415dc70 2024-04-25T08:38:20.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.009+0000 7fcc69fb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640d0c90 con 0x7fcc6406ce10 2024-04-25T08:38:20.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.009+0000 7fcc63fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fcc6409a450 0x7fcc640cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:38672/0 (socket says 172.21.15.60:38672) 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.009+0000 7fcc63fff640 1 -- 172.21.15.60:0/4027637803 learned_addr learned my addr 172.21.15.60:0/4027637803 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 0x7fcc640d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512696430 0 0) 0x7fcc6406c220 con 0x7fcc6409a450 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc4c003520 con 0x7fcc6409a450 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3782364643 0 0) 0x7fcc4c003520 con 0x7fcc6409a450 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 msgr2=0x7fcc640d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 0x7fcc640d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 >> v1:172.21.15.100:6789/0 conn(0x7fcc6415dc70 legacy=0x7fcc6406c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640d0360 con 0x7fcc6409a450 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fcc5c0019f0 con 0x7fcc6409a450 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 >> v1:172.21.15.60:6789/0 conn(0x7fcc6409a450 legacy=0x7fcc640cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 0x7fcc4c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4c003f20 0x7fcc4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc4c004d40 0x7fcc4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640d0c90 con 0x7fcc4c003f20 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00ba20 con 0x7fcc4c004d40 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00bc20 con 0x7fcc6406ce10 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4c003f20 0x7fcc4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc63fff640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc4c004d40 0x7fcc4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.010+0000 7fcc637fe640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 0x7fcc4c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc62ffd640 1 -- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 msgr2=0x7fcc4c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6406ce10 0x7fcc4c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc62ffd640 1 -- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc4c004d40 msgr2=0x7fcc4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc4c004d40 0x7fcc4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc62ffd640 1 -- 172.21.15.60:0/4027637803 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c003520 con 0x7fcc4c003f20 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4c003f20 0x7fcc4c004800 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009ac0 tx=0x7fcc4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09fd04735ca4211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58036070 con 0x7fcc4c003f20 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58031400 con 0x7fcc4c003f20 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.011+0000 7fcc627fc640 1 -- 172.21.15.60:0/4027637803 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58009690 con 0x7fcc4c003f20 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc69fb7640 1 -- 172.21.15.60:0/4027637803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4c003f20 msgr2=0x7fcc4c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/4027637803 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc4c003f20 0x7fcc4c004800 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009ac0 tx=0x7fcc4c005830 comp rx=0 tx=0).stop 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc63fff640 1 -- 172.21.15.60:0/4027637803 reap_dead start 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc69fb7640 1 -- 172.21.15.60:0/4027637803 shutdown_connections 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc69fb7640 1 -- 172.21.15.60:0/4027637803 >> 172.21.15.60:0/4027637803 conn(0x7fcc640b3270 msgr2=0x7fcc640b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc69fb7640 1 -- 172.21.15.60:0/4027637803 shutdown_connections 2024-04-25T08:38:20.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.012+0000 7fcc69fb7640 1 -- 172.21.15.60:0/4027637803 wait complete. 2024-04-25T08:38:20.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.013+0000 7fcc69fb7640 1 Processor -- start 2024-04-25T08:38:20.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.013+0000 7fcc69fb7640 1 -- start start 2024-04-25T08:38:20.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc69fb7640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc4c003f20 0x7fcc6415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc69fb7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 0x7fcc6409b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc69fb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6409bae0 0x7fcc640ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc69fb7640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6415e050 con 0x7fcc6415c920 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc69fb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640d0920 con 0x7fcc6409bae0 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc69fb7640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640d1120 con 0x7fcc4c003f20 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc62ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 0x7fcc6409b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc62ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 0x7fcc6409b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59358/0 (socket says 172.21.15.60:59358) 2024-04-25T08:38:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.014+0000 7fcc62ffd640 1 -- 172.21.15.60:0/3240664980 learned_addr learned my addr 172.21.15.60:0/3240664980 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc63fff640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6409bae0 0x7fcc640ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc637fe640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc4c003f20 0x7fcc6415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc62ffd640 1 -- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc4c003f20 msgr2=0x7fcc6415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc4c003f20 0x7fcc6415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc62ffd640 1 -- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6409bae0 msgr2=0x7fcc640ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6409bae0 0x7fcc640ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc62ffd640 1 -- 172.21.15.60:0/3240664980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640b23c0 con 0x7fcc6415c920 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc63fff640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6409bae0 0x7fcc640ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc637fe640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc4c003f20 0x7fcc6415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc62ffd640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 0x7fcc6409b5a0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fcc580099e0 tx=0x7fcc58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58040020 con 0x7fcc6415c920 2024-04-25T08:38:20.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58003c40 con 0x7fcc6415c920 2024-04-25T08:38:20.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc641f38f0 con 0x7fcc6415c920 2024-04-25T08:38:20.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58036040 con 0x7fcc6415c920 2024-04-25T08:38:20.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.015+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc641f3eb0 con 0x7fcc6415c920 2024-04-25T08:38:20.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.016+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64073990 con 0x7fcc6415c920 2024-04-25T08:38:20.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.016+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc58043020 con 0x7fcc6415c920 2024-04-25T08:38:20.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.017+0000 7fcc4bfff640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5006b500 0x7fcc5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.017+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc580aa520 con 0x7fcc6415c920 2024-04-25T08:38:20.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.017+0000 7fcc637fe640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5006b500 0x7fcc5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.018+0000 7fcc637fe640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5006b500 0x7fcc5006d9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcc640c8780 tx=0x7fcc54009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.021+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc5804a070 con 0x7fcc6415c920 2024-04-25T08:38:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.240+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc640cfca0 con 0x7fcc6415c920 2024-04-25T08:38:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.241+0000 7fcc4bfff640 1 -- 172.21.15.60:0/3240664980 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fcc580392d0 con 0x7fcc6415c920 2024-04-25T08:38:20.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:20.235 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:20.235 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.242+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5006b500 msgr2=0x7fcc5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.236 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.242+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5006b500 0x7fcc5006d9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcc640c8780 tx=0x7fcc54009040 comp rx=0 tx=0).stop 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.242+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 msgr2=0x7fcc6409b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.242+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 0x7fcc6409b5a0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fcc580099e0 tx=0x7fcc58004730 comp rx=0 tx=0).stop 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 shutdown_connections 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5006b500 0x7fcc5006d9c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6409bae0 0x7fcc640ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6415c920 0x7fcc6409b5a0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 --2- 172.21.15.60:0/3240664980 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc4c003f20 0x7fcc6415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 >> 172.21.15.60:0/3240664980 conn(0x7fcc640b3270 msgr2=0x7fcc640740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 shutdown_connections 2024-04-25T08:38:20.237 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.243+0000 7fcc69fb7640 1 -- 172.21.15.60:0/3240664980 wait complete. 2024-04-25T08:38:20.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:20.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.354+0000 7f5e47c6b640 1 Processor -- start 2024-04-25T08:38:20.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.355+0000 7f5e47c6b640 1 -- start start 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.355+0000 7f5e47c6b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e4015db60 0x7f5e400ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.355+0000 7f5e47c6b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.355+0000 7f5e47c6b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e400cec00 con 0x7f5e4015df40 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.355+0000 7f5e47c6b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400ce930 con 0x7f5e4015db60 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.355+0000 7f5e47c6b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400b6450 con 0x7f5e400b5f00 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e4015db60 0x7f5e400ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5e4015db60 0x7f5e400ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59370/0 (socket says 172.21.15.60:59370) 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 -- 172.21.15.60:0/1048614130 learned_addr learned my addr 172.21.15.60:0/1048614130 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:20.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 -- 172.21.15.60:0/1048614130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 msgr2=0x7f5e400d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45206640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 -- 172.21.15.60:0/1048614130 >> v1:172.21.15.100:6789/0 conn(0x7f5e4015df40 legacy=0x7f5e40161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45a07640 1 -- 172.21.15.60:0/1048614130 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400ceee0 con 0x7f5e4015db60 2024-04-25T08:38:20.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.356+0000 7f5e45206640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e45a07640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015db60 0x7f5e400ce3f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0098a0 tx=0x7f5e3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59bfe46d239a398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e44a05640 1 -- 172.21.15.60:0/1048614130 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037070 con 0x7f5e4015db60 2024-04-25T08:38:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e44a05640 1 -- 172.21.15.60:0/1048614130 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c032440 con 0x7f5e4015db60 2024-04-25T08:38:20.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e44a05640 1 -- 172.21.15.60:0/1048614130 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c032810 con 0x7f5e4015db60 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 -- 172.21.15.60:0/1048614130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015db60 msgr2=0x7f5e400ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015db60 0x7f5e400ce3f0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0098a0 tx=0x7f5e3c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 -- 172.21.15.60:0/1048614130 shutdown_connections 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/1048614130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015db60 0x7f5e400ce3f0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 -- 172.21.15.60:0/1048614130 >> 172.21.15.60:0/1048614130 conn(0x7f5e40061600 msgr2=0x7f5e40061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 -- 172.21.15.60:0/1048614130 shutdown_connections 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.357+0000 7f5e47c6b640 1 -- 172.21.15.60:0/1048614130 wait complete. 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 Processor -- start 2024-04-25T08:38:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 -- start start 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4015db60 0x7f5e401f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 0x7f5e400cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400d3280 con 0x7f5e4015df40 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400d1d50 con 0x7f5e4015db60 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e47c6b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400d3400 con 0x7f5e400b5f00 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e46208640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 0x7f5e400cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e46208640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 0x7f5e400cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59384/0 (socket says 172.21.15.60:59384) 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.359+0000 7f5e46208640 1 -- 172.21.15.60:0/2245631853 learned_addr learned my addr 172.21.15.60:0/2245631853 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e45a07640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e45206640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4015db60 0x7f5e401f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e46208640 1 -- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 msgr2=0x7f5e400c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e46208640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e46208640 1 -- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4015db60 msgr2=0x7f5e401f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e46208640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4015db60 0x7f5e401f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e46208640 1 -- 172.21.15.60:0/2245631853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e401f3950 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e45a07640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e45206640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4015db60 0x7f5e401f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e46208640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 0x7f5e400cca10 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800c780 tx=0x7f5e3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38004b60 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38004d00 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e400caeb0 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800aa30 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.360+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e400cb470 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.361+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e40073270 con 0x7f5e4015df40 2024-04-25T08:38:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.362+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e3800abd0 con 0x7f5e4015df40 2024-04-25T08:38:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.362+0000 7f5e367fc640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.362+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e3808a670 con 0x7f5e4015df40 2024-04-25T08:38:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.362+0000 7f5e45a07640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.363+0000 7f5e45a07640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.366+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e38020070 con 0x7f5e4015df40 2024-04-25T08:38:20.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.548+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e40073990 con 0x7f5e4015df40 2024-04-25T08:38:20.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.549+0000 7f5e367fc640 1 -- 172.21.15.60:0/2245631853 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f5e38059370 con 0x7f5e4015df40 2024-04-25T08:38:20.543 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:20.544 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":134,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:38:18.871661+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 msgr2=0x7f5e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 msgr2=0x7f5e400cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 0x7f5e400cca10 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800c780 tx=0x7f5e3800cc40 comp rx=0 tx=0).stop 2024-04-25T08:38:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 shutdown_connections 2024-04-25T08:38:20.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e4015df40 0x7f5e400cca10 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e4015db60 0x7f5e401f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 --2- 172.21.15.60:0/2245631853 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5e400b5f00 0x7f5e400c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 >> 172.21.15.60:0/2245631853 conn(0x7f5e40061600 msgr2=0x7f5e4015b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 shutdown_connections 2024-04-25T08:38:20.547 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.551+0000 7f5e47c6b640 1 -- 172.21.15.60:0/2245631853 wait complete. 2024-04-25T08:38:20.557 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:38:20.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:20.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.663+0000 7f97dde4d640 1 Processor -- start 2024-04-25T08:38:20.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.663+0000 7f97dde4d640 1 -- start start 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97dde4d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97d806ce10 0x7f97d80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97dde4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97dde4d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d806c220 con 0x7f97d815dc70 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97dde4d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d80d0c90 con 0x7f97d806ce10 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97dde4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80d0080 con 0x7f97d809a450 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97d6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97d806ce10 0x7f97d80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97d6ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97d806ce10 0x7f97d80d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59390/0 (socket says 172.21.15.60:59390) 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.664+0000 7f97d6ffd640 1 -- 172.21.15.60:0/1451892938 learned_addr learned my addr 172.21.15.60:0/1451892938 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.665+0000 7f97d7fff640 1 --2- 172.21.15.60:0/1451892938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.665+0000 7f97d6ffd640 1 -- 172.21.15.60:0/1451892938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 msgr2=0x7f97d80cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.665+0000 7f97d6ffd640 1 --2- 172.21.15.60:0/1451892938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.665+0000 7f97d6ffd640 1 -- 172.21.15.60:0/1451892938 >> v1:172.21.15.100:6789/0 conn(0x7f97d815dc70 legacy=0x7f97d806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.665+0000 7f97d6ffd640 1 -- 172.21.15.60:0/1451892938 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80d0360 con 0x7f97d806ce10 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.665+0000 7f97d6ffd640 1 --2- 172.21.15.60:0/1451892938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d806ce10 0x7f97d80d1120 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009870 tx=0x7f97cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324b2763fee245cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97d67fc640 1 -- 172.21.15.60:0/1451892938 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc037070 con 0x7f97d806ce10 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97d67fc640 1 -- 172.21.15.60:0/1451892938 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc032440 con 0x7f97d806ce10 2024-04-25T08:38:20.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97d67fc640 1 -- 172.21.15.60:0/1451892938 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc032810 con 0x7f97d806ce10 2024-04-25T08:38:20.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 -- 172.21.15.60:0/1451892938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d806ce10 msgr2=0x7f97d80d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 --2- 172.21.15.60:0/1451892938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d806ce10 0x7f97d80d1120 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009870 tx=0x7f97cc02a210 comp rx=0 tx=0).stop 2024-04-25T08:38:20.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 -- 172.21.15.60:0/1451892938 shutdown_connections 2024-04-25T08:38:20.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 --2- 172.21.15.60:0/1451892938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d806ce10 0x7f97d80d1120 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 --2- 172.21.15.60:0/1451892938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 -- 172.21.15.60:0/1451892938 >> 172.21.15.60:0/1451892938 conn(0x7f97d80b3270 msgr2=0x7f97d80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 -- 172.21.15.60:0/1451892938 shutdown_connections 2024-04-25T08:38:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.666+0000 7f97dde4d640 1 -- 172.21.15.60:0/1451892938 wait complete. 2024-04-25T08:38:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.667+0000 7f97dde4d640 1 Processor -- start 2024-04-25T08:38:20.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 -- start start 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97d806ce10 0x7f97d8156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 0x7f97d8158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d815e050 con 0x7f97d815dc70 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80d0920 con 0x7f97d806ce10 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97dde4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d80cbdd0 con 0x7f97d809a450 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 0x7f97d8158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 0x7f97d8158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59402/0 (socket says 172.21.15.60:59402) 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97d7fff640 1 -- 172.21.15.60:0/4028901737 learned_addr learned my addr 172.21.15.60:0/4028901737 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.668+0000 7f97d77fe640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97d806ce10 0x7f97d8156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d6ffd640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d7fff640 1 -- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 msgr2=0x7f97d80b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d7fff640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d7fff640 1 -- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97d806ce10 msgr2=0x7f97d8156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d7fff640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97d806ce10 0x7f97d8156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d7fff640 1 -- 172.21.15.60:0/4028901737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80b5c60 con 0x7f97d815dc70 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d77fe640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97d806ce10 0x7f97d8156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d6ffd640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97d7fff640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 0x7f97d8158030 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f97d0007e40 tx=0x7f97d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0014620 con 0x7f97d815dc70 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d0019070 con 0x7f97d815dc70 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0014a20 con 0x7f97d815dc70 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d80b5f40 con 0x7f97d815dc70 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.669+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d80c5350 con 0x7f97d815dc70 2024-04-25T08:38:20.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.670+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d8073990 con 0x7f97d815dc70 2024-04-25T08:38:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.671+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97d0014bc0 con 0x7f97d815dc70 2024-04-25T08:38:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.671+0000 7f97bffff640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97c406b500 0x7f97c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.671+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97d008ac20 con 0x7f97d815dc70 2024-04-25T08:38:20.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.671+0000 7f97d77fe640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97c406b500 0x7f97c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.672+0000 7f97d77fe640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97c406b500 0x7f97c406d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f97c8006fd0 tx=0x7f97c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.675+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97d00599a0 con 0x7f97d815dc70 2024-04-25T08:38:20.844 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.851+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97d80cdae0 con 0x7f97c406b500 2024-04-25T08:38:20.851 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.857+0000 7f97bffff640 1 -- 172.21.15.60:0/4028901737 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238401 (secure 0 0 0) 0x7f97d80cdae0 con 0x7f97c406b500 2024-04-25T08:38:20.854 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:20.855 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:38:20.857 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97c406b500 msgr2=0x7f97c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97c406b500 0x7f97c406d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f97c8006fd0 tx=0x7f97c8008040 comp rx=0 tx=0).stop 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 msgr2=0x7f97d8158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 0x7f97d8158030 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f97d0007e40 tx=0x7f97d000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 shutdown_connections 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97d815dc70 0x7f97d8158030 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97d809a450 0x7f97d80b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97c406b500 0x7f97c406d9c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 --2- 172.21.15.60:0/4028901737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97d806ce10 0x7f97d8156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 >> 172.21.15.60:0/4028901737 conn(0x7f97d80b3270 msgr2=0x7f97d8070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 shutdown_connections 2024-04-25T08:38:20.859 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:20.860+0000 7f97dde4d640 1 -- 172.21.15.60:0/4028901737 wait complete. 2024-04-25T08:38:20.865 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-04-25T08:38:19.521757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":30,"acting":30,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":428484,"kb_used_data":13820,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754546236,"statfs":{"total":773094113280,"available":772655345664,"internally_reserved":0,"allocated":14151680,"data_stored":7207843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000989"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853765+0000","last_change":"2024-04-25T08:38:15.853765+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853765+0000","last_undegraded":"2024-04-25T08:38:15.853765+0000","last_fullsized":"2024-04-25T08:38:15.853765+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942873+0000","last_change":"2024-04-25T08:38:15.942873+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942873+0000","last_undegraded":"2024-04-25T08:38:15.942873+0000","last_fullsized":"2024-04-25T08:38:15.942873+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942729+0000","last_change":"2024-04-25T08:38:15.942729+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942729+0000","last_undegraded":"2024-04-25T08:38:15.942729+0000","last_fullsized":"2024-04-25T08:38:15.942729+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853468+0000","last_change":"2024-04-25T08:38:15.853468+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853468+0000","last_undegraded":"2024-04-25T08:38:15.853468+0000","last_fullsized":"2024-04-25T08:38:15.853468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.856507+0000","last_change":"2024-04-25T08:38:15.856507+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.856507+0000","last_undegraded":"2024-04-25T08:38:15.856507+0000","last_fullsized":"2024-04-25T08:38:15.856507+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942269+0000","last_change":"2024-04-25T08:38:15.942269+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942269+0000","last_undegraded":"2024-04-25T08:38:15.942269+0000","last_fullsized":"2024-04-25T08:38:15.942269+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942210+0000","last_change":"2024-04-25T08:38:15.942210+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942210+0000","last_undegraded":"2024-04-25T08:38:15.942210+0000","last_fullsized":"2024-04-25T08:38:15.942210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853444+0000","last_change":"2024-04-25T08:38:15.853444+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853444+0000","last_undegraded":"2024-04-25T08:38:15.853444+0000","last_fullsized":"2024-04-25T08:38:15.853444+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.856037+0000","last_change":"2024-04-25T08:38:15.856037+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.856037+0000","last_undegraded":"2024-04-25T08:38:15.856037+0000","last_fullsized":"2024-04-25T08:38:15.856037+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853438+0000","last_change":"2024-04-25T08:38:15.853438+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853438+0000","last_undegraded":"2024-04-25T08:38:15.853438+0000","last_fullsized":"2024-04-25T08:38:15.853438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942182+0000","last_change":"2024-04-25T08:38:15.942182+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942182+0000","last_undegraded":"2024-04-25T08:38:15.942182+0000","last_fullsized":"2024-04-25T08:38:15.942182+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.855971+0000","last_change":"2024-04-25T08:38:15.855971+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.855971+0000","last_undegraded":"2024-04-25T08:38:15.855971+0000","last_fullsized":"2024-04-25T08:38:15.855971+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942267+0000","last_change":"2024-04-25T08:38:15.942267+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942267+0000","last_undegraded":"2024-04-25T08:38:15.942267+0000","last_fullsized":"2024-04-25T08:38:15.942267+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942727+0000","last_change":"2024-04-25T08:38:15.942727+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942727+0000","last_undegraded":"2024-04-25T08:38:15.942727+0000","last_fullsized":"2024-04-25T08:38:15.942727+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":375,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T08:38:15.943285+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:38:15.943285+0000","last_peered":"2024-04-25T08:38:15.943285+0000","last_clean":"2024-04-25T08:38:15.943285+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:38:15.943285+0000","last_undegraded":"2024-04-25T08:38:15.943285+0000","last_fullsized":"2024-04-25T08:38:15.943285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542257,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47248,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324592,"statfs":{"total":96636764160,"available":96588382208,"internally_reserved":0,"allocated":2228224,"data_stored":1350381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50824,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321016,"statfs":{"total":96636764160,"available":96584720384,"internally_reserved":0,"allocated":1634304,"data_stored":760013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542257,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":1634304,"data_stored":760013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50824,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321016,"statfs":{"total":96636764160,"available":96584720384,"internally_reserved":0,"allocated":1622016,"data_stored":753804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59088,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312752,"statfs":{"total":96636764160,"available":96576258048,"internally_reserved":0,"allocated":1572864,"data_stored":731852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59136,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312704,"statfs":{"total":96636764160,"available":96576208896,"internally_reserved":0,"allocated":1622016,"data_stored":753804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54984,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316856,"statfs":{"total":96636764160,"available":96580460544,"internally_reserved":0,"allocated":1622016,"data_stored":753804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59712,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312128,"statfs":{"total":96636764160,"available":96575619072,"internally_reserved":0,"allocated":2215936,"data_stored":1344172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:20.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 Processor -- start 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 -- start start 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015db60 0x7efda00ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda00d0b10 0x7efda00d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda00cec00 con 0x7efda00b5f00 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda00ce930 con 0x7efda00d0b10 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.013+0000 7efda683a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda00b6450 con 0x7efda015db60 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda00d0b10 0x7efda00d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7efda00d0b10 0x7efda00d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59412/0 (socket says 172.21.15.60:59412) 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 -- 172.21.15.60:0/2117801881 learned_addr learned my addr 172.21.15.60:0/2117801881 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efd9ffff640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015db60 0x7efda00ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 -- 172.21.15.60:0/2117801881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015db60 msgr2=0x7efda00ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015db60 0x7efda00ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 -- 172.21.15.60:0/2117801881 >> v1:172.21.15.100:6789/0 conn(0x7efda00b5f00 legacy=0x7efda00d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efda4dd7640 1 -- 172.21.15.60:0/2117801881 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00ceee0 con 0x7efda00d0b10 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.014+0000 7efd9ffff640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015db60 0x7efda00ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda4dd7640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda00d0b10 0x7efda00d2270 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e50 tx=0x7efd9802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa33e91ce34f0c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efd9effd640 1 -- 172.21.15.60:0/2117801881 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98036070 con 0x7efda00d0b10 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efd9effd640 1 -- 172.21.15.60:0/2117801881 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98031600 con 0x7efda00d0b10 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efd9effd640 1 -- 172.21.15.60:0/2117801881 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980319d0 con 0x7efda00d0b10 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 -- 172.21.15.60:0/2117801881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda00d0b10 msgr2=0x7efda00d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda00d0b10 0x7efda00d2270 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e50 tx=0x7efd9802a680 comp rx=0 tx=0).stop 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 -- 172.21.15.60:0/2117801881 shutdown_connections 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda00d0b10 0x7efda00d2270 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 --2- 172.21.15.60:0/2117801881 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda015db60 0x7efda00ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 -- 172.21.15.60:0/2117801881 >> 172.21.15.60:0/2117801881 conn(0x7efda0061600 msgr2=0x7efda0061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 -- 172.21.15.60:0/2117801881 shutdown_connections 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.015+0000 7efda683a640 1 -- 172.21.15.60:0/2117801881 wait complete. 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.016+0000 7efda683a640 1 Processor -- start 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 -- start start 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda00b5f00 0x7efda00b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda00d0b10 0x7efda0161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 0x7efda0161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda015df40 con 0x7efda015db60 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda00d1d50 con 0x7efda00b5f00 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda683a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda015e0c0 con 0x7efda00d0b10 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.017+0000 7efda4dd7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 0x7efda0161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 0x7efda0161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59420/0 (socket says 172.21.15.60:59420) 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 -- 172.21.15.60:0/701355012 learned_addr learned my addr 172.21.15.60:0/701355012 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efd9ffff640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda00b5f00 0x7efda00b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efd9f7fe640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda00d0b10 0x7efda0161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 -- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda00d0b10 msgr2=0x7efda0161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda00d0b10 0x7efda0161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 -- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda00b5f00 msgr2=0x7efda00b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda00b5f00 0x7efda00b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 -- 172.21.15.60:0/701355012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda01f46d0 con 0x7efda015db60 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efd9ffff640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda00b5f00 0x7efda00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda4dd7640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 0x7efda0161e40 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efd98034040 tx=0x7efd98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98036050 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98042070 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda0162550 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98009510 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.018+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda00cbf10 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.019+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda0073270 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.020+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd98007bd0 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.020+0000 7efd9cff9640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd8c06b4d0 0x7efd8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.020+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd980ab2e0 con 0x7efda015db60 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.020+0000 7efd9ffff640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd8c06b4d0 0x7efd8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.021+0000 7efd9ffff640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd8c06b4d0 0x7efd8c06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efd90006fd0 tx=0x7efd90008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.024+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd98045360 con 0x7efda015db60 2024-04-25T08:38:21.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.201+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efda00d1860 con 0x7efd8c06b4d0 2024-04-25T08:38:21.201 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.207+0000 7efd9cff9640 1 -- 172.21.15.60:0/701355012 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+238401 (secure 0 0 0) 0x7efda00d1860 con 0x7efd8c06b4d0 2024-04-25T08:38:21.203 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:21.204 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd8c06b4d0 msgr2=0x7efd8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd8c06b4d0 0x7efd8c06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efd90006fd0 tx=0x7efd90008040 comp rx=0 tx=0).stop 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 msgr2=0x7efda0161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 0x7efda0161e40 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efd98034040 tx=0x7efd98004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 shutdown_connections 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda015db60 0x7efda0161e40 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda00d0b10 0x7efda0161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd8c06b4d0 0x7efd8c06d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.206 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 --2- 172.21.15.60:0/701355012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda00b5f00 0x7efda00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 >> 172.21.15.60:0/701355012 conn(0x7efda0061600 msgr2=0x7efda0061bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 shutdown_connections 2024-04-25T08:38:21.207 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.210+0000 7efda683a640 1 -- 172.21.15.60:0/701355012 wait complete. 2024-04-25T08:38:21.215 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-04-25T08:38:19.521757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":30,"acting":30,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":428484,"kb_used_data":13820,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754546236,"statfs":{"total":773094113280,"available":772655345664,"internally_reserved":0,"allocated":14151680,"data_stored":7207843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000989"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853765+0000","last_change":"2024-04-25T08:38:15.853765+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853765+0000","last_undegraded":"2024-04-25T08:38:15.853765+0000","last_fullsized":"2024-04-25T08:38:15.853765+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942873+0000","last_change":"2024-04-25T08:38:15.942873+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942873+0000","last_undegraded":"2024-04-25T08:38:15.942873+0000","last_fullsized":"2024-04-25T08:38:15.942873+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942729+0000","last_change":"2024-04-25T08:38:15.942729+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942729+0000","last_undegraded":"2024-04-25T08:38:15.942729+0000","last_fullsized":"2024-04-25T08:38:15.942729+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853468+0000","last_change":"2024-04-25T08:38:15.853468+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853468+0000","last_undegraded":"2024-04-25T08:38:15.853468+0000","last_fullsized":"2024-04-25T08:38:15.853468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.856507+0000","last_change":"2024-04-25T08:38:15.856507+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.856507+0000","last_undegraded":"2024-04-25T08:38:15.856507+0000","last_fullsized":"2024-04-25T08:38:15.856507+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942269+0000","last_change":"2024-04-25T08:38:15.942269+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942269+0000","last_undegraded":"2024-04-25T08:38:15.942269+0000","last_fullsized":"2024-04-25T08:38:15.942269+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942210+0000","last_change":"2024-04-25T08:38:15.942210+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942210+0000","last_undegraded":"2024-04-25T08:38:15.942210+0000","last_fullsized":"2024-04-25T08:38:15.942210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853444+0000","last_change":"2024-04-25T08:38:15.853444+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853444+0000","last_undegraded":"2024-04-25T08:38:15.853444+0000","last_fullsized":"2024-04-25T08:38:15.853444+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.856037+0000","last_change":"2024-04-25T08:38:15.856037+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.856037+0000","last_undegraded":"2024-04-25T08:38:15.856037+0000","last_fullsized":"2024-04-25T08:38:15.856037+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.853438+0000","last_change":"2024-04-25T08:38:15.853438+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.853438+0000","last_undegraded":"2024-04-25T08:38:15.853438+0000","last_fullsized":"2024-04-25T08:38:15.853438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942182+0000","last_change":"2024-04-25T08:38:15.942182+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942182+0000","last_undegraded":"2024-04-25T08:38:15.942182+0000","last_fullsized":"2024-04-25T08:38:15.942182+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.855971+0000","last_change":"2024-04-25T08:38:15.855971+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.855971+0000","last_undegraded":"2024-04-25T08:38:15.855971+0000","last_fullsized":"2024-04-25T08:38:15.855971+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942267+0000","last_change":"2024-04-25T08:38:15.942267+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942267+0000","last_undegraded":"2024-04-25T08:38:15.942267+0000","last_fullsized":"2024-04-25T08:38:15.942267+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:15.846850+0000","last_change":"2024-04-25T08:38:15.846850+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.846850+0000","last_undegraded":"2024-04-25T08:38:15.846850+0000","last_fullsized":"2024-04-25T08:38:15.846850+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:38:16.850893+0000","last_change":"2024-04-25T08:38:16.850893+0000","last_active":"2024-04-25T08:38:16.850893+0000","last_peered":"2024-04-25T08:38:16.850893+0000","last_clean":"2024-04-25T08:38:16.850893+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:16.850893+0000","last_undegraded":"2024-04-25T08:38:16.850893+0000","last_fullsized":"2024-04-25T08:38:16.850893+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-04-25T08:38:15.942727+0000","last_change":"2024-04-25T08:38:15.942727+0000","last_active":"2024-04-25T08:38:15.846850+0000","last_peered":"2024-04-25T08:38:15.846850+0000","last_clean":"2024-04-25T08:38:15.846850+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:38:15.942727+0000","last_undegraded":"2024-04-25T08:38:15.942727+0000","last_fullsized":"2024-04-25T08:38:15.942727+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":375,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-04-25T08:38:15.943285+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:38:15.943285+0000","last_peered":"2024-04-25T08:38:15.943285+0000","last_clean":"2024-04-25T08:38:15.943285+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:38:15.943285+0000","last_undegraded":"2024-04-25T08:38:15.943285+0000","last_fullsized":"2024-04-25T08:38:15.943285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":28,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542257,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47248,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324592,"statfs":{"total":96636764160,"available":96588382208,"internally_reserved":0,"allocated":2228224,"data_stored":1350381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542257,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50824,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321016,"statfs":{"total":96636764160,"available":96584720384,"internally_reserved":0,"allocated":1634304,"data_stored":760013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542257,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":1634304,"data_stored":760013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50824,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321016,"statfs":{"total":96636764160,"available":96584720384,"internally_reserved":0,"allocated":1622016,"data_stored":753804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59088,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312752,"statfs":{"total":96636764160,"available":96576258048,"internally_reserved":0,"allocated":1572864,"data_stored":731852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59136,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312704,"statfs":{"total":96636764160,"available":96576208896,"internally_reserved":0,"allocated":1622016,"data_stored":753804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54984,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316856,"statfs":{"total":96636764160,"available":96580460544,"internally_reserved":0,"allocated":1622016,"data_stored":753804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59712,"kb_used_data":2164,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312128,"statfs":{"total":96636764160,"available":96575619072,"internally_reserved":0,"allocated":2215936,"data_stored":1344172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:21.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.363+0000 7f69b2316640 1 Processor -- start 2024-04-25T08:38:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.363+0000 7f69b2316640 1 -- start start 2024-04-25T08:38:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.364+0000 7f69b2316640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 0x7f69ac162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.364+0000 7f69b2316640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69ac0b5c30 0x7f69ac0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.364+0000 7f69b2316640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ac162690 con 0x7f69ac0b6cb0 2024-04-25T08:38:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.364+0000 7f69b2316640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ac0ceee0 con 0x7f69ac0b5c30 2024-04-25T08:38:21.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.364+0000 7f69b2316640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0d1d10 con 0x7f69ac15db70 2024-04-25T08:38:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69abfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 0x7f69ac162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69b08b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69ac0b5c30 0x7f69ac0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69ab7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f69ac0b6cb0 0x7f69ac0b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59880/0 (socket says 172.21.15.60:59880) 2024-04-25T08:38:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69abfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 0x7f69ac162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54300/0 (socket says 172.21.15.60:54300) 2024-04-25T08:38:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69b08b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f69ac0b5c30 0x7f69ac0d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:58676/0 (socket says 172.21.15.60:58676) 2024-04-25T08:38:21.368 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69ab7fe640 1 -- 172.21.15.60:0/1126230366 learned_addr learned my addr 172.21.15.60:0/1126230366 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69b08b3640 1 -- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 msgr2=0x7f69ac162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69b08b3640 1 --2- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 0x7f69ac162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69b08b3640 1 -- 172.21.15.60:0/1126230366 >> v1:172.21.15.100:6789/0 conn(0x7f69ac0b6cb0 legacy=0x7f69ac0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.369 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69b08b3640 1 -- 172.21.15.60:0/1126230366 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0d1ff0 con 0x7f69ac0b5c30 2024-04-25T08:38:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69aaffd640 1 -- 172.21.15.60:0/1126230366 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948353647 0 0) 0x7f69ac162690 con 0x7f69ac0b6cb0 2024-04-25T08:38:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.365+0000 7f69abfff640 1 --2- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 0x7f69ac162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.366+0000 7f69b08b3640 1 --2- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac0b5c30 0x7f69ac0d27f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009d90 tx=0x7f69a402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a49bc48b4abdfd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.366+0000 7f69aaffd640 1 -- 172.21.15.60:0/1126230366 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4031600 con 0x7f69ac0b5c30 2024-04-25T08:38:21.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.366+0000 7f69aaffd640 1 -- 172.21.15.60:0/1126230366 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a40317a0 con 0x7f69ac0b5c30 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.366+0000 7f69aaffd640 1 -- 172.21.15.60:0/1126230366 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4031b70 con 0x7f69ac0b5c30 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.366+0000 7f69b2316640 1 -- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac0b5c30 msgr2=0x7f69ac0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.366+0000 7f69b2316640 1 --2- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac0b5c30 0x7f69ac0d27f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009d90 tx=0x7f69a402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.367+0000 7f69b2316640 1 -- 172.21.15.60:0/1126230366 shutdown_connections 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.367+0000 7f69b2316640 1 --2- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac0b5c30 0x7f69ac0d27f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.367+0000 7f69b2316640 1 --2- 172.21.15.60:0/1126230366 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac15db70 0x7f69ac162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.367+0000 7f69b2316640 1 -- 172.21.15.60:0/1126230366 >> 172.21.15.60:0/1126230366 conn(0x7f69ac0d0520 msgr2=0x7f69ac0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.367+0000 7f69b2316640 1 -- 172.21.15.60:0/1126230366 shutdown_connections 2024-04-25T08:38:21.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.367+0000 7f69b2316640 1 -- 172.21.15.60:0/1126230366 wait complete. 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.368+0000 7f69b2316640 1 Processor -- start 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.368+0000 7f69b2316640 1 -- start start 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 0x7f69ac145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac0b6cb0 0x7f69ac143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac15db70 0x7f69ac15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0ce530 con 0x7f69ac0b5c30 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac15df50 con 0x7f69ac15db70 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac0cf1c0 con 0x7f69ac0b6cb0 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 0x7f69ac145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 0x7f69ac145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:59432/0 (socket says 172.21.15.60:59432) 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 -- 172.21.15.60:0/538479380 learned_addr learned my addr 172.21.15.60:0/538479380 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b08b3640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac15db70 0x7f69ac15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69ab7fe640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac0b6cb0 0x7f69ac143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 -- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac0b6cb0 msgr2=0x7f69ac143b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac0b6cb0 0x7f69ac143b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 -- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac15db70 msgr2=0x7f69ac15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac15db70 0x7f69ac15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 -- 172.21.15.60:0/538479380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac145960 con 0x7f69ac0b5c30 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69abfff640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 0x7f69ac145420 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f699c00e960 tx=0x7f699c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c017070 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699c00ca00 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ac144380 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.369+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c00cdd0 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.370+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac144940 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.370+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ac073990 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.371+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f699c01a070 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.371+0000 7f69a8ff9640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f699806b500 0x7f699806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.371+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f699c08a5b0 con 0x7f69ac0b5c30 2024-04-25T08:38:21.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.372+0000 7f69ab7fe640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f699806b500 0x7f699806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.372+0000 7f69ab7fe640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f699806b500 0x7f699806d9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0caa30 tx=0x7f69a0007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.376 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.376+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f699c058bb0 con 0x7f69ac0b5c30 2024-04-25T08:38:21.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.552+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69ac0d1820 con 0x7f699806b500 2024-04-25T08:38:21.553 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.559+0000 7f69a8ff9640 1 -- 172.21.15.60:0/538479380 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f69ac0d1820 con 0x7f699806b500 2024-04-25T08:38:21.556 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:21.558 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:38:21.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f699806b500 msgr2=0x7f699806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f699806b500 0x7f699806d9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0caa30 tx=0x7f69a0007d50 comp rx=0 tx=0).stop 2024-04-25T08:38:21.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 msgr2=0x7f69ac145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 0x7f69ac145420 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f699c00e960 tx=0x7f699c00ee20 comp rx=0 tx=0).stop 2024-04-25T08:38:21.561 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 shutdown_connections 2024-04-25T08:38:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f699806b500 0x7f699806d9c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f69ac15db70 0x7f69ac15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f69ac0b6cb0 0x7f69ac143b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 --2- 172.21.15.60:0/538479380 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69ac0b5c30 0x7f69ac145420 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 >> 172.21.15.60:0/538479380 conn(0x7f69ac0d0520 msgr2=0x7f69ac0d2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.562 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 shutdown_connections 2024-04-25T08:38:21.563 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.562+0000 7f69b2316640 1 -- 172.21.15.60:0/538479380 wait complete. 2024-04-25T08:38:21.567 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-25T08:38:21.522262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":428912,"kb_used_data":14268,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754545808,"statfs":{"total":773094113280,"available":772654907392,"internally_reserved":0,"allocated":14610432,"data_stored":7423112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.670599"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944892+0000","last_change":"2024-04-25T08:38:17.888831+0000","last_active":"2024-04-25T08:38:19.944892+0000","last_peered":"2024-04-25T08:38:19.944892+0000","last_clean":"2024-04-25T08:38:19.944892+0000","last_became_active":"2024-04-25T08:38:17.888425+0000","last_became_peered":"2024-04-25T08:38:17.888425+0000","last_unstale":"2024-04-25T08:38:19.944892+0000","last_undegraded":"2024-04-25T08:38:19.944892+0000","last_fullsized":"2024-04-25T08:38:19.944892+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:22.446450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882289+0000","last_change":"2024-04-25T08:38:17.878980+0000","last_active":"2024-04-25T08:38:19.882289+0000","last_peered":"2024-04-25T08:38:19.882289+0000","last_clean":"2024-04-25T08:38:19.882289+0000","last_became_active":"2024-04-25T08:38:17.878667+0000","last_became_peered":"2024-04-25T08:38:17.878667+0000","last_unstale":"2024-04-25T08:38:19.882289+0000","last_undegraded":"2024-04-25T08:38:19.882289+0000","last_fullsized":"2024-04-25T08:38:19.882289+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876695+0000","last_change":"2024-04-25T08:38:17.879831+0000","last_active":"2024-04-25T08:38:18.876695+0000","last_peered":"2024-04-25T08:38:18.876695+0000","last_clean":"2024-04-25T08:38:18.876695+0000","last_became_active":"2024-04-25T08:38:17.879554+0000","last_became_peered":"2024-04-25T08:38:17.879554+0000","last_unstale":"2024-04-25T08:38:18.876695+0000","last_undegraded":"2024-04-25T08:38:18.876695+0000","last_fullsized":"2024-04-25T08:38:18.876695+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882016+0000","last_change":"2024-04-25T08:38:17.891476+0000","last_active":"2024-04-25T08:38:19.882016+0000","last_peered":"2024-04-25T08:38:19.882016+0000","last_clean":"2024-04-25T08:38:19.882016+0000","last_became_active":"2024-04-25T08:38:17.891162+0000","last_became_peered":"2024-04-25T08:38:17.891162+0000","last_unstale":"2024-04-25T08:38:19.882016+0000","last_undegraded":"2024-04-25T08:38:19.882016+0000","last_fullsized":"2024-04-25T08:38:19.882016+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.057228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874423+0000","last_change":"2024-04-25T08:38:17.881621+0000","last_active":"2024-04-25T08:38:18.874423+0000","last_peered":"2024-04-25T08:38:18.874423+0000","last_clean":"2024-04-25T08:38:18.874423+0000","last_became_active":"2024-04-25T08:38:17.881287+0000","last_became_peered":"2024-04-25T08:38:17.881287+0000","last_unstale":"2024-04-25T08:38:18.874423+0000","last_undegraded":"2024-04-25T08:38:18.874423+0000","last_fullsized":"2024-04-25T08:38:18.874423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943745+0000","last_change":"2024-04-25T08:38:17.874836+0000","last_active":"2024-04-25T08:38:19.943745+0000","last_peered":"2024-04-25T08:38:19.943745+0000","last_clean":"2024-04-25T08:38:19.943745+0000","last_became_active":"2024-04-25T08:38:17.874196+0000","last_became_peered":"2024-04-25T08:38:17.874196+0000","last_unstale":"2024-04-25T08:38:19.943745+0000","last_undegraded":"2024-04-25T08:38:19.943745+0000","last_fullsized":"2024-04-25T08:38:19.943745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875655+0000","last_change":"2024-04-25T08:38:17.894302+0000","last_active":"2024-04-25T08:38:18.875655+0000","last_peered":"2024-04-25T08:38:18.875655+0000","last_clean":"2024-04-25T08:38:18.875655+0000","last_became_active":"2024-04-25T08:38:17.893594+0000","last_became_peered":"2024-04-25T08:38:17.893594+0000","last_unstale":"2024-04-25T08:38:18.875655+0000","last_undegraded":"2024-04-25T08:38:18.875655+0000","last_fullsized":"2024-04-25T08:38:18.875655+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:58.149171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879387+0000","last_change":"2024-04-25T08:38:17.885426+0000","last_active":"2024-04-25T08:38:19.879387+0000","last_peered":"2024-04-25T08:38:19.879387+0000","last_clean":"2024-04-25T08:38:19.879387+0000","last_became_active":"2024-04-25T08:38:17.885142+0000","last_became_peered":"2024-04-25T08:38:17.885142+0000","last_unstale":"2024-04-25T08:38:19.879387+0000","last_undegraded":"2024-04-25T08:38:19.879387+0000","last_fullsized":"2024-04-25T08:38:19.879387+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:17.119223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944491+0000","last_change":"2024-04-25T08:38:17.881404+0000","last_active":"2024-04-25T08:38:19.944491+0000","last_peered":"2024-04-25T08:38:19.944491+0000","last_clean":"2024-04-25T08:38:19.944491+0000","last_became_active":"2024-04-25T08:38:17.881129+0000","last_became_peered":"2024-04-25T08:38:17.881129+0000","last_unstale":"2024-04-25T08:38:19.944491+0000","last_undegraded":"2024-04-25T08:38:19.944491+0000","last_fullsized":"2024-04-25T08:38:19.944491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.466242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943737+0000","last_change":"2024-04-25T08:38:17.886835+0000","last_active":"2024-04-25T08:38:19.943737+0000","last_peered":"2024-04-25T08:38:19.943737+0000","last_clean":"2024-04-25T08:38:19.943737+0000","last_became_active":"2024-04-25T08:38:17.886530+0000","last_became_peered":"2024-04-25T08:38:17.886530+0000","last_unstale":"2024-04-25T08:38:19.943737+0000","last_undegraded":"2024-04-25T08:38:19.943737+0000","last_fullsized":"2024-04-25T08:38:19.943737+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:02.753734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881989+0000","last_change":"2024-04-25T08:38:17.887506+0000","last_active":"2024-04-25T08:38:19.881989+0000","last_peered":"2024-04-25T08:38:19.881989+0000","last_clean":"2024-04-25T08:38:19.881989+0000","last_became_active":"2024-04-25T08:38:17.884413+0000","last_became_peered":"2024-04-25T08:38:17.884413+0000","last_unstale":"2024-04-25T08:38:19.881989+0000","last_undegraded":"2024-04-25T08:38:19.881989+0000","last_fullsized":"2024-04-25T08:38:19.881989+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:55.340733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944346+0000","last_change":"2024-04-25T08:38:17.882319+0000","last_active":"2024-04-25T08:38:19.944346+0000","last_peered":"2024-04-25T08:38:19.944346+0000","last_clean":"2024-04-25T08:38:19.944346+0000","last_became_active":"2024-04-25T08:38:17.881864+0000","last_became_peered":"2024-04-25T08:38:17.881864+0000","last_unstale":"2024-04-25T08:38:19.944346+0000","last_undegraded":"2024-04-25T08:38:19.944346+0000","last_fullsized":"2024-04-25T08:38:19.944346+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:35.454814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944626+0000","last_change":"2024-04-25T08:38:17.875563+0000","last_active":"2024-04-25T08:38:19.944626+0000","last_peered":"2024-04-25T08:38:19.944626+0000","last_clean":"2024-04-25T08:38:19.944626+0000","last_became_active":"2024-04-25T08:38:17.875187+0000","last_became_peered":"2024-04-25T08:38:17.875187+0000","last_unstale":"2024-04-25T08:38:19.944626+0000","last_undegraded":"2024-04-25T08:38:19.944626+0000","last_fullsized":"2024-04-25T08:38:19.944626+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:31.918313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944527+0000","last_change":"2024-04-25T08:38:17.880229+0000","last_active":"2024-04-25T08:38:19.944527+0000","last_peered":"2024-04-25T08:38:19.944527+0000","last_clean":"2024-04-25T08:38:19.944527+0000","last_became_active":"2024-04-25T08:38:17.879911+0000","last_became_peered":"2024-04-25T08:38:17.879911+0000","last_unstale":"2024-04-25T08:38:19.944527+0000","last_undegraded":"2024-04-25T08:38:19.944527+0000","last_fullsized":"2024-04-25T08:38:19.944527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.772260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943620+0000","last_change":"2024-04-25T08:38:17.881918+0000","last_active":"2024-04-25T08:38:19.943620+0000","last_peered":"2024-04-25T08:38:19.943620+0000","last_clean":"2024-04-25T08:38:19.943620+0000","last_became_active":"2024-04-25T08:38:17.881591+0000","last_became_peered":"2024-04-25T08:38:17.881591+0000","last_unstale":"2024-04-25T08:38:19.943620+0000","last_undegraded":"2024-04-25T08:38:19.943620+0000","last_fullsized":"2024-04-25T08:38:19.943620+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:41.225407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943933+0000","last_change":"2024-04-25T08:38:17.882032+0000","last_active":"2024-04-25T08:38:19.943933+0000","last_peered":"2024-04-25T08:38:19.943933+0000","last_clean":"2024-04-25T08:38:19.943933+0000","last_became_active":"2024-04-25T08:38:17.881535+0000","last_became_peered":"2024-04-25T08:38:17.881535+0000","last_unstale":"2024-04-25T08:38:19.943933+0000","last_undegraded":"2024-04-25T08:38:19.943933+0000","last_fullsized":"2024-04-25T08:38:19.943933+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944573+0000","last_change":"2024-04-25T08:38:17.882058+0000","last_active":"2024-04-25T08:38:19.944573+0000","last_peered":"2024-04-25T08:38:19.944573+0000","last_clean":"2024-04-25T08:38:19.944573+0000","last_became_active":"2024-04-25T08:38:17.881728+0000","last_became_peered":"2024-04-25T08:38:17.881728+0000","last_unstale":"2024-04-25T08:38:19.944573+0000","last_undegraded":"2024-04-25T08:38:19.944573+0000","last_fullsized":"2024-04-25T08:38:19.944573+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:13.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874594+0000","last_change":"2024-04-25T08:38:17.882056+0000","last_active":"2024-04-25T08:38:18.874594+0000","last_peered":"2024-04-25T08:38:18.874594+0000","last_clean":"2024-04-25T08:38:18.874594+0000","last_became_active":"2024-04-25T08:38:17.881737+0000","last_became_peered":"2024-04-25T08:38:17.881737+0000","last_unstale":"2024-04-25T08:38:18.874594+0000","last_undegraded":"2024-04-25T08:38:18.874594+0000","last_fullsized":"2024-04-25T08:38:18.874594+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877346+0000","last_change":"2024-04-25T08:38:17.896592+0000","last_active":"2024-04-25T08:38:18.877346+0000","last_peered":"2024-04-25T08:38:18.877346+0000","last_clean":"2024-04-25T08:38:18.877346+0000","last_became_active":"2024-04-25T08:38:17.894196+0000","last_became_peered":"2024-04-25T08:38:17.894196+0000","last_unstale":"2024-04-25T08:38:18.877346+0000","last_undegraded":"2024-04-25T08:38:18.877346+0000","last_fullsized":"2024-04-25T08:38:18.877346+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.093419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874582+0000","last_change":"2024-04-25T08:38:17.894097+0000","last_active":"2024-04-25T08:38:18.874582+0000","last_peered":"2024-04-25T08:38:18.874582+0000","last_clean":"2024-04-25T08:38:18.874582+0000","last_became_active":"2024-04-25T08:38:17.893173+0000","last_became_peered":"2024-04-25T08:38:17.893173+0000","last_unstale":"2024-04-25T08:38:18.874582+0000","last_undegraded":"2024-04-25T08:38:18.874582+0000","last_fullsized":"2024-04-25T08:38:18.874582+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881956+0000","last_change":"2024-04-25T08:38:17.879586+0000","last_active":"2024-04-25T08:38:19.881956+0000","last_peered":"2024-04-25T08:38:19.881956+0000","last_clean":"2024-04-25T08:38:19.881956+0000","last_became_active":"2024-04-25T08:38:17.879289+0000","last_became_peered":"2024-04-25T08:38:17.879289+0000","last_unstale":"2024-04-25T08:38:19.881956+0000","last_undegraded":"2024-04-25T08:38:19.881956+0000","last_fullsized":"2024-04-25T08:38:19.881956+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.675286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944386+0000","last_change":"2024-04-25T08:38:17.880984+0000","last_active":"2024-04-25T08:38:19.944386+0000","last_peered":"2024-04-25T08:38:19.944386+0000","last_clean":"2024-04-25T08:38:19.944386+0000","last_became_active":"2024-04-25T08:38:17.880594+0000","last_became_peered":"2024-04-25T08:38:17.880594+0000","last_unstale":"2024-04-25T08:38:19.944386+0000","last_undegraded":"2024-04-25T08:38:19.944386+0000","last_fullsized":"2024-04-25T08:38:19.944386+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:06.858902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882372+0000","last_change":"2024-04-25T08:38:17.881329+0000","last_active":"2024-04-25T08:38:19.882372+0000","last_peered":"2024-04-25T08:38:19.882372+0000","last_clean":"2024-04-25T08:38:19.882372+0000","last_became_active":"2024-04-25T08:38:17.881014+0000","last_became_peered":"2024-04-25T08:38:17.881014+0000","last_unstale":"2024-04-25T08:38:19.882372+0000","last_undegraded":"2024-04-25T08:38:19.882372+0000","last_fullsized":"2024-04-25T08:38:19.882372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944094+0000","last_change":"2024-04-25T08:38:17.876698+0000","last_active":"2024-04-25T08:38:19.944094+0000","last_peered":"2024-04-25T08:38:19.944094+0000","last_clean":"2024-04-25T08:38:19.944094+0000","last_became_active":"2024-04-25T08:38:17.876364+0000","last_became_peered":"2024-04-25T08:38:17.876364+0000","last_unstale":"2024-04-25T08:38:19.944094+0000","last_undegraded":"2024-04-25T08:38:19.944094+0000","last_fullsized":"2024-04-25T08:38:19.944094+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:31.656833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876611+0000","last_change":"2024-04-25T08:38:17.871784+0000","last_active":"2024-04-25T08:38:18.876611+0000","last_peered":"2024-04-25T08:38:18.876611+0000","last_clean":"2024-04-25T08:38:18.876611+0000","last_became_active":"2024-04-25T08:38:17.870733+0000","last_became_peered":"2024-04-25T08:38:17.870733+0000","last_unstale":"2024-04-25T08:38:18.876611+0000","last_undegraded":"2024-04-25T08:38:18.876611+0000","last_fullsized":"2024-04-25T08:38:18.876611+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876372+0000","last_change":"2024-04-25T08:38:17.887048+0000","last_active":"2024-04-25T08:38:18.876372+0000","last_peered":"2024-04-25T08:38:18.876372+0000","last_clean":"2024-04-25T08:38:18.876372+0000","last_became_active":"2024-04-25T08:38:17.886769+0000","last_became_peered":"2024-04-25T08:38:17.886769+0000","last_unstale":"2024-04-25T08:38:18.876372+0000","last_undegraded":"2024-04-25T08:38:18.876372+0000","last_fullsized":"2024-04-25T08:38:18.876372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:06.919441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877229+0000","last_change":"2024-04-25T08:38:17.898751+0000","last_active":"2024-04-25T08:38:18.877229+0000","last_peered":"2024-04-25T08:38:18.877229+0000","last_clean":"2024-04-25T08:38:18.877229+0000","last_became_active":"2024-04-25T08:38:17.898466+0000","last_became_peered":"2024-04-25T08:38:17.898466+0000","last_unstale":"2024-04-25T08:38:18.877229+0000","last_undegraded":"2024-04-25T08:38:18.877229+0000","last_fullsized":"2024-04-25T08:38:18.877229+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:05.202461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943893+0000","last_change":"2024-04-25T08:38:17.884795+0000","last_active":"2024-04-25T08:38:19.943893+0000","last_peered":"2024-04-25T08:38:19.943893+0000","last_clean":"2024-04-25T08:38:19.943893+0000","last_became_active":"2024-04-25T08:38:17.884380+0000","last_became_peered":"2024-04-25T08:38:17.884380+0000","last_unstale":"2024-04-25T08:38:19.943893+0000","last_undegraded":"2024-04-25T08:38:19.943893+0000","last_fullsized":"2024-04-25T08:38:19.943893+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:44.628191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876361+0000","last_change":"2024-04-25T08:38:17.872093+0000","last_active":"2024-04-25T08:38:18.876361+0000","last_peered":"2024-04-25T08:38:18.876361+0000","last_clean":"2024-04-25T08:38:18.876361+0000","last_became_active":"2024-04-25T08:38:17.871801+0000","last_became_peered":"2024-04-25T08:38:17.871801+0000","last_unstale":"2024-04-25T08:38:18.876361+0000","last_undegraded":"2024-04-25T08:38:18.876361+0000","last_fullsized":"2024-04-25T08:38:18.876361+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.057228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876355+0000","last_change":"2024-04-25T08:38:17.871321+0000","last_active":"2024-04-25T08:38:18.876355+0000","last_peered":"2024-04-25T08:38:18.876355+0000","last_clean":"2024-04-25T08:38:18.876355+0000","last_became_active":"2024-04-25T08:38:17.870448+0000","last_became_peered":"2024-04-25T08:38:17.870448+0000","last_unstale":"2024-04-25T08:38:18.876355+0000","last_undegraded":"2024-04-25T08:38:18.876355+0000","last_fullsized":"2024-04-25T08:38:18.876355+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:55.340733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879425+0000","last_change":"2024-04-25T08:38:17.874726+0000","last_active":"2024-04-25T08:38:19.879425+0000","last_peered":"2024-04-25T08:38:19.879425+0000","last_clean":"2024-04-25T08:38:19.879425+0000","last_became_active":"2024-04-25T08:38:17.874458+0000","last_became_peered":"2024-04-25T08:38:17.874458+0000","last_unstale":"2024-04-25T08:38:19.879425+0000","last_undegraded":"2024-04-25T08:38:19.879425+0000","last_fullsized":"2024-04-25T08:38:19.879425+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:47.779597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943170+0000","last_change":"2024-04-25T08:38:17.884450+0000","last_active":"2024-04-25T08:38:19.943170+0000","last_peered":"2024-04-25T08:38:19.943170+0000","last_clean":"2024-04-25T08:38:19.943170+0000","last_became_active":"2024-04-25T08:38:17.884060+0000","last_became_peered":"2024-04-25T08:38:17.884060+0000","last_unstale":"2024-04-25T08:38:19.943170+0000","last_undegraded":"2024-04-25T08:38:19.943170+0000","last_fullsized":"2024-04-25T08:38:19.943170+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:21.455456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879358+0000","last_change":"2024-04-25T08:38:17.885953+0000","last_active":"2024-04-25T08:38:19.879358+0000","last_peered":"2024-04-25T08:38:19.879358+0000","last_clean":"2024-04-25T08:38:19.879358+0000","last_became_active":"2024-04-25T08:38:17.885687+0000","last_became_peered":"2024-04-25T08:38:17.885687+0000","last_unstale":"2024-04-25T08:38:19.879358+0000","last_undegraded":"2024-04-25T08:38:19.879358+0000","last_fullsized":"2024-04-25T08:38:19.879358+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.490629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876440+0000","last_change":"2024-04-25T08:38:16.875570+0000","last_active":"2024-04-25T08:38:18.876440+0000","last_peered":"2024-04-25T08:38:18.876440+0000","last_clean":"2024-04-25T08:38:18.876440+0000","last_became_active":"2024-04-25T08:38:16.875258+0000","last_became_peered":"2024-04-25T08:38:16.875258+0000","last_unstale":"2024-04-25T08:38:18.876440+0000","last_undegraded":"2024-04-25T08:38:18.876440+0000","last_fullsized":"2024-04-25T08:38:18.876440+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:45.186789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943861+0000","last_change":"2024-04-25T08:38:17.877731+0000","last_active":"2024-04-25T08:38:19.943861+0000","last_peered":"2024-04-25T08:38:19.943861+0000","last_clean":"2024-04-25T08:38:19.943861+0000","last_became_active":"2024-04-25T08:38:17.877437+0000","last_became_peered":"2024-04-25T08:38:17.877437+0000","last_unstale":"2024-04-25T08:38:19.943861+0000","last_undegraded":"2024-04-25T08:38:19.943861+0000","last_fullsized":"2024-04-25T08:38:19.943861+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:27.127276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874830+0000","last_change":"2024-04-25T08:38:16.869588+0000","last_active":"2024-04-25T08:38:18.874830+0000","last_peered":"2024-04-25T08:38:18.874830+0000","last_clean":"2024-04-25T08:38:18.874830+0000","last_became_active":"2024-04-25T08:38:16.868936+0000","last_became_peered":"2024-04-25T08:38:16.868936+0000","last_unstale":"2024-04-25T08:38:18.874830+0000","last_undegraded":"2024-04-25T08:38:18.874830+0000","last_fullsized":"2024-04-25T08:38:18.874830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:29.799312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882046+0000","last_change":"2024-04-25T08:38:17.880219+0000","last_active":"2024-04-25T08:38:19.882046+0000","last_peered":"2024-04-25T08:38:19.882046+0000","last_clean":"2024-04-25T08:38:19.882046+0000","last_became_active":"2024-04-25T08:38:17.879938+0000","last_became_peered":"2024-04-25T08:38:17.879938+0000","last_unstale":"2024-04-25T08:38:19.882046+0000","last_undegraded":"2024-04-25T08:38:19.882046+0000","last_fullsized":"2024-04-25T08:38:19.882046+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:06.919441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877294+0000","last_change":"2024-04-25T08:38:16.877474+0000","last_active":"2024-04-25T08:38:18.877294+0000","last_peered":"2024-04-25T08:38:18.877294+0000","last_clean":"2024-04-25T08:38:18.877294+0000","last_became_active":"2024-04-25T08:38:16.877245+0000","last_became_peered":"2024-04-25T08:38:16.877245+0000","last_unstale":"2024-04-25T08:38:18.877294+0000","last_undegraded":"2024-04-25T08:38:18.877294+0000","last_fullsized":"2024-04-25T08:38:18.877294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.336690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943370+0000","last_change":"2024-04-25T08:38:17.886473+0000","last_active":"2024-04-25T08:38:19.943370+0000","last_peered":"2024-04-25T08:38:19.943370+0000","last_clean":"2024-04-25T08:38:19.943370+0000","last_became_active":"2024-04-25T08:38:17.885797+0000","last_became_peered":"2024-04-25T08:38:17.885797+0000","last_unstale":"2024-04-25T08:38:19.943370+0000","last_undegraded":"2024-04-25T08:38:19.943370+0000","last_fullsized":"2024-04-25T08:38:19.943370+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:16.987654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877062+0000","last_change":"2024-04-25T08:38:16.868822+0000","last_active":"2024-04-25T08:38:18.877062+0000","last_peered":"2024-04-25T08:38:18.877062+0000","last_clean":"2024-04-25T08:38:18.877062+0000","last_became_active":"2024-04-25T08:38:16.868564+0000","last_became_peered":"2024-04-25T08:38:16.868564+0000","last_unstale":"2024-04-25T08:38:18.877062+0000","last_undegraded":"2024-04-25T08:38:18.877062+0000","last_fullsized":"2024-04-25T08:38:18.877062+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:27.367777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879613+0000","last_change":"2024-04-25T08:38:17.872175+0000","last_active":"2024-04-25T08:38:19.879613+0000","last_peered":"2024-04-25T08:38:19.879613+0000","last_clean":"2024-04-25T08:38:19.879613+0000","last_became_active":"2024-04-25T08:38:17.871909+0000","last_became_peered":"2024-04-25T08:38:17.871909+0000","last_unstale":"2024-04-25T08:38:19.879613+0000","last_undegraded":"2024-04-25T08:38:19.879613+0000","last_fullsized":"2024-04-25T08:38:19.879613+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:32.701570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879631+0000","last_change":"2024-04-25T08:38:16.869047+0000","last_active":"2024-04-25T08:38:19.879631+0000","last_peered":"2024-04-25T08:38:19.879631+0000","last_clean":"2024-04-25T08:38:19.879631+0000","last_became_active":"2024-04-25T08:38:16.868570+0000","last_became_peered":"2024-04-25T08:38:16.868570+0000","last_unstale":"2024-04-25T08:38:19.879631+0000","last_undegraded":"2024-04-25T08:38:19.879631+0000","last_fullsized":"2024-04-25T08:38:19.879631+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:39.591232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879783+0000","last_change":"2024-04-25T08:38:17.887891+0000","last_active":"2024-04-25T08:38:19.879783+0000","last_peered":"2024-04-25T08:38:19.879783+0000","last_clean":"2024-04-25T08:38:19.879783+0000","last_became_active":"2024-04-25T08:38:17.887612+0000","last_became_peered":"2024-04-25T08:38:17.887612+0000","last_unstale":"2024-04-25T08:38:19.879783+0000","last_undegraded":"2024-04-25T08:38:19.879783+0000","last_fullsized":"2024-04-25T08:38:19.879783+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.803355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874718+0000","last_change":"2024-04-25T08:38:16.868606+0000","last_active":"2024-04-25T08:38:18.874718+0000","last_peered":"2024-04-25T08:38:18.874718+0000","last_clean":"2024-04-25T08:38:18.874718+0000","last_became_active":"2024-04-25T08:38:16.868290+0000","last_became_peered":"2024-04-25T08:38:16.868290+0000","last_unstale":"2024-04-25T08:38:18.874718+0000","last_undegraded":"2024-04-25T08:38:18.874718+0000","last_fullsized":"2024-04-25T08:38:18.874718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.697527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874750+0000","last_change":"2024-04-25T08:38:17.887163+0000","last_active":"2024-04-25T08:38:18.874750+0000","last_peered":"2024-04-25T08:38:18.874750+0000","last_clean":"2024-04-25T08:38:18.874750+0000","last_became_active":"2024-04-25T08:38:17.886853+0000","last_became_peered":"2024-04-25T08:38:17.886853+0000","last_unstale":"2024-04-25T08:38:18.874750+0000","last_undegraded":"2024-04-25T08:38:18.874750+0000","last_fullsized":"2024-04-25T08:38:18.874750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:31.656833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944174+0000","last_change":"2024-04-25T08:38:16.879982+0000","last_active":"2024-04-25T08:38:19.944174+0000","last_peered":"2024-04-25T08:38:19.944174+0000","last_clean":"2024-04-25T08:38:19.944174+0000","last_became_active":"2024-04-25T08:38:16.879692+0000","last_became_peered":"2024-04-25T08:38:16.879692+0000","last_unstale":"2024-04-25T08:38:19.944174+0000","last_undegraded":"2024-04-25T08:38:19.944174+0000","last_fullsized":"2024-04-25T08:38:19.944174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.742158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877301+0000","last_change":"2024-04-25T08:38:17.895788+0000","last_active":"2024-04-25T08:38:18.877301+0000","last_peered":"2024-04-25T08:38:18.877301+0000","last_clean":"2024-04-25T08:38:18.877301+0000","last_became_active":"2024-04-25T08:38:17.895479+0000","last_became_peered":"2024-04-25T08:38:17.895479+0000","last_unstale":"2024-04-25T08:38:18.877301+0000","last_undegraded":"2024-04-25T08:38:18.877301+0000","last_fullsized":"2024-04-25T08:38:18.877301+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:04.128752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876368+0000","last_change":"2024-04-25T08:38:16.865526+0000","last_active":"2024-04-25T08:38:18.876368+0000","last_peered":"2024-04-25T08:38:18.876368+0000","last_clean":"2024-04-25T08:38:18.876368+0000","last_became_active":"2024-04-25T08:38:16.865135+0000","last_became_peered":"2024-04-25T08:38:16.865135+0000","last_unstale":"2024-04-25T08:38:18.876368+0000","last_undegraded":"2024-04-25T08:38:18.876368+0000","last_fullsized":"2024-04-25T08:38:18.876368+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.197963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943446+0000","last_change":"2024-04-25T08:38:17.887207+0000","last_active":"2024-04-25T08:38:19.943446+0000","last_peered":"2024-04-25T08:38:19.943446+0000","last_clean":"2024-04-25T08:38:19.943446+0000","last_became_active":"2024-04-25T08:38:17.886920+0000","last_became_peered":"2024-04-25T08:38:17.886920+0000","last_unstale":"2024-04-25T08:38:19.943446+0000","last_undegraded":"2024-04-25T08:38:19.943446+0000","last_fullsized":"2024-04-25T08:38:19.943446+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:45.695945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882289+0000","last_change":"2024-04-25T08:38:16.870861+0000","last_active":"2024-04-25T08:38:19.882289+0000","last_peered":"2024-04-25T08:38:19.882289+0000","last_clean":"2024-04-25T08:38:19.882289+0000","last_became_active":"2024-04-25T08:38:16.870598+0000","last_became_peered":"2024-04-25T08:38:16.870598+0000","last_unstale":"2024-04-25T08:38:19.882289+0000","last_undegraded":"2024-04-25T08:38:19.882289+0000","last_fullsized":"2024-04-25T08:38:19.882289+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:21.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944319+0000","last_change":"2024-04-25T08:38:17.890425+0000","last_active":"2024-04-25T08:38:19.944319+0000","last_peered":"2024-04-25T08:38:19.944319+0000","last_clean":"2024-04-25T08:38:19.944319+0000","last_became_active":"2024-04-25T08:38:17.890108+0000","last_became_peered":"2024-04-25T08:38:17.890108+0000","last_unstale":"2024-04-25T08:38:19.944319+0000","last_undegraded":"2024-04-25T08:38:19.944319+0000","last_fullsized":"2024-04-25T08:38:19.944319+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:26.696133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944170+0000","last_change":"2024-04-25T08:38:16.868916+0000","last_active":"2024-04-25T08:38:19.944170+0000","last_peered":"2024-04-25T08:38:19.944170+0000","last_clean":"2024-04-25T08:38:19.944170+0000","last_became_active":"2024-04-25T08:38:16.868648+0000","last_became_peered":"2024-04-25T08:38:16.868648+0000","last_unstale":"2024-04-25T08:38:19.944170+0000","last_undegraded":"2024-04-25T08:38:19.944170+0000","last_fullsized":"2024-04-25T08:38:19.944170+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:41.331503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876563+0000","last_change":"2024-04-25T08:38:17.877450+0000","last_active":"2024-04-25T08:38:18.876563+0000","last_peered":"2024-04-25T08:38:18.876563+0000","last_clean":"2024-04-25T08:38:18.876563+0000","last_became_active":"2024-04-25T08:38:17.877096+0000","last_became_peered":"2024-04-25T08:38:17.877096+0000","last_unstale":"2024-04-25T08:38:18.876563+0000","last_undegraded":"2024-04-25T08:38:18.876563+0000","last_fullsized":"2024-04-25T08:38:18.876563+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877437+0000","last_change":"2024-04-25T08:38:16.872592+0000","last_active":"2024-04-25T08:38:18.877437+0000","last_peered":"2024-04-25T08:38:18.877437+0000","last_clean":"2024-04-25T08:38:18.877437+0000","last_became_active":"2024-04-25T08:38:16.872012+0000","last_became_peered":"2024-04-25T08:38:16.872012+0000","last_unstale":"2024-04-25T08:38:18.877437+0000","last_undegraded":"2024-04-25T08:38:18.877437+0000","last_fullsized":"2024-04-25T08:38:18.877437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:12.053186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876280+0000","last_change":"2024-04-25T08:38:17.889252+0000","last_active":"2024-04-25T08:38:18.876280+0000","last_peered":"2024-04-25T08:38:18.876280+0000","last_clean":"2024-04-25T08:38:18.876280+0000","last_became_active":"2024-04-25T08:38:17.888591+0000","last_became_peered":"2024-04-25T08:38:17.888591+0000","last_unstale":"2024-04-25T08:38:18.876280+0000","last_undegraded":"2024-04-25T08:38:18.876280+0000","last_fullsized":"2024-04-25T08:38:18.876280+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.675286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877066+0000","last_change":"2024-04-25T08:38:16.871721+0000","last_active":"2024-04-25T08:38:18.877066+0000","last_peered":"2024-04-25T08:38:18.877066+0000","last_clean":"2024-04-25T08:38:18.877066+0000","last_became_active":"2024-04-25T08:38:16.871135+0000","last_became_peered":"2024-04-25T08:38:16.871135+0000","last_unstale":"2024-04-25T08:38:18.877066+0000","last_undegraded":"2024-04-25T08:38:18.877066+0000","last_fullsized":"2024-04-25T08:38:18.877066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:05.915398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944043+0000","last_change":"2024-04-25T08:38:17.884680+0000","last_active":"2024-04-25T08:38:19.944043+0000","last_peered":"2024-04-25T08:38:19.944043+0000","last_clean":"2024-04-25T08:38:19.944043+0000","last_became_active":"2024-04-25T08:38:17.884023+0000","last_became_peered":"2024-04-25T08:38:17.884023+0000","last_unstale":"2024-04-25T08:38:19.944043+0000","last_undegraded":"2024-04-25T08:38:19.944043+0000","last_fullsized":"2024-04-25T08:38:19.944043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:30.754146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943481+0000","last_change":"2024-04-25T08:38:16.873738+0000","last_active":"2024-04-25T08:38:19.943481+0000","last_peered":"2024-04-25T08:38:19.943481+0000","last_clean":"2024-04-25T08:38:19.943481+0000","last_became_active":"2024-04-25T08:38:16.873338+0000","last_became_peered":"2024-04-25T08:38:16.873338+0000","last_unstale":"2024-04-25T08:38:19.943481+0000","last_undegraded":"2024-04-25T08:38:19.943481+0000","last_fullsized":"2024-04-25T08:38:19.943481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:11.883106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877409+0000","last_change":"2024-04-25T08:38:17.895519+0000","last_active":"2024-04-25T08:38:18.877409+0000","last_peered":"2024-04-25T08:38:18.877409+0000","last_clean":"2024-04-25T08:38:18.877409+0000","last_became_active":"2024-04-25T08:38:17.895149+0000","last_became_peered":"2024-04-25T08:38:17.895149+0000","last_unstale":"2024-04-25T08:38:18.877409+0000","last_undegraded":"2024-04-25T08:38:18.877409+0000","last_fullsized":"2024-04-25T08:38:18.877409+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.397082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875906+0000","last_change":"2024-04-25T08:38:16.868510+0000","last_active":"2024-04-25T08:38:18.875906+0000","last_peered":"2024-04-25T08:38:18.875906+0000","last_clean":"2024-04-25T08:38:18.875906+0000","last_became_active":"2024-04-25T08:38:16.868241+0000","last_became_peered":"2024-04-25T08:38:16.868241+0000","last_unstale":"2024-04-25T08:38:18.875906+0000","last_undegraded":"2024-04-25T08:38:18.875906+0000","last_fullsized":"2024-04-25T08:38:18.875906+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:55.755478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874750+0000","last_change":"2024-04-25T08:38:17.884410+0000","last_active":"2024-04-25T08:38:18.874750+0000","last_peered":"2024-04-25T08:38:18.874750+0000","last_clean":"2024-04-25T08:38:18.874750+0000","last_became_active":"2024-04-25T08:38:17.884122+0000","last_became_peered":"2024-04-25T08:38:17.884122+0000","last_unstale":"2024-04-25T08:38:18.874750+0000","last_undegraded":"2024-04-25T08:38:18.874750+0000","last_fullsized":"2024-04-25T08:38:18.874750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:26.696133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944197+0000","last_change":"2024-04-25T08:38:16.876363+0000","last_active":"2024-04-25T08:38:19.944197+0000","last_peered":"2024-04-25T08:38:19.944197+0000","last_clean":"2024-04-25T08:38:19.944197+0000","last_became_active":"2024-04-25T08:38:16.876049+0000","last_became_peered":"2024-04-25T08:38:16.876049+0000","last_unstale":"2024-04-25T08:38:19.944197+0000","last_undegraded":"2024-04-25T08:38:19.944197+0000","last_fullsized":"2024-04-25T08:38:19.944197+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:21.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876466+0000","last_change":"2024-04-25T08:38:17.881462+0000","last_active":"2024-04-25T08:38:18.876466+0000","last_peered":"2024-04-25T08:38:18.876466+0000","last_clean":"2024-04-25T08:38:18.876466+0000","last_became_active":"2024-04-25T08:38:17.880097+0000","last_became_peered":"2024-04-25T08:38:17.880097+0000","last_unstale":"2024-04-25T08:38:18.876466+0000","last_undegraded":"2024-04-25T08:38:18.876466+0000","last_fullsized":"2024-04-25T08:38:18.876466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882404+0000","last_change":"2024-04-25T08:38:16.866476+0000","last_active":"2024-04-25T08:38:19.882404+0000","last_peered":"2024-04-25T08:38:19.882404+0000","last_clean":"2024-04-25T08:38:19.882404+0000","last_became_active":"2024-04-25T08:38:16.865868+0000","last_became_peered":"2024-04-25T08:38:16.865868+0000","last_unstale":"2024-04-25T08:38:19.882404+0000","last_undegraded":"2024-04-25T08:38:19.882404+0000","last_fullsized":"2024-04-25T08:38:19.882404+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:46.775554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875571+0000","last_change":"2024-04-25T08:38:17.896892+0000","last_active":"2024-04-25T08:38:18.875571+0000","last_peered":"2024-04-25T08:38:18.875571+0000","last_clean":"2024-04-25T08:38:18.875571+0000","last_became_active":"2024-04-25T08:38:17.895902+0000","last_became_peered":"2024-04-25T08:38:17.895902+0000","last_unstale":"2024-04-25T08:38:18.875571+0000","last_undegraded":"2024-04-25T08:38:18.875571+0000","last_fullsized":"2024-04-25T08:38:18.875571+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:34.550562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876229+0000","last_change":"2024-04-25T08:38:16.868952+0000","last_active":"2024-04-25T08:38:18.876229+0000","last_peered":"2024-04-25T08:38:18.876229+0000","last_clean":"2024-04-25T08:38:18.876229+0000","last_became_active":"2024-04-25T08:38:16.868689+0000","last_became_peered":"2024-04-25T08:38:16.868689+0000","last_unstale":"2024-04-25T08:38:18.876229+0000","last_undegraded":"2024-04-25T08:38:18.876229+0000","last_fullsized":"2024-04-25T08:38:18.876229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:20.443300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875867+0000","last_change":"2024-04-25T08:38:17.897649+0000","last_active":"2024-04-25T08:38:18.875867+0000","last_peered":"2024-04-25T08:38:18.875867+0000","last_clean":"2024-04-25T08:38:18.875867+0000","last_became_active":"2024-04-25T08:38:17.896745+0000","last_became_peered":"2024-04-25T08:38:17.896745+0000","last_unstale":"2024-04-25T08:38:18.875867+0000","last_undegraded":"2024-04-25T08:38:18.875867+0000","last_fullsized":"2024-04-25T08:38:18.875867+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:03.164735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944209+0000","last_change":"2024-04-25T08:38:16.879316+0000","last_active":"2024-04-25T08:38:19.944209+0000","last_peered":"2024-04-25T08:38:19.944209+0000","last_clean":"2024-04-25T08:38:19.944209+0000","last_became_active":"2024-04-25T08:38:16.879048+0000","last_became_peered":"2024-04-25T08:38:16.879048+0000","last_unstale":"2024-04-25T08:38:19.944209+0000","last_undegraded":"2024-04-25T08:38:19.944209+0000","last_fullsized":"2024-04-25T08:38:19.944209+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943504+0000","last_change":"2024-04-25T08:38:17.887226+0000","last_active":"2024-04-25T08:38:19.943504+0000","last_peered":"2024-04-25T08:38:19.943504+0000","last_clean":"2024-04-25T08:38:19.943504+0000","last_became_active":"2024-04-25T08:38:17.886924+0000","last_became_peered":"2024-04-25T08:38:17.886924+0000","last_unstale":"2024-04-25T08:38:19.943504+0000","last_undegraded":"2024-04-25T08:38:19.943504+0000","last_fullsized":"2024-04-25T08:38:19.943504+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:58.314369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874682+0000","last_change":"2024-04-25T08:38:16.878410+0000","last_active":"2024-04-25T08:38:18.874682+0000","last_peered":"2024-04-25T08:38:18.874682+0000","last_clean":"2024-04-25T08:38:18.874682+0000","last_became_active":"2024-04-25T08:38:16.878111+0000","last_became_peered":"2024-04-25T08:38:16.878111+0000","last_unstale":"2024-04-25T08:38:18.874682+0000","last_undegraded":"2024-04-25T08:38:18.874682+0000","last_fullsized":"2024-04-25T08:38:18.874682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874630+0000","last_change":"2024-04-25T08:38:17.881756+0000","last_active":"2024-04-25T08:38:18.874630+0000","last_peered":"2024-04-25T08:38:18.874630+0000","last_clean":"2024-04-25T08:38:18.874630+0000","last_became_active":"2024-04-25T08:38:17.881349+0000","last_became_peered":"2024-04-25T08:38:17.881349+0000","last_unstale":"2024-04-25T08:38:18.874630+0000","last_undegraded":"2024-04-25T08:38:18.874630+0000","last_fullsized":"2024-04-25T08:38:18.874630+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:07.729869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879784+0000","last_change":"2024-04-25T08:38:16.865428+0000","last_active":"2024-04-25T08:38:19.879784+0000","last_peered":"2024-04-25T08:38:19.879784+0000","last_clean":"2024-04-25T08:38:19.879784+0000","last_became_active":"2024-04-25T08:38:16.865137+0000","last_became_peered":"2024-04-25T08:38:16.865137+0000","last_unstale":"2024-04-25T08:38:19.879784+0000","last_undegraded":"2024-04-25T08:38:19.879784+0000","last_fullsized":"2024-04-25T08:38:19.879784+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:55.755478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944757+0000","last_change":"2024-04-25T08:38:17.889169+0000","last_active":"2024-04-25T08:38:19.944757+0000","last_peered":"2024-04-25T08:38:19.944757+0000","last_clean":"2024-04-25T08:38:19.944757+0000","last_became_active":"2024-04-25T08:38:17.888859+0000","last_became_peered":"2024-04-25T08:38:17.888859+0000","last_unstale":"2024-04-25T08:38:19.944757+0000","last_undegraded":"2024-04-25T08:38:19.944757+0000","last_fullsized":"2024-04-25T08:38:19.944757+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:21.979826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944552+0000","last_change":"2024-04-25T08:38:16.870881+0000","last_active":"2024-04-25T08:38:19.944552+0000","last_peered":"2024-04-25T08:38:19.944552+0000","last_clean":"2024-04-25T08:38:19.944552+0000","last_became_active":"2024-04-25T08:38:16.870440+0000","last_became_peered":"2024-04-25T08:38:16.870440+0000","last_unstale":"2024-04-25T08:38:19.944552+0000","last_undegraded":"2024-04-25T08:38:19.944552+0000","last_fullsized":"2024-04-25T08:38:19.944552+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:25.178803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874779+0000","last_change":"2024-04-25T08:38:17.885425+0000","last_active":"2024-04-25T08:38:18.874779+0000","last_peered":"2024-04-25T08:38:18.874779+0000","last_clean":"2024-04-25T08:38:18.874779+0000","last_became_active":"2024-04-25T08:38:17.885118+0000","last_became_peered":"2024-04-25T08:38:17.885118+0000","last_unstale":"2024-04-25T08:38:18.874779+0000","last_undegraded":"2024-04-25T08:38:18.874779+0000","last_fullsized":"2024-04-25T08:38:18.874779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.466242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875707+0000","last_change":"2024-04-25T08:38:16.869530+0000","last_active":"2024-04-25T08:38:18.875707+0000","last_peered":"2024-04-25T08:38:18.875707+0000","last_clean":"2024-04-25T08:38:18.875707+0000","last_became_active":"2024-04-25T08:38:16.869225+0000","last_became_peered":"2024-04-25T08:38:16.869225+0000","last_unstale":"2024-04-25T08:38:18.875707+0000","last_undegraded":"2024-04-25T08:38:18.875707+0000","last_fullsized":"2024-04-25T08:38:18.875707+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:21.300604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882377+0000","last_change":"2024-04-25T08:38:17.884940+0000","last_active":"2024-04-25T08:38:19.882377+0000","last_peered":"2024-04-25T08:38:19.882377+0000","last_clean":"2024-04-25T08:38:19.882377+0000","last_became_active":"2024-04-25T08:38:17.884659+0000","last_became_peered":"2024-04-25T08:38:17.884659+0000","last_unstale":"2024-04-25T08:38:19.882377+0000","last_undegraded":"2024-04-25T08:38:19.882377+0000","last_fullsized":"2024-04-25T08:38:19.882377+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944287+0000","last_change":"2024-04-25T08:38:16.877836+0000","last_active":"2024-04-25T08:38:19.944287+0000","last_peered":"2024-04-25T08:38:19.944287+0000","last_clean":"2024-04-25T08:38:19.944287+0000","last_became_active":"2024-04-25T08:38:16.876856+0000","last_became_peered":"2024-04-25T08:38:16.876856+0000","last_unstale":"2024-04-25T08:38:19.944287+0000","last_undegraded":"2024-04-25T08:38:19.944287+0000","last_fullsized":"2024-04-25T08:38:19.944287+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:46.775554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874465+0000","last_change":"2024-04-25T08:38:17.885787+0000","last_active":"2024-04-25T08:38:18.874465+0000","last_peered":"2024-04-25T08:38:18.874465+0000","last_clean":"2024-04-25T08:38:18.874465+0000","last_became_active":"2024-04-25T08:38:17.885447+0000","last_became_peered":"2024-04-25T08:38:17.885447+0000","last_unstale":"2024-04-25T08:38:18.874465+0000","last_undegraded":"2024-04-25T08:38:18.874465+0000","last_fullsized":"2024-04-25T08:38:18.874465+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944775+0000","last_change":"2024-04-25T08:38:16.874920+0000","last_active":"2024-04-25T08:38:19.944775+0000","last_peered":"2024-04-25T08:38:19.944775+0000","last_clean":"2024-04-25T08:38:19.944775+0000","last_became_active":"2024-04-25T08:38:16.874423+0000","last_became_peered":"2024-04-25T08:38:16.874423+0000","last_unstale":"2024-04-25T08:38:19.944775+0000","last_undegraded":"2024-04-25T08:38:19.944775+0000","last_fullsized":"2024-04-25T08:38:19.944775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:00.856047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876026+0000","last_change":"2024-04-25T08:38:17.888010+0000","last_active":"2024-04-25T08:38:18.876026+0000","last_peered":"2024-04-25T08:38:18.876026+0000","last_clean":"2024-04-25T08:38:18.876026+0000","last_became_active":"2024-04-25T08:38:17.887721+0000","last_became_peered":"2024-04-25T08:38:17.887721+0000","last_unstale":"2024-04-25T08:38:18.876026+0000","last_undegraded":"2024-04-25T08:38:18.876026+0000","last_fullsized":"2024-04-25T08:38:18.876026+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:26.152377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"134'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881249+0000","last_change":"2024-04-25T08:38:16.863623+0000","last_active":"2024-04-25T08:38:19.881249+0000","last_peered":"2024-04-25T08:38:19.881249+0000","last_clean":"2024-04-25T08:38:19.881249+0000","last_became_active":"2024-04-25T08:38:16.863233+0000","last_became_peered":"2024-04-25T08:38:16.863233+0000","last_unstale":"2024-04-25T08:38:19.881249+0000","last_undegraded":"2024-04-25T08:38:19.881249+0000","last_fullsized":"2024-04-25T08:38:19.881249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:21.069051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875571+0000","last_change":"2024-04-25T08:38:17.892580+0000","last_active":"2024-04-25T08:38:18.875571+0000","last_peered":"2024-04-25T08:38:18.875571+0000","last_clean":"2024-04-25T08:38:18.875571+0000","last_became_active":"2024-04-25T08:38:17.889740+0000","last_became_peered":"2024-04-25T08:38:17.889740+0000","last_unstale":"2024-04-25T08:38:18.875571+0000","last_undegraded":"2024-04-25T08:38:18.875571+0000","last_fullsized":"2024-04-25T08:38:18.875571+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:42.269427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879793+0000","last_change":"2024-04-25T08:38:16.870790+0000","last_active":"2024-04-25T08:38:19.879793+0000","last_peered":"2024-04-25T08:38:19.879793+0000","last_clean":"2024-04-25T08:38:19.879793+0000","last_became_active":"2024-04-25T08:38:16.870489+0000","last_became_peered":"2024-04-25T08:38:16.870489+0000","last_unstale":"2024-04-25T08:38:19.879793+0000","last_undegraded":"2024-04-25T08:38:19.879793+0000","last_fullsized":"2024-04-25T08:38:19.879793+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.197963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943648+0000","last_change":"2024-04-25T08:38:17.884365+0000","last_active":"2024-04-25T08:38:19.943648+0000","last_peered":"2024-04-25T08:38:19.943648+0000","last_clean":"2024-04-25T08:38:19.943648+0000","last_became_active":"2024-04-25T08:38:17.884041+0000","last_became_peered":"2024-04-25T08:38:17.884041+0000","last_unstale":"2024-04-25T08:38:19.943648+0000","last_undegraded":"2024-04-25T08:38:19.943648+0000","last_fullsized":"2024-04-25T08:38:19.943648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:42.402577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943793+0000","last_change":"2024-04-25T08:38:16.878761+0000","last_active":"2024-04-25T08:38:19.943793+0000","last_peered":"2024-04-25T08:38:19.943793+0000","last_clean":"2024-04-25T08:38:19.943793+0000","last_became_active":"2024-04-25T08:38:16.878118+0000","last_became_peered":"2024-04-25T08:38:16.878118+0000","last_unstale":"2024-04-25T08:38:19.943793+0000","last_undegraded":"2024-04-25T08:38:19.943793+0000","last_fullsized":"2024-04-25T08:38:19.943793+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:46.498729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943685+0000","last_change":"2024-04-25T08:38:17.877565+0000","last_active":"2024-04-25T08:38:19.943685+0000","last_peered":"2024-04-25T08:38:19.943685+0000","last_clean":"2024-04-25T08:38:19.943685+0000","last_became_active":"2024-04-25T08:38:17.876848+0000","last_became_peered":"2024-04-25T08:38:17.876848+0000","last_unstale":"2024-04-25T08:38:19.943685+0000","last_undegraded":"2024-04-25T08:38:19.943685+0000","last_fullsized":"2024-04-25T08:38:19.943685+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882003+0000","last_change":"2024-04-25T08:38:16.875734+0000","last_active":"2024-04-25T08:38:19.882003+0000","last_peered":"2024-04-25T08:38:19.882003+0000","last_clean":"2024-04-25T08:38:19.882003+0000","last_became_active":"2024-04-25T08:38:16.875418+0000","last_became_peered":"2024-04-25T08:38:16.875418+0000","last_unstale":"2024-04-25T08:38:19.882003+0000","last_undegraded":"2024-04-25T08:38:19.882003+0000","last_fullsized":"2024-04-25T08:38:19.882003+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:46.498729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944097+0000","last_change":"2024-04-25T08:38:17.878723+0000","last_active":"2024-04-25T08:38:19.944097+0000","last_peered":"2024-04-25T08:38:19.944097+0000","last_clean":"2024-04-25T08:38:19.944097+0000","last_became_active":"2024-04-25T08:38:17.878200+0000","last_became_peered":"2024-04-25T08:38:17.878200+0000","last_unstale":"2024-04-25T08:38:19.944097+0000","last_undegraded":"2024-04-25T08:38:19.944097+0000","last_fullsized":"2024-04-25T08:38:19.944097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:07.729869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875920+0000","last_change":"2024-04-25T08:38:16.873456+0000","last_active":"2024-04-25T08:38:18.875920+0000","last_peered":"2024-04-25T08:38:18.875920+0000","last_clean":"2024-04-25T08:38:18.875920+0000","last_became_active":"2024-04-25T08:38:16.873176+0000","last_became_peered":"2024-04-25T08:38:16.873176+0000","last_unstale":"2024-04-25T08:38:18.875920+0000","last_undegraded":"2024-04-25T08:38:18.875920+0000","last_fullsized":"2024-04-25T08:38:18.875920+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.941893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943958+0000","last_change":"2024-04-25T08:38:17.892178+0000","last_active":"2024-04-25T08:38:19.943958+0000","last_peered":"2024-04-25T08:38:19.943958+0000","last_clean":"2024-04-25T08:38:19.943958+0000","last_became_active":"2024-04-25T08:38:17.891887+0000","last_became_peered":"2024-04-25T08:38:17.891887+0000","last_unstale":"2024-04-25T08:38:19.943958+0000","last_undegraded":"2024-04-25T08:38:19.943958+0000","last_fullsized":"2024-04-25T08:38:19.943958+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944410+0000","last_change":"2024-04-25T08:38:16.878410+0000","last_active":"2024-04-25T08:38:19.944410+0000","last_peered":"2024-04-25T08:38:19.944410+0000","last_clean":"2024-04-25T08:38:19.944410+0000","last_became_active":"2024-04-25T08:38:16.877139+0000","last_became_peered":"2024-04-25T08:38:16.877139+0000","last_unstale":"2024-04-25T08:38:19.944410+0000","last_undegraded":"2024-04-25T08:38:19.944410+0000","last_fullsized":"2024-04-25T08:38:19.944410+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:16.115180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877343+0000","last_change":"2024-04-25T08:38:17.897234+0000","last_active":"2024-04-25T08:38:18.877343+0000","last_peered":"2024-04-25T08:38:18.877343+0000","last_clean":"2024-04-25T08:38:18.877343+0000","last_became_active":"2024-04-25T08:38:17.896958+0000","last_became_peered":"2024-04-25T08:38:17.896958+0000","last_unstale":"2024-04-25T08:38:18.877343+0000","last_undegraded":"2024-04-25T08:38:18.877343+0000","last_fullsized":"2024-04-25T08:38:18.877343+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:19.842177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874473+0000","last_change":"2024-04-25T08:38:16.866696+0000","last_active":"2024-04-25T08:38:18.874473+0000","last_peered":"2024-04-25T08:38:18.874473+0000","last_clean":"2024-04-25T08:38:18.874473+0000","last_became_active":"2024-04-25T08:38:16.866421+0000","last_became_peered":"2024-04-25T08:38:16.866421+0000","last_unstale":"2024-04-25T08:38:18.874473+0000","last_undegraded":"2024-04-25T08:38:18.874473+0000","last_fullsized":"2024-04-25T08:38:18.874473+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:16.115180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876675+0000","last_change":"2024-04-25T08:38:17.879750+0000","last_active":"2024-04-25T08:38:18.876675+0000","last_peered":"2024-04-25T08:38:18.876675+0000","last_clean":"2024-04-25T08:38:18.876675+0000","last_became_active":"2024-04-25T08:38:17.879471+0000","last_became_peered":"2024-04-25T08:38:17.879471+0000","last_unstale":"2024-04-25T08:38:18.876675+0000","last_undegraded":"2024-04-25T08:38:18.876675+0000","last_fullsized":"2024-04-25T08:38:18.876675+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875673+0000","last_change":"2024-04-25T08:38:16.870075+0000","last_active":"2024-04-25T08:38:18.875673+0000","last_peered":"2024-04-25T08:38:18.875673+0000","last_clean":"2024-04-25T08:38:18.875673+0000","last_became_active":"2024-04-25T08:38:16.869730+0000","last_became_peered":"2024-04-25T08:38:16.869730+0000","last_unstale":"2024-04-25T08:38:18.875673+0000","last_undegraded":"2024-04-25T08:38:18.875673+0000","last_fullsized":"2024-04-25T08:38:18.875673+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:57.363947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":381,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875376+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:38:18.875376+0000","last_peered":"2024-04-25T08:38:18.875376+0000","last_clean":"2024-04-25T08:38:18.875376+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:38:18.875376+0000","last_undegraded":"2024-04-25T08:38:18.875376+0000","last_fullsized":"2024-04-25T08:38:18.875376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542258,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47304,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324536,"statfs":{"total":96636764160,"available":96588324864,"internally_reserved":0,"allocated":2281472,"data_stored":1372601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542258,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50868,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320972,"statfs":{"total":96636764160,"available":96584675328,"internally_reserved":0,"allocated":1679360,"data_stored":781203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542258,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46704,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325136,"statfs":{"total":96636764160,"available":96588939264,"internally_reserved":0,"allocated":1675264,"data_stored":780553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574961,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50860,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320980,"statfs":{"total":96636764160,"available":96584683520,"internally_reserved":0,"allocated":1671168,"data_stored":780531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59200,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312640,"statfs":{"total":96636764160,"available":96576143360,"internally_reserved":0,"allocated":1687552,"data_stored":782213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59188,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312652,"statfs":{"total":96636764160,"available":96576155648,"internally_reserved":0,"allocated":1679360,"data_stored":781147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55024,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316816,"statfs":{"total":96636764160,"available":96580419584,"internally_reserved":0,"allocated":1675264,"data_stored":780601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59764,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312076,"statfs":{"total":96636764160,"available":96575565824,"internally_reserved":0,"allocated":2260992,"data_stored":1364263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:21.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.714+0000 7f79d607c640 1 Processor -- start 2024-04-25T08:38:21.708 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.714+0000 7f79d607c640 1 -- start start 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79d607c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d0154ed0 0x7f79d00aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79d607c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d00ad410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79d607c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f79d006c1d0 0x7f79d00ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79d607c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d015ec60 con 0x7f79d0154ed0 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79d607c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d015ef40 con 0x7f79d006c1d0 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79d607c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d006db90 con 0x7f79d0071ac0 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d0154ed0 0x7f79d00aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f79d0154ed0 0x7f79d00aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60736/0 (socket says 172.21.15.60:60736) 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 -- 172.21.15.60:0/2982392780 learned_addr learned my addr 172.21.15.60:0/2982392780 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cffff640 1 --2- 172.21.15.60:0/2982392780 >> v2:172.21.15.100:3300/0 conn(0x7f79d006c1d0 0x7f79d00ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79ceffd640 1 --2- 172.21.15.60:0/2982392780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d00ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.710 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 -- 172.21.15.60:0/2982392780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 msgr2=0x7f79d00ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 --2- 172.21.15.60:0/2982392780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d00ad410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 -- 172.21.15.60:0/2982392780 >> v2:172.21.15.100:3300/0 conn(0x7f79d006c1d0 msgr2=0x7f79d00ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 --2- 172.21.15.60:0/2982392780 >> v2:172.21.15.100:3300/0 conn(0x7f79d006c1d0 0x7f79d00ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cf7fe640 1 -- 172.21.15.60:0/2982392780 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d006de70 con 0x7f79d0154ed0 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.715+0000 7f79cffff640 1 --2- 172.21.15.60:0/2982392780 >> v2:172.21.15.100:3300/0 conn(0x7f79d006c1d0 0x7f79d00ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.716+0000 7f79cf7fe640 1 --2- 172.21.15.60:0/2982392780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d00aced0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f79d0162890 tx=0x7f79c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379c38fd8452ee1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.716+0000 7f79ce7fc640 1 -- 172.21.15.60:0/2982392780 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0037070 con 0x7f79d0154ed0 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.716+0000 7f79ce7fc640 1 -- 172.21.15.60:0/2982392780 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c0032440 con 0x7f79d0154ed0 2024-04-25T08:38:21.711 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.716+0000 7f79ce7fc640 1 -- 172.21.15.60:0/2982392780 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c0032810 con 0x7f79d0154ed0 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 -- 172.21.15.60:0/2982392780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 msgr2=0x7f79d00aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 --2- 172.21.15.60:0/2982392780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d00aced0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f79d0162890 tx=0x7f79c002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 -- 172.21.15.60:0/2982392780 shutdown_connections 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 --2- 172.21.15.60:0/2982392780 >> v2:172.21.15.100:3300/0 conn(0x7f79d006c1d0 0x7f79d00ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 --2- 172.21.15.60:0/2982392780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d00ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 --2- 172.21.15.60:0/2982392780 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d00aced0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 -- 172.21.15.60:0/2982392780 >> 172.21.15.60:0/2982392780 conn(0x7f79d00af670 msgr2=0x7f79d00b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 -- 172.21.15.60:0/2982392780 shutdown_connections 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.717+0000 7f79d607c640 1 -- 172.21.15.60:0/2982392780 wait complete. 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.718+0000 7f79d607c640 1 Processor -- start 2024-04-25T08:38:21.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.718+0000 7f79d607c640 1 -- start start 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79d607c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d0146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79d607c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d01510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79d607c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d0151630 0x7f79d00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79d607c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d0161ba0 con 0x7f79d0154ed0 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79d607c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d0161a20 con 0x7f79d0151630 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79d607c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d0161d20 con 0x7f79d0071ac0 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d01510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d01510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60744/0 (socket says 172.21.15.60:60744) 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 -- 172.21.15.60:0/3045474896 learned_addr learned my addr 172.21.15.60:0/3045474896 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79cf7fe640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d0146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79cffff640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d0151630 0x7f79d00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 -- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 msgr2=0x7f79d0146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d0146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 -- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d0151630 msgr2=0x7f79d00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d0151630 0x7f79d00ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 -- 172.21.15.60:0/3045474896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0146f70 con 0x7f79d0154ed0 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79cffff640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d0151630 0x7f79d00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.719+0000 7f79ceffd640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d01510f0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f79c400d800 tx=0x7f79c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.720+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c400b6c0 con 0x7f79d0154ed0 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.720+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c4019070 con 0x7f79d0154ed0 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.720+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d0155d50 con 0x7f79d0154ed0 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.720+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c400ba80 con 0x7f79d0154ed0 2024-04-25T08:38:21.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.720+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d0156310 con 0x7f79d0154ed0 2024-04-25T08:38:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.721+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d00b2b10 con 0x7f79d0154ed0 2024-04-25T08:38:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.721+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79c400bc20 con 0x7f79d0154ed0 2024-04-25T08:38:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.722+0000 7f79b7fff640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79bc06b4d0 0x7f79bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.722+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79c408a750 con 0x7f79d0154ed0 2024-04-25T08:38:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.722+0000 7f79cf7fe640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79bc06b4d0 0x7f79bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.722+0000 7f79cf7fe640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79bc06b4d0 0x7f79bc06d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f79c0034000 tx=0x7f79c002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.726+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79d0146f70 con 0x7f79d0154ed0 2024-04-25T08:38:21.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.978+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f79d0151a40 con 0x7f79d0154ed0 2024-04-25T08:38:21.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.978+0000 7f79b7fff640 1 -- 172.21.15.60:0/3045474896 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1563 (secure 0 0 0) 0x7f79d0151a40 con 0x7f79d0154ed0 2024-04-25T08:38:21.972 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:21.972 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":439205888,"bytes_avail":772654907392,"bytes_total":773094113280,"write_bytes_sec":2528,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T08:37:57.516705+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79bc06b4d0 msgr2=0x7f79bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79bc06b4d0 0x7f79bc06d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f79c0034000 tx=0x7f79c002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 msgr2=0x7f79d01510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d01510f0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f79c400d800 tx=0x7f79c400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 shutdown_connections 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f79bc06b4d0 0x7f79bc06d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79d0151630 0x7f79d00ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79d0154ed0 0x7f79d01510f0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 --2- 172.21.15.60:0/3045474896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f79d0071ac0 0x7f79d0146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 >> 172.21.15.60:0/3045474896 conn(0x7f79d006c1d0 msgr2=0x7f79d0160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 shutdown_connections 2024-04-25T08:38:21.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:21.980+0000 7f79d607c640 1 -- 172.21.15.60:0/3045474896 wait complete. 2024-04-25T08:38:21.985 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:38:21.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.091+0000 7fd927efc640 1 Processor -- start 2024-04-25T08:38:22.085 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.091+0000 7fd927efc640 1 -- start start 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd927efc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd92006abe0 0x7fd92006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd927efc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd9200ce330 0x7fd92006fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd927efc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9201626e0 con 0x7fd9200700a0 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd927efc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92006b500 con 0x7fd92006abe0 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd927efc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200cd270 con 0x7fd9200ce330 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd925c98640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd92006abe0 0x7fd92006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd926499640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd9200700a0 0x7fd92006e3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57562/0 (socket says 172.21.15.60:57562) 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd925c98640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd92006abe0 0x7fd92006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60752/0 (socket says 172.21.15.60:60752) 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.092+0000 7fd926499640 1 -- 172.21.15.60:0/3709972615 learned_addr learned my addr 172.21.15.60:0/3709972615 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925c98640 1 -- 172.21.15.60:0/3709972615 >> v2:172.21.15.100:3300/0 conn(0x7fd9200ce330 msgr2=0x7fd92006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925497640 1 --2- 172.21.15.60:0/3709972615 >> v2:172.21.15.100:3300/0 conn(0x7fd9200ce330 0x7fd92006fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd924c96640 1 -- 172.21.15.60:0/3709972615 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815625107 0 0) 0x7fd9201626e0 con 0x7fd9200700a0 2024-04-25T08:38:22.087 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925c98640 1 --2- 172.21.15.60:0/3709972615 >> v2:172.21.15.100:3300/0 conn(0x7fd9200ce330 0x7fd92006fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925c98640 1 -- 172.21.15.60:0/3709972615 >> v1:172.21.15.60:6789/0 conn(0x7fd9200700a0 legacy=0x7fd92006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925c98640 1 -- 172.21.15.60:0/3709972615 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200cd550 con 0x7fd92006abe0 2024-04-25T08:38:22.088 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925497640 1 --2- 172.21.15.60:0/3709972615 >> v2:172.21.15.100:3300/0 conn(0x7fd9200ce330 0x7fd92006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.093+0000 7fd925c98640 1 --2- 172.21.15.60:0/3709972615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd92006abe0 0x7fd92006afc0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd92006bbf0 tx=0x7fd91802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5ba783ad3758f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd924c96640 1 -- 172.21.15.60:0/3709972615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918037070 con 0x7fd92006abe0 2024-04-25T08:38:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd924c96640 1 -- 172.21.15.60:0/3709972615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918032440 con 0x7fd92006abe0 2024-04-25T08:38:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd924c96640 1 -- 172.21.15.60:0/3709972615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91802ae60 con 0x7fd92006abe0 2024-04-25T08:38:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 -- 172.21.15.60:0/3709972615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd92006abe0 msgr2=0x7fd92006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.089 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 --2- 172.21.15.60:0/3709972615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd92006abe0 0x7fd92006afc0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd92006bbf0 tx=0x7fd91802a110 comp rx=0 tx=0).stop 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 -- 172.21.15.60:0/3709972615 shutdown_connections 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 --2- 172.21.15.60:0/3709972615 >> v2:172.21.15.100:3300/0 conn(0x7fd9200ce330 0x7fd92006fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 --2- 172.21.15.60:0/3709972615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd92006abe0 0x7fd92006afc0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 -- 172.21.15.60:0/3709972615 >> 172.21.15.60:0/3709972615 conn(0x7fd9200b2220 msgr2=0x7fd9200b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 -- 172.21.15.60:0/3709972615 shutdown_connections 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.094+0000 7fd927efc640 1 -- 172.21.15.60:0/3709972615 wait complete. 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.095+0000 7fd927efc640 1 Processor -- start 2024-04-25T08:38:22.090 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 -- start start 2024-04-25T08:38:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd92006abe0 0x7fd920159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 0x7fd9200b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9200ce330 0x7fd9200b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920161780 con 0x7fd9200700a0 2024-04-25T08:38:22.092 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200cdb10 con 0x7fd92006abe0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd927efc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92006eb00 con 0x7fd9200ce330 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd925497640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 0x7fd9200b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd925497640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 0x7fd9200b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60768/0 (socket says 172.21.15.60:60768) 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.096+0000 7fd925497640 1 -- 172.21.15.60:0/1675394639 learned_addr learned my addr 172.21.15.60:0/1675394639 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925497640 1 -- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9200ce330 msgr2=0x7fd9200b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925497640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9200ce330 0x7fd9200b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925497640 1 -- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd92006abe0 msgr2=0x7fd920159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925c98640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd92006abe0 0x7fd920159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925497640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd92006abe0 0x7fd920159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925497640 1 -- 172.21.15.60:0/1675394639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92015a690 con 0x7fd9200700a0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd925497640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 0x7fd9200b4320 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd91400c7f0 tx=0x7fd91400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914015070 con 0x7fd9200700a0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd914007820 con 0x7fd9200700a0 2024-04-25T08:38:22.093 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914007bf0 con 0x7fd9200700a0 2024-04-25T08:38:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92015a970 con 0x7fd9200700a0 2024-04-25T08:38:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.097+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92015af30 con 0x7fd9200700a0 2024-04-25T08:38:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.098+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9200b2d50 con 0x7fd9200700a0 2024-04-25T08:38:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.098+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd914018070 con 0x7fd9200700a0 2024-04-25T08:38:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.099+0000 7fd91e7fc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd90c06b4d0 0x7fd90c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.094 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.099+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd91408a580 con 0x7fd9200700a0 2024-04-25T08:38:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.099+0000 7fd925c98640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd90c06b4d0 0x7fd90c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.095 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.100+0000 7fd925c98640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd90c06b4d0 0x7fd90c06d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd918034000 tx=0x7fd91802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:22.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.103+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9140591d0 con 0x7fd9200700a0 2024-04-25T08:38:22.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.279+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9200b2ff0 con 0x7fd90c06b4d0 2024-04-25T08:38:22.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.286+0000 7fd91e7fc640 1 -- 172.21.15.60:0/1675394639 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7fd9200b2ff0 con 0x7fd90c06b4d0 2024-04-25T08:38:22.283 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:22.284 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:38:22.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd90c06b4d0 msgr2=0x7fd90c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd90c06b4d0 0x7fd90c06d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd918034000 tx=0x7fd91802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:22.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 msgr2=0x7fd9200b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.286 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 0x7fd9200b4320 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd91400c7f0 tx=0x7fd91400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 shutdown_connections 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9200ce330 0x7fd9200b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9200700a0 0x7fd9200b4320 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd90c06b4d0 0x7fd90c06d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 --2- 172.21.15.60:0/1675394639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd92006abe0 0x7fd920159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 >> 172.21.15.60:0/1675394639 conn(0x7fd9200b2220 msgr2=0x7fd9200af150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:22.287 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 shutdown_connections 2024-04-25T08:38:22.288 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:22.289+0000 7fd927efc640 1 -- 172.21.15.60:0/1675394639 wait complete. 2024-04-25T08:38:22.294 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-25T08:38:21.522262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":428912,"kb_used_data":14268,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754545808,"statfs":{"total":773094113280,"available":772654907392,"internally_reserved":0,"allocated":14610432,"data_stored":7423112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.670599"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944892+0000","last_change":"2024-04-25T08:38:17.888831+0000","last_active":"2024-04-25T08:38:19.944892+0000","last_peered":"2024-04-25T08:38:19.944892+0000","last_clean":"2024-04-25T08:38:19.944892+0000","last_became_active":"2024-04-25T08:38:17.888425+0000","last_became_peered":"2024-04-25T08:38:17.888425+0000","last_unstale":"2024-04-25T08:38:19.944892+0000","last_undegraded":"2024-04-25T08:38:19.944892+0000","last_fullsized":"2024-04-25T08:38:19.944892+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:22.446450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882289+0000","last_change":"2024-04-25T08:38:17.878980+0000","last_active":"2024-04-25T08:38:19.882289+0000","last_peered":"2024-04-25T08:38:19.882289+0000","last_clean":"2024-04-25T08:38:19.882289+0000","last_became_active":"2024-04-25T08:38:17.878667+0000","last_became_peered":"2024-04-25T08:38:17.878667+0000","last_unstale":"2024-04-25T08:38:19.882289+0000","last_undegraded":"2024-04-25T08:38:19.882289+0000","last_fullsized":"2024-04-25T08:38:19.882289+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876695+0000","last_change":"2024-04-25T08:38:17.879831+0000","last_active":"2024-04-25T08:38:18.876695+0000","last_peered":"2024-04-25T08:38:18.876695+0000","last_clean":"2024-04-25T08:38:18.876695+0000","last_became_active":"2024-04-25T08:38:17.879554+0000","last_became_peered":"2024-04-25T08:38:17.879554+0000","last_unstale":"2024-04-25T08:38:18.876695+0000","last_undegraded":"2024-04-25T08:38:18.876695+0000","last_fullsized":"2024-04-25T08:38:18.876695+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882016+0000","last_change":"2024-04-25T08:38:17.891476+0000","last_active":"2024-04-25T08:38:19.882016+0000","last_peered":"2024-04-25T08:38:19.882016+0000","last_clean":"2024-04-25T08:38:19.882016+0000","last_became_active":"2024-04-25T08:38:17.891162+0000","last_became_peered":"2024-04-25T08:38:17.891162+0000","last_unstale":"2024-04-25T08:38:19.882016+0000","last_undegraded":"2024-04-25T08:38:19.882016+0000","last_fullsized":"2024-04-25T08:38:19.882016+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.057228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874423+0000","last_change":"2024-04-25T08:38:17.881621+0000","last_active":"2024-04-25T08:38:18.874423+0000","last_peered":"2024-04-25T08:38:18.874423+0000","last_clean":"2024-04-25T08:38:18.874423+0000","last_became_active":"2024-04-25T08:38:17.881287+0000","last_became_peered":"2024-04-25T08:38:17.881287+0000","last_unstale":"2024-04-25T08:38:18.874423+0000","last_undegraded":"2024-04-25T08:38:18.874423+0000","last_fullsized":"2024-04-25T08:38:18.874423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943745+0000","last_change":"2024-04-25T08:38:17.874836+0000","last_active":"2024-04-25T08:38:19.943745+0000","last_peered":"2024-04-25T08:38:19.943745+0000","last_clean":"2024-04-25T08:38:19.943745+0000","last_became_active":"2024-04-25T08:38:17.874196+0000","last_became_peered":"2024-04-25T08:38:17.874196+0000","last_unstale":"2024-04-25T08:38:19.943745+0000","last_undegraded":"2024-04-25T08:38:19.943745+0000","last_fullsized":"2024-04-25T08:38:19.943745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875655+0000","last_change":"2024-04-25T08:38:17.894302+0000","last_active":"2024-04-25T08:38:18.875655+0000","last_peered":"2024-04-25T08:38:18.875655+0000","last_clean":"2024-04-25T08:38:18.875655+0000","last_became_active":"2024-04-25T08:38:17.893594+0000","last_became_peered":"2024-04-25T08:38:17.893594+0000","last_unstale":"2024-04-25T08:38:18.875655+0000","last_undegraded":"2024-04-25T08:38:18.875655+0000","last_fullsized":"2024-04-25T08:38:18.875655+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:58.149171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879387+0000","last_change":"2024-04-25T08:38:17.885426+0000","last_active":"2024-04-25T08:38:19.879387+0000","last_peered":"2024-04-25T08:38:19.879387+0000","last_clean":"2024-04-25T08:38:19.879387+0000","last_became_active":"2024-04-25T08:38:17.885142+0000","last_became_peered":"2024-04-25T08:38:17.885142+0000","last_unstale":"2024-04-25T08:38:19.879387+0000","last_undegraded":"2024-04-25T08:38:19.879387+0000","last_fullsized":"2024-04-25T08:38:19.879387+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:17.119223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944491+0000","last_change":"2024-04-25T08:38:17.881404+0000","last_active":"2024-04-25T08:38:19.944491+0000","last_peered":"2024-04-25T08:38:19.944491+0000","last_clean":"2024-04-25T08:38:19.944491+0000","last_became_active":"2024-04-25T08:38:17.881129+0000","last_became_peered":"2024-04-25T08:38:17.881129+0000","last_unstale":"2024-04-25T08:38:19.944491+0000","last_undegraded":"2024-04-25T08:38:19.944491+0000","last_fullsized":"2024-04-25T08:38:19.944491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.466242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943737+0000","last_change":"2024-04-25T08:38:17.886835+0000","last_active":"2024-04-25T08:38:19.943737+0000","last_peered":"2024-04-25T08:38:19.943737+0000","last_clean":"2024-04-25T08:38:19.943737+0000","last_became_active":"2024-04-25T08:38:17.886530+0000","last_became_peered":"2024-04-25T08:38:17.886530+0000","last_unstale":"2024-04-25T08:38:19.943737+0000","last_undegraded":"2024-04-25T08:38:19.943737+0000","last_fullsized":"2024-04-25T08:38:19.943737+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:02.753734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881989+0000","last_change":"2024-04-25T08:38:17.887506+0000","last_active":"2024-04-25T08:38:19.881989+0000","last_peered":"2024-04-25T08:38:19.881989+0000","last_clean":"2024-04-25T08:38:19.881989+0000","last_became_active":"2024-04-25T08:38:17.884413+0000","last_became_peered":"2024-04-25T08:38:17.884413+0000","last_unstale":"2024-04-25T08:38:19.881989+0000","last_undegraded":"2024-04-25T08:38:19.881989+0000","last_fullsized":"2024-04-25T08:38:19.881989+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:55.340733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944346+0000","last_change":"2024-04-25T08:38:17.882319+0000","last_active":"2024-04-25T08:38:19.944346+0000","last_peered":"2024-04-25T08:38:19.944346+0000","last_clean":"2024-04-25T08:38:19.944346+0000","last_became_active":"2024-04-25T08:38:17.881864+0000","last_became_peered":"2024-04-25T08:38:17.881864+0000","last_unstale":"2024-04-25T08:38:19.944346+0000","last_undegraded":"2024-04-25T08:38:19.944346+0000","last_fullsized":"2024-04-25T08:38:19.944346+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:35.454814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944626+0000","last_change":"2024-04-25T08:38:17.875563+0000","last_active":"2024-04-25T08:38:19.944626+0000","last_peered":"2024-04-25T08:38:19.944626+0000","last_clean":"2024-04-25T08:38:19.944626+0000","last_became_active":"2024-04-25T08:38:17.875187+0000","last_became_peered":"2024-04-25T08:38:17.875187+0000","last_unstale":"2024-04-25T08:38:19.944626+0000","last_undegraded":"2024-04-25T08:38:19.944626+0000","last_fullsized":"2024-04-25T08:38:19.944626+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:31.918313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944527+0000","last_change":"2024-04-25T08:38:17.880229+0000","last_active":"2024-04-25T08:38:19.944527+0000","last_peered":"2024-04-25T08:38:19.944527+0000","last_clean":"2024-04-25T08:38:19.944527+0000","last_became_active":"2024-04-25T08:38:17.879911+0000","last_became_peered":"2024-04-25T08:38:17.879911+0000","last_unstale":"2024-04-25T08:38:19.944527+0000","last_undegraded":"2024-04-25T08:38:19.944527+0000","last_fullsized":"2024-04-25T08:38:19.944527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.772260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943620+0000","last_change":"2024-04-25T08:38:17.881918+0000","last_active":"2024-04-25T08:38:19.943620+0000","last_peered":"2024-04-25T08:38:19.943620+0000","last_clean":"2024-04-25T08:38:19.943620+0000","last_became_active":"2024-04-25T08:38:17.881591+0000","last_became_peered":"2024-04-25T08:38:17.881591+0000","last_unstale":"2024-04-25T08:38:19.943620+0000","last_undegraded":"2024-04-25T08:38:19.943620+0000","last_fullsized":"2024-04-25T08:38:19.943620+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:41.225407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943933+0000","last_change":"2024-04-25T08:38:17.882032+0000","last_active":"2024-04-25T08:38:19.943933+0000","last_peered":"2024-04-25T08:38:19.943933+0000","last_clean":"2024-04-25T08:38:19.943933+0000","last_became_active":"2024-04-25T08:38:17.881535+0000","last_became_peered":"2024-04-25T08:38:17.881535+0000","last_unstale":"2024-04-25T08:38:19.943933+0000","last_undegraded":"2024-04-25T08:38:19.943933+0000","last_fullsized":"2024-04-25T08:38:19.943933+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944573+0000","last_change":"2024-04-25T08:38:17.882058+0000","last_active":"2024-04-25T08:38:19.944573+0000","last_peered":"2024-04-25T08:38:19.944573+0000","last_clean":"2024-04-25T08:38:19.944573+0000","last_became_active":"2024-04-25T08:38:17.881728+0000","last_became_peered":"2024-04-25T08:38:17.881728+0000","last_unstale":"2024-04-25T08:38:19.944573+0000","last_undegraded":"2024-04-25T08:38:19.944573+0000","last_fullsized":"2024-04-25T08:38:19.944573+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:13.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874594+0000","last_change":"2024-04-25T08:38:17.882056+0000","last_active":"2024-04-25T08:38:18.874594+0000","last_peered":"2024-04-25T08:38:18.874594+0000","last_clean":"2024-04-25T08:38:18.874594+0000","last_became_active":"2024-04-25T08:38:17.881737+0000","last_became_peered":"2024-04-25T08:38:17.881737+0000","last_unstale":"2024-04-25T08:38:18.874594+0000","last_undegraded":"2024-04-25T08:38:18.874594+0000","last_fullsized":"2024-04-25T08:38:18.874594+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877346+0000","last_change":"2024-04-25T08:38:17.896592+0000","last_active":"2024-04-25T08:38:18.877346+0000","last_peered":"2024-04-25T08:38:18.877346+0000","last_clean":"2024-04-25T08:38:18.877346+0000","last_became_active":"2024-04-25T08:38:17.894196+0000","last_became_peered":"2024-04-25T08:38:17.894196+0000","last_unstale":"2024-04-25T08:38:18.877346+0000","last_undegraded":"2024-04-25T08:38:18.877346+0000","last_fullsized":"2024-04-25T08:38:18.877346+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.093419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874582+0000","last_change":"2024-04-25T08:38:17.894097+0000","last_active":"2024-04-25T08:38:18.874582+0000","last_peered":"2024-04-25T08:38:18.874582+0000","last_clean":"2024-04-25T08:38:18.874582+0000","last_became_active":"2024-04-25T08:38:17.893173+0000","last_became_peered":"2024-04-25T08:38:17.893173+0000","last_unstale":"2024-04-25T08:38:18.874582+0000","last_undegraded":"2024-04-25T08:38:18.874582+0000","last_fullsized":"2024-04-25T08:38:18.874582+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881956+0000","last_change":"2024-04-25T08:38:17.879586+0000","last_active":"2024-04-25T08:38:19.881956+0000","last_peered":"2024-04-25T08:38:19.881956+0000","last_clean":"2024-04-25T08:38:19.881956+0000","last_became_active":"2024-04-25T08:38:17.879289+0000","last_became_peered":"2024-04-25T08:38:17.879289+0000","last_unstale":"2024-04-25T08:38:19.881956+0000","last_undegraded":"2024-04-25T08:38:19.881956+0000","last_fullsized":"2024-04-25T08:38:19.881956+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.675286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944386+0000","last_change":"2024-04-25T08:38:17.880984+0000","last_active":"2024-04-25T08:38:19.944386+0000","last_peered":"2024-04-25T08:38:19.944386+0000","last_clean":"2024-04-25T08:38:19.944386+0000","last_became_active":"2024-04-25T08:38:17.880594+0000","last_became_peered":"2024-04-25T08:38:17.880594+0000","last_unstale":"2024-04-25T08:38:19.944386+0000","last_undegraded":"2024-04-25T08:38:19.944386+0000","last_fullsized":"2024-04-25T08:38:19.944386+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:06.858902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882372+0000","last_change":"2024-04-25T08:38:17.881329+0000","last_active":"2024-04-25T08:38:19.882372+0000","last_peered":"2024-04-25T08:38:19.882372+0000","last_clean":"2024-04-25T08:38:19.882372+0000","last_became_active":"2024-04-25T08:38:17.881014+0000","last_became_peered":"2024-04-25T08:38:17.881014+0000","last_unstale":"2024-04-25T08:38:19.882372+0000","last_undegraded":"2024-04-25T08:38:19.882372+0000","last_fullsized":"2024-04-25T08:38:19.882372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944094+0000","last_change":"2024-04-25T08:38:17.876698+0000","last_active":"2024-04-25T08:38:19.944094+0000","last_peered":"2024-04-25T08:38:19.944094+0000","last_clean":"2024-04-25T08:38:19.944094+0000","last_became_active":"2024-04-25T08:38:17.876364+0000","last_became_peered":"2024-04-25T08:38:17.876364+0000","last_unstale":"2024-04-25T08:38:19.944094+0000","last_undegraded":"2024-04-25T08:38:19.944094+0000","last_fullsized":"2024-04-25T08:38:19.944094+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:31.656833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876611+0000","last_change":"2024-04-25T08:38:17.871784+0000","last_active":"2024-04-25T08:38:18.876611+0000","last_peered":"2024-04-25T08:38:18.876611+0000","last_clean":"2024-04-25T08:38:18.876611+0000","last_became_active":"2024-04-25T08:38:17.870733+0000","last_became_peered":"2024-04-25T08:38:17.870733+0000","last_unstale":"2024-04-25T08:38:18.876611+0000","last_undegraded":"2024-04-25T08:38:18.876611+0000","last_fullsized":"2024-04-25T08:38:18.876611+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876372+0000","last_change":"2024-04-25T08:38:17.887048+0000","last_active":"2024-04-25T08:38:18.876372+0000","last_peered":"2024-04-25T08:38:18.876372+0000","last_clean":"2024-04-25T08:38:18.876372+0000","last_became_active":"2024-04-25T08:38:17.886769+0000","last_became_peered":"2024-04-25T08:38:17.886769+0000","last_unstale":"2024-04-25T08:38:18.876372+0000","last_undegraded":"2024-04-25T08:38:18.876372+0000","last_fullsized":"2024-04-25T08:38:18.876372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:06.919441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877229+0000","last_change":"2024-04-25T08:38:17.898751+0000","last_active":"2024-04-25T08:38:18.877229+0000","last_peered":"2024-04-25T08:38:18.877229+0000","last_clean":"2024-04-25T08:38:18.877229+0000","last_became_active":"2024-04-25T08:38:17.898466+0000","last_became_peered":"2024-04-25T08:38:17.898466+0000","last_unstale":"2024-04-25T08:38:18.877229+0000","last_undegraded":"2024-04-25T08:38:18.877229+0000","last_fullsized":"2024-04-25T08:38:18.877229+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:05.202461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943893+0000","last_change":"2024-04-25T08:38:17.884795+0000","last_active":"2024-04-25T08:38:19.943893+0000","last_peered":"2024-04-25T08:38:19.943893+0000","last_clean":"2024-04-25T08:38:19.943893+0000","last_became_active":"2024-04-25T08:38:17.884380+0000","last_became_peered":"2024-04-25T08:38:17.884380+0000","last_unstale":"2024-04-25T08:38:19.943893+0000","last_undegraded":"2024-04-25T08:38:19.943893+0000","last_fullsized":"2024-04-25T08:38:19.943893+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:44.628191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876361+0000","last_change":"2024-04-25T08:38:17.872093+0000","last_active":"2024-04-25T08:38:18.876361+0000","last_peered":"2024-04-25T08:38:18.876361+0000","last_clean":"2024-04-25T08:38:18.876361+0000","last_became_active":"2024-04-25T08:38:17.871801+0000","last_became_peered":"2024-04-25T08:38:17.871801+0000","last_unstale":"2024-04-25T08:38:18.876361+0000","last_undegraded":"2024-04-25T08:38:18.876361+0000","last_fullsized":"2024-04-25T08:38:18.876361+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.057228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876355+0000","last_change":"2024-04-25T08:38:17.871321+0000","last_active":"2024-04-25T08:38:18.876355+0000","last_peered":"2024-04-25T08:38:18.876355+0000","last_clean":"2024-04-25T08:38:18.876355+0000","last_became_active":"2024-04-25T08:38:17.870448+0000","last_became_peered":"2024-04-25T08:38:17.870448+0000","last_unstale":"2024-04-25T08:38:18.876355+0000","last_undegraded":"2024-04-25T08:38:18.876355+0000","last_fullsized":"2024-04-25T08:38:18.876355+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:55.340733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879425+0000","last_change":"2024-04-25T08:38:17.874726+0000","last_active":"2024-04-25T08:38:19.879425+0000","last_peered":"2024-04-25T08:38:19.879425+0000","last_clean":"2024-04-25T08:38:19.879425+0000","last_became_active":"2024-04-25T08:38:17.874458+0000","last_became_peered":"2024-04-25T08:38:17.874458+0000","last_unstale":"2024-04-25T08:38:19.879425+0000","last_undegraded":"2024-04-25T08:38:19.879425+0000","last_fullsized":"2024-04-25T08:38:19.879425+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:47.779597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943170+0000","last_change":"2024-04-25T08:38:17.884450+0000","last_active":"2024-04-25T08:38:19.943170+0000","last_peered":"2024-04-25T08:38:19.943170+0000","last_clean":"2024-04-25T08:38:19.943170+0000","last_became_active":"2024-04-25T08:38:17.884060+0000","last_became_peered":"2024-04-25T08:38:17.884060+0000","last_unstale":"2024-04-25T08:38:19.943170+0000","last_undegraded":"2024-04-25T08:38:19.943170+0000","last_fullsized":"2024-04-25T08:38:19.943170+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:21.455456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879358+0000","last_change":"2024-04-25T08:38:17.885953+0000","last_active":"2024-04-25T08:38:19.879358+0000","last_peered":"2024-04-25T08:38:19.879358+0000","last_clean":"2024-04-25T08:38:19.879358+0000","last_became_active":"2024-04-25T08:38:17.885687+0000","last_became_peered":"2024-04-25T08:38:17.885687+0000","last_unstale":"2024-04-25T08:38:19.879358+0000","last_undegraded":"2024-04-25T08:38:19.879358+0000","last_fullsized":"2024-04-25T08:38:19.879358+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.490629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876440+0000","last_change":"2024-04-25T08:38:16.875570+0000","last_active":"2024-04-25T08:38:18.876440+0000","last_peered":"2024-04-25T08:38:18.876440+0000","last_clean":"2024-04-25T08:38:18.876440+0000","last_became_active":"2024-04-25T08:38:16.875258+0000","last_became_peered":"2024-04-25T08:38:16.875258+0000","last_unstale":"2024-04-25T08:38:18.876440+0000","last_undegraded":"2024-04-25T08:38:18.876440+0000","last_fullsized":"2024-04-25T08:38:18.876440+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:45.186789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943861+0000","last_change":"2024-04-25T08:38:17.877731+0000","last_active":"2024-04-25T08:38:19.943861+0000","last_peered":"2024-04-25T08:38:19.943861+0000","last_clean":"2024-04-25T08:38:19.943861+0000","last_became_active":"2024-04-25T08:38:17.877437+0000","last_became_peered":"2024-04-25T08:38:17.877437+0000","last_unstale":"2024-04-25T08:38:19.943861+0000","last_undegraded":"2024-04-25T08:38:19.943861+0000","last_fullsized":"2024-04-25T08:38:19.943861+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:27.127276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874830+0000","last_change":"2024-04-25T08:38:16.869588+0000","last_active":"2024-04-25T08:38:18.874830+0000","last_peered":"2024-04-25T08:38:18.874830+0000","last_clean":"2024-04-25T08:38:18.874830+0000","last_became_active":"2024-04-25T08:38:16.868936+0000","last_became_peered":"2024-04-25T08:38:16.868936+0000","last_unstale":"2024-04-25T08:38:18.874830+0000","last_undegraded":"2024-04-25T08:38:18.874830+0000","last_fullsized":"2024-04-25T08:38:18.874830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:29.799312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882046+0000","last_change":"2024-04-25T08:38:17.880219+0000","last_active":"2024-04-25T08:38:19.882046+0000","last_peered":"2024-04-25T08:38:19.882046+0000","last_clean":"2024-04-25T08:38:19.882046+0000","last_became_active":"2024-04-25T08:38:17.879938+0000","last_became_peered":"2024-04-25T08:38:17.879938+0000","last_unstale":"2024-04-25T08:38:19.882046+0000","last_undegraded":"2024-04-25T08:38:19.882046+0000","last_fullsized":"2024-04-25T08:38:19.882046+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:06.919441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877294+0000","last_change":"2024-04-25T08:38:16.877474+0000","last_active":"2024-04-25T08:38:18.877294+0000","last_peered":"2024-04-25T08:38:18.877294+0000","last_clean":"2024-04-25T08:38:18.877294+0000","last_became_active":"2024-04-25T08:38:16.877245+0000","last_became_peered":"2024-04-25T08:38:16.877245+0000","last_unstale":"2024-04-25T08:38:18.877294+0000","last_undegraded":"2024-04-25T08:38:18.877294+0000","last_fullsized":"2024-04-25T08:38:18.877294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.336690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943370+0000","last_change":"2024-04-25T08:38:17.886473+0000","last_active":"2024-04-25T08:38:19.943370+0000","last_peered":"2024-04-25T08:38:19.943370+0000","last_clean":"2024-04-25T08:38:19.943370+0000","last_became_active":"2024-04-25T08:38:17.885797+0000","last_became_peered":"2024-04-25T08:38:17.885797+0000","last_unstale":"2024-04-25T08:38:19.943370+0000","last_undegraded":"2024-04-25T08:38:19.943370+0000","last_fullsized":"2024-04-25T08:38:19.943370+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:16.987654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877062+0000","last_change":"2024-04-25T08:38:16.868822+0000","last_active":"2024-04-25T08:38:18.877062+0000","last_peered":"2024-04-25T08:38:18.877062+0000","last_clean":"2024-04-25T08:38:18.877062+0000","last_became_active":"2024-04-25T08:38:16.868564+0000","last_became_peered":"2024-04-25T08:38:16.868564+0000","last_unstale":"2024-04-25T08:38:18.877062+0000","last_undegraded":"2024-04-25T08:38:18.877062+0000","last_fullsized":"2024-04-25T08:38:18.877062+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:27.367777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879613+0000","last_change":"2024-04-25T08:38:17.872175+0000","last_active":"2024-04-25T08:38:19.879613+0000","last_peered":"2024-04-25T08:38:19.879613+0000","last_clean":"2024-04-25T08:38:19.879613+0000","last_became_active":"2024-04-25T08:38:17.871909+0000","last_became_peered":"2024-04-25T08:38:17.871909+0000","last_unstale":"2024-04-25T08:38:19.879613+0000","last_undegraded":"2024-04-25T08:38:19.879613+0000","last_fullsized":"2024-04-25T08:38:19.879613+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:32.701570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879631+0000","last_change":"2024-04-25T08:38:16.869047+0000","last_active":"2024-04-25T08:38:19.879631+0000","last_peered":"2024-04-25T08:38:19.879631+0000","last_clean":"2024-04-25T08:38:19.879631+0000","last_became_active":"2024-04-25T08:38:16.868570+0000","last_became_peered":"2024-04-25T08:38:16.868570+0000","last_unstale":"2024-04-25T08:38:19.879631+0000","last_undegraded":"2024-04-25T08:38:19.879631+0000","last_fullsized":"2024-04-25T08:38:19.879631+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:39.591232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879783+0000","last_change":"2024-04-25T08:38:17.887891+0000","last_active":"2024-04-25T08:38:19.879783+0000","last_peered":"2024-04-25T08:38:19.879783+0000","last_clean":"2024-04-25T08:38:19.879783+0000","last_became_active":"2024-04-25T08:38:17.887612+0000","last_became_peered":"2024-04-25T08:38:17.887612+0000","last_unstale":"2024-04-25T08:38:19.879783+0000","last_undegraded":"2024-04-25T08:38:19.879783+0000","last_fullsized":"2024-04-25T08:38:19.879783+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.803355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874718+0000","last_change":"2024-04-25T08:38:16.868606+0000","last_active":"2024-04-25T08:38:18.874718+0000","last_peered":"2024-04-25T08:38:18.874718+0000","last_clean":"2024-04-25T08:38:18.874718+0000","last_became_active":"2024-04-25T08:38:16.868290+0000","last_became_peered":"2024-04-25T08:38:16.868290+0000","last_unstale":"2024-04-25T08:38:18.874718+0000","last_undegraded":"2024-04-25T08:38:18.874718+0000","last_fullsized":"2024-04-25T08:38:18.874718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.697527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874750+0000","last_change":"2024-04-25T08:38:17.887163+0000","last_active":"2024-04-25T08:38:18.874750+0000","last_peered":"2024-04-25T08:38:18.874750+0000","last_clean":"2024-04-25T08:38:18.874750+0000","last_became_active":"2024-04-25T08:38:17.886853+0000","last_became_peered":"2024-04-25T08:38:17.886853+0000","last_unstale":"2024-04-25T08:38:18.874750+0000","last_undegraded":"2024-04-25T08:38:18.874750+0000","last_fullsized":"2024-04-25T08:38:18.874750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:31.656833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944174+0000","last_change":"2024-04-25T08:38:16.879982+0000","last_active":"2024-04-25T08:38:19.944174+0000","last_peered":"2024-04-25T08:38:19.944174+0000","last_clean":"2024-04-25T08:38:19.944174+0000","last_became_active":"2024-04-25T08:38:16.879692+0000","last_became_peered":"2024-04-25T08:38:16.879692+0000","last_unstale":"2024-04-25T08:38:19.944174+0000","last_undegraded":"2024-04-25T08:38:19.944174+0000","last_fullsized":"2024-04-25T08:38:19.944174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.742158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877301+0000","last_change":"2024-04-25T08:38:17.895788+0000","last_active":"2024-04-25T08:38:18.877301+0000","last_peered":"2024-04-25T08:38:18.877301+0000","last_clean":"2024-04-25T08:38:18.877301+0000","last_became_active":"2024-04-25T08:38:17.895479+0000","last_became_peered":"2024-04-25T08:38:17.895479+0000","last_unstale":"2024-04-25T08:38:18.877301+0000","last_undegraded":"2024-04-25T08:38:18.877301+0000","last_fullsized":"2024-04-25T08:38:18.877301+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:04.128752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876368+0000","last_change":"2024-04-25T08:38:16.865526+0000","last_active":"2024-04-25T08:38:18.876368+0000","last_peered":"2024-04-25T08:38:18.876368+0000","last_clean":"2024-04-25T08:38:18.876368+0000","last_became_active":"2024-04-25T08:38:16.865135+0000","last_became_peered":"2024-04-25T08:38:16.865135+0000","last_unstale":"2024-04-25T08:38:18.876368+0000","last_undegraded":"2024-04-25T08:38:18.876368+0000","last_fullsized":"2024-04-25T08:38:18.876368+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.197963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943446+0000","last_change":"2024-04-25T08:38:17.887207+0000","last_active":"2024-04-25T08:38:19.943446+0000","last_peered":"2024-04-25T08:38:19.943446+0000","last_clean":"2024-04-25T08:38:19.943446+0000","last_became_active":"2024-04-25T08:38:17.886920+0000","last_became_peered":"2024-04-25T08:38:17.886920+0000","last_unstale":"2024-04-25T08:38:19.943446+0000","last_undegraded":"2024-04-25T08:38:19.943446+0000","last_fullsized":"2024-04-25T08:38:19.943446+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:45.695945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882289+0000","last_change":"2024-04-25T08:38:16.870861+0000","last_active":"2024-04-25T08:38:19.882289+0000","last_peered":"2024-04-25T08:38:19.882289+0000","last_clean":"2024-04-25T08:38:19.882289+0000","last_became_active":"2024-04-25T08:38:16.870598+0000","last_became_peered":"2024-04-25T08:38:16.870598+0000","last_unstale":"2024-04-25T08:38:19.882289+0000","last_undegraded":"2024-04-25T08:38:19.882289+0000","last_fullsized":"2024-04-25T08:38:19.882289+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:21.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944319+0000","last_change":"2024-04-25T08:38:17.890425+0000","last_active":"2024-04-25T08:38:19.944319+0000","last_peered":"2024-04-25T08:38:19.944319+0000","last_clean":"2024-04-25T08:38:19.944319+0000","last_became_active":"2024-04-25T08:38:17.890108+0000","last_became_peered":"2024-04-25T08:38:17.890108+0000","last_unstale":"2024-04-25T08:38:19.944319+0000","last_undegraded":"2024-04-25T08:38:19.944319+0000","last_fullsized":"2024-04-25T08:38:19.944319+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:26.696133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944170+0000","last_change":"2024-04-25T08:38:16.868916+0000","last_active":"2024-04-25T08:38:19.944170+0000","last_peered":"2024-04-25T08:38:19.944170+0000","last_clean":"2024-04-25T08:38:19.944170+0000","last_became_active":"2024-04-25T08:38:16.868648+0000","last_became_peered":"2024-04-25T08:38:16.868648+0000","last_unstale":"2024-04-25T08:38:19.944170+0000","last_undegraded":"2024-04-25T08:38:19.944170+0000","last_fullsized":"2024-04-25T08:38:19.944170+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:41.331503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876563+0000","last_change":"2024-04-25T08:38:17.877450+0000","last_active":"2024-04-25T08:38:18.876563+0000","last_peered":"2024-04-25T08:38:18.876563+0000","last_clean":"2024-04-25T08:38:18.876563+0000","last_became_active":"2024-04-25T08:38:17.877096+0000","last_became_peered":"2024-04-25T08:38:17.877096+0000","last_unstale":"2024-04-25T08:38:18.876563+0000","last_undegraded":"2024-04-25T08:38:18.876563+0000","last_fullsized":"2024-04-25T08:38:18.876563+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877437+0000","last_change":"2024-04-25T08:38:16.872592+0000","last_active":"2024-04-25T08:38:18.877437+0000","last_peered":"2024-04-25T08:38:18.877437+0000","last_clean":"2024-04-25T08:38:18.877437+0000","last_became_active":"2024-04-25T08:38:16.872012+0000","last_became_peered":"2024-04-25T08:38:16.872012+0000","last_unstale":"2024-04-25T08:38:18.877437+0000","last_undegraded":"2024-04-25T08:38:18.877437+0000","last_fullsized":"2024-04-25T08:38:18.877437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:12.053186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876280+0000","last_change":"2024-04-25T08:38:17.889252+0000","last_active":"2024-04-25T08:38:18.876280+0000","last_peered":"2024-04-25T08:38:18.876280+0000","last_clean":"2024-04-25T08:38:18.876280+0000","last_became_active":"2024-04-25T08:38:17.888591+0000","last_became_peered":"2024-04-25T08:38:17.888591+0000","last_unstale":"2024-04-25T08:38:18.876280+0000","last_undegraded":"2024-04-25T08:38:18.876280+0000","last_fullsized":"2024-04-25T08:38:18.876280+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.675286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877066+0000","last_change":"2024-04-25T08:38:16.871721+0000","last_active":"2024-04-25T08:38:18.877066+0000","last_peered":"2024-04-25T08:38:18.877066+0000","last_clean":"2024-04-25T08:38:18.877066+0000","last_became_active":"2024-04-25T08:38:16.871135+0000","last_became_peered":"2024-04-25T08:38:16.871135+0000","last_unstale":"2024-04-25T08:38:18.877066+0000","last_undegraded":"2024-04-25T08:38:18.877066+0000","last_fullsized":"2024-04-25T08:38:18.877066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:05.915398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944043+0000","last_change":"2024-04-25T08:38:17.884680+0000","last_active":"2024-04-25T08:38:19.944043+0000","last_peered":"2024-04-25T08:38:19.944043+0000","last_clean":"2024-04-25T08:38:19.944043+0000","last_became_active":"2024-04-25T08:38:17.884023+0000","last_became_peered":"2024-04-25T08:38:17.884023+0000","last_unstale":"2024-04-25T08:38:19.944043+0000","last_undegraded":"2024-04-25T08:38:19.944043+0000","last_fullsized":"2024-04-25T08:38:19.944043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:30.754146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943481+0000","last_change":"2024-04-25T08:38:16.873738+0000","last_active":"2024-04-25T08:38:19.943481+0000","last_peered":"2024-04-25T08:38:19.943481+0000","last_clean":"2024-04-25T08:38:19.943481+0000","last_became_active":"2024-04-25T08:38:16.873338+0000","last_became_peered":"2024-04-25T08:38:16.873338+0000","last_unstale":"2024-04-25T08:38:19.943481+0000","last_undegraded":"2024-04-25T08:38:19.943481+0000","last_fullsized":"2024-04-25T08:38:19.943481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:11.883106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877409+0000","last_change":"2024-04-25T08:38:17.895519+0000","last_active":"2024-04-25T08:38:18.877409+0000","last_peered":"2024-04-25T08:38:18.877409+0000","last_clean":"2024-04-25T08:38:18.877409+0000","last_became_active":"2024-04-25T08:38:17.895149+0000","last_became_peered":"2024-04-25T08:38:17.895149+0000","last_unstale":"2024-04-25T08:38:18.877409+0000","last_undegraded":"2024-04-25T08:38:18.877409+0000","last_fullsized":"2024-04-25T08:38:18.877409+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.397082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875906+0000","last_change":"2024-04-25T08:38:16.868510+0000","last_active":"2024-04-25T08:38:18.875906+0000","last_peered":"2024-04-25T08:38:18.875906+0000","last_clean":"2024-04-25T08:38:18.875906+0000","last_became_active":"2024-04-25T08:38:16.868241+0000","last_became_peered":"2024-04-25T08:38:16.868241+0000","last_unstale":"2024-04-25T08:38:18.875906+0000","last_undegraded":"2024-04-25T08:38:18.875906+0000","last_fullsized":"2024-04-25T08:38:18.875906+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:55.755478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874750+0000","last_change":"2024-04-25T08:38:17.884410+0000","last_active":"2024-04-25T08:38:18.874750+0000","last_peered":"2024-04-25T08:38:18.874750+0000","last_clean":"2024-04-25T08:38:18.874750+0000","last_became_active":"2024-04-25T08:38:17.884122+0000","last_became_peered":"2024-04-25T08:38:17.884122+0000","last_unstale":"2024-04-25T08:38:18.874750+0000","last_undegraded":"2024-04-25T08:38:18.874750+0000","last_fullsized":"2024-04-25T08:38:18.874750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:26.696133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944197+0000","last_change":"2024-04-25T08:38:16.876363+0000","last_active":"2024-04-25T08:38:19.944197+0000","last_peered":"2024-04-25T08:38:19.944197+0000","last_clean":"2024-04-25T08:38:19.944197+0000","last_became_active":"2024-04-25T08:38:16.876049+0000","last_became_peered":"2024-04-25T08:38:16.876049+0000","last_unstale":"2024-04-25T08:38:19.944197+0000","last_undegraded":"2024-04-25T08:38:19.944197+0000","last_fullsized":"2024-04-25T08:38:19.944197+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:21.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876466+0000","last_change":"2024-04-25T08:38:17.881462+0000","last_active":"2024-04-25T08:38:18.876466+0000","last_peered":"2024-04-25T08:38:18.876466+0000","last_clean":"2024-04-25T08:38:18.876466+0000","last_became_active":"2024-04-25T08:38:17.880097+0000","last_became_peered":"2024-04-25T08:38:17.880097+0000","last_unstale":"2024-04-25T08:38:18.876466+0000","last_undegraded":"2024-04-25T08:38:18.876466+0000","last_fullsized":"2024-04-25T08:38:18.876466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882404+0000","last_change":"2024-04-25T08:38:16.866476+0000","last_active":"2024-04-25T08:38:19.882404+0000","last_peered":"2024-04-25T08:38:19.882404+0000","last_clean":"2024-04-25T08:38:19.882404+0000","last_became_active":"2024-04-25T08:38:16.865868+0000","last_became_peered":"2024-04-25T08:38:16.865868+0000","last_unstale":"2024-04-25T08:38:19.882404+0000","last_undegraded":"2024-04-25T08:38:19.882404+0000","last_fullsized":"2024-04-25T08:38:19.882404+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:46.775554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875571+0000","last_change":"2024-04-25T08:38:17.896892+0000","last_active":"2024-04-25T08:38:18.875571+0000","last_peered":"2024-04-25T08:38:18.875571+0000","last_clean":"2024-04-25T08:38:18.875571+0000","last_became_active":"2024-04-25T08:38:17.895902+0000","last_became_peered":"2024-04-25T08:38:17.895902+0000","last_unstale":"2024-04-25T08:38:18.875571+0000","last_undegraded":"2024-04-25T08:38:18.875571+0000","last_fullsized":"2024-04-25T08:38:18.875571+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:34.550562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876229+0000","last_change":"2024-04-25T08:38:16.868952+0000","last_active":"2024-04-25T08:38:18.876229+0000","last_peered":"2024-04-25T08:38:18.876229+0000","last_clean":"2024-04-25T08:38:18.876229+0000","last_became_active":"2024-04-25T08:38:16.868689+0000","last_became_peered":"2024-04-25T08:38:16.868689+0000","last_unstale":"2024-04-25T08:38:18.876229+0000","last_undegraded":"2024-04-25T08:38:18.876229+0000","last_fullsized":"2024-04-25T08:38:18.876229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:20.443300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875867+0000","last_change":"2024-04-25T08:38:17.897649+0000","last_active":"2024-04-25T08:38:18.875867+0000","last_peered":"2024-04-25T08:38:18.875867+0000","last_clean":"2024-04-25T08:38:18.875867+0000","last_became_active":"2024-04-25T08:38:17.896745+0000","last_became_peered":"2024-04-25T08:38:17.896745+0000","last_unstale":"2024-04-25T08:38:18.875867+0000","last_undegraded":"2024-04-25T08:38:18.875867+0000","last_fullsized":"2024-04-25T08:38:18.875867+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:03.164735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944209+0000","last_change":"2024-04-25T08:38:16.879316+0000","last_active":"2024-04-25T08:38:19.944209+0000","last_peered":"2024-04-25T08:38:19.944209+0000","last_clean":"2024-04-25T08:38:19.944209+0000","last_became_active":"2024-04-25T08:38:16.879048+0000","last_became_peered":"2024-04-25T08:38:16.879048+0000","last_unstale":"2024-04-25T08:38:19.944209+0000","last_undegraded":"2024-04-25T08:38:19.944209+0000","last_fullsized":"2024-04-25T08:38:19.944209+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943504+0000","last_change":"2024-04-25T08:38:17.887226+0000","last_active":"2024-04-25T08:38:19.943504+0000","last_peered":"2024-04-25T08:38:19.943504+0000","last_clean":"2024-04-25T08:38:19.943504+0000","last_became_active":"2024-04-25T08:38:17.886924+0000","last_became_peered":"2024-04-25T08:38:17.886924+0000","last_unstale":"2024-04-25T08:38:19.943504+0000","last_undegraded":"2024-04-25T08:38:19.943504+0000","last_fullsized":"2024-04-25T08:38:19.943504+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:58.314369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874682+0000","last_change":"2024-04-25T08:38:16.878410+0000","last_active":"2024-04-25T08:38:18.874682+0000","last_peered":"2024-04-25T08:38:18.874682+0000","last_clean":"2024-04-25T08:38:18.874682+0000","last_became_active":"2024-04-25T08:38:16.878111+0000","last_became_peered":"2024-04-25T08:38:16.878111+0000","last_unstale":"2024-04-25T08:38:18.874682+0000","last_undegraded":"2024-04-25T08:38:18.874682+0000","last_fullsized":"2024-04-25T08:38:18.874682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874630+0000","last_change":"2024-04-25T08:38:17.881756+0000","last_active":"2024-04-25T08:38:18.874630+0000","last_peered":"2024-04-25T08:38:18.874630+0000","last_clean":"2024-04-25T08:38:18.874630+0000","last_became_active":"2024-04-25T08:38:17.881349+0000","last_became_peered":"2024-04-25T08:38:17.881349+0000","last_unstale":"2024-04-25T08:38:18.874630+0000","last_undegraded":"2024-04-25T08:38:18.874630+0000","last_fullsized":"2024-04-25T08:38:18.874630+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:07.729869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879784+0000","last_change":"2024-04-25T08:38:16.865428+0000","last_active":"2024-04-25T08:38:19.879784+0000","last_peered":"2024-04-25T08:38:19.879784+0000","last_clean":"2024-04-25T08:38:19.879784+0000","last_became_active":"2024-04-25T08:38:16.865137+0000","last_became_peered":"2024-04-25T08:38:16.865137+0000","last_unstale":"2024-04-25T08:38:19.879784+0000","last_undegraded":"2024-04-25T08:38:19.879784+0000","last_fullsized":"2024-04-25T08:38:19.879784+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:55.755478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944757+0000","last_change":"2024-04-25T08:38:17.889169+0000","last_active":"2024-04-25T08:38:19.944757+0000","last_peered":"2024-04-25T08:38:19.944757+0000","last_clean":"2024-04-25T08:38:19.944757+0000","last_became_active":"2024-04-25T08:38:17.888859+0000","last_became_peered":"2024-04-25T08:38:17.888859+0000","last_unstale":"2024-04-25T08:38:19.944757+0000","last_undegraded":"2024-04-25T08:38:19.944757+0000","last_fullsized":"2024-04-25T08:38:19.944757+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:21.979826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944552+0000","last_change":"2024-04-25T08:38:16.870881+0000","last_active":"2024-04-25T08:38:19.944552+0000","last_peered":"2024-04-25T08:38:19.944552+0000","last_clean":"2024-04-25T08:38:19.944552+0000","last_became_active":"2024-04-25T08:38:16.870440+0000","last_became_peered":"2024-04-25T08:38:16.870440+0000","last_unstale":"2024-04-25T08:38:19.944552+0000","last_undegraded":"2024-04-25T08:38:19.944552+0000","last_fullsized":"2024-04-25T08:38:19.944552+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:25.178803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874779+0000","last_change":"2024-04-25T08:38:17.885425+0000","last_active":"2024-04-25T08:38:18.874779+0000","last_peered":"2024-04-25T08:38:18.874779+0000","last_clean":"2024-04-25T08:38:18.874779+0000","last_became_active":"2024-04-25T08:38:17.885118+0000","last_became_peered":"2024-04-25T08:38:17.885118+0000","last_unstale":"2024-04-25T08:38:18.874779+0000","last_undegraded":"2024-04-25T08:38:18.874779+0000","last_fullsized":"2024-04-25T08:38:18.874779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.466242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875707+0000","last_change":"2024-04-25T08:38:16.869530+0000","last_active":"2024-04-25T08:38:18.875707+0000","last_peered":"2024-04-25T08:38:18.875707+0000","last_clean":"2024-04-25T08:38:18.875707+0000","last_became_active":"2024-04-25T08:38:16.869225+0000","last_became_peered":"2024-04-25T08:38:16.869225+0000","last_unstale":"2024-04-25T08:38:18.875707+0000","last_undegraded":"2024-04-25T08:38:18.875707+0000","last_fullsized":"2024-04-25T08:38:18.875707+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:21.300604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882377+0000","last_change":"2024-04-25T08:38:17.884940+0000","last_active":"2024-04-25T08:38:19.882377+0000","last_peered":"2024-04-25T08:38:19.882377+0000","last_clean":"2024-04-25T08:38:19.882377+0000","last_became_active":"2024-04-25T08:38:17.884659+0000","last_became_peered":"2024-04-25T08:38:17.884659+0000","last_unstale":"2024-04-25T08:38:19.882377+0000","last_undegraded":"2024-04-25T08:38:19.882377+0000","last_fullsized":"2024-04-25T08:38:19.882377+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944287+0000","last_change":"2024-04-25T08:38:16.877836+0000","last_active":"2024-04-25T08:38:19.944287+0000","last_peered":"2024-04-25T08:38:19.944287+0000","last_clean":"2024-04-25T08:38:19.944287+0000","last_became_active":"2024-04-25T08:38:16.876856+0000","last_became_peered":"2024-04-25T08:38:16.876856+0000","last_unstale":"2024-04-25T08:38:19.944287+0000","last_undegraded":"2024-04-25T08:38:19.944287+0000","last_fullsized":"2024-04-25T08:38:19.944287+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:46.775554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874465+0000","last_change":"2024-04-25T08:38:17.885787+0000","last_active":"2024-04-25T08:38:18.874465+0000","last_peered":"2024-04-25T08:38:18.874465+0000","last_clean":"2024-04-25T08:38:18.874465+0000","last_became_active":"2024-04-25T08:38:17.885447+0000","last_became_peered":"2024-04-25T08:38:17.885447+0000","last_unstale":"2024-04-25T08:38:18.874465+0000","last_undegraded":"2024-04-25T08:38:18.874465+0000","last_fullsized":"2024-04-25T08:38:18.874465+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944775+0000","last_change":"2024-04-25T08:38:16.874920+0000","last_active":"2024-04-25T08:38:19.944775+0000","last_peered":"2024-04-25T08:38:19.944775+0000","last_clean":"2024-04-25T08:38:19.944775+0000","last_became_active":"2024-04-25T08:38:16.874423+0000","last_became_peered":"2024-04-25T08:38:16.874423+0000","last_unstale":"2024-04-25T08:38:19.944775+0000","last_undegraded":"2024-04-25T08:38:19.944775+0000","last_fullsized":"2024-04-25T08:38:19.944775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:00.856047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876026+0000","last_change":"2024-04-25T08:38:17.888010+0000","last_active":"2024-04-25T08:38:18.876026+0000","last_peered":"2024-04-25T08:38:18.876026+0000","last_clean":"2024-04-25T08:38:18.876026+0000","last_became_active":"2024-04-25T08:38:17.887721+0000","last_became_peered":"2024-04-25T08:38:17.887721+0000","last_unstale":"2024-04-25T08:38:18.876026+0000","last_undegraded":"2024-04-25T08:38:18.876026+0000","last_fullsized":"2024-04-25T08:38:18.876026+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:26.152377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"134'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881249+0000","last_change":"2024-04-25T08:38:16.863623+0000","last_active":"2024-04-25T08:38:19.881249+0000","last_peered":"2024-04-25T08:38:19.881249+0000","last_clean":"2024-04-25T08:38:19.881249+0000","last_became_active":"2024-04-25T08:38:16.863233+0000","last_became_peered":"2024-04-25T08:38:16.863233+0000","last_unstale":"2024-04-25T08:38:19.881249+0000","last_undegraded":"2024-04-25T08:38:19.881249+0000","last_fullsized":"2024-04-25T08:38:19.881249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:21.069051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875571+0000","last_change":"2024-04-25T08:38:17.892580+0000","last_active":"2024-04-25T08:38:18.875571+0000","last_peered":"2024-04-25T08:38:18.875571+0000","last_clean":"2024-04-25T08:38:18.875571+0000","last_became_active":"2024-04-25T08:38:17.889740+0000","last_became_peered":"2024-04-25T08:38:17.889740+0000","last_unstale":"2024-04-25T08:38:18.875571+0000","last_undegraded":"2024-04-25T08:38:18.875571+0000","last_fullsized":"2024-04-25T08:38:18.875571+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:42.269427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879793+0000","last_change":"2024-04-25T08:38:16.870790+0000","last_active":"2024-04-25T08:38:19.879793+0000","last_peered":"2024-04-25T08:38:19.879793+0000","last_clean":"2024-04-25T08:38:19.879793+0000","last_became_active":"2024-04-25T08:38:16.870489+0000","last_became_peered":"2024-04-25T08:38:16.870489+0000","last_unstale":"2024-04-25T08:38:19.879793+0000","last_undegraded":"2024-04-25T08:38:19.879793+0000","last_fullsized":"2024-04-25T08:38:19.879793+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.197963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943648+0000","last_change":"2024-04-25T08:38:17.884365+0000","last_active":"2024-04-25T08:38:19.943648+0000","last_peered":"2024-04-25T08:38:19.943648+0000","last_clean":"2024-04-25T08:38:19.943648+0000","last_became_active":"2024-04-25T08:38:17.884041+0000","last_became_peered":"2024-04-25T08:38:17.884041+0000","last_unstale":"2024-04-25T08:38:19.943648+0000","last_undegraded":"2024-04-25T08:38:19.943648+0000","last_fullsized":"2024-04-25T08:38:19.943648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:42.402577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943793+0000","last_change":"2024-04-25T08:38:16.878761+0000","last_active":"2024-04-25T08:38:19.943793+0000","last_peered":"2024-04-25T08:38:19.943793+0000","last_clean":"2024-04-25T08:38:19.943793+0000","last_became_active":"2024-04-25T08:38:16.878118+0000","last_became_peered":"2024-04-25T08:38:16.878118+0000","last_unstale":"2024-04-25T08:38:19.943793+0000","last_undegraded":"2024-04-25T08:38:19.943793+0000","last_fullsized":"2024-04-25T08:38:19.943793+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:46.498729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943685+0000","last_change":"2024-04-25T08:38:17.877565+0000","last_active":"2024-04-25T08:38:19.943685+0000","last_peered":"2024-04-25T08:38:19.943685+0000","last_clean":"2024-04-25T08:38:19.943685+0000","last_became_active":"2024-04-25T08:38:17.876848+0000","last_became_peered":"2024-04-25T08:38:17.876848+0000","last_unstale":"2024-04-25T08:38:19.943685+0000","last_undegraded":"2024-04-25T08:38:19.943685+0000","last_fullsized":"2024-04-25T08:38:19.943685+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882003+0000","last_change":"2024-04-25T08:38:16.875734+0000","last_active":"2024-04-25T08:38:19.882003+0000","last_peered":"2024-04-25T08:38:19.882003+0000","last_clean":"2024-04-25T08:38:19.882003+0000","last_became_active":"2024-04-25T08:38:16.875418+0000","last_became_peered":"2024-04-25T08:38:16.875418+0000","last_unstale":"2024-04-25T08:38:19.882003+0000","last_undegraded":"2024-04-25T08:38:19.882003+0000","last_fullsized":"2024-04-25T08:38:19.882003+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:46.498729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944097+0000","last_change":"2024-04-25T08:38:17.878723+0000","last_active":"2024-04-25T08:38:19.944097+0000","last_peered":"2024-04-25T08:38:19.944097+0000","last_clean":"2024-04-25T08:38:19.944097+0000","last_became_active":"2024-04-25T08:38:17.878200+0000","last_became_peered":"2024-04-25T08:38:17.878200+0000","last_unstale":"2024-04-25T08:38:19.944097+0000","last_undegraded":"2024-04-25T08:38:19.944097+0000","last_fullsized":"2024-04-25T08:38:19.944097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:07.729869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875920+0000","last_change":"2024-04-25T08:38:16.873456+0000","last_active":"2024-04-25T08:38:18.875920+0000","last_peered":"2024-04-25T08:38:18.875920+0000","last_clean":"2024-04-25T08:38:18.875920+0000","last_became_active":"2024-04-25T08:38:16.873176+0000","last_became_peered":"2024-04-25T08:38:16.873176+0000","last_unstale":"2024-04-25T08:38:18.875920+0000","last_undegraded":"2024-04-25T08:38:18.875920+0000","last_fullsized":"2024-04-25T08:38:18.875920+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.941893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943958+0000","last_change":"2024-04-25T08:38:17.892178+0000","last_active":"2024-04-25T08:38:19.943958+0000","last_peered":"2024-04-25T08:38:19.943958+0000","last_clean":"2024-04-25T08:38:19.943958+0000","last_became_active":"2024-04-25T08:38:17.891887+0000","last_became_peered":"2024-04-25T08:38:17.891887+0000","last_unstale":"2024-04-25T08:38:19.943958+0000","last_undegraded":"2024-04-25T08:38:19.943958+0000","last_fullsized":"2024-04-25T08:38:19.943958+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944410+0000","last_change":"2024-04-25T08:38:16.878410+0000","last_active":"2024-04-25T08:38:19.944410+0000","last_peered":"2024-04-25T08:38:19.944410+0000","last_clean":"2024-04-25T08:38:19.944410+0000","last_became_active":"2024-04-25T08:38:16.877139+0000","last_became_peered":"2024-04-25T08:38:16.877139+0000","last_unstale":"2024-04-25T08:38:19.944410+0000","last_undegraded":"2024-04-25T08:38:19.944410+0000","last_fullsized":"2024-04-25T08:38:19.944410+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:16.115180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877343+0000","last_change":"2024-04-25T08:38:17.897234+0000","last_active":"2024-04-25T08:38:18.877343+0000","last_peered":"2024-04-25T08:38:18.877343+0000","last_clean":"2024-04-25T08:38:18.877343+0000","last_became_active":"2024-04-25T08:38:17.896958+0000","last_became_peered":"2024-04-25T08:38:17.896958+0000","last_unstale":"2024-04-25T08:38:18.877343+0000","last_undegraded":"2024-04-25T08:38:18.877343+0000","last_fullsized":"2024-04-25T08:38:18.877343+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:19.842177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874473+0000","last_change":"2024-04-25T08:38:16.866696+0000","last_active":"2024-04-25T08:38:18.874473+0000","last_peered":"2024-04-25T08:38:18.874473+0000","last_clean":"2024-04-25T08:38:18.874473+0000","last_became_active":"2024-04-25T08:38:16.866421+0000","last_became_peered":"2024-04-25T08:38:16.866421+0000","last_unstale":"2024-04-25T08:38:18.874473+0000","last_undegraded":"2024-04-25T08:38:18.874473+0000","last_fullsized":"2024-04-25T08:38:18.874473+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:16.115180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876675+0000","last_change":"2024-04-25T08:38:17.879750+0000","last_active":"2024-04-25T08:38:18.876675+0000","last_peered":"2024-04-25T08:38:18.876675+0000","last_clean":"2024-04-25T08:38:18.876675+0000","last_became_active":"2024-04-25T08:38:17.879471+0000","last_became_peered":"2024-04-25T08:38:17.879471+0000","last_unstale":"2024-04-25T08:38:18.876675+0000","last_undegraded":"2024-04-25T08:38:18.876675+0000","last_fullsized":"2024-04-25T08:38:18.876675+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875673+0000","last_change":"2024-04-25T08:38:16.870075+0000","last_active":"2024-04-25T08:38:18.875673+0000","last_peered":"2024-04-25T08:38:18.875673+0000","last_clean":"2024-04-25T08:38:18.875673+0000","last_became_active":"2024-04-25T08:38:16.869730+0000","last_became_peered":"2024-04-25T08:38:16.869730+0000","last_unstale":"2024-04-25T08:38:18.875673+0000","last_undegraded":"2024-04-25T08:38:18.875673+0000","last_fullsized":"2024-04-25T08:38:18.875673+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:57.363947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":381,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875376+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:38:18.875376+0000","last_peered":"2024-04-25T08:38:18.875376+0000","last_clean":"2024-04-25T08:38:18.875376+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:38:18.875376+0000","last_undegraded":"2024-04-25T08:38:18.875376+0000","last_fullsized":"2024-04-25T08:38:18.875376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542258,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47304,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324536,"statfs":{"total":96636764160,"available":96588324864,"internally_reserved":0,"allocated":2281472,"data_stored":1372601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542258,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50868,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320972,"statfs":{"total":96636764160,"available":96584675328,"internally_reserved":0,"allocated":1679360,"data_stored":781203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542258,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46704,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325136,"statfs":{"total":96636764160,"available":96588939264,"internally_reserved":0,"allocated":1675264,"data_stored":780553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574961,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50860,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320980,"statfs":{"total":96636764160,"available":96584683520,"internally_reserved":0,"allocated":1671168,"data_stored":780531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59200,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312640,"statfs":{"total":96636764160,"available":96576143360,"internally_reserved":0,"allocated":1687552,"data_stored":782213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59188,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312652,"statfs":{"total":96636764160,"available":96576155648,"internally_reserved":0,"allocated":1679360,"data_stored":781147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55024,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316816,"statfs":{"total":96636764160,"available":96580419584,"internally_reserved":0,"allocated":1675264,"data_stored":780601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59764,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312076,"statfs":{"total":96636764160,"available":96575565824,"internally_reserved":0,"allocated":2260992,"data_stored":1364263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:25.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.421+0000 7f1a2a709640 1 Processor -- start 2024-04-25T08:38:25.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.421+0000 7f1a2a709640 1 -- start start 2024-04-25T08:38:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a2a709640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a2415dbe0 0x7f1a240d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a2a709640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a240b2cb0 0x7f1a240d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a2a709640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a240b97a0 con 0x7f1a240b92a0 2024-04-25T08:38:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a2a709640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2415dfc0 con 0x7f1a2415dbe0 2024-04-25T08:38:25.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a2a709640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a240d1a00 con 0x7f1a240b2cb0 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a2415dbe0 0x7f1a240d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1a2415dbe0 0x7f1a240d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60774/0 (socket says 172.21.15.60:60774) 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 -- 172.21.15.60:0/3107912709 learned_addr learned my addr 172.21.15.60:0/3107912709 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a237fe640 1 --2- 172.21.15.60:0/3107912709 >> v2:172.21.15.100:3300/0 conn(0x7f1a240b2cb0 0x7f1a240d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 -- 172.21.15.60:0/3107912709 >> v2:172.21.15.100:3300/0 conn(0x7f1a240b2cb0 msgr2=0x7f1a240d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 --2- 172.21.15.60:0/3107912709 >> v2:172.21.15.100:3300/0 conn(0x7f1a240b2cb0 0x7f1a240d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 -- 172.21.15.60:0/3107912709 >> v1:172.21.15.100:6789/0 conn(0x7f1a240b92a0 legacy=0x7f1a240b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.422+0000 7f1a23fff640 1 -- 172.21.15.60:0/3107912709 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240d1c50 con 0x7f1a2415dbe0 2024-04-25T08:38:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.423+0000 7f1a23fff640 1 --2- 172.21.15.60:0/3107912709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240d0910 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1a2406cff0 tx=0x7f1a1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198f017291145e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.423+0000 7f1a22ffd640 1 -- 172.21.15.60:0/3107912709 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14037070 con 0x7f1a2415dbe0 2024-04-25T08:38:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.423+0000 7f1a22ffd640 1 -- 172.21.15.60:0/3107912709 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a14032440 con 0x7f1a2415dbe0 2024-04-25T08:38:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.423+0000 7f1a22ffd640 1 -- 172.21.15.60:0/3107912709 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14032810 con 0x7f1a2415dbe0 2024-04-25T08:38:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 -- 172.21.15.60:0/3107912709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 msgr2=0x7f1a240d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 --2- 172.21.15.60:0/3107912709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240d0910 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1a2406cff0 tx=0x7f1a1402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 -- 172.21.15.60:0/3107912709 shutdown_connections 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 --2- 172.21.15.60:0/3107912709 >> v2:172.21.15.100:3300/0 conn(0x7f1a240b2cb0 0x7f1a240d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 --2- 172.21.15.60:0/3107912709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240d0910 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 -- 172.21.15.60:0/3107912709 >> 172.21.15.60:0/3107912709 conn(0x7f1a24061a20 msgr2=0x7f1a24061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 -- 172.21.15.60:0/3107912709 shutdown_connections 2024-04-25T08:38:25.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.424+0000 7f1a2a709640 1 -- 172.21.15.60:0/3107912709 wait complete. 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.425+0000 7f1a2a709640 1 Processor -- start 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 -- start start 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a240b2cb0 0x7f1a2414b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a240b92a0 0x7f1a240caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b6ab0 con 0x7f1a2415dbe0 2024-04-25T08:38:25.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240d2210 con 0x7f1a240b92a0 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a2a709640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b6c30 con 0x7f1a240b2cb0 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a28ca6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a28ca6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60782/0 (socket says 172.21.15.60:60782) 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.426+0000 7f1a28ca6640 1 -- 172.21.15.60:0/1226573045 learned_addr learned my addr 172.21.15.60:0/1226573045 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a237fe640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a240b92a0 0x7f1a240caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a23fff640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a240b2cb0 0x7f1a2414b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a28ca6640 1 -- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a240b2cb0 msgr2=0x7f1a2414b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a28ca6640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a240b2cb0 0x7f1a2414b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a28ca6640 1 -- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a240b92a0 msgr2=0x7f1a240caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a28ca6640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a240b92a0 0x7f1a240caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a28ca6640 1 -- 172.21.15.60:0/1226573045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2414bbd0 con 0x7f1a2415dbe0 2024-04-25T08:38:25.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a28ca6640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240cc3f0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c730 tx=0x7f1a1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c016020 con 0x7f1a2415dbe0 2024-04-25T08:38:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c006d50 con 0x7f1a2415dbe0 2024-04-25T08:38:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240cb350 con 0x7f1a2415dbe0 2024-04-25T08:38:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c003bb0 con 0x7f1a2415dbe0 2024-04-25T08:38:25.423 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.427+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a240cbbf0 con 0x7f1a2415dbe0 2024-04-25T08:38:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.428+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a24073990 con 0x7f1a2415dbe0 2024-04-25T08:38:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.429+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a1c00a7f0 con 0x7f1a2415dbe0 2024-04-25T08:38:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.429+0000 7f1a20ff9640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a1006b4d0 0x7f1a1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.429+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a1c089f90 con 0x7f1a2415dbe0 2024-04-25T08:38:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.430+0000 7f1a23fff640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a1006b4d0 0x7f1a1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.430+0000 7f1a23fff640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a1006b4d0 0x7f1a1006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1a14034000 tx=0x7f1a1402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.434+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a1c020020 con 0x7f1a2415dbe0 2024-04-25T08:38:25.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.610+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a240ca0e0 con 0x7f1a1006b4d0 2024-04-25T08:38:25.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.617+0000 7f1a20ff9640 1 -- 172.21.15.60:0/1226573045 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252422 (secure 0 0 0) 0x7f1a240ca0e0 con 0x7f1a1006b4d0 2024-04-25T08:38:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:25.614 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:38:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a1006b4d0 msgr2=0x7f1a1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a1006b4d0 0x7f1a1006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1a14034000 tx=0x7f1a1402e000 comp rx=0 tx=0).stop 2024-04-25T08:38:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 msgr2=0x7f1a240cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240cc3f0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00c730 tx=0x7f1a1c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 shutdown_connections 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a2415dbe0 0x7f1a240cc3f0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1a1006b4d0 0x7f1a1006d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a240b92a0 0x7f1a240caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 --2- 172.21.15.60:0/1226573045 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1a240b2cb0 0x7f1a2414b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 >> 172.21.15.60:0/1226573045 conn(0x7f1a24061a20 msgr2=0x7f1a2415ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 shutdown_connections 2024-04-25T08:38:25.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.620+0000 7f1a2a709640 1 -- 172.21.15.60:0/1226573045 wait complete. 2024-04-25T08:38:25.624 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-04-25T08:38:25.523286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":428916,"kb_used_data":14280,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754545804,"statfs":{"total":773094113280,"available":772654903296,"internally_reserved":0,"allocated":14622720,"data_stored":7429804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002109"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944892+0000","last_change":"2024-04-25T08:38:17.888831+0000","last_active":"2024-04-25T08:38:19.944892+0000","last_peered":"2024-04-25T08:38:19.944892+0000","last_clean":"2024-04-25T08:38:19.944892+0000","last_became_active":"2024-04-25T08:38:17.888425+0000","last_became_peered":"2024-04-25T08:38:17.888425+0000","last_unstale":"2024-04-25T08:38:19.944892+0000","last_undegraded":"2024-04-25T08:38:19.944892+0000","last_fullsized":"2024-04-25T08:38:19.944892+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:22.446450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882289+0000","last_change":"2024-04-25T08:38:17.878980+0000","last_active":"2024-04-25T08:38:19.882289+0000","last_peered":"2024-04-25T08:38:19.882289+0000","last_clean":"2024-04-25T08:38:19.882289+0000","last_became_active":"2024-04-25T08:38:17.878667+0000","last_became_peered":"2024-04-25T08:38:17.878667+0000","last_unstale":"2024-04-25T08:38:19.882289+0000","last_undegraded":"2024-04-25T08:38:19.882289+0000","last_fullsized":"2024-04-25T08:38:19.882289+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876695+0000","last_change":"2024-04-25T08:38:17.879831+0000","last_active":"2024-04-25T08:38:18.876695+0000","last_peered":"2024-04-25T08:38:18.876695+0000","last_clean":"2024-04-25T08:38:18.876695+0000","last_became_active":"2024-04-25T08:38:17.879554+0000","last_became_peered":"2024-04-25T08:38:17.879554+0000","last_unstale":"2024-04-25T08:38:18.876695+0000","last_undegraded":"2024-04-25T08:38:18.876695+0000","last_fullsized":"2024-04-25T08:38:18.876695+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882016+0000","last_change":"2024-04-25T08:38:17.891476+0000","last_active":"2024-04-25T08:38:19.882016+0000","last_peered":"2024-04-25T08:38:19.882016+0000","last_clean":"2024-04-25T08:38:19.882016+0000","last_became_active":"2024-04-25T08:38:17.891162+0000","last_became_peered":"2024-04-25T08:38:17.891162+0000","last_unstale":"2024-04-25T08:38:19.882016+0000","last_undegraded":"2024-04-25T08:38:19.882016+0000","last_fullsized":"2024-04-25T08:38:19.882016+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.057228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874423+0000","last_change":"2024-04-25T08:38:17.881621+0000","last_active":"2024-04-25T08:38:18.874423+0000","last_peered":"2024-04-25T08:38:18.874423+0000","last_clean":"2024-04-25T08:38:18.874423+0000","last_became_active":"2024-04-25T08:38:17.881287+0000","last_became_peered":"2024-04-25T08:38:17.881287+0000","last_unstale":"2024-04-25T08:38:18.874423+0000","last_undegraded":"2024-04-25T08:38:18.874423+0000","last_fullsized":"2024-04-25T08:38:18.874423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943745+0000","last_change":"2024-04-25T08:38:17.874836+0000","last_active":"2024-04-25T08:38:19.943745+0000","last_peered":"2024-04-25T08:38:19.943745+0000","last_clean":"2024-04-25T08:38:19.943745+0000","last_became_active":"2024-04-25T08:38:17.874196+0000","last_became_peered":"2024-04-25T08:38:17.874196+0000","last_unstale":"2024-04-25T08:38:19.943745+0000","last_undegraded":"2024-04-25T08:38:19.943745+0000","last_fullsized":"2024-04-25T08:38:19.943745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875655+0000","last_change":"2024-04-25T08:38:17.894302+0000","last_active":"2024-04-25T08:38:18.875655+0000","last_peered":"2024-04-25T08:38:18.875655+0000","last_clean":"2024-04-25T08:38:18.875655+0000","last_became_active":"2024-04-25T08:38:17.893594+0000","last_became_peered":"2024-04-25T08:38:17.893594+0000","last_unstale":"2024-04-25T08:38:18.875655+0000","last_undegraded":"2024-04-25T08:38:18.875655+0000","last_fullsized":"2024-04-25T08:38:18.875655+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:58.149171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879387+0000","last_change":"2024-04-25T08:38:17.885426+0000","last_active":"2024-04-25T08:38:19.879387+0000","last_peered":"2024-04-25T08:38:19.879387+0000","last_clean":"2024-04-25T08:38:19.879387+0000","last_became_active":"2024-04-25T08:38:17.885142+0000","last_became_peered":"2024-04-25T08:38:17.885142+0000","last_unstale":"2024-04-25T08:38:19.879387+0000","last_undegraded":"2024-04-25T08:38:19.879387+0000","last_fullsized":"2024-04-25T08:38:19.879387+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:17.119223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944491+0000","last_change":"2024-04-25T08:38:17.881404+0000","last_active":"2024-04-25T08:38:19.944491+0000","last_peered":"2024-04-25T08:38:19.944491+0000","last_clean":"2024-04-25T08:38:19.944491+0000","last_became_active":"2024-04-25T08:38:17.881129+0000","last_became_peered":"2024-04-25T08:38:17.881129+0000","last_unstale":"2024-04-25T08:38:19.944491+0000","last_undegraded":"2024-04-25T08:38:19.944491+0000","last_fullsized":"2024-04-25T08:38:19.944491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.466242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943737+0000","last_change":"2024-04-25T08:38:17.886835+0000","last_active":"2024-04-25T08:38:19.943737+0000","last_peered":"2024-04-25T08:38:19.943737+0000","last_clean":"2024-04-25T08:38:19.943737+0000","last_became_active":"2024-04-25T08:38:17.886530+0000","last_became_peered":"2024-04-25T08:38:17.886530+0000","last_unstale":"2024-04-25T08:38:19.943737+0000","last_undegraded":"2024-04-25T08:38:19.943737+0000","last_fullsized":"2024-04-25T08:38:19.943737+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:02.753734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881989+0000","last_change":"2024-04-25T08:38:17.887506+0000","last_active":"2024-04-25T08:38:19.881989+0000","last_peered":"2024-04-25T08:38:19.881989+0000","last_clean":"2024-04-25T08:38:19.881989+0000","last_became_active":"2024-04-25T08:38:17.884413+0000","last_became_peered":"2024-04-25T08:38:17.884413+0000","last_unstale":"2024-04-25T08:38:19.881989+0000","last_undegraded":"2024-04-25T08:38:19.881989+0000","last_fullsized":"2024-04-25T08:38:19.881989+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:55.340733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944346+0000","last_change":"2024-04-25T08:38:17.882319+0000","last_active":"2024-04-25T08:38:19.944346+0000","last_peered":"2024-04-25T08:38:19.944346+0000","last_clean":"2024-04-25T08:38:19.944346+0000","last_became_active":"2024-04-25T08:38:17.881864+0000","last_became_peered":"2024-04-25T08:38:17.881864+0000","last_unstale":"2024-04-25T08:38:19.944346+0000","last_undegraded":"2024-04-25T08:38:19.944346+0000","last_fullsized":"2024-04-25T08:38:19.944346+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:35.454814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944626+0000","last_change":"2024-04-25T08:38:17.875563+0000","last_active":"2024-04-25T08:38:19.944626+0000","last_peered":"2024-04-25T08:38:19.944626+0000","last_clean":"2024-04-25T08:38:19.944626+0000","last_became_active":"2024-04-25T08:38:17.875187+0000","last_became_peered":"2024-04-25T08:38:17.875187+0000","last_unstale":"2024-04-25T08:38:19.944626+0000","last_undegraded":"2024-04-25T08:38:19.944626+0000","last_fullsized":"2024-04-25T08:38:19.944626+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:31.918313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944527+0000","last_change":"2024-04-25T08:38:17.880229+0000","last_active":"2024-04-25T08:38:19.944527+0000","last_peered":"2024-04-25T08:38:19.944527+0000","last_clean":"2024-04-25T08:38:19.944527+0000","last_became_active":"2024-04-25T08:38:17.879911+0000","last_became_peered":"2024-04-25T08:38:17.879911+0000","last_unstale":"2024-04-25T08:38:19.944527+0000","last_undegraded":"2024-04-25T08:38:19.944527+0000","last_fullsized":"2024-04-25T08:38:19.944527+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.772260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943620+0000","last_change":"2024-04-25T08:38:17.881918+0000","last_active":"2024-04-25T08:38:19.943620+0000","last_peered":"2024-04-25T08:38:19.943620+0000","last_clean":"2024-04-25T08:38:19.943620+0000","last_became_active":"2024-04-25T08:38:17.881591+0000","last_became_peered":"2024-04-25T08:38:17.881591+0000","last_unstale":"2024-04-25T08:38:19.943620+0000","last_undegraded":"2024-04-25T08:38:19.943620+0000","last_fullsized":"2024-04-25T08:38:19.943620+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:41.225407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943933+0000","last_change":"2024-04-25T08:38:17.882032+0000","last_active":"2024-04-25T08:38:19.943933+0000","last_peered":"2024-04-25T08:38:19.943933+0000","last_clean":"2024-04-25T08:38:19.943933+0000","last_became_active":"2024-04-25T08:38:17.881535+0000","last_became_peered":"2024-04-25T08:38:17.881535+0000","last_unstale":"2024-04-25T08:38:19.943933+0000","last_undegraded":"2024-04-25T08:38:19.943933+0000","last_fullsized":"2024-04-25T08:38:19.943933+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944573+0000","last_change":"2024-04-25T08:38:17.882058+0000","last_active":"2024-04-25T08:38:19.944573+0000","last_peered":"2024-04-25T08:38:19.944573+0000","last_clean":"2024-04-25T08:38:19.944573+0000","last_became_active":"2024-04-25T08:38:17.881728+0000","last_became_peered":"2024-04-25T08:38:17.881728+0000","last_unstale":"2024-04-25T08:38:19.944573+0000","last_undegraded":"2024-04-25T08:38:19.944573+0000","last_fullsized":"2024-04-25T08:38:19.944573+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:13.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874594+0000","last_change":"2024-04-25T08:38:17.882056+0000","last_active":"2024-04-25T08:38:18.874594+0000","last_peered":"2024-04-25T08:38:18.874594+0000","last_clean":"2024-04-25T08:38:18.874594+0000","last_became_active":"2024-04-25T08:38:17.881737+0000","last_became_peered":"2024-04-25T08:38:17.881737+0000","last_unstale":"2024-04-25T08:38:18.874594+0000","last_undegraded":"2024-04-25T08:38:18.874594+0000","last_fullsized":"2024-04-25T08:38:18.874594+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877346+0000","last_change":"2024-04-25T08:38:17.896592+0000","last_active":"2024-04-25T08:38:18.877346+0000","last_peered":"2024-04-25T08:38:18.877346+0000","last_clean":"2024-04-25T08:38:18.877346+0000","last_became_active":"2024-04-25T08:38:17.894196+0000","last_became_peered":"2024-04-25T08:38:17.894196+0000","last_unstale":"2024-04-25T08:38:18.877346+0000","last_undegraded":"2024-04-25T08:38:18.877346+0000","last_fullsized":"2024-04-25T08:38:18.877346+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:34.093419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874582+0000","last_change":"2024-04-25T08:38:17.894097+0000","last_active":"2024-04-25T08:38:18.874582+0000","last_peered":"2024-04-25T08:38:18.874582+0000","last_clean":"2024-04-25T08:38:18.874582+0000","last_became_active":"2024-04-25T08:38:17.893173+0000","last_became_peered":"2024-04-25T08:38:17.893173+0000","last_unstale":"2024-04-25T08:38:18.874582+0000","last_undegraded":"2024-04-25T08:38:18.874582+0000","last_fullsized":"2024-04-25T08:38:18.874582+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881956+0000","last_change":"2024-04-25T08:38:17.879586+0000","last_active":"2024-04-25T08:38:19.881956+0000","last_peered":"2024-04-25T08:38:19.881956+0000","last_clean":"2024-04-25T08:38:19.881956+0000","last_became_active":"2024-04-25T08:38:17.879289+0000","last_became_peered":"2024-04-25T08:38:17.879289+0000","last_unstale":"2024-04-25T08:38:19.881956+0000","last_undegraded":"2024-04-25T08:38:19.881956+0000","last_fullsized":"2024-04-25T08:38:19.881956+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.675286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944386+0000","last_change":"2024-04-25T08:38:17.880984+0000","last_active":"2024-04-25T08:38:19.944386+0000","last_peered":"2024-04-25T08:38:19.944386+0000","last_clean":"2024-04-25T08:38:19.944386+0000","last_became_active":"2024-04-25T08:38:17.880594+0000","last_became_peered":"2024-04-25T08:38:17.880594+0000","last_unstale":"2024-04-25T08:38:19.944386+0000","last_undegraded":"2024-04-25T08:38:19.944386+0000","last_fullsized":"2024-04-25T08:38:19.944386+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:06.858902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882372+0000","last_change":"2024-04-25T08:38:17.881329+0000","last_active":"2024-04-25T08:38:19.882372+0000","last_peered":"2024-04-25T08:38:19.882372+0000","last_clean":"2024-04-25T08:38:19.882372+0000","last_became_active":"2024-04-25T08:38:17.881014+0000","last_became_peered":"2024-04-25T08:38:17.881014+0000","last_unstale":"2024-04-25T08:38:19.882372+0000","last_undegraded":"2024-04-25T08:38:19.882372+0000","last_fullsized":"2024-04-25T08:38:19.882372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944094+0000","last_change":"2024-04-25T08:38:17.876698+0000","last_active":"2024-04-25T08:38:19.944094+0000","last_peered":"2024-04-25T08:38:19.944094+0000","last_clean":"2024-04-25T08:38:19.944094+0000","last_became_active":"2024-04-25T08:38:17.876364+0000","last_became_peered":"2024-04-25T08:38:17.876364+0000","last_unstale":"2024-04-25T08:38:19.944094+0000","last_undegraded":"2024-04-25T08:38:19.944094+0000","last_fullsized":"2024-04-25T08:38:19.944094+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:31.656833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876611+0000","last_change":"2024-04-25T08:38:17.871784+0000","last_active":"2024-04-25T08:38:18.876611+0000","last_peered":"2024-04-25T08:38:18.876611+0000","last_clean":"2024-04-25T08:38:18.876611+0000","last_became_active":"2024-04-25T08:38:17.870733+0000","last_became_peered":"2024-04-25T08:38:17.870733+0000","last_unstale":"2024-04-25T08:38:18.876611+0000","last_undegraded":"2024-04-25T08:38:18.876611+0000","last_fullsized":"2024-04-25T08:38:18.876611+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876372+0000","last_change":"2024-04-25T08:38:17.887048+0000","last_active":"2024-04-25T08:38:18.876372+0000","last_peered":"2024-04-25T08:38:18.876372+0000","last_clean":"2024-04-25T08:38:18.876372+0000","last_became_active":"2024-04-25T08:38:17.886769+0000","last_became_peered":"2024-04-25T08:38:17.886769+0000","last_unstale":"2024-04-25T08:38:18.876372+0000","last_undegraded":"2024-04-25T08:38:18.876372+0000","last_fullsized":"2024-04-25T08:38:18.876372+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:06.919441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877229+0000","last_change":"2024-04-25T08:38:17.898751+0000","last_active":"2024-04-25T08:38:18.877229+0000","last_peered":"2024-04-25T08:38:18.877229+0000","last_clean":"2024-04-25T08:38:18.877229+0000","last_became_active":"2024-04-25T08:38:17.898466+0000","last_became_peered":"2024-04-25T08:38:17.898466+0000","last_unstale":"2024-04-25T08:38:18.877229+0000","last_undegraded":"2024-04-25T08:38:18.877229+0000","last_fullsized":"2024-04-25T08:38:18.877229+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:05.202461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943893+0000","last_change":"2024-04-25T08:38:17.884795+0000","last_active":"2024-04-25T08:38:19.943893+0000","last_peered":"2024-04-25T08:38:19.943893+0000","last_clean":"2024-04-25T08:38:19.943893+0000","last_became_active":"2024-04-25T08:38:17.884380+0000","last_became_peered":"2024-04-25T08:38:17.884380+0000","last_unstale":"2024-04-25T08:38:19.943893+0000","last_undegraded":"2024-04-25T08:38:19.943893+0000","last_fullsized":"2024-04-25T08:38:19.943893+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:44.628191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876361+0000","last_change":"2024-04-25T08:38:17.872093+0000","last_active":"2024-04-25T08:38:18.876361+0000","last_peered":"2024-04-25T08:38:18.876361+0000","last_clean":"2024-04-25T08:38:18.876361+0000","last_became_active":"2024-04-25T08:38:17.871801+0000","last_became_peered":"2024-04-25T08:38:17.871801+0000","last_unstale":"2024-04-25T08:38:18.876361+0000","last_undegraded":"2024-04-25T08:38:18.876361+0000","last_fullsized":"2024-04-25T08:38:18.876361+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:13.057228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876355+0000","last_change":"2024-04-25T08:38:17.871321+0000","last_active":"2024-04-25T08:38:18.876355+0000","last_peered":"2024-04-25T08:38:18.876355+0000","last_clean":"2024-04-25T08:38:18.876355+0000","last_became_active":"2024-04-25T08:38:17.870448+0000","last_became_peered":"2024-04-25T08:38:17.870448+0000","last_unstale":"2024-04-25T08:38:18.876355+0000","last_undegraded":"2024-04-25T08:38:18.876355+0000","last_fullsized":"2024-04-25T08:38:18.876355+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:55.340733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879425+0000","last_change":"2024-04-25T08:38:17.874726+0000","last_active":"2024-04-25T08:38:19.879425+0000","last_peered":"2024-04-25T08:38:19.879425+0000","last_clean":"2024-04-25T08:38:19.879425+0000","last_became_active":"2024-04-25T08:38:17.874458+0000","last_became_peered":"2024-04-25T08:38:17.874458+0000","last_unstale":"2024-04-25T08:38:19.879425+0000","last_undegraded":"2024-04-25T08:38:19.879425+0000","last_fullsized":"2024-04-25T08:38:19.879425+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:47.779597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943170+0000","last_change":"2024-04-25T08:38:17.884450+0000","last_active":"2024-04-25T08:38:19.943170+0000","last_peered":"2024-04-25T08:38:19.943170+0000","last_clean":"2024-04-25T08:38:19.943170+0000","last_became_active":"2024-04-25T08:38:17.884060+0000","last_became_peered":"2024-04-25T08:38:17.884060+0000","last_unstale":"2024-04-25T08:38:19.943170+0000","last_undegraded":"2024-04-25T08:38:19.943170+0000","last_fullsized":"2024-04-25T08:38:19.943170+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:21.455456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879358+0000","last_change":"2024-04-25T08:38:17.885953+0000","last_active":"2024-04-25T08:38:19.879358+0000","last_peered":"2024-04-25T08:38:19.879358+0000","last_clean":"2024-04-25T08:38:19.879358+0000","last_became_active":"2024-04-25T08:38:17.885687+0000","last_became_peered":"2024-04-25T08:38:17.885687+0000","last_unstale":"2024-04-25T08:38:19.879358+0000","last_undegraded":"2024-04-25T08:38:19.879358+0000","last_fullsized":"2024-04-25T08:38:19.879358+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.490629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"133'4","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876440+0000","last_change":"2024-04-25T08:38:16.875570+0000","last_active":"2024-04-25T08:38:18.876440+0000","last_peered":"2024-04-25T08:38:18.876440+0000","last_clean":"2024-04-25T08:38:18.876440+0000","last_became_active":"2024-04-25T08:38:16.875258+0000","last_became_peered":"2024-04-25T08:38:16.875258+0000","last_unstale":"2024-04-25T08:38:18.876440+0000","last_undegraded":"2024-04-25T08:38:18.876440+0000","last_fullsized":"2024-04-25T08:38:18.876440+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:45.186789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943861+0000","last_change":"2024-04-25T08:38:17.877731+0000","last_active":"2024-04-25T08:38:19.943861+0000","last_peered":"2024-04-25T08:38:19.943861+0000","last_clean":"2024-04-25T08:38:19.943861+0000","last_became_active":"2024-04-25T08:38:17.877437+0000","last_became_peered":"2024-04-25T08:38:17.877437+0000","last_unstale":"2024-04-25T08:38:19.943861+0000","last_undegraded":"2024-04-25T08:38:19.943861+0000","last_fullsized":"2024-04-25T08:38:19.943861+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:27.127276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874830+0000","last_change":"2024-04-25T08:38:16.869588+0000","last_active":"2024-04-25T08:38:18.874830+0000","last_peered":"2024-04-25T08:38:18.874830+0000","last_clean":"2024-04-25T08:38:18.874830+0000","last_became_active":"2024-04-25T08:38:16.868936+0000","last_became_peered":"2024-04-25T08:38:16.868936+0000","last_unstale":"2024-04-25T08:38:18.874830+0000","last_undegraded":"2024-04-25T08:38:18.874830+0000","last_fullsized":"2024-04-25T08:38:18.874830+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:29.799312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882046+0000","last_change":"2024-04-25T08:38:17.880219+0000","last_active":"2024-04-25T08:38:19.882046+0000","last_peered":"2024-04-25T08:38:19.882046+0000","last_clean":"2024-04-25T08:38:19.882046+0000","last_became_active":"2024-04-25T08:38:17.879938+0000","last_became_peered":"2024-04-25T08:38:17.879938+0000","last_unstale":"2024-04-25T08:38:19.882046+0000","last_undegraded":"2024-04-25T08:38:19.882046+0000","last_fullsized":"2024-04-25T08:38:19.882046+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:06.919441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877294+0000","last_change":"2024-04-25T08:38:16.877474+0000","last_active":"2024-04-25T08:38:18.877294+0000","last_peered":"2024-04-25T08:38:18.877294+0000","last_clean":"2024-04-25T08:38:18.877294+0000","last_became_active":"2024-04-25T08:38:16.877245+0000","last_became_peered":"2024-04-25T08:38:16.877245+0000","last_unstale":"2024-04-25T08:38:18.877294+0000","last_undegraded":"2024-04-25T08:38:18.877294+0000","last_fullsized":"2024-04-25T08:38:18.877294+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:54.336690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943370+0000","last_change":"2024-04-25T08:38:17.886473+0000","last_active":"2024-04-25T08:38:19.943370+0000","last_peered":"2024-04-25T08:38:19.943370+0000","last_clean":"2024-04-25T08:38:19.943370+0000","last_became_active":"2024-04-25T08:38:17.885797+0000","last_became_peered":"2024-04-25T08:38:17.885797+0000","last_unstale":"2024-04-25T08:38:19.943370+0000","last_undegraded":"2024-04-25T08:38:19.943370+0000","last_fullsized":"2024-04-25T08:38:19.943370+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:16.987654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877062+0000","last_change":"2024-04-25T08:38:16.868822+0000","last_active":"2024-04-25T08:38:18.877062+0000","last_peered":"2024-04-25T08:38:18.877062+0000","last_clean":"2024-04-25T08:38:18.877062+0000","last_became_active":"2024-04-25T08:38:16.868564+0000","last_became_peered":"2024-04-25T08:38:16.868564+0000","last_unstale":"2024-04-25T08:38:18.877062+0000","last_undegraded":"2024-04-25T08:38:18.877062+0000","last_fullsized":"2024-04-25T08:38:18.877062+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:27.367777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879613+0000","last_change":"2024-04-25T08:38:17.872175+0000","last_active":"2024-04-25T08:38:19.879613+0000","last_peered":"2024-04-25T08:38:19.879613+0000","last_clean":"2024-04-25T08:38:19.879613+0000","last_became_active":"2024-04-25T08:38:17.871909+0000","last_became_peered":"2024-04-25T08:38:17.871909+0000","last_unstale":"2024-04-25T08:38:19.879613+0000","last_undegraded":"2024-04-25T08:38:19.879613+0000","last_fullsized":"2024-04-25T08:38:19.879613+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:32.701570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879631+0000","last_change":"2024-04-25T08:38:16.869047+0000","last_active":"2024-04-25T08:38:19.879631+0000","last_peered":"2024-04-25T08:38:19.879631+0000","last_clean":"2024-04-25T08:38:19.879631+0000","last_became_active":"2024-04-25T08:38:16.868570+0000","last_became_peered":"2024-04-25T08:38:16.868570+0000","last_unstale":"2024-04-25T08:38:19.879631+0000","last_undegraded":"2024-04-25T08:38:19.879631+0000","last_fullsized":"2024-04-25T08:38:19.879631+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:39.591232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879783+0000","last_change":"2024-04-25T08:38:17.887891+0000","last_active":"2024-04-25T08:38:19.879783+0000","last_peered":"2024-04-25T08:38:19.879783+0000","last_clean":"2024-04-25T08:38:19.879783+0000","last_became_active":"2024-04-25T08:38:17.887612+0000","last_became_peered":"2024-04-25T08:38:17.887612+0000","last_unstale":"2024-04-25T08:38:19.879783+0000","last_undegraded":"2024-04-25T08:38:19.879783+0000","last_fullsized":"2024-04-25T08:38:19.879783+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:30.803355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874718+0000","last_change":"2024-04-25T08:38:16.868606+0000","last_active":"2024-04-25T08:38:18.874718+0000","last_peered":"2024-04-25T08:38:18.874718+0000","last_clean":"2024-04-25T08:38:18.874718+0000","last_became_active":"2024-04-25T08:38:16.868290+0000","last_became_peered":"2024-04-25T08:38:16.868290+0000","last_unstale":"2024-04-25T08:38:18.874718+0000","last_undegraded":"2024-04-25T08:38:18.874718+0000","last_fullsized":"2024-04-25T08:38:18.874718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.697527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874750+0000","last_change":"2024-04-25T08:38:17.887163+0000","last_active":"2024-04-25T08:38:18.874750+0000","last_peered":"2024-04-25T08:38:18.874750+0000","last_clean":"2024-04-25T08:38:18.874750+0000","last_became_active":"2024-04-25T08:38:17.886853+0000","last_became_peered":"2024-04-25T08:38:17.886853+0000","last_unstale":"2024-04-25T08:38:18.874750+0000","last_undegraded":"2024-04-25T08:38:18.874750+0000","last_fullsized":"2024-04-25T08:38:18.874750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:31.656833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944174+0000","last_change":"2024-04-25T08:38:16.879982+0000","last_active":"2024-04-25T08:38:19.944174+0000","last_peered":"2024-04-25T08:38:19.944174+0000","last_clean":"2024-04-25T08:38:19.944174+0000","last_became_active":"2024-04-25T08:38:16.879692+0000","last_became_peered":"2024-04-25T08:38:16.879692+0000","last_unstale":"2024-04-25T08:38:19.944174+0000","last_undegraded":"2024-04-25T08:38:19.944174+0000","last_fullsized":"2024-04-25T08:38:19.944174+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:36.742158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877301+0000","last_change":"2024-04-25T08:38:17.895788+0000","last_active":"2024-04-25T08:38:18.877301+0000","last_peered":"2024-04-25T08:38:18.877301+0000","last_clean":"2024-04-25T08:38:18.877301+0000","last_became_active":"2024-04-25T08:38:17.895479+0000","last_became_peered":"2024-04-25T08:38:17.895479+0000","last_unstale":"2024-04-25T08:38:18.877301+0000","last_undegraded":"2024-04-25T08:38:18.877301+0000","last_fullsized":"2024-04-25T08:38:18.877301+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:04.128752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876368+0000","last_change":"2024-04-25T08:38:16.865526+0000","last_active":"2024-04-25T08:38:18.876368+0000","last_peered":"2024-04-25T08:38:18.876368+0000","last_clean":"2024-04-25T08:38:18.876368+0000","last_became_active":"2024-04-25T08:38:16.865135+0000","last_became_peered":"2024-04-25T08:38:16.865135+0000","last_unstale":"2024-04-25T08:38:18.876368+0000","last_undegraded":"2024-04-25T08:38:18.876368+0000","last_fullsized":"2024-04-25T08:38:18.876368+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.197963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943446+0000","last_change":"2024-04-25T08:38:17.887207+0000","last_active":"2024-04-25T08:38:19.943446+0000","last_peered":"2024-04-25T08:38:19.943446+0000","last_clean":"2024-04-25T08:38:19.943446+0000","last_became_active":"2024-04-25T08:38:17.886920+0000","last_became_peered":"2024-04-25T08:38:17.886920+0000","last_unstale":"2024-04-25T08:38:19.943446+0000","last_undegraded":"2024-04-25T08:38:19.943446+0000","last_fullsized":"2024-04-25T08:38:19.943446+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:45.695945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882289+0000","last_change":"2024-04-25T08:38:16.870861+0000","last_active":"2024-04-25T08:38:19.882289+0000","last_peered":"2024-04-25T08:38:19.882289+0000","last_clean":"2024-04-25T08:38:19.882289+0000","last_became_active":"2024-04-25T08:38:16.870598+0000","last_became_peered":"2024-04-25T08:38:16.870598+0000","last_unstale":"2024-04-25T08:38:19.882289+0000","last_undegraded":"2024-04-25T08:38:19.882289+0000","last_fullsized":"2024-04-25T08:38:19.882289+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:21.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944319+0000","last_change":"2024-04-25T08:38:17.890425+0000","last_active":"2024-04-25T08:38:19.944319+0000","last_peered":"2024-04-25T08:38:19.944319+0000","last_clean":"2024-04-25T08:38:19.944319+0000","last_became_active":"2024-04-25T08:38:17.890108+0000","last_became_peered":"2024-04-25T08:38:17.890108+0000","last_unstale":"2024-04-25T08:38:19.944319+0000","last_undegraded":"2024-04-25T08:38:19.944319+0000","last_fullsized":"2024-04-25T08:38:19.944319+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:26.696133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944170+0000","last_change":"2024-04-25T08:38:16.868916+0000","last_active":"2024-04-25T08:38:19.944170+0000","last_peered":"2024-04-25T08:38:19.944170+0000","last_clean":"2024-04-25T08:38:19.944170+0000","last_became_active":"2024-04-25T08:38:16.868648+0000","last_became_peered":"2024-04-25T08:38:16.868648+0000","last_unstale":"2024-04-25T08:38:19.944170+0000","last_undegraded":"2024-04-25T08:38:19.944170+0000","last_fullsized":"2024-04-25T08:38:19.944170+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:41.331503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876563+0000","last_change":"2024-04-25T08:38:17.877450+0000","last_active":"2024-04-25T08:38:18.876563+0000","last_peered":"2024-04-25T08:38:18.876563+0000","last_clean":"2024-04-25T08:38:18.876563+0000","last_became_active":"2024-04-25T08:38:17.877096+0000","last_became_peered":"2024-04-25T08:38:17.877096+0000","last_unstale":"2024-04-25T08:38:18.876563+0000","last_undegraded":"2024-04-25T08:38:18.876563+0000","last_fullsized":"2024-04-25T08:38:18.876563+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"133'3","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877437+0000","last_change":"2024-04-25T08:38:16.872592+0000","last_active":"2024-04-25T08:38:18.877437+0000","last_peered":"2024-04-25T08:38:18.877437+0000","last_clean":"2024-04-25T08:38:18.877437+0000","last_became_active":"2024-04-25T08:38:16.872012+0000","last_became_peered":"2024-04-25T08:38:16.872012+0000","last_unstale":"2024-04-25T08:38:18.877437+0000","last_undegraded":"2024-04-25T08:38:18.877437+0000","last_fullsized":"2024-04-25T08:38:18.877437+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:12.053186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876280+0000","last_change":"2024-04-25T08:38:17.889252+0000","last_active":"2024-04-25T08:38:18.876280+0000","last_peered":"2024-04-25T08:38:18.876280+0000","last_clean":"2024-04-25T08:38:18.876280+0000","last_became_active":"2024-04-25T08:38:17.888591+0000","last_became_peered":"2024-04-25T08:38:17.888591+0000","last_unstale":"2024-04-25T08:38:18.876280+0000","last_undegraded":"2024-04-25T08:38:18.876280+0000","last_fullsized":"2024-04-25T08:38:18.876280+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:41.675286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"133'6","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877066+0000","last_change":"2024-04-25T08:38:16.871721+0000","last_active":"2024-04-25T08:38:18.877066+0000","last_peered":"2024-04-25T08:38:18.877066+0000","last_clean":"2024-04-25T08:38:18.877066+0000","last_became_active":"2024-04-25T08:38:16.871135+0000","last_became_peered":"2024-04-25T08:38:16.871135+0000","last_unstale":"2024-04-25T08:38:18.877066+0000","last_undegraded":"2024-04-25T08:38:18.877066+0000","last_fullsized":"2024-04-25T08:38:18.877066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:05.915398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944043+0000","last_change":"2024-04-25T08:38:17.884680+0000","last_active":"2024-04-25T08:38:19.944043+0000","last_peered":"2024-04-25T08:38:19.944043+0000","last_clean":"2024-04-25T08:38:19.944043+0000","last_became_active":"2024-04-25T08:38:17.884023+0000","last_became_peered":"2024-04-25T08:38:17.884023+0000","last_unstale":"2024-04-25T08:38:19.944043+0000","last_undegraded":"2024-04-25T08:38:19.944043+0000","last_fullsized":"2024-04-25T08:38:19.944043+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:30.754146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943481+0000","last_change":"2024-04-25T08:38:16.873738+0000","last_active":"2024-04-25T08:38:19.943481+0000","last_peered":"2024-04-25T08:38:19.943481+0000","last_clean":"2024-04-25T08:38:19.943481+0000","last_became_active":"2024-04-25T08:38:16.873338+0000","last_became_peered":"2024-04-25T08:38:16.873338+0000","last_unstale":"2024-04-25T08:38:19.943481+0000","last_undegraded":"2024-04-25T08:38:19.943481+0000","last_fullsized":"2024-04-25T08:38:19.943481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:11.883106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877409+0000","last_change":"2024-04-25T08:38:17.895519+0000","last_active":"2024-04-25T08:38:18.877409+0000","last_peered":"2024-04-25T08:38:18.877409+0000","last_clean":"2024-04-25T08:38:18.877409+0000","last_became_active":"2024-04-25T08:38:17.895149+0000","last_became_peered":"2024-04-25T08:38:17.895149+0000","last_unstale":"2024-04-25T08:38:18.877409+0000","last_undegraded":"2024-04-25T08:38:18.877409+0000","last_fullsized":"2024-04-25T08:38:18.877409+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.397082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875906+0000","last_change":"2024-04-25T08:38:16.868510+0000","last_active":"2024-04-25T08:38:18.875906+0000","last_peered":"2024-04-25T08:38:18.875906+0000","last_clean":"2024-04-25T08:38:18.875906+0000","last_became_active":"2024-04-25T08:38:16.868241+0000","last_became_peered":"2024-04-25T08:38:16.868241+0000","last_unstale":"2024-04-25T08:38:18.875906+0000","last_undegraded":"2024-04-25T08:38:18.875906+0000","last_fullsized":"2024-04-25T08:38:18.875906+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:55.755478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874750+0000","last_change":"2024-04-25T08:38:17.884410+0000","last_active":"2024-04-25T08:38:18.874750+0000","last_peered":"2024-04-25T08:38:18.874750+0000","last_clean":"2024-04-25T08:38:18.874750+0000","last_became_active":"2024-04-25T08:38:17.884122+0000","last_became_peered":"2024-04-25T08:38:17.884122+0000","last_unstale":"2024-04-25T08:38:18.874750+0000","last_undegraded":"2024-04-25T08:38:18.874750+0000","last_fullsized":"2024-04-25T08:38:18.874750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:26.696133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"133'8","reported_seq":22,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944197+0000","last_change":"2024-04-25T08:38:16.876363+0000","last_active":"2024-04-25T08:38:19.944197+0000","last_peered":"2024-04-25T08:38:19.944197+0000","last_clean":"2024-04-25T08:38:19.944197+0000","last_became_active":"2024-04-25T08:38:16.876049+0000","last_became_peered":"2024-04-25T08:38:16.876049+0000","last_unstale":"2024-04-25T08:38:19.944197+0000","last_undegraded":"2024-04-25T08:38:19.944197+0000","last_fullsized":"2024-04-25T08:38:19.944197+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:21.826411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876466+0000","last_change":"2024-04-25T08:38:17.881462+0000","last_active":"2024-04-25T08:38:18.876466+0000","last_peered":"2024-04-25T08:38:18.876466+0000","last_clean":"2024-04-25T08:38:18.876466+0000","last_became_active":"2024-04-25T08:38:17.880097+0000","last_became_peered":"2024-04-25T08:38:17.880097+0000","last_unstale":"2024-04-25T08:38:18.876466+0000","last_undegraded":"2024-04-25T08:38:18.876466+0000","last_fullsized":"2024-04-25T08:38:18.876466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:28.371820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882404+0000","last_change":"2024-04-25T08:38:16.866476+0000","last_active":"2024-04-25T08:38:19.882404+0000","last_peered":"2024-04-25T08:38:19.882404+0000","last_clean":"2024-04-25T08:38:19.882404+0000","last_became_active":"2024-04-25T08:38:16.865868+0000","last_became_peered":"2024-04-25T08:38:16.865868+0000","last_unstale":"2024-04-25T08:38:19.882404+0000","last_undegraded":"2024-04-25T08:38:19.882404+0000","last_fullsized":"2024-04-25T08:38:19.882404+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:46.775554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875571+0000","last_change":"2024-04-25T08:38:17.896892+0000","last_active":"2024-04-25T08:38:18.875571+0000","last_peered":"2024-04-25T08:38:18.875571+0000","last_clean":"2024-04-25T08:38:18.875571+0000","last_became_active":"2024-04-25T08:38:17.895902+0000","last_became_peered":"2024-04-25T08:38:17.895902+0000","last_unstale":"2024-04-25T08:38:18.875571+0000","last_undegraded":"2024-04-25T08:38:18.875571+0000","last_fullsized":"2024-04-25T08:38:18.875571+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:34.550562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876229+0000","last_change":"2024-04-25T08:38:16.868952+0000","last_active":"2024-04-25T08:38:18.876229+0000","last_peered":"2024-04-25T08:38:18.876229+0000","last_clean":"2024-04-25T08:38:18.876229+0000","last_became_active":"2024-04-25T08:38:16.868689+0000","last_became_peered":"2024-04-25T08:38:16.868689+0000","last_unstale":"2024-04-25T08:38:18.876229+0000","last_undegraded":"2024-04-25T08:38:18.876229+0000","last_fullsized":"2024-04-25T08:38:18.876229+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:20.443300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875867+0000","last_change":"2024-04-25T08:38:17.897649+0000","last_active":"2024-04-25T08:38:18.875867+0000","last_peered":"2024-04-25T08:38:18.875867+0000","last_clean":"2024-04-25T08:38:18.875867+0000","last_became_active":"2024-04-25T08:38:17.896745+0000","last_became_peered":"2024-04-25T08:38:17.896745+0000","last_unstale":"2024-04-25T08:38:18.875867+0000","last_undegraded":"2024-04-25T08:38:18.875867+0000","last_fullsized":"2024-04-25T08:38:18.875867+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:03.164735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"133'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944209+0000","last_change":"2024-04-25T08:38:16.879316+0000","last_active":"2024-04-25T08:38:19.944209+0000","last_peered":"2024-04-25T08:38:19.944209+0000","last_clean":"2024-04-25T08:38:19.944209+0000","last_became_active":"2024-04-25T08:38:16.879048+0000","last_became_peered":"2024-04-25T08:38:16.879048+0000","last_unstale":"2024-04-25T08:38:19.944209+0000","last_undegraded":"2024-04-25T08:38:19.944209+0000","last_fullsized":"2024-04-25T08:38:19.944209+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943504+0000","last_change":"2024-04-25T08:38:17.887226+0000","last_active":"2024-04-25T08:38:19.943504+0000","last_peered":"2024-04-25T08:38:19.943504+0000","last_clean":"2024-04-25T08:38:19.943504+0000","last_became_active":"2024-04-25T08:38:17.886924+0000","last_became_peered":"2024-04-25T08:38:17.886924+0000","last_unstale":"2024-04-25T08:38:19.943504+0000","last_undegraded":"2024-04-25T08:38:19.943504+0000","last_fullsized":"2024-04-25T08:38:19.943504+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:58.314369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874682+0000","last_change":"2024-04-25T08:38:16.878410+0000","last_active":"2024-04-25T08:38:18.874682+0000","last_peered":"2024-04-25T08:38:18.874682+0000","last_clean":"2024-04-25T08:38:18.874682+0000","last_became_active":"2024-04-25T08:38:16.878111+0000","last_became_peered":"2024-04-25T08:38:16.878111+0000","last_unstale":"2024-04-25T08:38:18.874682+0000","last_undegraded":"2024-04-25T08:38:18.874682+0000","last_fullsized":"2024-04-25T08:38:18.874682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:30.486586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874630+0000","last_change":"2024-04-25T08:38:17.881756+0000","last_active":"2024-04-25T08:38:18.874630+0000","last_peered":"2024-04-25T08:38:18.874630+0000","last_clean":"2024-04-25T08:38:18.874630+0000","last_became_active":"2024-04-25T08:38:17.881349+0000","last_became_peered":"2024-04-25T08:38:17.881349+0000","last_unstale":"2024-04-25T08:38:18.874630+0000","last_undegraded":"2024-04-25T08:38:18.874630+0000","last_fullsized":"2024-04-25T08:38:18.874630+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:07.729869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879784+0000","last_change":"2024-04-25T08:38:16.865428+0000","last_active":"2024-04-25T08:38:19.879784+0000","last_peered":"2024-04-25T08:38:19.879784+0000","last_clean":"2024-04-25T08:38:19.879784+0000","last_became_active":"2024-04-25T08:38:16.865137+0000","last_became_peered":"2024-04-25T08:38:16.865137+0000","last_unstale":"2024-04-25T08:38:19.879784+0000","last_undegraded":"2024-04-25T08:38:19.879784+0000","last_fullsized":"2024-04-25T08:38:19.879784+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:55.755478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944757+0000","last_change":"2024-04-25T08:38:17.889169+0000","last_active":"2024-04-25T08:38:19.944757+0000","last_peered":"2024-04-25T08:38:19.944757+0000","last_clean":"2024-04-25T08:38:19.944757+0000","last_became_active":"2024-04-25T08:38:17.888859+0000","last_became_peered":"2024-04-25T08:38:17.888859+0000","last_unstale":"2024-04-25T08:38:19.944757+0000","last_undegraded":"2024-04-25T08:38:19.944757+0000","last_fullsized":"2024-04-25T08:38:19.944757+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:21.979826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944552+0000","last_change":"2024-04-25T08:38:16.870881+0000","last_active":"2024-04-25T08:38:19.944552+0000","last_peered":"2024-04-25T08:38:19.944552+0000","last_clean":"2024-04-25T08:38:19.944552+0000","last_became_active":"2024-04-25T08:38:16.870440+0000","last_became_peered":"2024-04-25T08:38:16.870440+0000","last_unstale":"2024-04-25T08:38:19.944552+0000","last_undegraded":"2024-04-25T08:38:19.944552+0000","last_fullsized":"2024-04-25T08:38:19.944552+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:25.178803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874779+0000","last_change":"2024-04-25T08:38:17.885425+0000","last_active":"2024-04-25T08:38:18.874779+0000","last_peered":"2024-04-25T08:38:18.874779+0000","last_clean":"2024-04-25T08:38:18.874779+0000","last_became_active":"2024-04-25T08:38:17.885118+0000","last_became_peered":"2024-04-25T08:38:17.885118+0000","last_unstale":"2024-04-25T08:38:18.874779+0000","last_undegraded":"2024-04-25T08:38:18.874779+0000","last_fullsized":"2024-04-25T08:38:18.874779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:41.466242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875707+0000","last_change":"2024-04-25T08:38:16.869530+0000","last_active":"2024-04-25T08:38:18.875707+0000","last_peered":"2024-04-25T08:38:18.875707+0000","last_clean":"2024-04-25T08:38:18.875707+0000","last_became_active":"2024-04-25T08:38:16.869225+0000","last_became_peered":"2024-04-25T08:38:16.869225+0000","last_unstale":"2024-04-25T08:38:18.875707+0000","last_undegraded":"2024-04-25T08:38:18.875707+0000","last_fullsized":"2024-04-25T08:38:18.875707+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:21.300604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882377+0000","last_change":"2024-04-25T08:38:17.884940+0000","last_active":"2024-04-25T08:38:19.882377+0000","last_peered":"2024-04-25T08:38:19.882377+0000","last_clean":"2024-04-25T08:38:19.882377+0000","last_became_active":"2024-04-25T08:38:17.884659+0000","last_became_peered":"2024-04-25T08:38:17.884659+0000","last_unstale":"2024-04-25T08:38:19.882377+0000","last_undegraded":"2024-04-25T08:38:19.882377+0000","last_fullsized":"2024-04-25T08:38:19.882377+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944287+0000","last_change":"2024-04-25T08:38:16.877836+0000","last_active":"2024-04-25T08:38:19.944287+0000","last_peered":"2024-04-25T08:38:19.944287+0000","last_clean":"2024-04-25T08:38:19.944287+0000","last_became_active":"2024-04-25T08:38:16.876856+0000","last_became_peered":"2024-04-25T08:38:16.876856+0000","last_unstale":"2024-04-25T08:38:19.944287+0000","last_undegraded":"2024-04-25T08:38:19.944287+0000","last_fullsized":"2024-04-25T08:38:19.944287+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:46.775554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874465+0000","last_change":"2024-04-25T08:38:17.885787+0000","last_active":"2024-04-25T08:38:18.874465+0000","last_peered":"2024-04-25T08:38:18.874465+0000","last_clean":"2024-04-25T08:38:18.874465+0000","last_became_active":"2024-04-25T08:38:17.885447+0000","last_became_peered":"2024-04-25T08:38:17.885447+0000","last_unstale":"2024-04-25T08:38:18.874465+0000","last_undegraded":"2024-04-25T08:38:18.874465+0000","last_fullsized":"2024-04-25T08:38:18.874465+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:58.367990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"133'2","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944775+0000","last_change":"2024-04-25T08:38:16.874920+0000","last_active":"2024-04-25T08:38:19.944775+0000","last_peered":"2024-04-25T08:38:19.944775+0000","last_clean":"2024-04-25T08:38:19.944775+0000","last_became_active":"2024-04-25T08:38:16.874423+0000","last_became_peered":"2024-04-25T08:38:16.874423+0000","last_unstale":"2024-04-25T08:38:19.944775+0000","last_undegraded":"2024-04-25T08:38:19.944775+0000","last_fullsized":"2024-04-25T08:38:19.944775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:00.856047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876026+0000","last_change":"2024-04-25T08:38:17.888010+0000","last_active":"2024-04-25T08:38:18.876026+0000","last_peered":"2024-04-25T08:38:18.876026+0000","last_clean":"2024-04-25T08:38:18.876026+0000","last_became_active":"2024-04-25T08:38:17.887721+0000","last_became_peered":"2024-04-25T08:38:17.887721+0000","last_unstale":"2024-04-25T08:38:18.876026+0000","last_undegraded":"2024-04-25T08:38:18.876026+0000","last_fullsized":"2024-04-25T08:38:18.876026+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:26.152377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"134'1","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.881249+0000","last_change":"2024-04-25T08:38:16.863623+0000","last_active":"2024-04-25T08:38:19.881249+0000","last_peered":"2024-04-25T08:38:19.881249+0000","last_clean":"2024-04-25T08:38:19.881249+0000","last_became_active":"2024-04-25T08:38:16.863233+0000","last_became_peered":"2024-04-25T08:38:16.863233+0000","last_unstale":"2024-04-25T08:38:19.881249+0000","last_undegraded":"2024-04-25T08:38:19.881249+0000","last_fullsized":"2024-04-25T08:38:19.881249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:21.069051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875571+0000","last_change":"2024-04-25T08:38:17.892580+0000","last_active":"2024-04-25T08:38:18.875571+0000","last_peered":"2024-04-25T08:38:18.875571+0000","last_clean":"2024-04-25T08:38:18.875571+0000","last_became_active":"2024-04-25T08:38:17.889740+0000","last_became_peered":"2024-04-25T08:38:17.889740+0000","last_unstale":"2024-04-25T08:38:18.875571+0000","last_undegraded":"2024-04-25T08:38:18.875571+0000","last_fullsized":"2024-04-25T08:38:18.875571+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:42.269427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.879793+0000","last_change":"2024-04-25T08:38:16.870790+0000","last_active":"2024-04-25T08:38:19.879793+0000","last_peered":"2024-04-25T08:38:19.879793+0000","last_clean":"2024-04-25T08:38:19.879793+0000","last_became_active":"2024-04-25T08:38:16.870489+0000","last_became_peered":"2024-04-25T08:38:16.870489+0000","last_unstale":"2024-04-25T08:38:19.879793+0000","last_undegraded":"2024-04-25T08:38:19.879793+0000","last_fullsized":"2024-04-25T08:38:19.879793+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.197963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943648+0000","last_change":"2024-04-25T08:38:17.884365+0000","last_active":"2024-04-25T08:38:19.943648+0000","last_peered":"2024-04-25T08:38:19.943648+0000","last_clean":"2024-04-25T08:38:19.943648+0000","last_became_active":"2024-04-25T08:38:17.884041+0000","last_became_peered":"2024-04-25T08:38:17.884041+0000","last_unstale":"2024-04-25T08:38:19.943648+0000","last_undegraded":"2024-04-25T08:38:19.943648+0000","last_fullsized":"2024-04-25T08:38:19.943648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:42.402577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943793+0000","last_change":"2024-04-25T08:38:16.878761+0000","last_active":"2024-04-25T08:38:19.943793+0000","last_peered":"2024-04-25T08:38:19.943793+0000","last_clean":"2024-04-25T08:38:19.943793+0000","last_became_active":"2024-04-25T08:38:16.878118+0000","last_became_peered":"2024-04-25T08:38:16.878118+0000","last_unstale":"2024-04-25T08:38:19.943793+0000","last_undegraded":"2024-04-25T08:38:19.943793+0000","last_fullsized":"2024-04-25T08:38:19.943793+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:46.498729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943685+0000","last_change":"2024-04-25T08:38:17.877565+0000","last_active":"2024-04-25T08:38:19.943685+0000","last_peered":"2024-04-25T08:38:19.943685+0000","last_clean":"2024-04-25T08:38:19.943685+0000","last_became_active":"2024-04-25T08:38:17.876848+0000","last_became_peered":"2024-04-25T08:38:17.876848+0000","last_unstale":"2024-04-25T08:38:19.943685+0000","last_undegraded":"2024-04-25T08:38:19.943685+0000","last_fullsized":"2024-04-25T08:38:19.943685+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:22.304647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.882003+0000","last_change":"2024-04-25T08:38:16.875734+0000","last_active":"2024-04-25T08:38:19.882003+0000","last_peered":"2024-04-25T08:38:19.882003+0000","last_clean":"2024-04-25T08:38:19.882003+0000","last_became_active":"2024-04-25T08:38:16.875418+0000","last_became_peered":"2024-04-25T08:38:16.875418+0000","last_unstale":"2024-04-25T08:38:19.882003+0000","last_undegraded":"2024-04-25T08:38:19.882003+0000","last_fullsized":"2024-04-25T08:38:19.882003+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:46.498729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944097+0000","last_change":"2024-04-25T08:38:17.878723+0000","last_active":"2024-04-25T08:38:19.944097+0000","last_peered":"2024-04-25T08:38:19.944097+0000","last_clean":"2024-04-25T08:38:19.944097+0000","last_became_active":"2024-04-25T08:38:17.878200+0000","last_became_peered":"2024-04-25T08:38:17.878200+0000","last_unstale":"2024-04-25T08:38:19.944097+0000","last_undegraded":"2024-04-25T08:38:19.944097+0000","last_fullsized":"2024-04-25T08:38:19.944097+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:07.729869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875920+0000","last_change":"2024-04-25T08:38:16.873456+0000","last_active":"2024-04-25T08:38:18.875920+0000","last_peered":"2024-04-25T08:38:18.875920+0000","last_clean":"2024-04-25T08:38:18.875920+0000","last_became_active":"2024-04-25T08:38:16.873176+0000","last_became_peered":"2024-04-25T08:38:16.873176+0000","last_unstale":"2024-04-25T08:38:18.875920+0000","last_undegraded":"2024-04-25T08:38:18.875920+0000","last_fullsized":"2024-04-25T08:38:18.875920+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:58.941893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.943958+0000","last_change":"2024-04-25T08:38:17.892178+0000","last_active":"2024-04-25T08:38:19.943958+0000","last_peered":"2024-04-25T08:38:19.943958+0000","last_clean":"2024-04-25T08:38:19.943958+0000","last_became_active":"2024-04-25T08:38:17.891887+0000","last_became_peered":"2024-04-25T08:38:17.891887+0000","last_unstale":"2024-04-25T08:38:19.943958+0000","last_undegraded":"2024-04-25T08:38:19.943958+0000","last_fullsized":"2024-04-25T08:38:19.943958+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:40.077874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:19.944410+0000","last_change":"2024-04-25T08:38:16.878410+0000","last_active":"2024-04-25T08:38:19.944410+0000","last_peered":"2024-04-25T08:38:19.944410+0000","last_clean":"2024-04-25T08:38:19.944410+0000","last_became_active":"2024-04-25T08:38:16.877139+0000","last_became_peered":"2024-04-25T08:38:16.877139+0000","last_unstale":"2024-04-25T08:38:19.944410+0000","last_undegraded":"2024-04-25T08:38:19.944410+0000","last_fullsized":"2024-04-25T08:38:19.944410+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:16.115180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.877343+0000","last_change":"2024-04-25T08:38:17.897234+0000","last_active":"2024-04-25T08:38:18.877343+0000","last_peered":"2024-04-25T08:38:18.877343+0000","last_clean":"2024-04-25T08:38:18.877343+0000","last_became_active":"2024-04-25T08:38:17.896958+0000","last_became_peered":"2024-04-25T08:38:17.896958+0000","last_unstale":"2024-04-25T08:38:18.877343+0000","last_undegraded":"2024-04-25T08:38:18.877343+0000","last_fullsized":"2024-04-25T08:38:18.877343+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:19.842177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.874473+0000","last_change":"2024-04-25T08:38:16.866696+0000","last_active":"2024-04-25T08:38:18.874473+0000","last_peered":"2024-04-25T08:38:18.874473+0000","last_clean":"2024-04-25T08:38:18.874473+0000","last_became_active":"2024-04-25T08:38:16.866421+0000","last_became_peered":"2024-04-25T08:38:16.866421+0000","last_unstale":"2024-04-25T08:38:18.874473+0000","last_undegraded":"2024-04-25T08:38:18.874473+0000","last_fullsized":"2024-04-25T08:38:18.874473+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:16.115180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.876675+0000","last_change":"2024-04-25T08:38:17.879750+0000","last_active":"2024-04-25T08:38:18.876675+0000","last_peered":"2024-04-25T08:38:18.876675+0000","last_clean":"2024-04-25T08:38:18.876675+0000","last_became_active":"2024-04-25T08:38:17.879471+0000","last_became_peered":"2024-04-25T08:38:17.879471+0000","last_unstale":"2024-04-25T08:38:18.876675+0000","last_undegraded":"2024-04-25T08:38:18.876675+0000","last_fullsized":"2024-04-25T08:38:18.876675+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:16.850893+0000","last_clean_scrub_stamp":"2024-04-25T08:38:16.850893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:59.945936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875673+0000","last_change":"2024-04-25T08:38:16.870075+0000","last_active":"2024-04-25T08:38:18.875673+0000","last_peered":"2024-04-25T08:38:18.875673+0000","last_clean":"2024-04-25T08:38:18.875673+0000","last_became_active":"2024-04-25T08:38:16.869730+0000","last_became_peered":"2024-04-25T08:38:16.869730+0000","last_unstale":"2024-04-25T08:38:18.875673+0000","last_undegraded":"2024-04-25T08:38:18.875673+0000","last_fullsized":"2024-04-25T08:38:18.875673+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:15.846850+0000","last_clean_scrub_stamp":"2024-04-25T08:38:15.846850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:57.363947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":381,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-04-25T08:38:18.875376+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:38:18.875376+0000","last_peered":"2024-04-25T08:38:18.875376+0000","last_clean":"2024-04-25T08:38:18.875376+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:38:18.875376+0000","last_undegraded":"2024-04-25T08:38:18.875376+0000","last_fullsized":"2024-04-25T08:38:18.875376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542258,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47304,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324536,"statfs":{"total":96636764160,"available":96588324864,"internally_reserved":0,"allocated":2281472,"data_stored":1372601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542258,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50868,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320972,"statfs":{"total":96636764160,"available":96584675328,"internally_reserved":0,"allocated":1679360,"data_stored":781203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542258,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46704,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325136,"statfs":{"total":96636764160,"available":96588939264,"internally_reserved":0,"allocated":1675264,"data_stored":780553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50860,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320980,"statfs":{"total":96636764160,"available":96584683520,"internally_reserved":0,"allocated":1671168,"data_stored":780531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574961,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59200,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312640,"statfs":{"total":96636764160,"available":96576143360,"internally_reserved":0,"allocated":1687552,"data_stored":782213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59188,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312652,"statfs":{"total":96636764160,"available":96576155648,"internally_reserved":0,"allocated":1679360,"data_stored":781147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55024,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316816,"statfs":{"total":96636764160,"available":96580419584,"internally_reserved":0,"allocated":1675264,"data_stored":780601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59768,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312072,"statfs":{"total":96636764160,"available":96575561728,"internally_reserved":0,"allocated":2273280,"data_stored":1370955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:25.628 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:38:25.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:38:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.779+0000 7fe49a44a640 1 Processor -- start 2024-04-25T08:38:25.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.779+0000 7fe49a44a640 1 -- start start 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.780+0000 7fe49a44a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 0x7fe4940ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.780+0000 7fe49a44a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe4940d32f0 0x7fe4940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.780+0000 7fe49a44a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49415e000 con 0x7fe494161710 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.780+0000 7fe49a44a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4940d1de0 con 0x7fe4940d32f0 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.780+0000 7fe49a44a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe494161c60 con 0x7fe49415dc20 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe493fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 0x7fe4940ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe4989e7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe4940d32f0 0x7fe4940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe493fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 0x7fe4940ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:42512/0 (socket says 172.21.15.60:42512) 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe493fff640 1 -- 172.21.15.60:0/3805881329 learned_addr learned my addr 172.21.15.60:0/3805881329 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe4937fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe494161710 0x7fe49406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:52308/0 (socket says 172.21.15.60:52308) 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe4989e7640 1 -- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 msgr2=0x7fe4940ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe4989e7640 1 --2- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 0x7fe4940ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe4989e7640 1 -- 172.21.15.60:0/3805881329 >> v1:172.21.15.100:6789/0 conn(0x7fe494161710 legacy=0x7fe49406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe4989e7640 1 -- 172.21.15.60:0/3805881329 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494161f40 con 0x7fe4940d32f0 2024-04-25T08:38:25.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.781+0000 7fe493fff640 1 --2- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 0x7fe4940ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.782+0000 7fe4989e7640 1 --2- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4940d32f0 0x7fe4940cebb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009d90 tx=0x7fe48c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f6af14b4e8d0da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.782+0000 7fe492ffd640 1 -- 172.21.15.60:0/3805881329 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c007740 con 0x7fe4940d32f0 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.782+0000 7fe492ffd640 1 -- 172.21.15.60:0/3805881329 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c0078e0 con 0x7fe4940d32f0 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.782+0000 7fe492ffd640 1 -- 172.21.15.60:0/3805881329 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c007cb0 con 0x7fe4940d32f0 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.782+0000 7fe49a44a640 1 -- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4940d32f0 msgr2=0x7fe4940cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.782+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4940d32f0 0x7fe4940cebb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009d90 tx=0x7fe48c02a660 comp rx=0 tx=0).stop 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.783+0000 7fe49a44a640 1 -- 172.21.15.60:0/3805881329 shutdown_connections 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.783+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4940d32f0 0x7fe4940cebb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.783+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3805881329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe49415dc20 0x7fe4940ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.777 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.783+0000 7fe49a44a640 1 -- 172.21.15.60:0/3805881329 >> 172.21.15.60:0/3805881329 conn(0x7fe4940d0420 msgr2=0x7fe4940d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:25.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.783+0000 7fe49a44a640 1 -- 172.21.15.60:0/3805881329 shutdown_connections 2024-04-25T08:38:25.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.783+0000 7fe49a44a640 1 -- 172.21.15.60:0/3805881329 wait complete. 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.784+0000 7fe49a44a640 1 Processor -- start 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.784+0000 7fe49a44a640 1 -- start start 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe49a44a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4940d32f0 0x7fe494152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe49a44a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe49415dc20 0x7fe49415fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe49a44a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 0x7fe49414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe49a44a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940b5e50 con 0x7fe494161710 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe49a44a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494162500 con 0x7fe49415dc20 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe49a44a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940b5fd0 con 0x7fe4940d32f0 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 0x7fe49414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 0x7fe49414ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60792/0 (socket says 172.21.15.60:60792) 2024-04-25T08:38:25.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 -- 172.21.15.60:0/3319728406 learned_addr learned my addr 172.21.15.60:0/3319728406 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4937fe640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe49415dc20 0x7fe49415fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe493fff640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4940d32f0 0x7fe494152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 -- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4940d32f0 msgr2=0x7fe494152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4940d32f0 0x7fe494152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 -- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe49415dc20 msgr2=0x7fe49415fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe49415dc20 0x7fe49415fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 -- 172.21.15.60:0/3319728406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494160380 con 0x7fe494161710 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.785+0000 7fe4989e7640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 0x7fe49414ea10 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009eb0 tx=0x7fe48c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.786+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c040070 con 0x7fe494161710 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.786+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c031bc0 con 0x7fe494161710 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.786+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe494160660 con 0x7fe494161710 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.786+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c009450 con 0x7fe494161710 2024-04-25T08:38:25.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.786+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49414cb60 con 0x7fe494161710 2024-04-25T08:38:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.787+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe494160b80 con 0x7fe494161710 2024-04-25T08:38:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.787+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe48c039040 con 0x7fe494161710 2024-04-25T08:38:25.782 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.788+0000 7fe490ff9640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe47406b500 0x7fe47406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.788+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe48c0ab970 con 0x7fe494161710 2024-04-25T08:38:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.788+0000 7fe493fff640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe47406b500 0x7fe47406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.789+0000 7fe493fff640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe47406b500 0x7fe47406d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006ca0 tx=0x7fe47c009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:25.792+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe494160b80 con 0x7fe494161710 2024-04-25T08:38:26.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.043+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe4940d16b0 con 0x7fe494161710 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.046+0000 7fe490ff9640 1 -- 172.21.15.60:0/3319728406 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe48c07a010 con 0x7fe494161710 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:38:26.039 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe47406b500 msgr2=0x7fe47406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe47406b500 0x7fe47406d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006ca0 tx=0x7fe47c009240 comp rx=0 tx=0).stop 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 msgr2=0x7fe49414ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 0x7fe49414ea10 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009eb0 tx=0x7fe48c004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 shutdown_connections 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe494161710 0x7fe49414ea10 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe47406b500 0x7fe47406d9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe49415dc20 0x7fe49415fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 --2- 172.21.15.60:0/3319728406 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4940d32f0 0x7fe494152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 >> 172.21.15.60:0/3319728406 conn(0x7fe4940d0420 msgr2=0x7fe4940b86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 shutdown_connections 2024-04-25T08:38:26.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.047+0000 7fe49a44a640 1 -- 172.21.15.60:0/3319728406 wait complete. 2024-04-25T08:38:26.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.159+0000 7f1205bab640 1 Processor -- start 2024-04-25T08:38:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.159+0000 7f1205bab640 1 -- start start 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f1205bab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12000cfec0 0x7f12000cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f1205bab640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1200071480 con 0x7f12000ce3c0 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f1205bab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1200071760 con 0x7f1200154ee0 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f1205bab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1200072090 con 0x7f12000cfec0 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11fffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f12000ce3c0 0x7f120006d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57572/0 (socket says 172.21.15.60:57572) 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11fffff640 1 -- 172.21.15.60:0/2449695090 learned_addr learned my addr 172.21.15.60:0/2449695090 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11feffd640 1 --2- 172.21.15.60:0/2449695090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12000cfec0 0x7f12000cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192294172 0 0) 0x7f1200071480 con 0x7f12000ce3c0 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e8003520 con 0x7f12000ce3c0 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228003233 0 0) 0x7f1200071760 con 0x7f1200154ee0 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1200071480 con 0x7f1200154ee0 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.160+0000 7f11feffd640 1 -- 172.21.15.60:0/2449695090 >> v1:172.21.15.100:6789/0 conn(0x7f1200154ee0 legacy=0x7f12000cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.161+0000 7f11feffd640 1 -- 172.21.15.60:0/2449695090 >> v1:172.21.15.60:6789/0 conn(0x7f12000ce3c0 legacy=0x7f120006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.161+0000 7f11feffd640 1 -- 172.21.15.60:0/2449695090 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200071a40 con 0x7f12000cfec0 2024-04-25T08:38:26.156 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.161+0000 7f11feffd640 1 --2- 172.21.15.60:0/2449695090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12000cfec0 0x7f12000cde80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f11f40097d0 tx=0x7f11f402c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be31784c4ae39b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4038070 con 0x7f12000cfec0 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4031710 con 0x7f12000cfec0 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f11fe7fc640 1 -- 172.21.15.60:0/2449695090 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4031ae0 con 0x7f12000cfec0 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 -- 172.21.15.60:0/2449695090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12000cfec0 msgr2=0x7f12000cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 --2- 172.21.15.60:0/2449695090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12000cfec0 0x7f12000cde80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f11f40097d0 tx=0x7f11f402c420 comp rx=0 tx=0).stop 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 -- 172.21.15.60:0/2449695090 shutdown_connections 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 --2- 172.21.15.60:0/2449695090 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12000cfec0 0x7f12000cde80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 -- 172.21.15.60:0/2449695090 >> 172.21.15.60:0/2449695090 conn(0x7f120006abe0 msgr2=0x7f120006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 -- 172.21.15.60:0/2449695090 shutdown_connections 2024-04-25T08:38:26.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.162+0000 7f1205bab640 1 -- 172.21.15.60:0/2449695090 wait complete. 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.163+0000 7f1205bab640 1 Processor -- start 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.163+0000 7f1205bab640 1 -- start start 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f1205bab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 0x7f1200140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f1205bab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12000cfec0 0x7f1200151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f1205bab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1200154ee0 0x7f12001599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f1205bab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000ce800 con 0x7f12000ce3c0 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f1205bab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200071230 con 0x7f12000cfec0 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f1205bab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12001552c0 con 0x7f1200154ee0 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 0x7f1200140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f11ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 0x7f1200140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60802/0 (socket says 172.21.15.60:60802) 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f11ff7fe640 1 -- 172.21.15.60:0/2631545765 learned_addr learned my addr 172.21.15.60:0/2631545765 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.164+0000 7f11fffff640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1200154ee0 0x7f12001599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11feffd640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12000cfec0 0x7f1200151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11ff7fe640 1 -- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1200154ee0 msgr2=0x7f12001599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11ff7fe640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1200154ee0 0x7f12001599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11ff7fe640 1 -- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12000cfec0 msgr2=0x7f1200151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11ff7fe640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12000cfec0 0x7f1200151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11ff7fe640 1 -- 172.21.15.60:0/2631545765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12001e32b0 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11ff7fe640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 0x7f1200140640 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c530 tx=0x7f11f000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f00052d0 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f0005490 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12001e3590 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012870 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.165+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12001e3b50 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.166+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1200073990 con 0x7f12000ce3c0 2024-04-25T08:38:26.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.166+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11f0012a10 con 0x7f12000ce3c0 2024-04-25T08:38:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.167+0000 7f11e7fff640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b4d0 0x7f11ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.167+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11f0089ff0 con 0x7f12000ce3c0 2024-04-25T08:38:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.167+0000 7f11feffd640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b4d0 0x7f11ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.163 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.168+0000 7f11feffd640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b4d0 0x7f11ec06d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f11f4035000 tx=0x7f11f4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.171+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11f0058cf0 con 0x7f12000ce3c0 2024-04-25T08:38:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.393+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12000b3c30 con 0x7f12000ce3c0 2024-04-25T08:38:26.387 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.394+0000 7f11e7fff640 1 -- 172.21.15.60:0/2631545765 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f11f00585f0 con 0x7f12000ce3c0 2024-04-25T08:38:26.388 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:26.388 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:26.388 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b4d0 msgr2=0x7f11ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b4d0 0x7f11ec06d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f11f4035000 tx=0x7f11f4008030 comp rx=0 tx=0).stop 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 msgr2=0x7f1200140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 0x7f1200140640 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c530 tx=0x7f11f000c9f0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 shutdown_connections 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1200154ee0 0x7f12001599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f11ec06b4d0 0x7f11ec06d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12000cfec0 0x7f1200151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 --2- 172.21.15.60:0/2631545765 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f12000ce3c0 0x7f1200140640 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 >> 172.21.15.60:0/2631545765 conn(0x7f120006abe0 msgr2=0x7f12000b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 shutdown_connections 2024-04-25T08:38:26.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.396+0000 7f1205bab640 1 -- 172.21.15.60:0/2631545765 wait complete. 2024-04-25T08:38:26.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:38:17.221740+0000', 'modified': '2024-04-25T08:38:19.224480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9381': {'gid': 9381, 'name': 'd', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.100:6836/879630460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 879630460}, {'type': 'v1', 'addr': '172.21.15.100:6836', 'nonce': 879630460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:38:26.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:38:26.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:38:26.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.917+0000 7ff9feaf5640 1 Processor -- start 2024-04-25T08:38:26.911 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.917+0000 7ff9feaf5640 1 -- start start 2024-04-25T08:38:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9feaf5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9f815bb50 0x7ff9f80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9feaf5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9feaf5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80c75e0 con 0x7ff9f80cfb40 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9feaf5640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80c8680 con 0x7ff9f815bb50 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9feaf5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c78c0 con 0x7ff9f80c70d0 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9fc891640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9f815bb50 0x7ff9f80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9fc891640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff9f815bb50 0x7ff9f80c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60818/0 (socket says 172.21.15.60:60818) 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9fd092640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff9f80cfb40 0x7ff9f8070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57574/0 (socket says 172.21.15.60:57574) 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.918+0000 7ff9fc891640 1 -- 172.21.15.60:0/1127030119 learned_addr learned my addr 172.21.15.60:0/1127030119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9fc891640 1 -- 172.21.15.60:0/1127030119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 msgr2=0x7ff9f80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9f77fe640 1 -- 172.21.15.60:0/1127030119 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760620472 0 0) 0x7ff9f80c75e0 con 0x7ff9f80cfb40 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9fc891640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9fc891640 1 -- 172.21.15.60:0/1127030119 >> v1:172.21.15.60:6789/0 conn(0x7ff9f80cfb40 legacy=0x7ff9f8070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9fc891640 1 -- 172.21.15.60:0/1127030119 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80c7ba0 con 0x7ff9f815bb50 2024-04-25T08:38:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:26.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.919+0000 7ff9fc891640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f80c6b90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009870 tx=0x7ff9e802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb854dcb8b5b0093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9f77fe640 1 -- 172.21.15.60:0/1127030119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8037070 con 0x7ff9f815bb50 2024-04-25T08:38:26.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9f77fe640 1 -- 172.21.15.60:0/1127030119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e8032640 con 0x7ff9f815bb50 2024-04-25T08:38:26.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9f77fe640 1 -- 172.21.15.60:0/1127030119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8032a10 con 0x7ff9f815bb50 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1127030119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 msgr2=0x7ff9f80c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f80c6b90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009870 tx=0x7ff9e802a2e0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1127030119 shutdown_connections 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1127030119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f80c6b90 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1127030119 >> 172.21.15.60:0/1127030119 conn(0x7ff9f806f500 msgr2=0x7ff9f806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1127030119 shutdown_connections 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.920+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1127030119 wait complete. 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.921+0000 7ff9feaf5640 1 Processor -- start 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.921+0000 7ff9feaf5640 1 -- start start 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9feaf5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9feaf5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9f80cfb40 0x7ff9f8148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9feaf5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f81f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9feaf5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815f6d0 con 0x7ff9f815bb50 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9feaf5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80c8160 con 0x7ff9f80cfb40 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9feaf5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815f850 con 0x7ff9f80c70d0 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.922+0000 7ff9fd092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f81f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f81f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60830/0 (socket says 172.21.15.60:60830) 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 -- 172.21.15.60:0/1839151831 learned_addr learned my addr 172.21.15.60:0/1839151831 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fc891640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9f80cfb40 0x7ff9f8148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 -- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 msgr2=0x7ff9f80c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 -- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9f80cfb40 msgr2=0x7ff9f8148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.919 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9f80cfb40 0x7ff9f8148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 -- 172.21.15.60:0/1839151831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f815da40 con 0x7ff9f815bb50 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9f80cfb40 0x7ff9f8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9fd092640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f81f1db0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c7a0 tx=0x7ff9f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9f57fa640 1 -- 172.21.15.60:0/1839151831 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0018070 con 0x7ff9f815bb50 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9f57fa640 1 -- 172.21.15.60:0/1839151831 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000a640 con 0x7ff9f815bb50 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f815c060 con 0x7ff9f815bb50 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9f57fa640 1 -- 172.21.15.60:0/1839151831 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000aab0 con 0x7ff9f815bb50 2024-04-25T08:38:26.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.923+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f815dce0 con 0x7ff9f815bb50 2024-04-25T08:38:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.925+0000 7ff9f57fa640 1 -- 172.21.15.60:0/1839151831 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9f001b070 con 0x7ff9f815bb50 2024-04-25T08:38:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.925+0000 7ff9f57fa640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9e406b4d0 0x7ff9e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.925+0000 7ff9f57fa640 1 -- 172.21.15.60:0/1839151831 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9f0089870 con 0x7ff9f815bb50 2024-04-25T08:38:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.925+0000 7ff9fc891640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9e406b4d0 0x7ff9e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.926+0000 7ff9fc891640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9e406b4d0 0x7ff9e406d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034000 tx=0x7ff9e802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.926+0000 7ffa017f9740 1 Processor -- start 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.926+0000 7ffa017f9740 1 -- start start 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ebad0 0x555e0b9ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 0x555e0b9ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ffa017f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b906a10 con 0x555e0b9a3f10 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b8afb30 con 0x555e0b9ce230 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e0b902c50 con 0x555e0b9ebad0 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 0x555e0b9ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 0x555e0b9ea190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60840/0 (socket says 172.21.15.60:60840) 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 -- 172.21.15.60:0/2243272453 learned_addr learned my addr 172.21.15.60:0/2243272453 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fc891640 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ebad0 0x555e0b9ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 -- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ebad0 msgr2=0x555e0b9ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ebad0 0x555e0b9ec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 -- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 msgr2=0x555e0b9ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 -- 172.21.15.60:0/2243272453 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0b8c4e10 con 0x555e0b9a3f10 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9fd092640 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 0x555e0b9ea190 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0004fd0 tx=0x7ff9f0057e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be1a6618760681f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.927+0000 7ff9de7fc640 1 -- 172.21.15.60:0/2243272453 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00562a0 con 0x555e0b9a3f10 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ff9de7fc640 1 -- 172.21.15.60:0/2243272453 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0054610 con 0x555e0b9a3f10 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ff9de7fc640 1 -- 172.21.15.60:0/2243272453 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000ae50 con 0x555e0b9a3f10 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 -- 172.21.15.60:0/2243272453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 msgr2=0x555e0b9ea190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 0x555e0b9ea190 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0004fd0 tx=0x7ff9f0057e80 comp rx=0 tx=0).stop 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 -- 172.21.15.60:0/2243272453 shutdown_connections 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9a3f10 0x555e0b9ea190 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ebad0 0x555e0b9ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2243272453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 -- 172.21.15.60:0/2243272453 >> 172.21.15.60:0/2243272453 conn(0x555e0b9a0840 msgr2=0x555e0b9e9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 -- 172.21.15.60:0/2243272453 shutdown_connections 2024-04-25T08:38:26.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.928+0000 7ffa017f9740 1 -- 172.21.15.60:0/2243272453 wait complete. 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.929+0000 7ffa017f9740 1 -- start start 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.929+0000 7ffa017f9740 1 Processor -- start 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 0x555e0b9bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0b9d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b9a4cb0 con 0x555e0b9ce230 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b8c30c0 con 0x555e0b9ebad0 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e0b99e770 con 0x555e0b9bd320 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 0x555e0b9bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ffa017f9740 10 client.0 _ll_register_callbacks cb 0x555e0b8b6ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 0x555e0b9bc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60844/0 (socket says 172.21.15.60:60844) 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 -- 172.21.15.60:0/2906601018 learned_addr learned my addr 172.21.15.60:0/2906601018 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fd092640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0b9d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 -- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 msgr2=0x555e0b9d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0b9d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 -- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 msgr2=0x555e0b9e1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9e1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 -- 172.21.15.60:0/2906601018 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e0b61b2d0 con 0x555e0b9ce230 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9fc891640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 0x555e0b9bc9d0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009990 tx=0x7ff9e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23f50102c48bcc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.930+0000 7ff9d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ff9d77fe640 1 -- 172.21.15.60:0/2906601018 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e804f020 con 0x555e0b9ce230 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ff9d77fe640 1 -- 172.21.15.60:0/2906601018 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9e803d720 con 0x555e0b9ce230 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9f0054610 con 0x555e0b9ce230 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ff9d77fe640 1 -- 172.21.15.60:0/2906601018 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9e8059050 con 0x555e0b9ce230 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 10 client.9540 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:26.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x555e0b6e56b0 con 0x555e0b9ce230 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ff9d77fe640 1 -- 172.21.15.60:0/2906601018 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0054610 con 0x555e0b9ce230 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ff9d77fe640 1 -- 172.21.15.60:0/2906601018 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff9e805c020 con 0x555e0b9ce230 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 10 client.9540 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 10 client.9540 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 20 client.9540 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b99d4a0 0x555e0b9d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 4 client.9540 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e0b965920 con 0x555e0b99d4a0 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.931+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b99d4a0 0x555e0b9d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.932+0000 7ff9d77fe640 10 client.9540 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.932+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b99d4a0 0x555e0b9d0770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d5b7478a940013 server_cookie=befb500cce9e9bd9 in_seq=0 out_seq=0 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.932+0000 7ff9d77fe640 1 -- 172.21.15.60:0/2906601018 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e0b965920 con 0x555e0b99d4a0 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ff9d77fe640 10 client.9540 handle_command_reply: tid=0 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 client.9540 shutdown 2024-04-25T08:38:26.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 msgr2=0x555e0b9bc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 0x555e0b9bc9d0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009990 tx=0x7ff9e8004730 comp rx=0 tx=0).stop 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 shutdown_connections 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b99d4a0 0x555e0b9d0770 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0b9d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9e1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ff9d77fe640 0 client.9540 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 --2- 172.21.15.60:0/2906601018 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ce230 0x555e0b9bc9d0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 >> 172.21.15.60:0/2906601018 conn(0x555e0b9a3f10 msgr2=0x555e0b9ec5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 shutdown_connections 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 1 -- 172.21.15.60:0/2906601018 wait complete. 2024-04-25T08:38:26.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.933+0000 7ffa017f9740 20 client.9540 trim_cache size 0 max 16384 2024-04-25T08:38:26.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 Processor -- start 2024-04-25T08:38:26.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 -- start start 2024-04-25T08:38:26.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 0x555e0b9eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ce230 0x555e0b9ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81fe050 con 0x555e0b9bd320 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81451f0 con 0x555e0b9ebad0 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.949+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81eef40 con 0x555e0b9ce230 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 0x555e0b9eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 0x555e0b9eae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60852/0 (socket says 172.21.15.60:60852) 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 -- 172.21.15.60:0/782160232 learned_addr learned my addr 172.21.15.60:0/782160232 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ce230 0x555e0b9ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fd092640 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 -- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ce230 msgr2=0x555e0b9ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ce230 0x555e0b9ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 -- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 msgr2=0x555e0b9d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 -- 172.21.15.60:0/782160232 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0b9d0f30 con 0x555e0b9bd320 2024-04-25T08:38:26.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fd092640 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9fc891640 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 0x555e0b9eae60 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009960 tx=0x7ff9e8049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862f8e3c9f8a682c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9ddffb640 1 -- 172.21.15.60:0/782160232 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e804f040 con 0x555e0b9bd320 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9ddffb640 1 -- 172.21.15.60:0/782160232 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e803ddc0 con 0x555e0b9bd320 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.950+0000 7ff9ddffb640 1 -- 172.21.15.60:0/782160232 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8037040 con 0x555e0b9bd320 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 -- 172.21.15.60:0/782160232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 msgr2=0x555e0b9eae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 0x555e0b9eae60 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009960 tx=0x7ff9e8049000 comp rx=0 tx=0).stop 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 -- 172.21.15.60:0/782160232 shutdown_connections 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ebad0 0x555e0b9d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9ce230 0x555e0b9ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 --2- 172.21.15.60:0/782160232 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9bd320 0x555e0b9eae60 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 -- 172.21.15.60:0/782160232 >> 172.21.15.60:0/782160232 conn(0x555e0b9a3f10 msgr2=0x555e0b9bbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 -- 172.21.15.60:0/782160232 shutdown_connections 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 -- 172.21.15.60:0/782160232 wait complete. 2024-04-25T08:38:26.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.951+0000 7ffa017f9740 1 -- start start 2024-04-25T08:38:26.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 Processor -- start 2024-04-25T08:38:26.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0ba53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 0x555e0b9a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b9bdd90 con 0x555e0b9ebad0 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815f270 con 0x555e0b9ce230 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e0b99d8f0 con 0x555e0b9bd320 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ffa017f9740 10 client.0 _ll_register_callbacks cb 0x555e0b99ec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ff9fd092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 0x555e0b9a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ff9fd092640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 0x555e0b9a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60854/0 (socket says 172.21.15.60:60854) 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.952+0000 7ff9fd092640 1 -- 172.21.15.60:0/3430156944 learned_addr learned my addr 172.21.15.60:0/3430156944 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fc891640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0ba53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fd092640 1 -- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 msgr2=0x555e0ba53db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fd092640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0ba53db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fd092640 1 -- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 msgr2=0x555e0b9a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fd092640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fd092640 1 -- 172.21.15.60:0/3430156944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e0b9bc3f0 con 0x555e0b9ebad0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9f7fff640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9fd092640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 0x555e0b9a4890 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0087c70 tx=0x7ff9f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263caa4798ac79b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9d77fe640 1 -- 172.21.15.60:0/3430156944 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0073070 con 0x555e0b9ebad0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9d77fe640 1 -- 172.21.15.60:0/3430156944 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9f005e070 con 0x555e0b9ebad0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9e803ddc0 con 0x555e0b9ebad0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9d77fe640 1 -- 172.21.15.60:0/3430156944 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f000ae50 con 0x555e0b9ebad0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 10 client.9546 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x555e0b9ea000 con 0x555e0b9ebad0 2024-04-25T08:38:26.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9d77fe640 1 -- 172.21.15.60:0/3430156944 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e803ddc0 con 0x555e0b9ebad0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ff9d77fe640 1 -- 172.21.15.60:0/3430156944 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff9f00573f0 con 0x555e0b9ebad0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 10 client.9546 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 10 client.9546 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 20 client.9546 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b9defd0 0x555e0b9e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 4 client.9546 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.953+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e0b913f50 con 0x555e0b9defd0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ff9fc891640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b9defd0 0x555e0b9e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ff9d77fe640 10 client.9546 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ff9fc891640 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b9defd0 0x555e0b9e1de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4706cc1f026c2d6a server_cookie=6947d3a5b0129293 in_seq=0 out_seq=0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ff9d77fe640 1 -- 172.21.15.60:0/3430156944 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555e0b913f50 con 0x555e0b9defd0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ff9d77fe640 10 client.9546 handle_command_reply: tid=0 2024-04-25T08:38:26.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 client.9546 shutdown 2024-04-25T08:38:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 msgr2=0x555e0b9a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 0x555e0b9a4890 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0087c70 tx=0x7ff9f0014000 comp rx=0 tx=0).stop 2024-04-25T08:38:26.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 shutdown_connections 2024-04-25T08:38:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x555e0b9defd0 0x555e0b9e1de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x555e0b9ebad0 0x555e0b9a4890 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555e0b9ce230 0x555e0b9a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.954+0000 7ffa017f9740 1 --2- 172.21.15.60:0/3430156944 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x555e0b9bd320 0x555e0ba53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 >> 172.21.15.60:0/3430156944 conn(0x555e0b9a3f10 msgr2=0x555e0b9d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9d77fe640 0 client.9546 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 shutdown_connections 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ffa017f9740 1 -- 172.21.15.60:0/3430156944 wait complete. 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ffa017f9740 20 client.9546 trim_cache size 0 max 16384 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:38:26.958 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9381, 2024-04-25T08:38:26.959 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:38:26.959 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.0825410039999994, 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 78, 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 134, 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 134, 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.093110853000001 2024-04-25T08:38:26.960 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:38:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9e406b4d0 msgr2=0x7ff9e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9e406b4d0 0x7ff9e406d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8034000 tx=0x7ff9e802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 msgr2=0x7ff9f81f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f81f1db0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c7a0 tx=0x7ff9f000cc60 comp rx=0 tx=0).stop 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 shutdown_connections 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9f815bb50 0x7ff9f81f1db0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff9e406b4d0 0x7ff9e406d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9f80cfb40 0x7ff9f8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 --2- 172.21.15.60:0/1839151831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff9f80c70d0 0x7ff9f80c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 >> 172.21.15.60:0/1839151831 conn(0x7ff9f806f500 msgr2=0x7ff9f8156180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 shutdown_connections 2024-04-25T08:38:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:26.955+0000 7ff9feaf5640 1 -- 172.21.15.60:0/1839151831 wait complete. 2024-04-25T08:38:26.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:38:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.070+0000 7f06b965d640 1 Processor -- start 2024-04-25T08:38:27.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.070+0000 7f06b965d640 1 -- start start 2024-04-25T08:38:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b965d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06b415dbe0 0x7f06b40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b965d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06b40b92a0 0x7f06b40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b965d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b40b97a0 con 0x7f06b40b2cb0 2024-04-25T08:38:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b965d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b415dfc0 con 0x7f06b415dbe0 2024-04-25T08:38:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b965d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b40d1a00 con 0x7f06b40b92a0 2024-04-25T08:38:27.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06b415dbe0 0x7f06b40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f06b415dbe0 0x7f06b40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60860/0 (socket says 172.21.15.60:60860) 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 -- 172.21.15.60:0/2683938925 learned_addr learned my addr 172.21.15.60:0/2683938925 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b37fe640 1 --2- 172.21.15.60:0/2683938925 >> v2:172.21.15.100:3300/0 conn(0x7f06b40b92a0 0x7f06b40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 -- 172.21.15.60:0/2683938925 >> v2:172.21.15.100:3300/0 conn(0x7f06b40b92a0 msgr2=0x7f06b40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 --2- 172.21.15.60:0/2683938925 >> v2:172.21.15.100:3300/0 conn(0x7f06b40b92a0 0x7f06b40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 -- 172.21.15.60:0/2683938925 >> v1:172.21.15.100:6789/0 conn(0x7f06b40b2cb0 legacy=0x7f06b40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b2ffd640 1 -- 172.21.15.60:0/2683938925 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b40d1c50 con 0x7f06b415dbe0 2024-04-25T08:38:27.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.071+0000 7f06b37fe640 1 --2- 172.21.15.60:0/2683938925 >> v2:172.21.15.100:3300/0 conn(0x7f06b40b92a0 0x7f06b40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.072+0000 7f06b2ffd640 1 --2- 172.21.15.60:0/2683938925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b415dbe0 0x7f06b40d0910 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f06b40cfb90 tx=0x7f06a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9204dd56b2d7ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.072+0000 7f06b1ffb640 1 -- 172.21.15.60:0/2683938925 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4037070 con 0x7f06b415dbe0 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.072+0000 7f06b1ffb640 1 -- 172.21.15.60:0/2683938925 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a4032440 con 0x7f06b415dbe0 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.072+0000 7f06b1ffb640 1 -- 172.21.15.60:0/2683938925 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4032810 con 0x7f06b415dbe0 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.072+0000 7f06b965d640 1 -- 172.21.15.60:0/2683938925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b415dbe0 msgr2=0x7f06b40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.072+0000 7f06b965d640 1 --2- 172.21.15.60:0/2683938925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b415dbe0 0x7f06b40d0910 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f06b40cfb90 tx=0x7f06a402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.073+0000 7f06b965d640 1 -- 172.21.15.60:0/2683938925 shutdown_connections 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.073+0000 7f06b965d640 1 --2- 172.21.15.60:0/2683938925 >> v2:172.21.15.100:3300/0 conn(0x7f06b40b92a0 0x7f06b40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.073+0000 7f06b965d640 1 --2- 172.21.15.60:0/2683938925 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b415dbe0 0x7f06b40d0910 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.073+0000 7f06b965d640 1 -- 172.21.15.60:0/2683938925 >> 172.21.15.60:0/2683938925 conn(0x7f06b4061a20 msgr2=0x7f06b4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.073+0000 7f06b965d640 1 -- 172.21.15.60:0/2683938925 shutdown_connections 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.073+0000 7f06b965d640 1 -- 172.21.15.60:0/2683938925 wait complete. 2024-04-25T08:38:27.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.074+0000 7f06b965d640 1 Processor -- start 2024-04-25T08:38:27.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.074+0000 7f06b965d640 1 -- start start 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b40b2cb0 0x7f06b41f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 0x7f06b41f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b415dbe0 0x7f06b40c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40b6ab0 con 0x7f06b40b92a0 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40d2210 con 0x7f06b40b2cb0 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40b6c30 con 0x7f06b415dbe0 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 0x7f06b41f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 0x7f06b41f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60872/0 (socket says 172.21.15.60:60872) 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 -- 172.21.15.60:0/1857052755 learned_addr learned my addr 172.21.15.60:0/1857052755 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b2ffd640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b40b2cb0 0x7f06b41f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b37fe640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b415dbe0 0x7f06b40c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 -- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b415dbe0 msgr2=0x7f06b40c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b415dbe0 0x7f06b40c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 -- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b40b2cb0 msgr2=0x7f06b41f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b40b2cb0 0x7f06b41f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 -- 172.21.15.60:0/1857052755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b41f7730 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b27fc640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 0x7f06b41f6ee0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c440 tx=0x7f06a8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80052d0 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8005470 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b41f7a10 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.075+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8007260 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.076+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b4144650 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.076+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b4073990 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.077+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06a8007400 con 0x7f06b40b92a0 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.077+0000 7f069b7fe640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.078+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f06a8089820 con 0x7f06b40b92a0 2024-04-25T08:38:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.078+0000 7f06b2ffd640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.078+0000 7f06b2ffd640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f06a006b4d0 0x7f06a006d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f06a4034000 tx=0x7f06a402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.082+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06a8057ec0 con 0x7f06b40b92a0 2024-04-25T08:38:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.304+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06b40b28f0 con 0x7f06b40b92a0 2024-04-25T08:38:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.305+0000 7f069b7fe640 1 -- 172.21.15.60:0/1857052755 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06a805bf60 con 0x7f06b40b92a0 2024-04-25T08:38:27.299 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:38:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.306+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f06a006b4d0 msgr2=0x7f06a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.306+0000 7f06b965d640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f06a006b4d0 0x7f06a006d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f06a4034000 tx=0x7f06a402e000 comp rx=0 tx=0).stop 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 msgr2=0x7f06b41f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 0x7f06b41f6ee0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c440 tx=0x7f06a8001e70 comp rx=0 tx=0).stop 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 shutdown_connections 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06b415dbe0 0x7f06b40c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06b40b92a0 0x7f06b41f6ee0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 --2- 172.21.15.60:0/1857052755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b40b2cb0 0x7f06b41f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 >> 172.21.15.60:0/1857052755 conn(0x7f06b4061a20 msgr2=0x7f06b415b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 shutdown_connections 2024-04-25T08:38:27.302 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:27.307+0000 7f06b965d640 1 -- 172.21.15.60:0/1857052755 wait complete. 2024-04-25T08:38:27.311 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:38:27.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:38:27.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:38:27.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:38:27.312 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:27.312 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:38:27.312 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:38:27.312 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:38:27.312 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:38:27.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.312 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.328 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42203sec preferred_lft 42203sec 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:27.329 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:38:27.330 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:27.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.331 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:38:27.391 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:38:27.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.393 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:38:27.449 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:38:27.449 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:38:27.450 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:38:27.450 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:38:27.450 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:38:27.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.450 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:38:27.450 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:27.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:38:27.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:38:27.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:38:27.450 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:27.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.535 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:38:27.591 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:38:27.591 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:38:27.592 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:38:27.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:38:27.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:27.635 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:27.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:38:27.657 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:27.742 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:38:27.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:38:27.743 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:27.797 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:38:27.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:38:27.880 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:38:27.881 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:38:27.881 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:38:27.889 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:38:27.889 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:38:27.889 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:38:27.889 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:38:27.889 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:38:27.907 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:38:27.908 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:38:27.908 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:38:27.908 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:38:27.908 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:38:27.908 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:38:27.908 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:38:27.909 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:38:27.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:27.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:27.975 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:38:28.042 INFO:teuthology.orchestra.run.smithi060.stdout:client8383 2024-04-25T08:38:28.042 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:38:28.042 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:28.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:28.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.245+0000 7fa5065a5640 1 Processor -- start 2024-04-25T08:38:28.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.245+0000 7fa5065a5640 1 -- start start 2024-04-25T08:38:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa5065a5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa500154ed0 0x7fa5000aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa5065a5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50015ec60 con 0x7fa50006c1d0 2024-04-25T08:38:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa5065a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50015ef40 con 0x7fa500071ac0 2024-04-25T08:38:28.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa5065a5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50006db90 con 0x7fa500154ed0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa504b42640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa50006c1d0 0x7fa50006afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57580/0 (socket says 172.21.15.60:57580) 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa504b42640 1 -- 172.21.15.60:0/2464550868 learned_addr learned my addr 172.21.15.60:0/2464550868 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081892277 0 0) 0x7fa50015ec60 con 0x7fa50006c1d0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e8003520 con 0x7fa50006c1d0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4fffff640 1 --2- 172.21.15.60:0/2464550868 >> v2:172.21.15.100:3300/0 conn(0x7fa500154ed0 0x7fa5000aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782375031 0 0) 0x7fa4e8003520 con 0x7fa50006c1d0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 >> v2:172.21.15.100:3300/0 conn(0x7fa500154ed0 msgr2=0x7fa5000aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 --2- 172.21.15.60:0/2464550868 >> v2:172.21.15.100:3300/0 conn(0x7fa500154ed0 0x7fa5000aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 >> v1:172.21.15.100:6789/0 conn(0x7fa500071ac0 legacy=0x7fa5000ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50006de70 con 0x7fa50006c1d0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fa4f80019f0 con 0x7fa50006c1d0 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 >> v1:172.21.15.60:6789/0 conn(0x7fa50006c1d0 legacy=0x7fa50006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.246+0000 7fa4feffd640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4e80051c0 0x7fa4e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50006db90 con 0x7fa4e80039b0 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e800c160 con 0x7fa4e8004800 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e800c360 con 0x7fa4e80051c0 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4fffff640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4ff7fe640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa504b42640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4e80051c0 0x7fa4e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4fffff640 1 -- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4e80051c0 msgr2=0x7fa4e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4fffff640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa4e80051c0 0x7fa4e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4fffff640 1 -- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8004800 msgr2=0x7fa4e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4fffff640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa4fffff640 1 -- 172.21.15.60:0/2464550868 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8003520 con 0x7fa4e80039b0 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.247+0000 7fa504b42640 1 -- 172.21.15.60:0/2464550868 reap_dead start 2024-04-25T08:38:28.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa4fffff640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009ac0 tx=0x7fa4e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf1ccb6a5a94cca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4036070 con 0x7fa4e80039b0 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f4031400 con 0x7fa4e80039b0 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa4feffd640 1 -- 172.21.15.60:0/2464550868 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4009690 con 0x7fa4e80039b0 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 -- 172.21.15.60:0/2464550868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4e80039b0 msgr2=0x7fa4e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009ac0 tx=0x7fa4e8005830 comp rx=0 tx=0).stop 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 -- 172.21.15.60:0/2464550868 shutdown_connections 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 --2- 172.21.15.60:0/2464550868 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa4e80039b0 0x7fa4e8003dd0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 -- 172.21.15.60:0/2464550868 >> 172.21.15.60:0/2464550868 conn(0x7fa5000af670 msgr2=0x7fa5000b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 -- 172.21.15.60:0/2464550868 shutdown_connections 2024-04-25T08:38:28.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.248+0000 7fa5065a5640 1 -- 172.21.15.60:0/2464550868 wait complete. 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.249+0000 7fa5065a5640 1 Processor -- start 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 -- start start 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa50014cb20 0x7fa5000cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 0x7fa500157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa500158060 0x7fa5001f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000ad260 con 0x7fa5000b4d30 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000ad0e0 con 0x7fa50014cb20 2024-04-25T08:38:28.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa5065a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000ad3e0 con 0x7fa500158060 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.250+0000 7fa4ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 0x7fa500157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 0x7fa500157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60884/0 (socket says 172.21.15.60:60884) 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 -- 172.21.15.60:0/3037862949 learned_addr learned my addr 172.21.15.60:0/3037862949 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa504b42640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa500158060 0x7fa5001f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4fffff640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa50014cb20 0x7fa5000cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 -- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa500158060 msgr2=0x7fa5001f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa500158060 0x7fa5001f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 -- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa50014cb20 msgr2=0x7fa5000cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa50014cb20 0x7fa5000cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 -- 172.21.15.60:0/3037862949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000ccf80 con 0x7fa5000b4d30 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4ff7fe640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 0x7fa500157b20 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000f5f0 tx=0x7fa4f000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00052e0 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0018070 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5000cd260 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0003810 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.251+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5000cd820 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.252+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa500073990 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.252+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4f00039b0 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.253+0000 7fa4fcff9640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa4ec06b4d0 0x7fa4ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.253+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4f008a170 con 0x7fa5000b4d30 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.253+0000 7fa4fffff640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa4ec06b4d0 0x7fa4ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.254+0000 7fa4fffff640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa4ec06b4d0 0x7fa4ec06d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4033000 tx=0x7fa4f4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.257+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4f0058f20 con 0x7fa5000b4d30 2024-04-25T08:38:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.477+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa500156340 con 0x7fa5000b4d30 2024-04-25T08:38:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.477+0000 7fa4fcff9640 1 -- 172.21.15.60:0/3037862949 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa4f0058820 con 0x7fa5000b4d30 2024-04-25T08:38:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:28.472 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:28.472 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa4ec06b4d0 msgr2=0x7fa4ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa4ec06b4d0 0x7fa4ec06d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4033000 tx=0x7fa4f4008030 comp rx=0 tx=0).stop 2024-04-25T08:38:28.473 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 msgr2=0x7fa500157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 0x7fa500157b20 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000f5f0 tx=0x7fa4f000fab0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 shutdown_connections 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa500158060 0x7fa5001f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5000b4d30 0x7fa500157b20 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa4ec06b4d0 0x7fa4ec06d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 --2- 172.21.15.60:0/3037862949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa50014cb20 0x7fa5000cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 >> 172.21.15.60:0/3037862949 conn(0x7fa4e80039b0 msgr2=0x7fa500160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.479+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 shutdown_connections 2024-04-25T08:38:28.474 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.480+0000 7fa5065a5640 1 -- 172.21.15.60:0/3037862949 wait complete. 2024-04-25T08:38:28.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:28.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.593+0000 7f4b0d6c6640 1 Processor -- start 2024-04-25T08:38:28.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b0d6c6640 1 -- start start 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b0d6c6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b0806b580 0x7f4b080cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b0d6c6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b08161960 con 0x7f4b0806b1a0 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b0d6c6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b08161c40 con 0x7f4b080cffa0 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b0d6c6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080cbd50 con 0x7f4b0806b580 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b06ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4b0806b1a0 0x7f4b0806f9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:57586/0 (socket says 172.21.15.60:57586) 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.594+0000 7f4b06ffd640 1 -- 172.21.15.60:0/1424597314 learned_addr learned my addr 172.21.15.60:0/1424597314 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086128866 0 0) 0x7f4b08161960 con 0x7f4b0806b1a0 2024-04-25T08:38:28.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b067fc640 1 --2- 172.21.15.60:0/1424597314 >> v2:172.21.15.100:3300/0 conn(0x7f4b0806b580 0x7f4b080cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af0003520 con 0x7f4b0806b1a0 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 774148838 0 0) 0x7f4af0003520 con 0x7f4b0806b1a0 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 >> v2:172.21.15.100:3300/0 conn(0x7f4b0806b580 msgr2=0x7f4b080cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 --2- 172.21.15.60:0/1424597314 >> v2:172.21.15.100:3300/0 conn(0x7f4b0806b580 0x7f4b080cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 >> v1:172.21.15.100:6789/0 conn(0x7f4b080cffa0 legacy=0x7f4b0815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08161f20 con 0x7f4b0806b1a0 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f4af8001670 con 0x7f4b0806b1a0 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 >> v1:172.21.15.60:6789/0 conn(0x7f4b0806b1a0 legacy=0x7f4b0806f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4af00039b0 0x7f4af0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080cbd50 con 0x7f4af00039b0 2024-04-25T08:38:28.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af000c160 con 0x7f4af00051c0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af000c360 con 0x7f4af0004800 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4af00039b0 0x7f4af0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b077fe640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b067fc640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.595+0000 7f4b06ffd640 1 -- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af0004800 msgr2=0x7f4af0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b06ffd640 1 -- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af00051c0 msgr2=0x7f4af000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4af00051c0 0x7f4af000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b077fe640 1 -- 172.21.15.60:0/1424597314 reap_dead start 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b06ffd640 1 -- 172.21.15.60:0/1424597314 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0003520 con 0x7f4af00039b0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b067fc640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af0004800 0x7f4af0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4af00039b0 0x7f4af0003dd0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c240 tx=0x7f4af0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e556c9edb492081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8007d10 con 0x7f4af00039b0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8005ad0 con 0x7f4af00039b0 2024-04-25T08:38:28.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.596+0000 7f4b05ffb640 1 -- 172.21.15.60:0/1424597314 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8007070 con 0x7f4af00039b0 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/1424597314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4af00039b0 msgr2=0x7f4af0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4af00039b0 0x7f4af0003dd0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c240 tx=0x7f4af0005830 comp rx=0 tx=0).stop 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/1424597314 shutdown_connections 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/1424597314 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4af00039b0 0x7f4af0003dd0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/1424597314 >> 172.21.15.60:0/1424597314 conn(0x7f4b080b2220 msgr2=0x7f4b080b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/1424597314 shutdown_connections 2024-04-25T08:38:28.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.597+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/1424597314 wait complete. 2024-04-25T08:38:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.598+0000 7f4b0d6c6640 1 Processor -- start 2024-04-25T08:38:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.598+0000 7f4b0d6c6640 1 -- start start 2024-04-25T08:38:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b0d6c6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af00039b0 0x7f4b08156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b0d6c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 0x7f4b080c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b0d6c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0815ec30 0x7f4b0815f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b0d6c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08162720 con 0x7f4b080c5460 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b0d6c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080d03f0 con 0x7f4b0815ec30 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b0d6c6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08070330 con 0x7f4af00039b0 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 0x7f4b080c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 0x7f4b080c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:60906/0 (socket says 172.21.15.60:60906) 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 -- 172.21.15.60:0/3316008897 learned_addr learned my addr 172.21.15.60:0/3316008897 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af00039b0 0x7f4b08156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b077fe640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0815ec30 0x7f4b0815f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 -- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af00039b0 msgr2=0x7f4b08156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af00039b0 0x7f4b08156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 -- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0815ec30 msgr2=0x7f4b0815f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0815ec30 0x7f4b0815f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.599+0000 7f4b067fc640 1 -- 172.21.15.60:0/3316008897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080c5eb0 con 0x7f4b080c5460 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.600+0000 7f4b067fc640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 0x7f4b080c58c0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0107c0 tx=0x7f4afc010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.600+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00f040 con 0x7f4b080c5460 2024-04-25T08:38:28.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.600+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc00eb00 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.600+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b080c6190 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.600+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc010e70 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.600+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b080c6750 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.601+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08073990 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.601+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4afc01c400 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.602+0000 7f4aef7fe640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4af406b500 0x7f4af406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.602+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4afc08a640 con 0x7f4b080c5460 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.602+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4af406b500 0x7f4af406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.603+0000 7f4b06ffd640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4af406b500 0x7f4af406d9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4af8033000 tx=0x7f4af802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.606+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4afc018030 con 0x7f4b080c5460 2024-04-25T08:38:28.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.827+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b0806d370 con 0x7f4b080c5460 2024-04-25T08:38:28.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.827+0000 7f4aef7fe640 1 -- 172.21.15.60:0/3316008897 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4afc058c30 con 0x7f4b080c5460 2024-04-25T08:38:28.821 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:28.822 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:28.822 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4af406b500 msgr2=0x7f4af406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4af406b500 0x7f4af406d9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4af8033000 tx=0x7f4af802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 msgr2=0x7f4b080c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 0x7f4b080c58c0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0107c0 tx=0x7f4afc010c80 comp rx=0 tx=0).stop 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 shutdown_connections 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4af406b500 0x7f4af406d9c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b0815ec30 0x7f4b0815f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b080c5460 0x7f4b080c58c0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 --2- 172.21.15.60:0/3316008897 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4af00039b0 0x7f4b08156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.829+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 >> 172.21.15.60:0/3316008897 conn(0x7f4b080b2220 msgr2=0x7f4b080cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.830+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 shutdown_connections 2024-04-25T08:38:28.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:28.830+0000 7f4b0d6c6640 1 -- 172.21.15.60:0/3316008897 wait complete. 2024-04-25T08:38:28.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:28.835 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:28.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943172 s, 111 MB/s 2024-04-25T08:38:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762525 s, 138 MB/s 2024-04-25T08:38:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661948 s, 158 MB/s 2024-04-25T08:38:28.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.899 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082588 s, 127 MB/s 2024-04-25T08:38:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.908 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789357 s, 133 MB/s 2024-04-25T08:38:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.917 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816175 s, 128 MB/s 2024-04-25T08:38:28.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.925 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740342 s, 142 MB/s 2024-04-25T08:38:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.932 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572622 s, 183 MB/s 2024-04-25T08:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632155 s, 166 MB/s 2024-04-25T08:38:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772618 s, 136 MB/s 2024-04-25T08:38:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.955 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670202 s, 156 MB/s 2024-04-25T08:38:28.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.963 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006448 s, 163 MB/s 2024-04-25T08:38:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.971 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746365 s, 140 MB/s 2024-04-25T08:38:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.978 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647963 s, 162 MB/s 2024-04-25T08:38:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676459 s, 155 MB/s 2024-04-25T08:38:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:28.994 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721288 s, 145 MB/s 2024-04-25T08:38:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661912 s, 158 MB/s 2024-04-25T08:38:29.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063071 s, 166 MB/s 2024-04-25T08:38:29.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633397 s, 166 MB/s 2024-04-25T08:38:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678048 s, 155 MB/s 2024-04-25T08:38:29.032 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.032 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.032 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736162 s, 142 MB/s 2024-04-25T08:38:29.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.039 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665831 s, 157 MB/s 2024-04-25T08:38:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.047 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659456 s, 159 MB/s 2024-04-25T08:38:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.055 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732755 s, 143 MB/s 2024-04-25T08:38:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062533 s, 168 MB/s 2024-04-25T08:38:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691963 s, 152 MB/s 2024-04-25T08:38:29.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.077 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640279 s, 164 MB/s 2024-04-25T08:38:29.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.084 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.084 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642174 s, 163 MB/s 2024-04-25T08:38:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639474 s, 164 MB/s 2024-04-25T08:38:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616406 s, 170 MB/s 2024-04-25T08:38:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.107 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762189 s, 138 MB/s 2024-04-25T08:38:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655401 s, 160 MB/s 2024-04-25T08:38:29.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:29.116 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.181 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066117 s, 159 MB/s 2024-04-25T08:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.188 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606929 s, 173 MB/s 2024-04-25T08:38:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567451 s, 185 MB/s 2024-04-25T08:38:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.201 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588269 s, 178 MB/s 2024-04-25T08:38:29.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577667 s, 182 MB/s 2024-04-25T08:38:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.216 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701587 s, 149 MB/s 2024-04-25T08:38:29.223 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.223 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.223 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638405 s, 164 MB/s 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685358 s, 153 MB/s 2024-04-25T08:38:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.238 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616634 s, 170 MB/s 2024-04-25T08:38:29.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792958 s, 132 MB/s 2024-04-25T08:38:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.254 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662096 s, 158 MB/s 2024-04-25T08:38:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733429 s, 143 MB/s 2024-04-25T08:38:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.269 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614947 s, 171 MB/s 2024-04-25T08:38:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656661 s, 160 MB/s 2024-04-25T08:38:29.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.285 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731394 s, 143 MB/s 2024-04-25T08:38:29.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618623 s, 170 MB/s 2024-04-25T08:38:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618704 s, 169 MB/s 2024-04-25T08:38:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.306 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598824 s, 175 MB/s 2024-04-25T08:38:29.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.313 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.313 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062293 s, 168 MB/s 2024-04-25T08:38:29.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700272 s, 150 MB/s 2024-04-25T08:38:29.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683421 s, 153 MB/s 2024-04-25T08:38:29.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722565 s, 145 MB/s 2024-04-25T08:38:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641675 s, 163 MB/s 2024-04-25T08:38:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647206 s, 162 MB/s 2024-04-25T08:38:29.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613062 s, 171 MB/s 2024-04-25T08:38:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636221 s, 165 MB/s 2024-04-25T08:38:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.374 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742173 s, 141 MB/s 2024-04-25T08:38:29.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696928 s, 150 MB/s 2024-04-25T08:38:29.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653785 s, 160 MB/s 2024-04-25T08:38:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652539 s, 161 MB/s 2024-04-25T08:38:29.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614169 s, 171 MB/s 2024-04-25T08:38:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703343 s, 149 MB/s 2024-04-25T08:38:29.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:29.413 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:29.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574942 s, 182 MB/s 2024-04-25T08:38:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.485 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693646 s, 151 MB/s 2024-04-25T08:38:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.492 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064306 s, 163 MB/s 2024-04-25T08:38:29.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.499 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064329 s, 163 MB/s 2024-04-25T08:38:29.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.506 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626899 s, 167 MB/s 2024-04-25T08:38:29.513 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.513 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.513 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617545 s, 170 MB/s 2024-04-25T08:38:29.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.521 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666776 s, 157 MB/s 2024-04-25T08:38:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.529 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705157 s, 149 MB/s 2024-04-25T08:38:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655742 s, 160 MB/s 2024-04-25T08:38:29.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629376 s, 167 MB/s 2024-04-25T08:38:29.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.550 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570128 s, 184 MB/s 2024-04-25T08:38:29.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.557 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622092 s, 169 MB/s 2024-04-25T08:38:29.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.565 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.565 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712236 s, 147 MB/s 2024-04-25T08:38:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611039 s, 172 MB/s 2024-04-25T08:38:29.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.580 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659941 s, 159 MB/s 2024-04-25T08:38:29.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.588 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709262 s, 148 MB/s 2024-04-25T08:38:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.595 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064135 s, 163 MB/s 2024-04-25T08:38:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622902 s, 168 MB/s 2024-04-25T08:38:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620247 s, 169 MB/s 2024-04-25T08:38:29.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.617 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649698 s, 161 MB/s 2024-04-25T08:38:29.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.624 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623146 s, 168 MB/s 2024-04-25T08:38:29.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.631 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609036 s, 172 MB/s 2024-04-25T08:38:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638402 s, 164 MB/s 2024-04-25T08:38:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614399 s, 171 MB/s 2024-04-25T08:38:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.652 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589911 s, 178 MB/s 2024-04-25T08:38:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615632 s, 170 MB/s 2024-04-25T08:38:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645982 s, 162 MB/s 2024-04-25T08:38:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064587 s, 162 MB/s 2024-04-25T08:38:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594361 s, 176 MB/s 2024-04-25T08:38:29.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.688 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644314 s, 163 MB/s 2024-04-25T08:38:29.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.696 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.696 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706676 s, 148 MB/s 2024-04-25T08:38:29.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.703 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617752 s, 170 MB/s 2024-04-25T08:38:29.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:29.704 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:29.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604353 s, 174 MB/s 2024-04-25T08:38:29.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616832 s, 170 MB/s 2024-04-25T08:38:29.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620271 s, 169 MB/s 2024-04-25T08:38:29.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.789 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585557 s, 179 MB/s 2024-04-25T08:38:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.796 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638135 s, 164 MB/s 2024-04-25T08:38:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585374 s, 179 MB/s 2024-04-25T08:38:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630509 s, 166 MB/s 2024-04-25T08:38:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.818 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072065 s, 146 MB/s 2024-04-25T08:38:29.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.825 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062564 s, 168 MB/s 2024-04-25T08:38:29.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702423 s, 149 MB/s 2024-04-25T08:38:29.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.840 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604623 s, 173 MB/s 2024-04-25T08:38:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.846 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057596 s, 182 MB/s 2024-04-25T08:38:29.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.855 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699947 s, 150 MB/s 2024-04-25T08:38:29.864 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.864 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.864 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714229 s, 147 MB/s 2024-04-25T08:38:29.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650881 s, 161 MB/s 2024-04-25T08:38:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055101 s, 190 MB/s 2024-04-25T08:38:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689925 s, 152 MB/s 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705113 s, 149 MB/s 2024-04-25T08:38:29.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.899 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.899 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629876 s, 166 MB/s 2024-04-25T08:38:29.906 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.907 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627237 s, 167 MB/s 2024-04-25T08:38:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642639 s, 163 MB/s 2024-04-25T08:38:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628726 s, 167 MB/s 2024-04-25T08:38:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586933 s, 179 MB/s 2024-04-25T08:38:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007008 s, 150 MB/s 2024-04-25T08:38:29.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694416 s, 151 MB/s 2024-04-25T08:38:29.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694958 s, 151 MB/s 2024-04-25T08:38:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.959 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625431 s, 168 MB/s 2024-04-25T08:38:29.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.966 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637751 s, 164 MB/s 2024-04-25T08:38:29.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.974 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692585 s, 151 MB/s 2024-04-25T08:38:29.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.981 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692442 s, 151 MB/s 2024-04-25T08:38:29.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.988 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613803 s, 171 MB/s 2024-04-25T08:38:29.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:29.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:29.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720283 s, 146 MB/s 2024-04-25T08:38:29.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:29.998 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:30.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.063 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064973 s, 161 MB/s 2024-04-25T08:38:30.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.069 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605312 s, 173 MB/s 2024-04-25T08:38:30.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.076 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005657 s, 185 MB/s 2024-04-25T08:38:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625191 s, 168 MB/s 2024-04-25T08:38:30.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637034 s, 165 MB/s 2024-04-25T08:38:30.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.098 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649813 s, 161 MB/s 2024-04-25T08:38:30.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573619 s, 183 MB/s 2024-04-25T08:38:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.112 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619054 s, 169 MB/s 2024-04-25T08:38:30.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.119 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612957 s, 171 MB/s 2024-04-25T08:38:30.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.127 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695484 s, 151 MB/s 2024-04-25T08:38:30.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568328 s, 185 MB/s 2024-04-25T08:38:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.140 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595422 s, 176 MB/s 2024-04-25T08:38:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689471 s, 152 MB/s 2024-04-25T08:38:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.156 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718221 s, 146 MB/s 2024-04-25T08:38:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.164 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709876 s, 148 MB/s 2024-04-25T08:38:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665747 s, 158 MB/s 2024-04-25T08:38:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.178 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606709 s, 173 MB/s 2024-04-25T08:38:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.185 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611174 s, 172 MB/s 2024-04-25T08:38:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716094 s, 146 MB/s 2024-04-25T08:38:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.200 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592235 s, 177 MB/s 2024-04-25T08:38:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574407 s, 183 MB/s 2024-04-25T08:38:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606315 s, 173 MB/s 2024-04-25T08:38:30.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738654 s, 142 MB/s 2024-04-25T08:38:30.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.229 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643414 s, 163 MB/s 2024-04-25T08:38:30.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619372 s, 169 MB/s 2024-04-25T08:38:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575549 s, 182 MB/s 2024-04-25T08:38:30.252 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646238 s, 162 MB/s 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633606 s, 165 MB/s 2024-04-25T08:38:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.265 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630884 s, 166 MB/s 2024-04-25T08:38:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.272 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618596 s, 170 MB/s 2024-04-25T08:38:30.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.279 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604751 s, 173 MB/s 2024-04-25T08:38:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579253 s, 181 MB/s 2024-04-25T08:38:30.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:30.287 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.352 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666826 s, 157 MB/s 2024-04-25T08:38:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.359 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061014 s, 172 MB/s 2024-04-25T08:38:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607763 s, 173 MB/s 2024-04-25T08:38:30.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737526 s, 142 MB/s 2024-04-25T08:38:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576542 s, 182 MB/s 2024-04-25T08:38:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.388 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603995 s, 174 MB/s 2024-04-25T08:38:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582987 s, 180 MB/s 2024-04-25T08:38:30.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564631 s, 186 MB/s 2024-04-25T08:38:30.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071791 s, 146 MB/s 2024-04-25T08:38:30.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.418 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721063 s, 145 MB/s 2024-04-25T08:38:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657362 s, 160 MB/s 2024-04-25T08:38:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.432 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614361 s, 171 MB/s 2024-04-25T08:38:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.439 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623484 s, 168 MB/s 2024-04-25T08:38:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.449 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651562 s, 161 MB/s 2024-04-25T08:38:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628949 s, 167 MB/s 2024-04-25T08:38:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649508 s, 161 MB/s 2024-04-25T08:38:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.469 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710312 s, 148 MB/s 2024-04-25T08:38:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068516 s, 153 MB/s 2024-04-25T08:38:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.485 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693946 s, 151 MB/s 2024-04-25T08:38:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.492 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633717 s, 165 MB/s 2024-04-25T08:38:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.500 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694353 s, 151 MB/s 2024-04-25T08:38:30.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.507 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.507 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593206 s, 177 MB/s 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733888 s, 143 MB/s 2024-04-25T08:38:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.521 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560327 s, 187 MB/s 2024-04-25T08:38:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.529 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639006 s, 164 MB/s 2024-04-25T08:38:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.537 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707857 s, 148 MB/s 2024-04-25T08:38:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.545 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710131 s, 148 MB/s 2024-04-25T08:38:30.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.553 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725824 s, 144 MB/s 2024-04-25T08:38:30.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.560 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624439 s, 168 MB/s 2024-04-25T08:38:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.567 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649127 s, 162 MB/s 2024-04-25T08:38:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.574 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566195 s, 185 MB/s 2024-04-25T08:38:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700212 s, 150 MB/s 2024-04-25T08:38:30.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:30.583 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:30.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.649 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629394 s, 167 MB/s 2024-04-25T08:38:30.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662731 s, 158 MB/s 2024-04-25T08:38:30.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.663 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649225 s, 162 MB/s 2024-04-25T08:38:30.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560786 s, 187 MB/s 2024-04-25T08:38:30.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.677 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640141 s, 164 MB/s 2024-04-25T08:38:30.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.684 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649704 s, 161 MB/s 2024-04-25T08:38:30.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.691 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629483 s, 167 MB/s 2024-04-25T08:38:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703963 s, 149 MB/s 2024-04-25T08:38:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707112 s, 148 MB/s 2024-04-25T08:38:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667901 s, 157 MB/s 2024-04-25T08:38:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582458 s, 180 MB/s 2024-04-25T08:38:30.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624659 s, 168 MB/s 2024-04-25T08:38:30.735 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.736 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613364 s, 171 MB/s 2024-04-25T08:38:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571064 s, 184 MB/s 2024-04-25T08:38:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640102 s, 164 MB/s 2024-04-25T08:38:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594421 s, 176 MB/s 2024-04-25T08:38:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626176 s, 167 MB/s 2024-04-25T08:38:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638315 s, 164 MB/s 2024-04-25T08:38:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554217 s, 189 MB/s 2024-04-25T08:38:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621644 s, 169 MB/s 2024-04-25T08:38:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650444 s, 161 MB/s 2024-04-25T08:38:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572082 s, 183 MB/s 2024-04-25T08:38:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702309 s, 149 MB/s 2024-04-25T08:38:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567231 s, 185 MB/s 2024-04-25T08:38:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648102 s, 162 MB/s 2024-04-25T08:38:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664739 s, 158 MB/s 2024-04-25T08:38:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727233 s, 144 MB/s 2024-04-25T08:38:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630251 s, 166 MB/s 2024-04-25T08:38:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614828 s, 171 MB/s 2024-04-25T08:38:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063116 s, 166 MB/s 2024-04-25T08:38:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695365 s, 151 MB/s 2024-04-25T08:38:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699919 s, 150 MB/s 2024-04-25T08:38:30.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:30.875 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:30.875 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:30.875 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:38:30.875 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:30.875 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:30.875 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:30.876 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.940 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581066 s, 180 MB/s 2024-04-25T08:38:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720159 s, 146 MB/s 2024-04-25T08:38:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664052 s, 158 MB/s 2024-04-25T08:38:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644542 s, 163 MB/s 2024-04-25T08:38:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.970 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569282 s, 184 MB/s 2024-04-25T08:38:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.977 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609147 s, 172 MB/s 2024-04-25T08:38:30.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.984 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591396 s, 177 MB/s 2024-04-25T08:38:30.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.991 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.991 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006395 s, 164 MB/s 2024-04-25T08:38:30.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:30.999 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690527 s, 152 MB/s 2024-04-25T08:38:31.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.006 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620859 s, 169 MB/s 2024-04-25T08:38:31.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.014 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701917 s, 149 MB/s 2024-04-25T08:38:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644059 s, 163 MB/s 2024-04-25T08:38:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.028 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611076 s, 172 MB/s 2024-04-25T08:38:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.035 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649637 s, 161 MB/s 2024-04-25T08:38:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.042 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579958 s, 181 MB/s 2024-04-25T08:38:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.049 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619389 s, 169 MB/s 2024-04-25T08:38:31.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.056 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567807 s, 185 MB/s 2024-04-25T08:38:31.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562037 s, 187 MB/s 2024-04-25T08:38:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703108 s, 149 MB/s 2024-04-25T08:38:31.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.077 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.077 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057644 s, 182 MB/s 2024-04-25T08:38:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.085 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700525 s, 150 MB/s 2024-04-25T08:38:31.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648448 s, 162 MB/s 2024-04-25T08:38:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580179 s, 181 MB/s 2024-04-25T08:38:31.106 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.106 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.106 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613513 s, 171 MB/s 2024-04-25T08:38:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614244 s, 171 MB/s 2024-04-25T08:38:31.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.121 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695937 s, 151 MB/s 2024-04-25T08:38:31.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.129 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698109 s, 150 MB/s 2024-04-25T08:38:31.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.136 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602447 s, 174 MB/s 2024-04-25T08:38:31.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.143 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607507 s, 173 MB/s 2024-04-25T08:38:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.150 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699277 s, 150 MB/s 2024-04-25T08:38:31.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.158 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626433 s, 167 MB/s 2024-04-25T08:38:31.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.165 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638512 s, 164 MB/s 2024-04-25T08:38:31.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:31.166 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.231 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629163 s, 167 MB/s 2024-04-25T08:38:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.238 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592505 s, 177 MB/s 2024-04-25T08:38:31.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.245 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606437 s, 173 MB/s 2024-04-25T08:38:31.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.252 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059603 s, 176 MB/s 2024-04-25T08:38:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.259 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707646 s, 148 MB/s 2024-04-25T08:38:31.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.267 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711213 s, 147 MB/s 2024-04-25T08:38:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.274 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.275 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615655 s, 170 MB/s 2024-04-25T08:38:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601673 s, 174 MB/s 2024-04-25T08:38:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701911 s, 149 MB/s 2024-04-25T08:38:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.296 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640145 s, 164 MB/s 2024-04-25T08:38:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.303 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600273 s, 175 MB/s 2024-04-25T08:38:31.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634917 s, 165 MB/s 2024-04-25T08:38:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.317 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593921 s, 177 MB/s 2024-04-25T08:38:31.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.324 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.324 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551956 s, 190 MB/s 2024-04-25T08:38:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569218 s, 184 MB/s 2024-04-25T08:38:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633324 s, 166 MB/s 2024-04-25T08:38:31.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554382 s, 189 MB/s 2024-04-25T08:38:31.350 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551718 s, 190 MB/s 2024-04-25T08:38:31.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629605 s, 167 MB/s 2024-04-25T08:38:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.365 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629589 s, 167 MB/s 2024-04-25T08:38:31.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.372 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.372 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622443 s, 168 MB/s 2024-04-25T08:38:31.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555358 s, 189 MB/s 2024-04-25T08:38:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.386 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631952 s, 166 MB/s 2024-04-25T08:38:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.393 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612147 s, 171 MB/s 2024-04-25T08:38:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630436 s, 166 MB/s 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069334 s, 151 MB/s 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610297 s, 172 MB/s 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659594 s, 159 MB/s 2024-04-25T08:38:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.429 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057106 s, 184 MB/s 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696768 s, 150 MB/s 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.445 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.445 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707807 s, 148 MB/s 2024-04-25T08:38:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.452 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611199 s, 172 MB/s 2024-04-25T08:38:31.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:31.453 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:31.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675634 s, 155 MB/s 2024-04-25T08:38:31.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.527 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673782 s, 156 MB/s 2024-04-25T08:38:31.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.534 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630552 s, 166 MB/s 2024-04-25T08:38:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592252 s, 177 MB/s 2024-04-25T08:38:31.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.548 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642245 s, 163 MB/s 2024-04-25T08:38:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.555 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598538 s, 175 MB/s 2024-04-25T08:38:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.563 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726853 s, 144 MB/s 2024-04-25T08:38:31.570 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.571 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683269 s, 153 MB/s 2024-04-25T08:38:31.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.578 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651117 s, 161 MB/s 2024-04-25T08:38:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.585 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063195 s, 166 MB/s 2024-04-25T08:38:31.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.592 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626978 s, 167 MB/s 2024-04-25T08:38:31.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.600 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.600 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632657 s, 166 MB/s 2024-04-25T08:38:31.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.607 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612136 s, 171 MB/s 2024-04-25T08:38:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056428 s, 186 MB/s 2024-04-25T08:38:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627286 s, 167 MB/s 2024-04-25T08:38:31.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624148 s, 168 MB/s 2024-04-25T08:38:31.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.636 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.636 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073311 s, 143 MB/s 2024-04-25T08:38:31.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702679 s, 149 MB/s 2024-04-25T08:38:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710279 s, 148 MB/s 2024-04-25T08:38:31.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616156 s, 170 MB/s 2024-04-25T08:38:31.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647655 s, 162 MB/s 2024-04-25T08:38:31.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.673 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621368 s, 169 MB/s 2024-04-25T08:38:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057137 s, 184 MB/s 2024-04-25T08:38:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.688 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719592 s, 146 MB/s 2024-04-25T08:38:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.695 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610886 s, 172 MB/s 2024-04-25T08:38:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.702 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631706 s, 166 MB/s 2024-04-25T08:38:31.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.710 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686828 s, 153 MB/s 2024-04-25T08:38:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.717 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599598 s, 175 MB/s 2024-04-25T08:38:31.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572979 s, 183 MB/s 2024-04-25T08:38:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625844 s, 168 MB/s 2024-04-25T08:38:31.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557472 s, 188 MB/s 2024-04-25T08:38:31.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637643 s, 164 MB/s 2024-04-25T08:38:31.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.745 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:31.745 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:31.746 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:38:31.746 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:31.746 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:31.746 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:31.746 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:31.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600379 s, 175 MB/s 2024-04-25T08:38:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.818 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684362 s, 153 MB/s 2024-04-25T08:38:31.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.825 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.825 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654211 s, 160 MB/s 2024-04-25T08:38:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.833 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071787 s, 146 MB/s 2024-04-25T08:38:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.840 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059989 s, 175 MB/s 2024-04-25T08:38:31.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.847 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605971 s, 173 MB/s 2024-04-25T08:38:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.854 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604048 s, 174 MB/s 2024-04-25T08:38:31.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.862 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720397 s, 146 MB/s 2024-04-25T08:38:31.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642227 s, 163 MB/s 2024-04-25T08:38:31.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623893 s, 168 MB/s 2024-04-25T08:38:31.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607219 s, 173 MB/s 2024-04-25T08:38:31.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605521 s, 173 MB/s 2024-04-25T08:38:31.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628267 s, 167 MB/s 2024-04-25T08:38:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656328 s, 160 MB/s 2024-04-25T08:38:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574401 s, 183 MB/s 2024-04-25T08:38:31.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657679 s, 159 MB/s 2024-04-25T08:38:31.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.926 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625135 s, 168 MB/s 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557138 s, 188 MB/s 2024-04-25T08:38:31.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552256 s, 190 MB/s 2024-04-25T08:38:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.946 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059497 s, 176 MB/s 2024-04-25T08:38:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.953 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619385 s, 169 MB/s 2024-04-25T08:38:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558178 s, 188 MB/s 2024-04-25T08:38:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725221 s, 145 MB/s 2024-04-25T08:38:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652429 s, 161 MB/s 2024-04-25T08:38:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589124 s, 178 MB/s 2024-04-25T08:38:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605055 s, 173 MB/s 2024-04-25T08:38:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:31.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629583 s, 167 MB/s 2024-04-25T08:38:32.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691373 s, 152 MB/s 2024-04-25T08:38:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.011 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594991 s, 176 MB/s 2024-04-25T08:38:32.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.019 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.019 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637118 s, 165 MB/s 2024-04-25T08:38:32.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641942 s, 163 MB/s 2024-04-25T08:38:32.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.034 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712961 s, 147 MB/s 2024-04-25T08:38:32.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.035 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:32.035 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:32.035 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:38:32.035 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:32.035 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:32.036 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:32.036 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.101 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654498 s, 160 MB/s 2024-04-25T08:38:32.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.107 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059238 s, 177 MB/s 2024-04-25T08:38:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669296 s, 157 MB/s 2024-04-25T08:38:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.121 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056484 s, 186 MB/s 2024-04-25T08:38:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651423 s, 161 MB/s 2024-04-25T08:38:32.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.136 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617357 s, 170 MB/s 2024-04-25T08:38:32.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.143 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621494 s, 169 MB/s 2024-04-25T08:38:32.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.151 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699547 s, 150 MB/s 2024-04-25T08:38:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630034 s, 166 MB/s 2024-04-25T08:38:32.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.165 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618181 s, 170 MB/s 2024-04-25T08:38:32.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.172 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062891 s, 167 MB/s 2024-04-25T08:38:32.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.179 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640271 s, 164 MB/s 2024-04-25T08:38:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.187 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070003 s, 150 MB/s 2024-04-25T08:38:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633767 s, 165 MB/s 2024-04-25T08:38:32.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.201 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.201 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567429 s, 185 MB/s 2024-04-25T08:38:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559244 s, 187 MB/s 2024-04-25T08:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629416 s, 167 MB/s 2024-04-25T08:38:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648312 s, 162 MB/s 2024-04-25T08:38:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.229 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632254 s, 166 MB/s 2024-04-25T08:38:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645116 s, 163 MB/s 2024-04-25T08:38:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611566 s, 171 MB/s 2024-04-25T08:38:32.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.252 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731076 s, 143 MB/s 2024-04-25T08:38:32.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.259 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602158 s, 174 MB/s 2024-04-25T08:38:32.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.266 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627356 s, 167 MB/s 2024-04-25T08:38:32.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.273 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653384 s, 160 MB/s 2024-04-25T08:38:32.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707053 s, 148 MB/s 2024-04-25T08:38:32.288 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655933 s, 160 MB/s 2024-04-25T08:38:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.295 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593872 s, 177 MB/s 2024-04-25T08:38:32.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.302 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584214 s, 179 MB/s 2024-04-25T08:38:32.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.309 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560855 s, 187 MB/s 2024-04-25T08:38:32.315 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061805 s, 170 MB/s 2024-04-25T08:38:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.323 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692765 s, 151 MB/s 2024-04-25T08:38:32.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:32.325 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584204 s, 179 MB/s 2024-04-25T08:38:32.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701507 s, 149 MB/s 2024-04-25T08:38:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631015 s, 166 MB/s 2024-04-25T08:38:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720447 s, 146 MB/s 2024-04-25T08:38:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.420 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070758 s, 148 MB/s 2024-04-25T08:38:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.427 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.428 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684745 s, 153 MB/s 2024-04-25T08:38:32.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.435 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.435 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642736 s, 163 MB/s 2024-04-25T08:38:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.443 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703066 s, 149 MB/s 2024-04-25T08:38:32.450 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694761 s, 151 MB/s 2024-04-25T08:38:32.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.458 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636339 s, 165 MB/s 2024-04-25T08:38:32.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.465 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668329 s, 157 MB/s 2024-04-25T08:38:32.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.472 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608913 s, 172 MB/s 2024-04-25T08:38:32.480 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.480 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.480 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739962 s, 142 MB/s 2024-04-25T08:38:32.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.488 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638302 s, 164 MB/s 2024-04-25T08:38:32.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602706 s, 174 MB/s 2024-04-25T08:38:32.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717433 s, 146 MB/s 2024-04-25T08:38:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064745 s, 162 MB/s 2024-04-25T08:38:32.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616853 s, 170 MB/s 2024-04-25T08:38:32.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590634 s, 178 MB/s 2024-04-25T08:38:32.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.531 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.531 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613656 s, 171 MB/s 2024-04-25T08:38:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.539 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722976 s, 145 MB/s 2024-04-25T08:38:32.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.545 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.545 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565141 s, 186 MB/s 2024-04-25T08:38:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.552 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055655 s, 188 MB/s 2024-04-25T08:38:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.560 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703585 s, 149 MB/s 2024-04-25T08:38:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717191 s, 146 MB/s 2024-04-25T08:38:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.576 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719036 s, 146 MB/s 2024-04-25T08:38:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.583 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00551087 s, 190 MB/s 2024-04-25T08:38:32.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632485 s, 166 MB/s 2024-04-25T08:38:32.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622148 s, 169 MB/s 2024-04-25T08:38:32.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698653 s, 150 MB/s 2024-04-25T08:38:32.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633257 s, 166 MB/s 2024-04-25T08:38:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649867 s, 161 MB/s 2024-04-25T08:38:32.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.620 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:32.621 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:32.621 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:38:32.621 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:32.621 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:32.621 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:32.621 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745339 s, 141 MB/s 2024-04-25T08:38:32.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630491 s, 166 MB/s 2024-04-25T08:38:32.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638033 s, 164 MB/s 2024-04-25T08:38:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593452 s, 177 MB/s 2024-04-25T08:38:32.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624031 s, 168 MB/s 2024-04-25T08:38:32.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.721 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057355 s, 183 MB/s 2024-04-25T08:38:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.729 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689592 s, 152 MB/s 2024-04-25T08:38:32.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.736 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611871 s, 171 MB/s 2024-04-25T08:38:32.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718343 s, 146 MB/s 2024-04-25T08:38:32.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583215 s, 180 MB/s 2024-04-25T08:38:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.759 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680787 s, 154 MB/s 2024-04-25T08:38:32.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.767 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.767 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707824 s, 148 MB/s 2024-04-25T08:38:32.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.774 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.774 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687236 s, 153 MB/s 2024-04-25T08:38:32.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.781 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563044 s, 186 MB/s 2024-04-25T08:38:32.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.788 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617184 s, 170 MB/s 2024-04-25T08:38:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605548 s, 173 MB/s 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683742 s, 153 MB/s 2024-04-25T08:38:32.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638235 s, 164 MB/s 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635368 s, 165 MB/s 2024-04-25T08:38:32.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611253 s, 172 MB/s 2024-04-25T08:38:32.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.831 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.831 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620644 s, 169 MB/s 2024-04-25T08:38:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.839 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711761 s, 147 MB/s 2024-04-25T08:38:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.846 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617376 s, 170 MB/s 2024-04-25T08:38:32.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.853 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625538 s, 168 MB/s 2024-04-25T08:38:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.862 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793428 s, 132 MB/s 2024-04-25T08:38:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.870 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659994 s, 159 MB/s 2024-04-25T08:38:32.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.878 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719965 s, 146 MB/s 2024-04-25T08:38:32.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.885 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.885 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621538 s, 169 MB/s 2024-04-25T08:38:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.893 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731846 s, 143 MB/s 2024-04-25T08:38:32.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.901 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.901 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630116 s, 166 MB/s 2024-04-25T08:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.910 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791347 s, 133 MB/s 2024-04-25T08:38:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.922 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011304 s, 92.8 MB/s 2024-04-25T08:38:32.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.923 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:32.923 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:32.923 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:38:32.924 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:32.924 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:32.924 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:32.924 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655316 s, 160 MB/s 2024-04-25T08:38:32.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:32.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:32.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699021 s, 150 MB/s 2024-04-25T08:38:33.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.003 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628787 s, 167 MB/s 2024-04-25T08:38:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.010 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609272 s, 172 MB/s 2024-04-25T08:38:33.017 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625057 s, 168 MB/s 2024-04-25T08:38:33.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.025 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604363 s, 174 MB/s 2024-04-25T08:38:33.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.031 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599975 s, 175 MB/s 2024-04-25T08:38:33.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614286 s, 171 MB/s 2024-04-25T08:38:33.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608338 s, 172 MB/s 2024-04-25T08:38:33.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.053 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643826 s, 163 MB/s 2024-04-25T08:38:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.060 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575903 s, 182 MB/s 2024-04-25T08:38:33.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.066 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608182 s, 172 MB/s 2024-04-25T08:38:33.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.073 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586762 s, 179 MB/s 2024-04-25T08:38:33.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.081 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697315 s, 150 MB/s 2024-04-25T08:38:33.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609254 s, 172 MB/s 2024-04-25T08:38:33.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.097 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.097 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727758 s, 144 MB/s 2024-04-25T08:38:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.104 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654652 s, 160 MB/s 2024-04-25T08:38:33.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643868 s, 163 MB/s 2024-04-25T08:38:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.119 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701966 s, 149 MB/s 2024-04-25T08:38:33.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.126 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057531 s, 182 MB/s 2024-04-25T08:38:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.134 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741707 s, 141 MB/s 2024-04-25T08:38:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688014 s, 152 MB/s 2024-04-25T08:38:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.149 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680318 s, 154 MB/s 2024-04-25T08:38:33.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.157 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699665 s, 150 MB/s 2024-04-25T08:38:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.164 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622504 s, 168 MB/s 2024-04-25T08:38:33.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.172 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721233 s, 145 MB/s 2024-04-25T08:38:33.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.179 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.180 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586501 s, 179 MB/s 2024-04-25T08:38:33.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605341 s, 173 MB/s 2024-04-25T08:38:33.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705748 s, 149 MB/s 2024-04-25T08:38:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685245 s, 153 MB/s 2024-04-25T08:38:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.209 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654551 s, 160 MB/s 2024-04-25T08:38:33.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621698 s, 169 MB/s 2024-04-25T08:38:33.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:33.218 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.283 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634992 s, 165 MB/s 2024-04-25T08:38:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589811 s, 178 MB/s 2024-04-25T08:38:33.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.296 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644739 s, 163 MB/s 2024-04-25T08:38:33.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.309 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120611 s, 86.9 MB/s 2024-04-25T08:38:33.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.317 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689111 s, 152 MB/s 2024-04-25T08:38:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.325 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071612 s, 146 MB/s 2024-04-25T08:38:33.332 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.332 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.332 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622417 s, 168 MB/s 2024-04-25T08:38:33.339 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642311 s, 163 MB/s 2024-04-25T08:38:33.346 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614301 s, 171 MB/s 2024-04-25T08:38:33.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.355 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713028 s, 147 MB/s 2024-04-25T08:38:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.362 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626346 s, 167 MB/s 2024-04-25T08:38:33.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.369 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590383 s, 178 MB/s 2024-04-25T08:38:33.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575186 s, 182 MB/s 2024-04-25T08:38:33.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.383 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553266 s, 190 MB/s 2024-04-25T08:38:33.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611029 s, 172 MB/s 2024-04-25T08:38:33.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711176 s, 147 MB/s 2024-04-25T08:38:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626964 s, 167 MB/s 2024-04-25T08:38:33.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.412 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.412 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007327 s, 143 MB/s 2024-04-25T08:38:33.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.420 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646774 s, 162 MB/s 2024-04-25T08:38:33.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590058 s, 178 MB/s 2024-04-25T08:38:33.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.434 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.434 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628775 s, 167 MB/s 2024-04-25T08:38:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.441 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701782 s, 149 MB/s 2024-04-25T08:38:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.448 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00546269 s, 192 MB/s 2024-04-25T08:38:33.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.456 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.456 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693191 s, 151 MB/s 2024-04-25T08:38:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.463 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641747 s, 163 MB/s 2024-04-25T08:38:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.471 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695843 s, 151 MB/s 2024-04-25T08:38:33.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056056 s, 187 MB/s 2024-04-25T08:38:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.484 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611913 s, 171 MB/s 2024-04-25T08:38:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.492 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690202 s, 152 MB/s 2024-04-25T08:38:33.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.500 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.500 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714918 s, 147 MB/s 2024-04-25T08:38:33.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634287 s, 165 MB/s 2024-04-25T08:38:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.514 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608174 s, 172 MB/s 2024-04-25T08:38:33.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:33.515 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:33.516 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:33.516 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:38:33.516 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:33.516 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:33.516 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:33.516 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:33.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.581 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.581 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642008 s, 163 MB/s 2024-04-25T08:38:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.588 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696953 s, 150 MB/s 2024-04-25T08:38:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.595 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565143 s, 186 MB/s 2024-04-25T08:38:33.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611138 s, 172 MB/s 2024-04-25T08:38:33.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605244 s, 173 MB/s 2024-04-25T08:38:33.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.616 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656899 s, 160 MB/s 2024-04-25T08:38:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.623 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614354 s, 171 MB/s 2024-04-25T08:38:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.630 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061804 s, 170 MB/s 2024-04-25T08:38:33.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.637 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061265 s, 171 MB/s 2024-04-25T08:38:33.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.644 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572715 s, 183 MB/s 2024-04-25T08:38:33.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607169 s, 173 MB/s 2024-04-25T08:38:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.658 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690654 s, 152 MB/s 2024-04-25T08:38:33.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.665 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562606 s, 186 MB/s 2024-04-25T08:38:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.672 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635002 s, 165 MB/s 2024-04-25T08:38:33.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644204 s, 163 MB/s 2024-04-25T08:38:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555261 s, 189 MB/s 2024-04-25T08:38:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709465 s, 148 MB/s 2024-04-25T08:38:33.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633494 s, 166 MB/s 2024-04-25T08:38:33.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699721 s, 150 MB/s 2024-04-25T08:38:33.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583121 s, 180 MB/s 2024-04-25T08:38:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603855 s, 174 MB/s 2024-04-25T08:38:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.730 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605266 s, 173 MB/s 2024-04-25T08:38:33.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.737 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602981 s, 174 MB/s 2024-04-25T08:38:33.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.745 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695917 s, 151 MB/s 2024-04-25T08:38:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.752 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655293 s, 160 MB/s 2024-04-25T08:38:33.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.759 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617364 s, 170 MB/s 2024-04-25T08:38:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.766 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604206 s, 174 MB/s 2024-04-25T08:38:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.772 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056149 s, 187 MB/s 2024-04-25T08:38:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.779 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589086 s, 178 MB/s 2024-04-25T08:38:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613339 s, 171 MB/s 2024-04-25T08:38:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.793 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588131 s, 178 MB/s 2024-04-25T08:38:33.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.800 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057065 s, 184 MB/s 2024-04-25T08:38:33.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:33.801 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:33.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.866 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654391 s, 160 MB/s 2024-04-25T08:38:33.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.873 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673507 s, 156 MB/s 2024-04-25T08:38:33.881 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.881 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.881 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657987 s, 159 MB/s 2024-04-25T08:38:33.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.889 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738847 s, 142 MB/s 2024-04-25T08:38:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.896 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595168 s, 176 MB/s 2024-04-25T08:38:33.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657703 s, 159 MB/s 2024-04-25T08:38:33.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.910 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589179 s, 178 MB/s 2024-04-25T08:38:33.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.917 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.918 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604096 s, 174 MB/s 2024-04-25T08:38:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.925 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635581 s, 165 MB/s 2024-04-25T08:38:33.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.932 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615368 s, 170 MB/s 2024-04-25T08:38:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607861 s, 173 MB/s 2024-04-25T08:38:33.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.946 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649075 s, 162 MB/s 2024-04-25T08:38:33.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.954 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719318 s, 146 MB/s 2024-04-25T08:38:33.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.961 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613815 s, 171 MB/s 2024-04-25T08:38:33.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.968 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607228 s, 173 MB/s 2024-04-25T08:38:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729768 s, 144 MB/s 2024-04-25T08:38:33.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.984 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633812 s, 165 MB/s 2024-04-25T08:38:33.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.990 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059704 s, 176 MB/s 2024-04-25T08:38:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:33.998 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684408 s, 153 MB/s 2024-04-25T08:38:34.005 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.005 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.005 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609457 s, 172 MB/s 2024-04-25T08:38:34.012 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.013 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.013 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640592 s, 164 MB/s 2024-04-25T08:38:34.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.020 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633991 s, 165 MB/s 2024-04-25T08:38:34.028 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.028 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.028 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694345 s, 151 MB/s 2024-04-25T08:38:34.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.036 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693339 s, 151 MB/s 2024-04-25T08:38:34.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.043 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693212 s, 151 MB/s 2024-04-25T08:38:34.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609683 s, 172 MB/s 2024-04-25T08:38:34.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731339 s, 143 MB/s 2024-04-25T08:38:34.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.067 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689893 s, 152 MB/s 2024-04-25T08:38:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.073 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602489 s, 174 MB/s 2024-04-25T08:38:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.080 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625941 s, 168 MB/s 2024-04-25T08:38:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070573 s, 149 MB/s 2024-04-25T08:38:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637741 s, 164 MB/s 2024-04-25T08:38:34.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.096 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.097 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:34.097 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:38:34.097 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.097 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.097 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:34.097 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.162 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676709 s, 155 MB/s 2024-04-25T08:38:34.169 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.169 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.169 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071098 s, 147 MB/s 2024-04-25T08:38:34.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.177 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.177 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638926 s, 164 MB/s 2024-04-25T08:38:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.184 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696165 s, 151 MB/s 2024-04-25T08:38:34.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.191 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582751 s, 180 MB/s 2024-04-25T08:38:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061805 s, 170 MB/s 2024-04-25T08:38:34.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.206 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716375 s, 146 MB/s 2024-04-25T08:38:34.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630325 s, 166 MB/s 2024-04-25T08:38:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062802 s, 167 MB/s 2024-04-25T08:38:34.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632973 s, 166 MB/s 2024-04-25T08:38:34.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625931 s, 168 MB/s 2024-04-25T08:38:34.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.242 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606373 s, 173 MB/s 2024-04-25T08:38:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.249 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613159 s, 171 MB/s 2024-04-25T08:38:34.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.258 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742607 s, 141 MB/s 2024-04-25T08:38:34.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.265 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608257 s, 172 MB/s 2024-04-25T08:38:34.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.272 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061916 s, 169 MB/s 2024-04-25T08:38:34.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.278 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587184 s, 179 MB/s 2024-04-25T08:38:34.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.285 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609129 s, 172 MB/s 2024-04-25T08:38:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058695 s, 179 MB/s 2024-04-25T08:38:34.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005781 s, 181 MB/s 2024-04-25T08:38:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550153 s, 191 MB/s 2024-04-25T08:38:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.312 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613222 s, 171 MB/s 2024-04-25T08:38:34.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.319 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637592 s, 164 MB/s 2024-04-25T08:38:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616262 s, 170 MB/s 2024-04-25T08:38:34.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.333 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619656 s, 169 MB/s 2024-04-25T08:38:34.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628266 s, 167 MB/s 2024-04-25T08:38:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698961 s, 150 MB/s 2024-04-25T08:38:34.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.356 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.356 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615182 s, 170 MB/s 2024-04-25T08:38:34.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.362 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057228 s, 183 MB/s 2024-04-25T08:38:34.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.370 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653756 s, 160 MB/s 2024-04-25T08:38:34.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.377 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.377 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623556 s, 168 MB/s 2024-04-25T08:38:34.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.383 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057539 s, 182 MB/s 2024-04-25T08:38:34.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.384 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.384 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:34.385 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:38:34.385 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.385 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.385 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:34.385 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.449 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593843 s, 177 MB/s 2024-04-25T08:38:34.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.455 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581 s, 180 MB/s 2024-04-25T08:38:34.462 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.463 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613505 s, 171 MB/s 2024-04-25T08:38:34.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.470 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615641 s, 170 MB/s 2024-04-25T08:38:34.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574162 s, 183 MB/s 2024-04-25T08:38:34.483 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.483 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.483 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603797 s, 174 MB/s 2024-04-25T08:38:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620058 s, 169 MB/s 2024-04-25T08:38:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632304 s, 166 MB/s 2024-04-25T08:38:34.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.505 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.505 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064563 s, 162 MB/s 2024-04-25T08:38:34.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601395 s, 174 MB/s 2024-04-25T08:38:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060321 s, 174 MB/s 2024-04-25T08:38:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617714 s, 170 MB/s 2024-04-25T08:38:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605072 s, 173 MB/s 2024-04-25T08:38:34.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595985 s, 176 MB/s 2024-04-25T08:38:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061305 s, 171 MB/s 2024-04-25T08:38:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624141 s, 168 MB/s 2024-04-25T08:38:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.561 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060972 s, 172 MB/s 2024-04-25T08:38:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.567 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587174 s, 179 MB/s 2024-04-25T08:38:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.575 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625979 s, 168 MB/s 2024-04-25T08:38:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.582 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616551 s, 170 MB/s 2024-04-25T08:38:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.590 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747884 s, 140 MB/s 2024-04-25T08:38:34.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622297 s, 169 MB/s 2024-04-25T08:38:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.605 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072626 s, 144 MB/s 2024-04-25T08:38:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623782 s, 168 MB/s 2024-04-25T08:38:34.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612081 s, 171 MB/s 2024-04-25T08:38:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.626 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603395 s, 174 MB/s 2024-04-25T08:38:34.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.634 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605519 s, 173 MB/s 2024-04-25T08:38:34.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.641 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691563 s, 152 MB/s 2024-04-25T08:38:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.648 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597156 s, 176 MB/s 2024-04-25T08:38:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637561 s, 164 MB/s 2024-04-25T08:38:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.663 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701751 s, 149 MB/s 2024-04-25T08:38:34.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577711 s, 182 MB/s 2024-04-25T08:38:34.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.671 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.671 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:34.671 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:38:34.671 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.671 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.671 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:34.672 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:34.737 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.738 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610806 s, 172 MB/s 2024-04-25T08:38:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608571 s, 172 MB/s 2024-04-25T08:38:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060949 s, 172 MB/s 2024-04-25T08:38:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.758 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635357 s, 165 MB/s 2024-04-25T08:38:34.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.765 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574839 s, 182 MB/s 2024-04-25T08:38:34.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.771 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.771 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592741 s, 177 MB/s 2024-04-25T08:38:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.780 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690941 s, 152 MB/s 2024-04-25T08:38:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.787 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695739 s, 151 MB/s 2024-04-25T08:38:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.796 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736324 s, 142 MB/s 2024-04-25T08:38:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617142 s, 170 MB/s 2024-04-25T08:38:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.809 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056092 s, 187 MB/s 2024-04-25T08:38:34.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005856 s, 179 MB/s 2024-04-25T08:38:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625486 s, 168 MB/s 2024-04-25T08:38:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638862 s, 164 MB/s 2024-04-25T08:38:34.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.838 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692856 s, 151 MB/s 2024-04-25T08:38:34.845 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.845 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.845 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640829 s, 164 MB/s 2024-04-25T08:38:34.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.852 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605801 s, 173 MB/s 2024-04-25T08:38:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074445 s, 141 MB/s 2024-04-25T08:38:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.868 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060013 s, 175 MB/s 2024-04-25T08:38:34.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.875 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692803 s, 151 MB/s 2024-04-25T08:38:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616491 s, 170 MB/s 2024-04-25T08:38:34.888 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.889 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00545192 s, 192 MB/s 2024-04-25T08:38:34.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.896 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624322 s, 168 MB/s 2024-04-25T08:38:34.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.903 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609911 s, 172 MB/s 2024-04-25T08:38:34.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.911 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724839 s, 145 MB/s 2024-04-25T08:38:34.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699717 s, 150 MB/s 2024-04-25T08:38:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708249 s, 148 MB/s 2024-04-25T08:38:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611112 s, 172 MB/s 2024-04-25T08:38:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696351 s, 151 MB/s 2024-04-25T08:38:34.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.950 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695754 s, 151 MB/s 2024-04-25T08:38:34.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061301 s, 171 MB/s 2024-04-25T08:38:34.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:34.964 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:34.964 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694856 s, 151 MB/s 2024-04-25T08:38:34.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.965 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.966 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:34.966 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:38:34.966 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.966 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.966 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:34.966 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:35.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.031 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.031 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608443 s, 172 MB/s 2024-04-25T08:38:35.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.039 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.039 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711677 s, 147 MB/s 2024-04-25T08:38:35.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.046 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.046 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623311 s, 168 MB/s 2024-04-25T08:38:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.053 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607925 s, 172 MB/s 2024-04-25T08:38:35.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.060 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605496 s, 173 MB/s 2024-04-25T08:38:35.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.066 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572897 s, 183 MB/s 2024-04-25T08:38:35.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.073 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.073 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620459 s, 169 MB/s 2024-04-25T08:38:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716655 s, 146 MB/s 2024-04-25T08:38:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557875 s, 188 MB/s 2024-04-25T08:38:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650604 s, 161 MB/s 2024-04-25T08:38:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614818 s, 171 MB/s 2024-04-25T08:38:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586106 s, 179 MB/s 2024-04-25T08:38:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.116 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615248 s, 170 MB/s 2024-04-25T08:38:35.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643031 s, 163 MB/s 2024-04-25T08:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621453 s, 169 MB/s 2024-04-25T08:38:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681393 s, 154 MB/s 2024-04-25T08:38:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.145 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592355 s, 177 MB/s 2024-04-25T08:38:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.152 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624237 s, 168 MB/s 2024-04-25T08:38:35.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621653 s, 169 MB/s 2024-04-25T08:38:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621202 s, 169 MB/s 2024-04-25T08:38:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.174 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720258 s, 146 MB/s 2024-04-25T08:38:35.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648314 s, 162 MB/s 2024-04-25T08:38:35.189 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.189 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710606 s, 148 MB/s 2024-04-25T08:38:35.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.196 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586961 s, 179 MB/s 2024-04-25T08:38:35.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.203 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005995 s, 175 MB/s 2024-04-25T08:38:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614801 s, 171 MB/s 2024-04-25T08:38:35.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564158 s, 186 MB/s 2024-04-25T08:38:35.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607746 s, 173 MB/s 2024-04-25T08:38:35.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.232 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693271 s, 151 MB/s 2024-04-25T08:38:35.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.238 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578096 s, 181 MB/s 2024-04-25T08:38:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.246 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646545 s, 162 MB/s 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605676 s, 173 MB/s 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:35.260 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:35.260 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:35.260 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:38:35.261 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:35.261 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:35.261 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:35.261 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705286 s, 149 MB/s 2024-04-25T08:38:35.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.333 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611077 s, 172 MB/s 2024-04-25T08:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564323 s, 186 MB/s 2024-04-25T08:38:35.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.347 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.348 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642779 s, 163 MB/s 2024-04-25T08:38:35.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.354 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058013 s, 181 MB/s 2024-04-25T08:38:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.361 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616422 s, 170 MB/s 2024-04-25T08:38:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.368 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.369 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647734 s, 162 MB/s 2024-04-25T08:38:35.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.376 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628282 s, 167 MB/s 2024-04-25T08:38:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565307 s, 185 MB/s 2024-04-25T08:38:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.388 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552092 s, 190 MB/s 2024-04-25T08:38:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677618 s, 155 MB/s 2024-04-25T08:38:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.403 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623859 s, 168 MB/s 2024-04-25T08:38:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.411 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069716 s, 150 MB/s 2024-04-25T08:38:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726741 s, 144 MB/s 2024-04-25T08:38:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601225 s, 174 MB/s 2024-04-25T08:38:35.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.433 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603376 s, 174 MB/s 2024-04-25T08:38:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.440 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616813 s, 170 MB/s 2024-04-25T08:38:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.447 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055619 s, 189 MB/s 2024-04-25T08:38:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595622 s, 176 MB/s 2024-04-25T08:38:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624646 s, 168 MB/s 2024-04-25T08:38:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627742 s, 167 MB/s 2024-04-25T08:38:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.475 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618984 s, 169 MB/s 2024-04-25T08:38:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575979 s, 182 MB/s 2024-04-25T08:38:35.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597083 s, 176 MB/s 2024-04-25T08:38:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555074 s, 189 MB/s 2024-04-25T08:38:35.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693922 s, 151 MB/s 2024-04-25T08:38:35.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627011 s, 167 MB/s 2024-04-25T08:38:35.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619526 s, 169 MB/s 2024-04-25T08:38:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574121 s, 183 MB/s 2024-04-25T08:38:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694582 s, 151 MB/s 2024-04-25T08:38:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.539 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613119 s, 171 MB/s 2024-04-25T08:38:35.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.546 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.546 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623253 s, 168 MB/s 2024-04-25T08:38:35.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:35.548 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:35.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633092 s, 166 MB/s 2024-04-25T08:38:35.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585856 s, 179 MB/s 2024-04-25T08:38:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600794 s, 175 MB/s 2024-04-25T08:38:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693997 s, 151 MB/s 2024-04-25T08:38:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622002 s, 169 MB/s 2024-04-25T08:38:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.649 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609164 s, 172 MB/s 2024-04-25T08:38:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630787 s, 166 MB/s 2024-04-25T08:38:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567583 s, 185 MB/s 2024-04-25T08:38:35.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620089 s, 169 MB/s 2024-04-25T08:38:35.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.678 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074756 s, 140 MB/s 2024-04-25T08:38:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.685 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061871 s, 169 MB/s 2024-04-25T08:38:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.692 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609918 s, 172 MB/s 2024-04-25T08:38:35.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588935 s, 178 MB/s 2024-04-25T08:38:35.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.705 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558736 s, 188 MB/s 2024-04-25T08:38:35.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.713 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621935 s, 169 MB/s 2024-04-25T08:38:35.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.719 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555454 s, 189 MB/s 2024-04-25T08:38:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.727 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632217 s, 166 MB/s 2024-04-25T08:38:35.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636671 s, 165 MB/s 2024-04-25T08:38:35.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.741 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.741 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071837 s, 146 MB/s 2024-04-25T08:38:35.749 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.749 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.749 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635477 s, 165 MB/s 2024-04-25T08:38:35.757 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.757 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.757 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751445 s, 140 MB/s 2024-04-25T08:38:35.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.765 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064413 s, 163 MB/s 2024-04-25T08:38:35.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.772 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636515 s, 165 MB/s 2024-04-25T08:38:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555004 s, 189 MB/s 2024-04-25T08:38:35.785 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.785 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.785 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636413 s, 165 MB/s 2024-04-25T08:38:35.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.793 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708536 s, 148 MB/s 2024-04-25T08:38:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704145 s, 149 MB/s 2024-04-25T08:38:35.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605296 s, 173 MB/s 2024-04-25T08:38:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.816 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649502 s, 161 MB/s 2024-04-25T08:38:35.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674717 s, 155 MB/s 2024-04-25T08:38:35.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620004 s, 169 MB/s 2024-04-25T08:38:35.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582827 s, 180 MB/s 2024-04-25T08:38:35.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:35.838 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.903 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059653 s, 176 MB/s 2024-04-25T08:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.911 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718389 s, 146 MB/s 2024-04-25T08:38:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754702 s, 139 MB/s 2024-04-25T08:38:35.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629278 s, 167 MB/s 2024-04-25T08:38:35.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707578 s, 148 MB/s 2024-04-25T08:38:35.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.942 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.942 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648204 s, 162 MB/s 2024-04-25T08:38:35.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.949 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.949 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062276 s, 168 MB/s 2024-04-25T08:38:35.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.956 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.956 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635576 s, 165 MB/s 2024-04-25T08:38:35.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.966 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.966 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630493 s, 166 MB/s 2024-04-25T08:38:35.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.971 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.971 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634483 s, 165 MB/s 2024-04-25T08:38:35.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.978 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.978 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629131 s, 167 MB/s 2024-04-25T08:38:35.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.985 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649011 s, 162 MB/s 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060882 s, 172 MB/s 2024-04-25T08:38:36.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.000 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.000 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702924 s, 149 MB/s 2024-04-25T08:38:36.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.007 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629401 s, 167 MB/s 2024-04-25T08:38:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.015 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628123 s, 167 MB/s 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.022 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625026 s, 168 MB/s 2024-04-25T08:38:36.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730679 s, 144 MB/s 2024-04-25T08:38:36.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.037 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645516 s, 162 MB/s 2024-04-25T08:38:36.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.044 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061957 s, 169 MB/s 2024-04-25T08:38:36.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.051 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603104 s, 174 MB/s 2024-04-25T08:38:36.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.058 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061456 s, 171 MB/s 2024-04-25T08:38:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.065 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610748 s, 172 MB/s 2024-04-25T08:38:36.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602704 s, 174 MB/s 2024-04-25T08:38:36.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605092 s, 173 MB/s 2024-04-25T08:38:36.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.087 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645664 s, 162 MB/s 2024-04-25T08:38:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.094 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638964 s, 164 MB/s 2024-04-25T08:38:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729221 s, 144 MB/s 2024-04-25T08:38:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.110 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700638 s, 150 MB/s 2024-04-25T08:38:36.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.117 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601169 s, 174 MB/s 2024-04-25T08:38:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643202 s, 163 MB/s 2024-04-25T08:38:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.132 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073343 s, 143 MB/s 2024-04-25T08:38:36.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:36.133 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:36.134 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:36.134 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:38:36.134 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:36.134 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:36.134 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:36.134 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.199 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711984 s, 147 MB/s 2024-04-25T08:38:36.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756677 s, 139 MB/s 2024-04-25T08:38:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.216 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696359 s, 151 MB/s 2024-04-25T08:38:36.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.222 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.222 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00565003 s, 186 MB/s 2024-04-25T08:38:36.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.229 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056226 s, 186 MB/s 2024-04-25T08:38:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.237 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705567 s, 149 MB/s 2024-04-25T08:38:36.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.244 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657781 s, 159 MB/s 2024-04-25T08:38:36.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.252 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621978 s, 169 MB/s 2024-04-25T08:38:36.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.258 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.259 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635107 s, 165 MB/s 2024-04-25T08:38:36.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.267 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715318 s, 147 MB/s 2024-04-25T08:38:36.274 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.274 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.274 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635267 s, 165 MB/s 2024-04-25T08:38:36.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628313 s, 167 MB/s 2024-04-25T08:38:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606159 s, 173 MB/s 2024-04-25T08:38:36.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.294 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.295 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596452 s, 176 MB/s 2024-04-25T08:38:36.301 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.302 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.302 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617963 s, 170 MB/s 2024-04-25T08:38:36.308 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.309 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.309 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604164 s, 174 MB/s 2024-04-25T08:38:36.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.316 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.316 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607301 s, 173 MB/s 2024-04-25T08:38:36.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.323 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.323 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642845 s, 163 MB/s 2024-04-25T08:38:36.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.330 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.330 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063355 s, 166 MB/s 2024-04-25T08:38:36.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603613 s, 174 MB/s 2024-04-25T08:38:36.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609683 s, 172 MB/s 2024-04-25T08:38:36.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607983 s, 172 MB/s 2024-04-25T08:38:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620688 s, 169 MB/s 2024-04-25T08:38:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700229 s, 150 MB/s 2024-04-25T08:38:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627183 s, 167 MB/s 2024-04-25T08:38:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.380 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650105 s, 161 MB/s 2024-04-25T08:38:36.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.387 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.387 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613848 s, 171 MB/s 2024-04-25T08:38:36.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.394 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635081 s, 165 MB/s 2024-04-25T08:38:36.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.401 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579017 s, 181 MB/s 2024-04-25T08:38:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585456 s, 179 MB/s 2024-04-25T08:38:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614758 s, 171 MB/s 2024-04-25T08:38:36.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.422 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577619 s, 182 MB/s 2024-04-25T08:38:36.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:36.423 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:36.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734041 s, 143 MB/s 2024-04-25T08:38:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.496 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611 s, 172 MB/s 2024-04-25T08:38:36.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630045 s, 166 MB/s 2024-04-25T08:38:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732478 s, 143 MB/s 2024-04-25T08:38:36.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637626 s, 164 MB/s 2024-04-25T08:38:36.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.527 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723148 s, 145 MB/s 2024-04-25T08:38:36.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.534 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699516 s, 150 MB/s 2024-04-25T08:38:36.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.541 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00555106 s, 189 MB/s 2024-04-25T08:38:36.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.548 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611027 s, 172 MB/s 2024-04-25T08:38:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.555 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600046 s, 175 MB/s 2024-04-25T08:38:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.562 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599176 s, 175 MB/s 2024-04-25T08:38:36.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.569 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637806 s, 164 MB/s 2024-04-25T08:38:36.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.577 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065377 s, 160 MB/s 2024-04-25T08:38:36.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632969 s, 166 MB/s 2024-04-25T08:38:36.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.591 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640097 s, 164 MB/s 2024-04-25T08:38:36.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.598 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.598 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606188 s, 173 MB/s 2024-04-25T08:38:36.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.604 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599182 s, 175 MB/s 2024-04-25T08:38:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.612 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632954 s, 166 MB/s 2024-04-25T08:38:36.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.619 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.619 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635133 s, 165 MB/s 2024-04-25T08:38:36.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.627 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.627 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710702 s, 148 MB/s 2024-04-25T08:38:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.634 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639723 s, 164 MB/s 2024-04-25T08:38:36.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.641 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637415 s, 165 MB/s 2024-04-25T08:38:36.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.649 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.649 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702319 s, 149 MB/s 2024-04-25T08:38:36.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559455 s, 187 MB/s 2024-04-25T08:38:36.663 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.664 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685842 s, 153 MB/s 2024-04-25T08:38:36.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.671 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618759 s, 169 MB/s 2024-04-25T08:38:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721432 s, 145 MB/s 2024-04-25T08:38:36.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637875 s, 164 MB/s 2024-04-25T08:38:36.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056618 s, 185 MB/s 2024-04-25T08:38:36.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.700 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604077 s, 174 MB/s 2024-04-25T08:38:36.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648502 s, 162 MB/s 2024-04-25T08:38:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.714 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616344 s, 170 MB/s 2024-04-25T08:38:36.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:36.715 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:36.715 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:36.715 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:38:36.715 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:36.715 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:36.715 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:36.716 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653137 s, 161 MB/s 2024-04-25T08:38:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607943 s, 172 MB/s 2024-04-25T08:38:36.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.794 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594574 s, 176 MB/s 2024-04-25T08:38:36.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629129 s, 167 MB/s 2024-04-25T08:38:36.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.808 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.808 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570841 s, 184 MB/s 2024-04-25T08:38:36.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.815 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006008 s, 175 MB/s 2024-04-25T08:38:36.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708634 s, 148 MB/s 2024-04-25T08:38:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601859 s, 174 MB/s 2024-04-25T08:38:36.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596811 s, 176 MB/s 2024-04-25T08:38:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.844 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584688 s, 179 MB/s 2024-04-25T08:38:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708654 s, 148 MB/s 2024-04-25T08:38:36.859 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690763 s, 152 MB/s 2024-04-25T08:38:36.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.867 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061581 s, 170 MB/s 2024-04-25T08:38:36.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621534 s, 169 MB/s 2024-04-25T08:38:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730956 s, 143 MB/s 2024-04-25T08:38:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.889 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613466 s, 171 MB/s 2024-04-25T08:38:36.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.896 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608469 s, 172 MB/s 2024-04-25T08:38:36.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.903 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607713 s, 173 MB/s 2024-04-25T08:38:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.910 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669965 s, 157 MB/s 2024-04-25T08:38:36.918 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.918 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.918 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713983 s, 147 MB/s 2024-04-25T08:38:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.925 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608336 s, 172 MB/s 2024-04-25T08:38:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.933 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641023 s, 164 MB/s 2024-04-25T08:38:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.940 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062243 s, 168 MB/s 2024-04-25T08:38:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.947 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607375 s, 173 MB/s 2024-04-25T08:38:36.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.954 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061023 s, 172 MB/s 2024-04-25T08:38:36.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.960 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573012 s, 183 MB/s 2024-04-25T08:38:36.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.968 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637884 s, 164 MB/s 2024-04-25T08:38:36.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707855 s, 148 MB/s 2024-04-25T08:38:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.983 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622839 s, 168 MB/s 2024-04-25T08:38:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598515 s, 175 MB/s 2024-04-25T08:38:36.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:36.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:36.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635379 s, 165 MB/s 2024-04-25T08:38:37.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605425 s, 173 MB/s 2024-04-25T08:38:37.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:37.005 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739064 s, 142 MB/s 2024-04-25T08:38:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632383 s, 166 MB/s 2024-04-25T08:38:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.087 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735173 s, 143 MB/s 2024-04-25T08:38:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717965 s, 146 MB/s 2024-04-25T08:38:37.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630886 s, 166 MB/s 2024-04-25T08:38:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.109 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603117 s, 174 MB/s 2024-04-25T08:38:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.117 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645123 s, 163 MB/s 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636222 s, 165 MB/s 2024-04-25T08:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608392 s, 172 MB/s 2024-04-25T08:38:37.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655755 s, 160 MB/s 2024-04-25T08:38:37.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.146 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.146 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655919 s, 160 MB/s 2024-04-25T08:38:37.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.154 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.154 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739583 s, 142 MB/s 2024-04-25T08:38:37.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618966 s, 169 MB/s 2024-04-25T08:38:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631976 s, 166 MB/s 2024-04-25T08:38:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.176 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694969 s, 151 MB/s 2024-04-25T08:38:37.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633158 s, 166 MB/s 2024-04-25T08:38:37.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.191 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658407 s, 159 MB/s 2024-04-25T08:38:37.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671577 s, 156 MB/s 2024-04-25T08:38:37.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.205 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602853 s, 174 MB/s 2024-04-25T08:38:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.212 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607496 s, 173 MB/s 2024-04-25T08:38:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625933 s, 168 MB/s 2024-04-25T08:38:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.226 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612259 s, 171 MB/s 2024-04-25T08:38:37.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.234 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706878 s, 148 MB/s 2024-04-25T08:38:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.242 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708103 s, 148 MB/s 2024-04-25T08:38:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704417 s, 149 MB/s 2024-04-25T08:38:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.257 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575109 s, 182 MB/s 2024-04-25T08:38:37.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.264 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.264 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601476 s, 174 MB/s 2024-04-25T08:38:37.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.271 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.271 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597268 s, 176 MB/s 2024-04-25T08:38:37.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.278 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.278 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622378 s, 168 MB/s 2024-04-25T08:38:37.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.285 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614639 s, 171 MB/s 2024-04-25T08:38:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.293 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710354 s, 148 MB/s 2024-04-25T08:38:37.300 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.300 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.300 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630751 s, 166 MB/s 2024-04-25T08:38:37.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.301 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.301 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:37.301 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:38:37.301 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.301 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.301 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:37.302 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:37.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.366 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.366 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562614 s, 186 MB/s 2024-04-25T08:38:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.373 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627562 s, 167 MB/s 2024-04-25T08:38:37.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.381 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.381 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070637 s, 148 MB/s 2024-04-25T08:38:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.388 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598303 s, 175 MB/s 2024-04-25T08:38:37.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711115 s, 147 MB/s 2024-04-25T08:38:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598319 s, 175 MB/s 2024-04-25T08:38:37.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.411 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691534 s, 152 MB/s 2024-04-25T08:38:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.419 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757676 s, 138 MB/s 2024-04-25T08:38:37.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591579 s, 177 MB/s 2024-04-25T08:38:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619815 s, 169 MB/s 2024-04-25T08:38:37.441 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.441 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.441 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723744 s, 145 MB/s 2024-04-25T08:38:37.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.449 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.449 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659322 s, 159 MB/s 2024-04-25T08:38:37.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.456 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.456 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600264 s, 175 MB/s 2024-04-25T08:38:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.463 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624345 s, 168 MB/s 2024-04-25T08:38:37.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.470 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060836 s, 172 MB/s 2024-04-25T08:38:37.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.478 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701334 s, 150 MB/s 2024-04-25T08:38:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.485 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.486 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710318 s, 148 MB/s 2024-04-25T08:38:37.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.493 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.493 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616229 s, 170 MB/s 2024-04-25T08:38:37.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.500 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597795 s, 175 MB/s 2024-04-25T08:38:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.507 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.508 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708586 s, 148 MB/s 2024-04-25T08:38:37.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.515 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632677 s, 166 MB/s 2024-04-25T08:38:37.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.522 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006093 s, 172 MB/s 2024-04-25T08:38:37.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.529 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657036 s, 160 MB/s 2024-04-25T08:38:37.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624875 s, 168 MB/s 2024-04-25T08:38:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618224 s, 170 MB/s 2024-04-25T08:38:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619587 s, 169 MB/s 2024-04-25T08:38:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.557 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607688 s, 173 MB/s 2024-04-25T08:38:37.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.564 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618248 s, 170 MB/s 2024-04-25T08:38:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649234 s, 162 MB/s 2024-04-25T08:38:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.579 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062701 s, 167 MB/s 2024-04-25T08:38:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.587 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697997 s, 150 MB/s 2024-04-25T08:38:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.594 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610628 s, 172 MB/s 2024-04-25T08:38:37.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.595 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.595 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:37.595 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:38:37.595 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.595 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.595 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:37.596 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:37.661 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.661 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.661 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625947 s, 168 MB/s 2024-04-25T08:38:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.669 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703864 s, 149 MB/s 2024-04-25T08:38:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.676 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619147 s, 169 MB/s 2024-04-25T08:38:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.683 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623984 s, 168 MB/s 2024-04-25T08:38:37.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.691 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730885 s, 143 MB/s 2024-04-25T08:38:37.698 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.698 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621742 s, 169 MB/s 2024-04-25T08:38:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.706 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638608 s, 164 MB/s 2024-04-25T08:38:37.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604694 s, 173 MB/s 2024-04-25T08:38:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.720 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610897 s, 172 MB/s 2024-04-25T08:38:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.728 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696997 s, 150 MB/s 2024-04-25T08:38:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.735 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.736 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711689 s, 147 MB/s 2024-04-25T08:38:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.743 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622168 s, 169 MB/s 2024-04-25T08:38:37.750 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715735 s, 147 MB/s 2024-04-25T08:38:37.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.758 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.758 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631682 s, 166 MB/s 2024-04-25T08:38:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722389 s, 145 MB/s 2024-04-25T08:38:37.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.773 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617499 s, 170 MB/s 2024-04-25T08:38:37.780 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.781 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634855 s, 165 MB/s 2024-04-25T08:38:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.787 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554136 s, 189 MB/s 2024-04-25T08:38:37.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699388 s, 150 MB/s 2024-04-25T08:38:37.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063263 s, 166 MB/s 2024-04-25T08:38:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703521 s, 149 MB/s 2024-04-25T08:38:37.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.818 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630883 s, 166 MB/s 2024-04-25T08:38:37.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612937 s, 171 MB/s 2024-04-25T08:38:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.832 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660368 s, 159 MB/s 2024-04-25T08:38:37.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.838 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.838 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609915 s, 172 MB/s 2024-04-25T08:38:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.846 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705791 s, 149 MB/s 2024-04-25T08:38:37.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.853 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608019 s, 172 MB/s 2024-04-25T08:38:37.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650025 s, 161 MB/s 2024-04-25T08:38:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005557 s, 189 MB/s 2024-04-25T08:38:37.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764945 s, 137 MB/s 2024-04-25T08:38:37.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726205 s, 144 MB/s 2024-04-25T08:38:37.892 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.892 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.892 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721805 s, 145 MB/s 2024-04-25T08:38:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.893 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.894 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:38:37.894 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:38:37.894 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.894 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.894 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:38:37.894 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702264 s, 149 MB/s 2024-04-25T08:38:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:37.992 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640475 s, 164 MB/s 2024-04-25T08:38:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.000 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643769 s, 163 MB/s 2024-04-25T08:38:38.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630095 s, 166 MB/s 2024-04-25T08:38:38.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.014 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615332 s, 170 MB/s 2024-04-25T08:38:38.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.022 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744576 s, 141 MB/s 2024-04-25T08:38:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.029 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637422 s, 165 MB/s 2024-04-25T08:38:38.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.037 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709846 s, 148 MB/s 2024-04-25T08:38:38.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.044 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.044 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578201 s, 181 MB/s 2024-04-25T08:38:38.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.051 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586029 s, 179 MB/s 2024-04-25T08:38:38.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.058 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656745 s, 160 MB/s 2024-04-25T08:38:38.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.065 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612538 s, 171 MB/s 2024-04-25T08:38:38.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598868 s, 175 MB/s 2024-04-25T08:38:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.079 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604439 s, 173 MB/s 2024-04-25T08:38:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559374 s, 187 MB/s 2024-04-25T08:38:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.094 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760906 s, 138 MB/s 2024-04-25T08:38:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629887 s, 166 MB/s 2024-04-25T08:38:38.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.110 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007043 s, 149 MB/s 2024-04-25T08:38:38.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.117 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591663 s, 177 MB/s 2024-04-25T08:38:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055945 s, 187 MB/s 2024-04-25T08:38:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662547 s, 158 MB/s 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633588 s, 165 MB/s 2024-04-25T08:38:38.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.144 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578978 s, 181 MB/s 2024-04-25T08:38:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.152 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659517 s, 159 MB/s 2024-04-25T08:38:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620773 s, 169 MB/s 2024-04-25T08:38:38.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.167 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701568 s, 149 MB/s 2024-04-25T08:38:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.174 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622467 s, 168 MB/s 2024-04-25T08:38:38.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.181 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619884 s, 169 MB/s 2024-04-25T08:38:38.189 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.189 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.189 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722743 s, 145 MB/s 2024-04-25T08:38:38.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.196 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.196 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621327 s, 169 MB/s 2024-04-25T08:38:38.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.204 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636251 s, 165 MB/s 2024-04-25T08:38:38.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:38:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:38:38.211 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622845 s, 168 MB/s 2024-04-25T08:38:38.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:38.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:38:38.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.324+0000 7f57de110640 1 Processor -- start 2024-04-25T08:38:38.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.324+0000 7f57de110640 1 -- start start 2024-04-25T08:38:38.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57de110640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57d815dac0 0x7f57d80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57de110640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d80d0ab0 0x7f57d80b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57de110640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d80d2420 con 0x7f57d80d1f30 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57de110640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d80d3030 con 0x7f57d815dac0 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57de110640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d80d2700 con 0x7f57d80d0ab0 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57d815dac0 0x7f57d80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f57d815dac0 0x7f57d80d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40016/0 (socket says 172.21.15.60:40016) 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d6ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f57d80d1f30 0x7f57d80d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59222/0 (socket says 172.21.15.60:59222) 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 -- 172.21.15.60:0/3748712434 learned_addr learned my addr 172.21.15.60:0/3748712434 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d7fff640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d80d0ab0 0x7f57d80b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 -- 172.21.15.60:0/3748712434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d80d0ab0 msgr2=0x7f57d80b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d80d0ab0 0x7f57d80b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 -- 172.21.15.60:0/3748712434 >> v1:172.21.15.60:6789/0 conn(0x7f57d80d1f30 legacy=0x7f57d80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d77fe640 1 -- 172.21.15.60:0/3748712434 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80d29e0 con 0x7f57d815dac0 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d67fc640 1 -- 172.21.15.60:0/3748712434 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824355470 0 0) 0x7f57d80d2420 con 0x7f57d80d1f30 2024-04-25T08:38:38.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.325+0000 7f57d7fff640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d80d0ab0 0x7f57d80b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.326+0000 7f57d77fe640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d815dac0 0x7f57d80d19f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a6972535815242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.326+0000 7f57d67fc640 1 -- 172.21.15.60:0/3748712434 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8037070 con 0x7f57d815dac0 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.326+0000 7f57d67fc640 1 -- 172.21.15.60:0/3748712434 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c802ae50 con 0x7f57d815dac0 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.326+0000 7f57d67fc640 1 -- 172.21.15.60:0/3748712434 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8032860 con 0x7f57d815dac0 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 -- 172.21.15.60:0/3748712434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d815dac0 msgr2=0x7f57d80d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d815dac0 0x7f57d80d19f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c802a300 comp rx=0 tx=0).stop 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 -- 172.21.15.60:0/3748712434 shutdown_connections 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d80d0ab0 0x7f57d80b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 --2- 172.21.15.60:0/3748712434 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d815dac0 0x7f57d80d19f0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 -- 172.21.15.60:0/3748712434 >> 172.21.15.60:0/3748712434 conn(0x7f57d8061c30 msgr2=0x7f57d806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 -- 172.21.15.60:0/3748712434 shutdown_connections 2024-04-25T08:38:38.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.327+0000 7f57de110640 1 -- 172.21.15.60:0/3748712434 wait complete. 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.328+0000 7f57de110640 1 Processor -- start 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.328+0000 7f57de110640 1 -- start start 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57de110640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 0x7f57d8152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57de110640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d80d1f30 0x7f57d815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57de110640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d815dac0 0x7f57d80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57de110640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d80b97d0 con 0x7f57d80d0ab0 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57de110640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d815dea0 con 0x7f57d80d1f30 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57de110640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d80b9950 con 0x7f57d815dac0 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 0x7f57d8152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 0x7f57d8152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40030/0 (socket says 172.21.15.60:40030) 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 -- 172.21.15.60:0/2571480003 learned_addr learned my addr 172.21.15.60:0/2571480003 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d7fff640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d815dac0 0x7f57d80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d80d1f30 0x7f57d815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 -- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d815dac0 msgr2=0x7f57d80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d815dac0 0x7f57d80ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 -- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d80d1f30 msgr2=0x7f57d815e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d80d1f30 0x7f57d815e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.329+0000 7f57d77fe640 1 -- 172.21.15.60:0/2571480003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d81f7db0 con 0x7f57d80d0ab0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.330+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 0x7f57d8152aa0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.330+0000 7f57bffff640 1 -- 172.21.15.60:0/2571480003 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8041070 con 0x7f57d80d0ab0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.330+0000 7f57bffff640 1 -- 172.21.15.60:0/2571480003 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c803c670 con 0x7f57d80d0ab0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.330+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d8152fe0 con 0x7f57d80d0ab0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.330+0000 7f57bffff640 1 -- 172.21.15.60:0/2571480003 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8037040 con 0x7f57d80d0ab0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.330+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d81f7230 con 0x7f57d80d0ab0 2024-04-25T08:38:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.331+0000 7f57bffff640 1 -- 172.21.15.60:0/2571480003 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57c8044070 con 0x7f57d80d0ab0 2024-04-25T08:38:38.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.332+0000 7f57bffff640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57c406b4d0 0x7f57c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.332+0000 7f57bffff640 1 -- 172.21.15.60:0/2571480003 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57c80aa920 con 0x7f57d80d0ab0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.332+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57c406b4d0 0x7f57c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.332+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57c406b4d0 0x7f57c406d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.333+0000 7f57e0e10740 1 Processor -- start 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.333+0000 7f57e0e10740 1 -- start start 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.333+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fbb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.333+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 0x558d43fb95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.333+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43fe4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.333+0000 7f57e0e10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d43f1d000 con 0x558d43ff58d0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d43ec5f20 con 0x558d43fb9200 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d43f190a0 con 0x558d43fe3cc0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 0x558d43fb95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 0x558d43fb95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40042/0 (socket says 172.21.15.60:40042) 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 -- 172.21.15.60:0/162113613 learned_addr learned my addr 172.21.15.60:0/162113613 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d77fe640 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fbb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d7fff640 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43fe4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 -- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 msgr2=0x558d43fe4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43fe4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 -- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 msgr2=0x558d43fbb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fbb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 -- 172.21.15.60:0/162113613 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d43cfdba0 con 0x558d43ff58d0 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d77fe640 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fbb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 0x558d43fb95e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00f960 tx=0x7f57cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f35364dc02bfe36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57bd7fa640 1 -- 172.21.15.60:0/162113613 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc025070 con 0x558d43ff58d0 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57bd7fa640 1 -- 172.21.15.60:0/162113613 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc01b050 con 0x558d43ff58d0 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.334+0000 7f57bd7fa640 1 -- 172.21.15.60:0/162113613 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc014de0 con 0x558d43ff58d0 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 -- 172.21.15.60:0/162113613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 msgr2=0x558d43fb95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 0x558d43fb95e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00f960 tx=0x7f57cc012000 comp rx=0 tx=0).stop 2024-04-25T08:38:38.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 -- 172.21.15.60:0/162113613 shutdown_connections 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43fe4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43ff58d0 0x558d43fb95e0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 --2- 172.21.15.60:0/162113613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fbb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 -- 172.21.15.60:0/162113613 >> 172.21.15.60:0/162113613 conn(0x558d43f81050 msgr2=0x558d43ffecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 -- 172.21.15.60:0/162113613 shutdown_connections 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.335+0000 7f57e0e10740 1 -- 172.21.15.60:0/162113613 wait complete. 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 -- start start 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 Processor -- start 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fd04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43ff58d0 0x558d43fe0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d43ebb610 con 0x558d43fe3cc0 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d43fe1810 con 0x558d43fb9200 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.336+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d43fe6140 con 0x558d43ff58d0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57e0e10740 10 client.0 _ll_register_callbacks cb 0x558d43ed3f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40052/0 (socket says 172.21.15.60:40052) 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 -- 172.21.15.60:0/4209490445 learned_addr learned my addr 172.21.15.60:0/4209490445 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d77fe640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fd04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d7fff640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43ff58d0 0x558d43fe0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 -- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43ff58d0 msgr2=0x558d43fe0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43ff58d0 0x558d43fe0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 -- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 msgr2=0x558d43fd04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fd04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 -- 172.21.15.60:0/4209490445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d43bf1490 con 0x558d43fe3cc0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe0780 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f57cc023040 tx=0x7f57cc007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47d960430151541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57b67fc640 1 -- 172.21.15.60:0/4209490445 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc030020 con 0x558d43fe3cc0 2024-04-25T08:38:38.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57b67fc640 1 -- 172.21.15.60:0/4209490445 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57cc020a50 con 0x558d43fe3cc0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57cc01b050 con 0x558d43fe3cc0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57b67fc640 1 -- 172.21.15.60:0/4209490445 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc03a050 con 0x558d43fe3cc0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57e0e10740 10 client.9579 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x558d43d5e330 con 0x558d43fe3cc0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.337+0000 7f57b67fc640 1 -- 172.21.15.60:0/4209490445 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc01b050 con 0x558d43fe3cc0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57b67fc640 1 -- 172.21.15.60:0/4209490445 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57cc03d020 con 0x558d43fe3cc0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57e0e10740 10 client.9579 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57e0e10740 10 client.9579 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57e0e10740 20 client.9579 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57e0e10740 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43ff6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57e0e10740 4 client.9579 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d43f2a540 con 0x558d43ff6160 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57d77fe640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43ff6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57d77fe640 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43ff6590 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9f96b47c409cb2 server_cookie=d02384f357506ee0 in_seq=0 out_seq=0 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.338+0000 7f57b67fc640 10 client.9579 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57b67fc640 1 -- 172.21.15.60:0/4209490445 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d43f2a540 con 0x558d43ff6160 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57b67fc640 10 client.9579 handle_command_reply: tid=0 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 client.9579 shutdown 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 msgr2=0x558d43fe0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe0780 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f57cc023040 tx=0x7f57cc007c60 comp rx=0 tx=0).stop 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 shutdown_connections 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43ff6590 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57b67fc640 0 client.9579 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43ff58d0 0x558d43fe0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe0780 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 --2- 172.21.15.60:0/4209490445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43fb9200 0x558d43fd04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 >> 172.21.15.60:0/4209490445 conn(0x558d43f81050 msgr2=0x558d43fe1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 shutdown_connections 2024-04-25T08:38:38.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 1 -- 172.21.15.60:0/4209490445 wait complete. 2024-04-25T08:38:38.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.339+0000 7f57e0e10740 20 client.9579 trim_cache size 0 max 16384 2024-04-25T08:38:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.354+0000 7f57e0e10740 1 Processor -- start 2024-04-25T08:38:38.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.354+0000 7f57e0e10740 1 -- start start 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 0x558d43fb33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43ff6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57e0e10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8200340 con 0x558d43fb9200 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81ff5a0 con 0x558d43ff58d0 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8160980 con 0x558d43fe3cc0 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 0x558d43fb33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 0x558d43fb33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40064/0 (socket says 172.21.15.60:40064) 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d77fe640 1 -- 172.21.15.60:0/2500753108 learned_addr learned my addr 172.21.15.60:0/2500753108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d7fff640 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43ff6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d77fe640 1 -- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 msgr2=0x558d43ff6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.355+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43ff6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57d77fe640 1 -- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 msgr2=0x558d43fe1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57d77fe640 1 -- 172.21.15.60:0/2500753108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d43bca610 con 0x558d43fb9200 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57d7fff640 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 0x558d43fb33c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f57c8078de0 tx=0x7f57c80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9a7e14672f5da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57bcff9640 1 -- 172.21.15.60:0/2500753108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c80a7070 con 0x558d43fb9200 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57bcff9640 1 -- 172.21.15.60:0/2500753108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c802ae50 con 0x558d43fb9200 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57bcff9640 1 -- 172.21.15.60:0/2500753108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c804b040 con 0x558d43fb9200 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 -- 172.21.15.60:0/2500753108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 msgr2=0x558d43fb33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 0x558d43fb33c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f57c8078de0 tx=0x7f57c80095f0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.351 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 -- 172.21.15.60:0/2500753108 shutdown_connections 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fe3cc0 0x558d43ff6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2500753108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fb9200 0x558d43fb33c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.356+0000 7f57e0e10740 1 -- 172.21.15.60:0/2500753108 >> 172.21.15.60:0/2500753108 conn(0x558d43f81050 msgr2=0x558d43fd0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.357+0000 7f57e0e10740 1 -- 172.21.15.60:0/2500753108 shutdown_connections 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.357+0000 7f57e0e10740 1 -- 172.21.15.60:0/2500753108 wait complete. 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.357+0000 7f57e0e10740 1 -- start start 2024-04-25T08:38:38.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.357+0000 7f57e0e10740 1 Processor -- start 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fb9200 0x558d4406c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d43fffc10 con 0x558d43fe3cc0 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8160f80 con 0x558d43ff58d0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d44002b20 con 0x558d43fb9200 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57e0e10740 10 client.0 _ll_register_callbacks cb 0x7f57cc017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40080/0 (socket says 172.21.15.60:40080) 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 -- 172.21.15.60:0/2983628068 learned_addr learned my addr 172.21.15.60:0/2983628068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d7fff640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fb9200 0x558d4406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 -- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fb9200 msgr2=0x558d4406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fb9200 0x558d4406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 -- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 msgr2=0x558d43fe2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 -- 172.21.15.60:0/2983628068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d43fe3aa0 con 0x558d43fe3cc0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d7fff640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57d6ffd640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe2620 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f57cc001020 tx=0x7f57cc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a1284e063852ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.358+0000 7f57b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57b67fc640 1 -- 172.21.15.60:0/2983628068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc042020 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57b67fc640 1 -- 172.21.15.60:0/2983628068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57cc02d9d0 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57c802ae50 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57b67fc640 1 -- 172.21.15.60:0/2983628068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57cc04c050 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 10 client.9585 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x558d43fe6d80 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57b67fc640 1 -- 172.21.15.60:0/2983628068 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c802ae50 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57b67fc640 1 -- 172.21.15.60:0/2983628068 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f57cc04f020 con 0x558d43fe3cc0 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 10 client.9585 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 10 client.9585 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 20 client.9585 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43fe30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 4 client.9585 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x558d43ff8ef0 con 0x558d43ff6160 2024-04-25T08:38:38.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.359+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43fe30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.360+0000 7f57b67fc640 10 client.9585 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.360+0000 7f57d77fe640 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43fe30a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ac77f4e33e6d60 server_cookie=265b7c376702f930 in_seq=0 out_seq=0 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57b67fc640 1 -- 172.21.15.60:0/2983628068 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558d43ff8ef0 con 0x558d43ff6160 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57b67fc640 10 client.9585 handle_command_reply: tid=0 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 client.9585 shutdown 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 msgr2=0x558d43fe2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe2620 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f57cc001020 tx=0x7f57cc00a000 comp rx=0 tx=0).stop 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 shutdown_connections 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x558d43ff6160 0x558d43fe30a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558d43ff58d0 0x558d43fe2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558d43fe3cc0 0x558d43fe2620 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57b67fc640 0 client.9585 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 --2- 172.21.15.60:0/2983628068 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x558d43fb9200 0x558d4406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 >> 172.21.15.60:0/2983628068 conn(0x558d43f81050 msgr2=0x558d43fe1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.359 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 shutdown_connections 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 1 -- 172.21.15.60:0/2983628068 wait complete. 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.362+0000 7f57e0e10740 20 client.9585 trim_cache size 0 max 16384 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0, 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_tag": "4f96fef4-98b3-406b-8e54-cbf1e541226d", 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stdout: "mode": "asynchronous" 2024-04-25T08:38:38.360 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57c406b4d0 msgr2=0x7f57c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57c406b4d0 0x7f57c406d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).stop 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 msgr2=0x7f57d8152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 0x7f57d8152aa0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 shutdown_connections 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f57d815dac0 0x7f57d80ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f57c406b4d0 0x7f57c406d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57d80d1f30 0x7f57d815e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 --2- 172.21.15.60:0/2571480003 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f57d80d0ab0 0x7f57d8152aa0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 >> 172.21.15.60:0/2571480003 conn(0x7f57d8061c30 msgr2=0x7f57d80c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 shutdown_connections 2024-04-25T08:38:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.363+0000 7f57de110640 1 -- 172.21.15.60:0/2571480003 wait complete. 2024-04-25T08:38:38.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:38.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T08:38:38.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.493+0000 7fdefe89a640 1 Processor -- start 2024-04-25T08:38:38.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.493+0000 7fdefe89a640 1 -- start start 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefe89a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdef80cfa40 0x7fdef80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefe89a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef80c75e0 con 0x7fdef815bb50 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefe89a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef80c78c0 con 0x7fdef80cf600 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefe89a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80c8680 con 0x7fdef80cfa40 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefce37640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdef80cfa40 0x7fdef80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdef7fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdef815bb50 0x7fdef80c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59230/0 (socket says 172.21.15.60:59230) 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdef7fff640 1 -- 172.21.15.60:0/49670938 learned_addr learned my addr 172.21.15.60:0/49670938 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:38.487 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefce37640 1 -- 172.21.15.60:0/49670938 >> v1:172.21.15.100:6789/0 conn(0x7fdef80cf600 legacy=0x7fdef80cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefce37640 1 -- 172.21.15.60:0/49670938 >> v1:172.21.15.60:6789/0 conn(0x7fdef815bb50 legacy=0x7fdef80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdefce37640 1 -- 172.21.15.60:0/49670938 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80c7ba0 con 0x7fdef80cfa40 2024-04-25T08:38:38.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.494+0000 7fdef6ffd640 1 -- 172.21.15.60:0/49670938 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543099092 0 0) 0x7fdef80c75e0 con 0x7fdef815bb50 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.495+0000 7fdefce37640 1 --2- 172.21.15.60:0/49670938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cfa40 0x7fdef80cc630 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009d90 tx=0x7fdef002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1277fdf0bda564c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.495+0000 7fdef6ffd640 1 -- 172.21.15.60:0/49670938 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0037070 con 0x7fdef80cfa40 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.495+0000 7fdef6ffd640 1 -- 172.21.15.60:0/49670938 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0007740 con 0x7fdef80cfa40 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.495+0000 7fdef6ffd640 1 -- 172.21.15.60:0/49670938 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0007b10 con 0x7fdef80cfa40 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 -- 172.21.15.60:0/49670938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cfa40 msgr2=0x7fdef80cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 --2- 172.21.15.60:0/49670938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cfa40 0x7fdef80cc630 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009d90 tx=0x7fdef002a780 comp rx=0 tx=0).stop 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 -- 172.21.15.60:0/49670938 shutdown_connections 2024-04-25T08:38:38.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 --2- 172.21.15.60:0/49670938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cfa40 0x7fdef80cc630 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 -- 172.21.15.60:0/49670938 >> 172.21.15.60:0/49670938 conn(0x7fdef806f500 msgr2=0x7fdef806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 -- 172.21.15.60:0/49670938 shutdown_connections 2024-04-25T08:38:38.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.496+0000 7fdefe89a640 1 -- 172.21.15.60:0/49670938 wait complete. 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.497+0000 7fdefe89a640 1 Processor -- start 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.497+0000 7fdefe89a640 1 -- start start 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefe89a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 0x7fdef8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefe89a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdef80cfa40 0x7fdef814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefe89a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdef815bb50 0x7fdef814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefe89a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80c72a0 con 0x7fdef80cf600 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefe89a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80c8160 con 0x7fdef80cfa40 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefe89a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80cfe90 con 0x7fdef815bb50 2024-04-25T08:38:38.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 0x7fdef8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 0x7fdef8154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40092/0 (socket says 172.21.15.60:40092) 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 -- 172.21.15.60:0/3387843550 learned_addr learned my addr 172.21.15.60:0/3387843550 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef77fe640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdef80cfa40 0x7fdef814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefce37640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdef815bb50 0x7fdef814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 -- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdef815bb50 msgr2=0x7fdef814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdef815bb50 0x7fdef814eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 -- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdef80cfa40 msgr2=0x7fdef814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdef80cfa40 0x7fdef814e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdef7fff640 1 -- 172.21.15.60:0/3387843550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef814f420 con 0x7fdef80cf600 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.498+0000 7fdefce37640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdef815bb50 0x7fdef814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:38.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.499+0000 7fdef7fff640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 0x7fdef8154510 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c3f0 tx=0x7fdee800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.499+0000 7fdef4ff9640 1 -- 172.21.15.60:0/3387843550 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80074e0 con 0x7fdef80cf600 2024-04-25T08:38:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.499+0000 7fdef4ff9640 1 -- 172.21.15.60:0/3387843550 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee80047e0 con 0x7fdef80cf600 2024-04-25T08:38:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.499+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef81e53b0 con 0x7fdef80cf600 2024-04-25T08:38:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.499+0000 7fdef4ff9640 1 -- 172.21.15.60:0/3387843550 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8004c10 con 0x7fdef80cf600 2024-04-25T08:38:38.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.499+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef81e5970 con 0x7fdef80cf600 2024-04-25T08:38:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.500+0000 7fdef4ff9640 1 -- 172.21.15.60:0/3387843550 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdee8011070 con 0x7fdef80cf600 2024-04-25T08:38:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.501+0000 7fdef4ff9640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fded806b4d0 0x7fded806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.501+0000 7fdef4ff9640 1 -- 172.21.15.60:0/3387843550 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdee8089fd0 con 0x7fdef80cf600 2024-04-25T08:38:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.501+0000 7fdef77fe640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fded806b4d0 0x7fded806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.502+0000 7fdef77fe640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fded806b4d0 0x7fded806d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdef806f4d0 tx=0x7fdee0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.502+0000 7fdf01597740 1 Processor -- start 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.502+0000 7fdf01597740 1 -- start start 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd0291ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdf01597740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 0x55fd0293d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0293ddf0 0x55fd028f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdf01597740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd02859b90 con 0x55fd028f8e40 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd02802e40 con 0x55fd0293ddf0 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd02855c30 con 0x55fd0291ea90 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 0x55fd0293d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef7fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd0291ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 0x55fd0293d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40096/0 (socket says 172.21.15.60:40096) 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 -- 172.21.15.60:0/790051477 learned_addr learned my addr 172.21.15.60:0/790051477 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdefce37640 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0293ddf0 0x55fd028f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 -- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 msgr2=0x55fd0291ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd0291ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 -- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0293ddf0 msgr2=0x55fd028f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0293ddf0 0x55fd028f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 -- 172.21.15.60:0/790051477 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd026387d0 con 0x55fd028f8e40 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdef77fe640 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 0x55fd0293d8b0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001db0 tx=0x7fdee0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7709066586e38c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdeedffb640 1 -- 172.21.15.60:0/790051477 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee00035f0 con 0x55fd028f8e40 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdeedffb640 1 -- 172.21.15.60:0/790051477 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0003790 con 0x55fd028f8e40 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.503+0000 7fdeedffb640 1 -- 172.21.15.60:0/790051477 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0003b40 con 0x55fd028f8e40 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 -- 172.21.15.60:0/790051477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 msgr2=0x55fd0293d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 0x55fd0293d8b0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fdee0001db0 tx=0x7fdee0011000 comp rx=0 tx=0).stop 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 -- 172.21.15.60:0/790051477 shutdown_connections 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0293ddf0 0x55fd028f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd028f8e40 0x55fd0293d8b0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 --2- 172.21.15.60:0/790051477 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd0291ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 -- 172.21.15.60:0/790051477 >> 172.21.15.60:0/790051477 conn(0x55fd028be870 msgr2=0x55fd0293c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 -- 172.21.15.60:0/790051477 shutdown_connections 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.504+0000 7fdf01597740 1 -- 172.21.15.60:0/790051477 wait complete. 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.505+0000 7fdf01597740 1 -- start start 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.505+0000 7fdf01597740 1 Processor -- start 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02954120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291ea90 0x55fd029216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd02915d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd028f32a0 con 0x55fd0291f420 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd028163e0 con 0x55fd0291ea90 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd028f7dd0 con 0x55fd028f8e40 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 10 client.0 _ll_register_callbacks cb 0x55fd02815db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd02915d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd02915d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40110/0 (socket says 172.21.15.60:40110) 2024-04-25T08:38:38.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 -- 172.21.15.60:0/1159661856 learned_addr learned my addr 172.21.15.60:0/1159661856 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdef7fff640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02954120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdef77fe640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291ea90 0x55fd029216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 -- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 msgr2=0x55fd02954120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02954120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 -- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291ea90 msgr2=0x55fd029216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291ea90 0x55fd029216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 -- 172.21.15.60:0/1159661856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd0252e0f0 con 0x55fd0291f420 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdefce37640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd02915d20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009eb0 tx=0x7fdef0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfbb51fc47623a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fded6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fded6ffd640 1 -- 172.21.15.60:0/1159661856 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0037040 con 0x55fd0291f420 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fded6ffd640 1 -- 172.21.15.60:0/1159661856 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdef0032440 con 0x55fd0291f420 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.506+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee0003790 con 0x55fd0291f420 2024-04-25T08:38:38.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fded6ffd640 1 -- 172.21.15.60:0/1159661856 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef0050050 con 0x55fd0291f420 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 10 client.9600 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55fd02698f60 con 0x55fd0291f420 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fded6ffd640 1 -- 172.21.15.60:0/1159661856 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0003790 con 0x55fd0291f420 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fded6ffd640 1 -- 172.21.15.60:0/1159661856 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdef0053020 con 0x55fd0291f420 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 10 client.9600 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 10 client.9600 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 20 client.9600 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd02917190 0x55fd02918e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 4 client.9600 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd028670d0 con 0x55fd02917190 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdef7fff640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd02917190 0x55fd02918e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fdef7fff640 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd02917190 0x55fd02918e50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff0a9e3c2832719 server_cookie=2042a89e6f8e3c0f in_seq=0 out_seq=0 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.507+0000 7fded6ffd640 10 client.9600 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.506 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fded6ffd640 1 -- 172.21.15.60:0/1159661856 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd028670d0 con 0x55fd02917190 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fded6ffd640 10 client.9600 handle_command_reply: tid=0 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 client.9600 shutdown 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 msgr2=0x55fd02915d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd02915d20 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009eb0 tx=0x7fdef0004ab0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 shutdown_connections 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd02917190 0x55fd02918e50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd02915d20 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291ea90 0x55fd029216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fded6ffd640 0 client.9600 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 --2- 172.21.15.60:0/1159661856 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02954120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 >> 172.21.15.60:0/1159661856 conn(0x55fd028be870 msgr2=0x55fd0293fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 shutdown_connections 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 1 -- 172.21.15.60:0/1159661856 wait complete. 2024-04-25T08:38:38.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.508+0000 7fdf01597740 20 client.9600 trim_cache size 0 max 16384 2024-04-25T08:38:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.523+0000 7fdf01597740 1 Processor -- start 2024-04-25T08:38:38.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.523+0000 7fdf01597740 1 -- start start 2024-04-25T08:38:38.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02918ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdf01597740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 0x55fd02921010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291f420 0x55fd02921550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdf01597740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81fc020 con 0x55fd0291ea90 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef806d670 con 0x55fd0291f420 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef80b61d0 con 0x55fd028f8e40 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 0x55fd02921010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 0x55fd02921010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40114/0 (socket says 172.21.15.60:40114) 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 -- 172.21.15.60:0/2208712025 learned_addr learned my addr 172.21.15.60:0/2208712025 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef7fff640 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02918ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdefce37640 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291f420 0x55fd02921550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 -- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 msgr2=0x55fd02918ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02918ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 -- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291f420 msgr2=0x55fd02921550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291f420 0x55fd02921550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 -- 172.21.15.60:0/2208712025 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd025071b0 con 0x55fd0291ea90 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.524+0000 7fdef77fe640 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 0x55fd02921010 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fdee0003780 tx=0x7fdee0005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc005a5e8674ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdeed7fa640 1 -- 172.21.15.60:0/2208712025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0030070 con 0x55fd0291ea90 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdeed7fa640 1 -- 172.21.15.60:0/2208712025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0003b60 con 0x55fd0291ea90 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdeed7fa640 1 -- 172.21.15.60:0/2208712025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0021450 con 0x55fd0291ea90 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 -- 172.21.15.60:0/2208712025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 msgr2=0x55fd02921010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 0x55fd02921010 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fdee0003780 tx=0x7fdee0005000 comp rx=0 tx=0).stop 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 -- 172.21.15.60:0/2208712025 shutdown_connections 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd0291f420 0x55fd02921550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291ea90 0x55fd02921010 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 --2- 172.21.15.60:0/2208712025 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd028f8e40 0x55fd02918ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 -- 172.21.15.60:0/2208712025 >> 172.21.15.60:0/2208712025 conn(0x55fd028be870 msgr2=0x55fd028f6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 -- 172.21.15.60:0/2208712025 shutdown_connections 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.525+0000 7fdf01597740 1 -- 172.21.15.60:0/2208712025 wait complete. 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.526+0000 7fdf01597740 1 -- start start 2024-04-25T08:38:38.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.526+0000 7fdf01597740 1 Processor -- start 2024-04-25T08:38:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd028f8e40 0x55fd0293fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd02940020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd028f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd028f1410 con 0x55fd0291f420 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef806d070 con 0x55fd028f8e40 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd02925c90 con 0x55fd0291ea90 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 10 client.0 _ll_register_callbacks cb 0x7fdef0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd028f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd028f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40130/0 (socket says 172.21.15.60:40130) 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 -- 172.21.15.60:0/4288378888 learned_addr learned my addr 172.21.15.60:0/4288378888 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdef7fff640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd028f8e40 0x55fd0293fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdef77fe640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd02940020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 -- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 msgr2=0x55fd02940020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd02940020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 -- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd028f8e40 msgr2=0x55fd0293fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd028f8e40 0x55fd0293fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 -- 172.21.15.60:0/4288378888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd028f0dc0 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdefce37640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd028f52d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fdef002a7c0 tx=0x7fdef0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abacb6750151f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fded6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fded6ffd640 1 -- 172.21.15.60:0/4288378888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0037040 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fded6ffd640 1 -- 172.21.15.60:0/4288378888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdef0009070 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.527+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee0003b60 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fded6ffd640 1 -- 172.21.15.60:0/4288378888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef0060050 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 10 client.9606 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55fd028f60e0 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fded6ffd640 1 -- 172.21.15.60:0/4288378888 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0003b60 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fded6ffd640 1 -- 172.21.15.60:0/4288378888 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdef0063020 con 0x55fd0291f420 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 10 client.9606 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 10 client.9606 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T08:38:38.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 20 client.9606 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd0293b780 0x55fd028f5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 4 client.9606 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55fd02861420 con 0x55fd0293b780 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdef7fff640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd0293b780 0x55fd028f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fded6ffd640 10 client.9606 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.528+0000 7fdef7fff640 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd0293b780 0x55fd028f5810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b3a2406ff74681 server_cookie=a3d50c16e0c7b73e in_seq=0 out_seq=0 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fded6ffd640 1 -- 172.21.15.60:0/4288378888 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55fd02861420 con 0x55fd0293b780 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fded6ffd640 10 client.9606 handle_command_reply: tid=0 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 client.9606 shutdown 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 msgr2=0x55fd028f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd028f52d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fdef002a7c0 tx=0x7fdef0009750 comp rx=0 tx=0).stop 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 shutdown_connections 2024-04-25T08:38:38.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x55fd0293b780 0x55fd028f5810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fd0291f420 0x55fd028f52d0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55fd0291ea90 0x55fd02940020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 --2- 172.21.15.60:0/4288378888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fd028f8e40 0x55fd0293fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 >> 172.21.15.60:0/4288378888 conn(0x55fd028be870 msgr2=0x55fd02921430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fded6ffd640 0 client.9606 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 shutdown_connections 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 1 -- 172.21.15.60:0/4288378888 wait complete. 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdf01597740 20 client.9606 trim_cache size 0 max 16384 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stdout: "return_code": 0 2024-04-25T08:38:38.528 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:38:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fded806b4d0 msgr2=0x7fded806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdefe89a640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fded806b4d0 0x7fded806d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdef806f4d0 tx=0x7fdee0009040 comp rx=0 tx=0).stop 2024-04-25T08:38:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 msgr2=0x7fdef8154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdefe89a640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 0x7fdef8154510 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c3f0 tx=0x7fdee800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.529+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 shutdown_connections 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdef815bb50 0x7fdef814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fded806b4d0 0x7fded806d990 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdef80cfa40 0x7fdef814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 --2- 172.21.15.60:0/3387843550 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdef80cf600 0x7fdef8154510 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 >> 172.21.15.60:0/3387843550 conn(0x7fdef806f500 msgr2=0x7fdef8070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 shutdown_connections 2024-04-25T08:38:38.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.530+0000 7fdefe89a640 1 -- 172.21.15.60:0/3387843550 wait complete. 2024-04-25T08:38:38.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:38.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T08:38:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.642+0000 7f2a0b39e640 1 Processor -- start 2024-04-25T08:38:38.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a0b39e640 1 -- start start 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a0b39e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2a0415dc20 0x7f2a040ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a0b39e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a04161710 0x7f2a040cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a0b39e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0415e000 con 0x7f2a040b9330 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a0b39e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a040d1de0 con 0x7f2a04161710 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a0b39e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a04161c60 con 0x7f2a0415dc20 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.643+0000 7f2a08939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a04161710 0x7f2a040cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a08939640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2a04161710 0x7f2a040cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40136/0 (socket says 172.21.15.60:40136) 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a08939640 1 -- 172.21.15.60:0/3885216498 learned_addr learned my addr 172.21.15.60:0/3885216498 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a0913a640 1 --2- 172.21.15.60:0/3885216498 >> v2:172.21.15.100:3300/0 conn(0x7f2a0415dc20 0x7f2a040ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a08939640 1 -- 172.21.15.60:0/3885216498 >> v2:172.21.15.100:3300/0 conn(0x7f2a0415dc20 msgr2=0x7f2a040ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a08939640 1 --2- 172.21.15.60:0/3885216498 >> v2:172.21.15.100:3300/0 conn(0x7f2a0415dc20 0x7f2a040ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a08939640 1 -- 172.21.15.60:0/3885216498 >> v1:172.21.15.100:6789/0 conn(0x7f2a040b9330 legacy=0x7f2a0406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.644+0000 7f2a08939640 1 -- 172.21.15.60:0/3885216498 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04161f40 con 0x7f2a04161710 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a08939640 1 --2- 172.21.15.60:0/3885216498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a04161710 0x7f2a040cebb0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f2a040d1520 tx=0x7f29f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906e6d1441bf651f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a03fff640 1 -- 172.21.15.60:0/3885216498 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8037070 con 0x7f2a04161710 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a03fff640 1 -- 172.21.15.60:0/3885216498 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f8032440 con 0x7f2a04161710 2024-04-25T08:38:38.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a03fff640 1 -- 172.21.15.60:0/3885216498 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8032810 con 0x7f2a04161710 2024-04-25T08:38:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3885216498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a04161710 msgr2=0x7f2a040cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3885216498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a04161710 0x7f2a040cebb0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f2a040d1520 tx=0x7f29f802a230 comp rx=0 tx=0).stop 2024-04-25T08:38:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3885216498 shutdown_connections 2024-04-25T08:38:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3885216498 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a04161710 0x7f2a040cebb0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3885216498 >> v2:172.21.15.100:3300/0 conn(0x7f2a0415dc20 0x7f2a040ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3885216498 >> 172.21.15.60:0/3885216498 conn(0x7f2a040d0420 msgr2=0x7f2a040d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3885216498 shutdown_connections 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.645+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3885216498 wait complete. 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.646+0000 7f2a0b39e640 1 Processor -- start 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 -- start start 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a040b9330 0x7f2a040b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 0x7f2a041e9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a04161710 0x7f2a0414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040b5e50 con 0x7f2a0415dc20 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04162500 con 0x7f2a04161710 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a0b39e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040b5fd0 con 0x7f2a040b9330 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a08939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 0x7f2a041e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a08939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 0x7f2a041e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40144/0 (socket says 172.21.15.60:40144) 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.647+0000 7f2a08939640 1 -- 172.21.15.60:0/3842580479 learned_addr learned my addr 172.21.15.60:0/3842580479 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a0913a640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a040b9330 0x7f2a040b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a0993b640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a04161710 0x7f2a0414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a08939640 1 -- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a040b9330 msgr2=0x7f2a040b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a08939640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a040b9330 0x7f2a040b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a08939640 1 -- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a04161710 msgr2=0x7f2a0414f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a08939640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a04161710 0x7f2a0414f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a08939640 1 -- 172.21.15.60:0/3842580479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a040ccf80 con 0x7f2a0415dc20 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a08939640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 0x7f2a041e9950 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f29f802a270 tx=0x7f29f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a01ffb640 1 -- 172.21.15.60:0/3842580479 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8041020 con 0x7f2a0415dc20 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a01ffb640 1 -- 172.21.15.60:0/3842580479 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f80306c0 con 0x7f2a0415dc20 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a01ffb640 1 -- 172.21.15.60:0/3842580479 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8037040 con 0x7f2a0415dc20 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a040cc6e0 con 0x7f2a0415dc20 2024-04-25T08:38:38.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.648+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a040ccca0 con 0x7f2a0415dc20 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.650+0000 7f2a01ffb640 1 -- 172.21.15.60:0/3842580479 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29f8044020 con 0x7f2a0415dc20 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.650+0000 7f2a01ffb640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29f006b4d0 0x7f29f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.650+0000 7f2a01ffb640 1 -- 172.21.15.60:0/3842580479 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29f80aaca0 con 0x7f2a0415dc20 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.650+0000 7f2a0913a640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29f006b4d0 0x7f29f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.651+0000 7f2a0913a640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29f006b4d0 0x7f29f006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006fd0 tx=0x7f29fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.651+0000 7f2a0e17a740 1 Processor -- start 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.651+0000 7f2a0e17a740 1 -- start start 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2ab10 0x56471cc2d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 0x56471cc00f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471cb66da0 con 0x56471cbfd770 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471cb0fb80 con 0x56471cc2aef0 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56471cb62e40 con 0x56471cc2ab10 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 0x56471cc00f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 0x56471cc00f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40160/0 (socket says 172.21.15.60:40160) 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 -- 172.21.15.60:0/1326699381 learned_addr learned my addr 172.21.15.60:0/1326699381 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0913a640 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2ab10 0x56471cc2d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a08939640 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 -- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2ab10 msgr2=0x56471cc2d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2ab10 0x56471cc2d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 -- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 msgr2=0x56471cc2d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc2d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0993b640 1 -- 172.21.15.60:0/1326699381 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56471cb19f10 con 0x56471cbfd770 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.652+0000 7f2a0913a640 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2ab10 0x56471cc2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0993b640 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 0x56471cc00f80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009ef0 tx=0x7f29f400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf1d08a919e29f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f29eaffd640 1 -- 172.21.15.60:0/1326699381 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f401a070 con 0x56471cbfd770 2024-04-25T08:38:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f29eaffd640 1 -- 172.21.15.60:0/1326699381 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f400a5f0 con 0x56471cbfd770 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f29eaffd640 1 -- 172.21.15.60:0/1326699381 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f400a9c0 con 0x56471cbfd770 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1326699381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 msgr2=0x56471cc00f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 0x56471cc00f80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009ef0 tx=0x7f29f400c7d0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1326699381 shutdown_connections 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cbfd770 0x56471cc00f80 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc2d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1326699381 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2ab10 0x56471cc2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1326699381 >> 172.21.15.60:0/1326699381 conn(0x56471cbff9e0 msgr2=0x56471cc48320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1326699381 shutdown_connections 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.653+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1326699381 wait complete. 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.654+0000 7f2a0e17a740 1 -- start start 2024-04-25T08:38:38.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.654+0000 7f2a0e17a740 1 Processor -- start 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc22970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc25400 0x56471cc2ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471cc02b90 con 0x56471cc2ab10 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471cb054b0 con 0x56471cc2aef0 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56471cc02e70 con 0x56471cc25400 2024-04-25T08:38:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0e17a740 10 client.0 _ll_register_callbacks cb 0x56471cb45180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc22970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc22970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40170/0 (socket says 172.21.15.60:40170) 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 -- 172.21.15.60:0/1640962207 learned_addr learned my addr 172.21.15.60:0/1640962207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a08939640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0993b640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc25400 0x56471cc2ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 -- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc25400 msgr2=0x56471cc2ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc25400 0x56471cc2ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 -- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 msgr2=0x56471cc29590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc29590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0913a640 1 -- 172.21.15.60:0/1640962207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56471c9457d0 con 0x56471cc2ab10 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a08939640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc29590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.655+0000 7f2a0993b640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc25400 0x56471cc2ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0913a640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc22970 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f29fc014710 tx=0x7f29fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5224a355fe2d93b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f29dffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f29dffff640 1 -- 172.21.15.60:0/1640962207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc025070 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f29dffff640 1 -- 172.21.15.60:0/1640962207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29fc0158e0 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29f400a5f0 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f29dffff640 1 -- 172.21.15.60:0/1640962207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29fc030050 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 10 client.9618 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56471c87b3f0 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f29dffff640 1 -- 172.21.15.60:0/1640962207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f400a5f0 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f29dffff640 1 -- 172.21.15.60:0/1640962207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29fc033020 con 0x56471cc2ab10 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 10 client.9618 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 10 client.9618 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 20 client.9618 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc21f00 0x56471cc49450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 4 client.9618 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56471cb742e0 con 0x56471cc21f00 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.656+0000 7f2a08939640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc21f00 0x56471cc49450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f29dffff640 10 client.9618 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a08939640 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc21f00 0x56471cc49450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9cbd09b20acfd49 server_cookie=9675715a16f637e4 in_seq=0 out_seq=0 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f29dffff640 1 -- 172.21.15.60:0/1640962207 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56471cb742e0 con 0x56471cc21f00 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f29dffff640 10 client.9618 handle_command_reply: tid=0 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 client.9618 shutdown 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 msgr2=0x56471cc22970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc22970 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f29fc014710 tx=0x7f29fc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 shutdown_connections 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc21f00 0x56471cc49450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc25400 0x56471cc2ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2aef0 0x56471cc29590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1640962207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc22970 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.657+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 >> 172.21.15.60:0/1640962207 conn(0x56471cbfd770 msgr2=0x56471cbfe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.658+0000 7f29dffff640 0 client.9618 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.658+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 shutdown_connections 2024-04-25T08:38:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.658+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1640962207 wait complete. 2024-04-25T08:38:38.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.658+0000 7f2a0e17a740 20 client.9618 trim_cache size 0 max 16384 2024-04-25T08:38:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.673+0000 7f2a0e17a740 1 Processor -- start 2024-04-25T08:38:38.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.673+0000 7f2a0e17a740 1 -- start start 2024-04-25T08:38:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc25400 0x56471cc6ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc4b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc4b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a042006e0 con 0x56471cc2ab10 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041ff840 con 0x56471cc25400 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041f5010 con 0x56471cc2aef0 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc4b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc4b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40184/0 (socket says 172.21.15.60:40184) 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 -- 172.21.15.60:0/1853905128 learned_addr learned my addr 172.21.15.60:0/1853905128 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0913a640 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc25400 0x56471cc6ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a0993b640 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc4b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 -- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 msgr2=0x56471cc4b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc4b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 -- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc25400 msgr2=0x56471cc6ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc25400 0x56471cc6ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 -- 172.21.15.60:0/1853905128 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56471c83b0f0 con 0x56471cc2ab10 2024-04-25T08:38:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f2a08939640 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc4b280 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f29f80090b0 tx=0x7f29f8048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2a11b4153ab74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f29ea7fc640 1 -- 172.21.15.60:0/1853905128 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f807d350 con 0x56471cc2ab10 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f29ea7fc640 1 -- 172.21.15.60:0/1853905128 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f80819a0 con 0x56471cc2ab10 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.674+0000 7f29ea7fc640 1 -- 172.21.15.60:0/1853905128 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8077500 con 0x56471cc2ab10 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1853905128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 msgr2=0x56471cc4b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc4b280 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f29f80090b0 tx=0x7f29f8048040 comp rx=0 tx=0).stop 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1853905128 shutdown_connections 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc4b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc2ab10 0x56471cc4b280 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/1853905128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc25400 0x56471cc6ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1853905128 >> 172.21.15.60:0/1853905128 conn(0x56471cbfd770 msgr2=0x56471cc48320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1853905128 shutdown_connections 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.675+0000 7f2a0e17a740 1 -- 172.21.15.60:0/1853905128 wait complete. 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.676+0000 7f2a0e17a740 1 -- start start 2024-04-25T08:38:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.676+0000 7f2a0e17a740 1 Processor -- start 2024-04-25T08:38:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 0x56471cc2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2ab10 0x56471ccb2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc5a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56471cc22370 con 0x56471cc25400 2024-04-25T08:38:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041ff2d0 con 0x56471cc2ab10 2024-04-25T08:38:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x56471cc6a9f0 con 0x56471cc2aef0 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 10 client.0 _ll_register_callbacks cb 0x7f2a04062560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 0x56471cc2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 0x56471cc2d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40198/0 (socket says 172.21.15.60:40198) 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 -- 172.21.15.60:0/2110954547 learned_addr learned my addr 172.21.15.60:0/2110954547 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a08939640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2ab10 0x56471ccb2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0993b640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc5a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 -- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 msgr2=0x56471cc5a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc5a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 -- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2ab10 msgr2=0x56471ccb2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2ab10 0x56471ccb2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 -- 172.21.15.60:0/2110954547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56471cc5a5b0 con 0x56471cc25400 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0913a640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 0x56471cc2d7f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006ec0 tx=0x7f29fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cdc2fc8c43e955 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f29dffff640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f29dffff640 1 -- 172.21.15.60:0/2110954547 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc025070 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f29dffff640 1 -- 172.21.15.60:0/2110954547 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29fc02a440 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29f80819a0 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f29dffff640 1 -- 172.21.15.60:0/2110954547 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29fc03f050 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 10 client.9624 fetch_fsmap learned FSMap version 78 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.677+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56471cc56d10 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f29dffff640 1 -- 172.21.15.60:0/2110954547 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f80819a0 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f29dffff640 1 -- 172.21.15.60:0/2110954547 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 78) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29fc042020 con 0x56471cc25400 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 10 client.9624 fetch_fsmap finished waiting for FSMap version 78 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 10 client.9624 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.d 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 20 client.9624 populate_metadata read hostname 'smithi060' 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc241a0 0x56471cc57510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 4 client.9624 mds_command: new command op to 9381 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 --> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56471cb6e630 con 0x56471cc241a0 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a08939640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc241a0 0x56471cc57510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f29dffff640 10 client.9624 ms_handle_connect on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a08939640 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc241a0 0x56471cc57510 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe2b93b5456d235 server_cookie=fd3e6655a26b6cc3 in_seq=0 out_seq=0 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f29dffff640 1 -- 172.21.15.60:0/2110954547 <== mds.0 v2:172.21.15.100:6834/879630460 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x56471cb6e630 con 0x56471cc241a0 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f29dffff640 10 client.9624 handle_command_reply: tid=0 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.678+0000 7f2a0e17a740 1 client.9624 shutdown 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 msgr2=0x56471cc2d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 0x56471cc2d7f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006ec0 tx=0x7f29fc016000 comp rx=0 tx=0).stop 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 shutdown_connections 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:6834/879630460,v1:172.21.15.100:6836/879630460] conn(0x56471cc241a0 0x56471cc57510 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56471cc2aef0 0x56471cc5a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56471cc2ab10 0x56471ccb2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f29dffff640 0 client.9624 ms_handle_reset on v2:172.21.15.100:6834/879630460 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 --2- 172.21.15.60:0/2110954547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56471cc25400 0x56471cc2d7f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 >> 172.21.15.60:0/2110954547 conn(0x56471cbfd770 msgr2=0x56471cbf8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 shutdown_connections 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 1 -- 172.21.15.60:0/2110954547 wait complete. 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0e17a740 20 client.9624 trim_cache size 0 max 16384 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:38:38.678 INFO:teuthology.orchestra.run.smithi060.stdout: "scrubs": { 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "4f96fef4-98b3-406b-8e54-cbf1e541226d": { 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tag": "4f96fef4-98b3-406b-8e54-cbf1e541226d", 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: "options": "recursive" 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:38:38.679 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29f006b4d0 msgr2=0x7f29f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29f006b4d0 0x7f29f006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006fd0 tx=0x7f29fc008040 comp rx=0 tx=0).stop 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 msgr2=0x7f2a041e9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 0x7f2a041e9950 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f29f802a270 tx=0x7f29f80046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 shutdown_connections 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f29f006b4d0 0x7f29f006d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a04161710 0x7f2a0414f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2a0415dc20 0x7f2a041e9950 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 --2- 172.21.15.60:0/3842580479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2a040b9330 0x7f2a040b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.679+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 >> 172.21.15.60:0/3842580479 conn(0x7f2a040d0420 msgr2=0x7f2a041601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.680+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 shutdown_connections 2024-04-25T08:38:38.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.680+0000 7f2a0b39e640 1 -- 172.21.15.60:0/3842580479 wait complete. 2024-04-25T08:38:38.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:38.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.791+0000 7f8e8e1d5640 1 Processor -- start 2024-04-25T08:38:38.784 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.791+0000 7f8e8e1d5640 1 -- start start 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e8e1d5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e880d0ad0 0x7f8e88168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e8e1d5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e880b9940 con 0x7f8e880d2e90 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e8e1d5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e880d1df0 con 0x7f8e8815dd30 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e8e1d5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e880b7a50 con 0x7f8e880d0ad0 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e86ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8e880d2e90 0x7f8e880d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59244/0 (socket says 172.21.15.60:59244) 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e86ffd640 1 -- 172.21.15.60:0/258978152 learned_addr learned my addr 172.21.15.60:0/258978152 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e87fff640 1 --2- 172.21.15.60:0/258978152 >> v2:172.21.15.100:3300/0 conn(0x7f8e880d0ad0 0x7f8e88168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507106812 0 0) 0x7f8e880b9940 con 0x7f8e880d2e90 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e70003520 con 0x7f8e880d2e90 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581165506 0 0) 0x7f8e880d1df0 con 0x7f8e8815dd30 2024-04-25T08:38:38.786 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e880b9940 con 0x7f8e8815dd30 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e87fff640 1 -- 172.21.15.60:0/258978152 >> v1:172.21.15.100:6789/0 conn(0x7f8e8815dd30 legacy=0x7f8e880b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e87fff640 1 -- 172.21.15.60:0/258978152 >> v1:172.21.15.60:6789/0 conn(0x7f8e880d2e90 legacy=0x7f8e880d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.792+0000 7f8e87fff640 1 -- 172.21.15.60:0/258978152 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880d20d0 con 0x7f8e880d0ad0 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.793+0000 7f8e87fff640 1 --2- 172.21.15.60:0/258978152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d0ad0 0x7f8e88168760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009d90 tx=0x7f8e8002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ace232b33956ff44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.793+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031600 con 0x7f8e880d0ad0 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.793+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e800317a0 con 0x7f8e880d0ad0 2024-04-25T08:38:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.793+0000 7f8e867fc640 1 -- 172.21.15.60:0/258978152 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031b50 con 0x7f8e880d0ad0 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/258978152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d0ad0 msgr2=0x7f8e88168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/258978152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d0ad0 0x7f8e88168760 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009d90 tx=0x7f8e8002a5d0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/258978152 shutdown_connections 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/258978152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d0ad0 0x7f8e88168760 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/258978152 >> 172.21.15.60:0/258978152 conn(0x7f8e88061c30 msgr2=0x7f8e8806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/258978152 shutdown_connections 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.794+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/258978152 wait complete. 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.795+0000 7f8e8e1d5640 1 Processor -- start 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.795+0000 7f8e8e1d5640 1 -- start start 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e8e1d5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880d0ad0 0x7f8e88156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e8e1d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d2e90 0x7f8e880b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.790 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e8e1d5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 0x7f8e880b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e8e1d5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880d2970 con 0x7f8e8815dd30 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e8e1d5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880d2690 con 0x7f8e880d2e90 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e8e1d5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e880d1a10 con 0x7f8e880d0ad0 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 0x7f8e880b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e86ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d2e90 0x7f8e880b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e877fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880d0ad0 0x7f8e88156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 0x7f8e880b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40202/0 (socket says 172.21.15.60:40202) 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e86ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d2e90 0x7f8e880b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44372/0 (socket says 172.21.15.60:44372) 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 -- 172.21.15.60:0/3161576215 learned_addr learned my addr 172.21.15.60:0/3161576215 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 -- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880d0ad0 msgr2=0x7f8e88156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880d0ad0 0x7f8e88156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.791 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 -- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d2e90 msgr2=0x7f8e880b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d2e90 0x7f8e880b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 -- 172.21.15.60:0/3161576215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e880b31f0 con 0x7f8e8815dd30 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e87fff640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 0x7f8e880b4290 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009eb0 tx=0x7f8e80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.796+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003f070 con 0x7f8e8815dd30 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.797+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e80030850 con 0x7f8e8815dd30 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.797+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e880b34d0 con 0x7f8e8815dd30 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.797+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80030c40 con 0x7f8e8815dd30 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.797+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e880b3a90 con 0x7f8e8815dd30 2024-04-25T08:38:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.798+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e88073990 con 0x7f8e8815dd30 2024-04-25T08:38:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.798+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e80043410 con 0x7f8e8815dd30 2024-04-25T08:38:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.798+0000 7f8e6ffff640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b4d0 0x7f8e7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.799+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e800ab2e0 con 0x7f8e8815dd30 2024-04-25T08:38:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.799+0000 7f8e877fe640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b4d0 0x7f8e7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:38.793 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.799+0000 7f8e877fe640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b4d0 0x7f8e7406d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8e78007fd0 tx=0x7f8e78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:38.803+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e80079990 con 0x7f8e8815dd30 2024-04-25T08:38:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.019+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e880ceac0 con 0x7f8e8815dd30 2024-04-25T08:38:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.020+0000 7f8e6ffff640 1 -- 172.21.15.60:0/3161576215 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f8e8007da30 con 0x7f8e8815dd30 2024-04-25T08:38:39.013 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:39.013 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:39.013 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b4d0 msgr2=0x7f8e7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b4d0 0x7f8e7406d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8e78007fd0 tx=0x7f8e78009040 comp rx=0 tx=0).stop 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 msgr2=0x7f8e880b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 0x7f8e880b4290 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009eb0 tx=0x7f8e80004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 shutdown_connections 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e8815dd30 0x7f8e880b4290 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e7406b4d0 0x7f8e7406d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e880d2e90 0x7f8e880b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 --2- 172.21.15.60:0/3161576215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e880d0ad0 0x7f8e88156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 >> 172.21.15.60:0/3161576215 conn(0x7f8e88061c30 msgr2=0x7f8e880c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 shutdown_connections 2024-04-25T08:38:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.022+0000 7f8e8e1d5640 1 -- 172.21.15.60:0/3161576215 wait complete. 2024-04-25T08:38:39.025 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:38:39.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.132+0000 7fab74322640 1 Processor -- start 2024-04-25T08:38:39.125 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.132+0000 7fab74322640 1 -- start start 2024-04-25T08:38:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab74322640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fab6c0d0ad0 0x7fab6c168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab74322640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0b9940 con 0x7fab6c0d2e90 2024-04-25T08:38:39.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab74322640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0d1df0 con 0x7fab6c15dd30 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab74322640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c0b7a50 con 0x7fab6c0d0ad0 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab728bf640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fab6c0d0ad0 0x7fab6c168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab718bd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fab6c0d2e90 0x7fab6c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:59252/0 (socket says 172.21.15.60:59252) 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.133+0000 7fab718bd640 1 -- 172.21.15.60:0/376885584 learned_addr learned my addr 172.21.15.60:0/376885584 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab728bf640 1 -- 172.21.15.60:0/376885584 >> v1:172.21.15.100:6789/0 conn(0x7fab6c15dd30 legacy=0x7fab6c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab728bf640 1 -- 172.21.15.60:0/376885584 >> v1:172.21.15.60:6789/0 conn(0x7fab6c0d2e90 legacy=0x7fab6c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab710bc640 1 -- 172.21.15.60:0/376885584 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211264614 0 0) 0x7fab6c0b9940 con 0x7fab6c0d2e90 2024-04-25T08:38:39.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab728bf640 1 -- 172.21.15.60:0/376885584 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0d20d0 con 0x7fab6c0d0ad0 2024-04-25T08:38:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab728bf640 1 --2- 172.21.15.60:0/376885584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c168760 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009d90 tx=0x7fab5c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73081003e0886b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab710bc640 1 -- 172.21.15.60:0/376885584 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031600 con 0x7fab6c0d0ad0 2024-04-25T08:38:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab710bc640 1 -- 172.21.15.60:0/376885584 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c0317a0 con 0x7fab6c0d0ad0 2024-04-25T08:38:39.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.134+0000 7fab710bc640 1 -- 172.21.15.60:0/376885584 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031b70 con 0x7fab6c0d0ad0 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 -- 172.21.15.60:0/376885584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c0d0ad0 msgr2=0x7fab6c168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 --2- 172.21.15.60:0/376885584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c168760 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009d90 tx=0x7fab5c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 -- 172.21.15.60:0/376885584 shutdown_connections 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 --2- 172.21.15.60:0/376885584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c168760 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 -- 172.21.15.60:0/376885584 >> 172.21.15.60:0/376885584 conn(0x7fab6c061c30 msgr2=0x7fab6c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 -- 172.21.15.60:0/376885584 shutdown_connections 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.135+0000 7fab74322640 1 -- 172.21.15.60:0/376885584 wait complete. 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.136+0000 7fab74322640 1 Processor -- start 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.136+0000 7fab74322640 1 -- start start 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab74322640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab74322640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab6c0d2e90 0x7fab6c1582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab74322640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 0x7fab6c14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab74322640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0d2970 con 0x7fab6c15dd30 2024-04-25T08:38:39.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab74322640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0d2690 con 0x7fab6c0d0ad0 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab74322640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0d1a10 con 0x7fab6c0d2e90 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 0x7fab6c14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 0x7fab6c14ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40222/0 (socket says 172.21.15.60:40222) 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 -- 172.21.15.60:0/1182208551 learned_addr learned my addr 172.21.15.60:0/1182208551 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab720be640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab718bd640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab6c0d2e90 0x7fab6c1582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 -- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab6c0d2e90 msgr2=0x7fab6c1582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab6c0d2e90 0x7fab6c1582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 -- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab6c0d0ad0 msgr2=0x7fab6c159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab728bf640 1 -- 172.21.15.60:0/1182208551 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c1589d0 con 0x7fab6c15dd30 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.137+0000 7fab720be640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:39.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.138+0000 7fab728bf640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 0x7fab6c14ae50 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009eb0 tx=0x7fab5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.138+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c03f070 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.138+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c030b60 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.138+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c158cb0 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.138+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c043410 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.138+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c0b31f0 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.139+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c073990 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.139+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab5c03b410 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.139+0000 7fab6affd640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fab5806b4d0 0x7fab5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.140+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab5c0ab690 con 0x7fab6c15dd30 2024-04-25T08:38:39.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.140+0000 7fab720be640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fab5806b4d0 0x7fab5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:39.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.140+0000 7fab720be640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fab5806b4d0 0x7fab5806d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fab64007fd0 tx=0x7fab64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:39.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.144+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab5c07a440 con 0x7fab6c15dd30 2024-04-25T08:38:39.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.397+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fab6c15d3e0 con 0x7fab6c15dd30 2024-04-25T08:38:39.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.397+0000 7fab6affd640 1 -- 172.21.15.60:0/1182208551 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fab5c079d40 con 0x7fab6c15dd30 2024-04-25T08:38:39.390 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:39.390 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":797,"data_bytes":812100643,"bytes_used":1957388288,"bytes_avail":771136724992,"bytes_total":773094113280,"write_bytes_sec":67616338,"read_op_per_sec":0,"write_op_per_sec":65},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T08:37:57.516705+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:38:39.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fab5806b4d0 msgr2=0x7fab5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fab5806b4d0 0x7fab5806d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fab64007fd0 tx=0x7fab64009040 comp rx=0 tx=0).stop 2024-04-25T08:38:39.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 msgr2=0x7fab6c14ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 0x7fab6c14ae50 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009eb0 tx=0x7fab5c004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 shutdown_connections 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fab6c15dd30 0x7fab6c14ae50 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fab6c0d2e90 0x7fab6c1582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fab5806b4d0 0x7fab5806d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 --2- 172.21.15.60:0/1182208551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fab6c0d0ad0 0x7fab6c159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 >> 172.21.15.60:0/1182208551 conn(0x7fab6c061c30 msgr2=0x7fab6c160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 shutdown_connections 2024-04-25T08:38:39.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:39.399+0000 7fab74322640 1 -- 172.21.15.60:0/1182208551 wait complete. 2024-04-25T08:38:39.403 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:38:40.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:40.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.525+0000 7f97c93ce640 1 Processor -- start 2024-04-25T08:38:40.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.525+0000 7f97c93ce640 1 -- start start 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c93ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c93ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97c415e100 0x7f97c40d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c93ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97c415e560 0x7f97c40d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c93ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c40d1c00 con 0x7f97c415e100 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c93ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c415eac0 con 0x7f97c415e560 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c93ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c415eda0 con 0x7f97c415dd20 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97c415e100 0x7f97c40d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f97c415e100 0x7f97c40d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40230/0 (socket says 172.21.15.60:40230) 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 -- 172.21.15.60:0/774008644 learned_addr learned my addr 172.21.15.60:0/774008644 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c37fe640 1 --2- 172.21.15.60:0/774008644 >> v2:172.21.15.100:3300/0 conn(0x7f97c415e560 0x7f97c40d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c2ffd640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 -- 172.21.15.60:0/774008644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 msgr2=0x7f97c40b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 -- 172.21.15.60:0/774008644 >> v2:172.21.15.100:3300/0 conn(0x7f97c415e560 msgr2=0x7f97c40d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 --2- 172.21.15.60:0/774008644 >> v2:172.21.15.100:3300/0 conn(0x7f97c415e560 0x7f97c40d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c27fc640 1 -- 172.21.15.60:0/774008644 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c415f080 con 0x7f97c415e100 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.526+0000 7f97c2ffd640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.527+0000 7f97c27fc640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40d0aa0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f97c40cab60 tx=0x7f97b802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffcbeda94b99ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.527+0000 7f97c1ffb640 1 -- 172.21.15.60:0/774008644 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8037070 con 0x7f97c415e100 2024-04-25T08:38:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.527+0000 7f97c1ffb640 1 -- 172.21.15.60:0/774008644 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8032440 con 0x7f97c415e100 2024-04-25T08:38:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.527+0000 7f97c1ffb640 1 -- 172.21.15.60:0/774008644 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8032810 con 0x7f97c415e100 2024-04-25T08:38:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 -- 172.21.15.60:0/774008644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 msgr2=0x7f97c40d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40d0aa0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f97c40cab60 tx=0x7f97b802a140 comp rx=0 tx=0).stop 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 -- 172.21.15.60:0/774008644 shutdown_connections 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 --2- 172.21.15.60:0/774008644 >> v2:172.21.15.100:3300/0 conn(0x7f97c415e560 0x7f97c40d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40d0aa0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 --2- 172.21.15.60:0/774008644 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 -- 172.21.15.60:0/774008644 >> 172.21.15.60:0/774008644 conn(0x7f97c4061a20 msgr2=0x7f97c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 -- 172.21.15.60:0/774008644 shutdown_connections 2024-04-25T08:38:40.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.528+0000 7f97c93ce640 1 -- 172.21.15.60:0/774008644 wait complete. 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.529+0000 7f97c93ce640 1 Processor -- start 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.529+0000 7f97c93ce640 1 -- start start 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c93ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c93ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c93ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c415e560 0x7f97c4152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c93ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40d3c20 con 0x7f97c415e100 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c93ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40d3aa0 con 0x7f97c415e560 2024-04-25T08:38:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c93ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40d3da0 con 0x7f97c415dd20 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40236/0 (socket says 172.21.15.60:40236) 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 -- 172.21.15.60:0/2213868545 learned_addr learned my addr 172.21.15.60:0/2213868545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c2ffd640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c37fe640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c415e560 0x7f97c4152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 -- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 msgr2=0x7f97c40ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 -- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c415e560 msgr2=0x7f97c4152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c415e560 0x7f97c4152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c27fc640 1 -- 172.21.15.60:0/2213868545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c40b1ab0 con 0x7f97c415e100 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.530+0000 7f97c37fe640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c415e560 0x7f97c4152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.531+0000 7f97c27fc640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40aea20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f97b802a180 tx=0x7f97b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.531+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8041020 con 0x7f97c415e100 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.531+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b80306c0 con 0x7f97c415e100 2024-04-25T08:38:40.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.531+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c40b1d90 con 0x7f97c415e100 2024-04-25T08:38:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.531+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8037040 con 0x7f97c415e100 2024-04-25T08:38:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.531+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c40b2350 con 0x7f97c415e100 2024-04-25T08:38:40.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.532+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c4073990 con 0x7f97c415e100 2024-04-25T08:38:40.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.532+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97b8044020 con 0x7f97c415e100 2024-04-25T08:38:40.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.533+0000 7f97ab7fe640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97b006b4d0 0x7f97b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.533+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97b80aaaf0 con 0x7f97c415e100 2024-04-25T08:38:40.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.533+0000 7f97c2ffd640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97b006b4d0 0x7f97b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.534+0000 7f97c2ffd640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97b006b4d0 0x7f97b006d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f97b4006b10 tx=0x7f97b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:40.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.537+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97b804b030 con 0x7f97c415e100 2024-04-25T08:38:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.762+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97c4062050 con 0x7f97c415e100 2024-04-25T08:38:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.763+0000 7f97ab7fe640 1 -- 172.21.15.60:0/2213868545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f97b80797e0 con 0x7f97c415e100 2024-04-25T08:38:40.755 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:40.756 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:40.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97b006b4d0 msgr2=0x7f97b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97b006b4d0 0x7f97b006d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f97b4006b10 tx=0x7f97b4008040 comp rx=0 tx=0).stop 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 msgr2=0x7f97c40aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40aea20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f97b802a180 tx=0x7f97b80046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 shutdown_connections 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f97b006b4d0 0x7f97b006d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97c415e560 0x7f97c4152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f97c415e100 0x7f97c40aea20 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 --2- 172.21.15.60:0/2213868545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f97c415dd20 0x7f97c40ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 >> 172.21.15.60:0/2213868545 conn(0x7f97c4061a20 msgr2=0x7f97c415f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 shutdown_connections 2024-04-25T08:38:40.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.765+0000 7f97c93ce640 1 -- 172.21.15.60:0/2213868545 wait complete. 2024-04-25T08:38:40.768 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:38:40.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.876+0000 7f83ee8fc640 1 Processor -- start 2024-04-25T08:38:40.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.876+0000 7f83ee8fc640 1 -- start start 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83ee8fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83e80ad5a0 0x7f83e806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83ee8fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83e806df50 0x7f83e806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83ee8fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e80b3760 con 0x7f83e815da80 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83ee8fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80b3520 con 0x7f83e80ad5a0 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83ee8fc640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80b4370 con 0x7f83e806df50 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83e80ad5a0 0x7f83e806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83e80ad5a0 0x7f83e806da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40248/0 (socket says 172.21.15.60:40248) 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 -- 172.21.15.60:0/4205261675 learned_addr learned my addr 172.21.15.60:0/4205261675 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83ece99640 1 --2- 172.21.15.60:0/4205261675 >> v2:172.21.15.100:3300/0 conn(0x7f83e806df50 0x7f83e806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 -- 172.21.15.60:0/4205261675 >> v2:172.21.15.100:3300/0 conn(0x7f83e806df50 msgr2=0x7f83e806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 --2- 172.21.15.60:0/4205261675 >> v2:172.21.15.100:3300/0 conn(0x7f83e806df50 0x7f83e806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 -- 172.21.15.60:0/4205261675 >> v1:172.21.15.100:6789/0 conn(0x7f83e815da80 legacy=0x7f83e80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.877+0000 7f83e77fe640 1 -- 172.21.15.60:0/4205261675 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80b3a40 con 0x7f83e80ad5a0 2024-04-25T08:38:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.878+0000 7f83e77fe640 1 --2- 172.21.15.60:0/4205261675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e80ad5a0 0x7f83e806da10 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b42e0 tx=0x7f83dc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23343671e51e457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.878+0000 7f83e6ffd640 1 -- 172.21.15.60:0/4205261675 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc037070 con 0x7f83e80ad5a0 2024-04-25T08:38:40.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.878+0000 7f83e6ffd640 1 -- 172.21.15.60:0/4205261675 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc02ae50 con 0x7f83e80ad5a0 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.878+0000 7f83e6ffd640 1 -- 172.21.15.60:0/4205261675 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc032860 con 0x7f83e80ad5a0 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 -- 172.21.15.60:0/4205261675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e80ad5a0 msgr2=0x7f83e806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/4205261675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e80ad5a0 0x7f83e806da10 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b42e0 tx=0x7f83dc02a300 comp rx=0 tx=0).stop 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 -- 172.21.15.60:0/4205261675 shutdown_connections 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/4205261675 >> v2:172.21.15.100:3300/0 conn(0x7f83e806df50 0x7f83e806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/4205261675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e80ad5a0 0x7f83e806da10 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 -- 172.21.15.60:0/4205261675 >> 172.21.15.60:0/4205261675 conn(0x7f83e8071b00 msgr2=0x7f83e8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 -- 172.21.15.60:0/4205261675 shutdown_connections 2024-04-25T08:38:40.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.879+0000 7f83ee8fc640 1 -- 172.21.15.60:0/4205261675 wait complete. 2024-04-25T08:38:40.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.880+0000 7f83ee8fc640 1 Processor -- start 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 -- start start 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 0x7f83e8147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83e80ad5a0 0x7f83e80b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e815da80 0x7f83e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80b4000 con 0x7f83e806df50 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e815de60 con 0x7f83e815da80 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ee8fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e806c130 con 0x7f83e80ad5a0 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 0x7f83e8147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 0x7f83e8147820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:40264/0 (socket says 172.21.15.60:40264) 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83e7fff640 1 -- 172.21.15.60:0/3309576303 learned_addr learned my addr 172.21.15.60:0/3309576303 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83e77fe640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83e80ad5a0 0x7f83e80b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.881+0000 7f83ece99640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e815da80 0x7f83e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e7fff640 1 -- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83e80ad5a0 msgr2=0x7f83e80b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e7fff640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83e80ad5a0 0x7f83e80b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e7fff640 1 -- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e815da80 msgr2=0x7f83e80b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e7fff640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e815da80 0x7f83e80b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e7fff640 1 -- 172.21.15.60:0/3309576303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80ce9f0 con 0x7f83e806df50 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e7fff640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 0x7f83e8147820 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f83d800c3f0 tx=0x7f83d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d80052d0 con 0x7f83e806df50 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8005490 con 0x7f83e806df50 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e815a520 con 0x7f83e806df50 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8007260 con 0x7f83e806df50 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.882+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e8147d60 con 0x7f83e806df50 2024-04-25T08:38:40.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.883+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e8073990 con 0x7f83e806df50 2024-04-25T08:38:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.884+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83d8007400 con 0x7f83e806df50 2024-04-25T08:38:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.884+0000 7f83e4ff9640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83d406b4d0 0x7f83d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.884+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83d80897c0 con 0x7f83e806df50 2024-04-25T08:38:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.885+0000 7f83e77fe640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83d406b4d0 0x7f83d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:40.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.885+0000 7f83e77fe640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83d406b4d0 0x7f83d406d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:40.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:40.889+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83d8057e60 con 0x7f83e806df50 2024-04-25T08:38:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.138+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f83e80729a0 con 0x7f83e806df50 2024-04-25T08:38:41.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.138+0000 7f83e4ff9640 1 -- 172.21.15.60:0/3309576303 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f83d805ce20 con 0x7f83e806df50 2024-04-25T08:38:41.131 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:41.131 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":581,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":797,"data_bytes":812100643,"bytes_used":1957388288,"bytes_avail":771136724992,"bytes_total":773094113280,"write_bytes_sec":67617225,"read_op_per_sec":0,"write_op_per_sec":65},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T08:37:57.516705+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:38:41.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83d406b4d0 msgr2=0x7f83d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83d406b4d0 0x7f83d406d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 msgr2=0x7f83e8147820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 0x7f83e8147820 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f83d800c3f0 tx=0x7f83d800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 shutdown_connections 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f83d406b4d0 0x7f83d406d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83e815da80 0x7f83e80b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83e80ad5a0 0x7f83e80b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 --2- 172.21.15.60:0/3309576303 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83e806df50 0x7f83e8147820 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 >> 172.21.15.60:0/3309576303 conn(0x7f83e8071b00 msgr2=0x7f83e806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 shutdown_connections 2024-04-25T08:38:41.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:41.140+0000 7f83ee8fc640 1 -- 172.21.15.60:0/3309576303 wait complete. 2024-04-25T08:38:41.144 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:38:42.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.266+0000 7fec26e07640 1 Processor -- start 2024-04-25T08:38:42.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.266+0000 7fec26e07640 1 -- start start 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec26e07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec200d2e90 0x7fec200d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec26e07640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fec200d0900 0x7fec200b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec26e07640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200b9940 con 0x7fec2015dd30 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec26e07640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec200d0d40 con 0x7fec200d2e90 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec26e07640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec200d1df0 con 0x7fec200d0900 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec200d2e90 0x7fec200d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec1ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fec200d2e90 0x7fec200d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39010/0 (socket says 172.21.15.60:39010) 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec1ffff640 1 -- 172.21.15.60:0/2355273773 learned_addr learned my addr 172.21.15.60:0/2355273773 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.267+0000 7fec24ba3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fec2015dd30 0x7fec200b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60914/0 (socket says 172.21.15.60:60914) 2024-04-25T08:38:42.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.268+0000 7fec253a4640 1 --2- 172.21.15.60:0/2355273773 >> v2:172.21.15.100:3300/0 conn(0x7fec200d0900 0x7fec200b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.268+0000 7fec1ffff640 1 -- 172.21.15.60:0/2355273773 >> v2:172.21.15.100:3300/0 conn(0x7fec200d0900 msgr2=0x7fec200b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.268+0000 7fec1ffff640 1 --2- 172.21.15.60:0/2355273773 >> v2:172.21.15.100:3300/0 conn(0x7fec200d0900 0x7fec200b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.268+0000 7fec1ffff640 1 -- 172.21.15.60:0/2355273773 >> v1:172.21.15.60:6789/0 conn(0x7fec2015dd30 legacy=0x7fec200b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.268+0000 7fec1ffff640 1 -- 172.21.15.60:0/2355273773 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200d20d0 con 0x7fec200d2e90 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.268+0000 7fec253a4640 1 --2- 172.21.15.60:0/2355273773 >> v2:172.21.15.100:3300/0 conn(0x7fec200d0900 0x7fec200b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec1ffff640 1 --2- 172.21.15.60:0/2355273773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d2e90 0x7fec200d03c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fec200ce1d0 tx=0x7fec1002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946140b3a83af221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec1f7fe640 1 -- 172.21.15.60:0/2355273773 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10004030 con 0x7fec200d2e90 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec1f7fe640 1 -- 172.21.15.60:0/2355273773 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec100041d0 con 0x7fec200d2e90 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec1f7fe640 1 -- 172.21.15.60:0/2355273773 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10003610 con 0x7fec200d2e90 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 -- 172.21.15.60:0/2355273773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d2e90 msgr2=0x7fec200d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 --2- 172.21.15.60:0/2355273773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d2e90 0x7fec200d03c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fec200ce1d0 tx=0x7fec1002a140 comp rx=0 tx=0).stop 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 -- 172.21.15.60:0/2355273773 shutdown_connections 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 --2- 172.21.15.60:0/2355273773 >> v2:172.21.15.100:3300/0 conn(0x7fec200d0900 0x7fec200b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 --2- 172.21.15.60:0/2355273773 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d2e90 0x7fec200d03c0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 -- 172.21.15.60:0/2355273773 >> 172.21.15.60:0/2355273773 conn(0x7fec20061c30 msgr2=0x7fec2006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 -- 172.21.15.60:0/2355273773 shutdown_connections 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.269+0000 7fec26e07640 1 -- 172.21.15.60:0/2355273773 wait complete. 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.270+0000 7fec26e07640 1 Processor -- start 2024-04-25T08:38:42.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 -- start start 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 0x7fec200ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec200d2e90 0x7fec200caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015dd30 0x7fec2014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d2970 con 0x7fec200d0900 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d2690 con 0x7fec2015dd30 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.271+0000 7fec26e07640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec200d1a10 con 0x7fec200d2e90 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 0x7fec200ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 0x7fec200ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39020/0 (socket says 172.21.15.60:39020) 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 -- 172.21.15.60:0/155059256 learned_addr learned my addr 172.21.15.60:0/155059256 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec253a4640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015dd30 0x7fec2014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec1ffff640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec200d2e90 0x7fec200caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 -- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec200d2e90 msgr2=0x7fec200caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec200d2e90 0x7fec200caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 -- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015dd30 msgr2=0x7fec2014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015dd30 0x7fec2014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 -- 172.21.15.60:0/155059256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200cafc0 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec1ffff640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec200d2e90 0x7fec200caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec24ba3640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 0x7fec200ce620 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c760 tx=0x7fec1800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18017070 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800a640 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec200cd580 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800aa10 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.272+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec200cdb40 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.273+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec200c6e10 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.274+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec1801a070 con 0x7fec200d0900 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.274+0000 7fec1d7fa640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec0006b4d0 0x7fec0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.274+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec18089b00 con 0x7fec200d0900 2024-04-25T08:38:42.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.275+0000 7fec1ffff640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec0006b4d0 0x7fec0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.275+0000 7fec1ffff640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec0006b4d0 0x7fec0006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fec10033000 tx=0x7fec1002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:42.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.279+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec200c6e10 con 0x7fec200d0900 2024-04-25T08:38:42.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.496+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec20073990 con 0x7fec200d0900 2024-04-25T08:38:42.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.497+0000 7fec1d7fa640 1 -- 172.21.15.60:0/155059256 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fec18089e20 con 0x7fec200d0900 2024-04-25T08:38:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:42.490 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:42.490 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec0006b4d0 msgr2=0x7fec0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec0006b4d0 0x7fec0006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fec10033000 tx=0x7fec1002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 msgr2=0x7fec200ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 0x7fec200ce620 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c760 tx=0x7fec1800cc20 comp rx=0 tx=0).stop 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 shutdown_connections 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fec0006b4d0 0x7fec0006d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec2015dd30 0x7fec2014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fec200d2e90 0x7fec200caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 --2- 172.21.15.60:0/155059256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fec200d0900 0x7fec200ce620 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 >> 172.21.15.60:0/155059256 conn(0x7fec20061c30 msgr2=0x7fec200b85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 shutdown_connections 2024-04-25T08:38:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.499+0000 7fec26e07640 1 -- 172.21.15.60:0/155059256 wait complete. 2024-04-25T08:38:42.502 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:38:42.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.610+0000 7f6af633b640 1 Processor -- start 2024-04-25T08:38:42.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.610+0000 7f6af633b640 1 -- start start 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6af633b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6af015db70 0x7f6af0162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6af633b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af0162690 con 0x7f6af00b5c30 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6af633b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af00ceee0 con 0x7f6af00b6cb0 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6af633b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00d1d10 con 0x7f6af015db70 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6aeffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6af015db70 0x7f6af0162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6af48d8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6af00b5c30 0x7f6af00d27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60922/0 (socket says 172.21.15.60:60922) 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6af48d8640 1 -- 172.21.15.60:0/3748579467 learned_addr learned my addr 172.21.15.60:0/3748579467 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.611+0000 7f6aeffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6af015db70 0x7f6af0162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39032/0 (socket says 172.21.15.60:39032) 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeffff640 1 -- 172.21.15.60:0/3748579467 >> v1:172.21.15.100:6789/0 conn(0x7f6af00b6cb0 legacy=0x7f6af00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeffff640 1 -- 172.21.15.60:0/3748579467 >> v1:172.21.15.60:6789/0 conn(0x7f6af00b5c30 legacy=0x7f6af00d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeffff640 1 -- 172.21.15.60:0/3748579467 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00d1ff0 con 0x7f6af015db70 2024-04-25T08:38:42.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeffff640 1 --2- 172.21.15.60:0/3748579467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af015db70 0x7f6af0162150 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097a0 tx=0x7f6ae002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e51a76ccc47efbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:42.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeeffd640 1 -- 172.21.15.60:0/3748579467 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0037070 con 0x7f6af015db70 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeeffd640 1 -- 172.21.15.60:0/3748579467 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0032440 con 0x7f6af015db70 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.612+0000 7f6aeeffd640 1 -- 172.21.15.60:0/3748579467 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0032810 con 0x7f6af015db70 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 -- 172.21.15.60:0/3748579467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af015db70 msgr2=0x7f6af0162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 --2- 172.21.15.60:0/3748579467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af015db70 0x7f6af0162150 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097a0 tx=0x7f6ae002a230 comp rx=0 tx=0).stop 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 -- 172.21.15.60:0/3748579467 shutdown_connections 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 --2- 172.21.15.60:0/3748579467 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af015db70 0x7f6af0162150 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 -- 172.21.15.60:0/3748579467 >> 172.21.15.60:0/3748579467 conn(0x7f6af00d0520 msgr2=0x7f6af00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 -- 172.21.15.60:0/3748579467 shutdown_connections 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.613+0000 7f6af633b640 1 -- 172.21.15.60:0/3748579467 wait complete. 2024-04-25T08:38:42.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.614+0000 7f6af633b640 1 Processor -- start 2024-04-25T08:38:42.611 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.614+0000 7f6af633b640 1 -- start start 2024-04-25T08:38:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af633b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6af00b5c30 0x7f6af00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af633b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 0x7f6af0155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af633b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af015db70 0x7f6af0155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af633b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00ce530 con 0x7f6af00b6cb0 2024-04-25T08:38:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af633b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af015df50 con 0x7f6af015db70 2024-04-25T08:38:42.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af633b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00cf1c0 con 0x7f6af00b5c30 2024-04-25T08:38:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 0x7f6af0155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6aef7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 0x7f6af0155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39042/0 (socket says 172.21.15.60:39042) 2024-04-25T08:38:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6aef7fe640 1 -- 172.21.15.60:0/816403935 learned_addr learned my addr 172.21.15.60:0/816403935 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6aeffff640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6af00b5c30 0x7f6af00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6af48d8640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af015db70 0x7f6af0155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.615+0000 7f6aef7fe640 1 -- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6af00b5c30 msgr2=0x7f6af00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aef7fe640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6af00b5c30 0x7f6af00cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aef7fe640 1 -- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af015db70 msgr2=0x7f6af0155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aef7fe640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af015db70 0x7f6af0155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aef7fe640 1 -- 172.21.15.60:0/816403935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af0153a30 con 0x7f6af00b6cb0 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6af48d8640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af015db70 0x7f6af0155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aef7fe640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 0x7f6af0155a50 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c440 tx=0x7f6ae4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40052d0 con 0x7f6af00b6cb0 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae4005470 con 0x7f6af00b6cb0 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af015f070 con 0x7f6af00b6cb0 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4007260 con 0x7f6af00b6cb0 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.616+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af015f630 con 0x7f6af00b6cb0 2024-04-25T08:38:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.617+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af0153c00 con 0x7f6af00b6cb0 2024-04-25T08:38:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.617+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ae4007400 con 0x7f6af00b6cb0 2024-04-25T08:38:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.618+0000 7f6aecff9640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6adc06b4d0 0x7f6adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.618+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ae40897f0 con 0x7f6af00b6cb0 2024-04-25T08:38:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.618+0000 7f6aeffff640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6adc06b4d0 0x7f6adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:42.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.619+0000 7f6aeffff640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6adc06b4d0 0x7f6adc06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0034000 tx=0x7f6ae002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:42.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.622+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ae4057e90 con 0x7f6af00b6cb0 2024-04-25T08:38:42.866 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.874+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6af00d1820 con 0x7f6af00b6cb0 2024-04-25T08:38:42.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.874+0000 7f6aecff9640 1 -- 172.21.15.60:0/816403935 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f6ae405bf30 con 0x7f6af00b6cb0 2024-04-25T08:38:42.867 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:42.867 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":583,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076492608,"bytes_used":2595225600,"bytes_avail":770498887680,"bytes_total":773094113280,"write_bytes_sec":89650141,"read_op_per_sec":0,"write_op_per_sec":87},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T08:37:57.516705+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6adc06b4d0 msgr2=0x7f6adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6adc06b4d0 0x7f6adc06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0034000 tx=0x7f6ae002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 msgr2=0x7f6af0155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 0x7f6af0155a50 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c440 tx=0x7f6ae4001e70 comp rx=0 tx=0).stop 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 shutdown_connections 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6adc06b4d0 0x7f6adc06d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6af015db70 0x7f6af0155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6af00b6cb0 0x7f6af0155a50 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 --2- 172.21.15.60:0/816403935 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6af00b5c30 0x7f6af00cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:42.869 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 >> 172.21.15.60:0/816403935 conn(0x7f6af00d0520 msgr2=0x7f6af00b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:42.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 shutdown_connections 2024-04-25T08:38:42.870 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:42.876+0000 7f6af633b640 1 -- 172.21.15.60:0/816403935 wait complete. 2024-04-25T08:38:42.879 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:38:43.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:43.994 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.000+0000 7f3b39efc640 1 Processor -- start 2024-04-25T08:38:43.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.000+0000 7f3b39efc640 1 -- start start 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.001+0000 7f3b39efc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 0x7f3b341625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.001+0000 7f3b39efc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340cce50 con 0x7f3b340cc8e0 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.001+0000 7f3b39efc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b34161af0 con 0x7f3b340d1810 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.001+0000 7f3b39efc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34161dd0 con 0x7f3b3415dc90 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.001+0000 7f3b32ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3b340cc8e0 0x7f3b3415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60930/0 (socket says 172.21.15.60:60930) 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.001+0000 7f3b32ffd640 1 -- 172.21.15.60:0/110390329 learned_addr learned my addr 172.21.15.60:0/110390329 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b337fe640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 0x7f3b341625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023137929 0 0) 0x7f3b340cce50 con 0x7f3b340cc8e0 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b1c003520 con 0x7f3b340cc8e0 2024-04-25T08:38:43.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634535522 0 0) 0x7f3b34161af0 con 0x7f3b340d1810 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b340cce50 con 0x7f3b340d1810 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104988074 0 0) 0x7f3b1c003520 con 0x7f3b340cc8e0 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 msgr2=0x7f3b341625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 0x7f3b341625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 >> v1:172.21.15.100:6789/0 conn(0x7f3b340d1810 legacy=0x7f3b340cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b341620b0 con 0x7f3b340cc8e0 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f3b28001670 con 0x7f3b340cc8e0 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 >> v1:172.21.15.60:6789/0 conn(0x7f3b340cc8e0 legacy=0x7f3b3415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 0x7f3b1c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c0040d0 0x7f3b1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1c004d40 0x7f3b1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34161dd0 con 0x7f3b1c0040d0 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c00bb50 con 0x7f3b1c004d40 2024-04-25T08:38:44.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c00bd20 con 0x7f3b3415dc90 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b32ffd640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c0040d0 0x7f3b1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b337fe640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 0x7f3b1c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b33fff640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1c004d40 0x7f3b1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b32ffd640 1 -- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 msgr2=0x7f3b1c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b32ffd640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b3415dc90 0x7f3b1c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b32ffd640 1 -- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1c004d40 msgr2=0x7f3b1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b32ffd640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b1c004d40 0x7f3b1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.002+0000 7f3b32ffd640 1 -- 172.21.15.60:0/110390329 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c003520 con 0x7f3b1c0040d0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.003+0000 7f3b32ffd640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c0040d0 0x7f3b1c004800 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c160 tx=0x7f3b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65378c17aec0355c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.003+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28007cc0 con 0x7f3b1c0040d0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.003+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b28005af0 con 0x7f3b1c0040d0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.003+0000 7f3b327fc640 1 -- 172.21.15.60:0/110390329 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28007070 con 0x7f3b1c0040d0 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b39efc640 1 -- 172.21.15.60:0/110390329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c0040d0 msgr2=0x7f3b1c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b39efc640 1 --2- 172.21.15.60:0/110390329 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b1c0040d0 0x7f3b1c004800 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c160 tx=0x7f3b1c005830 comp rx=0 tx=0).stop 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b33fff640 1 -- 172.21.15.60:0/110390329 reap_dead start 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b39efc640 1 -- 172.21.15.60:0/110390329 shutdown_connections 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b39efc640 1 -- 172.21.15.60:0/110390329 >> 172.21.15.60:0/110390329 conn(0x7f3b3406f590 msgr2=0x7f3b3406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b39efc640 1 -- 172.21.15.60:0/110390329 shutdown_connections 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.004+0000 7f3b39efc640 1 -- 172.21.15.60:0/110390329 wait complete. 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.005+0000 7f3b39efc640 1 Processor -- start 2024-04-25T08:38:44.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 -- start start 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0040d0 0x7f3b341f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b340cb020 0x7f3b34153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 0x7f3b340b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3415e1f0 con 0x7f3b340b3280 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3415e070 con 0x7f3b340cb020 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b39efc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341625d0 con 0x7f3b1c0040d0 2024-04-25T08:38:44.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 0x7f3b340b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 0x7f3b340b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39072/0 (socket says 172.21.15.60:39072) 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.006+0000 7f3b33fff640 1 -- 172.21.15.60:0/2766799757 learned_addr learned my addr 172.21.15.60:0/2766799757 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b32ffd640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b340cb020 0x7f3b34153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b337fe640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0040d0 0x7f3b341f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b33fff640 1 -- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0040d0 msgr2=0x7f3b341f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b33fff640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0040d0 0x7f3b341f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b33fff640 1 -- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b340cb020 msgr2=0x7f3b34153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b33fff640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b340cb020 0x7f3b34153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b33fff640 1 -- 172.21.15.60:0/2766799757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340cb790 con 0x7f3b340b3280 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b32ffd640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b340cb020 0x7f3b34153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b33fff640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 0x7f3b340b3730 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f3b340cd060 tx=0x7f3b2c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c019070 con 0x7f3b340b3280 2024-04-25T08:38:44.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c006d60 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b341f76c0 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c003bd0 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.007+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b341f7c40 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.008+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34073990 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.009+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b2c01c070 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.009+0000 7f3b1bfff640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b2006b4d0 0x7f3b2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.009+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b2c089750 con 0x7f3b340b3280 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.009+0000 7f3b337fe640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b2006b4d0 0x7f3b2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.010+0000 7f3b337fe640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b2006b4d0 0x7f3b2006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009a20 tx=0x7f3b24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:44.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.014+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b2c0584d0 con 0x7f3b340b3280 2024-04-25T08:38:44.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.229+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b340cc070 con 0x7f3b340b3280 2024-04-25T08:38:44.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.229+0000 7f3b1bfff640 1 -- 172.21.15.60:0/2766799757 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f3b2c057dd0 con 0x7f3b340b3280 2024-04-25T08:38:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:44.223 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:44.223 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:44.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b2006b4d0 msgr2=0x7f3b2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b2006b4d0 0x7f3b2006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009a20 tx=0x7f3b24008040 comp rx=0 tx=0).stop 2024-04-25T08:38:44.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 msgr2=0x7f3b340b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 0x7f3b340b3730 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f3b340cd060 tx=0x7f3b2c00fd40 comp rx=0 tx=0).stop 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 shutdown_connections 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3b340b3280 0x7f3b340b3730 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3b2006b4d0 0x7f3b2006d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b340cb020 0x7f3b34153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 --2- 172.21.15.60:0/2766799757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3b1c0040d0 0x7f3b341f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 >> 172.21.15.60:0/2766799757 conn(0x7f3b3406f590 msgr2=0x7f3b3415cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 shutdown_connections 2024-04-25T08:38:44.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.231+0000 7f3b39efc640 1 -- 172.21.15.60:0/2766799757 wait complete. 2024-04-25T08:38:44.236 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:38:44.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:44.335 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.342+0000 7f67126c4640 1 Processor -- start 2024-04-25T08:38:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.342+0000 7f67126c4640 1 -- start start 2024-04-25T08:38:44.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f67126c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c15dd30 0x7f670c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f67126c4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f670c0d0ad0 0x7f670c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f67126c4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670c0d1df0 con 0x7f670c0b9770 2024-04-25T08:38:44.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f67126c4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c0d20d0 con 0x7f670c0d0ad0 2024-04-25T08:38:44.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f67126c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d23b0 con 0x7f670c15dd30 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f670b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f670c0b9770 0x7f670c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60938/0 (socket says 172.21.15.60:60938) 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f670b7fe640 1 -- 172.21.15.60:0/1126913152 learned_addr learned my addr 172.21.15.60:0/1126913152 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f6710c61640 1 --2- 172.21.15.60:0/1126913152 >> v2:172.21.15.100:3300/0 conn(0x7f670c0d0ad0 0x7f670c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f670bfff640 1 --2- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c15dd30 0x7f670c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f670affd640 1 -- 172.21.15.60:0/1126913152 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619906518 0 0) 0x7f670c0d1df0 con 0x7f670c0b9770 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f670affd640 1 -- 172.21.15.60:0/1126913152 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f4003520 con 0x7f670c0b9770 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f6710c61640 1 -- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c15dd30 msgr2=0x7f670c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f6710c61640 1 --2- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c15dd30 0x7f670c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f6710c61640 1 -- 172.21.15.60:0/1126913152 >> v1:172.21.15.60:6789/0 conn(0x7f670c0b9770 legacy=0x7f670c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.343+0000 7f6710c61640 1 -- 172.21.15.60:0/1126913152 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0d2690 con 0x7f670c0d0ad0 2024-04-25T08:38:44.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.344+0000 7f6710c61640 1 --2- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0d0ad0 0x7f670c0b6a40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6704009d90 tx=0x7f670402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f28ca878193412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:44.339 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.344+0000 7f670affd640 1 -- 172.21.15.60:0/1126913152 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031600 con 0x7f670c0d0ad0 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.344+0000 7f670affd640 1 -- 172.21.15.60:0/1126913152 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040317a0 con 0x7f670c0d0ad0 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.344+0000 7f670affd640 1 -- 172.21.15.60:0/1126913152 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031b70 con 0x7f670c0d0ad0 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 -- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0d0ad0 msgr2=0x7f670c0b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 --2- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0d0ad0 0x7f670c0b6a40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6704009d90 tx=0x7f670402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 -- 172.21.15.60:0/1126913152 shutdown_connections 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 --2- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0d0ad0 0x7f670c0b6a40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 --2- 172.21.15.60:0/1126913152 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c15dd30 0x7f670c0b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 -- 172.21.15.60:0/1126913152 >> 172.21.15.60:0/1126913152 conn(0x7f670c061c30 msgr2=0x7f670c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 -- 172.21.15.60:0/1126913152 shutdown_connections 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.345+0000 7f67126c4640 1 -- 172.21.15.60:0/1126913152 wait complete. 2024-04-25T08:38:44.340 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.346+0000 7f67126c4640 1 Processor -- start 2024-04-25T08:38:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.346+0000 7f67126c4640 1 -- start start 2024-04-25T08:38:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f67126c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0b9770 0x7f670c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f67126c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c0d0ad0 0x7f670c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f67126c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 0x7f670c1f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f67126c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d1b90 con 0x7f670c15dd30 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f67126c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d1a10 con 0x7f670c0b9770 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f67126c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0d2e90 con 0x7f670c0d0ad0 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f6710c61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 0x7f670c1f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f6710c61640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 0x7f670c1f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39086/0 (socket says 172.21.15.60:39086) 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f6710c61640 1 -- 172.21.15.60:0/3799217526 learned_addr learned my addr 172.21.15.60:0/3799217526 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f670bfff640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0b9770 0x7f670c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.347+0000 7f670b7fe640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c0d0ad0 0x7f670c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6710c61640 1 -- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c0d0ad0 msgr2=0x7f670c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6710c61640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c0d0ad0 0x7f670c0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6710c61640 1 -- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0b9770 msgr2=0x7f670c0cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6710c61640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0b9770 0x7f670c0cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6710c61640 1 -- 172.21.15.60:0/3799217526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0cca50 con 0x7f670c15dd30 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f670bfff640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0b9770 0x7f670c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f670b7fe640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c0d0ad0 0x7f670c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6710c61640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 0x7f670c1f6200 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f670402abc0 tx=0x7f6704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704040070 con 0x7f670c15dd30 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670403c850 con 0x7f670c15dd30 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c0ccd30 con 0x7f670c15dd30 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670403cc40 con 0x7f670c15dd30 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.348+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c0cd2f0 con 0x7f670c15dd30 2024-04-25T08:38:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.349+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c1f47b0 con 0x7f670c15dd30 2024-04-25T08:38:44.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.350+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6704039040 con 0x7f670c15dd30 2024-04-25T08:38:44.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.350+0000 7f6708ff9640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f806b4d0 0x7f66f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:44.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.350+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67040ab4c0 con 0x7f670c15dd30 2024-04-25T08:38:44.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.350+0000 7f670bfff640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f806b4d0 0x7f66f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:44.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.351+0000 7f670bfff640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f806b4d0 0x7f66f806d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009a20 tx=0x7f66fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:44.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.355+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6704031770 con 0x7f670c15dd30 2024-04-25T08:38:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.603+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f670c0b7c40 con 0x7f670c15dd30 2024-04-25T08:38:44.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.604+0000 7f6708ff9640 1 -- 172.21.15.60:0/3799217526 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f670407a240 con 0x7f670c15dd30 2024-04-25T08:38:44.597 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:44.598 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":585,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076492608,"bytes_used":2595225600,"bytes_avail":770498887680,"bytes_total":773094113280,"write_bytes_sec":70602224,"read_op_per_sec":0,"write_op_per_sec":69},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T08:37:57.516705+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:38:44.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.605+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f806b4d0 msgr2=0x7f66f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.605+0000 7f67126c4640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f806b4d0 0x7f66f806d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009a20 tx=0x7f66fc008040 comp rx=0 tx=0).stop 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 msgr2=0x7f670c1f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 0x7f670c1f6200 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f670402abc0 tx=0x7f6704004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 shutdown_connections 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f670c15dd30 0x7f670c1f6200 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f670c0d0ad0 0x7f670c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f66f806b4d0 0x7f66f806d990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 --2- 172.21.15.60:0/3799217526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f670c0b9770 0x7f670c0cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 >> 172.21.15.60:0/3799217526 conn(0x7f670c061c30 msgr2=0x7f670c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 shutdown_connections 2024-04-25T08:38:44.600 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:44.606+0000 7f67126c4640 1 -- 172.21.15.60:0/3799217526 wait complete. 2024-04-25T08:38:44.610 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:38:45.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:45.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.731+0000 7f42462c4640 1 Processor -- start 2024-04-25T08:38:45.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.731+0000 7f42462c4640 1 -- start start 2024-04-25T08:38:45.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f42462c4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f424015db70 0x7f4240162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f42462c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b5c30 0x7f42400d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f42462c4640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4240162690 con 0x7f42400b6cb0 2024-04-25T08:38:45.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f42462c4640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42400ceee0 con 0x7f424015db70 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f42462c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400d1d10 con 0x7f42400b5c30 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f423ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f424015db70 0x7f4240162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f423f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f42400b6cb0 0x7f42400b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60944/0 (socket says 172.21.15.60:60944) 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.732+0000 7f423f7fe640 1 -- 172.21.15.60:0/724096086 learned_addr learned my addr 172.21.15.60:0/724096086 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f4244861640 1 --2- 172.21.15.60:0/724096086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b5c30 0x7f42400d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f423ffff640 1 -- 172.21.15.60:0/724096086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b5c30 msgr2=0x7f42400d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f423ffff640 1 --2- 172.21.15.60:0/724096086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b5c30 0x7f42400d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f423ffff640 1 -- 172.21.15.60:0/724096086 >> v1:172.21.15.60:6789/0 conn(0x7f42400b6cb0 legacy=0x7f42400b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f423ffff640 1 -- 172.21.15.60:0/724096086 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400d1ff0 con 0x7f424015db70 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f423effd640 1 -- 172.21.15.60:0/724096086 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630230857 0 0) 0x7f4240162690 con 0x7f42400b6cb0 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.733+0000 7f423ffff640 1 --2- 172.21.15.60:0/724096086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424015db70 0x7f4240162150 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f42340097a0 tx=0x7f423402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451703383f4de3a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f423effd640 1 -- 172.21.15.60:0/724096086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234037070 con 0x7f424015db70 2024-04-25T08:38:45.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f423effd640 1 -- 172.21.15.60:0/724096086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234032440 con 0x7f424015db70 2024-04-25T08:38:45.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f423effd640 1 -- 172.21.15.60:0/724096086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234032810 con 0x7f424015db70 2024-04-25T08:38:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 -- 172.21.15.60:0/724096086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424015db70 msgr2=0x7f4240162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 --2- 172.21.15.60:0/724096086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424015db70 0x7f4240162150 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f42340097a0 tx=0x7f423402a180 comp rx=0 tx=0).stop 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 -- 172.21.15.60:0/724096086 shutdown_connections 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 --2- 172.21.15.60:0/724096086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b5c30 0x7f42400d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 --2- 172.21.15.60:0/724096086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f424015db70 0x7f4240162150 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 -- 172.21.15.60:0/724096086 >> 172.21.15.60:0/724096086 conn(0x7f42400d0520 msgr2=0x7f42400d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 -- 172.21.15.60:0/724096086 shutdown_connections 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.734+0000 7f42462c4640 1 -- 172.21.15.60:0/724096086 wait complete. 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.735+0000 7f42462c4640 1 Processor -- start 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 -- start start 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 0x7f424014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b6cb0 0x7f42400cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424015db70 0x7f42400cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.732 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400ce530 con 0x7f42400b5c30 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424015df50 con 0x7f424015db70 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.736+0000 7f42462c4640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400cf1c0 con 0x7f42400b6cb0 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 0x7f424014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 0x7f424014bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39110/0 (socket says 172.21.15.60:39110) 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 -- 172.21.15.60:0/156670183 learned_addr learned my addr 172.21.15.60:0/156670183 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423f7fe640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b6cb0 0x7f42400cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f4244861640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424015db70 0x7f42400cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 -- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b6cb0 msgr2=0x7f42400cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b6cb0 0x7f42400cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 -- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424015db70 msgr2=0x7f42400cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.733 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424015db70 0x7f42400cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 -- 172.21.15.60:0/156670183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424015fce0 con 0x7f42400b5c30 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423ffff640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 0x7f424014bfc0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4234000c00 tx=0x7f42340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234037040 con 0x7f42400b5c30 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234042070 con 0x7f42400b5c30 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424015ffc0 con 0x7f42400b5c30 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234030bb0 con 0x7f42400b5c30 2024-04-25T08:38:45.734 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.737+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42400cae50 con 0x7f42400b5c30 2024-04-25T08:38:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.739+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4240073990 con 0x7f42400b5c30 2024-04-25T08:38:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.739+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4234047460 con 0x7f42400b5c30 2024-04-25T08:38:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.739+0000 7f423cff9640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.740+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42340aaf40 con 0x7f42400b5c30 2024-04-25T08:38:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.740+0000 7f423f7fe640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.735 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.740+0000 7f423f7fe640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f422c06b4d0 0x7f422c06d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4240160320 tx=0x7f4230007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.737 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.744+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4234032e20 con 0x7f42400b5c30 2024-04-25T08:38:45.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.961+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42400d1820 con 0x7f42400b5c30 2024-04-25T08:38:45.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.961+0000 7f423cff9640 1 -- 172.21.15.60:0/156670183 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4234079c40 con 0x7f42400b5c30 2024-04-25T08:38:45.955 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:45.956 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:45.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f422c06b4d0 msgr2=0x7f422c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f422c06b4d0 0x7f422c06d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4240160320 tx=0x7f4230007b50 comp rx=0 tx=0).stop 2024-04-25T08:38:45.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 msgr2=0x7f424014bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 0x7f424014bfc0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f4234000c00 tx=0x7f42340046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 shutdown_connections 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f424015db70 0x7f42400cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f42400b6cb0 0x7f42400cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 --2- 172.21.15.60:0/156670183 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f42400b5c30 0x7f424014bfc0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 >> 172.21.15.60:0/156670183 conn(0x7f42400d0520 msgr2=0x7f4240162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 shutdown_connections 2024-04-25T08:38:45.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:45.963+0000 7f42462c4640 1 -- 172.21.15.60:0/156670183 wait complete. 2024-04-25T08:38:45.968 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:38:45.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:46.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.074+0000 7f550310a640 1 Processor -- start 2024-04-25T08:38:46.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.074+0000 7f550310a640 1 -- start start 2024-04-25T08:38:46.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f550310a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54fc0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f550310a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f54fc06d290 0x7f54fc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f550310a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc144b30 con 0x7f54fc144750 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f550310a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0712d0 con 0x7f54fc06d290 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f550310a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0715b0 con 0x7f54fc0ce920 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f5500ea6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f54fc144750 0x7f54fc06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:60954/0 (socket says 172.21.15.60:60954) 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f5500ea6640 1 -- 172.21.15.60:0/423680665 learned_addr learned my addr 172.21.15.60:0/423680665 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f55016a7640 1 --2- 172.21.15.60:0/423680665 >> v2:172.21.15.100:3300/0 conn(0x7f54fc06d290 0x7f54fc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fbfff640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54fc0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350015439 0 0) 0x7f54fc144b30 con 0x7f54fc144750 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e8003520 con 0x7f54fc144750 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61462049 0 0) 0x7f54e8003520 con 0x7f54fc144750 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 msgr2=0x7f54fc0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54fc0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 >> v2:172.21.15.100:3300/0 conn(0x7f54fc06d290 msgr2=0x7f54fc06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 --2- 172.21.15.60:0/423680665 >> v2:172.21.15.100:3300/0 conn(0x7f54fc06d290 0x7f54fc06d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc071890 con 0x7f54fc144750 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f54f40019f0 con 0x7f54fc144750 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 >> v1:172.21.15.60:6789/0 conn(0x7f54fc144750 legacy=0x7f54fc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e8004800 0x7f54e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.075+0000 7f54fb7fe640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0712d0 con 0x7f54e80039b0 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0715b0 con 0x7f54e8004800 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e800b9c0 con 0x7f54fc0ce920 2024-04-25T08:38:46.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f5500ea6640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f54fbfff640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e8004800 0x7f54e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f55016a7640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f5500ea6640 1 -- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 msgr2=0x7f54e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f5500ea6640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f5500ea6640 1 -- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e8004800 msgr2=0x7f54e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f5500ea6640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54e8004800 0x7f54e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f5500ea6640 1 -- 172.21.15.60:0/423680665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8003520 con 0x7f54e80039b0 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.076+0000 7f55016a7640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc0ce920 0x7f54e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f5500ea6640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54e8003dd0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f54f400a870 tx=0x7f54e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c36f102a53e753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400a4c0 con 0x7f54e80039b0 2024-04-25T08:38:46.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f402f070 con 0x7f54e80039b0 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f54fb7fe640 1 -- 172.21.15.60:0/423680665 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f402bcc0 con 0x7f54e80039b0 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f550310a640 1 -- 172.21.15.60:0/423680665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 msgr2=0x7f54e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f550310a640 1 --2- 172.21.15.60:0/423680665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54e8003dd0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f54f400a870 tx=0x7f54e8005830 comp rx=0 tx=0).stop 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f55016a7640 1 -- 172.21.15.60:0/423680665 reap_dead start 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f550310a640 1 -- 172.21.15.60:0/423680665 shutdown_connections 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f550310a640 1 -- 172.21.15.60:0/423680665 >> 172.21.15.60:0/423680665 conn(0x7f54fc06abe0 msgr2=0x7f54fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f550310a640 1 -- 172.21.15.60:0/423680665 shutdown_connections 2024-04-25T08:38:46.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.077+0000 7f550310a640 1 -- 172.21.15.60:0/423680665 wait complete. 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.078+0000 7f550310a640 1 Processor -- start 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.078+0000 7f550310a640 1 -- start start 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f550310a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54fc1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f550310a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc143c80 0x7f54fc15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f550310a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc15a940 0x7f54fc0a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f550310a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc145450 con 0x7f54e80039b0 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f550310a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc06f020 con 0x7f54fc15a940 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f550310a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1455d0 con 0x7f54fc143c80 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f5500ea6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54fc1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f5500ea6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54fc1419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39118/0 (socket says 172.21.15.60:39118) 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.079+0000 7f5500ea6640 1 -- 172.21.15.60:0/3403080917 learned_addr learned my addr 172.21.15.60:0/3403080917 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f54fbfff640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc143c80 0x7f54fc15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f55016a7640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc15a940 0x7f54fc0a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f5500ea6640 1 -- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc143c80 msgr2=0x7f54fc15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f5500ea6640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc143c80 0x7f54fc15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f5500ea6640 1 -- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc15a940 msgr2=0x7f54fc0a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f5500ea6640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc15a940 0x7f54fc0a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f5500ea6640 1 -- 172.21.15.60:0/3403080917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc1422b0 con 0x7f54e80039b0 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f54fbfff640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc143c80 0x7f54fc15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f5500ea6640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54fc1419d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f54f402b530 tx=0x7f54f4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f402ba80 con 0x7f54e80039b0 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f402bc20 con 0x7f54e80039b0 2024-04-25T08:38:46.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc142590 con 0x7f54e80039b0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4004e30 con 0x7f54e80039b0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.080+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc0a86f0 con 0x7f54e80039b0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.081+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc073990 con 0x7f54e80039b0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.082+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54f4034470 con 0x7f54e80039b0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.082+0000 7f54f97fa640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f54dc06b4f0 0x7f54dc06d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.082+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54f40aa6f0 con 0x7f54e80039b0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.082+0000 7f54fbfff640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f54dc06b4f0 0x7f54dc06d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.083+0000 7f54fbfff640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f54dc06b4f0 0x7f54dc06d9b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f54fc15b910 tx=0x7f54e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.086+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54f4079470 con 0x7f54e80039b0 2024-04-25T08:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.337+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f54fc0aa0f0 con 0x7f54e80039b0 2024-04-25T08:38:46.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.338+0000 7f54f97fa640 1 -- 172.21.15.60:0/3403080917 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f54f4078d70 con 0x7f54e80039b0 2024-04-25T08:38:46.331 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:46.331 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":587,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":134,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076492608,"bytes_used":2595291136,"bytes_avail":770498822144,"bytes_total":773094113280,"write_bytes_sec":70604536,"read_op_per_sec":0,"write_op_per_sec":69},"fsmap":{"epoch":78,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":9381}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":28,"modified":"2024-04-25T08:38:45.528175+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.339+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f54dc06b4f0 msgr2=0x7f54dc06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.333 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.339+0000 7f550310a640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f54dc06b4f0 0x7f54dc06d9b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f54fc15b910 tx=0x7f54e4008040 comp rx=0 tx=0).stop 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.339+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 msgr2=0x7f54fc1419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.339+0000 7f550310a640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54fc1419d0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f54f402b530 tx=0x7f54f4001360 comp rx=0 tx=0).stop 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 shutdown_connections 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f54dc06b4f0 0x7f54dc06d9b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f54fc15a940 0x7f54fc0a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f54fc143c80 0x7f54fc15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 --2- 172.21.15.60:0/3403080917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f54e80039b0 0x7f54fc1419d0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 >> 172.21.15.60:0/3403080917 conn(0x7f54fc06abe0 msgr2=0x7f54fc0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 shutdown_connections 2024-04-25T08:38:46.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.340+0000 7f550310a640 1 -- 172.21.15.60:0/3403080917 wait complete. 2024-04-25T08:38:46.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:38:46.345 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:38:46.345 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:46.357 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:38:46.357 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'c', 'a', 'b'} 2024-04-25T08:38:46.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T08:38:46.465 INFO:teuthology.orchestra.run.smithi060.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T08:38:46.465 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T08:38:46.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.574+0000 7f3898989640 1 Processor -- start 2024-04-25T08:38:46.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.574+0000 7f3898989640 1 -- start start 2024-04-25T08:38:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f3898989640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f389415dd30 0x7f38940b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f3898989640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38940d0ad0 0x7f38940b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f3898989640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940d1df0 con 0x7f38940b9770 2024-04-25T08:38:46.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f3898989640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940d20d0 con 0x7f389415dd30 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f3898989640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940d23b0 con 0x7f38940d0ad0 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f389259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f389415dd30 0x7f38940b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f389259c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f389415dd30 0x7f38940b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39120/0 (socket says 172.21.15.60:39120) 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f389259c640 1 -- 172.21.15.60:0/520545330 learned_addr learned my addr 172.21.15.60:0/520545330 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f3892d9d640 1 --2- 172.21.15.60:0/520545330 >> v2:172.21.15.100:3300/0 conn(0x7f38940d0ad0 0x7f38940b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f389259c640 1 -- 172.21.15.60:0/520545330 >> v2:172.21.15.100:3300/0 conn(0x7f38940d0ad0 msgr2=0x7f38940b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.575+0000 7f389259c640 1 --2- 172.21.15.60:0/520545330 >> v2:172.21.15.100:3300/0 conn(0x7f38940d0ad0 0x7f38940b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.576+0000 7f389259c640 1 -- 172.21.15.60:0/520545330 >> v1:172.21.15.100:6789/0 conn(0x7f38940b9770 legacy=0x7f38940d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.576+0000 7f389259c640 1 -- 172.21.15.60:0/520545330 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940d2690 con 0x7f389415dd30 2024-04-25T08:38:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.576+0000 7f389259c640 1 --2- 172.21.15.60:0/520545330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f38940b9230 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f38940ce200 tx=0x7f388402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5ee2dd1587bcfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.576+0000 7f389159a640 1 -- 172.21.15.60:0/520545330 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884037070 con 0x7f389415dd30 2024-04-25T08:38:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.576+0000 7f389159a640 1 -- 172.21.15.60:0/520545330 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3884032440 con 0x7f389415dd30 2024-04-25T08:38:46.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.576+0000 7f389159a640 1 -- 172.21.15.60:0/520545330 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884032810 con 0x7f389415dd30 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 -- 172.21.15.60:0/520545330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 msgr2=0x7f38940b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 --2- 172.21.15.60:0/520545330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f38940b9230 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f38940ce200 tx=0x7f388402a150 comp rx=0 tx=0).stop 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 -- 172.21.15.60:0/520545330 shutdown_connections 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 --2- 172.21.15.60:0/520545330 >> v2:172.21.15.100:3300/0 conn(0x7f38940d0ad0 0x7f38940b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 --2- 172.21.15.60:0/520545330 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f38940b9230 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 -- 172.21.15.60:0/520545330 >> 172.21.15.60:0/520545330 conn(0x7f3894061c30 msgr2=0x7f389406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 -- 172.21.15.60:0/520545330 shutdown_connections 2024-04-25T08:38:46.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.577+0000 7f3898989640 1 -- 172.21.15.60:0/520545330 wait complete. 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.578+0000 7f3898989640 1 Processor -- start 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.578+0000 7f3898989640 1 -- start start 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3898989640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38940b9770 0x7f389414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3898989640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38940d0ad0 0x7f3894159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3898989640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f3894153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3898989640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d1b90 con 0x7f389415dd30 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3898989640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d1a10 con 0x7f38940b9770 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3898989640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38940d2e90 con 0x7f38940d0ad0 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3892d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f3894153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3892d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f3894153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:39126/0 (socket says 172.21.15.60:39126) 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3892d9d640 1 -- 172.21.15.60:0/2701830711 learned_addr learned my addr 172.21.15.60:0/2701830711 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f3891d9b640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38940d0ad0 0x7f3894159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.579+0000 7f389259c640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38940b9770 0x7f389414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3892d9d640 1 -- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38940d0ad0 msgr2=0x7f3894159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3892d9d640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38940d0ad0 0x7f3894159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3892d9d640 1 -- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38940b9770 msgr2=0x7f389414e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3892d9d640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38940b9770 0x7f389414e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3892d9d640 1 -- 172.21.15.60:0/2701830711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38941608a0 con 0x7f389415dd30 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f389259c640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38940b9770 0x7f389414e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3891d9b640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38940d0ad0 0x7f3894159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3892d9d640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f3894153a20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f388c007e70 tx=0x7f388c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c014620 con 0x7f389415dd30 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c019070 con 0x7f389415dd30 2024-04-25T08:38:46.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38941549c0 con 0x7f389415dd30 2024-04-25T08:38:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c014a20 con 0x7f389415dd30 2024-04-25T08:38:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.580+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941600a0 con 0x7f389415dd30 2024-04-25T08:38:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.581+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3894073990 con 0x7f389415dd30 2024-04-25T08:38:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.581+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f388c014bc0 con 0x7f389415dd30 2024-04-25T08:38:46.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.582+0000 7f3882ffd640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.582+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f388c08ab90 con 0x7f389415dd30 2024-04-25T08:38:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.582+0000 7f389259c640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:46.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.583+0000 7f389259c640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3884034000 tx=0x7f388402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:46.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.586+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f388c059910 con 0x7f389415dd30 2024-04-25T08:38:46.800 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.807+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38940b7540 con 0x7f389415dd30 2024-04-25T08:38:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.807+0000 7f3882ffd640 1 -- 172.21.15.60:0/2701830711 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f388c059210 con 0x7f389415dd30 2024-04-25T08:38:46.801 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:46.801 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:46.801 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f387406b4d0 msgr2=0x7f387406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3884034000 tx=0x7f388402e000 comp rx=0 tx=0).stop 2024-04-25T08:38:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 msgr2=0x7f3894153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f3894153a20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f388c007e70 tx=0x7f388c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 shutdown_connections 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f389415dd30 0x7f3894153a20 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f38940d0ad0 0x7f3894159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 --2- 172.21.15.60:0/2701830711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38940b9770 0x7f389414e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.809+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 >> 172.21.15.60:0/2701830711 conn(0x7f3894061c30 msgr2=0x7f38940ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.810+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 shutdown_connections 2024-04-25T08:38:46.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:46.810+0000 7f3898989640 1 -- 172.21.15.60:0/2701830711 wait complete. 2024-04-25T08:38:46.814 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:38:51.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:51.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.935+0000 7f67f1407640 1 Processor -- start 2024-04-25T08:38:51.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.935+0000 7f67f1407640 1 -- start start 2024-04-25T08:38:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67f1407640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec15db60 0x7f67ec0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67f1407640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0cec00 con 0x7f67ec0d0b10 2024-04-25T08:38:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67f1407640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec0ceee0 con 0x7f67ec0b5f00 2024-04-25T08:38:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67f1407640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec0ce930 con 0x7f67ec15db60 2024-04-25T08:38:51.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67eb7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f67ec0d0b10 0x7f67ec0d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44924/0 (socket says 172.21.15.60:44924) 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67eb7fe640 1 -- 172.21.15.60:0/1804523580 learned_addr learned my addr 172.21.15.60:0/1804523580 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67eaffd640 1 --2- 172.21.15.60:0/1804523580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec15db60 0x7f67ec0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363017882 0 0) 0x7f67ec0cec00 con 0x7f67ec0d0b10 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d8003520 con 0x7f67ec0d0b10 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534362850 0 0) 0x7f67ec0ceee0 con 0x7f67ec0b5f00 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec0cec00 con 0x7f67ec0b5f00 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67eaffd640 1 -- 172.21.15.60:0/1804523580 >> v1:172.21.15.100:6789/0 conn(0x7f67ec0b5f00 legacy=0x7f67ec0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67eaffd640 1 -- 172.21.15.60:0/1804523580 >> v1:172.21.15.60:6789/0 conn(0x7f67ec0d0b10 legacy=0x7f67ec0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1753579186 0 0) 0x7f67d8003520 con 0x7f67ec0d0b10 2024-04-25T08:38:51.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.936+0000 7f67eaffd640 1 -- 172.21.15.60:0/1804523580 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0cf1c0 con 0x7f67ec15db60 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.937+0000 7f67eaffd640 1 --2- 172.21.15.60:0/1804523580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec15db60 0x7f67ec0ce3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0097d0 tx=0x7f67dc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c5b33c77dddc58f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc037070 con 0x7f67ec15db60 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc02ab90 con 0x7f67ec15db60 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67ea7fc640 1 -- 172.21.15.60:0/1804523580 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e40019f0 con 0x7f67ec15db60 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 -- 172.21.15.60:0/1804523580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec15db60 msgr2=0x7f67ec0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 --2- 172.21.15.60:0/1804523580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec15db60 0x7f67ec0ce3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0097d0 tx=0x7f67dc02a100 comp rx=0 tx=0).stop 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 -- 172.21.15.60:0/1804523580 shutdown_connections 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 --2- 172.21.15.60:0/1804523580 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec15db60 0x7f67ec0ce3f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 -- 172.21.15.60:0/1804523580 >> 172.21.15.60:0/1804523580 conn(0x7f67ec061600 msgr2=0x7f67ec061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 -- 172.21.15.60:0/1804523580 shutdown_connections 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.938+0000 7f67f1407640 1 -- 172.21.15.60:0/1804523580 wait complete. 2024-04-25T08:38:51.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.939+0000 7f67f1407640 1 Processor -- start 2024-04-25T08:38:51.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.939+0000 7f67f1407640 1 -- start start 2024-04-25T08:38:51.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67f1407640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec0b5f00 0x7f67ec0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67f1407640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec0d0b10 0x7f67ec160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67f1407640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 0x7f67ec0cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67f1407640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec15df40 con 0x7f67ec15db60 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67f1407640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec0b6450 con 0x7f67ec0b5f00 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67f1407640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec15e0c0 con 0x7f67ec0d0b10 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 0x7f67ec0cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67eb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 0x7f67ec0cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42888/0 (socket says 172.21.15.60:42888) 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67eb7fe640 1 -- 172.21.15.60:0/1734348282 learned_addr learned my addr 172.21.15.60:0/1734348282 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67eaffd640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec0b5f00 0x7f67ec0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.940+0000 7f67e3fff640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec0d0b10 0x7f67ec160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67eb7fe640 1 -- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec0d0b10 msgr2=0x7f67ec160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67eb7fe640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec0d0b10 0x7f67ec160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67eb7fe640 1 -- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec0b5f00 msgr2=0x7f67ec0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67eb7fe640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec0b5f00 0x7f67ec0cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67eb7fe640 1 -- 172.21.15.60:0/1734348282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0cd000 con 0x7f67ec15db60 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67e3fff640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec0d0b10 0x7f67ec160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67eb7fe640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 0x7f67ec0cab40 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005c00 tx=0x7f67e400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4017070 con 0x7f67ec15db60 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e4012400 con 0x7f67ec15db60 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec0cd2e0 con 0x7f67ec15db60 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4012810 con 0x7f67ec15db60 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.941+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec15e810 con 0x7f67ec15db60 2024-04-25T08:38:51.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.942+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec073270 con 0x7f67ec15db60 2024-04-25T08:38:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.943+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67e401a070 con 0x7f67ec15db60 2024-04-25T08:38:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.943+0000 7f67e37fe640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67cc06b4d0 0x7f67cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.943+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67e408a360 con 0x7f67ec15db60 2024-04-25T08:38:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.943+0000 7f67eaffd640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67cc06b4d0 0x7f67cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.944+0000 7f67eaffd640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67cc06b4d0 0x7f67cc06d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f67dc034000 tx=0x7f67dc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:51.949+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67e4059060 con 0x7f67ec15db60 2024-04-25T08:38:52.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.168+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67ec15ea10 con 0x7f67ec15db60 2024-04-25T08:38:52.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.168+0000 7f67e37fe640 1 -- 172.21.15.60:0/1734348282 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f67ec15ea10 con 0x7f67ec15db60 2024-04-25T08:38:52.162 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:52.162 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:52.163 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:52.164 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67cc06b4d0 msgr2=0x7f67cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67cc06b4d0 0x7f67cc06d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f67dc034000 tx=0x7f67dc02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 msgr2=0x7f67ec0cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 0x7f67ec0cab40 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005c00 tx=0x7f67e400a430 comp rx=0 tx=0).stop 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 shutdown_connections 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f67ec15db60 0x7f67ec0cab40 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f67ec0d0b10 0x7f67ec160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f67cc06b4d0 0x7f67cc06d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 --2- 172.21.15.60:0/1734348282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67ec0b5f00 0x7f67ec0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 >> 172.21.15.60:0/1734348282 conn(0x7f67ec061600 msgr2=0x7f67ec0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 shutdown_connections 2024-04-25T08:38:52.165 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:52.171+0000 7f67f1407640 1 -- 172.21.15.60:0/1734348282 wait complete. 2024-04-25T08:38:52.175 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:38:57.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 Processor -- start 2024-04-25T08:38:57.290 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 -- start start 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdf280cfec0 0x7fdf280cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdf280ce3c0 0x7fdf2806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf28071480 con 0x7fdf28154ee0 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf280ce830 con 0x7fdf280ce3c0 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.297+0000 7fdf2d596640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf28072090 con 0x7fdf280cfec0 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf277fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdf280ce3c0 0x7fdf2806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf26ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fdf28154ee0 0x7fdf280cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:44930/0 (socket says 172.21.15.60:44930) 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf26ffd640 1 -- 172.21.15.60:0/3763234753 learned_addr learned my addr 172.21.15.60:0/3763234753 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf267fc640 1 --2- 172.21.15.60:0/3763234753 >> v2:172.21.15.100:3300/0 conn(0x7fdf280cfec0 0x7fdf280cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf25ffb640 1 -- 172.21.15.60:0/3763234753 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118510470 0 0) 0x7fdf28071480 con 0x7fdf28154ee0 2024-04-25T08:38:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf25ffb640 1 -- 172.21.15.60:0/3763234753 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf14003520 con 0x7fdf28154ee0 2024-04-25T08:38:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf277fe640 1 -- 172.21.15.60:0/3763234753 >> v2:172.21.15.100:3300/0 conn(0x7fdf280cfec0 msgr2=0x7fdf280cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf277fe640 1 --2- 172.21.15.60:0/3763234753 >> v2:172.21.15.100:3300/0 conn(0x7fdf280cfec0 0x7fdf280cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf277fe640 1 -- 172.21.15.60:0/3763234753 >> v1:172.21.15.60:6789/0 conn(0x7fdf28154ee0 legacy=0x7fdf280cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.298+0000 7fdf277fe640 1 -- 172.21.15.60:0/3763234753 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28071760 con 0x7fdf280ce3c0 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf277fe640 1 --2- 172.21.15.60:0/3763234753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280ce3c0 0x7fdf2806d6f0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fdf280cf560 tx=0x7fdf1802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b6d797552f50e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf25ffb640 1 -- 172.21.15.60:0/3763234753 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007740 con 0x7fdf280ce3c0 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf25ffb640 1 -- 172.21.15.60:0/3763234753 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf180078e0 con 0x7fdf280ce3c0 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf25ffb640 1 -- 172.21.15.60:0/3763234753 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007cb0 con 0x7fdf280ce3c0 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf2d596640 1 -- 172.21.15.60:0/3763234753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280ce3c0 msgr2=0x7fdf2806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf2d596640 1 --2- 172.21.15.60:0/3763234753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280ce3c0 0x7fdf2806d6f0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fdf280cf560 tx=0x7fdf1802a680 comp rx=0 tx=0).stop 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf2d596640 1 -- 172.21.15.60:0/3763234753 shutdown_connections 2024-04-25T08:38:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf2d596640 1 --2- 172.21.15.60:0/3763234753 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280ce3c0 0x7fdf2806d6f0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.299+0000 7fdf2d596640 1 --2- 172.21.15.60:0/3763234753 >> v2:172.21.15.100:3300/0 conn(0x7fdf280cfec0 0x7fdf280cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.300+0000 7fdf2d596640 1 -- 172.21.15.60:0/3763234753 >> 172.21.15.60:0/3763234753 conn(0x7fdf2806abe0 msgr2=0x7fdf2806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.300+0000 7fdf2d596640 1 -- 172.21.15.60:0/3763234753 shutdown_connections 2024-04-25T08:38:57.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.300+0000 7fdf2d596640 1 -- 172.21.15.60:0/3763234753 wait complete. 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.301+0000 7fdf2d596640 1 Processor -- start 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.301+0000 7fdf2d596640 1 -- start start 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf2d596640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf280ce3c0 0x7fdf28152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf2d596640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 0x7fdf280b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf2d596640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf28154ee0 0x7fdf28156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf2d596640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28071d20 con 0x7fdf280cfec0 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf2d596640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28071230 con 0x7fdf280ce3c0 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf2d596640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281552c0 con 0x7fdf28154ee0 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 0x7fdf280b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 0x7fdf280b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:42920/0 (socket says 172.21.15.60:42920) 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 -- 172.21.15.60:0/4228067637 learned_addr learned my addr 172.21.15.60:0/4228067637 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf26ffd640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf280ce3c0 0x7fdf28152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf277fe640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf28154ee0 0x7fdf28156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 -- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf28154ee0 msgr2=0x7fdf28156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf28154ee0 0x7fdf28156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 -- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf280ce3c0 msgr2=0x7fdf28152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf280ce3c0 0x7fdf28152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 -- 172.21.15.60:0/4228067637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf280b1410 con 0x7fdf280cfec0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf267fc640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 0x7fdf280b4200 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000d830 tx=0x7fdf1000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.302+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000b6c0 con 0x7fdf280cfec0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.303+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf10019070 con 0x7fdf280cfec0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.303+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf281513d0 con 0x7fdf280cfec0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.303+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000ba80 con 0x7fdf280cfec0 2024-04-25T08:38:57.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.303+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf28151c10 con 0x7fdf280cfec0 2024-04-25T08:38:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.304+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28073990 con 0x7fdf280cfec0 2024-04-25T08:38:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.304+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf1001e460 con 0x7fdf280cfec0 2024-04-25T08:38:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.305+0000 7fdf1f7fe640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdf0806b4d0 0x7fdf0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.305+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdf1008a970 con 0x7fdf280cfec0 2024-04-25T08:38:57.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.305+0000 7fdf26ffd640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdf0806b4d0 0x7fdf0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.305+0000 7fdf26ffd640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdf0806b4d0 0x7fdf0806d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005880 tx=0x7fdf20009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:57.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.309+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf281513d0 con 0x7fdf280cfec0 2024-04-25T08:38:57.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.525+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf2806d360 con 0x7fdf280cfec0 2024-04-25T08:38:57.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.526+0000 7fdf1f7fe640 1 -- 172.21.15.60:0/4228067637 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fdf100596f0 con 0x7fdf280cfec0 2024-04-25T08:38:57.520 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:38:57.520 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:38:57.520 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:38:57.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.527+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdf0806b4d0 msgr2=0x7fdf0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.527+0000 7fdf2d596640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdf0806b4d0 0x7fdf0806d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005880 tx=0x7fdf20009040 comp rx=0 tx=0).stop 2024-04-25T08:38:57.521 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.527+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 msgr2=0x7fdf280b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.527+0000 7fdf2d596640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 0x7fdf280b4200 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000d830 tx=0x7fdf1000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 shutdown_connections 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdf28154ee0 0x7fdf28156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdf280cfec0 0x7fdf280b4200 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdf0806b4d0 0x7fdf0806d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 --2- 172.21.15.60:0/4228067637 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf280ce3c0 0x7fdf28152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 >> 172.21.15.60:0/4228067637 conn(0x7fdf2806abe0 msgr2=0x7fdf280b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 shutdown_connections 2024-04-25T08:38:57.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:38:57.528+0000 7fdf2d596640 1 -- 172.21.15.60:0/4228067637 wait complete. 2024-04-25T08:38:57.532 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:38:57.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:57.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:02.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:02.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 Processor -- start 2024-04-25T08:39:02.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 -- start start 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e4415dd30 0x7f8e440b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e440b9770 0x7f8e440d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440d2e90 0x7f8e440d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440d1df0 con 0x7f8e440b9770 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440d20d0 con 0x7f8e4415dd30 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.654+0000 7f8e4a7f9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440d23b0 con 0x7f8e440d2e90 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e440b9770 0x7f8e440d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8e440b9770 0x7f8e440d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45930/0 (socket says 172.21.15.60:45930) 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 -- 172.21.15.60:0/3510348799 learned_addr learned my addr 172.21.15.60:0/3510348799 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e43fff640 1 --2- 172.21.15.60:0/3510348799 >> v2:172.21.15.100:3300/0 conn(0x7f8e4415dd30 0x7f8e440b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e48d96640 1 --2- 172.21.15.60:0/3510348799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440d2e90 0x7f8e440d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 -- 172.21.15.60:0/3510348799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440d2e90 msgr2=0x7f8e440d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:02.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 --2- 172.21.15.60:0/3510348799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440d2e90 0x7f8e440d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 -- 172.21.15.60:0/3510348799 >> v2:172.21.15.100:3300/0 conn(0x7f8e4415dd30 msgr2=0x7f8e440b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 --2- 172.21.15.60:0/3510348799 >> v2:172.21.15.100:3300/0 conn(0x7f8e4415dd30 0x7f8e440b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.655+0000 7f8e437fe640 1 -- 172.21.15.60:0/3510348799 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440d2690 con 0x7f8e440b9770 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e437fe640 1 --2- 172.21.15.60:0/3510348799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440b9770 0x7f8e440d03c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8e4406d000 tx=0x7f8e3802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ed08b921fd95ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e42ffd640 1 -- 172.21.15.60:0/3510348799 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004030 con 0x7f8e440b9770 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e42ffd640 1 -- 172.21.15.60:0/3510348799 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e380041d0 con 0x7f8e440b9770 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e42ffd640 1 -- 172.21.15.60:0/3510348799 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38003610 con 0x7f8e440b9770 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/3510348799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440b9770 msgr2=0x7f8e440d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/3510348799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440b9770 0x7f8e440d03c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8e4406d000 tx=0x7f8e3802a130 comp rx=0 tx=0).stop 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/3510348799 shutdown_connections 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/3510348799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440d2e90 0x7f8e440d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/3510348799 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440b9770 0x7f8e440d03c0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/3510348799 >> v2:172.21.15.100:3300/0 conn(0x7f8e4415dd30 0x7f8e440b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.656+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/3510348799 >> 172.21.15.60:0/3510348799 conn(0x7f8e44061c30 msgr2=0x7f8e4406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.657+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/3510348799 shutdown_connections 2024-04-25T08:39:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.657+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/3510348799 wait complete. 2024-04-25T08:39:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.658+0000 7f8e4a7f9640 1 Processor -- start 2024-04-25T08:39:02.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.658+0000 7f8e4a7f9640 1 -- start start 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e4a7f9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440b9770 0x7f8e441e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e4a7f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 0x7f8e4414a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e4a7f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4415dd30 0x7f8e44149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e4a7f9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440d1b90 con 0x7f8e440d2e90 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e4a7f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440d1a10 con 0x7f8e4415dd30 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e4a7f9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b5430 con 0x7f8e440b9770 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 0x7f8e4414a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 0x7f8e4414a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45944/0 (socket says 172.21.15.60:45944) 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 -- 172.21.15.60:0/468321242 learned_addr learned my addr 172.21.15.60:0/468321242 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e43fff640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440b9770 0x7f8e441e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e48d96640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4415dd30 0x7f8e44149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 -- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440b9770 msgr2=0x7f8e441e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440b9770 0x7f8e441e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 -- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4415dd30 msgr2=0x7f8e44149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4415dd30 0x7f8e44149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 -- 172.21.15.60:0/468321242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4415f670 con 0x7f8e440d2e90 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e437fe640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 0x7f8e4414a800 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38041020 con 0x7f8e440d2e90 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38003c50 con 0x7f8e440d2e90 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.659+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004030 con 0x7f8e440d2e90 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.660+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4414ad40 con 0x7f8e440d2e90 2024-04-25T08:39:02.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.660+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4415f8e0 con 0x7f8e440d2e90 2024-04-25T08:39:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.660+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e44073990 con 0x7f8e440d2e90 2024-04-25T08:39:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.661+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e38044020 con 0x7f8e440d2e90 2024-04-25T08:39:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.661+0000 7f8e40ff9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.662+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e380aae40 con 0x7f8e440d2e90 2024-04-25T08:39:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.662+0000 7f8e43fff640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.662+0000 7f8e43fff640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e3006b4d0 0x7f8e3006d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e440c97c0 tx=0x7f8e34008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.666+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e3804b030 con 0x7f8e440d2e90 2024-04-25T08:39:02.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.889+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e440cecb0 con 0x7f8e440d2e90 2024-04-25T08:39:02.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.890+0000 7f8e40ff9640 1 -- 172.21.15.60:0/468321242 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f8e380453b0 con 0x7f8e440d2e90 2024-04-25T08:39:02.883 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:02.883 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9375,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:38:19.224480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9381":{"gid":9381,"name":"d","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.100:6836/879630460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":879630460},{"type":"v1","addr":"172.21.15.100:6836","nonce":879630460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:02.884 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 78 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e3006b4d0 msgr2=0x7f8e3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e3006b4d0 0x7f8e3006d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8e440c97c0 tx=0x7f8e34008040 comp rx=0 tx=0).stop 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 msgr2=0x7f8e4414a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 0x7f8e4414a800 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e380046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 shutdown_connections 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e4415dd30 0x7f8e44149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e440d2e90 0x7f8e4414a800 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 --2- 172.21.15.60:0/468321242 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8e440b9770 0x7f8e441e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 >> 172.21.15.60:0/468321242 conn(0x7f8e44061c30 msgr2=0x7f8e440b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:02.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 shutdown_connections 2024-04-25T08:39:02.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:02.892+0000 7f8e4a7f9640 1 -- 172.21.15.60:0/468321242 wait complete. 2024-04-25T08:39:02.896 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:39:07.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:08.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.006+0000 7f9f005e0640 1 Processor -- start 2024-04-25T08:39:08.000 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.006+0000 7f9f005e0640 1 -- start start 2024-04-25T08:39:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9f005e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9f005e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ef80cfcf0 0x7f9ef80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9f005e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef80ce3c0 0x7f9ef815e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9f005e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef80ce830 con 0x7f9ef80ce3c0 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9f005e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef8072090 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9f005e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef80d0150 con 0x7f9ef8154ee0 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efeb7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef80ce3c0 0x7f9ef815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efeb7d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ef80ce3c0 0x7f9ef815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45948/0 (socket says 172.21.15.60:45948) 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efeb7d640 1 -- 172.21.15.60:0/1260418867 learned_addr learned my addr 172.21.15.60:0/1260418867 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efe37c640 1 --2- 172.21.15.60:0/1260418867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efdb7b640 1 --2- 172.21.15.60:0/1260418867 >> v2:172.21.15.100:3300/0 conn(0x7f9ef80cfcf0 0x7f9ef80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efeb7d640 1 -- 172.21.15.60:0/1260418867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 msgr2=0x7f9ef80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.007+0000 7f9efeb7d640 1 --2- 172.21.15.60:0/1260418867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efeb7d640 1 -- 172.21.15.60:0/1260418867 >> v2:172.21.15.100:3300/0 conn(0x7f9ef80cfcf0 msgr2=0x7f9ef80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efeb7d640 1 --2- 172.21.15.60:0/1260418867 >> v2:172.21.15.100:3300/0 conn(0x7f9ef80cfcf0 0x7f9ef80cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efeb7d640 1 -- 172.21.15.60:0/1260418867 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8071480 con 0x7f9ef80ce3c0 2024-04-25T08:39:08.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efeb7d640 1 --2- 172.21.15.60:0/1260418867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef815e570 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80cd350 tx=0x7f9ee802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28a35e8f58ce6be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efd37a640 1 -- 172.21.15.60:0/1260418867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8036070 con 0x7f9ef80ce3c0 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efd37a640 1 -- 172.21.15.60:0/1260418867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8031600 con 0x7f9ef80ce3c0 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.008+0000 7f9efd37a640 1 -- 172.21.15.60:0/1260418867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80319d0 con 0x7f9ef80ce3c0 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 -- 172.21.15.60:0/1260418867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80ce3c0 msgr2=0x7f9ef815e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1260418867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef815e570 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80cd350 tx=0x7f9ee802a6c0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 -- 172.21.15.60:0/1260418867 shutdown_connections 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1260418867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef815e570 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1260418867 >> v2:172.21.15.100:3300/0 conn(0x7f9ef80cfcf0 0x7f9ef80cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1260418867 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 -- 172.21.15.60:0/1260418867 >> 172.21.15.60:0/1260418867 conn(0x7f9ef806abe0 msgr2=0x7f9ef806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 -- 172.21.15.60:0/1260418867 shutdown_connections 2024-04-25T08:39:08.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.009+0000 7f9f005e0640 1 -- 172.21.15.60:0/1260418867 wait complete. 2024-04-25T08:39:08.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.010+0000 7f9f005e0640 1 Processor -- start 2024-04-25T08:39:08.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.010+0000 7f9f005e0640 1 -- start start 2024-04-25T08:39:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9f005e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9f005e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 0x7f9ef80b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9f005e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9f005e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8071a40 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9f005e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8071230 con 0x7f9ef80ce3c0 2024-04-25T08:39:08.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9f005e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8071d20 con 0x7f9ef8154ee0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 0x7f9ef80b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 0x7f9ef80b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45952/0 (socket says 172.21.15.60:45952) 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 -- 172.21.15.60:0/1216768109 learned_addr learned my addr 172.21.15.60:0/1216768109 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efeb7d640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efe37c640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 -- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 msgr2=0x7f9ef80b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 -- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef80ce3c0 msgr2=0x7f9ef813cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef813cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.011+0000 7f9efdb7b640 1 -- 172.21.15.60:0/1216768109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef8152190 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.012+0000 7f9efdb7b640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 0x7f9ef80b40b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00e890 tx=0x7f9eec00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.012+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec017070 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.012+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec00c940 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.012+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef8152470 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.012+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00cd10 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.012+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef8152750 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.013+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef8073990 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.013+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9eec01a070 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.014+0000 7f9ef6ffd640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9ee406b4d0 0x7f9ee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.014+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9eec08a680 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.014+0000 7f9efe37c640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9ee406b4d0 0x7f9ee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.015+0000 7f9efe37c640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9ee406b4d0 0x7f9ee406d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009aa0 tx=0x7f9ef0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.018+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9eec058cd0 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.237+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ef806d360 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.237+0000 7f9ef6ffd640 1 -- 172.21.15.60:0/1216768109 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f9eec05cd70 con 0x7f9ef80cfcf0 2024-04-25T08:39:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:08.231 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:39:06.172452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:08.231 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T08:39:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9ee406b4d0 msgr2=0x7f9ee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9ee406b4d0 0x7f9ee406d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0009aa0 tx=0x7f9ef0009190 comp rx=0 tx=0).stop 2024-04-25T08:39:08.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 msgr2=0x7f9ef80b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 0x7f9ef80b40b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00e890 tx=0x7f9eec00ed50 comp rx=0 tx=0).stop 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 shutdown_connections 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ef8154ee0 0x7f9ef80b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ef80cfcf0 0x7f9ef80b40b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9ee406b4d0 0x7f9ee406d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 --2- 172.21.15.60:0/1216768109 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ef80ce3c0 0x7f9ef813cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.239+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 >> 172.21.15.60:0/1216768109 conn(0x7f9ef806abe0 msgr2=0x7f9ef80cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.240+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 shutdown_connections 2024-04-25T08:39:08.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.240+0000 7f9f005e0640 1 -- 172.21.15.60:0/1216768109 wait complete. 2024-04-25T08:39:08.244 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T08:39:08.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:08.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.351+0000 7f4ef1f28640 1 Processor -- start 2024-04-25T08:39:08.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.351+0000 7f4ef1f28640 1 -- start start 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4ef1f28640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4eec0c6df0 0x7f4eec0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4ef1f28640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0d1c00 0x7f4eec0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4ef1f28640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec15dfe0 con 0x7f4eec15dc00 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4ef1f28640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eec0d2090 con 0x7f4eec0c6df0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4ef1f28640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0d00c0 con 0x7f4eec0d1c00 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4eebfff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0d1c00 0x7f4eec0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4eeaffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4eec0c6df0 0x7f4eec0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4eeb7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4eec15dc00 0x7f4eec0ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:52648/0 (socket says 172.21.15.60:52648) 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.352+0000 7f4eeb7fe640 1 -- 172.21.15.60:0/709835210 learned_addr learned my addr 172.21.15.60:0/709835210 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eebfff640 1 -- 172.21.15.60:0/709835210 >> v2:172.21.15.100:3300/0 conn(0x7f4eec0c6df0 msgr2=0x7f4eec0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eebfff640 1 --2- 172.21.15.60:0/709835210 >> v2:172.21.15.100:3300/0 conn(0x7f4eec0c6df0 0x7f4eec0d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eebfff640 1 -- 172.21.15.60:0/709835210 >> v1:172.21.15.100:6789/0 conn(0x7f4eec15dc00 legacy=0x7f4eec0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eebfff640 1 -- 172.21.15.60:0/709835210 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0d03a0 con 0x7f4eec0d1c00 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eea7fc640 1 -- 172.21.15.60:0/709835210 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908689589 0 0) 0x7f4edc001110 con 0x7f4eec15dc00 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eeaffd640 1 --2- 172.21.15.60:0/709835210 >> v2:172.21.15.100:3300/0 conn(0x7f4eec0c6df0 0x7f4eec0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eebfff640 1 --2- 172.21.15.60:0/709835210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0d1c00 0x7f4eec0ccc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009d90 tx=0x7f4ee402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919971551b5ba276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eea7fc640 1 -- 172.21.15.60:0/709835210 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4007740 con 0x7f4eec0d1c00 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.353+0000 7f4eea7fc640 1 -- 172.21.15.60:0/709835210 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee40078e0 con 0x7f4eec0d1c00 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4eea7fc640 1 -- 172.21.15.60:0/709835210 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4007cb0 con 0x7f4eec0d1c00 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 -- 172.21.15.60:0/709835210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0d1c00 msgr2=0x7f4eec0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/709835210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0d1c00 0x7f4eec0ccc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009d90 tx=0x7f4ee402a660 comp rx=0 tx=0).stop 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 -- 172.21.15.60:0/709835210 shutdown_connections 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/709835210 >> v2:172.21.15.100:3300/0 conn(0x7f4eec0c6df0 0x7f4eec0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/709835210 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0d1c00 0x7f4eec0ccc40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 -- 172.21.15.60:0/709835210 >> 172.21.15.60:0/709835210 conn(0x7f4eec06f5d0 msgr2=0x7f4eec06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 -- 172.21.15.60:0/709835210 shutdown_connections 2024-04-25T08:39:08.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.354+0000 7f4ef1f28640 1 -- 172.21.15.60:0/709835210 wait complete. 2024-04-25T08:39:08.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.355+0000 7f4ef1f28640 1 Processor -- start 2024-04-25T08:39:08.350 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.355+0000 7f4ef1f28640 1 -- start start 2024-04-25T08:39:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4ef1f28640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0c6df0 0x7f4eec0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4ef1f28640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 0x7f4eec0b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4ef1f28640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec15dc00 0x7f4eec1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4ef1f28640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0cc130 con 0x7f4eec0d1c00 2024-04-25T08:39:08.352 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4ef1f28640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0cc720 con 0x7f4eec15dc00 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4ef1f28640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0cfd90 con 0x7f4eec0c6df0 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 0x7f4eec0b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4eeaffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 0x7f4eec0b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45954/0 (socket says 172.21.15.60:45954) 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4eeaffd640 1 -- 172.21.15.60:0/2405646263 learned_addr learned my addr 172.21.15.60:0/2405646263 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4eeb7fe640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0c6df0 0x7f4eec0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.356+0000 7f4eebfff640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec15dc00 0x7f4eec1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4eeaffd640 1 -- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0c6df0 msgr2=0x7f4eec0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4eeaffd640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0c6df0 0x7f4eec0b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4eeaffd640 1 -- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec15dc00 msgr2=0x7f4eec1f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4eeaffd640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec15dc00 0x7f4eec1f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4eeaffd640 1 -- 172.21.15.60:0/2405646263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec1f7a90 con 0x7f4eec0d1c00 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4eeaffd640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 0x7f4eec0b72c0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000ec10 tx=0x7f4ee000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0017070 con 0x7f4eec0d1c00 2024-04-25T08:39:08.353 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee000cc30 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec1f7d70 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0013470 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.357+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec0b5220 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.358+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec073990 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.358+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ee001a070 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.359+0000 7f4ed3fff640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.359+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4ee008a750 con 0x7f4eec0d1c00 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.359+0000 7f4eeb7fe640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.359+0000 7f4eeb7fe640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ed806b4d0 0x7f4ed806d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4edc000f60 tx=0x7f4edc007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.356 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.363+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ee0059370 con 0x7f4eec0d1c00 2024-04-25T08:39:08.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.581+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4eec15a8b0 con 0x7f4eec0d1c00 2024-04-25T08:39:08.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.581+0000 7f4ed3fff640 1 -- 172.21.15.60:0/2405646263 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f4ee0058c70 con 0x7f4eec0d1c00 2024-04-25T08:39:08.575 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:08.575 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:39:06.172452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:08.575 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ed806b4d0 msgr2=0x7f4ed806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ed806b4d0 0x7f4ed806d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4edc000f60 tx=0x7f4edc007c00 comp rx=0 tx=0).stop 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 msgr2=0x7f4eec0b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 0x7f4eec0b72c0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000ec10 tx=0x7f4ee000c520 comp rx=0 tx=0).stop 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 shutdown_connections 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4eec15dc00 0x7f4eec1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4eec0d1c00 0x7f4eec0b72c0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 --2- 172.21.15.60:0/2405646263 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4eec0c6df0 0x7f4eec0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 >> 172.21.15.60:0/2405646263 conn(0x7f4eec06f5d0 msgr2=0x7f4eec0b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 shutdown_connections 2024-04-25T08:39:08.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.583+0000 7f4ef1f28640 1 -- 172.21.15.60:0/2405646263 wait complete. 2024-04-25T08:39:08.588 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:39:08.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:08.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.694+0000 7fe774b70640 1 Processor -- start 2024-04-25T08:39:08.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.694+0000 7fe774b70640 1 -- start start 2024-04-25T08:39:08.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.695+0000 7fe774b70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7701602e0 0x7fe7700ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.695+0000 7fe774b70640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe77015f5f0 0x7fe77015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.695+0000 7fe774b70640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7700cd710 con 0x7fe77015ba00 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.695+0000 7fe774b70640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe770099b70 con 0x7fe77015f5f0 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.695+0000 7fe774b70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe770099e50 con 0x7fe7701602e0 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe77015f5f0 0x7fe77015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe77015f5f0 0x7fe77015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45960/0 (socket says 172.21.15.60:45960) 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 -- 172.21.15.60:0/3712038224 learned_addr learned my addr 172.21.15.60:0/3712038224 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76dd9b640 1 --2- 172.21.15.60:0/3712038224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7701602e0 0x7fe7700ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 -- 172.21.15.60:0/3712038224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7701602e0 msgr2=0x7fe7700ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 --2- 172.21.15.60:0/3712038224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7701602e0 0x7fe7700ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 -- 172.21.15.60:0/3712038224 >> v1:172.21.15.100:6789/0 conn(0x7fe77015ba00 legacy=0x7fe77009a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76ed9d640 1 -- 172.21.15.60:0/3712038224 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77009a130 con 0x7fe77015f5f0 2024-04-25T08:39:08.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.696+0000 7fe76d59a640 1 -- 172.21.15.60:0/3712038224 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160546216 0 0) 0x7fe768001110 con 0x7fe77015ba00 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe76ed9d640 1 --2- 172.21.15.60:0/3712038224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015faa0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe760009e50 tx=0x7fe76002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b463f3294a554cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe76d59a640 1 -- 172.21.15.60:0/3712038224 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760036070 con 0x7fe77015f5f0 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe76d59a640 1 -- 172.21.15.60:0/3712038224 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760031600 con 0x7fe77015f5f0 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe76d59a640 1 -- 172.21.15.60:0/3712038224 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600319d0 con 0x7fe77015f5f0 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 -- 172.21.15.60:0/3712038224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 msgr2=0x7fe77015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 --2- 172.21.15.60:0/3712038224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015faa0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe760009e50 tx=0x7fe76002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 -- 172.21.15.60:0/3712038224 shutdown_connections 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 --2- 172.21.15.60:0/3712038224 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015faa0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 --2- 172.21.15.60:0/3712038224 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe7701602e0 0x7fe7700ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 -- 172.21.15.60:0/3712038224 >> 172.21.15.60:0/3712038224 conn(0x7fe7700b3270 msgr2=0x7fe7700b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 -- 172.21.15.60:0/3712038224 shutdown_connections 2024-04-25T08:39:08.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.697+0000 7fe774b70640 1 -- 172.21.15.60:0/3712038224 wait complete. 2024-04-25T08:39:08.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 Processor -- start 2024-04-25T08:39:08.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 -- start start 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe77015ba00 0x7fe77015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7701602e0 0x7fe7701f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cd460 con 0x7fe77015f5f0 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cd2e0 con 0x7fe7701602e0 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.699+0000 7fe774b70640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c52d0 con 0x7fe77015ba00 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45974/0 (socket says 172.21.15.60:45974) 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 -- 172.21.15.60:0/4105385486 learned_addr learned my addr 172.21.15.60:0/4105385486 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76ed9d640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7701602e0 0x7fe7701f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 -- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe77015ba00 msgr2=0x7fe77015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76e59c640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe77015ba00 0x7fe77015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe77015ba00 0x7fe77015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 -- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7701602e0 msgr2=0x7fe7701f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7701602e0 0x7fe7701f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 -- 172.21.15.60:0/4105385486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77015c680 con 0x7fe77015f5f0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76ed9d640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7701602e0 0x7fe7701f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe76dd9b640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015e9e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe75800e870 tx=0x7fe75800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe758017070 con 0x7fe77015f5f0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75800c940 con 0x7fe77015f5f0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe770145640 con 0x7fe77015f5f0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75800cd30 con 0x7fe77015f5f0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.700+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770145c00 con 0x7fe77015f5f0 2024-04-25T08:39:08.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.702+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe770073990 con 0x7fe77015f5f0 2024-04-25T08:39:08.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.702+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe75801a070 con 0x7fe77015f5f0 2024-04-25T08:39:08.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.702+0000 7fe766ffd640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe75006b500 0x7fe75006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.702+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe75808a600 con 0x7fe77015f5f0 2024-04-25T08:39:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.703+0000 7fe76e59c640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe75006b500 0x7fe75006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.703+0000 7fe76e59c640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe75006b500 0x7fe75006d9c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe768000f60 tx=0x7fe768009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.707+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7580592a0 con 0x7fe77015f5f0 2024-04-25T08:39:08.846 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:39:08.852+0000 7f358c940640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:39:08.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.957+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe7700c55c0 con 0x7fe77015f5f0 2024-04-25T08:39:08.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.957+0000 7fe766ffd640 1 -- 172.21.15.60:0/4105385486 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1464 (secure 0 0 0) 0x7fe758058ba0 con 0x7fe77015f5f0 2024-04-25T08:39:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:08.951 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":609,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":135,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076561899,"bytes_used":2595487744,"bytes_avail":770498625536,"bytes_total":773094113280,"read_bytes_sec":230748,"write_bytes_sec":204,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":82,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9375}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T08:39:05.533541+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe75006b500 msgr2=0x7fe75006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe75006b500 0x7fe75006d9c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe768000f60 tx=0x7fe768009040 comp rx=0 tx=0).stop 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 msgr2=0x7fe77015e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015e9e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe75800e870 tx=0x7fe75800ed30 comp rx=0 tx=0).stop 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 shutdown_connections 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe75006b500 0x7fe75006d9c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7701602e0 0x7fe7701f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe77015f5f0 0x7fe77015e9e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 --2- 172.21.15.60:0/4105385486 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe77015ba00 0x7fe77015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 >> 172.21.15.60:0/4105385486 conn(0x7fe7700b3270 msgr2=0x7fe77006fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 shutdown_connections 2024-04-25T08:39:08.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:08.959+0000 7fe774b70640 1 -- 172.21.15.60:0/4105385486 wait complete. 2024-04-25T08:39:08.963 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:39:08.964 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:39:08.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:09.001 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:09.045 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:39:09.045 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:09.045 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:09.061 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:09.061 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:09.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:09.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.264+0000 7f388693b640 1 Processor -- start 2024-04-25T08:39:09.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.264+0000 7f388693b640 1 -- start start 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f388693b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f38800d2e90 0x7f38800d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f388693b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f38800d0900 0x7f38800b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f388693b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38800b9940 con 0x7f388015dd30 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f388693b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800d0d40 con 0x7f38800d0900 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f388693b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38800d1df0 con 0x7f38800d2e90 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f387ffff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f388015dd30 0x7f38800b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43136/0 (socket says 172.21.15.60:43136) 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f387ffff640 1 -- 172.21.15.60:0/1886218638 learned_addr learned my addr 172.21.15.60:0/1886218638 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:09.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f3884ed8640 1 --2- 172.21.15.60:0/1886218638 >> v2:172.21.15.60:3300/0 conn(0x7f38800d0900 0x7f38800b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f3884ed8640 1 -- 172.21.15.60:0/1886218638 >> v2:172.21.15.100:3300/0 conn(0x7f38800d2e90 msgr2=0x7f38800d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f3884ed8640 1 --2- 172.21.15.60:0/1886218638 >> v2:172.21.15.100:3300/0 conn(0x7f38800d2e90 0x7f38800d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f3884ed8640 1 -- 172.21.15.60:0/1886218638 >> v1:172.21.15.60:6789/0 conn(0x7f388015dd30 legacy=0x7f38800b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.266+0000 7f3884ed8640 1 -- 172.21.15.60:0/1886218638 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38800d20d0 con 0x7f38800d0900 2024-04-25T08:39:09.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.265+0000 7f387effd640 1 -- 172.21.15.60:0/1886218638 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631873919 0 0) 0x7f38800b9940 con 0x7f388015dd30 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.266+0000 7f3884ed8640 1 --2- 172.21.15.60:0/1886218638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d0900 0x7f38800b6a40 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f38800b6540 tx=0x7f387802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6248af041a935037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.266+0000 7f387effd640 1 -- 172.21.15.60:0/1886218638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878007740 con 0x7f38800d0900 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.266+0000 7f387effd640 1 -- 172.21.15.60:0/1886218638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38780078e0 con 0x7f38800d0900 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.266+0000 7f387effd640 1 -- 172.21.15.60:0/1886218638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878007cb0 con 0x7f38800d0900 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 -- 172.21.15.60:0/1886218638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d0900 msgr2=0x7f38800b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 --2- 172.21.15.60:0/1886218638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d0900 0x7f38800b6a40 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f38800b6540 tx=0x7f387802a560 comp rx=0 tx=0).stop 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 -- 172.21.15.60:0/1886218638 shutdown_connections 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 --2- 172.21.15.60:0/1886218638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d0900 0x7f38800b6a40 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 --2- 172.21.15.60:0/1886218638 >> v2:172.21.15.100:3300/0 conn(0x7f38800d2e90 0x7f38800d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 -- 172.21.15.60:0/1886218638 >> 172.21.15.60:0/1886218638 conn(0x7f3880061c30 msgr2=0x7f388006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 -- 172.21.15.60:0/1886218638 shutdown_connections 2024-04-25T08:39:09.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.267+0000 7f388693b640 1 -- 172.21.15.60:0/1886218638 wait complete. 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.268+0000 7f388693b640 1 Processor -- start 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.268+0000 7f388693b640 1 -- start start 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f388693b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38800d0900 0x7f38800b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f388693b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 0x7f38801f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f388693b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f388015dd30 0x7f38801f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f388693b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38800d2970 con 0x7f38800d2e90 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f388693b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38800d2690 con 0x7f38800d0900 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f388693b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38800d1a10 con 0x7f388015dd30 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 0x7f38801f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 0x7f38801f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45984/0 (socket says 172.21.15.60:45984) 2024-04-25T08:39:09.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 -- 172.21.15.60:0/2496739258 learned_addr learned my addr 172.21.15.60:0/2496739258 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387ffff640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38800d0900 0x7f38800b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 -- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f388015dd30 msgr2=0x7f38801f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f388015dd30 0x7f38801f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 -- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38800d0900 msgr2=0x7f38800b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38800d0900 0x7f38800b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387f7fe640 1 -- 172.21.15.60:0/2496739258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38801f3fc0 con 0x7f38800d2e90 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.269+0000 7f387ffff640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38800d0900 0x7f38800b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.270+0000 7f387f7fe640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 0x7f38801f7900 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f387000b4d0 tx=0x7f387000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.270+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870014070 con 0x7f38800d2e90 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.270+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870005c10 con 0x7f38800d2e90 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.270+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38801f42a0 con 0x7f38800d2e90 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.270+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870010670 con 0x7f38800d2e90 2024-04-25T08:39:09.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.270+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38801f3cb0 con 0x7f38800d2e90 2024-04-25T08:39:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.271+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880157730 con 0x7f38800d2e90 2024-04-25T08:39:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.271+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3870010810 con 0x7f38800d2e90 2024-04-25T08:39:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.272+0000 7f387cff9640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f386006b4d0 0x7f386006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.272+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f387008a130 con 0x7f38800d2e90 2024-04-25T08:39:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.272+0000 7f387ffff640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f386006b4d0 0x7f386006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.272+0000 7f387ffff640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f386006b4d0 0x7f386006d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3868000f60 tx=0x7f3868009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.276+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3870058de0 con 0x7f38800d2e90 2024-04-25T08:39:09.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.458+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38800b2960 con 0x7f38800d2e90 2024-04-25T08:39:09.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.459+0000 7f387cff9640 1 -- 172.21.15.60:0/2496739258 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17017 (secure 0 0 0) 0x7f38700586e0 con 0x7f38800d2e90 2024-04-25T08:39:09.452 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:09.452 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":135,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:04.147601+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6836/879630460":"2024-04-26T08:39:04.147468+0000","172.21.15.100:6834/879630460":"2024-04-26T08:39:04.147468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.460+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f386006b4d0 msgr2=0x7f386006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.460+0000 7f388693b640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f386006b4d0 0x7f386006d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3868000f60 tx=0x7f3868009040 comp rx=0 tx=0).stop 2024-04-25T08:39:09.454 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.460+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 msgr2=0x7f38801f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.460+0000 7f388693b640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 0x7f38801f7900 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f387000b4d0 tx=0x7f387000b990 comp rx=0 tx=0).stop 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 shutdown_connections 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f388015dd30 0x7f38801f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38800d2e90 0x7f38801f7900 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f386006b4d0 0x7f386006d990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 --2- 172.21.15.60:0/2496739258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f38800d0900 0x7f38800b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 >> 172.21.15.60:0/2496739258 conn(0x7f3880061c30 msgr2=0x7f388006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 shutdown_connections 2024-04-25T08:39:09.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.461+0000 7f388693b640 1 -- 172.21.15.60:0/2496739258 wait complete. 2024-04-25T08:39:09.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:38:15.506220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:38:16.201959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:09.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:09.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.577+0000 7ff34349a640 1 Processor -- start 2024-04-25T08:39:09.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.577+0000 7ff34349a640 1 -- start start 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff34349a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff34349a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff33c161710 0x7ff33c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff34349a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c15e000 con 0x7ff33c15dc20 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff34349a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c0d1de0 con 0x7ff33c161710 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff34349a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c161c60 con 0x7ff33c0ced80 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:55280/0 (socket says 172.21.15.60:55280) 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 -- 172.21.15.60:0/2576010116 learned_addr learned my addr 172.21.15.60:0/2576010116 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff341a37640 1 --2- 172.21.15.60:0/2576010116 >> v2:172.21.15.100:3300/0 conn(0x7ff33c161710 0x7ff33c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 -- 172.21.15.60:0/2576010116 >> v2:172.21.15.100:3300/0 conn(0x7ff33c161710 msgr2=0x7ff33c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:09.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 --2- 172.21.15.60:0/2576010116 >> v2:172.21.15.100:3300/0 conn(0x7ff33c161710 0x7ff33c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 -- 172.21.15.60:0/2576010116 >> v1:172.21.15.100:6789/0 conn(0x7ff33c15dc20 legacy=0x7ff33c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff333fff640 1 -- 172.21.15.60:0/2576010116 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659179915 0 0) 0x7ff33c15e000 con 0x7ff33c15dc20 2024-04-25T08:39:09.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.578+0000 7ff340a35640 1 -- 172.21.15.60:0/2576010116 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c161f40 con 0x7ff33c0ced80 2024-04-25T08:39:09.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.579+0000 7ff340a35640 1 --2- 172.21.15.60:0/2576010116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c0d22e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009870 tx=0x7ff32c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c10bf63cf2440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.579+0000 7ff333fff640 1 -- 172.21.15.60:0/2576010116 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c037070 con 0x7ff33c0ced80 2024-04-25T08:39:09.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.579+0000 7ff333fff640 1 -- 172.21.15.60:0/2576010116 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c032440 con 0x7ff33c0ced80 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.579+0000 7ff333fff640 1 -- 172.21.15.60:0/2576010116 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c032810 con 0x7ff33c0ced80 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 -- 172.21.15.60:0/2576010116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 msgr2=0x7ff33c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 --2- 172.21.15.60:0/2576010116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c0d22e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009870 tx=0x7ff32c02a270 comp rx=0 tx=0).stop 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 -- 172.21.15.60:0/2576010116 shutdown_connections 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 --2- 172.21.15.60:0/2576010116 >> v2:172.21.15.100:3300/0 conn(0x7ff33c161710 0x7ff33c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 --2- 172.21.15.60:0/2576010116 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c0d22e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 -- 172.21.15.60:0/2576010116 >> 172.21.15.60:0/2576010116 conn(0x7ff33c0d0420 msgr2=0x7ff33c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 -- 172.21.15.60:0/2576010116 shutdown_connections 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.580+0000 7ff34349a640 1 -- 172.21.15.60:0/2576010116 wait complete. 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.581+0000 7ff34349a640 1 Processor -- start 2024-04-25T08:39:09.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.581+0000 7ff34349a640 1 -- start start 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff34349a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff34349a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 0x7ff33c0ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff34349a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff33c161710 0x7ff33c0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff34349a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0b5e50 con 0x7ff33c15dc20 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff34349a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c162500 con 0x7ff33c161710 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff34349a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0b5fd0 con 0x7ff33c0ced80 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 0x7ff33c0ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 0x7ff33c0ac5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:45996/0 (socket says 172.21.15.60:45996) 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 -- 172.21.15.60:0/309695033 learned_addr learned my addr 172.21.15.60:0/309695033 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff341a37640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff33c161710 0x7ff33c0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff341236640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 -- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 msgr2=0x7ff33c15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c15ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 -- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff33c161710 msgr2=0x7ff33c0acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff33c161710 0x7ff33c0acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 -- 172.21.15.60:0/309695033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0aa5d0 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.582+0000 7ff340a35640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 0x7ff33c0ac5e0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0090b0 tx=0x7ff32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.583+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c037040 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.583+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c004030 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.583+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c004420 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.583+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c0aa8b0 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.583+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c0aae70 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.584+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c073a00 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.584+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff32c03a040 con 0x7ff33c15dc20 2024-04-25T08:39:09.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.585+0000 7ff331ffb640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.585+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff32c0aa8e0 con 0x7ff33c15dc20 2024-04-25T08:39:09.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.585+0000 7ff341236640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.580 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.585+0000 7ff341236640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff32006b4d0 0x7ff32006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff338000f60 tx=0x7ff338009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.589+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff32c042020 con 0x7ff33c15dc20 2024-04-25T08:39:09.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.772+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff33c0d16b0 con 0x7ff33c15dc20 2024-04-25T08:39:09.766 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.772+0000 7ff331ffb640 1 -- 172.21.15.60:0/309695033 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff32c078f20 con 0x7ff33c15dc20 2024-04-25T08:39:09.766 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:39:09.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.774+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff32006b4d0 msgr2=0x7ff32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.774+0000 7ff34349a640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff32006b4d0 0x7ff32006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff338000f60 tx=0x7ff338009040 comp rx=0 tx=0).stop 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.774+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 msgr2=0x7ff33c0ac5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.774+0000 7ff34349a640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 0x7ff33c0ac5e0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0090b0 tx=0x7ff32c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 shutdown_connections 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff32006b4d0 0x7ff32006d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff33c161710 0x7ff33c0acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff33c15dc20 0x7ff33c0ac5e0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 --2- 172.21.15.60:0/309695033 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff33c0ced80 0x7ff33c15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 >> 172.21.15.60:0/309695033 conn(0x7ff33c0d0420 msgr2=0x7ff33c0b83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 shutdown_connections 2024-04-25T08:39:09.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.775+0000 7ff34349a640 1 -- 172.21.15.60:0/309695033 wait complete. 2024-04-25T08:39:09.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:39:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.888+0000 7fcb8fae1640 1 Processor -- start 2024-04-25T08:39:09.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 -- start start 2024-04-25T08:39:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb88161240 0x7fcb880b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcb88161890 0x7fcb880ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb880b54a0 0x7fcb880b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880b7120 con 0x7fcb880b54a0 2024-04-25T08:39:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb88161620 con 0x7fcb88161890 2024-04-25T08:39:09.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.889+0000 7fcb8fae1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88161e00 con 0x7fcb88161240 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb880b54a0 0x7fcb880b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcb880b54a0 0x7fcb880b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46000/0 (socket says 172.21.15.60:46000) 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 -- 172.21.15.60:0/166365383 learned_addr learned my addr 172.21.15.60:0/166365383 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8d07c640 1 --2- 172.21.15.60:0/166365383 >> v2:172.21.15.100:3300/0 conn(0x7fcb88161890 0x7fcb880ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8d87d640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb88161240 0x7fcb880b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 -- 172.21.15.60:0/166365383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb88161240 msgr2=0x7fcb880b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb88161240 0x7fcb880b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 -- 172.21.15.60:0/166365383 >> v2:172.21.15.100:3300/0 conn(0x7fcb88161890 msgr2=0x7fcb880ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 --2- 172.21.15.60:0/166365383 >> v2:172.21.15.100:3300/0 conn(0x7fcb88161890 0x7fcb880ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8e07e640 1 -- 172.21.15.60:0/166365383 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb881620e0 con 0x7fcb880b54a0 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.890+0000 7fcb8d87d640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb88161240 0x7fcb880b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8e07e640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb880b54a0 0x7fcb880b5930 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcb880d4ae0 tx=0x7fcb7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be69e927bcefae12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8c87b640 1 -- 172.21.15.60:0/166365383 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78036070 con 0x7fcb880b54a0 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8c87b640 1 -- 172.21.15.60:0/166365383 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78031600 con 0x7fcb880b54a0 2024-04-25T08:39:09.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8c87b640 1 -- 172.21.15.60:0/166365383 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb780319d0 con 0x7fcb880b54a0 2024-04-25T08:39:09.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 -- 172.21.15.60:0/166365383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb880b54a0 msgr2=0x7fcb880b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb880b54a0 0x7fcb880b5930 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcb880d4ae0 tx=0x7fcb7802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 -- 172.21.15.60:0/166365383 shutdown_connections 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb880b54a0 0x7fcb880b5930 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/166365383 >> v2:172.21.15.100:3300/0 conn(0x7fcb88161890 0x7fcb880ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/166365383 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb88161240 0x7fcb880b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.891+0000 7fcb8fae1640 1 -- 172.21.15.60:0/166365383 >> 172.21.15.60:0/166365383 conn(0x7fcb880d04a0 msgr2=0x7fcb880d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.892+0000 7fcb8fae1640 1 -- 172.21.15.60:0/166365383 shutdown_connections 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.892+0000 7fcb8fae1640 1 -- 172.21.15.60:0/166365383 wait complete. 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.893+0000 7fcb8fae1640 1 Processor -- start 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.893+0000 7fcb8fae1640 1 -- start start 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.893+0000 7fcb8fae1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb880b54a0 0x7fcb880c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.893+0000 7fcb8fae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb88161240 0x7fcb88150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8fae1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 0x7fcb881f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8fae1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880d15f0 con 0x7fcb88161890 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8fae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881626a0 con 0x7fcb88161240 2024-04-25T08:39:09.889 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8fae1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880cf000 con 0x7fcb880b54a0 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 0x7fcb881f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 0x7fcb881f3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46014/0 (socket says 172.21.15.60:46014) 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 -- 172.21.15.60:0/2722497924 learned_addr learned my addr 172.21.15.60:0/2722497924 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8d07c640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb88161240 0x7fcb88150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8d87d640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb880b54a0 0x7fcb880c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 -- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb880b54a0 msgr2=0x7fcb880c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb880b54a0 0x7fcb880c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 -- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb88161240 msgr2=0x7fcb88150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb88161240 0x7fcb88150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 -- 172.21.15.60:0/2722497924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8815fd30 con 0x7fcb88161890 2024-04-25T08:39:09.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8e07e640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 0x7fcb881f3e00 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fcb78034040 tx=0x7fcb78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78041020 con 0x7fcb88161890 2024-04-25T08:39:09.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb780077a0 con 0x7fcb88161890 2024-04-25T08:39:09.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb88160590 con 0x7fcb88161890 2024-04-25T08:39:09.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78036040 con 0x7fcb88161890 2024-04-25T08:39:09.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.894+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb881602b0 con 0x7fcb88161890 2024-04-25T08:39:09.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.896+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb88073a00 con 0x7fcb88161890 2024-04-25T08:39:09.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.896+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb78044020 con 0x7fcb88161890 2024-04-25T08:39:09.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.896+0000 7fcb867fc640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.896+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb780ab000 con 0x7fcb88161890 2024-04-25T08:39:09.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.896+0000 7fcb8d87d640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.897+0000 7fcb8d87d640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcb7406b4d0 0x7fcb7406d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006fd0 tx=0x7fcb80008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:09.901+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb7804b030 con 0x7fcb88161890 2024-04-25T08:39:10.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.080+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb8806d240 con 0x7fcb88161890 2024-04-25T08:39:10.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.080+0000 7fcb867fc640 1 -- 172.21.15.60:0/2722497924 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb78079d30 con 0x7fcb88161890 2024-04-25T08:39:10.074 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:39:10.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcb7406b4d0 msgr2=0x7fcb7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcb7406b4d0 0x7fcb7406d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006fd0 tx=0x7fcb80008040 comp rx=0 tx=0).stop 2024-04-25T08:39:10.076 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 msgr2=0x7fcb881f3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 0x7fcb881f3e00 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fcb78034040 tx=0x7fcb78004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 shutdown_connections 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcb88161890 0x7fcb881f3e00 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb88161240 0x7fcb88150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 --2- 172.21.15.60:0/2722497924 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb880b54a0 0x7fcb880c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.082+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 >> 172.21.15.60:0/2722497924 conn(0x7fcb880d04a0 msgr2=0x7fcb8815c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.083+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 shutdown_connections 2024-04-25T08:39:10.077 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.083+0000 7fcb8fae1640 1 -- 172.21.15.60:0/2722497924 wait complete. 2024-04-25T08:39:10.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:39:10.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.197+0000 7ff2e715c640 1 Processor -- start 2024-04-25T08:39:10.191 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.197+0000 7ff2e715c640 1 -- start start 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e715c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2e01446d0 0x7ff2e00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e715c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff2e0144d50 0x7ff2e00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e715c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e00cccc0 con 0x7ff2e01452a0 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e715c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00b2b20 con 0x7ff2e01446d0 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e715c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e006d400 con 0x7ff2e0144d50 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e4ef8640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff2e01446d0 0x7ff2e00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e56f9640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ff2e01452a0 0x7ff2e00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43150/0 (socket says 172.21.15.60:43150) 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2e56f9640 1 -- 172.21.15.60:0/3807372645 learned_addr learned my addr 172.21.15.60:0/3807372645 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.198+0000 7ff2dffff640 1 --2- 172.21.15.60:0/3807372645 >> v2:172.21.15.100:3300/0 conn(0x7ff2e0144d50 0x7ff2e00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.199+0000 7ff2e4ef8640 1 -- 172.21.15.60:0/3807372645 >> v2:172.21.15.100:3300/0 conn(0x7ff2e0144d50 msgr2=0x7ff2e00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.199+0000 7ff2e4ef8640 1 --2- 172.21.15.60:0/3807372645 >> v2:172.21.15.100:3300/0 conn(0x7ff2e0144d50 0x7ff2e00b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.199+0000 7ff2e4ef8640 1 -- 172.21.15.60:0/3807372645 >> v1:172.21.15.60:6789/0 conn(0x7ff2e01452a0 legacy=0x7ff2e00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.193 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.199+0000 7ff2e4ef8640 1 -- 172.21.15.60:0/3807372645 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00cc140 con 0x7ff2e01446d0 2024-04-25T08:39:10.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.199+0000 7ff2df7fe640 1 -- 172.21.15.60:0/3807372645 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455121545 0 0) 0x7ff2e00cccc0 con 0x7ff2e01452a0 2024-04-25T08:39:10.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.199+0000 7ff2e4ef8640 1 --2- 172.21.15.60:0/3807372645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e01446d0 0x7ff2e00b3f20 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0071c70 tx=0x7ff2d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f515380f33d5f083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2df7fe640 1 -- 172.21.15.60:0/3807372645 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8007740 con 0x7ff2e01446d0 2024-04-25T08:39:10.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2df7fe640 1 -- 172.21.15.60:0/3807372645 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d80078e0 con 0x7ff2e01446d0 2024-04-25T08:39:10.194 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2df7fe640 1 -- 172.21.15.60:0/3807372645 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8007cb0 con 0x7ff2e01446d0 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 -- 172.21.15.60:0/3807372645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e01446d0 msgr2=0x7ff2e00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3807372645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e01446d0 0x7ff2e00b3f20 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0071c70 tx=0x7ff2d802a680 comp rx=0 tx=0).stop 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 -- 172.21.15.60:0/3807372645 shutdown_connections 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3807372645 >> v2:172.21.15.100:3300/0 conn(0x7ff2e0144d50 0x7ff2e00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3807372645 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e01446d0 0x7ff2e00b3f20 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 -- 172.21.15.60:0/3807372645 >> 172.21.15.60:0/3807372645 conn(0x7ff2e006abe0 msgr2=0x7ff2e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 -- 172.21.15.60:0/3807372645 shutdown_connections 2024-04-25T08:39:10.195 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.200+0000 7ff2e715c640 1 -- 172.21.15.60:0/3807372645 wait complete. 2024-04-25T08:39:10.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.201+0000 7ff2e715c640 1 Processor -- start 2024-04-25T08:39:10.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.201+0000 7ff2e715c640 1 -- start start 2024-04-25T08:39:10.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2e715c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e01446d0 0x7ff2e00c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2e715c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 0x7ff2e01f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2e715c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e01452a0 0x7ff2e013ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2e715c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00cc700 con 0x7ff2e0144d50 2024-04-25T08:39:10.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2e715c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00b49a0 con 0x7ff2e01446d0 2024-04-25T08:39:10.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2e715c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00cc9e0 con 0x7ff2e01452a0 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 0x7ff2e01f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2dffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 0x7ff2e01f1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46034/0 (socket says 172.21.15.60:46034) 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.202+0000 7ff2dffff640 1 -- 172.21.15.60:0/3357174845 learned_addr learned my addr 172.21.15.60:0/3357174845 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2e4ef8640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e01446d0 0x7ff2e00c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2e56f9640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e01452a0 0x7ff2e013ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dffff640 1 -- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e01452a0 msgr2=0x7ff2e013ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dffff640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e01452a0 0x7ff2e013ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dffff640 1 -- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e01446d0 msgr2=0x7ff2e00c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dffff640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e01446d0 0x7ff2e00c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dffff640 1 -- 172.21.15.60:0/3357174845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e01f22d0 con 0x7ff2e0144d50 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dffff640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 0x7ff2e01f1810 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000e870 tx=0x7ff2d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0017070 con 0x7ff2e0144d50 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d000c940 con 0x7ff2e0144d50 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e01f25b0 con 0x7ff2e0144d50 2024-04-25T08:39:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d000cd30 con 0x7ff2e0144d50 2024-04-25T08:39:10.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.203+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e00ac7e0 con 0x7ff2e0144d50 2024-04-25T08:39:10.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.204+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e0073a00 con 0x7ff2e0144d50 2024-04-25T08:39:10.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.204+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2d001a070 con 0x7ff2e0144d50 2024-04-25T08:39:10.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.205+0000 7ff2dd7fa640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff2c006b4d0 0x7ff2c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.205+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff2d008a5d0 con 0x7ff2e0144d50 2024-04-25T08:39:10.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.205+0000 7ff2e4ef8640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff2c006b4d0 0x7ff2c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.206+0000 7ff2e4ef8640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff2c006b4d0 0x7ff2c006d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8034000 tx=0x7ff2d8033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.202 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.209+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2d0059300 con 0x7ff2e0144d50 2024-04-25T08:39:10.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.387+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2e01f2780 con 0x7ff2e0144d50 2024-04-25T08:39:10.380 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.387+0000 7ff2dd7fa640 1 -- 172.21.15.60:0/3357174845 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v135) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2d0058c00 con 0x7ff2e0144d50 2024-04-25T08:39:10.381 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff2c006b4d0 msgr2=0x7ff2c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff2c006b4d0 0x7ff2c006d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8034000 tx=0x7ff2d8033040 comp rx=0 tx=0).stop 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 msgr2=0x7ff2e01f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 0x7ff2e01f1810 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000e870 tx=0x7ff2d000ed30 comp rx=0 tx=0).stop 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 shutdown_connections 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff2e01452a0 0x7ff2e013ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff2e0144d50 0x7ff2e01f1810 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff2c006b4d0 0x7ff2c006d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 --2- 172.21.15.60:0/3357174845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e01446d0 0x7ff2e00c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 >> 172.21.15.60:0/3357174845 conn(0x7ff2e006abe0 msgr2=0x7ff2e006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 shutdown_connections 2024-04-25T08:39:10.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.389+0000 7ff2e715c640 1 -- 172.21.15.60:0/3357174845 wait complete. 2024-04-25T08:39:10.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:39:10.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.502+0000 7f9acd8ac640 1 Processor -- start 2024-04-25T08:39:10.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.502+0000 7f9acd8ac640 1 -- start start 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9acd8ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ac815bb50 0x7f9ac80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9acd8ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ac80cfd10 0x7f9ac80cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9acd8ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80c75e0 con 0x7f9ac80c70d0 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9acd8ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80c8680 con 0x7f9ac80cfd10 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9acd8ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80c78c0 con 0x7f9ac815bb50 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9ac77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ac80cfd10 0x7f9ac80cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9ac77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f9ac80cfd10 0x7f9ac80cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46036/0 (socket says 172.21.15.60:46036) 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9ac77fe640 1 -- 172.21.15.60:0/2058474519 learned_addr learned my addr 172.21.15.60:0/2058474519 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:10.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.503+0000 7f9ac6ffd640 1 --2- 172.21.15.60:0/2058474519 >> v2:172.21.15.100:3300/0 conn(0x7f9ac815bb50 0x7f9ac80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac77fe640 1 -- 172.21.15.60:0/2058474519 >> v2:172.21.15.100:3300/0 conn(0x7f9ac815bb50 msgr2=0x7f9ac80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac77fe640 1 --2- 172.21.15.60:0/2058474519 >> v2:172.21.15.100:3300/0 conn(0x7f9ac815bb50 0x7f9ac80c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac77fe640 1 -- 172.21.15.60:0/2058474519 >> v1:172.21.15.100:6789/0 conn(0x7f9ac80c70d0 legacy=0x7f9ac80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac77fe640 1 -- 172.21.15.60:0/2058474519 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80c7ba0 con 0x7f9ac80cfd10 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac77fe640 1 --2- 172.21.15.60:0/2058474519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80cfd10 0x7f9ac80cbf20 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8070ab0 tx=0x7f9ab802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd369b424376f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac5ffb640 1 -- 172.21.15.60:0/2058474519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8036070 con 0x7f9ac80cfd10 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac5ffb640 1 -- 172.21.15.60:0/2058474519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8031600 con 0x7f9ac80cfd10 2024-04-25T08:39:10.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.504+0000 7f9ac5ffb640 1 -- 172.21.15.60:0/2058474519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80319f0 con 0x7f9ac80cfd10 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 -- 172.21.15.60:0/2058474519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80cfd10 msgr2=0x7f9ac80cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/2058474519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80cfd10 0x7f9ac80cbf20 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8070ab0 tx=0x7f9ab802a7c0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 -- 172.21.15.60:0/2058474519 shutdown_connections 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/2058474519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80cfd10 0x7f9ac80cbf20 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/2058474519 >> v2:172.21.15.100:3300/0 conn(0x7f9ac815bb50 0x7f9ac80c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 -- 172.21.15.60:0/2058474519 >> 172.21.15.60:0/2058474519 conn(0x7f9ac806f500 msgr2=0x7f9ac806f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 -- 172.21.15.60:0/2058474519 shutdown_connections 2024-04-25T08:39:10.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.505+0000 7f9acd8ac640 1 -- 172.21.15.60:0/2058474519 wait complete. 2024-04-25T08:39:10.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.506+0000 7f9acd8ac640 1 Processor -- start 2024-04-25T08:39:10.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.506+0000 7f9acd8ac640 1 -- start start 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9acd8ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 0x7f9ac8147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9acd8ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ac80cfd10 0x7f9ac8156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9acd8ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac815bb50 0x7f9ac80b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9acd8ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815f6d0 con 0x7f9ac80c70d0 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9acd8ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80c8160 con 0x7f9ac815bb50 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9acd8ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815f850 con 0x7f9ac80cfd10 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 0x7f9ac8147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9ac6ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 0x7f9ac8147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46052/0 (socket says 172.21.15.60:46052) 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9ac6ffd640 1 -- 172.21.15.60:0/1391677647 learned_addr learned my addr 172.21.15.60:0/1391677647 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9ac67fc640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ac80cfd10 0x7f9ac8156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.507+0000 7f9ac77fe640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac815bb50 0x7f9ac80b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac6ffd640 1 -- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ac80cfd10 msgr2=0x7f9ac8156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac6ffd640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ac80cfd10 0x7f9ac8156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac6ffd640 1 -- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac815bb50 msgr2=0x7f9ac80b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac6ffd640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac815bb50 0x7f9ac80b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac6ffd640 1 -- 172.21.15.60:0/1391677647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac81575d0 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac67fc640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ac80cfd10 0x7f9ac8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9ac6ffd640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 0x7f9ac8147150 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e870 tx=0x7f9ac000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0017070 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac000c940 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac815d730 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000cd30 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.508+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac8147690 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.509+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac8073a00 con 0x7f9ac80c70d0 2024-04-25T08:39:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.509+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ac001a070 con 0x7f9ac80c70d0 2024-04-25T08:39:10.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.510+0000 7f9abffff640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9aa806b500 0x7f9aa806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.510+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ac008a600 con 0x7f9ac80c70d0 2024-04-25T08:39:10.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.510+0000 7f9ac67fc640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9aa806b500 0x7f9aa806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.510+0000 7f9ac67fc640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9aa806b500 0x7f9aa806d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80b4a00 tx=0x7f9ab0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.514+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ac00592a0 con 0x7f9ac80c70d0 2024-04-25T08:39:10.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.707+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ac80cf150 con 0x7f9ac80c70d0 2024-04-25T08:39:10.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.708+0000 7f9abffff640 1 -- 172.21.15.60:0/1391677647 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v135) v1 ==== 80+0+122 (secure 0 0 0) 0x7f9ac0058ba0 con 0x7f9ac80c70d0 2024-04-25T08:39:10.701 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.100:6836/879630460 2024-04-26T08:39:04.147468+0000 2024-04-25T08:39:10.701 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.15.100:6834/879630460 2024-04-26T08:39:04.147468+0000 2024-04-25T08:39:10.702 INFO:teuthology.orchestra.run.smithi060.stderr:listed 2 entries 2024-04-25T08:39:10.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.709+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9aa806b500 msgr2=0x7f9aa806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.709+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9aa806b500 0x7f9aa806d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80b4a00 tx=0x7f9ab0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:10.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.709+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 msgr2=0x7f9ac8147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.709+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 0x7f9ac8147150 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000e870 tx=0x7f9ac000ed30 comp rx=0 tx=0).stop 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 shutdown_connections 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9aa806b500 0x7f9aa806d9c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ac815bb50 0x7f9ac80b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9ac80cfd10 0x7f9ac8156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 --2- 172.21.15.60:0/1391677647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9ac80c70d0 0x7f9ac8147150 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 >> 172.21.15.60:0/1391677647 conn(0x7f9ac806f500 msgr2=0x7f9ac8154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 shutdown_connections 2024-04-25T08:39:10.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.710+0000 7f9acd8ac640 1 -- 172.21.15.60:0/1391677647 wait complete. 2024-04-25T08:39:10.714 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:39:10.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:10.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:10.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:39:10.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:10.766 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:39:10.781 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:10.781 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:10.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:10.800 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:10.800 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:10.836 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:10.836 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:10.836 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:10.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:10.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 Processor -- start 2024-04-25T08:39:10.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 -- start start 2024-04-25T08:39:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30f815dc00 0x7f30f80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80d1dd0 0x7f30f80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f815dfe0 con 0x7f30f80ceb20 2024-04-25T08:39:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30f80d00c0 con 0x7f30f815dc00 2024-04-25T08:39:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.984+0000 7f30fe66b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80d03a0 con 0x7f30f80d1dd0 2024-04-25T08:39:10.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30f815dc00 0x7f30f80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f30f815dc00 0x7f30f80c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46058/0 (socket says 172.21.15.60:46058) 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 -- 172.21.15.60:0/617805118 learned_addr learned my addr 172.21.15.60:0/617805118 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30fcc08640 1 --2- 172.21.15.60:0/617805118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80d1dd0 0x7f30f80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 -- 172.21.15.60:0/617805118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80d1dd0 msgr2=0x7f30f80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 --2- 172.21.15.60:0/617805118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80d1dd0 0x7f30f80ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 -- 172.21.15.60:0/617805118 >> v1:172.21.15.100:6789/0 conn(0x7f30f80ceb20 legacy=0x7f30f80d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.985+0000 7f30f7fff640 1 -- 172.21.15.60:0/617805118 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80cc440 con 0x7f30f815dc00 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30f7fff640 1 --2- 172.21.15.60:0/617805118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f80c6df0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009870 tx=0x7f30e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9a90e948d5bcf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30f6ffd640 1 -- 172.21.15.60:0/617805118 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8037070 con 0x7f30f815dc00 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30f6ffd640 1 -- 172.21.15.60:0/617805118 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e8032440 con 0x7f30f815dc00 2024-04-25T08:39:10.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30f6ffd640 1 -- 172.21.15.60:0/617805118 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8032810 con 0x7f30f815dc00 2024-04-25T08:39:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30fe66b640 1 -- 172.21.15.60:0/617805118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 msgr2=0x7f30f80c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30fe66b640 1 --2- 172.21.15.60:0/617805118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f80c6df0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009870 tx=0x7f30e802a270 comp rx=0 tx=0).stop 2024-04-25T08:39:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30fe66b640 1 -- 172.21.15.60:0/617805118 shutdown_connections 2024-04-25T08:39:10.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30fe66b640 1 --2- 172.21.15.60:0/617805118 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80d1dd0 0x7f30f80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30fe66b640 1 --2- 172.21.15.60:0/617805118 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f80c6df0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.986+0000 7f30fe66b640 1 -- 172.21.15.60:0/617805118 >> 172.21.15.60:0/617805118 conn(0x7f30f806f5d0 msgr2=0x7f30f806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.987+0000 7f30fe66b640 1 -- 172.21.15.60:0/617805118 shutdown_connections 2024-04-25T08:39:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.987+0000 7f30fe66b640 1 -- 172.21.15.60:0/617805118 wait complete. 2024-04-25T08:39:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.988+0000 7f30fe66b640 1 Processor -- start 2024-04-25T08:39:10.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.988+0000 7f30fe66b640 1 -- start start 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80ceb20 0x7f30f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f80d1dd0 0x7f30f81f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f8159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80cfd90 con 0x7f30f815dc00 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80cc130 con 0x7f30f80d1dd0 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80cdd70 con 0x7f30f80ceb20 2024-04-25T08:39:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f8159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f8159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46068/0 (socket says 172.21.15.60:46068) 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 -- 172.21.15.60:0/3712827168 learned_addr learned my addr 172.21.15.60:0/3712827168 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30f7fff640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80ceb20 0x7f30f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30f77fe640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f80d1dd0 0x7f30f81f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 -- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80ceb20 msgr2=0x7f30f80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80ceb20 0x7f30f80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 -- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f80d1dd0 msgr2=0x7f30f81f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f80d1dd0 0x7f30f81f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 -- 172.21.15.60:0/3712827168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80b66b0 con 0x7f30f815dc00 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30f77fe640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f80d1dd0 0x7f30f81f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fcc08640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f8159f30 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009e30 tx=0x7f30f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0018020 con 0x7f30f815dc00 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0014480 con 0x7f30f815dc00 2024-04-25T08:39:10.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f80b6990 con 0x7f30f815dc00 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0014870 con 0x7f30f815dc00 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.989+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f81f3c70 con 0x7f30f815dc00 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.991+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f80c7500 con 0x7f30f815dc00 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.991+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30f0005d10 con 0x7f30f815dc00 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.991+0000 7f30f4ff9640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30e406b4d0 0x7f30e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.991+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f30f008b240 con 0x7f30f815dc00 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.992+0000 7f30f7fff640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30e406b4d0 0x7f30e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.992+0000 7f30f7fff640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30e406b4d0 0x7f30e406d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f30e8034000 tx=0x7f30e802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:10.996+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30f80c7500 con 0x7f30f815dc00 2024-04-25T08:39:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.214+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30f8073990 con 0x7f30f815dc00 2024-04-25T08:39:11.208 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.215+0000 7f30f4ff9640 1 -- 172.21.15.60:0/3712827168 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f30f0022030 con 0x7f30f815dc00 2024-04-25T08:39:11.208 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:11.208 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:39:06.172452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:11.209 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T08:39:11.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.216+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30e406b4d0 msgr2=0x7f30e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.216+0000 7f30fe66b640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30e406b4d0 0x7f30e406d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f30e8034000 tx=0x7f30e802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:11.210 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.216+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 msgr2=0x7f30f8159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.216+0000 7f30fe66b640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f8159f30 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f30f0009e30 tx=0x7f30f000c490 comp rx=0 tx=0).stop 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.216+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 shutdown_connections 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f30f815dc00 0x7f30f8159f30 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f30e406b4d0 0x7f30e406d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f80d1dd0 0x7f30f81f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 --2- 172.21.15.60:0/3712827168 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f30f80ceb20 0x7f30f80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 >> 172.21.15.60:0/3712827168 conn(0x7f30f806f5d0 msgr2=0x7f30f80c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 shutdown_connections 2024-04-25T08:39:11.211 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.217+0000 7f30fe66b640 1 -- 172.21.15.60:0/3712827168 wait complete. 2024-04-25T08:39:11.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:11.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.329+0000 7f9b3d3e2640 1 Processor -- start 2024-04-25T08:39:11.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.329+0000 7f9b3d3e2640 1 -- start start 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.329+0000 7f9b3d3e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.329+0000 7f9b3d3e2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b380d0ad0 0x7f9b380b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b3d3e2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b380d1df0 con 0x7f9b380b9770 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b3d3e2640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b380d20d0 con 0x7f9b380d0ad0 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b3d3e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380d23b0 con 0x7f9b3815dd30 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b36ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b377fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b380d0ad0 0x7f9b380b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b2e7fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9b380b9770 0x7f9b380d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:52686/0 (socket says 172.21.15.60:52686) 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b2e7fc640 1 -- 172.21.15.60:0/3843505879 learned_addr learned my addr 172.21.15.60:0/3843505879 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b367fc640 1 -- 172.21.15.60:0/3843505879 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482701006 0 0) 0x7f9b380d1df0 con 0x7f9b380b9770 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b367fc640 1 -- 172.21.15.60:0/3843505879 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b24003520 con 0x7f9b380b9770 2024-04-25T08:39:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b36ffd640 1 -- 172.21.15.60:0/3843505879 >> v2:172.21.15.100:3300/0 conn(0x7f9b380d0ad0 msgr2=0x7f9b380b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:11.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b36ffd640 1 --2- 172.21.15.60:0/3843505879 >> v2:172.21.15.100:3300/0 conn(0x7f9b380d0ad0 0x7f9b380b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b36ffd640 1 -- 172.21.15.60:0/3843505879 >> v1:172.21.15.100:6789/0 conn(0x7f9b380b9770 legacy=0x7f9b380d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b36ffd640 1 -- 172.21.15.60:0/3843505879 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380d2690 con 0x7f9b3815dd30 2024-04-25T08:39:11.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.330+0000 7f9b377fe640 1 --2- 172.21.15.60:0/3843505879 >> v2:172.21.15.100:3300/0 conn(0x7f9b380d0ad0 0x7f9b380b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:11.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.331+0000 7f9b36ffd640 1 --2- 172.21.15.60:0/3843505879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380b9230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9b280097a0 tx=0x7f9b2802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca33094a9095da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.331+0000 7f9b367fc640 1 -- 172.21.15.60:0/3843505879 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28037070 con 0x7f9b3815dd30 2024-04-25T08:39:11.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.331+0000 7f9b367fc640 1 -- 172.21.15.60:0/3843505879 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2802ae50 con 0x7f9b3815dd30 2024-04-25T08:39:11.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.331+0000 7f9b367fc640 1 -- 172.21.15.60:0/3843505879 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28032640 con 0x7f9b3815dd30 2024-04-25T08:39:11.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/3843505879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 msgr2=0x7f9b380b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/3843505879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380b9230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9b280097a0 tx=0x7f9b2802a100 comp rx=0 tx=0).stop 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/3843505879 shutdown_connections 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/3843505879 >> v2:172.21.15.100:3300/0 conn(0x7f9b380d0ad0 0x7f9b380b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/3843505879 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380b9230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/3843505879 >> 172.21.15.60:0/3843505879 conn(0x7f9b38061c30 msgr2=0x7f9b3806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/3843505879 shutdown_connections 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.332+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/3843505879 wait complete. 2024-04-25T08:39:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.333+0000 7f9b3d3e2640 1 Processor -- start 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.333+0000 7f9b3d3e2640 1 -- start start 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b3d3e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b380b9770 0x7f9b38155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b3d3e2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 0x7f9b38156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b3d3e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b3d3e2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380d1b90 con 0x7f9b380d0ad0 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b3d3e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380d1a10 con 0x7f9b380b9770 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b3d3e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380d2e90 con 0x7f9b3815dd30 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 0x7f9b38156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 0x7f9b38156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46078/0 (socket says 172.21.15.60:46078) 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 -- 172.21.15.60:0/865200739 learned_addr learned my addr 172.21.15.60:0/865200739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b377fe640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b36ffd640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b380b9770 0x7f9b38155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 -- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 msgr2=0x7f9b380cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 -- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b380b9770 msgr2=0x7f9b38155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b380b9770 0x7f9b38155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.334+0000 7f9b2e7fc640 1 -- 172.21.15.60:0/865200739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380cd960 con 0x7f9b380d0ad0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.335+0000 7f9b2e7fc640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 0x7f9b38156900 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c4d0 tx=0x7f9b2000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.335+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20017070 con 0x7f9b380d0ad0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.335+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20005d30 con 0x7f9b380d0ad0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.335+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20007310 con 0x7f9b380d0ad0 2024-04-25T08:39:11.330 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.335+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b380cc8c0 con 0x7f9b380d0ad0 2024-04-25T08:39:11.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.335+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b380cce80 con 0x7f9b380d0ad0 2024-04-25T08:39:11.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.336+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b38073990 con 0x7f9b380d0ad0 2024-04-25T08:39:11.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.336+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b2001a070 con 0x7f9b380d0ad0 2024-04-25T08:39:11.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.337+0000 7f9b2ffff640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b1806b4d0 0x7f9b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.337+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9b20089680 con 0x7f9b380d0ad0 2024-04-25T08:39:11.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.337+0000 7f9b36ffd640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b1806b4d0 0x7f9b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.338+0000 7f9b36ffd640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b1806b4d0 0x7f9b1806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9b28034000 tx=0x7f9b2802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.341+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b20057d60 con 0x7f9b380d0ad0 2024-04-25T08:39:11.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.526+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b380ca080 con 0x7f9b380d0ad0 2024-04-25T08:39:11.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.527+0000 7f9b2ffff640 1 -- 172.21.15.60:0/865200739 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17017 (secure 0 0 0) 0x7f9b2005be00 con 0x7f9b380d0ad0 2024-04-25T08:39:11.521 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:11.521 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":135,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:04.147601+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6836/879630460":"2024-04-26T08:39:04.147468+0000","172.21.15.100:6834/879630460":"2024-04-26T08:39:04.147468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b1806b4d0 msgr2=0x7f9b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b1806b4d0 0x7f9b1806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9b28034000 tx=0x7f9b2802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 msgr2=0x7f9b38156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 0x7f9b38156900 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c4d0 tx=0x7f9b2000c8d0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 shutdown_connections 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9b3815dd30 0x7f9b380cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b380d0ad0 0x7f9b38156900 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f9b1806b4d0 0x7f9b1806d990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 --2- 172.21.15.60:0/865200739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b380b9770 0x7f9b38155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 >> 172.21.15.60:0/865200739 conn(0x7f9b38061c30 msgr2=0x7f9b3806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 shutdown_connections 2024-04-25T08:39:11.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.529+0000 7f9b3d3e2640 1 -- 172.21.15.60:0/865200739 wait complete. 2024-04-25T08:39:11.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:38:15.506220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:38:16.201959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:11.534 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:11.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.645+0000 7f77050b3640 1 Processor -- start 2024-04-25T08:39:11.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.645+0000 7f77050b3640 1 -- start start 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f77050b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f77050b3640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7700161710 0x7f77000cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f77050b3640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f77000b9330 0x7f770006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f77050b3640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770015e000 con 0x7f77000b9330 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f77050b3640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000d1de0 con 0x7f7700161710 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f77050b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7700161c60 con 0x7f770015dc20 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f77000b9330 0x7f770006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f77000b9330 0x7f770006bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46094/0 (socket says 172.21.15.60:46094) 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 -- 172.21.15.60:0/3465285002 learned_addr learned my addr 172.21.15.60:0/3465285002 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:11.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76f7fff640 1 --2- 172.21.15.60:0/3465285002 >> v2:172.21.15.100:3300/0 conn(0x7f7700161710 0x7f77000cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76fed9d640 1 --2- 172.21.15.60:0/3465285002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 -- 172.21.15.60:0/3465285002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 msgr2=0x7f77000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:11.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 --2- 172.21.15.60:0/3465285002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77000ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 -- 172.21.15.60:0/3465285002 >> v2:172.21.15.100:3300/0 conn(0x7f7700161710 msgr2=0x7f77000cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:11.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 --2- 172.21.15.60:0/3465285002 >> v2:172.21.15.100:3300/0 conn(0x7f7700161710 0x7f77000cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.646+0000 7f76ff59e640 1 -- 172.21.15.60:0/3465285002 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700161f40 con 0x7f77000b9330 2024-04-25T08:39:11.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.647+0000 7f76ff59e640 1 --2- 172.21.15.60:0/3465285002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77000b9330 0x7f770006bf80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f770006d9a0 tx=0x7f76f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c21020865da7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.647+0000 7f76fe59c640 1 -- 172.21.15.60:0/3465285002 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8036070 con 0x7f77000b9330 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.647+0000 7f76fe59c640 1 -- 172.21.15.60:0/3465285002 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8031600 con 0x7f77000b9330 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.647+0000 7f76fe59c640 1 -- 172.21.15.60:0/3465285002 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80319d0 con 0x7f77000b9330 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 -- 172.21.15.60:0/3465285002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77000b9330 msgr2=0x7f770006bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 --2- 172.21.15.60:0/3465285002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77000b9330 0x7f770006bf80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f770006d9a0 tx=0x7f76f802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 -- 172.21.15.60:0/3465285002 shutdown_connections 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 --2- 172.21.15.60:0/3465285002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f77000b9330 0x7f770006bf80 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 --2- 172.21.15.60:0/3465285002 >> v2:172.21.15.100:3300/0 conn(0x7f7700161710 0x7f77000cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 --2- 172.21.15.60:0/3465285002 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 -- 172.21.15.60:0/3465285002 >> 172.21.15.60:0/3465285002 conn(0x7f77000d0420 msgr2=0x7f77000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 -- 172.21.15.60:0/3465285002 shutdown_connections 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.648+0000 7f77050b3640 1 -- 172.21.15.60:0/3465285002 wait complete. 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.649+0000 7f77050b3640 1 Processor -- start 2024-04-25T08:39:11.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.649+0000 7f77050b3640 1 -- start start 2024-04-25T08:39:11.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f77050b3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77000b9330 0x7f77001f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f77050b3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77001f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f77050b3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 0x7f770015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f77050b3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b5e50 con 0x7f7700161710 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f77050b3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7700162500 con 0x7f77000b9330 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f77050b3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b5fd0 con 0x7f770015dc20 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 0x7f770015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 0x7f770015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:46108/0 (socket says 172.21.15.60:46108) 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 -- 172.21.15.60:0/2409837038 learned_addr learned my addr 172.21.15.60:0/2409837038 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76fed9d640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77000b9330 0x7f77001f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76f7fff640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77001f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 -- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 msgr2=0x7f77001f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77001f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 -- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77000b9330 msgr2=0x7f77001f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77000b9330 0x7f77001f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 -- 172.21.15.60:0/2409837038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000aa380 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76fed9d640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77000b9330 0x7f77001f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76f7fff640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77001f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76ff59e640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 0x7f770015cc10 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009f40 tx=0x7f76f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.650+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8041020 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.651+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80077a0 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.651+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001f3ec0 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.651+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8036040 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.651+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77000aa620 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.652+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7700073a00 con 0x7f7700161710 2024-04-25T08:39:11.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.652+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76f8044020 con 0x7f7700161710 2024-04-25T08:39:11.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.652+0000 7f76f77fe640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f76e006b4d0 0x7f76e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.653+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f76f80aae50 con 0x7f7700161710 2024-04-25T08:39:11.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.653+0000 7f76fed9d640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f76e006b4d0 0x7f76e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.653+0000 7f76fed9d640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f76e006b4d0 0x7f76e006d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009a70 tx=0x7f76f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.657+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76f804b030 con 0x7f7700161710 2024-04-25T08:39:11.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.841+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77000d16b0 con 0x7f7700161710 2024-04-25T08:39:11.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.841+0000 7f76f77fe640 1 -- 172.21.15.60:0/2409837038 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76f8079c40 con 0x7f7700161710 2024-04-25T08:39:11.835 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:39:11.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.843+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f76e006b4d0 msgr2=0x7f76e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f76e006b4d0 0x7f76e006d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009a70 tx=0x7f76f0008040 comp rx=0 tx=0).stop 2024-04-25T08:39:11.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 msgr2=0x7f770015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 0x7f770015cc10 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009f40 tx=0x7f76f8004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 shutdown_connections 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7700161710 0x7f770015cc10 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f770015dc20 0x7f77001f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f76e006b4d0 0x7f76e006d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 --2- 172.21.15.60:0/2409837038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77000b9330 0x7f77001f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 >> 172.21.15.60:0/2409837038 conn(0x7f77000d0420 msgr2=0x7f77000d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 shutdown_connections 2024-04-25T08:39:11.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.844+0000 7f77050b3640 1 -- 172.21.15.60:0/2409837038 wait complete. 2024-04-25T08:39:11.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:39:11.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.958+0000 7fbf6d612640 1 Processor -- start 2024-04-25T08:39:11.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.958+0000 7fbf6d612640 1 -- start start 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf6d612640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbf6815da90 0x7fbf68161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf6d612640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf680b9750 0x7fbf680ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf6d612640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf68162040 con 0x7fbf68161ba0 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf6d612640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf68162600 con 0x7fbf680b9750 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf6d612640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf68162320 con 0x7fbf6815da90 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf667fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fbf680b9750 0x7fbf680ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf677fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fbf68161ba0 0x7fbf6806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50954/0 (socket says 172.21.15.60:50954) 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf677fe640 1 -- 172.21.15.60:0/1136468867 learned_addr learned my addr 172.21.15.60:0/1136468867 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf66ffd640 1 --2- 172.21.15.60:0/1136468867 >> v2:172.21.15.100:3300/0 conn(0x7fbf6815da90 0x7fbf68161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf667fc640 1 -- 172.21.15.60:0/1136468867 >> v2:172.21.15.100:3300/0 conn(0x7fbf6815da90 msgr2=0x7fbf68161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf667fc640 1 --2- 172.21.15.60:0/1136468867 >> v2:172.21.15.100:3300/0 conn(0x7fbf6815da90 0x7fbf68161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.959+0000 7fbf667fc640 1 -- 172.21.15.60:0/1136468867 >> v1:172.21.15.60:6789/0 conn(0x7fbf68161ba0 legacy=0x7fbf6806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.960+0000 7fbf65ffb640 1 -- 172.21.15.60:0/1136468867 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228674987 0 0) 0x7fbf68162040 con 0x7fbf68161ba0 2024-04-25T08:39:11.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.960+0000 7fbf667fc640 1 -- 172.21.15.60:0/1136468867 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680d1b10 con 0x7fbf680b9750 2024-04-25T08:39:11.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.960+0000 7fbf667fc640 1 --2- 172.21.15.60:0/1136468867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf680b9750 0x7fbf680ce670 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fbf680b61c0 tx=0x7fbf5002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8797b737d4cd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.960+0000 7fbf65ffb640 1 -- 172.21.15.60:0/1136468867 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50037070 con 0x7fbf680b9750 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.960+0000 7fbf65ffb640 1 -- 172.21.15.60:0/1136468867 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf50032440 con 0x7fbf680b9750 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.960+0000 7fbf65ffb640 1 -- 172.21.15.60:0/1136468867 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5002ae60 con 0x7fbf680b9750 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 -- 172.21.15.60:0/1136468867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf680b9750 msgr2=0x7fbf680ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1136468867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf680b9750 0x7fbf680ce670 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fbf680b61c0 tx=0x7fbf5002a110 comp rx=0 tx=0).stop 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 -- 172.21.15.60:0/1136468867 shutdown_connections 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1136468867 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf680b9750 0x7fbf680ce670 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1136468867 >> v2:172.21.15.100:3300/0 conn(0x7fbf6815da90 0x7fbf68161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 -- 172.21.15.60:0/1136468867 >> 172.21.15.60:0/1136468867 conn(0x7fbf68061a20 msgr2=0x7fbf68061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 -- 172.21.15.60:0/1136468867 shutdown_connections 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.961+0000 7fbf6d612640 1 -- 172.21.15.60:0/1136468867 wait complete. 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.962+0000 7fbf6d612640 1 Processor -- start 2024-04-25T08:39:11.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.962+0000 7fbf6d612640 1 -- start start 2024-04-25T08:39:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf6d612640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 0x7fbf681f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf6d612640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf68161ba0 0x7fbf681f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.958 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf6d612640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf680c81d0 0x7fbf6815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf6d612640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680d3320 con 0x7fbf6815da90 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf6d612640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6815de70 con 0x7fbf68161ba0 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf6d612640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680d34a0 con 0x7fbf680c81d0 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 0x7fbf681f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 0x7fbf681f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57000/0 (socket says 172.21.15.60:57000) 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 -- 172.21.15.60:0/1842768923 learned_addr learned my addr 172.21.15.60:0/1842768923 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 -- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf680c81d0 msgr2=0x7fbf6815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf677fe640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf680c81d0 0x7fbf6815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf667fc640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf68161ba0 0x7fbf681f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf680c81d0 0x7fbf6815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 -- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf68161ba0 msgr2=0x7fbf681f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf68161ba0 0x7fbf681f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.963+0000 7fbf66ffd640 1 -- 172.21.15.60:0/1842768923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf681f67e0 con 0x7fbf6815da90 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf677fe640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf680c81d0 0x7fbf6815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:11.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf667fc640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf68161ba0 0x7fbf681f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf66ffd640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 0x7fbf681f3470 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e9e0 tx=0x7fbf6000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60017070 con 0x7fbf6815da90 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6000c9d0 con 0x7fbf6815da90 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6815f410 con 0x7fbf6815da90 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000cdc0 con 0x7fbf6815da90 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.964+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf680c88c0 con 0x7fbf6815da90 2024-04-25T08:39:11.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.965+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf68073a00 con 0x7fbf6815da90 2024-04-25T08:39:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.966+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf6001a070 con 0x7fbf6815da90 2024-04-25T08:39:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.966+0000 7fbf5f7fe640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b4d0 0x7fbf4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.966+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbf6008a400 con 0x7fbf6815da90 2024-04-25T08:39:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.966+0000 7fbf667fc640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b4d0 0x7fbf4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.967+0000 7fbf667fc640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b4d0 0x7fbf4806d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbf50034000 tx=0x7fbf5002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:11.970+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf600590b0 con 0x7fbf6815da90 2024-04-25T08:39:12.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.150+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf6815c6e0 con 0x7fbf6815da90 2024-04-25T08:39:12.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.151+0000 7fbf5f7fe640 1 -- 172.21.15.60:0/1842768923 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf600589b0 con 0x7fbf6815da90 2024-04-25T08:39:12.144 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:39:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b4d0 msgr2=0x7fbf4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b4d0 0x7fbf4806d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbf50034000 tx=0x7fbf5002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 msgr2=0x7fbf681f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 0x7fbf681f3470 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e9e0 tx=0x7fbf6000eea0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 shutdown_connections 2024-04-25T08:39:12.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbf680c81d0 0x7fbf6815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbf4806b4d0 0x7fbf4806d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbf68161ba0 0x7fbf681f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.152+0000 7fbf6d612640 1 --2- 172.21.15.60:0/1842768923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbf6815da90 0x7fbf681f3470 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.153+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 >> 172.21.15.60:0/1842768923 conn(0x7fbf68061a20 msgr2=0x7fbf680b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.153+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 shutdown_connections 2024-04-25T08:39:12.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.153+0000 7fbf6d612640 1 -- 172.21.15.60:0/1842768923 wait complete. 2024-04-25T08:39:12.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:39:12.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.267+0000 7fd5dacb3640 1 Processor -- start 2024-04-25T08:39:12.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5dacb3640 1 -- start start 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 0x7fd5d40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5dacb3640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d406cda0 con 0x7fd5d4161710 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5dacb3640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d415e000 con 0x7fd5d40d32f0 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40d1de0 con 0x7fd5d415dc20 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5d3fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd5d4161710 0x7fd5d406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50958/0 (socket says 172.21.15.60:50958) 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.268+0000 7fd5d3fff640 1 -- 172.21.15.60:0/10450397 learned_addr learned my addr 172.21.15.60:0/10450397 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 0x7fd5d40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834751359 0 0) 0x7fd5d406cda0 con 0x7fd5d4161710 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5c4003520 con 0x7fd5d4161710 2024-04-25T08:39:12.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597895810 0 0) 0x7fd5d415e000 con 0x7fd5d40d32f0 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d406cda0 con 0x7fd5d40d32f0 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1083083060 0 0) 0x7fd5c4003520 con 0x7fd5d4161710 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 msgr2=0x7fd5d40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 0x7fd5d40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 >> v1:172.21.15.100:6789/0 conn(0x7fd5d40d32f0 legacy=0x7fd5d406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4161c60 con 0x7fd5d4161710 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fd5c0001670 con 0x7fd5d4161710 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 >> v1:172.21.15.60:6789/0 conn(0x7fd5d4161710 legacy=0x7fd5d406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5c4003b70 0x7fd5c4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 0x7fd5c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4004d40 0x7fd5c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40d1de0 con 0x7fd5c4003b70 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c400ba70 con 0x7fd5c4004d40 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c400bc20 con 0x7fd5d415dc20 2024-04-25T08:39:12.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5c4003b70 0x7fd5c4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d3fff640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 0x7fd5c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.269+0000 7fd5d9250640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4004d40 0x7fd5c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d8a4f640 1 -- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 msgr2=0x7fd5c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5d415dc20 0x7fd5c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d8a4f640 1 -- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4004d40 msgr2=0x7fd5c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4004d40 0x7fd5c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d8a4f640 1 -- 172.21.15.60:0/10450397 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4003520 con 0x7fd5c4003b70 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5c4003b70 0x7fd5c4003f70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009ac0 tx=0x7fd5c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19678a6853c1d76b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8036070 con 0x7fd5c4003b70 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c8031400 con 0x7fd5c4003b70 2024-04-25T08:39:12.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.270+0000 7fd5d37fe640 1 -- 172.21.15.60:0/10450397 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8009690 con 0x7fd5c4003b70 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5dacb3640 1 -- 172.21.15.60:0/10450397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5c4003b70 msgr2=0x7fd5c4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/10450397 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5c4003b70 0x7fd5c4003f70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009ac0 tx=0x7fd5c4005830 comp rx=0 tx=0).stop 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5d9250640 1 -- 172.21.15.60:0/10450397 reap_dead start 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5dacb3640 1 -- 172.21.15.60:0/10450397 shutdown_connections 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5dacb3640 1 -- 172.21.15.60:0/10450397 >> 172.21.15.60:0/10450397 conn(0x7fd5d40d0420 msgr2=0x7fd5d40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5dacb3640 1 -- 172.21.15.60:0/10450397 shutdown_connections 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.271+0000 7fd5dacb3640 1 -- 172.21.15.60:0/10450397 wait complete. 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.272+0000 7fd5dacb3640 1 Processor -- start 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.272+0000 7fd5dacb3640 1 -- start start 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5c4003b70 0x7fd5d40cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cc310 0x7fd5d414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5dacb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 0x7fd5d41f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4162500 con 0x7fd5d414cb20 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4162220 con 0x7fd5d40cc310 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5dacb3640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40b5e50 con 0x7fd5c4003b70 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5d9250640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 0x7fd5d41f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5d9250640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 0x7fd5d41f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57022/0 (socket says 172.21.15.60:57022) 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5d9250640 1 -- 172.21.15.60:0/3847867885 learned_addr learned my addr 172.21.15.60:0/3847867885 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5c4003b70 0x7fd5d40cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.273+0000 7fd5d3fff640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cc310 0x7fd5d414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d9250640 1 -- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5c4003b70 msgr2=0x7fd5d40cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d9250640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5c4003b70 0x7fd5d40cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d9250640 1 -- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cc310 msgr2=0x7fd5d414c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d9250640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cc310 0x7fd5d414c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d9250640 1 -- 172.21.15.60:0/3847867885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d40cad30 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d9250640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 0x7fd5d41f3460 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40d3780 tx=0x7fd5cc00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc019070 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc006d60 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc003b90 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d40cb010 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.274+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d40cb5d0 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.275+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d4073a00 con 0x7fd5d414cb20 2024-04-25T08:39:12.271 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.275+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5cc0050d0 con 0x7fd5d414cb20 2024-04-25T08:39:12.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.276+0000 7fd5d17fa640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b806b500 0x7fd5b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.276+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd5cc089780 con 0x7fd5d414cb20 2024-04-25T08:39:12.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.276+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b806b500 0x7fd5b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.277+0000 7fd5d8a4f640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b806b500 0x7fd5b806d9c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8033000 tx=0x7fd5c8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.281+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5cc057d30 con 0x7fd5d414cb20 2024-04-25T08:39:12.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.459+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5d40d16b0 con 0x7fd5d414cb20 2024-04-25T08:39:12.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.460+0000 7fd5d17fa640 1 -- 172.21.15.60:0/3847867885 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v135) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5cc05bdd0 con 0x7fd5d414cb20 2024-04-25T08:39:12.453 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:39:12.455 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.461+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b806b500 msgr2=0x7fd5b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b806b500 0x7fd5b806d9c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8033000 tx=0x7fd5c8008030 comp rx=0 tx=0).stop 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 msgr2=0x7fd5d41f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 0x7fd5d41f3460 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd5d40d3780 tx=0x7fd5cc00fd40 comp rx=0 tx=0).stop 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 shutdown_connections 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd5d414cb20 0x7fd5d41f3460 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd5b806b500 0x7fd5b806d9c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d40cc310 0x7fd5d414c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 --2- 172.21.15.60:0/3847867885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd5c4003b70 0x7fd5d40cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 >> 172.21.15.60:0/3847867885 conn(0x7fd5d40d0420 msgr2=0x7fd5d40c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 shutdown_connections 2024-04-25T08:39:12.456 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.462+0000 7fd5dacb3640 1 -- 172.21.15.60:0/3847867885 wait complete. 2024-04-25T08:39:12.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:12.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.573+0000 7fd64b04c640 1 Processor -- start 2024-04-25T08:39:12.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.573+0000 7fd64b04c640 1 -- start start 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd64b04c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6440d1020 0x7fd6440d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd64b04c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6440b97a0 con 0x7fd64415dbe0 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd64b04c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6440d1c50 con 0x7fd6440b2cb0 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd64b04c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64415dfc0 con 0x7fd6440d1020 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd6495e9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd6440d1020 0x7fd6440d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd648de8640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd64415dbe0 0x7fd6440d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50966/0 (socket says 172.21.15.60:50966) 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.574+0000 7fd648de8640 1 -- 172.21.15.60:0/2509655009 learned_addr learned my addr 172.21.15.60:0/2509655009 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6495e9640 1 -- 172.21.15.60:0/2509655009 >> v1:172.21.15.100:6789/0 conn(0x7fd6440b2cb0 legacy=0x7fd6440d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6495e9640 1 -- 172.21.15.60:0/2509655009 >> v1:172.21.15.60:6789/0 conn(0x7fd64415dbe0 legacy=0x7fd6440d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6495e9640 1 -- 172.21.15.60:0/2509655009 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440d1f30 con 0x7fd6440d1020 2024-04-25T08:39:12.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6437fe640 1 -- 172.21.15.60:0/2509655009 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21982775 0 0) 0x7fd6440b97a0 con 0x7fd64415dbe0 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6495e9640 1 --2- 172.21.15.60:0/2509655009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440d1020 0x7fd6440d2e40 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd6340097a0 tx=0x7fd63402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad1841bd72cde08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6437fe640 1 -- 172.21.15.60:0/2509655009 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634037070 con 0x7fd6440d1020 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6437fe640 1 -- 172.21.15.60:0/2509655009 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634032440 con 0x7fd6440d1020 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.575+0000 7fd6437fe640 1 -- 172.21.15.60:0/2509655009 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634032810 con 0x7fd6440d1020 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 -- 172.21.15.60:0/2509655009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440d1020 msgr2=0x7fd6440d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 --2- 172.21.15.60:0/2509655009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440d1020 0x7fd6440d2e40 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd6340097a0 tx=0x7fd63402a230 comp rx=0 tx=0).stop 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 -- 172.21.15.60:0/2509655009 shutdown_connections 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 --2- 172.21.15.60:0/2509655009 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd6440d1020 0x7fd6440d2e40 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 -- 172.21.15.60:0/2509655009 >> 172.21.15.60:0/2509655009 conn(0x7fd644061a20 msgr2=0x7fd644061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 -- 172.21.15.60:0/2509655009 shutdown_connections 2024-04-25T08:39:12.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.576+0000 7fd64b04c640 1 -- 172.21.15.60:0/2509655009 wait complete. 2024-04-25T08:39:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.577+0000 7fd64b04c640 1 Processor -- start 2024-04-25T08:39:12.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.577+0000 7fd64b04c640 1 -- start start 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd64b04c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6440b2cb0 0x7fd6441f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd64b04c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440d1020 0x7fd6440c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd64b04c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 0x7fd6440b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd64b04c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440b92a0 con 0x7fd64415dbe0 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd64b04c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440d1a00 con 0x7fd6440d1020 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd64b04c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440b9420 con 0x7fd6440b2cb0 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd6495e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 0x7fd6440b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd6495e9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 0x7fd6440b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57040/0 (socket says 172.21.15.60:57040) 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd6495e9640 1 -- 172.21.15.60:0/1590007173 learned_addr learned my addr 172.21.15.60:0/1590007173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd643fff640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440d1020 0x7fd6440c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.573 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.578+0000 7fd648de8640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6440b2cb0 0x7fd6441f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6495e9640 1 -- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6440b2cb0 msgr2=0x7fd6441f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6495e9640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6440b2cb0 0x7fd6441f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6495e9640 1 -- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440d1020 msgr2=0x7fd6440c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6495e9640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440d1020 0x7fd6440c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6495e9640 1 -- 172.21.15.60:0/1590007173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440cd4b0 con 0x7fd64415dbe0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd643fff640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440d1020 0x7fd6440c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6495e9640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 0x7fd6440b4b10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd6340098c0 tx=0x7fd6340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634037040 con 0x7fd64415dbe0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634042070 con 0x7fd64415dbe0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6440cd790 con 0x7fd64415dbe0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634030b30 con 0x7fd64415dbe0 2024-04-25T08:39:12.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.579+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6440b3610 con 0x7fd64415dbe0 2024-04-25T08:39:12.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.580+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644073990 con 0x7fd64415dbe0 2024-04-25T08:39:12.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.580+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd634030cd0 con 0x7fd64415dbe0 2024-04-25T08:39:12.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.581+0000 7fd6417fa640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd63006b500 0x7fd63006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.581+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd6340aaf80 con 0x7fd64415dbe0 2024-04-25T08:39:12.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.581+0000 7fd648de8640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd63006b500 0x7fd63006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.581+0000 7fd648de8640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd63006b500 0x7fd63006d9c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0012e0 tx=0x7fd63c005bf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.578 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.585+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd634045360 con 0x7fd64415dbe0 2024-04-25T08:39:12.794 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.801+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd644062180 con 0x7fd64415dbe0 2024-04-25T08:39:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.801+0000 7fd6417fa640 1 -- 172.21.15.60:0/1590007173 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4152 (secure 0 0 0) 0x7fd634079cb0 con 0x7fd64415dbe0 2024-04-25T08:39:12.795 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:12.795 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:39:06.172452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:12.795 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T08:39:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.803+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd63006b500 msgr2=0x7fd63006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.803+0000 7fd64b04c640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd63006b500 0x7fd63006d9c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0012e0 tx=0x7fd63c005bf0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.803+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 msgr2=0x7fd6440b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.803+0000 7fd64b04c640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 0x7fd6440b4b10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd6340098c0 tx=0x7fd6340046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.803+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 shutdown_connections 2024-04-25T08:39:12.797 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd64415dbe0 0x7fd6440b4b10 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd63006b500 0x7fd63006d9c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6440d1020 0x7fd6440c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 --2- 172.21.15.60:0/1590007173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6440b2cb0 0x7fd6441f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 >> 172.21.15.60:0/1590007173 conn(0x7fd644061a20 msgr2=0x7fd6441590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 shutdown_connections 2024-04-25T08:39:12.798 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.804+0000 7fd64b04c640 1 -- 172.21.15.60:0/1590007173 wait complete. 2024-04-25T08:39:12.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.943+0000 7f5923563640 1 Processor -- start 2024-04-25T08:39:12.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.943+0000 7f5923563640 1 -- start start 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f5923563640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f591c06abe0 0x7f591c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f5923563640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c15c990 0x7f591c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f5923563640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c06b500 con 0x7f591c0cc1d0 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f5923563640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0ce6e0 con 0x7f591c06abe0 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f5923563640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0cfe10 con 0x7f591c15c990 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f59212ff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f591c06abe0 0x7f591c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f59212ff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f591c06abe0 0x7f591c0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57042/0 (socket says 172.21.15.60:57042) 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.944+0000 7f59212ff640 1 -- 172.21.15.60:0/4130298946 learned_addr learned my addr 172.21.15.60:0/4130298946 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.945+0000 7f5920afe640 1 --2- 172.21.15.60:0/4130298946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c15c990 0x7f591c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.945+0000 7f59212ff640 1 -- 172.21.15.60:0/4130298946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c15c990 msgr2=0x7f591c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.945+0000 7f59212ff640 1 --2- 172.21.15.60:0/4130298946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c15c990 0x7f591c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.945+0000 7f59212ff640 1 -- 172.21.15.60:0/4130298946 >> v1:172.21.15.100:6789/0 conn(0x7f591c0cc1d0 legacy=0x7f591c0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.945+0000 7f59212ff640 1 -- 172.21.15.60:0/4130298946 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0d00f0 con 0x7f591c06abe0 2024-04-25T08:39:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.945+0000 7f59212ff640 1 --2- 172.21.15.60:0/4130298946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c06abe0 0x7f591c0cf780 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5918009870 tx=0x7f591802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7376734ffd4103ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5913fff640 1 -- 172.21.15.60:0/4130298946 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918037070 con 0x7f591c06abe0 2024-04-25T08:39:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5913fff640 1 -- 172.21.15.60:0/4130298946 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918032440 con 0x7f591c06abe0 2024-04-25T08:39:12.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5913fff640 1 -- 172.21.15.60:0/4130298946 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918032810 con 0x7f591c06abe0 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 -- 172.21.15.60:0/4130298946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c06abe0 msgr2=0x7f591c0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 --2- 172.21.15.60:0/4130298946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c06abe0 0x7f591c0cf780 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5918009870 tx=0x7f591802a210 comp rx=0 tx=0).stop 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 -- 172.21.15.60:0/4130298946 shutdown_connections 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 --2- 172.21.15.60:0/4130298946 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c15c990 0x7f591c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 --2- 172.21.15.60:0/4130298946 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c06abe0 0x7f591c0cf780 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 -- 172.21.15.60:0/4130298946 >> 172.21.15.60:0/4130298946 conn(0x7f591c0b2290 msgr2=0x7f591c0b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 -- 172.21.15.60:0/4130298946 shutdown_connections 2024-04-25T08:39:12.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.946+0000 7f5923563640 1 -- 172.21.15.60:0/4130298946 wait complete. 2024-04-25T08:39:12.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.947+0000 7f5923563640 1 Processor -- start 2024-04-25T08:39:12.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.947+0000 7f5923563640 1 -- start start 2024-04-25T08:39:12.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5923563640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c06abe0 0x7f591c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5923563640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c0cc1d0 0x7f591c15a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5923563640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 0x7f591c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5923563640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c15cdd0 con 0x7f591c15c990 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5923563640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0ce370 con 0x7f591c0cc1d0 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5923563640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0cf280 con 0x7f591c06abe0 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5921b00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 0x7f591c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5921b00640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 0x7f591c06f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57052/0 (socket says 172.21.15.60:57052) 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5921b00640 1 -- 172.21.15.60:0/3885428355 learned_addr learned my addr 172.21.15.60:0/3885428355 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f5920afe640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c0cc1d0 0x7f591c15a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.948+0000 7f59212ff640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c06abe0 0x7f591c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5921b00640 1 -- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c06abe0 msgr2=0x7f591c158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5921b00640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c06abe0 0x7f591c158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5921b00640 1 -- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c0cc1d0 msgr2=0x7f591c15a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5921b00640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c0cc1d0 0x7f591c15a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5921b00640 1 -- 172.21.15.60:0/3885428355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0b4b90 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5920afe640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c0cc1d0 0x7f591c15a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5921b00640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 0x7f591c06f490 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f591400c7b0 tx=0x7f591400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914004b60 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914004d00 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400aa90 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c0b4e70 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.949+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c0b5430 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.950+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c073990 con 0x7f591c15c990 2024-04-25T08:39:12.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.950+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f591400ac30 con 0x7f591c15c990 2024-04-25T08:39:12.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.951+0000 7f5911ffb640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f590006b4d0 0x7f590006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.951+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f591408a620 con 0x7f591c15c990 2024-04-25T08:39:12.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.951+0000 7f59212ff640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f590006b4d0 0x7f590006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.952+0000 7f59212ff640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f590006b4d0 0x7f590006d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5918034000 tx=0x7f591802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:12.955+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5914020070 con 0x7f591c15c990 2024-04-25T08:39:13.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.136+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f591c0b65c0 con 0x7f591c15c990 2024-04-25T08:39:13.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.137+0000 7f5911ffb640 1 -- 172.21.15.60:0/3885428355 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17017 (secure 0 0 0) 0x7f5914059350 con 0x7f591c15c990 2024-04-25T08:39:13.130 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:13.131 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":135,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:04.147601+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6836/879630460":"2024-04-26T08:39:04.147468+0000","172.21.15.100:6834/879630460":"2024-04-26T08:39:04.147468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.138+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f590006b4d0 msgr2=0x7f590006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.138+0000 7f5923563640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f590006b4d0 0x7f590006d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5918034000 tx=0x7f591802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:13.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 msgr2=0x7f591c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 0x7f591c06f490 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f591400c7b0 tx=0x7f591400cc70 comp rx=0 tx=0).stop 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 shutdown_connections 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f591c15c990 0x7f591c06f490 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f590006b4d0 0x7f590006d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f591c0cc1d0 0x7f591c15a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 --2- 172.21.15.60:0/3885428355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f591c06abe0 0x7f591c158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 >> 172.21.15.60:0/3885428355 conn(0x7f591c0b2290 msgr2=0x7f591c15cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 shutdown_connections 2024-04-25T08:39:13.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.139+0000 7f5923563640 1 -- 172.21.15.60:0/3885428355 wait complete. 2024-04-25T08:39:13.144 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:39:13.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:13.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.251+0000 7fe6ca3c1640 1 Processor -- start 2024-04-25T08:39:13.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.251+0000 7fe6ca3c1640 1 -- start start 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.252+0000 7fe6ca3c1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 0x7fe6c40b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.252+0000 7fe6ca3c1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.252+0000 7fe6ca3c1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c415eac0 con 0x7fe6c415e100 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.252+0000 7fe6ca3c1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40d1c00 con 0x7fe6c415dd20 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.252+0000 7fe6ca3c1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415eda0 con 0x7fe6c40d11b0 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 0x7fe6c40b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c895e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 0x7fe6c40b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44708/0 (socket says 172.21.15.60:44708) 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c3fff640 1 -- 172.21.15.60:0/3895266481 learned_addr learned my addr 172.21.15.60:0/3895266481 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c895e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:39830/0 (socket says 172.21.15.60:39830) 2024-04-25T08:39:13.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c37fe640 1 --1- 172.21.15.60:0/3895266481 >> v1:172.21.15.100:6789/0 conn(0x7fe6c415e100 0x7fe6c40d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:32956/0 (socket says 172.21.15.60:32956) 2024-04-25T08:39:13.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c2ffd640 1 -- 172.21.15.60:0/3895266481 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635423279 0 0) 0x7fe6c415eac0 con 0x7fe6c415e100 2024-04-25T08:39:13.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c2ffd640 1 -- 172.21.15.60:0/3895266481 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b0003520 con 0x7fe6c415e100 2024-04-25T08:39:13.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c895e640 1 -- 172.21.15.60:0/3895266481 >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 msgr2=0x7fe6c40b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:13.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c895e640 1 --2- 172.21.15.60:0/3895266481 >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 0x7fe6c40b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c895e640 1 -- 172.21.15.60:0/3895266481 >> v1:172.21.15.100:6789/0 conn(0x7fe6c415e100 legacy=0x7fe6c40d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c895e640 1 -- 172.21.15.60:0/3895266481 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c415f080 con 0x7fe6c40d11b0 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.253+0000 7fe6c3fff640 1 --2- 172.21.15.60:0/3895266481 >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 0x7fe6c40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.254+0000 7fe6c895e640 1 --2- 172.21.15.60:0/3895266481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40d2370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009d90 tx=0x7fe6bc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d3a85b345895b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.254+0000 7fe6c2ffd640 1 -- 172.21.15.60:0/3895266481 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc031600 con 0x7fe6c40d11b0 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.254+0000 7fe6c2ffd640 1 -- 172.21.15.60:0/3895266481 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0317a0 con 0x7fe6c40d11b0 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.254+0000 7fe6c2ffd640 1 -- 172.21.15.60:0/3895266481 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc031b50 con 0x7fe6c40d11b0 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.254+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/3895266481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 msgr2=0x7fe6c40d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.254+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/3895266481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40d2370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009d90 tx=0x7fe6bc02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.255+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/3895266481 shutdown_connections 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.255+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/3895266481 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40d2370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.255+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/3895266481 >> v2:172.21.15.100:3300/0 conn(0x7fe6c415dd20 0x7fe6c40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.255+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/3895266481 >> 172.21.15.60:0/3895266481 conn(0x7fe6c4061a20 msgr2=0x7fe6c4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.255+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/3895266481 shutdown_connections 2024-04-25T08:39:13.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.255+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/3895266481 wait complete. 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.256+0000 7fe6ca3c1640 1 Processor -- start 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.256+0000 7fe6ca3c1640 1 -- start start 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6ca3c1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6ca3c1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c415dd20 0x7fe6c40c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6ca3c1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 0x7fe6c40c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6ca3c1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415e710 con 0x7fe6c415e100 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6ca3c1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415e590 con 0x7fe6c415dd20 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6ca3c1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40c9fd0 con 0x7fe6c40d11b0 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 0x7fe6c40c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 0x7fe6c40c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57064/0 (socket says 172.21.15.60:57064) 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 -- 172.21.15.60:0/1164615065 learned_addr learned my addr 172.21.15.60:0/1164615065 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:13.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c3fff640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c37fe640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c415dd20 0x7fe6c40c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 -- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 msgr2=0x7fe6c40aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 -- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c415dd20 msgr2=0x7fe6c40c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c415dd20 0x7fe6c40c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 -- 172.21.15.60:0/1164615065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40c9cf0 con 0x7fe6c415e100 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c37fe640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c415dd20 0x7fe6c40c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.257+0000 7fe6c895e640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 0x7fe6c40c95e0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009eb0 tx=0x7fe6bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.258+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc040070 con 0x7fe6c415e100 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.258+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc004260 con 0x7fe6c415e100 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.258+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c41508c0 con 0x7fe6c415e100 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.258+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc009450 con 0x7fe6c415e100 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.258+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c4150e80 con 0x7fe6c415e100 2024-04-25T08:39:13.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.259+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c4151080 con 0x7fe6c415e100 2024-04-25T08:39:13.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.259+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6bc039040 con 0x7fe6c415e100 2024-04-25T08:39:13.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.259+0000 7fe6c0ff9640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.260+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe6bc0ab3b0 con 0x7fe6c415e100 2024-04-25T08:39:13.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.260+0000 7fe6c3fff640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.260+0000 7fe6c3fff640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe6a406b4d0 0x7fe6a406d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac006ca0 tx=0x7fe6ac009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.264+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6c4151080 con 0x7fe6c415e100 2024-04-25T08:39:13.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.481+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6c4073990 con 0x7fe6c415e100 2024-04-25T08:39:13.475 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.482+0000 7fe6c0ff9640 1 -- 172.21.15.60:0/1164615065 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4152 (secure 0 0 0) 0x7fe6bc079b50 con 0x7fe6c415e100 2024-04-25T08:39:13.476 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:13.476 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:39:06.172452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:13.476 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe6a406b4d0 msgr2=0x7fe6a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe6a406b4d0 0x7fe6a406d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac006ca0 tx=0x7fe6ac009240 comp rx=0 tx=0).stop 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 msgr2=0x7fe6c40c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 0x7fe6c40c95e0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009eb0 tx=0x7fe6bc004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 shutdown_connections 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6c415e100 0x7fe6c40c95e0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe6c415dd20 0x7fe6c40c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 --2- 172.21.15.60:0/1164615065 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe6c40d11b0 0x7fe6c40aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 >> 172.21.15.60:0/1164615065 conn(0x7fe6c4061a20 msgr2=0x7fe6c40b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:13.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 shutdown_connections 2024-04-25T08:39:13.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.484+0000 7fe6ca3c1640 1 -- 172.21.15.60:0/1164615065 wait complete. 2024-04-25T08:39:13.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:13.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.596+0000 7f726e658640 1 Processor -- start 2024-04-25T08:39:13.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.596+0000 7f726e658640 1 -- start start 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726e658640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f726815dd30 0x7f72680b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726e658640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72680d0ad0 0x7f72680b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726e658640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680d1df0 con 0x7f72680b9770 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726e658640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680d20d0 con 0x7f72680d0ad0 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726e658640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680d23b0 con 0x7f726815dd30 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72680d0ad0 0x7f72680b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f72680d0ad0 0x7f72680b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57066/0 (socket says 172.21.15.60:57066) 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 -- 172.21.15.60:0/3791493942 learned_addr learned my addr 172.21.15.60:0/3791493942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f7267fff640 1 --2- 172.21.15.60:0/3791493942 >> v2:172.21.15.100:3300/0 conn(0x7f726815dd30 0x7f72680b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 -- 172.21.15.60:0/3791493942 >> v2:172.21.15.100:3300/0 conn(0x7f726815dd30 msgr2=0x7f72680b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 --2- 172.21.15.60:0/3791493942 >> v2:172.21.15.100:3300/0 conn(0x7f726815dd30 0x7f72680b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 -- 172.21.15.60:0/3791493942 >> v1:172.21.15.100:6789/0 conn(0x7f72680b9770 legacy=0x7f72680d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.591 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.597+0000 7f726cbf5640 1 -- 172.21.15.60:0/3791493942 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680d2690 con 0x7f72680d0ad0 2024-04-25T08:39:13.592 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.598+0000 7f726cbf5640 1 --2- 172.21.15.60:0/3791493942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680d0ad0 0x7f72680b6a40 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f726806d000 tx=0x7f726002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cfed10e86beca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.598+0000 7f7266ffd640 1 -- 172.21.15.60:0/3791493942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260036070 con 0x7f72680d0ad0 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.598+0000 7f7266ffd640 1 -- 172.21.15.60:0/3791493942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260031600 con 0x7f72680d0ad0 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.598+0000 7f7266ffd640 1 -- 172.21.15.60:0/3791493942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600319d0 con 0x7f72680d0ad0 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 -- 172.21.15.60:0/3791493942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680d0ad0 msgr2=0x7f72680b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 --2- 172.21.15.60:0/3791493942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680d0ad0 0x7f72680b6a40 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f726806d000 tx=0x7f726002a610 comp rx=0 tx=0).stop 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 -- 172.21.15.60:0/3791493942 shutdown_connections 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 --2- 172.21.15.60:0/3791493942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680d0ad0 0x7f72680b6a40 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 --2- 172.21.15.60:0/3791493942 >> v2:172.21.15.100:3300/0 conn(0x7f726815dd30 0x7f72680b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 -- 172.21.15.60:0/3791493942 >> 172.21.15.60:0/3791493942 conn(0x7f7268061c30 msgr2=0x7f726806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 -- 172.21.15.60:0/3791493942 shutdown_connections 2024-04-25T08:39:13.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.599+0000 7f726e658640 1 -- 172.21.15.60:0/3791493942 wait complete. 2024-04-25T08:39:13.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.600+0000 7f726e658640 1 Processor -- start 2024-04-25T08:39:13.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.600+0000 7f726e658640 1 -- start start 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726e658640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 0x7f7268159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726e658640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72680d0ad0 0x7f72680b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726e658640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f726815dd30 0x7f72681f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726e658640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680d1b90 con 0x7f72680b9770 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726e658640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680d1a10 con 0x7f726815dd30 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726e658640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72680d2e90 con 0x7f72680d0ad0 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 0x7f7268159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 0x7f7268159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57076/0 (socket says 172.21.15.60:57076) 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 -- 172.21.15.60:0/9840178 learned_addr learned my addr 172.21.15.60:0/9840178 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726cbf5640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f726815dd30 0x7f72681f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f72677fe640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72680d0ad0 0x7f72680b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 -- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72680d0ad0 msgr2=0x7f72680b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72680d0ad0 0x7f72680b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 -- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f726815dd30 msgr2=0x7f72681f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f726815dd30 0x7f72681f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 -- 172.21.15.60:0/9840178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726815a460 con 0x7f72680b9770 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f726cbf5640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f726815dd30 0x7f72681f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7267fff640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 0x7f7268159f20 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f725800e8a0 tx=0x7f725800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.601+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258017070 con 0x7f72680b9770 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.602+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725800c940 con 0x7f72680b9770 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.602+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680cac40 con 0x7f72680b9770 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.602+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800cd30 con 0x7f72680b9770 2024-04-25T08:39:13.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.602+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72680cb1e0 con 0x7f72680b9770 2024-04-25T08:39:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.603+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7268067df0 con 0x7f72680b9770 2024-04-25T08:39:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.603+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f725801a070 con 0x7f72680b9770 2024-04-25T08:39:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.603+0000 7f7264ff9640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f725406b4d0 0x7f725406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.603+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f725808a270 con 0x7f72680b9770 2024-04-25T08:39:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.604+0000 7f72677fe640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f725406b4d0 0x7f725406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.604+0000 7f72677fe640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f725406b4d0 0x7f725406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f72681f6dd0 tx=0x7f725c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.608+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7268067df0 con 0x7f72680b9770 2024-04-25T08:39:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.825+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7268073990 con 0x7f72680b9770 2024-04-25T08:39:13.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.825+0000 7f7264ff9640 1 -- 172.21.15.60:0/9840178 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f726815a460 con 0x7f72680b9770 2024-04-25T08:39:13.819 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:13.819 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":82,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:17.221740+0000","modified":"2024-04-25T08:39:06.172452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9375":{"gid":9375,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":16,"addr":"172.21.15.60:6834/313583008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":313583008},{"type":"v1","addr":"172.21.15.60:6834","nonce":313583008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:39:13.819 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 82 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f725406b4d0 msgr2=0x7f725406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f725406b4d0 0x7f725406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f72681f6dd0 tx=0x7f725c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 msgr2=0x7f7268159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 0x7f7268159f20 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f725800e8a0 tx=0x7f725800ed60 comp rx=0 tx=0).stop 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 shutdown_connections 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f725406b4d0 0x7f725406d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f726815dd30 0x7f72681f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72680d0ad0 0x7f72680b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 --2- 172.21.15.60:0/9840178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f72680b9770 0x7f7268159f20 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 >> 172.21.15.60:0/9840178 conn(0x7f7268061c30 msgr2=0x7f72680ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 shutdown_connections 2024-04-25T08:39:13.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.827+0000 7f726e658640 1 -- 172.21.15.60:0/9840178 wait complete. 2024-04-25T08:39:13.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.941+0000 7f3bcc5ce640 1 Processor -- start 2024-04-25T08:39:13.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.941+0000 7f3bcc5ce640 1 -- start start 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcc5ce640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3bc415dd30 0x7f3bc40b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcc5ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3bc40d0ad0 0x7f3bc40b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcc5ce640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc40d1df0 con 0x7f3bc40b9770 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcc5ce640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc40d20d0 con 0x7f3bc415dd30 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcc5ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc40d23b0 con 0x7f3bc40d0ad0 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3bc415dd30 0x7f3bc40b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3bc415dd30 0x7f3bc40b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57092/0 (socket says 172.21.15.60:57092) 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 -- 172.21.15.60:0/507126430 learned_addr learned my addr 172.21.15.60:0/507126430 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 -- 172.21.15.60:0/507126430 >> v2:172.21.15.100:3300/0 conn(0x7f3bc40d0ad0 msgr2=0x7f3bc40b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcab6b640 1 --2- 172.21.15.60:0/507126430 >> v2:172.21.15.100:3300/0 conn(0x7f3bc40d0ad0 0x7f3bc40b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 --2- 172.21.15.60:0/507126430 >> v2:172.21.15.100:3300/0 conn(0x7f3bc40d0ad0 0x7f3bc40b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 -- 172.21.15.60:0/507126430 >> v1:172.21.15.100:6789/0 conn(0x7f3bc40b9770 legacy=0x7f3bc40d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bca36a640 1 -- 172.21.15.60:0/507126430 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc40d2690 con 0x7f3bc415dd30 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.942+0000 7f3bcab6b640 1 --2- 172.21.15.60:0/507126430 >> v2:172.21.15.100:3300/0 conn(0x7f3bc40d0ad0 0x7f3bc40b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.943+0000 7f3bca36a640 1 --2- 172.21.15.60:0/507126430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40b9230 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40ce200 tx=0x7f3bc002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3b79148e273d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.943+0000 7f3bc9368640 1 -- 172.21.15.60:0/507126430 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0037070 con 0x7f3bc415dd30 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.943+0000 7f3bc9368640 1 -- 172.21.15.60:0/507126430 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc002ae10 con 0x7f3bc415dd30 2024-04-25T08:39:13.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.943+0000 7f3bc9368640 1 -- 172.21.15.60:0/507126430 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0032660 con 0x7f3bc415dd30 2024-04-25T08:39:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.943+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/507126430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 msgr2=0x7f3bc40b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.943+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/507126430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40b9230 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40ce200 tx=0x7f3bc002a100 comp rx=0 tx=0).stop 2024-04-25T08:39:13.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.944+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/507126430 shutdown_connections 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.944+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/507126430 >> v2:172.21.15.100:3300/0 conn(0x7f3bc40d0ad0 0x7f3bc40b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.944+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/507126430 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40b9230 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.944+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/507126430 >> 172.21.15.60:0/507126430 conn(0x7f3bc4061c30 msgr2=0x7f3bc406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.944+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/507126430 shutdown_connections 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.944+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/507126430 wait complete. 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 Processor -- start 2024-04-25T08:39:13.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 -- start start 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc40b9770 0x7f3bc41f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc40d0ad0 0x7f3bc4155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40d1b90 con 0x7f3bc415dd30 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40d1a10 con 0x7f3bc40b9770 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.945+0000 7f3bcc5ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc40d2e90 con 0x7f3bc40d0ad0 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57094/0 (socket says 172.21.15.60:57094) 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 -- 172.21.15.60:0/1622940060 learned_addr learned my addr 172.21.15.60:0/1622940060 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bca36a640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc40b9770 0x7f3bc41f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bc9b69640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc40d0ad0 0x7f3bc4155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 -- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc40d0ad0 msgr2=0x7f3bc4155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc40d0ad0 0x7f3bc4155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 -- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc40b9770 msgr2=0x7f3bc41f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc40b9770 0x7f3bc41f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 -- 172.21.15.60:0/1622940060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc4156050 con 0x7f3bc415dd30 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bca36a640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc40b9770 0x7f3bc41f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcab6b640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40c7f10 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00cef0 tx=0x7f3bbc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc01a070 con 0x7f3bc415dd30 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00d860 con 0x7f3bc415dd30 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc4156330 con 0x7f3bc415dd30 2024-04-25T08:39:13.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00dc50 con 0x7f3bc415dd30 2024-04-25T08:39:13.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.946+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc4153380 con 0x7f3bc415dd30 2024-04-25T08:39:13.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.947+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc40b6790 con 0x7f3bc415dd30 2024-04-25T08:39:13.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.948+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bbc01d070 con 0x7f3bc415dd30 2024-04-25T08:39:13.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.948+0000 7f3bbaffd640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ba806b4d0 0x7f3ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:13.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.948+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3bbc08ad10 con 0x7f3bc415dd30 2024-04-25T08:39:13.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.949+0000 7f3bca36a640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ba806b4d0 0x7f3ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:13.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.949+0000 7f3bca36a640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ba806b4d0 0x7f3ba806d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0034000 tx=0x7f3bc002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:13.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:13.953+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bc40b6790 con 0x7f3bc415dd30 2024-04-25T08:39:14.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.136+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bc4073990 con 0x7f3bc415dd30 2024-04-25T08:39:14.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.137+0000 7f3bbaffd640 1 -- 172.21.15.60:0/1622940060 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17017 (secure 0 0 0) 0x7f3bbc090020 con 0x7f3bc415dd30 2024-04-25T08:39:14.131 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:14.131 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":135,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:04.147601+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:15.506220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:16.201959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:6836/879630460":"2024-04-26T08:39:04.147468+0000","172.21.15.100:6834/879630460":"2024-04-26T08:39:04.147468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ba806b4d0 msgr2=0x7f3ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ba806b4d0 0x7f3ba806d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0034000 tx=0x7f3bc002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 msgr2=0x7f3bc40c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40c7f10 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00cef0 tx=0x7f3bbc00a710 comp rx=0 tx=0).stop 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 shutdown_connections 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3bc415dd30 0x7f3bc40c7f10 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3bc40d0ad0 0x7f3bc4155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3ba806b4d0 0x7f3ba806d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 --2- 172.21.15.60:0/1622940060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3bc40b9770 0x7f3bc41f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 >> 172.21.15.60:0/1622940060 conn(0x7f3bc4061c30 msgr2=0x7f3bc40c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 shutdown_connections 2024-04-25T08:39:14.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.139+0000 7f3bcc5ce640 1 -- 172.21.15.60:0/1622940060 wait complete. 2024-04-25T08:39:14.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:39:14.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 Processor -- start 2024-04-25T08:39:14.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 -- start start 2024-04-25T08:39:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fc8161710 0x7f3fc80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc815e000 con 0x7f3fc80b9330 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc80d1de0 con 0x7f3fc8161710 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.254+0000 7f3fce922640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8161c60 con 0x7f3fc815dc20 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fc8161710 0x7f3fc80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3fc8161710 0x7f3fc80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57102/0 (socket says 172.21.15.60:57102) 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 -- 172.21.15.60:0/4265592667 learned_addr learned my addr 172.21.15.60:0/4265592667 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc7fff640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 -- 172.21.15.60:0/4265592667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 msgr2=0x7f3fc80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 -- 172.21.15.60:0/4265592667 >> v1:172.21.15.100:6789/0 conn(0x7f3fc80b9330 legacy=0x7f3fc806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc77fe640 1 -- 172.21.15.60:0/4265592667 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8161f40 con 0x7f3fc8161710 2024-04-25T08:39:14.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.255+0000 7f3fc7fff640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8161710 0x7f3fc80cebb0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009870 tx=0x7f3fbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece393dfee7823e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fc6ffd640 1 -- 172.21.15.60:0/4265592667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc037070 con 0x7f3fc8161710 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fc6ffd640 1 -- 172.21.15.60:0/4265592667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc032440 con 0x7f3fc8161710 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fc6ffd640 1 -- 172.21.15.60:0/4265592667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc032810 con 0x7f3fc8161710 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 -- 172.21.15.60:0/4265592667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8161710 msgr2=0x7f3fc80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8161710 0x7f3fc80cebb0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009870 tx=0x7f3fbc02a270 comp rx=0 tx=0).stop 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 -- 172.21.15.60:0/4265592667 shutdown_connections 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc8161710 0x7f3fc80cebb0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 --2- 172.21.15.60:0/4265592667 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 -- 172.21.15.60:0/4265592667 >> 172.21.15.60:0/4265592667 conn(0x7f3fc80d0420 msgr2=0x7f3fc80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 -- 172.21.15.60:0/4265592667 shutdown_connections 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.256+0000 7f3fce922640 1 -- 172.21.15.60:0/4265592667 wait complete. 2024-04-25T08:39:14.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.257+0000 7f3fce922640 1 Processor -- start 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 -- start start 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 0x7f3fc81e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8161710 0x7f3fc814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80b5e50 con 0x7f3fc80b9330 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8162500 con 0x7f3fc8161710 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.258+0000 7f3fce922640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80b5fd0 con 0x7f3fc815dc20 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 0x7f3fc81e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 0x7f3fc81e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57108/0 (socket says 172.21.15.60:57108) 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 -- 172.21.15.60:0/883949207 learned_addr learned my addr 172.21.15.60:0/883949207 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:14.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fccebf640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8161710 0x7f3fc814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 -- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 msgr2=0x7f3fc80c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 -- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8161710 msgr2=0x7f3fc814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8161710 0x7f3fc814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 -- 172.21.15.60:0/883949207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80c90c0 con 0x7f3fc80b9330 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fccebf640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8161710 0x7f3fc814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc7fff640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 0x7f3fc81e77c0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800d800 tx=0x7f3fb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8004030 con 0x7f3fc80b9330 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb80041d0 con 0x7f3fc80b9330 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80096a0 con 0x7f3fc80b9330 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc80c93a0 con 0x7f3fc80b9330 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.259+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc80c9960 con 0x7f3fc80b9330 2024-04-25T08:39:14.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.260+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8073a00 con 0x7f3fc80b9330 2024-04-25T08:39:14.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.261+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fb8009840 con 0x7f3fc80b9330 2024-04-25T08:39:14.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.261+0000 7f3fc4ff9640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.261+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3fb808a770 con 0x7f3fc80b9330 2024-04-25T08:39:14.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.262+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.262+0000 7f3fc77fe640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fb406b4d0 0x7f3fb406d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:14.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.266+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fb8059380 con 0x7f3fc80b9330 2024-04-25T08:39:14.478 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.484+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3fc80d16b0 con 0x7f3fc80b9330 2024-04-25T08:39:14.478 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:39:14.485+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:39:14.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.486+0000 7f3fc4ff9640 1 -- 172.21.15.60:0/883949207 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3fb8058c80 con 0x7f3fc80b9330 2024-04-25T08:39:14.480 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:39:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fb406b4d0 msgr2=0x7f3fb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fb406b4d0 0x7f3fb406d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc034000 tx=0x7f3fbc02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 msgr2=0x7f3fc81e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 0x7f3fc81e77c0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800d800 tx=0x7f3fb800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.483 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 shutdown_connections 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3fc8161710 0x7f3fc814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3fc815dc20 0x7f3fc80c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 --2- 172.21.15.60:0/883949207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fc80b9330 0x7f3fc81e77c0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 >> 172.21.15.60:0/883949207 conn(0x7f3fc80d0420 msgr2=0x7f3fc80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 shutdown_connections 2024-04-25T08:39:14.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.489+0000 7f3fce922640 1 -- 172.21.15.60:0/883949207 wait complete. 2024-04-25T08:39:14.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:39:14.521 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:39:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.606+0000 7fd7c6f09640 1 Processor -- start 2024-04-25T08:39:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.606+0000 7fd7c6f09640 1 -- start start 2024-04-25T08:39:14.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c6f09640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7c015b7d0 0x7fd7c00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c6f09640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7c00b5620 0x7fd7c00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c6f09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c006c2d0 0x7fd7c006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.601 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c6f09640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c006cc80 con 0x7fd7c015b7d0 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c6f09640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c006cf50 con 0x7fd7c00b5620 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c6f09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00ce530 con 0x7fd7c006c2d0 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7c015b7d0 0x7fd7c00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd7c015b7d0 0x7fd7c00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57118/0 (socket says 172.21.15.60:57118) 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 -- 172.21.15.60:0/3254099342 learned_addr learned my addr 172.21.15.60:0/3254099342 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7b7fff640 1 --2- 172.21.15.60:0/3254099342 >> v2:172.21.15.100:3300/0 conn(0x7fd7c00b5620 0x7fd7c00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c54a6640 1 --2- 172.21.15.60:0/3254099342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c006c2d0 0x7fd7c006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 -- 172.21.15.60:0/3254099342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c006c2d0 msgr2=0x7fd7c006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 --2- 172.21.15.60:0/3254099342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c006c2d0 0x7fd7c006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 -- 172.21.15.60:0/3254099342 >> v2:172.21.15.100:3300/0 conn(0x7fd7c00b5620 msgr2=0x7fd7c00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 --2- 172.21.15.60:0/3254099342 >> v2:172.21.15.100:3300/0 conn(0x7fd7c00b5620 0x7fd7c00b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.602 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7c4ca5640 1 -- 172.21.15.60:0/3254099342 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00ce810 con 0x7fd7c015b7d0 2024-04-25T08:39:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.607+0000 7fd7b7fff640 1 --2- 172.21.15.60:0/3254099342 >> v2:172.21.15.100:3300/0 conn(0x7fd7c00b5620 0x7fd7c00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c4ca5640 1 --2- 172.21.15.60:0/3254099342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c015b7d0 0x7fd7c00cf010 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00ce180 tx=0x7fd7bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5b397becc9c097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7b77fe640 1 -- 172.21.15.60:0/3254099342 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc037070 con 0x7fd7c015b7d0 2024-04-25T08:39:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7b77fe640 1 -- 172.21.15.60:0/3254099342 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc032440 con 0x7fd7c015b7d0 2024-04-25T08:39:14.603 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7b77fe640 1 -- 172.21.15.60:0/3254099342 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc032810 con 0x7fd7c015b7d0 2024-04-25T08:39:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3254099342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c015b7d0 msgr2=0x7fd7c00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3254099342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c015b7d0 0x7fd7c00cf010 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00ce180 tx=0x7fd7bc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3254099342 shutdown_connections 2024-04-25T08:39:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3254099342 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c006c2d0 0x7fd7c006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3254099342 >> v2:172.21.15.100:3300/0 conn(0x7fd7c00b5620 0x7fd7c00b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.606 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3254099342 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c015b7d0 0x7fd7c00cf010 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.608+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3254099342 >> 172.21.15.60:0/3254099342 conn(0x7fd7c00d0800 msgr2=0x7fd7c00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.609+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3254099342 shutdown_connections 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.609+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3254099342 wait complete. 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.610+0000 7fd7c6f09640 1 Processor -- start 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.610+0000 7fd7c6f09640 1 -- start start 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c006c2d0 0x7fd7c01f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 0x7fd7c00c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c015b7d0 0x7fd7c0159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cf550 con 0x7fd7c00b5620 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c006d230 con 0x7fd7c006c2d0 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cf6d0 con 0x7fd7c015b7d0 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 0x7fd7c00c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 0x7fd7c00c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57122/0 (socket says 172.21.15.60:57122) 2024-04-25T08:39:14.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 -- 172.21.15.60:0/3124367695 learned_addr learned my addr 172.21.15.60:0/3124367695 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c4ca5640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c006c2d0 0x7fd7c01f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c54a6640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c015b7d0 0x7fd7c0159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 -- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c015b7d0 msgr2=0x7fd7c0159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c015b7d0 0x7fd7c0159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 -- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c006c2d0 msgr2=0x7fd7c01f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c006c2d0 0x7fd7c01f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 -- 172.21.15.60:0/3124367695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00c99d0 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b7fff640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 0x7fd7c00c9060 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000e7e0 tx=0x7fd7b000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000f040 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0004030 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c01f6960 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0004400 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.611+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01f2f80 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.612+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0073a00 con 0x7fd7c00b5620 2024-04-25T08:39:14.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.613+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7b000c6d0 con 0x7fd7c00b5620 2024-04-25T08:39:14.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.613+0000 7fd7b57fa640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7ac06b500 0x7fd7ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:14.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.613+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd7b008a180 con 0x7fd7c00b5620 2024-04-25T08:39:14.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.613+0000 7fd7c4ca5640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7ac06b500 0x7fd7ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:14.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.614+0000 7fd7c4ca5640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7ac06b500 0x7fd7ac06d9c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc034000 tx=0x7fd7bc02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:14.610 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.617+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7b0017020 con 0x7fd7c00b5620 2024-04-25T08:39:14.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:14.835+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd7c006b210 con 0x7fd7c00b5620 2024-04-25T08:39:15.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.488+0000 7fd7b57fa640 1 -- 172.21.15.60:0/3124367695 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7b0058e50 con 0x7fd7c00b5620 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7ac06b500 msgr2=0x7fd7ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7ac06b500 0x7fd7ac06d9c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc034000 tx=0x7fd7bc02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 msgr2=0x7fd7c00c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 0x7fd7c00c9060 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000e7e0 tx=0x7fd7b000eca0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 shutdown_connections 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd7c015b7d0 0x7fd7c0159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c00b5620 0x7fd7c00c9060 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd7ac06b500 0x7fd7ac06d9c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 --2- 172.21.15.60:0/3124367695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7c006c2d0 0x7fd7c01f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 >> 172.21.15.60:0/3124367695 conn(0x7fd7c00d0800 msgr2=0x7fd7c00b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:15.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 shutdown_connections 2024-04-25T08:39:15.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.490+0000 7fd7c6f09640 1 -- 172.21.15.60:0/3124367695 wait complete. 2024-04-25T08:39:15.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:39:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.656+0000 7fc81b370640 1 Processor -- start 2024-04-25T08:39:15.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.656+0000 7fc81b370640 1 -- start start 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81b370640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81415dac0 0x7fc8140d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81b370640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8140d0ab0 0x7fc8140b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81b370640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8140d2420 con 0x7fc8140d1f30 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81b370640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8140d3030 con 0x7fc8140d0ab0 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81b370640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140d2700 con 0x7fc81415dac0 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81990d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8140d0ab0 0x7fc8140b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81890b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc8140d1f30 0x7fc8140d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50978/0 (socket says 172.21.15.60:50978) 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81890b640 1 -- 172.21.15.60:0/732571611 learned_addr learned my addr 172.21.15.60:0/732571611 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81910c640 1 --2- 172.21.15.60:0/732571611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81415dac0 0x7fc8140d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81990d640 1 -- 172.21.15.60:0/732571611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81415dac0 msgr2=0x7fc8140d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81990d640 1 --2- 172.21.15.60:0/732571611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81415dac0 0x7fc8140d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81990d640 1 -- 172.21.15.60:0/732571611 >> v1:172.21.15.60:6789/0 conn(0x7fc8140d1f30 legacy=0x7fc8140d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc80bfff640 1 -- 172.21.15.60:0/732571611 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725787273 0 0) 0x7fc8140d2420 con 0x7fc8140d1f30 2024-04-25T08:39:15.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.657+0000 7fc81990d640 1 -- 172.21.15.60:0/732571611 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8140d29e0 con 0x7fc8140d0ab0 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.658+0000 7fc81990d640 1 --2- 172.21.15.60:0/732571611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d0ab0 0x7fc8140b6990 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009d90 tx=0x7fc80c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44caab909861d66e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.658+0000 7fc80bfff640 1 -- 172.21.15.60:0/732571611 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c007740 con 0x7fc8140d0ab0 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.658+0000 7fc80bfff640 1 -- 172.21.15.60:0/732571611 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c0078e0 con 0x7fc8140d0ab0 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.658+0000 7fc80bfff640 1 -- 172.21.15.60:0/732571611 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c007cb0 con 0x7fc8140d0ab0 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 -- 172.21.15.60:0/732571611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d0ab0 msgr2=0x7fc8140b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 --2- 172.21.15.60:0/732571611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d0ab0 0x7fc8140b6990 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009d90 tx=0x7fc80c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 -- 172.21.15.60:0/732571611 shutdown_connections 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 --2- 172.21.15.60:0/732571611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d0ab0 0x7fc8140b6990 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 --2- 172.21.15.60:0/732571611 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc81415dac0 0x7fc8140d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 -- 172.21.15.60:0/732571611 >> 172.21.15.60:0/732571611 conn(0x7fc814061c30 msgr2=0x7fc81406c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 -- 172.21.15.60:0/732571611 shutdown_connections 2024-04-25T08:39:15.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.659+0000 7fc81b370640 1 -- 172.21.15.60:0/732571611 wait complete. 2024-04-25T08:39:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.660+0000 7fc81b370640 1 Processor -- start 2024-04-25T08:39:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.660+0000 7fc81b370640 1 -- start start 2024-04-25T08:39:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81b370640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8140d0ab0 0x7fc814150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81b370640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 0x7fc814160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81b370640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81415dac0 0x7fc8141607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81b370640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140b97d0 con 0x7fc8140d1f30 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81b370640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81415dea0 con 0x7fc81415dac0 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81b370640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8140b9950 con 0x7fc8140d0ab0 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 0x7fc814160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 0x7fc814160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57146/0 (socket says 172.21.15.60:57146) 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 -- 172.21.15.60:0/773398301 learned_addr learned my addr 172.21.15.60:0/773398301 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81910c640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8140d0ab0 0x7fc814150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81990d640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81415dac0 0x7fc8141607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 -- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8140d0ab0 msgr2=0x7fc814150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8140d0ab0 0x7fc814150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 -- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81415dac0 msgr2=0x7fc8141607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81415dac0 0x7fc8141607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 -- 172.21.15.60:0/773398301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814154df0 con 0x7fc8140d1f30 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc81890b640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 0x7fc814160280 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c730 tx=0x7fc80400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:15.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.661+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804007270 con 0x7fc8140d1f30 2024-04-25T08:39:15.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.662+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804007410 con 0x7fc8140d1f30 2024-04-25T08:39:15.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.662+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc814151840 con 0x7fc8140d1f30 2024-04-25T08:39:15.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.662+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804001d00 con 0x7fc8140d1f30 2024-04-25T08:39:15.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.662+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc814151e00 con 0x7fc8140d1f30 2024-04-25T08:39:15.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.663+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc814073a00 con 0x7fc8140d1f30 2024-04-25T08:39:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.663+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8040057a0 con 0x7fc8140d1f30 2024-04-25T08:39:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.664+0000 7fc809ffb640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f806b500 0x7fc7f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.664+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc80408a150 con 0x7fc8140d1f30 2024-04-25T08:39:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.664+0000 7fc81910c640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f806b500 0x7fc7f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.664+0000 7fc81910c640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f806b500 0x7fc7f806d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc810006fd0 tx=0x7fc810008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:15.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.668+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc804058d80 con 0x7fc8140d1f30 2024-04-25T08:39:15.843 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:15.849+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc81415b680 con 0x7fc8140d1f30 2024-04-25T08:39:16.486 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.493+0000 7fc809ffb640 1 -- 172.21.15.60:0/773398301 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc804058680 con 0x7fc8140d1f30 2024-04-25T08:39:16.487 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f806b500 msgr2=0x7fc7f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f806b500 0x7fc7f806d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc810006fd0 tx=0x7fc810008040 comp rx=0 tx=0).stop 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 msgr2=0x7fc814160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 0x7fc814160280 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c730 tx=0x7fc80400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 shutdown_connections 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc7f806b500 0x7fc7f806d9c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc81415dac0 0x7fc8141607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8140d1f30 0x7fc814160280 secure :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c730 tx=0x7fc80400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 --2- 172.21.15.60:0/773398301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8140d0ab0 0x7fc814150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.497+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 >> 172.21.15.60:0/773398301 conn(0x7fc814061c30 msgr2=0x7fc8140b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.498+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 shutdown_connections 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.498+0000 7fc81b370640 1 -- 172.21.15.60:0/773398301 wait complete. 2024-04-25T08:39:16.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:39:16.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.623+0000 7f3dc796a640 1 Processor -- start 2024-04-25T08:39:16.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.624+0000 7f3dc796a640 1 -- start start 2024-04-25T08:39:16.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.624+0000 7f3dc796a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc796a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3dc006c9e0 0x7f3dc00b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc796a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc006cf50 con 0x7f3dc00b5620 2024-04-25T08:39:16.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc796a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc00b5fc0 con 0x7f3dc006c9e0 2024-04-25T08:39:16.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc796a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00ce530 con 0x7f3dc015b7d0 2024-04-25T08:39:16.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc4f05640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f3dc00b5620 0x7f3dc006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50980/0 (socket says 172.21.15.60:50980) 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc5f07640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3dc006c9e0 0x7f3dc00b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc4f05640 1 -- 172.21.15.60:0/155681403 learned_addr learned my addr 172.21.15.60:0/155681403 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3db7fff640 1 -- 172.21.15.60:0/155681403 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096267253 0 0) 0x7f3dc006cf50 con 0x7f3dc00b5620 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3db7fff640 1 -- 172.21.15.60:0/155681403 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dac003520 con 0x7f3dc00b5620 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc5706640 1 --2- 172.21.15.60:0/155681403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc5f07640 1 -- 172.21.15.60:0/155681403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 msgr2=0x7f3dc00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc5f07640 1 --2- 172.21.15.60:0/155681403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc5f07640 1 -- 172.21.15.60:0/155681403 >> v1:172.21.15.60:6789/0 conn(0x7f3dc00b5620 legacy=0x7f3dc006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.625+0000 7f3dc5f07640 1 -- 172.21.15.60:0/155681403 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc00ce810 con 0x7f3dc006c9e0 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.626+0000 7f3dc5f07640 1 --2- 172.21.15.60:0/155681403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc00b5a80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009d90 tx=0x7f3db802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bf547fa78e71f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.626+0000 7f3db7fff640 1 -- 172.21.15.60:0/155681403 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8007740 con 0x7f3dc006c9e0 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.626+0000 7f3db7fff640 1 -- 172.21.15.60:0/155681403 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db80078e0 con 0x7f3dc006c9e0 2024-04-25T08:39:16.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.626+0000 7f3db7fff640 1 -- 172.21.15.60:0/155681403 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8007cb0 con 0x7f3dc006c9e0 2024-04-25T08:39:16.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.626+0000 7f3dc796a640 1 -- 172.21.15.60:0/155681403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 msgr2=0x7f3dc00b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.626+0000 7f3dc796a640 1 --2- 172.21.15.60:0/155681403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc00b5a80 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009d90 tx=0x7f3db802a660 comp rx=0 tx=0).stop 2024-04-25T08:39:16.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.627+0000 7f3dc796a640 1 -- 172.21.15.60:0/155681403 shutdown_connections 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.627+0000 7f3dc796a640 1 --2- 172.21.15.60:0/155681403 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc00b5a80 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.627+0000 7f3dc796a640 1 --2- 172.21.15.60:0/155681403 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.627+0000 7f3dc796a640 1 -- 172.21.15.60:0/155681403 >> 172.21.15.60:0/155681403 conn(0x7f3dc00d0800 msgr2=0x7f3dc00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.627+0000 7f3dc796a640 1 -- 172.21.15.60:0/155681403 shutdown_connections 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.627+0000 7f3dc796a640 1 -- 172.21.15.60:0/155681403 wait complete. 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.628+0000 7f3dc796a640 1 Processor -- start 2024-04-25T08:39:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.628+0000 7f3dc796a640 1 -- start start 2024-04-25T08:39:16.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.628+0000 7f3dc796a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc796a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc00b5620 0x7f3dc00b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc796a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc796a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00cf550 con 0x7f3dc006c9e0 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc796a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc006d230 con 0x7f3dc00b5620 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc796a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00cf6d0 con 0x7f3dc015b7d0 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc01599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57164/0 (socket says 172.21.15.60:57164) 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 -- 172.21.15.60:0/143422945 learned_addr learned my addr 172.21.15.60:0/143422945 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5f07640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc4f05640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc00b5620 0x7f3dc00b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 -- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 msgr2=0x7f3dc00b8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00b8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 -- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc00b5620 msgr2=0x7f3dc00b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc00b5620 0x7f3dc00b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 -- 172.21.15.60:0/143422945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc01f7a00 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc4f05640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc00b5620 0x7f3dc00b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc5706640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc01599e0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c7f0 tx=0x7f3dbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc015070 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc004220 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc01f3960 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.629+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc007620 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.630+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc01f3f20 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.630+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc00d3990 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.631+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3dbc0079d0 con 0x7f3dc006c9e0 2024-04-25T08:39:16.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.631+0000 7f3db5ffb640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3da406b4d0 0x7f3da406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.631+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f3dbc08a2f0 con 0x7f3dc006c9e0 2024-04-25T08:39:16.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.631+0000 7f3dc4f05640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3da406b4d0 0x7f3da406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.632+0000 7f3dc4f05640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3da406b4d0 0x7f3da406d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3dc01f6ca0 tx=0x7f3db0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.635+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3dc00d3990 con 0x7f3dc006c9e0 2024-04-25T08:39:16.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:16.818+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3dc0073a00 con 0x7f3dc006c9e0 2024-04-25T08:39:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.494+0000 7f3db5ffb640 1 -- 172.21.15.60:0/143422945 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3dbc0590f0 con 0x7f3dc006c9e0 2024-04-25T08:39:17.490 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:39:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.499+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3da406b4d0 msgr2=0x7f3da406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.499+0000 7f3dc796a640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3da406b4d0 0x7f3da406d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3dc01f6ca0 tx=0x7f3db0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.499+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 msgr2=0x7f3dc01599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.499+0000 7f3dc796a640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc01599e0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c7f0 tx=0x7f3dbc00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 shutdown_connections 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3dc015b7d0 0x7f3dc00b8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3da406b4d0 0x7f3da406d990 secure :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3dc01f6ca0 tx=0x7f3db0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dc00b5620 0x7f3dc00b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 --2- 172.21.15.60:0/143422945 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3dc006c9e0 0x7f3dc01599e0 secure :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00c7f0 tx=0x7f3dbc00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 >> 172.21.15.60:0/143422945 conn(0x7f3dc00d0800 msgr2=0x7f3dc01604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 shutdown_connections 2024-04-25T08:39:17.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.500+0000 7f3dc796a640 1 -- 172.21.15.60:0/143422945 wait complete. 2024-04-25T08:39:17.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:39:17.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.650+0000 7ff6ebc93640 1 Processor -- start 2024-04-25T08:39:17.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.650+0000 7ff6ebc93640 1 -- start start 2024-04-25T08:39:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.652+0000 7ff6ebc93640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff6e41434e0 0x7ff6e4071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.652+0000 7ff6ebc93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e406d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.652+0000 7ff6ebc93640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e406e050 con 0x7ff6e406dbd0 2024-04-25T08:39:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.652+0000 7ff6e9a2f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff6e41434e0 0x7ff6e4071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.652+0000 7ff6ebc93640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e4071fb0 con 0x7ff6e41434e0 2024-04-25T08:39:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.652+0000 7ff6ebc93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406e310 con 0x7ff6e4071b70 2024-04-25T08:39:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e9a2f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff6e41434e0 0x7ff6e4071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44822/0 (socket says 172.21.15.60:44822) 2024-04-25T08:39:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/2149095890 learned_addr learned my addr 172.21.15.60:0/2149095890 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:17.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e922e640 1 --2- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 msgr2=0x7ff6e406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e9a2f640 1 --2- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/2149095890 >> v1:172.21.15.60:6789/0 conn(0x7ff6e406dbd0 legacy=0x7ff6e415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.653+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/2149095890 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e406e5f0 con 0x7ff6e41434e0 2024-04-25T08:39:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.655+0000 7ff6e9a2f640 1 --2- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e41434e0 0x7ff6e4071630 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0007360 tx=0x7ff6e002b310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9727600d578b12fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.655+0000 7ff6e8a2d640 1 -- 172.21.15.60:0/2149095890 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e002be60 con 0x7ff6e41434e0 2024-04-25T08:39:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.655+0000 7ff6e8a2d640 1 -- 172.21.15.60:0/2149095890 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e0008be0 con 0x7ff6e41434e0 2024-04-25T08:39:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.655+0000 7ff6e8a2d640 1 -- 172.21.15.60:0/2149095890 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000a0d0 con 0x7ff6e41434e0 2024-04-25T08:39:17.649 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.656+0000 7ff6ebc93640 1 -- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e41434e0 msgr2=0x7ff6e4071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.657+0000 7ff6ebc93640 1 --2- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e41434e0 0x7ff6e4071630 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0007360 tx=0x7ff6e002b310 comp rx=0 tx=0).stop 2024-04-25T08:39:17.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.657+0000 7ff6ebc93640 1 -- 172.21.15.60:0/2149095890 shutdown_connections 2024-04-25T08:39:17.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.657+0000 7ff6ebc93640 1 --2- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.657+0000 7ff6ebc93640 1 --2- 172.21.15.60:0/2149095890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e41434e0 0x7ff6e4071630 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.657+0000 7ff6ebc93640 1 -- 172.21.15.60:0/2149095890 >> 172.21.15.60:0/2149095890 conn(0x7ff6e40b1790 msgr2=0x7ff6e40b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.651 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.657+0000 7ff6ebc93640 1 -- 172.21.15.60:0/2149095890 shutdown_connections 2024-04-25T08:39:17.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.659+0000 7ff6ebc93640 1 -- 172.21.15.60:0/2149095890 wait complete. 2024-04-25T08:39:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.661+0000 7ff6ebc93640 1 Processor -- start 2024-04-25T08:39:17.654 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.661+0000 7ff6ebc93640 1 -- start start 2024-04-25T08:39:17.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.662+0000 7ff6ebc93640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 0x7ff6e40ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.662+0000 7ff6ebc93640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e41e2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.662+0000 7ff6ebc93640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6e41434e0 0x7ff6e415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.662+0000 7ff6ebc93640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406ee90 con 0x7ff6e41434e0 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.662+0000 7ff6ebc93640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406ebb0 con 0x7ff6e406dbd0 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.662+0000 7ff6ebc93640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e41438c0 con 0x7ff6e4071b70 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.663+0000 7ff6e9a2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 0x7ff6e40ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.663+0000 7ff6e9a2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 0x7ff6e40ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:44830/0 (socket says 172.21.15.60:44830) 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.663+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/123248069 learned_addr learned my addr 172.21.15.60:0/123248069 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.663+0000 7ff6e922e640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e41e2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.663+0000 7ff6ea230640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6e41434e0 0x7ff6e415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 msgr2=0x7ff6e41e2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6e9a2f640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e41e2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/123248069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6e41434e0 msgr2=0x7ff6e415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6e9a2f640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6e41434e0 0x7ff6e415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6e9a2f640 1 -- 172.21.15.60:0/123248069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40c6900 con 0x7ff6e406dbd0 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6ea230640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6e41434e0 0x7ff6e415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.664+0000 7ff6e9a2f640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 0x7ff6e40ce100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0035040 tx=0x7ff6e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.665+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0042020 con 0x7ff6e406dbd0 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.665+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e003e880 con 0x7ff6e406dbd0 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.665+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0042020 con 0x7ff6e406dbd0 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.665+0000 7ff6ebc93640 1 -- 172.21.15.60:0/123248069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40c6bc0 con 0x7ff6e406dbd0 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.665+0000 7ff6ebc93640 1 -- 172.21.15.60:0/123248069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e40c7180 con 0x7ff6e406dbd0 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.667+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e40c75c0 con 0x7ff6e406dbd0 2024-04-25T08:39:17.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.668+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6e0045020 con 0x7ff6e406dbd0 2024-04-25T08:39:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.669+0000 7ff6da7fc640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff6c806b8d0 0x7ff6c806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.669+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff6e003e680 con 0x7ff6e406dbd0 2024-04-25T08:39:17.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.671+0000 7ff6e922e640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff6c806b8d0 0x7ff6c806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.673+0000 7ff6e922e640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff6c806b8d0 0x7ff6c806dd90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40ca400 tx=0x7ff6d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.674+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6e004c030 con 0x7ff6e406dbd0 2024-04-25T08:39:17.852 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.859+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff6e4073a00 con 0x7ff6e406dbd0 2024-04-25T08:39:17.858 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.865+0000 7ff6da7fc640 1 -- 172.21.15.60:0/123248069 <== mon.1 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff6e004c030 con 0x7ff6e406dbd0 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff6c806b8d0 msgr2=0x7ff6c806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff6c806b8d0 0x7ff6c806dd90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40ca400 tx=0x7ff6d4008040 comp rx=0 tx=0).stop 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 msgr2=0x7ff6e40ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 0x7ff6e40ce100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0035040 tx=0x7ff6e00046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 shutdown_connections 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff6e41434e0 0x7ff6e415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff6e4071b70 0x7ff6e41e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.861 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff6c806b8d0 0x7ff6c806dd90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 --2- 172.21.15.60:0/123248069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6e406dbd0 0x7ff6e40ce100 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 >> 172.21.15.60:0/123248069 conn(0x7ff6e40b1790 msgr2=0x7ff6e415cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 shutdown_connections 2024-04-25T08:39:17.862 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.867+0000 7ff6c7fff640 1 -- 172.21.15.60:0/123248069 wait complete. 2024-04-25T08:39:17.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T08:39:17.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.979+0000 7f088ce4d640 1 Processor -- start 2024-04-25T08:39:17.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.979+0000 7f088ce4d640 1 -- start start 2024-04-25T08:39:17.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088ce4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f08881616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088ce4d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08880b93a0 0x7f08880ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088ce4d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f08880cec20 0x7f088806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088ce4d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088815e000 con 0x7f08880b93a0 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088ce4d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880b6bb0 con 0x7f08880cec20 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088ce4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880b6030 con 0x7f088815dc20 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08880b93a0 0x7f08880ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f08880b93a0 0x7f08880ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57182/0 (socket says 172.21.15.60:57182) 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 -- 172.21.15.60:0/944073590 learned_addr learned my addr 172.21.15.60:0/944073590 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f088659c640 1 --2- 172.21.15.60:0/944073590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f08881616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f0886d9d640 1 --2- 172.21.15.60:0/944073590 >> v2:172.21.15.100:3300/0 conn(0x7f08880cec20 0x7f088806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 -- 172.21.15.60:0/944073590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 msgr2=0x7f08881616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 --2- 172.21.15.60:0/944073590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f08881616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 -- 172.21.15.60:0/944073590 >> v2:172.21.15.100:3300/0 conn(0x7f08880cec20 msgr2=0x7f088806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 --2- 172.21.15.60:0/944073590 >> v2:172.21.15.100:3300/0 conn(0x7f08880cec20 0x7f088806c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.975 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.980+0000 7f087ffff640 1 -- 172.21.15.60:0/944073590 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0888161de0 con 0x7f08880b93a0 2024-04-25T08:39:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.981+0000 7f087ffff640 1 --2- 172.21.15.60:0/944073590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880b93a0 0x7f08880ce6e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f08880d1ff0 tx=0x7f087002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daaa76ea703fc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.981+0000 7f0885d9b640 1 -- 172.21.15.60:0/944073590 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870037070 con 0x7f08880b93a0 2024-04-25T08:39:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.981+0000 7f0885d9b640 1 -- 172.21.15.60:0/944073590 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870032440 con 0x7f08880b93a0 2024-04-25T08:39:17.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.981+0000 7f0885d9b640 1 -- 172.21.15.60:0/944073590 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870032810 con 0x7f08880b93a0 2024-04-25T08:39:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 -- 172.21.15.60:0/944073590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880b93a0 msgr2=0x7f08880ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 --2- 172.21.15.60:0/944073590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880b93a0 0x7f08880ce6e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f08880d1ff0 tx=0x7f087002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 -- 172.21.15.60:0/944073590 shutdown_connections 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 --2- 172.21.15.60:0/944073590 >> v2:172.21.15.100:3300/0 conn(0x7f08880cec20 0x7f088806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 --2- 172.21.15.60:0/944073590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880b93a0 0x7f08880ce6e0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 --2- 172.21.15.60:0/944073590 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f08881616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 -- 172.21.15.60:0/944073590 >> 172.21.15.60:0/944073590 conn(0x7f08880d0490 msgr2=0x7f08880d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 -- 172.21.15.60:0/944073590 shutdown_connections 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.982+0000 7f088ce4d640 1 -- 172.21.15.60:0/944073590 wait complete. 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.983+0000 7f088ce4d640 1 Processor -- start 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.983+0000 7f088ce4d640 1 -- start start 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088ce4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08880b93a0 0x7f088815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088ce4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 0x7f08880c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088ce4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f088814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088ce4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088806c610 con 0x7f08880cec20 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088ce4d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881623a0 con 0x7f08880b93a0 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088ce4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088806c790 con 0x7f088815dc20 2024-04-25T08:39:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 0x7f08880c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 0x7f08880c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57188/0 (socket says 172.21.15.60:57188) 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 -- 172.21.15.60:0/1747389990 learned_addr learned my addr 172.21.15.60:0/1747389990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f088659c640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08880b93a0 0x7f088815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f0886d9d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f088814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 -- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 msgr2=0x7f088814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f088814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 -- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08880b93a0 msgr2=0x7f088815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08880b93a0 0x7f088815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 -- 172.21.15.60:0/1747389990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08881e8df0 con 0x7f08880cec20 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.984+0000 7f087ffff640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 0x7f08880c9b30 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f087002a220 tx=0x7f08700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.985+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870041070 con 0x7f08880cec20 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.985+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870032e50 con 0x7f08880cec20 2024-04-25T08:39:17.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.985+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870037040 con 0x7f08880cec20 2024-04-25T08:39:17.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.985+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08881e90d0 con 0x7f08880cec20 2024-04-25T08:39:17.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.985+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08881e9690 con 0x7f08880cec20 2024-04-25T08:39:17.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.986+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0888073990 con 0x7f08880cec20 2024-04-25T08:39:17.981 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.986+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0870044070 con 0x7f08880cec20 2024-04-25T08:39:17.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.987+0000 7f087effd640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f086806b4d0 0x7f086806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.987+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f08700aaba0 con 0x7f08880cec20 2024-04-25T08:39:17.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.987+0000 7f088659c640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f086806b4d0 0x7f086806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.987+0000 7f088659c640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f086806b4d0 0x7f086806d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0878009a30 tx=0x7f0878008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:17.991+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0870079cf0 con 0x7f08880cec20 2024-04-25T08:39:18.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.239+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f08881e9d90 con 0x7f08880cec20 2024-04-25T08:39:18.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.498+0000 7f087effd640 1 -- 172.21.15.60:0/1747389990 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v488) v1 ==== 166+0+0 (secure 0 0 0) 0x7f08700795f0 con 0x7f08880cec20 2024-04-25T08:39:18.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f086806b4d0 msgr2=0x7f086806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f086806b4d0 0x7f086806d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f0878009a30 tx=0x7f0878008040 comp rx=0 tx=0).stop 2024-04-25T08:39:18.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 msgr2=0x7f08880c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 0x7f08880c9b30 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f087002a220 tx=0x7f08700046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.494 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 shutdown_connections 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f088815dc20 0x7f088814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f08880cec20 0x7f08880c9b30 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f086806b4d0 0x7f086806d990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 --2- 172.21.15.60:0/1747389990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08880b93a0 0x7f088815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 >> 172.21.15.60:0/1747389990 conn(0x7f08880d0490 msgr2=0x7f08880b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 shutdown_connections 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.500+0000 7f088ce4d640 1 -- 172.21.15.60:0/1747389990 wait complete. 2024-04-25T08:39:18.505 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:39:18.506 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:18.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:18.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.618+0000 7fc2ad09a640 1 Processor -- start 2024-04-25T08:39:18.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.618+0000 7fc2ad09a640 1 -- start start 2024-04-25T08:39:18.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2ad09a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a80b6cb0 0x7fc2a80b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2ad09a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2a80b5a60 0x7fc2a80d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2ad09a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80ceee0 con 0x7fc2a815db70 2024-04-25T08:39:18.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2ad09a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a80d1d10 con 0x7fc2a80b5a60 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2ad09a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80d1ff0 con 0x7fc2a80b6cb0 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2a759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2a80b5a60 0x7fc2a80d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2a759e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc2a80b5a60 0x7fc2a80d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57202/0 (socket says 172.21.15.60:57202) 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.619+0000 7fc2a759e640 1 -- 172.21.15.60:0/1883096994 learned_addr learned my addr 172.21.15.60:0/1883096994 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a759e640 1 -- 172.21.15.60:0/1883096994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a80b6cb0 msgr2=0x7fc2a80b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a659c640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a80b6cb0 0x7fc2a80b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a759e640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a80b6cb0 0x7fc2a80b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a759e640 1 -- 172.21.15.60:0/1883096994 >> v1:172.21.15.100:6789/0 conn(0x7fc2a815db70 legacy=0x7fc2a8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a759e640 1 -- 172.21.15.60:0/1883096994 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80d22d0 con 0x7fc2a80b5a60 2024-04-25T08:39:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a659c640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a80b6cb0 0x7fc2a80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.620+0000 7fc2a759e640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a80d27f0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc298009e50 tx=0x7fc29802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0b0d7e171c7232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2a5d9b640 1 -- 172.21.15.60:0/1883096994 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298036070 con 0x7fc2a80b5a60 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2a5d9b640 1 -- 172.21.15.60:0/1883096994 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298031600 con 0x7fc2a80b5a60 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2a5d9b640 1 -- 172.21.15.60:0/1883096994 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980319d0 con 0x7fc2a80b5a60 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1883096994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 msgr2=0x7fc2a80d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a80d27f0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc298009e50 tx=0x7fc29802a710 comp rx=0 tx=0).stop 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1883096994 shutdown_connections 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a80d27f0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1883096994 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a80b6cb0 0x7fc2a80b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1883096994 >> 172.21.15.60:0/1883096994 conn(0x7fc2a80d0520 msgr2=0x7fc2a80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1883096994 shutdown_connections 2024-04-25T08:39:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.621+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1883096994 wait complete. 2024-04-25T08:39:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.622+0000 7fc2ad09a640 1 Processor -- start 2024-04-25T08:39:18.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 -- start start 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a81f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80b6cb0 0x7fc2a8160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a815db70 0x7fc2a80c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80cf1c0 con 0x7fc2a80b5a60 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80ce530 con 0x7fc2a80b6cb0 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2ad09a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80cf340 con 0x7fc2a815db70 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2a6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a81f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2a6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a81f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57214/0 (socket says 172.21.15.60:57214) 2024-04-25T08:39:18.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.623+0000 7fc2a6d9d640 1 -- 172.21.15.60:0/1211631776 learned_addr learned my addr 172.21.15.60:0/1211631776 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a659c640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80b6cb0 0x7fc2a8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a759e640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a815db70 0x7fc2a80c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a6d9d640 1 -- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a815db70 msgr2=0x7fc2a80c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a6d9d640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a815db70 0x7fc2a80c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a6d9d640 1 -- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80b6cb0 msgr2=0x7fc2a8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a6d9d640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80b6cb0 0x7fc2a8160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a6d9d640 1 -- 172.21.15.60:0/1211631776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8160e80 con 0x7fc2a80b5a60 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a659c640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80b6cb0 0x7fc2a8160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2a6d9d640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a81f4880 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c400 tx=0x7fc2a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00052d0 con 0x7fc2a80b5a60 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0005490 con 0x7fc2a80b5a60 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a81f39c0 con 0x7fc2a80b5a60 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0007260 con 0x7fc2a80b5a60 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.624+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81f3f80 con 0x7fc2a80b5a60 2024-04-25T08:39:18.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.625+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a8073990 con 0x7fc2a80b5a60 2024-04-25T08:39:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.625+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2a0007400 con 0x7fc2a80b5a60 2024-04-25T08:39:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.626+0000 7fc29f7fe640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc28806b4d0 0x7fc28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.626+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc2a00895d0 con 0x7fc2a80b5a60 2024-04-25T08:39:18.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.626+0000 7fc2a659c640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc28806b4d0 0x7fc28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.627+0000 7fc2a659c640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc28806b4d0 0x7fc28806d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc2a81f7db0 tx=0x7fc290008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:18.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.630+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2a0058600 con 0x7fc2a80b5a60 2024-04-25T08:39:18.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.814+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2a80d1820 con 0x7fc2a80b5a60 2024-04-25T08:39:18.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.814+0000 7fc29f7fe640 1 -- 172.21.15.60:0/1211631776 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+13419 (secure 0 0 0) 0x7fc2a0057f00 con 0x7fc2a80b5a60 2024-04-25T08:39:18.808 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:18.808 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":138,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:17.494219+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6834/313583008":"2024-04-26T08:39:14.486236+0000","172.21.15.60:6832/313583008":"2024-04-26T08:39:14.486236+0000","172.21.15.100:6836/879630460":"2024-04-26T08:39:04.147468+0000","172.21.15.100:6834/879630460":"2024-04-26T08:39:04.147468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:18.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.816+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc28806b4d0 msgr2=0x7fc28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.816+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc28806b4d0 0x7fc28806d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc2a81f7db0 tx=0x7fc290008040 comp rx=0 tx=0).stop 2024-04-25T08:39:18.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.816+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 msgr2=0x7fc2a81f4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.816+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a81f4880 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c400 tx=0x7fc2a000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 shutdown_connections 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc2a815db70 0x7fc2a80c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc28806b4d0 0x7fc28806d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2a80b6cb0 0x7fc2a8160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 --2- 172.21.15.60:0/1211631776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc2a80b5a60 0x7fc2a81f4880 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 >> 172.21.15.60:0/1211631776 conn(0x7fc2a80d0520 msgr2=0x7fc2a80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 shutdown_connections 2024-04-25T08:39:18.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.817+0000 7fc2ad09a640 1 -- 172.21.15.60:0/1211631776 wait complete. 2024-04-25T08:39:18.821 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:18.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:18.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.932+0000 7fbb36c4d640 1 Processor -- start 2024-04-25T08:39:18.926 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.932+0000 7fbb36c4d640 1 -- start start 2024-04-25T08:39:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.933+0000 7fbb36c4d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbb3006c330 0x7fbb300b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.933+0000 7fbb36c4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c7e0 0x7fbb300b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.933+0000 7fbb36c4d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300d1d50 con 0x7fbb30158230 2024-04-25T08:39:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.933+0000 7fbb36c4d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb30158610 con 0x7fbb3006c330 2024-04-25T08:39:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.933+0000 7fbb36c4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006bea0 con 0x7fbb3006c7e0 2024-04-25T08:39:18.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb2ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbb3006c330 0x7fbb300b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb351ea640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c7e0 0x7fbb300b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb349e9640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbb30158230 0x7fbb300d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33010/0 (socket says 172.21.15.60:33010) 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb349e9640 1 -- 172.21.15.60:0/3403017363 learned_addr learned my addr 172.21.15.60:0/3403017363 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb351ea640 1 -- 172.21.15.60:0/3403017363 >> v2:172.21.15.100:3300/0 conn(0x7fbb3006c330 msgr2=0x7fbb300b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb351ea640 1 --2- 172.21.15.60:0/3403017363 >> v2:172.21.15.100:3300/0 conn(0x7fbb3006c330 0x7fbb300b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb351ea640 1 -- 172.21.15.60:0/3403017363 >> v1:172.21.15.100:6789/0 conn(0x7fbb30158230 legacy=0x7fbb300d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb351ea640 1 -- 172.21.15.60:0/3403017363 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb300d2020 con 0x7fbb3006c7e0 2024-04-25T08:39:18.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.934+0000 7fbb2ffff640 1 --2- 172.21.15.60:0/3403017363 >> v2:172.21.15.100:3300/0 conn(0x7fbb3006c330 0x7fbb300b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:18.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb351ea640 1 --2- 172.21.15.60:0/3403017363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c7e0 0x7fbb300b53a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097a0 tx=0x7fbb1802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83cb56a6262050b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb2f7fe640 1 -- 172.21.15.60:0/3403017363 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18037070 con 0x7fbb3006c7e0 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb2f7fe640 1 -- 172.21.15.60:0/3403017363 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18032440 con 0x7fbb3006c7e0 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb2f7fe640 1 -- 172.21.15.60:0/3403017363 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18032810 con 0x7fbb3006c7e0 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb36c4d640 1 -- 172.21.15.60:0/3403017363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c7e0 msgr2=0x7fbb300b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/3403017363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c7e0 0x7fbb300b53a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb180097a0 tx=0x7fbb1802a180 comp rx=0 tx=0).stop 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb36c4d640 1 -- 172.21.15.60:0/3403017363 shutdown_connections 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/3403017363 >> v2:172.21.15.100:3300/0 conn(0x7fbb3006c330 0x7fbb300b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/3403017363 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c7e0 0x7fbb300b53a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.935+0000 7fbb36c4d640 1 -- 172.21.15.60:0/3403017363 >> 172.21.15.60:0/3403017363 conn(0x7fbb30061600 msgr2=0x7fbb300619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.936+0000 7fbb36c4d640 1 -- 172.21.15.60:0/3403017363 shutdown_connections 2024-04-25T08:39:18.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.936+0000 7fbb36c4d640 1 -- 172.21.15.60:0/3403017363 wait complete. 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.937+0000 7fbb36c4d640 1 Processor -- start 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.937+0000 7fbb36c4d640 1 -- start start 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c330 0x7fbb301520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 0x7fbb300b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb30158230 0x7fbb300b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300d25e0 con 0x7fbb3006c7e0 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006bb30 con 0x7fbb30158230 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3015e320 con 0x7fbb3006c330 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 0x7fbb300b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 0x7fbb300b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57224/0 (socket says 172.21.15.60:57224) 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 -- 172.21.15.60:0/1440591125 learned_addr learned my addr 172.21.15.60:0/1440591125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb351ea640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb30158230 0x7fbb300b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb349e9640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c330 0x7fbb301520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 -- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c330 msgr2=0x7fbb301520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c330 0x7fbb301520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 -- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb30158230 msgr2=0x7fbb300b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb30158230 0x7fbb300b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 -- 172.21.15.60:0/1440591125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30158850 con 0x7fbb3006c7e0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb351ea640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb30158230 0x7fbb300b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2ffff640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 0x7fbb300b5940 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000e9b0 tx=0x7fbb2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20005990 con 0x7fbb3006c7e0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20005b30 con 0x7fbb3006c7e0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb30158b30 con 0x7fbb3006c7e0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.938+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2000c9a0 con 0x7fbb3006c7e0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.939+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb301590f0 con 0x7fbb3006c7e0 2024-04-25T08:39:18.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.939+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb30073a00 con 0x7fbb3006c7e0 2024-04-25T08:39:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.940+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb2000cb40 con 0x7fbb3006c7e0 2024-04-25T08:39:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.940+0000 7fbb2d7fa640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb1006b500 0x7fbb1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.940+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fbb2008a510 con 0x7fbb3006c7e0 2024-04-25T08:39:18.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.941+0000 7fbb349e9640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb1006b500 0x7fbb1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:18.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.941+0000 7fbb349e9640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb1006b500 0x7fbb1006d9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbb280012e0 tx=0x7fbb28009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:18.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:18.945+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb20017020 con 0x7fbb3006c7e0 2024-04-25T08:39:19.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.122+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb30061b80 con 0x7fbb3006c7e0 2024-04-25T08:39:19.116 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.123+0000 7fbb2d7fa640 1 -- 172.21.15.60:0/1440591125 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb20059410 con 0x7fbb3006c7e0 2024-04-25T08:39:19.117 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:39:19.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb1006b500 msgr2=0x7fbb1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:19.121 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb1006b500 0x7fbb1006d9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fbb280012e0 tx=0x7fbb28009040 comp rx=0 tx=0).stop 2024-04-25T08:39:19.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 msgr2=0x7fbb300b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:19.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 0x7fbb300b5940 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000e9b0 tx=0x7fbb2000ee70 comp rx=0 tx=0).stop 2024-04-25T08:39:19.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 shutdown_connections 2024-04-25T08:39:19.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fbb1006b500 0x7fbb1006d9c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbb30158230 0x7fbb300b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.122 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbb3006c7e0 0x7fbb300b5940 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 --2- 172.21.15.60:0/1440591125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbb3006c330 0x7fbb301520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 >> 172.21.15.60:0/1440591125 conn(0x7fbb30061600 msgr2=0x7fbb300ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:19.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 shutdown_connections 2024-04-25T08:39:19.123 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.125+0000 7fbb36c4d640 1 -- 172.21.15.60:0/1440591125 wait complete. 2024-04-25T08:39:19.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T08:39:19.230 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.237+0000 7fd662bb6640 1 Processor -- start 2024-04-25T08:39:19.231 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.237+0000 7fd662bb6640 1 -- start start 2024-04-25T08:39:19.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd662bb6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d0170 0x7fd65c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:19.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd662bb6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd65c0ce700 0x7fd65c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:19.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd662bb6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65c0c8030 con 0x7fd65c15da70 2024-04-25T08:39:19.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd662bb6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd65c0ceb50 con 0x7fd65c0ce700 2024-04-25T08:39:19.233 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd662bb6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c0c8310 con 0x7fd65c0d0170 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd661153640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd65c0ce700 0x7fd65c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd660952640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd65c15da70 0x7fd65c0c7730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50996/0 (socket says 172.21.15.60:50996) 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd660952640 1 -- 172.21.15.60:0/2883655320 learned_addr learned my addr 172.21.15.60:0/2883655320 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd65bfff640 1 --2- 172.21.15.60:0/2883655320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d0170 0x7fd65c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd661153640 1 -- 172.21.15.60:0/2883655320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d0170 msgr2=0x7fd65c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd661153640 1 --2- 172.21.15.60:0/2883655320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d0170 0x7fd65c0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd661153640 1 -- 172.21.15.60:0/2883655320 >> v1:172.21.15.60:6789/0 conn(0x7fd65c15da70 legacy=0x7fd65c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd661153640 1 -- 172.21.15.60:0/2883655320 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0c85f0 con 0x7fd65c0ce700 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.238+0000 7fd65b7fe640 1 -- 172.21.15.60:0/2883655320 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342753452 0 0) 0x7fd65c0c8030 con 0x7fd65c15da70 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.239+0000 7fd661153640 1 --2- 172.21.15.60:0/2883655320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0ce700 0x7fd65c06c330 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd654009d90 tx=0x7fd65402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d0e4fbf392f320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:19.234 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.239+0000 7fd65b7fe640 1 -- 172.21.15.60:0/2883655320 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654007740 con 0x7fd65c0ce700 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.239+0000 7fd65b7fe640 1 -- 172.21.15.60:0/2883655320 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6540078e0 con 0x7fd65c0ce700 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.239+0000 7fd65b7fe640 1 -- 172.21.15.60:0/2883655320 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654007cb0 con 0x7fd65c0ce700 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.239+0000 7fd662bb6640 1 -- 172.21.15.60:0/2883655320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0ce700 msgr2=0x7fd65c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.239+0000 7fd662bb6640 1 --2- 172.21.15.60:0/2883655320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0ce700 0x7fd65c06c330 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fd654009d90 tx=0x7fd65402a660 comp rx=0 tx=0).stop 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.240+0000 7fd662bb6640 1 -- 172.21.15.60:0/2883655320 shutdown_connections 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.240+0000 7fd662bb6640 1 --2- 172.21.15.60:0/2883655320 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c0ce700 0x7fd65c06c330 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.240+0000 7fd662bb6640 1 --2- 172.21.15.60:0/2883655320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0d0170 0x7fd65c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.240+0000 7fd662bb6640 1 -- 172.21.15.60:0/2883655320 >> 172.21.15.60:0/2883655320 conn(0x7fd65c0b33e0 msgr2=0x7fd65c0b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.240+0000 7fd662bb6640 1 -- 172.21.15.60:0/2883655320 shutdown_connections 2024-04-25T08:39:19.235 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.240+0000 7fd662bb6640 1 -- 172.21.15.60:0/2883655320 wait complete. 2024-04-25T08:39:19.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.241+0000 7fd662bb6640 1 Processor -- start 2024-04-25T08:39:19.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.241+0000 7fd662bb6640 1 -- start start 2024-04-25T08:39:19.238 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd662bb6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0ce700 0x7fd65c160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd662bb6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0d0170 0x7fd65c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd662bb6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 0x7fd65c0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd662bb6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c15de50 con 0x7fd65c15da70 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd662bb6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c06be10 con 0x7fd65c0d0170 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd662bb6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c06c870 con 0x7fd65c0ce700 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 0x7fd65c0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 0x7fd65c0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57238/0 (socket says 172.21.15.60:57238) 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 -- 172.21.15.60:0/3509825190 learned_addr learned my addr 172.21.15.60:0/3509825190 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd660952640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0ce700 0x7fd65c160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd65bfff640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0d0170 0x7fd65c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 -- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0ce700 msgr2=0x7fd65c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0ce700 0x7fd65c160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 -- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0d0170 msgr2=0x7fd65c147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0d0170 0x7fd65c147070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 -- 172.21.15.60:0/3509825190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c159ce0 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd65bfff640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0d0170 0x7fd65c147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd661153640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 0x7fd65c0caec0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd654009eb0 tx=0x7fd654004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.242+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654040070 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.243+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654031bc0 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.243+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c159fc0 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.243+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654009450 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.243+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c15a580 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.244+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c06de80 con 0x7fd65c15da70 2024-04-25T08:39:19.240 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.244+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd654039040 con 0x7fd65c15da70 2024-04-25T08:39:19.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.244+0000 7fd6597fa640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd64006b4d0 0x7fd64006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:19.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.245+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd6540ab900 con 0x7fd65c15da70 2024-04-25T08:39:19.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.245+0000 7fd660952640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd64006b4d0 0x7fd64006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:19.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.245+0000 7fd660952640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd64006b4d0 0x7fd64006d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd650000f60 tx=0x7fd650007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:19.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.249+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd65407a8a0 con 0x7fd65c15da70 2024-04-25T08:39:19.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:19.499+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7fd65c073990 con 0x7fd65c15da70 2024-04-25T08:39:20.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.504+0000 7fd6597fa640 1 -- 172.21.15.60:0/3509825190 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v490) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd65407a1a0 con 0x7fd65c15da70 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd64006b4d0 msgr2=0x7fd64006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd64006b4d0 0x7fd64006d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd650000f60 tx=0x7fd650007c00 comp rx=0 tx=0).stop 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 msgr2=0x7fd65c0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 0x7fd65c0caec0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fd654009eb0 tx=0x7fd654004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 shutdown_connections 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd65c15da70 0x7fd65c0caec0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd64006b4d0 0x7fd64006d990 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd65c0d0170 0x7fd65c147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 --2- 172.21.15.60:0/3509825190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd65c0ce700 0x7fd65c160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 >> 172.21.15.60:0/3509825190 conn(0x7fd65c0b33e0 msgr2=0x7fd65c15c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 shutdown_connections 2024-04-25T08:39:20.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.507+0000 7fd662bb6640 1 -- 172.21.15.60:0/3509825190 wait complete. 2024-04-25T08:39:20.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:39:20.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.630+0000 7f3f5a1ce640 1 Processor -- start 2024-04-25T08:39:20.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.630+0000 7f3f5a1ce640 1 -- start start 2024-04-25T08:39:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.631+0000 7f3f5a1ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415bb50 0x7f3f540c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.631+0000 7f3f5a1ce640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f540cfd10 0x7f3f540cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:20.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.631+0000 7f3f5a1ce640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f540c75e0 con 0x7f3f540c70d0 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.631+0000 7f3f5a1ce640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540c8680 con 0x7f3f540cfd10 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.631+0000 7f3f5a1ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540c78c0 con 0x7f3f5415bb50 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f537fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415bb50 0x7f3f540c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f53fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f540cfd10 0x7f3f540cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f52ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3f540c70d0 0x7f3f540cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:33020/0 (socket says 172.21.15.60:33020) 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f52ffd640 1 -- 172.21.15.60:0/2413633535 learned_addr learned my addr 172.21.15.60:0/2413633535 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f527fc640 1 -- 172.21.15.60:0/2413633535 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169870943 0 0) 0x7f3f540c75e0 con 0x7f3f540c70d0 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f527fc640 1 -- 172.21.15.60:0/2413633535 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f3c003520 con 0x7f3f540c70d0 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f537fe640 1 -- 172.21.15.60:0/2413633535 >> v2:172.21.15.100:3300/0 conn(0x7f3f540cfd10 msgr2=0x7f3f540cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f537fe640 1 --2- 172.21.15.60:0/2413633535 >> v2:172.21.15.100:3300/0 conn(0x7f3f540cfd10 0x7f3f540cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f537fe640 1 -- 172.21.15.60:0/2413633535 >> v1:172.21.15.100:6789/0 conn(0x7f3f540c70d0 legacy=0x7f3f540cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.632+0000 7f3f537fe640 1 -- 172.21.15.60:0/2413633535 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540c7ba0 con 0x7f3f5415bb50 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.633+0000 7f3f537fe640 1 --2- 172.21.15.60:0/2413633535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415bb50 0x7f3f540c6b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097a0 tx=0x7f3f4402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f38531298e4609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.633+0000 7f3f527fc640 1 -- 172.21.15.60:0/2413633535 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44037070 con 0x7f3f5415bb50 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.633+0000 7f3f527fc640 1 -- 172.21.15.60:0/2413633535 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44032440 con 0x7f3f5415bb50 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.633+0000 7f3f527fc640 1 -- 172.21.15.60:0/2413633535 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44032810 con 0x7f3f5415bb50 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/2413633535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415bb50 msgr2=0x7f3f540c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/2413633535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415bb50 0x7f3f540c6b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097a0 tx=0x7f3f4402a210 comp rx=0 tx=0).stop 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/2413633535 shutdown_connections 2024-04-25T08:39:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/2413633535 >> v2:172.21.15.100:3300/0 conn(0x7f3f540cfd10 0x7f3f540cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/2413633535 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f5415bb50 0x7f3f540c6b90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/2413633535 >> 172.21.15.60:0/2413633535 conn(0x7f3f5406f500 msgr2=0x7f3f5406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/2413633535 shutdown_connections 2024-04-25T08:39:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.634+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/2413633535 wait complete. 2024-04-25T08:39:20.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.635+0000 7f3f5a1ce640 1 Processor -- start 2024-04-25T08:39:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.635+0000 7f3f5a1ce640 1 -- start start 2024-04-25T08:39:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f5a1ce640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f540c70d0 0x7f3f54146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f5a1ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f540cfd10 0x7f3f5415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f5a1ce640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 0x7f3f5415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f5a1ce640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5415f6d0 con 0x7f3f5415bb50 2024-04-25T08:39:20.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f5a1ce640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540c8160 con 0x7f3f540cfd10 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f5a1ce640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5415f850 con 0x7f3f540c70d0 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 0x7f3f5415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 0x7f3f5415c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57242/0 (socket says 172.21.15.60:57242) 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 -- 172.21.15.60:0/3452286217 learned_addr learned my addr 172.21.15.60:0/3452286217 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f540c70d0 0x7f3f54146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f52ffd640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f540cfd10 0x7f3f5415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 -- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f540c70d0 msgr2=0x7f3f54146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f540c70d0 0x7f3f54146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 -- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f540cfd10 msgr2=0x7f3f5415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f540cfd10 0x7f3f5415c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 -- 172.21.15.60:0/3452286217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f541467d0 con 0x7f3f5415bb50 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.636+0000 7f3f53fff640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 0x7f3f5415c6e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00efc0 tx=0x7f3f4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.637+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c018020 con 0x7f3f5415bb50 2024-04-25T08:39:20.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.637+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c014480 con 0x7f3f5415bb50 2024-04-25T08:39:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.637+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540b6f40 con 0x7f3f5415bb50 2024-04-25T08:39:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.637+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c014870 con 0x7f3f5415bb50 2024-04-25T08:39:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.637+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f540c9820 con 0x7f3f5415bb50 2024-04-25T08:39:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.638+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f540d0e50 con 0x7f3f5415bb50 2024-04-25T08:39:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.638+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f4c005d10 con 0x7f3f5415bb50 2024-04-25T08:39:20.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.639+0000 7f3f3bfff640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f4006b4d0 0x7f3f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.639+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3f4c08ada0 con 0x7f3f5415bb50 2024-04-25T08:39:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.639+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f4006b4d0 0x7f3f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:20.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.639+0000 7f3f537fe640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f4006b4d0 0x7f3f4006d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3f44034000 tx=0x7f3f4402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:20.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.643+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f540d0e50 con 0x7f3f5415bb50 2024-04-25T08:39:20.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.822+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3f540cf150 con 0x7f3f5415bb50 2024-04-25T08:39:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.825+0000 7f3f3bfff640 1 -- 172.21.15.60:0/3452286217 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3f4c022030 con 0x7f3f5415bb50 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f4006b4d0 msgr2=0x7f3f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f4006b4d0 0x7f3f4006d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3f44034000 tx=0x7f3f4402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 msgr2=0x7f3f5415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 0x7f3f5415c6e0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00efc0 tx=0x7f3f4c00c490 comp rx=0 tx=0).stop 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 shutdown_connections 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3f5415bb50 0x7f3f5415c6e0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3f4006b4d0 0x7f3f4006d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f540cfd10 0x7f3f5415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 --2- 172.21.15.60:0/3452286217 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3f540c70d0 0x7f3f54146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 >> 172.21.15.60:0/3452286217 conn(0x7f3f5406f500 msgr2=0x7f3f540b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.827+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 shutdown_connections 2024-04-25T08:39:20.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:20.828+0000 7f3f5a1ce640 1 -- 172.21.15.60:0/3452286217 wait complete. 2024-04-25T08:39:20.832 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:39:20.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:20.869 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:20.917 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:39:20.917 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:20.917 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:20.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:20.932 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:20.932 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:20.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:20.949 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:20.949 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:20.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:20.987 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:20.988 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:21.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:21.006 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:39:21.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:21.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.134+0000 7ff069d53640 1 Processor -- start 2024-04-25T08:39:21.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.134+0000 7ff069d53640 1 -- start start 2024-04-25T08:39:21.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff069d53640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff0640d2e90 0x7ff0640d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff069d53640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0640d0900 0x7ff0640b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff069d53640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640b9940 con 0x7ff06415dd30 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff069d53640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640d0d40 con 0x7ff0640d0900 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff069d53640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640d1df0 con 0x7ff0640d2e90 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0640d0900 0x7ff0640b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ff0640d0900 0x7ff0640b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57254/0 (socket says 172.21.15.60:57254) 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 -- 172.21.15.60:0/2724858427 learned_addr learned my addr 172.21.15.60:0/2724858427 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 -- 172.21.15.60:0/2724858427 >> v2:172.21.15.100:3300/0 conn(0x7ff0640d2e90 msgr2=0x7ff0640d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff062ffd640 1 --2- 172.21.15.60:0/2724858427 >> v2:172.21.15.100:3300/0 conn(0x7ff0640d2e90 0x7ff0640d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 --2- 172.21.15.60:0/2724858427 >> v2:172.21.15.100:3300/0 conn(0x7ff0640d2e90 0x7ff0640d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 -- 172.21.15.60:0/2724858427 >> v1:172.21.15.100:6789/0 conn(0x7ff06415dd30 legacy=0x7ff0640b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.135+0000 7ff063fff640 1 -- 172.21.15.60:0/2724858427 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640d20d0 con 0x7ff0640d0900 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.136+0000 7ff063fff640 1 --2- 172.21.15.60:0/2724858427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0640b6a40 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7ff0640b6540 tx=0x7ff05c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3a1711029c641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.136+0000 7ff0627fc640 1 -- 172.21.15.60:0/2724858427 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c036070 con 0x7ff0640d0900 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.136+0000 7ff0627fc640 1 -- 172.21.15.60:0/2724858427 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c031600 con 0x7ff0640d0900 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.136+0000 7ff0627fc640 1 -- 172.21.15.60:0/2724858427 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0319d0 con 0x7ff0640d0900 2024-04-25T08:39:21.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 -- 172.21.15.60:0/2724858427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 msgr2=0x7ff0640b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 --2- 172.21.15.60:0/2724858427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0640b6a40 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7ff0640b6540 tx=0x7ff05c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 -- 172.21.15.60:0/2724858427 shutdown_connections 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 --2- 172.21.15.60:0/2724858427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0640b6a40 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 --2- 172.21.15.60:0/2724858427 >> v2:172.21.15.100:3300/0 conn(0x7ff0640d2e90 0x7ff0640d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 -- 172.21.15.60:0/2724858427 >> 172.21.15.60:0/2724858427 conn(0x7ff064061c30 msgr2=0x7ff06406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 -- 172.21.15.60:0/2724858427 shutdown_connections 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.137+0000 7ff069d53640 1 -- 172.21.15.60:0/2724858427 wait complete. 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.138+0000 7ff069d53640 1 Processor -- start 2024-04-25T08:39:21.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.138+0000 7ff069d53640 1 -- start start 2024-04-25T08:39:21.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff069d53640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0641f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff069d53640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0640d2e90 0x7ff0641f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff069d53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06415dd30 0x7ff06414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff069d53640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d2970 con 0x7ff0640d0900 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff069d53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d2690 con 0x7ff06415dd30 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff069d53640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640d1a10 con 0x7ff0640d2e90 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0641f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0641f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57270/0 (socket says 172.21.15.60:57270) 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 -- 172.21.15.60:0/864083985 learned_addr learned my addr 172.21.15.60:0/864083985 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff063fff640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06415dd30 0x7ff06414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff062ffd640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0640d2e90 0x7ff0641f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 -- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0640d2e90 msgr2=0x7ff0641f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0640d2e90 0x7ff0641f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 -- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06415dd30 msgr2=0x7ff06414e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06415dd30 0x7ff06414e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 -- 172.21.15.60:0/864083985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06414f040 con 0x7ff0640d0900 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.139+0000 7ff0637fe640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0641f7350 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b3f0 tx=0x7ff05400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:21.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.140+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054015020 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.140+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054011830 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.140+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06414f320 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.140+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054011c20 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.140+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0641f6e30 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.141+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff064073990 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.141+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff054018020 con 0x7ff0640d0900 2024-04-25T08:39:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.141+0000 7ff05bfff640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff04406b4d0 0x7ff04406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.142+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff054089c10 con 0x7ff0640d0900 2024-04-25T08:39:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.142+0000 7ff062ffd640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff04406b4d0 0x7ff04406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.142+0000 7ff062ffd640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff04406b4d0 0x7ff04406d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff0640b6200 tx=0x7ff04c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:21.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.146+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff05401f030 con 0x7ff0640d0900 2024-04-25T08:39:21.354 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.361+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0641f7070 con 0x7ff0640d0900 2024-04-25T08:39:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.361+0000 7ff05bfff640 1 -- 172.21.15.60:0/864083985 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+2911 (secure 0 0 0) 0x7ff054058550 con 0x7ff0640d0900 2024-04-25T08:39:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:21.355 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/2196553149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6833","nonce":2196553149},{"type":"v1","addr":"172.21.15.60:6835","nonce":2196553149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/1174913732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6835","nonce":1174913732},{"type":"v1","addr":"172.21.15.100:6837","nonce":1174913732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":9843,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6834/2351850133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2351850133},{"type":"v1","addr":"172.21.15.60:6834","nonce":2351850133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[]} 2024-04-25T08:39:21.355 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 84 2024-04-25T08:39:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff04406b4d0 msgr2=0x7ff04406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff04406b4d0 0x7ff04406d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff0640b6200 tx=0x7ff04c008040 comp rx=0 tx=0).stop 2024-04-25T08:39:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 msgr2=0x7ff0641f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0641f7350 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b3f0 tx=0x7ff05400b8b0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.357 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 shutdown_connections 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ff04406b4d0 0x7ff04406d990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff06415dd30 0x7ff06414e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0640d2e90 0x7ff0641f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 --2- 172.21.15.60:0/864083985 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff0640d0900 0x7ff0641f7350 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 >> 172.21.15.60:0/864083985 conn(0x7ff064061c30 msgr2=0x7ff0640c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 shutdown_connections 2024-04-25T08:39:21.358 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.363+0000 7ff069d53640 1 -- 172.21.15.60:0/864083985 wait complete. 2024-04-25T08:39:21.368 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:39:21.368 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:39:21.369 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:39:21.369 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:21.369 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:39:21.369 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:39:21.369 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:39:21.370 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:21.370 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:39:21.370 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:39:21.370 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:39:21.370 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:21.374 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:39:21.374 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:39:21.376 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:39:21.377 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:21.378 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:39:21.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:39:21.379 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:39:21.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:39:21.382 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:39:21.382 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:39:21.421 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:39:21.421 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:21.422 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:39:21.422 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:21.424 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:39:21.425 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:21.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:39:21.457 INFO:tasks.ceph.mds.d.smithi100.stdout:starting mds.d at 2024-04-25T08:39:21.496 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:39:21.497 INFO:tasks.ceph.mds.c.smithi060.stdout:starting mds.c at 2024-04-25T08:39:21.503 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:39:21.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.538+0000 7fe5501ab640 1 Processor -- start 2024-04-25T08:39:21.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.538+0000 7fe5501ab640 1 -- start start 2024-04-25T08:39:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe5501ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5480c6010 0x7fe5480ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe5501ab640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe5480cd520 0x7fe5480cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe5501ab640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5480c6940 con 0x7fe54815a9f0 2024-04-25T08:39:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe5501ab640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5480b0020 con 0x7fe5480cd520 2024-04-25T08:39:21.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe5501ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480cb5a0 con 0x7fe5480c6010 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe54df47640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe54815a9f0 0x7fe5480cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:51016/0 (socket says 172.21.15.60:51016) 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe54df47640 1 -- 172.21.15.60:0/1329637898 learned_addr learned my addr 172.21.15.60:0/1329637898 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe54e748640 1 --2- 172.21.15.60:0/1329637898 >> v2:172.21.15.100:3300/0 conn(0x7fe5480cd520 0x7fe5480cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.539+0000 7fe54d746640 1 --2- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5480c6010 0x7fe5480ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54cf45640 1 -- 172.21.15.60:0/1329637898 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166596608 0 0) 0x7fe538001110 con 0x7fe54815a9f0 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54cf45640 1 -- 172.21.15.60:0/1329637898 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe534005730 con 0x7fe54815a9f0 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54e748640 1 -- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5480c6010 msgr2=0x7fe5480ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54e748640 1 --2- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5480c6010 0x7fe5480ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54e748640 1 -- 172.21.15.60:0/1329637898 >> v1:172.21.15.60:6789/0 conn(0x7fe54815a9f0 legacy=0x7fe5480cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54e748640 1 -- 172.21.15.60:0/1329637898 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54815add0 con 0x7fe5480cd520 2024-04-25T08:39:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.540+0000 7fe54e748640 1 --2- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5480cba30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe540009d90 tx=0x7fe54002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875237f23067584d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe54cf45640 1 -- 172.21.15.60:0/1329637898 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540007740 con 0x7fe5480cd520 2024-04-25T08:39:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe54cf45640 1 -- 172.21.15.60:0/1329637898 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5400078e0 con 0x7fe5480cd520 2024-04-25T08:39:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe54cf45640 1 -- 172.21.15.60:0/1329637898 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540007cb0 con 0x7fe5480cd520 2024-04-25T08:39:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 -- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 msgr2=0x7fe5480cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5480cba30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe540009d90 tx=0x7fe54002a660 comp rx=0 tx=0).stop 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 -- 172.21.15.60:0/1329637898 shutdown_connections 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5480cba30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1329637898 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5480c6010 0x7fe5480ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 -- 172.21.15.60:0/1329637898 >> 172.21.15.60:0/1329637898 conn(0x7fe5480b2390 msgr2=0x7fe5480b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 -- 172.21.15.60:0/1329637898 shutdown_connections 2024-04-25T08:39:21.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.541+0000 7fe5501ab640 1 -- 172.21.15.60:0/1329637898 wait complete. 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.542+0000 7fe5501ab640 1 Processor -- start 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.542+0000 7fe5501ab640 1 -- start start 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe5501ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 0x7fe548148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe5501ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5481495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe5501ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815a9f0 0x7fe54814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe5501ab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480cb230 con 0x7fe5480c6010 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe5501ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480caf50 con 0x7fe5480cd520 2024-04-25T08:39:21.538 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe5501ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5480cbf70 con 0x7fe54815a9f0 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe54df47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 0x7fe548148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe54df47640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 0x7fe548148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:57290/0 (socket says 172.21.15.60:57290) 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe54df47640 1 -- 172.21.15.60:0/1117913048 learned_addr learned my addr 172.21.15.60:0/1117913048 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe54e748640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815a9f0 0x7fe54814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.543+0000 7fe54d746640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5481495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54df47640 1 -- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815a9f0 msgr2=0x7fe54814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54df47640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815a9f0 0x7fe54814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54df47640 1 -- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 msgr2=0x7fe5481495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54df47640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5481495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54df47640 1 -- 172.21.15.60:0/1117913048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480c9fb0 con 0x7fe5480c6010 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54d746640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5481495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe54df47640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 0x7fe548148ed0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c3a0 tx=0x7fe53800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538017070 con 0x7fe5480c6010 2024-04-25T08:39:21.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538005ce0 con 0x7fe5480c6010 2024-04-25T08:39:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538007310 con 0x7fe5480c6010 2024-04-25T08:39:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5480ca290 con 0x7fe5480c6010 2024-04-25T08:39:21.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.544+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5480ae010 con 0x7fe5480c6010 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.545+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe548073a00 con 0x7fe5480c6010 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.546+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe53801a070 con 0x7fe5480c6010 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.546+0000 7fe53e7fc640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe52c06b4d0 0x7fe52c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.546+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe538089470 con 0x7fe5480c6010 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.546+0000 7fe54d746640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe52c06b4d0 0x7fe52c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.547+0000 7fe54d746640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe52c06b4d0 0x7fe52c06d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe5480c7150 tx=0x7fe544008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:21.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.550+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5480c6940 con 0x7fe5480c6010 2024-04-25T08:39:21.738 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:21.744+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe5480cf3f0 con 0x7fe5480c6010 2024-04-25T08:39:22.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.511+0000 7fe53e7fc640 1 -- 172.21.15.60:0/1117913048 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v140) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe538057dc0 con 0x7fe5480c6010 2024-04-25T08:39:22.504 INFO:teuthology.orchestra.run.smithi060.stderr: removed all blocklist entries 2024-04-25T08:39:22.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe52c06b4d0 msgr2=0x7fe52c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe52c06b4d0 0x7fe52c06d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe5480c7150 tx=0x7fe544008040 comp rx=0 tx=0).stop 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 msgr2=0x7fe548148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 0x7fe548148ed0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c3a0 tx=0x7fe53800c860 comp rx=0 tx=0).stop 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 shutdown_connections 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe54815a9f0 0x7fe54814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe52c06b4d0 0x7fe52c06d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5480cd520 0x7fe5481495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 --2- 172.21.15.60:0/1117913048 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5480c6010 0x7fe548148ed0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 >> 172.21.15.60:0/1117913048 conn(0x7fe5480b2390 msgr2=0x7fe5480cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 shutdown_connections 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.514+0000 7fe5501ab640 1 -- 172.21.15.60:0/1117913048 wait complete. 2024-04-25T08:39:22.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:39:22.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.638+0000 7fc8f6038640 1 Processor -- start 2024-04-25T08:39:22.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.638+0000 7fc8f6038640 1 -- start start 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8f6038640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8f015da70 0x7fc8f00d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8f6038640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f00c8030 con 0x7fc8f00c7730 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8f6038640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f00c8310 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8f6038640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f00c85f0 con 0x7fc8f015da70 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8ef7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8f015da70 0x7fc8f00d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8eeffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fc8f00c7730 0x7fc8f00ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34480/0 (socket says 172.21.15.60:34480) 2024-04-25T08:39:22.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8ef7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fc8f015da70 0x7fc8f00d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52178/0 (socket says 172.21.15.60:52178) 2024-04-25T08:39:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8eeffd640 1 -- 172.21.15.60:0/251026170 learned_addr learned my addr 172.21.15.60:0/251026170 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8ef7fe640 1 -- 172.21.15.60:0/251026170 >> v1:172.21.15.100:6789/0 conn(0x7fc8f00ce8d0 legacy=0x7fc8f006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8ef7fe640 1 -- 172.21.15.60:0/251026170 >> v1:172.21.15.60:6789/0 conn(0x7fc8f00c7730 legacy=0x7fc8f00ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.640+0000 7fc8ef7fe640 1 -- 172.21.15.60:0/251026170 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f006bb30 con 0x7fc8f015da70 2024-04-25T08:39:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.639+0000 7fc8ee7fc640 1 -- 172.21.15.60:0/251026170 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014813206 0 0) 0x7fc8f00c8030 con 0x7fc8f00c7730 2024-04-25T08:39:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.640+0000 7fc8ef7fe640 1 --2- 172.21.15.60:0/251026170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f015da70 0x7fc8f00d0170 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00097a0 tx=0x7fc8e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f920080d68a495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.640+0000 7fc8ee7fc640 1 -- 172.21.15.60:0/251026170 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0037070 con 0x7fc8f015da70 2024-04-25T08:39:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.640+0000 7fc8ee7fc640 1 -- 172.21.15.60:0/251026170 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e0032440 con 0x7fc8f015da70 2024-04-25T08:39:22.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.640+0000 7fc8ee7fc640 1 -- 172.21.15.60:0/251026170 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0032810 con 0x7fc8f015da70 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 -- 172.21.15.60:0/251026170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f015da70 msgr2=0x7fc8f00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 --2- 172.21.15.60:0/251026170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f015da70 0x7fc8f00d0170 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00097a0 tx=0x7fc8e002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 -- 172.21.15.60:0/251026170 shutdown_connections 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 --2- 172.21.15.60:0/251026170 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f015da70 0x7fc8f00d0170 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 -- 172.21.15.60:0/251026170 >> 172.21.15.60:0/251026170 conn(0x7fc8f00b33e0 msgr2=0x7fc8f00b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 -- 172.21.15.60:0/251026170 shutdown_connections 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.641+0000 7fc8f6038640 1 -- 172.21.15.60:0/251026170 wait complete. 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.642+0000 7fc8f6038640 1 Processor -- start 2024-04-25T08:39:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.642+0000 7fc8f6038640 1 -- start start 2024-04-25T08:39:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8f6038640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8f00c7730 0x7fc8f015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8f6038640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 0x7fc8f00cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8f6038640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f015da70 0x7fc8f014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8f6038640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00c7bc0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8f6038640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015de50 con 0x7fc8f015da70 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8f6038640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00c7d40 con 0x7fc8f00c7730 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8eeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 0x7fc8f00cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8eeffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 0x7fc8f00cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52182/0 (socket says 172.21.15.60:52182) 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.643+0000 7fc8eeffd640 1 -- 172.21.15.60:0/2757816913 learned_addr learned my addr 172.21.15.60:0/2757816913 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8ef7fe640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8f00c7730 0x7fc8f015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8effff640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f015da70 0x7fc8f014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8eeffd640 1 -- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8f00c7730 msgr2=0x7fc8f015f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8eeffd640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8f00c7730 0x7fc8f015f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8eeffd640 1 -- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f015da70 msgr2=0x7fc8f014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8eeffd640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f015da70 0x7fc8f014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8eeffd640 1 -- 172.21.15.60:0/2757816913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00c9430 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8ef7fe640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8f00c7730 0x7fc8f015f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8eeffd640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 0x7fc8f00cafe0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c430 tx=0x7fc8e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40074e0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e40047e0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f00c96d0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4004bd0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.644+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f00c9c90 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.645+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f0073990 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.646+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8e4011070 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.646+0000 7fc8d7fff640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc8dc06b4d0 0x7fc8dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.646+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc8e4089c20 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.646+0000 7fc8ef7fe640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc8dc06b4d0 0x7fc8dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:22.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.647+0000 7fc8ef7fe640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc8dc06b4d0 0x7fc8dc06d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0034000 tx=0x7fc8e002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:22.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.650+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8e4058cf0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.898+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc8f006c4d0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.898+0000 7fc8d7fff640 1 -- 172.21.15.60:0/2757816913 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc8e40585f0 con 0x7fc8f00ce8d0 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_dump": [ 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.a", 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmhvQUMhAAZ6RGv2gupQhaxRr/nLKMUQ==", 2024-04-25T08:39:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.b", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmslbNARAAbKoxdkbRaQapt2Mss4AwiA==", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:39:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.c", 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpm2i7xOBAAKsAm53eYbbCLq/eFNDhOPQ==", 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mds.d", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpm77mTCBAA2QogRQJ3CeahtmnclVTK4Q==", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.0", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAtFCpmaljUKhAA67Nxed3bEj2MbxTMHS8e6A==", 2024-04-25T08:39:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.1", 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAwFCpmUeAIKRAAfXG/E0hngdrZfByXXD1zvg==", 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.2", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAzFCpmE4nzJhAAju+kIdBlCZ14BY4j3cnc6Q==", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.3", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA2FCpmS33QJBAA2w+k8KYj1BnVXXFTZhkKpA==", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.4", 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA8FCpmBw71IhAAjFMT8Gfjw7cyEUSe7rFYTw==", 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.5", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQA/FCpmTLyRIRAAhD/I8Y63eRtAgqLLBBNyKg==", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.6", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBCFCpmqeElJBAA0H7GxCardBTyyRhKkzwpFg==", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "osd.7", 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQBFFCpmBOlMIRAAu8sH/5WB/brNIjpbQUdxiQ==", 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile osd", 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.0", 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg==", 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow r", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.1", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmBOYTFRAALyKKSHXmEK+1HY9VPQtvbw==", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.2", 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmF8oEEhAA48MYIg5RwcERRLKwc55upA==", 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.3", 2024-04-25T08:39:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQAsFCpmGItzGhAAKnj/zEONdZqFGF2vOzWeag==", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow r", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow rw", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow rwx" 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "client.admin", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmvY48FxAA4SFFlZgldYEBKlH+QhX8TA==", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": "allow *", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow *", 2024-04-25T08:39:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.x", 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmM3zmLhAA+mq5cqdytwY7/VymvZWbOA==", 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "entity": "mgr.y", 2024-04-25T08:39:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "AQArFCpmcFKAKxAA2NSvYoPicy3flz1n+oX15A==", 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": { 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": "allow *", 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "mon": "allow profile mgr", 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": "allow *" 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-04-25T08:39:22.905 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:39:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc8dc06b4d0 msgr2=0x7fc8dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.906 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc8dc06b4d0 0x7fc8dc06d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0034000 tx=0x7fc8e002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 msgr2=0x7fc8f00cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 0x7fc8f00cafe0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c430 tx=0x7fc8e400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 shutdown_connections 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fc8dc06b4d0 0x7fc8dc06d990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f015da70 0x7fc8f014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc8f00ce8d0 0x7fc8f00cafe0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 --2- 172.21.15.60:0/2757816913 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fc8f00c7730 0x7fc8f015f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 >> 172.21.15.60:0/2757816913 conn(0x7fc8f00b33e0 msgr2=0x7fc8f00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 shutdown_connections 2024-04-25T08:39:22.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:22.900+0000 7fc8f6038640 1 -- 172.21.15.60:0/2757816913 wait complete. 2024-04-25T08:39:22.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:23.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.015+0000 7f17d7d30640 1 Processor -- start 2024-04-25T08:39:23.009 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.015+0000 7f17d7d30640 1 -- start start 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d7d30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d2e90 0x7f17d00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d7d30640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17d00d0900 0x7f17d00b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d7d30640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00b9940 con 0x7f17d015dd30 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d7d30640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d00d0d70 con 0x7f17d00d0900 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d7d30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00d1df0 con 0x7f17d00d2e90 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d62cd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17d00d0900 0x7f17d00b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d62cd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f17d00d0900 0x7f17d00b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52186/0 (socket says 172.21.15.60:52186) 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d62cd640 1 -- 172.21.15.60:0/1269761696 learned_addr learned my addr 172.21.15.60:0/1269761696 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.016+0000 7f17d52cb640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d2e90 0x7f17d00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d62cd640 1 -- 172.21.15.60:0/1269761696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d2e90 msgr2=0x7f17d00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d62cd640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d2e90 0x7f17d00d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d62cd640 1 -- 172.21.15.60:0/1269761696 >> v1:172.21.15.100:6789/0 conn(0x7f17d015dd30 legacy=0x7f17d00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d62cd640 1 -- 172.21.15.60:0/1269761696 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00d20d0 con 0x7f17d00d0900 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d52cb640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d2e90 0x7f17d00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d62cd640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d0900 0x7f17d00b6a40 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009e50 tx=0x7f17cc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b19a2069352d8e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d4aca640 1 -- 172.21.15.60:0/1269761696 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc036070 con 0x7f17d00d0900 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d4aca640 1 -- 172.21.15.60:0/1269761696 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc031600 con 0x7f17d00d0900 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.017+0000 7f17d4aca640 1 -- 172.21.15.60:0/1269761696 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0319d0 con 0x7f17d00d0900 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 -- 172.21.15.60:0/1269761696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d0900 msgr2=0x7f17d00b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d0900 0x7f17d00b6a40 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009e50 tx=0x7f17cc02a680 comp rx=0 tx=0).stop 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 -- 172.21.15.60:0/1269761696 shutdown_connections 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d0900 0x7f17d00b6a40 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1269761696 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d2e90 0x7f17d00d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 -- 172.21.15.60:0/1269761696 >> 172.21.15.60:0/1269761696 conn(0x7f17d0061c30 msgr2=0x7f17d006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 -- 172.21.15.60:0/1269761696 shutdown_connections 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.018+0000 7f17d7d30640 1 -- 172.21.15.60:0/1269761696 wait complete. 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.019+0000 7f17d7d30640 1 Processor -- start 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.019+0000 7f17d7d30640 1 -- start start 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d7d30640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d0900 0x7f17d0152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d7d30640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 0x7f17d01568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d7d30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d015dd30 0x7f17d015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d7d30640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00d2970 con 0x7f17d00d2e90 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d7d30640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00d2690 con 0x7f17d015dd30 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d7d30640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00d1a10 con 0x7f17d00d0900 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 0x7f17d01568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 0x7f17d01568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52192/0 (socket says 172.21.15.60:52192) 2024-04-25T08:39:23.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 -- 172.21.15.60:0/1800702248 learned_addr learned my addr 172.21.15.60:0/1800702248 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d62cd640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d015dd30 0x7f17d015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d5acc640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d0900 0x7f17d0152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 -- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d0900 msgr2=0x7f17d0152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d0900 0x7f17d0152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 -- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d015dd30 msgr2=0x7f17d015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d015dd30 0x7f17d015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.020+0000 7f17d52cb640 1 -- 172.21.15.60:0/1800702248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00b3e70 con 0x7f17d00d2e90 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.021+0000 7f17d52cb640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 0x7f17d01568b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f17c000e810 tx=0x7f17c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.021+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000f040 con 0x7f17d00d2e90 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.021+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0004230 con 0x7f17d00d2e90 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.021+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d00b4150 con 0x7f17d00d2e90 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.021+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0005990 con 0x7f17d00d2e90 2024-04-25T08:39:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.021+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d00cc190 con 0x7f17d00d2e90 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.022+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d0073990 con 0x7f17d00d2e90 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.022+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17c0005b30 con 0x7f17d00d2e90 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.023+0000 7f17c67fc640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f17b406b4d0 0x7f17b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.023+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f17c0089fd0 con 0x7f17d00d2e90 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.023+0000 7f17d5acc640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f17b406b4d0 0x7f17b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.023+0000 7f17d5acc640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f17b406b4d0 0x7f17b406d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f17c80016e0 tx=0x7f17c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.027+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17c0020020 con 0x7f17d00d2e90 2024-04-25T08:39:23.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.206+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17d00b4320 con 0x7f17d00d2e90 2024-04-25T08:39:23.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.207+0000 7f17c67fc640 1 -- 172.21.15.60:0/1800702248 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+13162 (secure 0 0 0) 0x7f17c0017030 con 0x7f17d00d2e90 2024-04-25T08:39:23.201 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:23.201 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":141,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:22.516029+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:23.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f17b406b4d0 msgr2=0x7f17b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f17b406b4d0 0x7f17b406d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f17c80016e0 tx=0x7f17c8009040 comp rx=0 tx=0).stop 2024-04-25T08:39:23.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 msgr2=0x7f17d01568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 0x7f17d01568b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f17c000e810 tx=0x7f17c000ecd0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.203 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 shutdown_connections 2024-04-25T08:39:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f17b406b4d0 0x7f17b406d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17d015dd30 0x7f17d015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17d00d2e90 0x7f17d01568b0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 --2- 172.21.15.60:0/1800702248 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f17d00d0900 0x7f17d0152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 >> 172.21.15.60:0/1800702248 conn(0x7f17d0061c30 msgr2=0x7f17d00c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.204 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.208+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 shutdown_connections 2024-04-25T08:39:23.205 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.209+0000 7f17d7d30640 1 -- 172.21.15.60:0/1800702248 wait complete. 2024-04-25T08:39:23.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:23.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:23.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.323+0000 7f7cd3ecb640 1 Processor -- start 2024-04-25T08:39:23.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.324+0000 7f7cd3ecb640 1 -- start start 2024-04-25T08:39:23.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.324+0000 7f7cd3ecb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7ccc15dc20 0x7f7ccc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.324+0000 7f7cd3ecb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ccc161710 0x7f7ccc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.324+0000 7f7cd3ecb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc15e000 con 0x7f7ccc0b9330 2024-04-25T08:39:23.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.324+0000 7f7cd3ecb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0d1de0 con 0x7f7ccc161710 2024-04-25T08:39:23.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.324+0000 7f7cd3ecb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc161c60 con 0x7f7ccc15dc20 2024-04-25T08:39:23.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1466640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7ccc161710 0x7f7ccc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd2468640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f7ccc0b9330 0x7f7ccc06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34482/0 (socket says 172.21.15.60:34482) 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd2468640 1 -- 172.21.15.60:0/2126188255 learned_addr learned my addr 172.21.15.60:0/2126188255 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1c67640 1 --2- 172.21.15.60:0/2126188255 >> v2:172.21.15.100:3300/0 conn(0x7f7ccc15dc20 0x7f7ccc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1466640 1 -- 172.21.15.60:0/2126188255 >> v2:172.21.15.100:3300/0 conn(0x7f7ccc15dc20 msgr2=0x7f7ccc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1466640 1 --2- 172.21.15.60:0/2126188255 >> v2:172.21.15.100:3300/0 conn(0x7f7ccc15dc20 0x7f7ccc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1466640 1 -- 172.21.15.60:0/2126188255 >> v1:172.21.15.60:6789/0 conn(0x7f7ccc0b9330 legacy=0x7f7ccc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1466640 1 -- 172.21.15.60:0/2126188255 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc161f40 con 0x7f7ccc161710 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.325+0000 7f7cd1c67640 1 --2- 172.21.15.60:0/2126188255 >> v2:172.21.15.100:3300/0 conn(0x7f7ccc15dc20 0x7f7ccc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd1466640 1 --2- 172.21.15.60:0/2126188255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc161710 0x7f7ccc0cebb0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0d1520 tx=0x7f7cbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e76e2cbe46cb657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd0c65640 1 -- 172.21.15.60:0/2126188255 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc037070 con 0x7f7ccc161710 2024-04-25T08:39:23.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd0c65640 1 -- 172.21.15.60:0/2126188255 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc032440 con 0x7f7ccc161710 2024-04-25T08:39:23.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd0c65640 1 -- 172.21.15.60:0/2126188255 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc032810 con 0x7f7ccc161710 2024-04-25T08:39:23.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2126188255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc161710 msgr2=0x7f7ccc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2126188255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc161710 0x7f7ccc0cebb0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0d1520 tx=0x7f7cbc02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2126188255 shutdown_connections 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2126188255 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc161710 0x7f7ccc0cebb0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2126188255 >> v2:172.21.15.100:3300/0 conn(0x7f7ccc15dc20 0x7f7ccc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2126188255 >> 172.21.15.60:0/2126188255 conn(0x7f7ccc0d0420 msgr2=0x7f7ccc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.326+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2126188255 shutdown_connections 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.327+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2126188255 wait complete. 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 Processor -- start 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 -- start start 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ccc0b9330 0x7f7ccc14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.324 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 0x7f7ccc0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ccc161710 0x7f7ccc0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b5e50 con 0x7f7ccc15dc20 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc162500 con 0x7f7ccc0b9330 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd3ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b5fd0 con 0x7f7ccc161710 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd1466640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 0x7f7ccc0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.328+0000 7f7cd1466640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 0x7f7ccc0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52204/0 (socket says 172.21.15.60:52204) 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 -- 172.21.15.60:0/2480154482 learned_addr learned my addr 172.21.15.60:0/2480154482 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd2468640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ccc161710 0x7f7ccc0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1c67640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ccc0b9330 0x7f7ccc14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 -- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ccc161710 msgr2=0x7f7ccc0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ccc161710 0x7f7ccc0cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 -- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ccc0b9330 msgr2=0x7f7ccc14b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ccc0b9330 0x7f7ccc14b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 -- 172.21.15.60:0/2480154482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc14baa0 con 0x7f7ccc15dc20 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd1466640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 0x7f7ccc0c9b40 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc000c00 tx=0x7f7cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc037040 con 0x7f7ccc15dc20 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc042070 con 0x7f7ccc15dc20 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc0caf30 con 0x7f7ccc15dc20 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc030bd0 con 0x7f7ccc15dc20 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.329+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc0cb4f0 con 0x7f7ccc15dc20 2024-04-25T08:39:23.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.330+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc073a00 con 0x7f7ccc15dc20 2024-04-25T08:39:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.331+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cbc047460 con 0x7f7ccc15dc20 2024-04-25T08:39:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.331+0000 7f7cc27fc640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cb006b4d0 0x7f7cb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.331+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7cbc0aac50 con 0x7f7ccc15dc20 2024-04-25T08:39:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.331+0000 7f7cd1c67640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cb006b4d0 0x7f7cb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.327 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.332+0000 7f7cd1c67640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cb006b4d0 0x7f7cb006d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc15abb0 tx=0x7f7cc8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.329 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.336+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cbc032e20 con 0x7f7ccc15dc20 2024-04-25T08:39:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.518+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ccc0d16b0 con 0x7f7ccc15dc20 2024-04-25T08:39:23.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.519+0000 7f7cc27fc640 1 -- 172.21.15.60:0/2480154482 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cbc045360 con 0x7f7ccc15dc20 2024-04-25T08:39:23.512 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:39:23.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cb006b4d0 msgr2=0x7f7cb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cb006b4d0 0x7f7cb006d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc15abb0 tx=0x7f7cc8008040 comp rx=0 tx=0).stop 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 msgr2=0x7f7ccc0c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 0x7f7ccc0c9b40 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc000c00 tx=0x7f7cbc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 shutdown_connections 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7ccc161710 0x7f7ccc0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ccc15dc20 0x7f7ccc0c9b40 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cb006b4d0 0x7f7cb006d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 --2- 172.21.15.60:0/2480154482 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ccc0b9330 0x7f7ccc14b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 >> 172.21.15.60:0/2480154482 conn(0x7f7ccc0d0420 msgr2=0x7f7ccc0b88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 shutdown_connections 2024-04-25T08:39:23.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.521+0000 7f7cd3ecb640 1 -- 172.21.15.60:0/2480154482 wait complete. 2024-04-25T08:39:23.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:39:23.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.635+0000 7f36b4f49640 1 Processor -- start 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.635+0000 7f36b4f49640 1 -- start start 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36b4f49640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f36b015dbe0 0x7f36b00d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36b4f49640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36b00b92a0 0x7f36b00d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36b4f49640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b00b97a0 con 0x7f36b00b2cb0 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36b4f49640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b015dfc0 con 0x7f36b00b92a0 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36b4f49640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b00d1a00 con 0x7f36b015dbe0 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36aed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36b00b92a0 0x7f36b00d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36a5d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f36b00b2cb0 0x7f36b00b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34496/0 (socket says 172.21.15.60:34496) 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36aed9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f36b00b92a0 0x7f36b00d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52206/0 (socket says 172.21.15.60:52206) 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36a5d9b640 1 -- 172.21.15.60:0/2352479875 learned_addr learned my addr 172.21.15.60:0/2352479875 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36ae59c640 1 --2- 172.21.15.60:0/2352479875 >> v2:172.21.15.100:3300/0 conn(0x7f36b015dbe0 0x7f36b00d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36aed9d640 1 -- 172.21.15.60:0/2352479875 >> v2:172.21.15.100:3300/0 conn(0x7f36b015dbe0 msgr2=0x7f36b00d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36aed9d640 1 --2- 172.21.15.60:0/2352479875 >> v2:172.21.15.100:3300/0 conn(0x7f36b015dbe0 0x7f36b00d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36aed9d640 1 -- 172.21.15.60:0/2352479875 >> v1:172.21.15.60:6789/0 conn(0x7f36b00b2cb0 legacy=0x7f36b00b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.636+0000 7f36aed9d640 1 -- 172.21.15.60:0/2352479875 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00d1c50 con 0x7f36b00b92a0 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36aed9d640 1 --2- 172.21.15.60:0/2352479875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00d0e50 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f36b006c140 tx=0x7f36a802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d8573d0f313df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36add9b640 1 -- 172.21.15.60:0/2352479875 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8007740 con 0x7f36b00b92a0 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36add9b640 1 -- 172.21.15.60:0/2352479875 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a80078e0 con 0x7f36b00b92a0 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36add9b640 1 -- 172.21.15.60:0/2352479875 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8007cb0 con 0x7f36b00b92a0 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36b4f49640 1 -- 172.21.15.60:0/2352479875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 msgr2=0x7f36b00d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36b4f49640 1 --2- 172.21.15.60:0/2352479875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00d0e50 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f36b006c140 tx=0x7f36a802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.637+0000 7f36b4f49640 1 -- 172.21.15.60:0/2352479875 shutdown_connections 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.638+0000 7f36b4f49640 1 --2- 172.21.15.60:0/2352479875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00d0e50 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.638+0000 7f36b4f49640 1 --2- 172.21.15.60:0/2352479875 >> v2:172.21.15.100:3300/0 conn(0x7f36b015dbe0 0x7f36b00d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.638+0000 7f36b4f49640 1 -- 172.21.15.60:0/2352479875 >> 172.21.15.60:0/2352479875 conn(0x7f36b0061a20 msgr2=0x7f36b0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.638+0000 7f36b4f49640 1 -- 172.21.15.60:0/2352479875 shutdown_connections 2024-04-25T08:39:23.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.638+0000 7f36b4f49640 1 -- 172.21.15.60:0/2352479875 wait complete. 2024-04-25T08:39:23.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.639+0000 7f36b4f49640 1 Processor -- start 2024-04-25T08:39:23.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.639+0000 7f36b4f49640 1 -- start start 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36b4f49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b00b2cb0 0x7f36b00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36b4f49640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36b4f49640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36b015dbe0 0x7f36b00ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36b4f49640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00b6ab0 con 0x7f36b00b92a0 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36b4f49640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00d2210 con 0x7f36b00b2cb0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36b4f49640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00b6c30 con 0x7f36b015dbe0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52214/0 (socket says 172.21.15.60:52214) 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 -- 172.21.15.60:0/3425339386 learned_addr learned my addr 172.21.15.60:0/3425339386 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36aed9d640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36b015dbe0 0x7f36b00ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36ae59c640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b00b2cb0 0x7f36b00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 -- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36b015dbe0 msgr2=0x7f36b00ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36b015dbe0 0x7f36b00ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 -- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b00b2cb0 msgr2=0x7f36b00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b00b2cb0 0x7f36b00cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 -- 172.21.15.60:0/3425339386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b00cd5d0 con 0x7f36b00b92a0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.640+0000 7f36a5d9b640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00cca80 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f369800c440 tx=0x7f3698001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.641+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698017070 con 0x7f36b00b92a0 2024-04-25T08:39:23.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.641+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3698005ce0 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.641+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b00cd8b0 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.641+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36980072f0 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.641+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b0156580 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.642+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0073990 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.642+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f369801a070 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.643+0000 7f36a77fe640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.643+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f36980893a0 con 0x7f36b00b92a0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.643+0000 7f36ae59c640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.644+0000 7f36ae59c640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f369006b4d0 0x7f369006d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006fd0 tx=0x7f36a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.647+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3698058470 con 0x7f36b00b92a0 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.829+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f36b00b7540 con 0x7f36b00b92a0 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.829+0000 7f36a77fe640 1 -- 172.21.15.60:0/3425339386 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3698057d70 con 0x7f36b00b92a0 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stdout:[ 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: "poolnum": 1, 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: "poolname": ".mgr" 2024-04-25T08:39:23.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-04-25T08:39:23.824 INFO:teuthology.orchestra.run.smithi060.stdout:] 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f369006b4d0 msgr2=0x7f369006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f369006b4d0 0x7f369006d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006fd0 tx=0x7f36a0008040 comp rx=0 tx=0).stop 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 msgr2=0x7f36b00cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00cca80 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f369800c440 tx=0x7f3698001e70 comp rx=0 tx=0).stop 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 shutdown_connections 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f36b015dbe0 0x7f36b00ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f36b00b92a0 0x7f36b00cca80 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 --2- 172.21.15.60:0/3425339386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f36b00b2cb0 0x7f36b00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 >> 172.21.15.60:0/3425339386 conn(0x7f36b0061a20 msgr2=0x7f36b0160160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 shutdown_connections 2024-04-25T08:39:23.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.831+0000 7f36b4f49640 1 -- 172.21.15.60:0/3425339386 wait complete. 2024-04-25T08:39:23.838 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:39:23.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:39:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.950+0000 7fde0779e640 1 Processor -- start 2024-04-25T08:39:23.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.950+0000 7fde0779e640 1 -- start start 2024-04-25T08:39:23.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0779e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde0015b5a0 0x7fde000b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0779e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fde00162450 0x7fde000cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0779e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde000cf320 con 0x7fde0015b980 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0779e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde000cf040 con 0x7fde0015b5a0 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0779e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde00161970 con 0x7fde00162450 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0553a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fde0015b5a0 0x7fde000b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde04d39640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fde0015b980 0x7fde000ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34500/0 (socket says 172.21.15.60:34500) 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde04d39640 1 -- 172.21.15.60:0/3791484319 learned_addr learned my addr 172.21.15.60:0/3791484319 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0553a640 1 --2- 172.21.15.60:0/3791484319 >> v2:172.21.15.60:3300/0 conn(0x7fde0015b5a0 0x7fde000b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52220/0 (socket says 172.21.15.60:52220) 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde05d3b640 1 --2- 172.21.15.60:0/3791484319 >> v2:172.21.15.100:3300/0 conn(0x7fde00162450 0x7fde000cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0553a640 1 -- 172.21.15.60:0/3791484319 >> v2:172.21.15.100:3300/0 conn(0x7fde00162450 msgr2=0x7fde000cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0553a640 1 --2- 172.21.15.60:0/3791484319 >> v2:172.21.15.100:3300/0 conn(0x7fde00162450 0x7fde000cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0553a640 1 -- 172.21.15.60:0/3791484319 >> v1:172.21.15.60:6789/0 conn(0x7fde0015b980 legacy=0x7fde000ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fddfffff640 1 -- 172.21.15.60:0/3791484319 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814904388 0 0) 0x7fddf4001290 con 0x7fde0015b980 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.951+0000 7fde0553a640 1 -- 172.21.15.60:0/3791484319 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00161c50 con 0x7fde0015b5a0 2024-04-25T08:39:23.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.952+0000 7fde0553a640 1 --2- 172.21.15.60:0/3791484319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde0015b5a0 0x7fde000b5530 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fde000cfd30 tx=0x7fddf802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72171693cb3c000e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.952+0000 7fddfffff640 1 -- 172.21.15.60:0/3791484319 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8007740 con 0x7fde0015b5a0 2024-04-25T08:39:23.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.952+0000 7fddfffff640 1 -- 172.21.15.60:0/3791484319 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf80078e0 con 0x7fde0015b5a0 2024-04-25T08:39:23.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.952+0000 7fddfffff640 1 -- 172.21.15.60:0/3791484319 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8007cb0 con 0x7fde0015b5a0 2024-04-25T08:39:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.952+0000 7fde0779e640 1 -- 172.21.15.60:0/3791484319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde0015b5a0 msgr2=0x7fde000b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.952+0000 7fde0779e640 1 --2- 172.21.15.60:0/3791484319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde0015b5a0 0x7fde000b5530 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fde000cfd30 tx=0x7fddf802a680 comp rx=0 tx=0).stop 2024-04-25T08:39:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.953+0000 7fde0779e640 1 -- 172.21.15.60:0/3791484319 shutdown_connections 2024-04-25T08:39:23.949 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.953+0000 7fde0779e640 1 --2- 172.21.15.60:0/3791484319 >> v2:172.21.15.100:3300/0 conn(0x7fde00162450 0x7fde000cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.953+0000 7fde0779e640 1 --2- 172.21.15.60:0/3791484319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde0015b5a0 0x7fde000b5530 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.953+0000 7fde0779e640 1 -- 172.21.15.60:0/3791484319 >> 172.21.15.60:0/3791484319 conn(0x7fde000d0530 msgr2=0x7fde000d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.953+0000 7fde0779e640 1 -- 172.21.15.60:0/3791484319 shutdown_connections 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.953+0000 7fde0779e640 1 -- 172.21.15.60:0/3791484319 wait complete. 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.954+0000 7fde0779e640 1 Processor -- start 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.954+0000 7fde0779e640 1 -- start start 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde0015b5a0 0x7fde00160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0015b980 0x7fde0014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 0x7fde000c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b5bf0 con 0x7fde00162450 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b5a70 con 0x7fde0015b980 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde000b5d70 con 0x7fde0015b5a0 2024-04-25T08:39:23.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 0x7fde000c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 0x7fde000c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52230/0 (socket says 172.21.15.60:52230) 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 -- 172.21.15.60:0/929838629 learned_addr learned my addr 172.21.15.60:0/929838629 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0553a640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde0015b5a0 0x7fde00160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 -- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde0015b5a0 msgr2=0x7fde00160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde04d39640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0015b980 0x7fde0014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde0015b5a0 0x7fde00160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 -- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0015b980 msgr2=0x7fde0014b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0015b980 0x7fde0014b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 -- 172.21.15.60:0/929838629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0015fbe0 con 0x7fde00162450 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde05d3b640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 0x7fde000c9830 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d600 tx=0x7fddf000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf00125f0 con 0x7fde00162450 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0019070 con 0x7fde00162450 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0015fec0 con 0x7fde00162450 2024-04-25T08:39:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.955+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf00129f0 con 0x7fde00162450 2024-04-25T08:39:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.956+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde00160480 con 0x7fde00162450 2024-04-25T08:39:23.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.957+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00073a00 con 0x7fde00162450 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.957+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddf0012b90 con 0x7fde00162450 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.957+0000 7fddfdffb640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.957+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fddf008a3c0 con 0x7fde00162450 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.958+0000 7fde0553a640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.958+0000 7fde0553a640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddec06b4d0 0x7fddec06d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:23.955 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:23.962+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddf0059410 con 0x7fde00162450 2024-04-25T08:39:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.144+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fde000d2140 con 0x7fde00162450 2024-04-25T08:39:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.516+0000 7fddfdffb640 1 -- 172.21.15.60:0/929838629 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) v1 ==== 138+0+0 (secure 0 0 0) 0x7fddf0058d10 con 0x7fde00162450 2024-04-25T08:39:24.510 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' created 2024-04-25T08:39:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddec06b4d0 msgr2=0x7fddec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddec06b4d0 0x7fddec06d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fddf8034000 tx=0x7fddf802f000 comp rx=0 tx=0).stop 2024-04-25T08:39:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 msgr2=0x7fde000c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:24.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 0x7fde000c9830 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d600 tx=0x7fddf000dac0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 shutdown_connections 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fde00162450 0x7fde000c9830 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde0015b980 0x7fde0014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 --2- 172.21.15.60:0/929838629 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fde0015b5a0 0x7fde00160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.518+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 >> 172.21.15.60:0/929838629 conn(0x7fde000d0530 msgr2=0x7fde0015be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.519+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 shutdown_connections 2024-04-25T08:39:24.513 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.519+0000 7fde0779e640 1 -- 172.21.15.60:0/929838629 wait complete. 2024-04-25T08:39:24.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:39:24.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.642+0000 7f1e1b041640 1 Processor -- start 2024-04-25T08:39:24.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.642+0000 7f1e1b041640 1 -- start start 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e1b041640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e140b5bb0 0x7f1e140b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e1b041640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140cf1d0 con 0x7f1e14161240 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e1b041640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e14161620 con 0x7f1e14161890 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e1b041640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e140b7120 con 0x7f1e140b5bb0 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e195de640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f1e140b5bb0 0x7f1e140b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e18ddd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1e14161240 0x7f1e140ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34512/0 (socket says 172.21.15.60:34512) 2024-04-25T08:39:24.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.643+0000 7f1e18ddd640 1 -- 172.21.15.60:0/1957187025 learned_addr learned my addr 172.21.15.60:0/1957187025 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.644+0000 7f1e195de640 1 -- 172.21.15.60:0/1957187025 >> v1:172.21.15.100:6789/0 conn(0x7f1e14161890 legacy=0x7f1e140b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.644+0000 7f1e137fe640 1 -- 172.21.15.60:0/1957187025 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197573442 0 0) 0x7f1e140cf1d0 con 0x7f1e14161240 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.644+0000 7f1e195de640 1 -- 172.21.15.60:0/1957187025 >> v1:172.21.15.60:6789/0 conn(0x7f1e14161240 legacy=0x7f1e140ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.644+0000 7f1e195de640 1 -- 172.21.15.60:0/1957187025 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14161e00 con 0x7f1e140b5bb0 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.644+0000 7f1e195de640 1 --2- 172.21.15.60:0/1957187025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e140b5bb0 0x7f1e140b6be0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097a0 tx=0x7f1dfc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d0516735e1ff54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e137fe640 1 -- 172.21.15.60:0/1957187025 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc037070 con 0x7f1e140b5bb0 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e137fe640 1 -- 172.21.15.60:0/1957187025 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc032440 con 0x7f1e140b5bb0 2024-04-25T08:39:24.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e137fe640 1 -- 172.21.15.60:0/1957187025 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc032810 con 0x7f1e140b5bb0 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 -- 172.21.15.60:0/1957187025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e140b5bb0 msgr2=0x7f1e140b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 --2- 172.21.15.60:0/1957187025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e140b5bb0 0x7f1e140b6be0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097a0 tx=0x7f1dfc02a230 comp rx=0 tx=0).stop 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 -- 172.21.15.60:0/1957187025 shutdown_connections 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 --2- 172.21.15.60:0/1957187025 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e140b5bb0 0x7f1e140b6be0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 -- 172.21.15.60:0/1957187025 >> 172.21.15.60:0/1957187025 conn(0x7f1e140d04a0 msgr2=0x7f1e140d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 -- 172.21.15.60:0/1957187025 shutdown_connections 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.645+0000 7f1e1b041640 1 -- 172.21.15.60:0/1957187025 wait complete. 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.646+0000 7f1e1b041640 1 Processor -- start 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.646+0000 7f1e1b041640 1 -- start start 2024-04-25T08:39:24.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e1b041640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e140b5bb0 0x7f1e1414adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e1b041640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 0x7f1e140d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e1b041640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e14161890 0x7f1e140d3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e1b041640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141626a0 con 0x7f1e14161240 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e1b041640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141623c0 con 0x7f1e14161890 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e1b041640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140d15f0 con 0x7f1e140b5bb0 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 0x7f1e140d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 0x7f1e140d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52258/0 (socket says 172.21.15.60:52258) 2024-04-25T08:39:24.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.647+0000 7f1e13fff640 1 -- 172.21.15.60:0/2320398615 learned_addr learned my addr 172.21.15.60:0/2320398615 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e18ddd640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e140b5bb0 0x7f1e1414adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e195de640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e14161890 0x7f1e140d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e13fff640 1 -- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e140b5bb0 msgr2=0x7f1e1414adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e13fff640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e140b5bb0 0x7f1e1414adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e13fff640 1 -- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e14161890 msgr2=0x7f1e140d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e13fff640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e14161890 0x7f1e140d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e13fff640 1 -- 172.21.15.60:0/2320398615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140cd5d0 con 0x7f1e14161240 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e18ddd640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e140b5bb0 0x7f1e1414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e195de640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e14161890 0x7f1e140d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e13fff640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 0x7f1e140d2ae0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400c400 tx=0x7f1e0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04017070 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e040052d0 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e140b44a0 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04003830 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.648+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e140b4a60 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.649+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e140b65c0 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.650+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e0401a070 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.650+0000 7f1e117fa640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.650+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1e04089480 con 0x7f1e14161240 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.651+0000 7f1e18ddd640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:24.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.651+0000 7f1e18ddd640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0012e0 tx=0x7f1e0c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:24.650 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.655+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e140cd5d0 con 0x7f1e14161240 2024-04-25T08:39:24.828 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:24.834+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1e1406d240 con 0x7f1e14161240 2024-04-25T08:39:25.512 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.519+0000 7f1e117fa640 1 -- 172.21.15.60:0/2320398615 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1e04087e20 con 0x7f1e14161240 2024-04-25T08:39:25.513 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' created 2024-04-25T08:39:25.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 msgr2=0x7f1df406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0012e0 tx=0x7f1e0c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 msgr2=0x7f1e140d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 0x7f1e140d2ae0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400c400 tx=0x7f1e0400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 shutdown_connections 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1df406b4d0 0x7f1df406d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e14161890 0x7f1e140d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e14161240 0x7f1e140d2ae0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 --2- 172.21.15.60:0/2320398615 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1e140b5bb0 0x7f1e1414adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 >> 172.21.15.60:0/2320398615 conn(0x7f1e140d04a0 msgr2=0x7f1e1415e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 shutdown_connections 2024-04-25T08:39:25.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.522+0000 7f1e1b041640 1 -- 172.21.15.60:0/2320398615 wait complete. 2024-04-25T08:39:25.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:39:25.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.659+0000 7f1c1b716640 1 Processor -- start 2024-04-25T08:39:25.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.659+0000 7f1c1b716640 1 -- start start 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.660+0000 7f1c1b716640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1c14161890 0x7f1c140b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.660+0000 7f1c1b716640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c140cf1d0 con 0x7f1c140b54a0 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.660+0000 7f1c1b716640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c14161620 con 0x7f1c14161240 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.660+0000 7f1c1b716640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c140b7120 con 0x7f1c14161890 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.660+0000 7f1c19cb3640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f1c140b54a0 0x7f1c140d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34528/0 (socket says 172.21.15.60:34528) 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.660+0000 7f1c19cb3640 1 -- 172.21.15.60:0/3744132540 learned_addr learned my addr 172.21.15.60:0/3744132540 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429717176 0 0) 0x7f1c140cf1d0 con 0x7f1c140b54a0 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c18cb1640 1 --2- 172.21.15.60:0/3744132540 >> v2:172.21.15.100:3300/0 conn(0x7f1c14161890 0x7f1c140b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bfc003520 con 0x7f1c140b54a0 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58907400 0 0) 0x7f1bfc003520 con 0x7f1c140b54a0 2024-04-25T08:39:25.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 >> v2:172.21.15.100:3300/0 conn(0x7f1c14161890 msgr2=0x7f1c140b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 --2- 172.21.15.60:0/3744132540 >> v2:172.21.15.100:3300/0 conn(0x7f1c14161890 0x7f1c140b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 >> v1:172.21.15.100:6789/0 conn(0x7f1c14161240 legacy=0x7f1c140ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c14161e00 con 0x7f1c140b54a0 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f1c040019f0 con 0x7f1c140b54a0 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 >> v1:172.21.15.60:6789/0 conn(0x7f1c140b54a0 legacy=0x7f1c140d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bfc003a00 0x7f1bfc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bfc004800 0x7f1bfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc0051c0 0x7f1bfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c090 con 0x7f1bfc003a00 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c240 con 0x7f1bfc004800 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c440 con 0x7f1bfc0051c0 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c194b2640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bfc003a00 0x7f1bfc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c18cb1640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bfc004800 0x7f1bfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.661+0000 7f1c19cb3640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc0051c0 0x7f1bfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c194b2640 1 -- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc0051c0 msgr2=0x7f1bfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c194b2640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc0051c0 0x7f1bfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c194b2640 1 -- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bfc004800 msgr2=0x7f1bfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c194b2640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1bfc004800 0x7f1bfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c194b2640 1 -- 172.21.15.60:0/3744132540 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc003520 con 0x7f1bfc003a00 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c19cb3640 1 -- 172.21.15.60:0/3744132540 reap_dead start 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c194b2640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bfc003a00 0x7f1bfc003e20 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c001750 tx=0x7f1bfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8eae4fd0717692f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c001370 con 0x7f1bfc003a00 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0052d0 con 0x7f1bfc003a00 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.662+0000 7f1c13fff640 1 -- 172.21.15.60:0/3744132540 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c02abc0 con 0x7f1bfc003a00 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 -- 172.21.15.60:0/3744132540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bfc003a00 msgr2=0x7f1bfc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bfc003a00 0x7f1bfc003e20 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c001750 tx=0x7f1bfc005830 comp rx=0 tx=0).stop 2024-04-25T08:39:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 -- 172.21.15.60:0/3744132540 shutdown_connections 2024-04-25T08:39:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 --2- 172.21.15.60:0/3744132540 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1bfc003a00 0x7f1bfc003e20 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 -- 172.21.15.60:0/3744132540 >> 172.21.15.60:0/3744132540 conn(0x7f1c140d04a0 msgr2=0x7f1c140d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 -- 172.21.15.60:0/3744132540 shutdown_connections 2024-04-25T08:39:25.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.663+0000 7f1c1b716640 1 -- 172.21.15.60:0/3744132540 wait complete. 2024-04-25T08:39:25.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.664+0000 7f1c1b716640 1 Processor -- start 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.664+0000 7f1c1b716640 1 -- start start 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c1b716640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc003a00 0x7f1c1415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c1b716640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1415dc60 0x7f1c141f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c1b716640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 0x7f1c1415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c1b716640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141626a0 con 0x7f1c141f4130 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c1b716640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141623c0 con 0x7f1c1415dc60 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c1b716640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140d15f0 con 0x7f1bfc003a00 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 0x7f1c1415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 0x7f1c1415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52274/0 (socket says 172.21.15.60:52274) 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 -- 172.21.15.60:0/1966986777 learned_addr learned my addr 172.21.15.60:0/1966986777 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c18cb1640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1415dc60 0x7f1c141f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c194b2640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc003a00 0x7f1c1415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 -- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc003a00 msgr2=0x7f1c1415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc003a00 0x7f1c1415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 -- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1415dc60 msgr2=0x7f1c141f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1415dc60 0x7f1c141f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c19cb3640 1 -- 172.21.15.60:0/1966986777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1415e1b0 con 0x7f1c141f4130 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.665+0000 7f1c18cb1640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1415dc60 0x7f1c141f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.666+0000 7f1c19cb3640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 0x7f1c1415d120 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f1c140ce570 tx=0x7f1c04007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.666+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400aa70 con 0x7f1c141f4130 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.666+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c04018070 con 0x7f1c141f4130 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.666+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1415e490 con 0x7f1c141f4130 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.666+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0400ae10 con 0x7f1c141f4130 2024-04-25T08:39:25.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.666+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1415cc00 con 0x7f1c141f4130 2024-04-25T08:39:25.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.667+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1415ce00 con 0x7f1c141f4130 2024-04-25T08:39:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.668+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c0401d460 con 0x7f1c141f4130 2024-04-25T08:39:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.668+0000 7f1c11ffb640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1c0006b500 0x7f1c0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.668+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f1c0408a7f0 con 0x7f1c141f4130 2024-04-25T08:39:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.668+0000 7f1c194b2640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1c0006b500 0x7f1c0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.669+0000 7f1c194b2640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1c0006b500 0x7f1c0006d9c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c032000 tx=0x7f1c0c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.673+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c1415ce00 con 0x7f1c141f4130 2024-04-25T08:39:25.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:25.889+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1c1406d240 con 0x7f1c141f4130 2024-04-25T08:39:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.530+0000 7f1c11ffb640 1 -- 172.21.15.60:0/1966986777 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1c04059540 con 0x7f1c141f4130 2024-04-25T08:39:26.524 INFO:teuthology.orchestra.run.smithi060.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T08:39:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.535+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1c0006b500 msgr2=0x7f1c0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.535+0000 7f1c1b716640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1c0006b500 0x7f1c0006d9c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c032000 tx=0x7f1c0c02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:26.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.535+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 msgr2=0x7f1c1415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.535+0000 7f1c1b716640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 0x7f1c1415d120 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f1c140ce570 tx=0x7f1c04007b20 comp rx=0 tx=0).stop 2024-04-25T08:39:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.536+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 shutdown_connections 2024-04-25T08:39:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.536+0000 7f1c1b716640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1c141f4130 0x7f1c1415d120 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.536+0000 7f1c1b716640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f1c0006b500 0x7f1c0006d9c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.536+0000 7f1c1b716640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c1415dc60 0x7f1c141f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.536+0000 7f1c1b716640 1 --2- 172.21.15.60:0/1966986777 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1bfc003a00 0x7f1c1415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.536+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 >> 172.21.15.60:0/1966986777 conn(0x7f1c140d04a0 msgr2=0x7f1c140b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.537+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 shutdown_connections 2024-04-25T08:39:26.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.538+0000 7f1c1b716640 1 -- 172.21.15.60:0/1966986777 wait complete. 2024-04-25T08:39:26.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:26.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.677+0000 7f221d6ac640 1 Processor -- start 2024-04-25T08:39:26.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.677+0000 7f221d6ac640 1 -- start start 2024-04-25T08:39:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f221d6ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f221815dd20 0x7f22180b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f221d6ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e100 0x7f22180d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f221d6ac640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f221815e560 0x7f22180d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f221d6ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180d1c00 con 0x7f221815dd20 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f221d6ac640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221815eac0 con 0x7f221815e560 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f221d6ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221815eda0 con 0x7f221815e100 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f221815dd20 0x7f22180b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f221815dd20 0x7f22180b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52286/0 (socket says 172.21.15.60:52286) 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 -- 172.21.15.60:0/2486833858 learned_addr learned my addr 172.21.15.60:0/2486833858 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f22167fc640 1 --2- 172.21.15.60:0/2486833858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e100 0x7f22180d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f22177fe640 1 --2- 172.21.15.60:0/2486833858 >> v2:172.21.15.100:3300/0 conn(0x7f221815e560 0x7f22180d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 -- 172.21.15.60:0/2486833858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e100 msgr2=0x7f22180d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 --2- 172.21.15.60:0/2486833858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e100 0x7f22180d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 -- 172.21.15.60:0/2486833858 >> v2:172.21.15.100:3300/0 conn(0x7f221815e560 msgr2=0x7f22180d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 --2- 172.21.15.60:0/2486833858 >> v2:172.21.15.100:3300/0 conn(0x7f221815e560 0x7f22180d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.678+0000 7f2216ffd640 1 -- 172.21.15.60:0/2486833858 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221815f080 con 0x7f221815dd20 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.679+0000 7f2216ffd640 1 --2- 172.21.15.60:0/2486833858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f22180b96d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f22180cab60 tx=0x7f220802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72251f367db5a695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.679+0000 7f2215ffb640 1 -- 172.21.15.60:0/2486833858 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208037070 con 0x7f221815dd20 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.679+0000 7f2215ffb640 1 -- 172.21.15.60:0/2486833858 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2208032440 con 0x7f221815dd20 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.679+0000 7f2215ffb640 1 -- 172.21.15.60:0/2486833858 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208032810 con 0x7f221815dd20 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 -- 172.21.15.60:0/2486833858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 msgr2=0x7f22180b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 --2- 172.21.15.60:0/2486833858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f22180b96d0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f22180cab60 tx=0x7f220802a140 comp rx=0 tx=0).stop 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 -- 172.21.15.60:0/2486833858 shutdown_connections 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 --2- 172.21.15.60:0/2486833858 >> v2:172.21.15.100:3300/0 conn(0x7f221815e560 0x7f22180d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 --2- 172.21.15.60:0/2486833858 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e100 0x7f22180d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 --2- 172.21.15.60:0/2486833858 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f22180b96d0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 -- 172.21.15.60:0/2486833858 >> 172.21.15.60:0/2486833858 conn(0x7f2218061a20 msgr2=0x7f2218061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:26.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 -- 172.21.15.60:0/2486833858 shutdown_connections 2024-04-25T08:39:26.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.680+0000 7f221d6ac640 1 -- 172.21.15.60:0/2486833858 wait complete. 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.681+0000 7f221d6ac640 1 Processor -- start 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.681+0000 7f221d6ac640 1 -- start start 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f221d6ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f221814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f221d6ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221815e100 0x7f22180cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f221d6ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e560 0x7f2218156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f221d6ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180d3c20 con 0x7f221815dd20 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f221d6ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180d3aa0 con 0x7f221815e100 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f221d6ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22180d3da0 con 0x7f221815e560 2024-04-25T08:39:26.679 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f221814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f221814e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52298/0 (socket says 172.21.15.60:52298) 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 -- 172.21.15.60:0/3651443889 learned_addr learned my addr 172.21.15.60:0/3651443889 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f22167fc640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221815e100 0x7f22180cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f22177fe640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e560 0x7f2218156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 -- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e560 msgr2=0x7f2218156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e560 0x7f2218156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 -- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221815e100 msgr2=0x7f22180cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221815e100 0x7f22180cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 -- 172.21.15.60:0/3651443889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180cc510 con 0x7f221815dd20 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f22167fc640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221815e100 0x7f22180cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f22177fe640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e560 0x7f2218156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.682+0000 7f2216ffd640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f221814e0e0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f2208000c00 tx=0x7f22080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:26.680 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.683+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208041020 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.683+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22080306c0 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.683+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22180caf30 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.683+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208037040 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.683+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22180cb4f0 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.684+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2218073990 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.684+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2208044020 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.685+0000 7f21ff7fe640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f220406b4d0 0x7f220406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.685+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22080aab90 con 0x7f221815dd20 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.685+0000 7f22167fc640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f220406b4d0 0x7f220406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.681 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.685+0000 7f22167fc640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f220406b4d0 0x7f220406d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f22181595b0 tx=0x7f220c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:26.682 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.689+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f220804b030 con 0x7f221815dd20 2024-04-25T08:39:26.864 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.870+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22180b4ef0 con 0x7f221815dd20 2024-04-25T08:39:26.865 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.872+0000 7f21ff7fe640 1 -- 172.21.15.60:0/3651443889 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f2208079880 con 0x7f221815dd20 2024-04-25T08:39:26.865 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:26.865 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":144,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:25.890501+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f220406b4d0 msgr2=0x7f220406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f220406b4d0 0x7f220406d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f22181595b0 tx=0x7f220c008040 comp rx=0 tx=0).stop 2024-04-25T08:39:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 msgr2=0x7f221814e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f221814e0e0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f2208000c00 tx=0x7f22080046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 shutdown_connections 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221815e560 0x7f2218156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f220406b4d0 0x7f220406d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f221815e100 0x7f22180cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 --2- 172.21.15.60:0/3651443889 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221815dd20 0x7f221814e0e0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.873+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 >> 172.21.15.60:0/3651443889 conn(0x7f2218061a20 msgr2=0x7f22180b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.874+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 shutdown_connections 2024-04-25T08:39:26.868 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.874+0000 7f221d6ac640 1 -- 172.21.15.60:0/3651443889 wait complete. 2024-04-25T08:39:26.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:26.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.986+0000 7f2d6e0c6640 1 Processor -- start 2024-04-25T08:39:26.980 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.986+0000 7f2d6e0c6640 1 -- start start 2024-04-25T08:39:26.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d6e0c6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d6815dd30 0x7f2d680b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d6e0c6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 0x7f2d680b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d6e0c6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d680d1df0 con 0x7f2d680b9770 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d6e0c6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680d20d0 con 0x7f2d6815dd30 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d6e0c6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680d23b0 con 0x7f2d680d0ad0 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d67fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 0x7f2d680b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d677fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d6815dd30 0x7f2d680b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d677fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d6815dd30 0x7f2d680b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:47998/0 (socket says 172.21.15.60:47998) 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d67fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 0x7f2d680b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37504/0 (socket says 172.21.15.60:37504) 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d677fe640 1 -- 172.21.15.60:0/3849941942 learned_addr learned my addr 172.21.15.60:0/3849941942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.987+0000 7f2d66ffd640 1 --1- 172.21.15.60:0/3849941942 >> v1:172.21.15.100:6789/0 conn(0x7f2d680b9770 0x7f2d680d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59914/0 (socket says 172.21.15.60:59914) 2024-04-25T08:39:26.983 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d677fe640 1 -- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 msgr2=0x7f2d680b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d677fe640 1 --2- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 0x7f2d680b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d677fe640 1 -- 172.21.15.60:0/3849941942 >> v1:172.21.15.100:6789/0 conn(0x7f2d680b9770 legacy=0x7f2d680d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d677fe640 1 -- 172.21.15.60:0/3849941942 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680d2690 con 0x7f2d6815dd30 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d667fc640 1 -- 172.21.15.60:0/3849941942 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617848169 0 0) 0x7f2d680d1df0 con 0x7f2d680b9770 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d67fff640 1 --2- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 0x7f2d680b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d677fe640 1 --2- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815dd30 0x7f2d680b9230 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009870 tx=0x7f2d5802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d65157df93872b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.988+0000 7f2d667fc640 1 -- 172.21.15.60:0/3849941942 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58037070 con 0x7f2d6815dd30 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d667fc640 1 -- 172.21.15.60:0/3849941942 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58032440 con 0x7f2d6815dd30 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d667fc640 1 -- 172.21.15.60:0/3849941942 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58032810 con 0x7f2d6815dd30 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815dd30 msgr2=0x7f2d680b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815dd30 0x7f2d680b9230 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009870 tx=0x7f2d5802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3849941942 shutdown_connections 2024-04-25T08:39:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d680d0ad0 0x7f2d680b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3849941942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d6815dd30 0x7f2d680b9230 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3849941942 >> 172.21.15.60:0/3849941942 conn(0x7f2d68061c30 msgr2=0x7f2d6806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3849941942 shutdown_connections 2024-04-25T08:39:26.985 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.989+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3849941942 wait complete. 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.990+0000 7f2d6e0c6640 1 Processor -- start 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.990+0000 7f2d6e0c6640 1 -- start start 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d6e0c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 0x7f2d680cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d6e0c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d680d0ad0 0x7f2d680b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d6e0c6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6815dd30 0x7f2d680b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d6e0c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680d1b90 con 0x7f2d680b9770 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d6e0c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680d1a10 con 0x7f2d680d0ad0 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d6e0c6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680d2e90 con 0x7f2d6815dd30 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 0x7f2d680cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d677fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 0x7f2d680cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52300/0 (socket says 172.21.15.60:52300) 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d677fe640 1 -- 172.21.15.60:0/3209759285 learned_addr learned my addr 172.21.15.60:0/3209759285 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.991+0000 7f2d67fff640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6815dd30 0x7f2d680b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d66ffd640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d680d0ad0 0x7f2d680b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d677fe640 1 -- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6815dd30 msgr2=0x7f2d680b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d677fe640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6815dd30 0x7f2d680b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d677fe640 1 -- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d680d0ad0 msgr2=0x7f2d680b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d677fe640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d680d0ad0 0x7f2d680b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d677fe640 1 -- 172.21.15.60:0/3209759285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680cb8d0 con 0x7f2d680b9770 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d67fff640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6815dd30 0x7f2d680b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d677fe640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 0x7f2d680cd410 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f2d58000c00 tx=0x7f2d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:26.989 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58037040 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58042070 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d681f37c0 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58030880 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.992+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d681f3d80 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.993+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d68073990 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.994+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d5802ad10 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.994+0000 7f2d4ffff640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2d5406b4d0 0x7f2d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.994+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d580aa8d0 con 0x7f2d680b9770 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.994+0000 7f2d66ffd640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2d5406b4d0 0x7f2d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:26.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.995+0000 7f2d66ffd640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2d5406b4d0 0x7f2d5406d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c007fd0 tx=0x7f2d5c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:26.992 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:26.998+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d58079650 con 0x7f2d680b9770 2024-04-25T08:39:27.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.178+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d680c97e0 con 0x7f2d680b9770 2024-04-25T08:39:27.173 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.180+0000 7f2d4ffff640 1 -- 172.21.15.60:0/3209759285 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f2d58078f50 con 0x7f2d680b9770 2024-04-25T08:39:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:27.174 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":144,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:25.890501+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:27.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2d5406b4d0 msgr2=0x7f2d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2d5406b4d0 0x7f2d5406d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c007fd0 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 msgr2=0x7f2d680cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 0x7f2d680cd410 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f2d58000c00 tx=0x7f2d580046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 shutdown_connections 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2d6815dd30 0x7f2d680b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2d5406b4d0 0x7f2d5406d990 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d680d0ad0 0x7f2d680b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 --2- 172.21.15.60:0/3209759285 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d680b9770 0x7f2d680cd410 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 >> 172.21.15.60:0/3209759285 conn(0x7f2d68061c30 msgr2=0x7f2d680c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 shutdown_connections 2024-04-25T08:39:27.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.182+0000 7f2d6e0c6640 1 -- 172.21.15.60:0/3209759285 wait complete. 2024-04-25T08:39:27.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:39:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.298+0000 7f637cb22640 1 Processor -- start 2024-04-25T08:39:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.298+0000 7f637cb22640 1 -- start start 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637cb22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f637815b7d0 0x7f63780cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637cb22640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637806cf50 con 0x7f637806c9e0 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637cb22640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63780ce530 con 0x7f63780b5620 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637cb22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63780ce810 con 0x7f637815b7d0 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637659c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f637815b7d0 0x7f63780cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f6376d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f637806c9e0 0x7f63780d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34542/0 (socket says 172.21.15.60:34542) 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f6376d9d640 1 -- 172.21.15.60:0/877027573 learned_addr learned my addr 172.21.15.60:0/877027573 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637659c640 1 -- 172.21.15.60:0/877027573 >> v1:172.21.15.100:6789/0 conn(0x7f63780b5620 legacy=0x7f637806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637659c640 1 -- 172.21.15.60:0/877027573 >> v1:172.21.15.60:6789/0 conn(0x7f637806c9e0 legacy=0x7f63780d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f637659c640 1 -- 172.21.15.60:0/877027573 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63780ceaf0 con 0x7f637815b7d0 2024-04-25T08:39:27.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.299+0000 7f6375d9b640 1 -- 172.21.15.60:0/877027573 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121503218 0 0) 0x7f637806cf50 con 0x7f637806c9e0 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.300+0000 7f637659c640 1 --2- 172.21.15.60:0/877027573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780cf010 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f63680097a0 tx=0x7f636802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce3e83f59bd0383 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.300+0000 7f6375d9b640 1 -- 172.21.15.60:0/877027573 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368037070 con 0x7f637815b7d0 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.300+0000 7f6375d9b640 1 -- 172.21.15.60:0/877027573 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368032440 con 0x7f637815b7d0 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.300+0000 7f6375d9b640 1 -- 172.21.15.60:0/877027573 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368032810 con 0x7f637815b7d0 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.300+0000 7f637cb22640 1 -- 172.21.15.60:0/877027573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 msgr2=0x7f63780cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.300+0000 7f637cb22640 1 --2- 172.21.15.60:0/877027573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780cf010 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f63680097a0 tx=0x7f636802a230 comp rx=0 tx=0).stop 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.301+0000 7f637cb22640 1 -- 172.21.15.60:0/877027573 shutdown_connections 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.301+0000 7f637cb22640 1 --2- 172.21.15.60:0/877027573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780cf010 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.301+0000 7f637cb22640 1 -- 172.21.15.60:0/877027573 >> 172.21.15.60:0/877027573 conn(0x7f63780d0800 msgr2=0x7f63780d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.301+0000 7f637cb22640 1 -- 172.21.15.60:0/877027573 shutdown_connections 2024-04-25T08:39:27.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.301+0000 7f637cb22640 1 -- 172.21.15.60:0/877027573 wait complete. 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.302+0000 7f637cb22640 1 Processor -- start 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.302+0000 7f637cb22640 1 -- start start 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637cb22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637806c9e0 0x7f63781f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637cb22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63780b5620 0x7f63781f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637cb22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637cb22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780cf6d0 con 0x7f637815b7d0 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637cb22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780cf550 con 0x7f637806c9e0 2024-04-25T08:39:27.298 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637cb22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637815bbb0 con 0x7f63780b5620 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52320/0 (socket says 172.21.15.60:52320) 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 -- 172.21.15.60:0/2249969810 learned_addr learned my addr 172.21.15.60:0/2249969810 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637659c640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637806c9e0 0x7f63781f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f636dd9b640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63780b5620 0x7f63781f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 -- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63780b5620 msgr2=0x7f63781f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63780b5620 0x7f63781f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 -- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637806c9e0 msgr2=0x7f63781f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637806c9e0 0x7f63781f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 -- 172.21.15.60:0/2249969810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63780cd690 con 0x7f637815b7d0 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f637659c640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637806c9e0 0x7f63781f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.303+0000 7f6376d9d640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780ccd40 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f637000c830 tx=0x7f637000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:27.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.304+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370018070 con 0x7f637815b7d0 2024-04-25T08:39:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.304+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637000ce20 con 0x7f637815b7d0 2024-04-25T08:39:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.304+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000a880 con 0x7f637815b7d0 2024-04-25T08:39:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.304+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63780cd970 con 0x7f637815b7d0 2024-04-25T08:39:27.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.304+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63780cdf30 con 0x7f637815b7d0 2024-04-25T08:39:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.305+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63780ce310 con 0x7f637815b7d0 2024-04-25T08:39:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.305+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f637000aa20 con 0x7f637815b7d0 2024-04-25T08:39:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.306+0000 7f636f7fe640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f635806b4d0 0x7f635806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.306+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6370089820 con 0x7f637815b7d0 2024-04-25T08:39:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.306+0000 7f637659c640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f635806b4d0 0x7f635806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.306+0000 7f637659c640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f635806b4d0 0x7f635806d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6368034000 tx=0x7f636802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:27.303 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.310+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63780ce310 con 0x7f637815b7d0 2024-04-25T08:39:27.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.529+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f63781f4930 con 0x7f637815b7d0 2024-04-25T08:39:27.539 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.546+0000 7f636f7fe640 1 -- 172.21.15.60:0/2249969810 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) v1 ==== 118+0+0 (secure 0 0 0) 0x7f63781f4930 con 0x7f637815b7d0 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f635806b4d0 msgr2=0x7f635806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f635806b4d0 0x7f635806d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6368034000 tx=0x7f636802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 msgr2=0x7f63780ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780ccd40 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f637000c830 tx=0x7f637000ccf0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 shutdown_connections 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f637815b7d0 0x7f63780ccd40 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63780b5620 0x7f63781f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f635806b4d0 0x7f635806d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 --2- 172.21.15.60:0/2249969810 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f637806c9e0 0x7f63781f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 >> 172.21.15.60:0/2249969810 conn(0x7f63780d0800 msgr2=0x7f637815fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 shutdown_connections 2024-04-25T08:39:27.543 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.548+0000 7f637cb22640 1 -- 172.21.15.60:0/2249969810 wait complete. 2024-04-25T08:39:27.553 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:39:27.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:39:27.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.663+0000 7f0bd9676640 1 Processor -- start 2024-04-25T08:39:27.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.663+0000 7f0bd9676640 1 -- start start 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd9676640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0bd40ce230 0x7f0bd40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd9676640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bd415b270 0x7f0bd415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd9676640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd4072280 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd9676640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd4071270 con 0x7f0bd415b270 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd9676640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd40714c0 con 0x7f0bd40ce230 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd37fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0bd415b270 0x7f0bd415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd2ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0bd40cc4f0 0x7f0bd40cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34550/0 (socket says 172.21.15.60:34550) 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd2ffd640 1 -- 172.21.15.60:0/2149298327 learned_addr learned my addr 172.21.15.60:0/2149298327 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd37fe640 1 -- 172.21.15.60:0/2149298327 >> v2:172.21.15.100:3300/0 conn(0x7f0bd40ce230 msgr2=0x7f0bd40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:27.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd37fe640 1 --2- 172.21.15.60:0/2149298327 >> v2:172.21.15.100:3300/0 conn(0x7f0bd40ce230 0x7f0bd40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd37fe640 1 -- 172.21.15.60:0/2149298327 >> v1:172.21.15.60:6789/0 conn(0x7f0bd40cc4f0 legacy=0x7f0bd40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.664+0000 7f0bd37fe640 1 -- 172.21.15.60:0/2149298327 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40717a0 con 0x7f0bd415b270 2024-04-25T08:39:27.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.665+0000 7f0bd37fe640 1 --2- 172.21.15.60:0/2149298327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415b270 0x7f0bd415b6e0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40b4c00 tx=0x7f0bcc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df873c5ad0fd792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:27.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.665+0000 7f0bd1ffb640 1 -- 172.21.15.60:0/2149298327 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc037070 con 0x7f0bd415b270 2024-04-25T08:39:27.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.665+0000 7f0bd1ffb640 1 -- 172.21.15.60:0/2149298327 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc032440 con 0x7f0bd415b270 2024-04-25T08:39:27.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.665+0000 7f0bd1ffb640 1 -- 172.21.15.60:0/2149298327 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc032810 con 0x7f0bd415b270 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 -- 172.21.15.60:0/2149298327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415b270 msgr2=0x7f0bd415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 --2- 172.21.15.60:0/2149298327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415b270 0x7f0bd415b6e0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40b4c00 tx=0x7f0bcc02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 -- 172.21.15.60:0/2149298327 shutdown_connections 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 --2- 172.21.15.60:0/2149298327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd415b270 0x7f0bd415b6e0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 --2- 172.21.15.60:0/2149298327 >> v2:172.21.15.100:3300/0 conn(0x7f0bd40ce230 0x7f0bd40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 -- 172.21.15.60:0/2149298327 >> 172.21.15.60:0/2149298327 conn(0x7f0bd406abe0 msgr2=0x7f0bd406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 -- 172.21.15.60:0/2149298327 shutdown_connections 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.666+0000 7f0bd9676640 1 -- 172.21.15.60:0/2149298327 wait complete. 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.667+0000 7f0bd9676640 1 Processor -- start 2024-04-25T08:39:27.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.667+0000 7f0bd9676640 1 -- start start 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd9676640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 0x7f0bd40cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd9676640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40ce230 0x7f0bd40cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd9676640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415b270 0x7f0bd41f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd9676640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415bc20 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd9676640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4071d60 con 0x7f0bd415b270 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd9676640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415bda0 con 0x7f0bd40ce230 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 0x7f0bd40cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 0x7f0bd40cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52330/0 (socket says 172.21.15.60:52330) 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 -- 172.21.15.60:0/1142463557 learned_addr learned my addr 172.21.15.60:0/1142463557 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd37fe640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415b270 0x7f0bd41f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd27fc640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40ce230 0x7f0bd40cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 -- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40ce230 msgr2=0x7f0bd40cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40ce230 0x7f0bd40cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 -- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415b270 msgr2=0x7f0bd41f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415b270 0x7f0bd41f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 -- 172.21.15.60:0/1142463557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4139f70 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd27fc640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40ce230 0x7f0bd40cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.668+0000 7f0bd2ffd640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 0x7f0bd40cf710 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400ca70 tx=0x7f0bc400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.669+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4017070 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.669+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc400abd0 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.669+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd413b860 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.669+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4012400 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.669+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd413be20 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.670+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd413d730 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.670+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bc401a070 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.670+0000 7f0bcb7fe640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bb406b4d0 0x7f0bb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:27.665 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.670+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bc408a040 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.671+0000 7f0bd27fc640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bb406b4d0 0x7f0bb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:27.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.671+0000 7f0bd27fc640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bb406b4d0 0x7f0bb406d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4151440 tx=0x7f0bbc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:27.668 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.675+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bc4058dc0 con 0x7f0bd40cc4f0 2024-04-25T08:39:27.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:27.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:27.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:27.894+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0bd4073a00 con 0x7f0bd40cc4f0 2024-04-25T08:39:28.546 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.552+0000 7f0bcb7fe640 1 -- 172.21.15.60:0/1142463557 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0bc40586c0 con 0x7f0bd40cc4f0 2024-04-25T08:39:28.548 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bb406b4d0 msgr2=0x7f0bb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bb406b4d0 0x7f0bb406d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4151440 tx=0x7f0bbc008040 comp rx=0 tx=0).stop 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 msgr2=0x7f0bd40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 0x7f0bd40cf710 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400ca70 tx=0x7f0bc400cf30 comp rx=0 tx=0).stop 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 shutdown_connections 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0bb406b4d0 0x7f0bb406d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bd415b270 0x7f0bd41f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0bd40ce230 0x7f0bd40cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 --2- 172.21.15.60:0/1142463557 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0bd40cc4f0 0x7f0bd40cf710 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 >> 172.21.15.60:0/1142463557 conn(0x7f0bd406abe0 msgr2=0x7f0bd40b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 shutdown_connections 2024-04-25T08:39:28.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.555+0000 7f0bd9676640 1 -- 172.21.15.60:0/1142463557 wait complete. 2024-04-25T08:39:28.562 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:39:28.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.719+0000 7f564c812640 1 Processor -- start 2024-04-25T08:39:28.712 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.719+0000 7f564c812640 1 -- start start 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f564c812640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56480d2e90 0x7f56480d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f564c812640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56480b9940 con 0x7f56480d0900 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f564c812640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56480d1df0 con 0x7f564815dd30 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f564c812640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56480d0d40 con 0x7f56480d2e90 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f5646d9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f56480d0900 0x7f56480b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34566/0 (socket says 172.21.15.60:34566) 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f5646d9d640 1 -- 172.21.15.60:0/570263805 learned_addr learned my addr 172.21.15.60:0/570263805 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434993900 0 0) 0x7f56480b9940 con 0x7f56480d0900 2024-04-25T08:39:28.716 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f563ffff640 1 --2- 172.21.15.60:0/570263805 >> v2:172.21.15.100:3300/0 conn(0x7f56480d2e90 0x7f56480d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.720+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5634003520 con 0x7f56480d0900 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134498277 0 0) 0x7f5634003520 con 0x7f56480d0900 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 >> v2:172.21.15.100:3300/0 conn(0x7f56480d2e90 msgr2=0x7f56480d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 --2- 172.21.15.60:0/570263805 >> v2:172.21.15.100:3300/0 conn(0x7f56480d2e90 0x7f56480d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 >> v1:172.21.15.100:6789/0 conn(0x7f564815dd30 legacy=0x7f56480b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480d20d0 con 0x7f56480d0900 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f56400019f0 con 0x7f56480d0900 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 >> v1:172.21.15.60:6789/0 conn(0x7f56480d0900 legacy=0x7f56480b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340039b0 0x7f5634003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56340052e0 0x7f563400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480d0d40 con 0x7f56340052e0 2024-04-25T08:39:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563400c1b0 con 0x7f5634004800 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563400c3b0 con 0x7f56340039b0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5646d9d640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56340052e0 0x7f563400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f564659c640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340039b0 0x7f5634003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f563ffff640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5646d9d640 1 -- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340039b0 msgr2=0x7f5634003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5646d9d640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340039b0 0x7f5634003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5646d9d640 1 -- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5634004800 msgr2=0x7f5634004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5646d9d640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.721+0000 7f5646d9d640 1 -- 172.21.15.60:0/570263805 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5634003520 con 0x7f56340052e0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.722+0000 7f5646d9d640 1 -- 172.21.15.60:0/570263805 reap_dead start 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.722+0000 7f5646d9d640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56340052e0 0x7f563400b9c0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f564000a830 tx=0x7f5634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e70c6b682493521 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.722+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564000a480 con 0x7f56340052e0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.722+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564002f070 con 0x7f56340052e0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.722+0000 7f5645d9b640 1 -- 172.21.15.60:0/570263805 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564002bc90 con 0x7f56340052e0 2024-04-25T08:39:28.718 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 -- 172.21.15.60:0/570263805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56340052e0 msgr2=0x7f563400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56340052e0 0x7f563400b9c0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f564000a830 tx=0x7f5634005830 comp rx=0 tx=0).stop 2024-04-25T08:39:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 -- 172.21.15.60:0/570263805 shutdown_connections 2024-04-25T08:39:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 --2- 172.21.15.60:0/570263805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f56340052e0 0x7f563400b9c0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 -- 172.21.15.60:0/570263805 >> 172.21.15.60:0/570263805 conn(0x7f5648061c30 msgr2=0x7f564806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 -- 172.21.15.60:0/570263805 shutdown_connections 2024-04-25T08:39:28.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.723+0000 7f564c812640 1 -- 172.21.15.60:0/570263805 wait complete. 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.724+0000 7f564c812640 1 Processor -- start 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.724+0000 7f564c812640 1 -- start start 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564c812640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340052e0 0x7f56480b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564c812640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56480b57c0 0x7f56481531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564c812640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 0x7f56481566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564c812640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480d2970 con 0x7f5648155350 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564c812640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480d2690 con 0x7f56480b57c0 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564c812640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56480d1a10 con 0x7f56340052e0 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 0x7f56481566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 0x7f56481566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52362/0 (socket says 172.21.15.60:52362) 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 -- 172.21.15.60:0/2770944133 learned_addr learned my addr 172.21.15.60:0/2770944133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f564659c640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340052e0 0x7f56480b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f563ffff640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56480b57c0 0x7f56481531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 -- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340052e0 msgr2=0x7f56480b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340052e0 0x7f56480b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 -- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56480b57c0 msgr2=0x7f56481531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56480b57c0 0x7f56481531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 -- 172.21.15.60:0/2770944133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480b4720 con 0x7f5648155350 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.725+0000 7f5646d9d640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 0x7f56481566d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5640000f80 tx=0x7f56480cfab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.726+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564000a480 con 0x7f5648155350 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.726+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564002bd90 con 0x7f5648155350 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.726+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004d80 con 0x7f5648155350 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.726+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56480b4a00 con 0x7f5648155350 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.726+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56480b4fc0 con 0x7f5648155350 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.727+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5648073990 con 0x7f5648155350 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.727+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5640034470 con 0x7f5648155350 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.728+0000 7f563effd640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.728+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56400aa840 con 0x7f5648155350 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.728+0000 7f564659c640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:28.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.728+0000 7f564659c640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5638000c00 tx=0x7f5638009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:28.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.732+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5640078c00 con 0x7f5648155350 2024-04-25T08:39:28.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.951+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5648153b20 con 0x7f5648155350 2024-04-25T08:39:28.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.951+0000 7f563effd640 1 -- 172.21.15.60:0/2770944133 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f5640078500 con 0x7f5648155350 2024-04-25T08:39:28.945 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:28.945 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:27.896176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:28.945 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 89 2024-04-25T08:39:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 msgr2=0x7f562806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5638000c00 tx=0x7f5638009040 comp rx=0 tx=0).stop 2024-04-25T08:39:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 msgr2=0x7f56481566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 0x7f56481566d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5640000f80 tx=0x7f56480cfab0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 shutdown_connections 2024-04-25T08:39:28.947 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5648155350 0x7f56481566d0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f562806b500 0x7f562806d9c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56480b57c0 0x7f56481531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 --2- 172.21.15.60:0/2770944133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f56340052e0 0x7f56480b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.953+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 >> 172.21.15.60:0/2770944133 conn(0x7f5648061c30 msgr2=0x7f5648160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.954+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 shutdown_connections 2024-04-25T08:39:28.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:28.954+0000 7f564c812640 1 -- 172.21.15.60:0/2770944133 wait complete. 2024-04-25T08:39:28.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:29.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.066+0000 7f220be31640 1 Processor -- start 2024-04-25T08:39:29.059 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.066+0000 7f220be31640 1 -- start start 2024-04-25T08:39:29.060 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f220be31640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22040d2e90 0x7f22040d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f220be31640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040b9940 con 0x7f220415dd30 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f220be31640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040d1df0 con 0x7f22040d0900 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f220be31640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040d0d40 con 0x7f22040d2e90 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f22093cc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22040d2e90 0x7f22040d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f2209bcd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f220415dd30 0x7f22040b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34574/0 (socket says 172.21.15.60:34574) 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f2209bcd640 1 -- 172.21.15.60:0/698329452 learned_addr learned my addr 172.21.15.60:0/698329452 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f2208bcb640 1 -- 172.21.15.60:0/698329452 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013457124 0 0) 0x7f22040b9940 con 0x7f220415dd30 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f2208bcb640 1 -- 172.21.15.60:0/698329452 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f0003520 con 0x7f220415dd30 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f22093cc640 1 -- 172.21.15.60:0/698329452 >> v1:172.21.15.100:6789/0 conn(0x7f22040d0900 legacy=0x7f22040b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f22093cc640 1 -- 172.21.15.60:0/698329452 >> v1:172.21.15.60:6789/0 conn(0x7f220415dd30 legacy=0x7f22040b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.067+0000 7f22093cc640 1 -- 172.21.15.60:0/698329452 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040d20d0 con 0x7f22040d2e90 2024-04-25T08:39:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.068+0000 7f22093cc640 1 --2- 172.21.15.60:0/698329452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22040d03c0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009790 tx=0x7f21f402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94444cfe4f26e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.068+0000 7f2208bcb640 1 -- 172.21.15.60:0/698329452 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037070 con 0x7f22040d2e90 2024-04-25T08:39:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.068+0000 7f2208bcb640 1 -- 172.21.15.60:0/698329452 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f402ae50 con 0x7f22040d2e90 2024-04-25T08:39:29.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.068+0000 7f2208bcb640 1 -- 172.21.15.60:0/698329452 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4032660 con 0x7f22040d2e90 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.068+0000 7f220be31640 1 -- 172.21.15.60:0/698329452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 msgr2=0x7f22040d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.068+0000 7f220be31640 1 --2- 172.21.15.60:0/698329452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22040d03c0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009790 tx=0x7f21f402a100 comp rx=0 tx=0).stop 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.069+0000 7f220be31640 1 -- 172.21.15.60:0/698329452 shutdown_connections 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.069+0000 7f220be31640 1 --2- 172.21.15.60:0/698329452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22040d03c0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.069+0000 7f220be31640 1 -- 172.21.15.60:0/698329452 >> 172.21.15.60:0/698329452 conn(0x7f2204061c30 msgr2=0x7f220406c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.069+0000 7f220be31640 1 -- 172.21.15.60:0/698329452 shutdown_connections 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.069+0000 7f220be31640 1 -- 172.21.15.60:0/698329452 wait complete. 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 Processor -- start 2024-04-25T08:39:29.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 -- start start 2024-04-25T08:39:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22040d0900 0x7f220414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22041598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f220415dd30 0x7f2204159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040d2970 con 0x7f22040d2e90 2024-04-25T08:39:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040d2690 con 0x7f22040d0900 2024-04-25T08:39:29.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.070+0000 7f220be31640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040d1a10 con 0x7f220415dd30 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22041598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22041598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52384/0 (socket says 172.21.15.60:52384) 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 -- 172.21.15.60:0/1715226075 learned_addr learned my addr 172.21.15.60:0/1715226075 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f2209bcd640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22040d0900 0x7f220414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f220a3ce640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f220415dd30 0x7f2204159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 -- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f220415dd30 msgr2=0x7f2204159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f220415dd30 0x7f2204159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 -- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22040d0900 msgr2=0x7f220414ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22040d0900 0x7f220414ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 -- 172.21.15.60:0/1715226075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204156880 con 0x7f22040d2e90 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f220a3ce640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f220415dd30 0x7f2204159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:29.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f22093cc640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22041598d0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f21f40098b0 tx=0x7f21f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037040 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4042070 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2204156b60 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4006030 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.071+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040b4580 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.072+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204073990 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.073+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21f403a070 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.073+0000 7f21fa7fc640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.073+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21f40aa480 con 0x7f22040d2e90 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.073+0000 7f2209bcd640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.074+0000 7f2209bcd640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21e806b4d0 0x7f21e806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22000011e0 tx=0x7f2200007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.077+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21f4079200 con 0x7f22040d2e90 2024-04-25T08:39:29.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.258+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22040b7d90 con 0x7f22040d2e90 2024-04-25T08:39:29.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.259+0000 7f21fa7fc640 1 -- 172.21.15.60:0/1715226075 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f21f4078b00 con 0x7f22040d2e90 2024-04-25T08:39:29.253 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:29.253 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":145,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:27.531391+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21e806b4d0 msgr2=0x7f21e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21e806b4d0 0x7f21e806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f22000011e0 tx=0x7f2200007d30 comp rx=0 tx=0).stop 2024-04-25T08:39:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 msgr2=0x7f22041598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22041598d0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f21f40098b0 tx=0x7f21f40046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 shutdown_connections 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f220415dd30 0x7f2204159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22040d2e90 0x7f22041598d0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21e806b4d0 0x7f21e806d990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.261+0000 7f220be31640 1 --2- 172.21.15.60:0/1715226075 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22040d0900 0x7f220414ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.262+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 >> 172.21.15.60:0/1715226075 conn(0x7f2204061c30 msgr2=0x7f22040b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.262+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 shutdown_connections 2024-04-25T08:39:29.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.262+0000 7f220be31640 1 -- 172.21.15.60:0/1715226075 wait complete. 2024-04-25T08:39:29.266 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:39:29.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.373+0000 7f5dbd207640 1 Processor -- start 2024-04-25T08:39:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.373+0000 7f5dbd207640 1 -- start start 2024-04-25T08:39:29.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.374+0000 7f5dbd207640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5db80ce590 0x7f5db8165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.374+0000 7f5dbd207640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db8071480 con 0x7f5db80cfec0 2024-04-25T08:39:29.370 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.374+0000 7f5dbd207640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db8071760 con 0x7f5db8154ee0 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.374+0000 7f5dbd207640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db8072090 con 0x7f5db80ce590 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.374+0000 7f5db659c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5db80cfec0 0x7f5db80cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34582/0 (socket says 172.21.15.60:34582) 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.374+0000 7f5db659c640 1 -- 172.21.15.60:0/882080092 learned_addr learned my addr 172.21.15.60:0/882080092 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896170351 0 0) 0x7f5db8071480 con 0x7f5db80cfec0 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da0003520 con 0x7f5db80cfec0 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db759e640 1 --2- 172.21.15.60:0/882080092 >> v2:172.21.15.100:3300/0 conn(0x7f5db80ce590 0x7f5db8165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1010334828 0 0) 0x7f5da0003520 con 0x7f5db80cfec0 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 >> v2:172.21.15.100:3300/0 conn(0x7f5db80ce590 msgr2=0x7f5db8165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 --2- 172.21.15.60:0/882080092 >> v2:172.21.15.100:3300/0 conn(0x7f5db80ce590 0x7f5db8165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 >> v1:172.21.15.100:6789/0 conn(0x7f5db8154ee0 legacy=0x7f5db80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8071a40 con 0x7f5db80cfec0 2024-04-25T08:39:29.371 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f5dac001670 con 0x7f5db80cfec0 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 >> v1:172.21.15.60:6789/0 conn(0x7f5db80cfec0 legacy=0x7f5db80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da00039b0 0x7f5da0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000c030 con 0x7f5da0004800 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000c1e0 con 0x7f5da00051c0 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000c3e0 con 0x7f5da00039b0 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db659c640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db759e640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db6d9d640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da00039b0 0x7f5da0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db659c640 1 -- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da00039b0 msgr2=0x7f5da0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db659c640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5da00039b0 0x7f5da0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db659c640 1 -- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00051c0 msgr2=0x7f5da000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.375+0000 7f5db659c640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5da00051c0 0x7f5da000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.376+0000 7f5db759e640 1 -- 172.21.15.60:0/882080092 reap_dead start 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.376+0000 7f5db659c640 1 -- 172.21.15.60:0/882080092 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da0003520 con 0x7f5da0004800 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.376+0000 7f5db659c640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c240 tx=0x7f5da0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d742846442fd1fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.376+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac007d10 con 0x7f5da0004800 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.376+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac005ad0 con 0x7f5da0004800 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.376+0000 7f5db5d9b640 1 -- 172.21.15.60:0/882080092 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac007070 con 0x7f5da0004800 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 -- 172.21.15.60:0/882080092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 msgr2=0x7f5da0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c240 tx=0x7f5da0005830 comp rx=0 tx=0).stop 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 -- 172.21.15.60:0/882080092 shutdown_connections 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 --2- 172.21.15.60:0/882080092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5da0004c80 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 -- 172.21.15.60:0/882080092 >> 172.21.15.60:0/882080092 conn(0x7f5db806abe0 msgr2=0x7f5db806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:29.373 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 -- 172.21.15.60:0/882080092 shutdown_connections 2024-04-25T08:39:29.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.377+0000 7f5dbd207640 1 -- 172.21.15.60:0/882080092 wait complete. 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.378+0000 7f5dbd207640 1 Processor -- start 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.378+0000 7f5dbd207640 1 -- start start 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5dbd207640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5db8144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5dbd207640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db8154170 0x7f5db80b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5dbd207640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5db815cfa0 0x7f5db815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5dbd207640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db81552c0 con 0x7f5da0004800 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5dbd207640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8071230 con 0x7f5db8154170 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5dbd207640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8155440 con 0x7f5db815cfa0 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5db8144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5db8144d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52406/0 (socket says 172.21.15.60:52406) 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 -- 172.21.15.60:0/3974280528 learned_addr learned my addr 172.21.15.60:0/3974280528 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:29.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db659c640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db8154170 0x7f5db80b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db759e640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5db815cfa0 0x7f5db815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 -- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5db815cfa0 msgr2=0x7f5db815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5db815cfa0 0x7f5db815d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 -- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db8154170 msgr2=0x7f5db80b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db8154170 0x7f5db80b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db6d9d640 1 -- 172.21.15.60:0/3974280528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8147510 con 0x7f5da0004800 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.379+0000 7f5db659c640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db8154170 0x7f5db80b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5db759e640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5db815cfa0 0x7f5db815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5db6d9d640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5db8144d70 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e470 tx=0x7f5db000e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0019070 con 0x7f5da0004800 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db00052e0 con 0x7f5da0004800 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db81477f0 con 0x7f5da0004800 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00037f0 con 0x7f5da0004800 2024-04-25T08:39:29.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.380+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8147db0 con 0x7f5da0004800 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.381+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db8073990 con 0x7f5da0004800 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.381+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5db001c070 con 0x7f5da0004800 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.382+0000 7f5d9f7fe640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5da406b500 0x7f5da406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.382+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5db0089740 con 0x7f5da0004800 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.382+0000 7f5db659c640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5da406b500 0x7f5da406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.383+0000 7f5db659c640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5da406b500 0x7f5da406d9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5dac032000 tx=0x7f5dac034040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.379 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.386+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5db0057e70 con 0x7f5da0004800 2024-04-25T08:39:29.557 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.564+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5db8148120 con 0x7f5da406b500 2024-04-25T08:39:29.565 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.571+0000 7f5d9f7fe640 1 -- 172.21.15.60:0/3974280528 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242992 (secure 0 0 0) 0x7f5db8148120 con 0x7f5da406b500 2024-04-25T08:39:29.568 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:29.569 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:39:29.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5da406b500 msgr2=0x7f5da406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5da406b500 0x7f5da406d9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5dac032000 tx=0x7f5dac034040 comp rx=0 tx=0).stop 2024-04-25T08:39:29.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 msgr2=0x7f5db8144d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5db8144d70 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e470 tx=0x7f5db000e930 comp rx=0 tx=0).stop 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 shutdown_connections 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5db815cfa0 0x7f5db815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5da406b500 0x7f5da406d9c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5db8154170 0x7f5db80b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 --2- 172.21.15.60:0/3974280528 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5da0004800 0x7f5db8144d70 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.574+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 >> 172.21.15.60:0/3974280528 conn(0x7f5db806abe0 msgr2=0x7f5db806d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.575+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 shutdown_connections 2024-04-25T08:39:29.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.575+0000 7f5dbd207640 1 -- 172.21.15.60:0/3974280528 wait complete. 2024-04-25T08:39:29.579 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-04-25T08:39:29.538661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425596,"kb_used_data":15084,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754549124,"statfs":{"total":773094113280,"available":772658302976,"internally_reserved":0,"allocated":15446016,"data_stored":7832377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.019512"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532972+0000","last_change":"2024-04-25T08:39:25.532972+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532972+0000","last_undegraded":"2024-04-25T08:39:25.532972+0000","last_fullsized":"2024-04-25T08:39:25.532972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.533598+0000","last_change":"2024-04-25T08:39:25.533598+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.533598+0000","last_undegraded":"2024-04-25T08:39:25.533598+0000","last_fullsized":"2024-04-25T08:39:25.533598+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531925+0000","last_change":"2024-04-25T08:39:25.531925+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531925+0000","last_undegraded":"2024-04-25T08:39:25.531925+0000","last_fullsized":"2024-04-25T08:39:25.531925+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527182+0000","last_change":"2024-04-25T08:39:25.527182+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527182+0000","last_undegraded":"2024-04-25T08:39:25.527182+0000","last_fullsized":"2024-04-25T08:39:25.527182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525519+0000","last_change":"2024-04-25T08:39:25.525519+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525519+0000","last_undegraded":"2024-04-25T08:39:25.525519+0000","last_fullsized":"2024-04-25T08:39:25.525519+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.528068+0000","last_change":"2024-04-25T08:39:25.528068+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.528068+0000","last_undegraded":"2024-04-25T08:39:25.528068+0000","last_fullsized":"2024-04-25T08:39:25.528068+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531168+0000","last_change":"2024-04-25T08:39:25.531168+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531168+0000","last_undegraded":"2024-04-25T08:39:25.531168+0000","last_fullsized":"2024-04-25T08:39:25.531168+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529355+0000","last_change":"2024-04-25T08:39:25.529355+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529355+0000","last_undegraded":"2024-04-25T08:39:25.529355+0000","last_fullsized":"2024-04-25T08:39:25.529355+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532541+0000","last_change":"2024-04-25T08:39:25.532541+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532541+0000","last_undegraded":"2024-04-25T08:39:25.532541+0000","last_fullsized":"2024-04-25T08:39:25.532541+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530852+0000","last_change":"2024-04-25T08:39:25.530852+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530852+0000","last_undegraded":"2024-04-25T08:39:25.530852+0000","last_fullsized":"2024-04-25T08:39:25.530852+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521260+0000","last_change":"2024-04-25T08:39:24.521260+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521260+0000","last_undegraded":"2024-04-25T08:39:24.521260+0000","last_fullsized":"2024-04-25T08:39:24.521260+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542880+0000","last_change":"2024-04-25T08:39:25.541869+0000","last_active":"2024-04-25T08:39:25.542880+0000","last_peered":"2024-04-25T08:39:25.542880+0000","last_clean":"2024-04-25T08:39:25.542880+0000","last_became_active":"2024-04-25T08:39:25.541531+0000","last_became_peered":"2024-04-25T08:39:25.541531+0000","last_unstale":"2024-04-25T08:39:25.542880+0000","last_undegraded":"2024-04-25T08:39:25.542880+0000","last_fullsized":"2024-04-25T08:39:25.542880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:57.694019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542218+0000","last_change":"2024-04-25T08:39:25.540429+0000","last_active":"2024-04-25T08:39:25.542218+0000","last_peered":"2024-04-25T08:39:25.542218+0000","last_clean":"2024-04-25T08:39:25.542218+0000","last_became_active":"2024-04-25T08:39:25.539563+0000","last_became_peered":"2024-04-25T08:39:25.539563+0000","last_unstale":"2024-04-25T08:39:25.542218+0000","last_undegraded":"2024-04-25T08:39:25.542218+0000","last_fullsized":"2024-04-25T08:39:25.542218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:52.290637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530037+0000","last_change":"2024-04-25T08:39:25.530037+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530037+0000","last_undegraded":"2024-04-25T08:39:25.530037+0000","last_fullsized":"2024-04-25T08:39:25.530037+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.522955+0000","last_change":"2024-04-25T08:39:24.522955+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.522955+0000","last_undegraded":"2024-04-25T08:39:24.522955+0000","last_fullsized":"2024-04-25T08:39:24.522955+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524974+0000","last_change":"2024-04-25T08:39:24.524974+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524974+0000","last_undegraded":"2024-04-25T08:39:24.524974+0000","last_fullsized":"2024-04-25T08:39:24.524974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525161+0000","last_change":"2024-04-25T08:39:25.525161+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525161+0000","last_undegraded":"2024-04-25T08:39:25.525161+0000","last_fullsized":"2024-04-25T08:39:25.525161+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521231+0000","last_change":"2024-04-25T08:39:24.521231+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521231+0000","last_undegraded":"2024-04-25T08:39:24.521231+0000","last_fullsized":"2024-04-25T08:39:24.521231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.539017+0000","last_change":"2024-04-25T08:39:25.537615+0000","last_active":"2024-04-25T08:39:25.539017+0000","last_peered":"2024-04-25T08:39:25.539017+0000","last_clean":"2024-04-25T08:39:25.539017+0000","last_became_active":"2024-04-25T08:39:25.537345+0000","last_became_peered":"2024-04-25T08:39:25.537345+0000","last_unstale":"2024-04-25T08:39:25.539017+0000","last_undegraded":"2024-04-25T08:39:25.539017+0000","last_fullsized":"2024-04-25T08:39:25.539017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:50.758570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529484+0000","last_change":"2024-04-25T08:39:25.529484+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529484+0000","last_undegraded":"2024-04-25T08:39:25.529484+0000","last_fullsized":"2024-04-25T08:39:25.529484+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521239+0000","last_change":"2024-04-25T08:39:24.521239+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521239+0000","last_undegraded":"2024-04-25T08:39:24.521239+0000","last_fullsized":"2024-04-25T08:39:24.521239+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542936+0000","last_change":"2024-04-25T08:39:25.541562+0000","last_active":"2024-04-25T08:39:25.542936+0000","last_peered":"2024-04-25T08:39:25.542936+0000","last_clean":"2024-04-25T08:39:25.542936+0000","last_became_active":"2024-04-25T08:39:25.541283+0000","last_became_peered":"2024-04-25T08:39:25.541283+0000","last_unstale":"2024-04-25T08:39:25.542936+0000","last_undegraded":"2024-04-25T08:39:25.542936+0000","last_fullsized":"2024-04-25T08:39:25.542936+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.416592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543099+0000","last_change":"2024-04-25T08:39:25.540300+0000","last_active":"2024-04-25T08:39:25.543099+0000","last_peered":"2024-04-25T08:39:25.543099+0000","last_clean":"2024-04-25T08:39:25.543099+0000","last_became_active":"2024-04-25T08:39:25.539840+0000","last_became_peered":"2024-04-25T08:39:25.539840+0000","last_unstale":"2024-04-25T08:39:25.543099+0000","last_undegraded":"2024-04-25T08:39:25.543099+0000","last_fullsized":"2024-04-25T08:39:25.543099+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523795+0000","last_change":"2024-04-25T08:39:24.523795+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523795+0000","last_undegraded":"2024-04-25T08:39:24.523795+0000","last_fullsized":"2024-04-25T08:39:24.523795+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521328+0000","last_change":"2024-04-25T08:39:24.521328+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521328+0000","last_undegraded":"2024-04-25T08:39:24.521328+0000","last_fullsized":"2024-04-25T08:39:24.521328+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.531819+0000","last_change":"2024-04-25T08:39:25.531155+0000","last_active":"2024-04-25T08:39:25.531819+0000","last_peered":"2024-04-25T08:39:25.531819+0000","last_clean":"2024-04-25T08:39:25.531819+0000","last_became_active":"2024-04-25T08:39:25.530883+0000","last_became_peered":"2024-04-25T08:39:25.530883+0000","last_unstale":"2024-04-25T08:39:25.531819+0000","last_undegraded":"2024-04-25T08:39:25.531819+0000","last_fullsized":"2024-04-25T08:39:25.531819+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543391+0000","last_change":"2024-04-25T08:39:25.542778+0000","last_active":"2024-04-25T08:39:25.543391+0000","last_peered":"2024-04-25T08:39:25.543391+0000","last_clean":"2024-04-25T08:39:25.543391+0000","last_became_active":"2024-04-25T08:39:25.542474+0000","last_became_peered":"2024-04-25T08:39:25.542474+0000","last_unstale":"2024-04-25T08:39:25.543391+0000","last_undegraded":"2024-04-25T08:39:25.543391+0000","last_fullsized":"2024-04-25T08:39:25.543391+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543025+0000","last_change":"2024-04-25T08:39:25.540523+0000","last_active":"2024-04-25T08:39:25.543025+0000","last_peered":"2024-04-25T08:39:25.543025+0000","last_clean":"2024-04-25T08:39:25.543025+0000","last_became_active":"2024-04-25T08:39:25.540179+0000","last_became_peered":"2024-04-25T08:39:25.540179+0000","last_unstale":"2024-04-25T08:39:25.543025+0000","last_undegraded":"2024-04-25T08:39:25.543025+0000","last_fullsized":"2024-04-25T08:39:25.543025+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529469+0000","last_change":"2024-04-25T08:39:25.529469+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529469+0000","last_undegraded":"2024-04-25T08:39:25.529469+0000","last_fullsized":"2024-04-25T08:39:25.529469+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523172+0000","last_change":"2024-04-25T08:39:24.523172+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523172+0000","last_undegraded":"2024-04-25T08:39:24.523172+0000","last_fullsized":"2024-04-25T08:39:24.523172+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530763+0000","last_change":"2024-04-25T08:39:25.530763+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530763+0000","last_undegraded":"2024-04-25T08:39:25.530763+0000","last_fullsized":"2024-04-25T08:39:25.530763+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544961+0000","last_change":"2024-04-25T08:39:25.544537+0000","last_active":"2024-04-25T08:39:25.544961+0000","last_peered":"2024-04-25T08:39:25.544961+0000","last_clean":"2024-04-25T08:39:25.544961+0000","last_became_active":"2024-04-25T08:39:25.544239+0000","last_became_peered":"2024-04-25T08:39:25.544239+0000","last_unstale":"2024-04-25T08:39:25.544961+0000","last_undegraded":"2024-04-25T08:39:25.544961+0000","last_fullsized":"2024-04-25T08:39:25.544961+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:09.533163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530428+0000","last_change":"2024-04-25T08:39:25.530428+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530428+0000","last_undegraded":"2024-04-25T08:39:25.530428+0000","last_fullsized":"2024-04-25T08:39:25.530428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523778+0000","last_change":"2024-04-25T08:39:24.523778+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523778+0000","last_undegraded":"2024-04-25T08:39:24.523778+0000","last_fullsized":"2024-04-25T08:39:24.523778+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529985+0000","last_change":"2024-04-25T08:39:25.529985+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529985+0000","last_undegraded":"2024-04-25T08:39:25.529985+0000","last_fullsized":"2024-04-25T08:39:25.529985+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.536302+0000","last_change":"2024-04-25T08:39:25.534723+0000","last_active":"2024-04-25T08:39:25.536302+0000","last_peered":"2024-04-25T08:39:25.536302+0000","last_clean":"2024-04-25T08:39:25.536302+0000","last_became_active":"2024-04-25T08:39:25.534462+0000","last_became_peered":"2024-04-25T08:39:25.534462+0000","last_unstale":"2024-04-25T08:39:25.536302+0000","last_undegraded":"2024-04-25T08:39:25.536302+0000","last_fullsized":"2024-04-25T08:39:25.536302+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529814+0000","last_change":"2024-04-25T08:39:25.529814+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529814+0000","last_undegraded":"2024-04-25T08:39:25.529814+0000","last_fullsized":"2024-04-25T08:39:25.529814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524065+0000","last_change":"2024-04-25T08:39:24.524065+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524065+0000","last_undegraded":"2024-04-25T08:39:24.524065+0000","last_fullsized":"2024-04-25T08:39:24.524065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529165+0000","last_change":"2024-04-25T08:39:25.529165+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529165+0000","last_undegraded":"2024-04-25T08:39:25.529165+0000","last_fullsized":"2024-04-25T08:39:25.529165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.525390+0000","last_change":"2024-04-25T08:39:24.525390+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.525390+0000","last_undegraded":"2024-04-25T08:39:24.525390+0000","last_fullsized":"2024-04-25T08:39:24.525390+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.535065+0000","last_change":"2024-04-25T08:39:25.533867+0000","last_active":"2024-04-25T08:39:25.535065+0000","last_peered":"2024-04-25T08:39:25.535065+0000","last_clean":"2024-04-25T08:39:25.535065+0000","last_became_active":"2024-04-25T08:39:25.533567+0000","last_became_peered":"2024-04-25T08:39:25.533567+0000","last_unstale":"2024-04-25T08:39:25.535065+0000","last_undegraded":"2024-04-25T08:39:25.535065+0000","last_fullsized":"2024-04-25T08:39:25.535065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527139+0000","last_change":"2024-04-25T08:39:25.527139+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527139+0000","last_undegraded":"2024-04-25T08:39:25.527139+0000","last_fullsized":"2024-04-25T08:39:25.527139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543080+0000","last_change":"2024-04-25T08:39:25.540746+0000","last_active":"2024-04-25T08:39:25.543080+0000","last_peered":"2024-04-25T08:39:25.543080+0000","last_clean":"2024-04-25T08:39:25.543080+0000","last_became_active":"2024-04-25T08:39:25.540230+0000","last_became_peered":"2024-04-25T08:39:25.540230+0000","last_unstale":"2024-04-25T08:39:25.543080+0000","last_undegraded":"2024-04-25T08:39:25.543080+0000","last_fullsized":"2024-04-25T08:39:25.543080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529530+0000","last_change":"2024-04-25T08:39:25.529530+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529530+0000","last_undegraded":"2024-04-25T08:39:25.529530+0000","last_fullsized":"2024-04-25T08:39:25.529530+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523429+0000","last_change":"2024-04-25T08:39:24.523429+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523429+0000","last_undegraded":"2024-04-25T08:39:24.523429+0000","last_fullsized":"2024-04-25T08:39:24.523429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521968+0000","last_change":"2024-04-25T08:39:24.521968+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521968+0000","last_undegraded":"2024-04-25T08:39:24.521968+0000","last_fullsized":"2024-04-25T08:39:24.521968+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.541132+0000","last_change":"2024-04-25T08:39:25.539417+0000","last_active":"2024-04-25T08:39:25.541132+0000","last_peered":"2024-04-25T08:39:25.541132+0000","last_clean":"2024-04-25T08:39:25.541132+0000","last_became_active":"2024-04-25T08:39:25.539015+0000","last_became_peered":"2024-04-25T08:39:25.539015+0000","last_unstale":"2024-04-25T08:39:25.541132+0000","last_undegraded":"2024-04-25T08:39:25.541132+0000","last_fullsized":"2024-04-25T08:39:25.541132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.524638+0000","last_change":"2024-04-25T08:39:25.524638+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.524638+0000","last_undegraded":"2024-04-25T08:39:25.524638+0000","last_fullsized":"2024-04-25T08:39:25.524638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523976+0000","last_change":"2024-04-25T08:39:24.523976+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523976+0000","last_undegraded":"2024-04-25T08:39:24.523976+0000","last_fullsized":"2024-04-25T08:39:24.523976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544089+0000","last_change":"2024-04-25T08:39:25.543519+0000","last_active":"2024-04-25T08:39:25.544089+0000","last_peered":"2024-04-25T08:39:25.544089+0000","last_clean":"2024-04-25T08:39:25.544089+0000","last_became_active":"2024-04-25T08:39:25.543148+0000","last_became_peered":"2024-04-25T08:39:25.543148+0000","last_unstale":"2024-04-25T08:39:25.544089+0000","last_undegraded":"2024-04-25T08:39:25.544089+0000","last_fullsized":"2024-04-25T08:39:25.544089+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.721871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.538234+0000","last_change":"2024-04-25T08:39:25.536756+0000","last_active":"2024-04-25T08:39:25.538234+0000","last_peered":"2024-04-25T08:39:25.538234+0000","last_clean":"2024-04-25T08:39:25.538234+0000","last_became_active":"2024-04-25T08:39:25.535762+0000","last_became_peered":"2024-04-25T08:39:25.535762+0000","last_unstale":"2024-04-25T08:39:25.538234+0000","last_undegraded":"2024-04-25T08:39:25.538234+0000","last_fullsized":"2024-04-25T08:39:25.538234+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:10.416181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522508+0000","last_change":"2024-04-25T08:39:25.522508+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522508+0000","last_undegraded":"2024-04-25T08:39:25.522508+0000","last_fullsized":"2024-04-25T08:39:25.522508+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":398,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.532864+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:25.532864+0000","last_peered":"2024-04-25T08:39:25.532864+0000","last_clean":"2024-04-25T08:39:25.532864+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:25.532864+0000","last_undegraded":"2024-04-25T08:39:25.532864+0000","last_fullsized":"2024-04-25T08:39:25.532864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1425990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542271,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50980,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320860,"statfs":{"total":96636764160,"available":96584560640,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542271,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46824,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325016,"statfs":{"total":96636764160,"available":96588816384,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46804,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325036,"statfs":{"total":96636764160,"available":96588836864,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59292,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312548,"statfs":{"total":96636764160,"available":96576049152,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2371584,"data_stored":1419323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:29.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.726+0000 7f0c4bd7e640 1 Processor -- start 2024-04-25T08:39:29.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.727+0000 7f0c4bd7e640 1 -- start start 2024-04-25T08:39:29.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.727+0000 7f0c4bd7e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c440d0b10 0x7f0c440ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.727+0000 7f0c4bd7e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c440cec00 con 0x7f0c4415db60 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.727+0000 7f0c4bd7e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c440ceee0 con 0x7f0c440b5f00 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.727+0000 7f0c4bd7e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c440ce930 con 0x7f0c440d0b10 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c49b1a640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0c4415db60 0x7f0c440d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34594/0 (socket says 172.21.15.60:34594) 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c49b1a640 1 -- 172.21.15.60:0/3889023744 learned_addr learned my addr 172.21.15.60:0/3889023744 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c49319640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f0c440d0b10 0x7f0c440ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c49319640 1 -- 172.21.15.60:0/3889023744 >> v1:172.21.15.100:6789/0 conn(0x7f0c440b5f00 legacy=0x7f0c440d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c49319640 1 -- 172.21.15.60:0/3889023744 >> v1:172.21.15.60:6789/0 conn(0x7f0c4415db60 legacy=0x7f0c440d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c49319640 1 -- 172.21.15.60:0/3889023744 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440cf1c0 con 0x7f0c440d0b10 2024-04-25T08:39:29.722 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.728+0000 7f0c48b18640 1 -- 172.21.15.60:0/3889023744 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697534799 0 0) 0x7f0c440cec00 con 0x7f0c4415db60 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c49319640 1 --2- 172.21.15.60:0/3889023744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440ce3f0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097d0 tx=0x7f0c3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63b0e35a55ae833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c48b18640 1 -- 172.21.15.60:0/3889023744 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34037070 con 0x7f0c440d0b10 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c48b18640 1 -- 172.21.15.60:0/3889023744 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34032440 con 0x7f0c440d0b10 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c48b18640 1 -- 172.21.15.60:0/3889023744 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34032810 con 0x7f0c440d0b10 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/3889023744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 msgr2=0x7f0c440ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/3889023744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440ce3f0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f0c340097d0 tx=0x7f0c3402a180 comp rx=0 tx=0).stop 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/3889023744 shutdown_connections 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/3889023744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440ce3f0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/3889023744 >> 172.21.15.60:0/3889023744 conn(0x7f0c44061600 msgr2=0x7f0c44061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/3889023744 shutdown_connections 2024-04-25T08:39:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.729+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/3889023744 wait complete. 2024-04-25T08:39:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 Processor -- start 2024-04-25T08:39:29.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 -- start start 2024-04-25T08:39:29.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c440b5f00 0x7f0c441f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c4415db60 0x7f0c44152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4415df40 con 0x7f0c440d0b10 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c440b6450 con 0x7f0c440b5f00 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.731+0000 7f0c4bd7e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4415e0c0 con 0x7f0c4415db60 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52416/0 (socket says 172.21.15.60:52416) 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 -- 172.21.15.60:0/186472729 learned_addr learned my addr 172.21.15.60:0/186472729 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49b1a640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c440b5f00 0x7f0c441f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c4a31b640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c4415db60 0x7f0c44152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 -- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c4415db60 msgr2=0x7f0c44152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c4415db60 0x7f0c44152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 -- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c440b5f00 msgr2=0x7f0c441f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c440b5f00 0x7f0c441f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 -- 172.21.15.60:0/186472729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440c8b60 con 0x7f0c440d0b10 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c49319640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440c82a0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009720 tx=0x7f0c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34037040 con 0x7f0c440d0b10 2024-04-25T08:39:29.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34004220 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c440c8e40 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3403c730 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.732+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c440c96e0 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.733+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c44073270 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.734+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c3403a070 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.734+0000 7f0c3a7fc640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c2806b4d0 0x7f0c2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.734+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c340aacd0 con 0x7f0c440d0b10 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.734+0000 7f0c49b1a640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c2806b4d0 0x7f0c2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:29.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.735+0000 7f0c49b1a640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c2806b4d0 0x7f0c2806d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0c40000f60 tx=0x7f0c40007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.738+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c3404a030 con 0x7f0c440d0b10 2024-04-25T08:39:29.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.915+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c44073990 con 0x7f0c2806b4d0 2024-04-25T08:39:29.914 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.921+0000 7f0c3a7fc640 1 -- 172.21.15.60:0/186472729 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242992 (secure 0 0 0) 0x7f0c44073990 con 0x7f0c2806b4d0 2024-04-25T08:39:29.917 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:29.918 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c2806b4d0 msgr2=0x7f0c2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c2806b4d0 0x7f0c2806d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0c40000f60 tx=0x7f0c40007c00 comp rx=0 tx=0).stop 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 msgr2=0x7f0c440c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440c82a0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009720 tx=0x7f0c340046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 shutdown_connections 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0c4415db60 0x7f0c44152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0c440d0b10 0x7f0c440c82a0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f0c2806b4d0 0x7f0c2806d990 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 --2- 172.21.15.60:0/186472729 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c440b5f00 0x7f0c441f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 >> 172.21.15.60:0/186472729 conn(0x7f0c44061600 msgr2=0x7f0c4415d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 shutdown_connections 2024-04-25T08:39:29.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:29.924+0000 7f0c4bd7e640 1 -- 172.21.15.60:0/186472729 wait complete. 2024-04-25T08:39:29.929 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-04-25T08:39:29.538661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425596,"kb_used_data":15084,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754549124,"statfs":{"total":773094113280,"available":772658302976,"internally_reserved":0,"allocated":15446016,"data_stored":7832377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.019512"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532972+0000","last_change":"2024-04-25T08:39:25.532972+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532972+0000","last_undegraded":"2024-04-25T08:39:25.532972+0000","last_fullsized":"2024-04-25T08:39:25.532972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.533598+0000","last_change":"2024-04-25T08:39:25.533598+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.533598+0000","last_undegraded":"2024-04-25T08:39:25.533598+0000","last_fullsized":"2024-04-25T08:39:25.533598+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531925+0000","last_change":"2024-04-25T08:39:25.531925+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531925+0000","last_undegraded":"2024-04-25T08:39:25.531925+0000","last_fullsized":"2024-04-25T08:39:25.531925+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527182+0000","last_change":"2024-04-25T08:39:25.527182+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527182+0000","last_undegraded":"2024-04-25T08:39:25.527182+0000","last_fullsized":"2024-04-25T08:39:25.527182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525519+0000","last_change":"2024-04-25T08:39:25.525519+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525519+0000","last_undegraded":"2024-04-25T08:39:25.525519+0000","last_fullsized":"2024-04-25T08:39:25.525519+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.528068+0000","last_change":"2024-04-25T08:39:25.528068+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.528068+0000","last_undegraded":"2024-04-25T08:39:25.528068+0000","last_fullsized":"2024-04-25T08:39:25.528068+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531168+0000","last_change":"2024-04-25T08:39:25.531168+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531168+0000","last_undegraded":"2024-04-25T08:39:25.531168+0000","last_fullsized":"2024-04-25T08:39:25.531168+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529355+0000","last_change":"2024-04-25T08:39:25.529355+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529355+0000","last_undegraded":"2024-04-25T08:39:25.529355+0000","last_fullsized":"2024-04-25T08:39:25.529355+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532541+0000","last_change":"2024-04-25T08:39:25.532541+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532541+0000","last_undegraded":"2024-04-25T08:39:25.532541+0000","last_fullsized":"2024-04-25T08:39:25.532541+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530852+0000","last_change":"2024-04-25T08:39:25.530852+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530852+0000","last_undegraded":"2024-04-25T08:39:25.530852+0000","last_fullsized":"2024-04-25T08:39:25.530852+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521260+0000","last_change":"2024-04-25T08:39:24.521260+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521260+0000","last_undegraded":"2024-04-25T08:39:24.521260+0000","last_fullsized":"2024-04-25T08:39:24.521260+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542880+0000","last_change":"2024-04-25T08:39:25.541869+0000","last_active":"2024-04-25T08:39:25.542880+0000","last_peered":"2024-04-25T08:39:25.542880+0000","last_clean":"2024-04-25T08:39:25.542880+0000","last_became_active":"2024-04-25T08:39:25.541531+0000","last_became_peered":"2024-04-25T08:39:25.541531+0000","last_unstale":"2024-04-25T08:39:25.542880+0000","last_undegraded":"2024-04-25T08:39:25.542880+0000","last_fullsized":"2024-04-25T08:39:25.542880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:57.694019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542218+0000","last_change":"2024-04-25T08:39:25.540429+0000","last_active":"2024-04-25T08:39:25.542218+0000","last_peered":"2024-04-25T08:39:25.542218+0000","last_clean":"2024-04-25T08:39:25.542218+0000","last_became_active":"2024-04-25T08:39:25.539563+0000","last_became_peered":"2024-04-25T08:39:25.539563+0000","last_unstale":"2024-04-25T08:39:25.542218+0000","last_undegraded":"2024-04-25T08:39:25.542218+0000","last_fullsized":"2024-04-25T08:39:25.542218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:52.290637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530037+0000","last_change":"2024-04-25T08:39:25.530037+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530037+0000","last_undegraded":"2024-04-25T08:39:25.530037+0000","last_fullsized":"2024-04-25T08:39:25.530037+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.522955+0000","last_change":"2024-04-25T08:39:24.522955+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.522955+0000","last_undegraded":"2024-04-25T08:39:24.522955+0000","last_fullsized":"2024-04-25T08:39:24.522955+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524974+0000","last_change":"2024-04-25T08:39:24.524974+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524974+0000","last_undegraded":"2024-04-25T08:39:24.524974+0000","last_fullsized":"2024-04-25T08:39:24.524974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525161+0000","last_change":"2024-04-25T08:39:25.525161+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525161+0000","last_undegraded":"2024-04-25T08:39:25.525161+0000","last_fullsized":"2024-04-25T08:39:25.525161+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521231+0000","last_change":"2024-04-25T08:39:24.521231+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521231+0000","last_undegraded":"2024-04-25T08:39:24.521231+0000","last_fullsized":"2024-04-25T08:39:24.521231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.539017+0000","last_change":"2024-04-25T08:39:25.537615+0000","last_active":"2024-04-25T08:39:25.539017+0000","last_peered":"2024-04-25T08:39:25.539017+0000","last_clean":"2024-04-25T08:39:25.539017+0000","last_became_active":"2024-04-25T08:39:25.537345+0000","last_became_peered":"2024-04-25T08:39:25.537345+0000","last_unstale":"2024-04-25T08:39:25.539017+0000","last_undegraded":"2024-04-25T08:39:25.539017+0000","last_fullsized":"2024-04-25T08:39:25.539017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:50.758570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529484+0000","last_change":"2024-04-25T08:39:25.529484+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529484+0000","last_undegraded":"2024-04-25T08:39:25.529484+0000","last_fullsized":"2024-04-25T08:39:25.529484+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521239+0000","last_change":"2024-04-25T08:39:24.521239+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521239+0000","last_undegraded":"2024-04-25T08:39:24.521239+0000","last_fullsized":"2024-04-25T08:39:24.521239+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542936+0000","last_change":"2024-04-25T08:39:25.541562+0000","last_active":"2024-04-25T08:39:25.542936+0000","last_peered":"2024-04-25T08:39:25.542936+0000","last_clean":"2024-04-25T08:39:25.542936+0000","last_became_active":"2024-04-25T08:39:25.541283+0000","last_became_peered":"2024-04-25T08:39:25.541283+0000","last_unstale":"2024-04-25T08:39:25.542936+0000","last_undegraded":"2024-04-25T08:39:25.542936+0000","last_fullsized":"2024-04-25T08:39:25.542936+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.416592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543099+0000","last_change":"2024-04-25T08:39:25.540300+0000","last_active":"2024-04-25T08:39:25.543099+0000","last_peered":"2024-04-25T08:39:25.543099+0000","last_clean":"2024-04-25T08:39:25.543099+0000","last_became_active":"2024-04-25T08:39:25.539840+0000","last_became_peered":"2024-04-25T08:39:25.539840+0000","last_unstale":"2024-04-25T08:39:25.543099+0000","last_undegraded":"2024-04-25T08:39:25.543099+0000","last_fullsized":"2024-04-25T08:39:25.543099+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523795+0000","last_change":"2024-04-25T08:39:24.523795+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523795+0000","last_undegraded":"2024-04-25T08:39:24.523795+0000","last_fullsized":"2024-04-25T08:39:24.523795+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521328+0000","last_change":"2024-04-25T08:39:24.521328+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521328+0000","last_undegraded":"2024-04-25T08:39:24.521328+0000","last_fullsized":"2024-04-25T08:39:24.521328+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.531819+0000","last_change":"2024-04-25T08:39:25.531155+0000","last_active":"2024-04-25T08:39:25.531819+0000","last_peered":"2024-04-25T08:39:25.531819+0000","last_clean":"2024-04-25T08:39:25.531819+0000","last_became_active":"2024-04-25T08:39:25.530883+0000","last_became_peered":"2024-04-25T08:39:25.530883+0000","last_unstale":"2024-04-25T08:39:25.531819+0000","last_undegraded":"2024-04-25T08:39:25.531819+0000","last_fullsized":"2024-04-25T08:39:25.531819+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543391+0000","last_change":"2024-04-25T08:39:25.542778+0000","last_active":"2024-04-25T08:39:25.543391+0000","last_peered":"2024-04-25T08:39:25.543391+0000","last_clean":"2024-04-25T08:39:25.543391+0000","last_became_active":"2024-04-25T08:39:25.542474+0000","last_became_peered":"2024-04-25T08:39:25.542474+0000","last_unstale":"2024-04-25T08:39:25.543391+0000","last_undegraded":"2024-04-25T08:39:25.543391+0000","last_fullsized":"2024-04-25T08:39:25.543391+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543025+0000","last_change":"2024-04-25T08:39:25.540523+0000","last_active":"2024-04-25T08:39:25.543025+0000","last_peered":"2024-04-25T08:39:25.543025+0000","last_clean":"2024-04-25T08:39:25.543025+0000","last_became_active":"2024-04-25T08:39:25.540179+0000","last_became_peered":"2024-04-25T08:39:25.540179+0000","last_unstale":"2024-04-25T08:39:25.543025+0000","last_undegraded":"2024-04-25T08:39:25.543025+0000","last_fullsized":"2024-04-25T08:39:25.543025+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529469+0000","last_change":"2024-04-25T08:39:25.529469+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529469+0000","last_undegraded":"2024-04-25T08:39:25.529469+0000","last_fullsized":"2024-04-25T08:39:25.529469+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523172+0000","last_change":"2024-04-25T08:39:24.523172+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523172+0000","last_undegraded":"2024-04-25T08:39:24.523172+0000","last_fullsized":"2024-04-25T08:39:24.523172+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530763+0000","last_change":"2024-04-25T08:39:25.530763+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530763+0000","last_undegraded":"2024-04-25T08:39:25.530763+0000","last_fullsized":"2024-04-25T08:39:25.530763+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544961+0000","last_change":"2024-04-25T08:39:25.544537+0000","last_active":"2024-04-25T08:39:25.544961+0000","last_peered":"2024-04-25T08:39:25.544961+0000","last_clean":"2024-04-25T08:39:25.544961+0000","last_became_active":"2024-04-25T08:39:25.544239+0000","last_became_peered":"2024-04-25T08:39:25.544239+0000","last_unstale":"2024-04-25T08:39:25.544961+0000","last_undegraded":"2024-04-25T08:39:25.544961+0000","last_fullsized":"2024-04-25T08:39:25.544961+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:09.533163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530428+0000","last_change":"2024-04-25T08:39:25.530428+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530428+0000","last_undegraded":"2024-04-25T08:39:25.530428+0000","last_fullsized":"2024-04-25T08:39:25.530428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523778+0000","last_change":"2024-04-25T08:39:24.523778+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523778+0000","last_undegraded":"2024-04-25T08:39:24.523778+0000","last_fullsized":"2024-04-25T08:39:24.523778+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529985+0000","last_change":"2024-04-25T08:39:25.529985+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529985+0000","last_undegraded":"2024-04-25T08:39:25.529985+0000","last_fullsized":"2024-04-25T08:39:25.529985+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.536302+0000","last_change":"2024-04-25T08:39:25.534723+0000","last_active":"2024-04-25T08:39:25.536302+0000","last_peered":"2024-04-25T08:39:25.536302+0000","last_clean":"2024-04-25T08:39:25.536302+0000","last_became_active":"2024-04-25T08:39:25.534462+0000","last_became_peered":"2024-04-25T08:39:25.534462+0000","last_unstale":"2024-04-25T08:39:25.536302+0000","last_undegraded":"2024-04-25T08:39:25.536302+0000","last_fullsized":"2024-04-25T08:39:25.536302+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529814+0000","last_change":"2024-04-25T08:39:25.529814+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529814+0000","last_undegraded":"2024-04-25T08:39:25.529814+0000","last_fullsized":"2024-04-25T08:39:25.529814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524065+0000","last_change":"2024-04-25T08:39:24.524065+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524065+0000","last_undegraded":"2024-04-25T08:39:24.524065+0000","last_fullsized":"2024-04-25T08:39:24.524065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529165+0000","last_change":"2024-04-25T08:39:25.529165+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529165+0000","last_undegraded":"2024-04-25T08:39:25.529165+0000","last_fullsized":"2024-04-25T08:39:25.529165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.525390+0000","last_change":"2024-04-25T08:39:24.525390+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.525390+0000","last_undegraded":"2024-04-25T08:39:24.525390+0000","last_fullsized":"2024-04-25T08:39:24.525390+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.535065+0000","last_change":"2024-04-25T08:39:25.533867+0000","last_active":"2024-04-25T08:39:25.535065+0000","last_peered":"2024-04-25T08:39:25.535065+0000","last_clean":"2024-04-25T08:39:25.535065+0000","last_became_active":"2024-04-25T08:39:25.533567+0000","last_became_peered":"2024-04-25T08:39:25.533567+0000","last_unstale":"2024-04-25T08:39:25.535065+0000","last_undegraded":"2024-04-25T08:39:25.535065+0000","last_fullsized":"2024-04-25T08:39:25.535065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527139+0000","last_change":"2024-04-25T08:39:25.527139+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527139+0000","last_undegraded":"2024-04-25T08:39:25.527139+0000","last_fullsized":"2024-04-25T08:39:25.527139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543080+0000","last_change":"2024-04-25T08:39:25.540746+0000","last_active":"2024-04-25T08:39:25.543080+0000","last_peered":"2024-04-25T08:39:25.543080+0000","last_clean":"2024-04-25T08:39:25.543080+0000","last_became_active":"2024-04-25T08:39:25.540230+0000","last_became_peered":"2024-04-25T08:39:25.540230+0000","last_unstale":"2024-04-25T08:39:25.543080+0000","last_undegraded":"2024-04-25T08:39:25.543080+0000","last_fullsized":"2024-04-25T08:39:25.543080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529530+0000","last_change":"2024-04-25T08:39:25.529530+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529530+0000","last_undegraded":"2024-04-25T08:39:25.529530+0000","last_fullsized":"2024-04-25T08:39:25.529530+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523429+0000","last_change":"2024-04-25T08:39:24.523429+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523429+0000","last_undegraded":"2024-04-25T08:39:24.523429+0000","last_fullsized":"2024-04-25T08:39:24.523429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521968+0000","last_change":"2024-04-25T08:39:24.521968+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521968+0000","last_undegraded":"2024-04-25T08:39:24.521968+0000","last_fullsized":"2024-04-25T08:39:24.521968+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.541132+0000","last_change":"2024-04-25T08:39:25.539417+0000","last_active":"2024-04-25T08:39:25.541132+0000","last_peered":"2024-04-25T08:39:25.541132+0000","last_clean":"2024-04-25T08:39:25.541132+0000","last_became_active":"2024-04-25T08:39:25.539015+0000","last_became_peered":"2024-04-25T08:39:25.539015+0000","last_unstale":"2024-04-25T08:39:25.541132+0000","last_undegraded":"2024-04-25T08:39:25.541132+0000","last_fullsized":"2024-04-25T08:39:25.541132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.524638+0000","last_change":"2024-04-25T08:39:25.524638+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.524638+0000","last_undegraded":"2024-04-25T08:39:25.524638+0000","last_fullsized":"2024-04-25T08:39:25.524638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523976+0000","last_change":"2024-04-25T08:39:24.523976+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523976+0000","last_undegraded":"2024-04-25T08:39:24.523976+0000","last_fullsized":"2024-04-25T08:39:24.523976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544089+0000","last_change":"2024-04-25T08:39:25.543519+0000","last_active":"2024-04-25T08:39:25.544089+0000","last_peered":"2024-04-25T08:39:25.544089+0000","last_clean":"2024-04-25T08:39:25.544089+0000","last_became_active":"2024-04-25T08:39:25.543148+0000","last_became_peered":"2024-04-25T08:39:25.543148+0000","last_unstale":"2024-04-25T08:39:25.544089+0000","last_undegraded":"2024-04-25T08:39:25.544089+0000","last_fullsized":"2024-04-25T08:39:25.544089+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.721871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.538234+0000","last_change":"2024-04-25T08:39:25.536756+0000","last_active":"2024-04-25T08:39:25.538234+0000","last_peered":"2024-04-25T08:39:25.538234+0000","last_clean":"2024-04-25T08:39:25.538234+0000","last_became_active":"2024-04-25T08:39:25.535762+0000","last_became_peered":"2024-04-25T08:39:25.535762+0000","last_unstale":"2024-04-25T08:39:25.538234+0000","last_undegraded":"2024-04-25T08:39:25.538234+0000","last_fullsized":"2024-04-25T08:39:25.538234+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:10.416181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522508+0000","last_change":"2024-04-25T08:39:25.522508+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522508+0000","last_undegraded":"2024-04-25T08:39:25.522508+0000","last_fullsized":"2024-04-25T08:39:25.522508+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":398,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.532864+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:25.532864+0000","last_peered":"2024-04-25T08:39:25.532864+0000","last_clean":"2024-04-25T08:39:25.532864+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:25.532864+0000","last_undegraded":"2024-04-25T08:39:25.532864+0000","last_fullsized":"2024-04-25T08:39:25.532864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1425990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542271,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50980,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320860,"statfs":{"total":96636764160,"available":96584560640,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542271,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46824,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325016,"statfs":{"total":96636764160,"available":96588816384,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46804,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325036,"statfs":{"total":96636764160,"available":96588836864,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59292,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312548,"statfs":{"total":96636764160,"available":96576049152,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2371584,"data_stored":1419323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:29.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:30.070 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.077+0000 7fac3c789640 1 Processor -- start 2024-04-25T08:39:30.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.077+0000 7fac3c789640 1 -- start start 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3c789640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac3415db70 0x7fac34162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3c789640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac340b6cb0 0x7fac340b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3c789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b5a60 0x7fac3406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3c789640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac34162690 con 0x7fac3415db70 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3c789640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac340ceee0 con 0x7fac340b6cb0 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3c789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac340d1d10 con 0x7fac340b5a60 2024-04-25T08:39:30.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac3415db70 0x7fac34162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fac3415db70 0x7fac34162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52426/0 (socket says 172.21.15.60:52426) 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 -- 172.21.15.60:0/3244249245 learned_addr learned my addr 172.21.15.60:0/3244249245 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3ad26640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b5a60 0x7fac3406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac39d24640 1 --2- 172.21.15.60:0/3244249245 >> v2:172.21.15.100:3300/0 conn(0x7fac340b6cb0 0x7fac340b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 -- 172.21.15.60:0/3244249245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b5a60 msgr2=0x7fac3406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b5a60 0x7fac3406c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 -- 172.21.15.60:0/3244249245 >> v2:172.21.15.100:3300/0 conn(0x7fac340b6cb0 msgr2=0x7fac340b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 --2- 172.21.15.60:0/3244249245 >> v2:172.21.15.100:3300/0 conn(0x7fac340b6cb0 0x7fac340b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3a525640 1 -- 172.21.15.60:0/3244249245 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340d1ff0 con 0x7fac3415db70 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac39d24640 1 --2- 172.21.15.60:0/3244249245 >> v2:172.21.15.100:3300/0 conn(0x7fac340b6cb0 0x7fac340b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.078+0000 7fac3ad26640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b5a60 0x7fac3406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.079+0000 7fac3a525640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac3415db70 0x7fac34162150 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fac3408c600 tx=0x7fac3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2659e7a217858577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.079+0000 7fac39523640 1 -- 172.21.15.60:0/3244249245 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30037070 con 0x7fac3415db70 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.079+0000 7fac39523640 1 -- 172.21.15.60:0/3244249245 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac30032440 con 0x7fac3415db70 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.079+0000 7fac39523640 1 -- 172.21.15.60:0/3244249245 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30032810 con 0x7fac3415db70 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.079+0000 7fac3c789640 1 -- 172.21.15.60:0/3244249245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac3415db70 msgr2=0x7fac34162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.079+0000 7fac3c789640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac3415db70 0x7fac34162150 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fac3408c600 tx=0x7fac3002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 -- 172.21.15.60:0/3244249245 shutdown_connections 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b5a60 0x7fac3406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 --2- 172.21.15.60:0/3244249245 >> v2:172.21.15.100:3300/0 conn(0x7fac340b6cb0 0x7fac340b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 --2- 172.21.15.60:0/3244249245 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac3415db70 0x7fac34162150 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 -- 172.21.15.60:0/3244249245 >> 172.21.15.60:0/3244249245 conn(0x7fac340d0520 msgr2=0x7fac340d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 -- 172.21.15.60:0/3244249245 shutdown_connections 2024-04-25T08:39:30.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.080+0000 7fac3c789640 1 -- 172.21.15.60:0/3244249245 wait complete. 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.081+0000 7fac3c789640 1 Processor -- start 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.081+0000 7fac3c789640 1 -- start start 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.081+0000 7fac3c789640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 0x7fac34156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.081+0000 7fac3c789640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b6cb0 0x7fac340aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3c789640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac3415db70 0x7fac3406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3c789640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340ce530 con 0x7fac340b5a60 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3c789640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3415df50 con 0x7fac3415db70 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3c789640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac340cf1c0 con 0x7fac340b6cb0 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 0x7fac34156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 0x7fac34156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52434/0 (socket says 172.21.15.60:52434) 2024-04-25T08:39:30.078 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 -- 172.21.15.60:0/2108731074 learned_addr learned my addr 172.21.15.60:0/2108731074 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3ad26640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac3415db70 0x7fac3406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac39d24640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b6cb0 0x7fac340aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 -- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b6cb0 msgr2=0x7fac340aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b6cb0 0x7fac340aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 -- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac3415db70 msgr2=0x7fac3406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac3415db70 0x7fac3406c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 -- 172.21.15.60:0/2108731074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac341574e0 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3a525640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 0x7fac34156bb0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fac3002a220 tx=0x7fac300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30041020 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac300306c0 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac341f79e0 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30037040 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.082+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac341f7fa0 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.083+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34073990 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.084+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac30044020 con 0x7fac340b5a60 2024-04-25T08:39:30.079 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.084+0000 7fac2affd640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac1806b4d0 0x7fac1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.084+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac300aaa40 con 0x7fac340b5a60 2024-04-25T08:39:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.085+0000 7fac39d24640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac1806b4d0 0x7fac1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.080 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.085+0000 7fac39d24640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac1806b4d0 0x7fac1806d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fac340b6a30 tx=0x7fac24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.082 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.089+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac3004b030 con 0x7fac340b5a60 2024-04-25T08:39:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.263+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac340ce310 con 0x7fac1806b4d0 2024-04-25T08:39:30.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.270+0000 7fac2affd640 1 -- 172.21.15.60:0/2108731074 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242992 (secure 0 0 0) 0x7fac340ce310 con 0x7fac1806b4d0 2024-04-25T08:39:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:30.266 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:39:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac1806b4d0 msgr2=0x7fac1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac1806b4d0 0x7fac1806d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fac340b6a30 tx=0x7fac24008040 comp rx=0 tx=0).stop 2024-04-25T08:39:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 msgr2=0x7fac34156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 0x7fac34156bb0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fac3002a220 tx=0x7fac300046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 shutdown_connections 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fac1806b4d0 0x7fac1806d990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac3415db70 0x7fac3406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac340b6cb0 0x7fac340aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 --2- 172.21.15.60:0/2108731074 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac340b5a60 0x7fac34156bb0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 >> 172.21.15.60:0/2108731074 conn(0x7fac340d0520 msgr2=0x7fac3415a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 shutdown_connections 2024-04-25T08:39:30.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.272+0000 7fac3c789640 1 -- 172.21.15.60:0/2108731074 wait complete. 2024-04-25T08:39:30.277 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-04-25T08:39:29.538661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425596,"kb_used_data":15084,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754549124,"statfs":{"total":773094113280,"available":772658302976,"internally_reserved":0,"allocated":15446016,"data_stored":7832377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.019512"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532972+0000","last_change":"2024-04-25T08:39:25.532972+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532972+0000","last_undegraded":"2024-04-25T08:39:25.532972+0000","last_fullsized":"2024-04-25T08:39:25.532972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.533598+0000","last_change":"2024-04-25T08:39:25.533598+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.533598+0000","last_undegraded":"2024-04-25T08:39:25.533598+0000","last_fullsized":"2024-04-25T08:39:25.533598+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531925+0000","last_change":"2024-04-25T08:39:25.531925+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531925+0000","last_undegraded":"2024-04-25T08:39:25.531925+0000","last_fullsized":"2024-04-25T08:39:25.531925+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527182+0000","last_change":"2024-04-25T08:39:25.527182+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527182+0000","last_undegraded":"2024-04-25T08:39:25.527182+0000","last_fullsized":"2024-04-25T08:39:25.527182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525519+0000","last_change":"2024-04-25T08:39:25.525519+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525519+0000","last_undegraded":"2024-04-25T08:39:25.525519+0000","last_fullsized":"2024-04-25T08:39:25.525519+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.528068+0000","last_change":"2024-04-25T08:39:25.528068+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.528068+0000","last_undegraded":"2024-04-25T08:39:25.528068+0000","last_fullsized":"2024-04-25T08:39:25.528068+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531168+0000","last_change":"2024-04-25T08:39:25.531168+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531168+0000","last_undegraded":"2024-04-25T08:39:25.531168+0000","last_fullsized":"2024-04-25T08:39:25.531168+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529355+0000","last_change":"2024-04-25T08:39:25.529355+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529355+0000","last_undegraded":"2024-04-25T08:39:25.529355+0000","last_fullsized":"2024-04-25T08:39:25.529355+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532541+0000","last_change":"2024-04-25T08:39:25.532541+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532541+0000","last_undegraded":"2024-04-25T08:39:25.532541+0000","last_fullsized":"2024-04-25T08:39:25.532541+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530852+0000","last_change":"2024-04-25T08:39:25.530852+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530852+0000","last_undegraded":"2024-04-25T08:39:25.530852+0000","last_fullsized":"2024-04-25T08:39:25.530852+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521260+0000","last_change":"2024-04-25T08:39:24.521260+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521260+0000","last_undegraded":"2024-04-25T08:39:24.521260+0000","last_fullsized":"2024-04-25T08:39:24.521260+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542880+0000","last_change":"2024-04-25T08:39:25.541869+0000","last_active":"2024-04-25T08:39:25.542880+0000","last_peered":"2024-04-25T08:39:25.542880+0000","last_clean":"2024-04-25T08:39:25.542880+0000","last_became_active":"2024-04-25T08:39:25.541531+0000","last_became_peered":"2024-04-25T08:39:25.541531+0000","last_unstale":"2024-04-25T08:39:25.542880+0000","last_undegraded":"2024-04-25T08:39:25.542880+0000","last_fullsized":"2024-04-25T08:39:25.542880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:57.694019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542218+0000","last_change":"2024-04-25T08:39:25.540429+0000","last_active":"2024-04-25T08:39:25.542218+0000","last_peered":"2024-04-25T08:39:25.542218+0000","last_clean":"2024-04-25T08:39:25.542218+0000","last_became_active":"2024-04-25T08:39:25.539563+0000","last_became_peered":"2024-04-25T08:39:25.539563+0000","last_unstale":"2024-04-25T08:39:25.542218+0000","last_undegraded":"2024-04-25T08:39:25.542218+0000","last_fullsized":"2024-04-25T08:39:25.542218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:52.290637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530037+0000","last_change":"2024-04-25T08:39:25.530037+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530037+0000","last_undegraded":"2024-04-25T08:39:25.530037+0000","last_fullsized":"2024-04-25T08:39:25.530037+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.522955+0000","last_change":"2024-04-25T08:39:24.522955+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.522955+0000","last_undegraded":"2024-04-25T08:39:24.522955+0000","last_fullsized":"2024-04-25T08:39:24.522955+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524974+0000","last_change":"2024-04-25T08:39:24.524974+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524974+0000","last_undegraded":"2024-04-25T08:39:24.524974+0000","last_fullsized":"2024-04-25T08:39:24.524974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525161+0000","last_change":"2024-04-25T08:39:25.525161+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525161+0000","last_undegraded":"2024-04-25T08:39:25.525161+0000","last_fullsized":"2024-04-25T08:39:25.525161+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521231+0000","last_change":"2024-04-25T08:39:24.521231+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521231+0000","last_undegraded":"2024-04-25T08:39:24.521231+0000","last_fullsized":"2024-04-25T08:39:24.521231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.539017+0000","last_change":"2024-04-25T08:39:25.537615+0000","last_active":"2024-04-25T08:39:25.539017+0000","last_peered":"2024-04-25T08:39:25.539017+0000","last_clean":"2024-04-25T08:39:25.539017+0000","last_became_active":"2024-04-25T08:39:25.537345+0000","last_became_peered":"2024-04-25T08:39:25.537345+0000","last_unstale":"2024-04-25T08:39:25.539017+0000","last_undegraded":"2024-04-25T08:39:25.539017+0000","last_fullsized":"2024-04-25T08:39:25.539017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:50.758570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529484+0000","last_change":"2024-04-25T08:39:25.529484+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529484+0000","last_undegraded":"2024-04-25T08:39:25.529484+0000","last_fullsized":"2024-04-25T08:39:25.529484+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521239+0000","last_change":"2024-04-25T08:39:24.521239+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521239+0000","last_undegraded":"2024-04-25T08:39:24.521239+0000","last_fullsized":"2024-04-25T08:39:24.521239+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542936+0000","last_change":"2024-04-25T08:39:25.541562+0000","last_active":"2024-04-25T08:39:25.542936+0000","last_peered":"2024-04-25T08:39:25.542936+0000","last_clean":"2024-04-25T08:39:25.542936+0000","last_became_active":"2024-04-25T08:39:25.541283+0000","last_became_peered":"2024-04-25T08:39:25.541283+0000","last_unstale":"2024-04-25T08:39:25.542936+0000","last_undegraded":"2024-04-25T08:39:25.542936+0000","last_fullsized":"2024-04-25T08:39:25.542936+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.416592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543099+0000","last_change":"2024-04-25T08:39:25.540300+0000","last_active":"2024-04-25T08:39:25.543099+0000","last_peered":"2024-04-25T08:39:25.543099+0000","last_clean":"2024-04-25T08:39:25.543099+0000","last_became_active":"2024-04-25T08:39:25.539840+0000","last_became_peered":"2024-04-25T08:39:25.539840+0000","last_unstale":"2024-04-25T08:39:25.543099+0000","last_undegraded":"2024-04-25T08:39:25.543099+0000","last_fullsized":"2024-04-25T08:39:25.543099+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523795+0000","last_change":"2024-04-25T08:39:24.523795+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523795+0000","last_undegraded":"2024-04-25T08:39:24.523795+0000","last_fullsized":"2024-04-25T08:39:24.523795+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521328+0000","last_change":"2024-04-25T08:39:24.521328+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521328+0000","last_undegraded":"2024-04-25T08:39:24.521328+0000","last_fullsized":"2024-04-25T08:39:24.521328+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.531819+0000","last_change":"2024-04-25T08:39:25.531155+0000","last_active":"2024-04-25T08:39:25.531819+0000","last_peered":"2024-04-25T08:39:25.531819+0000","last_clean":"2024-04-25T08:39:25.531819+0000","last_became_active":"2024-04-25T08:39:25.530883+0000","last_became_peered":"2024-04-25T08:39:25.530883+0000","last_unstale":"2024-04-25T08:39:25.531819+0000","last_undegraded":"2024-04-25T08:39:25.531819+0000","last_fullsized":"2024-04-25T08:39:25.531819+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543391+0000","last_change":"2024-04-25T08:39:25.542778+0000","last_active":"2024-04-25T08:39:25.543391+0000","last_peered":"2024-04-25T08:39:25.543391+0000","last_clean":"2024-04-25T08:39:25.543391+0000","last_became_active":"2024-04-25T08:39:25.542474+0000","last_became_peered":"2024-04-25T08:39:25.542474+0000","last_unstale":"2024-04-25T08:39:25.543391+0000","last_undegraded":"2024-04-25T08:39:25.543391+0000","last_fullsized":"2024-04-25T08:39:25.543391+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543025+0000","last_change":"2024-04-25T08:39:25.540523+0000","last_active":"2024-04-25T08:39:25.543025+0000","last_peered":"2024-04-25T08:39:25.543025+0000","last_clean":"2024-04-25T08:39:25.543025+0000","last_became_active":"2024-04-25T08:39:25.540179+0000","last_became_peered":"2024-04-25T08:39:25.540179+0000","last_unstale":"2024-04-25T08:39:25.543025+0000","last_undegraded":"2024-04-25T08:39:25.543025+0000","last_fullsized":"2024-04-25T08:39:25.543025+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529469+0000","last_change":"2024-04-25T08:39:25.529469+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529469+0000","last_undegraded":"2024-04-25T08:39:25.529469+0000","last_fullsized":"2024-04-25T08:39:25.529469+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523172+0000","last_change":"2024-04-25T08:39:24.523172+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523172+0000","last_undegraded":"2024-04-25T08:39:24.523172+0000","last_fullsized":"2024-04-25T08:39:24.523172+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530763+0000","last_change":"2024-04-25T08:39:25.530763+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530763+0000","last_undegraded":"2024-04-25T08:39:25.530763+0000","last_fullsized":"2024-04-25T08:39:25.530763+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544961+0000","last_change":"2024-04-25T08:39:25.544537+0000","last_active":"2024-04-25T08:39:25.544961+0000","last_peered":"2024-04-25T08:39:25.544961+0000","last_clean":"2024-04-25T08:39:25.544961+0000","last_became_active":"2024-04-25T08:39:25.544239+0000","last_became_peered":"2024-04-25T08:39:25.544239+0000","last_unstale":"2024-04-25T08:39:25.544961+0000","last_undegraded":"2024-04-25T08:39:25.544961+0000","last_fullsized":"2024-04-25T08:39:25.544961+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:09.533163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530428+0000","last_change":"2024-04-25T08:39:25.530428+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530428+0000","last_undegraded":"2024-04-25T08:39:25.530428+0000","last_fullsized":"2024-04-25T08:39:25.530428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523778+0000","last_change":"2024-04-25T08:39:24.523778+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523778+0000","last_undegraded":"2024-04-25T08:39:24.523778+0000","last_fullsized":"2024-04-25T08:39:24.523778+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529985+0000","last_change":"2024-04-25T08:39:25.529985+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529985+0000","last_undegraded":"2024-04-25T08:39:25.529985+0000","last_fullsized":"2024-04-25T08:39:25.529985+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.536302+0000","last_change":"2024-04-25T08:39:25.534723+0000","last_active":"2024-04-25T08:39:25.536302+0000","last_peered":"2024-04-25T08:39:25.536302+0000","last_clean":"2024-04-25T08:39:25.536302+0000","last_became_active":"2024-04-25T08:39:25.534462+0000","last_became_peered":"2024-04-25T08:39:25.534462+0000","last_unstale":"2024-04-25T08:39:25.536302+0000","last_undegraded":"2024-04-25T08:39:25.536302+0000","last_fullsized":"2024-04-25T08:39:25.536302+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529814+0000","last_change":"2024-04-25T08:39:25.529814+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529814+0000","last_undegraded":"2024-04-25T08:39:25.529814+0000","last_fullsized":"2024-04-25T08:39:25.529814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524065+0000","last_change":"2024-04-25T08:39:24.524065+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524065+0000","last_undegraded":"2024-04-25T08:39:24.524065+0000","last_fullsized":"2024-04-25T08:39:24.524065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529165+0000","last_change":"2024-04-25T08:39:25.529165+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529165+0000","last_undegraded":"2024-04-25T08:39:25.529165+0000","last_fullsized":"2024-04-25T08:39:25.529165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.525390+0000","last_change":"2024-04-25T08:39:24.525390+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.525390+0000","last_undegraded":"2024-04-25T08:39:24.525390+0000","last_fullsized":"2024-04-25T08:39:24.525390+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.535065+0000","last_change":"2024-04-25T08:39:25.533867+0000","last_active":"2024-04-25T08:39:25.535065+0000","last_peered":"2024-04-25T08:39:25.535065+0000","last_clean":"2024-04-25T08:39:25.535065+0000","last_became_active":"2024-04-25T08:39:25.533567+0000","last_became_peered":"2024-04-25T08:39:25.533567+0000","last_unstale":"2024-04-25T08:39:25.535065+0000","last_undegraded":"2024-04-25T08:39:25.535065+0000","last_fullsized":"2024-04-25T08:39:25.535065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527139+0000","last_change":"2024-04-25T08:39:25.527139+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527139+0000","last_undegraded":"2024-04-25T08:39:25.527139+0000","last_fullsized":"2024-04-25T08:39:25.527139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543080+0000","last_change":"2024-04-25T08:39:25.540746+0000","last_active":"2024-04-25T08:39:25.543080+0000","last_peered":"2024-04-25T08:39:25.543080+0000","last_clean":"2024-04-25T08:39:25.543080+0000","last_became_active":"2024-04-25T08:39:25.540230+0000","last_became_peered":"2024-04-25T08:39:25.540230+0000","last_unstale":"2024-04-25T08:39:25.543080+0000","last_undegraded":"2024-04-25T08:39:25.543080+0000","last_fullsized":"2024-04-25T08:39:25.543080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529530+0000","last_change":"2024-04-25T08:39:25.529530+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529530+0000","last_undegraded":"2024-04-25T08:39:25.529530+0000","last_fullsized":"2024-04-25T08:39:25.529530+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523429+0000","last_change":"2024-04-25T08:39:24.523429+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523429+0000","last_undegraded":"2024-04-25T08:39:24.523429+0000","last_fullsized":"2024-04-25T08:39:24.523429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521968+0000","last_change":"2024-04-25T08:39:24.521968+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521968+0000","last_undegraded":"2024-04-25T08:39:24.521968+0000","last_fullsized":"2024-04-25T08:39:24.521968+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.541132+0000","last_change":"2024-04-25T08:39:25.539417+0000","last_active":"2024-04-25T08:39:25.541132+0000","last_peered":"2024-04-25T08:39:25.541132+0000","last_clean":"2024-04-25T08:39:25.541132+0000","last_became_active":"2024-04-25T08:39:25.539015+0000","last_became_peered":"2024-04-25T08:39:25.539015+0000","last_unstale":"2024-04-25T08:39:25.541132+0000","last_undegraded":"2024-04-25T08:39:25.541132+0000","last_fullsized":"2024-04-25T08:39:25.541132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.524638+0000","last_change":"2024-04-25T08:39:25.524638+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.524638+0000","last_undegraded":"2024-04-25T08:39:25.524638+0000","last_fullsized":"2024-04-25T08:39:25.524638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523976+0000","last_change":"2024-04-25T08:39:24.523976+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523976+0000","last_undegraded":"2024-04-25T08:39:24.523976+0000","last_fullsized":"2024-04-25T08:39:24.523976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544089+0000","last_change":"2024-04-25T08:39:25.543519+0000","last_active":"2024-04-25T08:39:25.544089+0000","last_peered":"2024-04-25T08:39:25.544089+0000","last_clean":"2024-04-25T08:39:25.544089+0000","last_became_active":"2024-04-25T08:39:25.543148+0000","last_became_peered":"2024-04-25T08:39:25.543148+0000","last_unstale":"2024-04-25T08:39:25.544089+0000","last_undegraded":"2024-04-25T08:39:25.544089+0000","last_fullsized":"2024-04-25T08:39:25.544089+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.721871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.538234+0000","last_change":"2024-04-25T08:39:25.536756+0000","last_active":"2024-04-25T08:39:25.538234+0000","last_peered":"2024-04-25T08:39:25.538234+0000","last_clean":"2024-04-25T08:39:25.538234+0000","last_became_active":"2024-04-25T08:39:25.535762+0000","last_became_peered":"2024-04-25T08:39:25.535762+0000","last_unstale":"2024-04-25T08:39:25.538234+0000","last_undegraded":"2024-04-25T08:39:25.538234+0000","last_fullsized":"2024-04-25T08:39:25.538234+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:10.416181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522508+0000","last_change":"2024-04-25T08:39:25.522508+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522508+0000","last_undegraded":"2024-04-25T08:39:25.522508+0000","last_fullsized":"2024-04-25T08:39:25.522508+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":398,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.532864+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:25.532864+0000","last_peered":"2024-04-25T08:39:25.532864+0000","last_clean":"2024-04-25T08:39:25.532864+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:25.532864+0000","last_undegraded":"2024-04-25T08:39:25.532864+0000","last_fullsized":"2024-04-25T08:39:25.532864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1425990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542271,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50980,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320860,"statfs":{"total":96636764160,"available":96584560640,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542271,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46824,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325016,"statfs":{"total":96636764160,"available":96588816384,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46804,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325036,"statfs":{"total":96636764160,"available":96588836864,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59292,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312548,"statfs":{"total":96636764160,"available":96576049152,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2371584,"data_stored":1419323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.532972+0000', 'last_change': '2024-04-25T08:39:25.532972+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.532972+0000', 'last_undegraded': '2024-04-25T08:39:25.532972+0000', 'last_fullsized': '2024-04-25T08:39:25.532972+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.533598+0000', 'last_change': '2024-04-25T08:39:25.533598+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.533598+0000', 'last_undegraded': '2024-04-25T08:39:25.533598+0000', 'last_fullsized': '2024-04-25T08:39:25.533598+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.531925+0000', 'last_change': '2024-04-25T08:39:25.531925+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.531925+0000', 'last_undegraded': '2024-04-25T08:39:25.531925+0000', 'last_fullsized': '2024-04-25T08:39:25.531925+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2024-04-25T08:39:30.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.527182+0000', 'last_change': '2024-04-25T08:39:25.527182+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.527182+0000', 'last_undegraded': '2024-04-25T08:39:25.527182+0000', 'last_fullsized': '2024-04-25T08:39:25.527182+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.525519+0000', 'last_change': '2024-04-25T08:39:25.525519+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.525519+0000', 'last_undegraded': '2024-04-25T08:39:25.525519+0000', 'last_fullsized': '2024-04-25T08:39:25.525519+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.528068+0000', 'last_change': '2024-04-25T08:39:25.528068+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.528068+0000', 'last_undegraded': '2024-04-25T08:39:25.528068+0000', 'last_fullsized': '2024-04-25T08:39:25.528068+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2024-04-25T08:39:30.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.531168+0000', 'last_change': '2024-04-25T08:39:25.531168+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.531168+0000', 'last_undegraded': '2024-04-25T08:39:25.531168+0000', 'last_fullsized': '2024-04-25T08:39:25.531168+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529355+0000', 'last_change': '2024-04-25T08:39:25.529355+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529355+0000', 'last_undegraded': '2024-04-25T08:39:25.529355+0000', 'last_fullsized': '2024-04-25T08:39:25.529355+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2024-04-25T08:39:30.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.532541+0000', 'last_change': '2024-04-25T08:39:25.532541+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.532541+0000', 'last_undegraded': '2024-04-25T08:39:25.532541+0000', 'last_fullsized': '2024-04-25T08:39:25.532541+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.530852+0000', 'last_change': '2024-04-25T08:39:25.530852+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.530852+0000', 'last_undegraded': '2024-04-25T08:39:25.530852+0000', 'last_fullsized': '2024-04-25T08:39:25.530852+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2024-04-25T08:39:30.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.521260+0000', 'last_change': '2024-04-25T08:39:24.521260+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.521260+0000', 'last_undegraded': '2024-04-25T08:39:24.521260+0000', 'last_fullsized': '2024-04-25T08:39:24.521260+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2024-04-25T08:39:30.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.530037+0000', 'last_change': '2024-04-25T08:39:25.530037+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.530037+0000', 'last_undegraded': '2024-04-25T08:39:25.530037+0000', 'last_fullsized': '2024-04-25T08:39:25.530037+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.522955+0000', 'last_change': '2024-04-25T08:39:24.522955+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.522955+0000', 'last_undegraded': '2024-04-25T08:39:24.522955+0000', 'last_fullsized': '2024-04-25T08:39:24.522955+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.524974+0000', 'last_change': '2024-04-25T08:39:24.524974+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.524974+0000', 'last_undegraded': '2024-04-25T08:39:24.524974+0000', 'last_fullsized': '2024-04-25T08:39:24.524974+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2024-04-25T08:39:30.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.525161+0000', 'last_change': '2024-04-25T08:39:25.525161+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.525161+0000', 'last_undegraded': '2024-04-25T08:39:25.525161+0000', 'last_fullsized': '2024-04-25T08:39:25.525161+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.521231+0000', 'last_change': '2024-04-25T08:39:24.521231+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.521231+0000', 'last_undegraded': '2024-04-25T08:39:24.521231+0000', 'last_fullsized': '2024-04-25T08:39:24.521231+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529484+0000', 'last_change': '2024-04-25T08:39:25.529484+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529484+0000', 'last_undegraded': '2024-04-25T08:39:25.529484+0000', 'last_fullsized': '2024-04-25T08:39:25.529484+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.521239+0000', 'last_change': '2024-04-25T08:39:24.521239+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.521239+0000', 'last_undegraded': '2024-04-25T08:39:24.521239+0000', 'last_fullsized': '2024-04-25T08:39:24.521239+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2024-04-25T08:39:30.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.523795+0000', 'last_change': '2024-04-25T08:39:24.523795+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.523795+0000', 'last_undegraded': '2024-04-25T08:39:24.523795+0000', 'last_fullsized': '2024-04-25T08:39:24.523795+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.521328+0000', 'last_change': '2024-04-25T08:39:24.521328+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.521328+0000', 'last_undegraded': '2024-04-25T08:39:24.521328+0000', 'last_fullsized': '2024-04-25T08:39:24.521328+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2024-04-25T08:39:30.290 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529469+0000', 'last_change': '2024-04-25T08:39:25.529469+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529469+0000', 'last_undegraded': '2024-04-25T08:39:25.529469+0000', 'last_fullsized': '2024-04-25T08:39:25.529469+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.523172+0000', 'last_change': '2024-04-25T08:39:24.523172+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.523172+0000', 'last_undegraded': '2024-04-25T08:39:24.523172+0000', 'last_fullsized': '2024-04-25T08:39:24.523172+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.530763+0000', 'last_change': '2024-04-25T08:39:25.530763+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.530763+0000', 'last_undegraded': '2024-04-25T08:39:25.530763+0000', 'last_fullsized': '2024-04-25T08:39:25.530763+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:30.291 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.530428+0000', 'last_change': '2024-04-25T08:39:25.530428+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.530428+0000', 'last_undegraded': '2024-04-25T08:39:25.530428+0000', 'last_fullsized': '2024-04-25T08:39:25.530428+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.523778+0000', 'last_change': '2024-04-25T08:39:24.523778+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.523778+0000', 'last_undegraded': '2024-04-25T08:39:24.523778+0000', 'last_fullsized': '2024-04-25T08:39:24.523778+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529985+0000', 'last_change': '2024-04-25T08:39:25.529985+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529985+0000', 'last_undegraded': '2024-04-25T08:39:25.529985+0000', 'last_fullsized': '2024-04-25T08:39:25.529985+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529814+0000', 'last_change': '2024-04-25T08:39:25.529814+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529814+0000', 'last_undegraded': '2024-04-25T08:39:25.529814+0000', 'last_fullsized': '2024-04-25T08:39:25.529814+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 143, 'state': 'creating+activating', 'last_fresh': '2024-04-25T08:39:25.522548+0000', 'last_change': '2024-04-25T08:39:25.522548+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.522548+0000', 'last_undegraded': '2024-04-25T08:39:25.522548+0000', 'last_fullsized': '2024-04-25T08:39:25.522548+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2024-04-25T08:39:30.292 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.524065+0000', 'last_change': '2024-04-25T08:39:24.524065+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.524065+0000', 'last_undegraded': '2024-04-25T08:39:24.524065+0000', 'last_fullsized': '2024-04-25T08:39:24.524065+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529165+0000', 'last_change': '2024-04-25T08:39:25.529165+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529165+0000', 'last_undegraded': '2024-04-25T08:39:25.529165+0000', 'last_fullsized': '2024-04-25T08:39:25.529165+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 143, 'state': 'creating+activating', 'last_fresh': '2024-04-25T08:39:25.522548+0000', 'last_change': '2024-04-25T08:39:25.522548+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.522548+0000', 'last_undegraded': '2024-04-25T08:39:25.522548+0000', 'last_fullsized': '2024-04-25T08:39:25.522548+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.525390+0000', 'last_change': '2024-04-25T08:39:24.525390+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.525390+0000', 'last_undegraded': '2024-04-25T08:39:24.525390+0000', 'last_fullsized': '2024-04-25T08:39:24.525390+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2024-04-25T08:39:30.293 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.527139+0000', 'last_change': '2024-04-25T08:39:25.527139+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.527139+0000', 'last_undegraded': '2024-04-25T08:39:25.527139+0000', 'last_fullsized': '2024-04-25T08:39:25.527139+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.529530+0000', 'last_change': '2024-04-25T08:39:25.529530+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.529530+0000', 'last_undegraded': '2024-04-25T08:39:25.529530+0000', 'last_fullsized': '2024-04-25T08:39:25.529530+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.523429+0000', 'last_change': '2024-04-25T08:39:24.523429+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.523429+0000', 'last_undegraded': '2024-04-25T08:39:24.523429+0000', 'last_fullsized': '2024-04-25T08:39:24.523429+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.521968+0000', 'last_change': '2024-04-25T08:39:24.521968+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.521968+0000', 'last_undegraded': '2024-04-25T08:39:24.521968+0000', 'last_fullsized': '2024-04-25T08:39:24.521968+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:30.294 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:25.524638+0000', 'last_change': '2024-04-25T08:39:25.524638+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.524638+0000', 'last_undegraded': '2024-04-25T08:39:25.524638+0000', 'last_fullsized': '2024-04-25T08:39:25.524638+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:24.523976+0000', 'last_change': '2024-04-25T08:39:24.523976+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:24.523976+0000', 'last_undegraded': '2024-04-25T08:39:24.523976+0000', 'last_fullsized': '2024-04-25T08:39:24.523976+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2024-04-25T08:39:30.295 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:39:25.517518+0000', 'last_change': '2024-04-25T08:39:25.517518+0000', 'last_active': '2024-04-25T08:39:25.517518+0000', 'last_peered': '2024-04-25T08:39:25.517518+0000', 'last_clean': '2024-04-25T08:39:25.517518+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.517518+0000', 'last_undegraded': '2024-04-25T08:39:25.517518+0000', 'last_fullsized': '2024-04-25T08:39:25.517518+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:25.517518+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:39:30.296 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2024-04-25T08:39:30.296 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 143, 'state': 'creating+activating', 'last_fresh': '2024-04-25T08:39:25.522508+0000', 'last_change': '2024-04-25T08:39:25.522508+0000', 'last_active': '2024-04-25T08:39:24.513339+0000', 'last_peered': '2024-04-25T08:39:24.513339+0000', 'last_clean': '2024-04-25T08:39:24.513339+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:25.522508+0000', 'last_undegraded': '2024-04-25T08:39:25.522508+0000', 'last_fullsized': '2024-04-25T08:39:25.522508+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:24.513339+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:30.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:30.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.424+0000 7f3360a9e640 1 Processor -- start 2024-04-25T08:39:30.418 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.424+0000 7f3360a9e640 1 -- start start 2024-04-25T08:39:30.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f3360a9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 0x7f335c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f3360a9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f335c0b6cb0 0x7f335c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f3360a9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c162690 con 0x7f335c0b5a60 2024-04-25T08:39:30.419 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f3360a9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f335c0d3800 con 0x7f335c0b6cb0 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f3360a9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0ceee0 con 0x7f335c15db70 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f335a59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 0x7f335c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.425+0000 7f3359d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f335c0b6cb0 0x7f335c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335a59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 0x7f335c162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:37640/0 (socket says 172.21.15.60:37640) 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335ad9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f335c0b5a60 0x7f335c06c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:59976/0 (socket says 172.21.15.60:59976) 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335a59c640 1 -- 172.21.15.60:0/1752361523 learned_addr learned my addr 172.21.15.60:0/1752361523 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335a59c640 1 -- 172.21.15.60:0/1752361523 >> v2:172.21.15.100:3300/0 conn(0x7f335c0b6cb0 msgr2=0x7f335c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335a59c640 1 --2- 172.21.15.60:0/1752361523 >> v2:172.21.15.100:3300/0 conn(0x7f335c0b6cb0 0x7f335c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335a59c640 1 -- 172.21.15.60:0/1752361523 >> v1:172.21.15.100:6789/0 conn(0x7f335c0b5a60 legacy=0x7f335c06c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f335a59c640 1 -- 172.21.15.60:0/1752361523 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0d1d10 con 0x7f335c15db70 2024-04-25T08:39:30.420 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.426+0000 7f3359d9b640 1 --2- 172.21.15.60:0/1752361523 >> v2:172.21.15.100:3300/0 conn(0x7f335c0b6cb0 0x7f335c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f335a59c640 1 --2- 172.21.15.60:0/1752361523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 0x7f335c162150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3344009870 tx=0x7f334402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eca415025e3d67a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.421 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f335959a640 1 -- 172.21.15.60:0/1752361523 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344037070 con 0x7f335c15db70 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f335959a640 1 -- 172.21.15.60:0/1752361523 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344032440 con 0x7f335c15db70 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f335959a640 1 -- 172.21.15.60:0/1752361523 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344032810 con 0x7f335c15db70 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 -- 172.21.15.60:0/1752361523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 msgr2=0x7f335c162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1752361523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 0x7f335c162150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3344009870 tx=0x7f334402a270 comp rx=0 tx=0).stop 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 -- 172.21.15.60:0/1752361523 shutdown_connections 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1752361523 >> v2:172.21.15.100:3300/0 conn(0x7f335c0b6cb0 0x7f335c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1752361523 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c15db70 0x7f335c162150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 -- 172.21.15.60:0/1752361523 >> 172.21.15.60:0/1752361523 conn(0x7f335c0d0520 msgr2=0x7f335c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.427+0000 7f3360a9e640 1 -- 172.21.15.60:0/1752361523 shutdown_connections 2024-04-25T08:39:30.422 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.428+0000 7f3360a9e640 1 -- 172.21.15.60:0/1752361523 wait complete. 2024-04-25T08:39:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.429+0000 7f3360a9e640 1 Processor -- start 2024-04-25T08:39:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.429+0000 7f3360a9e640 1 -- start start 2024-04-25T08:39:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.429+0000 7f3360a9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c0b5a60 0x7f335c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.424 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.429+0000 7f3360a9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 0x7f335c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3360a9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f335c15db70 0x7f335c15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3360a9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c15df50 con 0x7f335c0b6cb0 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3360a9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0d22d0 con 0x7f335c15db70 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3360a9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0ce530 con 0x7f335c0b5a60 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 0x7f335c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 0x7f335c15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52444/0 (socket says 172.21.15.60:52444) 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 -- 172.21.15.60:0/1979503545 learned_addr learned my addr 172.21.15.60:0/1979503545 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f335a59c640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c0b5a60 0x7f335c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f335ad9d640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f335c15db70 0x7f335c15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 -- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c0b5a60 msgr2=0x7f335c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c0b5a60 0x7f335c0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 -- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f335c15db70 msgr2=0x7f335c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f335c15db70 0x7f335c15c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.425 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 -- 172.21.15.60:0/1979503545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0cc5b0 con 0x7f335c0b6cb0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f335ad9d640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f335c15db70 0x7f335c15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3359d9b640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 0x7f335c15c090 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e9b0 tx=0x7f334c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c017070 con 0x7f335c0b6cb0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c00ca00 con 0x7f335c0b6cb0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00cdf0 con 0x7f335c0b6cb0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c0cc890 con 0x7f335c0b6cb0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.430+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c149410 con 0x7f335c0b6cb0 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.431+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c073990 con 0x7f335c0b6cb0 2024-04-25T08:39:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.432+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f334c01a070 con 0x7f335c0b6cb0 2024-04-25T08:39:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.432+0000 7f3352ffd640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f333c06b4d0 0x7f333c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.433+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f334c08a340 con 0x7f335c0b6cb0 2024-04-25T08:39:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.433+0000 7f335a59c640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f333c06b4d0 0x7f333c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.427 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.433+0000 7f335a59c640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f333c06b4d0 0x7f333c06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3344034000 tx=0x7f334402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.437+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f334c058960 con 0x7f335c0b6cb0 2024-04-25T08:39:30.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.693+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f335c0d1820 con 0x7f335c0b6cb0 2024-04-25T08:39:30.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.694+0000 7f3352ffd640 1 -- 172.21.15.60:0/1979503545 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1321 (secure 0 0 0) 0x7f334c05ca00 con 0x7f335c0b6cb0 2024-04-25T08:39:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:30.688 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":631,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":145,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033750,"num_in_osds":8,"osd_in_since":1714033741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":42},{"state_name":"creating+peering","count":36},{"state_name":"active+clean","count":16},{"state_name":"creating+activating","count":3}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":435810304,"bytes_avail":772658302976,"bytes_total":773094113280,"unknown_pgs_ratio":0.4329896867275238,"inactive_pgs_ratio":0.40206184983253479},"fsmap":{"epoch":89,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":9924}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T08:39:15.536051+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f333c06b4d0 msgr2=0x7f333c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f333c06b4d0 0x7f333c06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3344034000 tx=0x7f334402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 msgr2=0x7f335c15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 0x7f335c15c090 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e9b0 tx=0x7f334c00ee70 comp rx=0 tx=0).stop 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 shutdown_connections 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f333c06b4d0 0x7f333c06d990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f335c15db70 0x7f335c15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f335c0b6cb0 0x7f335c15c090 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 --2- 172.21.15.60:0/1979503545 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f335c0b5a60 0x7f335c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 >> 172.21.15.60:0/1979503545 conn(0x7f335c0d0520 msgr2=0x7f335c0c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 shutdown_connections 2024-04-25T08:39:30.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.696+0000 7f3360a9e640 1 -- 172.21.15.60:0/1979503545 wait complete. 2024-04-25T08:39:30.700 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:39:30.701 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.808+0000 7fd04e1e0640 1 Processor -- start 2024-04-25T08:39:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.808+0000 7fd04e1e0640 1 -- start start 2024-04-25T08:39:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd04e1e0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd04815dd20 0x7fd0480b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd04e1e0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0480d11b0 0x7fd0480d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd04e1e0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04815eac0 con 0x7fd04815e100 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd04e1e0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0480d1c00 con 0x7fd0480d11b0 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd04e1e0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04815eda0 con 0x7fd04815dd20 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd047fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd0480d11b0 0x7fd0480d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd046ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd04815e100 0x7fd0480d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:34608/0 (socket says 172.21.15.60:34608) 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd046ffd640 1 -- 172.21.15.60:0/3226781589 learned_addr learned my addr 172.21.15.60:0/3226781589 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.809+0000 7fd0477fe640 1 --2- 172.21.15.60:0/3226781589 >> v2:172.21.15.100:3300/0 conn(0x7fd04815dd20 0x7fd0480b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd047fff640 1 -- 172.21.15.60:0/3226781589 >> v2:172.21.15.100:3300/0 conn(0x7fd04815dd20 msgr2=0x7fd0480b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd047fff640 1 --2- 172.21.15.60:0/3226781589 >> v2:172.21.15.100:3300/0 conn(0x7fd04815dd20 0x7fd0480b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd047fff640 1 -- 172.21.15.60:0/3226781589 >> v1:172.21.15.60:6789/0 conn(0x7fd04815e100 legacy=0x7fd0480d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd047fff640 1 -- 172.21.15.60:0/3226781589 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04815f080 con 0x7fd0480d11b0 2024-04-25T08:39:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd047fff640 1 --2- 172.21.15.60:0/3226781589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0480d11b0 0x7fd0480d2370 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fd0480b6520 tx=0x7fd04002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965ed45dfe1b387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd0467fc640 1 -- 172.21.15.60:0/3226781589 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040007740 con 0x7fd0480d11b0 2024-04-25T08:39:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd0467fc640 1 -- 172.21.15.60:0/3226781589 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400078e0 con 0x7fd0480d11b0 2024-04-25T08:39:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.810+0000 7fd0467fc640 1 -- 172.21.15.60:0/3226781589 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040007cb0 con 0x7fd0480d11b0 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 -- 172.21.15.60:0/3226781589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0480d11b0 msgr2=0x7fd0480d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/3226781589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0480d11b0 0x7fd0480d2370 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fd0480b6520 tx=0x7fd04002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 -- 172.21.15.60:0/3226781589 shutdown_connections 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/3226781589 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd0480d11b0 0x7fd0480d2370 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/3226781589 >> v2:172.21.15.100:3300/0 conn(0x7fd04815dd20 0x7fd0480b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 -- 172.21.15.60:0/3226781589 >> 172.21.15.60:0/3226781589 conn(0x7fd048061a20 msgr2=0x7fd048061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 -- 172.21.15.60:0/3226781589 shutdown_connections 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.811+0000 7fd04e1e0640 1 -- 172.21.15.60:0/3226781589 wait complete. 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.812+0000 7fd04e1e0640 1 Processor -- start 2024-04-25T08:39:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.812+0000 7fd04e1e0640 1 -- start start 2024-04-25T08:39:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd04e1e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0480d11b0 0x7fd048151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd04e1e0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04815dd20 0x7fd0481f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd04e1e0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 0x7fd0480cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd04e1e0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04815e710 con 0x7fd04815e100 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd04e1e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04815e590 con 0x7fd0480d11b0 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd04e1e0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0481f50d0 con 0x7fd04815dd20 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 0x7fd0480cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 0x7fd0480cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:52456/0 (socket says 172.21.15.60:52456) 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 -- 172.21.15.60:0/1927769746 learned_addr learned my addr 172.21.15.60:0/1927769746 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd0477fe640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0480d11b0 0x7fd048151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 -- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04815dd20 msgr2=0x7fd0481f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd046ffd640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04815dd20 0x7fd0481f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04815dd20 0x7fd0481f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 -- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0480d11b0 msgr2=0x7fd048151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.813+0000 7fd047fff640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0480d11b0 0x7fd048151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd047fff640 1 -- 172.21.15.60:0/1927769746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0481f53b0 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd0477fe640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0480d11b0 0x7fd048151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd047fff640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 0x7fd0480cabe0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd04002abe0 tx=0x7fd040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040040070 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040031bc0 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0481f5690 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040045460 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.814+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd048152440 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.815+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0480b4ac0 con 0x7fd04815e100 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.815+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd040039040 con 0x7fd04815e100 2024-04-25T08:39:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.816+0000 7fd03ffff640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd02806b4d0 0x7fd02806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.816+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0400abc80 con 0x7fd04815e100 2024-04-25T08:39:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.816+0000 7fd0477fe640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd02806b4d0 0x7fd02806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.817+0000 7fd0477fe640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd02806b4d0 0x7fd02806d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.820+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0480b4ac0 con 0x7fd04815e100 2024-04-25T08:39:30.990 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:30.996+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd048073990 con 0x7fd02806b4d0 2024-04-25T08:39:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.002+0000 7fd03ffff640 1 -- 172.21.15.60:0/1927769746 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242992 (secure 0 0 0) 0x7fd048073990 con 0x7fd02806b4d0 2024-04-25T08:39:30.999 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:31.000 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd02806b4d0 msgr2=0x7fd02806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd02806b4d0 0x7fd02806d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038008040 comp rx=0 tx=0).stop 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 msgr2=0x7fd0480cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 0x7fd0480cabe0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fd04002abe0 tx=0x7fd040004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 shutdown_connections 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd04815e100 0x7fd0480cabe0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd04815dd20 0x7fd0481f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd02806b4d0 0x7fd02806d990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 --2- 172.21.15.60:0/1927769746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0480d11b0 0x7fd048151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.005+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 >> 172.21.15.60:0/1927769746 conn(0x7fd048061a20 msgr2=0x7fd048061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.006+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 shutdown_connections 2024-04-25T08:39:31.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:31.006+0000 7fd04e1e0640 1 -- 172.21.15.60:0/1927769746 wait complete. 2024-04-25T08:39:31.010 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-04-25T08:39:29.538661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425596,"kb_used_data":15084,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754549124,"statfs":{"total":773094113280,"available":772658302976,"internally_reserved":0,"allocated":15446016,"data_stored":7832377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.019512"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532972+0000","last_change":"2024-04-25T08:39:25.532972+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532972+0000","last_undegraded":"2024-04-25T08:39:25.532972+0000","last_fullsized":"2024-04-25T08:39:25.532972+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.533598+0000","last_change":"2024-04-25T08:39:25.533598+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.533598+0000","last_undegraded":"2024-04-25T08:39:25.533598+0000","last_fullsized":"2024-04-25T08:39:25.533598+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531925+0000","last_change":"2024-04-25T08:39:25.531925+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531925+0000","last_undegraded":"2024-04-25T08:39:25.531925+0000","last_fullsized":"2024-04-25T08:39:25.531925+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527182+0000","last_change":"2024-04-25T08:39:25.527182+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527182+0000","last_undegraded":"2024-04-25T08:39:25.527182+0000","last_fullsized":"2024-04-25T08:39:25.527182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525519+0000","last_change":"2024-04-25T08:39:25.525519+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525519+0000","last_undegraded":"2024-04-25T08:39:25.525519+0000","last_fullsized":"2024-04-25T08:39:25.525519+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.528068+0000","last_change":"2024-04-25T08:39:25.528068+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.528068+0000","last_undegraded":"2024-04-25T08:39:25.528068+0000","last_fullsized":"2024-04-25T08:39:25.528068+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.531168+0000","last_change":"2024-04-25T08:39:25.531168+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.531168+0000","last_undegraded":"2024-04-25T08:39:25.531168+0000","last_fullsized":"2024-04-25T08:39:25.531168+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529355+0000","last_change":"2024-04-25T08:39:25.529355+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529355+0000","last_undegraded":"2024-04-25T08:39:25.529355+0000","last_fullsized":"2024-04-25T08:39:25.529355+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.532541+0000","last_change":"2024-04-25T08:39:25.532541+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.532541+0000","last_undegraded":"2024-04-25T08:39:25.532541+0000","last_fullsized":"2024-04-25T08:39:25.532541+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530852+0000","last_change":"2024-04-25T08:39:25.530852+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530852+0000","last_undegraded":"2024-04-25T08:39:25.530852+0000","last_fullsized":"2024-04-25T08:39:25.530852+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521260+0000","last_change":"2024-04-25T08:39:24.521260+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521260+0000","last_undegraded":"2024-04-25T08:39:24.521260+0000","last_fullsized":"2024-04-25T08:39:24.521260+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542880+0000","last_change":"2024-04-25T08:39:25.541869+0000","last_active":"2024-04-25T08:39:25.542880+0000","last_peered":"2024-04-25T08:39:25.542880+0000","last_clean":"2024-04-25T08:39:25.542880+0000","last_became_active":"2024-04-25T08:39:25.541531+0000","last_became_peered":"2024-04-25T08:39:25.541531+0000","last_unstale":"2024-04-25T08:39:25.542880+0000","last_undegraded":"2024-04-25T08:39:25.542880+0000","last_fullsized":"2024-04-25T08:39:25.542880+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:57.694019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542218+0000","last_change":"2024-04-25T08:39:25.540429+0000","last_active":"2024-04-25T08:39:25.542218+0000","last_peered":"2024-04-25T08:39:25.542218+0000","last_clean":"2024-04-25T08:39:25.542218+0000","last_became_active":"2024-04-25T08:39:25.539563+0000","last_became_peered":"2024-04-25T08:39:25.539563+0000","last_unstale":"2024-04-25T08:39:25.542218+0000","last_undegraded":"2024-04-25T08:39:25.542218+0000","last_fullsized":"2024-04-25T08:39:25.542218+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:52.290637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530037+0000","last_change":"2024-04-25T08:39:25.530037+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530037+0000","last_undegraded":"2024-04-25T08:39:25.530037+0000","last_fullsized":"2024-04-25T08:39:25.530037+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.522955+0000","last_change":"2024-04-25T08:39:24.522955+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.522955+0000","last_undegraded":"2024-04-25T08:39:24.522955+0000","last_fullsized":"2024-04-25T08:39:24.522955+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524974+0000","last_change":"2024-04-25T08:39:24.524974+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524974+0000","last_undegraded":"2024-04-25T08:39:24.524974+0000","last_fullsized":"2024-04-25T08:39:24.524974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.525161+0000","last_change":"2024-04-25T08:39:25.525161+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.525161+0000","last_undegraded":"2024-04-25T08:39:25.525161+0000","last_fullsized":"2024-04-25T08:39:25.525161+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521231+0000","last_change":"2024-04-25T08:39:24.521231+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521231+0000","last_undegraded":"2024-04-25T08:39:24.521231+0000","last_fullsized":"2024-04-25T08:39:24.521231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.539017+0000","last_change":"2024-04-25T08:39:25.537615+0000","last_active":"2024-04-25T08:39:25.539017+0000","last_peered":"2024-04-25T08:39:25.539017+0000","last_clean":"2024-04-25T08:39:25.539017+0000","last_became_active":"2024-04-25T08:39:25.537345+0000","last_became_peered":"2024-04-25T08:39:25.537345+0000","last_unstale":"2024-04-25T08:39:25.539017+0000","last_undegraded":"2024-04-25T08:39:25.539017+0000","last_fullsized":"2024-04-25T08:39:25.539017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:50.758570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529484+0000","last_change":"2024-04-25T08:39:25.529484+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529484+0000","last_undegraded":"2024-04-25T08:39:25.529484+0000","last_fullsized":"2024-04-25T08:39:25.529484+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521239+0000","last_change":"2024-04-25T08:39:24.521239+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521239+0000","last_undegraded":"2024-04-25T08:39:24.521239+0000","last_fullsized":"2024-04-25T08:39:24.521239+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.542936+0000","last_change":"2024-04-25T08:39:25.541562+0000","last_active":"2024-04-25T08:39:25.542936+0000","last_peered":"2024-04-25T08:39:25.542936+0000","last_clean":"2024-04-25T08:39:25.542936+0000","last_became_active":"2024-04-25T08:39:25.541283+0000","last_became_peered":"2024-04-25T08:39:25.541283+0000","last_unstale":"2024-04-25T08:39:25.542936+0000","last_undegraded":"2024-04-25T08:39:25.542936+0000","last_fullsized":"2024-04-25T08:39:25.542936+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:38.416592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543099+0000","last_change":"2024-04-25T08:39:25.540300+0000","last_active":"2024-04-25T08:39:25.543099+0000","last_peered":"2024-04-25T08:39:25.543099+0000","last_clean":"2024-04-25T08:39:25.543099+0000","last_became_active":"2024-04-25T08:39:25.539840+0000","last_became_peered":"2024-04-25T08:39:25.539840+0000","last_unstale":"2024-04-25T08:39:25.543099+0000","last_undegraded":"2024-04-25T08:39:25.543099+0000","last_fullsized":"2024-04-25T08:39:25.543099+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523795+0000","last_change":"2024-04-25T08:39:24.523795+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523795+0000","last_undegraded":"2024-04-25T08:39:24.523795+0000","last_fullsized":"2024-04-25T08:39:24.523795+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521328+0000","last_change":"2024-04-25T08:39:24.521328+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521328+0000","last_undegraded":"2024-04-25T08:39:24.521328+0000","last_fullsized":"2024-04-25T08:39:24.521328+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.531819+0000","last_change":"2024-04-25T08:39:25.531155+0000","last_active":"2024-04-25T08:39:25.531819+0000","last_peered":"2024-04-25T08:39:25.531819+0000","last_clean":"2024-04-25T08:39:25.531819+0000","last_became_active":"2024-04-25T08:39:25.530883+0000","last_became_peered":"2024-04-25T08:39:25.530883+0000","last_unstale":"2024-04-25T08:39:25.531819+0000","last_undegraded":"2024-04-25T08:39:25.531819+0000","last_fullsized":"2024-04-25T08:39:25.531819+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543391+0000","last_change":"2024-04-25T08:39:25.542778+0000","last_active":"2024-04-25T08:39:25.543391+0000","last_peered":"2024-04-25T08:39:25.543391+0000","last_clean":"2024-04-25T08:39:25.543391+0000","last_became_active":"2024-04-25T08:39:25.542474+0000","last_became_peered":"2024-04-25T08:39:25.542474+0000","last_unstale":"2024-04-25T08:39:25.543391+0000","last_undegraded":"2024-04-25T08:39:25.543391+0000","last_fullsized":"2024-04-25T08:39:25.543391+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543025+0000","last_change":"2024-04-25T08:39:25.540523+0000","last_active":"2024-04-25T08:39:25.543025+0000","last_peered":"2024-04-25T08:39:25.543025+0000","last_clean":"2024-04-25T08:39:25.543025+0000","last_became_active":"2024-04-25T08:39:25.540179+0000","last_became_peered":"2024-04-25T08:39:25.540179+0000","last_unstale":"2024-04-25T08:39:25.543025+0000","last_undegraded":"2024-04-25T08:39:25.543025+0000","last_fullsized":"2024-04-25T08:39:25.543025+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529469+0000","last_change":"2024-04-25T08:39:25.529469+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529469+0000","last_undegraded":"2024-04-25T08:39:25.529469+0000","last_fullsized":"2024-04-25T08:39:25.529469+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523172+0000","last_change":"2024-04-25T08:39:24.523172+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523172+0000","last_undegraded":"2024-04-25T08:39:24.523172+0000","last_fullsized":"2024-04-25T08:39:24.523172+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530763+0000","last_change":"2024-04-25T08:39:25.530763+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530763+0000","last_undegraded":"2024-04-25T08:39:25.530763+0000","last_fullsized":"2024-04-25T08:39:25.530763+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544961+0000","last_change":"2024-04-25T08:39:25.544537+0000","last_active":"2024-04-25T08:39:25.544961+0000","last_peered":"2024-04-25T08:39:25.544961+0000","last_clean":"2024-04-25T08:39:25.544961+0000","last_became_active":"2024-04-25T08:39:25.544239+0000","last_became_peered":"2024-04-25T08:39:25.544239+0000","last_unstale":"2024-04-25T08:39:25.544961+0000","last_undegraded":"2024-04-25T08:39:25.544961+0000","last_fullsized":"2024-04-25T08:39:25.544961+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:09.533163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.530428+0000","last_change":"2024-04-25T08:39:25.530428+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.530428+0000","last_undegraded":"2024-04-25T08:39:25.530428+0000","last_fullsized":"2024-04-25T08:39:25.530428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523778+0000","last_change":"2024-04-25T08:39:24.523778+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523778+0000","last_undegraded":"2024-04-25T08:39:24.523778+0000","last_fullsized":"2024-04-25T08:39:24.523778+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529985+0000","last_change":"2024-04-25T08:39:25.529985+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529985+0000","last_undegraded":"2024-04-25T08:39:25.529985+0000","last_fullsized":"2024-04-25T08:39:25.529985+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.536302+0000","last_change":"2024-04-25T08:39:25.534723+0000","last_active":"2024-04-25T08:39:25.536302+0000","last_peered":"2024-04-25T08:39:25.536302+0000","last_clean":"2024-04-25T08:39:25.536302+0000","last_became_active":"2024-04-25T08:39:25.534462+0000","last_became_peered":"2024-04-25T08:39:25.534462+0000","last_unstale":"2024-04-25T08:39:25.536302+0000","last_undegraded":"2024-04-25T08:39:25.536302+0000","last_fullsized":"2024-04-25T08:39:25.536302+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:46.964885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529814+0000","last_change":"2024-04-25T08:39:25.529814+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529814+0000","last_undegraded":"2024-04-25T08:39:25.529814+0000","last_fullsized":"2024-04-25T08:39:25.529814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.524065+0000","last_change":"2024-04-25T08:39:24.524065+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.524065+0000","last_undegraded":"2024-04-25T08:39:24.524065+0000","last_fullsized":"2024-04-25T08:39:24.524065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529165+0000","last_change":"2024-04-25T08:39:25.529165+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529165+0000","last_undegraded":"2024-04-25T08:39:25.529165+0000","last_fullsized":"2024-04-25T08:39:25.529165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522548+0000","last_change":"2024-04-25T08:39:25.522548+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522548+0000","last_undegraded":"2024-04-25T08:39:25.522548+0000","last_fullsized":"2024-04-25T08:39:25.522548+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.525390+0000","last_change":"2024-04-25T08:39:24.525390+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.525390+0000","last_undegraded":"2024-04-25T08:39:24.525390+0000","last_fullsized":"2024-04-25T08:39:24.525390+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.535065+0000","last_change":"2024-04-25T08:39:25.533867+0000","last_active":"2024-04-25T08:39:25.535065+0000","last_peered":"2024-04-25T08:39:25.535065+0000","last_clean":"2024-04-25T08:39:25.535065+0000","last_became_active":"2024-04-25T08:39:25.533567+0000","last_became_peered":"2024-04-25T08:39:25.533567+0000","last_unstale":"2024-04-25T08:39:25.535065+0000","last_undegraded":"2024-04-25T08:39:25.535065+0000","last_fullsized":"2024-04-25T08:39:25.535065+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:46.200944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.527139+0000","last_change":"2024-04-25T08:39:25.527139+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.527139+0000","last_undegraded":"2024-04-25T08:39:25.527139+0000","last_fullsized":"2024-04-25T08:39:25.527139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.543080+0000","last_change":"2024-04-25T08:39:25.540746+0000","last_active":"2024-04-25T08:39:25.543080+0000","last_peered":"2024-04-25T08:39:25.543080+0000","last_clean":"2024-04-25T08:39:25.543080+0000","last_became_active":"2024-04-25T08:39:25.540230+0000","last_became_peered":"2024-04-25T08:39:25.540230+0000","last_unstale":"2024-04-25T08:39:25.543080+0000","last_undegraded":"2024-04-25T08:39:25.543080+0000","last_fullsized":"2024-04-25T08:39:25.543080+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.994223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.529530+0000","last_change":"2024-04-25T08:39:25.529530+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.529530+0000","last_undegraded":"2024-04-25T08:39:25.529530+0000","last_fullsized":"2024-04-25T08:39:25.529530+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523429+0000","last_change":"2024-04-25T08:39:24.523429+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523429+0000","last_undegraded":"2024-04-25T08:39:24.523429+0000","last_fullsized":"2024-04-25T08:39:24.523429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.521968+0000","last_change":"2024-04-25T08:39:24.521968+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.521968+0000","last_undegraded":"2024-04-25T08:39:24.521968+0000","last_fullsized":"2024-04-25T08:39:24.521968+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.541132+0000","last_change":"2024-04-25T08:39:25.539417+0000","last_active":"2024-04-25T08:39:25.541132+0000","last_peered":"2024-04-25T08:39:25.541132+0000","last_clean":"2024-04-25T08:39:25.541132+0000","last_became_active":"2024-04-25T08:39:25.539015+0000","last_became_peered":"2024-04-25T08:39:25.539015+0000","last_unstale":"2024-04-25T08:39:25.541132+0000","last_undegraded":"2024-04-25T08:39:25.541132+0000","last_fullsized":"2024-04-25T08:39:25.541132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:56.925675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-04-25T08:39:25.524638+0000","last_change":"2024-04-25T08:39:25.524638+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.524638+0000","last_undegraded":"2024-04-25T08:39:25.524638+0000","last_fullsized":"2024-04-25T08:39:25.524638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2024-04-25T08:39:24.523976+0000","last_change":"2024-04-25T08:39:24.523976+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:24.523976+0000","last_undegraded":"2024-04-25T08:39:24.523976+0000","last_fullsized":"2024-04-25T08:39:24.523976+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.544089+0000","last_change":"2024-04-25T08:39:25.543519+0000","last_active":"2024-04-25T08:39:25.544089+0000","last_peered":"2024-04-25T08:39:25.544089+0000","last_clean":"2024-04-25T08:39:25.544089+0000","last_became_active":"2024-04-25T08:39:25.543148+0000","last_became_peered":"2024-04-25T08:39:25.543148+0000","last_unstale":"2024-04-25T08:39:25.544089+0000","last_undegraded":"2024-04-25T08:39:25.544089+0000","last_fullsized":"2024-04-25T08:39:25.544089+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.721871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.538234+0000","last_change":"2024-04-25T08:39:25.536756+0000","last_active":"2024-04-25T08:39:25.538234+0000","last_peered":"2024-04-25T08:39:25.538234+0000","last_clean":"2024-04-25T08:39:25.538234+0000","last_became_active":"2024-04-25T08:39:25.535762+0000","last_became_peered":"2024-04-25T08:39:25.535762+0000","last_unstale":"2024-04-25T08:39:25.538234+0000","last_undegraded":"2024-04-25T08:39:25.538234+0000","last_fullsized":"2024-04-25T08:39:25.538234+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:10.416181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:39:25.517518+0000","last_change":"2024-04-25T08:39:25.517518+0000","last_active":"2024-04-25T08:39:25.517518+0000","last_peered":"2024-04-25T08:39:25.517518+0000","last_clean":"2024-04-25T08:39:25.517518+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.517518+0000","last_undegraded":"2024-04-25T08:39:25.517518+0000","last_fullsized":"2024-04-25T08:39:25.517518+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":3,"reported_epoch":143,"state":"creating+activating","last_fresh":"2024-04-25T08:39:25.522508+0000","last_change":"2024-04-25T08:39:25.522508+0000","last_active":"2024-04-25T08:39:24.513339+0000","last_peered":"2024-04-25T08:39:24.513339+0000","last_clean":"2024-04-25T08:39:24.513339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:25.522508+0000","last_undegraded":"2024-04-25T08:39:25.522508+0000","last_fullsized":"2024-04-25T08:39:25.522508+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":398,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:39:25.532864+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:25.532864+0000","last_peered":"2024-04-25T08:39:25.532864+0000","last_clean":"2024-04-25T08:39:25.532864+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:25.532864+0000","last_undegraded":"2024-04-25T08:39:25.532864+0000","last_fullsized":"2024-04-25T08:39:25.532864+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":3},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51560,"kb_used_data":2328,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320280,"statfs":{"total":96636764160,"available":96583966720,"internally_reserved":0,"allocated":2383872,"data_stored":1425990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542271,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50980,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320860,"statfs":{"total":96636764160,"available":96584560640,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542271,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46824,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325016,"statfs":{"total":96636764160,"available":96588816384,"internally_reserved":0,"allocated":1789952,"data_stored":835622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46804,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325036,"statfs":{"total":96636764160,"available":96588836864,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59292,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312548,"statfs":{"total":96636764160,"available":96576049152,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55132,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1777664,"data_stored":828955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2371584,"data_stored":1419323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:31.824 INFO:tasks.ceph.mds.a.smithi060.stderr:2024-04-25T08:39:31.831+0000 7f8a4b65a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:39:34.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.136+0000 7f88997b0640 1 Processor -- start 2024-04-25T08:39:34.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.136+0000 7f88997b0640 1 -- start start 2024-04-25T08:39:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88997b0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f88940d1020 0x7f88940d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88997b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88940b97a0 con 0x7f88940b2cb0 2024-04-25T08:39:34.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88997b0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88940d1c50 con 0x7f889415dbe0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88997b0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889415dfc0 con 0x7f88940d1020 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f888bfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f88940b2cb0 0x7f88940d2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36458/0 (socket says 172.21.15.60:36458) 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f888bfff640 1 -- 172.21.15.60:0/1811474447 learned_addr learned my addr 172.21.15.60:0/1811474447 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88937fe640 1 --2- 172.21.15.60:0/1811474447 >> v2:172.21.15.100:3300/0 conn(0x7f88940d1020 0x7f88940d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411193457 0 0) 0x7f88940b97a0 con 0x7f88940b2cb0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8880003520 con 0x7f88940b2cb0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2108393608 0 0) 0x7f8880003520 con 0x7f88940b2cb0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 >> v2:172.21.15.100:3300/0 conn(0x7f88940d1020 msgr2=0x7f88940d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 --2- 172.21.15.60:0/1811474447 >> v2:172.21.15.100:3300/0 conn(0x7f88940d1020 0x7f88940d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 >> v1:172.21.15.100:6789/0 conn(0x7f889415dbe0 legacy=0x7f88940d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940d1f30 con 0x7f88940b2cb0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.137+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f887c001670 con 0x7f88940b2cb0 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 >> v1:172.21.15.60:6789/0 conn(0x7f88940b2cb0 legacy=0x7f88940d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.133 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88800039b0 0x7f8880003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8880004800 0x7f8880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88800051c0 0x7f888000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888000c030 con 0x7f8880004800 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888000c1e0 con 0x7f88800051c0 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888000c3e0 con 0x7f88800039b0 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f888bfff640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8880004800 0x7f8880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88937fe640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88800051c0 0x7f888000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f8892ffd640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88800039b0 0x7f8880003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f888bfff640 1 -- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88800039b0 msgr2=0x7f8880003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f888bfff640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f88800039b0 0x7f8880003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f888bfff640 1 -- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88800051c0 msgr2=0x7f888000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f888bfff640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88800051c0 0x7f888000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.134 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f88937fe640 1 -- 172.21.15.60:0/1811474447 reap_dead start 2024-04-25T08:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.138+0000 7f888bfff640 1 -- 172.21.15.60:0/1811474447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8880003520 con 0x7f8880004800 2024-04-25T08:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.139+0000 7f888bfff640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8880004800 0x7f8880004c80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c210 tx=0x7f8880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad1430c55cd2446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.139+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c007d10 con 0x7f8880004800 2024-04-25T08:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.139+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c005ad0 con 0x7f8880004800 2024-04-25T08:39:34.135 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.139+0000 7f88927fc640 1 -- 172.21.15.60:0/1811474447 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c007070 con 0x7f8880004800 2024-04-25T08:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 -- 172.21.15.60:0/1811474447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8880004800 msgr2=0x7f8880004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8880004800 0x7f8880004c80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c210 tx=0x7f8880005830 comp rx=0 tx=0).stop 2024-04-25T08:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 -- 172.21.15.60:0/1811474447 shutdown_connections 2024-04-25T08:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 --2- 172.21.15.60:0/1811474447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8880004800 0x7f8880004c80 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.138 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 -- 172.21.15.60:0/1811474447 >> 172.21.15.60:0/1811474447 conn(0x7f8894061a20 msgr2=0x7f8894061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 -- 172.21.15.60:0/1811474447 shutdown_connections 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.140+0000 7f88997b0640 1 -- 172.21.15.60:0/1811474447 wait complete. 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.141+0000 7f88997b0640 1 Processor -- start 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.141+0000 7f88997b0640 1 -- start start 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88997b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8880004800 0x7f88940c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88997b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 0x7f88941f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88997b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f889415bf70 0x7f889415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88997b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88940b92a0 con 0x7f8894160a00 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88997b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88940d1a00 con 0x7f889415bf70 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88997b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88940b9420 con 0x7f8880004800 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 0x7f88941f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 0x7f88941f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54228/0 (socket says 172.21.15.60:54228) 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 -- 172.21.15.60:0/1947099108 learned_addr learned my addr 172.21.15.60:0/1947099108 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f8892ffd640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8880004800 0x7f88940c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88937fe640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f889415bf70 0x7f889415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 -- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8880004800 msgr2=0x7f88940c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8880004800 0x7f88940c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 -- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f889415bf70 msgr2=0x7f889415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f889415bf70 0x7f889415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 -- 172.21.15.60:0/1947099108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940ccc10 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f88937fe640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f889415bf70 0x7f889415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888bfff640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 0x7f88941f3640 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c330 tx=0x7f88940d1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c03f020 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.142+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c001470 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.143+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0049e0 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.143+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88940ccef0 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.143+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88941f3b80 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.144+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894073990 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.144+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f887c042020 con 0x7f8894160a00 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.144+0000 7f888b7fe640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f887406b4d0 0x7f887406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.144+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f887c0a9df0 con 0x7f8894160a00 2024-04-25T08:39:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.145+0000 7f8892ffd640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f887406b4d0 0x7f887406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.145+0000 7f8892ffd640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f887406b4d0 0x7f887406d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.149+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f887c078b70 con 0x7f8894160a00 2024-04-25T08:39:34.319 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.326+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88940d0d00 con 0x7f887406b4d0 2024-04-25T08:39:34.326 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.333+0000 7f888b7fe640 1 -- 172.21.15.60:0/1947099108 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f88940d0d00 con 0x7f887406b4d0 2024-04-25T08:39:34.328 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:34.330 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:39:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f887406b4d0 msgr2=0x7f887406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f887406b4d0 0x7f887406d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f8884009040 comp rx=0 tx=0).stop 2024-04-25T08:39:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 msgr2=0x7f88941f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.331 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 0x7f88941f3640 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c330 tx=0x7f88940d1e20 comp rx=0 tx=0).stop 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 shutdown_connections 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f887406b4d0 0x7f887406d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f889415bf70 0x7f889415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8894160a00 0x7f88941f3640 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 --2- 172.21.15.60:0/1947099108 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8880004800 0x7f88940c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 >> 172.21.15.60:0/1947099108 conn(0x7f8894061a20 msgr2=0x7f88940d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 shutdown_connections 2024-04-25T08:39:34.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.335+0000 7f88997b0640 1 -- 172.21.15.60:0/1947099108 wait complete. 2024-04-25T08:39:34.340 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-04-25T08:39:33.539690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":425896,"kb_used_data":15400,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754548824,"statfs":{"total":773094113280,"available":772657995776,"internally_reserved":0,"allocated":15769600,"data_stored":7981404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":420204080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001786"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853619+0000","last_change":"2024-04-25T08:39:26.552236+0000","last_active":"2024-04-25T08:39:27.853619+0000","last_peered":"2024-04-25T08:39:27.853619+0000","last_clean":"2024-04-25T08:39:27.853619+0000","last_became_active":"2024-04-25T08:39:26.551963+0000","last_became_peered":"2024-04-25T08:39:26.551963+0000","last_unstale":"2024-04-25T08:39:27.853619+0000","last_undegraded":"2024-04-25T08:39:27.853619+0000","last_fullsized":"2024-04-25T08:39:27.853619+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:42.412035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534673+0000","last_change":"2024-04-25T08:39:26.546253+0000","last_active":"2024-04-25T08:39:27.534673+0000","last_peered":"2024-04-25T08:39:27.534673+0000","last_clean":"2024-04-25T08:39:27.534673+0000","last_became_active":"2024-04-25T08:39:26.545951+0000","last_became_peered":"2024-04-25T08:39:26.545951+0000","last_unstale":"2024-04-25T08:39:27.534673+0000","last_undegraded":"2024-04-25T08:39:27.534673+0000","last_fullsized":"2024-04-25T08:39:27.534673+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:21.724428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534305+0000","last_change":"2024-04-25T08:39:26.549606+0000","last_active":"2024-04-25T08:39:27.534305+0000","last_peered":"2024-04-25T08:39:27.534305+0000","last_clean":"2024-04-25T08:39:27.534305+0000","last_became_active":"2024-04-25T08:39:26.548971+0000","last_became_peered":"2024-04-25T08:39:26.548971+0000","last_unstale":"2024-04-25T08:39:27.534305+0000","last_undegraded":"2024-04-25T08:39:27.534305+0000","last_fullsized":"2024-04-25T08:39:27.534305+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:59.355782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535389+0000","last_change":"2024-04-25T08:39:26.545791+0000","last_active":"2024-04-25T08:39:27.535389+0000","last_peered":"2024-04-25T08:39:27.535389+0000","last_clean":"2024-04-25T08:39:27.535389+0000","last_became_active":"2024-04-25T08:39:26.545498+0000","last_became_peered":"2024-04-25T08:39:26.545498+0000","last_unstale":"2024-04-25T08:39:27.535389+0000","last_undegraded":"2024-04-25T08:39:27.535389+0000","last_fullsized":"2024-04-25T08:39:27.535389+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:30.122081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757904+0000","last_change":"2024-04-25T08:39:26.553847+0000","last_active":"2024-04-25T08:39:28.757904+0000","last_peered":"2024-04-25T08:39:28.757904+0000","last_clean":"2024-04-25T08:39:28.757904+0000","last_became_active":"2024-04-25T08:39:26.553570+0000","last_became_peered":"2024-04-25T08:39:26.553570+0000","last_unstale":"2024-04-25T08:39:28.757904+0000","last_undegraded":"2024-04-25T08:39:28.757904+0000","last_fullsized":"2024-04-25T08:39:28.757904+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:48.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757175+0000","last_change":"2024-04-25T08:39:26.556266+0000","last_active":"2024-04-25T08:39:28.757175+0000","last_peered":"2024-04-25T08:39:28.757175+0000","last_clean":"2024-04-25T08:39:28.757175+0000","last_became_active":"2024-04-25T08:39:26.555943+0000","last_became_peered":"2024-04-25T08:39:26.555943+0000","last_unstale":"2024-04-25T08:39:28.757175+0000","last_undegraded":"2024-04-25T08:39:28.757175+0000","last_fullsized":"2024-04-25T08:39:28.757175+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:53.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853487+0000","last_change":"2024-04-25T08:39:26.548935+0000","last_active":"2024-04-25T08:39:27.853487+0000","last_peered":"2024-04-25T08:39:27.853487+0000","last_clean":"2024-04-25T08:39:27.853487+0000","last_became_active":"2024-04-25T08:39:26.547947+0000","last_became_peered":"2024-04-25T08:39:26.547947+0000","last_unstale":"2024-04-25T08:39:27.853487+0000","last_undegraded":"2024-04-25T08:39:27.853487+0000","last_fullsized":"2024-04-25T08:39:27.853487+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:58.698198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535562+0000","last_change":"2024-04-25T08:39:26.546540+0000","last_active":"2024-04-25T08:39:27.535562+0000","last_peered":"2024-04-25T08:39:27.535562+0000","last_clean":"2024-04-25T08:39:27.535562+0000","last_became_active":"2024-04-25T08:39:26.545488+0000","last_became_peered":"2024-04-25T08:39:26.545488+0000","last_unstale":"2024-04-25T08:39:27.535562+0000","last_undegraded":"2024-04-25T08:39:27.535562+0000","last_fullsized":"2024-04-25T08:39:27.535562+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:58.726050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.758113+0000","last_change":"2024-04-25T08:39:26.565543+0000","last_active":"2024-04-25T08:39:28.758113+0000","last_peered":"2024-04-25T08:39:28.758113+0000","last_clean":"2024-04-25T08:39:28.758113+0000","last_became_active":"2024-04-25T08:39:26.564891+0000","last_became_peered":"2024-04-25T08:39:26.564891+0000","last_unstale":"2024-04-25T08:39:28.758113+0000","last_undegraded":"2024-04-25T08:39:28.758113+0000","last_fullsized":"2024-04-25T08:39:28.758113+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:52.723530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757947+0000","last_change":"2024-04-25T08:39:26.557964+0000","last_active":"2024-04-25T08:39:28.757947+0000","last_peered":"2024-04-25T08:39:28.757947+0000","last_clean":"2024-04-25T08:39:28.757947+0000","last_became_active":"2024-04-25T08:39:26.557642+0000","last_became_peered":"2024-04-25T08:39:26.557642+0000","last_unstale":"2024-04-25T08:39:28.757947+0000","last_undegraded":"2024-04-25T08:39:28.757947+0000","last_fullsized":"2024-04-25T08:39:28.757947+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:21.724428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559109+0000","last_change":"2024-04-25T08:39:26.548310+0000","last_active":"2024-04-25T08:39:27.559109+0000","last_peered":"2024-04-25T08:39:27.559109+0000","last_clean":"2024-04-25T08:39:27.559109+0000","last_became_active":"2024-04-25T08:39:26.548018+0000","last_became_peered":"2024-04-25T08:39:26.548018+0000","last_unstale":"2024-04-25T08:39:27.559109+0000","last_undegraded":"2024-04-25T08:39:27.559109+0000","last_fullsized":"2024-04-25T08:39:27.559109+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:56.446222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535416+0000","last_change":"2024-04-25T08:39:26.559457+0000","last_active":"2024-04-25T08:39:27.535416+0000","last_peered":"2024-04-25T08:39:27.535416+0000","last_clean":"2024-04-25T08:39:27.535416+0000","last_became_active":"2024-04-25T08:39:26.558887+0000","last_became_peered":"2024-04-25T08:39:26.558887+0000","last_unstale":"2024-04-25T08:39:27.535416+0000","last_undegraded":"2024-04-25T08:39:27.535416+0000","last_fullsized":"2024-04-25T08:39:27.535416+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:06.980994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757589+0000","last_change":"2024-04-25T08:39:26.561096+0000","last_active":"2024-04-25T08:39:28.757589+0000","last_peered":"2024-04-25T08:39:28.757589+0000","last_clean":"2024-04-25T08:39:28.757589+0000","last_became_active":"2024-04-25T08:39:26.560436+0000","last_became_peered":"2024-04-25T08:39:26.560436+0000","last_unstale":"2024-04-25T08:39:28.757589+0000","last_undegraded":"2024-04-25T08:39:28.757589+0000","last_fullsized":"2024-04-25T08:39:28.757589+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:25.601013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536030+0000","last_change":"2024-04-25T08:39:26.560171+0000","last_active":"2024-04-25T08:39:27.536030+0000","last_peered":"2024-04-25T08:39:27.536030+0000","last_clean":"2024-04-25T08:39:27.536030+0000","last_became_active":"2024-04-25T08:39:26.559865+0000","last_became_peered":"2024-04-25T08:39:26.559865+0000","last_unstale":"2024-04-25T08:39:27.536030+0000","last_undegraded":"2024-04-25T08:39:27.536030+0000","last_fullsized":"2024-04-25T08:39:27.536030+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:39.420771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534228+0000","last_change":"2024-04-25T08:39:26.541904+0000","last_active":"2024-04-25T08:39:27.534228+0000","last_peered":"2024-04-25T08:39:27.534228+0000","last_clean":"2024-04-25T08:39:27.534228+0000","last_became_active":"2024-04-25T08:39:26.541228+0000","last_became_peered":"2024-04-25T08:39:26.541228+0000","last_unstale":"2024-04-25T08:39:27.534228+0000","last_undegraded":"2024-04-25T08:39:27.534228+0000","last_fullsized":"2024-04-25T08:39:27.534228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:46.244387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757100+0000","last_change":"2024-04-25T08:39:26.564174+0000","last_active":"2024-04-25T08:39:28.757100+0000","last_peered":"2024-04-25T08:39:28.757100+0000","last_clean":"2024-04-25T08:39:28.757100+0000","last_became_active":"2024-04-25T08:39:26.563831+0000","last_became_peered":"2024-04-25T08:39:26.563831+0000","last_unstale":"2024-04-25T08:39:28.757100+0000","last_undegraded":"2024-04-25T08:39:28.757100+0000","last_fullsized":"2024-04-25T08:39:28.757100+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:59.355782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536273+0000","last_change":"2024-04-25T08:39:26.542617+0000","last_active":"2024-04-25T08:39:27.536273+0000","last_peered":"2024-04-25T08:39:27.536273+0000","last_clean":"2024-04-25T08:39:27.536273+0000","last_became_active":"2024-04-25T08:39:26.542347+0000","last_became_peered":"2024-04-25T08:39:26.542347+0000","last_unstale":"2024-04-25T08:39:27.536273+0000","last_undegraded":"2024-04-25T08:39:27.536273+0000","last_fullsized":"2024-04-25T08:39:27.536273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:58.698198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559238+0000","last_change":"2024-04-25T08:39:26.552785+0000","last_active":"2024-04-25T08:39:27.559238+0000","last_peered":"2024-04-25T08:39:27.559238+0000","last_clean":"2024-04-25T08:39:27.559238+0000","last_became_active":"2024-04-25T08:39:26.552512+0000","last_became_peered":"2024-04-25T08:39:26.552512+0000","last_unstale":"2024-04-25T08:39:27.559238+0000","last_undegraded":"2024-04-25T08:39:27.559238+0000","last_fullsized":"2024-04-25T08:39:27.559238+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:41.368195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539233+0000","last_change":"2024-04-25T08:39:26.556148+0000","last_active":"2024-04-25T08:39:28.539233+0000","last_peered":"2024-04-25T08:39:28.539233+0000","last_clean":"2024-04-25T08:39:28.539233+0000","last_became_active":"2024-04-25T08:39:26.555727+0000","last_became_peered":"2024-04-25T08:39:26.555727+0000","last_unstale":"2024-04-25T08:39:28.539233+0000","last_undegraded":"2024-04-25T08:39:28.539233+0000","last_fullsized":"2024-04-25T08:39:28.539233+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:38.027963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757493+0000","last_change":"2024-04-25T08:39:26.554223+0000","last_active":"2024-04-25T08:39:28.757493+0000","last_peered":"2024-04-25T08:39:28.757493+0000","last_clean":"2024-04-25T08:39:28.757493+0000","last_became_active":"2024-04-25T08:39:26.553895+0000","last_became_peered":"2024-04-25T08:39:26.553895+0000","last_unstale":"2024-04-25T08:39:28.757493+0000","last_undegraded":"2024-04-25T08:39:28.757493+0000","last_fullsized":"2024-04-25T08:39:28.757493+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:27.547598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757745+0000","last_change":"2024-04-25T08:39:26.565257+0000","last_active":"2024-04-25T08:39:28.757745+0000","last_peered":"2024-04-25T08:39:28.757745+0000","last_clean":"2024-04-25T08:39:28.757745+0000","last_became_active":"2024-04-25T08:39:26.564792+0000","last_became_peered":"2024-04-25T08:39:26.564792+0000","last_unstale":"2024-04-25T08:39:28.757745+0000","last_undegraded":"2024-04-25T08:39:28.757745+0000","last_fullsized":"2024-04-25T08:39:28.757745+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:48.515895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853545+0000","last_change":"2024-04-25T08:39:26.548644+0000","last_active":"2024-04-25T08:39:27.853545+0000","last_peered":"2024-04-25T08:39:27.853545+0000","last_clean":"2024-04-25T08:39:27.853545+0000","last_became_active":"2024-04-25T08:39:26.548381+0000","last_became_peered":"2024-04-25T08:39:26.548381+0000","last_unstale":"2024-04-25T08:39:27.853545+0000","last_undegraded":"2024-04-25T08:39:27.853545+0000","last_fullsized":"2024-04-25T08:39:27.853545+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:10.537342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535743+0000","last_change":"2024-04-25T08:39:26.544220+0000","last_active":"2024-04-25T08:39:27.535743+0000","last_peered":"2024-04-25T08:39:27.535743+0000","last_clean":"2024-04-25T08:39:27.535743+0000","last_became_active":"2024-04-25T08:39:26.543936+0000","last_became_peered":"2024-04-25T08:39:26.543936+0000","last_unstale":"2024-04-25T08:39:27.535743+0000","last_undegraded":"2024-04-25T08:39:27.535743+0000","last_fullsized":"2024-04-25T08:39:27.535743+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:29.515498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559330+0000","last_change":"2024-04-25T08:39:26.566108+0000","last_active":"2024-04-25T08:39:27.559330+0000","last_peered":"2024-04-25T08:39:27.559330+0000","last_clean":"2024-04-25T08:39:27.559330+0000","last_became_active":"2024-04-25T08:39:26.565778+0000","last_became_peered":"2024-04-25T08:39:26.565778+0000","last_unstale":"2024-04-25T08:39:27.559330+0000","last_undegraded":"2024-04-25T08:39:27.559330+0000","last_fullsized":"2024-04-25T08:39:27.559330+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:39.469980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853659+0000","last_change":"2024-04-25T08:39:26.549953+0000","last_active":"2024-04-25T08:39:27.853659+0000","last_peered":"2024-04-25T08:39:27.853659+0000","last_clean":"2024-04-25T08:39:27.853659+0000","last_became_active":"2024-04-25T08:39:26.549674+0000","last_became_peered":"2024-04-25T08:39:26.549674+0000","last_unstale":"2024-04-25T08:39:27.853659+0000","last_undegraded":"2024-04-25T08:39:27.853659+0000","last_fullsized":"2024-04-25T08:39:27.853659+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:46.244387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853657+0000","last_change":"2024-04-25T08:39:26.554227+0000","last_active":"2024-04-25T08:39:27.853657+0000","last_peered":"2024-04-25T08:39:27.853657+0000","last_clean":"2024-04-25T08:39:27.853657+0000","last_became_active":"2024-04-25T08:39:26.553903+0000","last_became_peered":"2024-04-25T08:39:26.553903+0000","last_unstale":"2024-04-25T08:39:27.853657+0000","last_undegraded":"2024-04-25T08:39:27.853657+0000","last_fullsized":"2024-04-25T08:39:27.853657+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:43.630525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757668+0000","last_change":"2024-04-25T08:39:26.565344+0000","last_active":"2024-04-25T08:39:28.757668+0000","last_peered":"2024-04-25T08:39:28.757668+0000","last_clean":"2024-04-25T08:39:28.757668+0000","last_became_active":"2024-04-25T08:39:26.564951+0000","last_became_peered":"2024-04-25T08:39:26.564951+0000","last_unstale":"2024-04-25T08:39:28.757668+0000","last_undegraded":"2024-04-25T08:39:28.757668+0000","last_fullsized":"2024-04-25T08:39:28.757668+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:26.244665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559130+0000","last_change":"2024-04-25T08:39:26.556976+0000","last_active":"2024-04-25T08:39:27.559130+0000","last_peered":"2024-04-25T08:39:27.559130+0000","last_clean":"2024-04-25T08:39:27.559130+0000","last_became_active":"2024-04-25T08:39:26.556686+0000","last_became_peered":"2024-04-25T08:39:26.556686+0000","last_unstale":"2024-04-25T08:39:27.559130+0000","last_undegraded":"2024-04-25T08:39:27.559130+0000","last_fullsized":"2024-04-25T08:39:27.559130+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:40.157254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536961+0000","last_change":"2024-04-25T08:39:26.560425+0000","last_active":"2024-04-25T08:39:27.536961+0000","last_peered":"2024-04-25T08:39:27.536961+0000","last_clean":"2024-04-25T08:39:27.536961+0000","last_became_active":"2024-04-25T08:39:26.559801+0000","last_became_peered":"2024-04-25T08:39:26.559801+0000","last_unstale":"2024-04-25T08:39:27.536961+0000","last_undegraded":"2024-04-25T08:39:27.536961+0000","last_fullsized":"2024-04-25T08:39:27.536961+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:53.420233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559233+0000","last_change":"2024-04-25T08:39:26.553341+0000","last_active":"2024-04-25T08:39:27.559233+0000","last_peered":"2024-04-25T08:39:27.559233+0000","last_clean":"2024-04-25T08:39:27.559233+0000","last_became_active":"2024-04-25T08:39:26.552074+0000","last_became_peered":"2024-04-25T08:39:26.552074+0000","last_unstale":"2024-04-25T08:39:27.559233+0000","last_undegraded":"2024-04-25T08:39:27.559233+0000","last_fullsized":"2024-04-25T08:39:27.559233+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:25.785848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757971+0000","last_change":"2024-04-25T08:39:26.564074+0000","last_active":"2024-04-25T08:39:28.757971+0000","last_peered":"2024-04-25T08:39:28.757971+0000","last_clean":"2024-04-25T08:39:28.757971+0000","last_became_active":"2024-04-25T08:39:26.563763+0000","last_became_peered":"2024-04-25T08:39:26.563763+0000","last_unstale":"2024-04-25T08:39:28.757971+0000","last_undegraded":"2024-04-25T08:39:28.757971+0000","last_fullsized":"2024-04-25T08:39:28.757971+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:15.957839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535700+0000","last_change":"2024-04-25T08:39:26.545998+0000","last_active":"2024-04-25T08:39:27.535700+0000","last_peered":"2024-04-25T08:39:27.535700+0000","last_clean":"2024-04-25T08:39:27.535700+0000","last_became_active":"2024-04-25T08:39:26.545718+0000","last_became_peered":"2024-04-25T08:39:26.545718+0000","last_unstale":"2024-04-25T08:39:27.535700+0000","last_undegraded":"2024-04-25T08:39:27.535700+0000","last_fullsized":"2024-04-25T08:39:27.535700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:53.422818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534323+0000","last_change":"2024-04-25T08:39:26.549381+0000","last_active":"2024-04-25T08:39:27.534323+0000","last_peered":"2024-04-25T08:39:27.534323+0000","last_clean":"2024-04-25T08:39:27.534323+0000","last_became_active":"2024-04-25T08:39:26.549032+0000","last_became_peered":"2024-04-25T08:39:26.549032+0000","last_unstale":"2024-04-25T08:39:27.534323+0000","last_undegraded":"2024-04-25T08:39:27.534323+0000","last_fullsized":"2024-04-25T08:39:27.534323+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:26.828290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"144'4","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.758169+0000","last_change":"2024-04-25T08:39:25.543530+0000","last_active":"2024-04-25T08:39:28.758169+0000","last_peered":"2024-04-25T08:39:28.758169+0000","last_clean":"2024-04-25T08:39:28.758169+0000","last_became_active":"2024-04-25T08:39:25.541304+0000","last_became_peered":"2024-04-25T08:39:25.541304+0000","last_unstale":"2024-04-25T08:39:28.758169+0000","last_undegraded":"2024-04-25T08:39:28.758169+0000","last_fullsized":"2024-04-25T08:39:28.758169+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:44.005644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534524+0000","last_change":"2024-04-25T08:39:26.542527+0000","last_active":"2024-04-25T08:39:27.534524+0000","last_peered":"2024-04-25T08:39:27.534524+0000","last_clean":"2024-04-25T08:39:27.534524+0000","last_became_active":"2024-04-25T08:39:26.540986+0000","last_became_peered":"2024-04-25T08:39:26.540986+0000","last_unstale":"2024-04-25T08:39:27.534524+0000","last_undegraded":"2024-04-25T08:39:27.534524+0000","last_fullsized":"2024-04-25T08:39:27.534524+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:48.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539067+0000","last_change":"2024-04-25T08:39:25.541869+0000","last_active":"2024-04-25T08:39:28.539067+0000","last_peered":"2024-04-25T08:39:28.539067+0000","last_clean":"2024-04-25T08:39:28.539067+0000","last_became_active":"2024-04-25T08:39:25.541531+0000","last_became_peered":"2024-04-25T08:39:25.541531+0000","last_unstale":"2024-04-25T08:39:28.539067+0000","last_undegraded":"2024-04-25T08:39:28.539067+0000","last_fullsized":"2024-04-25T08:39:28.539067+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:45.240208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536160+0000","last_change":"2024-04-25T08:39:26.539077+0000","last_active":"2024-04-25T08:39:27.536160+0000","last_peered":"2024-04-25T08:39:27.536160+0000","last_clean":"2024-04-25T08:39:27.536160+0000","last_became_active":"2024-04-25T08:39:26.538669+0000","last_became_peered":"2024-04-25T08:39:26.538669+0000","last_unstale":"2024-04-25T08:39:27.536160+0000","last_undegraded":"2024-04-25T08:39:27.536160+0000","last_fullsized":"2024-04-25T08:39:27.536160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:20.394454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539350+0000","last_change":"2024-04-25T08:39:25.540429+0000","last_active":"2024-04-25T08:39:28.539350+0000","last_peered":"2024-04-25T08:39:28.539350+0000","last_clean":"2024-04-25T08:39:28.539350+0000","last_became_active":"2024-04-25T08:39:25.539563+0000","last_became_peered":"2024-04-25T08:39:25.539563+0000","last_unstale":"2024-04-25T08:39:28.539350+0000","last_undegraded":"2024-04-25T08:39:28.539350+0000","last_fullsized":"2024-04-25T08:39:28.539350+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:42.626346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539257+0000","last_change":"2024-04-25T08:39:26.551103+0000","last_active":"2024-04-25T08:39:28.539257+0000","last_peered":"2024-04-25T08:39:28.539257+0000","last_clean":"2024-04-25T08:39:28.539257+0000","last_became_active":"2024-04-25T08:39:26.550819+0000","last_became_peered":"2024-04-25T08:39:26.550819+0000","last_unstale":"2024-04-25T08:39:28.539257+0000","last_undegraded":"2024-04-25T08:39:28.539257+0000","last_fullsized":"2024-04-25T08:39:28.539257+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:07.090120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534318+0000","last_change":"2024-04-25T08:39:25.536731+0000","last_active":"2024-04-25T08:39:27.534318+0000","last_peered":"2024-04-25T08:39:27.534318+0000","last_clean":"2024-04-25T08:39:27.534318+0000","last_became_active":"2024-04-25T08:39:25.536385+0000","last_became_peered":"2024-04-25T08:39:25.536385+0000","last_unstale":"2024-04-25T08:39:27.534318+0000","last_undegraded":"2024-04-25T08:39:27.534318+0000","last_fullsized":"2024-04-25T08:39:27.534318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:55.622854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757125+0000","last_change":"2024-04-25T08:39:26.545690+0000","last_active":"2024-04-25T08:39:28.757125+0000","last_peered":"2024-04-25T08:39:28.757125+0000","last_clean":"2024-04-25T08:39:28.757125+0000","last_became_active":"2024-04-25T08:39:26.545388+0000","last_became_peered":"2024-04-25T08:39:26.545388+0000","last_unstale":"2024-04-25T08:39:28.757125+0000","last_undegraded":"2024-04-25T08:39:28.757125+0000","last_fullsized":"2024-04-25T08:39:28.757125+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:40.496471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536905+0000","last_change":"2024-04-25T08:39:25.537798+0000","last_active":"2024-04-25T08:39:27.536905+0000","last_peered":"2024-04-25T08:39:27.536905+0000","last_clean":"2024-04-25T08:39:27.536905+0000","last_became_active":"2024-04-25T08:39:25.536841+0000","last_became_peered":"2024-04-25T08:39:25.536841+0000","last_unstale":"2024-04-25T08:39:27.536905+0000","last_undegraded":"2024-04-25T08:39:27.536905+0000","last_fullsized":"2024-04-25T08:39:27.536905+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:11.791198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535531+0000","last_change":"2024-04-25T08:39:26.553435+0000","last_active":"2024-04-25T08:39:27.535531+0000","last_peered":"2024-04-25T08:39:27.535531+0000","last_clean":"2024-04-25T08:39:27.535531+0000","last_became_active":"2024-04-25T08:39:26.552951+0000","last_became_peered":"2024-04-25T08:39:26.552951+0000","last_unstale":"2024-04-25T08:39:27.535531+0000","last_undegraded":"2024-04-25T08:39:27.535531+0000","last_fullsized":"2024-04-25T08:39:27.535531+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:51.069202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757407+0000","last_change":"2024-04-25T08:39:25.543306+0000","last_active":"2024-04-25T08:39:28.757407+0000","last_peered":"2024-04-25T08:39:28.757407+0000","last_clean":"2024-04-25T08:39:28.757407+0000","last_became_active":"2024-04-25T08:39:25.542975+0000","last_became_peered":"2024-04-25T08:39:25.542975+0000","last_unstale":"2024-04-25T08:39:28.757407+0000","last_undegraded":"2024-04-25T08:39:28.757407+0000","last_fullsized":"2024-04-25T08:39:28.757407+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:25.490860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534357+0000","last_change":"2024-04-25T08:39:26.541992+0000","last_active":"2024-04-25T08:39:27.534357+0000","last_peered":"2024-04-25T08:39:27.534357+0000","last_clean":"2024-04-25T08:39:27.534357+0000","last_became_active":"2024-04-25T08:39:26.541680+0000","last_became_peered":"2024-04-25T08:39:26.541680+0000","last_unstale":"2024-04-25T08:39:27.534357+0000","last_undegraded":"2024-04-25T08:39:27.534357+0000","last_fullsized":"2024-04-25T08:39:27.534357+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:25.601013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853289+0000","last_change":"2024-04-25T08:39:25.537615+0000","last_active":"2024-04-25T08:39:27.853289+0000","last_peered":"2024-04-25T08:39:27.853289+0000","last_clean":"2024-04-25T08:39:27.853289+0000","last_became_active":"2024-04-25T08:39:25.537345+0000","last_became_peered":"2024-04-25T08:39:25.537345+0000","last_unstale":"2024-04-25T08:39:27.853289+0000","last_undegraded":"2024-04-25T08:39:27.853289+0000","last_fullsized":"2024-04-25T08:39:27.853289+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:40.816894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853452+0000","last_change":"2024-04-25T08:39:26.543693+0000","last_active":"2024-04-25T08:39:27.853452+0000","last_peered":"2024-04-25T08:39:27.853452+0000","last_clean":"2024-04-25T08:39:27.853452+0000","last_became_active":"2024-04-25T08:39:26.543439+0000","last_became_peered":"2024-04-25T08:39:26.543439+0000","last_unstale":"2024-04-25T08:39:27.853452+0000","last_undegraded":"2024-04-25T08:39:27.853452+0000","last_fullsized":"2024-04-25T08:39:27.853452+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:39.420771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757155+0000","last_change":"2024-04-25T08:39:25.542707+0000","last_active":"2024-04-25T08:39:28.757155+0000","last_peered":"2024-04-25T08:39:28.757155+0000","last_clean":"2024-04-25T08:39:28.757155+0000","last_became_active":"2024-04-25T08:39:25.542406+0000","last_became_peered":"2024-04-25T08:39:25.542406+0000","last_unstale":"2024-04-25T08:39:28.757155+0000","last_undegraded":"2024-04-25T08:39:28.757155+0000","last_fullsized":"2024-04-25T08:39:28.757155+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:55.622854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536768+0000","last_change":"2024-04-25T08:39:26.544746+0000","last_active":"2024-04-25T08:39:27.536768+0000","last_peered":"2024-04-25T08:39:27.536768+0000","last_clean":"2024-04-25T08:39:27.536768+0000","last_became_active":"2024-04-25T08:39:26.544423+0000","last_became_peered":"2024-04-25T08:39:26.544423+0000","last_unstale":"2024-04-25T08:39:27.536768+0000","last_undegraded":"2024-04-25T08:39:27.536768+0000","last_fullsized":"2024-04-25T08:39:27.536768+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:26.495039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539231+0000","last_change":"2024-04-25T08:39:25.541562+0000","last_active":"2024-04-25T08:39:28.539231+0000","last_peered":"2024-04-25T08:39:28.539231+0000","last_clean":"2024-04-25T08:39:28.539231+0000","last_became_active":"2024-04-25T08:39:25.541283+0000","last_became_peered":"2024-04-25T08:39:25.541283+0000","last_unstale":"2024-04-25T08:39:28.539231+0000","last_undegraded":"2024-04-25T08:39:28.539231+0000","last_fullsized":"2024-04-25T08:39:28.539231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:52.828151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559111+0000","last_change":"2024-04-25T08:39:26.548069+0000","last_active":"2024-04-25T08:39:27.559111+0000","last_peered":"2024-04-25T08:39:27.559111+0000","last_clean":"2024-04-25T08:39:27.559111+0000","last_became_active":"2024-04-25T08:39:26.547801+0000","last_became_peered":"2024-04-25T08:39:26.547801+0000","last_unstale":"2024-04-25T08:39:27.559111+0000","last_undegraded":"2024-04-25T08:39:27.559111+0000","last_fullsized":"2024-04-25T08:39:27.559111+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:31.497078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853799+0000","last_change":"2024-04-25T08:39:25.540300+0000","last_active":"2024-04-25T08:39:27.853799+0000","last_peered":"2024-04-25T08:39:27.853799+0000","last_clean":"2024-04-25T08:39:27.853799+0000","last_became_active":"2024-04-25T08:39:25.539840+0000","last_became_peered":"2024-04-25T08:39:25.539840+0000","last_unstale":"2024-04-25T08:39:27.853799+0000","last_undegraded":"2024-04-25T08:39:27.853799+0000","last_fullsized":"2024-04-25T08:39:27.853799+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:55.622854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559372+0000","last_change":"2024-04-25T08:39:26.552891+0000","last_active":"2024-04-25T08:39:27.559372+0000","last_peered":"2024-04-25T08:39:27.559372+0000","last_clean":"2024-04-25T08:39:27.559372+0000","last_became_active":"2024-04-25T08:39:26.552104+0000","last_became_peered":"2024-04-25T08:39:26.552104+0000","last_unstale":"2024-04-25T08:39:27.559372+0000","last_undegraded":"2024-04-25T08:39:27.559372+0000","last_fullsized":"2024-04-25T08:39:27.559372+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:45.559291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"144'3","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559168+0000","last_change":"2024-04-25T08:39:25.534628+0000","last_active":"2024-04-25T08:39:27.559168+0000","last_peered":"2024-04-25T08:39:27.559168+0000","last_clean":"2024-04-25T08:39:27.559168+0000","last_became_active":"2024-04-25T08:39:25.534334+0000","last_became_peered":"2024-04-25T08:39:25.534334+0000","last_unstale":"2024-04-25T08:39:27.559168+0000","last_undegraded":"2024-04-25T08:39:27.559168+0000","last_fullsized":"2024-04-25T08:39:27.559168+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:06.500619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535834+0000","last_change":"2024-04-25T08:39:26.544207+0000","last_active":"2024-04-25T08:39:27.535834+0000","last_peered":"2024-04-25T08:39:27.535834+0000","last_clean":"2024-04-25T08:39:27.535834+0000","last_became_active":"2024-04-25T08:39:26.543896+0000","last_became_peered":"2024-04-25T08:39:26.543896+0000","last_unstale":"2024-04-25T08:39:27.535834+0000","last_undegraded":"2024-04-25T08:39:27.535834+0000","last_fullsized":"2024-04-25T08:39:27.535834+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:41.821073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757966+0000","last_change":"2024-04-25T08:39:25.541301+0000","last_active":"2024-04-25T08:39:28.757966+0000","last_peered":"2024-04-25T08:39:28.757966+0000","last_clean":"2024-04-25T08:39:28.757966+0000","last_became_active":"2024-04-25T08:39:25.540985+0000","last_became_peered":"2024-04-25T08:39:25.540985+0000","last_unstale":"2024-04-25T08:39:28.757966+0000","last_undegraded":"2024-04-25T08:39:28.757966+0000","last_fullsized":"2024-04-25T08:39:28.757966+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.416054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534409+0000","last_change":"2024-04-25T08:39:26.542211+0000","last_active":"2024-04-25T08:39:27.534409+0000","last_peered":"2024-04-25T08:39:27.534409+0000","last_clean":"2024-04-25T08:39:27.534409+0000","last_became_active":"2024-04-25T08:39:26.541620+0000","last_became_peered":"2024-04-25T08:39:26.541620+0000","last_unstale":"2024-04-25T08:39:27.534409+0000","last_undegraded":"2024-04-25T08:39:27.534409+0000","last_fullsized":"2024-04-25T08:39:27.534409+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:48.515895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536127+0000","last_change":"2024-04-25T08:39:25.531155+0000","last_active":"2024-04-25T08:39:27.536127+0000","last_peered":"2024-04-25T08:39:27.536127+0000","last_clean":"2024-04-25T08:39:27.536127+0000","last_became_active":"2024-04-25T08:39:25.530883+0000","last_became_peered":"2024-04-25T08:39:25.530883+0000","last_unstale":"2024-04-25T08:39:27.536127+0000","last_undegraded":"2024-04-25T08:39:27.536127+0000","last_fullsized":"2024-04-25T08:39:27.536127+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:15.645600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757335+0000","last_change":"2024-04-25T08:39:26.565355+0000","last_active":"2024-04-25T08:39:28.757335+0000","last_peered":"2024-04-25T08:39:28.757335+0000","last_clean":"2024-04-25T08:39:28.757335+0000","last_became_active":"2024-04-25T08:39:26.565004+0000","last_became_peered":"2024-04-25T08:39:26.565004+0000","last_unstale":"2024-04-25T08:39:28.757335+0000","last_undegraded":"2024-04-25T08:39:28.757335+0000","last_fullsized":"2024-04-25T08:39:28.757335+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:26.828290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535826+0000","last_change":"2024-04-25T08:39:25.542778+0000","last_active":"2024-04-25T08:39:27.535826+0000","last_peered":"2024-04-25T08:39:27.535826+0000","last_clean":"2024-04-25T08:39:27.535826+0000","last_became_active":"2024-04-25T08:39:25.542474+0000","last_became_peered":"2024-04-25T08:39:25.542474+0000","last_unstale":"2024-04-25T08:39:27.535826+0000","last_undegraded":"2024-04-25T08:39:27.535826+0000","last_fullsized":"2024-04-25T08:39:27.535826+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:28.511320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534501+0000","last_change":"2024-04-25T08:39:26.560555+0000","last_active":"2024-04-25T08:39:27.534501+0000","last_peered":"2024-04-25T08:39:27.534501+0000","last_clean":"2024-04-25T08:39:27.534501+0000","last_became_active":"2024-04-25T08:39:26.560232+0000","last_became_peered":"2024-04-25T08:39:26.560232+0000","last_unstale":"2024-04-25T08:39:27.534501+0000","last_undegraded":"2024-04-25T08:39:27.534501+0000","last_fullsized":"2024-04-25T08:39:27.534501+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:48.052559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"144'8","reported_seq":22,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853151+0000","last_change":"2024-04-25T08:39:25.540523+0000","last_active":"2024-04-25T08:39:27.853151+0000","last_peered":"2024-04-25T08:39:27.853151+0000","last_clean":"2024-04-25T08:39:27.853151+0000","last_became_active":"2024-04-25T08:39:25.540179+0000","last_became_peered":"2024-04-25T08:39:25.540179+0000","last_unstale":"2024-04-25T08:39:27.853151+0000","last_undegraded":"2024-04-25T08:39:27.853151+0000","last_fullsized":"2024-04-25T08:39:27.853151+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:52.418639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539031+0000","last_change":"2024-04-25T08:39:26.562106+0000","last_active":"2024-04-25T08:39:28.539031+0000","last_peered":"2024-04-25T08:39:28.539031+0000","last_clean":"2024-04-25T08:39:28.539031+0000","last_became_active":"2024-04-25T08:39:26.561792+0000","last_became_peered":"2024-04-25T08:39:26.561792+0000","last_unstale":"2024-04-25T08:39:28.539031+0000","last_undegraded":"2024-04-25T08:39:28.539031+0000","last_fullsized":"2024-04-25T08:39:28.539031+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:52.723530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559016+0000","last_change":"2024-04-25T08:39:25.534674+0000","last_active":"2024-04-25T08:39:27.559016+0000","last_peered":"2024-04-25T08:39:27.559016+0000","last_clean":"2024-04-25T08:39:27.559016+0000","last_became_active":"2024-04-25T08:39:25.534386+0000","last_became_peered":"2024-04-25T08:39:25.534386+0000","last_unstale":"2024-04-25T08:39:27.559016+0000","last_undegraded":"2024-04-25T08:39:27.559016+0000","last_fullsized":"2024-04-25T08:39:27.559016+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:58.249712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539258+0000","last_change":"2024-04-25T08:39:26.556890+0000","last_active":"2024-04-25T08:39:28.539258+0000","last_peered":"2024-04-25T08:39:28.539258+0000","last_clean":"2024-04-25T08:39:28.539258+0000","last_became_active":"2024-04-25T08:39:26.555998+0000","last_became_peered":"2024-04-25T08:39:26.555998+0000","last_unstale":"2024-04-25T08:39:28.539258+0000","last_undegraded":"2024-04-25T08:39:28.539258+0000","last_fullsized":"2024-04-25T08:39:28.539258+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:47.600446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539133+0000","last_change":"2024-04-25T08:39:25.544537+0000","last_active":"2024-04-25T08:39:28.539133+0000","last_peered":"2024-04-25T08:39:28.539133+0000","last_clean":"2024-04-25T08:39:28.539133+0000","last_became_active":"2024-04-25T08:39:25.544239+0000","last_became_peered":"2024-04-25T08:39:25.544239+0000","last_unstale":"2024-04-25T08:39:28.539133+0000","last_undegraded":"2024-04-25T08:39:28.539133+0000","last_fullsized":"2024-04-25T08:39:28.539133+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:58.351603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853688+0000","last_change":"2024-04-25T08:39:26.560686+0000","last_active":"2024-04-25T08:39:27.853688+0000","last_peered":"2024-04-25T08:39:27.853688+0000","last_clean":"2024-04-25T08:39:27.853688+0000","last_became_active":"2024-04-25T08:39:26.560432+0000","last_became_peered":"2024-04-25T08:39:26.560432+0000","last_unstale":"2024-04-25T08:39:27.853688+0000","last_undegraded":"2024-04-25T08:39:27.853688+0000","last_fullsized":"2024-04-25T08:39:27.853688+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:59.355782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"144'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534464+0000","last_change":"2024-04-25T08:39:25.544139+0000","last_active":"2024-04-25T08:39:27.534464+0000","last_peered":"2024-04-25T08:39:27.534464+0000","last_clean":"2024-04-25T08:39:27.534464+0000","last_became_active":"2024-04-25T08:39:25.543864+0000","last_became_peered":"2024-04-25T08:39:25.543864+0000","last_unstale":"2024-04-25T08:39:27.534464+0000","last_undegraded":"2024-04-25T08:39:27.534464+0000","last_fullsized":"2024-04-25T08:39:27.534464+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:25.490860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853749+0000","last_change":"2024-04-25T08:39:26.555880+0000","last_active":"2024-04-25T08:39:27.853749+0000","last_peered":"2024-04-25T08:39:27.853749+0000","last_clean":"2024-04-25T08:39:27.853749+0000","last_became_active":"2024-04-25T08:39:26.555527+0000","last_became_peered":"2024-04-25T08:39:26.555527+0000","last_unstale":"2024-04-25T08:39:27.853749+0000","last_undegraded":"2024-04-25T08:39:27.853749+0000","last_fullsized":"2024-04-25T08:39:27.853749+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:53.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535637+0000","last_change":"2024-04-25T08:39:25.534723+0000","last_active":"2024-04-25T08:39:27.535637+0000","last_peered":"2024-04-25T08:39:27.535637+0000","last_clean":"2024-04-25T08:39:27.535637+0000","last_became_active":"2024-04-25T08:39:25.534462+0000","last_became_peered":"2024-04-25T08:39:25.534462+0000","last_unstale":"2024-04-25T08:39:27.535637+0000","last_undegraded":"2024-04-25T08:39:27.535637+0000","last_fullsized":"2024-04-25T08:39:27.535637+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:40.816894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.538867+0000","last_change":"2024-04-25T08:39:26.552017+0000","last_active":"2024-04-25T08:39:28.538867+0000","last_peered":"2024-04-25T08:39:28.538867+0000","last_clean":"2024-04-25T08:39:28.538867+0000","last_became_active":"2024-04-25T08:39:26.551743+0000","last_became_peered":"2024-04-25T08:39:26.551743+0000","last_unstale":"2024-04-25T08:39:28.538867+0000","last_undegraded":"2024-04-25T08:39:28.538867+0000","last_fullsized":"2024-04-25T08:39:28.538867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:21.724428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757231+0000","last_change":"2024-04-25T08:39:25.530825+0000","last_active":"2024-04-25T08:39:28.757231+0000","last_peered":"2024-04-25T08:39:28.757231+0000","last_clean":"2024-04-25T08:39:28.757231+0000","last_became_active":"2024-04-25T08:39:25.530455+0000","last_became_peered":"2024-04-25T08:39:25.530455+0000","last_unstale":"2024-04-25T08:39:28.757231+0000","last_undegraded":"2024-04-25T08:39:28.757231+0000","last_fullsized":"2024-04-25T08:39:28.757231+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:13.940588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534405+0000","last_change":"2024-04-25T08:39:26.551194+0000","last_active":"2024-04-25T08:39:27.534405+0000","last_peered":"2024-04-25T08:39:27.534405+0000","last_clean":"2024-04-25T08:39:27.534405+0000","last_became_active":"2024-04-25T08:39:26.549862+0000","last_became_peered":"2024-04-25T08:39:26.549862+0000","last_unstale":"2024-04-25T08:39:27.534405+0000","last_undegraded":"2024-04-25T08:39:27.534405+0000","last_fullsized":"2024-04-25T08:39:27.534405+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:26.244665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536122+0000","last_change":"2024-04-25T08:39:25.539082+0000","last_active":"2024-04-25T08:39:27.536122+0000","last_peered":"2024-04-25T08:39:27.536122+0000","last_clean":"2024-04-25T08:39:27.536122+0000","last_became_active":"2024-04-25T08:39:25.538736+0000","last_became_peered":"2024-04-25T08:39:25.538736+0000","last_unstale":"2024-04-25T08:39:27.536122+0000","last_undegraded":"2024-04-25T08:39:27.536122+0000","last_fullsized":"2024-04-25T08:39:27.536122+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:12.864908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.538889+0000","last_change":"2024-04-25T08:39:26.547365+0000","last_active":"2024-04-25T08:39:28.538889+0000","last_peered":"2024-04-25T08:39:28.538889+0000","last_clean":"2024-04-25T08:39:28.538889+0000","last_became_active":"2024-04-25T08:39:26.547067+0000","last_became_peered":"2024-04-25T08:39:26.547067+0000","last_unstale":"2024-04-25T08:39:28.538889+0000","last_undegraded":"2024-04-25T08:39:28.538889+0000","last_fullsized":"2024-04-25T08:39:28.538889+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:15.957839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757087+0000","last_change":"2024-04-25T08:39:25.528679+0000","last_active":"2024-04-25T08:39:28.757087+0000","last_peered":"2024-04-25T08:39:28.757087+0000","last_clean":"2024-04-25T08:39:28.757087+0000","last_became_active":"2024-04-25T08:39:25.528354+0000","last_became_peered":"2024-04-25T08:39:25.528354+0000","last_unstale":"2024-04-25T08:39:28.757087+0000","last_undegraded":"2024-04-25T08:39:28.757087+0000","last_fullsized":"2024-04-25T08:39:28.757087+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:36.036034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534320+0000","last_change":"2024-04-25T08:39:26.546237+0000","last_active":"2024-04-25T08:39:27.534320+0000","last_peered":"2024-04-25T08:39:27.534320+0000","last_clean":"2024-04-25T08:39:27.534320+0000","last_became_active":"2024-04-25T08:39:26.545854+0000","last_became_peered":"2024-04-25T08:39:26.545854+0000","last_unstale":"2024-04-25T08:39:27.534320+0000","last_undegraded":"2024-04-25T08:39:27.534320+0000","last_fullsized":"2024-04-25T08:39:27.534320+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:53.832329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536037+0000","last_change":"2024-04-25T08:39:25.531069+0000","last_active":"2024-04-25T08:39:27.536037+0000","last_peered":"2024-04-25T08:39:27.536037+0000","last_clean":"2024-04-25T08:39:27.536037+0000","last_became_active":"2024-04-25T08:39:25.528111+0000","last_became_peered":"2024-04-25T08:39:25.528111+0000","last_unstale":"2024-04-25T08:39:27.536037+0000","last_undegraded":"2024-04-25T08:39:27.536037+0000","last_fullsized":"2024-04-25T08:39:27.536037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:41.755865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757763+0000","last_change":"2024-04-25T08:39:26.558666+0000","last_active":"2024-04-25T08:39:28.757763+0000","last_peered":"2024-04-25T08:39:28.757763+0000","last_clean":"2024-04-25T08:39:28.757763+0000","last_became_active":"2024-04-25T08:39:26.558170+0000","last_became_peered":"2024-04-25T08:39:26.558170+0000","last_unstale":"2024-04-25T08:39:28.757763+0000","last_undegraded":"2024-04-25T08:39:28.757763+0000","last_fullsized":"2024-04-25T08:39:28.757763+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:48.052559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"144'2","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536374+0000","last_change":"2024-04-25T08:39:25.533867+0000","last_active":"2024-04-25T08:39:27.536374+0000","last_peered":"2024-04-25T08:39:27.536374+0000","last_clean":"2024-04-25T08:39:27.536374+0000","last_became_active":"2024-04-25T08:39:25.533567+0000","last_became_peered":"2024-04-25T08:39:25.533567+0000","last_unstale":"2024-04-25T08:39:27.536374+0000","last_undegraded":"2024-04-25T08:39:27.536374+0000","last_fullsized":"2024-04-25T08:39:27.536374+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:52.418639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535910+0000","last_change":"2024-04-25T08:39:26.559439+0000","last_active":"2024-04-25T08:39:27.535910+0000","last_peered":"2024-04-25T08:39:27.535910+0000","last_clean":"2024-04-25T08:39:27.535910+0000","last_became_active":"2024-04-25T08:39:26.558833+0000","last_became_peered":"2024-04-25T08:39:26.558833+0000","last_unstale":"2024-04-25T08:39:27.535910+0000","last_undegraded":"2024-04-25T08:39:27.535910+0000","last_fullsized":"2024-04-25T08:39:27.535910+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:53.294816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"145'1","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853896+0000","last_change":"2024-04-25T08:39:25.540746+0000","last_active":"2024-04-25T08:39:27.853896+0000","last_peered":"2024-04-25T08:39:27.853896+0000","last_clean":"2024-04-25T08:39:27.853896+0000","last_became_active":"2024-04-25T08:39:25.540230+0000","last_became_peered":"2024-04-25T08:39:25.540230+0000","last_unstale":"2024-04-25T08:39:27.853896+0000","last_undegraded":"2024-04-25T08:39:27.853896+0000","last_fullsized":"2024-04-25T08:39:27.853896+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:25.490860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853818+0000","last_change":"2024-04-25T08:39:26.557011+0000","last_active":"2024-04-25T08:39:27.853818+0000","last_peered":"2024-04-25T08:39:27.853818+0000","last_clean":"2024-04-25T08:39:27.853818+0000","last_became_active":"2024-04-25T08:39:26.556654+0000","last_became_peered":"2024-04-25T08:39:26.556654+0000","last_unstale":"2024-04-25T08:39:27.853818+0000","last_undegraded":"2024-04-25T08:39:27.853818+0000","last_fullsized":"2024-04-25T08:39:27.853818+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:26.828290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534529+0000","last_change":"2024-04-25T08:39:25.541989+0000","last_active":"2024-04-25T08:39:27.534529+0000","last_peered":"2024-04-25T08:39:27.534529+0000","last_clean":"2024-04-25T08:39:27.534529+0000","last_became_active":"2024-04-25T08:39:25.541700+0000","last_became_peered":"2024-04-25T08:39:25.541700+0000","last_unstale":"2024-04-25T08:39:27.534529+0000","last_undegraded":"2024-04-25T08:39:27.534529+0000","last_fullsized":"2024-04-25T08:39:27.534529+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:44.005644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.758183+0000","last_change":"2024-04-25T08:39:26.558984+0000","last_active":"2024-04-25T08:39:28.758183+0000","last_peered":"2024-04-25T08:39:28.758183+0000","last_clean":"2024-04-25T08:39:28.758183+0000","last_became_active":"2024-04-25T08:39:26.558633+0000","last_became_peered":"2024-04-25T08:39:26.558633+0000","last_unstale":"2024-04-25T08:39:28.758183+0000","last_undegraded":"2024-04-25T08:39:28.758183+0000","last_fullsized":"2024-04-25T08:39:28.758183+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:38.027963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757818+0000","last_change":"2024-04-25T08:39:25.545129+0000","last_active":"2024-04-25T08:39:28.757818+0000","last_peered":"2024-04-25T08:39:28.757818+0000","last_clean":"2024-04-25T08:39:28.757818+0000","last_became_active":"2024-04-25T08:39:25.544818+0000","last_became_peered":"2024-04-25T08:39:25.544818+0000","last_unstale":"2024-04-25T08:39:28.757818+0000","last_undegraded":"2024-04-25T08:39:28.757818+0000","last_fullsized":"2024-04-25T08:39:28.757818+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:32.365375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535977+0000","last_change":"2024-04-25T08:39:26.545782+0000","last_active":"2024-04-25T08:39:27.535977+0000","last_peered":"2024-04-25T08:39:27.535977+0000","last_clean":"2024-04-25T08:39:27.535977+0000","last_became_active":"2024-04-25T08:39:26.545352+0000","last_became_peered":"2024-04-25T08:39:26.545352+0000","last_unstale":"2024-04-25T08:39:27.535977+0000","last_undegraded":"2024-04-25T08:39:27.535977+0000","last_fullsized":"2024-04-25T08:39:27.535977+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:16.649779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853393+0000","last_change":"2024-04-25T08:39:25.539417+0000","last_active":"2024-04-25T08:39:27.853393+0000","last_peered":"2024-04-25T08:39:27.853393+0000","last_clean":"2024-04-25T08:39:27.853393+0000","last_became_active":"2024-04-25T08:39:25.539015+0000","last_became_peered":"2024-04-25T08:39:25.539015+0000","last_unstale":"2024-04-25T08:39:27.853393+0000","last_undegraded":"2024-04-25T08:39:27.853393+0000","last_fullsized":"2024-04-25T08:39:27.853393+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:28.511320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.535699+0000","last_change":"2024-04-25T08:39:26.553666+0000","last_active":"2024-04-25T08:39:27.535699+0000","last_peered":"2024-04-25T08:39:27.535699+0000","last_clean":"2024-04-25T08:39:27.535699+0000","last_became_active":"2024-04-25T08:39:26.553397+0000","last_became_peered":"2024-04-25T08:39:26.553397+0000","last_unstale":"2024-04-25T08:39:27.535699+0000","last_undegraded":"2024-04-25T08:39:27.535699+0000","last_fullsized":"2024-04-25T08:39:27.535699+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:11.420359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.559242+0000","last_change":"2024-04-25T08:39:25.537465+0000","last_active":"2024-04-25T08:39:27.559242+0000","last_peered":"2024-04-25T08:39:27.559242+0000","last_clean":"2024-04-25T08:39:27.559242+0000","last_became_active":"2024-04-25T08:39:25.537097+0000","last_became_peered":"2024-04-25T08:39:25.537097+0000","last_unstale":"2024-04-25T08:39:27.559242+0000","last_undegraded":"2024-04-25T08:39:27.559242+0000","last_fullsized":"2024-04-25T08:39:27.559242+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:11.131598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.534278+0000","last_change":"2024-04-25T08:39:26.549276+0000","last_active":"2024-04-25T08:39:27.534278+0000","last_peered":"2024-04-25T08:39:27.534278+0000","last_clean":"2024-04-25T08:39:27.534278+0000","last_became_active":"2024-04-25T08:39:26.548936+0000","last_became_peered":"2024-04-25T08:39:26.548936+0000","last_unstale":"2024-04-25T08:39:27.534278+0000","last_undegraded":"2024-04-25T08:39:27.534278+0000","last_fullsized":"2024-04-25T08:39:27.534278+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:43.630525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.853430+0000","last_change":"2024-04-25T08:39:25.543519+0000","last_active":"2024-04-25T08:39:27.853430+0000","last_peered":"2024-04-25T08:39:27.853430+0000","last_clean":"2024-04-25T08:39:27.853430+0000","last_became_active":"2024-04-25T08:39:25.543148+0000","last_became_peered":"2024-04-25T08:39:25.543148+0000","last_unstale":"2024-04-25T08:39:27.853430+0000","last_undegraded":"2024-04-25T08:39:27.853430+0000","last_fullsized":"2024-04-25T08:39:27.853430+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:15.645600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536864+0000","last_change":"2024-04-25T08:39:26.560447+0000","last_active":"2024-04-25T08:39:27.536864+0000","last_peered":"2024-04-25T08:39:27.536864+0000","last_clean":"2024-04-25T08:39:27.536864+0000","last_became_active":"2024-04-25T08:39:26.560160+0000","last_became_peered":"2024-04-25T08:39:26.560160+0000","last_unstale":"2024-04-25T08:39:27.536864+0000","last_undegraded":"2024-04-25T08:39:27.536864+0000","last_fullsized":"2024-04-25T08:39:27.536864+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:45.009823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.538855+0000","last_change":"2024-04-25T08:39:25.536756+0000","last_active":"2024-04-25T08:39:28.538855+0000","last_peered":"2024-04-25T08:39:28.538855+0000","last_clean":"2024-04-25T08:39:28.538855+0000","last_became_active":"2024-04-25T08:39:25.535762+0000","last_became_peered":"2024-04-25T08:39:25.535762+0000","last_unstale":"2024-04-25T08:39:28.538855+0000","last_undegraded":"2024-04-25T08:39:28.538855+0000","last_fullsized":"2024-04-25T08:39:28.538855+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:41.407857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:27.536554+0000","last_change":"2024-04-25T08:39:26.549565+0000","last_active":"2024-04-25T08:39:27.536554+0000","last_peered":"2024-04-25T08:39:27.536554+0000","last_clean":"2024-04-25T08:39:27.536554+0000","last_became_active":"2024-04-25T08:39:26.549281+0000","last_became_peered":"2024-04-25T08:39:26.549281+0000","last_unstale":"2024-04-25T08:39:27.536554+0000","last_undegraded":"2024-04-25T08:39:27.536554+0000","last_fullsized":"2024-04-25T08:39:27.536554+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:25.517518+0000","last_clean_scrub_stamp":"2024-04-25T08:39:25.517518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:56.627033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"144'6","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.757550+0000","last_change":"2024-04-25T08:39:25.528176+0000","last_active":"2024-04-25T08:39:28.757550+0000","last_peered":"2024-04-25T08:39:28.757550+0000","last_clean":"2024-04-25T08:39:28.757550+0000","last_became_active":"2024-04-25T08:39:25.527834+0000","last_became_peered":"2024-04-25T08:39:25.527834+0000","last_unstale":"2024-04-25T08:39:28.757550+0000","last_undegraded":"2024-04-25T08:39:28.757550+0000","last_fullsized":"2024-04-25T08:39:28.757550+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:24.513339+0000","last_clean_scrub_stamp":"2024-04-25T08:39:24.513339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":402,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T08:39:28.539000+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:28.539000+0000","last_peered":"2024-04-25T08:39:28.539000+0000","last_clean":"2024-04-25T08:39:28.539000+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:28.539000+0000","last_undegraded":"2024-04-25T08:39:28.539000+0000","last_fullsized":"2024-04-25T08:39:28.539000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542272,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51588,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320252,"statfs":{"total":96636764160,"available":96583938048,"internally_reserved":0,"allocated":2412544,"data_stored":1439871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542272,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51004,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320836,"statfs":{"total":96636764160,"available":96584536064,"internally_reserved":0,"allocated":1818624,"data_stored":849551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542272,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46840,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325000,"statfs":{"total":96636764160,"available":96588800000,"internally_reserved":0,"allocated":1814528,"data_stored":849481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55180,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316660,"statfs":{"total":96636764160,"available":96580259840,"internally_reserved":0,"allocated":1826816,"data_stored":851093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46852,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324988,"statfs":{"total":96636764160,"available":96588787712,"internally_reserved":0,"allocated":1822720,"data_stored":850097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59336,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312504,"statfs":{"total":96636764160,"available":96576004096,"internally_reserved":0,"allocated":1822720,"data_stored":849593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55172,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316668,"statfs":{"total":96636764160,"available":96580268032,"internally_reserved":0,"allocated":1818624,"data_stored":849551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59924,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311916,"statfs":{"total":96636764160,"available":96575401984,"internally_reserved":0,"allocated":2433024,"data_stored":1442167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:34.344 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:39:34.344 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:39:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.495+0000 7fe58a4ac640 1 Processor -- start 2024-04-25T08:39:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.495+0000 7fe58a4ac640 1 -- start start 2024-04-25T08:39:34.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe58a4ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5840ced80 0x7fe5840d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe58a4ac640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe584161710 0x7fe5840d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe58a4ac640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58415e000 con 0x7fe58415dc20 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe58a4ac640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5840d1de0 con 0x7fe584161710 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe58a4ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe584161c60 con 0x7fe5840ced80 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe584161710 0x7fe5840d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe584161710 0x7fe5840d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54232/0 (socket says 172.21.15.60:54232) 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 -- 172.21.15.60:0/1602734760 learned_addr learned my addr 172.21.15.60:0/1602734760 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe5837fe640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5840ced80 0x7fe5840d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 -- 172.21.15.60:0/1602734760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5840ced80 msgr2=0x7fe5840d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5840ced80 0x7fe5840d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 -- 172.21.15.60:0/1602734760 >> v1:172.21.15.100:6789/0 conn(0x7fe58415dc20 legacy=0x7fe5840ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe588a49640 1 -- 172.21.15.60:0/1602734760 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584161f40 con 0x7fe584161710 2024-04-25T08:39:34.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.496+0000 7fe5837fe640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5840ced80 0x7fe5840d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe588a49640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe584161710 0x7fe5840d2820 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009e50 tx=0x7fe57c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea505958d9022eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe582ffd640 1 -- 172.21.15.60:0/1602734760 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c036070 con 0x7fe584161710 2024-04-25T08:39:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe582ffd640 1 -- 172.21.15.60:0/1602734760 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c031600 con 0x7fe584161710 2024-04-25T08:39:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe582ffd640 1 -- 172.21.15.60:0/1602734760 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0319d0 con 0x7fe584161710 2024-04-25T08:39:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe58a4ac640 1 -- 172.21.15.60:0/1602734760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe584161710 msgr2=0x7fe5840d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe584161710 0x7fe5840d2820 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009e50 tx=0x7fe57c02a6c0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe58a4ac640 1 -- 172.21.15.60:0/1602734760 shutdown_connections 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe584161710 0x7fe5840d2820 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/1602734760 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe5840ced80 0x7fe5840d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.497+0000 7fe58a4ac640 1 -- 172.21.15.60:0/1602734760 >> 172.21.15.60:0/1602734760 conn(0x7fe5840d0420 msgr2=0x7fe5840d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.498+0000 7fe58a4ac640 1 -- 172.21.15.60:0/1602734760 shutdown_connections 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.498+0000 7fe58a4ac640 1 -- 172.21.15.60:0/1602734760 wait complete. 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.499+0000 7fe58a4ac640 1 Processor -- start 2024-04-25T08:39:34.493 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.499+0000 7fe58a4ac640 1 -- start start 2024-04-25T08:39:34.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.499+0000 7fe58a4ac640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 0x7fe584157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.499+0000 7fe58a4ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58415dc20 0x7fe5840c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.499+0000 7fe58a4ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe584161710 0x7fe58415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe58a4ac640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b5e50 con 0x7fe5840ced80 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe58a4ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe584162500 con 0x7fe58415dc20 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe58a4ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b5fd0 con 0x7fe584161710 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 0x7fe584157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 0x7fe584157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54236/0 (socket says 172.21.15.60:54236) 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 -- 172.21.15.60:0/358470934 learned_addr learned my addr 172.21.15.60:0/358470934 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe588a49640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe584161710 0x7fe58415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe5837fe640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58415dc20 0x7fe5840c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 -- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe584161710 msgr2=0x7fe58415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe584161710 0x7fe58415e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 -- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58415dc20 msgr2=0x7fe5840c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58415dc20 0x7fe5840c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 -- 172.21.15.60:0/358470934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5840ca300 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe583fff640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 0x7fe584157840 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00b3f0 tx=0x7fe56c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c015020 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c011830 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58415f1d0 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c011c20 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.500+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58415f790 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.501+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5840cf380 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.502+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe56c018020 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.502+0000 7fe580ff9640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe56406b4d0 0x7fe56406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.502+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe56c089f90 con 0x7fe5840ced80 2024-04-25T08:39:34.497 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.502+0000 7fe5837fe640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe56406b4d0 0x7fe56406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.503+0000 7fe5837fe640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe56406b4d0 0x7fe56406d990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe58415a630 tx=0x7fe574008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.507+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5840cf380 con 0x7fe5840ced80 2024-04-25T08:39:34.751 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.758+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe5840d16b0 con 0x7fe5840ced80 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.760+0000 7fe580ff9640 1 -- 172.21.15.60:0/358470934 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe56c01f030 con 0x7fe5840ced80 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAsFCpme+Z7DBAA0amRWpgqshgQHGAcUS1xAg== 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stdout: caps mds = "allow" 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stdout: caps mon = "allow r" 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:39:34.754 INFO:teuthology.orchestra.run.smithi060.stderr:updated caps for client.0 2024-04-25T08:39:34.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe56406b4d0 msgr2=0x7fe56406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe56406b4d0 0x7fe56406d990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe58415a630 tx=0x7fe574008040 comp rx=0 tx=0).stop 2024-04-25T08:39:34.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 msgr2=0x7fe584157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.756 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 0x7fe584157840 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00b3f0 tx=0x7fe56c00b8b0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 shutdown_connections 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe584161710 0x7fe58415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe56406b4d0 0x7fe56406d990 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe58415dc20 0x7fe5840c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 --2- 172.21.15.60:0/358470934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe5840ced80 0x7fe584157840 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.762+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 >> 172.21.15.60:0/358470934 conn(0x7fe5840d0420 msgr2=0x7fe5840d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.763+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 shutdown_connections 2024-04-25T08:39:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.763+0000 7fe58a4ac640 1 -- 172.21.15.60:0/358470934 wait complete. 2024-04-25T08:39:34.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.874+0000 7efd61b29640 1 Processor -- start 2024-04-25T08:39:34.867 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.874+0000 7efd61b29640 1 -- start start 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd61b29640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efd5c0b6cb0 0x7efd5c0b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd61b29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd5c0b5a60 0x7efd5c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd61b29640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5c0ceee0 con 0x7efd5c15db70 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd61b29640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0d1d10 con 0x7efd5c0b6cb0 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd61b29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0d1ff0 con 0x7efd5c0b5a60 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd5b7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efd5c15db70 0x7efd5c162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36470/0 (socket says 172.21.15.60:36470) 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd5b7fe640 1 -- 172.21.15.60:0/2267839888 learned_addr learned my addr 172.21.15.60:0/2267839888 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd5affd640 1 --2- 172.21.15.60:0/2267839888 >> v2:172.21.15.100:3300/0 conn(0x7efd5c0b6cb0 0x7efd5c0b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.871 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd5bfff640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd5c0b5a60 0x7efd5c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274785044 0 0) 0x7efd5c0ceee0 con 0x7efd5c15db70 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.875+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd44003520 con 0x7efd5c15db70 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407120550 0 0) 0x7efd44003520 con 0x7efd5c15db70 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd5c0b5a60 msgr2=0x7efd5c0d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd5c0b5a60 0x7efd5c0d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 >> v2:172.21.15.100:3300/0 conn(0x7efd5c0b6cb0 msgr2=0x7efd5c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 --2- 172.21.15.60:0/2267839888 >> v2:172.21.15.100:3300/0 conn(0x7efd5c0b6cb0 0x7efd5c0b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0d22d0 con 0x7efd5c15db70 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7efd4c001670 con 0x7efd5c15db70 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5affd640 1 --2- 172.21.15.60:0/2267839888 >> v2:172.21.15.100:3300/0 conn(0x7efd5c0b6cb0 0x7efd5c0b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 >> v1:172.21.15.60:6789/0 conn(0x7efd5c15db70 legacy=0x7efd5c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd440039b0 0x7efd44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd440043a0 0x7efd44004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd44004ef0 0x7efd4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0d1d10 con 0x7efd44004ef0 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0d1ff0 con 0x7efd440043a0 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4400bf00 con 0x7efd440039b0 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd5c0b5a60 0x7efd5c0d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5affd640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd440043a0 0x7efd44004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd44004ef0 0x7efd4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5b7fe640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd440039b0 0x7efd44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 -- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd440039b0 msgr2=0x7efd44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd440039b0 0x7efd44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 -- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd440043a0 msgr2=0x7efd44004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd440043a0 0x7efd44004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.876+0000 7efd5bfff640 1 -- 172.21.15.60:0/2267839888 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44003520 con 0x7efd44004ef0 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd5bfff640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd44004ef0 0x7efd4400b9c0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7efd5400e1f0 tx=0x7efd44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effff04fd7f82f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54009d40 con 0x7efd44004ef0 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54035070 con 0x7efd44004ef0 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd5a7fc640 1 -- 172.21.15.60:0/2267839888 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54004030 con 0x7efd44004ef0 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd61b29640 1 -- 172.21.15.60:0/2267839888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd44004ef0 msgr2=0x7efd4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd61b29640 1 --2- 172.21.15.60:0/2267839888 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd44004ef0 0x7efd4400b9c0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7efd5400e1f0 tx=0x7efd44005830 comp rx=0 tx=0).stop 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.877+0000 7efd5bfff640 1 -- 172.21.15.60:0/2267839888 reap_dead start 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.878+0000 7efd61b29640 1 -- 172.21.15.60:0/2267839888 shutdown_connections 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.878+0000 7efd61b29640 1 -- 172.21.15.60:0/2267839888 >> 172.21.15.60:0/2267839888 conn(0x7efd5c0d0520 msgr2=0x7efd5c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.878+0000 7efd61b29640 1 -- 172.21.15.60:0/2267839888 shutdown_connections 2024-04-25T08:39:34.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.878+0000 7efd61b29640 1 -- 172.21.15.60:0/2267839888 wait complete. 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.879+0000 7efd61b29640 1 Processor -- start 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.879+0000 7efd61b29640 1 -- start start 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd44004ef0 0x7efd5c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1f6920 0x7efd5c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 0x7efd5c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0cf1c0 con 0x7efd5c15b6a0 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0ce530 con 0x7efd5c1f6920 2024-04-25T08:39:34.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0cf340 con 0x7efd44004ef0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 0x7efd5c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 0x7efd5c14cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54256/0 (socket says 172.21.15.60:54256) 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 -- 172.21.15.60:0/1986225316 learned_addr learned my addr 172.21.15.60:0/1986225316 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5b7fe640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd44004ef0 0x7efd5c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5affd640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1f6920 0x7efd5c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 -- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd44004ef0 msgr2=0x7efd5c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd44004ef0 0x7efd5c0cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 -- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1f6920 msgr2=0x7efd5c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1f6920 0x7efd5c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 -- 172.21.15.60:0/1986225316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c1f6da0 con 0x7efd5c15b6a0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd5bfff640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 0x7efd5c14cf60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7efd5402afc0 tx=0x7efd54009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54009d40 con 0x7efd5c15b6a0 2024-04-25T08:39:34.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5403c420 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c14dd20 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.880+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5403c810 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.881+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c0cd3b0 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.881+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c073990 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.882+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd54005d10 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.882+0000 7efd43fff640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd4806b500 0x7efd4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.882+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd540ab670 con 0x7efd5c15b6a0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.883+0000 7efd5b7fe640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd4806b500 0x7efd4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.883+0000 7efd5b7fe640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd4806b500 0x7efd4806d9c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efd4c000c00 tx=0x7efd4c0078f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:34.887+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd54079c80 con 0x7efd5c15b6a0 2024-04-25T08:39:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.103+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd5c0d1820 con 0x7efd5c15b6a0 2024-04-25T08:39:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.104+0000 7efd43fff640 1 -- 172.21.15.60:0/1986225316 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4945 (secure 0 0 0) 0x7efd5407dd20 con 0x7efd5c15b6a0 2024-04-25T08:39:35.097 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:35.098 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":89,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8716,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:27.896176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:35.098 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 89 2024-04-25T08:39:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd4806b500 msgr2=0x7efd4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd4806b500 0x7efd4806d9c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efd4c000c00 tx=0x7efd4c0078f0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.099 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 msgr2=0x7efd5c14cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 0x7efd5c14cf60 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7efd5402afc0 tx=0x7efd54009750 comp rx=0 tx=0).stop 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 shutdown_connections 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efd5c15b6a0 0x7efd5c14cf60 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efd4806b500 0x7efd4806d9c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd5c1f6920 0x7efd5c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 --2- 172.21.15.60:0/1986225316 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efd44004ef0 0x7efd5c0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 >> 172.21.15.60:0/1986225316 conn(0x7efd5c0d0520 msgr2=0x7efd5c158130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 shutdown_connections 2024-04-25T08:39:35.100 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.106+0000 7efd61b29640 1 -- 172.21.15.60:0/1986225316 wait complete. 2024-04-25T08:39:35.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:39:25.890816+0000', 'modified': '2024-04-25T08:39:27.896176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9924}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9924': {'gid': 9924, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6833/4231971347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 4231971347}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 4231971347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:39:35.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:39:35.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:39:35.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.627+0000 7faeb8d32640 1 Processor -- start 2024-04-25T08:39:35.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.627+0000 7faeb8d32640 1 -- start start 2024-04-25T08:39:35.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.628+0000 7faeb8d32640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41610f0 0x7faeb406c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.628+0000 7faeb8d32640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faeb40ce7b0 0x7faeb4161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.628+0000 7faeb8d32640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb40c6960 con 0x7faeb41616f0 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.628+0000 7faeb8d32640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb41614d0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.628+0000 7faeb8d32640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4162260 con 0x7faeb41610f0 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.628+0000 7faeb2d9d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7faeb40ce7b0 0x7faeb4161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb1d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7faeb41616f0 0x7faeb40ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36480/0 (socket says 172.21.15.60:36480) 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb1d9b640 1 -- 172.21.15.60:0/345732347 learned_addr learned my addr 172.21.15.60:0/345732347 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb259c640 1 --2- 172.21.15.60:0/345732347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41610f0 0x7faeb406c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb2d9d640 1 -- 172.21.15.60:0/345732347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41610f0 msgr2=0x7faeb406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/345732347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41610f0 0x7faeb406c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb2d9d640 1 -- 172.21.15.60:0/345732347 >> v1:172.21.15.60:6789/0 conn(0x7faeb41616f0 legacy=0x7faeb40ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.629+0000 7faeb2d9d640 1 -- 172.21.15.60:0/345732347 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4162540 con 0x7faeb40ce7b0 2024-04-25T08:39:35.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/345732347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4161b50 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7faea4009d90 tx=0x7faea402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129398b865d0e38a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb159a640 1 -- 172.21.15.60:0/345732347 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4007740 con 0x7faeb40ce7b0 2024-04-25T08:39:35.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb159a640 1 -- 172.21.15.60:0/345732347 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea40078e0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb159a640 1 -- 172.21.15.60:0/345732347 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4007cb0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 -- 172.21.15.60:0/345732347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 msgr2=0x7faeb4161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 --2- 172.21.15.60:0/345732347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4161b50 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7faea4009d90 tx=0x7faea402a660 comp rx=0 tx=0).stop 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 -- 172.21.15.60:0/345732347 shutdown_connections 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 --2- 172.21.15.60:0/345732347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4161b50 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 --2- 172.21.15.60:0/345732347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41610f0 0x7faeb406c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 -- 172.21.15.60:0/345732347 >> 172.21.15.60:0/345732347 conn(0x7faeb406f500 msgr2=0x7faeb406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 -- 172.21.15.60:0/345732347 shutdown_connections 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.630+0000 7faeb8d32640 1 -- 172.21.15.60:0/345732347 wait complete. 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 Processor -- start 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 -- start start 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faeb41610f0 0x7faeb415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41616f0 0x7faeb4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40cf690 con 0x7faeb40ce7b0 2024-04-25T08:39:35.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40cfce0 con 0x7faeb41610f0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.632+0000 7faeb8d32640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb406c700 con 0x7faeb41616f0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54278/0 (socket says 172.21.15.60:54278) 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 -- 172.21.15.60:0/4159610086 learned_addr learned my addr 172.21.15.60:0/4159610086 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41616f0 0x7faeb4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faeb41610f0 0x7faeb415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 -- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41616f0 msgr2=0x7faeb4159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41616f0 0x7faeb4159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 -- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faeb41610f0 msgr2=0x7faeb415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faeb41610f0 0x7faeb415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 -- 172.21.15.60:0/4159610086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb40b6200 con 0x7faeb40ce7b0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb259c640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4153f50 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d800 tx=0x7faeac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeaaffd640 1 -- 172.21.15.60:0/4159610086 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00b6c0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeaaffd640 1 -- 172.21.15.60:0/4159610086 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac019070 con 0x7faeb40ce7b0 2024-04-25T08:39:35.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb40b64e0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeaaffd640 1 -- 172.21.15.60:0/4159610086 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac00ba80 con 0x7faeb40ce7b0 2024-04-25T08:39:35.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.633+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb40b6aa0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.635+0000 7faeaaffd640 1 -- 172.21.15.60:0/4159610086 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faeac00bc20 con 0x7faeb40ce7b0 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.635+0000 7faeaaffd640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae9406b4d0 0x7fae9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.635+0000 7faeaaffd640 1 -- 172.21.15.60:0/4159610086 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faeac08a7b0 con 0x7faeb40ce7b0 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.635+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae9406b4d0 0x7fae9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.636+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae9406b4d0 0x7fae9406d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7faeb415a9f0 tx=0x7fae9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.636+0000 7faebba30740 1 Processor -- start 2024-04-25T08:39:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.636+0000 7faebba30740 1 -- start start 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39de230 0x55a0a39de610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39fbad0 0x55a0a39fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faebba30740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 0x55a0a39fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faebba30740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a3916a10 con 0x55a0a39b3f10 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a38bfb30 con 0x55a0a39fbad0 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a3912c50 con 0x55a0a39de230 2024-04-25T08:39:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faeb2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 0x55a0a39fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faeb2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 0x55a0a39fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54284/0 (socket says 172.21.15.60:54284) 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faeb2d9d640 1 -- 172.21.15.60:0/276723772 learned_addr learned my addr 172.21.15.60:0/276723772 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faeb259c640 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39de230 0x55a0a39de610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.637+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39fbad0 0x55a0a39fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7faeb2d9d640 1 -- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39de230 msgr2=0x55a0a39de610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39de230 0x55a0a39de610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7faeb2d9d640 1 -- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39fbad0 msgr2=0x55a0a39fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39fbad0 0x55a0a39fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7faeb2d9d640 1 -- 172.21.15.60:0/276723772 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0a38d4e10 con 0x55a0a39b3f10 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 0x55a0a39fa190 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7faea4009eb0 tx=0x7faea402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c422979c67dd19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7fae93fff640 1 -- 172.21.15.60:0/276723772 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40078c0 con 0x55a0a39b3f10 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7fae93fff640 1 -- 172.21.15.60:0/276723772 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4007a60 con 0x55a0a39b3f10 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.638+0000 7fae93fff640 1 -- 172.21.15.60:0/276723772 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4007e10 con 0x55a0a39b3f10 2024-04-25T08:39:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 -- 172.21.15.60:0/276723772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 msgr2=0x55a0a39fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 0x55a0a39fa190 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7faea4009eb0 tx=0x7faea402f000 comp rx=0 tx=0).stop 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 -- 172.21.15.60:0/276723772 shutdown_connections 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39b3f10 0x55a0a39fa190 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39fbad0 0x55a0a39fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 --2- 172.21.15.60:0/276723772 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39de230 0x55a0a39de610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 -- 172.21.15.60:0/276723772 >> 172.21.15.60:0/276723772 conn(0x55a0a39b0840 msgr2=0x55a0a39f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 -- 172.21.15.60:0/276723772 shutdown_connections 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 -- 172.21.15.60:0/276723772 wait complete. 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 -- start start 2024-04-25T08:39:35.635 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.639+0000 7faebba30740 1 Processor -- start 2024-04-25T08:39:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 0x55a0a39e0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a39b4cb0 con 0x55a0a39de230 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a38d30c0 con 0x55a0a39e0cb0 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a39ae770 con 0x55a0a39fbad0 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faebba30740 10 client.0 _ll_register_callbacks cb 0x55a0a38c6ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faeb259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 0x55a0a39e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faeb259c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 0x55a0a39e0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54288/0 (socket says 172.21.15.60:54288) 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.640+0000 7faeb259c640 1 -- 172.21.15.60:0/2984732273 learned_addr learned my addr 172.21.15.60:0/2984732273 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb259c640 1 -- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 msgr2=0x55a0a39cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb259c640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb259c640 1 -- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39e0cb0 msgr2=0x55a0a39ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb259c640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb259c640 1 -- 172.21.15.60:0/2984732273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0a362b2d0 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faeb259c640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 0x55a0a39e0770 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d7d0 tx=0x7faeac0595f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc95f800614553fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7fae917fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7fae917fa640 1 -- 172.21.15.60:0/2984732273 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac087060 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7fae917fa640 1 -- 172.21.15.60:0/2984732273 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faeac0554a0 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faea4007a60 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7fae917fa640 1 -- 172.21.15.60:0/2984732273 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7faeac070070 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 10 client.10101 fetch_fsmap learned FSMap version 89 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55a0a36f56b0 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7fae917fa640 1 -- 172.21.15.60:0/2984732273 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4007a60 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7fae917fa640 1 -- 172.21.15.60:0/2984732273 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faeac013070 con 0x55a0a39de230 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 10 client.10101 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 10 client.10101 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 20 client.10101 populate_metadata read hostname 'smithi060' 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a39f1990 0x55a0a3a170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 4 client.10101 mds_command: new command op to 9924 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.641+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 --> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0a3975920 con 0x55a0a39f1990 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.642+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a39f1990 0x55a0a3a170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.642+0000 7fae917fa640 10 client.10101 ms_handle_connect on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.642+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a39f1990 0x55a0a3a170d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d111a2cff56ce711 server_cookie=af443bcb94552c4b in_seq=0 out_seq=0 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.642+0000 7fae917fa640 1 -- 172.21.15.60:0/2984732273 <== mds.0 v2:172.21.15.60:6832/4231971347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0a3975920 con 0x55a0a39f1990 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.642+0000 7fae917fa640 10 client.10101 handle_command_reply: tid=0 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.642+0000 7faebba30740 1 client.10101 shutdown 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 msgr2=0x55a0a39e0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 0x55a0a39e0770 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d7d0 tx=0x7faeac0595f0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 shutdown_connections 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a39f1990 0x55a0a3a170d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 --2- 172.21.15.60:0/2984732273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39de230 0x55a0a39e0770 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7fae917fa640 0 client.10101 ms_handle_reset on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 >> 172.21.15.60:0/2984732273 conn(0x55a0a39b3f10 msgr2=0x55a0a39fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 shutdown_connections 2024-04-25T08:39:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 1 -- 172.21.15.60:0/2984732273 wait complete. 2024-04-25T08:39:35.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.643+0000 7faebba30740 20 client.10101 trim_cache size 0 max 16384 2024-04-25T08:39:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.659+0000 7faebba30740 1 Processor -- start 2024-04-25T08:39:35.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.659+0000 7faebba30740 1 -- start start 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a08980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faebba30740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faebba30740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4203a30 con 0x55a0a39e0cb0 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4153430 con 0x55a0a39de230 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4203430 con 0x55a0a39fbad0 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54304/0 (socket says 172.21.15.60:54304) 2024-04-25T08:39:35.655 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 -- 172.21.15.60:0/1109908938 learned_addr learned my addr 172.21.15.60:0/1109908938 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb259c640 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a08980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 -- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 msgr2=0x55a0a39b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 -- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 msgr2=0x55a0a3a08980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a08980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 -- 172.21.15.60:0/1109908938 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0a39f2e60 con 0x55a0a39e0cb0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb259c640 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ad690 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7faeb415a2d0 tx=0x7fae9c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92afff98f6ad8f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7fae937fe640 1 -- 172.21.15.60:0/1109908938 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c01e040 con 0x55a0a39e0cb0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7fae937fe640 1 -- 172.21.15.60:0/1109908938 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c029020 con 0x55a0a39e0cb0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.660+0000 7fae937fe640 1 -- 172.21.15.60:0/1109908938 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c005cb0 con 0x55a0a39e0cb0 2024-04-25T08:39:35.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 -- 172.21.15.60:0/1109908938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 msgr2=0x55a0a39ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ad690 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7faeb415a2d0 tx=0x7fae9c017040 comp rx=0 tx=0).stop 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 -- 172.21.15.60:0/1109908938 shutdown_connections 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39fbad0 0x55a0a39b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39e0cb0 0x55a0a39ad690 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 --2- 172.21.15.60:0/1109908938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 -- 172.21.15.60:0/1109908938 >> 172.21.15.60:0/1109908938 conn(0x55a0a39b3f10 msgr2=0x55a0a39b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 -- 172.21.15.60:0/1109908938 shutdown_connections 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 -- 172.21.15.60:0/1109908938 wait complete. 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.661+0000 7faebba30740 1 -- start start 2024-04-25T08:39:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 Processor -- start 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a07760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39e0cb0 0x55a0a3a07ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 0x55a0a3a58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a39f1810 con 0x55a0a39fbad0 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0a39f13e0 con 0x55a0a39de230 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0a39f1e10 con 0x55a0a39e0cb0 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faebba30740 10 client.0 _ll_register_callbacks cb 0x55a0a39aec50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faeb2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 0x55a0a3a58370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.662+0000 7faeb2d9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 0x55a0a3a58370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54306/0 (socket says 172.21.15.60:54306) 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 -- 172.21.15.60:0/795765809 learned_addr learned my addr 172.21.15.60:0/795765809 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb1d9b640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39e0cb0 0x55a0a3a07ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb259c640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a07760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 -- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39e0cb0 msgr2=0x55a0a3a07ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39e0cb0 0x55a0a3a07ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 -- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 msgr2=0x55a0a3a07760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a07760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 -- 172.21.15.60:0/795765809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0a39dab20 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faeb2d9d640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 0x55a0a3a58370 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7faea402ac50 tx=0x7faea4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca37f8231867881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7fae917fa640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7fae917fa640 1 -- 172.21.15.60:0/795765809 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4003990 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7fae917fa640 1 -- 172.21.15.60:0/795765809 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faea4003b30 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae9c029020 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7fae917fa640 1 -- 172.21.15.60:0/795765809 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea405b050 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faebba30740 10 client.10107 fetch_fsmap learned FSMap version 89 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.663+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55a0a39da320 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7fae917fa640 1 -- 172.21.15.60:0/795765809 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c029020 con 0x55a0a39fbad0 2024-04-25T08:39:35.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7fae917fa640 1 -- 172.21.15.60:0/795765809 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faea405e020 con 0x55a0a39fbad0 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 10 client.10107 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 10 client.10107 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 20 client.10107 populate_metadata read hostname 'smithi060' 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a3a589d0 0x55a0a3a644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 4 client.10107 mds_command: new command op to 9924 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 --> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a0a39af660 con 0x55a0a3a589d0 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faeb259c640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a3a589d0 0x55a0a3a644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7fae917fa640 10 client.10107 ms_handle_connect on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faeb259c640 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a3a589d0 0x55a0a3a644a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43529dddbcab462e server_cookie=3ec949f8a70a90ae in_seq=0 out_seq=0 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7fae917fa640 1 -- 172.21.15.60:0/795765809 <== mds.0 v2:172.21.15.60:6832/4231971347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a0a39af660 con 0x55a0a3a589d0 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7fae917fa640 10 client.10107 handle_command_reply: tid=0 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 1 client.10107 shutdown 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 msgr2=0x55a0a3a58370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.664+0000 7faebba30740 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 0x55a0a3a58370 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7faea402ac50 tx=0x7faea4004ab0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 shutdown_connections 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x55a0a3a589d0 0x55a0a3a644a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55a0a39fbad0 0x55a0a3a58370 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x55a0a39e0cb0 0x55a0a3a07ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 --2- 172.21.15.60:0/795765809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a0a39de230 0x55a0a3a07760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7fae917fa640 0 client.10107 ms_handle_reset on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 >> 172.21.15.60:0/795765809 conn(0x55a0a39b3f10 msgr2=0x55a0a3a1b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 shutdown_connections 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 1 -- 172.21.15.60:0/795765809 wait complete. 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faebba30740 20 client.10107 trim_cache size 0 max 16384 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:39:35.664 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9924, 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 9.1336602209999995, 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 89, 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 145, 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 145, 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 14.159717618 2024-04-25T08:39:35.665 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae9406b4d0 msgr2=0x7fae9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae9406b4d0 0x7fae9406d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7faeb415a9f0 tx=0x7fae9c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 msgr2=0x7faeb4153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4153f50 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d800 tx=0x7faeac00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 shutdown_connections 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faeb41616f0 0x7faeb4159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.666 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fae9406b4d0 0x7fae9406d990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faeb41610f0 0x7faeb415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 --2- 172.21.15.60:0/4159610086 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faeb40ce7b0 0x7faeb4153f50 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 >> 172.21.15.60:0/4159610086 conn(0x7faeb406f500 msgr2=0x7faeb40d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 shutdown_connections 2024-04-25T08:39:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.665+0000 7faeb8d32640 1 -- 172.21.15.60:0/4159610086 wait complete. 2024-04-25T08:39:35.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:39:35.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.778+0000 7f83100fc640 1 Processor -- start 2024-04-25T08:39:35.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.778+0000 7f83100fc640 1 -- start start 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f83100fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f830815dd30 0x7f83080b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f83100fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83080b9770 0x7f83080d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f83100fc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83080d1df0 con 0x7f83080d2e90 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f83100fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83080b7a50 con 0x7f83080b9770 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f83100fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83080d20d0 con 0x7f830815dd30 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83080b9770 0x7f83080d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f83080b9770 0x7f83080d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54314/0 (socket says 172.21.15.60:54314) 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 -- 172.21.15.60:0/1046667731 learned_addr learned my addr 172.21.15.60:0/1046667731 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830de98640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f830815dd30 0x7f83080b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 -- 172.21.15.60:0/1046667731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f830815dd30 msgr2=0x7f83080b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f830815dd30 0x7f83080b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 -- 172.21.15.60:0/1046667731 >> v1:172.21.15.100:6789/0 conn(0x7f83080d2e90 legacy=0x7f83080b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830d697640 1 -- 172.21.15.60:0/1046667731 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83080d23b0 con 0x7f83080b9770 2024-04-25T08:39:35.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.779+0000 7f830de98640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f830815dd30 0x7f83080b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.780+0000 7f830d697640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83080d03c0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009850 tx=0x7f82f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1563c8c8be973cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.780+0000 7f830ce96640 1 -- 172.21.15.60:0/1046667731 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8004030 con 0x7f83080b9770 2024-04-25T08:39:35.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.780+0000 7f830ce96640 1 -- 172.21.15.60:0/1046667731 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f80041d0 con 0x7f83080b9770 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.780+0000 7f830ce96640 1 -- 172.21.15.60:0/1046667731 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8003610 con 0x7f83080b9770 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 -- 172.21.15.60:0/1046667731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 msgr2=0x7f83080d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83080d03c0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009850 tx=0x7f82f802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 -- 172.21.15.60:0/1046667731 shutdown_connections 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83080d03c0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 --2- 172.21.15.60:0/1046667731 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f830815dd30 0x7f83080b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 -- 172.21.15.60:0/1046667731 >> 172.21.15.60:0/1046667731 conn(0x7f8308061c30 msgr2=0x7f830806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 -- 172.21.15.60:0/1046667731 shutdown_connections 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.781+0000 7f83100fc640 1 -- 172.21.15.60:0/1046667731 wait complete. 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.782+0000 7f83100fc640 1 Processor -- start 2024-04-25T08:39:35.776 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.782+0000 7f83100fc640 1 -- start start 2024-04-25T08:39:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f83100fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83081f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f83100fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83080d2e90 0x7f8308153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f83100fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f830815dd30 0x7f8308145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f83100fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080d1a10 con 0x7f83080b9770 2024-04-25T08:39:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f83100fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080d2970 con 0x7f830815dd30 2024-04-25T08:39:35.778 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f83100fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83080d1b90 con 0x7f83080d2e90 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83081f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83081f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54318/0 (socket says 172.21.15.60:54318) 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 -- 172.21.15.60:0/768356870 learned_addr learned my addr 172.21.15.60:0/768356870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830d697640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83080d2e90 0x7f8308153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830e699640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f830815dd30 0x7f8308145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 -- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83080d2e90 msgr2=0x7f8308153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83080d2e90 0x7f8308153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 -- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f830815dd30 msgr2=0x7f8308145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f830815dd30 0x7f8308145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 -- 172.21.15.60:0/768356870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83081f69e0 con 0x7f83080b9770 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f830de98640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83081f64a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f830400d800 tx=0x7f830400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.783+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304004030 con 0x7f83080b9770 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.784+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83040041d0 con 0x7f83080b9770 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.784+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8308156330 con 0x7f83080b9770 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.784+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830400b6c0 con 0x7f83080b9770 2024-04-25T08:39:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.784+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081568d0 con 0x7f83080b9770 2024-04-25T08:39:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.784+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8308073990 con 0x7f83080b9770 2024-04-25T08:39:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.785+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f830400b860 con 0x7f83080b9770 2024-04-25T08:39:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.785+0000 7f82fe7fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f82ec06b4d0 0x7f82ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.786+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f830408a5f0 con 0x7f83080b9770 2024-04-25T08:39:35.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.786+0000 7f830d697640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f82ec06b4d0 0x7f82ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.781 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.786+0000 7f830d697640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f82ec06b4d0 0x7f82ec06d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f82f8033000 tx=0x7f82f802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.783 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:35.790+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8304059370 con 0x7f83080b9770 2024-04-25T08:39:35.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.006+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f830815d350 con 0x7f83080b9770 2024-04-25T08:39:35.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.006+0000 7f82fe7fc640 1 -- 172.21.15.60:0/768356870 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8304058c70 con 0x7f83080b9770 2024-04-25T08:39:36.000 INFO:teuthology.orchestra.run.smithi060.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f82ec06b4d0 msgr2=0x7f82ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f82ec06b4d0 0x7f82ec06d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f82f8033000 tx=0x7f82f802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 msgr2=0x7f83081f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83081f64a0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f830400d800 tx=0x7f830400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 shutdown_connections 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f82ec06b4d0 0x7f82ec06d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f830815dd30 0x7f8308145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f83080d2e90 0x7f8308153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 --2- 172.21.15.60:0/768356870 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f83080b9770 0x7f83081f64a0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 >> 172.21.15.60:0/768356870 conn(0x7f8308061c30 msgr2=0x7f83080d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 shutdown_connections 2024-04-25T08:39:36.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:36.008+0000 7f83100fc640 1 -- 172.21.15.60:0/768356870 wait complete. 2024-04-25T08:39:36.012 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi060.front.sepia.ceph.com 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:39:36.013 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:39:36.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.013 DEBUG:teuthology.orchestra.run.smithi060:> ip addr 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 ::1/128 scope host 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:36.029 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f0 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:6b:fe:cd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp7s0f1 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:60 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f0 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: inet 172.21.15.60/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft 42134sec preferred_lft 42134sec 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::ec4:7aff:febd:1560/64 scope link noprefixroute 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 0c:c4:7a:bd:15:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: altname enp1s0f1 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:39:36.030 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::a872:6dff:fefc:a795/64 scope link 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether 5a:a4:32:45:7e:61 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::58a4:32ff:fe45:7e61/64 scope link 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: link/ether c6:13:a0:5e:3b:57 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: inet6 fe80::c413:a0ff:fe5e:3b57/64 scope link 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run.smithi060.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:36.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.032 DEBUG:teuthology.orchestra.run.smithi060:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:39:36.091 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-04-25T08:39:36.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.093 DEBUG:teuthology.orchestra.run.smithi060:> route 2024-04-25T08:39:36.149 INFO:teuthology.orchestra.run.smithi060.stdout:Kernel IP routing table 2024-04-25T08:39:36.149 INFO:teuthology.orchestra.run.smithi060.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:39:36.149 INFO:teuthology.orchestra.run.smithi060.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:39:36.149 INFO:teuthology.orchestra.run.smithi060.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:39:36.150 INFO:teuthology.orchestra.run.smithi060.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:39:36.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.150 DEBUG:teuthology.orchestra.run.smithi060:> (cd / && exec bash -c ' 2024-04-25T08:39:36.150 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:36.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:39:36.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:39:36.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:39:36.150 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:36.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.234 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:39:36.250 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:39:36.250 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:39:36.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:39:36.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link set brx.0 up 2024-04-25T08:39:36.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:39:36.315 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:36.370 INFO:teuthology.orchestra.run.smithi060.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:39:36.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:39:36.371 DEBUG:teuthology.orchestra.run.smithi060:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:36.426 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:39:36.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:39:36.509 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:39:36.509 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:39:36.509 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:39:36.517 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:39:36.517 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:39:36.517 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:39:36.517 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:39:36.518 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:39:36.526 INFO:teuthology.orchestra.run.smithi060.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:39:36.526 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:39:36.526 INFO:teuthology.orchestra.run.smithi060.stdout:invalid new device string format 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:3300,172.21.15.100:3300,172.21.15.100:3301" 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: resolved to: "172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790" 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.60:6789,172.21.15.100:6789,172.21.15.100:6790:/ 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de" will pass to kernel 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout: You just mounted a file system that supports labels which does not 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T08:39:36.527 INFO:teuthology.orchestra.run.smithi060.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T08:39:36.527 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:39:36.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:36.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.593 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:39:36.656 INFO:teuthology.orchestra.run.smithi060.stdout:client8845 2024-04-25T08:39:36.656 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:36.656 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:36.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:36.712 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.788 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796374 s, 132 MB/s 2024-04-25T08:39:36.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.797 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.797 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083763 s, 125 MB/s 2024-04-25T08:39:36.804 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.805 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069731 s, 150 MB/s 2024-04-25T08:39:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.813 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774617 s, 135 MB/s 2024-04-25T08:39:36.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.821 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067803 s, 155 MB/s 2024-04-25T08:39:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.829 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067378 s, 156 MB/s 2024-04-25T08:39:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.835 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058662 s, 179 MB/s 2024-04-25T08:39:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.843 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714605 s, 147 MB/s 2024-04-25T08:39:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.851 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.852 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690423 s, 152 MB/s 2024-04-25T08:39:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.860 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770144 s, 136 MB/s 2024-04-25T08:39:36.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.867 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642012 s, 163 MB/s 2024-04-25T08:39:36.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078792 s, 133 MB/s 2024-04-25T08:39:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066426 s, 158 MB/s 2024-04-25T08:39:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556358 s, 188 MB/s 2024-04-25T08:39:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648971 s, 162 MB/s 2024-04-25T08:39:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065199 s, 161 MB/s 2024-04-25T08:39:36.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.913 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719713 s, 146 MB/s 2024-04-25T08:39:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634852 s, 165 MB/s 2024-04-25T08:39:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647734 s, 162 MB/s 2024-04-25T08:39:36.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609496 s, 172 MB/s 2024-04-25T08:39:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.943 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773133 s, 136 MB/s 2024-04-25T08:39:36.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665443 s, 158 MB/s 2024-04-25T08:39:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593474 s, 177 MB/s 2024-04-25T08:39:36.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659376 s, 159 MB/s 2024-04-25T08:39:36.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.973 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714547 s, 147 MB/s 2024-04-25T08:39:36.980 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.980 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.980 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626009 s, 168 MB/s 2024-04-25T08:39:36.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.988 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652112 s, 161 MB/s 2024-04-25T08:39:36.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:36.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:36.995 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580526 s, 181 MB/s 2024-04-25T08:39:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.002 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697651 s, 150 MB/s 2024-04-25T08:39:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630575 s, 166 MB/s 2024-04-25T08:39:37.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.016 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.016 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613073 s, 171 MB/s 2024-04-25T08:39:37.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.024 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.024 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619353 s, 169 MB/s 2024-04-25T08:39:37.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:37.025 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:37.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757906 s, 138 MB/s 2024-04-25T08:39:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621818 s, 169 MB/s 2024-04-25T08:39:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.106 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717799 s, 146 MB/s 2024-04-25T08:39:37.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718551 s, 146 MB/s 2024-04-25T08:39:37.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.122 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660978 s, 159 MB/s 2024-04-25T08:39:37.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645858 s, 162 MB/s 2024-04-25T08:39:37.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.139 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.139 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061912 s, 169 MB/s 2024-04-25T08:39:37.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.144 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601881 s, 174 MB/s 2024-04-25T08:39:37.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.151 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063892 s, 164 MB/s 2024-04-25T08:39:37.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697642 s, 150 MB/s 2024-04-25T08:39:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794441 s, 132 MB/s 2024-04-25T08:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064776 s, 162 MB/s 2024-04-25T08:39:37.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715031 s, 147 MB/s 2024-04-25T08:39:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610749 s, 172 MB/s 2024-04-25T08:39:37.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745884 s, 141 MB/s 2024-04-25T08:39:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.206 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697007 s, 150 MB/s 2024-04-25T08:39:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.213 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620152 s, 169 MB/s 2024-04-25T08:39:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.220 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.221 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625932 s, 168 MB/s 2024-04-25T08:39:37.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615579 s, 170 MB/s 2024-04-25T08:39:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732388 s, 143 MB/s 2024-04-25T08:39:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.244 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722834 s, 145 MB/s 2024-04-25T08:39:37.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676783 s, 155 MB/s 2024-04-25T08:39:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.259 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650296 s, 161 MB/s 2024-04-25T08:39:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.267 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701443 s, 149 MB/s 2024-04-25T08:39:37.274 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.274 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.274 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622794 s, 168 MB/s 2024-04-25T08:39:37.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606591 s, 173 MB/s 2024-04-25T08:39:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723679 s, 145 MB/s 2024-04-25T08:39:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.297 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653967 s, 160 MB/s 2024-04-25T08:39:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.304 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660084 s, 159 MB/s 2024-04-25T08:39:37.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.311 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651101 s, 161 MB/s 2024-04-25T08:39:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.319 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733464 s, 143 MB/s 2024-04-25T08:39:37.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.327 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710174 s, 148 MB/s 2024-04-25T08:39:37.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:37.329 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:37.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.395 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659581 s, 159 MB/s 2024-04-25T08:39:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.402 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640209 s, 164 MB/s 2024-04-25T08:39:37.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.409 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.409 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632493 s, 166 MB/s 2024-04-25T08:39:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.416 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657462 s, 159 MB/s 2024-04-25T08:39:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061005 s, 172 MB/s 2024-04-25T08:39:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.431 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605372 s, 173 MB/s 2024-04-25T08:39:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.439 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751583 s, 140 MB/s 2024-04-25T08:39:37.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.446 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.447 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068796 s, 152 MB/s 2024-04-25T08:39:37.453 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.453 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.453 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00554405 s, 189 MB/s 2024-04-25T08:39:37.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.460 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621968 s, 169 MB/s 2024-04-25T08:39:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733877 s, 143 MB/s 2024-04-25T08:39:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.475 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611851 s, 171 MB/s 2024-04-25T08:39:37.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.483 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614764 s, 171 MB/s 2024-04-25T08:39:37.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.489 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588087 s, 178 MB/s 2024-04-25T08:39:37.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664126 s, 158 MB/s 2024-04-25T08:39:37.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625662 s, 168 MB/s 2024-04-25T08:39:37.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063178 s, 166 MB/s 2024-04-25T08:39:37.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.519 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713374 s, 147 MB/s 2024-04-25T08:39:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.527 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602058 s, 174 MB/s 2024-04-25T08:39:37.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632627 s, 166 MB/s 2024-04-25T08:39:37.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.542 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736366 s, 142 MB/s 2024-04-25T08:39:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.550 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699207 s, 150 MB/s 2024-04-25T08:39:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.556 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557144 s, 188 MB/s 2024-04-25T08:39:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.564 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645148 s, 163 MB/s 2024-04-25T08:39:37.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.571 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638765 s, 164 MB/s 2024-04-25T08:39:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.578 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060946 s, 172 MB/s 2024-04-25T08:39:37.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580865 s, 181 MB/s 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659228 s, 159 MB/s 2024-04-25T08:39:37.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.599 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633153 s, 166 MB/s 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639231 s, 164 MB/s 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600575 s, 175 MB/s 2024-04-25T08:39:37.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.620 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564887 s, 186 MB/s 2024-04-25T08:39:37.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:37.621 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:37.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.686 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724256 s, 145 MB/s 2024-04-25T08:39:37.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.694 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.694 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704051 s, 149 MB/s 2024-04-25T08:39:37.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.703 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746422 s, 140 MB/s 2024-04-25T08:39:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.710 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641024 s, 164 MB/s 2024-04-25T08:39:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.718 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732908 s, 143 MB/s 2024-04-25T08:39:37.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746215 s, 141 MB/s 2024-04-25T08:39:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.734 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634097 s, 165 MB/s 2024-04-25T08:39:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708744 s, 148 MB/s 2024-04-25T08:39:37.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621359 s, 169 MB/s 2024-04-25T08:39:37.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.756 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.756 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655685 s, 160 MB/s 2024-04-25T08:39:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.764 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711145 s, 147 MB/s 2024-04-25T08:39:37.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.772 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.772 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685508 s, 153 MB/s 2024-04-25T08:39:37.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.779 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.779 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607527 s, 173 MB/s 2024-04-25T08:39:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.786 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610898 s, 172 MB/s 2024-04-25T08:39:37.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.793 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.793 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059581 s, 176 MB/s 2024-04-25T08:39:37.800 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.800 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.800 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612928 s, 171 MB/s 2024-04-25T08:39:37.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.807 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060374 s, 174 MB/s 2024-04-25T08:39:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.814 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600621 s, 175 MB/s 2024-04-25T08:39:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.821 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657444 s, 159 MB/s 2024-04-25T08:39:37.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.829 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064546 s, 162 MB/s 2024-04-25T08:39:37.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758007 s, 138 MB/s 2024-04-25T08:39:37.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.844 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612668 s, 171 MB/s 2024-04-25T08:39:37.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.852 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667149 s, 157 MB/s 2024-04-25T08:39:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580856 s, 181 MB/s 2024-04-25T08:39:37.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.866 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.866 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063744 s, 164 MB/s 2024-04-25T08:39:37.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.874 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.874 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719022 s, 146 MB/s 2024-04-25T08:39:37.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711215 s, 147 MB/s 2024-04-25T08:39:37.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.889 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622956 s, 168 MB/s 2024-04-25T08:39:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715201 s, 147 MB/s 2024-04-25T08:39:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621693 s, 169 MB/s 2024-04-25T08:39:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.911 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061403 s, 171 MB/s 2024-04-25T08:39:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.918 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623062 s, 168 MB/s 2024-04-25T08:39:37.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:37.920 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:37.985 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.986 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639825 s, 164 MB/s 2024-04-25T08:39:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.993 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621476 s, 169 MB/s 2024-04-25T08:39:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:37.999 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005948 s, 176 MB/s 2024-04-25T08:39:38.007 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.008 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723124 s, 145 MB/s 2024-04-25T08:39:38.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.015 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674169 s, 156 MB/s 2024-04-25T08:39:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721853 s, 145 MB/s 2024-04-25T08:39:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627453 s, 167 MB/s 2024-04-25T08:39:38.037 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621524 s, 169 MB/s 2024-04-25T08:39:38.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631035 s, 166 MB/s 2024-04-25T08:39:38.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.052 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062196 s, 169 MB/s 2024-04-25T08:39:38.060 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.060 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.060 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782337 s, 134 MB/s 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695054 s, 151 MB/s 2024-04-25T08:39:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638612 s, 164 MB/s 2024-04-25T08:39:38.082 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.082 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612019 s, 171 MB/s 2024-04-25T08:39:38.090 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701792 s, 149 MB/s 2024-04-25T08:39:38.098 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724152 s, 145 MB/s 2024-04-25T08:39:38.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.105 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.105 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587605 s, 178 MB/s 2024-04-25T08:39:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.112 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.113 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618603 s, 170 MB/s 2024-04-25T08:39:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.120 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733381 s, 143 MB/s 2024-04-25T08:39:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.127 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602233 s, 174 MB/s 2024-04-25T08:39:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.135 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723439 s, 145 MB/s 2024-04-25T08:39:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.143 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067315 s, 156 MB/s 2024-04-25T08:39:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.150 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653979 s, 160 MB/s 2024-04-25T08:39:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.157 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572137 s, 183 MB/s 2024-04-25T08:39:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.164 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584045 s, 180 MB/s 2024-04-25T08:39:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645424 s, 162 MB/s 2024-04-25T08:39:38.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.179 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632994 s, 166 MB/s 2024-04-25T08:39:38.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070528 s, 149 MB/s 2024-04-25T08:39:38.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070645 s, 148 MB/s 2024-04-25T08:39:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650412 s, 161 MB/s 2024-04-25T08:39:38.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560272 s, 187 MB/s 2024-04-25T08:39:38.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.215 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586891 s, 179 MB/s 2024-04-25T08:39:38.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:38.216 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:38.216 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:38.216 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:39:38.216 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:38.217 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:38.217 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:38.217 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.281 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704424 s, 149 MB/s 2024-04-25T08:39:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688566 s, 152 MB/s 2024-04-25T08:39:38.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.295 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.296 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570709 s, 184 MB/s 2024-04-25T08:39:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.303 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648188 s, 162 MB/s 2024-04-25T08:39:38.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.310 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.310 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610777 s, 172 MB/s 2024-04-25T08:39:38.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.317 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.317 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647823 s, 162 MB/s 2024-04-25T08:39:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.325 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681271 s, 154 MB/s 2024-04-25T08:39:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.333 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706138 s, 148 MB/s 2024-04-25T08:39:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.340 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620062 s, 169 MB/s 2024-04-25T08:39:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.348 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699518 s, 150 MB/s 2024-04-25T08:39:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.355 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644093 s, 163 MB/s 2024-04-25T08:39:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.362 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572492 s, 183 MB/s 2024-04-25T08:39:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.369 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591581 s, 177 MB/s 2024-04-25T08:39:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.376 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641908 s, 163 MB/s 2024-04-25T08:39:38.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.383 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570103 s, 184 MB/s 2024-04-25T08:39:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.391 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711378 s, 147 MB/s 2024-04-25T08:39:38.398 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.399 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691996 s, 152 MB/s 2024-04-25T08:39:38.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.406 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.406 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647117 s, 162 MB/s 2024-04-25T08:39:38.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.413 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.413 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612172 s, 171 MB/s 2024-04-25T08:39:38.421 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.421 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.421 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745995 s, 141 MB/s 2024-04-25T08:39:38.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.428 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.428 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623194 s, 168 MB/s 2024-04-25T08:39:38.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737078 s, 142 MB/s 2024-04-25T08:39:38.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061349 s, 171 MB/s 2024-04-25T08:39:38.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652251 s, 161 MB/s 2024-04-25T08:39:38.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.458 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.458 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063368 s, 165 MB/s 2024-04-25T08:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.466 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642739 s, 163 MB/s 2024-04-25T08:39:38.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.473 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585281 s, 179 MB/s 2024-04-25T08:39:38.480 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.481 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725041 s, 145 MB/s 2024-04-25T08:39:38.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.488 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.488 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659037 s, 159 MB/s 2024-04-25T08:39:38.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580946 s, 180 MB/s 2024-04-25T08:39:38.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.502 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633644 s, 165 MB/s 2024-04-25T08:39:38.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.509 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.509 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646439 s, 162 MB/s 2024-04-25T08:39:38.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:38.511 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.576 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588859 s, 178 MB/s 2024-04-25T08:39:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.583 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617116 s, 170 MB/s 2024-04-25T08:39:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.589 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607195 s, 173 MB/s 2024-04-25T08:39:38.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.597 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.597 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695158 s, 151 MB/s 2024-04-25T08:39:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.604 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605357 s, 173 MB/s 2024-04-25T08:39:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.611 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595522 s, 176 MB/s 2024-04-25T08:39:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.618 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592196 s, 177 MB/s 2024-04-25T08:39:38.625 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.626 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665845 s, 157 MB/s 2024-04-25T08:39:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.633 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062683 s, 167 MB/s 2024-04-25T08:39:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.640 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631293 s, 166 MB/s 2024-04-25T08:39:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.648 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757684 s, 138 MB/s 2024-04-25T08:39:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.655 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593931 s, 177 MB/s 2024-04-25T08:39:38.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.662 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.663 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636094 s, 165 MB/s 2024-04-25T08:39:38.669 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.670 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622618 s, 168 MB/s 2024-04-25T08:39:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.677 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618489 s, 170 MB/s 2024-04-25T08:39:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.684 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634315 s, 165 MB/s 2024-04-25T08:39:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.691 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614868 s, 171 MB/s 2024-04-25T08:39:38.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.697 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550083 s, 191 MB/s 2024-04-25T08:39:38.704 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.705 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622651 s, 168 MB/s 2024-04-25T08:39:38.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.713 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.713 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739941 s, 142 MB/s 2024-04-25T08:39:38.720 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.720 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.720 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621088 s, 169 MB/s 2024-04-25T08:39:38.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.728 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708971 s, 148 MB/s 2024-04-25T08:39:38.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.736 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709167 s, 148 MB/s 2024-04-25T08:39:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725883 s, 144 MB/s 2024-04-25T08:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.751 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609551 s, 172 MB/s 2024-04-25T08:39:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.759 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665635 s, 158 MB/s 2024-04-25T08:39:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.766 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643159 s, 163 MB/s 2024-04-25T08:39:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.773 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596803 s, 176 MB/s 2024-04-25T08:39:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.780 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.781 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655772 s, 160 MB/s 2024-04-25T08:39:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.787 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619575 s, 169 MB/s 2024-04-25T08:39:38.794 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626985 s, 167 MB/s 2024-04-25T08:39:38.801 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062393 s, 168 MB/s 2024-04-25T08:39:38.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:38.803 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.868 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073041 s, 144 MB/s 2024-04-25T08:39:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.875 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624956 s, 168 MB/s 2024-04-25T08:39:38.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071289 s, 147 MB/s 2024-04-25T08:39:38.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.891 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618267 s, 170 MB/s 2024-04-25T08:39:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700992 s, 150 MB/s 2024-04-25T08:39:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.907 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740075 s, 142 MB/s 2024-04-25T08:39:38.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.915 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.915 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733048 s, 143 MB/s 2024-04-25T08:39:38.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.922 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.922 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579597 s, 181 MB/s 2024-04-25T08:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.929 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633637 s, 165 MB/s 2024-04-25T08:39:38.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640648 s, 164 MB/s 2024-04-25T08:39:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.943 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608146 s, 172 MB/s 2024-04-25T08:39:38.950 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672594 s, 156 MB/s 2024-04-25T08:39:38.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.957 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.957 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570314 s, 184 MB/s 2024-04-25T08:39:38.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645284 s, 162 MB/s 2024-04-25T08:39:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.975 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620985 s, 169 MB/s 2024-04-25T08:39:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.979 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662744 s, 158 MB/s 2024-04-25T08:39:38.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.987 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070997 s, 148 MB/s 2024-04-25T08:39:38.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:38.994 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:38.994 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612734 s, 171 MB/s 2024-04-25T08:39:39.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.001 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.001 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610476 s, 172 MB/s 2024-04-25T08:39:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.008 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605144 s, 173 MB/s 2024-04-25T08:39:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.015 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593336 s, 177 MB/s 2024-04-25T08:39:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.022 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634993 s, 165 MB/s 2024-04-25T08:39:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.029 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061349 s, 171 MB/s 2024-04-25T08:39:39.035 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.036 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.036 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559292 s, 187 MB/s 2024-04-25T08:39:39.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.043 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647815 s, 162 MB/s 2024-04-25T08:39:39.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560322 s, 187 MB/s 2024-04-25T08:39:39.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.057 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608075 s, 172 MB/s 2024-04-25T08:39:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.064 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627406 s, 167 MB/s 2024-04-25T08:39:39.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.071 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059864 s, 175 MB/s 2024-04-25T08:39:39.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.078 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617028 s, 170 MB/s 2024-04-25T08:39:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.085 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600685 s, 175 MB/s 2024-04-25T08:39:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.092 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622521 s, 168 MB/s 2024-04-25T08:39:39.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:39.093 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.159 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730578 s, 144 MB/s 2024-04-25T08:39:39.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725973 s, 144 MB/s 2024-04-25T08:39:39.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.174 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574109 s, 183 MB/s 2024-04-25T08:39:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.181 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636995 s, 165 MB/s 2024-04-25T08:39:39.188 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.189 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.189 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063722 s, 165 MB/s 2024-04-25T08:39:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.195 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059173 s, 177 MB/s 2024-04-25T08:39:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.203 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643733 s, 163 MB/s 2024-04-25T08:39:39.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631745 s, 166 MB/s 2024-04-25T08:39:39.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060573 s, 173 MB/s 2024-04-25T08:39:39.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.224 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.224 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061052 s, 172 MB/s 2024-04-25T08:39:39.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.231 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606778 s, 173 MB/s 2024-04-25T08:39:39.238 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.239 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651531 s, 161 MB/s 2024-04-25T08:39:39.245 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.246 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632683 s, 166 MB/s 2024-04-25T08:39:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641658 s, 163 MB/s 2024-04-25T08:39:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.260 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061952 s, 169 MB/s 2024-04-25T08:39:39.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.267 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646632 s, 162 MB/s 2024-04-25T08:39:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.275 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681154 s, 154 MB/s 2024-04-25T08:39:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.282 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605702 s, 173 MB/s 2024-04-25T08:39:39.288 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613411 s, 171 MB/s 2024-04-25T08:39:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.296 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.297 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069545 s, 151 MB/s 2024-04-25T08:39:39.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.304 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.304 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687575 s, 153 MB/s 2024-04-25T08:39:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.311 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612842 s, 171 MB/s 2024-04-25T08:39:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.319 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699611 s, 150 MB/s 2024-04-25T08:39:39.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.326 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.327 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644274 s, 163 MB/s 2024-04-25T08:39:39.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.334 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.334 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692728 s, 151 MB/s 2024-04-25T08:39:39.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.341 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650078 s, 161 MB/s 2024-04-25T08:39:39.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685571 s, 153 MB/s 2024-04-25T08:39:39.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.357 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067963 s, 154 MB/s 2024-04-25T08:39:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.365 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713717 s, 147 MB/s 2024-04-25T08:39:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.373 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723466 s, 145 MB/s 2024-04-25T08:39:39.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.380 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.380 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616719 s, 170 MB/s 2024-04-25T08:39:39.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705688 s, 149 MB/s 2024-04-25T08:39:39.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:39.389 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.454 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699385 s, 150 MB/s 2024-04-25T08:39:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.461 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064448 s, 163 MB/s 2024-04-25T08:39:39.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.468 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.468 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606044 s, 173 MB/s 2024-04-25T08:39:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.475 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614217 s, 171 MB/s 2024-04-25T08:39:39.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626799 s, 167 MB/s 2024-04-25T08:39:39.489 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.490 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656698 s, 160 MB/s 2024-04-25T08:39:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.496 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621132 s, 169 MB/s 2024-04-25T08:39:39.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621047 s, 169 MB/s 2024-04-25T08:39:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639887 s, 164 MB/s 2024-04-25T08:39:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.518 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644489 s, 163 MB/s 2024-04-25T08:39:39.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.525 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594739 s, 176 MB/s 2024-04-25T08:39:39.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.534 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.534 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739297 s, 142 MB/s 2024-04-25T08:39:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.541 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641327 s, 164 MB/s 2024-04-25T08:39:39.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.549 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699167 s, 150 MB/s 2024-04-25T08:39:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.557 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728893 s, 144 MB/s 2024-04-25T08:39:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.564 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064523 s, 163 MB/s 2024-04-25T08:39:39.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.572 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.572 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073358 s, 143 MB/s 2024-04-25T08:39:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716775 s, 146 MB/s 2024-04-25T08:39:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.588 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693896 s, 151 MB/s 2024-04-25T08:39:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.596 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653244 s, 161 MB/s 2024-04-25T08:39:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.603 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613927 s, 171 MB/s 2024-04-25T08:39:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.610 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611977 s, 171 MB/s 2024-04-25T08:39:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.617 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616947 s, 170 MB/s 2024-04-25T08:39:39.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.624 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062746 s, 167 MB/s 2024-04-25T08:39:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.631 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063606 s, 165 MB/s 2024-04-25T08:39:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580795 s, 181 MB/s 2024-04-25T08:39:39.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.646 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671705 s, 156 MB/s 2024-04-25T08:39:39.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.653 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598075 s, 175 MB/s 2024-04-25T08:39:39.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.660 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637296 s, 165 MB/s 2024-04-25T08:39:39.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607538 s, 173 MB/s 2024-04-25T08:39:39.675 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.675 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.675 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072891 s, 144 MB/s 2024-04-25T08:39:39.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.682 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0059707 s, 176 MB/s 2024-04-25T08:39:39.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:39.683 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:39.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.747 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623445 s, 168 MB/s 2024-04-25T08:39:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.754 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600067 s, 175 MB/s 2024-04-25T08:39:39.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609805 s, 172 MB/s 2024-04-25T08:39:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636739 s, 165 MB/s 2024-04-25T08:39:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.775 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00576934 s, 182 MB/s 2024-04-25T08:39:39.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.781 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.781 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542985 s, 193 MB/s 2024-04-25T08:39:39.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.788 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.788 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617931 s, 170 MB/s 2024-04-25T08:39:39.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.795 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.795 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613341 s, 171 MB/s 2024-04-25T08:39:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.802 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618431 s, 170 MB/s 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638818 s, 164 MB/s 2024-04-25T08:39:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.817 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627667 s, 167 MB/s 2024-04-25T08:39:39.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620997 s, 169 MB/s 2024-04-25T08:39:39.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.832 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741267 s, 141 MB/s 2024-04-25T08:39:39.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.839 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605268 s, 173 MB/s 2024-04-25T08:39:39.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.846 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.846 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626909 s, 167 MB/s 2024-04-25T08:39:39.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.853 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.853 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560381 s, 187 MB/s 2024-04-25T08:39:39.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700069 s, 150 MB/s 2024-04-25T08:39:39.867 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.868 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602169 s, 174 MB/s 2024-04-25T08:39:39.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724706 s, 145 MB/s 2024-04-25T08:39:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.883 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637587 s, 164 MB/s 2024-04-25T08:39:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604514 s, 173 MB/s 2024-04-25T08:39:39.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070672 s, 148 MB/s 2024-04-25T08:39:39.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634173 s, 165 MB/s 2024-04-25T08:39:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619119 s, 169 MB/s 2024-04-25T08:39:39.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.920 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684606 s, 153 MB/s 2024-04-25T08:39:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601995 s, 174 MB/s 2024-04-25T08:39:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.935 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692822 s, 151 MB/s 2024-04-25T08:39:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.943 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071275 s, 147 MB/s 2024-04-25T08:39:39.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.951 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.951 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721161 s, 145 MB/s 2024-04-25T08:39:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610909 s, 172 MB/s 2024-04-25T08:39:39.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.965 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.965 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647709 s, 162 MB/s 2024-04-25T08:39:39.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:39.973 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:39.973 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711093 s, 147 MB/s 2024-04-25T08:39:39.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:39.974 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:39.974 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:39.975 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:39:39.975 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:39.975 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:39.975 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:39.975 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0057826 s, 181 MB/s 2024-04-25T08:39:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627094 s, 167 MB/s 2024-04-25T08:39:40.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.052 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.053 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630854 s, 166 MB/s 2024-04-25T08:39:40.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.060 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618462 s, 170 MB/s 2024-04-25T08:39:40.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.068 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.068 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754297 s, 139 MB/s 2024-04-25T08:39:40.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.075 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.075 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635122 s, 165 MB/s 2024-04-25T08:39:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.089 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125151 s, 83.8 MB/s 2024-04-25T08:39:40.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.096 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627959 s, 167 MB/s 2024-04-25T08:39:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611189 s, 172 MB/s 2024-04-25T08:39:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707854 s, 148 MB/s 2024-04-25T08:39:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.118 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696529 s, 151 MB/s 2024-04-25T08:39:40.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.126 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633569 s, 166 MB/s 2024-04-25T08:39:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062359 s, 168 MB/s 2024-04-25T08:39:40.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.141 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.141 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710382 s, 148 MB/s 2024-04-25T08:39:40.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.149 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720352 s, 146 MB/s 2024-04-25T08:39:40.156 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.157 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686797 s, 153 MB/s 2024-04-25T08:39:40.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.164 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625237 s, 168 MB/s 2024-04-25T08:39:40.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.170 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577008 s, 182 MB/s 2024-04-25T08:39:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.178 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688058 s, 152 MB/s 2024-04-25T08:39:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.185 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622118 s, 169 MB/s 2024-04-25T08:39:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.193 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627064 s, 167 MB/s 2024-04-25T08:39:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.200 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619138 s, 169 MB/s 2024-04-25T08:39:40.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.207 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.207 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606129 s, 173 MB/s 2024-04-25T08:39:40.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595637 s, 176 MB/s 2024-04-25T08:39:40.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.221 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651461 s, 161 MB/s 2024-04-25T08:39:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.228 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633187 s, 166 MB/s 2024-04-25T08:39:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705687 s, 149 MB/s 2024-04-25T08:39:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613697 s, 171 MB/s 2024-04-25T08:39:40.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.251 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.251 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707783 s, 148 MB/s 2024-04-25T08:39:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.259 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705554 s, 149 MB/s 2024-04-25T08:39:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.266 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585126 s, 179 MB/s 2024-04-25T08:39:40.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.273 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.273 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645701 s, 162 MB/s 2024-04-25T08:39:40.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:40.274 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:40.275 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:40.275 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:39:40.275 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:40.275 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:40.275 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:40.275 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:40.340 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.341 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.341 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759217 s, 138 MB/s 2024-04-25T08:39:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.348 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646542 s, 162 MB/s 2024-04-25T08:39:40.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.355 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.355 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670454 s, 156 MB/s 2024-04-25T08:39:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.363 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706748 s, 148 MB/s 2024-04-25T08:39:40.370 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.371 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064594 s, 162 MB/s 2024-04-25T08:39:40.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069878 s, 150 MB/s 2024-04-25T08:39:40.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.386 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619613 s, 169 MB/s 2024-04-25T08:39:40.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.394 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.394 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705389 s, 149 MB/s 2024-04-25T08:39:40.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.401 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.401 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647233 s, 162 MB/s 2024-04-25T08:39:40.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575574 s, 182 MB/s 2024-04-25T08:39:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.415 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624728 s, 168 MB/s 2024-04-25T08:39:40.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670837 s, 156 MB/s 2024-04-25T08:39:40.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.430 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700641 s, 150 MB/s 2024-04-25T08:39:40.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061873 s, 169 MB/s 2024-04-25T08:39:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.445 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062414 s, 168 MB/s 2024-04-25T08:39:40.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603863 s, 174 MB/s 2024-04-25T08:39:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.459 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637874 s, 164 MB/s 2024-04-25T08:39:40.465 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.466 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598547 s, 175 MB/s 2024-04-25T08:39:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.472 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00587196 s, 179 MB/s 2024-04-25T08:39:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.479 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586626 s, 179 MB/s 2024-04-25T08:39:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670439 s, 156 MB/s 2024-04-25T08:39:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071364 s, 147 MB/s 2024-04-25T08:39:40.502 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.503 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690264 s, 152 MB/s 2024-04-25T08:39:40.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.510 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.510 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625401 s, 168 MB/s 2024-04-25T08:39:40.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.517 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.517 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591442 s, 177 MB/s 2024-04-25T08:39:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.524 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600481 s, 175 MB/s 2024-04-25T08:39:40.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.532 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723149 s, 145 MB/s 2024-04-25T08:39:40.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733867 s, 143 MB/s 2024-04-25T08:39:40.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.548 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715917 s, 146 MB/s 2024-04-25T08:39:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.555 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607385 s, 173 MB/s 2024-04-25T08:39:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.563 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734949 s, 143 MB/s 2024-04-25T08:39:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.571 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636424 s, 165 MB/s 2024-04-25T08:39:40.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:40.572 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:40.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627873 s, 167 MB/s 2024-04-25T08:39:40.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659809 s, 159 MB/s 2024-04-25T08:39:40.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566076 s, 185 MB/s 2024-04-25T08:39:40.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643268 s, 163 MB/s 2024-04-25T08:39:40.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575492 s, 182 MB/s 2024-04-25T08:39:40.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730675 s, 144 MB/s 2024-04-25T08:39:40.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.681 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598374 s, 175 MB/s 2024-04-25T08:39:40.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.688 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616219 s, 170 MB/s 2024-04-25T08:39:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.695 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623393 s, 168 MB/s 2024-04-25T08:39:40.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.702 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.702 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687029 s, 153 MB/s 2024-04-25T08:39:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.709 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564897 s, 186 MB/s 2024-04-25T08:39:40.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631504 s, 166 MB/s 2024-04-25T08:39:40.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.723 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.723 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060853 s, 172 MB/s 2024-04-25T08:39:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.732 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759506 s, 138 MB/s 2024-04-25T08:39:40.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.739 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652317 s, 161 MB/s 2024-04-25T08:39:40.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.747 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656292 s, 160 MB/s 2024-04-25T08:39:40.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596716 s, 176 MB/s 2024-04-25T08:39:40.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634093 s, 165 MB/s 2024-04-25T08:39:40.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.769 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676957 s, 155 MB/s 2024-04-25T08:39:40.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.776 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00622943 s, 168 MB/s 2024-04-25T08:39:40.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.782 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.782 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563694 s, 186 MB/s 2024-04-25T08:39:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.789 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581222 s, 180 MB/s 2024-04-25T08:39:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.796 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615904 s, 170 MB/s 2024-04-25T08:39:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.803 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618452 s, 170 MB/s 2024-04-25T08:39:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.810 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571255 s, 184 MB/s 2024-04-25T08:39:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.817 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061312 s, 171 MB/s 2024-04-25T08:39:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655143 s, 160 MB/s 2024-04-25T08:39:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.832 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706011 s, 149 MB/s 2024-04-25T08:39:40.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.839 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.839 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610776 s, 172 MB/s 2024-04-25T08:39:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.847 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673806 s, 156 MB/s 2024-04-25T08:39:40.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.854 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.854 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667552 s, 157 MB/s 2024-04-25T08:39:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.861 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604781 s, 173 MB/s 2024-04-25T08:39:40.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:40.863 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:40.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.928 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.928 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719143 s, 146 MB/s 2024-04-25T08:39:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.936 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747149 s, 140 MB/s 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713115 s, 147 MB/s 2024-04-25T08:39:40.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.953 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073746 s, 142 MB/s 2024-04-25T08:39:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.960 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692312 s, 151 MB/s 2024-04-25T08:39:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.967 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630105 s, 166 MB/s 2024-04-25T08:39:40.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.974 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579475 s, 181 MB/s 2024-04-25T08:39:40.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658081 s, 159 MB/s 2024-04-25T08:39:40.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.990 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741198 s, 141 MB/s 2024-04-25T08:39:40.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:40.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:40.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663191 s, 158 MB/s 2024-04-25T08:39:41.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.006 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.006 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755915 s, 139 MB/s 2024-04-25T08:39:41.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.014 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.014 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706726 s, 148 MB/s 2024-04-25T08:39:41.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.022 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.022 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704764 s, 149 MB/s 2024-04-25T08:39:41.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.030 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.030 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729433 s, 144 MB/s 2024-04-25T08:39:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.038 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707714 s, 148 MB/s 2024-04-25T08:39:41.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.045 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.045 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649225 s, 162 MB/s 2024-04-25T08:39:41.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.051 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.052 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552974 s, 190 MB/s 2024-04-25T08:39:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.059 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626538 s, 167 MB/s 2024-04-25T08:39:41.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.066 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.066 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654267 s, 160 MB/s 2024-04-25T08:39:41.074 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.074 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.074 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705021 s, 149 MB/s 2024-04-25T08:39:41.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.081 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.081 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640056 s, 164 MB/s 2024-04-25T08:39:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006041 s, 174 MB/s 2024-04-25T08:39:41.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.096 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.096 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623851 s, 168 MB/s 2024-04-25T08:39:41.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615032 s, 170 MB/s 2024-04-25T08:39:41.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.110 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701342 s, 150 MB/s 2024-04-25T08:39:41.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.117 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.117 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598093 s, 175 MB/s 2024-04-25T08:39:41.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.125 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629491 s, 167 MB/s 2024-04-25T08:39:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.132 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691069 s, 152 MB/s 2024-04-25T08:39:41.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.140 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677766 s, 155 MB/s 2024-04-25T08:39:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706603 s, 148 MB/s 2024-04-25T08:39:41.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.155 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618855 s, 169 MB/s 2024-04-25T08:39:41.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.162 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636977 s, 165 MB/s 2024-04-25T08:39:41.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:41.163 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:41.163 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:41.163 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:39:41.164 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:41.164 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:41.164 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:41.164 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:41.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.227 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621345 s, 169 MB/s 2024-04-25T08:39:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.235 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698621 s, 150 MB/s 2024-04-25T08:39:41.241 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.241 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.241 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578133 s, 181 MB/s 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641863 s, 163 MB/s 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570748 s, 184 MB/s 2024-04-25T08:39:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.263 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616283 s, 170 MB/s 2024-04-25T08:39:41.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.270 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600184 s, 175 MB/s 2024-04-25T08:39:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672691 s, 156 MB/s 2024-04-25T08:39:41.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00578702 s, 181 MB/s 2024-04-25T08:39:41.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.292 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727628 s, 144 MB/s 2024-04-25T08:39:41.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621329 s, 169 MB/s 2024-04-25T08:39:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.306 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.307 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633967 s, 165 MB/s 2024-04-25T08:39:41.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695361 s, 151 MB/s 2024-04-25T08:39:41.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.321 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.321 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639968 s, 164 MB/s 2024-04-25T08:39:41.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629995 s, 166 MB/s 2024-04-25T08:39:41.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.337 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.337 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713053 s, 147 MB/s 2024-04-25T08:39:41.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648821 s, 162 MB/s 2024-04-25T08:39:41.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.351 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.351 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592943 s, 177 MB/s 2024-04-25T08:39:41.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.358 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574545 s, 183 MB/s 2024-04-25T08:39:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.365 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634803 s, 165 MB/s 2024-04-25T08:39:41.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.372 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583433 s, 180 MB/s 2024-04-25T08:39:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632866 s, 166 MB/s 2024-04-25T08:39:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.386 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063176 s, 166 MB/s 2024-04-25T08:39:41.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.393 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607435 s, 173 MB/s 2024-04-25T08:39:41.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.400 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066126 s, 159 MB/s 2024-04-25T08:39:41.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.408 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.408 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704153 s, 149 MB/s 2024-04-25T08:39:41.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.415 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.416 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627098 s, 167 MB/s 2024-04-25T08:39:41.422 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.423 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.423 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628895 s, 167 MB/s 2024-04-25T08:39:41.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.430 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688377 s, 152 MB/s 2024-04-25T08:39:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592598 s, 177 MB/s 2024-04-25T08:39:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00558349 s, 188 MB/s 2024-04-25T08:39:41.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.451 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618339 s, 170 MB/s 2024-04-25T08:39:41.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:41.452 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:41.452 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:41.452 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:39:41.452 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:41.452 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:41.453 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:41.453 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757309 s, 138 MB/s 2024-04-25T08:39:41.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.527 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.527 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780044 s, 134 MB/s 2024-04-25T08:39:41.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.535 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694117 s, 151 MB/s 2024-04-25T08:39:41.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.542 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.543 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604323 s, 174 MB/s 2024-04-25T08:39:41.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.549 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.550 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618612 s, 170 MB/s 2024-04-25T08:39:41.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.556 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.556 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604129 s, 174 MB/s 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710673 s, 148 MB/s 2024-04-25T08:39:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.571 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598892 s, 175 MB/s 2024-04-25T08:39:41.578 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.579 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.579 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633244 s, 166 MB/s 2024-04-25T08:39:41.586 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.587 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712964 s, 147 MB/s 2024-04-25T08:39:41.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.594 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670331 s, 156 MB/s 2024-04-25T08:39:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.602 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666679 s, 157 MB/s 2024-04-25T08:39:41.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.609 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.609 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00601032 s, 174 MB/s 2024-04-25T08:39:41.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.616 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.616 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656238 s, 160 MB/s 2024-04-25T08:39:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.624 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719564 s, 146 MB/s 2024-04-25T08:39:41.631 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.632 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642473 s, 163 MB/s 2024-04-25T08:39:41.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.638 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056783 s, 185 MB/s 2024-04-25T08:39:41.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.645 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645246 s, 163 MB/s 2024-04-25T08:39:41.652 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.653 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.653 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615278 s, 170 MB/s 2024-04-25T08:39:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.660 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722583 s, 145 MB/s 2024-04-25T08:39:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00600168 s, 175 MB/s 2024-04-25T08:39:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613362 s, 171 MB/s 2024-04-25T08:39:41.681 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.682 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626747 s, 167 MB/s 2024-04-25T08:39:41.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.689 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610701 s, 172 MB/s 2024-04-25T08:39:41.696 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.697 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070382 s, 149 MB/s 2024-04-25T08:39:41.704 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.704 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.704 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693321 s, 151 MB/s 2024-04-25T08:39:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.712 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623168 s, 168 MB/s 2024-04-25T08:39:41.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.719 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618397 s, 170 MB/s 2024-04-25T08:39:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.726 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062804 s, 167 MB/s 2024-04-25T08:39:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610964 s, 172 MB/s 2024-04-25T08:39:41.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.740 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617649 s, 170 MB/s 2024-04-25T08:39:41.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.747 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639613 s, 164 MB/s 2024-04-25T08:39:41.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:41.748 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:41.748 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:41.749 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:39:41.749 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:41.749 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:41.749 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:41.749 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.813 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623288 s, 168 MB/s 2024-04-25T08:39:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.820 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652202 s, 161 MB/s 2024-04-25T08:39:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.826 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055874 s, 188 MB/s 2024-04-25T08:39:41.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.834 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704395 s, 149 MB/s 2024-04-25T08:39:41.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.842 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.842 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624539 s, 168 MB/s 2024-04-25T08:39:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058744 s, 178 MB/s 2024-04-25T08:39:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.855 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.856 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620014 s, 169 MB/s 2024-04-25T08:39:41.862 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.863 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611016 s, 172 MB/s 2024-04-25T08:39:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.869 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608306 s, 172 MB/s 2024-04-25T08:39:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.877 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656034 s, 160 MB/s 2024-04-25T08:39:41.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.884 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.884 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638895 s, 164 MB/s 2024-04-25T08:39:41.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.890 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00563715 s, 186 MB/s 2024-04-25T08:39:41.897 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.898 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.898 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619279 s, 169 MB/s 2024-04-25T08:39:41.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.905 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.905 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631796 s, 166 MB/s 2024-04-25T08:39:41.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.913 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.913 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695522 s, 151 MB/s 2024-04-25T08:39:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.920 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.921 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069927 s, 150 MB/s 2024-04-25T08:39:41.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.927 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.927 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583874 s, 180 MB/s 2024-04-25T08:39:41.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.934 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.934 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620357 s, 169 MB/s 2024-04-25T08:39:41.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.941 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564887 s, 186 MB/s 2024-04-25T08:39:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00553173 s, 190 MB/s 2024-04-25T08:39:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.954 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615556 s, 170 MB/s 2024-04-25T08:39:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.961 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00534759 s, 196 MB/s 2024-04-25T08:39:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.968 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621417 s, 169 MB/s 2024-04-25T08:39:41.975 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.975 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.975 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614237 s, 171 MB/s 2024-04-25T08:39:41.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.982 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589558 s, 178 MB/s 2024-04-25T08:39:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564972 s, 186 MB/s 2024-04-25T08:39:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:41.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068802 s, 152 MB/s 2024-04-25T08:39:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.003 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596175 s, 176 MB/s 2024-04-25T08:39:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.010 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632738 s, 166 MB/s 2024-04-25T08:39:42.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669702 s, 157 MB/s 2024-04-25T08:39:42.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690875 s, 152 MB/s 2024-04-25T08:39:42.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.033 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.033 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687361 s, 153 MB/s 2024-04-25T08:39:42.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:42.035 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:42.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.101 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735177 s, 143 MB/s 2024-04-25T08:39:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.108 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573409 s, 183 MB/s 2024-04-25T08:39:42.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.115 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628186 s, 167 MB/s 2024-04-25T08:39:42.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.122 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.122 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630111 s, 166 MB/s 2024-04-25T08:39:42.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.129 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.129 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609604 s, 172 MB/s 2024-04-25T08:39:42.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.136 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.137 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630966 s, 166 MB/s 2024-04-25T08:39:42.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.143 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.143 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612776 s, 171 MB/s 2024-04-25T08:39:42.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.151 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.151 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627683 s, 167 MB/s 2024-04-25T08:39:42.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.157 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.157 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00577546 s, 182 MB/s 2024-04-25T08:39:42.164 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.165 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611215 s, 172 MB/s 2024-04-25T08:39:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.171 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630354 s, 166 MB/s 2024-04-25T08:39:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.178 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061522 s, 170 MB/s 2024-04-25T08:39:42.185 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.186 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.186 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618459 s, 170 MB/s 2024-04-25T08:39:42.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.193 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.193 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645588 s, 162 MB/s 2024-04-25T08:39:42.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.200 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627902 s, 167 MB/s 2024-04-25T08:39:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.208 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714072 s, 147 MB/s 2024-04-25T08:39:42.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.215 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.215 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634812 s, 165 MB/s 2024-04-25T08:39:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.223 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652809 s, 161 MB/s 2024-04-25T08:39:42.230 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.231 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.231 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691626 s, 152 MB/s 2024-04-25T08:39:42.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.239 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.239 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725344 s, 145 MB/s 2024-04-25T08:39:42.246 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.247 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.247 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695148 s, 151 MB/s 2024-04-25T08:39:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.253 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568197 s, 185 MB/s 2024-04-25T08:39:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.260 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617562 s, 170 MB/s 2024-04-25T08:39:42.267 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.268 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647072 s, 162 MB/s 2024-04-25T08:39:42.275 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.276 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716417 s, 146 MB/s 2024-04-25T08:39:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.283 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637753 s, 164 MB/s 2024-04-25T08:39:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.289 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567915 s, 185 MB/s 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075558 s, 139 MB/s 2024-04-25T08:39:42.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.305 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.305 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612299 s, 171 MB/s 2024-04-25T08:39:42.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.312 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.312 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064318 s, 163 MB/s 2024-04-25T08:39:42.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.320 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.320 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720135 s, 146 MB/s 2024-04-25T08:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720013 s, 146 MB/s 2024-04-25T08:39:42.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:42.330 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699525 s, 150 MB/s 2024-04-25T08:39:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.403 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620793 s, 169 MB/s 2024-04-25T08:39:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.410 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655731 s, 160 MB/s 2024-04-25T08:39:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710814 s, 148 MB/s 2024-04-25T08:39:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709775 s, 148 MB/s 2024-04-25T08:39:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.433 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608302 s, 172 MB/s 2024-04-25T08:39:42.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.440 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589236 s, 178 MB/s 2024-04-25T08:39:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.447 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061017 s, 172 MB/s 2024-04-25T08:39:42.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.455 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704972 s, 149 MB/s 2024-04-25T08:39:42.462 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.463 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640957 s, 164 MB/s 2024-04-25T08:39:42.469 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.470 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00605987 s, 173 MB/s 2024-04-25T08:39:42.476 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621532 s, 169 MB/s 2024-04-25T08:39:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.485 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708678 s, 148 MB/s 2024-04-25T08:39:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.492 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.493 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710532 s, 148 MB/s 2024-04-25T08:39:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.500 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610428 s, 172 MB/s 2024-04-25T08:39:42.507 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.508 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713423 s, 147 MB/s 2024-04-25T08:39:42.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.515 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.515 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674612 s, 155 MB/s 2024-04-25T08:39:42.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.522 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.522 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625241 s, 168 MB/s 2024-04-25T08:39:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.529 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602729 s, 174 MB/s 2024-04-25T08:39:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.536 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00579632 s, 181 MB/s 2024-04-25T08:39:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.544 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750831 s, 140 MB/s 2024-04-25T08:39:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.552 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065716 s, 160 MB/s 2024-04-25T08:39:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.560 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694379 s, 151 MB/s 2024-04-25T08:39:42.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726507 s, 144 MB/s 2024-04-25T08:39:42.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.576 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722961 s, 145 MB/s 2024-04-25T08:39:42.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646152 s, 162 MB/s 2024-04-25T08:39:42.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.592 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.592 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697416 s, 150 MB/s 2024-04-25T08:39:42.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.599 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662881 s, 158 MB/s 2024-04-25T08:39:42.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.606 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624336 s, 168 MB/s 2024-04-25T08:39:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.614 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661451 s, 159 MB/s 2024-04-25T08:39:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594824 s, 176 MB/s 2024-04-25T08:39:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648029 s, 162 MB/s 2024-04-25T08:39:42.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:42.629 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:42.629 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:42.629 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:39:42.629 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:42.629 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:42.629 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:42.630 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628267 s, 167 MB/s 2024-04-25T08:39:42.700 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.701 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635772 s, 165 MB/s 2024-04-25T08:39:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.708 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630263 s, 166 MB/s 2024-04-25T08:39:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.716 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713698 s, 147 MB/s 2024-04-25T08:39:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.724 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714261 s, 147 MB/s 2024-04-25T08:39:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.732 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682898 s, 154 MB/s 2024-04-25T08:39:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699858 s, 150 MB/s 2024-04-25T08:39:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.747 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687832 s, 152 MB/s 2024-04-25T08:39:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.754 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.755 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647212 s, 162 MB/s 2024-04-25T08:39:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.762 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615512 s, 170 MB/s 2024-04-25T08:39:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.769 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668858 s, 157 MB/s 2024-04-25T08:39:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.777 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703769 s, 149 MB/s 2024-04-25T08:39:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.785 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670919 s, 156 MB/s 2024-04-25T08:39:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.792 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619266 s, 169 MB/s 2024-04-25T08:39:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.799 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595917 s, 176 MB/s 2024-04-25T08:39:42.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.807 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067811 s, 155 MB/s 2024-04-25T08:39:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.814 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616419 s, 170 MB/s 2024-04-25T08:39:42.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.822 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.822 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699691 s, 150 MB/s 2024-04-25T08:39:42.828 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.829 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.829 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00603981 s, 174 MB/s 2024-04-25T08:39:42.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.836 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.836 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621187 s, 169 MB/s 2024-04-25T08:39:42.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.843 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639749 s, 164 MB/s 2024-04-25T08:39:42.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.850 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.850 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646993 s, 162 MB/s 2024-04-25T08:39:42.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.857 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0056881 s, 184 MB/s 2024-04-25T08:39:42.864 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.864 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.864 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610288 s, 172 MB/s 2024-04-25T08:39:42.871 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.871 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.872 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669692 s, 157 MB/s 2024-04-25T08:39:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.878 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590668 s, 178 MB/s 2024-04-25T08:39:42.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.886 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.886 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666568 s, 157 MB/s 2024-04-25T08:39:42.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.893 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.893 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656418 s, 160 MB/s 2024-04-25T08:39:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.900 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604685 s, 173 MB/s 2024-04-25T08:39:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.907 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615129 s, 170 MB/s 2024-04-25T08:39:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.914 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586948 s, 179 MB/s 2024-04-25T08:39:42.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.921 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.921 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618566 s, 170 MB/s 2024-04-25T08:39:42.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:42.922 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:42.922 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:42.923 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:39:42.923 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:42.923 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:42.923 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:42.923 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.987 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711208 s, 147 MB/s 2024-04-25T08:39:42.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:42.995 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:42.995 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690314 s, 152 MB/s 2024-04-25T08:39:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.003 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748042 s, 140 MB/s 2024-04-25T08:39:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.010 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00609706 s, 172 MB/s 2024-04-25T08:39:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712011 s, 147 MB/s 2024-04-25T08:39:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.025 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641773 s, 163 MB/s 2024-04-25T08:39:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.032 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574248 s, 183 MB/s 2024-04-25T08:39:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.040 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729823 s, 144 MB/s 2024-04-25T08:39:43.048 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.049 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737888 s, 142 MB/s 2024-04-25T08:39:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.055 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00561321 s, 187 MB/s 2024-04-25T08:39:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.062 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063079 s, 166 MB/s 2024-04-25T08:39:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.070 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639993 s, 164 MB/s 2024-04-25T08:39:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.076 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055296 s, 190 MB/s 2024-04-25T08:39:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.083 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598794 s, 175 MB/s 2024-04-25T08:39:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.091 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667147 s, 157 MB/s 2024-04-25T08:39:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.099 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753802 s, 139 MB/s 2024-04-25T08:39:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.107 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715159 s, 147 MB/s 2024-04-25T08:39:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.115 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635257 s, 165 MB/s 2024-04-25T08:39:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.121 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595868 s, 176 MB/s 2024-04-25T08:39:43.128 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.128 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.128 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054859 s, 191 MB/s 2024-04-25T08:39:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.135 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640566 s, 164 MB/s 2024-04-25T08:39:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.142 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652584 s, 161 MB/s 2024-04-25T08:39:43.149 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.150 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610488 s, 172 MB/s 2024-04-25T08:39:43.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.158 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.158 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730953 s, 143 MB/s 2024-04-25T08:39:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.165 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663668 s, 158 MB/s 2024-04-25T08:39:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.172 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.173 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627564 s, 167 MB/s 2024-04-25T08:39:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.180 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659184 s, 159 MB/s 2024-04-25T08:39:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.187 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610692 s, 172 MB/s 2024-04-25T08:39:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.194 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645423 s, 162 MB/s 2024-04-25T08:39:43.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.202 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.202 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070897 s, 148 MB/s 2024-04-25T08:39:43.209 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.210 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667621 s, 157 MB/s 2024-04-25T08:39:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.217 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642039 s, 163 MB/s 2024-04-25T08:39:43.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:43.218 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:43.218 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:43.218 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:39:43.218 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:43.218 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:43.218 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:43.219 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:43.285 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.286 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616216 s, 170 MB/s 2024-04-25T08:39:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.293 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614588 s, 171 MB/s 2024-04-25T08:39:43.300 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.300 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.300 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706373 s, 148 MB/s 2024-04-25T08:39:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594205 s, 176 MB/s 2024-04-25T08:39:43.315 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.315 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.315 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715423 s, 147 MB/s 2024-04-25T08:39:43.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.322 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629169 s, 167 MB/s 2024-04-25T08:39:43.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612434 s, 171 MB/s 2024-04-25T08:39:43.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.336 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060564 s, 173 MB/s 2024-04-25T08:39:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694434 s, 151 MB/s 2024-04-25T08:39:43.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.352 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.352 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661362 s, 159 MB/s 2024-04-25T08:39:43.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.359 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.359 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646034 s, 162 MB/s 2024-04-25T08:39:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.367 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724322 s, 145 MB/s 2024-04-25T08:39:43.374 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.375 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637766 s, 164 MB/s 2024-04-25T08:39:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.382 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638819 s, 164 MB/s 2024-04-25T08:39:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.389 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628461 s, 167 MB/s 2024-04-25T08:39:43.395 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.396 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571812 s, 183 MB/s 2024-04-25T08:39:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.403 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.404 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690134 s, 152 MB/s 2024-04-25T08:39:43.411 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.411 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.411 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695442 s, 151 MB/s 2024-04-25T08:39:43.418 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.419 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636813 s, 165 MB/s 2024-04-25T08:39:43.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.426 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.426 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062496 s, 168 MB/s 2024-04-25T08:39:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.433 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619281 s, 169 MB/s 2024-04-25T08:39:43.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.440 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.440 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061445 s, 171 MB/s 2024-04-25T08:39:43.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.448 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.448 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070061 s, 150 MB/s 2024-04-25T08:39:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.455 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618079 s, 170 MB/s 2024-04-25T08:39:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.466 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698587 s, 150 MB/s 2024-04-25T08:39:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.470 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627661 s, 167 MB/s 2024-04-25T08:39:43.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.477 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.477 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063392 s, 165 MB/s 2024-04-25T08:39:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.484 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00629617 s, 167 MB/s 2024-04-25T08:39:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060702 s, 173 MB/s 2024-04-25T08:39:43.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.499 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.499 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663497 s, 158 MB/s 2024-04-25T08:39:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.506 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612393 s, 171 MB/s 2024-04-25T08:39:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.514 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722993 s, 145 MB/s 2024-04-25T08:39:43.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:43.515 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:43.515 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:43.515 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:39:43.516 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:43.516 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:43.516 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:43.516 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638987 s, 164 MB/s 2024-04-25T08:39:43.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.587 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.587 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618937 s, 169 MB/s 2024-04-25T08:39:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.594 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00613657 s, 171 MB/s 2024-04-25T08:39:43.600 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.600 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.600 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572164 s, 183 MB/s 2024-04-25T08:39:43.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.607 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.608 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00632466 s, 166 MB/s 2024-04-25T08:39:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.614 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.615 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620365 s, 169 MB/s 2024-04-25T08:39:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606863 s, 173 MB/s 2024-04-25T08:39:43.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.630 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.630 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742389 s, 141 MB/s 2024-04-25T08:39:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.638 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685928 s, 153 MB/s 2024-04-25T08:39:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.644 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.645 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607594 s, 173 MB/s 2024-04-25T08:39:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.651 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582052 s, 180 MB/s 2024-04-25T08:39:43.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.659 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.659 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723056 s, 145 MB/s 2024-04-25T08:39:43.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.666 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.666 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006017 s, 174 MB/s 2024-04-25T08:39:43.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.674 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.674 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691541 s, 152 MB/s 2024-04-25T08:39:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.682 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706697 s, 148 MB/s 2024-04-25T08:39:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.690 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713603 s, 147 MB/s 2024-04-25T08:39:43.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.697 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.697 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00631689 s, 166 MB/s 2024-04-25T08:39:43.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.705 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.705 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648984 s, 162 MB/s 2024-04-25T08:39:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.712 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614418 s, 171 MB/s 2024-04-25T08:39:43.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.718 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.719 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583307 s, 180 MB/s 2024-04-25T08:39:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.726 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645165 s, 163 MB/s 2024-04-25T08:39:43.732 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.733 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.733 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566453 s, 185 MB/s 2024-04-25T08:39:43.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.740 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.740 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633695 s, 165 MB/s 2024-04-25T08:39:43.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.748 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.748 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713344 s, 147 MB/s 2024-04-25T08:39:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620057 s, 169 MB/s 2024-04-25T08:39:43.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.762 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.762 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646174 s, 162 MB/s 2024-04-25T08:39:43.770 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.770 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.770 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721784 s, 145 MB/s 2024-04-25T08:39:43.777 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.778 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621632 s, 169 MB/s 2024-04-25T08:39:43.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.784 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.784 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591148 s, 177 MB/s 2024-04-25T08:39:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.792 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636694 s, 165 MB/s 2024-04-25T08:39:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.799 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612007 s, 171 MB/s 2024-04-25T08:39:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.807 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709378 s, 148 MB/s 2024-04-25T08:39:43.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:43.811 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.876 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618825 s, 169 MB/s 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574993 s, 182 MB/s 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.890 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635335 s, 165 MB/s 2024-04-25T08:39:43.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.896 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607831 s, 173 MB/s 2024-04-25T08:39:43.903 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617703 s, 170 MB/s 2024-04-25T08:39:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.911 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062317 s, 168 MB/s 2024-04-25T08:39:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.918 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638413 s, 164 MB/s 2024-04-25T08:39:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.925 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614849 s, 171 MB/s 2024-04-25T08:39:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.932 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628813 s, 167 MB/s 2024-04-25T08:39:43.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.939 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.939 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611347 s, 172 MB/s 2024-04-25T08:39:43.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.946 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.946 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062079 s, 169 MB/s 2024-04-25T08:39:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.953 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630255 s, 166 MB/s 2024-04-25T08:39:43.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.960 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.960 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610049 s, 172 MB/s 2024-04-25T08:39:43.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.967 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.967 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602893 s, 174 MB/s 2024-04-25T08:39:43.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.975 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.975 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065197 s, 161 MB/s 2024-04-25T08:39:43.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.982 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.983 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703783 s, 149 MB/s 2024-04-25T08:39:43.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.990 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.990 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694304 s, 151 MB/s 2024-04-25T08:39:43.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:43.997 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:43.997 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061611 s, 170 MB/s 2024-04-25T08:39:44.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00569193 s, 184 MB/s 2024-04-25T08:39:44.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.011 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.011 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624792 s, 168 MB/s 2024-04-25T08:39:44.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.018 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.018 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621815 s, 169 MB/s 2024-04-25T08:39:44.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.026 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.026 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694007 s, 151 MB/s 2024-04-25T08:39:44.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.035 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746402 s, 140 MB/s 2024-04-25T08:39:44.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.043 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740191 s, 142 MB/s 2024-04-25T08:39:44.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581385 s, 180 MB/s 2024-04-25T08:39:44.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.056 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.056 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573775 s, 183 MB/s 2024-04-25T08:39:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.063 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612829 s, 171 MB/s 2024-04-25T08:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.071 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690136 s, 152 MB/s 2024-04-25T08:39:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.078 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651764 s, 161 MB/s 2024-04-25T08:39:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.086 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662125 s, 158 MB/s 2024-04-25T08:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.093 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637213 s, 165 MB/s 2024-04-25T08:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.100 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643254 s, 163 MB/s 2024-04-25T08:39:44.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:44.101 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:44.102 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:44.102 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:39:44.102 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:44.102 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:44.102 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:44.102 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:44.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.167 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633781 s, 165 MB/s 2024-04-25T08:39:44.174 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.175 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.175 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703411 s, 149 MB/s 2024-04-25T08:39:44.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.182 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.182 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666387 s, 157 MB/s 2024-04-25T08:39:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.190 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069782 s, 150 MB/s 2024-04-25T08:39:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685746 s, 153 MB/s 2024-04-25T08:39:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.205 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615763 s, 170 MB/s 2024-04-25T08:39:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667215 s, 157 MB/s 2024-04-25T08:39:44.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.219 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.219 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618401 s, 170 MB/s 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653005 s, 161 MB/s 2024-04-25T08:39:44.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.234 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.234 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651491 s, 161 MB/s 2024-04-25T08:39:44.241 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.241 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.241 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628949 s, 167 MB/s 2024-04-25T08:39:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.248 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607695 s, 173 MB/s 2024-04-25T08:39:44.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.255 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.255 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593256 s, 177 MB/s 2024-04-25T08:39:44.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.262 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.262 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00592368 s, 177 MB/s 2024-04-25T08:39:44.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.269 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.269 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637482 s, 164 MB/s 2024-04-25T08:39:44.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.276 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.277 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628644 s, 167 MB/s 2024-04-25T08:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693968 s, 151 MB/s 2024-04-25T08:39:44.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.291 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.291 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623201 s, 168 MB/s 2024-04-25T08:39:44.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.299 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.299 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703045 s, 149 MB/s 2024-04-25T08:39:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.307 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698653 s, 150 MB/s 2024-04-25T08:39:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.314 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574198 s, 183 MB/s 2024-04-25T08:39:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698819 s, 150 MB/s 2024-04-25T08:39:44.328 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.329 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.329 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596917 s, 176 MB/s 2024-04-25T08:39:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.336 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628444 s, 167 MB/s 2024-04-25T08:39:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.342 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.343 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575029 s, 182 MB/s 2024-04-25T08:39:44.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.349 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.349 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599586 s, 175 MB/s 2024-04-25T08:39:44.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.357 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.357 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693505 s, 151 MB/s 2024-04-25T08:39:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.364 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575907 s, 182 MB/s 2024-04-25T08:39:44.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.371 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.371 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060117 s, 174 MB/s 2024-04-25T08:39:44.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.379 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.379 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707436 s, 148 MB/s 2024-04-25T08:39:44.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.386 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.386 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656911 s, 160 MB/s 2024-04-25T08:39:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.393 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617486 s, 170 MB/s 2024-04-25T08:39:44.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:44.395 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.460 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066114 s, 159 MB/s 2024-04-25T08:39:44.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.467 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599131 s, 175 MB/s 2024-04-25T08:39:44.473 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.473 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.473 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0054757 s, 191 MB/s 2024-04-25T08:39:44.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.482 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.482 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073542 s, 143 MB/s 2024-04-25T08:39:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.490 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070032 s, 150 MB/s 2024-04-25T08:39:44.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.497 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.497 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665036 s, 158 MB/s 2024-04-25T08:39:44.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.504 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.504 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596272 s, 176 MB/s 2024-04-25T08:39:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.512 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714189 s, 147 MB/s 2024-04-25T08:39:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.519 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564825 s, 186 MB/s 2024-04-25T08:39:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.526 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623724 s, 168 MB/s 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066237 s, 158 MB/s 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00566001 s, 185 MB/s 2024-04-25T08:39:44.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.548 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.548 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690424 s, 152 MB/s 2024-04-25T08:39:44.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.555 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.555 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636345 s, 165 MB/s 2024-04-25T08:39:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.562 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066533 s, 158 MB/s 2024-04-25T08:39:44.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.569 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.569 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607032 s, 173 MB/s 2024-04-25T08:39:44.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.577 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.577 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663294 s, 158 MB/s 2024-04-25T08:39:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.584 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611467 s, 171 MB/s 2024-04-25T08:39:44.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.591 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.591 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626229 s, 167 MB/s 2024-04-25T08:39:44.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.599 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.599 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690869 s, 152 MB/s 2024-04-25T08:39:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.606 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644516 s, 163 MB/s 2024-04-25T08:39:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.613 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063183 s, 166 MB/s 2024-04-25T08:39:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.620 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.621 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617751 s, 170 MB/s 2024-04-25T08:39:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.628 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709277 s, 148 MB/s 2024-04-25T08:39:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.635 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614757 s, 171 MB/s 2024-04-25T08:39:44.642 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.643 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637839 s, 164 MB/s 2024-04-25T08:39:44.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.650 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.650 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635951 s, 165 MB/s 2024-04-25T08:39:44.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.657 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.657 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597683 s, 175 MB/s 2024-04-25T08:39:44.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.665 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.665 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703134 s, 149 MB/s 2024-04-25T08:39:44.672 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.673 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.673 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694274 s, 151 MB/s 2024-04-25T08:39:44.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.680 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.680 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653185 s, 161 MB/s 2024-04-25T08:39:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614889 s, 171 MB/s 2024-04-25T08:39:44.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:44.688 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:44.688 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:44.688 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:39:44.688 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:44.689 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:44.689 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:44.689 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621254 s, 169 MB/s 2024-04-25T08:39:44.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713904 s, 147 MB/s 2024-04-25T08:39:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.769 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753611 s, 139 MB/s 2024-04-25T08:39:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.778 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770914 s, 136 MB/s 2024-04-25T08:39:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.785 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067275 s, 156 MB/s 2024-04-25T08:39:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.792 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.793 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624738 s, 168 MB/s 2024-04-25T08:39:44.799 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.800 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619579 s, 169 MB/s 2024-04-25T08:39:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.806 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005889 s, 178 MB/s 2024-04-25T08:39:44.814 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.815 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728779 s, 144 MB/s 2024-04-25T08:39:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.823 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739796 s, 142 MB/s 2024-04-25T08:39:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.830 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657216 s, 160 MB/s 2024-04-25T08:39:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.837 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620823 s, 169 MB/s 2024-04-25T08:39:44.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.844 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00571406 s, 184 MB/s 2024-04-25T08:39:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.852 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652137 s, 161 MB/s 2024-04-25T08:39:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662726 s, 158 MB/s 2024-04-25T08:39:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.868 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773236 s, 136 MB/s 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607518 s, 173 MB/s 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624222 s, 168 MB/s 2024-04-25T08:39:44.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.889 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.889 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058368 s, 180 MB/s 2024-04-25T08:39:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.896 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658307 s, 159 MB/s 2024-04-25T08:39:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.904 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719684 s, 146 MB/s 2024-04-25T08:39:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.912 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00628977 s, 167 MB/s 2024-04-25T08:39:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.919 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635887 s, 165 MB/s 2024-04-25T08:39:44.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.926 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.926 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060835 s, 172 MB/s 2024-04-25T08:39:44.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.933 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.933 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639902 s, 164 MB/s 2024-04-25T08:39:44.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.941 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.941 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705093 s, 149 MB/s 2024-04-25T08:39:44.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.948 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.948 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619765 s, 169 MB/s 2024-04-25T08:39:44.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.955 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.956 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066405 s, 158 MB/s 2024-04-25T08:39:44.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.962 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.962 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597044 s, 176 MB/s 2024-04-25T08:39:44.969 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.970 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618354 s, 170 MB/s 2024-04-25T08:39:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623517 s, 168 MB/s 2024-04-25T08:39:44.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:44.984 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:44.984 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668562 s, 157 MB/s 2024-04-25T08:39:44.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:44.985 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:44.985 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:44.985 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:39:44.985 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:44.985 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:44.986 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:44.986 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:45.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.049 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635715 s, 165 MB/s 2024-04-25T08:39:45.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.058 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073477 s, 143 MB/s 2024-04-25T08:39:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.065 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649265 s, 162 MB/s 2024-04-25T08:39:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00619952 s, 169 MB/s 2024-04-25T08:39:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.080 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717263 s, 146 MB/s 2024-04-25T08:39:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.088 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686632 s, 153 MB/s 2024-04-25T08:39:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064734 s, 162 MB/s 2024-04-25T08:39:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.102 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610787 s, 172 MB/s 2024-04-25T08:39:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.111 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744076 s, 141 MB/s 2024-04-25T08:39:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.118 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00612251 s, 171 MB/s 2024-04-25T08:39:45.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.125 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.125 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00617302 s, 170 MB/s 2024-04-25T08:39:45.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.132 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.133 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698647 s, 150 MB/s 2024-04-25T08:39:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.140 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623363 s, 168 MB/s 2024-04-25T08:39:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.148 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072017 s, 146 MB/s 2024-04-25T08:39:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.155 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651391 s, 161 MB/s 2024-04-25T08:39:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.162 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.163 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063108 s, 166 MB/s 2024-04-25T08:39:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.169 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611944 s, 171 MB/s 2024-04-25T08:39:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.176 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00556101 s, 189 MB/s 2024-04-25T08:39:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.183 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651352 s, 161 MB/s 2024-04-25T08:39:45.191 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.192 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.192 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732814 s, 143 MB/s 2024-04-25T08:39:45.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.198 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.198 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608957 s, 172 MB/s 2024-04-25T08:39:45.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.206 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.206 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006319 s, 166 MB/s 2024-04-25T08:39:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.213 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.214 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716295 s, 146 MB/s 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705954 s, 149 MB/s 2024-04-25T08:39:45.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.229 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.229 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639007 s, 164 MB/s 2024-04-25T08:39:45.235 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.236 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.236 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596335 s, 176 MB/s 2024-04-25T08:39:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062666 s, 167 MB/s 2024-04-25T08:39:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.250 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657444 s, 159 MB/s 2024-04-25T08:39:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.257 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00570978 s, 184 MB/s 2024-04-25T08:39:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.265 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643411 s, 163 MB/s 2024-04-25T08:39:45.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.272 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.273 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725792 s, 144 MB/s 2024-04-25T08:39:45.279 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.280 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.280 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637553 s, 164 MB/s 2024-04-25T08:39:45.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:45.281 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.346 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.347 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719408 s, 146 MB/s 2024-04-25T08:39:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.354 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716441 s, 146 MB/s 2024-04-25T08:39:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.362 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643101 s, 163 MB/s 2024-04-25T08:39:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.369 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638704 s, 164 MB/s 2024-04-25T08:39:45.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.376 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.376 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00610446 s, 172 MB/s 2024-04-25T08:39:45.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.384 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.384 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752971 s, 139 MB/s 2024-04-25T08:39:45.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.392 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.392 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070712 s, 148 MB/s 2024-04-25T08:39:45.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.400 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.400 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694559 s, 151 MB/s 2024-04-25T08:39:45.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.407 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.407 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615848 s, 170 MB/s 2024-04-25T08:39:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.414 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00559573 s, 187 MB/s 2024-04-25T08:39:45.421 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.421 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.421 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060897 s, 172 MB/s 2024-04-25T08:39:45.429 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.429 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.429 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692598 s, 151 MB/s 2024-04-25T08:39:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.436 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.437 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705378 s, 149 MB/s 2024-04-25T08:39:45.443 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.444 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.444 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630443 s, 166 MB/s 2024-04-25T08:39:45.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.451 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.452 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070961 s, 148 MB/s 2024-04-25T08:39:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.459 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625327 s, 168 MB/s 2024-04-25T08:39:45.466 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.467 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070569 s, 149 MB/s 2024-04-25T08:39:45.473 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.474 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.474 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00608208 s, 172 MB/s 2024-04-25T08:39:45.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.481 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.481 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625307 s, 168 MB/s 2024-04-25T08:39:45.487 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.487 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.487 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581945 s, 180 MB/s 2024-04-25T08:39:45.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.494 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.495 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606366 s, 173 MB/s 2024-04-25T08:39:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.501 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.502 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616021 s, 170 MB/s 2024-04-25T08:39:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.511 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714022 s, 147 MB/s 2024-04-25T08:39:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.518 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732268 s, 143 MB/s 2024-04-25T08:39:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.525 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060354 s, 174 MB/s 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061894 s, 169 MB/s 2024-04-25T08:39:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.540 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696122 s, 151 MB/s 2024-04-25T08:39:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.547 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643989 s, 163 MB/s 2024-04-25T08:39:45.553 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.554 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572946 s, 183 MB/s 2024-04-25T08:39:45.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.561 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.561 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657211 s, 160 MB/s 2024-04-25T08:39:45.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.568 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.568 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00611171 s, 172 MB/s 2024-04-25T08:39:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.575 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591345 s, 177 MB/s 2024-04-25T08:39:45.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:45.576 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:45.576 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:45.576 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:39:45.576 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:45.576 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:45.576 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:45.577 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.641 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.641 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616572 s, 170 MB/s 2024-04-25T08:39:45.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.648 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.649 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705641 s, 149 MB/s 2024-04-25T08:39:45.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.656 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.656 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653837 s, 160 MB/s 2024-04-25T08:39:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.664 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722042 s, 145 MB/s 2024-04-25T08:39:45.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.671 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.671 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642411 s, 163 MB/s 2024-04-25T08:39:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.678 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.679 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638117 s, 164 MB/s 2024-04-25T08:39:45.686 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.687 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.687 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061717 s, 170 MB/s 2024-04-25T08:39:45.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.692 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.693 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00590264 s, 178 MB/s 2024-04-25T08:39:45.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.699 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.699 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00575549 s, 182 MB/s 2024-04-25T08:39:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.707 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664708 s, 158 MB/s 2024-04-25T08:39:45.714 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.715 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.715 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00625189 s, 168 MB/s 2024-04-25T08:39:45.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.721 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.722 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641301 s, 164 MB/s 2024-04-25T08:39:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.728 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0062338 s, 168 MB/s 2024-04-25T08:39:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.736 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681726 s, 154 MB/s 2024-04-25T08:39:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.744 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683917 s, 153 MB/s 2024-04-25T08:39:45.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.753 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.753 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776818 s, 135 MB/s 2024-04-25T08:39:45.760 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726992 s, 144 MB/s 2024-04-25T08:39:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.768 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704163 s, 149 MB/s 2024-04-25T08:39:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.776 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637746 s, 164 MB/s 2024-04-25T08:39:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652332 s, 161 MB/s 2024-04-25T08:39:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.790 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647192 s, 162 MB/s 2024-04-25T08:39:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.798 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663851 s, 158 MB/s 2024-04-25T08:39:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.805 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621814 s, 169 MB/s 2024-04-25T08:39:45.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.813 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.813 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006736 s, 156 MB/s 2024-04-25T08:39:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.819 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591746 s, 177 MB/s 2024-04-25T08:39:45.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.827 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.827 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711284 s, 147 MB/s 2024-04-25T08:39:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.834 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560138 s, 187 MB/s 2024-04-25T08:39:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.841 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00620625 s, 169 MB/s 2024-04-25T08:39:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.849 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069819 s, 150 MB/s 2024-04-25T08:39:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.857 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713902 s, 147 MB/s 2024-04-25T08:39:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.864 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00597003 s, 176 MB/s 2024-04-25T08:39:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.871 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.872 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066341 s, 158 MB/s 2024-04-25T08:39:45.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> set -e 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> done 2024-04-25T08:39:45.873 DEBUG:teuthology.orchestra.run.smithi060:> ') 2024-04-25T08:39:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.938 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639531 s, 164 MB/s 2024-04-25T08:39:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.945 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640139 s, 164 MB/s 2024-04-25T08:39:45.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.952 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.952 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656291 s, 160 MB/s 2024-04-25T08:39:45.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.959 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.960 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00630162 s, 166 MB/s 2024-04-25T08:39:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.968 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736996 s, 142 MB/s 2024-04-25T08:39:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.975 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00580755 s, 181 MB/s 2024-04-25T08:39:45.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.981 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.981 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591976 s, 177 MB/s 2024-04-25T08:39:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.988 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.989 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615828 s, 170 MB/s 2024-04-25T08:39:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:45.996 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710299 s, 148 MB/s 2024-04-25T08:39:46.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.004 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.004 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705614 s, 149 MB/s 2024-04-25T08:39:46.012 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.013 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.013 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730778 s, 143 MB/s 2024-04-25T08:39:46.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.020 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.020 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638004 s, 164 MB/s 2024-04-25T08:39:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.027 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640805 s, 164 MB/s 2024-04-25T08:39:46.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.034 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.034 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00591803 s, 177 MB/s 2024-04-25T08:39:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.042 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704794 s, 149 MB/s 2024-04-25T08:39:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.050 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721658 s, 145 MB/s 2024-04-25T08:39:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.057 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00626923 s, 167 MB/s 2024-04-25T08:39:46.064 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.065 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.065 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640432 s, 164 MB/s 2024-04-25T08:39:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.072 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693671 s, 151 MB/s 2024-04-25T08:39:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.080 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689242 s, 152 MB/s 2024-04-25T08:39:46.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.087 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.087 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650349 s, 161 MB/s 2024-04-25T08:39:46.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.094 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.095 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00623112 s, 168 MB/s 2024-04-25T08:39:46.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.102 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.103 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702584 s, 149 MB/s 2024-04-25T08:39:46.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.109 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.109 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060603 s, 173 MB/s 2024-04-25T08:39:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.116 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572959 s, 183 MB/s 2024-04-25T08:39:46.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.124 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.124 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638741 s, 164 MB/s 2024-04-25T08:39:46.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.130 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.130 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560501 s, 187 MB/s 2024-04-25T08:39:46.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.138 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.138 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689039 s, 152 MB/s 2024-04-25T08:39:46.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.144 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.144 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573589 s, 183 MB/s 2024-04-25T08:39:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.153 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764569 s, 137 MB/s 2024-04-25T08:39:46.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.161 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.161 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707129 s, 148 MB/s 2024-04-25T08:39:46.167 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-04-25T08:39:46.168 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-04-25T08:39:46.168 INFO:teuthology.orchestra.run.smithi060.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00567852 s, 185 MB/s 2024-04-25T08:39:46.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T08:39:46.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.319+0000 7f4a9c893640 1 Processor -- start 2024-04-25T08:39:46.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.319+0000 7f4a9c893640 1 -- start start 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a9c893640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a980d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a9c893640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980d2420 con 0x7f4a980d1f30 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a9c893640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a980d2700 con 0x7f4a980d0ab0 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a9c893640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980d3030 con 0x7f4a9815dac0 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a95d9b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f4a980d1f30 0x7f4a980d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36440/0 (socket says 172.21.15.60:36440) 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a95d9b640 1 -- 172.21.15.60:0/1484320773 learned_addr learned my addr 172.21.15.60:0/1484320773 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.320+0000 7f4a9659c640 1 --2- 172.21.15.60:0/1484320773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a980d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270081092 0 0) 0x7f4a980d2420 con 0x7f4a980d1f30 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a84003520 con 0x7f4a980d1f30 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016023147 0 0) 0x7f4a980d2700 con 0x7f4a980d0ab0 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a980d2420 con 0x7f4a980d0ab0 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9659c640 1 -- 172.21.15.60:0/1484320773 >> v1:172.21.15.100:6789/0 conn(0x7f4a980d0ab0 legacy=0x7f4a980b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9659c640 1 -- 172.21.15.60:0/1484320773 >> v1:172.21.15.60:6789/0 conn(0x7f4a980d1f30 legacy=0x7f4a980d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.321+0000 7f4a9659c640 1 -- 172.21.15.60:0/1484320773 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980d29e0 con 0x7f4a9815dac0 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9659c640 1 --2- 172.21.15.60:0/1484320773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a980d19f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a980d31b0 tx=0x7f4a9002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad8e2ce66c15ad58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90037070 con 0x7f4a9815dac0 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a90032440 con 0x7f4a9815dac0 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9559a640 1 -- 172.21.15.60:0/1484320773 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a900327f0 con 0x7f4a9815dac0 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 -- 172.21.15.60:0/1484320773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 msgr2=0x7f4a980d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 --2- 172.21.15.60:0/1484320773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a980d19f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a980d31b0 tx=0x7f4a9002a290 comp rx=0 tx=0).stop 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 -- 172.21.15.60:0/1484320773 shutdown_connections 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 --2- 172.21.15.60:0/1484320773 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a980d19f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 -- 172.21.15.60:0/1484320773 >> 172.21.15.60:0/1484320773 conn(0x7f4a98061c30 msgr2=0x7f4a9806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 -- 172.21.15.60:0/1484320773 shutdown_connections 2024-04-25T08:39:46.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.322+0000 7f4a9c893640 1 -- 172.21.15.60:0/1484320773 wait complete. 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 Processor -- start 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 -- start start 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 0x7f4a98160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a980d1f30 0x7f4a980cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a98143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980b97d0 con 0x7f4a980d0ab0 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9815dea0 con 0x7f4a980d1f30 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9c893640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980b9950 con 0x7f4a9815dac0 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.324+0000 7f4a9659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 0x7f4a98160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 0x7f4a98160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54708/0 (socket says 172.21.15.60:54708) 2024-04-25T08:39:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 -- 172.21.15.60:0/3593163147 learned_addr learned my addr 172.21.15.60:0/3593163147 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a96d9d640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a98143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a95d9b640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a980d1f30 0x7f4a980cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 -- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 msgr2=0x7f4a98143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a98143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 -- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a980d1f30 msgr2=0x7f4a980cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a980d1f30 0x7f4a980cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 -- 172.21.15.60:0/3593163147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980c75d0 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9659c640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 0x7f4a98160690 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f4a90035040 tx=0x7f4a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90037050 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a90043070 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9815fbb0 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9003ca50 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.325+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a98160170 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.326+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a98073a00 con 0x7f4a980d0ab0 2024-04-25T08:39:46.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.327+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a9003cbf0 con 0x7f4a980d0ab0 2024-04-25T08:39:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.327+0000 7f4a8effd640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4a7806b4d0 0x7f4a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.327+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a900ab140 con 0x7f4a980d0ab0 2024-04-25T08:39:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.327+0000 7f4a95d9b640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4a7806b4d0 0x7f4a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.328+0000 7f4a95d9b640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4a7806b4d0 0x7f4a7806d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4a80007a60 tx=0x7f4a80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.332+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a90046360 con 0x7f4a980d0ab0 2024-04-25T08:39:46.549 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.555+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f4a980b8bd0 con 0x7f4a980d0ab0 2024-04-25T08:39:46.767 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.774+0000 7f4a8effd640 1 -- 172.21.15.60:0/3593163147 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v90) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4a90079d10 con 0x7f4a980d0ab0 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4a7806b4d0 msgr2=0x7f4a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4a7806b4d0 0x7f4a7806d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4a80007a60 tx=0x7f4a80009040 comp rx=0 tx=0).stop 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 msgr2=0x7f4a98160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 0x7f4a98160690 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f4a90035040 tx=0x7f4a900046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 shutdown_connections 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4a9815dac0 0x7f4a98143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f4a7806b4d0 0x7f4a7806d990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a980d1f30 0x7f4a980cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 --2- 172.21.15.60:0/3593163147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a980d0ab0 0x7f4a98160690 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 >> 172.21.15.60:0/3593163147 conn(0x7f4a98061c30 msgr2=0x7f4a980ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 shutdown_connections 2024-04-25T08:39:46.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.776+0000 7f4a9c893640 1 -- 172.21.15.60:0/3593163147 wait complete. 2024-04-25T08:39:46.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T08:39:46.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.891+0000 7f21df2c6640 1 Processor -- start 2024-04-25T08:39:46.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.891+0000 7f21df2c6640 1 -- start start 2024-04-25T08:39:46.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21df2c6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21d815dac0 0x7f21d80d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21df2c6640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d80d1f30 0x7f21d80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21df2c6640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80d2420 con 0x7f21d80d34c0 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21df2c6640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80d3030 con 0x7f21d80d1f30 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21df2c6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80d2700 con 0x7f21d815dac0 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dc861640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f21d80d1f30 0x7f21d80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dd863640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f21d80d34c0 0x7f21d80b5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36456/0 (socket says 172.21.15.60:36456) 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dd863640 1 -- 172.21.15.60:0/2568211345 learned_addr learned my addr 172.21.15.60:0/2568211345 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dd062640 1 --2- 172.21.15.60:0/2568211345 >> v2:172.21.15.100:3300/0 conn(0x7f21d815dac0 0x7f21d80d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dc861640 1 -- 172.21.15.60:0/2568211345 >> v2:172.21.15.100:3300/0 conn(0x7f21d815dac0 msgr2=0x7f21d80d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dc861640 1 --2- 172.21.15.60:0/2568211345 >> v2:172.21.15.100:3300/0 conn(0x7f21d815dac0 0x7f21d80d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.892+0000 7f21dc861640 1 -- 172.21.15.60:0/2568211345 >> v1:172.21.15.60:6789/0 conn(0x7f21d80d34c0 legacy=0x7f21d80b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.893+0000 7f21cffff640 1 -- 172.21.15.60:0/2568211345 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883786920 0 0) 0x7f21d80d2420 con 0x7f21d80d34c0 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.893+0000 7f21dc861640 1 -- 172.21.15.60:0/2568211345 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80d29e0 con 0x7f21d80d1f30 2024-04-25T08:39:46.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.893+0000 7f21dc861640 1 --2- 172.21.15.60:0/2568211345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d1f30 0x7f21d80d03a0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f21d80b6520 tx=0x7f21c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc86c3fe15469ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.893+0000 7f21cffff640 1 -- 172.21.15.60:0/2568211345 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8037070 con 0x7f21d80d1f30 2024-04-25T08:39:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.893+0000 7f21cffff640 1 -- 172.21.15.60:0/2568211345 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8032440 con 0x7f21d80d1f30 2024-04-25T08:39:46.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.893+0000 7f21cffff640 1 -- 172.21.15.60:0/2568211345 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8032810 con 0x7f21d80d1f30 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 -- 172.21.15.60:0/2568211345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d1f30 msgr2=0x7f21d80d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 --2- 172.21.15.60:0/2568211345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d1f30 0x7f21d80d03a0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f21d80b6520 tx=0x7f21c802a230 comp rx=0 tx=0).stop 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 -- 172.21.15.60:0/2568211345 shutdown_connections 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 --2- 172.21.15.60:0/2568211345 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d1f30 0x7f21d80d03a0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 --2- 172.21.15.60:0/2568211345 >> v2:172.21.15.100:3300/0 conn(0x7f21d815dac0 0x7f21d80d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 -- 172.21.15.60:0/2568211345 >> 172.21.15.60:0/2568211345 conn(0x7f21d8061c30 msgr2=0x7f21d806c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 -- 172.21.15.60:0/2568211345 shutdown_connections 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.894+0000 7f21df2c6640 1 -- 172.21.15.60:0/2568211345 wait complete. 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.895+0000 7f21df2c6640 1 Processor -- start 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.895+0000 7f21df2c6640 1 -- start start 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21df2c6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d80d1f30 0x7f21d814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21df2c6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 0x7f21d8156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21df2c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d815dac0 0x7f21d815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21df2c6640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80b97d0 con 0x7f21d80d34c0 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21df2c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d815dea0 con 0x7f21d815dac0 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21df2c6640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80b9950 con 0x7f21d80d1f30 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dc861640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 0x7f21d8156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dc861640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 0x7f21d8156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54728/0 (socket says 172.21.15.60:54728) 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dc861640 1 -- 172.21.15.60:0/1533259275 learned_addr learned my addr 172.21.15.60:0/1533259275 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dd062640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d80d1f30 0x7f21d814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dd863640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d815dac0 0x7f21d815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dc861640 1 -- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d80d1f30 msgr2=0x7f21d814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.896+0000 7f21dc861640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d80d1f30 0x7f21d814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21dc861640 1 -- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d815dac0 msgr2=0x7f21d815e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21dc861640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d815dac0 0x7f21d815e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21dc861640 1 -- 172.21.15.60:0/1533259275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8157940 con 0x7f21d80d34c0 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21dd863640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d815dac0 0x7f21d815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21dc861640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 0x7f21d8156ef0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f21c8000c00 tx=0x7f21c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8037040 con 0x7f21d80d34c0 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8042070 con 0x7f21d80d34c0 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d8157c20 con 0x7f21d80d34c0 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8030bd0 con 0x7f21d80d34c0 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.897+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d81581e0 con 0x7f21d80d34c0 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.898+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8073a00 con 0x7f21d80d34c0 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.899+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21c8047460 con 0x7f21d80d34c0 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.899+0000 7f21cdffb640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.899+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21c80aae50 con 0x7f21d80d34c0 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.899+0000 7f21dd062640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.900+0000 7f21dd062640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f21d40079a0 tx=0x7f21d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:46.904+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21c8032e20 con 0x7f21d80d34c0 2024-04-25T08:39:47.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.122+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f21d815d5e0 con 0x7f21d80d34c0 2024-04-25T08:39:47.775 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.782+0000 7f21cdffb640 1 -- 172.21.15.60:0/1533259275 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v91) v1 ==== 121+0+0 (secure 0 0 0) 0x7f21c8079b40 con 0x7f21d80d34c0 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21bc06b4d0 msgr2=0x7f21bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21bc06b4d0 0x7f21bc06d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f21d40079a0 tx=0x7f21d4008040 comp rx=0 tx=0).stop 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 msgr2=0x7f21d8156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 0x7f21d8156ef0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f21c8000c00 tx=0x7f21c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 shutdown_connections 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21bc06b4d0 0x7f21bc06d990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21d815dac0 0x7f21d815e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21d80d34c0 0x7f21d8156ef0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 --2- 172.21.15.60:0/1533259275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f21d80d1f30 0x7f21d814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 >> 172.21.15.60:0/1533259275 conn(0x7f21d8061c30 msgr2=0x7f21d80c7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:47.779 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 shutdown_connections 2024-04-25T08:39:47.780 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.785+0000 7f21df2c6640 1 -- 172.21.15.60:0/1533259275 wait complete. 2024-04-25T08:39:47.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:47.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.906+0000 7f2837809640 1 Processor -- start 2024-04-25T08:39:47.900 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.906+0000 7f2837809640 1 -- start start 2024-04-25T08:39:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2837809640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f283015dbe0 0x7f28300d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2837809640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28300b2cb0 0x7f28300d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2837809640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300b97a0 con 0x7f28300b92a0 2024-04-25T08:39:47.901 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2837809640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283015dfc0 con 0x7f28300b2cb0 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2837809640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28300d1a00 con 0x7f283015dbe0 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2834da4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28300b2cb0 0x7f28300d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2835da6640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f28300b92a0 0x7f28300b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36472/0 (socket says 172.21.15.60:36472) 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2834da4640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f28300b2cb0 0x7f28300d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54738/0 (socket says 172.21.15.60:54738) 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.907+0000 7f2835da6640 1 -- 172.21.15.60:0/827133950 learned_addr learned my addr 172.21.15.60:0/827133950 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.908+0000 7f28355a5640 1 --2- 172.21.15.60:0/827133950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f283015dbe0 0x7f28300d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.908+0000 7f2834da4640 1 -- 172.21.15.60:0/827133950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f283015dbe0 msgr2=0x7f28300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.908+0000 7f2834da4640 1 --2- 172.21.15.60:0/827133950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f283015dbe0 0x7f28300d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.908+0000 7f2834da4640 1 -- 172.21.15.60:0/827133950 >> v1:172.21.15.60:6789/0 conn(0x7f28300b92a0 legacy=0x7f28300b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:47.902 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.908+0000 7f2834da4640 1 -- 172.21.15.60:0/827133950 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300d1c50 con 0x7f28300b2cb0 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.908+0000 7f2834da4640 1 --2- 172.21.15.60:0/827133950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28300b2cb0 0x7f28300d0e50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f28240097a0 tx=0x7f282402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fadb8f46e0463d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f282ffff640 1 -- 172.21.15.60:0/827133950 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824037070 con 0x7f28300b2cb0 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f282ffff640 1 -- 172.21.15.60:0/827133950 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2824032440 con 0x7f28300b2cb0 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f282ffff640 1 -- 172.21.15.60:0/827133950 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824032810 con 0x7f28300b2cb0 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 -- 172.21.15.60:0/827133950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28300b2cb0 msgr2=0x7f28300d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 --2- 172.21.15.60:0/827133950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28300b2cb0 0x7f28300d0e50 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f28240097a0 tx=0x7f282402a180 comp rx=0 tx=0).stop 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 -- 172.21.15.60:0/827133950 shutdown_connections 2024-04-25T08:39:47.904 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 --2- 172.21.15.60:0/827133950 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28300b2cb0 0x7f28300d0e50 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 --2- 172.21.15.60:0/827133950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f283015dbe0 0x7f28300d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 -- 172.21.15.60:0/827133950 >> 172.21.15.60:0/827133950 conn(0x7f2830061a20 msgr2=0x7f2830061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:47.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 -- 172.21.15.60:0/827133950 shutdown_connections 2024-04-25T08:39:47.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.909+0000 7f2837809640 1 -- 172.21.15.60:0/827133950 wait complete. 2024-04-25T08:39:47.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 Processor -- start 2024-04-25T08:39:47.905 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 -- start start 2024-04-25T08:39:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28300b2cb0 0x7f283015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28300b92a0 0x7f28300c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 0x7f28300b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300b6ab0 con 0x7f283015dbe0 2024-04-25T08:39:47.907 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300d2210 con 0x7f28300b92a0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2837809640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28300b6c30 con 0x7f28300b2cb0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.911+0000 7f2835da6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 0x7f28300b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 0x7f28300b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54750/0 (socket says 172.21.15.60:54750) 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 -- 172.21.15.60:0/3170849714 learned_addr learned my addr 172.21.15.60:0/3170849714 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f28355a5640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28300b2cb0 0x7f283015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2834da4640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28300b92a0 0x7f28300c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 -- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28300b2cb0 msgr2=0x7f283015f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28300b2cb0 0x7f283015f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 -- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28300b92a0 msgr2=0x7f28300c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28300b92a0 0x7f28300c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 -- 172.21.15.60:0/3170849714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283015e710 con 0x7f283015dbe0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2835da6640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 0x7f28300b4370 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f282800c430 tx=0x7f282800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828017070 con 0x7f283015dbe0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2828005ce0 con 0x7f283015dbe0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280072d0 con 0x7f283015dbe0 2024-04-25T08:39:47.908 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283015e9f0 con 0x7f283015dbe0 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.912+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283015efb0 con 0x7f283015dbe0 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.913+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2830073990 con 0x7f283015dbe0 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.914+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f282801a070 con 0x7f283015dbe0 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.914+0000 7f282dffb640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f281c06b4d0 0x7f281c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.914+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28280895d0 con 0x7f283015dbe0 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.914+0000 7f28355a5640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f281c06b4d0 0x7f281c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.915+0000 7f28355a5640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f281c06b4d0 0x7f281c06d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2820003aa0 tx=0x7f2820009830 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:47.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:47.918+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2828057c70 con 0x7f283015dbe0 2024-04-25T08:39:48.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.135+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28300b48b0 con 0x7f283015dbe0 2024-04-25T08:39:48.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.135+0000 7f282dffb640 1 -- 172.21.15.60:0/3170849714 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f282805bd10 con 0x7f283015dbe0 2024-04-25T08:39:48.129 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:48.129 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:47.785553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8716":{"gid":8716,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:48.129 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f281c06b4d0 msgr2=0x7f281c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f281c06b4d0 0x7f281c06d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2820003aa0 tx=0x7f2820009830 comp rx=0 tx=0).stop 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 msgr2=0x7f28300b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 0x7f28300b4370 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f282800c430 tx=0x7f282800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 shutdown_connections 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f283015dbe0 0x7f28300b4370 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f281c06b4d0 0x7f281c06d990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.131 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28300b92a0 0x7f28300c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 --2- 172.21.15.60:0/3170849714 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f28300b2cb0 0x7f283015f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.137+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 >> 172.21.15.60:0/3170849714 conn(0x7f2830061a20 msgr2=0x7f28300b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.138+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 shutdown_connections 2024-04-25T08:39:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.138+0000 7f2837809640 1 -- 172.21.15.60:0/3170849714 wait complete. 2024-04-25T08:39:48.544 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T08:39:25.890816+0000', 'modified': '2024-04-25T08:39:47.785553+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9924}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8716': {'gid': 8716, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6837/4143083931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 4143083931}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 4143083931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9924': {'gid': 9924, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.60:6833/4231971347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 4231971347}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 4231971347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:39:48.544 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:39:48.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:39:48.652 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.659+0000 7fb5c2fd5640 1 Processor -- start 2024-04-25T08:39:48.653 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.659+0000 7fb5c2fd5640 1 -- start start 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5c2fd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5bc0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5c2fd5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc0d23a0 con 0x7fb5bc15dc20 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5c2fd5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc0d2680 con 0x7fb5bc0d1ce0 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5c2fd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0d2960 con 0x7fb5bc0ced80 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5c0d71640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb5bc15dc20 0x7fb5bc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36484/0 (socket says 172.21.15.60:36484) 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5c0d71640 1 -- 172.21.15.60:0/3311890178 learned_addr learned my addr 172.21.15.60:0/3311890178 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:48.656 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585144557 0 0) 0x7fb5bc0d23a0 con 0x7fb5bc15dc20 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5bc0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.660+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a4003520 con 0x7fb5bc15dc20 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384496677 0 0) 0x7fb5a4003520 con 0x7fb5bc15dc20 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 msgr2=0x7fb5bc0d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5bc0d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 >> v1:172.21.15.100:6789/0 conn(0x7fb5bc0d1ce0 legacy=0x7fb5bc0b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0d2c40 con 0x7fb5bc15dc20 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fb5b8001670 con 0x7fb5bc15dc20 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 >> v1:172.21.15.60:6789/0 conn(0x7fb5bc15dc20 legacy=0x7fb5bc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5a40039e0 0x7fb5a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a4004390 0x7fb5a4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5a4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0d2960 con 0x7fb5a4004390 2024-04-25T08:39:48.657 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a400ba70 con 0x7fb5a40039e0 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a400bc70 con 0x7fb5bc0ced80 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a4004390 0x7fb5a4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5a40039e0 0x7fb5a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5c1572640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5a4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b3fff640 1 -- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 msgr2=0x7fb5a4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5a4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b3fff640 1 -- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5a40039e0 msgr2=0x7fb5a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5a40039e0 0x7fb5a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5b3fff640 1 -- 172.21.15.60:0/3311890178 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a4003520 con 0x7fb5a4004390 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.661+0000 7fb5c1572640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5bc0ced80 0x7fb5a4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.662+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a4004390 0x7fb5a4004860 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009ac0 tx=0x7fb5a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412f6cad9c537c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.662+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac036070 con 0x7fb5a4004390 2024-04-25T08:39:48.658 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.662+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac031400 con 0x7fb5a4004390 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.662+0000 7fb5b37fe640 1 -- 172.21.15.60:0/3311890178 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac009690 con 0x7fb5a4004390 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.662+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/3311890178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a4004390 msgr2=0x7fb5a4004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.662+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/3311890178 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5a4004390 0x7fb5a4004860 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009ac0 tx=0x7fb5a4005830 comp rx=0 tx=0).stop 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.663+0000 7fb5c1572640 1 -- 172.21.15.60:0/3311890178 reap_dead start 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.663+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/3311890178 shutdown_connections 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.663+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/3311890178 >> 172.21.15.60:0/3311890178 conn(0x7fb5bc061a20 msgr2=0x7fb5bc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.663+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/3311890178 shutdown_connections 2024-04-25T08:39:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.663+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/3311890178 wait complete. 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.664+0000 7fb5c2fd5640 1 Processor -- start 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.664+0000 7fb5c2fd5640 1 -- start start 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c2fd5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5a4004390 0x7fb5bc15f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c2fd5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 0x7fb5bc0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c2fd5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5bc15ffe0 0x7fb5bc160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c2fd5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0d35c0 con 0x7fb5bc148fb0 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c2fd5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0d3440 con 0x7fb5bc15ffe0 2024-04-25T08:39:48.662 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c2fd5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0d3740 con 0x7fb5a4004390 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 0x7fb5bc0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 0x7fb5bc0b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54766/0 (socket says 172.21.15.60:54766) 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 -- 172.21.15.60:0/2002680537 learned_addr learned my addr 172.21.15.60:0/2002680537 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c1572640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5bc15ffe0 0x7fb5bc160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5a4004390 0x7fb5bc15f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 -- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5a4004390 msgr2=0x7fb5bc15f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5a4004390 0x7fb5bc15f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 -- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5bc15ffe0 msgr2=0x7fb5bc160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5bc15ffe0 0x7fb5bc160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 -- 172.21.15.60:0/2002680537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0cc4d0 con 0x7fb5bc148fb0 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5c1572640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5bc15ffe0 0x7fb5bc160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.665+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 0x7fb5bc0b55b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009be0 tx=0x7fb5ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.666+0000 7fb5b17fa640 1 -- 172.21.15.60:0/2002680537 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac040020 con 0x7fb5bc148fb0 2024-04-25T08:39:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.666+0000 7fb5b17fa640 1 -- 172.21.15.60:0/2002680537 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac003940 con 0x7fb5bc148fb0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.666+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc0cc7b0 con 0x7fb5bc148fb0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.666+0000 7fb5b17fa640 1 -- 172.21.15.60:0/2002680537 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac036040 con 0x7fb5bc148fb0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.666+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc0ccd70 con 0x7fb5bc148fb0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.667+0000 7fb5b17fa640 1 -- 172.21.15.60:0/2002680537 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5ac043020 con 0x7fb5bc148fb0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.668+0000 7fb5b17fa640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb5a806b500 0x7fb5a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.668+0000 7fb5b17fa640 1 -- 172.21.15.60:0/2002680537 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5ac0aa660 con 0x7fb5bc148fb0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.668+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb5a806b500 0x7fb5a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.668+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb5a806b500 0x7fb5a806d9c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8007f90 tx=0x7fb5b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 Processor -- start 2024-04-25T08:39:48.664 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 -- start start 2024-04-25T08:39:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 0x5622334e8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56223349ff10 0x5622334e6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.670 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562233402a10 con 0x5622334e7ad0 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622333abb30 con 0x5622334ca230 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.669+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622333fec50 con 0x56223349ff10 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 0x5622334e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 0x5622334e8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54768/0 (socket says 172.21.15.60:54768) 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1587938431 learned_addr learned my addr 172.21.15.60:0/1587938431 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5c1572640 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56223349ff10 0x5622334e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56223349ff10 msgr2=0x5622334e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56223349ff10 0x5622334e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 msgr2=0x5622334ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1587938431 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622333c0e10 con 0x5622334e7ad0 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 0x5622334e8020 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac080d90 tx=0x7fb5ac078bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244565aa7f164b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.671 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5a27fc640 1 -- 172.21.15.60:0/1587938431 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0812c0 con 0x5622334e7ad0 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5a27fc640 1 -- 172.21.15.60:0/1587938431 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac076eb0 con 0x5622334e7ad0 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.670+0000 7fb5a27fc640 1 -- 172.21.15.60:0/1587938431 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac075220 con 0x5622334e7ad0 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1587938431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 msgr2=0x5622334e8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 0x5622334e8020 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac080d90 tx=0x7fb5ac078bb0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1587938431 shutdown_connections 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x56223349ff10 0x5622334e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334e7ad0 0x5622334e8020 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1587938431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1587938431 >> 172.21.15.60:0/1587938431 conn(0x56223349c840 msgr2=0x5622334e5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1587938431 shutdown_connections 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1587938431 wait complete. 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.671+0000 7fb5c5dba740 1 -- start start 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 Processor -- start 2024-04-25T08:39:48.672 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334e7ad0 0x5622334c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622333fec50 con 0x5622334c6b20 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622333bf0c0 con 0x5622334ca230 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x562233402a10 con 0x5622334e7ad0 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c5dba740 10 client.0 _ll_register_callbacks cb 0x5622333b2ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c1572640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c1572640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54772/0 (socket says 172.21.15.60:54772) 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.672+0000 7fb5c1572640 1 -- 172.21.15.60:0/4254557331 learned_addr learned my addr 172.21.15.60:0/4254557331 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334e7ad0 0x5622334c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c1572640 1 -- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334e7ad0 msgr2=0x5622334c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c1572640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334e7ad0 0x5622334c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c1572640 1 -- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 msgr2=0x5622334e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.673 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c1572640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c1572640 1 -- 172.21.15.60:0/4254557331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622331172d0 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c1572640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334caef0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40058a0 tx=0x7fb5b4001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1de92fc2f9bae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb59b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb59b7fe640 1 -- 172.21.15.60:0/4254557331 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b401b060 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb59b7fe640 1 -- 172.21.15.60:0/4254557331 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5b4016480 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5ac076eb0 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb59b7fe640 1 -- 172.21.15.60:0/4254557331 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5b4021050 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c5dba740 10 client.10149 fetch_fsmap learned FSMap version 92 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.673+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5622331e16b0 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb59b7fe640 1 -- 172.21.15.60:0/4254557331 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac076eb0 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb59b7fe640 1 -- 172.21.15.60:0/4254557331 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 92) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fb5b4024020 con 0x5622334c6b20 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c5dba740 10 client.10149 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c5dba740 10 client.10149 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:39:48.674 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c5dba740 20 client.10149 populate_metadata read hostname 'smithi060' 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334f8bc0 0x5622334f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c5dba740 4 client.10149 mds_command: new command op to 9924 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 --> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562233461920 con 0x5622334f8bc0 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334f8bc0 0x5622334f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334f8bc0 0x5622334f8fa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d701b50008732d server_cookie=3f3adb55e5412d04 in_seq=0 out_seq=0 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.674+0000 7fb59b7fe640 10 client.10149 ms_handle_connect on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb59b7fe640 1 -- 172.21.15.60:0/4254557331 <== mds.0 v2:172.21.15.60:6832/4231971347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562233461920 con 0x5622334f8bc0 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb59b7fe640 10 client.10149 handle_command_reply: tid=0 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 client.10149 shutdown 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 msgr2=0x5622334caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334caef0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40058a0 tx=0x7fb5b4001b60 comp rx=0 tx=0).stop 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 shutdown_connections 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334f8bc0 0x5622334f8fa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334caef0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.675 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334e7ad0 0x5622334c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb59b7fe640 0 client.10149 ms_handle_reset on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:48.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/4254557331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334ca230 0x5622334e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 >> 172.21.15.60:0/4254557331 conn(0x56223349ff10 msgr2=0x5622334e8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 shutdown_connections 2024-04-25T08:39:48.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 1 -- 172.21.15.60:0/4254557331 wait complete. 2024-04-25T08:39:48.676 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.675+0000 7fb5c5dba740 20 client.10149 trim_cache size 0 max 16384 2024-04-25T08:39:48.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.691+0000 7fb5c5dba740 1 Processor -- start 2024-04-25T08:39:48.684 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.691+0000 7fb5c5dba740 1 -- start start 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc200b80 con 0x5622334c6b20 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1ff1b0 con 0x5622334e7ad0 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc1ff7b0 con 0x5622334ca230 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334debd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54774/0 (socket says 172.21.15.60:54774) 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 -- 172.21.15.60:0/1761434298 learned_addr learned my addr 172.21.15.60:0/1761434298 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 -- 172.21.15.60:0/1761434298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334ca230 msgr2=0x5622334f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.687 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 -- 172.21.15.60:0/1761434298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 msgr2=0x5622334f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 -- 172.21.15.60:0/1761434298 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56223349bd00 con 0x5622334c6b20 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334debd0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x5622334dc940 tx=0x7fb5b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549b6113f799afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5a1ffb640 1 -- 172.21.15.60:0/1761434298 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8016c20 con 0x5622334c6b20 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5a1ffb640 1 -- 172.21.15.60:0/1761434298 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8016dc0 con 0x5622334c6b20 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.692+0000 7fb5a1ffb640 1 -- 172.21.15.60:0/1761434298 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80245f0 con 0x5622334c6b20 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1761434298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 msgr2=0x5622334debd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334debd0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x5622334dc940 tx=0x7fb5b8014000 comp rx=0 tx=0).stop 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1761434298 shutdown_connections 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1761434298 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334c6b20 0x5622334debd0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1761434298 >> 172.21.15.60:0/1761434298 conn(0x56223349ff10 msgr2=0x5622334e32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1761434298 shutdown_connections 2024-04-25T08:39:48.688 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.693+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1761434298 wait complete. 2024-04-25T08:39:48.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 -- start start 2024-04-25T08:39:48.689 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 Processor -- start 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334c6b20 0x5622334ff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622334dc790 con 0x5622334ca230 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622333abb30 con 0x5622334e7ad0 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.694+0000 7fb5c5dba740 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622334cb480 con 0x5622334c6b20 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c5dba740 10 client.0 _ll_register_callbacks cb 0x56223349ac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:48.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54778/0 (socket says 172.21.15.60:54778) 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1620193802 learned_addr learned my addr 172.21.15.60:0/1620193802 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334c6b20 0x5622334ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c1572640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334c6b20 msgr2=0x5622334ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334c6b20 0x5622334ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 msgr2=0x5622334f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 -- 172.21.15.60:0/1620193802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622334dd290 con 0x5622334ca230 2024-04-25T08:39:48.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c1572640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5b3fff640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 0x5622334f7aa0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac079460 tx=0x7fb5ac0a8830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734221d2286dfb2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb59b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2024-04-25T08:39:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb59b7fe640 1 -- 172.21.15.60:0/1620193802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac074e50 con 0x5622334ca230 2024-04-25T08:39:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb59b7fe640 1 -- 172.21.15.60:0/1620193802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5ac080e30 con 0x5622334ca230 2024-04-25T08:39:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5b8016dc0 con 0x5622334ca230 2024-04-25T08:39:48.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb59b7fe640 1 -- 172.21.15.60:0/1620193802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5ac0812c0 con 0x5622334ca230 2024-04-25T08:39:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c5dba740 10 client.10155 fetch_fsmap learned FSMap version 92 2024-04-25T08:39:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.695+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) v3 -- 0x5622335468f0 con 0x5622334ca230 2024-04-25T08:39:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb59b7fe640 1 -- 172.21.15.60:0/1620193802 <== mon.0 v2:172.21.15.60:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b8016dc0 con 0x5622334ca230 2024-04-25T08:39:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb59b7fe640 1 -- 172.21.15.60:0/1620193802 <== mon.0 v2:172.21.15.60:3300/0 5 ==== fsmap(e 92) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fb5ac08cb70 con 0x5622334ca230 2024-04-25T08:39:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 10 client.10155 fetch_fsmap finished waiting for FSMap version 92 2024-04-25T08:39:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 10 client.10155 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:39:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 20 client.10155 populate_metadata read hostname 'smithi060' 2024-04-25T08:39:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334c94c0 0x5622334c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 4 client.10155 mds_command: new command op to 9924 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:39:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 --> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56223340ff50 con 0x5622334c94c0 2024-04-25T08:39:48.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334c94c0 0x5622334c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb59b7fe640 10 client.10155 ms_handle_connect on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c0d71640 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334c94c0 0x5622334c98a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bfade7a166625c server_cookie=ff3a40c095fd52ba in_seq=0 out_seq=0 2024-04-25T08:39:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb59b7fe640 1 -- 172.21.15.60:0/1620193802 <== mds.0 v2:172.21.15.60:6832/4231971347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x56223340ff50 con 0x5622334c94c0 2024-04-25T08:39:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb59b7fe640 10 client.10155 handle_command_reply: tid=0 2024-04-25T08:39:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.696+0000 7fb5c5dba740 1 client.10155 shutdown 2024-04-25T08:39:48.697 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 msgr2=0x5622334f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 0x5622334f7aa0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac079460 tx=0x7fb5ac0a8830 comp rx=0 tx=0).stop 2024-04-25T08:39:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 shutdown_connections 2024-04-25T08:39:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:6832/4231971347,v1:172.21.15.60:6833/4231971347] conn(0x5622334c94c0 0x5622334c98a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5622334e7ad0 0x5622334f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb59b7fe640 0 client.10155 ms_handle_reset on v2:172.21.15.60:6832/4231971347 2024-04-25T08:39:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5622334ca230 0x5622334f7aa0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 --2- 172.21.15.60:0/1620193802 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x5622334c6b20 0x5622334ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 >> 172.21.15.60:0/1620193802 conn(0x56223349ff10 msgr2=0x5622334e6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 shutdown_connections 2024-04-25T08:39:48.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 1 -- 172.21.15.60:0/1620193802 wait complete. 2024-04-25T08:39:48.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c5dba740 20 client.10155 trim_cache size 0 max 16384 2024-04-25T08:39:48.700 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:39:48.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_fsid": "46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de", 2024-04-25T08:39:48.700 INFO:teuthology.orchestra.run.smithi060.stdout: "whoami": 0, 2024-04-25T08:39:48.700 INFO:teuthology.orchestra.run.smithi060.stdout: "id": 9924, 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "want_state": "up:active", 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "state": "up:active", 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "fs_name": "cephfs", 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "rank_uptime": 22.165697001000002, 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "mdsmap_epoch": 92, 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch": 145, 2024-04-25T08:39:48.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_epoch_barrier": 145, 2024-04-25T08:39:48.702 INFO:teuthology.orchestra.run.smithi060.stdout: "uptime": 27.191755618999998 2024-04-25T08:39:48.702 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:39:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb5a806b500 msgr2=0x7fb5a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb5a806b500 0x7fb5a806d9c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8007f90 tx=0x7fb5b8009040 comp rx=0 tx=0).stop 2024-04-25T08:39:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 msgr2=0x7fb5bc0b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 0x7fb5bc0b55b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009be0 tx=0x7fb5ac004730 comp rx=0 tx=0).stop 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 shutdown_connections 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb5a806b500 0x7fb5a806d9c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5bc15ffe0 0x7fb5bc160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb5bc148fb0 0x7fb5bc0b55b0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 --2- 172.21.15.60:0/2002680537 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb5a4004390 0x7fb5bc15f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 >> 172.21.15.60:0/2002680537 conn(0x7fb5bc061a20 msgr2=0x7fb5bc0b8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 shutdown_connections 2024-04-25T08:39:48.705 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:48.697+0000 7fb5c2fd5640 1 -- 172.21.15.60:0/2002680537 wait complete. 2024-04-25T08:39:48.706 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:39:48.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:48.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:48.792 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:39:48.792 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:48.792 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:48.808 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:48.808 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:48.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:49.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.011+0000 7f47a1368640 1 Processor -- start 2024-04-25T08:39:49.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.011+0000 7f47a1368640 1 -- start start 2024-04-25T08:39:49.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f47a1368640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f479c15dd20 0x7f479c0b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f47a1368640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f47a1368640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479c15eac0 con 0x7f479c15e560 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f47a1368640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0d1c00 con 0x7f479c15dd20 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f47a1368640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c15eda0 con 0x7f479c15e100 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f479c15dd20 0x7f479c0b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f479c15dd20 0x7f479c0b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54780/0 (socket says 172.21.15.60:54780) 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 -- 172.21.15.60:0/474531739 learned_addr learned my addr 172.21.15.60:0/474531739 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f4793fff640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 -- 172.21.15.60:0/474531739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 msgr2=0x7f479c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c0d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 -- 172.21.15.60:0/474531739 >> v1:172.21.15.100:6789/0 conn(0x7f479c15e560 legacy=0x7f479c0d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f479affd640 1 -- 172.21.15.60:0/474531739 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c15f080 con 0x7f479c15dd20 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.012+0000 7f4793fff640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.013+0000 7f479affd640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c0b96d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f478c009870 tx=0x7f478c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94dfb2c70127e6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.013+0000 7f479a7fc640 1 -- 172.21.15.60:0/474531739 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c037070 con 0x7f479c15dd20 2024-04-25T08:39:49.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.013+0000 7f479a7fc640 1 -- 172.21.15.60:0/474531739 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c032440 con 0x7f479c15dd20 2024-04-25T08:39:49.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.013+0000 7f479a7fc640 1 -- 172.21.15.60:0/474531739 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c032810 con 0x7f479c15dd20 2024-04-25T08:39:49.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 -- 172.21.15.60:0/474531739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 msgr2=0x7f479c0b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c0b96d0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f478c009870 tx=0x7f478c02a180 comp rx=0 tx=0).stop 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 -- 172.21.15.60:0/474531739 shutdown_connections 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c0d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 --2- 172.21.15.60:0/474531739 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c0b96d0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 -- 172.21.15.60:0/474531739 >> 172.21.15.60:0/474531739 conn(0x7f479c061a20 msgr2=0x7f479c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 -- 172.21.15.60:0/474531739 shutdown_connections 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.014+0000 7f47a1368640 1 -- 172.21.15.60:0/474531739 wait complete. 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.015+0000 7f47a1368640 1 Processor -- start 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.015+0000 7f47a1368640 1 -- start start 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c1f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c1f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f479c15e560 0x7f479c1f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0d3500 con 0x7f479c15dd20 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0d3380 con 0x7f479c15e560 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0d3680 con 0x7f479c15e100 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c1f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c1f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54792/0 (socket says 172.21.15.60:54792) 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 -- 172.21.15.60:0/1633886752 learned_addr learned my addr 172.21.15.60:0/1633886752 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f4793fff640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c1f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479b7fe640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f479c15e560 0x7f479c1f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 -- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 msgr2=0x7f479c1f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c1f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 -- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f479c15e560 msgr2=0x7f479c1f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f479c15e560 0x7f479c1f76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 -- 172.21.15.60:0/1633886752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0afea0 con 0x7f479c15dd20 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f4793fff640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c1f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:49.012 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f479affd640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c1f3c20 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f478c009870 tx=0x7f478c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c041020 con 0x7f479c15dd20 2024-04-25T08:39:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c030970 con 0x7f479c15dd20 2024-04-25T08:39:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.016+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c0b0180 con 0x7f479c15dd20 2024-04-25T08:39:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.017+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c037040 con 0x7f479c15dd20 2024-04-25T08:39:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.017+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c0b0740 con 0x7f479c15dd20 2024-04-25T08:39:49.013 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.018+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c073990 con 0x7f479c15dd20 2024-04-25T08:39:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.018+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f478c044020 con 0x7f479c15dd20 2024-04-25T08:39:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.019+0000 7f47937fe640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f477c06b4d0 0x7f477c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.019+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f478c0aad20 con 0x7f479c15dd20 2024-04-25T08:39:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.019+0000 7f4793fff640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f477c06b4d0 0x7f477c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.019+0000 7f4793fff640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f477c06b4d0 0x7f477c06d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.016 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.023+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f478c04b030 con 0x7f479c15dd20 2024-04-25T08:39:49.196 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.203+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f479c06c310 con 0x7f479c15dd20 2024-04-25T08:39:49.197 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.204+0000 7f47937fe640 1 -- 172.21.15.60:0/1633886752 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f478c079a10 con 0x7f479c15dd20 2024-04-25T08:39:49.197 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:49.198 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":145,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:27.531391+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.205+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f477c06b4d0 msgr2=0x7f477c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.205+0000 7f47a1368640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f477c06b4d0 0x7f477c06d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784008040 comp rx=0 tx=0).stop 2024-04-25T08:39:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 msgr2=0x7f479c1f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c1f3c20 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f478c009870 tx=0x7f478c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 shutdown_connections 2024-04-25T08:39:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f477c06b4d0 0x7f477c06d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f479c15e560 0x7f479c1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f479c15e100 0x7f479c1f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 --2- 172.21.15.60:0/1633886752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f479c15dd20 0x7f479c1f3c20 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 >> 172.21.15.60:0/1633886752 conn(0x7f479c061a20 msgr2=0x7f479c15d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 shutdown_connections 2024-04-25T08:39:49.200 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.206+0000 7f47a1368640 1 -- 172.21.15.60:0/1633886752 wait complete. 2024-04-25T08:39:49.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:39:24.176721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:39:24.866339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:49.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:49.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.320+0000 7fdce7d63640 1 Processor -- start 2024-04-25T08:39:49.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.320+0000 7fdce7d63640 1 -- start start 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce7d63640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdce00d34c0 0x7fdce00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce7d63640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdce00d08e0 0x7fdce00b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce7d63640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00d2420 con 0x7fdce015dac0 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce7d63640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00d0d50 con 0x7fdce00d34c0 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce7d63640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00d3030 con 0x7fdce00d08e0 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce52fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdce00d34c0 0x7fdce00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce52fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fdce00d34c0 0x7fdce00d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54804/0 (socket says 172.21.15.60:54804) 2024-04-25T08:39:49.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce52fe640 1 -- 172.21.15.60:0/3104165525 learned_addr learned my addr 172.21.15.60:0/3104165525 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.321+0000 7fdce6300640 1 --2- 172.21.15.60:0/3104165525 >> v2:172.21.15.100:3300/0 conn(0x7fdce00d08e0 0x7fdce00b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce52fe640 1 -- 172.21.15.60:0/3104165525 >> v2:172.21.15.100:3300/0 conn(0x7fdce00d08e0 msgr2=0x7fdce00b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce52fe640 1 --2- 172.21.15.60:0/3104165525 >> v2:172.21.15.100:3300/0 conn(0x7fdce00d08e0 0x7fdce00b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce52fe640 1 -- 172.21.15.60:0/3104165525 >> v1:172.21.15.100:6789/0 conn(0x7fdce015dac0 legacy=0x7fdce00d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.316 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce52fe640 1 -- 172.21.15.60:0/3104165525 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce00d2700 con 0x7fdce00d34c0 2024-04-25T08:39:49.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce52fe640 1 --2- 172.21.15.60:0/3104165525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce00d03a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fdce00d2fa0 tx=0x7fdcd402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c177f32e4e3a546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce4afd640 1 -- 172.21.15.60:0/3104165525 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4004230 con 0x7fdce00d34c0 2024-04-25T08:39:49.317 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce4afd640 1 -- 172.21.15.60:0/3104165525 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd402ae50 con 0x7fdce00d34c0 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.322+0000 7fdce4afd640 1 -- 172.21.15.60:0/3104165525 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4003610 con 0x7fdce00d34c0 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 -- 172.21.15.60:0/3104165525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 msgr2=0x7fdce00d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 --2- 172.21.15.60:0/3104165525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce00d03a0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fdce00d2fa0 tx=0x7fdcd402a300 comp rx=0 tx=0).stop 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 -- 172.21.15.60:0/3104165525 shutdown_connections 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 --2- 172.21.15.60:0/3104165525 >> v2:172.21.15.100:3300/0 conn(0x7fdce00d08e0 0x7fdce00b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 --2- 172.21.15.60:0/3104165525 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce00d03a0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 -- 172.21.15.60:0/3104165525 >> 172.21.15.60:0/3104165525 conn(0x7fdce0061c30 msgr2=0x7fdce006c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 -- 172.21.15.60:0/3104165525 shutdown_connections 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.323+0000 7fdce7d63640 1 -- 172.21.15.60:0/3104165525 wait complete. 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.324+0000 7fdce7d63640 1 Processor -- start 2024-04-25T08:39:49.318 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.324+0000 7fdce7d63640 1 -- start start 2024-04-25T08:39:49.320 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce7d63640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdce00d08e0 0x7fdce00c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce7d63640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce7d63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce015dac0 0x7fdce015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce7d63640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00d2100 con 0x7fdce00d34c0 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce7d63640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00d2cc0 con 0x7fdce015dac0 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce7d63640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce015dea0 con 0x7fdce00d08e0 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54808/0 (socket says 172.21.15.60:54808) 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 -- 172.21.15.60:0/232542952 learned_addr learned my addr 172.21.15.60:0/232542952 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce5aff640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdce00d08e0 0x7fdce00c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce6300640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce015dac0 0x7fdce015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 -- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdce00d08e0 msgr2=0x7fdce00c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdce00d08e0 0x7fdce00c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 -- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce015dac0 msgr2=0x7fdce015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce015dac0 0x7fdce015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce52fe640 1 -- 172.21.15.60:0/232542952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce00c9000 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.325+0000 7fdce6300640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce015dac0 0x7fdce015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.326+0000 7fdce52fe640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce015f6a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fdcd402a340 tx=0x7fdcd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.326+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4042020 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.326+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd4003a20 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.326+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce00c92a0 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.326+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4003df0 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.326+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce00c9860 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.327+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce0073a00 con 0x7fdce00d34c0 2024-04-25T08:39:49.322 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.327+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcd4045020 con 0x7fdce00d34c0 2024-04-25T08:39:49.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.328+0000 7fdcde7fc640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdccc06b500 0x7fdccc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.328+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdcd40aaec0 con 0x7fdce00d34c0 2024-04-25T08:39:49.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.328+0000 7fdce5aff640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdccc06b500 0x7fdccc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.323 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.328+0000 7fdce5aff640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdccc06b500 0x7fdccc06d9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80016e0 tx=0x7fdcd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.325 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.332+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcd404c030 con 0x7fdce00d34c0 2024-04-25T08:39:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.514+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdce00ced60 con 0x7fdce00d34c0 2024-04-25T08:39:49.507 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.514+0000 7fdcde7fc640 1 -- 172.21.15.60:0/232542952 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcd4079bb0 con 0x7fdce00d34c0 2024-04-25T08:39:49.507 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdccc06b500 msgr2=0x7fdccc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdccc06b500 0x7fdccc06d9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80016e0 tx=0x7fdcd8009040 comp rx=0 tx=0).stop 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 msgr2=0x7fdce015f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce015f6a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fdcd402a340 tx=0x7fdcd40046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 shutdown_connections 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fdccc06b500 0x7fdccc06d9c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdce015dac0 0x7fdce015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdce00d34c0 0x7fdce015f6a0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 --2- 172.21.15.60:0/232542952 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdce00d08e0 0x7fdce00c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 >> 172.21.15.60:0/232542952 conn(0x7fdce0061c30 msgr2=0x7fdce00d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 shutdown_connections 2024-04-25T08:39:49.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.516+0000 7fdce7d63640 1 -- 172.21.15.60:0/232542952 wait complete. 2024-04-25T08:39:49.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:39:49.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.631+0000 7fa9c1a12640 1 Processor -- start 2024-04-25T08:39:49.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.631+0000 7fa9c1a12640 1 -- start start 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9c1a12640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9bc0ced80 0x7fa9bc0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9c1a12640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9bc161710 0x7fa9bc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9c1a12640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc15e000 con 0x7fa9bc15dc20 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9c1a12640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0d1de0 con 0x7fa9bc161710 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9c1a12640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc161c60 con 0x7fa9bc0ced80 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9bc161710 0x7fa9bc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa9bc161710 0x7fa9bc0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54812/0 (socket says 172.21.15.60:54812) 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 -- 172.21.15.60:0/3374367068 learned_addr learned my addr 172.21.15.60:0/3374367068 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9ba7fc640 1 --2- 172.21.15.60:0/3374367068 >> v2:172.21.15.100:3300/0 conn(0x7fa9bc0ced80 0x7fa9bc0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 -- 172.21.15.60:0/3374367068 >> v2:172.21.15.100:3300/0 conn(0x7fa9bc0ced80 msgr2=0x7fa9bc0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 --2- 172.21.15.60:0/3374367068 >> v2:172.21.15.100:3300/0 conn(0x7fa9bc0ced80 0x7fa9bc0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 -- 172.21.15.60:0/3374367068 >> v1:172.21.15.100:6789/0 conn(0x7fa9bc15dc20 legacy=0x7fa9bc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9bb7fe640 1 -- 172.21.15.60:0/3374367068 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc161f40 con 0x7fa9bc161710 2024-04-25T08:39:49.627 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.632+0000 7fa9ba7fc640 1 --2- 172.21.15.60:0/3374367068 >> v2:172.21.15.100:3300/0 conn(0x7fa9bc0ced80 0x7fa9bc0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.633+0000 7fa9bb7fe640 1 --2- 172.21.15.60:0/3374367068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc161710 0x7fa9bc0d2820 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc06da90 tx=0x7fa9b402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd74bba9ffca1342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.633+0000 7fa9b9ffb640 1 -- 172.21.15.60:0/3374367068 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4036070 con 0x7fa9bc161710 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.633+0000 7fa9b9ffb640 1 -- 172.21.15.60:0/3374367068 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b4031600 con 0x7fa9bc161710 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.633+0000 7fa9b9ffb640 1 -- 172.21.15.60:0/3374367068 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40319d0 con 0x7fa9bc161710 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 -- 172.21.15.60:0/3374367068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc161710 msgr2=0x7fa9bc0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/3374367068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc161710 0x7fa9bc0d2820 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc06da90 tx=0x7fa9b402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 -- 172.21.15.60:0/3374367068 shutdown_connections 2024-04-25T08:39:49.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/3374367068 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc161710 0x7fa9bc0d2820 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/3374367068 >> v2:172.21.15.100:3300/0 conn(0x7fa9bc0ced80 0x7fa9bc0d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 -- 172.21.15.60:0/3374367068 >> 172.21.15.60:0/3374367068 conn(0x7fa9bc0d0420 msgr2=0x7fa9bc0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 -- 172.21.15.60:0/3374367068 shutdown_connections 2024-04-25T08:39:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.634+0000 7fa9c1a12640 1 -- 172.21.15.60:0/3374367068 wait complete. 2024-04-25T08:39:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.635+0000 7fa9c1a12640 1 Processor -- start 2024-04-25T08:39:49.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.635+0000 7fa9c1a12640 1 -- start start 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9c1a12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9bc0ced80 0x7fa9bc1f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9c1a12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 0x7fa9bc1e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9c1a12640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9bc161710 0x7fa9bc0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9c1a12640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b5e50 con 0x7fa9bc15dc20 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9c1a12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc162500 con 0x7fa9bc0ced80 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9c1a12640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b5fd0 con 0x7fa9bc161710 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 0x7fa9bc1e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 0x7fa9bc1e7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54822/0 (socket says 172.21.15.60:54822) 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 -- 172.21.15.60:0/2237624465 learned_addr learned my addr 172.21.15.60:0/2237624465 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9baffd640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9bc0ced80 0x7fa9bc1f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9bb7fe640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9bc161710 0x7fa9bc0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.631 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 -- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9bc161710 msgr2=0x7fa9bc0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9bc161710 0x7fa9bc0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 -- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9bc0ced80 msgr2=0x7fa9bc1f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9bc0ced80 0x7fa9bc1f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.636+0000 7fa9ba7fc640 1 -- 172.21.15.60:0/2237624465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc1f3b10 con 0x7fa9bc15dc20 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.637+0000 7fa9ba7fc640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 0x7fa9bc1e7f20 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c820 tx=0x7fa9b000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.637+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0015070 con 0x7fa9bc15dc20 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.637+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b0007a30 con 0x7fa9bc15dc20 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.637+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b0007de0 con 0x7fa9bc15dc20 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.637+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc0c8a20 con 0x7fa9bc15dc20 2024-04-25T08:39:49.632 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.637+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc0c8fe0 con 0x7fa9bc15dc20 2024-04-25T08:39:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.638+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc0c9420 con 0x7fa9bc15dc20 2024-04-25T08:39:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.638+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9b0018070 con 0x7fa9bc15dc20 2024-04-25T08:39:49.633 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.639+0000 7fa9a3fff640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9a806b4d0 0x7fa9a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.639+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9b008a1b0 con 0x7fa9bc15dc20 2024-04-25T08:39:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.639+0000 7fa9baffd640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9a806b4d0 0x7fa9a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.634 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.640+0000 7fa9baffd640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9a806b4d0 0x7fa9a806d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.643+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9b0058f30 con 0x7fa9bc15dc20 2024-04-25T08:39:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.825+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9bc073a00 con 0x7fa9bc15dc20 2024-04-25T08:39:49.818 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.825+0000 7fa9a3fff640 1 -- 172.21.15.60:0/2237624465 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9b0058830 con 0x7fa9bc15dc20 2024-04-25T08:39:49.819 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9a806b4d0 msgr2=0x7fa9a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9a806b4d0 0x7fa9a806d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac008040 comp rx=0 tx=0).stop 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 msgr2=0x7fa9bc1e7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 0x7fa9bc1e7f20 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c820 tx=0x7fa9b000cce0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 shutdown_connections 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa9bc161710 0x7fa9bc0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9bc15dc20 0x7fa9bc1e7f20 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa9a806b4d0 0x7fa9a806d990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.821 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 --2- 172.21.15.60:0/2237624465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9bc0ced80 0x7fa9bc1f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 >> 172.21.15.60:0/2237624465 conn(0x7fa9bc0d0420 msgr2=0x7fa9bc0d2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 shutdown_connections 2024-04-25T08:39:49.822 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.827+0000 7fa9c1a12640 1 -- 172.21.15.60:0/2237624465 wait complete. 2024-04-25T08:39:49.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:39:49.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.941+0000 7f2bf1ee1640 1 Processor -- start 2024-04-25T08:39:49.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.941+0000 7f2bf1ee1640 1 -- start start 2024-04-25T08:39:49.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bf1ee1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2bec06e360 0x7f2bec06e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bf1ee1640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2bec06bd20 0x7f2bec06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bf1ee1640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec15e020 con 0x7f2bec06ec80 2024-04-25T08:39:49.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bf1ee1640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec06b890 con 0x7f2bec06bd20 2024-04-25T08:39:49.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bf1ee1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec06ac80 con 0x7f2bec06e360 2024-04-25T08:39:49.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bebfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f2bec06bd20 0x7f2bec06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2beaffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f2bec06ec80 0x7f2bec0b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:36498/0 (socket says 172.21.15.60:36498) 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2beaffd640 1 -- 172.21.15.60:0/3990878382 learned_addr learned my addr 172.21.15.60:0/3990878382 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.942+0000 7f2bebfff640 1 --2- 172.21.15.60:0/3990878382 >> v2:172.21.15.60:3300/0 conn(0x7f2bec06bd20 0x7f2bec06c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54824/0 (socket says 172.21.15.60:54824) 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2beb7fe640 1 --2- 172.21.15.60:0/3990878382 >> v2:172.21.15.100:3300/0 conn(0x7f2bec06e360 0x7f2bec06e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2bebfff640 1 -- 172.21.15.60:0/3990878382 >> v2:172.21.15.100:3300/0 conn(0x7f2bec06e360 msgr2=0x7f2bec06e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2bebfff640 1 --2- 172.21.15.60:0/3990878382 >> v2:172.21.15.100:3300/0 conn(0x7f2bec06e360 0x7f2bec06e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2bebfff640 1 -- 172.21.15.60:0/3990878382 >> v1:172.21.15.60:6789/0 conn(0x7f2bec06ec80 legacy=0x7f2bec0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2bebfff640 1 -- 172.21.15.60:0/3990878382 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec06af60 con 0x7f2bec06bd20 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2bea7fc640 1 -- 172.21.15.60:0/3990878382 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586271220 0 0) 0x7f2bec15e020 con 0x7f2bec06ec80 2024-04-25T08:39:49.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2beb7fe640 1 --2- 172.21.15.60:0/3990878382 >> v2:172.21.15.100:3300/0 conn(0x7f2bec06e360 0x7f2bec06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:49.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.943+0000 7f2bebfff640 1 --2- 172.21.15.60:0/3990878382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec06c1b0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f2bec072e90 tx=0x7f2be402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5f7e7e8f5913bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bea7fc640 1 -- 172.21.15.60:0/3990878382 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4007740 con 0x7f2bec06bd20 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bea7fc640 1 -- 172.21.15.60:0/3990878382 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be40078e0 con 0x7f2bec06bd20 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bea7fc640 1 -- 172.21.15.60:0/3990878382 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4007cb0 con 0x7f2bec06bd20 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/3990878382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 msgr2=0x7f2bec06c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/3990878382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec06c1b0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f2bec072e90 tx=0x7f2be402a710 comp rx=0 tx=0).stop 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/3990878382 shutdown_connections 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/3990878382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec06c1b0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/3990878382 >> v2:172.21.15.100:3300/0 conn(0x7f2bec06e360 0x7f2bec06e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/3990878382 >> 172.21.15.60:0/3990878382 conn(0x7f2bec0cfec0 msgr2=0x7f2bec0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/3990878382 shutdown_connections 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.944+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/3990878382 wait complete. 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.945+0000 7f2bf1ee1640 1 Processor -- start 2024-04-25T08:39:49.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 -- start start 2024-04-25T08:39:49.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec1482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bec06e360 0x7f2bec14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bec06ec80 0x7f2bec14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0b2fd0 con 0x7f2bec06bd20 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec06b520 con 0x7f2bec06ec80 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2bf1ee1640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0b3150 con 0x7f2bec06e360 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2beb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec1482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2beb7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec1482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54840/0 (socket says 172.21.15.60:54840) 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.946+0000 7f2beb7fe640 1 -- 172.21.15.60:0/1251162081 learned_addr learned my addr 172.21.15.60:0/1251162081 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2beb7fe640 1 -- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bec06e360 msgr2=0x7f2bec14acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2beb7fe640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bec06e360 0x7f2bec14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2beb7fe640 1 -- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bec06ec80 msgr2=0x7f2bec14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2beb7fe640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bec06ec80 0x7f2bec14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2beb7fe640 1 -- 172.21.15.60:0/1251162081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec149ce0 con 0x7f2bec06bd20 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2beb7fe640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec1482e0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c7f0 tx=0x7f2bdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc015070 con 0x7f2bec06bd20 2024-04-25T08:39:49.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc007a30 con 0x7f2bec06bd20 2024-04-25T08:39:49.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec149fc0 con 0x7f2bec06bd20 2024-04-25T08:39:49.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc007e00 con 0x7f2bec06bd20 2024-04-25T08:39:49.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.947+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec14a580 con 0x7f2bec06bd20 2024-04-25T08:39:49.943 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.948+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec073a00 con 0x7f2bec06bd20 2024-04-25T08:39:49.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.948+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bdc018070 con 0x7f2bec06bd20 2024-04-25T08:39:49.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.949+0000 7f2bd3fff640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2bd806b4d0 0x7f2bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.949+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bdc08a390 con 0x7f2bec06bd20 2024-04-25T08:39:49.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.949+0000 7f2beaffd640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2bd806b4d0 0x7f2bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.950+0000 7f2beaffd640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2bd806b4d0 0x7f2bd806d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2be0007fd0 tx=0x7f2be0007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:49.953+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bdc059080 con 0x7f2bec06bd20 2024-04-25T08:39:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.132+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bec0b3590 con 0x7f2bec06bd20 2024-04-25T08:39:50.126 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.133+0000 7f2bd3fff640 1 -- 172.21.15.60:0/1251162081 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bdc058980 con 0x7f2bec06bd20 2024-04-25T08:39:50.126 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:39:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.134+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2bd806b4d0 msgr2=0x7f2bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2bd806b4d0 0x7f2bd806d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2be0007fd0 tx=0x7f2be0007960 comp rx=0 tx=0).stop 2024-04-25T08:39:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 msgr2=0x7f2bec1482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec1482e0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c7f0 tx=0x7f2bdc00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.128 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 shutdown_connections 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f2bd806b4d0 0x7f2bd806d990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2bec06ec80 0x7f2bec14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2bec06e360 0x7f2bec14acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 --2- 172.21.15.60:0/1251162081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec06bd20 0x7f2bec1482e0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 >> 172.21.15.60:0/1251162081 conn(0x7f2bec0cfec0 msgr2=0x7f2bec15d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 shutdown_connections 2024-04-25T08:39:50.129 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.135+0000 7f2bf1ee1640 1 -- 172.21.15.60:0/1251162081 wait complete. 2024-04-25T08:39:50.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:39:50.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.247+0000 7f7fa537e640 1 Processor -- start 2024-04-25T08:39:50.241 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.248+0000 7f7fa537e640 1 -- start start 2024-04-25T08:39:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.248+0000 7f7fa537e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015dc20 0x7f7fa00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.248+0000 7f7fa537e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7fa00d32f0 0x7f7fa00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.248+0000 7f7fa537e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa015e000 con 0x7f7fa0161710 2024-04-25T08:39:50.242 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.248+0000 7f7fa537e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00d1de0 con 0x7f7fa00d32f0 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.248+0000 7f7fa537e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0161c60 con 0x7f7fa015dc20 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015dc20 0x7f7fa00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7fa00d32f0 0x7f7fa00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9e7fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7fa0161710 0x7f7fa006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:45134/0 (socket says 172.21.15.60:45134) 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9e7fc640 1 -- 172.21.15.60:0/2301686918 learned_addr learned my addr 172.21.15.60:0/2301686918 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015dc20 0x7f7fa00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45430/0 (socket says 172.21.15.60:45430) 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9f7fe640 1 -- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015dc20 msgr2=0x7f7fa00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9f7fe640 1 --2- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015dc20 0x7f7fa00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9f7fe640 1 -- 172.21.15.60:0/2301686918 >> v1:172.21.15.100:6789/0 conn(0x7f7fa0161710 legacy=0x7f7fa006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9dffb640 1 -- 172.21.15.60:0/2301686918 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363534878 0 0) 0x7f7fa015e000 con 0x7f7fa0161710 2024-04-25T08:39:50.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.249+0000 7f7f9f7fe640 1 -- 172.21.15.60:0/2301686918 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0161f40 con 0x7f7fa00d32f0 2024-04-25T08:39:50.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7f9f7fe640 1 --2- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00cebb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009d90 tx=0x7f7f9002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c73a6ba47bfba7e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7f9dffb640 1 -- 172.21.15.60:0/2301686918 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90007740 con 0x7f7fa00d32f0 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7f9dffb640 1 -- 172.21.15.60:0/2301686918 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f900078e0 con 0x7f7fa00d32f0 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7f9dffb640 1 -- 172.21.15.60:0/2301686918 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90007cb0 con 0x7f7fa00d32f0 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7fa537e640 1 -- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 msgr2=0x7f7fa00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7fa537e640 1 --2- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00cebb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009d90 tx=0x7f7f9002a660 comp rx=0 tx=0).stop 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.250+0000 7f7fa537e640 1 -- 172.21.15.60:0/2301686918 shutdown_connections 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.251+0000 7f7fa537e640 1 --2- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00cebb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.251+0000 7f7fa537e640 1 --2- 172.21.15.60:0/2301686918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa015dc20 0x7f7fa00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.251+0000 7f7fa537e640 1 -- 172.21.15.60:0/2301686918 >> 172.21.15.60:0/2301686918 conn(0x7f7fa00d0420 msgr2=0x7f7fa00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.251+0000 7f7fa537e640 1 -- 172.21.15.60:0/2301686918 shutdown_connections 2024-04-25T08:39:50.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.251+0000 7f7fa537e640 1 -- 172.21.15.60:0/2301686918 wait complete. 2024-04-25T08:39:50.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.252+0000 7f7fa537e640 1 Processor -- start 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.252+0000 7f7fa537e640 1 -- start start 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 0x7f7fa00cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa0161710 0x7f7fa00cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00b5e50 con 0x7f7fa015dc20 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0162500 con 0x7f7fa00d32f0 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00b5fd0 con 0x7f7fa0161710 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 0x7f7fa00cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 0x7f7fa00cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54842/0 (socket says 172.21.15.60:54842) 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 -- 172.21.15.60:0/4248250448 learned_addr learned my addr 172.21.15.60:0/4248250448 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9f7fe640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa0161710 0x7f7fa00cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9effd640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 -- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa0161710 msgr2=0x7f7fa00cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.248 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa0161710 0x7f7fa00cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 -- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 msgr2=0x7f7fa00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 -- 172.21.15.60:0/4248250448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa015fd80 con 0x7f7fa015dc20 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f9e7fc640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 0x7f7fa00cc340 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800c430 tx=0x7f7f8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f880074e0 con 0x7f7fa015dc20 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f880047e0 con 0x7f7fa015dc20 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa0160060 con 0x7f7fa015dc20 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.253+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88004bd0 con 0x7f7fa015dc20 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.254+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa0160620 con 0x7f7fa015dc20 2024-04-25T08:39:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.255+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa0160820 con 0x7f7fa015dc20 2024-04-25T08:39:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.255+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f88011070 con 0x7f7fa015dc20 2024-04-25T08:39:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.256+0000 7f7f977fe640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8006b4d0 0x7f7f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.256+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f88089f40 con 0x7f7fa015dc20 2024-04-25T08:39:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.256+0000 7f7f9effd640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8006b4d0 0x7f7f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.256+0000 7f7f9effd640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8006b4d0 0x7f7f8006d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009aa0 tx=0x7f7f98009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.260+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fa0160820 con 0x7f7fa015dc20 2024-04-25T08:39:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.456+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7fa0073a00 con 0x7f7fa015dc20 2024-04-25T08:39:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.457+0000 7f7f977fe640 1 -- 172.21.15.60:0/4248250448 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7f88058560 con 0x7f7fa015dc20 2024-04-25T08:39:50.450 INFO:teuthology.orchestra.run.smithi060.stderr:listed 0 entries 2024-04-25T08:39:50.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8006b4d0 msgr2=0x7f7f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8006b4d0 0x7f7f8006d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7f98009aa0 tx=0x7f7f98009190 comp rx=0 tx=0).stop 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 msgr2=0x7f7fa00cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 0x7f7fa00cc340 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800c430 tx=0x7f7f8800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 shutdown_connections 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7fa0161710 0x7f7fa00cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7fa015dc20 0x7f7fa00cc340 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7f8006b4d0 0x7f7f8006d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 --2- 172.21.15.60:0/4248250448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fa00d32f0 0x7f7fa00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 >> 172.21.15.60:0/4248250448 conn(0x7f7fa00d0420 msgr2=0x7f7fa00b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 shutdown_connections 2024-04-25T08:39:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:50.459+0000 7f7fa537e640 1 -- 172.21.15.60:0/4248250448 wait complete. 2024-04-25T08:39:50.463 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:39:50.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:50.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:51.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:39:51.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:51.082 DEBUG:teuthology.orchestra.run.smithi060:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:39:51.097 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:51.097 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:51.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:51.113 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:51.113 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:51.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:51.152 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:51.152 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:51.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:51.169 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.299+0000 7f20e9807640 1 Processor -- start 2024-04-25T08:39:51.292 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.299+0000 7f20e9807640 1 -- start start 2024-04-25T08:39:51.293 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e9807640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20e415dd20 0x7f20e40b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e9807640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e40d11b0 0x7f20e40d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e9807640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e415eac0 con 0x7f20e415e100 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e9807640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40d1c00 con 0x7f20e415dd20 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e9807640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e415eda0 con 0x7f20e40d11b0 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e37fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e40d11b0 0x7f20e40d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20e415dd20 0x7f20e40b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e37fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e40d11b0 0x7f20e40d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:45444/0 (socket says 172.21.15.60:45444) 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f20e415dd20 0x7f20e40b96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:43986/0 (socket says 172.21.15.60:43986) 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e37fe640 1 -- 172.21.15.60:0/4060926744 learned_addr learned my addr 172.21.15.60:0/4060926744 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e27fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f20e415e100 0x7f20e40d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:45138/0 (socket says 172.21.15.60:45138) 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e2ffd640 1 -- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e40d11b0 msgr2=0x7f20e40d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e2ffd640 1 --2- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e40d11b0 0x7f20e40d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.294 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e2ffd640 1 -- 172.21.15.60:0/4060926744 >> v1:172.21.15.100:6789/0 conn(0x7f20e415e100 legacy=0x7f20e40d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.295 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.300+0000 7f20e2ffd640 1 -- 172.21.15.60:0/4060926744 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e415f080 con 0x7f20e415dd20 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.301+0000 7f20e2ffd640 1 --2- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e40b96d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f20d40097a0 tx=0x7f20d402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7617cab64fc955e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.301+0000 7f20e1ffb640 1 -- 172.21.15.60:0/4060926744 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4037070 con 0x7f20e415dd20 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.301+0000 7f20e1ffb640 1 -- 172.21.15.60:0/4060926744 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d402ae50 con 0x7f20e415dd20 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.301+0000 7f20e1ffb640 1 -- 172.21.15.60:0/4060926744 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4032660 con 0x7f20e415dd20 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 -- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 msgr2=0x7f20e40b96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 --2- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e40b96d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f20d40097a0 tx=0x7f20d402a100 comp rx=0 tx=0).stop 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 -- 172.21.15.60:0/4060926744 shutdown_connections 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 --2- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e40d11b0 0x7f20e40d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 --2- 172.21.15.60:0/4060926744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e40b96d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 -- 172.21.15.60:0/4060926744 >> 172.21.15.60:0/4060926744 conn(0x7f20e4061a20 msgr2=0x7f20e4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:51.296 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 -- 172.21.15.60:0/4060926744 shutdown_connections 2024-04-25T08:39:51.297 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.302+0000 7f20e9807640 1 -- 172.21.15.60:0/4060926744 wait complete. 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.303+0000 7f20e9807640 1 Processor -- start 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.303+0000 7f20e9807640 1 -- start start 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e9807640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 0x7f20e414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e9807640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e9807640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e415e100 0x7f20e40cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e9807640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e415e710 con 0x7f20e40d11b0 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e9807640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e415e590 con 0x7f20e415dd20 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e9807640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41e7350 con 0x7f20e415e100 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 0x7f20e414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 0x7f20e414fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54844/0 (socket says 172.21.15.60:54844) 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 -- 172.21.15.60:0/1013477948 learned_addr learned my addr 172.21.15.60:0/1013477948 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e37fe640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e415e100 0x7f20e40cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.299 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e27fc640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 -- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e415e100 msgr2=0x7f20e40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e415e100 0x7f20e40cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 -- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 msgr2=0x7f20e415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.304+0000 7f20e2ffd640 1 -- 172.21.15.60:0/1013477948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40cbab0 con 0x7f20e40d11b0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.305+0000 7f20e2ffd640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 0x7f20e414fb20 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f20d40098c0 tx=0x7f20d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.305+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4041030 con 0x7f20e40d11b0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.305+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d402ac50 con 0x7f20e40d11b0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.305+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e40cbd90 con 0x7f20e40d11b0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.305+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d4037040 con 0x7f20e40d11b0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.305+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e41e6470 con 0x7f20e40d11b0 2024-04-25T08:39:51.300 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.306+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4073990 con 0x7f20e40d11b0 2024-04-25T08:39:51.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.306+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20d4044070 con 0x7f20e40d11b0 2024-04-25T08:39:51.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.307+0000 7f20cbfff640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20d006b4d0 0x7f20d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.307+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20d40aa8b0 con 0x7f20e40d11b0 2024-04-25T08:39:51.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.307+0000 7f20e27fc640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20d006b4d0 0x7f20d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.307+0000 7f20e27fc640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20d006b4d0 0x7f20d006d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f20e40ca440 tx=0x7f20d8007ce0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.311+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20d4044e40 con 0x7f20e40d11b0 2024-04-25T08:39:51.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.528+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20e40b9780 con 0x7f20e40d11b0 2024-04-25T08:39:51.522 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.528+0000 7f20cbfff640 1 -- 172.21.15.60:0/1013477948 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f20d4079750 con 0x7f20e40d11b0 2024-04-25T08:39:51.522 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:51.522 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:47.785553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8716":{"gid":8716,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:51.523 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-04-25T08:39:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20d006b4d0 msgr2=0x7f20d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20d006b4d0 0x7f20d006d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f20e40ca440 tx=0x7f20d8007ce0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 msgr2=0x7f20e414fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 0x7f20e414fb20 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f20d40098c0 tx=0x7f20d40046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 shutdown_connections 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f20e415e100 0x7f20e40cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f20d006b4d0 0x7f20d006d990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e415dd20 0x7f20e415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 --2- 172.21.15.60:0/1013477948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f20e40d11b0 0x7f20e414fb20 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 >> 172.21.15.60:0/1013477948 conn(0x7f20e4061a20 msgr2=0x7f20e40cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:51.525 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 shutdown_connections 2024-04-25T08:39:51.526 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.530+0000 7f20e9807640 1 -- 172.21.15.60:0/1013477948 wait complete. 2024-04-25T08:39:51.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:51.636 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.643+0000 7fcc72a12640 1 Processor -- start 2024-04-25T08:39:51.637 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.643+0000 7fcc72a12640 1 -- start start 2024-04-25T08:39:51.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc72a12640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcc6c15dd30 0x7fcc6c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.638 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc72a12640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcc6c0d0ad0 0x7fcc6c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc72a12640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0d1df0 con 0x7fcc6c0b9770 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc72a12640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c0d20d0 con 0x7fcc6c15dd30 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc72a12640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c0d23b0 con 0x7fcc6c0d0ad0 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcc6c15dd30 0x7fcc6c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fcc6c15dd30 0x7fcc6c0b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54856/0 (socket says 172.21.15.60:54856) 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 -- 172.21.15.60:0/3310391607 learned_addr learned my addr 172.21.15.60:0/3310391607 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc70faf640 1 --2- 172.21.15.60:0/3310391607 >> v2:172.21.15.100:3300/0 conn(0x7fcc6c0d0ad0 0x7fcc6c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 -- 172.21.15.60:0/3310391607 >> v2:172.21.15.100:3300/0 conn(0x7fcc6c0d0ad0 msgr2=0x7fcc6c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 --2- 172.21.15.60:0/3310391607 >> v2:172.21.15.100:3300/0 conn(0x7fcc6c0d0ad0 0x7fcc6c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 -- 172.21.15.60:0/3310391607 >> v1:172.21.15.100:6789/0 conn(0x7fcc6c0b9770 legacy=0x7fcc6c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc6bfff640 1 -- 172.21.15.60:0/3310391607 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0d2690 con 0x7fcc6c15dd30 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.644+0000 7fcc70faf640 1 --2- 172.21.15.60:0/3310391607 >> v2:172.21.15.100:3300/0 conn(0x7fcc6c0d0ad0 0x7fcc6c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.645+0000 7fcc6bfff640 1 --2- 172.21.15.60:0/3310391607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0b9230 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0ce200 tx=0x7fcc6002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e667154359d6df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.645+0000 7fcc6affd640 1 -- 172.21.15.60:0/3310391607 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60037070 con 0x7fcc6c15dd30 2024-04-25T08:39:51.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.645+0000 7fcc6affd640 1 -- 172.21.15.60:0/3310391607 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6002ae20 con 0x7fcc6c15dd30 2024-04-25T08:39:51.640 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.645+0000 7fcc6affd640 1 -- 172.21.15.60:0/3310391607 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60032660 con 0x7fcc6c15dd30 2024-04-25T08:39:51.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 -- 172.21.15.60:0/3310391607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 msgr2=0x7fcc6c0b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.642 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 --2- 172.21.15.60:0/3310391607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0b9230 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0ce200 tx=0x7fcc6002a110 comp rx=0 tx=0).stop 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 -- 172.21.15.60:0/3310391607 shutdown_connections 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 --2- 172.21.15.60:0/3310391607 >> v2:172.21.15.100:3300/0 conn(0x7fcc6c0d0ad0 0x7fcc6c0b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 --2- 172.21.15.60:0/3310391607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0b9230 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 -- 172.21.15.60:0/3310391607 >> 172.21.15.60:0/3310391607 conn(0x7fcc6c061c30 msgr2=0x7fcc6c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 -- 172.21.15.60:0/3310391607 shutdown_connections 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.646+0000 7fcc72a12640 1 -- 172.21.15.60:0/3310391607 wait complete. 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.647+0000 7fcc72a12640 1 Processor -- start 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.647+0000 7fcc72a12640 1 -- start start 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc72a12640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6c0b9770 0x7fcc6c1f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc72a12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6c0d0ad0 0x7fcc6c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc72a12640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc72a12640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0d1b90 con 0x7fcc6c15dd30 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc72a12640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0d1a10 con 0x7fcc6c0d0ad0 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc72a12640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0d2e90 con 0x7fcc6c0b9770 2024-04-25T08:39:51.643 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:54868/0 (socket says 172.21.15.60:54868) 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 -- 172.21.15.60:0/36975296 learned_addr learned my addr 172.21.15.60:0/36975296 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc6bfff640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6c0b9770 0x7fcc6c1f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc6b7fe640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6c0d0ad0 0x7fcc6c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 -- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6c0b9770 msgr2=0x7fcc6c1f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6c0b9770 0x7fcc6c1f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 -- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6c0d0ad0 msgr2=0x7fcc6c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6c0d0ad0 0x7fcc6c155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 -- 172.21.15.60:0/36975296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c156050 con 0x7fcc6c15dd30 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.648+0000 7fcc70faf640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0c7f10 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400cef0 tx=0x7fcc6400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.649+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6401a070 con 0x7fcc6c15dd30 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.649+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6400d860 con 0x7fcc6c15dd30 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.649+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c156330 con 0x7fcc6c15dd30 2024-04-25T08:39:51.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.649+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400dc50 con 0x7fcc6c15dd30 2024-04-25T08:39:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.649+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c153380 con 0x7fcc6c15dd30 2024-04-25T08:39:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.650+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c0b6790 con 0x7fcc6c15dd30 2024-04-25T08:39:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.650+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc6401d070 con 0x7fcc6c15dd30 2024-04-25T08:39:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.651+0000 7fcc68ff9640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.651+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc6408ac20 con 0x7fcc6c15dd30 2024-04-25T08:39:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.651+0000 7fcc6bfff640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.651+0000 7fcc6bfff640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5806b4d0 0x7fcc5806d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcc60034000 tx=0x7fcc6002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.655+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc6c0b6790 con 0x7fcc6c15dd30 2024-04-25T08:39:51.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.840+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc6c073990 con 0x7fcc6c15dd30 2024-04-25T08:39:51.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.841+0000 7fcc68ff9640 1 -- 172.21.15.60:0/36975296 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fcc640599d0 con 0x7fcc6c15dd30 2024-04-25T08:39:51.834 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:51.834 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":145,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:27.531391+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5806b4d0 msgr2=0x7fcc5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5806b4d0 0x7fcc5806d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcc60034000 tx=0x7fcc6002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:51.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 msgr2=0x7fcc6c0c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0c7f10 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400cef0 tx=0x7fcc6400a710 comp rx=0 tx=0).stop 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 shutdown_connections 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc6c15dd30 0x7fcc6c0c7f10 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fcc5806b4d0 0x7fcc5806d990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc6c0d0ad0 0x7fcc6c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 --2- 172.21.15.60:0/36975296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcc6c0b9770 0x7fcc6c1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 >> 172.21.15.60:0/36975296 conn(0x7fcc6c061c30 msgr2=0x7fcc6c0c9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 shutdown_connections 2024-04-25T08:39:51.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.843+0000 7fcc72a12640 1 -- 172.21.15.60:0/36975296 wait complete. 2024-04-25T08:39:51.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:29:11.440271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:39:24.176721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '144', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:39:24.866339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:51.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.958+0000 7f87cd142640 1 Processor -- start 2024-04-25T08:39:51.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.959+0000 7f87cd142640 1 -- start start 2024-04-25T08:39:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.959+0000 7f87cd142640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.959+0000 7f87cd142640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87c8161890 0x7f87c80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.959+0000 7f87cd142640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c8161620 con 0x7f87c80b54a0 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.959+0000 7f87cd142640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c80b7120 con 0x7f87c8161890 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.959+0000 7f87cd142640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c8161e00 con 0x7f87c8161240 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87c759e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f87c80b54a0 0x7f87c80d1b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50526/0 (socket says 172.21.15.60:50526) 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87bffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f87c8161890 0x7f87c80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87c759e640 1 -- 172.21.15.60:0/104561017 learned_addr learned my addr 172.21.15.60:0/104561017 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87c6d9d640 1 --2- 172.21.15.60:0/104561017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87c659c640 1 -- 172.21.15.60:0/104561017 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188996669 0 0) 0x7f87c8161620 con 0x7f87c80b54a0 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87c659c640 1 -- 172.21.15.60:0/104561017 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b4003520 con 0x7f87c80b54a0 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87bffff640 1 -- 172.21.15.60:0/104561017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 msgr2=0x7f87c80b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87bffff640 1 --2- 172.21.15.60:0/104561017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87bffff640 1 -- 172.21.15.60:0/104561017 >> v1:172.21.15.60:6789/0 conn(0x7f87c80b54a0 legacy=0x7f87c80d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.954 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.960+0000 7f87bffff640 1 -- 172.21.15.60:0/104561017 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c81620e0 con 0x7f87c8161890 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87bffff640 1 --2- 172.21.15.60:0/104561017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80ceac0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f87b00097a0 tx=0x7f87b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1de76cc215834e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87c659c640 1 -- 172.21.15.60:0/104561017 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0037070 con 0x7f87c8161890 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87c659c640 1 -- 172.21.15.60:0/104561017 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b0032440 con 0x7f87c8161890 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87c659c640 1 -- 172.21.15.60:0/104561017 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b0032810 con 0x7f87c8161890 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 -- 172.21.15.60:0/104561017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 msgr2=0x7f87c80ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 --2- 172.21.15.60:0/104561017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80ceac0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f87b00097a0 tx=0x7f87b002a180 comp rx=0 tx=0).stop 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 -- 172.21.15.60:0/104561017 shutdown_connections 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 --2- 172.21.15.60:0/104561017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80ceac0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 --2- 172.21.15.60:0/104561017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 -- 172.21.15.60:0/104561017 >> 172.21.15.60:0/104561017 conn(0x7f87c80d04a0 msgr2=0x7f87c80d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 -- 172.21.15.60:0/104561017 shutdown_connections 2024-04-25T08:39:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.961+0000 7f87cd142640 1 -- 172.21.15.60:0/104561017 wait complete. 2024-04-25T08:39:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 Processor -- start 2024-04-25T08:39:51.957 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 -- start start 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80b54a0 0x7f87c80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80d15f0 con 0x7f87c8161890 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c81626a0 con 0x7f87c80b54a0 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.963+0000 7f87cd142640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80cf000 con 0x7f87c8161240 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35736/0 (socket says 172.21.15.60:35736) 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 -- 172.21.15.60:0/956307105 learned_addr learned my addr 172.21.15.60:0/956307105 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 -- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 msgr2=0x7f87c80c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.959 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 -- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80b54a0 msgr2=0x7f87c80c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80b54a0 0x7f87c80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 -- 172.21.15.60:0/956307105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80b2980 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87c759e640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80c9e10 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f87c000cb70 tx=0x7f87c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0017070 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c000ad40 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0012620 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c80ca350 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.964+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c80c7ea0 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.965+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c8073a00 con 0x7f87c8161890 2024-04-25T08:39:51.960 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.966+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87c001a070 con 0x7f87c8161890 2024-04-25T08:39:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.966+0000 7f87bf7fe640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.966+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87c008a180 con 0x7f87c8161890 2024-04-25T08:39:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.967+0000 7f87c6d9d640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:51.961 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.967+0000 7f87c6d9d640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f87a806b4d0 0x7f87a806d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006a90 tx=0x7f87b8006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:51.963 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:51.970+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87c0058f00 con 0x7f87c8161890 2024-04-25T08:39:52.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.150+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87c80d2bc0 con 0x7f87c8161890 2024-04-25T08:39:52.143 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.150+0000 7f87bf7fe640 1 -- 172.21.15.60:0/956307105 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87c0058800 con 0x7f87c8161890 2024-04-25T08:39:52.144 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f87a806b4d0 msgr2=0x7f87a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f87a806b4d0 0x7f87a806d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006a90 tx=0x7f87b8006a20 comp rx=0 tx=0).stop 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 msgr2=0x7f87c80c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80c9e10 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f87c000cb70 tx=0x7f87c000a430 comp rx=0 tx=0).stop 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 shutdown_connections 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f87c8161890 0x7f87c80c9e10 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f87c8161240 0x7f87c80c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f87a806b4d0 0x7f87a806d990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 --2- 172.21.15.60:0/956307105 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87c80b54a0 0x7f87c80c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 >> 172.21.15.60:0/956307105 conn(0x7f87c80d04a0 msgr2=0x7f87c80b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 shutdown_connections 2024-04-25T08:39:52.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.152+0000 7f87cd142640 1 -- 172.21.15.60:0/956307105 wait complete. 2024-04-25T08:39:52.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:39:52.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.268+0000 7fb3ffd5c640 1 Processor -- start 2024-04-25T08:39:52.261 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.268+0000 7fb3ffd5c640 1 -- start start 2024-04-25T08:39:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3ffd5c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb3f81452a0 0x7fb3f806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3ffd5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f80cccc0 con 0x7fb3f8144d50 2024-04-25T08:39:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3ffd5c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f80cc140 con 0x7fb3f81446d0 2024-04-25T08:39:52.262 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3ffd5c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f80b2b20 con 0x7fb3f81452a0 2024-04-25T08:39:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3fe2f9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb3f81452a0 0x7fb3f806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3fd2f7640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fb3f8144d50 0x7fb3f806d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50538/0 (socket says 172.21.15.60:50538) 2024-04-25T08:39:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3fd2f7640 1 -- 172.21.15.60:0/4168517716 learned_addr learned my addr 172.21.15.60:0/4168517716 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/4168517716 >> v1:172.21.15.100:6789/0 conn(0x7fb3f81446d0 legacy=0x7fb3f80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/4168517716 >> v1:172.21.15.60:6789/0 conn(0x7fb3f8144d50 legacy=0x7fb3f806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.269+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/4168517716 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f80cc420 con 0x7fb3f81452a0 2024-04-25T08:39:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.270+0000 7fb3fe2f9640 1 --2- 172.21.15.60:0/4168517716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f806dfa0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009d90 tx=0x7fb3f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cca85e2750dfa28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.270+0000 7fb3fcaf6640 1 -- 172.21.15.60:0/4168517716 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0007740 con 0x7fb3f81452a0 2024-04-25T08:39:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.270+0000 7fb3fcaf6640 1 -- 172.21.15.60:0/4168517716 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f00078e0 con 0x7fb3f81452a0 2024-04-25T08:39:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.270+0000 7fb3fcaf6640 1 -- 172.21.15.60:0/4168517716 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0007cb0 con 0x7fb3f81452a0 2024-04-25T08:39:52.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.270+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/4168517716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 msgr2=0x7fb3f806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.270+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/4168517716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f806dfa0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009d90 tx=0x7fb3f002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.271+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/4168517716 shutdown_connections 2024-04-25T08:39:52.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.271+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/4168517716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f806dfa0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.271+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/4168517716 >> 172.21.15.60:0/4168517716 conn(0x7fb3f806abe0 msgr2=0x7fb3f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.271+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/4168517716 shutdown_connections 2024-04-25T08:39:52.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.271+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/4168517716 wait complete. 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.272+0000 7fb3ffd5c640 1 Processor -- start 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.272+0000 7fb3ffd5c640 1 -- start start 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.272+0000 7fb3ffd5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f81446d0 0x7fb3f80a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ffd5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3f8144d50 0x7fb3f80a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ffd5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f80ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ffd5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f80cc9e0 con 0x7fb3f81452a0 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ffd5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f806d400 con 0x7fb3f81446d0 2024-04-25T08:39:52.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ffd5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f8144ab0 con 0x7fb3f8144d50 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f80ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f80ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35760/0 (socket says 172.21.15.60:35760) 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/3975985939 learned_addr learned my addr 172.21.15.60:0/3975985939 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fd2f7640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3f8144d50 0x7fb3f80a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fdaf8640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f81446d0 0x7fb3f80a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3f8144d50 msgr2=0x7fb3f80a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3f8144d50 0x7fb3f80a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f81446d0 msgr2=0x7fb3f80a9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f81446d0 0x7fb3f80a9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 -- 172.21.15.60:0/3975985939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f80aa6c0 con 0x7fb3f81452a0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3fe2f9640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f80ab560 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009d90 tx=0x7fb3f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0040070 con 0x7fb3f81452a0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.273+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f0031c90 con 0x7fb3f81452a0 2024-04-25T08:39:52.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.274+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f80aa9a0 con 0x7fb3f81452a0 2024-04-25T08:39:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.274+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f003c640 con 0x7fb3f81452a0 2024-04-25T08:39:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.274+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f80cd5c0 con 0x7fb3f81452a0 2024-04-25T08:39:52.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.275+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f8073a00 con 0x7fb3f81452a0 2024-04-25T08:39:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.275+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3f0039040 con 0x7fb3f81452a0 2024-04-25T08:39:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.275+0000 7fb3ee7fc640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb3dc06b4d0 0x7fb3dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.275+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3f00ab880 con 0x7fb3f81452a0 2024-04-25T08:39:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.276+0000 7fb3fdaf8640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb3dc06b4d0 0x7fb3dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.276+0000 7fb3fdaf8640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb3dc06b4d0 0x7fb3dc06d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4006fd0 tx=0x7fb3f4009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.280+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3f0079fe0 con 0x7fb3f81452a0 2024-04-25T08:39:52.450 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.457+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3f80afc80 con 0x7fb3f81452a0 2024-04-25T08:39:52.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.457+0000 7fb3ee7fc640 1 -- 172.21.15.60:0/3975985939 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3f007e080 con 0x7fb3f81452a0 2024-04-25T08:39:52.451 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 32 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb3dc06b4d0 msgr2=0x7fb3dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb3dc06b4d0 0x7fb3dc06d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4006fd0 tx=0x7fb3f4009060 comp rx=0 tx=0).stop 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 msgr2=0x7fb3f80ab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f80ab560 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009d90 tx=0x7fb3f0004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 shutdown_connections 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb3f81452a0 0x7fb3f80ab560 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb3f8144d50 0x7fb3f80a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb3dc06b4d0 0x7fb3dc06d990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 --2- 172.21.15.60:0/3975985939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3f81446d0 0x7fb3f80a9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 >> 172.21.15.60:0/3975985939 conn(0x7fb3f806abe0 msgr2=0x7fb3f80af570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 shutdown_connections 2024-04-25T08:39:52.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.459+0000 7fb3ffd5c640 1 -- 172.21.15.60:0/3975985939 wait complete. 2024-04-25T08:39:52.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:39:52.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.574+0000 7fa5948cb640 1 Processor -- start 2024-04-25T08:39:52.567 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.574+0000 7fa5948cb640 1 -- start start 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa5948cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901610f0 0x7fa59006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa5948cb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5900c6960 con 0x7fa5900ce7b0 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa5948cb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa590162260 con 0x7fa5901616f0 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa5948cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5901614d0 con 0x7fa5901610f0 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa58ed9d640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fa5900ce7b0 0x7fa5900d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50544/0 (socket says 172.21.15.60:50544) 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa58ed9d640 1 -- 172.21.15.60:0/401983080 learned_addr learned my addr 172.21.15.60:0/401983080 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa58e59c640 1 --2- 172.21.15.60:0/401983080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901610f0 0x7fa59006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.569 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa58d59a640 1 -- 172.21.15.60:0/401983080 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891095517 0 0) 0x7fa5900c6960 con 0x7fa5900ce7b0 2024-04-25T08:39:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.575+0000 7fa58d59a640 1 -- 172.21.15.60:0/401983080 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c003520 con 0x7fa5900ce7b0 2024-04-25T08:39:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.576+0000 7fa58d59a640 1 -- 172.21.15.60:0/401983080 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771563824 0 0) 0x7fa57c003520 con 0x7fa5900ce7b0 2024-04-25T08:39:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.576+0000 7fa58e59c640 1 -- 172.21.15.60:0/401983080 >> v1:172.21.15.100:6789/0 conn(0x7fa5901616f0 legacy=0x7fa5900ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.576+0000 7fa58e59c640 1 -- 172.21.15.60:0/401983080 >> v1:172.21.15.60:6789/0 conn(0x7fa5900ce7b0 legacy=0x7fa5900d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.570 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.576+0000 7fa58e59c640 1 -- 172.21.15.60:0/401983080 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa590162540 con 0x7fa5901610f0 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa58e59c640 1 --2- 172.21.15.60:0/401983080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901610f0 0x7fa59006c1c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa5800097d0 tx=0x7fa58002a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8937422970d6b4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa58d59a640 1 -- 172.21.15.60:0/401983080 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580037070 con 0x7fa5901610f0 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa58d59a640 1 -- 172.21.15.60:0/401983080 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58002ab40 con 0x7fa5901610f0 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa58d59a640 1 -- 172.21.15.60:0/401983080 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5880019f0 con 0x7fa5901610f0 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 -- 172.21.15.60:0/401983080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901610f0 msgr2=0x7fa59006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 --2- 172.21.15.60:0/401983080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901610f0 0x7fa59006c1c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa5800097d0 tx=0x7fa58002a0b0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 -- 172.21.15.60:0/401983080 shutdown_connections 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 --2- 172.21.15.60:0/401983080 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901610f0 0x7fa59006c1c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.571 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 -- 172.21.15.60:0/401983080 >> 172.21.15.60:0/401983080 conn(0x7fa59006f500 msgr2=0x7fa59006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 -- 172.21.15.60:0/401983080 shutdown_connections 2024-04-25T08:39:52.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.577+0000 7fa5948cb640 1 -- 172.21.15.60:0/401983080 wait complete. 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 Processor -- start 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 -- start start 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 0x7fa5900b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5901610f0 0x7fa59014b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901616f0 0x7fa5900ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900cf690 con 0x7fa5900ce7b0 2024-04-25T08:39:52.574 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900cfce0 con 0x7fa5901610f0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.579+0000 7fa5948cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59006c700 con 0x7fa5901616f0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 0x7fa5900b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 0x7fa5900b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35768/0 (socket says 172.21.15.60:35768) 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 -- 172.21.15.60:0/315078554 learned_addr learned my addr 172.21.15.60:0/315078554 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58ed9d640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901616f0 0x7fa5900ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58dd9b640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5901610f0 0x7fa59014b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 -- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901616f0 msgr2=0x7fa5900ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901616f0 0x7fa5900ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 -- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5901610f0 msgr2=0x7fa59014b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5901610f0 0x7fa59014b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 -- 172.21.15.60:0/315078554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900b5e30 con 0x7fa5900ce7b0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58ed9d640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901616f0 0x7fa5900ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58dd9b640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5901610f0 0x7fa59014b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa58e59c640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 0x7fa5900b58f0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa5800097d0 tx=0x7fa5800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580042020 con 0x7fa5900ce7b0 2024-04-25T08:39:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa580003a40 con 0x7fa5900ce7b0 2024-04-25T08:39:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5900b4850 con 0x7fa5900ce7b0 2024-04-25T08:39:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580037040 con 0x7fa5900ce7b0 2024-04-25T08:39:52.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.580+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5900b4e10 con 0x7fa5900ce7b0 2024-04-25T08:39:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.581+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa590073a00 con 0x7fa5900ce7b0 2024-04-25T08:39:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.582+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa580045020 con 0x7fa5900ce7b0 2024-04-25T08:39:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.582+0000 7fa586ffd640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa57006b4d0 0x7fa57006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.582+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5800aaa50 con 0x7fa5900ce7b0 2024-04-25T08:39:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.582+0000 7fa58dd9b640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa57006b4d0 0x7fa57006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.577 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.583+0000 7fa58dd9b640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa57006b4d0 0x7fa57006d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa578007fd0 tx=0x7fa578009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.579 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.587+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa58004c030 con 0x7fa5900ce7b0 2024-04-25T08:39:52.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.765+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa590071220 con 0x7fa5900ce7b0 2024-04-25T08:39:52.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.766+0000 7fa586ffd640 1 -- 172.21.15.60:0/315078554 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa580079860 con 0x7fa5900ce7b0 2024-04-25T08:39:52.759 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 64 2024-04-25T08:39:52.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa57006b4d0 msgr2=0x7fa57006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa57006b4d0 0x7fa57006d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa578007fd0 tx=0x7fa578009040 comp rx=0 tx=0).stop 2024-04-25T08:39:52.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 msgr2=0x7fa5900b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 0x7fa5900b58f0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa5800097d0 tx=0x7fa5800046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 shutdown_connections 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa5901616f0 0x7fa5900ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa57006b4d0 0x7fa57006d990 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5901610f0 0x7fa59014b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 --2- 172.21.15.60:0/315078554 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5900ce7b0 0x7fa5900b58f0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 >> 172.21.15.60:0/315078554 conn(0x7fa59006f500 msgr2=0x7fa5900b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 shutdown_connections 2024-04-25T08:39:52.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.768+0000 7fa5948cb640 1 -- 172.21.15.60:0/315078554 wait complete. 2024-04-25T08:39:52.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:52.872 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.879+0000 7f331b9b0640 1 Processor -- start 2024-04-25T08:39:52.873 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.879+0000 7f331b9b0640 1 -- start start 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331b9b0640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f33141602e0 0x7f33140ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331b9b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f331415f5f0 0x7f331415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331b9b0640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33140cd710 con 0x7f331415ba00 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331b9b0640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3314099b70 con 0x7f33141602e0 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331b9b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3314099e50 con 0x7f331415f5f0 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f3318f4b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f33141602e0 0x7f33140ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331974c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f331415ba00 0x7f331409a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50556/0 (socket says 172.21.15.60:50556) 2024-04-25T08:39:52.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.880+0000 7f331974c640 1 -- 172.21.15.60:0/1260361918 learned_addr learned my addr 172.21.15.60:0/1260361918 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3319f4d640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f331415f5f0 0x7f331415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3318f4b640 1 -- 172.21.15.60:0/1260361918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f331415f5f0 msgr2=0x7f331415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3318f4b640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f331415f5f0 0x7f331415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3313fff640 1 -- 172.21.15.60:0/1260361918 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279383211 0 0) 0x7f3304001490 con 0x7f331415ba00 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3318f4b640 1 -- 172.21.15.60:0/1260361918 >> v1:172.21.15.60:6789/0 conn(0x7f331415ba00 legacy=0x7f331409a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3318f4b640 1 -- 172.21.15.60:0/1260361918 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331409a130 con 0x7f33141602e0 2024-04-25T08:39:52.875 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3319f4d640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f331415f5f0 0x7f331415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.881+0000 7f3318f4b640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33141602e0 0x7f33140ccda0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f33080097a0 tx=0x7f330802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e4aa25ed57868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f3313fff640 1 -- 172.21.15.60:0/1260361918 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308037070 con 0x7f33141602e0 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f3313fff640 1 -- 172.21.15.60:0/1260361918 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330802adb0 con 0x7f33141602e0 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f3313fff640 1 -- 172.21.15.60:0/1260361918 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308032660 con 0x7f33141602e0 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 -- 172.21.15.60:0/1260361918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33141602e0 msgr2=0x7f33140ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33141602e0 0x7f33140ccda0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f33080097a0 tx=0x7f330802a060 comp rx=0 tx=0).stop 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 -- 172.21.15.60:0/1260361918 shutdown_connections 2024-04-25T08:39:52.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f331415f5f0 0x7f331415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1260361918 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f33141602e0 0x7f33140ccda0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 -- 172.21.15.60:0/1260361918 >> 172.21.15.60:0/1260361918 conn(0x7f33140b3270 msgr2=0x7f33140b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 -- 172.21.15.60:0/1260361918 shutdown_connections 2024-04-25T08:39:52.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.882+0000 7f331b9b0640 1 -- 172.21.15.60:0/1260361918 wait complete. 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.883+0000 7f331b9b0640 1 Processor -- start 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.883+0000 7f331b9b0640 1 -- start start 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331b9b0640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 0x7f331415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331b9b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331415f5f0 0x7f331409b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331b9b0640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33141602e0 0x7f331406f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331b9b0640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140cd460 con 0x7f331415ba00 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331b9b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33140cd2e0 con 0x7f331415f5f0 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331b9b0640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33140c52d0 con 0x7f33141602e0 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331974c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 0x7f331415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331974c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 0x7f331415ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35786/0 (socket says 172.21.15.60:35786) 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331974c640 1 -- 172.21.15.60:0/1859557811 learned_addr learned my addr 172.21.15.60:0/1859557811 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f3319f4d640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33141602e0 0x7f331406f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f3318f4b640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331415f5f0 0x7f331409b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331974c640 1 -- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33141602e0 msgr2=0x7f331406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.884+0000 7f331974c640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33141602e0 0x7f331406f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f331974c640 1 -- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331415f5f0 msgr2=0x7f331409b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f331974c640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331415f5f0 0x7f331409b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f331974c640 1 -- 172.21.15.60:0/1859557811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331415b260 con 0x7f331415ba00 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f331974c640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 0x7f331415ad20 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f330400c850 tx=0x7f330400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304018070 con 0x7f331415ba00 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330400a640 con 0x7f331415ba00 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331406be90 con 0x7f331415ba00 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330400aa70 con 0x7f331415ba00 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.885+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331406c450 con 0x7f331415ba00 2024-04-25T08:39:52.880 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.886+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3314073990 con 0x7f331415ba00 2024-04-25T08:39:52.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.886+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f330401b070 con 0x7f331415ba00 2024-04-25T08:39:52.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.887+0000 7f3311ffb640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f330006b4d0 0x7f330006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.887+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3304089a00 con 0x7f331415ba00 2024-04-25T08:39:52.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.887+0000 7f3318f4b640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f330006b4d0 0x7f330006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.888+0000 7f3318f4b640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f330006b4d0 0x7f330006d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3308034000 tx=0x7f330802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.884 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:52.891+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f331415b260 con 0x7f331415ba00 2024-04-25T08:39:53.101 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.108+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33140b40a0 con 0x7f331415ba00 2024-04-25T08:39:53.102 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.109+0000 7f3311ffb640 1 -- 172.21.15.60:0/1859557811 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f33140cd710 con 0x7f331415ba00 2024-04-25T08:39:53.102 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:53.102 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:47.785553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8716":{"gid":8716,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:53.103 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-04-25T08:39:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f330006b4d0 msgr2=0x7f330006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f330006b4d0 0x7f330006d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3308034000 tx=0x7f330802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 msgr2=0x7f331415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.104 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 0x7f331415ad20 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f330400c850 tx=0x7f330400cd10 comp rx=0 tx=0).stop 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 shutdown_connections 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33141602e0 0x7f331406f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f330006b4d0 0x7f330006d990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f331415f5f0 0x7f331409b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 --2- 172.21.15.60:0/1859557811 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f331415ba00 0x7f331415ad20 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 >> 172.21.15.60:0/1859557811 conn(0x7f33140b3270 msgr2=0x7f331415e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 shutdown_connections 2024-04-25T08:39:53.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.111+0000 7f331b9b0640 1 -- 172.21.15.60:0/1859557811 wait complete. 2024-04-25T08:39:53.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:53.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.222+0000 7fe093729640 1 Processor -- start 2024-04-25T08:39:53.216 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.223+0000 7fe093729640 1 -- start start 2024-04-25T08:39:53.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.223+0000 7fe093729640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe08c0b2cb0 0x7fe08c0d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.218 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.223+0000 7fe093729640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0b97a0 con 0x7fe08c15dbe0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.223+0000 7fe093729640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0d1c50 con 0x7fe08c0d1020 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.223+0000 7fe093729640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c15dfc0 con 0x7fe08c0b2cb0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe0914c5640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe08c15dbe0 0x7fe08c0d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50558/0 (socket says 172.21.15.60:50558) 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe0914c5640 1 -- 172.21.15.60:0/4125148035 learned_addr learned my addr 172.21.15.60:0/4125148035 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83067040 0 0) 0x7fe08c0b97a0 con 0x7fe08c15dbe0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe078003520 con 0x7fe08c15dbe0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe090cc4640 1 --2- 172.21.15.60:0/4125148035 >> v2:172.21.15.100:3300/0 conn(0x7fe08c0b2cb0 0x7fe08c0d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3369045374 0 0) 0x7fe078003520 con 0x7fe08c15dbe0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 >> v2:172.21.15.100:3300/0 conn(0x7fe08c0b2cb0 msgr2=0x7fe08c0d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 --2- 172.21.15.60:0/4125148035 >> v2:172.21.15.100:3300/0 conn(0x7fe08c0b2cb0 0x7fe08c0d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 >> v1:172.21.15.100:6789/0 conn(0x7fe08c0d1020 legacy=0x7fe08c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0d1f30 con 0x7fe08c15dbe0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fe084001670 con 0x7fe08c15dbe0 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 >> v1:172.21.15.60:6789/0 conn(0x7fe08c15dbe0 legacy=0x7fe08c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.219 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780039b0 0x7fe078003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe078004800 0x7fe078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0780051c0 0x7fe07800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07800c030 con 0x7fe0780051c0 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07800c1e0 con 0x7fe078004800 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07800c3e0 con 0x7fe0780039b0 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe091cc6640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0780051c0 0x7fe07800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe0914c5640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780039b0 0x7fe078003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.224+0000 7fe090cc4640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe078004800 0x7fe078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 -- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780039b0 msgr2=0x7fe078003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780039b0 0x7fe078003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 -- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe078004800 msgr2=0x7fe078004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe078004800 0x7fe078004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 -- 172.21.15.60:0/4125148035 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe078003520 con 0x7fe0780051c0 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 -- 172.21.15.60:0/4125148035 reap_dead start 2024-04-25T08:39:53.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.225+0000 7fe091cc6640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0780051c0 0x7fe07800b9c0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fe088001e50 tx=0x7fe078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3c96920a929798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088001a70 con 0x7fe0780051c0 2024-04-25T08:39:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe088003970 con 0x7fe0780051c0 2024-04-25T08:39:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe083fff640 1 -- 172.21.15.60:0/4125148035 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088003d60 con 0x7fe0780051c0 2024-04-25T08:39:53.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 -- 172.21.15.60:0/4125148035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0780051c0 msgr2=0x7fe07800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0780051c0 0x7fe07800b9c0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fe088001e50 tx=0x7fe078005830 comp rx=0 tx=0).stop 2024-04-25T08:39:53.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 -- 172.21.15.60:0/4125148035 shutdown_connections 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 --2- 172.21.15.60:0/4125148035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0780051c0 0x7fe07800b9c0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 -- 172.21.15.60:0/4125148035 >> 172.21.15.60:0/4125148035 conn(0x7fe08c061a20 msgr2=0x7fe08c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 -- 172.21.15.60:0/4125148035 shutdown_connections 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.226+0000 7fe093729640 1 -- 172.21.15.60:0/4125148035 wait complete. 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.227+0000 7fe093729640 1 Processor -- start 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.227+0000 7fe093729640 1 -- start start 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe093729640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780051c0 0x7fe08c15e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe093729640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe08c155c40 0x7fe08c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe093729640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 0x7fe08c158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe093729640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0b92a0 con 0x7fe08c157890 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe093729640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0d1a00 con 0x7fe08c155c40 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe093729640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0b9420 con 0x7fe0780051c0 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe091cc6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 0x7fe08c158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe091cc6640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 0x7fe08c158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35808/0 (socket says 172.21.15.60:35808) 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe091cc6640 1 -- 172.21.15.60:0/1668003519 learned_addr learned my addr 172.21.15.60:0/1668003519 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:53.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe0914c5640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780051c0 0x7fe08c15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.228+0000 7fe090cc4640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe08c155c40 0x7fe08c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe091cc6640 1 -- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780051c0 msgr2=0x7fe08c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe091cc6640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780051c0 0x7fe08c15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe091cc6640 1 -- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe08c155c40 msgr2=0x7fe08c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe091cc6640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe08c155c40 0x7fe08c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe091cc6640 1 -- 172.21.15.60:0/1668003519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c157e30 con 0x7fe08c157890 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe090cc4640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe08c155c40 0x7fe08c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe091cc6640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 0x7fe08c158ed0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fe088001f70 tx=0x7fe0880014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088001a70 con 0x7fe08c157890 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08802ac70 con 0x7fe08c157890 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c158110 con 0x7fe08c157890 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088006d70 con 0x7fe08c157890 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.229+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c1586d0 con 0x7fe08c157890 2024-04-25T08:39:53.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.230+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c073990 con 0x7fe08c157890 2024-04-25T08:39:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.231+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe088007030 con 0x7fe08c157890 2024-04-25T08:39:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.231+0000 7fe081ffb640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe07006b500 0x7fe07006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.231+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0880a9b20 con 0x7fe08c157890 2024-04-25T08:39:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.231+0000 7fe0914c5640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe07006b500 0x7fe07006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.226 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.232+0000 7fe0914c5640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe07006b500 0x7fe07006d9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe084007cb0 tx=0x7fe084009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.235+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe088043060 con 0x7fe08c157890 2024-04-25T08:39:53.409 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.416+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe08c062050 con 0x7fe08c157890 2024-04-25T08:39:53.410 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.417+0000 7fe081ffb640 1 -- 172.21.15.60:0/1668003519 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fe088078810 con 0x7fe08c157890 2024-04-25T08:39:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:53.411 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":145,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:27.531391+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe07006b500 msgr2=0x7fe07006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe07006b500 0x7fe07006d9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe084007cb0 tx=0x7fe084009040 comp rx=0 tx=0).stop 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 msgr2=0x7fe08c158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 0x7fe08c158ed0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fe088001f70 tx=0x7fe0880014a0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 shutdown_connections 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe08c157890 0x7fe08c158ed0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe07006b500 0x7fe07006d9c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe08c155c40 0x7fe08c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 --2- 172.21.15.60:0/1668003519 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe0780051c0 0x7fe08c15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 >> 172.21.15.60:0/1668003519 conn(0x7fe08c061a20 msgr2=0x7fe08c0b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 shutdown_connections 2024-04-25T08:39:53.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.419+0000 7fe093729640 1 -- 172.21.15.60:0/1668003519 wait complete. 2024-04-25T08:39:53.424 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:39:53.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:53.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.531+0000 7f06a8688640 1 Processor -- start 2024-04-25T08:39:53.524 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.531+0000 7f06a8688640 1 -- start start 2024-04-25T08:39:53.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a8688640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 0x7f06a00aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a8688640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a015ec60 con 0x7f06a006c1d0 2024-04-25T08:39:53.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a8688640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a015ef40 con 0x7f06a0071ac0 2024-04-25T08:39:53.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a8688640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a006db90 con 0x7f06a0154ed0 2024-04-25T08:39:53.527 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a6c25640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f06a006c1d0 0x7f06a006afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50562/0 (socket says 172.21.15.60:50562) 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a6c25640 1 -- 172.21.15.60:0/2784459037 learned_addr learned my addr 172.21.15.60:0/2784459037 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a6424640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 0x7f06a00aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954456066 0 0) 0x7f06a015ec60 con 0x7f06a006c1d0 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.532+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0688003520 con 0x7f06a006c1d0 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2109409332 0 0) 0x7f0688003520 con 0x7f06a006c1d0 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 msgr2=0x7f06a00aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 0x7f06a00aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 >> v1:172.21.15.100:6789/0 conn(0x7f06a0071ac0 legacy=0x7f06a00ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a006de70 con 0x7f06a006c1d0 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7f06900019f0 con 0x7f06a006c1d0 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 >> v1:172.21.15.60:6789/0 conn(0x7f06a006c1d0 legacy=0x7f06a006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06880039e0 0x7f0688003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0688004800 0x7f0688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 0x7f06880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a006db90 con 0x7f0688004800 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068800ba70 con 0x7f06880039e0 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068800bc70 con 0x7f06a0154ed0 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5c23640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0688004800 0x7f0688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a6424640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06880039e0 0x7f0688003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a6c25640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 0x7f06880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5c23640 1 -- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 msgr2=0x7f06880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5c23640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0154ed0 0x7f06880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5c23640 1 -- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06880039e0 msgr2=0x7f0688003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5c23640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06880039e0 0x7f0688003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.533+0000 7f06a5c23640 1 -- 172.21.15.60:0/2784459037 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0688003520 con 0x7f0688004800 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.534+0000 7f06a5c23640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0688004800 0x7f0688004c80 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f06a0071ea0 tx=0x7f0688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0da091e91ef6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.534+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694001370 con 0x7f0688004800 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.534+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06940052d0 con 0x7f0688004800 2024-04-25T08:39:53.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.534+0000 7f06a5422640 1 -- 172.21.15.60:0/2784459037 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069402ab70 con 0x7f0688004800 2024-04-25T08:39:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a8688640 1 -- 172.21.15.60:0/2784459037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0688004800 msgr2=0x7f0688004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a8688640 1 --2- 172.21.15.60:0/2784459037 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0688004800 0x7f0688004c80 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f06a0071ea0 tx=0x7f0688005830 comp rx=0 tx=0).stop 2024-04-25T08:39:53.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a6c25640 1 -- 172.21.15.60:0/2784459037 reap_dead start 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a8688640 1 -- 172.21.15.60:0/2784459037 shutdown_connections 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a8688640 1 -- 172.21.15.60:0/2784459037 >> 172.21.15.60:0/2784459037 conn(0x7f06a00af670 msgr2=0x7f06a00b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a8688640 1 -- 172.21.15.60:0/2784459037 shutdown_connections 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.535+0000 7f06a8688640 1 -- 172.21.15.60:0/2784459037 wait complete. 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.536+0000 7f06a8688640 1 Processor -- start 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.536+0000 7f06a8688640 1 -- start start 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0157fb0 0x7f06a00cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 0x7f06a014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a00afa60 0x7f06a00afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00ad050 con 0x7f06a014e360 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00aced0 con 0x7f06a00afa60 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a00ad1d0 con 0x7f06a0157fb0 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 0x7f06a014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 0x7f06a014e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35824/0 (socket says 172.21.15.60:35824) 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 -- 172.21.15.60:0/1029929738 learned_addr learned my addr 172.21.15.60:0/1029929738 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:53.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a6424640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0157fb0 0x7f06a00cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a6c25640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a00afa60 0x7f06a00afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 -- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0157fb0 msgr2=0x7f06a00cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0157fb0 0x7f06a00cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 -- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a00afa60 msgr2=0x7f06a00afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a00afa60 0x7f06a00afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 -- 172.21.15.60:0/1029929738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a00cd370 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a5c23640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 0x7f06a014e7e0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f069402a520 tx=0x7f0694001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694037040 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694042070 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a00ce750 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.537+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694007a30 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.538+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a00cde20 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.539+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a00b2b10 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.539+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06940052d0 con 0x7f06a014e360 2024-04-25T08:39:53.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.539+0000 7f069effd640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f068c06b4d0 0x7f068c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.539+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f06940aa400 con 0x7f06a014e360 2024-04-25T08:39:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.539+0000 7f06a6424640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f068c06b4d0 0x7f068c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.540+0000 7f06a6424640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f068c06b4d0 0x7f068c06d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b2030 tx=0x7f0698009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.544+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06a00cd370 con 0x7f06a014e360 2024-04-25T08:39:53.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.766+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a0073990 con 0x7f06a014e360 2024-04-25T08:39:53.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.766+0000 7f069effd640 1 -- 172.21.15.60:0/1029929738 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f0694079180 con 0x7f06a014e360 2024-04-25T08:39:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:53.760 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:47.785553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8716":{"gid":8716,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-04-25T08:39:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.768+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f068c06b4d0 msgr2=0x7f068c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.768+0000 7f06a8688640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f068c06b4d0 0x7f068c06d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f06a00b2030 tx=0x7f0698009190 comp rx=0 tx=0).stop 2024-04-25T08:39:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.768+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 msgr2=0x7f06a014e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.768+0000 7f06a8688640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 0x7f06a014e7e0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f069402a520 tx=0x7f0694001120 comp rx=0 tx=0).stop 2024-04-25T08:39:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 shutdown_connections 2024-04-25T08:39:53.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f068c06b4d0 0x7f068c06d990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06a00afa60 0x7f06a00afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f06a014e360 0x7f06a014e7e0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 --2- 172.21.15.60:0/1029929738 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f06a0157fb0 0x7f06a00cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 >> 172.21.15.60:0/1029929738 conn(0x7f0688004800 msgr2=0x7f06a0154000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 shutdown_connections 2024-04-25T08:39:53.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.769+0000 7f06a8688640 1 -- 172.21.15.60:0/1029929738 wait complete. 2024-04-25T08:39:53.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:53.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.881+0000 7ffb0dbf9640 1 Processor -- start 2024-04-25T08:39:53.874 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.881+0000 7ffb0dbf9640 1 -- start start 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb0dbf9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb0815dc00 0x7ffb080c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb0dbf9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb080d1dd0 0x7ffb080ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb0dbf9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0815dfe0 con 0x7ffb080ceb20 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb0dbf9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb080d00c0 con 0x7ffb0815dc00 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb0dbf9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080d03a0 con 0x7ffb080d1dd0 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb077fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7ffb0815dc00 0x7ffb080c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb06ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7ffb080ceb20 0x7ffb080d16c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50564/0 (socket says 172.21.15.60:50564) 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb06ffd640 1 -- 172.21.15.60:0/2446300622 learned_addr learned my addr 172.21.15.60:0/2446300622 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb07fff640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb080d1dd0 0x7ffb080ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb077fe640 1 -- 172.21.15.60:0/2446300622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb080d1dd0 msgr2=0x7ffb080ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb077fe640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb080d1dd0 0x7ffb080ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.876 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb067fc640 1 -- 172.21.15.60:0/2446300622 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665024726 0 0) 0x7ffafc001110 con 0x7ffb080ceb20 2024-04-25T08:39:53.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb077fe640 1 -- 172.21.15.60:0/2446300622 >> v1:172.21.15.60:6789/0 conn(0x7ffb080ceb20 legacy=0x7ffb080d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb077fe640 1 -- 172.21.15.60:0/2446300622 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb080cc440 con 0x7ffb0815dc00 2024-04-25T08:39:53.877 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.882+0000 7ffb07fff640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb080d1dd0 0x7ffb080ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.883+0000 7ffb077fe640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb0815dc00 0x7ffb080c6df0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80097a0 tx=0x7ffaf802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b2da4a37678ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.883+0000 7ffb067fc640 1 -- 172.21.15.60:0/2446300622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8037070 con 0x7ffb0815dc00 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.883+0000 7ffb067fc640 1 -- 172.21.15.60:0/2446300622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8032440 con 0x7ffb0815dc00 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.883+0000 7ffb067fc640 1 -- 172.21.15.60:0/2446300622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8032810 con 0x7ffb0815dc00 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/2446300622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb0815dc00 msgr2=0x7ffb080c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb0815dc00 0x7ffb080c6df0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80097a0 tx=0x7ffaf802a180 comp rx=0 tx=0).stop 2024-04-25T08:39:53.878 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/2446300622 shutdown_connections 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb080d1dd0 0x7ffb080ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/2446300622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb0815dc00 0x7ffb080c6df0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/2446300622 >> 172.21.15.60:0/2446300622 conn(0x7ffb0806f5d0 msgr2=0x7ffb0806f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/2446300622 shutdown_connections 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.884+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/2446300622 wait complete. 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.885+0000 7ffb0dbf9640 1 Processor -- start 2024-04-25T08:39:53.879 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.885+0000 7ffb0dbf9640 1 -- start start 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb0dbf9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb080ceb20 0x7ffb08154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb0dbf9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 0x7ffb0815cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb0dbf9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb0815dc00 0x7ffb0815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb0dbf9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080cfd90 con 0x7ffb080d1dd0 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb0dbf9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080cc130 con 0x7ffb080ceb20 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb0dbf9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080cdd70 con 0x7ffb0815dc00 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 0x7ffb0815cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 0x7ffb0815cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35844/0 (socket says 172.21.15.60:35844) 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 -- 172.21.15.60:0/3542916173 learned_addr learned my addr 172.21.15.60:0/3542916173 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb077fe640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb080ceb20 0x7ffb08154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb07fff640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb0815dc00 0x7ffb0815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 -- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb0815dc00 msgr2=0x7ffb0815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb0815dc00 0x7ffb0815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 -- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb080ceb20 msgr2=0x7ffb08154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb080ceb20 0x7ffb08154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 -- 172.21.15.60:0/3542916173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb08156550 con 0x7ffb080d1dd0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.886+0000 7ffb06ffd640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 0x7ffb0815cb60 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c780 tx=0x7ffafc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.887+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc018070 con 0x7ffb080d1dd0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.887+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc0072d0 con 0x7ffb080d1dd0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.887+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb08156830 con 0x7ffb080d1dd0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.887+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0047e0 con 0x7ffb080d1dd0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.887+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb08156df0 con 0x7ffb080d1dd0 2024-04-25T08:39:53.882 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.888+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb08073990 con 0x7ffb080d1dd0 2024-04-25T08:39:53.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.888+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffafc01b070 con 0x7ffb080d1dd0 2024-04-25T08:39:53.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.889+0000 7ffaeffff640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffaf406b4d0 0x7ffaf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.889+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffafc0896e0 con 0x7ffb080d1dd0 2024-04-25T08:39:53.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.889+0000 7ffb077fe640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffaf406b4d0 0x7ffaf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.883 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.889+0000 7ffb077fe640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffaf406b4d0 0x7ffaf406d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8034000 tx=0x7ffaf802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:53.893+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb0815dfe0 con 0x7ffb080d1dd0 2024-04-25T08:39:54.105 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.112+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb080c66c0 con 0x7ffb080d1dd0 2024-04-25T08:39:54.106 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.113+0000 7ffaeffff640 1 -- 172.21.15.60:0/3542916173 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4952 (secure 0 0 0) 0x7ffafc057ce0 con 0x7ffb080d1dd0 2024-04-25T08:39:54.106 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:54.106 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":92,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:39:25.890816+0000","modified":"2024-04-25T08:39:47.785553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8716":{"gid":8716,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/4143083931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":4143083931},{"type":"v1","addr":"172.21.15.100:6837","nonce":4143083931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9924":{"gid":9924,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.60:6833/4231971347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":4231971347},{"type":"v1","addr":"172.21.15.60:6833","nonce":4231971347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:39:54.107 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 92 2024-04-25T08:39:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffaf406b4d0 msgr2=0x7ffaf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffaf406b4d0 0x7ffaf406d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8034000 tx=0x7ffaf802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 msgr2=0x7ffb0815cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 0x7ffb0815cb60 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00c780 tx=0x7ffafc00cc40 comp rx=0 tx=0).stop 2024-04-25T08:39:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 shutdown_connections 2024-04-25T08:39:54.108 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ffb0815dc00 0x7ffb0815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb080d1dd0 0x7ffb0815cb60 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7ffaf406b4d0 0x7ffaf406d990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 --2- 172.21.15.60:0/3542916173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb080ceb20 0x7ffb08154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 >> 172.21.15.60:0/3542916173 conn(0x7ffb0806f5d0 msgr2=0x7ffb080b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 shutdown_connections 2024-04-25T08:39:54.109 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.115+0000 7ffb0dbf9640 1 -- 172.21.15.60:0/3542916173 wait complete. 2024-04-25T08:39:54.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:54.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.227+0000 7f70d8170640 1 Processor -- start 2024-04-25T08:39:54.220 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.227+0000 7f70d8170640 1 -- start start 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d8170640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d8170640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70d00d0ad0 0x7f70d00b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d8170640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00d1df0 con 0x7f70d00b9770 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d8170640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00d20d0 con 0x7f70d00d0ad0 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d8170640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d23b0 con 0x7f70d015dd30 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70d00d0ad0 0x7f70d00b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f70d00d0ad0 0x7f70d00b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35850/0 (socket says 172.21.15.60:35850) 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d570b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f70d00b9770 0x7f70d00d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50578/0 (socket says 172.21.15.60:50578) 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 -- 172.21.15.60:0/2333213536 learned_addr learned my addr 172.21.15.60:0/2333213536 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d5f0c640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 -- 172.21.15.60:0/2333213536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 msgr2=0x7f70d00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.222 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d00b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 -- 172.21.15.60:0/2333213536 >> v1:172.21.15.60:6789/0 conn(0x7f70d00b9770 legacy=0x7f70d00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d670d640 1 -- 172.21.15.60:0/2333213536 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00d2690 con 0x7f70d00d0ad0 2024-04-25T08:39:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d4f0a640 1 -- 172.21.15.60:0/2333213536 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239026179 0 0) 0x7f70d00d1df0 con 0x7f70d00b9770 2024-04-25T08:39:54.223 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.228+0000 7f70d5f0c640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d00b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.229+0000 7f70d670d640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00b6a40 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009e50 tx=0x7f70c802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce93c456d432aa89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.229+0000 7f70d4f0a640 1 -- 172.21.15.60:0/2333213536 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8036070 con 0x7f70d00d0ad0 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.229+0000 7f70d4f0a640 1 -- 172.21.15.60:0/2333213536 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8031600 con 0x7f70d00d0ad0 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.229+0000 7f70d4f0a640 1 -- 172.21.15.60:0/2333213536 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c80319d0 con 0x7f70d00d0ad0 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 -- 172.21.15.60:0/2333213536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 msgr2=0x7f70d00b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00b6a40 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009e50 tx=0x7f70c802a680 comp rx=0 tx=0).stop 2024-04-25T08:39:54.224 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 -- 172.21.15.60:0/2333213536 shutdown_connections 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00b6a40 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 --2- 172.21.15.60:0/2333213536 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d00b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 -- 172.21.15.60:0/2333213536 >> 172.21.15.60:0/2333213536 conn(0x7f70d0061c30 msgr2=0x7f70d006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 -- 172.21.15.60:0/2333213536 shutdown_connections 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.230+0000 7f70d8170640 1 -- 172.21.15.60:0/2333213536 wait complete. 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.231+0000 7f70d8170640 1 Processor -- start 2024-04-25T08:39:54.225 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.231+0000 7f70d8170640 1 -- start start 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d8170640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d00b9770 0x7f70d00caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d8170640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d8170640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d8170640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d1b90 con 0x7f70d00d0ad0 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d8170640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d1a10 con 0x7f70d00b9770 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d8170640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d2e90 con 0x7f70d015dd30 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35856/0 (socket says 172.21.15.60:35856) 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 -- 172.21.15.60:0/443305622 learned_addr learned my addr 172.21.15.60:0/443305622 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d670d640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.227 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d5f0c640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d00b9770 0x7f70d00caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 -- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 msgr2=0x7f70d0156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d0156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 -- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d00b9770 msgr2=0x7f70d00caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d00b9770 0x7f70d00caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 -- 172.21.15.60:0/443305622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d015f490 con 0x7f70d00d0ad0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70d570b640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00cd530 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c4d0 tx=0x7f70c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007480 con 0x7f70d00d0ad0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.232+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00057a0 con 0x7f70d00d0ad0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.233+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d015f770 con 0x7f70d00d0ad0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.233+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007620 con 0x7f70d00d0ad0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.233+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d015fd30 con 0x7f70d00d0ad0 2024-04-25T08:39:54.228 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.234+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0073990 con 0x7f70d00d0ad0 2024-04-25T08:39:54.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.234+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70c0005b50 con 0x7f70d00d0ad0 2024-04-25T08:39:54.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.234+0000 7f70c67fc640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70b406b4d0 0x7f70b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.235+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70c0089ea0 con 0x7f70d00d0ad0 2024-04-25T08:39:54.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.235+0000 7f70d5f0c640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70b406b4d0 0x7f70b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.229 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.235+0000 7f70d5f0c640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70b406b4d0 0x7f70b406d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f70cc006e80 tx=0x7f70cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.232 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.239+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70c0058540 con 0x7f70d00d0ad0 2024-04-25T08:39:54.412 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.418+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d0157430 con 0x7f70d00d0ad0 2024-04-25T08:39:54.413 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.420+0000 7f70c67fc640 1 -- 172.21.15.60:0/443305622 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f70c005c5e0 con 0x7f70d00d0ad0 2024-04-25T08:39:54.413 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:39:54.413 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":145,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:27.531391+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:24.176721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:24.866339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70b406b4d0 msgr2=0x7f70b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70b406b4d0 0x7f70b406d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f70cc006e80 tx=0x7f70cc008040 comp rx=0 tx=0).stop 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 msgr2=0x7f70d00cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00cd530 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c4d0 tx=0x7f70c000c990 comp rx=0 tx=0).stop 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 shutdown_connections 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f70d015dd30 0x7f70d0156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f70d00d0ad0 0x7f70d00cd530 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f70b406b4d0 0x7f70b406d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.415 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 --2- 172.21.15.60:0/443305622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70d00b9770 0x7f70d00caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.421+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 >> 172.21.15.60:0/443305622 conn(0x7f70d0061c30 msgr2=0x7f70d00c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.422+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 shutdown_connections 2024-04-25T08:39:54.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.422+0000 7f70d8170640 1 -- 172.21.15.60:0/443305622 wait complete. 2024-04-25T08:39:54.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:39:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 Processor -- start 2024-04-25T08:39:54.528 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 -- start start 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76000ced80 0x7f76000d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7600161710 0x7f76000d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760015e000 con 0x7f760015dc20 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000d1de0 con 0x7f7600161710 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.535+0000 7f7606c6a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7600161c60 con 0x7f76000ced80 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7605207640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7600161710 0x7f76000d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.529 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7604a06640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f760015dc20 0x7f76000ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50584/0 (socket says 172.21.15.60:50584) 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7604a06640 1 -- 172.21.15.60:0/2913818807 learned_addr learned my addr 172.21.15.60:0/2913818807 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f75fffff640 1 --2- 172.21.15.60:0/2913818807 >> v2:172.21.15.100:3300/0 conn(0x7f76000ced80 0x7f76000d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7605207640 1 -- 172.21.15.60:0/2913818807 >> v2:172.21.15.100:3300/0 conn(0x7f76000ced80 msgr2=0x7f76000d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7605207640 1 --2- 172.21.15.60:0/2913818807 >> v2:172.21.15.100:3300/0 conn(0x7f76000ced80 0x7f76000d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7605207640 1 -- 172.21.15.60:0/2913818807 >> v1:172.21.15.60:6789/0 conn(0x7f760015dc20 legacy=0x7f76000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f75ff7fe640 1 -- 172.21.15.60:0/2913818807 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248563126 0 0) 0x7f760015e000 con 0x7f760015dc20 2024-04-25T08:39:54.530 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.536+0000 7f7605207640 1 -- 172.21.15.60:0/2913818807 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600161f40 con 0x7f7600161710 2024-04-25T08:39:54.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7605207640 1 --2- 172.21.15.60:0/2913818807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7600161710 0x7f76000d2820 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f760006da90 tx=0x7f75f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae41f5961fd2fca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f75ff7fe640 1 -- 172.21.15.60:0/2913818807 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8007740 con 0x7f7600161710 2024-04-25T08:39:54.531 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f75ff7fe640 1 -- 172.21.15.60:0/2913818807 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f80078e0 con 0x7f7600161710 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f75ff7fe640 1 -- 172.21.15.60:0/2913818807 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8007cb0 con 0x7f7600161710 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 -- 172.21.15.60:0/2913818807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7600161710 msgr2=0x7f76000d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 --2- 172.21.15.60:0/2913818807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7600161710 0x7f76000d2820 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f760006da90 tx=0x7f75f802a680 comp rx=0 tx=0).stop 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 -- 172.21.15.60:0/2913818807 shutdown_connections 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 --2- 172.21.15.60:0/2913818807 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7600161710 0x7f76000d2820 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 --2- 172.21.15.60:0/2913818807 >> v2:172.21.15.100:3300/0 conn(0x7f76000ced80 0x7f76000d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 -- 172.21.15.60:0/2913818807 >> 172.21.15.60:0/2913818807 conn(0x7f76000d0420 msgr2=0x7f76000d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 -- 172.21.15.60:0/2913818807 shutdown_connections 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.537+0000 7f7606c6a640 1 -- 172.21.15.60:0/2913818807 wait complete. 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.538+0000 7f7606c6a640 1 Processor -- start 2024-04-25T08:39:54.532 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 -- start start 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 0x7f760015ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f760015dc20 0x7f760015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7600161710 0x7f76001f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b5e50 con 0x7f76000ced80 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7600162500 con 0x7f7600161710 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7606c6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000b5fd0 con 0x7f760015dc20 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.539+0000 7f7604a06640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 0x7f760015ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 0x7f760015ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35876/0 (socket says 172.21.15.60:35876) 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 -- 172.21.15.60:0/364049591 learned_addr learned my addr 172.21.15.60:0/364049591 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f75fffff640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f760015dc20 0x7f760015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7605207640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7600161710 0x7f76001f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 -- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f760015dc20 msgr2=0x7f760015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f760015dc20 0x7f760015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 -- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7600161710 msgr2=0x7f76001f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7600161710 0x7f76001f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 -- 172.21.15.60:0/364049591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7600160160 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7604a06640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 0x7f760015ec00 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f75f400c430 tx=0x7f75f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f40074e0 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f40047e0 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760015f7e0 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4004bd0 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.540+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000c7570 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.541+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7600073a00 con 0x7f76000ced80 2024-04-25T08:39:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.542+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75f4011070 con 0x7f76000ced80 2024-04-25T08:39:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.542+0000 7f75fd7fa640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75e406b4d0 0x7f75e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.542+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f75f4089e90 con 0x7f76000ced80 2024-04-25T08:39:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.542+0000 7f75fffff640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75e406b4d0 0x7f75e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.537 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.543+0000 7f75fffff640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75e406b4d0 0x7f75e406d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f76001f68f0 tx=0x7f75ec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.540 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.546+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75f4058530 con 0x7f76000ced80 2024-04-25T08:39:54.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.766+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f76000d16b0 con 0x7f76000ced80 2024-04-25T08:39:54.760 INFO:tasks.ceph.mon.a.smithi060.stderr:2024-04-25T08:39:54.766+0000 7fcb37bb6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:39:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.768+0000 7f75fd7fa640 1 -- 172.21.15.60:0/364049591 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 161+0+0 (secure 0 0 0) 0x7f75f405c5d0 con 0x7f76000ced80 2024-04-25T08:39:54.761 INFO:teuthology.orchestra.run.smithi060.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75e406b4d0 msgr2=0x7f75e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75e406b4d0 0x7f75e406d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f76001f68f0 tx=0x7f75ec008040 comp rx=0 tx=0).stop 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 msgr2=0x7f760015ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 0x7f760015ec00 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f75f400c430 tx=0x7f75f400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 shutdown_connections 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f75e406b4d0 0x7f75e406d990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7600161710 0x7f76001f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f760015dc20 0x7f760015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 --2- 172.21.15.60:0/364049591 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f76000ced80 0x7f760015ec00 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 >> 172.21.15.60:0/364049591 conn(0x7f76000d0420 msgr2=0x7f76000d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 shutdown_connections 2024-04-25T08:39:54.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.771+0000 7f7606c6a640 1 -- 172.21.15.60:0/364049591 wait complete. 2024-04-25T08:39:54.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:39:54.802 INFO:tasks.ceph.mds.b.smithi100.stdout:starting mds.b at 2024-04-25T08:39:54.803 INFO:tasks.ceph.mds.a.smithi060.stdout:starting mds.a at 2024-04-25T08:39:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.888+0000 7fed20a15640 1 Processor -- start 2024-04-25T08:39:54.881 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.888+0000 7fed20a15640 1 -- start start 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed20a15640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fed1c161710 0x7fed1c0d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed20a15640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1c15e000 con 0x7fed1c15dc20 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed20a15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1c0d1de0 con 0x7fed1c0ced80 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed20a15640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1c161c60 con 0x7fed1c161710 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed1a59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fed1c15dc20 0x7fed1c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50612/0 (socket says 172.21.15.60:50612) 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed1a59c640 1 -- 172.21.15.60:0/374654215 learned_addr learned my addr 172.21.15.60:0/374654215 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed1ad9d640 1 --2- 172.21.15.60:0/374654215 >> v2:172.21.15.100:3300/0 conn(0x7fed1c161710 0x7fed1c0d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247276512 0 0) 0x7fed1c15e000 con 0x7fed1c15dc20 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed08003520 con 0x7fed1c15dc20 2024-04-25T08:39:54.885 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.889+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 50501504 0 0) 0x7fed08003520 con 0x7fed1c15dc20 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 >> v2:172.21.15.100:3300/0 conn(0x7fed1c161710 msgr2=0x7fed1c0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 --2- 172.21.15.60:0/374654215 >> v2:172.21.15.100:3300/0 conn(0x7fed1c161710 0x7fed1c0d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 >> v1:172.21.15.100:6789/0 conn(0x7fed1c0ced80 legacy=0x7fed1c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c161f40 con 0x7fed1c15dc20 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7fed0c001670 con 0x7fed1c15dc20 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 >> v1:172.21.15.60:6789/0 conn(0x7fed1c15dc20 legacy=0x7fed1c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080039b0 0x7fed08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080043b0 0x7fed08004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08005080 0x7fed0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c161c60 con 0x7fed080039b0 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0800bfe0 con 0x7fed08005080 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0800c1e0 con 0x7fed080043b0 2024-04-25T08:39:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1a59c640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080039b0 0x7fed08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed19d9b640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080043b0 0x7fed08004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1ad9d640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08005080 0x7fed0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1a59c640 1 -- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080043b0 msgr2=0x7fed08004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1a59c640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080043b0 0x7fed08004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1a59c640 1 -- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08005080 msgr2=0x7fed0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1a59c640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed08005080 0x7fed0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1a59c640 1 -- 172.21.15.60:0/374654215 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed08003520 con 0x7fed080039b0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.890+0000 7fed1ad9d640 1 -- 172.21.15.60:0/374654215 reap_dead start 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.891+0000 7fed1a59c640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080039b0 0x7fed08003d90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00c240 tx=0x7fed08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecd5015b7583fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.891+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c007d30 con 0x7fed080039b0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.891+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c005af0 con 0x7fed080039b0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.891+0000 7fed1959a640 1 -- 172.21.15.60:0/374654215 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c007070 con 0x7fed080039b0 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.891+0000 7fed20a15640 1 -- 172.21.15.60:0/374654215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080039b0 msgr2=0x7fed08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.891+0000 7fed20a15640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080039b0 0x7fed08003d90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00c240 tx=0x7fed08005830 comp rx=0 tx=0).stop 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.892+0000 7fed20a15640 1 -- 172.21.15.60:0/374654215 shutdown_connections 2024-04-25T08:39:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.892+0000 7fed20a15640 1 --2- 172.21.15.60:0/374654215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed080039b0 0x7fed08003d90 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.892+0000 7fed20a15640 1 -- 172.21.15.60:0/374654215 >> 172.21.15.60:0/374654215 conn(0x7fed1c0d0420 msgr2=0x7fed1c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.892+0000 7fed20a15640 1 -- 172.21.15.60:0/374654215 shutdown_connections 2024-04-25T08:39:54.888 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.892+0000 7fed20a15640 1 -- 172.21.15.60:0/374654215 wait complete. 2024-04-25T08:39:54.890 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.893+0000 7fed20a15640 1 Processor -- start 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.893+0000 7fed20a15640 1 -- start start 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed20a15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080039b0 0x7fed1c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed20a15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1c15c330 0x7fed1c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed20a15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 0x7fed1c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed20a15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c0b5e50 con 0x7fed1c15e210 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed20a15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c162500 con 0x7fed1c15c330 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed20a15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c0b5fd0 con 0x7fed080039b0 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 0x7fed1c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 0x7fed1c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35898/0 (socket says 172.21.15.60:35898) 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 -- 172.21.15.60:0/2543481369 learned_addr learned my addr 172.21.15.60:0/2543481369 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1a59c640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080039b0 0x7fed1c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed19d9b640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1c15c330 0x7fed1c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 -- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080039b0 msgr2=0x7fed1c0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080039b0 0x7fed1c0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.891 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 -- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1c15c330 msgr2=0x7fed1c15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1c15c330 0x7fed1c15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 -- 172.21.15.60:0/2543481369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c15ef00 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed1ad9d640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 0x7fed1c15e650 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0b90f0 tx=0x7fed1400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.894+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1400fce0 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.895+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1401d070 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.895+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1c15f1e0 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.895+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14015670 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.895+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1c15f7a0 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.896+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1c073a00 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.896+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed14015810 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.897+0000 7fed12ffd640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecfc06b500 0x7fecfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.897+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fed1408ab00 con 0x7fed1c15e210 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.897+0000 7fed1a59c640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecfc06b500 0x7fecfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.892 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.897+0000 7fed1a59c640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecfc06b500 0x7fecfc06d9c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fed0c032000 tx=0x7fed0c02d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.894 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:54.901+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed14010040 con 0x7fed1c15e210 2024-04-25T08:39:55.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.117+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fed1c15fbe0 con 0x7fed1c15e210 2024-04-25T08:39:55.799 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.806+0000 7fed12ffd640 1 -- 172.21.15.60:0/2543481369 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed14059150 con 0x7fed1c15e210 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecfc06b500 msgr2=0x7fecfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecfc06b500 0x7fecfc06d9c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fed0c032000 tx=0x7fed0c02d000 comp rx=0 tx=0).stop 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 msgr2=0x7fed1c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 0x7fed1c15e650 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0b90f0 tx=0x7fed1400f1b0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 shutdown_connections 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fed1c15e210 0x7fed1c15e650 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fecfc06b500 0x7fecfc06d9c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed1c15c330 0x7fed1c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 --2- 172.21.15.60:0/2543481369 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed080039b0 0x7fed1c0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 >> 172.21.15.60:0/2543481369 conn(0x7fed1c0d0420 msgr2=0x7fed1c0b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 shutdown_connections 2024-04-25T08:39:55.804 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.810+0000 7fed20a15640 1 -- 172.21.15.60:0/2543481369 wait complete. 2024-04-25T08:39:55.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:39:55.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.976+0000 7fa94dd22640 1 Processor -- start 2024-04-25T08:39:55.970 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa94dd22640 1 -- start start 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa94dd22640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa94815dc20 0x7fa9480ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa94dd22640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa948161710 0x7fa9480cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa94dd22640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94815e000 con 0x7fa9480b9330 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa94dd22640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9480d1de0 con 0x7fa94815dc20 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa94dd22640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa948161c60 con 0x7fa948161710 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa9477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa94815dc20 0x7fa9480ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.971 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa9477fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa94815dc20 0x7fa9480ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35910/0 (socket says 172.21.15.60:35910) 2024-04-25T08:39:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.977+0000 7fa9477fe640 1 -- 172.21.15.60:0/1706350325 learned_addr learned my addr 172.21.15.60:0/1706350325 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.978+0000 7fa946ffd640 1 --2- 172.21.15.60:0/1706350325 >> v2:172.21.15.100:3300/0 conn(0x7fa948161710 0x7fa9480cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.978+0000 7fa9477fe640 1 -- 172.21.15.60:0/1706350325 >> v2:172.21.15.100:3300/0 conn(0x7fa948161710 msgr2=0x7fa9480cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.978+0000 7fa9477fe640 1 --2- 172.21.15.60:0/1706350325 >> v2:172.21.15.100:3300/0 conn(0x7fa948161710 0x7fa9480cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.978+0000 7fa9477fe640 1 -- 172.21.15.60:0/1706350325 >> v1:172.21.15.100:6789/0 conn(0x7fa9480b9330 legacy=0x7fa94806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:55.972 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.978+0000 7fa9477fe640 1 -- 172.21.15.60:0/1706350325 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948161f40 con 0x7fa94815dc20 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa9477fe640 1 --2- 172.21.15.60:0/1706350325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa94815dc20 0x7fa9480ce670 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fa94806da90 tx=0x7fa93802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2554e3d13b58eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa9467fc640 1 -- 172.21.15.60:0/1706350325 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938037070 con 0x7fa94815dc20 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa9467fc640 1 -- 172.21.15.60:0/1706350325 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938032440 con 0x7fa94815dc20 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa9467fc640 1 -- 172.21.15.60:0/1706350325 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938032810 con 0x7fa94815dc20 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 -- 172.21.15.60:0/1706350325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa94815dc20 msgr2=0x7fa9480ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1706350325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa94815dc20 0x7fa9480ce670 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fa94806da90 tx=0x7fa93802a230 comp rx=0 tx=0).stop 2024-04-25T08:39:55.973 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 -- 172.21.15.60:0/1706350325 shutdown_connections 2024-04-25T08:39:55.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1706350325 >> v2:172.21.15.100:3300/0 conn(0x7fa948161710 0x7fa9480cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1706350325 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa94815dc20 0x7fa9480ce670 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 -- 172.21.15.60:0/1706350325 >> 172.21.15.60:0/1706350325 conn(0x7fa9480d0420 msgr2=0x7fa9480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:55.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 -- 172.21.15.60:0/1706350325 shutdown_connections 2024-04-25T08:39:55.974 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.979+0000 7fa94dd22640 1 -- 172.21.15.60:0/1706350325 wait complete. 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.980+0000 7fa94dd22640 1 Processor -- start 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 -- start start 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9480b9330 0x7fa9481f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa94815dc20 0x7fa9481f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 0x7fa9480cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480b5e50 con 0x7fa948161710 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948162500 con 0x7fa9480b9330 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.981+0000 7fa94dd22640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480b5fd0 con 0x7fa94815dc20 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 0x7fa9480cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 0x7fa9480cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35922/0 (socket says 172.21.15.60:35922) 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 -- 172.21.15.60:0/1214601139 learned_addr learned my addr 172.21.15.60:0/1214601139 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:55.976 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa946ffd640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa94815dc20 0x7fa9481f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa9477fe640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9480b9330 0x7fa9481f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 -- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa94815dc20 msgr2=0x7fa9481f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa94815dc20 0x7fa9481f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 -- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9480b9330 msgr2=0x7fa9481f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9480b9330 0x7fa9481f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 -- 172.21.15.60:0/1214601139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480cbfc0 con 0x7fa948161710 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa947fff640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 0x7fa9480cd230 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b9e0 tx=0x7fa94000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940019070 con 0x7fa948161710 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000caa0 con 0x7fa948161710 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9480cc2a0 con 0x7fa948161710 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940014440 con 0x7fa948161710 2024-04-25T08:39:55.977 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.982+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9480cc860 con 0x7fa948161710 2024-04-25T08:39:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.984+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa948073a00 con 0x7fa948161710 2024-04-25T08:39:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.984+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa94001c070 con 0x7fa948161710 2024-04-25T08:39:55.978 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.984+0000 7fa92ffff640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93406b4d0 0x7fa93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.985+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa940089fc0 con 0x7fa948161710 2024-04-25T08:39:55.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.985+0000 7fa9477fe640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93406b4d0 0x7fa93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.979 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.985+0000 7fa9477fe640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93406b4d0 0x7fa93406d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa938034000 tx=0x7fa93802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.982 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:55.989+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa940058610 con 0x7fa948161710 2024-04-25T08:39:56.160 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.167+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9480d16b0 con 0x7fa948161710 2024-04-25T08:39:56.801 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.808+0000 7fa92ffff640 1 -- 172.21.15.60:0/1214601139 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa94005c6b0 con 0x7fa948161710 2024-04-25T08:39:56.801 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:39:56.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.811+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93406b4d0 msgr2=0x7fa93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.811+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93406b4d0 0x7fa93406d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa938034000 tx=0x7fa93802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:56.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.811+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 msgr2=0x7fa9480cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.811+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 0x7fa9480cd230 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b9e0 tx=0x7fa94000bea0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 shutdown_connections 2024-04-25T08:39:56.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa948161710 0x7fa9480cd230 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa94815dc20 0x7fa9481f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa93406b4d0 0x7fa93406d990 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 --2- 172.21.15.60:0/1214601139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9480b9330 0x7fa9481f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 >> 172.21.15.60:0/1214601139 conn(0x7fa9480d0420 msgr2=0x7fa94815bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:56.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 shutdown_connections 2024-04-25T08:39:56.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.812+0000 7fa94dd22640 1 -- 172.21.15.60:0/1214601139 wait complete. 2024-04-25T08:39:56.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:39:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.939+0000 7f07d0ecb640 1 Processor -- start 2024-04-25T08:39:56.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.939+0000 7f07d0ecb640 1 -- start start 2024-04-25T08:39:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07d0ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07d0ecb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07cc06ec50 0x7f07cc15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07d0ecb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0cfd80 con 0x7f07cc15ee60 2024-04-25T08:39:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07d0ecb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc15e490 con 0x7f07cc06ec50 2024-04-25T08:39:56.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07d0ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0d0060 con 0x7f07cc15b710 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07c3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07cc06ec50 0x7f07cc15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07ca59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07c3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f07cc06ec50 0x7f07cc15e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38620/0 (socket says 172.21.15.60:38620) 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07cad9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f07cc15ee60 0x7f07cc0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:42310/0 (socket says 172.21.15.60:42310) 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07ca59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54554/0 (socket says 172.21.15.60:54554) 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.940+0000 7f07c3fff640 1 -- 172.21.15.60:0/3680066870 learned_addr learned my addr 172.21.15.60:0/3680066870 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07c3fff640 1 -- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 msgr2=0x7f07cc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:56.934 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07c3fff640 1 --2- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07c3fff640 1 -- 172.21.15.60:0/3680066870 >> v1:172.21.15.100:6789/0 conn(0x7f07cc15ee60 legacy=0x7f07cc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07c3fff640 1 -- 172.21.15.60:0/3680066870 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0d0340 con 0x7f07cc06ec50 2024-04-25T08:39:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07c9d9b640 1 -- 172.21.15.60:0/3680066870 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746907065 0 0) 0x7f07cc0cfd80 con 0x7f07cc15ee60 2024-04-25T08:39:56.935 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07ca59c640 1 --2- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.941+0000 7f07c3fff640 1 --2- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc15e920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009870 tx=0x7f07b402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f346d40b911f494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07c9d9b640 1 -- 172.21.15.60:0/3680066870 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4037070 con 0x7f07cc06ec50 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07c9d9b640 1 -- 172.21.15.60:0/3680066870 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b4032440 con 0x7f07cc06ec50 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07c9d9b640 1 -- 172.21.15.60:0/3680066870 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4032810 con 0x7f07cc06ec50 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 -- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 msgr2=0x7f07cc15e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc15e920 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009870 tx=0x7f07b402a270 comp rx=0 tx=0).stop 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 -- 172.21.15.60:0/3680066870 shutdown_connections 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc15e920 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/3680066870 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 -- 172.21.15.60:0/3680066870 >> 172.21.15.60:0/3680066870 conn(0x7f07cc06abe0 msgr2=0x7f07cc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 -- 172.21.15.60:0/3680066870 shutdown_connections 2024-04-25T08:39:56.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.942+0000 7f07d0ecb640 1 -- 172.21.15.60:0/3680066870 wait complete. 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.943+0000 7f07d0ecb640 1 Processor -- start 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 -- start start 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc0b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 0x7f07cc0c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15bc70 con 0x7f07cc15ee60 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15baf0 con 0x7f07cc06ec50 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07d0ecb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0acfc0 con 0x7f07cc15b710 2024-04-25T08:39:56.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07cad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 0x7f07cc0c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07cad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 0x7f07cc0c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35936/0 (socket says 172.21.15.60:35936) 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.944+0000 7f07cad9d640 1 -- 172.21.15.60:0/2063248347 learned_addr learned my addr 172.21.15.60:0/2063248347 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07c3fff640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07ca59c640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc0b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07cad9d640 1 -- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 msgr2=0x7f07cc156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07cad9d640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07cad9d640 1 -- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 msgr2=0x7f07cc0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07cad9d640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc0b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07cad9d640 1 -- 172.21.15.60:0/2063248347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0b0c70 con 0x7f07cc15ee60 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07ca59c640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07cad9d640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 0x7f07cc0c46d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f07c400c700 tx=0x7f07c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4018070 con 0x7f07cc15ee60 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c400ce10 con 0x7f07cc15ee60 2024-04-25T08:39:56.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc0b0f50 con 0x7f07cc15ee60 2024-04-25T08:39:56.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c400a860 con 0x7f07cc15ee60 2024-04-25T08:39:56.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.945+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc0b1510 con 0x7f07cc15ee60 2024-04-25T08:39:56.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.946+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc073a00 con 0x7f07cc15ee60 2024-04-25T08:39:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.947+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07c401b070 con 0x7f07cc15ee60 2024-04-25T08:39:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.947+0000 7f07c2ffd640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f07ac06b4d0 0x7f07ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.947+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f07c40897d0 con 0x7f07cc15ee60 2024-04-25T08:39:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.948+0000 7f07ca59c640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f07ac06b4d0 0x7f07ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.942 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.948+0000 7f07ca59c640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f07ac06b4d0 0x7f07ac06d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f07cc06e6f0 tx=0x7f07bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:56.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:56.952+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07c4057f30 con 0x7f07cc15ee60 2024-04-25T08:39:57.127 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.134+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f07cc0ac8b0 con 0x7f07cc15ee60 2024-04-25T08:39:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.818+0000 7f07c2ffd640 1 -- 172.21.15.60:0/2063248347 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f07c405bfd0 con 0x7f07cc15ee60 2024-04-25T08:39:57.812 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'cephfs_data' removed 2024-04-25T08:39:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.821+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f07ac06b4d0 msgr2=0x7f07ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.821+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f07ac06b4d0 0x7f07ac06d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f07cc06e6f0 tx=0x7f07bc008040 comp rx=0 tx=0).stop 2024-04-25T08:39:57.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.821+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 msgr2=0x7f07cc0c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.821+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 0x7f07cc0c46d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f07c400c700 tx=0x7f07c400cbc0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 shutdown_connections 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07cc15ee60 0x7f07cc0c46d0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07cc15b710 0x7f07cc156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f07ac06b4d0 0x7f07ac06d990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 --2- 172.21.15.60:0/2063248347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f07cc06ec50 0x7f07cc0b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 >> 172.21.15.60:0/2063248347 conn(0x7f07cc06abe0 msgr2=0x7f07cc15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 shutdown_connections 2024-04-25T08:39:57.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.822+0000 7f07d0ecb640 1 -- 172.21.15.60:0/2063248347 wait complete. 2024-04-25T08:39:57.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:39:57.944 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.951+0000 7fef3142a640 1 Processor -- start 2024-04-25T08:39:57.945 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.951+0000 7fef3142a640 1 -- start start 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef3142a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fef2c0ced80 0x7fef2c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef3142a640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2c15e000 con 0x7fef2c15dc20 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef3142a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2c0d1de0 con 0x7fef2c161710 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef3142a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c161c60 con 0x7fef2c0ced80 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef2affd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fef2c15dc20 0x7fef2c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50620/0 (socket says 172.21.15.60:50620) 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef2affd640 1 -- 172.21.15.60:0/2225348786 learned_addr learned my addr 172.21.15.60:0/2225348786 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef227fc640 1 --2- 172.21.15.60:0/2225348786 >> v2:172.21.15.60:3300/0 conn(0x7fef2c0ced80 0x7fef2c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.952+0000 7fef227fc640 1 -- 172.21.15.60:0/2225348786 >> v1:172.21.15.100:6789/0 conn(0x7fef2c161710 legacy=0x7fef2c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef227fc640 1 -- 172.21.15.60:0/2225348786 >> v1:172.21.15.60:6789/0 conn(0x7fef2c15dc20 legacy=0x7fef2c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef2a7fc640 1 -- 172.21.15.60:0/2225348786 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072462213 0 0) 0x7fef2c15e000 con 0x7fef2c15dc20 2024-04-25T08:39:57.946 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef227fc640 1 -- 172.21.15.60:0/2225348786 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c161f40 con 0x7fef2c0ced80 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef227fc640 1 --2- 172.21.15.60:0/2225348786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c0d22e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fef140097a0 tx=0x7fef1402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6546cade841b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef2a7fc640 1 -- 172.21.15.60:0/2225348786 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14037070 con 0x7fef2c0ced80 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef2a7fc640 1 -- 172.21.15.60:0/2225348786 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14032440 con 0x7fef2c0ced80 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.953+0000 7fef2a7fc640 1 -- 172.21.15.60:0/2225348786 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14032810 con 0x7fef2c0ced80 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 -- 172.21.15.60:0/2225348786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 msgr2=0x7fef2c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 --2- 172.21.15.60:0/2225348786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c0d22e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fef140097a0 tx=0x7fef1402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 -- 172.21.15.60:0/2225348786 shutdown_connections 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 --2- 172.21.15.60:0/2225348786 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c0d22e0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 -- 172.21.15.60:0/2225348786 >> 172.21.15.60:0/2225348786 conn(0x7fef2c0d0420 msgr2=0x7fef2c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 -- 172.21.15.60:0/2225348786 shutdown_connections 2024-04-25T08:39:57.948 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.954+0000 7fef3142a640 1 -- 172.21.15.60:0/2225348786 wait complete. 2024-04-25T08:39:57.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.955+0000 7fef3142a640 1 Processor -- start 2024-04-25T08:39:57.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.955+0000 7fef3142a640 1 -- start start 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef3142a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c1f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef3142a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef2c15dc20 0x7fef2c1e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef3142a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef2c161710 0x7fef2c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef3142a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0b5e50 con 0x7fef2c0ced80 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef3142a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c162500 con 0x7fef2c15dc20 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef3142a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0b5fd0 con 0x7fef2c161710 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef2affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c1f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef2affd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c1f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35948/0 (socket says 172.21.15.60:35948) 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef2affd640 1 -- 172.21.15.60:0/3863626172 learned_addr learned my addr 172.21.15.60:0/3863626172 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef227fc640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef2c15dc20 0x7fef2c1e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.956+0000 7fef2b7fe640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef2c161710 0x7fef2c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef2affd640 1 -- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef2c161710 msgr2=0x7fef2c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef2affd640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef2c161710 0x7fef2c15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef2affd640 1 -- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef2c15dc20 msgr2=0x7fef2c1e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.951 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef2affd640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef2c15dc20 0x7fef2c1e7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef2affd640 1 -- 172.21.15.60:0/3863626172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c15e390 con 0x7fef2c0ced80 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef2affd640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c1f6730 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00c500 tx=0x7fef1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c018070 con 0x7fef2c0ced80 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c007580 con 0x7fef2c0ced80 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2c15e670 con 0x7fef2c0ced80 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c004a20 con 0x7fef2c0ced80 2024-04-25T08:39:57.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.957+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c15ec30 con 0x7fef2c0ced80 2024-04-25T08:39:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.958+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c073a00 con 0x7fef2c0ced80 2024-04-25T08:39:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.959+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef1c01b070 con 0x7fef2c0ced80 2024-04-25T08:39:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.959+0000 7fef23fff640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef0c06b4d0 0x7fef0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.959+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fef1c089280 con 0x7fef2c0ced80 2024-04-25T08:39:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.959+0000 7fef227fc640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef0c06b4d0 0x7fef0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.960+0000 7fef227fc640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef0c06b4d0 0x7fef0c06d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fef14034000 tx=0x7fef1402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:57.956 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:57.963+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef1c057cb0 con 0x7fef2c0ced80 2024-04-25T08:39:57.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:57.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:58.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.146+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fef2c0d16b0 con 0x7fef2c0ced80 2024-04-25T08:39:58.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.149+0000 7fef23fff640 1 -- 172.21.15.60:0/3863626172 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef1c05bd50 con 0x7fef2c0ced80 2024-04-25T08:39:58.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef0c06b4d0 msgr2=0x7fef0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef0c06b4d0 0x7fef0c06d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fef14034000 tx=0x7fef1402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:58.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 msgr2=0x7fef2c1f6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c1f6730 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00c500 tx=0x7fef1c00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 shutdown_connections 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef2c161710 0x7fef2c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fef0c06b4d0 0x7fef0c06d990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef2c15dc20 0x7fef2c1e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 --2- 172.21.15.60:0/3863626172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fef2c0ced80 0x7fef2c1f6730 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 >> 172.21.15.60:0/3863626172 conn(0x7fef2c0d0420 msgr2=0x7fef2c15b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 shutdown_connections 2024-04-25T08:39:58.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.151+0000 7fef3142a640 1 -- 172.21.15.60:0/3863626172 wait complete. 2024-04-25T08:39:58.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.263+0000 7fe415f5c640 1 Processor -- start 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.263+0000 7fe415f5c640 1 -- start start 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe415f5c640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4100b9730 0x7fe4100ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe415f5c640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100cf040 con 0x7fe41015da70 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe415f5c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe410161d50 con 0x7fe4100ceb90 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe415f5c640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4101625e0 con 0x7fe4100b9730 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe40effd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fe4100b9730 0x7fe4100ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe40f7fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe41015da70 0x7fe410161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50636/0 (socket says 172.21.15.60:50636) 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe40f7fe640 1 -- 172.21.15.60:0/1515718024 learned_addr learned my addr 172.21.15.60:0/1515718024 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.264+0000 7fe40effd640 1 --2- 172.21.15.60:0/1515718024 >> v2:172.21.15.60:3300/0 conn(0x7fe4100b9730 0x7fe4100ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35958/0 (socket says 172.21.15.60:35958) 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.265+0000 7fe40effd640 1 -- 172.21.15.60:0/1515718024 >> v1:172.21.15.100:6789/0 conn(0x7fe4100ceb90 legacy=0x7fe4100d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.265+0000 7fe40effd640 1 -- 172.21.15.60:0/1515718024 >> v1:172.21.15.60:6789/0 conn(0x7fe41015da70 legacy=0x7fe410161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.259 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.265+0000 7fe40effd640 1 -- 172.21.15.60:0/1515718024 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410162020 con 0x7fe4100b9730 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.265+0000 7fe40effd640 1 --2- 172.21.15.60:0/1515718024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4100b9730 0x7fe4100ce650 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe40402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20281fc8d3dd6dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe40e7fc640 1 -- 172.21.15.60:0/1515718024 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404037070 con 0x7fe4100b9730 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe40e7fc640 1 -- 172.21.15.60:0/1515718024 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40402ae50 con 0x7fe4100b9730 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe40e7fc640 1 -- 172.21.15.60:0/1515718024 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404032660 con 0x7fe4100b9730 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 -- 172.21.15.60:0/1515718024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4100b9730 msgr2=0x7fe4100ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 --2- 172.21.15.60:0/1515718024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4100b9730 0x7fe4100ce650 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe40402a140 comp rx=0 tx=0).stop 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 -- 172.21.15.60:0/1515718024 shutdown_connections 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 --2- 172.21.15.60:0/1515718024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4100b9730 0x7fe4100ce650 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 -- 172.21.15.60:0/1515718024 >> 172.21.15.60:0/1515718024 conn(0x7fe410061a20 msgr2=0x7fe410061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 -- 172.21.15.60:0/1515718024 shutdown_connections 2024-04-25T08:39:58.260 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.266+0000 7fe415f5c640 1 -- 172.21.15.60:0/1515718024 wait complete. 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.267+0000 7fe415f5c640 1 Processor -- start 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.267+0000 7fe415f5c640 1 -- start start 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe415f5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4100ceb90 0x7fe41014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe415f5c640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 0x7fe4100cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe415f5c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4100ccbe0 0x7fe4100c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe415f5c640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100d1dd0 con 0x7fe41015da70 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe415f5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100d1af0 con 0x7fe4100ceb90 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe415f5c640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41015de50 con 0x7fe4100ccbe0 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe40effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 0x7fe4100cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe40effd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 0x7fe4100cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35960/0 (socket says 172.21.15.60:35960) 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe40effd640 1 -- 172.21.15.60:0/221861895 learned_addr learned my addr 172.21.15.60:0/221861895 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe40f7fe640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4100ceb90 0x7fe41014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.268+0000 7fe40ffff640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4100ccbe0 0x7fe4100c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40effd640 1 -- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4100ccbe0 msgr2=0x7fe4100c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40effd640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4100ccbe0 0x7fe4100c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40effd640 1 -- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4100ceb90 msgr2=0x7fe41014cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40effd640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4100ceb90 0x7fe41014cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40effd640 1 -- 172.21.15.60:0/221861895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100c8370 con 0x7fe41015da70 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40f7fe640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4100ceb90 0x7fe41014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe40effd640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 0x7fe4100cc6a0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404042070 con 0x7fe41015da70 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40403e670 con 0x7fe41015da70 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100c8650 con 0x7fe41015da70 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe404037040 con 0x7fe41015da70 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.269+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100c8c10 con 0x7fe41015da70 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.270+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4100ccff0 con 0x7fe41015da70 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.271+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe404045070 con 0x7fe41015da70 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.271+0000 7fe3f7fff640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3fc06b4d0 0x7fe3fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.271+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe4040aa6e0 con 0x7fe41015da70 2024-04-25T08:39:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.271+0000 7fe40f7fe640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3fc06b4d0 0x7fe3fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.266 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.272+0000 7fe40f7fe640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3fc06b4d0 0x7fe3fc06d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe4100cc080 tx=0x7fe400009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.268 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.275+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4100ccff0 con 0x7fe41015da70 2024-04-25T08:39:58.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.526+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7fe410073990 con 0x7fe41015da70 2024-04-25T08:39:58.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.819+0000 7fe3f7fff640 1 -- 172.21.15.60:0/221861895 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v522) v1 ==== 159+0+0 (secure 0 0 0) 0x7fe404079760 con 0x7fe41015da70 2024-04-25T08:39:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.821+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3fc06b4d0 msgr2=0x7fe3fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.821+0000 7fe415f5c640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3fc06b4d0 0x7fe3fc06d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe4100cc080 tx=0x7fe400009040 comp rx=0 tx=0).stop 2024-04-25T08:39:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.821+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 msgr2=0x7fe4100cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.821+0000 7fe415f5c640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 0x7fe4100cc6a0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe4040046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 shutdown_connections 2024-04-25T08:39:58.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe4100ccbe0 0x7fe4100c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe41015da70 0x7fe4100cc6a0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe3fc06b4d0 0x7fe3fc06d990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 --2- 172.21.15.60:0/221861895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4100ceb90 0x7fe41014cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 >> 172.21.15.60:0/221861895 conn(0x7fe410061a20 msgr2=0x7fe41015bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 shutdown_connections 2024-04-25T08:39:58.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:39:58.822+0000 7fe415f5c640 1 -- 172.21.15.60:0/221861895 wait complete. 2024-04-25T08:39:58.827 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ERROR 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner: 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner: 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:Ran 9 tests in 594.188s 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner: 2024-04-25T08:39:58.828 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner: 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T08:39:58.829 INFO:tasks.cephfs_test_runner: 2024-04-25T08:39:58.830 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:59.028 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c8e26fe140064d3ca9aab2212c3b6a85 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:59.029 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T08:39:59.039 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T08:39:59.047 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T08:39:59.047 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:59.047 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:59.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:59.067 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-04-25T08:39:59.067 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:59.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2854 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:59.122 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:59.123 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:59.140 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:59.140 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-04-25T08:39:59.140 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:59.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497628k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:59.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.197 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:39:59.212 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:39:59.212 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:39:59.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:59.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:59.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-04-25T08:39:59.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.449 DEBUG:teuthology.orchestra.run.smithi060:> ip netns list 2024-04-25T08:39:59.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo ip link delete ceph-brx 2024-04-25T08:39:59.532 INFO:teuthology.orchestra.run.smithi060.stderr:Cannot find device "ceph-brx" 2024-04-25T08:39:59.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:39:59.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.534 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T08:39:59.553 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T08:39:59.553 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:39:59.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.553 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:39:59.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:39:59.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.690 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-25T08:39:59.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.787 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-04-25T08:39:59.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:59.802 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-04-25T08:39:59.869 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-04-25T08:39:59.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:39:59.871 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T08:39:59.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:39:59.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.002+0000 7f8bcc35a640 1 Processor -- start 2024-04-25T08:39:59.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.002+0000 7f8bcc35a640 1 -- start start 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bcc35a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 0x7f8bc4162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bcc35a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8bc40b5c30 0x7f8bc40d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bcc35a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc4162690 con 0x7f8bc40b6cb0 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bcc35a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40ceee0 con 0x7f8bc40b5c30 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bcc35a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40d1d10 con 0x7f8bc415db70 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bca0f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 0x7f8bc4162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.997 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bca8f7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8bc40b5c30 0x7f8bc40d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bca0f6640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 0x7f8bc4162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:54598/0 (socket says 172.21.15.60:54598) 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bc98f5640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8bc40b6cb0 0x7f8bc40b5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:42336/0 (socket says 172.21.15.60:42336) 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bca8f7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8bc40b5c30 0x7f8bc40d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:38668/0 (socket says 172.21.15.60:38668) 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.003+0000 7f8bca0f6640 1 -- 172.21.15.60:0/394406740 learned_addr learned my addr 172.21.15.60:0/394406740 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.004+0000 7f8bca0f6640 1 -- 172.21.15.60:0/394406740 >> v2:172.21.15.100:3300/0 conn(0x7f8bc40b5c30 msgr2=0x7f8bc40d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.004+0000 7f8bca0f6640 1 --2- 172.21.15.60:0/394406740 >> v2:172.21.15.100:3300/0 conn(0x7f8bc40b5c30 0x7f8bc40d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.004+0000 7f8bca0f6640 1 -- 172.21.15.60:0/394406740 >> v1:172.21.15.100:6789/0 conn(0x7f8bc40b6cb0 legacy=0x7f8bc40b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.998 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.004+0000 7f8bc90f4640 1 -- 172.21.15.60:0/394406740 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122544442 0 0) 0x7f8bc4162690 con 0x7f8bc40b6cb0 2024-04-25T08:39:59.999 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.004+0000 7f8bca0f6640 1 -- 172.21.15.60:0/394406740 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40d1ff0 con 0x7f8bc415db70 2024-04-25T08:40:00.001 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.004+0000 7f8bca0f6640 1 --2- 172.21.15.60:0/394406740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 0x7f8bc4162150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009870 tx=0x7f8bc002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c1c69c86a6df5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bc90f4640 1 -- 172.21.15.60:0/394406740 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0037070 con 0x7f8bc415db70 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bc90f4640 1 -- 172.21.15.60:0/394406740 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0032440 con 0x7f8bc415db70 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bc90f4640 1 -- 172.21.15.60:0/394406740 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0032810 con 0x7f8bc415db70 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 -- 172.21.15.60:0/394406740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 msgr2=0x7f8bc4162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/394406740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 0x7f8bc4162150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009870 tx=0x7f8bc002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 -- 172.21.15.60:0/394406740 shutdown_connections 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/394406740 >> v2:172.21.15.100:3300/0 conn(0x7f8bc40b5c30 0x7f8bc40d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.002 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/394406740 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc415db70 0x7f8bc4162150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 -- 172.21.15.60:0/394406740 >> 172.21.15.60:0/394406740 conn(0x7f8bc40d0520 msgr2=0x7f8bc40d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 -- 172.21.15.60:0/394406740 shutdown_connections 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.005+0000 7f8bcc35a640 1 -- 172.21.15.60:0/394406740 wait complete. 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.006+0000 7f8bcc35a640 1 Processor -- start 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.006+0000 7f8bcc35a640 1 -- start start 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bcc35a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bc40b5c30 0x7f8bc40ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bcc35a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc40b6cb0 0x7f8bc415e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.003 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bcc35a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 0x7f8bc41f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bcc35a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40ce530 con 0x7f8bc415db70 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bcc35a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415df50 con 0x7f8bc40b5c30 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bcc35a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40cf1c0 con 0x7f8bc40b6cb0 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bca8f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 0x7f8bc41f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bca8f7640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 0x7f8bc41f63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35974/0 (socket says 172.21.15.60:35974) 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bca8f7640 1 -- 172.21.15.60:0/2882743526 learned_addr learned my addr 172.21.15.60:0/2882743526 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bc98f5640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc40b6cb0 0x7f8bc415e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.007+0000 7f8bca0f6640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bc40b5c30 0x7f8bc40ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.004 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bca8f7640 1 -- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc40b6cb0 msgr2=0x7f8bc415e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bca8f7640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc40b6cb0 0x7f8bc415e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bca8f7640 1 -- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bc40b5c30 msgr2=0x7f8bc40ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bca8f7640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bc40b5c30 0x7f8bc40ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bca8f7640 1 -- 172.21.15.60:0/2882743526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc415eb70 con 0x7f8bc415db70 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bc98f5640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc40b6cb0 0x7f8bc415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bca8f7640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 0x7f8bc41f63d0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009e30 tx=0x7f8bbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc018020 con 0x7f8bc415db70 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc014480 con 0x7f8bc415db70 2024-04-25T08:40:00.005 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc415ee50 con 0x7f8bc415db70 2024-04-25T08:40:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc014870 con 0x7f8bc415db70 2024-04-25T08:40:00.006 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.008+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc415f6f0 con 0x7f8bc415db70 2024-04-25T08:40:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.009+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc4073990 con 0x7f8bc415db70 2024-04-25T08:40:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.009+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bbc005d10 con 0x7f8bc415db70 2024-04-25T08:40:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.010+0000 7f8bbaffd640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ba806b4d0 0x7f8ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.010+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8bbc08af60 con 0x7f8bc415db70 2024-04-25T08:40:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.010+0000 7f8bca0f6640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ba806b4d0 0x7f8ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.007 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.010+0000 7f8bca0f6640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ba806b4d0 0x7f8ba806d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0034000 tx=0x7f8bc002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.008 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.014+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bbc022030 con 0x7f8bc415db70 2024-04-25T08:40:00.184 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.191+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bc415f020 con 0x7f8ba806b4d0 2024-04-25T08:40:00.186 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.192+0000 7f8bbaffd640 1 -- 172.21.15.60:0/2882743526 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13602 (secure 0 0 0) 0x7f8bc415f020 con 0x7f8ba806b4d0 2024-04-25T08:40:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:00.186 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:40:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ba806b4d0 msgr2=0x7f8ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ba806b4d0 0x7f8ba806d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0034000 tx=0x7f8bc002e000 comp rx=0 tx=0).stop 2024-04-25T08:40:00.188 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 msgr2=0x7f8bc41f63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 0x7f8bc41f63d0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009e30 tx=0x7f8bbc00c490 comp rx=0 tx=0).stop 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 shutdown_connections 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8bc415db70 0x7f8bc41f63d0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8bc40b6cb0 0x7f8bc415e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f8ba806b4d0 0x7f8ba806d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 --2- 172.21.15.60:0/2882743526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bc40b5c30 0x7f8bc40ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 >> 172.21.15.60:0/2882743526 conn(0x7f8bc40d0520 msgr2=0x7f8bc4159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 shutdown_connections 2024-04-25T08:40:00.189 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.195+0000 7f8bcc35a640 1 -- 172.21.15.60:0/2882743526 wait complete. 2024-04-25T08:40:00.199 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-25T08:39:59.545521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2536120,"kb_used_data":2121496,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":752438600,"statfs":{"total":773094113280,"available":770497126400,"internally_reserved":0,"allocated":2172411904,"data_stored":2164579108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1075969361,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-14,"num_read_kb":-2184,"num_write":-1101,"num_write_kb":-1050896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001175"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":404,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-25T08:39:54.860658+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:54.860658+0000","last_peered":"2024-04-25T08:39:54.860658+0000","last_clean":"2024-04-25T08:39:54.860658+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:54.860658+0000","last_undegraded":"2024-04-25T08:39:54.860658+0000","last_fullsized":"2024-04-25T08:39:54.860658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542277,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322956,"kb_used_data":273728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94048884,"statfs":{"total":96636764160,"available":96306057216,"internally_reserved":0,"allocated":280297472,"data_stored":279318984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542277,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302228,"kb_used_data":253000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94069612,"statfs":{"total":96636764160,"available":96327282688,"internally_reserved":0,"allocated":259072000,"data_stored":258094916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542277,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325444,"kb_used_data":276216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94046396,"statfs":{"total":96636764160,"available":96303509504,"internally_reserved":0,"allocated":282845184,"data_stored":281874322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":367832,"kb_used_data":314444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94004008,"statfs":{"total":96636764160,"available":96260104192,"internally_reserved":0,"allocated":321990656,"data_stored":321010078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415500,"kb_used_data":370432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":93956340,"statfs":{"total":96636764160,"available":96211292160,"internally_reserved":0,"allocated":379322368,"data_stored":378343930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287692,"kb_used_data":230144,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94084148,"statfs":{"total":96636764160,"available":96342167552,"internally_reserved":0,"allocated":235667456,"data_stored":234688514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289672,"kb_used_data":236284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94082168,"statfs":{"total":96636764160,"available":96340140032,"internally_reserved":0,"allocated":241954816,"data_stored":240979928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":224796,"kb_used_data":167248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94147044,"statfs":{"total":96636764160,"available":96406573056,"internally_reserved":0,"allocated":171261952,"data_stored":170268436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:00.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.308+0000 7fb837476640 1 Processor -- start 2024-04-25T08:40:00.301 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.308+0000 7fb837476640 1 -- start start 2024-04-25T08:40:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb837476640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb83015b320 0x7fb83013d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb837476640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8300cf620 0x7fb8300cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb837476640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb830071ed0 con 0x7fb8300cfaa0 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb837476640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83015a840 con 0x7fb8300cf620 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb837476640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83015ab20 con 0x7fb83015b320 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8300cf620 0x7fb8300cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fb8300cf620 0x7fb8300cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35990/0 (socket says 172.21.15.60:35990) 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 -- 172.21.15.60:0/2340902594 learned_addr learned my addr 172.21.15.60:0/2340902594 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb835212640 1 --2- 172.21.15.60:0/2340902594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb83015b320 0x7fb83013d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.305 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 -- 172.21.15.60:0/2340902594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb83015b320 msgr2=0x7fb83013d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 --2- 172.21.15.60:0/2340902594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb83015b320 0x7fb83013d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 -- 172.21.15.60:0/2340902594 >> v1:172.21.15.100:6789/0 conn(0x7fb8300cfaa0 legacy=0x7fb830071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.309+0000 7fb834a11640 1 -- 172.21.15.60:0/2340902594 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83015ae00 con 0x7fb8300cf620 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.310+0000 7fb834a11640 1 --2- 172.21.15.60:0/2340902594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300cf620 0x7fb8300cbfb0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fb82000bb10 tx=0x7fb82002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8d98dc2c2ad968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.310+0000 7fb827fff640 1 -- 172.21.15.60:0/2340902594 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820032790 con 0x7fb8300cf620 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.310+0000 7fb827fff640 1 -- 172.21.15.60:0/2340902594 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820038070 con 0x7fb8300cf620 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.310+0000 7fb827fff640 1 -- 172.21.15.60:0/2340902594 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820032b50 con 0x7fb8300cf620 2024-04-25T08:40:00.306 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 -- 172.21.15.60:0/2340902594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300cf620 msgr2=0x7fb8300cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 --2- 172.21.15.60:0/2340902594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300cf620 0x7fb8300cbfb0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fb82000bb10 tx=0x7fb82002c890 comp rx=0 tx=0).stop 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 -- 172.21.15.60:0/2340902594 shutdown_connections 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 --2- 172.21.15.60:0/2340902594 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb8300cf620 0x7fb8300cbfb0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 --2- 172.21.15.60:0/2340902594 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb83015b320 0x7fb83013d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 -- 172.21.15.60:0/2340902594 >> 172.21.15.60:0/2340902594 conn(0x7fb83006abe0 msgr2=0x7fb83006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 -- 172.21.15.60:0/2340902594 shutdown_connections 2024-04-25T08:40:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.311+0000 7fb837476640 1 -- 172.21.15.60:0/2340902594 wait complete. 2024-04-25T08:40:00.310 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.312+0000 7fb837476640 1 Processor -- start 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.312+0000 7fb837476640 1 -- start start 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb837476640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8300cf620 0x7fb8301ef000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb837476640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300cfaa0 0x7fb830155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb837476640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 0x7fb83013bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb837476640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cc4f0 con 0x7fb83015b320 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb837476640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830071b90 con 0x7fb8300cf620 2024-04-25T08:40:00.311 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb837476640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cc670 con 0x7fb8300cfaa0 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 0x7fb83013bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 0x7fb83013bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36002/0 (socket says 172.21.15.60:36002) 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 -- 172.21.15.60:0/976891410 learned_addr learned my addr 172.21.15.60:0/976891410 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb834a11640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300cfaa0 0x7fb830155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835212640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8300cf620 0x7fb8301ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 -- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300cfaa0 msgr2=0x7fb830155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300cfaa0 0x7fb830155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.312 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 -- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8300cf620 msgr2=0x7fb8301ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8300cf620 0x7fb8301ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 -- 172.21.15.60:0/976891410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8301f2ea0 con 0x7fb83015b320 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.313+0000 7fb835a13640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 0x7fb83013bc20 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c730 tx=0x7fb82c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.314+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c016020 con 0x7fb83015b320 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.314+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c006d50 con 0x7fb83015b320 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.314+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8301f3180 con 0x7fb83015b320 2024-04-25T08:40:00.313 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.314+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c003bb0 con 0x7fb83015b320 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.314+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8301f3740 con 0x7fb83015b320 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.315+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830073990 con 0x7fb83015b320 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.315+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82c00a7f0 con 0x7fb83015b320 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.316+0000 7fb825ffb640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb81406b4d0 0x7fb81406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.316+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb82c089c90 con 0x7fb83015b320 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.316+0000 7fb835212640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb81406b4d0 0x7fb81406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.314 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.317+0000 7fb835212640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb81406b4d0 0x7fb81406d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb8300aeb00 tx=0x7fb828008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.315 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.320+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb82c020020 con 0x7fb83015b320 2024-04-25T08:40:00.495 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.502+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8300cfeb0 con 0x7fb81406b4d0 2024-04-25T08:40:00.496 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.503+0000 7fb825ffb640 1 -- 172.21.15.60:0/976891410 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13602 (secure 0 0 0) 0x7fb8300cfeb0 con 0x7fb81406b4d0 2024-04-25T08:40:00.496 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:00.496 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb81406b4d0 msgr2=0x7fb81406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb81406b4d0 0x7fb81406d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb8300aeb00 tx=0x7fb828008040 comp rx=0 tx=0).stop 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 msgr2=0x7fb83013bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 0x7fb83013bc20 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00c730 tx=0x7fb82c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 shutdown_connections 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb83015b320 0x7fb83013bc20 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.499 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb8300cfaa0 0x7fb830155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fb81406b4d0 0x7fb81406d990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 --2- 172.21.15.60:0/976891410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8300cf620 0x7fb8301ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 >> 172.21.15.60:0/976891410 conn(0x7fb83006abe0 msgr2=0x7fb830157910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 shutdown_connections 2024-04-25T08:40:00.500 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.505+0000 7fb837476640 1 -- 172.21.15.60:0/976891410 wait complete. 2024-04-25T08:40:00.509 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-25T08:39:59.545521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2536120,"kb_used_data":2121496,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":752438600,"statfs":{"total":773094113280,"available":770497126400,"internally_reserved":0,"allocated":2172411904,"data_stored":2164579108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1075969361,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-14,"num_read_kb":-2184,"num_write":-1101,"num_write_kb":-1050896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001175"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":404,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-25T08:39:54.860658+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:54.860658+0000","last_peered":"2024-04-25T08:39:54.860658+0000","last_clean":"2024-04-25T08:39:54.860658+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:54.860658+0000","last_undegraded":"2024-04-25T08:39:54.860658+0000","last_fullsized":"2024-04-25T08:39:54.860658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542277,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322956,"kb_used_data":273728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94048884,"statfs":{"total":96636764160,"available":96306057216,"internally_reserved":0,"allocated":280297472,"data_stored":279318984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542277,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302228,"kb_used_data":253000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94069612,"statfs":{"total":96636764160,"available":96327282688,"internally_reserved":0,"allocated":259072000,"data_stored":258094916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542277,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325444,"kb_used_data":276216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94046396,"statfs":{"total":96636764160,"available":96303509504,"internally_reserved":0,"allocated":282845184,"data_stored":281874322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":367832,"kb_used_data":314444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94004008,"statfs":{"total":96636764160,"available":96260104192,"internally_reserved":0,"allocated":321990656,"data_stored":321010078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415500,"kb_used_data":370432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":93956340,"statfs":{"total":96636764160,"available":96211292160,"internally_reserved":0,"allocated":379322368,"data_stored":378343930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287692,"kb_used_data":230144,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94084148,"statfs":{"total":96636764160,"available":96342167552,"internally_reserved":0,"allocated":235667456,"data_stored":234688514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289672,"kb_used_data":236284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94082168,"statfs":{"total":96636764160,"available":96340140032,"internally_reserved":0,"allocated":241954816,"data_stored":240979928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":224796,"kb_used_data":167248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94147044,"statfs":{"total":96636764160,"available":96406573056,"internally_reserved":0,"allocated":171261952,"data_stored":170268436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:00.510 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:40:00.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.619+0000 7fd944b9e640 1 Processor -- start 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.619+0000 7fd944b9e640 1 -- start start 2024-04-25T08:40:00.613 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd944b9e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd9400cfec0 0x7fd9400cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd944b9e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940071480 con 0x7fd940154ee0 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd944b9e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940071760 con 0x7fd9400ce3c0 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd944b9e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9400ce830 con 0x7fd9400cfec0 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93e59c640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd940154ee0 0x7fd9400cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50650/0 (socket says 172.21.15.60:50650) 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93e59c640 1 -- 172.21.15.60:0/2345450372 learned_addr learned my addr 172.21.15.60:0/2345450372 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93dd9b640 1 --2- 172.21.15.60:0/2345450372 >> v2:172.21.15.100:3300/0 conn(0x7fd9400cfec0 0x7fd9400cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93d59a640 1 -- 172.21.15.60:0/2345450372 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160407159 0 0) 0x7fd940071480 con 0x7fd940154ee0 2024-04-25T08:40:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93d59a640 1 -- 172.21.15.60:0/2345450372 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd928003520 con 0x7fd940154ee0 2024-04-25T08:40:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93dd9b640 1 -- 172.21.15.60:0/2345450372 >> v1:172.21.15.100:6789/0 conn(0x7fd9400ce3c0 legacy=0x7fd94006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93d59a640 1 -- 172.21.15.60:0/2345450372 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141162493 0 0) 0x7fd928003520 con 0x7fd940154ee0 2024-04-25T08:40:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93dd9b640 1 -- 172.21.15.60:0/2345450372 >> v1:172.21.15.60:6789/0 conn(0x7fd940154ee0 legacy=0x7fd9400cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.620+0000 7fd93dd9b640 1 -- 172.21.15.60:0/2345450372 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd940071a40 con 0x7fd9400cfec0 2024-04-25T08:40:00.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.621+0000 7fd93dd9b640 1 --2- 172.21.15.60:0/2345450372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9400cfec0 0x7fd9400cde80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd9340097a0 tx=0x7fd93402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a647dd34d7058d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.621+0000 7fd93d59a640 1 -- 172.21.15.60:0/2345450372 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934037070 con 0x7fd9400cfec0 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.621+0000 7fd93d59a640 1 -- 172.21.15.60:0/2345450372 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93402ac50 con 0x7fd9400cfec0 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.621+0000 7fd93d59a640 1 -- 172.21.15.60:0/2345450372 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd938001670 con 0x7fd9400cfec0 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 -- 172.21.15.60:0/2345450372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9400cfec0 msgr2=0x7fd9400cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 --2- 172.21.15.60:0/2345450372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9400cfec0 0x7fd9400cde80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd9340097a0 tx=0x7fd93402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 -- 172.21.15.60:0/2345450372 shutdown_connections 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 --2- 172.21.15.60:0/2345450372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd9400cfec0 0x7fd9400cde80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 -- 172.21.15.60:0/2345450372 >> 172.21.15.60:0/2345450372 conn(0x7fd94006abe0 msgr2=0x7fd94006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 -- 172.21.15.60:0/2345450372 shutdown_connections 2024-04-25T08:40:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.622+0000 7fd944b9e640 1 -- 172.21.15.60:0/2345450372 wait complete. 2024-04-25T08:40:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.623+0000 7fd944b9e640 1 Processor -- start 2024-04-25T08:40:00.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.623+0000 7fd944b9e640 1 -- start start 2024-04-25T08:40:00.620 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd944b9e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9400ce3c0 0x7fd94013eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd944b9e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 0x7fd9401599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd944b9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd940154ee0 0x7fd940159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd944b9e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940071230 con 0x7fd9400cfec0 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd944b9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9401552c0 con 0x7fd940154ee0 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd944b9e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940155440 con 0x7fd9400ce3c0 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 0x7fd9401599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 0x7fd9401599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36006/0 (socket says 172.21.15.60:36006) 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 -- 172.21.15.60:0/4254554222 learned_addr learned my addr 172.21.15.60:0/4254554222 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93e59c640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9400ce3c0 0x7fd94013eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93ed9d640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd940154ee0 0x7fd940159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 -- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9400ce3c0 msgr2=0x7fd94013eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9400ce3c0 0x7fd94013eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 -- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd940154ee0 msgr2=0x7fd940159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd940154ee0 0x7fd940159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93dd9b640 1 -- 172.21.15.60:0/4254554222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9401e40b0 con 0x7fd9400cfec0 2024-04-25T08:40:00.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.624+0000 7fd93ed9d640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd940154ee0 0x7fd940159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.625+0000 7fd93dd9b640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 0x7fd9401599d0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd9340098c0 tx=0x7fd9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.625+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934041020 con 0x7fd9400cfec0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.625+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9340044c0 con 0x7fd9400cfec0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.625+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9401e4390 con 0x7fd9400cfec0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.625+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934037040 con 0x7fd9400cfec0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.625+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9401513e0 con 0x7fd9400cfec0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.626+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd940073990 con 0x7fd9400cfec0 2024-04-25T08:40:00.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.626+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd934044020 con 0x7fd9400cfec0 2024-04-25T08:40:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.627+0000 7fd92effd640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd92006b500 0x7fd92006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.627+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd9340aa3b0 con 0x7fd9400cfec0 2024-04-25T08:40:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.627+0000 7fd93e59c640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd92006b500 0x7fd92006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.627+0000 7fd93e59c640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd92006b500 0x7fd92006d9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd938005880 tx=0x7fd938009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.631+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd93404b030 con 0x7fd9400cfec0 2024-04-25T08:40:00.802 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.809+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd940151620 con 0x7fd92006b500 2024-04-25T08:40:00.803 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.811+0000 7fd92effd640 1 -- 172.21.15.60:0/4254554222 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13602 (secure 0 0 0) 0x7fd938001430 con 0x7fd92006b500 2024-04-25T08:40:00.803 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:00.804 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:40:00.805 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd92006b500 msgr2=0x7fd92006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd92006b500 0x7fd92006d9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd938005880 tx=0x7fd938009040 comp rx=0 tx=0).stop 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 msgr2=0x7fd9401599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 0x7fd9401599d0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd9340098c0 tx=0x7fd9340046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 shutdown_connections 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd92006b500 0x7fd92006d9c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd940154ee0 0x7fd940159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd9400cfec0 0x7fd9401599d0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 --2- 172.21.15.60:0/4254554222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd9400ce3c0 0x7fd94013eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 >> 172.21.15.60:0/4254554222 conn(0x7fd94006abe0 msgr2=0x7fd9400cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 shutdown_connections 2024-04-25T08:40:00.807 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.812+0000 7fd944b9e640 1 -- 172.21.15.60:0/4254554222 wait complete. 2024-04-25T08:40:00.816 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-25T08:39:59.545521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2536120,"kb_used_data":2121496,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":752438600,"statfs":{"total":773094113280,"available":770497126400,"internally_reserved":0,"allocated":2172411904,"data_stored":2164579108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":424463920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1075969361,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-14,"num_read_kb":-2184,"num_write":-1101,"num_write_kb":-1050896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001175"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":404,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-04-25T08:39:54.860658+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:54.860658+0000","last_peered":"2024-04-25T08:39:54.860658+0000","last_clean":"2024-04-25T08:39:54.860658+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:54.860658+0000","last_undegraded":"2024-04-25T08:39:54.860658+0000","last_fullsized":"2024-04-25T08:39:54.860658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542277,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":322956,"kb_used_data":273728,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94048884,"statfs":{"total":96636764160,"available":96306057216,"internally_reserved":0,"allocated":280297472,"data_stored":279318984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542277,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302228,"kb_used_data":253000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94069612,"statfs":{"total":96636764160,"available":96327282688,"internally_reserved":0,"allocated":259072000,"data_stored":258094916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542277,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325444,"kb_used_data":276216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94046396,"statfs":{"total":96636764160,"available":96303509504,"internally_reserved":0,"allocated":282845184,"data_stored":281874322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":367832,"kb_used_data":314444,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94004008,"statfs":{"total":96636764160,"available":96260104192,"internally_reserved":0,"allocated":321990656,"data_stored":321010078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":415500,"kb_used_data":370432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":93956340,"statfs":{"total":96636764160,"available":96211292160,"internally_reserved":0,"allocated":379322368,"data_stored":378343930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574979,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287692,"kb_used_data":230144,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94084148,"statfs":{"total":96636764160,"available":96342167552,"internally_reserved":0,"allocated":235667456,"data_stored":234688514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":289672,"kb_used_data":236284,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94082168,"statfs":{"total":96636764160,"available":96340140032,"internally_reserved":0,"allocated":241954816,"data_stored":240979928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":224796,"kb_used_data":167248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94147044,"statfs":{"total":96636764160,"available":96406573056,"internally_reserved":0,"allocated":171261952,"data_stored":170268436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:00.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:00.917 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.924+0000 7efe5269e640 1 Processor -- start 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.924+0000 7efe5269e640 1 -- start start 2024-04-25T08:40:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe5269e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 0x7efe4c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe5269e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0b6220 con 0x7efe4c0b92a0 2024-04-25T08:40:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe5269e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0b97a0 con 0x7efe4c0b2cb0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe5269e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c15dfc0 con 0x7efe4c15dbe0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe50c3b640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7efe4c0b92a0 0x7efe4c0b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:50664/0 (socket says 172.21.15.60:50664) 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe50c3b640 1 -- 172.21.15.60:0/3255982001 learned_addr learned my addr 172.21.15.60:0/3255982001 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.925+0000 7efe4bfff640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 0x7efe4c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287612242 0 0) 0x7efe4c0b6220 con 0x7efe4c0b92a0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe38003520 con 0x7efe4c0b92a0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178802947 0 0) 0x7efe4c0b97a0 con 0x7efe4c0b2cb0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4c0b6220 con 0x7efe4c0b2cb0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.0 v1:172.21.15.60:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996917056 0 0) 0x7efe38003520 con 0x7efe4c0b92a0 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 msgr2=0x7efe4c0d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 0x7efe4c0d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 >> v1:172.21.15.100:6789/0 conn(0x7efe4c0b2cb0 legacy=0x7efe4c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 --> v1:172.21.15.60:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0d1c50 con 0x7efe4c0b92a0 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.0 v1:172.21.15.60:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2067756222 0 0) 0x7efe440019f0 con 0x7efe4c0b92a0 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 >> v1:172.21.15.60:6789/0 conn(0x7efe4c0b92a0 legacy=0x7efe4c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe38003b70 0x7efe38003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 0x7efe38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe38004d40 0x7efe380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3800ba70 con 0x7efe38004d40 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3800bbf0 con 0x7efe38003b70 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3800bdf0 con 0x7efe4c15dbe0 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe50c3b640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe38004d40 0x7efe380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4bfff640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe38003b70 0x7efe38003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe4b7fe640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 0x7efe38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe50c3b640 1 -- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 msgr2=0x7efe38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe50c3b640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c15dbe0 0x7efe38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe50c3b640 1 -- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe38003b70 msgr2=0x7efe38003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe50c3b640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe38003b70 0x7efe38003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.926+0000 7efe50c3b640 1 -- 172.21.15.60:0/3255982001 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe38003520 con 0x7efe38004d40 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.927+0000 7efe50c3b640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe38004d40 0x7efe380051c0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b25c0 tx=0x7efe38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b836c40cc62a1ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.927+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400a350 con 0x7efe38004d40 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.927+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4402f070 con 0x7efe38004d40 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.927+0000 7efe4affd640 1 -- 172.21.15.60:0/3255982001 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4402bb30 con 0x7efe38004d40 2024-04-25T08:40:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe5269e640 1 -- 172.21.15.60:0/3255982001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe38004d40 msgr2=0x7efe380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.927 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe5269e640 1 --2- 172.21.15.60:0/3255982001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe38004d40 0x7efe380051c0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b25c0 tx=0x7efe38005830 comp rx=0 tx=0).stop 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe50c3b640 1 -- 172.21.15.60:0/3255982001 reap_dead start 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe5269e640 1 -- 172.21.15.60:0/3255982001 shutdown_connections 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe5269e640 1 -- 172.21.15.60:0/3255982001 >> 172.21.15.60:0/3255982001 conn(0x7efe4c061a20 msgr2=0x7efe4c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe5269e640 1 -- 172.21.15.60:0/3255982001 shutdown_connections 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.928+0000 7efe5269e640 1 -- 172.21.15.60:0/3255982001 wait complete. 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.929+0000 7efe5269e640 1 Processor -- start 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.929+0000 7efe5269e640 1 -- start start 2024-04-25T08:40:00.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe5269e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4c0b5b60 0x7efe4c1f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe5269e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c1f7370 0x7efe4c1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe5269e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 0x7efe4c0b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe5269e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0d2210 con 0x7efe4c0b3090 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe5269e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0d1a00 con 0x7efe4c0b5b60 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe5269e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0b6ab0 con 0x7efe4c1f7370 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 0x7efe4c0b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 0x7efe4c0b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36032/0 (socket says 172.21.15.60:36032) 2024-04-25T08:40:00.929 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 -- 172.21.15.60:0/1417377934 learned_addr learned my addr 172.21.15.60:0/1417377934 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe4b7fe640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c1f7370 0x7efe4c1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe4bfff640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4c0b5b60 0x7efe4c1f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 -- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c1f7370 msgr2=0x7efe4c1f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c1f7370 0x7efe4c1f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 -- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4c0b5b60 msgr2=0x7efe4c1f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4c0b5b60 0x7efe4c1f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 -- 172.21.15.60:0/1417377934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c1f7d30 con 0x7efe4c0b3090 2024-04-25T08:40:00.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.930+0000 7efe50c3b640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 0x7efe4c0b3510 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7efe44001780 tx=0x7efe4402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.931+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400a350 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.931+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4402bd90 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.931+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c0b4560 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.931+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44034670 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.931+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0b4b20 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.932+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c073990 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.932+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe44034810 con 0x7efe4c0b3090 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.933+0000 7efe48ff9640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.933+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efe440a9ce0 con 0x7efe4c0b3090 2024-04-25T08:40:00.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.933+0000 7efe4bfff640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.933+0000 7efe4bfff640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efe340099e0 tx=0x7efe34008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:00.937+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe44046060 con 0x7efe4c0b3090 2024-04-25T08:40:01.110 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.117+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe4c073990 con 0x7efe4c0b3090 2024-04-25T08:40:01.111 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.118+0000 7efe48ff9640 1 -- 172.21.15.60:0/1417377934 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+13291 (secure 0 0 0) 0x7efe44078d60 con 0x7efe4c0b3090 2024-04-25T08:40:01.111 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:01.112 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":148,"fsid":"46ba0e8c-f2cb-4e7c-bec7-f6587d7c72de","created":"2024-04-25T08:28:59.273560+0000","modified":"2024-04-25T08:39:57.814949+0000","last_up_change":"2024-04-25T08:29:10.511563+0000","last_in_change":"2024-04-25T08:29:01.427560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:29:11.440271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0309f754-7c34-473b-99a9-f12a336ecab5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6817","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6819","nonce":3630457049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6821","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6823","nonce":3630457049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6829","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6831","nonce":3630457049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6825","nonce":3630457049},{"type":"v1","addr":"172.21.15.60:6827","nonce":3630457049}]},"public_addr":"172.21.15.60:6819/3630457049","cluster_addr":"172.21.15.60:6823/3630457049","heartbeat_back_addr":"172.21.15.60:6831/3630457049","heartbeat_front_addr":"172.21.15.60:6827/3630457049","state":["exists","up"]},{"osd":1,"uuid":"7813a924-5ea3-47ac-a55e-cb91b1c687d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6801","nonce":3720682638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6803","nonce":3720682638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6807","nonce":3720682638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3720682638},{"type":"v1","addr":"172.21.15.60:6805","nonce":3720682638}]},"public_addr":"172.21.15.60:6801/3720682638","cluster_addr":"172.21.15.60:6803/3720682638","heartbeat_back_addr":"172.21.15.60:6807/3720682638","heartbeat_front_addr":"172.21.15.60:6805/3720682638","state":["exists","up"]},{"osd":2,"uuid":"ab9b54eb-8ee6-49bb-8cf9-6a3c745b4841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6809","nonce":1418327746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6811","nonce":1418327746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6815","nonce":1418327746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1418327746},{"type":"v1","addr":"172.21.15.60:6813","nonce":1418327746}]},"public_addr":"172.21.15.60:6809/1418327746","cluster_addr":"172.21.15.60:6811/1418327746","heartbeat_back_addr":"172.21.15.60:6815/1418327746","heartbeat_front_addr":"172.21.15.60:6813/1418327746","state":["exists","up"]},{"osd":3,"uuid":"7228adc9-f853-45ff-aa9b-3dc9e965833b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6818","nonce":426387744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6822","nonce":426387744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6830","nonce":426387744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":426387744},{"type":"v1","addr":"172.21.15.60:6826","nonce":426387744}]},"public_addr":"172.21.15.60:6818/426387744","cluster_addr":"172.21.15.60:6822/426387744","heartbeat_back_addr":"172.21.15.60:6830/426387744","heartbeat_front_addr":"172.21.15.60:6826/426387744","state":["exists","up"]},{"osd":4,"uuid":"55bf0b3b-3016-4d56-8b21-2c25c3ee741d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6809","nonce":2796877898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6811","nonce":2796877898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6815","nonce":2796877898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":2796877898},{"type":"v1","addr":"172.21.15.100:6813","nonce":2796877898}]},"public_addr":"172.21.15.100:6809/2796877898","cluster_addr":"172.21.15.100:6811/2796877898","heartbeat_back_addr":"172.21.15.100:6815/2796877898","heartbeat_front_addr":"172.21.15.100:6813/2796877898","state":["exists","up"]},{"osd":5,"uuid":"dd77d113-576d-4798-8ee1-6f46b49e7c6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6801","nonce":301366941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6803","nonce":301366941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6807","nonce":301366941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":301366941},{"type":"v1","addr":"172.21.15.100:6805","nonce":301366941}]},"public_addr":"172.21.15.100:6801/301366941","cluster_addr":"172.21.15.100:6803/301366941","heartbeat_back_addr":"172.21.15.100:6807/301366941","heartbeat_front_addr":"172.21.15.100:6805/301366941","state":["exists","up"]},{"osd":6,"uuid":"8d9944d9-fae5-4a43-a375-b3cccd237dcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6817","nonce":4212964990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6819","nonce":4212964990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6823","nonce":4212964990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":4212964990},{"type":"v1","addr":"172.21.15.100:6821","nonce":4212964990}]},"public_addr":"172.21.15.100:6817/4212964990","cluster_addr":"172.21.15.100:6819/4212964990","heartbeat_back_addr":"172.21.15.100:6823/4212964990","heartbeat_front_addr":"172.21.15.100:6821/4212964990","state":["exists","up"]},{"osd":7,"uuid":"989211bc-e8f0-418b-8acf-378707ab4de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6825","nonce":1558157189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6827","nonce":1558157189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6831","nonce":1558157189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1558157189},{"type":"v1","addr":"172.21.15.100:6829","nonce":1558157189}]},"public_addr":"172.21.15.100:6825/1558157189","cluster_addr":"172.21.15.100:6827/1558157189","heartbeat_back_addr":"172.21.15.100:6831/1558157189","heartbeat_front_addr":"172.21.15.100:6829/1558157189","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584805+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.507051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.619593+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.644125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.563441+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.587528+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.602907+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:29:06.584121+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:6833/4231971347":"2024-04-26T08:39:54.767792+0000","172.21.15.60:6832/4231971347":"2024-04-26T08:39:54.767792+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 msgr2=0x7efe2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efe340099e0 tx=0x7efe34008040 comp rx=0 tx=0).stop 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 msgr2=0x7efe4c0b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 0x7efe4c0b3510 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7efe44001780 tx=0x7efe4402d070 comp rx=0 tx=0).stop 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 shutdown_connections 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efe4c0b3090 0x7efe4c0b3510 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efe4c1f7370 0x7efe4c1f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7efe2c06b4d0 0x7efe2c06d990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 --2- 172.21.15.60:0/1417377934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe4c0b5b60 0x7efe4c1f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 >> 172.21.15.60:0/1417377934 conn(0x7efe4c061a20 msgr2=0x7efe4c0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 shutdown_connections 2024-04-25T08:40:01.115 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:01.120+0000 7efe5269e640 1 -- 172.21.15.60:0/1417377934 wait complete. 2024-04-25T08:40:02.125 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T08:40:02.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:02.243 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.250+0000 7f39a3ff5640 1 Processor -- start 2024-04-25T08:40:02.244 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.250+0000 7f39a3ff5640 1 -- start start 2024-04-25T08:40:02.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a3ff5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f399c0ced80 0x7f399c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a3ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 0x7f399c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.245 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a3ff5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c15e000 con 0x7f399c15dc20 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a3ff5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c0d1de0 con 0x7f399c0ced80 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a3ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c161c60 con 0x7f399c161710 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a2592640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 0x7f399c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a1590640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f399c0ced80 0x7f399c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a2592640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 0x7f399c0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35078/0 (socket says 172.21.15.60:35078) 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a1590640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f399c0ced80 0x7f399c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45150/0 (socket says 172.21.15.60:45150) 2024-04-25T08:40:02.246 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a1d91640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f399c15dc20 0x7f399c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38166/0 (socket says 172.21.15.60:38166) 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.251+0000 7f39a2592640 1 -- 172.21.15.60:0/3071359490 learned_addr learned my addr 172.21.15.60:0/3071359490 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.252+0000 7f39a1590640 1 -- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 msgr2=0x7f399c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.252+0000 7f39a1590640 1 --2- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 0x7f399c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.252+0000 7f39a1590640 1 -- 172.21.15.60:0/3071359490 >> v1:172.21.15.100:6789/0 conn(0x7f399c15dc20 legacy=0x7f399c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.252+0000 7f39a1590640 1 -- 172.21.15.60:0/3071359490 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c161f40 con 0x7f399c0ced80 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.252+0000 7f39a0d8f640 1 -- 172.21.15.60:0/3071359490 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109635269 0 0) 0x7f399c15e000 con 0x7f399c15dc20 2024-04-25T08:40:02.247 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.252+0000 7f39a2592640 1 --2- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 0x7f399c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:02.250 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a1590640 1 --2- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c0ced80 0x7f399c0d22e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f398c0097a0 tx=0x7f398c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848fc46be314a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a0d8f640 1 -- 172.21.15.60:0/3071359490 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c037070 con 0x7f399c0ced80 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a0d8f640 1 -- 172.21.15.60:0/3071359490 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c032440 con 0x7f399c0ced80 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a0d8f640 1 -- 172.21.15.60:0/3071359490 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c032810 con 0x7f399c0ced80 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a3ff5640 1 -- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c0ced80 msgr2=0x7f399c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a3ff5640 1 --2- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c0ced80 0x7f399c0d22e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f398c0097a0 tx=0x7f398c02a180 comp rx=0 tx=0).stop 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a3ff5640 1 -- 172.21.15.60:0/3071359490 shutdown_connections 2024-04-25T08:40:02.251 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a3ff5640 1 --2- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c0ced80 0x7f399c0d22e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a3ff5640 1 --2- 172.21.15.60:0/3071359490 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c161710 0x7f399c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.253+0000 7f39a3ff5640 1 -- 172.21.15.60:0/3071359490 >> 172.21.15.60:0/3071359490 conn(0x7f399c0d0420 msgr2=0x7f399c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.254+0000 7f39a3ff5640 1 -- 172.21.15.60:0/3071359490 shutdown_connections 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.254+0000 7f39a3ff5640 1 -- 172.21.15.60:0/3071359490 wait complete. 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.255+0000 7f39a3ff5640 1 Processor -- start 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.255+0000 7f39a3ff5640 1 -- start start 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.255+0000 7f39a3ff5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c0ced80 0x7f399c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.252 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.255+0000 7f39a3ff5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c15dc20 0x7f399c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a3ff5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c161710 0x7f399c0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a3ff5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b5e50 con 0x7f399c0ced80 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a3ff5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c162500 con 0x7f399c161710 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a3ff5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b5fd0 con 0x7f399c15dc20 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c0ced80 0x7f399c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c0ced80 0x7f399c14fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49154/0 (socket says 172.21.15.60:49154) 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 -- 172.21.15.60:0/1088800602 learned_addr learned my addr 172.21.15.60:0/1088800602 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1590640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c15dc20 0x7f399c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a2592640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c161710 0x7f399c0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 -- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c15dc20 msgr2=0x7f399c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f399c15dc20 0x7f399c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 -- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c161710 msgr2=0x7f399c0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f399c161710 0x7f399c0b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 -- 172.21.15.60:0/1088800602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0ccea0 con 0x7f399c0ced80 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a1d91640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c0ced80 0x7f399c14fd50 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f399800c430 tx=0x7f399800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39980074e0 con 0x7f399c0ced80 2024-04-25T08:40:02.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39980047e0 con 0x7f399c0ced80 2024-04-25T08:40:02.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c0cd180 con 0x7f399c0ced80 2024-04-25T08:40:02.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.256+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998004bd0 con 0x7f399c0ced80 2024-04-25T08:40:02.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.257+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c0cd740 con 0x7f399c0ced80 2024-04-25T08:40:02.255 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.257+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f399c073a00 con 0x7f399c0ced80 2024-04-25T08:40:02.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.258+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3998011070 con 0x7f399c0ced80 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39927fc640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398006b4d0 0x7f398006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3998089b00 con 0x7f399c0ced80 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39a1590640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398006b4d0 0x7f398006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39927fc640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7f3980071970 0x7f3980073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 --> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f399801ce80 con 0x7f3980071970 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f399808e2c0 con 0x7f399c0ced80 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39a2592640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7f3980071970 0x7f3980073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39a1590640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398006b4d0 0x7f398006d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f398c034000 tx=0x7f398c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.259+0000 7f39a2592640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7f3980071970 0x7f3980073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.260+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== osd.0 v2:172.21.15.60:6817/3630457049 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f399801ce80 con 0x7f3980071970 2024-04-25T08:40:02.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 --> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f399c0d16b0 con 0x7f3980071970 2024-04-25T08:40:02.267 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39927fc640 1 -- 172.21.15.60:0/1088800602 <== osd.0 v2:172.21.15.60:6817/3630457049 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f399c0d16b0 con 0x7f3980071970 2024-04-25T08:40:02.267 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:02.268 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:02.268 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7f3980071970 msgr2=0x7f3980073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:6817/3630457049,v1:172.21.15.60:6819/3630457049] conn(0x7f3980071970 0x7f3980073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398006b4d0 msgr2=0x7f398006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f398006b4d0 0x7f398006d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f398c034000 tx=0x7f398c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c0ced80 msgr2=0x7f399c14fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a3ff5640 1 --2- 172.21.15.60:0/1088800602 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f399c0ced80 0x7f399c14fd50 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f399800c430 tx=0x7f399800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.274+0000 7f39a2592640 1 -- 172.21.15.60:0/1088800602 reap_dead start 2024-04-25T08:40:02.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.275+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 shutdown_connections 2024-04-25T08:40:02.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.275+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 >> 172.21.15.60:0/1088800602 conn(0x7f399c0d0420 msgr2=0x7f399c0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.275+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 shutdown_connections 2024-04-25T08:40:02.270 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.275+0000 7f39a3ff5640 1 -- 172.21.15.60:0/1088800602 wait complete. 2024-04-25T08:40:02.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T08:40:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.387+0000 7f316549e640 1 Processor -- start 2024-04-25T08:40:02.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.387+0000 7f316549e640 1 -- start start 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f316549e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f316549e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 0x7f3160161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f316549e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600c6960 con 0x7f31601616f0 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f316549e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31601614d0 con 0x7f31600ce7b0 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f316549e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3160162260 con 0x7f31601610f0 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f315effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.388+0000 7f315f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 0x7f3160161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315effd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316006c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35094/0 (socket says 172.21.15.60:35094) 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 0x7f3160161b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45170/0 (socket says 172.21.15.60:45170) 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315effd640 1 -- 172.21.15.60:0/2462189051 learned_addr learned my addr 172.21.15.60:0/2462189051 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315effd640 1 -- 172.21.15.60:0/2462189051 >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 msgr2=0x7f3160161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315effd640 1 --2- 172.21.15.60:0/2462189051 >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 0x7f3160161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315effd640 1 -- 172.21.15.60:0/2462189051 >> v1:172.21.15.100:6789/0 conn(0x7f31601616f0 legacy=0x7f31600ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315effd640 1 -- 172.21.15.60:0/2462189051 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160162540 con 0x7f31601610f0 2024-04-25T08:40:02.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.389+0000 7f315f7fe640 1 --2- 172.21.15.60:0/2462189051 >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 0x7f3160161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f315effd640 1 --2- 172.21.15.60:0/2462189051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316006c1c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3158009870 tx=0x7f315802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30190a19b768ac81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f315dffb640 1 -- 172.21.15.60:0/2462189051 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158037070 con 0x7f31601610f0 2024-04-25T08:40:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f315dffb640 1 -- 172.21.15.60:0/2462189051 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3158032440 con 0x7f31601610f0 2024-04-25T08:40:02.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f315dffb640 1 -- 172.21.15.60:0/2462189051 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158032810 con 0x7f31601610f0 2024-04-25T08:40:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 -- 172.21.15.60:0/2462189051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 msgr2=0x7f316006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 --2- 172.21.15.60:0/2462189051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316006c1c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3158009870 tx=0x7f315802a270 comp rx=0 tx=0).stop 2024-04-25T08:40:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 -- 172.21.15.60:0/2462189051 shutdown_connections 2024-04-25T08:40:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 --2- 172.21.15.60:0/2462189051 >> v2:172.21.15.100:3300/0 conn(0x7f31600ce7b0 0x7f3160161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 --2- 172.21.15.60:0/2462189051 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316006c1c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.388 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 -- 172.21.15.60:0/2462189051 >> 172.21.15.60:0/2462189051 conn(0x7f316006f500 msgr2=0x7f316006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 -- 172.21.15.60:0/2462189051 shutdown_connections 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.390+0000 7f316549e640 1 -- 172.21.15.60:0/2462189051 wait complete. 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.392+0000 7f316549e640 1 Processor -- start 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.392+0000 7f316549e640 1 -- start start 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.392+0000 7f316549e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600ce7b0 0x7f31600b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.392+0000 7f316549e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.392+0000 7f316549e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 0x7f31600c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.389 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f316549e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600cf690 con 0x7f31601616f0 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f316549e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600cfce0 con 0x7f31600ce7b0 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f316549e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316006c700 con 0x7f31601610f0 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 0x7f31600c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600ce7b0 0x7f31600b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 0x7f31600c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49168/0 (socket says 172.21.15.60:49168) 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 -- 172.21.15.60:0/3500151125 learned_addr learned my addr 172.21.15.60:0/3500151125 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315e7fc640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 -- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 msgr2=0x7f316014b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.390 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316014b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 -- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600ce7b0 msgr2=0x7f31600b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600ce7b0 0x7f31600b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 -- 172.21.15.60:0/3500151125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316014b9b0 con 0x7f31601616f0 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f315f7fe640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 0x7f31600c9220 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f315000efc0 tx=0x7f3150009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150018030 con 0x7f31601616f0 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315000cde0 con 0x7f31601616f0 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31600af420 con 0x7f31601616f0 2024-04-25T08:40:02.391 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150003bb0 con 0x7f31601616f0 2024-04-25T08:40:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.393+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31600b0be0 con 0x7f31601616f0 2024-04-25T08:40:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.394+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3160073a00 con 0x7f31601616f0 2024-04-25T08:40:02.392 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.395+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3150005d10 con 0x7f31601616f0 2024-04-25T08:40:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.395+0000 7f31577fe640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f314006b4d0 0x7f314006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.395+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f315008a9d0 con 0x7f31601616f0 2024-04-25T08:40:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.395+0000 7f315effd640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f314006b4d0 0x7f314006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.396+0000 7f315effd640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f314006b4d0 0x7f314006d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f315802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.393 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.400+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3150022070 con 0x7f31601616f0 2024-04-25T08:40:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.578+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f31600af640 con 0x7f314006b4d0 2024-04-25T08:40:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.579+0000 7f31577fe640 1 -- 172.21.15.60:0/3500151125 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f31600af640 con 0x7f314006b4d0 2024-04-25T08:40:02.573 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f314006b4d0 msgr2=0x7f314006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f314006b4d0 0x7f314006d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f315802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 msgr2=0x7f31600c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 0x7f31600c9220 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f315000efc0 tx=0x7f3150009f90 comp rx=0 tx=0).stop 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 shutdown_connections 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f31601616f0 0x7f31600c9220 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f31601610f0 0x7f316014b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.575 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f314006b4d0 0x7f314006d990 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 --2- 172.21.15.60:0/3500151125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31600ce7b0 0x7f31600b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 >> 172.21.15.60:0/3500151125 conn(0x7f316006f500 msgr2=0x7f31600b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 shutdown_connections 2024-04-25T08:40:02.576 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.581+0000 7f316549e640 1 -- 172.21.15.60:0/3500151125 wait complete. 2024-04-25T08:40:02.585 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T08:40:02.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:02.690 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.697+0000 7f4863f6a640 1 Processor -- start 2024-04-25T08:40:02.691 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.697+0000 7f4863f6a640 1 -- start start 2024-04-25T08:40:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4863f6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4863f6a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f485c161710 0x7f485c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4863f6a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c15e000 con 0x7f485c0b9330 2024-04-25T08:40:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4863f6a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c0d1de0 con 0x7f485c161710 2024-04-25T08:40:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4863f6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c161c60 con 0x7f485c15dc20 2024-04-25T08:40:02.692 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4861d06640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4861505640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f485c161710 0x7f485c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4861d06640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35106/0 (socket says 172.21.15.60:35106) 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.698+0000 7f4861d06640 1 -- 172.21.15.60:0/3728186521 learned_addr learned my addr 172.21.15.60:0/3728186521 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.699+0000 7f4861d06640 1 -- 172.21.15.60:0/3728186521 >> v2:172.21.15.100:3300/0 conn(0x7f485c161710 msgr2=0x7f485c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.699+0000 7f4861d06640 1 --2- 172.21.15.60:0/3728186521 >> v2:172.21.15.100:3300/0 conn(0x7f485c161710 0x7f485c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.699+0000 7f4861d06640 1 -- 172.21.15.60:0/3728186521 >> v1:172.21.15.100:6789/0 conn(0x7f485c0b9330 legacy=0x7f485c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.699+0000 7f4861d06640 1 -- 172.21.15.60:0/3728186521 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c161f40 con 0x7f485c15dc20 2024-04-25T08:40:02.693 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.699+0000 7f4861505640 1 --2- 172.21.15.60:0/3728186521 >> v2:172.21.15.100:3300/0 conn(0x7f485c161710 0x7f485c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:02.694 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.699+0000 7f4861d06640 1 --2- 172.21.15.60:0/3728186521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ce670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4854009870 tx=0x7f485402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe008ff3b607c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4860d04640 1 -- 172.21.15.60:0/3728186521 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854037070 con 0x7f485c15dc20 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4860d04640 1 -- 172.21.15.60:0/3728186521 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854032440 con 0x7f485c15dc20 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4860d04640 1 -- 172.21.15.60:0/3728186521 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854032810 con 0x7f485c15dc20 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 -- 172.21.15.60:0/3728186521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 msgr2=0x7f485c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 --2- 172.21.15.60:0/3728186521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ce670 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4854009870 tx=0x7f485402a270 comp rx=0 tx=0).stop 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 -- 172.21.15.60:0/3728186521 shutdown_connections 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 --2- 172.21.15.60:0/3728186521 >> v2:172.21.15.100:3300/0 conn(0x7f485c161710 0x7f485c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.695 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 --2- 172.21.15.60:0/3728186521 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ce670 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 -- 172.21.15.60:0/3728186521 >> 172.21.15.60:0/3728186521 conn(0x7f485c0d0420 msgr2=0x7f485c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 -- 172.21.15.60:0/3728186521 shutdown_connections 2024-04-25T08:40:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.700+0000 7f4863f6a640 1 -- 172.21.15.60:0/3728186521 wait complete. 2024-04-25T08:40:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.701+0000 7f4863f6a640 1 Processor -- start 2024-04-25T08:40:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.701+0000 7f4863f6a640 1 -- start start 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4863f6a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f485c0b9330 0x7f485c160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4863f6a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4863f6a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f485c161710 0x7f485c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4863f6a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0b5e50 con 0x7f485c161710 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4863f6a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c162500 con 0x7f485c0b9330 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4863f6a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0b5fd0 con 0x7f485c15dc20 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4862507640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f485c161710 0x7f485c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.699 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4861505640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4861d06640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f485c0b9330 0x7f485c160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4861505640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35116/0 (socket says 172.21.15.60:35116) 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4861505640 1 -- 172.21.15.60:0/769964017 learned_addr learned my addr 172.21.15.60:0/769964017 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.702+0000 7f4862507640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f485c161710 0x7f485c152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49178/0 (socket says 172.21.15.60:49178) 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861d06640 1 -- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 msgr2=0x7f485c0ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861d06640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861d06640 1 -- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f485c161710 msgr2=0x7f485c152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861d06640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f485c161710 0x7f485c152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861d06640 1 -- 172.21.15.60:0/769964017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c0aa920 con 0x7f485c0b9330 2024-04-25T08:40:02.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861505640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f485c15dc20 0x7f485c0ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4861d06640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f485c0b9330 0x7f485c160bc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4854031000 tx=0x7f48540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854041020 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854030660 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c0aac00 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854037040 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.703+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c0ab1c0 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.704+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f485c073a00 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== mon.1 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4854066070 con 0x7f485c0b9330 2024-04-25T08:40:02.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f485a7fc640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f484806b4d0 0x7f484806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== mon.1 v2:172.21.15.100:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f48540ab4f0 con 0x7f485c0b9330 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f4861505640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f484806b4d0 0x7f484806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f485a7fc640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f4848071970 0x7f4848073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 --> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4854034ec0 con 0x7f4848071970 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== mon.1 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48540ab8a0 con 0x7f485c0b9330 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.705+0000 7f4862507640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f4848071970 0x7f4848073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.706+0000 7f4861505640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f484806b4d0 0x7f484806d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f485c0b8f70 tx=0x7f4850009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.706+0000 7f4862507640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f4848071970 0x7f4848073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.706+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== osd.1 v2:172.21.15.60:6800/3720682638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4854034ec0 con 0x7f4848071970 2024-04-25T08:40:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.719+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 --> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f485c0d16b0 con 0x7f4848071970 2024-04-25T08:40:02.713 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f485a7fc640 1 -- 172.21.15.60:0/769964017 <== osd.1 v2:172.21.15.60:6800/3720682638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f485c0d16b0 con 0x7f4848071970 2024-04-25T08:40:02.713 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:02.713 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:02.713 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f4848071970 msgr2=0x7f4848073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.60:6800/3720682638,v1:172.21.15.60:6801/3720682638] conn(0x7f4848071970 0x7f4848073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.714 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f484806b4d0 msgr2=0x7f484806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f484806b4d0 0x7f484806d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f485c0b8f70 tx=0x7f4850009190 comp rx=0 tx=0).stop 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f485c0b9330 msgr2=0x7f485c160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 --2- 172.21.15.60:0/769964017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f485c0b9330 0x7f485c160bc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4854031000 tx=0x7f48540046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4862507640 1 -- 172.21.15.60:0/769964017 reap_dead start 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 shutdown_connections 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 >> 172.21.15.60:0/769964017 conn(0x7f485c0d0420 msgr2=0x7f485c0c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 shutdown_connections 2024-04-25T08:40:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.720+0000 7f4863f6a640 1 -- 172.21.15.60:0/769964017 wait complete. 2024-04-25T08:40:02.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T08:40:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.832+0000 7fe9dc458640 1 Processor -- start 2024-04-25T08:40:02.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.832+0000 7fe9dc458640 1 -- start start 2024-04-25T08:40:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9dc458640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9dc458640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d40cf1d0 con 0x7fe9d4161240 2024-04-25T08:40:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9dc458640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d4161620 con 0x7fe9d4161890 2024-04-25T08:40:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9dc458640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40b7120 con 0x7fe9d40b5bb0 2024-04-25T08:40:02.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9da1f4640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fe9d4161240 0x7fe9d40ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43714/0 (socket says 172.21.15.60:43714) 2024-04-25T08:40:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9da1f4640 1 -- 172.21.15.60:0/95391285 learned_addr learned my addr 172.21.15.60:0/95391285 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9da9f5640 1 --2- 172.21.15.60:0/95391285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9da9f5640 1 -- 172.21.15.60:0/95391285 >> v1:172.21.15.100:6789/0 conn(0x7fe9d4161890 legacy=0x7fe9d40b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9da9f5640 1 -- 172.21.15.60:0/95391285 >> v1:172.21.15.60:6789/0 conn(0x7fe9d4161240 legacy=0x7fe9d40ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.833+0000 7fe9da9f5640 1 -- 172.21.15.60:0/95391285 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4161e00 con 0x7fe9d40b5bb0 2024-04-25T08:40:02.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.834+0000 7fe9da9f5640 1 --2- 172.21.15.60:0/95391285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40b6be0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009dc0 tx=0x7fe9cc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b5dd5e80d974370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.836+0000 7fe9d91f2640 1 -- 172.21.15.60:0/95391285 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc007740 con 0x7fe9d40b5bb0 2024-04-25T08:40:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.836+0000 7fe9d91f2640 1 -- 172.21.15.60:0/95391285 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc0078e0 con 0x7fe9d40b5bb0 2024-04-25T08:40:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.836+0000 7fe9d91f2640 1 -- 172.21.15.60:0/95391285 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc007cb0 con 0x7fe9d40b5bb0 2024-04-25T08:40:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.836+0000 7fe9dc458640 1 -- 172.21.15.60:0/95391285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 msgr2=0x7fe9d40b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.836+0000 7fe9dc458640 1 --2- 172.21.15.60:0/95391285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40b6be0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009dc0 tx=0x7fe9cc02a6d0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.837+0000 7fe9dc458640 1 -- 172.21.15.60:0/95391285 shutdown_connections 2024-04-25T08:40:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.837+0000 7fe9dc458640 1 --2- 172.21.15.60:0/95391285 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40b6be0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.837+0000 7fe9dc458640 1 -- 172.21.15.60:0/95391285 >> 172.21.15.60:0/95391285 conn(0x7fe9d40d04a0 msgr2=0x7fe9d40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.837+0000 7fe9dc458640 1 -- 172.21.15.60:0/95391285 shutdown_connections 2024-04-25T08:40:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.837+0000 7fe9dc458640 1 -- 172.21.15.60:0/95391285 wait complete. 2024-04-25T08:40:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.838+0000 7fe9dc458640 1 Processor -- start 2024-04-25T08:40:02.831 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.838+0000 7fe9dc458640 1 -- start start 2024-04-25T08:40:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.838+0000 7fe9dc458640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.833 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.838+0000 7fe9dc458640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d4161240 0x7fe9d4155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9dc458640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 0x7fe9d40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9dc458640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41626a0 con 0x7fe9d4161890 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9dc458640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41623c0 con 0x7fe9d4161240 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9dc458640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40d15f0 con 0x7fe9d40b5bb0 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 0x7fe9d40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 0x7fe9d40cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49186/0 (socket says 172.21.15.60:49186) 2024-04-25T08:40:02.834 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 -- 172.21.15.60:0/1984072119 learned_addr learned my addr 172.21.15.60:0/1984072119 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da1f4640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9d99f3640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d4161240 0x7fe9d4155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 -- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 msgr2=0x7fe9d40cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 -- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d4161240 msgr2=0x7fe9d4155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d4161240 0x7fe9d4155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 -- 172.21.15.60:0/1984072119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d41e7730 con 0x7fe9d4161890 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9da9f5640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 0x7fe9d40cb620 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009ee0 tx=0x7fe9cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.835 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc040070 con 0x7fe9d4161890 2024-04-25T08:40:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc031c90 con 0x7fe9d4161890 2024-04-25T08:40:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d41e79d0 con 0x7fe9d4161890 2024-04-25T08:40:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc045680 con 0x7fe9d4161890 2024-04-25T08:40:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.839+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d41e7f90 con 0x7fe9d4161890 2024-04-25T08:40:02.836 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.840+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d4073a00 con 0x7fe9d4161890 2024-04-25T08:40:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.841+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9cc039040 con 0x7fe9d4161890 2024-04-25T08:40:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.841+0000 7fe9caffd640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9b806b4d0 0x7fe9b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.841+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe9cc0ab920 con 0x7fe9d4161890 2024-04-25T08:40:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.841+0000 7fe9da1f4640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9b806b4d0 0x7fe9b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.837 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.842+0000 7fe9da1f4640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9b806b4d0 0x7fe9b806d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005880 tx=0x7fe9c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.838 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:02.845+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9cc07a410 con 0x7fe9d4161890 2024-04-25T08:40:03.018 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.025+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9d406d240 con 0x7fe9b806b4d0 2024-04-25T08:40:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.026+0000 7fe9caffd640 1 -- 172.21.15.60:0/1984072119 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe9d406d240 con 0x7fe9b806b4d0 2024-04-25T08:40:03.019 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T08:40:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9b806b4d0 msgr2=0x7fe9b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.021 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9b806b4d0 0x7fe9b806d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4005880 tx=0x7fe9c4009040 comp rx=0 tx=0).stop 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 msgr2=0x7fe9d40cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 0x7fe9d40cb620 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009ee0 tx=0x7fe9cc004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 shutdown_connections 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9d4161890 0x7fe9d40cb620 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fe9b806b4d0 0x7fe9b806d990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe9d4161240 0x7fe9d4155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 --2- 172.21.15.60:0/1984072119 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fe9d40b5bb0 0x7fe9d40cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.022 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 >> 172.21.15.60:0/1984072119 conn(0x7fe9d40d04a0 msgr2=0x7fe9d415f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 shutdown_connections 2024-04-25T08:40:03.023 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.028+0000 7fe9dc458640 1 -- 172.21.15.60:0/1984072119 wait complete. 2024-04-25T08:40:03.032 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T08:40:03.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:03.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.144+0000 7fa0534ab640 1 Processor -- start 2024-04-25T08:40:03.137 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.144+0000 7fa0534ab640 1 -- start start 2024-04-25T08:40:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa0534ab640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa04c0ced80 0x7fa04c0d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.139 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa0534ab640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa04c0d1ce0 0x7fa04c0b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa0534ab640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04c0d2120 con 0x7fa04c15dc20 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa0534ab640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c0d23a0 con 0x7fa04c0ced80 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa0534ab640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04c0d2680 con 0x7fa04c0d1ce0 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa04c0ced80 0x7fa04c0d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fa04c0ced80 0x7fa04c0d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49198/0 (socket says 172.21.15.60:49198) 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 -- 172.21.15.60:0/762857483 learned_addr learned my addr 172.21.15.60:0/762857483 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa051a48640 1 --2- 172.21.15.60:0/762857483 >> v2:172.21.15.100:3300/0 conn(0x7fa04c0d1ce0 0x7fa04c0b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.140 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 -- 172.21.15.60:0/762857483 >> v2:172.21.15.100:3300/0 conn(0x7fa04c0d1ce0 msgr2=0x7fa04c0b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 --2- 172.21.15.60:0/762857483 >> v2:172.21.15.100:3300/0 conn(0x7fa04c0d1ce0 0x7fa04c0b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 -- 172.21.15.60:0/762857483 >> v1:172.21.15.100:6789/0 conn(0x7fa04c15dc20 legacy=0x7fa04c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.145+0000 7fa050a46640 1 -- 172.21.15.60:0/762857483 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0d2960 con 0x7fa04c0ced80 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.146+0000 7fa050a46640 1 --2- 172.21.15.60:0/762857483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c0d3440 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0b6050 tx=0x7fa03c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e7ce991bbdb4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.146+0000 7fa043fff640 1 -- 172.21.15.60:0/762857483 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c037070 con 0x7fa04c0ced80 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.146+0000 7fa043fff640 1 -- 172.21.15.60:0/762857483 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c032440 con 0x7fa04c0ced80 2024-04-25T08:40:03.141 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.146+0000 7fa043fff640 1 -- 172.21.15.60:0/762857483 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c032810 con 0x7fa04c0ced80 2024-04-25T08:40:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 -- 172.21.15.60:0/762857483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 msgr2=0x7fa04c0d3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 --2- 172.21.15.60:0/762857483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c0d3440 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0b6050 tx=0x7fa03c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 -- 172.21.15.60:0/762857483 shutdown_connections 2024-04-25T08:40:03.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 --2- 172.21.15.60:0/762857483 >> v2:172.21.15.100:3300/0 conn(0x7fa04c0d1ce0 0x7fa04c0b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 --2- 172.21.15.60:0/762857483 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c0d3440 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 -- 172.21.15.60:0/762857483 >> 172.21.15.60:0/762857483 conn(0x7fa04c061a20 msgr2=0x7fa04c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 -- 172.21.15.60:0/762857483 shutdown_connections 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.147+0000 7fa0534ab640 1 -- 172.21.15.60:0/762857483 wait complete. 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.148+0000 7fa0534ab640 1 Processor -- start 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.148+0000 7fa0534ab640 1 -- start start 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa0534ab640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa0534ab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa04c0d1ce0 0x7fa04c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa0534ab640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa04c15dc20 0x7fa04c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa0534ab640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0b6a30 con 0x7fa04c0ced80 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa0534ab640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0d2f20 con 0x7fa04c0d1ce0 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa0534ab640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c0b6bb0 con 0x7fa04c15dc20 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49200/0 (socket says 172.21.15.60:49200) 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 -- 172.21.15.60:0/3645300133 learned_addr learned my addr 172.21.15.60:0/3645300133 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa050a46640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa04c0d1ce0 0x7fa04c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051a48640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa04c15dc20 0x7fa04c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 -- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa04c15dc20 msgr2=0x7fa04c06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa04c15dc20 0x7fa04c06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 -- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa04c0d1ce0 msgr2=0x7fa04c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa04c0d1ce0 0x7fa04c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 -- 172.21.15.60:0/3645300133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c1534f0 con 0x7fa04c0ced80 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa050a46640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa04c0d1ce0 0x7fa04c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa051247640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c152fb0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fa04800b3e0 tx=0x7fa04800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.149+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048015020 con 0x7fa04c0ced80 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.150+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa048011850 con 0x7fa04c0ced80 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.150+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c0cba10 con 0x7fa04c0ced80 2024-04-25T08:40:03.147 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.150+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048011c40 con 0x7fa04c0ced80 2024-04-25T08:40:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.150+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c0cbfd0 con 0x7fa04c0ced80 2024-04-25T08:40:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.150+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa04c0cc690 con 0x7fa04c0ced80 2024-04-25T08:40:03.148 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.151+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa048018020 con 0x7fa04c0ced80 2024-04-25T08:40:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.151+0000 7fa041ffb640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa03006b4d0 0x7fa03006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa04808a050 con 0x7fa04c0ced80 2024-04-25T08:40:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa041ffb640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7fa030071970 0x7fa030073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 --> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa030074480 con 0x7fa030071970 2024-04-25T08:40:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa048091050 con 0x7fa04c0ced80 2024-04-25T08:40:03.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa050a46640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa03006b4d0 0x7fa03006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa051a48640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7fa030071970 0x7fa030073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa051a48640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7fa030071970 0x7fa030073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.152+0000 7fa050a46640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa03006b4d0 0x7fa03006d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa03c034000 tx=0x7fa03c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.153+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== osd.2 v2:172.21.15.60:6808/1418327746 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa030074480 con 0x7fa030071970 2024-04-25T08:40:03.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.166+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 --> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa04c0ca730 con 0x7fa030071970 2024-04-25T08:40:03.159 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.166+0000 7fa041ffb640 1 -- 172.21.15.60:0/3645300133 <== osd.2 v2:172.21.15.60:6808/1418327746 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa04c0ca730 con 0x7fa030071970 2024-04-25T08:40:03.160 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:03.160 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.166+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7fa030071970 msgr2=0x7fa030073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.166+0000 7fa0534ab640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:6808/1418327746,v1:172.21.15.60:6809/1418327746] conn(0x7fa030071970 0x7fa030073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa03006b4d0 msgr2=0x7fa03006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa03006b4d0 0x7fa03006d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa03c034000 tx=0x7fa03c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:03.161 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 msgr2=0x7fa04c152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 --2- 172.21.15.60:0/3645300133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa04c0ced80 0x7fa04c152fb0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fa04800b3e0 tx=0x7fa04800b8a0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa051a48640 1 -- 172.21.15.60:0/3645300133 reap_dead start 2024-04-25T08:40:03.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 shutdown_connections 2024-04-25T08:40:03.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 >> 172.21.15.60:0/3645300133 conn(0x7fa04c061a20 msgr2=0x7fa04c15ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 shutdown_connections 2024-04-25T08:40:03.162 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.167+0000 7fa0534ab640 1 -- 172.21.15.60:0/3645300133 wait complete. 2024-04-25T08:40:03.170 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T08:40:03.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.279+0000 7f94914cb640 1 Processor -- start 2024-04-25T08:40:03.272 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.279+0000 7f94914cb640 1 -- start start 2024-04-25T08:40:03.273 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f94914cb640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f948c15dc20 0x7f948c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f94914cb640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c06cda0 con 0x7f948c161710 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f94914cb640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c15e000 con 0x7f948c0d32f0 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f94914cb640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0d1de0 con 0x7f948c15dc20 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f948affd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f948c15dc20 0x7f948c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f948a7fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f948c161710 0x7f948c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43726/0 (socket says 172.21.15.60:43726) 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f948a7fc640 1 -- 172.21.15.60:0/762617346 learned_addr learned my addr 172.21.15.60:0/762617346 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:03.274 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f948affd640 1 -- 172.21.15.60:0/762617346 >> v1:172.21.15.100:6789/0 conn(0x7f948c0d32f0 legacy=0x7f948c06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:03.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f948affd640 1 -- 172.21.15.60:0/762617346 >> v1:172.21.15.60:6789/0 conn(0x7f948c161710 legacy=0x7f948c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f948affd640 1 -- 172.21.15.60:0/762617346 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c161c60 con 0x7f948c15dc20 2024-04-25T08:40:03.275 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.280+0000 7f9489ffb640 1 -- 172.21.15.60:0/762617346 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944700697 0 0) 0x7f948c06cda0 con 0x7f948c161710 2024-04-25T08:40:03.276 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.281+0000 7f948affd640 1 --2- 172.21.15.60:0/762617346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0ce670 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f947c0097a0 tx=0x7f947c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f503c791c23b33aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.281+0000 7f9489ffb640 1 -- 172.21.15.60:0/762617346 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c037070 con 0x7f948c15dc20 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.281+0000 7f9489ffb640 1 -- 172.21.15.60:0/762617346 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c032440 con 0x7f948c15dc20 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.281+0000 7f9489ffb640 1 -- 172.21.15.60:0/762617346 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c032810 con 0x7f948c15dc20 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 -- 172.21.15.60:0/762617346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 msgr2=0x7f948c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 --2- 172.21.15.60:0/762617346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0ce670 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f947c0097a0 tx=0x7f947c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 -- 172.21.15.60:0/762617346 shutdown_connections 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 --2- 172.21.15.60:0/762617346 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0ce670 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 -- 172.21.15.60:0/762617346 >> 172.21.15.60:0/762617346 conn(0x7f948c0d0420 msgr2=0x7f948c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.277 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 -- 172.21.15.60:0/762617346 shutdown_connections 2024-04-25T08:40:03.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.282+0000 7f94914cb640 1 -- 172.21.15.60:0/762617346 wait complete. 2024-04-25T08:40:03.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.283+0000 7f94914cb640 1 Processor -- start 2024-04-25T08:40:03.278 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.283+0000 7f94914cb640 1 -- start start 2024-04-25T08:40:03.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f94914cb640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f948c0d32f0 0x7f948c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.280 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f94914cb640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f94914cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f948c161710 0x7f948c0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f94914cb640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c162500 con 0x7f948c15dc20 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f94914cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c162220 con 0x7f948c161710 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f94914cb640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0b5e50 con 0x7f948c0d32f0 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49220/0 (socket says 172.21.15.60:49220) 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 -- 172.21.15.60:0/4084984642 learned_addr learned my addr 172.21.15.60:0/4084984642 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:03.281 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948affd640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f948c0d32f0 0x7f948c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948b7fe640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f948c161710 0x7f948c0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 -- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f948c0d32f0 msgr2=0x7f948c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f948c0d32f0 0x7f948c15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 -- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f948c161710 msgr2=0x7f948c0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f948c161710 0x7f948c0cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 -- 172.21.15.60:0/4084984642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0cc5d0 con 0x7f948c15dc20 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948b7fe640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f948c161710 0x7f948c0cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948affd640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f948c0d32f0 0x7f948c15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.284+0000 7f948a7fc640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0cd670 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f948000c4d0 tx=0x7f948000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.285+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480018070 con 0x7f948c15dc20 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.285+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9480007500 con 0x7f948c15dc20 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.285+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c0cc8b0 con 0x7f948c15dc20 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.285+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480004a20 con 0x7f948c15dc20 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.285+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c0cce70 con 0x7f948c15dc20 2024-04-25T08:40:03.283 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.286+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c073a00 con 0x7f948c15dc20 2024-04-25T08:40:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.286+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f948001b070 con 0x7f948c15dc20 2024-04-25T08:40:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.286+0000 7f94737fe640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f947806b4d0 0x7f947806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.287+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f94800893f0 con 0x7f948c15dc20 2024-04-25T08:40:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.287+0000 7f948affd640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f947806b4d0 0x7f947806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.284 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.287+0000 7f948affd640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f947806b4d0 0x7f947806d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f947c034000 tx=0x7f947c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.285 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.291+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94800583e0 con 0x7f948c15dc20 2024-04-25T08:40:03.464 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.470+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f948c068070 con 0x7f947806b4d0 2024-04-25T08:40:03.465 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.471+0000 7f94737fe640 1 -- 172.21.15.60:0/4084984642 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f948c068070 con 0x7f947806b4d0 2024-04-25T08:40:03.465 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T08:40:03.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f947806b4d0 msgr2=0x7f947806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f947806b4d0 0x7f947806d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f947c034000 tx=0x7f947c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:03.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 msgr2=0x7f948c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.466 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0cd670 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f948000c4d0 tx=0x7f948000c990 comp rx=0 tx=0).stop 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 shutdown_connections 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f947806b4d0 0x7f947806d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f948c161710 0x7f948c0cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f948c15dc20 0x7f948c0cd670 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 --2- 172.21.15.60:0/4084984642 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f948c0d32f0 0x7f948c15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 >> 172.21.15.60:0/4084984642 conn(0x7f948c0d0420 msgr2=0x7f948c160b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 shutdown_connections 2024-04-25T08:40:03.467 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.473+0000 7f94914cb640 1 -- 172.21.15.60:0/4084984642 wait complete. 2024-04-25T08:40:03.477 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T08:40:03.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:03.582 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.589+0000 7f61901a5640 1 Processor -- start 2024-04-25T08:40:03.583 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.589+0000 7f61901a5640 1 -- start start 2024-04-25T08:40:03.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f61901a5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f618815b7d0 0x7f61880cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.585 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f61901a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f61901a5640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618806cf50 con 0x7f618806c2d0 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f61901a5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61880b5fc0 con 0x7f618815b7d0 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f61901a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880ce530 con 0x7f61880b5620 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f618e742640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f618806c2d0 0x7f618806c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43738/0 (socket says 172.21.15.60:43738) 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f618e742640 1 -- 172.21.15.60:0/714306320 learned_addr learned my addr 172.21.15.60:0/714306320 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f618df41640 1 --2- 172.21.15.60:0/714306320 >> v2:172.21.15.100:3300/0 conn(0x7f618815b7d0 0x7f61880cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.586 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f618d740640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f618cf3f640 1 -- 172.21.15.60:0/714306320 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358558366 0 0) 0x7f618806cf50 con 0x7f618806c2d0 2024-04-25T08:40:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.590+0000 7f618cf3f640 1 -- 172.21.15.60:0/714306320 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6174003520 con 0x7f618806c2d0 2024-04-25T08:40:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618df41640 1 -- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 msgr2=0x7f61880b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618df41640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618df41640 1 -- 172.21.15.60:0/714306320 >> v1:172.21.15.60:6789/0 conn(0x7f618806c2d0 legacy=0x7f618806c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.587 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618df41640 1 -- 172.21.15.60:0/714306320 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61880ce810 con 0x7f618815b7d0 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618d740640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618df41640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cf010 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f61800097a0 tx=0x7f618002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d895e61cd018ca09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618cf3f640 1 -- 172.21.15.60:0/714306320 <== mon.1 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180037070 con 0x7f618815b7d0 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618cf3f640 1 -- 172.21.15.60:0/714306320 <== mon.1 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180032440 con 0x7f618815b7d0 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.591+0000 7f618cf3f640 1 -- 172.21.15.60:0/714306320 <== mon.1 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61800327f0 con 0x7f618815b7d0 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 -- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 msgr2=0x7f61880cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cf010 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f61800097a0 tx=0x7f618002a180 comp rx=0 tx=0).stop 2024-04-25T08:40:03.588 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 -- 172.21.15.60:0/714306320 shutdown_connections 2024-04-25T08:40:03.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 --2- 172.21.15.60:0/714306320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cf010 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.589 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 -- 172.21.15.60:0/714306320 >> 172.21.15.60:0/714306320 conn(0x7f61880d0800 msgr2=0x7f61880d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 -- 172.21.15.60:0/714306320 shutdown_connections 2024-04-25T08:40:03.590 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.592+0000 7f61901a5640 1 -- 172.21.15.60:0/714306320 wait complete. 2024-04-25T08:40:03.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.593+0000 7f61901a5640 1 Processor -- start 2024-04-25T08:40:03.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.593+0000 7f61901a5640 1 -- start start 2024-04-25T08:40:03.593 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f61901a5640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f618806c2d0 0x7f61880cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f61901a5640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f61901a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f61901a5640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880cf550 con 0x7f618806c2d0 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f61901a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618806d230 con 0x7f618815b7d0 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f61901a5640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61880cf6d0 con 0x7f61880b5620 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f618806c2d0 0x7f61880cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.594 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f618806c2d0 0x7f61880cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49224/0 (socket says 172.21.15.60:49224) 2024-04-25T08:40:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 -- 172.21.15.60:0/3346505079 learned_addr learned my addr 172.21.15.60:0/3346505079 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618d740640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 -- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 msgr2=0x7f61880ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618e742640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f61880b5620 0x7f61880ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.595 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 -- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 msgr2=0x7f61880cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618df41640 1 -- 172.21.15.60:0/3346505079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61881f6740 con 0x7f618806c2d0 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.594+0000 7f618e742640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f618815b7d0 0x7f61880cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.595+0000 7f618df41640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f618806c2d0 0x7f61880cace0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f618002a1c0 tx=0x7f61800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.595+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180037040 con 0x7f618806c2d0 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.595+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180042070 con 0x7f618806c2d0 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.595+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61881f6a20 con 0x7f618806c2d0 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.595+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618003b930 con 0x7f618806c2d0 2024-04-25T08:40:03.596 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.595+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61881f6fe0 con 0x7f618806c2d0 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.596+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61880ca680 con 0x7f618806c2d0 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.596+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f618003bad0 con 0x7f618806c2d0 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f617e7fc640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f616c06b4d0 0x7f616c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f61800aa900 con 0x7f618806c2d0 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f618d740640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f616c06b4d0 0x7f616c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f617e7fc640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f616c071970 0x7f616c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 --> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f616c074480 con 0x7f616c071970 2024-04-25T08:40:03.597 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61800af2c0 con 0x7f618806c2d0 2024-04-25T08:40:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f618e742640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f616c071970 0x7f616c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f618e742640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f616c071970 0x7f616c073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.597+0000 7f618d740640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f616c06b4d0 0x7f616c06d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6178009560 tx=0x7f6178008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.598+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== osd.3 v2:172.21.15.60:6816/426387744 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f616c074480 con 0x7f616c071970 2024-04-25T08:40:03.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.611+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 --> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6188073a00 con 0x7f616c071970 2024-04-25T08:40:03.605 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f617e7fc640 1 -- 172.21.15.60:0/3346505079 <== osd.3 v2:172.21.15.60:6816/426387744 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6188073a00 con 0x7f616c071970 2024-04-25T08:40:03.605 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:03.605 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:03.606 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f616c071970 msgr2=0x7f616c073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:6816/426387744,v1:172.21.15.60:6818/426387744] conn(0x7f616c071970 0x7f616c073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f616c06b4d0 msgr2=0x7f616c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f616c06b4d0 0x7f616c06d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6178009560 tx=0x7f6178008040 comp rx=0 tx=0).stop 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f618806c2d0 msgr2=0x7f61880cace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 --2- 172.21.15.60:0/3346505079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f618806c2d0 0x7f61880cace0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f618002a1c0 tx=0x7f61800046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f618e742640 1 -- 172.21.15.60:0/3346505079 reap_dead start 2024-04-25T08:40:03.607 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 shutdown_connections 2024-04-25T08:40:03.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 >> 172.21.15.60:0/3346505079 conn(0x7f61880d0800 msgr2=0x7f618806c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 shutdown_connections 2024-04-25T08:40:03.608 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.612+0000 7f61901a5640 1 -- 172.21.15.60:0/3346505079 wait complete. 2024-04-25T08:40:03.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T08:40:03.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.724+0000 7f073ea38640 1 Processor -- start 2024-04-25T08:40:03.717 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.724+0000 7f073ea38640 1 -- start start 2024-04-25T08:40:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073ea38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073ea38640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07380ccfe0 0x7f07380cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073ea38640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07380afc80 con 0x7f073815de80 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073ea38640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380cb210 con 0x7f07380ccfe0 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073ea38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380ce4d0 con 0x7f07380cadb0 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07380ccfe0 0x7f07380cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f07380ccfe0 0x7f07380cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49238/0 (socket says 172.21.15.60:49238) 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 -- 172.21.15.60:0/3917263825 learned_addr learned my addr 172.21.15.60:0/3917263825 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f07377fe640 1 --2- 172.21.15.60:0/3917263825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 -- 172.21.15.60:0/3917263825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 msgr2=0x7f07380cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:03.720 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 --2- 172.21.15.60:0/3917263825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 -- 172.21.15.60:0/3917263825 >> v1:172.21.15.100:6789/0 conn(0x7f073815de80 legacy=0x7f07380cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:03.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.725+0000 7f073cfd5640 1 -- 172.21.15.60:0/3917263825 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380ce7b0 con 0x7f07380ccfe0 2024-04-25T08:40:03.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.726+0000 7f073cfd5640 1 --2- 172.21.15.60:0/3917263825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07380cd470 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f0730009e50 tx=0x7f073002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e4943429eb99d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.726+0000 7f0736ffd640 1 -- 172.21.15.60:0/3917263825 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730036070 con 0x7f07380ccfe0 2024-04-25T08:40:03.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.726+0000 7f0736ffd640 1 -- 172.21.15.60:0/3917263825 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0730031600 con 0x7f07380ccfe0 2024-04-25T08:40:03.721 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.726+0000 7f0736ffd640 1 -- 172.21.15.60:0/3917263825 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300319d0 con 0x7f07380ccfe0 2024-04-25T08:40:03.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 -- 172.21.15.60:0/3917263825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 msgr2=0x7f07380cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 --2- 172.21.15.60:0/3917263825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07380cd470 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f0730009e50 tx=0x7f073002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 -- 172.21.15.60:0/3917263825 shutdown_connections 2024-04-25T08:40:03.724 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 --2- 172.21.15.60:0/3917263825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07380cd470 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 --2- 172.21.15.60:0/3917263825 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 -- 172.21.15.60:0/3917263825 >> 172.21.15.60:0/3917263825 conn(0x7f07380b2390 msgr2=0x7f07380b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 -- 172.21.15.60:0/3917263825 shutdown_connections 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.727+0000 7f073ea38640 1 -- 172.21.15.60:0/3917263825 wait complete. 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.728+0000 7f073ea38640 1 Processor -- start 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.728+0000 7f073ea38640 1 -- start start 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07381558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.725 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f073815de80 0x7f07380b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073815e260 con 0x7f07380ccfe0 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380cfc00 con 0x7f073815de80 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380b2eb0 con 0x7f07380cadb0 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07381558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07381558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49242/0 (socket says 172.21.15.60:49242) 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 -- 172.21.15.60:0/2744469990 learned_addr learned my addr 172.21.15.60:0/2744469990 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f0737fff640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.726 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073cfd5640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f073815de80 0x7f07380b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 -- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 msgr2=0x7f07380b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 -- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f073815de80 msgr2=0x7f07380b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f073815de80 0x7f07380b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 -- 172.21.15.60:0/2744469990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380b5de0 con 0x7f07380ccfe0 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073cfd5640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f073815de80 0x7f07380b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f07377fe640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07381558b0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f072800d800 tx=0x7f072800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800b6c0 con 0x7f07380ccfe0 2024-04-25T08:40:03.727 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728019070 con 0x7f07380ccfe0 2024-04-25T08:40:03.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.729+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0738158220 con 0x7f07380ccfe0 2024-04-25T08:40:03.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.730+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800ba80 con 0x7f07380ccfe0 2024-04-25T08:40:03.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.730+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07380b4550 con 0x7f07380ccfe0 2024-04-25T08:40:03.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.731+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0738073a00 con 0x7f07380ccfe0 2024-04-25T08:40:03.728 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.731+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f072800bc20 con 0x7f07380ccfe0 2024-04-25T08:40:03.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.732+0000 7f0734ff9640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f072406b4d0 0x7f072406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:03.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.732+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f072808a4f0 con 0x7f07380ccfe0 2024-04-25T08:40:03.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.732+0000 7f0737fff640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f072406b4d0 0x7f072406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:03.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.732+0000 7f0737fff640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f072406b4d0 0x7f072406d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f072c006fd0 tx=0x7f072c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:03.729 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.736+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07380b5de0 con 0x7f07380ccfe0 2024-04-25T08:40:03.912 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.919+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f07380b2870 con 0x7f072406b4d0 2024-04-25T08:40:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.919+0000 7f0734ff9640 1 -- 172.21.15.60:0/2744469990 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f07380b2870 con 0x7f072406b4d0 2024-04-25T08:40:03.913 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f072406b4d0 msgr2=0x7f072406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f072406b4d0 0x7f072406d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f072c006fd0 tx=0x7f072c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 msgr2=0x7f07381558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07381558b0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f072800d800 tx=0x7f072800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 shutdown_connections 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f072406b4d0 0x7f072406d990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.915 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f073815de80 0x7f07380b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f07380ccfe0 0x7f07381558b0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 --2- 172.21.15.60:0/2744469990 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f07380cadb0 0x7f07380b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 >> 172.21.15.60:0/2744469990 conn(0x7f07380b2390 msgr2=0x7f07380cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 shutdown_connections 2024-04-25T08:40:03.916 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:03.921+0000 7f073ea38640 1 -- 172.21.15.60:0/2744469990 wait complete. 2024-04-25T08:40:03.925 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T08:40:03.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.037+0000 7fa0ae11d640 1 Processor -- start 2024-04-25T08:40:04.031 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.037+0000 7fa0ae11d640 1 -- start start 2024-04-25T08:40:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0ae11d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 0x7fa0a80d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0ae11d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 0x7fa0a80d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.032 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0ae11d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a815e000 con 0x7fa0a815dc20 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0ae11d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a80d1de0 con 0x7fa0a8161710 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0ae11d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8161c60 con 0x7fa0a80ced80 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0a6ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 0x7fa0a80d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0a7fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 0x7fa0a80d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0a6ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 0x7fa0a80d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.60:35196/0 (socket says 172.21.15.60:35196) 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0a77fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa0a815dc20 0x7fa0a80ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.60:38218/0 (socket says 172.21.15.60:38218) 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0a6ffd640 1 -- 172.21.15.60:0/1368397878 learned_addr learned my addr 172.21.15.60:0/1368397878 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.033 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.038+0000 7fa0a7fff640 1 --2- 172.21.15.60:0/1368397878 >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 0x7fa0a80d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.60:45270/0 (socket says 172.21.15.60:45270) 2024-04-25T08:40:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.039+0000 7fa0a6ffd640 1 -- 172.21.15.60:0/1368397878 >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 msgr2=0x7fa0a80d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.039+0000 7fa0a6ffd640 1 --2- 172.21.15.60:0/1368397878 >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 0x7fa0a80d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.039+0000 7fa0a6ffd640 1 -- 172.21.15.60:0/1368397878 >> v1:172.21.15.100:6789/0 conn(0x7fa0a815dc20 legacy=0x7fa0a80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.039+0000 7fa0a6ffd640 1 -- 172.21.15.60:0/1368397878 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a8161f40 con 0x7fa0a80ced80 2024-04-25T08:40:04.034 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.039+0000 7fa0a7fff640 1 --2- 172.21.15.60:0/1368397878 >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 0x7fa0a80d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:04.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0a6ffd640 1 --2- 172.21.15.60:0/1368397878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 0x7fa0a80d22e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009870 tx=0x7fa09c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61db9f4eac02e4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0a67fc640 1 -- 172.21.15.60:0/1368397878 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c037070 con 0x7fa0a80ced80 2024-04-25T08:40:04.037 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0a67fc640 1 -- 172.21.15.60:0/1368397878 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c032440 con 0x7fa0a80ced80 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0a67fc640 1 -- 172.21.15.60:0/1368397878 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c032810 con 0x7fa0a80ced80 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1368397878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 msgr2=0x7fa0a80d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 --2- 172.21.15.60:0/1368397878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 0x7fa0a80d22e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009870 tx=0x7fa09c02a270 comp rx=0 tx=0).stop 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1368397878 shutdown_connections 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 --2- 172.21.15.60:0/1368397878 >> v2:172.21.15.100:3300/0 conn(0x7fa0a8161710 0x7fa0a80d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 --2- 172.21.15.60:0/1368397878 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a80ced80 0x7fa0a80d22e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.038 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1368397878 >> 172.21.15.60:0/1368397878 conn(0x7fa0a80d0420 msgr2=0x7fa0a80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1368397878 shutdown_connections 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.040+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1368397878 wait complete. 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 Processor -- start 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 -- start start 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a80ced80 0x7fa0a815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a815dc20 0x7fa0a81e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a8161710 0x7fa0a80c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80b5e50 con 0x7fa0a80ced80 2024-04-25T08:40:04.039 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8162500 con 0x7fa0a815dc20 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.042+0000 7fa0ae11d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80b5fd0 con 0x7fa0a8161710 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a80ced80 0x7fa0a815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a80ced80 0x7fa0a815c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49254/0 (socket says 172.21.15.60:49254) 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 -- 172.21.15.60:0/1592540476 learned_addr learned my addr 172.21.15.60:0/1592540476 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a6ffd640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a815dc20 0x7fa0a81e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a7fff640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a8161710 0x7fa0a80c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 -- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a8161710 msgr2=0x7fa0a80c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a8161710 0x7fa0a80c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.040 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 -- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a815dc20 msgr2=0x7fa0a81e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0a815dc20 0x7fa0a81e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 -- 172.21.15.60:0/1592540476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a81e87c0 con 0x7fa0a80ced80 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a7fff640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fa0a8161710 0x7fa0a80c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0a77fe640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a80ced80 0x7fa0a815c870 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c3c0 tx=0x7fa09800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980074e0 con 0x7fa0a80ced80 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098007680 con 0x7fa0a80ced80 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098004a00 con 0x7fa0a80ced80 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a81e8aa0 con 0x7fa0a80ced80 2024-04-25T08:40:04.041 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.043+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a81e9060 con 0x7fa0a80ced80 2024-04-25T08:40:04.042 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.044+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0a8073a00 con 0x7fa0a80ced80 2024-04-25T08:40:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.045+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa09801a020 con 0x7fa0a80ced80 2024-04-25T08:40:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.045+0000 7fa08ffff640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa09406b4d0 0x7fa09406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.045+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa098089930 con 0x7fa0a80ced80 2024-04-25T08:40:04.043 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa08ffff640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7fa094071970 0x7fa094073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 --> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa094074480 con 0x7fa094071970 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa09808e2a0 con 0x7fa0a80ced80 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa0a6ffd640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa09406b4d0 0x7fa09406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa0a7fff640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7fa094071970 0x7fa094073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa0a6ffd640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa09406b4d0 0x7fa09406d990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa09c034000 tx=0x7fa09c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.046+0000 7fa0a7fff640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7fa094071970 0x7fa094073db0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.044 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.047+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== osd.4 v2:172.21.15.100:6808/2796877898 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa094074480 con 0x7fa094071970 2024-04-25T08:40:04.053 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.060+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 --> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0a80d16b0 con 0x7fa094071970 2024-04-25T08:40:04.054 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa08ffff640 1 -- 172.21.15.60:0/1592540476 <== osd.4 v2:172.21.15.100:6808/2796877898 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa0a80d16b0 con 0x7fa094071970 2024-04-25T08:40:04.054 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:04.054 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:04.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7fa094071970 msgr2=0x7fa094073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6808/2796877898,v1:172.21.15.100:6809/2796877898] conn(0x7fa094071970 0x7fa094073db0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.055 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa09406b4d0 msgr2=0x7fa09406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fa09406b4d0 0x7fa09406d990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa09c034000 tx=0x7fa09c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a80ced80 msgr2=0x7fa0a815c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 --2- 172.21.15.60:0/1592540476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0a80ced80 0x7fa0a815c870 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c3c0 tx=0x7fa09800c880 comp rx=0 tx=0).stop 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0a7fff640 1 -- 172.21.15.60:0/1592540476 reap_dead start 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 shutdown_connections 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 >> 172.21.15.60:0/1592540476 conn(0x7fa0a80d0420 msgr2=0x7fa0a80d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 shutdown_connections 2024-04-25T08:40:04.056 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.061+0000 7fa0ae11d640 1 -- 172.21.15.60:0/1592540476 wait complete. 2024-04-25T08:40:04.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T08:40:04.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.173+0000 7f018d21f640 1 Processor -- start 2024-04-25T08:40:04.166 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.173+0000 7f018d21f640 1 -- start start 2024-04-25T08:40:04.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018d21f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f01880cc4f0 0x7f01880cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018d21f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01880cdb20 0x7f01880cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018d21f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f018815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018d21f640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0188071270 con 0x7f01880cdb20 2024-04-25T08:40:04.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018d21f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0188072280 con 0x7f01880cc4f0 2024-04-25T08:40:04.169 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018d21f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01880714c0 con 0x7f018815b270 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01880cdb20 0x7f01880cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f01880cdb20 0x7f01880cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49270/0 (socket says 172.21.15.60:49270) 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 -- 172.21.15.60:0/2646185942 learned_addr learned my addr 172.21.15.60:0/2646185942 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f018759e640 1 --2- 172.21.15.60:0/2646185942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f018815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f0186d9d640 1 --2- 172.21.15.60:0/2646185942 >> v2:172.21.15.100:3300/0 conn(0x7f01880cc4f0 0x7f01880cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 -- 172.21.15.60:0/2646185942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 msgr2=0x7f018815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:04.170 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 --2- 172.21.15.60:0/2646185942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f018815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 -- 172.21.15.60:0/2646185942 >> v2:172.21.15.100:3300/0 conn(0x7f01880cc4f0 msgr2=0x7f01880cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 --2- 172.21.15.60:0/2646185942 >> v2:172.21.15.100:3300/0 conn(0x7f01880cc4f0 0x7f01880cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f017ffff640 1 -- 172.21.15.60:0/2646185942 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01880717a0 con 0x7f01880cdb20 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.174+0000 7f0186d9d640 1 --2- 172.21.15.60:0/2646185942 >> v2:172.21.15.100:3300/0 conn(0x7f01880cc4f0 0x7f01880cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.175+0000 7f017ffff640 1 --2- 172.21.15.60:0/2646185942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01880cdfb0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f01880cd290 tx=0x7f017002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b900faadfa924b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.175+0000 7f018659c640 1 -- 172.21.15.60:0/2646185942 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170037070 con 0x7f01880cdb20 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.175+0000 7f018659c640 1 -- 172.21.15.60:0/2646185942 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0170032440 con 0x7f01880cdb20 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.175+0000 7f018659c640 1 -- 172.21.15.60:0/2646185942 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170032810 con 0x7f01880cdb20 2024-04-25T08:40:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 -- 172.21.15.60:0/2646185942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 msgr2=0x7f01880cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 --2- 172.21.15.60:0/2646185942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01880cdfb0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f01880cd290 tx=0x7f017002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 -- 172.21.15.60:0/2646185942 shutdown_connections 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 --2- 172.21.15.60:0/2646185942 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f018815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 --2- 172.21.15.60:0/2646185942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01880cdfb0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 --2- 172.21.15.60:0/2646185942 >> v2:172.21.15.100:3300/0 conn(0x7f01880cc4f0 0x7f01880cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 -- 172.21.15.60:0/2646185942 >> 172.21.15.60:0/2646185942 conn(0x7f018806abe0 msgr2=0x7f018806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 -- 172.21.15.60:0/2646185942 shutdown_connections 2024-04-25T08:40:04.172 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.176+0000 7f018d21f640 1 -- 172.21.15.60:0/2646185942 wait complete. 2024-04-25T08:40:04.175 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.177+0000 7f018d21f640 1 Processor -- start 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.177+0000 7f018d21f640 1 -- start start 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018d21f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01880cc4f0 0x7f0188147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018d21f640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01881f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018d21f640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f0188154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018d21f640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018815bc00 con 0x7f01880cdb20 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018d21f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0188071d60 con 0x7f01880cc4f0 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018d21f640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018815bd80 con 0x7f018815b270 2024-04-25T08:40:04.176 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01881f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01881f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49286/0 (socket says 172.21.15.60:49286) 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 -- 172.21.15.60:0/2155301036 learned_addr learned my addr 172.21.15.60:0/2155301036 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f0186d9d640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01880cc4f0 0x7f0188147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018759e640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f0188154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 -- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 msgr2=0x7f0188154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f0188154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 -- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01880cc4f0 msgr2=0x7f0188147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01880cc4f0 0x7f0188147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.177 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 -- 172.21.15.60:0/2155301036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01881ede40 con 0x7f01880cdb20 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f0186d9d640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01880cc4f0 0x7f0188147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f018759e640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f0188154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017ffff640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01881f6fe0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f017002a220 tx=0x7f01700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170041020 con 0x7f01880cdb20 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01700306c0 con 0x7f01880cdb20 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.178+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170037040 con 0x7f01880cdb20 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.179+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01881f39a0 con 0x7f01880cdb20 2024-04-25T08:40:04.178 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.179+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0188152640 con 0x7f01880cdb20 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.179+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0188073a00 con 0x7f01880cdb20 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.180+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0170044020 con 0x7f01880cdb20 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.180+0000 7f017f7fe640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f016806b4d0 0x7f016806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.181+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f01700aa830 con 0x7f01880cdb20 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.181+0000 7f0186d9d640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f016806b4d0 0x7f016806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.181+0000 7f0186d9d640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f016806b4d0 0x7f016806d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f01880cbbd0 tx=0x7f0178008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.179 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.185+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f017004b030 con 0x7f01880cdb20 2024-04-25T08:40:04.360 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.367+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f018806d3a0 con 0x7f016806b4d0 2024-04-25T08:40:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.369+0000 7f017f7fe640 1 -- 172.21.15.60:0/2155301036 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f018806d3a0 con 0x7f016806b4d0 2024-04-25T08:40:04.361 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T08:40:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f016806b4d0 msgr2=0x7f016806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f016806b4d0 0x7f016806d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f01880cbbd0 tx=0x7f0178008040 comp rx=0 tx=0).stop 2024-04-25T08:40:04.363 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 msgr2=0x7f01881f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01881f6fe0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f017002a220 tx=0x7f01700046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 shutdown_connections 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f018815b270 0x7f0188154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f01880cdb20 0x7f01881f6fe0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f016806b4d0 0x7f016806d990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 --2- 172.21.15.60:0/2155301036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01880cc4f0 0x7f0188147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 >> 172.21.15.60:0/2155301036 conn(0x7f018806abe0 msgr2=0x7f018806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 shutdown_connections 2024-04-25T08:40:04.364 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.370+0000 7f018d21f640 1 -- 172.21.15.60:0/2155301036 wait complete. 2024-04-25T08:40:04.374 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T08:40:04.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:04.479 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.486+0000 7f819d45d640 1 Processor -- start 2024-04-25T08:40:04.480 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.486+0000 7f819d45d640 1 -- start start 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f819d45d640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8198161240 0x7f81980b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f819d45d640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980cf1d0 con 0x7f8198161890 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f819d45d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8198161620 con 0x7f81980b54a0 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f819d45d640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980b7120 con 0x7f8198161240 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f8196ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8198161240 0x7f81980b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f8196ffd640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f8198161240 0x7f81980b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49288/0 (socket says 172.21.15.60:49288) 2024-04-25T08:40:04.481 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f8196ffd640 1 -- 172.21.15.60:0/3838736450 learned_addr learned my addr 172.21.15.60:0/3838736450 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.487+0000 7f81967fc640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f8198161890 0x7f81980ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43748/0 (socket says 172.21.15.60:43748) 2024-04-25T08:40:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.488+0000 7f8196ffd640 1 -- 172.21.15.60:0/3838736450 >> v1:172.21.15.100:6789/0 conn(0x7f81980b54a0 legacy=0x7f81980d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.488+0000 7f8196ffd640 1 -- 172.21.15.60:0/3838736450 >> v1:172.21.15.60:6789/0 conn(0x7f8198161890 legacy=0x7f81980ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.488+0000 7f8195ffb640 1 -- 172.21.15.60:0/3838736450 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586409950 0 0) 0x7f81980cf1d0 con 0x7f8198161890 2024-04-25T08:40:04.482 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.488+0000 7f8196ffd640 1 -- 172.21.15.60:0/3838736450 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198161e00 con 0x7f8198161240 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.488+0000 7f8196ffd640 1 --2- 172.21.15.60:0/3838736450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980b6be0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f8188009870 tx=0x7f818802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69969d52980c65e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f8195ffb640 1 -- 172.21.15.60:0/3838736450 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188037070 con 0x7f8198161240 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f8195ffb640 1 -- 172.21.15.60:0/3838736450 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188032440 con 0x7f8198161240 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f8195ffb640 1 -- 172.21.15.60:0/3838736450 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188032810 con 0x7f8198161240 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 -- 172.21.15.60:0/3838736450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 msgr2=0x7f81980b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 --2- 172.21.15.60:0/3838736450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980b6be0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f8188009870 tx=0x7f818802a2a0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 -- 172.21.15.60:0/3838736450 shutdown_connections 2024-04-25T08:40:04.484 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 --2- 172.21.15.60:0/3838736450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980b6be0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 -- 172.21.15.60:0/3838736450 >> 172.21.15.60:0/3838736450 conn(0x7f81980d04a0 msgr2=0x7f81980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 -- 172.21.15.60:0/3838736450 shutdown_connections 2024-04-25T08:40:04.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.489+0000 7f819d45d640 1 -- 172.21.15.60:0/3838736450 wait complete. 2024-04-25T08:40:04.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.490+0000 7f819d45d640 1 Processor -- start 2024-04-25T08:40:04.485 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.490+0000 7f819d45d640 1 -- start start 2024-04-25T08:40:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f819d45d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980b54a0 0x7f81980ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f819d45d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f819d45d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8198161890 0x7f819815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f819d45d640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981626a0 con 0x7f8198161240 2024-04-25T08:40:04.488 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f819d45d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981623c0 con 0x7f81980b54a0 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f819d45d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980d15f0 con 0x7f8198161890 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f81967fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f81967fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49302/0 (socket says 172.21.15.60:49302) 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f81967fc640 1 -- 172.21.15.60:0/121986082 learned_addr learned my addr 172.21.15.60:0/121986082 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f8196ffd640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980b54a0 0x7f81980ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.491+0000 7f81977fe640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8198161890 0x7f819815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f81967fc640 1 -- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8198161890 msgr2=0x7f819815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.489 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f81967fc640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8198161890 0x7f819815fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f81967fc640 1 -- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980b54a0 msgr2=0x7f81980ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f81967fc640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f81980b54a0 0x7f81980ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f81967fc640 1 -- 172.21.15.60:0/121986082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81980c7ca0 con 0x7f8198161240 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f81967fc640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980c7550 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f818c00c420 tx=0x7f818c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0074e0 con 0x7f8198161240 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c0047e0 con 0x7f8198161240 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c004c10 con 0x7f8198161240 2024-04-25T08:40:04.490 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81980c7f80 con 0x7f8198161240 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.492+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81980c8820 con 0x7f8198161240 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.493+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8198073a00 con 0x7f8198161240 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.493+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f818c011070 con 0x7f8198161240 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f817f7fe640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f818c089cb0 con 0x7f8198161240 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f817f7fe640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8184071970 0x7f8184073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 --> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f818c00ee50 con 0x7f8184071970 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f818c08c3b0 con 0x7f8198161240 2024-04-25T08:40:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f8196ffd640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f818406b4d0 0x7f818406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f81977fe640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8184071970 0x7f8184073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.494+0000 7f8196ffd640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f818406b4d0 0x7f818406d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8188034000 tx=0x7f818802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.495+0000 7f81977fe640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8184071970 0x7f8184073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.495+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== osd.5 v2:172.21.15.100:6800/301366941 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f818c00ee50 con 0x7f8184071970 2024-04-25T08:40:04.501 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.508+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 --> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f819806d240 con 0x7f8184071970 2024-04-25T08:40:04.502 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f817f7fe640 1 -- 172.21.15.60:0/121986082 <== osd.5 v2:172.21.15.100:6800/301366941 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f819806d240 con 0x7f8184071970 2024-04-25T08:40:04.502 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:04.503 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:04.503 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8184071970 msgr2=0x7f8184073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f819d45d640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6800/301366941,v1:172.21.15.100:6801/301366941] conn(0x7f8184071970 0x7f8184073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f818406b4d0 msgr2=0x7f818406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f819d45d640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f818406b4d0 0x7f818406d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8188034000 tx=0x7f818802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 msgr2=0x7f81980c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f819d45d640 1 --2- 172.21.15.60:0/121986082 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8198161240 0x7f81980c7550 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f818c00c420 tx=0x7f818c00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.509+0000 7f81977fe640 1 -- 172.21.15.60:0/121986082 reap_dead start 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.510+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 shutdown_connections 2024-04-25T08:40:04.504 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.510+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 >> 172.21.15.60:0/121986082 conn(0x7f81980d04a0 msgr2=0x7f819806c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.510+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 shutdown_connections 2024-04-25T08:40:04.505 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.510+0000 7f819d45d640 1 -- 172.21.15.60:0/121986082 wait complete. 2024-04-25T08:40:04.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T08:40:04.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.621+0000 7f6eb74d2640 1 Processor -- start 2024-04-25T08:40:04.614 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.621+0000 7f6eb74d2640 1 -- start start 2024-04-25T08:40:04.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb74d2640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6eb0161710 0x7f6eb00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb74d2640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb015e000 con 0x7f6eb015dc20 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb74d2640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb00d1de0 con 0x7f6eb00ced80 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb74d2640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb0161c60 con 0x7f6eb0161710 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb526e640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f6eb015dc20 0x7f6eb00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43750/0 (socket says 172.21.15.60:43750) 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb5a6f640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f6eb0161710 0x7f6eb00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.622+0000 7f6eb526e640 1 -- 172.21.15.60:0/2327529161 learned_addr learned my addr 172.21.15.60:0/2327529161 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eb5a6f640 1 -- 172.21.15.60:0/2327529161 >> v1:172.21.15.100:6789/0 conn(0x7f6eb00ced80 legacy=0x7f6eb00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:04.616 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eb5a6f640 1 -- 172.21.15.60:0/2327529161 >> v1:172.21.15.60:6789/0 conn(0x7f6eb015dc20 legacy=0x7f6eb00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eb5a6f640 1 -- 172.21.15.60:0/2327529161 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb0161f40 con 0x7f6eb0161710 2024-04-25T08:40:04.617 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eaffff640 1 -- 172.21.15.60:0/2327529161 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421529690 0 0) 0x7f6eb015e000 con 0x7f6eb015dc20 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eb5a6f640 1 --2- 172.21.15.60:0/2327529161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb0161710 0x7f6eb00d29f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80097a0 tx=0x7f6ea802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bfeaf4a0b870d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eaffff640 1 -- 172.21.15.60:0/2327529161 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8037070 con 0x7f6eb0161710 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eaffff640 1 -- 172.21.15.60:0/2327529161 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea8032440 con 0x7f6eb0161710 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.623+0000 7f6eaffff640 1 -- 172.21.15.60:0/2327529161 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8032810 con 0x7f6eb0161710 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 -- 172.21.15.60:0/2327529161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb0161710 msgr2=0x7f6eb00d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/2327529161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb0161710 0x7f6eb00d29f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f6ea80097a0 tx=0x7f6ea802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.618 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 -- 172.21.15.60:0/2327529161 shutdown_connections 2024-04-25T08:40:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/2327529161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb0161710 0x7f6eb00d29f0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 -- 172.21.15.60:0/2327529161 >> 172.21.15.60:0/2327529161 conn(0x7f6eb00d0420 msgr2=0x7f6eb00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 -- 172.21.15.60:0/2327529161 shutdown_connections 2024-04-25T08:40:04.619 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.624+0000 7f6eb74d2640 1 -- 172.21.15.60:0/2327529161 wait complete. 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.625+0000 7f6eb74d2640 1 Processor -- start 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.625+0000 7f6eb74d2640 1 -- start start 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb74d2640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 0x7f6eb00c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb74d2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6eb015dc20 0x7f6eb015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb74d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eb0161710 0x7f6eb015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb74d2640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00b5e50 con 0x7f6eb00ced80 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb74d2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0162500 con 0x7f6eb0161710 2024-04-25T08:40:04.621 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb74d2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00b5fd0 con 0x7f6eb015dc20 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 0x7f6eb00c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 0x7f6eb00c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49312/0 (socket says 172.21.15.60:49312) 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 -- 172.21.15.60:0/224475882 learned_addr learned my addr 172.21.15.60:0/224475882 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb4a6d640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6eb015dc20 0x7f6eb015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb5a6f640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eb0161710 0x7f6eb015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 -- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6eb015dc20 msgr2=0x7f6eb015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6eb015dc20 0x7f6eb015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.622 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 -- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eb0161710 msgr2=0x7f6eb015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eb0161710 0x7f6eb015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.626+0000 7f6eb526e640 1 -- 172.21.15.60:0/224475882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb01f4690 con 0x7f6eb00ced80 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.627+0000 7f6eb526e640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 0x7f6eb00c9cb0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000c890 tx=0x7f6ea000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.627+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0018070 con 0x7f6eb00ced80 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.627+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea000a640 con 0x7f6eb00ced80 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.627+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb01f38d0 con 0x7f6eb00ced80 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.627+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000aa30 con 0x7f6eb00ced80 2024-04-25T08:40:04.623 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.627+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01f3e90 con 0x7f6eb00ced80 2024-04-25T08:40:04.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.628+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb01f3650 con 0x7f6eb00ced80 2024-04-25T08:40:04.624 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.628+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ea001b070 con 0x7f6eb00ced80 2024-04-25T08:40:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.629+0000 7f6eadffb640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.629+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6ea00895f0 con 0x7f6eb00ced80 2024-04-25T08:40:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.629+0000 7f6eb4a6d640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.625 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.630+0000 7f6eb4a6d640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00b5b50 tx=0x7f6ea4009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.633+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eb01f3650 con 0x7f6eb00ced80 2024-04-25T08:40:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.815+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb0073a00 con 0x7f6e9c06b4d0 2024-04-25T08:40:04.809 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.817+0000 7f6eadffb640 1 -- 172.21.15.60:0/224475882 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6eb0073a00 con 0x7f6e9c06b4d0 2024-04-25T08:40:04.810 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T08:40:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e9c06b4d0 msgr2=0x7f6e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.811 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00b5b50 tx=0x7f6ea4009060 comp rx=0 tx=0).stop 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 msgr2=0x7f6eb00c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 0x7f6eb00c9cb0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000c890 tx=0x7f6ea000cd50 comp rx=0 tx=0).stop 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 shutdown_connections 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f6e9c06b4d0 0x7f6e9c06d990 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6eb0161710 0x7f6eb015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6eb015dc20 0x7f6eb015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 --2- 172.21.15.60:0/224475882 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eb00ced80 0x7f6eb00c9cb0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.812 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 >> 172.21.15.60:0/224475882 conn(0x7f6eb00d0420 msgr2=0x7f6eb00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 shutdown_connections 2024-04-25T08:40:04.813 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.818+0000 7f6eb74d2640 1 -- 172.21.15.60:0/224475882 wait complete. 2024-04-25T08:40:04.822 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T08:40:04.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.934+0000 7f2215a15640 1 Processor -- start 2024-04-25T08:40:04.928 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.934+0000 7f2215a15640 1 -- start start 2024-04-25T08:40:04.930 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f2215a15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221015de80 0x7f22100cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f2215a15640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22100ccfe0 0x7f22100cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f2215a15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100afc80 con 0x7f22100cefb0 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f2215a15640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100ce4d0 con 0x7f22100ccfe0 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f2215a15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100ce7b0 con 0x7f221015de80 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f220f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22100ccfe0 0x7f22100cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f220f7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f22100ccfe0 0x7f22100cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49314/0 (socket says 172.21.15.60:49314) 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f220f7fe640 1 -- 172.21.15.60:0/135179755 learned_addr learned my addr 172.21.15.60:0/135179755 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.931 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.935+0000 7f220effd640 1 --2- 172.21.15.60:0/135179755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221015de80 0x7f22100cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.936+0000 7f220f7fe640 1 -- 172.21.15.60:0/135179755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221015de80 msgr2=0x7f22100cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.936+0000 7f220f7fe640 1 --2- 172.21.15.60:0/135179755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221015de80 0x7f22100cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.936+0000 7f220f7fe640 1 -- 172.21.15.60:0/135179755 >> v1:172.21.15.100:6789/0 conn(0x7f22100cefb0 legacy=0x7f22100cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.936+0000 7f220f7fe640 1 -- 172.21.15.60:0/135179755 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100cea90 con 0x7f22100ccfe0 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.936+0000 7f220f7fe640 1 --2- 172.21.15.60:0/135179755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22100ccfe0 0x7f22100cd470 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f220002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6293acd21049de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f220dffb640 1 -- 172.21.15.60:0/135179755 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200036070 con 0x7f22100ccfe0 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f220dffb640 1 -- 172.21.15.60:0/135179755 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200031600 con 0x7f22100ccfe0 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f220dffb640 1 -- 172.21.15.60:0/135179755 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000319d0 con 0x7f22100ccfe0 2024-04-25T08:40:04.932 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 -- 172.21.15.60:0/135179755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22100ccfe0 msgr2=0x7f22100cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 --2- 172.21.15.60:0/135179755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22100ccfe0 0x7f22100cd470 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f220002a710 comp rx=0 tx=0).stop 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 -- 172.21.15.60:0/135179755 shutdown_connections 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 --2- 172.21.15.60:0/135179755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f22100ccfe0 0x7f22100cd470 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 --2- 172.21.15.60:0/135179755 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f221015de80 0x7f22100cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 -- 172.21.15.60:0/135179755 >> 172.21.15.60:0/135179755 conn(0x7f22100b2390 msgr2=0x7f22100b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 -- 172.21.15.60:0/135179755 shutdown_connections 2024-04-25T08:40:04.933 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.937+0000 7f2215a15640 1 -- 172.21.15.60:0/135179755 wait complete. 2024-04-25T08:40:04.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.938+0000 7f2215a15640 1 Processor -- start 2024-04-25T08:40:04.936 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 -- start start 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22100ccfe0 0x7f221014e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22100cefb0 0x7f22100adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221015de80 0x7f2210155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100cfca0 con 0x7f221015de80 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100cfb20 con 0x7f22100cefb0 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f2215a15640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100b2eb0 con 0x7f22100ccfe0 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f220f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221015de80 0x7f2210155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.937 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f220f7fe640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221015de80 0x7f2210155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49316/0 (socket says 172.21.15.60:49316) 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f220f7fe640 1 -- 172.21.15.60:0/89729166 learned_addr learned my addr 172.21.15.60:0/89729166 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.939+0000 7f220effd640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22100ccfe0 0x7f221014e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220e7fc640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22100cefb0 0x7f22100adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220f7fe640 1 -- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22100ccfe0 msgr2=0x7f221014e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220f7fe640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f22100ccfe0 0x7f221014e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220f7fe640 1 -- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22100cefb0 msgr2=0x7f22100adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220f7fe640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22100cefb0 0x7f22100adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.938 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220f7fe640 1 -- 172.21.15.60:0/89729166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100ae8f0 con 0x7f221015de80 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220e7fc640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f22100cefb0 0x7f22100adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f220f7fe640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221015de80 0x7f2210155d90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f2200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200036050 con 0x7f221015de80 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200042070 con 0x7f221015de80 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200009510 con 0x7f221015de80 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22100aebd0 con 0x7f221015de80 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.940+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22100afef0 con 0x7f221015de80 2024-04-25T08:40:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.941+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22100b02d0 con 0x7f221015de80 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.941+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2200007bf0 con 0x7f221015de80 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f21f7fff640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21fc06b4d0 0x7f21fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f22000aada0 con 0x7f221015de80 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f21f7fff640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7f21fc071970 0x7f21fc073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 --> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22000020b0 con 0x7f21fc071970 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22000ad4a0 con 0x7f221015de80 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f220effd640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21fc06b4d0 0x7f21fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.940 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.942+0000 7f220e7fc640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7f21fc071970 0x7f21fc073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.943+0000 7f220effd640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21fc06b4d0 0x7f21fc06d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2208009aa0 tx=0x7f2208009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.943+0000 7f220e7fc640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7f21fc071970 0x7f21fc073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:04.941 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.943+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== osd.6 v2:172.21.15.100:6816/4212964990 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f22000020b0 con 0x7f21fc071970 2024-04-25T08:40:04.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.957+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 --> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2210073a00 con 0x7f21fc071970 2024-04-25T08:40:04.950 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.957+0000 7f21f7fff640 1 -- 172.21.15.60:0/89729166 <== osd.6 v2:172.21.15.100:6816/4212964990 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2210073a00 con 0x7f21fc071970 2024-04-25T08:40:04.951 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:04.951 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:04.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7f21fc071970 msgr2=0x7f21fc073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6816/4212964990,v1:172.21.15.100:6817/4212964990] conn(0x7f21fc071970 0x7f21fc073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:04.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21fc06b4d0 msgr2=0x7f21fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f21fc06b4d0 0x7f21fc06d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2208009aa0 tx=0x7f2208009190 comp rx=0 tx=0).stop 2024-04-25T08:40:04.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221015de80 msgr2=0x7f2210155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:04.952 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 --2- 172.21.15.60:0/89729166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f221015de80 0x7f2210155d90 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f2200009e50 tx=0x7f2200004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:04.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f220f7fe640 1 -- 172.21.15.60:0/89729166 reap_dead start 2024-04-25T08:40:04.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 shutdown_connections 2024-04-25T08:40:04.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 >> 172.21.15.60:0/89729166 conn(0x7f22100b2390 msgr2=0x7f221015c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:04.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 shutdown_connections 2024-04-25T08:40:04.953 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:04.958+0000 7f2215a15640 1 -- 172.21.15.60:0/89729166 wait complete. 2024-04-25T08:40:04.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T08:40:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 Processor -- start 2024-04-25T08:40:05.063 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 -- start start 2024-04-25T08:40:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13880b5620 0x7f13880b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f138806c2d0 0x7f138806c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138806cc80 con 0x7f138815b7d0 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138806cf50 con 0x7f13880b5620 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.070+0000 7f138e2fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880ce530 con 0x7f138806c2d0 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f13877fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f13880b5620 0x7f13880b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f1387fff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f138815b7d0 0x7f13880cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43758/0 (socket says 172.21.15.60:43758) 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f1387fff640 1 -- 172.21.15.60:0/109744993 learned_addr learned my addr 172.21.15.60:0/109744993 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f138c899640 1 --2- 172.21.15.60:0/109744993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f138806c2d0 0x7f138806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f13877fe640 1 -- 172.21.15.60:0/109744993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f138806c2d0 msgr2=0x7f138806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f13877fe640 1 --2- 172.21.15.60:0/109744993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f138806c2d0 0x7f138806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.065 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f13877fe640 1 -- 172.21.15.60:0/109744993 >> v1:172.21.15.60:6789/0 conn(0x7f138815b7d0 legacy=0x7f13880cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.066 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.071+0000 7f13877fe640 1 -- 172.21.15.60:0/109744993 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880ce810 con 0x7f13880b5620 2024-04-25T08:40:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f13877fe640 1 --2- 172.21.15.60:0/109744993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13880b5620 0x7f13880b5ab0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f13780097a0 tx=0x7f137802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef20617b3141e5f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f1386ffd640 1 -- 172.21.15.60:0/109744993 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378037070 con 0x7f13880b5620 2024-04-25T08:40:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f1386ffd640 1 -- 172.21.15.60:0/109744993 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378032440 con 0x7f13880b5620 2024-04-25T08:40:05.067 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f1386ffd640 1 -- 172.21.15.60:0/109744993 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378032810 con 0x7f13880b5620 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 -- 172.21.15.60:0/109744993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13880b5620 msgr2=0x7f13880b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 --2- 172.21.15.60:0/109744993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13880b5620 0x7f13880b5ab0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f13780097a0 tx=0x7f137802a180 comp rx=0 tx=0).stop 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 -- 172.21.15.60:0/109744993 shutdown_connections 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 --2- 172.21.15.60:0/109744993 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f138806c2d0 0x7f138806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 --2- 172.21.15.60:0/109744993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f13880b5620 0x7f13880b5ab0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 -- 172.21.15.60:0/109744993 >> 172.21.15.60:0/109744993 conn(0x7f13880d0800 msgr2=0x7f13880d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 -- 172.21.15.60:0/109744993 shutdown_connections 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.072+0000 7f138e2fc640 1 -- 172.21.15.60:0/109744993 wait complete. 2024-04-25T08:40:05.068 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.073+0000 7f138e2fc640 1 Processor -- start 2024-04-25T08:40:05.069 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 -- start start 2024-04-25T08:40:05.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 0x7f138814cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.071 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13880b5620 0x7f13880cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138815b7d0 0x7f13880cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138815bbb0 con 0x7f138806c2d0 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138806d230 con 0x7f138815b7d0 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f138e2fc640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138815bd30 con 0x7f13880b5620 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.074+0000 7f1387fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 0x7f138814cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 0x7f138814cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49332/0 (socket says 172.21.15.60:49332) 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 -- 172.21.15.60:0/2660739638 learned_addr learned my addr 172.21.15.60:0/2660739638 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f13877fe640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13880b5620 0x7f13880cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.072 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f138c899640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138815b7d0 0x7f13880cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 -- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13880b5620 msgr2=0x7f13880cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13880b5620 0x7f13880cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 -- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138815b7d0 msgr2=0x7f13880cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138815b7d0 0x7f13880cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 -- 172.21.15.60:0/2660739638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138814d6d0 con 0x7f138806c2d0 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f13877fe640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13880b5620 0x7f13880cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1387fff640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 0x7f138814cd60 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f137000c3c0 tx=0x7f137000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.073 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370017070 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1370005ce0 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138814d9b0 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700072f0 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.075+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138814df70 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.076+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1388073a00 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.077+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f137001a070 con 0x7f138806c2d0 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.077+0000 7f1384ff9640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f136806b4d0 0x7f136806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.074 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.077+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1370089400 con 0x7f138806c2d0 2024-04-25T08:40:05.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.077+0000 7f13877fe640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f136806b4d0 0x7f136806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.078+0000 7f13877fe640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f136806b4d0 0x7f136806d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1378034000 tx=0x7f137802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.075 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.081+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1370058480 con 0x7f138806c2d0 2024-04-25T08:40:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.260+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f138806b210 con 0x7f136806b4d0 2024-04-25T08:40:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.261+0000 7f1384ff9640 1 -- 172.21.15.60:0/2660739638 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f138806b210 con 0x7f136806b4d0 2024-04-25T08:40:05.254 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T08:40:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f136806b4d0 msgr2=0x7f136806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.256 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f136806b4d0 0x7f136806d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1378034000 tx=0x7f137802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 msgr2=0x7f138814cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 0x7f138814cd60 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f137000c3c0 tx=0x7f137000c880 comp rx=0 tx=0).stop 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 shutdown_connections 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f136806b4d0 0x7f136806d990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f138815b7d0 0x7f13880cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f13880b5620 0x7f13880cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 --2- 172.21.15.60:0/2660739638 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f138806c2d0 0x7f138814cd60 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 >> 172.21.15.60:0/2660739638 conn(0x7f13880d0800 msgr2=0x7f138815bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 shutdown_connections 2024-04-25T08:40:05.258 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.263+0000 7f138e2fc640 1 -- 172.21.15.60:0/2660739638 wait complete. 2024-04-25T08:40:05.267 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T08:40:05.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:40:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.379+0000 7f032a06b640 1 Processor -- start 2024-04-25T08:40:05.372 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.379+0000 7f032a06b640 1 -- start start 2024-04-25T08:40:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f032a06b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f032415dc20 0x7f03240ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f032a06b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032406cda0 con 0x7f0324161710 2024-04-25T08:40:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f032a06b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032415e000 con 0x7f03240d32f0 2024-04-25T08:40:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f032a06b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240d1de0 con 0x7f032415dc20 2024-04-25T08:40:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f0322ffd640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f0324161710 0x7f032406bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43762/0 (socket says 172.21.15.60:43762) 2024-04-25T08:40:05.374 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f0322ffd640 1 -- 172.21.15.60:0/167620683 learned_addr learned my addr 172.21.15.60:0/167620683 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03237fe640 1 --2- 172.21.15.60:0/167620683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f032415dc20 0x7f03240ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03227fc640 1 -- 172.21.15.60:0/167620683 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636109302 0 0) 0x7f032406cda0 con 0x7f0324161710 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03227fc640 1 -- 172.21.15.60:0/167620683 --> v1:172.21.15.60:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f030c003520 con 0x7f0324161710 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03237fe640 1 -- 172.21.15.60:0/167620683 >> v1:172.21.15.100:6789/0 conn(0x7f03240d32f0 legacy=0x7f032406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03237fe640 1 -- 172.21.15.60:0/167620683 >> v1:172.21.15.60:6789/0 conn(0x7f0324161710 legacy=0x7f032406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03227fc640 1 -- 172.21.15.60:0/167620683 <== mon.1 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024372752 0 0) 0x7f032415e000 con 0x7f03240d32f0 2024-04-25T08:40:05.375 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.380+0000 7f03237fe640 1 -- 172.21.15.60:0/167620683 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0324161c60 con 0x7f032415dc20 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.381+0000 7f03237fe640 1 --2- 172.21.15.60:0/167620683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f032415dc20 0x7f03240ce670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03140097d0 tx=0x7f031402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3df3be73a318d1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.381+0000 7f03227fc640 1 -- 172.21.15.60:0/167620683 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314037070 con 0x7f032415dc20 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.381+0000 7f03227fc640 1 -- 172.21.15.60:0/167620683 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314032440 con 0x7f032415dc20 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.381+0000 7f03227fc640 1 -- 172.21.15.60:0/167620683 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03140327f0 con 0x7f032415dc20 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 -- 172.21.15.60:0/167620683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f032415dc20 msgr2=0x7f03240ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 --2- 172.21.15.60:0/167620683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f032415dc20 0x7f03240ce670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03140097d0 tx=0x7f031402a210 comp rx=0 tx=0).stop 2024-04-25T08:40:05.377 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 -- 172.21.15.60:0/167620683 shutdown_connections 2024-04-25T08:40:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 --2- 172.21.15.60:0/167620683 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f032415dc20 0x7f03240ce670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 -- 172.21.15.60:0/167620683 >> 172.21.15.60:0/167620683 conn(0x7f03240d0420 msgr2=0x7f03240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 -- 172.21.15.60:0/167620683 shutdown_connections 2024-04-25T08:40:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.382+0000 7f032a06b640 1 -- 172.21.15.60:0/167620683 wait complete. 2024-04-25T08:40:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.383+0000 7f032a06b640 1 Processor -- start 2024-04-25T08:40:05.378 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.383+0000 7f032a06b640 1 -- start start 2024-04-25T08:40:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f032a06b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03240d32f0 0x7f03241f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f032a06b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f032415dc20 0x7f03241f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.381 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f032a06b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0324161710 0x7f0324157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f032a06b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0324162500 con 0x7f032415dc20 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f032a06b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0324162220 con 0x7f03240d32f0 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f032a06b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03240b5e50 con 0x7f0324161710 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f0322ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f032415dc20 0x7f03241f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f0322ffd640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f032415dc20 0x7f03241f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49338/0 (socket says 172.21.15.60:49338) 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f0322ffd640 1 -- 172.21.15.60:0/2924652275 learned_addr learned my addr 172.21.15.60:0/2924652275 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f03237fe640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03240d32f0 0x7f03241f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.384+0000 7f0323fff640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0324161710 0x7f0324157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.382 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f0322ffd640 1 -- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0324161710 msgr2=0x7f0324157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f0322ffd640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0324161710 0x7f0324157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f0322ffd640 1 -- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03240d32f0 msgr2=0x7f03241f0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f0322ffd640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03240d32f0 0x7f03241f0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f0322ffd640 1 -- 172.21.15.60:0/2924652275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03240cbe40 con 0x7f032415dc20 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f0322ffd640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f032415dc20 0x7f03241f3f80 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f031800c390 tx=0x7f031800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318017070 con 0x7f032415dc20 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318005ce0 con 0x7f032415dc20 2024-04-25T08:40:05.383 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03240cc120 con 0x7f032415dc20 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180072f0 con 0x7f032415dc20 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.385+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03240cc6e0 con 0x7f032415dc20 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.386+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0324073a00 con 0x7f032415dc20 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.386+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f031801a070 con 0x7f032415dc20 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f030bfff640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f03180893e0 con 0x7f032415dc20 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f03237fe640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f030bfff640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f0310071970 0x7f0310073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 --> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0318010e50 con 0x7f0310071970 2024-04-25T08:40:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0318090050 con 0x7f032415dc20 2024-04-25T08:40:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.387+0000 7f0323fff640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f0310071970 0x7f0310073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.388+0000 7f03237fe640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0314034000 tx=0x7f031402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.388+0000 7f0323fff640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f0310071970 0x7f0310073db0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.385 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.388+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== osd.7 v2:172.21.15.100:6824/1558157189 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0318010e50 con 0x7f0310071970 2024-04-25T08:40:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.402+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 --> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f03240d16b0 con 0x7f0310071970 2024-04-25T08:40:05.395 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.402+0000 7f030bfff640 1 -- 172.21.15.60:0/2924652275 <== osd.7 v2:172.21.15.100:6824/1558157189 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f03240d16b0 con 0x7f0310071970 2024-04-25T08:40:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-04-25T08:40:05.396 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:40:05.396 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-04-25T08:40:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.402+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f0310071970 msgr2=0x7f0310073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.402+0000 7f032a06b640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6824/1558157189,v1:172.21.15.100:6825/1558157189] conn(0x7f0310071970 0x7f0310073db0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031006b4d0 msgr2=0x7f031006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.397 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0314034000 tx=0x7f031402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f032415dc20 msgr2=0x7f03241f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 --2- 172.21.15.60:0/2924652275 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f032415dc20 0x7f03241f3f80 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f031800c390 tx=0x7f031800c850 comp rx=0 tx=0).stop 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f0323fff640 1 -- 172.21.15.60:0/2924652275 reap_dead start 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 shutdown_connections 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 >> 172.21.15.60:0/2924652275 conn(0x7f03240d0420 msgr2=0x7f03240b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 shutdown_connections 2024-04-25T08:40:05.398 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.403+0000 7f032a06b640 1 -- 172.21.15.60:0/2924652275 wait complete. 2024-04-25T08:40:05.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T08:40:05.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.515+0000 7fd2dc79b640 1 Processor -- start 2024-04-25T08:40:05.508 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.515+0000 7fd2dc79b640 1 -- start start 2024-04-25T08:40:05.509 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2dc79b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2d41446d0 0x7fd2d40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2dc79b640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d40cccc0 con 0x7fd2d406e8a0 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2dc79b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d40d0460 con 0x7fd2d4144d50 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2dc79b640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d40b2b20 con 0x7fd2d41446d0 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2da537640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7fd2d41446d0 0x7fd2d40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2dad38640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7fd2d406e8a0 0x7fd2d40cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43776/0 (socket says 172.21.15.60:43776) 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2dad38640 1 -- 172.21.15.60:0/3455917802 learned_addr learned my addr 172.21.15.60:0/3455917802 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:05.510 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2da537640 1 -- 172.21.15.60:0/3455917802 >> v1:172.21.15.100:6789/0 conn(0x7fd2d4144d50 legacy=0x7fd2d40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2da537640 1 -- 172.21.15.60:0/3455917802 >> v1:172.21.15.60:6789/0 conn(0x7fd2d406e8a0 legacy=0x7fd2d40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2da537640 1 -- 172.21.15.60:0/3455917802 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40cc140 con 0x7fd2d41446d0 2024-04-25T08:40:05.511 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.516+0000 7fd2d9535640 1 -- 172.21.15.60:0/3455917802 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314817232 0 0) 0x7fd2d40cccc0 con 0x7fd2d406e8a0 2024-04-25T08:40:05.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.517+0000 7fd2da537640 1 --2- 172.21.15.60:0/3455917802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d40b3f20 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00097a0 tx=0x7fd2d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d443b68e0204706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.517+0000 7fd2d9535640 1 -- 172.21.15.60:0/3455917802 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0037070 con 0x7fd2d41446d0 2024-04-25T08:40:05.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.517+0000 7fd2d9535640 1 -- 172.21.15.60:0/3455917802 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d0032440 con 0x7fd2d41446d0 2024-04-25T08:40:05.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.517+0000 7fd2d9535640 1 -- 172.21.15.60:0/3455917802 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0032810 con 0x7fd2d41446d0 2024-04-25T08:40:05.514 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.517+0000 7fd2dc79b640 1 -- 172.21.15.60:0/3455917802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 msgr2=0x7fd2d40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.517+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/3455917802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d40b3f20 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00097a0 tx=0x7fd2d002a230 comp rx=0 tx=0).stop 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.518+0000 7fd2dc79b640 1 -- 172.21.15.60:0/3455917802 shutdown_connections 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.518+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/3455917802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d40b3f20 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.518+0000 7fd2dc79b640 1 -- 172.21.15.60:0/3455917802 >> 172.21.15.60:0/3455917802 conn(0x7fd2d406abe0 msgr2=0x7fd2d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.518+0000 7fd2dc79b640 1 -- 172.21.15.60:0/3455917802 shutdown_connections 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.518+0000 7fd2dc79b640 1 -- 172.21.15.60:0/3455917802 wait complete. 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 Processor -- start 2024-04-25T08:40:05.515 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 -- start start 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d406e8a0 0x7fd2d414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d413d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2d4144d50 0x7fd2d413de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40cc700 con 0x7fd2d41446d0 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.519+0000 7fd2dc79b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d406d400 con 0x7fd2d406e8a0 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2dc79b640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d40cc9e0 con 0x7fd2d4144d50 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d413d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d413d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49344/0 (socket says 172.21.15.60:49344) 2024-04-25T08:40:05.516 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 -- 172.21.15.60:0/1511201046 learned_addr learned my addr 172.21.15.60:0/1511201046 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2dad38640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2d4144d50 0x7fd2d413de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 -- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2d4144d50 msgr2=0x7fd2d413de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2d4144d50 0x7fd2d413de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 -- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d406e8a0 msgr2=0x7fd2d414fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d406e8a0 0x7fd2d414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 -- 172.21.15.60:0/1511201046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d40ae8d0 con 0x7fd2d41446d0 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2d9d36640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d413d910 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c430 tx=0x7fd2c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4017070 con 0x7fd2d41446d0 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c40052d0 con 0x7fd2d41446d0 2024-04-25T08:40:05.517 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d40cd480 con 0x7fd2d41446d0 2024-04-25T08:40:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4003830 con 0x7fd2d41446d0 2024-04-25T08:40:05.518 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.520+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d40cda40 con 0x7fd2d41446d0 2024-04-25T08:40:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.521+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d40cdc40 con 0x7fd2d41446d0 2024-04-25T08:40:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.522+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2c401a070 con 0x7fd2d41446d0 2024-04-25T08:40:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.522+0000 7fd2caffd640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2b806b4d0 0x7fd2b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.522+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd2c4089450 con 0x7fd2d41446d0 2024-04-25T08:40:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.523+0000 7fd2da537640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2b806b4d0 0x7fd2b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.519 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.523+0000 7fd2da537640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2b806b4d0 0x7fd2b806d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0034000 tx=0x7fd2d002c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.520 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.527+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2d40cdc40 con 0x7fd2d41446d0 2024-04-25T08:40:05.700 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.707+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2d41399f0 con 0x7fd2b806b4d0 2024-04-25T08:40:05.701 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.708+0000 7fd2caffd640 1 -- 172.21.15.60:0/1511201046 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd2d0032670 con 0x7fd2b806b4d0 2024-04-25T08:40:05.701 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T08:40:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2b806b4d0 msgr2=0x7fd2b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2b806b4d0 0x7fd2b806d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0034000 tx=0x7fd2d002c000 comp rx=0 tx=0).stop 2024-04-25T08:40:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 msgr2=0x7fd2d413d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.703 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d413d910 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c430 tx=0x7fd2c400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 shutdown_connections 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd2d4144d50 0x7fd2d413de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd2d41446d0 0x7fd2d413d910 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7fd2b806b4d0 0x7fd2b806d990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 --2- 172.21.15.60:0/1511201046 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2d406e8a0 0x7fd2d414fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 >> 172.21.15.60:0/1511201046 conn(0x7fd2d406abe0 msgr2=0x7fd2d406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 shutdown_connections 2024-04-25T08:40:05.704 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.710+0000 7fd2dc79b640 1 -- 172.21.15.60:0/1511201046 wait complete. 2024-04-25T08:40:05.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:05.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.822+0000 7f5f3dff9640 1 Processor -- start 2024-04-25T08:40:05.815 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.822+0000 7f5f3dff9640 1 -- start start 2024-04-25T08:40:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.822+0000 7f5f3dff9640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f38164970 0x7f5f38166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.822+0000 7f5f3dff9640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f38144b30 con 0x7f5f38144750 2024-04-25T08:40:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.822+0000 7f5f3dff9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380712d0 con 0x7f5f380ce920 2024-04-25T08:40:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.822+0000 7f5f3dff9640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f380715b0 con 0x7f5f38164970 2024-04-25T08:40:05.816 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f37fff640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f5f38164970 0x7f5f38166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f377fe640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f5f38144750 0x7f5f3806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:43792/0 (socket says 172.21.15.60:43792) 2024-04-25T08:40:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f377fe640 1 -- 172.21.15.60:0/2679620262 learned_addr learned my addr 172.21.15.60:0/2679620262 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f37fff640 1 -- 172.21.15.60:0/2679620262 >> v1:172.21.15.100:6789/0 conn(0x7f5f380ce920 legacy=0x7f5f380cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f37fff640 1 -- 172.21.15.60:0/2679620262 >> v1:172.21.15.60:6789/0 conn(0x7f5f38144750 legacy=0x7f5f3806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f367fc640 1 -- 172.21.15.60:0/2679620262 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189928763 0 0) 0x7f5f38144b30 con 0x7f5f38144750 2024-04-25T08:40:05.817 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.823+0000 7f5f37fff640 1 -- 172.21.15.60:0/2679620262 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38071890 con 0x7f5f38164970 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f37fff640 1 --2- 172.21.15.60:0/2679620262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f38166d60 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009d90 tx=0x7f5f3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8f954478e48b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f367fc640 1 -- 172.21.15.60:0/2679620262 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30007740 con 0x7f5f38164970 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f367fc640 1 -- 172.21.15.60:0/2679620262 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f300078e0 con 0x7f5f38164970 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f367fc640 1 -- 172.21.15.60:0/2679620262 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30007cb0 con 0x7f5f38164970 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 -- 172.21.15.60:0/2679620262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 msgr2=0x7f5f38166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/2679620262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f38166d60 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009d90 tx=0x7f5f3002a660 comp rx=0 tx=0).stop 2024-04-25T08:40:05.819 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 -- 172.21.15.60:0/2679620262 shutdown_connections 2024-04-25T08:40:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/2679620262 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f38166d60 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 -- 172.21.15.60:0/2679620262 >> 172.21.15.60:0/2679620262 conn(0x7f5f3806abe0 msgr2=0x7f5f3806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 -- 172.21.15.60:0/2679620262 shutdown_connections 2024-04-25T08:40:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.824+0000 7f5f3dff9640 1 -- 172.21.15.60:0/2679620262 wait complete. 2024-04-25T08:40:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.826+0000 7f5f3dff9640 1 Processor -- start 2024-04-25T08:40:05.820 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.826+0000 7f5f3dff9640 1 -- start start 2024-04-25T08:40:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.826+0000 7f5f3dff9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f380ce920 0x7f5f3812ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f38144750 0x7f5f3815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f381437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38145450 con 0x7f5f38164970 2024-04-25T08:40:05.823 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3806f020 con 0x7f5f38144750 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381455d0 con 0x7f5f380ce920 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f381437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f381437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:49360/0 (socket says 172.21.15.60:49360) 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 -- 172.21.15.60:0/4069948425 learned_addr learned my addr 172.21.15.60:0/4069948425 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f377fe640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f380ce920 0x7f5f3812ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f36ffd640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f38144750 0x7f5f3815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 -- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f380ce920 msgr2=0x7f5f3812ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f380ce920 0x7f5f3812ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 -- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f38144750 msgr2=0x7f5f3815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f38144750 0x7f5f3815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 -- 172.21.15.60:0/4069948425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380b2aa0 con 0x7f5f38164970 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f36ffd640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f38144750 0x7f5f3815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f37fff640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f381437e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009d90 tx=0x7f5f30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30040070 con 0x7f5f38164970 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30031c90 con 0x7f5f38164970 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3003c640 con 0x7f5f38164970 2024-04-25T08:40:05.825 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f380b2d80 con 0x7f5f38164970 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.827+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f380b3340 con 0x7f5f38164970 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.828+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38073990 con 0x7f5f38164970 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.829+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f30039040 con 0x7f5f38164970 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.829+0000 7f5f2ffff640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f1806b4d0 0x7f5f1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.830+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5f300ab600 con 0x7f5f38164970 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.830+0000 7f5f377fe640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f1806b4d0 0x7f5f1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.826 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.830+0000 7f5f377fe640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f1806b4d0 0x7f5f1806d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f60 tx=0x7f5f28007b90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.827 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:05.834+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f3007a740 con 0x7f5f38164970 2024-04-25T08:40:06.010 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.017+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f380d0510 con 0x7f5f1806b4d0 2024-04-25T08:40:06.011 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.018+0000 7f5f2ffff640 1 -- 172.21.15.60:0/4069948425 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13457 (secure 0 0 0) 0x7f5f380d0510 con 0x7f5f1806b4d0 2024-04-25T08:40:06.011 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:06.012 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f1806b4d0 msgr2=0x7f5f1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f1806b4d0 0x7f5f1806d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f60 tx=0x7f5f28007b90 comp rx=0 tx=0).stop 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 msgr2=0x7f5f381437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f381437e0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009d90 tx=0x7f5f30004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 shutdown_connections 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5f38164970 0x7f5f381437e0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.014 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f5f1806b4d0 0x7f5f1806d990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f38144750 0x7f5f3815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 --2- 172.21.15.60:0/4069948425 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5f380ce920 0x7f5f3812ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 >> 172.21.15.60:0/4069948425 conn(0x7f5f3806abe0 msgr2=0x7f5f3806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:06.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.020+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 shutdown_connections 2024-04-25T08:40:06.015 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:06.021+0000 7f5f3dff9640 1 -- 172.21.15.60:0/4069948425 wait complete. 2024-04-25T08:40:06.024 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-04-25T08:40:05.546632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":434488,"kb_used_data":15688,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754540232,"statfs":{"total":773094113280,"available":772649197568,"internally_reserved":0,"allocated":16064512,"data_stored":8127184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1073741824,"num_objects":-1024,"num_object_clones":0,"num_object_copies":-2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1024,"num_write_kb":-1048576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.735422"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":408,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:39:58.865178+0000","last_change":"2024-04-25T08:29:12.725770+0000","last_active":"2024-04-25T08:39:58.865178+0000","last_peered":"2024-04-25T08:39:58.865178+0000","last_clean":"2024-04-25T08:39:58.865178+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:39:58.865178+0000","last_undegraded":"2024-04-25T08:39:58.865178+0000","last_fullsized":"2024-04-25T08:39:58.865178+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:29:11.513400+0000","last_clean_scrub_stamp":"2024-04-25T08:29:11.513400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:54.413796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51628,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320212,"statfs":{"total":96636764160,"available":96583897088,"internally_reserved":0,"allocated":2453504,"data_stored":1458674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542278,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55208,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46888,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324952,"statfs":{"total":96636764160,"available":96588750848,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55204,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64104,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307736,"statfs":{"total":96636764160,"available":96571121664,"internally_reserved":0,"allocated":2453504,"data_stored":1458674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:06.027 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T08:29:11.513400+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=29, tm_sec=11, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=40, tm_sec=1, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T08:40:06.027 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T08:40:26.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.150+0000 7f92f279e640 1 Processor -- start 2024-04-25T08:40:26.142 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.150+0000 7f92f279e640 1 -- start start 2024-04-25T08:40:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92f279e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f92ec0ce920 0x7f92ec0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92f279e640 1 -- --> v1:172.21.15.60:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ec144b30 con 0x7f92ec144750 2024-04-25T08:40:26.144 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92f279e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ec0712d0 con 0x7f92ec06d290 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92f279e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec0715b0 con 0x7f92ec0ce920 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92eb7fe640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f92ec0ce920 0x7f92ec0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92ebfff640 1 --1- >> v1:172.21.15.60:6789/0 conn(0x7f92ec144750 0x7f92ec06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.60:6789/0 says I am v1:172.21.15.60:48914/0 (socket says 172.21.15.60:48914) 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92ebfff640 1 -- 172.21.15.60:0/3454595667 learned_addr learned my addr 172.21.15.60:0/3454595667 (peer_addr_for_me v1:172.21.15.60:0/0) 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92eb7fe640 1 -- 172.21.15.60:0/3454595667 >> v1:172.21.15.100:6789/0 conn(0x7f92ec06d290 legacy=0x7f92ec06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92eb7fe640 1 -- 172.21.15.60:0/3454595667 >> v1:172.21.15.60:6789/0 conn(0x7f92ec144750 legacy=0x7f92ec06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92eb7fe640 1 -- 172.21.15.60:0/3454595667 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec071890 con 0x7f92ec0ce920 2024-04-25T08:40:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.151+0000 7f92eaffd640 1 -- 172.21.15.60:0/3454595667 <== mon.0 v1:172.21.15.60:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532423673 0 0) 0x7f92ec144b30 con 0x7f92ec144750 2024-04-25T08:40:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.152+0000 7f92eb7fe640 1 --2- 172.21.15.60:0/3454595667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec0ce920 0x7f92ec0cedb0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0097a0 tx=0x7f92dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fbdc215274d87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.152+0000 7f92eaffd640 1 -- 172.21.15.60:0/3454595667 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc037070 con 0x7f92ec0ce920 2024-04-25T08:40:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.152+0000 7f92eaffd640 1 -- 172.21.15.60:0/3454595667 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc032440 con 0x7f92ec0ce920 2024-04-25T08:40:26.146 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.152+0000 7f92eaffd640 1 -- 172.21.15.60:0/3454595667 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc032810 con 0x7f92ec0ce920 2024-04-25T08:40:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 -- 172.21.15.60:0/3454595667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec0ce920 msgr2=0x7f92ec0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 --2- 172.21.15.60:0/3454595667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec0ce920 0x7f92ec0cedb0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0097a0 tx=0x7f92dc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 -- 172.21.15.60:0/3454595667 shutdown_connections 2024-04-25T08:40:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 --2- 172.21.15.60:0/3454595667 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec0ce920 0x7f92ec0cedb0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 -- 172.21.15.60:0/3454595667 >> 172.21.15.60:0/3454595667 conn(0x7f92ec06abe0 msgr2=0x7f92ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.149 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 -- 172.21.15.60:0/3454595667 shutdown_connections 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.153+0000 7f92f279e640 1 -- 172.21.15.60:0/3454595667 wait complete. 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.154+0000 7f92f279e640 1 Processor -- start 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.154+0000 7f92f279e640 1 -- start start 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f279e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ec06d290 0x7f92ec1392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f279e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92ec0ce920 0x7f92ec15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f279e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 0x7f92ec15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f279e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06f020 con 0x7f92ec144750 2024-04-25T08:40:26.150 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f279e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06ece0 con 0x7f92ec06d290 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f279e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec145450 con 0x7f92ec0ce920 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f0d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 0x7f92ec15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f0d3b640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 0x7f92ec15c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36974/0 (socket says 172.21.15.60:36974) 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92f0d3b640 1 -- 172.21.15.60:0/3662409177 learned_addr learned my addr 172.21.15.60:0/3662409177 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92eb7fe640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92ec0ce920 0x7f92ec15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.155+0000 7f92ebfff640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ec06d290 0x7f92ec1392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f0d3b640 1 -- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92ec0ce920 msgr2=0x7f92ec15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.151 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f0d3b640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92ec0ce920 0x7f92ec15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f0d3b640 1 -- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ec06d290 msgr2=0x7f92ec1392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f0d3b640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ec06d290 0x7f92ec1392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f0d3b640 1 -- 172.21.15.60:0/3662409177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec15f120 con 0x7f92ec144750 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92ebfff640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ec06d290 0x7f92ec1392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f0d3b640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 0x7f92ec15c000 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f92e400b9a0 tx=0x7f92e400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4019070 con 0x7f92ec144750 2024-04-25T08:40:26.152 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e400cba0 con 0x7f92ec144750 2024-04-25T08:40:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec0b2be0 con 0x7f92ec144750 2024-04-25T08:40:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4014440 con 0x7f92ec144750 2024-04-25T08:40:26.153 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.156+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec0b31a0 con 0x7f92ec144750 2024-04-25T08:40:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.157+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec073990 con 0x7f92ec144750 2024-04-25T08:40:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.158+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92e401c070 con 0x7f92ec144750 2024-04-25T08:40:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.158+0000 7f92e8ff9640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92cc06b4d0 0x7f92cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.158+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f92e4089b90 con 0x7f92ec144750 2024-04-25T08:40:26.154 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.158+0000 7f92ebfff640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92cc06b4d0 0x7f92cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.159+0000 7f92ebfff640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92cc06b4d0 0x7f92cc06d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000f60 tx=0x7f92d4007c10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.155 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.163+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92e40585c0 con 0x7f92ec144750 2024-04-25T08:40:26.332 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.340+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 --> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92ec06ea80 con 0x7f92cc06b4d0 2024-04-25T08:40:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.342+0000 7f92e8ff9640 1 -- 172.21.15.60:0/3662409177 <== mgr.4111 v2:172.21.15.100:6832/416547562 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7f92ec06ea80 con 0x7f92cc06b4d0 2024-04-25T08:40:26.334 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:26.334 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-04-25T08:40:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92cc06b4d0 msgr2=0x7f92cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92cc06b4d0 0x7f92cc06d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f92d4000f60 tx=0x7f92d4007c10 comp rx=0 tx=0).stop 2024-04-25T08:40:26.336 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 msgr2=0x7f92ec15c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 0x7f92ec15c000 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f92e400b9a0 tx=0x7f92e400be60 comp rx=0 tx=0).stop 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 shutdown_connections 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f92ec144750 0x7f92ec15c000 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f92ec0ce920 0x7f92ec15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f92cc06b4d0 0x7f92cc06d990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 --2- 172.21.15.60:0/3662409177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92ec06d290 0x7f92ec1392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.337 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 >> 172.21.15.60:0/3662409177 conn(0x7f92ec06abe0 msgr2=0x7f92ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 shutdown_connections 2024-04-25T08:40:26.338 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.344+0000 7f92f279e640 1 -- 172.21.15.60:0/3662409177 wait complete. 2024-04-25T08:40:26.347 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-04-25T08:40:25.551360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":434476,"kb_used_data":15688,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754540244,"statfs":{"total":773094113280,"available":772649209856,"internally_reserved":0,"allocated":16064512,"data_stored":8127184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":428723760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002799"},"pg_stats":[{"pgid":"1.0","version":"148'119","reported_seq":419,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:40:06.377723+0000","last_change":"2024-04-25T08:40:06.377757+0000","last_active":"2024-04-25T08:40:06.377723+0000","last_peered":"2024-04-25T08:40:06.377723+0000","last_clean":"2024-04-25T08:40:06.377723+0000","last_became_active":"2024-04-25T08:29:12.724645+0000","last_became_peered":"2024-04-25T08:29:12.724645+0000","last_unstale":"2024-04-25T08:40:06.377723+0000","last_undegraded":"2024-04-25T08:40:06.377723+0000","last_fullsized":"2024-04-25T08:40:06.377723+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'119","last_scrub_stamp":"2024-04-25T08:40:06.377648+0000","last_deep_scrub":"148'119","last_deep_scrub_stamp":"2024-04-25T08:40:06.377648+0000","last_clean_scrub_stamp":"2024-04-25T08:40:06.377648+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:02.315278+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2453504,"data_stored":1458674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542282,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542282,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55208,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46888,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324952,"statfs":{"total":96636764160,"available":96588750848,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55204,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1859584,"data_stored":868306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64100,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307740,"statfs":{"total":96636764160,"available":96571125760,"internally_reserved":0,"allocated":2453504,"data_stored":1458674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:26.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T08:40:26.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.457+0000 7f3dbc79e640 1 Processor -- start 2024-04-25T08:40:26.449 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.457+0000 7f3dbc79e640 1 -- start start 2024-04-25T08:40:26.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dbc79e640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3db41446d0 0x7f3db40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dbc79e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db406e8a0 0x7f3db40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.451 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dbc79e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db40cccc0 con 0x7f3db4144d50 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dbc79e640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db40b2b20 con 0x7f3db41446d0 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dbc79e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db406d400 con 0x7f3db406e8a0 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3db41446d0 0x7f3db40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f3db41446d0 0x7f3db40b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36980/0 (socket says 172.21.15.60:36980) 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 -- 172.21.15.60:0/2844419123 learned_addr learned my addr 172.21.15.60:0/2844419123 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dbad3b640 1 --2- 172.21.15.60:0/2844419123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db406e8a0 0x7f3db40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.452 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 -- 172.21.15.60:0/2844419123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db406e8a0 msgr2=0x7f3db40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 --2- 172.21.15.60:0/2844419123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db406e8a0 0x7f3db40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 -- 172.21.15.60:0/2844419123 >> v1:172.21.15.100:6789/0 conn(0x7f3db4144d50 legacy=0x7f3db40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.458+0000 7f3dba53a640 1 -- 172.21.15.60:0/2844419123 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db40cc140 con 0x7f3db41446d0 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.459+0000 7f3dba53a640 1 --2- 172.21.15.60:0/2844419123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40b3f20 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009870 tx=0x7f3db002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8375233af0e6777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.459+0000 7f3db9538640 1 -- 172.21.15.60:0/2844419123 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0037070 con 0x7f3db41446d0 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.459+0000 7f3db9538640 1 -- 172.21.15.60:0/2844419123 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db0032440 con 0x7f3db41446d0 2024-04-25T08:40:26.453 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.459+0000 7f3db9538640 1 -- 172.21.15.60:0/2844419123 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0032810 con 0x7f3db41446d0 2024-04-25T08:40:26.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 -- 172.21.15.60:0/2844419123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 msgr2=0x7f3db40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/2844419123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40b3f20 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009870 tx=0x7f3db002a270 comp rx=0 tx=0).stop 2024-04-25T08:40:26.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 -- 172.21.15.60:0/2844419123 shutdown_connections 2024-04-25T08:40:26.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/2844419123 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db406e8a0 0x7f3db40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.457 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/2844419123 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40b3f20 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 -- 172.21.15.60:0/2844419123 >> 172.21.15.60:0/2844419123 conn(0x7f3db406abe0 msgr2=0x7f3db406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 -- 172.21.15.60:0/2844419123 shutdown_connections 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.460+0000 7f3dbc79e640 1 -- 172.21.15.60:0/2844419123 wait complete. 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.461+0000 7f3dbc79e640 1 Processor -- start 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.461+0000 7f3dbc79e640 1 -- start start 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbc79e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db406e8a0 0x7f3db413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbc79e640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbc79e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db4144d50 0x7f3db40ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbc79e640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cc700 con 0x7f3db41446d0 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbc79e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40b49a0 con 0x7f3db406e8a0 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbc79e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cc9e0 con 0x7f3db4144d50 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36992/0 (socket says 172.21.15.60:36992) 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 -- 172.21.15.60:0/3394034141 learned_addr learned my addr 172.21.15.60:0/3394034141 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dbad3b640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db4144d50 0x7f3db40ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3dba53a640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db406e8a0 0x7f3db413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.459 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 -- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db4144d50 msgr2=0x7f3db40ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db4144d50 0x7f3db40ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 -- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db406e8a0 msgr2=0x7f3db413b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db406e8a0 0x7f3db413b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 -- 172.21.15.60:0/3394034141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db413e410 con 0x7f3db41446d0 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.462+0000 7f3db9d39640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40aaf20 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f3da400c3c0 tx=0x7f3da400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.463+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40052d0 con 0x7f3db41446d0 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.463+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4005490 con 0x7f3db41446d0 2024-04-25T08:40:26.460 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.463+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db413e6f0 con 0x7f3db41446d0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.463+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4007260 con 0x7f3db41446d0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.463+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db40a8af0 con 0x7f3db41446d0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.464+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db40a8cf0 con 0x7f3db41446d0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.464+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3da4007400 con 0x7f3db41446d0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.464+0000 7f3daaffd640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d9806b4d0 0x7f3d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.465+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3da40894c0 con 0x7f3db41446d0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.465+0000 7f3dba53a640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d9806b4d0 0x7f3d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.461 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.465+0000 7f3dba53a640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d9806b4d0 0x7f3d9806d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3db0034000 tx=0x7f3db002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.462 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.469+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3db40a8cf0 con 0x7f3db41446d0 2024-04-25T08:40:26.641 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.649+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3db4073a00 con 0x7f3db41446d0 2024-04-25T08:40:26.644 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.652+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v28)=0 v28) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3da405ce20 con 0x7f3db41446d0 2024-04-25T08:40:26.645 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.653+0000 7f3daaffd640 1 -- 172.21.15.60:0/3394034141 <== mon.0 v2:172.21.15.60:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3da4057e60 con 0x7f3db41446d0 2024-04-25T08:40:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d9806b4d0 msgr2=0x7f3d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d9806b4d0 0x7f3d9806d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3db0034000 tx=0x7f3db002e000 comp rx=0 tx=0).stop 2024-04-25T08:40:26.647 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 msgr2=0x7f3db40aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40aaf20 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f3da400c3c0 tx=0x7f3da400c880 comp rx=0 tx=0).stop 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 shutdown_connections 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3db4144d50 0x7f3db40ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3db41446d0 0x7f3db40aaf20 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f3d9806b4d0 0x7f3d9806d990 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 --2- 172.21.15.60:0/3394034141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3db406e8a0 0x7f3db413b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 >> 172.21.15.60:0/3394034141 conn(0x7f3db406abe0 msgr2=0x7f3db41433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 shutdown_connections 2024-04-25T08:40:26.648 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.655+0000 7f3dbc79e640 1 -- 172.21.15.60:0/3394034141 wait complete. 2024-04-25T08:40:26.658 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:40:26.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:26.758 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.766+0000 7f7ce0c3a640 1 Processor -- start 2024-04-25T08:40:26.759 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.766+0000 7f7ce0c3a640 1 -- start start 2024-04-25T08:40:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7ce0c3a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7cdc15ba00 0x7f7cdc09a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.761 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7ce0c3a640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7cdc1602e0 0x7f7cdc0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7ce0c3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15f5f0 0x7f7cdc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7ce0c3a640 1 -- --> v2:172.21.15.60:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc0cd710 con 0x7f7cdc1602e0 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7ce0c3a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc099b70 con 0x7f7cdc15ba00 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7ce0c3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc099e50 con 0x7f7cdc15f5f0 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7cd9d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7cdc1602e0 0x7f7cdc0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7cd9d9b640 1 --2- >> v2:172.21.15.60:3300/0 conn(0x7f7cdc1602e0 0x7f7cdc0ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36994/0 (socket says 172.21.15.60:36994) 2024-04-25T08:40:26.762 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.767+0000 7f7cd9d9b640 1 -- 172.21.15.60:0/1775034226 learned_addr learned my addr 172.21.15.60:0/1775034226 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cda59c640 1 --2- 172.21.15.60:0/1775034226 >> v2:172.21.15.100:3300/0 conn(0x7f7cdc15ba00 0x7f7cdc09a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cdad9d640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15f5f0 0x7f7cdc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cd9d9b640 1 -- 172.21.15.60:0/1775034226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15f5f0 msgr2=0x7f7cdc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cd9d9b640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15f5f0 0x7f7cdc15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cd9d9b640 1 -- 172.21.15.60:0/1775034226 >> v2:172.21.15.100:3300/0 conn(0x7f7cdc15ba00 msgr2=0x7f7cdc09a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cd9d9b640 1 --2- 172.21.15.60:0/1775034226 >> v2:172.21.15.100:3300/0 conn(0x7f7cdc15ba00 0x7f7cdc09a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cd9d9b640 1 -- 172.21.15.60:0/1775034226 --> v2:172.21.15.60:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc09a130 con 0x7f7cdc1602e0 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cda59c640 1 --2- 172.21.15.60:0/1775034226 >> v2:172.21.15.100:3300/0 conn(0x7f7cdc15ba00 0x7f7cdc09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cdad9d640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15f5f0 0x7f7cdc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.768+0000 7f7cd9d9b640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc0ccda0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0b4370 tx=0x7f7cd002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294103b6b1b223de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7cd959a640 1 -- 172.21.15.60:0/1775034226 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0037070 con 0x7f7cdc1602e0 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7cd959a640 1 -- 172.21.15.60:0/1775034226 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7cd002ae00 con 0x7f7cdc1602e0 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7cd959a640 1 -- 172.21.15.60:0/1775034226 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0032680 con 0x7f7cdc1602e0 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/1775034226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 msgr2=0x7f7cdc0ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.764 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc0ccda0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0b4370 tx=0x7f7cd002a0b0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/1775034226 shutdown_connections 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15f5f0 0x7f7cdc15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/1775034226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc0ccda0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/1775034226 >> v2:172.21.15.100:3300/0 conn(0x7f7cdc15ba00 0x7f7cdc09a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/1775034226 >> 172.21.15.60:0/1775034226 conn(0x7f7cdc0b3270 msgr2=0x7f7cdc0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/1775034226 shutdown_connections 2024-04-25T08:40:26.765 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.769+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/1775034226 wait complete. 2024-04-25T08:40:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 Processor -- start 2024-04-25T08:40:26.768 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 -- start start 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15ba00 0x7f7cdc1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cdc15f5f0 0x7f7cdc158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc15da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc15bde0 con 0x7f7cdc1602e0 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0cd2e0 con 0x7f7cdc15f5f0 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7ce0c3a640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc0b4980 con 0x7f7cdc15ba00 2024-04-25T08:40:26.769 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7cdad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7cdad9d640 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc15da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:36996/0 (socket says 172.21.15.60:36996) 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.771+0000 7f7cdad9d640 1 -- 172.21.15.60:0/2098360958 learned_addr learned my addr 172.21.15.60:0/2098360958 (peer_addr_for_me v2:172.21.15.60:0/0) 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cd9d9b640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cdc15f5f0 0x7f7cdc158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cdad9d640 1 -- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15ba00 msgr2=0x7f7cdc1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cda59c640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15ba00 0x7f7cdc1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cdad9d640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15ba00 0x7f7cdc1591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cdad9d640 1 -- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cdc15f5f0 msgr2=0x7f7cdc158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.770 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cdad9d640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cdc15f5f0 0x7f7cdc158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cdad9d640 1 -- 172.21.15.60:0/2098360958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc15e270 con 0x7f7cdc1602e0 2024-04-25T08:40:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cd9d9b640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cdc15f5f0 0x7f7cdc158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7cdad9d640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc15da40 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4007df0 tx=0x7f7cd400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd401a070 con 0x7f7cdc1602e0 2024-04-25T08:40:26.771 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7cd4003990 con 0x7f7cdc1602e0 2024-04-25T08:40:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc15e550 con 0x7f7cdc1602e0 2024-04-25T08:40:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4003da0 con 0x7f7cdc1602e0 2024-04-25T08:40:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.772+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc0b4610 con 0x7f7cdc1602e0 2024-04-25T08:40:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.773+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc073990 con 0x7f7cdc1602e0 2024-04-25T08:40:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.774+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cd401d070 con 0x7f7cdc1602e0 2024-04-25T08:40:26.772 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.774+0000 7f7ccaffd640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cbc06b4d0 0x7f7cbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.774+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7cd408a420 con 0x7f7cdc1602e0 2024-04-25T08:40:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.774+0000 7f7cda59c640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cbc06b4d0 0x7f7cbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.775+0000 7f7cda59c640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cbc06b4d0 0x7f7cbc06d990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.774 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:26.778+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cd4010070 con 0x7f7cdc1602e0 2024-04-25T08:40:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.000+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cdc0b40a0 con 0x7f7cdc1602e0 2024-04-25T08:40:26.993 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.001+0000 7f7ccaffd640 1 -- 172.21.15.60:0/2098360958 <== mon.0 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3703 (secure 0 0 0) 0x7f7cd4059380 con 0x7f7cdc1602e0 2024-04-25T08:40:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-04-25T08:40:26.993 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7595,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/241856775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":241856775},{"type":"v1","addr":"172.21.15.100:6835","nonce":241856775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8713,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/1841757970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":1841757970},{"type":"v1","addr":"172.21.15.60:6835","nonce":1841757970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":8956,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3869699963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3869699963},{"type":"v1","addr":"172.21.15.100:6837","nonce":3869699963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":10275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/1436915809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":1436915809},{"type":"v1","addr":"172.21.15.60:6833","nonce":1436915809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[]} 2024-04-25T08:40:26.994 INFO:teuthology.orchestra.run.smithi060.stderr:dumped fsmap epoch 94 2024-04-25T08:40:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cbc06b4d0 msgr2=0x7f7cbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cbc06b4d0 0x7f7cbc06d990 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc008040 comp rx=0 tx=0).stop 2024-04-25T08:40:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 msgr2=0x7f7cdc15da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc15da40 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4007df0 tx=0x7f7cd400d4a0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.995 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 shutdown_connections 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7cdc1602e0 0x7f7cdc15da40 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:6832/416547562,v1:172.21.15.100:6833/416547562] conn(0x7f7cbc06b4d0 0x7f7cbc06d990 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7cdc15f5f0 0x7f7cdc158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 --2- 172.21.15.60:0/2098360958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7cdc15ba00 0x7f7cdc1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 >> 172.21.15.60:0/2098360958 conn(0x7f7cdc0b3270 msgr2=0x7f7cdc0cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 shutdown_connections 2024-04-25T08:40:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:2024-04-25T08:40:27.003+0000 7f7ce0c3a640 1 -- 172.21.15.60:0/2098360958 wait complete. 2024-04-25T08:40:27.006 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T08:40:27.006 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T08:40:27.007 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:40:27.007 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.016 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:40:27.016 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:40:27.016 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.065 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:40:27.065 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:40:27.065 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.077 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:40:27.077 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.128 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:40:27.129 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T08:40:27.129 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T08:40:27.129 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.166 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T08:40:27.167 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T08:40:27.167 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.242 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T08:40:27.242 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T08:40:27.242 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.324 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T08:40:27.325 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T08:40:27.325 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.364 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T08:40:27.364 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T08:40:27.364 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.411 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T08:40:27.411 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T08:40:27.411 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.451 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T08:40:27.451 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T08:40:27.451 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.522 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T08:40:27.523 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T08:40:27.523 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.606 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T08:40:27.607 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T08:40:27.607 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T08:40:27.607 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.632 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T08:40:27.632 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T08:40:27.632 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.661 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T08:40:27.661 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T08:40:27.661 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T08:40:27.661 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.683 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T08:40:27.684 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T08:40:27.684 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.713 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T08:40:27.713 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T08:40:27.714 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:27.766 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T08:40:27.766 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T08:40:27.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T08:40:27.811 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:40:27.811 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T08:40:28.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:28.037 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:40:28.037 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T08:40:28.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:28.168 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:40:28.168 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T08:40:28.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:40:28.234 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T08:40:28.353 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:40:28.353 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T08:40:28.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:40:28.528 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T08:40:28.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:40:28.626 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T08:40:28.716 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:40:28.716 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T08:40:28.765 INFO:tasks.ceph:Archiving mon data... 2024-04-25T08:40:28.766 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/data/mon.a.tgz 2024-04-25T08:40:28.766 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.11lsw2BskK 2024-04-25T08:40:28.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.11lsw2BskK 2024-04-25T08:40:28.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.11lsw2BskK 2024-04-25T08:40:29.038 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.11lsw2BskK is 454KB 2024-04-25T08:40:29.106 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.11lsw2BskK 2024-04-25T08:40:29.121 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/data/mon.b.tgz 2024-04-25T08:40:29.121 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-25T08:40:29.140 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.kYzYmDnaBr 2024-04-25T08:40:29.140 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.kYzYmDnaBr 2024-04-25T08:40:29.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.kYzYmDnaBr 2024-04-25T08:40:29.389 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.kYzYmDnaBr is 463KB 2024-04-25T08:40:29.486 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.kYzYmDnaBr 2024-04-25T08:40:29.500 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/data/mon.c.tgz 2024-04-25T08:40:29.500 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-04-25T08:40:29.554 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.pRg2aGEf9R 2024-04-25T08:40:29.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.pRg2aGEf9R 2024-04-25T08:40:29.704 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.pRg2aGEf9R 2024-04-25T08:40:29.784 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.pRg2aGEf9R is 463KB 2024-04-25T08:40:29.882 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.pRg2aGEf9R 2024-04-25T08:40:29.896 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T08:40:29.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T08:40:29.899 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T08:40:29.981 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T08:40:30.014 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T08:40:30.015 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T08:40:30.016 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/remote/smithi060/crash 2024-04-25T08:40:30.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T08:40:30.049 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/remote/smithi100/crash 2024-04-25T08:40:30.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T08:40:30.075 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T08:40:30.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:30.095 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:30.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:30.187 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:30.224 INFO:tasks.ceph:Compressing logs... 2024-04-25T08:40:30.224 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T08:40:30.229 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T08:40:30.253 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47332.log 2024-04-25T08:40:30.254 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80412.log 2024-04-25T08:40:30.313 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34478.log 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34478.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34558.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34478.log.gz 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T08:40:30.314 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34689.log 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.4.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40949.log 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34689.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34689.log.gz 2024-04-25T08:40:30.315 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.40949.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.1.40949.log.gz 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34769.log 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.6.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34609.log 2024-04-25T08:40:30.316 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34769.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34769.log.gz 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34609.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34609.log.gz 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40944.log 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34729.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41340.log 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.1.40944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40944.log.gz 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41336.log 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.3.41340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41340.log.gz 2024-04-25T08:40:30.317 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34518.log 2024-04-25T08:40:30.318 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.3.41336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41336.log.gz 2024-04-25T08:40:30.318 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T08:40:30.318 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.34518.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34518.log.gz 2024-04-25T08:40:30.318 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T08:40:30.454 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T08:40:30.637 INFO:teuthology.orchestra.run.smithi100.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94217.log 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80412.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47332.log.gz 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79095.log 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80412.log.gz 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65032.log 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79095.log: gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.99258.log 2024-04-25T08:40:30.714 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94217.log.gz 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65032.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79095.log.gz 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80456.log 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99258.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65032.log.gz 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47828.log 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99258.log.gz 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47626.log 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47828.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80456.log.gz 2024-04-25T08:40:30.715 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91059.log 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47828.log.gz 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45127.log 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91059.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47626.log.gz 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65387.log 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91059.log.gz 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78665.log 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65387.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45127.log.gz 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63529.log 2024-04-25T08:40:30.716 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78665.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.65387.log.gz --verbose 2024-04-25T08:40:30.717 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.43001.log 2024-04-25T08:40:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63529.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78665.log.gz 2024-04-25T08:40:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-04-25T08:40:30.717 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63529.log.gz 2024-04-25T08:40:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79053.log 2024-04-25T08:40:30.717 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43295.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58831.log 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79053.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62825.log 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79053.log.gz 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48280.log 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84495.log 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62825.log.gz 2024-04-25T08:40:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48280.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58831.log.gz 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46708.log 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48280.log.gz 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57609.log 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46708.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84495.log.gz 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80540.log 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46708.log.gz 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46036.log 2024-04-25T08:40:30.719 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57609.log.gz 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74653.log 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80540.log.gz 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89223.log 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74653.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46036.log.gz 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90718.log 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74653.log.gz 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57695.log 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90718.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89223.log.gz 2024-04-25T08:40:30.720 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100883.log 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41159.log 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90718.log.gz 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100883.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57695.log.gz 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86053.log 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100883.log.gz 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90931.log 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86053.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41159.log.gz 2024-04-25T08:40:30.721 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45631.log 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86053.log.gz 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62741.log 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90931.log.gz 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42300.log 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62741.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45631.log.gz 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85484.log 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42300.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62741.log.gz 2024-04-25T08:40:30.722 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65854.log 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42300.log.gz 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65555.log 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85484.log.gz 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96161.log 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65854.log.gz/var/log/ceph/ceph-client.admin.65555.log: 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65555.log.gz 2024-04-25T08:40:30.723 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47248.log 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45757.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96161.log.gz 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74315.log 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47248.log.gz 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61340.log 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74315.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-04-25T08:40:30.724 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58932.log 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74315.log.gz 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46786.log 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61340.log.gz 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98887.log 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58932.log.gz 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99772.log 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46786.log.gz 2024-04-25T08:40:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96825.log 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98887.log.gz 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96825.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99772.log.gz 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78528.log 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45994.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96825.log.gz 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56354.log 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-04-25T08:40:30.726 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56354.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78528.log.gz 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90000.log 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48631.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56354.log.gz 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42636.log 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48673.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.90000.log.gz 2024-04-25T08:40:30.727 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2024-04-25T08:40:30.728 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42636.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-04-25T08:40:30.728 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41657.log 2024-04-25T08:40:30.728 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42636.log.gz 2024-04-25T08:40:30.728 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93707.log 2024-04-25T08:40:30.728 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz 2024-04-25T08:40:30.728 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61821.log 2024-04-25T08:40:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41657.log.gz 2024-04-25T08:40:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95652.log 2024-04-25T08:40:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79137.log 2024-04-25T08:40:30.729 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.93707.log.gz 2024-04-25T08:40:30.729 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95652.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61821.log.gz 2024-04-25T08:40:30.730 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-04-25T08:40:30.730 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79137.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95652.log.gz 2024-04-25T08:40:30.730 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73538.log 2024-04-25T08:40:30.730 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41017.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79137.log.gz 2024-04-25T08:40:30.731 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2024-04-25T08:40:30.731 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73538.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-04-25T08:40:30.731 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94423.log 2024-04-25T08:40:30.731 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64382.log 2024-04-25T08:40:30.732 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.73538.log.gz 2024-04-25T08:40:30.732 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2024-04-25T08:40:30.732 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57863.log 2024-04-25T08:40:30.733 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64382.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94423.log.gzgzip 2024-04-25T08:40:30.733 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93923.log 2024-04-25T08:40:30.733 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83208.log 2024-04-25T08:40:30.733 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57863.log.gz 2024-04-25T08:40:30.734 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66548.log 2024-04-25T08:40:30.734 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64382.log.gz 2024-04-25T08:40:30.734 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56092.log 2024-04-25T08:40:30.734 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.93923.log.gz 2024-04-25T08:40:30.734 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83208.log.gz 2024-04-25T08:40:30.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96301.log 2024-04-25T08:40:30.735 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66548.log.gz 2024-04-25T08:40:30.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80020.log 2024-04-25T08:40:30.735 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96301.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56092.log.gz 2024-04-25T08:40:30.735 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83166.log 2024-04-25T08:40:30.736 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96301.log.gz 2024-04-25T08:40:30.736 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46870.log 2024-04-25T08:40:30.736 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80020.log.gz 2024-04-25T08:40:30.736 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63124.log 2024-04-25T08:40:30.736 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.95391.log 2024-04-25T08:40:30.737 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83166.log.gz 2024-04-25T08:40:30.737 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46870.log: /var/log/ceph/ceph-client.0.63124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58641.log 2024-04-25T08:40:30.737 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% 88.1% -- replaced with /var/log/ceph/ceph-client.0.63124.log.gz 2024-04-25T08:40:30.737 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46870.log.gz 2024-04-25T08:40:30.737 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47888.log 2024-04-25T08:40:30.738 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58641.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95391.log.gz 2024-04-25T08:40:30.738 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2024-04-25T08:40:30.738 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58641.log.gz 2024-04-25T08:40:30.738 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94657.log 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66464.log 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47888.log.gz 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94657.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz -- 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.80877.log 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94657.log.gz 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95339.log 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66464.log.gz 2024-04-25T08:40:30.739 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89402.log 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80877.log.gz 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56709.log 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95339.log.gz 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39650.log 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89402.log.gz 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94520.log 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56709.log.gz/var/log/ceph/ceph-client.admin.39650.log: 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85186.log 2024-04-25T08:40:30.740 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94520.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39650.log.gz 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74695.log 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94520.log.gz 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41961.log 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74695.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85186.log.gz 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61424.log 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74695.log.gz 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40092.log 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41961.log.gz 2024-04-25T08:40:30.741 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78402.log 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40092.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61424.log.gz 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77844.log 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40092.log.gz 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43085.log 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77844.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78402.log.gz 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62657.log 2024-04-25T08:40:30.742 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40807.log 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62657.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77844.log.gz 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43085.log.gz 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62394.log 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62657.log.gz 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100715.log 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62394.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40807.log.gz 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41489.log 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62394.log.gz 2024-04-25T08:40:30.743 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45910.log 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41489.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.100715.log.gz /var/log/ceph/ceph-client.admin.56919.log 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43379.log 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41489.log.gz 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56919.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45910.log.gz -5 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65260.log 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56919.log.gz 2024-04-25T08:40:30.744 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43379.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43379.log.gz 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59937.log 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65260.log.gz 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80793.log 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64560.log 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59937.log: /var/log/ceph/ceph-client.admin.80793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60669.log 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80793.log.gz 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89958.log 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.59937.log.gz 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64560.log.gz 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60429.log 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60669.log: /var/log/ceph/ceph-client.admin.89958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76148.log 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60669.log.gz 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89958.log.gz 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58246.log 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.76148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96429.log 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76148.log.gz 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.58246.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.60429.log.gz 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58246.log.gz 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46624.log 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78360.log 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96429.log.gz 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78360.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46624.log.gz 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83760.log 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78360.log.gz 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80582.log 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83760.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90168.log 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3%/var/log/ceph/ceph-client.admin.80582.log: -- replaced with /var/log/ceph/ceph-client.admin.83760.log.gz 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85611.log 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90168.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.80582.log.gz --verbose 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.58769.log 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90168.log.gz 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99132.log 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58769.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85611.log.gz 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55589.log 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58769.log.gz 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66066.log 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99132.log.gz 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85361.log 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85843.log 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66066.log.gz 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45673.log 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55589.log.gz 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85361.log: /var/log/ceph/ceph-client.admin.85843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85361.log.gz 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57386.log 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85843.log.gz 2024-04-25T08:40:30.750 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45673.log.gz 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99604.log 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57386.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99604.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57386.log.gz 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80328.log 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.2.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99604.log.gz 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41956.log 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47416.log 2024-04-25T08:40:30.751 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80328.log.gz 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74485.log 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41956.log.gz 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48109.log 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79397.log 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47416.log.gz 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66758.log 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74485.log.gz 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48109.log.gz 2024-04-25T08:40:30.752 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48324.log 2024-04-25T08:40:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81172.log 2024-04-25T08:40:30.753 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79397.log.gz 2024-04-25T08:40:30.753 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45211.log 2024-04-25T08:40:30.753 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48324.log.gz 2024-04-25T08:40:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.81172.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66758.log.gz 2024-04-25T08:40:30.754 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81172.log.gz 2024-04-25T08:40:30.754 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89163.log 2024-04-25T08:40:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64602.log 2024-04-25T08:40:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89163.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45211.log.gz 2024-04-25T08:40:30.755 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91671.log 2024-04-25T08:40:30.755 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89163.log.gz 2024-04-25T08:40:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47458.log 2024-04-25T08:40:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62699.log 2024-04-25T08:40:30.756 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64602.log.gz 2024-04-25T08:40:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47458.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91671.log.gz 2024-04-25T08:40:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67011.log 2024-04-25T08:40:30.757 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47458.log.gz 2024-04-25T08:40:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66170.log 2024-04-25T08:40:30.757 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.67011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99822.log 2024-04-25T08:40:30.758 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62699.log.gz 2024-04-25T08:40:30.758 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67011.log.gz 2024-04-25T08:40:30.758 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89528.log 2024-04-25T08:40:30.759 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66170.log.gz 2024-04-25T08:40:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57260.log 2024-04-25T08:40:30.759 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99822.log.gz 2024-04-25T08:40:30.759 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60910.log 2024-04-25T08:40:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57260.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89528.log.gz 2024-04-25T08:40:30.760 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95911.log 2024-04-25T08:40:30.760 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57260.log.gz 2024-04-25T08:40:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66022.log 2024-04-25T08:40:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95911.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.60910.log.gz 2024-04-25T08:40:30.761 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62573.log 2024-04-25T08:40:30.761 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66022.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95911.log.gz 2024-04-25T08:40:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61516.log 2024-04-25T08:40:30.762 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66022.log.gz 2024-04-25T08:40:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44706.log 2024-04-25T08:40:30.762 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58703.log 2024-04-25T08:40:30.763 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62573.log.gz 2024-04-25T08:40:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44706.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61516.log.gz 2024-04-25T08:40:30.763 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41285.log 2024-04-25T08:40:30.763 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44706.log.gz 2024-04-25T08:40:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78486.log 2024-04-25T08:40:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95484.log 2024-04-25T08:40:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78486.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41285.log.gz 2024-04-25T08:40:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57344.log 2024-04-25T08:40:30.765 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58703.log.gz 2024-04-25T08:40:30.765 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78486.log.gz 2024-04-25T08:40:30.765 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88947.log 2024-04-25T08:40:30.766 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95484.log.gz 2024-04-25T08:40:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88179.log 2024-04-25T08:40:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88947.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57344.log.gz 2024-04-25T08:40:30.766 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39564.log 2024-04-25T08:40:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78042.log 2024-04-25T08:40:30.767 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1%/var/log/ceph/ceph-client.admin.39564.log: -- replaced with /var/log/ceph/ceph-client.admin.88947.log.gz 2024-04-25T08:40:30.767 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91479.log 2024-04-25T08:40:30.767 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39564.log.gz 2024-04-25T08:40:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65429.log 2024-04-25T08:40:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78042.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.88179.log.gz 2024-04-25T08:40:30.768 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99394.log 2024-04-25T08:40:30.769 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91479.log.gz 2024-04-25T08:40:30.769 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65429.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78042.log.gz 2024-04-25T08:40:30.769 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88797.log 2024-04-25T08:40:30.769 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65429.log.gz 2024-04-25T08:40:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74527.log 2024-04-25T08:40:30.770 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99394.log.gz 2024-04-25T08:40:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76144.log 2024-04-25T08:40:30.770 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100631.log 2024-04-25T08:40:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.76144.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88797.log.gz 2024-04-25T08:40:30.771 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74527.log.gz 2024-04-25T08:40:30.771 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76144.log.gz 2024-04-25T08:40:30.771 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84663.log 2024-04-25T08:40:30.772 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65980.log 2024-04-25T08:40:30.772 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100631.log.gz 2024-04-25T08:40:30.772 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56480.log 2024-04-25T08:40:30.772 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62138.log 88.7% 2024-04-25T08:40:30.773 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84663.log.gz 2024-04-25T08:40:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56480.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65980.log.gz 2024-04-25T08:40:30.773 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T08:40:30.773 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56480.log.gz 2024-04-25T08:40:30.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96385.log 2024-04-25T08:40:30.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78927.log 2024-04-25T08:40:30.774 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62138.log.gz 2024-04-25T08:40:30.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62867.log 2024-04-25T08:40:30.774 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96385.log.gz 2024-04-25T08:40:30.774 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59275.log 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78927.log.gz 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46120.log 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63688.log 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59275.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.62867.log.gz 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59341.log 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63688.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46120.log.gz 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80370.log 2024-04-25T08:40:30.775 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.59275.log.gz 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59341.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63688.log.gz 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90336.log 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94007.log 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80370.log.gz 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91353.log 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90336.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.59341.log.gz 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T08:40:30.776 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79576.log 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90336.log.gz 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94007.log.gz 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44959.log 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79576.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91353.log.gz 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66716.log 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79576.log.gz 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89121.log 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44959.log.gz 2024-04-25T08:40:30.777 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95045.log 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66716.log.gz 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39736.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89121.log.gz 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28320.log 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99959.log 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.tmp-client.admin.28320.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.95045.log.gz -5 2024-04-25T08:40:30.778 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28320.log.gz 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99562.log 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99959.log.gz 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85526.log 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63410.log 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85526.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99562.log.gz 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88521.log 2024-04-25T08:40:30.779 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85526.log.gz 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80624.log 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63410.log.gz 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94091.log 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57947.log 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88521.log.gz 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80624.log.gz 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61298.log 2024-04-25T08:40:30.780 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94091.log.gz 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62096.log 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57947.log.gz 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42637.log 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61298.log.gz 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56312.log 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62096.log.gz 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43421.log 2024-04-25T08:40:30.781 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93839.log 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42637.log.gz 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56312.log.gz 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79179.log 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43421.log.gz 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61382.log 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85885.log 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93839.log.gz 2024-04-25T08:40:30.782 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79179.log.gz 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85569.log 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61382.log.gz 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89832.log 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85885.log.gz 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85569.log.gz 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56564.log 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89832.log.gz 2024-04-25T08:40:30.783 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96205.log 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96249.log 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56564.log.gz 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59515.log 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49009.log 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96205.log.gz 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96249.log: /var/log/ceph/ceph-client.admin.59515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47290.log 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96249.log.gz 2024-04-25T08:40:30.784 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74063.log 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47290.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59515.log.gz 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47500.log 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49009.log.gz -- replaced with /var/log/ceph/ceph-client.admin.47290.log.gz 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86245.log 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74063.log.gz 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73741.log 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47500.log.gz 2024-04-25T08:40:30.785 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75343.log 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73741.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.86245.log.gz 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73741.log.gz 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95526.log 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75343.log.gz 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59581.log 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64518.log 2024-04-25T08:40:30.786 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95526.log.gz 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96119.log 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75639.log 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64518.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59581.log.gz 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96119.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.64518.log.gz 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78707.log 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75639.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96119.log.gz 2024-04-25T08:40:30.787 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41615.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75639.log.gz 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46666.log 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45463.log 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78707.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48195.log 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78707.log.gz 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85102.log 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45463.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46666.log.gz 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48195.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45463.log.gz 2024-04-25T08:40:30.788 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57178.log 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90042.log 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85102.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48195.log.gz 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57178.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85102.log.gz 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43211.log 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57178.log.gz 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90042.log: /var/log/ceph/ceph-client.admin.43211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79492.log 2024-04-25T08:40:30.789 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90042.log.gz 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40627.log 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41964.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43211.log.gz 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90845.log 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79492.log.gz 87.9% 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61695.log 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95736.log 2024-04-25T08:40:30.790 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90845.log.gz 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85060.log 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61695.log.gz 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89570.log 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85144.log 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95736.log.gz 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85060.log: /var/log/ceph/ceph-client.admin.89570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41117.log 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85060.log.gz 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.40627.log.gz 2024-04-25T08:40:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61172.log 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89570.log.gz 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41117.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85144.log.gz 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47930.log 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41117.log.gz 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56008.log 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61172.log: /var/log/ceph/ceph-client.admin.47930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91185.log 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61172.log.gz 2024-04-25T08:40:30.792 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47930.log.gz 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80147.log 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56008.log.gz 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45337.log 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88353.log 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80147.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91185.log.gz 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45337.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80147.log.gz 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45169.log 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45337.log.gz 2024-04-25T08:40:30.793 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.88353.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42626.log 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95442.log 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88353.log.gz 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5%/var/log/ceph/ceph-client.admin.42626.log: -- replaced with /var/log/ceph/ceph-client.admin.45169.log.gz 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74611.log 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95442.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42626.log.gz 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82866.log 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95442.log.gz 2024-04-25T08:40:30.794 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2024-04-25T08:40:30.795 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74611.log: /var/log/ceph/ceph-client.admin.82866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-04-25T08:40:30.795 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74611.log.gz 2024-04-25T08:40:30.795 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79763.log 2024-04-25T08:40:30.795 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2024-04-25T08:40:30.795 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41243.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.82866.log.gz 2024-04-25T08:40:30.795 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99917.log 2024-04-25T08:40:30.796 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60952.log 2024-04-25T08:40:30.796 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79763.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-04-25T08:40:30.796 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.99917.log: -- /var/log/ceph/ceph-mds.c.log 2024-04-25T08:40:30.796 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79763.log.gz 2024-04-25T08:40:30.796 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75891.log 2024-04-25T08:40:30.797 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99917.log.gz 2024-04-25T08:40:30.797 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60952.log.gz 2024-04-25T08:40:30.797 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40134.log 2024-04-25T08:40:30.797 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88689.log 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75891.log: /var/log/ceph/ceph-client.admin.40134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61046.log 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88689.log: gzip -5 --verbose -- 91.4% -- replaced with /var/log/ceph/ceph-client.admin.75891.log.gz 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.79720.log 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61046.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88689.log.gz 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40134.log.gz 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55739.log 2024-04-25T08:40:30.798 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61046.log.gz 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48153.log 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90637.log 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55739.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79720.log.gz 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60194.log 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48153.log.gz 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45001.log 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55739.log.gz 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88437.log 2024-04-25T08:40:30.799 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60194.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90637.log.gz 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65896.log 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63284.log 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45001.log.gz 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60194.log.gz 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65692.log 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88437.log.gz 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98971.log 2024-04-25T08:40:30.800 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65896.log.gz 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63284.log: /var/log/ceph/ceph-client.admin.65692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63284.log.gz 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.98971.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65692.log.gz 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44459.log 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47374.log 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41363.log: /var/log/ceph/ceph-client.2.44459.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65597.log 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44459.log.gz 2024-04-25T08:40:30.801 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98971.log.gz 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75078.log 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41963.log 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47374.log.gz 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65597.log.gz 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96615.log 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59209.log 2024-04-25T08:40:30.802 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41963.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75078.log.gz 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93965.log 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41963.log.gz 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96615.log.gz 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46954.log 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91521.log 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93965.log: /var/log/ceph/ceph-client.admin.46954.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93965.log.gz 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59209.log.gz 2024-04-25T08:40:30.803 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46954.log.gz 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78833.log 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.1.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91521.log.gz 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75723.log 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80277.log 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75723.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78833.log.gz 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95610.log 2024-04-25T08:40:30.804 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75723.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79221.log 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80277.log: /var/log/ceph/ceph-client.admin.95610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48925.log 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%/var/log/ceph/ceph-client.admin.79221.log: -- replaced with /var/log/ceph/ceph-client.admin.95610.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80277.log.gz 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45589.log 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79221.log.gz 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48925.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48925.log.gz 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46240.log 2024-04-25T08:40:30.805 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96489.log 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46240.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45589.log.gz 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80062.log 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46240.log.gz 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60115.log 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96489.log: /var/log/ceph/ceph-client.admin.80062.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96489.log.gz 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80062.log.gz 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73256.log 2024-04-25T08:40:30.806 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79534.log 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73256.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60115.log.gz 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78234.log 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80104.log 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73256.log.gz 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79534.log.gz 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41962.log 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9%/var/log/ceph/ceph-client.admin.80104.log: -- replaced with /var/log/ceph/ceph-client.admin.78234.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56522.log 2024-04-25T08:40:30.807 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80104.log.gz 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56835.log 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41962.log: /var/log/ceph/ceph-client.admin.56522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73472.log 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41962.log.gz 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56835.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56522.log.gz 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56835.log.gz 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60868.log 2024-04-25T08:40:30.808 INFO:teuthology.orchestra.run.smithi060.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94741.log 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60868.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86137.log 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60868.log.gz 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73472.log.gz 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-osd.0.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94741.log.gz 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60603.log 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74147.log 2024-04-25T08:40:30.809 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4%/var/log/ceph/ceph-client.admin.60603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46282.log 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86137.log.gz 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60495.log 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60603.log.gz/var/log/ceph/ceph-client.admin.46282.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74147.log.gz 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46408.log 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46282.log.gz 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75681.log 2024-04-25T08:40:30.810 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73979.log 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46408.log: /var/log/ceph/ceph-client.admin.75681.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60495.log.gz 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46408.log.gz 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75681.log.gz 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83106.log 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91957.log 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83106.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73979.log.gz 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44025.log 2024-04-25T08:40:30.811 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83106.log.gz/var/log/ceph/ceph-client.admin.91957.log: 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91957.log.gz/var/log/ceph/ceph-client.0.44025.log: 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44025.log.gz 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74357.log 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100253.log 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74357.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74357.log.gz 2024-04-25T08:40:30.812 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100253.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100253.log.gz 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.45421.log 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61466.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45421.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.41573.log 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96741.log 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1%/var/log/ceph/ceph-client.admin.41573.log: -- replaced with /var/log/ceph/ceph-client.admin.61466.log.gz 2024-04-25T08:40:30.813 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41960.log 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96741.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96741.log.gz 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.88479.log 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100019.log 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41960.log.gz 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66824.log 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%/var/log/ceph/ceph-client.admin.100019.log: -- replaced with /var/log/ceph/ceph-client.admin.88479.log.gz 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44748.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100019.log.gz 2024-04-25T08:40:30.814 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66824.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66824.log.gz 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.57651.log 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84537.log 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44748.log.gz 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65158.log 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57651.log.gz 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84537.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84537.log.gz 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42293.log 2024-04-25T08:40:30.815 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65158.log.gz/var/log/ceph/ceph-client.admin.42293.log: 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89265.log 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62222.log 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85759.log 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89265.log.gz 2024-04-25T08:40:30.816 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41531.log 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62222.log.gz 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85759.log: gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85759.log.gz 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr: -- /var/log/ceph/ceph-client.admin.48016.log 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74399.log 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41531.log.gz 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48016.log: gzip -5 --verbose -- 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48016.log.gz 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.64644.log 2024-04-25T08:40:30.817 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34705.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74399.log.gz 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64644.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64644.log.gz 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77910.log 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60561.log 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34705.log.gz 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.77910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59677.log 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60020.log 2024-04-25T08:40:30.818 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77910.log.gz 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64780.log 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60561.log.gz 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59677.log: /var/log/ceph/ceph-client.admin.60020.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59677.log.gz 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60020.log.gz 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74273.log 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43711.log 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74273.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64780.log.gz 2024-04-25T08:40:30.819 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96783.log 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74273.log.gz 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64864.log 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43711.log: /var/log/ceph/ceph-client.admin.96783.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96783.log.gz 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42301.log 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47542.log 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42301.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64864.log.gz 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64822.log 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0%/var/log/ceph/ceph-client.admin.47542.log: -- replaced with /var/log/ceph/ceph-client.admin.42301.log.gz 2024-04-25T08:40:30.820 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.65345.log 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47542.log.gz 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41741.log 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65345.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64822.log.gz 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81176.log 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2%/var/log/ceph/ceph-client.admin.41741.log: -- replaced with /var/log/ceph/ceph-client.admin.65345.log.gz 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100421.log 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41741.log.gz 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99520.log 2024-04-25T08:40:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.81176.log: /var/log/ceph/ceph-client.admin.100421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85717.log 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81176.log.gz 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99520.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100421.log.gz 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90802.log 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99520.log.gz 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84107.log 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94919.log 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85717.log.gz 2024-04-25T08:40:30.822 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6%/var/log/ceph/ceph-client.admin.84107.log: -- replaced with /var/log/ceph/ceph-client.admin.90802.log.gz 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45379.log 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89664.log 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84107.log.gz 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45379.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62615.log 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94919.log.gz 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45379.log.gz 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96573.log 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89664.log: /var/log/ceph/ceph-client.admin.62615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74994.log 2024-04-25T08:40:30.823 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96573.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89664.log.gz 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62615.log.gz 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90887.log 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96573.log.gz 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74021.log 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45547.log 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74994.log.gz 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90887.log: /var/log/ceph/ceph-client.admin.74021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85442.log 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90887.log.gz 2024-04-25T08:40:30.824 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0%/var/log/ceph/ceph-client.admin.45547.log: -- replaced with /var/log/ceph/ceph-client.admin.74021.log.gz 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96699.log 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.45547.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.42296.log 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85442.log: /var/log/ceph/ceph-client.admin.96699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99478.log 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42296.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85442.log.gz 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96699.log.gz 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77976.log 2024-04-25T08:40:30.825 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42296.log.gz 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60241.log 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99478.log: /var/log/ceph/ceph-client.admin.77976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101009.log 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99478.log.gz 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78444.log 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1%/var/log/ceph/ceph-client.admin.101009.log: -- replaced with /var/log/ceph/ceph-client.admin.77976.log.gz 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60241.log.gz 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101009.log.gz 2024-04-25T08:40:30.826 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59027.log 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.59027.log: -- replaced with /var/log/ceph/ceph-client.admin.78444.log.gz 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55805.log 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59027.log.gz 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62487.log 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40050.log 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55805.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62487.log: gzip -5 --verbose -- 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55805.log.gz 2024-04-25T08:40:30.827 INFO:teuthology.orchestra.run.smithi060.stderr: /var/log/ceph/ceph-client.admin.79011.log 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40050.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62487.log.gz 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84012.log 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40050.log.gz 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98929.log 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79011.log: /var/log/ceph/ceph-client.admin.84012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79011.log.gz 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.84012.log.gz/var/log/ceph/ceph-client.admin.98929.log: 2024-04-25T08:40:30.828 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56186.log 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98929.log.gz/var/log/ceph/ceph-client.admin.48757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79859.log 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55924.log 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56186.log: /var/log/ceph/ceph-client.admin.79859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91227.log 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56186.log.gz 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55924.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79859.log.gz 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94363.log 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55924.log.gz 2024-04-25T08:40:30.829 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84892.log 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91227.log: /var/log/ceph/ceph-client.admin.94363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58575.log 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84892.log: 89.1% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94363.log.gz 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91227.log.gz 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84892.log.gz 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83970.log 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83544.log 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56228.log 2024-04-25T08:40:30.830 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58575.log.gz 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83970.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83886.log 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83544.log: /var/log/ceph/ceph-client.admin.56228.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83544.log.gz 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83886.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56228.log.gz 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99352.log 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95087.log 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83886.log.gz 2024-04-25T08:40:30.831 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.99352.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99352.log.gz 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56438.log 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95087.log.gz 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39694.log 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56438.log.gz 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94825.log 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39694.log.gz 2024-04-25T08:40:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88563.log 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94825.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94825.log.gz 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95568.log 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94279.log 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88563.log.gz 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79305.log 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94279.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95568.log.gz 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93641.log 2024-04-25T08:40:30.833 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94279.log.gz 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39924.log 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93641.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79305.log.gz 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48883.log 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.93641.log.gz 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40975.log 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39924.log.gz 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84934.log 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48883.log.gz 2024-04-25T08:40:30.834 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84275.log 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84934.log: 87.5% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40975.log.gz 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84934.log.gz 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48715.log 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45253.log 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84275.log.gz 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48715.log.gz 2024-04-25T08:40:30.835 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44030.log 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40891.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45253.log.gz 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99174.log 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.44030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44030.log.gz 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59473.log 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45295.log 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59473.log: /var/log/ceph/ceph-client.admin.99174.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99174.log.gz 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59473.log.gz 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73657.log 2024-04-25T08:40:30.836 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63686.log 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45295.log.gz/var/log/ceph/ceph-client.admin.73657.log: 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61256.log 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73657.log.gz/var/log/ceph/ceph-client.admin.63686.log: 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73825.log 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63686.log.gz 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75471.log 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5%/var/log/ceph/ceph-client.admin.73825.log: -- replaced with /var/log/ceph/ceph-client.admin.61256.log.gz 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84191.log 2024-04-25T08:40:30.837 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75471.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73825.log.gz 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74105.log 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75471.log.gz 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46498.log 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74105.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84191.log.gz 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74105.log.gz 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94049.log 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44917.log 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46498.log.gz 2024-04-25T08:40:30.838 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91101.log 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44917.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94049.log.gz 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66380.log 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44917.log.gz 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96975.log 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66380.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91101.log.gz 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66380.log.gz 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83844.log 2024-04-25T08:40:30.839 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39522.log 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96975.log.gz 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83844.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83844.log.gz 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75765.log 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79450.log 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39522.log.gz 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47748.log 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79450.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75765.log.gz 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79450.log.gz 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101051.log 2024-04-25T08:40:30.840 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63572.log 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101051.log: /var/log/ceph/ceph-client.admin.47748.log: 88.6% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47748.log.gz 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101051.log.gz 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80751.log 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91876.log 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3%/var/log/ceph/ceph-client.admin.80751.log: -- replaced with /var/log/ceph/ceph-client.admin.63572.log.gz 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95694.log 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80751.log.gz 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.91876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74189.log 2024-04-25T08:40:30.841 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.91876.log.gz 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101175.log 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95694.log.gz 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56396.log 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101175.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74189.log.gz 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58447.log 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101175.log.gz 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89444.log 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58447.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56396.log.gz 2024-04-25T08:40:30.842 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84579.log 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58447.log.gz 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84579.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89444.log.gz 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83292.log 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84579.log.gz 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95831.log 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83292.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83292.log.gz 2024-04-25T08:40:30.843 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86179.log 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73699.log 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95831.log.gz 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43337.log 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73699.log: 89.0% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.86179.log.gz 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73699.log.gz 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55523.log 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84705.log 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43337.log.gz 2024-04-25T08:40:30.844 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57821.log 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84705.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84705.log.gz 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55523.log.gz 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57821.log.gz 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96076.log 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75555.log 2024-04-25T08:40:30.845 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96076.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96076.log.gz 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95171.log 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56270.log 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75555.log.gz 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66590.log 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56270.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95171.log.gz 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63452.log 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56270.log.gz 2024-04-25T08:40:30.846 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65772.log 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66590.log.gz 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64990.log 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63452.log.gz 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74442.log 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65772.log.gz 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66212.log 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64990.log.gz/var/log/ceph/ceph-client.admin.74442.log: 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58055.log 2024-04-25T08:40:30.847 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66212.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74442.log.gz 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85281.log 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66212.log.gz 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83718.log 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85281.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58055.log.gz 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85281.log.gz 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83418.log 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74737.log 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83718.log.gz 2024-04-25T08:40:30.848 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59407.log 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3%/var/log/ceph/ceph-client.admin.74737.log: -- replaced with /var/log/ceph/ceph-client.admin.83418.log.gz 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74737.log.gz 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56751.log 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84149.log 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56751.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56751.log.gz 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59407.log.gz 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57472.log 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.84149.log: --verbose -- /var/log/ceph/ceph-client.admin.94562.log 2024-04-25T08:40:30.849 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84149.log.gz 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94562.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94562.log.gz 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57472.log.gz 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62350.log 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62180.log 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz/var/log/ceph/ceph-client.admin.62350.log: 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46324.log 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62350.log.gz 2024-04-25T08:40:30.850 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86436.log 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4%/var/log/ceph/ceph-client.admin.46324.log: -- replaced with /var/log/ceph/ceph-client.admin.62180.log.gz 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59153.log 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.86436.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46324.log.gz 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.86436.log.gz 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83586.log 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63885.log 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.59153.log.gz/var/log/ceph/ceph-client.admin.83586.log: 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75385.log 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63885.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63885.log.gz 2024-04-25T08:40:30.851 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83586.log.gz 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57428.log 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100841.log 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75385.log.gz 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100589.log 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57428.log.gz 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60784.log 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr: 87.2%/var/log/ceph/ceph-client.admin.100589.log: -- replaced with /var/log/ceph/ceph-client.admin.100841.log.gz 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90462.log 2024-04-25T08:40:30.852 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100589.log.gz/var/log/ceph/ceph-client.admin.60784.log: 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60784.log.gz 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63989.log 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46198.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90462.log.gz 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73388.log 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99646.log 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63989.log.gz 2024-04-25T08:40:30.853 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39395.log 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99646.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99646.log.gz 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.73388.log.gz 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83250.log 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39882.log 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39395.log.gz 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43547.log 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83250.log.gz 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96657.log 2024-04-25T08:40:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43547.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39882.log.gz 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47584.log 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96657.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43547.log.gz 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96657.log.gz 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39176.log 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83064.log 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47584.log.gz 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82998.log 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83064.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39176.log.gz 2024-04-25T08:40:30.855 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89079.log 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83064.log.gz 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.82998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94175.log 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89079.log.gz 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.82998.log.gz 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90760.log 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100799.log 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94175.log.gz 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91999.log 2024-04-25T08:40:30.856 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100799.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90760.log.gz 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41201.log 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100799.log.gz 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79939.log 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41201.log: 88.6% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91999.log.gz 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86432.log 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85673.log 2024-04-25T08:40:30.857 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79939.log.gz 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.86432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.86432.log.gz 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56961.log 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94699.log 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56961.log: /var/log/ceph/ceph-client.admin.85673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85673.log.gz 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56961.log.gz 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43127.log 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41405.log 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94699.log.gz 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41405.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43127.log.gz 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42632.log 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41405.log.gz 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39480.log 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42632.log: 88.1% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42632.log.gz 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100295.log 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83802.log 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39480.log.gz/var/log/ceph/ceph-client.admin.100295.log: 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83802.log: -- replaced with /var/log/ceph/ceph-client.admin.100295.log.gz 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.83802.log.gz 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47122.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gz 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91269.log 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40200.log 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91269.log: /var/log/ceph/ceph-client.admin.39840.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91269.log.gz 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85969.log 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80985.log 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40200.log.gz 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55655.log 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85969.log.gz 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39438.log 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55655.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80985.log.gz 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61088.log 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39438.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55655.log.gz 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39438.log.gz 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100757.log 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79678.log 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61088.log.gz 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48589.log 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79678.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100757.log.gz 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61653.log 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79678.log.gz 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85927.log 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48589.log.gz 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74912.log 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85927.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61653.log.gz 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41783.log 2024-04-25T08:40:30.863 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85927.log.gz 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96343.log 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74912.log.gz 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42833.log 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96343.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41783.log.gz 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40933.log 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96343.log.gz 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57003.log 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40933.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42833.log.gz 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40933.log.gz 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66296.log 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100337.log 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66296.log: /var/log/ceph/ceph-client.admin.57003.log: 89.4% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57003.log.gz 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66296.log.gz 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49075.log 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44592.log 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100337.log.gz 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83334.log 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44592.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49075.log.gz 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64686.log 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44592.log.gz 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61976.log 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64686.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83334.log.gz 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64686.log.gz 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91880.log 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75301.log 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.91880.log: /var/log/ceph/ceph-client.admin.61976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.91880.log.gz 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61976.log.gz 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57098.log 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100379.log 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75301.log.gz 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62933.log 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100379.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57098.log.gz 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100379.log.gz 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63368.log 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47668.log 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63368.log: /var/log/ceph/ceph-client.admin.62933.log: 88.2% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62933.log.gz 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63368.log.gz 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83502.log 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65471.log 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47668.log.gz 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83502.log.gz 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90084.log 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6%/var/log/ceph/ceph-client.admin.48799.log: -- replaced with /var/log/ceph/ceph-client.admin.65471.log.gz 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90420.log 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90084.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90084.log.gz 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr: --verbose -- /var/log/ceph/ceph-client.0.97166.log 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57989.log 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.97166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97166.log.gz 2024-04-25T08:40:30.869 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90420.log.gz 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42875.log 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82932.log 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42875.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57989.log.gz 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42875.log.gz 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.82932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84621.log 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41959.log: 91.3% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82932.log.gz 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93773.log 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42294.log 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84621.log.gz 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101093.log 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.93773.log.gz 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78791.log 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42294.log.gz 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84839.log 2024-04-25T08:40:30.871 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78791.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101093.log.gz 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78791.log.gz 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89013.log 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99730.log 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84839.log.gz 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39353.log 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99730.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99730.log.gz 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89013.log.gz 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46078.log 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75036.log 2024-04-25T08:40:30.872 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39353.log.gz 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46078.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58242.log 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46078.log.gz 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88647.log 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75036.log: /var/log/ceph/ceph-client.0.58242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58242.log.gz 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75036.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41075.log 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88647.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88647.log.gz 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42298.log 2024-04-25T08:40:30.873 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49262.log 2024-04-25T08:40:30.874 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2%/var/log/ceph/ceph-client.admin.42298.log: -- replaced with /var/log/ceph/ceph-client.admin.41075.log.gz 2024-04-25T08:40:30.874 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88905.log 2024-04-25T08:40:30.874 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42298.log.gz/var/log/ceph/ceph-client.0.49262.log: 2024-04-25T08:40:30.874 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49262.log.gz 2024-04-25T08:40:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78318.log 2024-04-25T08:40:30.874 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63687.log 2024-04-25T08:40:30.875 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88905.log.gz 2024-04-25T08:40:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86095.log 2024-04-25T08:40:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63687.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78318.log.gz 2024-04-25T08:40:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80919.log 2024-04-25T08:40:30.875 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63687.log.gz 2024-04-25T08:40:30.875 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80919.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86095.log.gz 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85801.log 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.80919.log.gz 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57567.log 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85801.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79347.log 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85801.log.gz 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56793.log 2024-04-25T08:40:30.876 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79347.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57567.log.gz 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79347.log.gz 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62264.log 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84976.log 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56793.log.gz/var/log/ceph/ceph-client.admin.62264.log: 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65074.log 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84976.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62264.log.gz 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100547.log 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84976.log.gz 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84233.log 2024-04-25T08:40:30.877 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100547.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65074.log.gz 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100547.log.gz 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49266.log 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67092.log 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.49266.log: /var/log/ceph/ceph-client.admin.84233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49266.log.gz 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84233.log.gz 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61214.log 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.67092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63326.log 2024-04-25T08:40:30.878 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67092.log.gz 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47080.log 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63326.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61214.log.gz 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63326.log.gz 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100210.log 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47080.log.gz 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34338.log 2024-04-25T08:40:30.879 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100210.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94783.log 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100210.log.gz 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.34338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78150.log 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr: 89.4%/var/log/ceph/ceph-client.admin.94783.log: -- replaced with /var/log/ceph/ceph-client.admin.34338.log.gz 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94783.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88605.log 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78150.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78150.log.gz 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78108.log 2024-04-25T08:40:30.880 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79636.log 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78108.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88605.log.gz 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78108.log.gzgzip 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57737.log 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78749.log 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57737.log: /var/log/ceph/ceph-client.admin.79636.log: 87.6% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79636.log.gz 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57737.log.gz 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61779.log 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2024-04-25T08:40:30.881 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78749.log.gz 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75849.log 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48238.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61779.log.gz 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74569.log 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96034.log 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74569.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75849.log.gz 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74569.log.gz 2024-04-25T08:40:30.882 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59871.log 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:gzip/var/log/ceph/ceph-client.admin.96034.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.89706.log 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%/var/log/ceph/ceph-client.admin.59871.log: -- replaced with /var/log/ceph/ceph-client.admin.96034.log.gz 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95297.log 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89706.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59871.log.gz 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63685.log 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89706.log.gz 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78570.log 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63685.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95297.log.gz 2024-04-25T08:40:30.883 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63685.log.gz 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88113.log 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90252.log 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78570.log.gz 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94961.log 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90252.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90252.log.gz 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88113.log.gz 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2024-04-25T08:40:30.884 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94961.log.gz 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75429.log 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45715.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57779.log 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5/var/log/ceph/ceph-client.admin.75429.log: --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75429.log.gz 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46996.log 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57779.log.gz 2024-04-25T08:40:30.885 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98845.log 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7%/var/log/ceph/ceph-client.admin.46996.log: -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66506.log 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46996.log.gz/var/log/ceph/ceph-client.admin.98845.log: 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98845.log.gz 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62531.log 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64097.log 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62531.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66506.log.gz 2024-04-25T08:40:30.886 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62531.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62445.log 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64097.log: 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99436.log 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64097.log.gz 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78192.log 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99436.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62445.log.gz 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99436.log.gz 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59805.log 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46540.log 2024-04-25T08:40:30.887 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78192.log: /var/log/ceph/ceph-client.admin.59805.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78192.log.gz 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46582.log 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46540.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59805.log.gz 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44652.log 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88245.log 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46582.log.gz 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44652.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100505.log 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44652.log.gz 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80233.log 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100505.log: /var/log/ceph/ceph-client.admin.88245.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100505.log.gz 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66254.log 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80233.log: -- replaced with /var/log/ceph/ceph-client.admin.88245.log.gz 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62306.log 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80233.log.gz 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65116.log 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62306.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66254.log.gz 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62306.log.gz 2024-04-25T08:40:30.889 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59739.log 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85018.log 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65116.log.gz 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75120.log 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.59739.log.gz 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.85018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89874.log 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85018.log.gz 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91605.log 2024-04-25T08:40:30.890 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89874.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75120.log.gz 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89874.log.gz 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44790.log 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89916.log 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44790.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.91605.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44790.log.gz 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61569.log 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44832.log 2024-04-25T08:40:30.891 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89916.log.gz 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80835.log 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44832.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61569.log.gz 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi060.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44832.log.gz 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89790.log 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94133.log 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89790.log: /var/log/ceph/ceph-client.admin.80835.log: 87.4% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80835.log.gz 2024-04-25T08:40:30.893 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89790.log.gz 2024-04-25T08:40:30.893 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49343.log 2024-04-25T08:40:30.893 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91437.log 2024-04-25T08:40:30.893 INFO:teuthology.orchestra.run.smithi060.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94133.log.gz 2024-04-25T08:40:30.893 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.49343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58513.log 2024-04-25T08:40:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91437.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49343.log.gz 2024-04-25T08:40:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101133.log 2024-04-25T08:40:30.894 INFO:teuthology.orchestra.run.smithi060.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91437.log.gz 2024-04-25T08:40:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60363.log 2024-04-25T08:40:30.894 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.101133.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58513.log.gz 2024-04-25T08:40:30.895 INFO:teuthology.orchestra.run.smithi060.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101133.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47038.log 2024-04-25T08:40:30.895 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.895 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57516.log 2024-04-25T08:40:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47038.log: /var/log/ceph/ceph-client.admin.60363.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47038.log.gz 2024-04-25T08:40:30.896 INFO:teuthology.orchestra.run.smithi060.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60363.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-04-25T08:40:30.896 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:30.896 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57516.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57516.log.gz 2024-04-25T08:40:30.897 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.84411.log 2024-04-25T08:40:30.897 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65513.log 2024-04-25T08:40:30.897 INFO:teuthology.orchestra.run.smithi060.stderr: 88.3%/var/log/ceph/ceph-client.admin.84411.log: -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-04-25T08:40:30.897 INFO:teuthology.orchestra.run.smithi060.stderr: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73322.log 2024-04-25T08:40:30.897 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84411.log.gz/var/log/ceph/ceph-client.admin.65513.log: 2024-04-25T08:40:30.898 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90294.log 2024-04-25T08:40:30.898 INFO:teuthology.orchestra.run.smithi060.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65513.log.gz 2024-04-25T08:40:30.898 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80666.log 2024-04-25T08:40:30.898 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90294.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90294.log.gz 2024-04-25T08:40:30.899 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73322.log.gz 2024-04-25T08:40:30.899 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78276.log 2024-04-25T08:40:30.899 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39606.log 2024-04-25T08:40:30.899 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78276.log: /var/log/ceph/ceph-client.admin.80666.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80666.log.gz 2024-04-25T08:40:30.899 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78276.log.gz 2024-04-25T08:40:30.900 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65303.log 2024-04-25T08:40:30.900 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96531.log 2024-04-25T08:40:30.900 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39606.log.gz 2024-04-25T08:40:30.900 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-04-25T08:40:30.901 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96531.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65303.log.gz 2024-04-25T08:40:30.901 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89307.log 2024-04-25T08:40:30.901 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96531.log.gz 2024-04-25T08:40:30.901 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-04-25T08:40:30.902 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz/var/log/ceph/ceph-client.admin.89307.log: 2024-04-25T08:40:30.902 INFO:teuthology.orchestra.run.smithi060.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83376.log 2024-04-25T08:40:30.902 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56877.log: -- replaced with /var/log/ceph/ceph-client.admin.89307.log.gz 2024-04-25T08:40:30.902 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42630.log 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91143.log 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83376.log.gz 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84789.log 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91143.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42630.log.gz 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91143.log.gz 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88311.log 2024-04-25T08:40:30.903 INFO:teuthology.orchestra.run.smithi060.stderr: 89.5%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.84789.log.gz 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75597.log 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88311.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88311.log.gz 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64906.log 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75597.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75597.log.gz 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89748.log 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64906.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64906.log.gz 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95129.log 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89748.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89748.log.gz 2024-04-25T08:40:30.904 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59106.log 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95129.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95129.log.gz 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93881.log 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.59106.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59106.log.gz 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42497.log 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.93881.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93881.log.gz 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97162.log 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42497.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42497.log.gz 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42290.log 2024-04-25T08:40:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.97162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97162.log.gz 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96867.log 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42290.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42290.log.gz 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65200.log 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96867.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96867.log.gz 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91017.log 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65200.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65200.log.gz 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41447.log 2024-04-25T08:40:30.906 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91017.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91017.log.gz 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.41447.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41447.log.gz 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48419.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90378.log 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48967.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96909.log 2024-04-25T08:40:30.907 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90378.log.gz 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94478.log 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.96909.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.96909.log.gz 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95992.log 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94478.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94478.log.gz 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47974.log 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95992.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95992.log.gz 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86011.log 2024-04-25T08:40:30.908 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.47974.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47974.log.gz 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84369.log 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.86011.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86011.log.gz 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84747.log 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84369.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84369.log.gz 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63945.log 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84747.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84747.log.gz 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74231.log 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.63945.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63945.log.gz 2024-04-25T08:40:30.909 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73783.log 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74231.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74231.log.gz 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88731.log 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73783.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73783.log.gz 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46912.log 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88731.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.88731.log.gz 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75250.log 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.46912.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46912.log.gz 2024-04-25T08:40:30.910 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99875.log 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75250.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75250.log.gz 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95003.log 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99875.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99875.log.gz 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56606.log 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95003.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95003.log.gz 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56606.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56606.log.gz 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99090.log 2024-04-25T08:40:30.911 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.55966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58385.log 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99090.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99090.log.gz 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80498.log 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.58385.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58385.log.gz 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90126.log 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80498.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80498.log.gz 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67015.log 2024-04-25T08:40:30.912 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90126.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90126.log.gz 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56656.log 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.67015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67015.log.gz 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84453.log 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56656.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56656.log.gz 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83652.log 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.84453.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84453.log.gz 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90975.log 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83652.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.83652.log.gz 2024-04-25T08:40:30.913 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61737.log 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90975.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90975.log.gz 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62783.log 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61737.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61737.log.gz 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75513.log 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.62783.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62783.log.gz 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64031.log 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75513.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75513.log.gz 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40849.log 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.64031.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.64031.log.gz 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100673.log 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40849.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40849.log.gz 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90557.log 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100673.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100673.log.gz 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91311.log 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90557.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90557.log.gz 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89486.log 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91311.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91311.log.gz 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80708.log 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.89486.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89486.log.gz 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45085.log 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80708.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80708.log.gz 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.45085.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45085.log.gz 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42633.log 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39272.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2024-04-25T08:40:30.916 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100967.log 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42633.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42633.log.gz 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61611.log 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100967.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100967.log.gz 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43505.log 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61611.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61611.log.gz 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61881.log 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.43505.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43505.log.gz 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74832.log 2024-04-25T08:40:30.917 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61881.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61881.log.gz 2024-04-25T08:40:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100463.log 2024-04-25T08:40:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.74832.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74832.log.gz 2024-04-25T08:40:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83460.log 2024-04-25T08:40:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100463.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100463.log.gz 2024-04-25T08:40:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56050.log 2024-04-25T08:40:30.918 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83460.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83460.log.gz 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42917.log 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.56050.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56050.log.gz 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100925.log 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.42917.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42917.log.gz 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66128.log 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.100925.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100925.log.gz 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75162.log 2024-04-25T08:40:30.919 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66128.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66128.log.gz 2024-04-25T08:40:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73937.log 2024-04-25T08:40:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75162.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75162.log.gz 2024-04-25T08:40:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66422.log 2024-04-25T08:40:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.73937.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73937.log.gz 2024-04-25T08:40:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90210.log 2024-04-25T08:40:30.920 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66422.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66422.log.gz 2024-04-25T08:40:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39778.log 2024-04-25T08:40:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.90210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90210.log.gz 2024-04-25T08:40:30.921 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94321.log 2024-04-25T08:40:30.922 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39778.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39778.log.gz 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57302.log 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.94321.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94321.log.gz 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60297.log 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.57302.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57302.log.gz 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64288.log 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60297.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.60297.log.gz 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95255.log 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.64288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64288.log.gz 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80189.log 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95255.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95255.log.gz 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40008.log 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.80189.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80189.log.gz 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66674.log 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.40008.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40008.log.gz 2024-04-25T08:40:30.954 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65938.log 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66674.log.gz 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66338.log 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.65938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65938.log.gz 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88395.log 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66338.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66338.log.gz 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75206.log 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88395.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88395.log.gz 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64284.log 2024-04-25T08:40:30.955 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75206.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75206.log.gz 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88863.log 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.64284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64284.log.gz 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78969.log 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.88863.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88863.log.gz 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61130.log 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.78969.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78969.log.gz 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99688.log 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.61130.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61130.log.gz 2024-04-25T08:40:30.956 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44463.log 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99688.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99688.log.gz 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95213.log 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.2.44463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44463.log.gz 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75957.log 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.95213.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95213.log.gz 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99216.log 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75957.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75957.log.gz 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44874.log 2024-04-25T08:40:30.957 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.99216.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99216.log.gz 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.44874.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44874.log.gz 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39095.log 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.48547.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60826.log 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.39095.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39095.log.gz 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75807.log 2024-04-25T08:40:30.958 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.60826.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60826.log.gz 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91395.log 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.75807.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75807.log.gz 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66632.log 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91395.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.91395.log.gz 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91563.log 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.66632.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66632.log.gz 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63120.log 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.91563.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91563.log.gz 2024-04-25T08:40:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79263.log 2024-04-25T08:40:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.0.63120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63120.log.gz 2024-04-25T08:40:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83928.log 2024-04-25T08:40:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.79263.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79263.log.gz 2024-04-25T08:40:30.960 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph-client.admin.83928.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83928.log.gz 2024-04-25T08:40:31.422 INFO:teuthology.orchestra.run.smithi060.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T08:40:31.693 INFO:teuthology.orchestra.run.smithi060.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T08:40:32.637 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T08:40:34.359 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T08:40:34.738 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T08:40:34.850 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T08:40:35.148 INFO:teuthology.orchestra.run.smithi060.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T08:40:35.431 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T08:40:35.532 INFO:teuthology.orchestra.run.smithi060.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T08:40:36.032 INFO:teuthology.orchestra.run.smithi060.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T08:40:36.033 INFO:teuthology.orchestra.run.smithi060.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T08:40:50.424 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T08:40:50.426 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-04-25T08:40:50.426 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m20.144s 2024-04-25T08:40:50.426 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m40.304s 2024-04-25T08:40:50.426 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m1.215s 2024-04-25T08:40:55.502 INFO:teuthology.orchestra.run.smithi060.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T08:40:55.504 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-04-25T08:40:55.504 INFO:teuthology.orchestra.run.smithi060.stderr:real 0m25.261s 2024-04-25T08:40:55.504 INFO:teuthology.orchestra.run.smithi060.stderr:user 0m47.676s 2024-04-25T08:40:55.504 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m1.786s 2024-04-25T08:40:55.505 INFO:tasks.ceph:Archiving logs... 2024-04-25T08:40:55.505 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/remote/smithi060/log 2024-04-25T08:40:55.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T08:41:02.276 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/remote/smithi100/log 2024-04-25T08:41:02.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T08:41:07.885 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T08:41:07.895 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T08:41:07.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T08:41:07.898 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T08:41:07.985 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T08:41:07.993 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T08:41:07.994 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:41:07.996 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:41:08.011 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:41:08.012 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:41:08.015 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:41:08.016 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-04-25T08:41:08.016 INFO:teuthology.orchestra.run.smithi060.stdout:^- hv01.front.sepia.ceph.com 2 6 377 31 +1562us[+1562us] +/- 639ms 2024-04-25T08:41:08.016 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 -608us[ -607us] +/- 58ms 2024-04-25T08:41:08.016 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +812us[ +813us] +/- 73ms 2024-04-25T08:41:08.016 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:41:08.017 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:41:08.018 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-04-25T08:41:08.018 INFO:teuthology.orchestra.run.smithi100.stdout:^- hv01.front.sepia.ceph.com 2 6 377 33 +1539us[+1539us] +/- 639ms 2024-04-25T08:41:08.018 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -622us[ -615us] +/- 58ms 2024-04-25T08:41:08.018 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 +789us[ +789us] +/- 73ms 2024-04-25T08:41:08.018 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:41:08.018 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T08:41:08.028 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T08:41:08.029 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T08:41:08.117 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T08:41:08.210 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T08:41:08.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:41:08.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:08.308 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:41:08.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:08.336 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T08:41:08.345 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T08:41:08.357 INFO:teuthology.task.internal:Duration was 1092.188797 seconds 2024-04-25T08:41:08.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T08:41:08.369 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T08:41:08.369 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T08:41:08.372 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T08:41:08.410 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:41:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:41:08.736 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T08:41:08.736 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-04-25T08:41:08.737 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T08:41:08.760 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-04-25T08:41:08.760 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T08:41:08.803 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T08:41:08.803 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T08:41:08.806 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T08:41:08.869 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T08:41:08.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T08:41:08.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T08:41:09.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T08:41:09.022 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T08:41:09.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T08:41:09.047 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T08:41:09.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T08:41:09.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:41:09.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:41:09.110 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-04-25T08:41:09.132 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-04-25T08:41:09.144 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:41:09.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:09.177 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:41:09.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:09.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T08:41:09.208 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T08:41:09.209 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/remote/smithi060 2024-04-25T08:41:09.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T08:41:09.252 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990/remote/smithi100 2024-04-25T08:41:09.253 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T08:41:09.284 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T08:41:09.284 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T08:41:09.289 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T08:41:09.333 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T08:41:09.345 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T08:41:09.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T08:41:09.359 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T08:41:09.359 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T08:41:09.361 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T08:41:09.374 INFO:teuthology.orchestra.run.smithi060.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 08:41 /home/ubuntu/cephtest 2024-04-25T08:41:09.388 INFO:teuthology.orchestra.run.smithi100.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 08:41 /home/ubuntu/cephtest 2024-04-25T08:41:09.389 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T08:41:09.400 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T08:41:09.424 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T08:41:09.448 DEBUG:teuthology.nuke:shortname: smithi060 2024-04-25T08:41:09.448 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:41:09.473 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:17:35.151676', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnV197zNM2keAFWgxLmeuHORJvZxFF/n/DwFsHB1+LnjfNnCv5zdZFeUzBQTY55RlhiFnnxVfbZD77KD4Qaj3o='} 2024-04-25T08:41:09.477 DEBUG:teuthology.nuke:shortname: smithi100 2024-04-25T08:41:09.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:41:09.499 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672990', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:17:35.149884', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP/E0Sv9UZ2S2pyRFzug2wA1AHOsjHxQQsUJUJBnVdiz9OdDQ+vMnFvwn8h50D04IPISNy9aXKRG5u6B1d8sO9w='} 2024-04-25T08:41:09.521 INFO:teuthology.orchestra.console.smithi060:Power off 2024-04-25T08:41:09.522 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T08:41:09.545 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2024-04-25T08:41:09.545 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:41:09.560 INFO:teuthology.orchestra.console.smithi100:Power off 2024-04-25T08:41:09.561 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T08:41:09.569 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-04-25T08:41:09.582 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-04-25T08:41:09.582 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:41:09.604 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-25T08:41:13.570 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:41:13.605 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:41:13.680 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2024-04-25T08:41:13.714 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-25T08:41:17.683 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:41:17.715 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:41:17.794 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2024-04-25T08:41:17.823 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-04-25T08:41:17.895 INFO:teuthology.orchestra.console.smithi060:Power off completed 2024-04-25T08:41:17.924 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-04-25T08:41:18.029 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1092.1887974739075 failure_reason: 'Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c8e26fe140064d3ca9aab2212c3b6a85 status: fail success: false 2024-04-25T08:41:18.030 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:41:18.102 INFO:teuthology.run:FAIL